gitignore by file size?

Question

I'm trying to implement Git to manage creative assets (Photoshop, Illustrator, Maya, etc.), and I'd like to exclude files from Git based on file size rather than extension, location, etc.

For example, I don't want to exclude all .avi files, but there are a handful of massive +1GB avi files in random directories that I don't want to commit.

Any suggestions?

Solution 1

I'm new to .gitignore, so there may be better ways to do this, but I've been excluding files by file size using:

find . -size +1G | cat >> .gitignore

Obviously you'll have to run this code frequently if you're generating a lot of large files.

Solution 2

Although the file size is very large and the following should not be an issue at all and provided that @abendine answer is correct, according to: https://stackoverflow.com/a/22057427/6466510

find * -size +1G | cat >> .gitignore

it would be far better. Have a look at this too: Difference between find . and find * in unix it turns out that replacing . with * here above, avoid to find things in .git directory.

Solution 3

To satisfy github's <100MB file limit, run this:

find . -size +100M | cat >> .gitignore

Solution 4

I wanted to also offer a Windows version of this as well.

forfiles /s /c "cmd /q /c if @fsize GTR 1073741824 echo @relpath" >> .gitignore

Solution 5

(Update 2020-05)

Microsoft released time ago Git-LFS as Open-Source. Probably this is what most people really are searching for:

https://git-lfs.github.com/ C&P from the project page: "Git Large File Storage (LFS) replaces large files such as audio samples, videos, datasets, and graphics with text pointers inside Git, while storing the file contents on a remote server like GitHub.com or GitHub Enterprise."

View more solutions

Comments

[aubreypwd]

This is a great way to control wp-content/uploads folder when deploying large sites built in WordPress. Thanks for this.

[IanB]

I found I needed to remove the leading ./ from the start of each file before gitignore would work

[antass]

This is a very old thread, but in case someone stumbled upon it and needed a pastable solution: find . -size +1G | sed 's|^\./||g' | cat >> .gitignore

[antass]

To avoid storing duplicate file names: find . -size +1G | sed 's|^\./||g' | cat >> .gitignore; awk '!NF || !seen[$0]++' .gitignore. sed will get rid of trailing ./, and awk will remove all duplicate lines ignoring empty ones by checking the number of fields present in a line (NF) - this is useful if your .gitignore is organized in sections separated by empty lines.

[Sam Watkins]

You all win a "useless use of cat award".

[baxx]

@SamWatkins do you have an example of how the above could be done without using cat? ( i think that it can just be completely dropped from the above? )

[Sida Zhou]

lol, took me a second to realize what is the cat award: find . -size +1G | sed 's|^\./||g' >> .gitignore

[user3710044]

You don't need sed either ... find . -size +1G -printf '%P\n' works too.

[Skyborg]

yes they shoud install git lfs and track the big files to automatic exclude them from the repositorys.

[Robin Kohrs]

Can I do this in some kind of automated way? I mean something like putting a function in my repo (or in my gitignore if this possbile) that continously collects files larger than some size and puts them in the gitignore?

[Kavidu Aloka Kodikara]

In windows, run the same command and it might add ./ to the start of each path as below ./android/app/build/outputs/apk/debug/app-universal-debug.ap‌​k . Removing the ./ as below solved my issue; android/app/build/outputs/apk/debug/app-universal-debug.apk

[KIC]

@RobinKohrs I have added yet another answer using a githook to add some automatic behavior