What does "Auto packing the repository for optimum performance" mean?

Question

I'm having a problem with my git repo. For the last couple of days whenever I do a push to the server I get this message: "Auto packing the repository for optimum performance", and it does not seem to go away and return the shell.

I also tried checking out to a new branch and then doing a rebase on my previous branch and then did git gc to remove the unused history objects and then did a push but still this message appears. Please let me know what's going on with my repo.

Solution 1

Short version: it means what it says, and if you just let it finish, all will be well.

During most operations which can potentially increase the number of loose (unpacked) objects in the repository (including pushes), Git invokes git gc --auto. If there are enough loose objects (by default, at least 6700), it will then invoke git repack -d -l to pack them. If there are too many separate packs, it will also repack them into one.

A pack is a delta-compressed single file, containing a large number of objects. It's more efficient to store objects in packs, but it takes time to pack (compress) objects, so Git initially creates loose objects, then packs them in batches now and then, via automatic invocation of git gc --auto.

If you let Git finish repacking, this won't happen again for a while. It can indeed take a while, especially if you have a lot of large binary objects, but if it's triggering, then it's a sign that it will probably drastically reduce the amount of disk space taken by the repo. If you really don't want it to happen, you can change the config parameter gc.auto. If you increase it to something much larger than 6700, it will happen less frequently, but take longer when it does. If you decrease it, it'll still have to do your current repack, but subsequently it will happen more often and finish more quickly. If you set it to 0, it will disable automatic repacking.

See man git-gc (under --auto) and man git-config (under gc.auto) for more information.

Solution 2

While Jefroni is correct that sometimes the auto-packing just needs time to complete, if the auto-packing message persists over multiple days as OP describes, there's a good chance that git's cleanup is missing dangling objects, as described in this question.

To see whether dangling objects are triggering ongoing messages about auto-packing, try running git fsck. If you get a long list of dangling commits, you can clean them with

git gc --prune=now

I usually have to run this on my repo every 2-3 months when the auto-packing message doesn't go away after a single pull.

Solution 3

To disable for one project:

cd your_project_dir
git config gc.auto 0

To disable globally:

git config --global gc.auto 0

Solution 4

Git is running git-repack, which packs many objects(=files, commits and trees) into one pack file. Git does this sometimes, when a heuristic says that there can be space saved (a pack file contains compressed object deltas, while each file in the objects/ directory contains the compressed full file content)

Solution 5

Hopefully, that git gc --auto step is now (git 2.0.1, June 25th, 2014) more efficient.
See commit 62aad18 by Nguyễn Thái Ngọc Duy (pclouds)

gc --auto: do not lock refs in the background

9f673f9 (gc: config option for running --auto in background - 2014-02-08, Git 2.0.0) puts "gc --auto" in background to reduce user's wait time.
Part of the garbage collecting is pack-refs and pruning reflogs. These require locking some refs and may abort other processes trying to lock the same ref.

If gc --auto is fired in the middle of a script, gc's holding locks in the background could fail the script, which could never happen before 9f673f9.

Keep running pack-refs and "reflog --prune" in foreground to stop parallel ref updates. The remaining background operations (repack, prune and rerere) should not impact running git processes.

And Git 2.22 (Q2 2019) further optimize git gc.

View more solutions

Comments

[Bachsau]

git config gc.autoDetach will disable this behaviour.

[Joshua Pinter]

Indeed, this took about 5 minutes for me, but it did finish up. Great answer.

[Admin]

We're seeing it happen with every push (making a while a few seconds, heh).

[Cascabel]

@dpk: That should not happen in normal circumstances - the number of objects in a single push should not be large enough to trigger it (unless your repository is enormous and/or you're pushing a ton of commits), so once it successfully completes (you are letting it complete, right?) it should not happen again until you build up to it. If you can't figure it out, ask a separate question.

[ruffin]

"If you let Git finish", and it can... fatal: Out of memory, malloc failed (tried to allocate 79610689 bytes) error: failed to run repack -- this is what I get for sticking our entire codebase into one git repo. Guess I'm going to kill apps and force repack "manually"

[Adrian Keister]

I think I found out how: go to the .git folder, open up the config file, and delete the text 'auto = 0', and save. That seems to re-enable autopacking.

[jtatum]

git config --unset gc.auto

[Barry Kelly]

I'm getting it every single time I do a git pull. I've done a manual git gc, but it still happens every time I pull. Weird.

[teashark]

Good rule: try to avoid binaries in your source

[jvriesem]

@cottsak: What about images or image-heavy PDFs?

[teashark]

@jvriesem same. they are binary. try to avoid them if you can. read: try.

[dbslone]

While not the accepted answer, this was exactly what I needed. I got the message every time I did a git pull, over several days, and fsck indeed showed a ton of dangling commits.

[eis]

I'm getting continuous Unlink of file '.git/objects/pack/pack-077ae8465e7c1511dc09692d0d0b11c1bf95‌​65ce.idx' failed. Should I try again? (y/n) when its doing this, never succeeding. Good thing I found how to disable this.

[Cascabel]

@eis If git is unable to delete (unlink) a file, that's kind of a problem, whether or not you want to gc your repository. You might want to double-check file permissions, at the very least.

[eis]

The repo was created by the same user, no other users on this machine (windows machine), so doesn't suggest file permissions problem.

[Cascabel]

@eis Well, the error does suggest permissions problems. Note that apparently on Windows, this can also happen because the OS refusees permission to unlink files when it thinks they're in use. There are several past questions on it. In any case, I wouldn't take disabling gc entirely as a solution.

[eis]

Yes, I am aware of the windows behavior, but disabling gc is the only working solution I've found so far. Also usually when I push I don't have time to go debugging windows, I need the fixes in to be able to do my job.

[dkellner]

To quote a classic: This is the way

[zyy]

Great answer, I let it be and checked in the next morning, it is gone.

[Han K]

Just an FYI - for the git gc --prune=now to work, make sure all the IDEs, Git bashes and Git UI programs are closed. Otherwise, it won't work as the resource/file are "locked". Happened to me as I forgot to close my GitKraken tool.