Git gc using excessive memory, unable to complete

Question

Final update and fix: The solution here turned out to be a combination of two things: using Windows Git rather than Cygwin Git as Graham Borland suggested, and the Git config settings pack.threads = 1 and gc.aggressiveWindow = 150.

I have a large local Git repository, a git svn clone of an SVN repository with about 40,000 commits. I'm trying to run git gc over this repository, and getting nowhere:

$ git gc --auto
Auto packing the repository for optimum performance. You may also
run "git gc" manually. See "git help gc" for more information.
Counting objects: 25966, done.
Compressing objects: 100% (25249/25249), done.
fatal: Out of memory, malloc failed (tried to allocate 426523986 bytes)
error: failed to run repack

I'm running Git 1.7.5.1 inside Cygwin on a 64-bit dual-core Win7 machine with 4GB RAM. The .git directory is currently a little over 6.1GB.

I've tried running git gc --aggressive, to see if the more complete system is able to fix it, but no luck: I get a similar message to the above, with the same size attempted malloc, but a considerably higher object count (508,485 counted, 493,506 compressed).

I've also tried—as suggested by Google—assorted twiddles to the [pack] part of my .gitconfig file; the most complete being from another StackOverflow question. My .gitconfig now has the following relevant lines, but setting these appears to have made no difference:

[pack]
        windowMemory = 16m
        threads = 1
        window = 1
        depth = 1
        deltaCacheSize = 1

Any suggestions on how I can get git to gc my repository?

Edit: Mark Longair suggested some more .gitconfig file changes. Which I made, new lines below. But the changes made no difference whatsoever.

[core]
        packedGitWindowSize = 1m
        packedGitLimit = 256m
[pack]
        packSizeLimit = 128m

Edit 2: Michael Krelin suggested increasing the swap/page file size (WinXP instructions here, and it's similar for Win7). I tried that, but it made no difference, and indeed I only increased the maximum size available, and it looks as if Windows never tried to increase the size of the page file it was using.

I'm now looking at whether this was caused by a memory limit within or imposed upon Cygwin. To check "imposed upon", I'm trying running Cygwin with administrator privileges. To check "within" (which looks more likely), I'm having a play with Cygwin's maximum memory settings.

Edit 3: Much though I may prefer using Cygwin, it turns out the Windows Git client deals with the memory issue just fine. Seems I'll be falling back to that every so often when my repository needs a tidy.

Solution 1

You might have more luck running a native Windows client such as msysGit, rather than trying to do it inside Cygwin.

Solution 2

I had the same problem, tried the solutions mentioned so far without success. But my problems with git gc began after I added big image files to repo, so I created .gitattributes file and turned off delta compression for those big files:

*.tga -delta
*.psd -delta

It worked.

Solution 3

Some other config options that you might want to try restricting to lower than default values include:

• pack.packSizeLimit
• core.packedGitWindowSize
• core.packedGitLimit

... all of which are documented in the git config documentation. It's particularly worth checking in each case what units are understood, which I've made mistakes with in the past.

Solution 4

The only thing that help to avoid this error on shared Linux hosting was to add

  [pack]
    packSizeLimit = 64m
    threads = 1

to

.gitconfig

Most important was "threads = 1"

Solution 5

Maybe temporarily adding a swap file bigger than life and going for a few cups of coffee elsewhere will help?

View more solutions