git-upload-pack hangs indefinitely

Question

I have the following call structure:

1. Jenkins runs fab -Huser@host set_repository_commit_hash:123abc.
2. set_repository_commit_hash runs git fetch with pty = False.
3. The child process ssh [email protected] git-upload-pack 'user/repository.git' never finishes.

I've tried running git fetch in a local clone and that succeeds, but running ssh [email protected] git-upload-pack 'user/repository.git' just returns the following and hangs:

00ab84249d3bb20930c185c08848c60b71f7b28990d6 HEADmulti_ack thin-pack side-band side-band-64k ofs-delta shallow no-progress include-tag multi_ack_detailed agent=git/1.8.4
0041cb34b1c8ca75d478df38c794fc15c5f01cc6377e refs/heads/branch_name
004012577068adf47015001bfa0cff9386d6cdf497ce refs/heads/[...]
003f84249d3bb20930c185c08848c60b71f7b28990d6 refs/heads/master
[a couple more lines like the ones above, then:]
0000

Is this a known SSH/Git/Fabric/Jenkins problem?

I did strace it, but I haven't recorded the session. I believe it was stuck on a read.

Possibly relevant links:

1. Jenkins issue 14752: SCM Polling / Max # of concurrent polling = 1 hangs github polling
2. Why would git-upload-pack (during git clone) hang?
3. tortoisegit issue 1880: tortoisegit fetch hangs due to running/never-exiting tortoisegitplink (especially comment #7)
4. What is this random never-ending 'git-upload-pack' process?

Solution

This problem appears to have gone away on its own, as can be expected by a rapidly evolving piece of software. Since I have not observed this issue for probably a couple years now I'd like to extend my thanks to whoever fixed it and consider this question obsolete.

If you are experiencing this issue with recent Git versions please consider asking a separate question, since it is likely not the exact same issue.

Comments

[slm]

Have tried any of the suggestions in the SO Q&A? Also what does strace -p <pid of hung git daemon> say it's blocking on?

[l0b0]

@slm Do you have any specific suggestions in mind?

[MauricioOtta]

What fixed for me was making sure Jenkins was running as the user with proper credentials to the GIT repos

[derobert]

That's what git-upload-pack is supposed to do, AFAICT. It's waiting for you to speak the git fetch-pack protocol and tell it which things to send (try running it on a local repository, you'll get the same output).

[derobert]

(Presuming this question isn't obsolete, as it's from 2013) Does a fresh git clone (from github) work on the host that Jenkins is trying to fetch on? I suspect it won't, and you probably have a Path MTU discovery issue caused by a broken firewall (which might be anywhere on the path—not just your side.)

[l0b0]

I remember seeing the same symptom a few months ago, but it has been a very rare occurrence since this was reported. I'm leaving it here in case it becomes more common again or others are having the same issue.

[ckujau]

Has this been reported to the Fabric bugtracker? Or to the Git project, if this is reproducible w/o fab?

[l0b0]

@ckujau No, because it's not reproducible, with or without fab. That is, I believe it's happened three times in as many years. Since it's not reproducible and it happens during Jenkins jobs which are usually urgent we generally just kill the job and restart.

[poige]

I'm voting to close this question as off-topic because it’s superuser’s subject rather