git server replication

Question

We are using git for a multi site project and both the repo and the team have grown substantially. Therefore remote sites are suffering. I would like to add new git servers to the remote sites to decrease the load on our git server and to make the downloads faster. However I am not sure how to keep these synchronized. Is there any way to replicate git servers atomically? And if not any other suggestions?

Solution 1

You could have a main "write" repo, and multiple remote, read only, repos pull from that. In turn, remote users would pull from those. That would give you increased download speeds from remote sites. Uploads would still be the same, but it seems like download speed is your main concern.

Solution 2

You can't replicate atomically. I'm kind of hoping that that's not what you mean.

I wrote gitmirror for automatically triggering a remote pull-based replication on repository change. It requires a tiny bit of setup, but I've got tons of stuff coming through github into it (and a few that don't use github).

Comments

[l.thee.a]

I was thinking about the same thing. However how do you avoid a situation where the commit is against a pull from a read-only server which is not up-to-date? It seems to me that this might cause some issues (merge/contention etc).

[l.thee.a]

I actually meant atomically. However gitmirror might solve the other part of my problem which was mirroring.

[Dustin]

Atomically is pretty well incompatible with globally distributed system with lots of moving parts. It's also undesirable. The purpose of a distributed revision control system is to allow these kinds of things to work even when there's no connection at all.

[Dustin]

The push will fail and the user will have to refetch and merge or rebase and retry. Encourage good, clean, frequent commits and this will never be an issue for you. If everyone works all day from that morning's pull and does one final "time to go home" commit and they all push it right when the whistle blows, the contention and additional work generated there are only symptoms. :)

[Andy]

That same problem could occur if you pulled it from the central repo and someone made commits after you pulled. You could easily setup a hook or cron that will make sure the satellite repos are never more than a few minutes behind the central repo