How do you merge two Git repositories?

Question

Consider the following scenario:

I have developed a small experimental project A in its own Git repo. It has now matured, and I'd like A to be part of larger project B, which has its own big repository. I'd now like to add A as a subdirectory of B.

How do I merge A into B, without losing history on any side?

Solution 1

A single branch of another repository can be easily placed under a subdirectory retaining its history. For example:

git subtree add --prefix=rails git://github.com/rails/rails.git master

This will appear as a single commit where all files of Rails master branch are added into "rails" directory. However the commit's title contains a reference to the old history tree:

Add 'rails/' from commit <rev>

Where <rev> is a SHA-1 commit hash. You can still see the history, blame some changes.

git log <rev>
git blame <rev> -- README.md

Note that you can't see the directory prefix from here since this is an actual old branch left intact. You should treat this like a usual file move commit: you will need an extra jump when reaching it.

# finishes with all files added at once commit
git log rails/README.md

# then continue from original tree
git log <rev> -- README.md

There are more complex solutions like doing this manually or rewriting the history as described in other answers.

The git-subtree command is a part of official git-contrib, some packet managers install it by default (OS X Homebrew). But you might have to install it by yourself in addition to git.

Solution 2

If you want to merge project-a into project-b:

cd path/to/project-b
git remote add project-a /path/to/project-a
git fetch project-a --tags
git merge --allow-unrelated-histories project-a/master # or whichever branch you want to merge
git remote remove project-a

Taken from: git merge different repositories?

This method worked pretty well for me, it's shorter and in my opinion a lot cleaner.

In case you want to put project-a into a subdirectory, you can use git-filter-repo (filter-branch is discouraged). Run the following commands before the commands above:

cd path/to/project-a
git filter-repo --to-subdirectory-filter project-a

An example of merging 2 big repositories, putting one of them into a subdirectory: https://gist.github.com/x-yuri/9890ab1079cf4357d6f269d073fd9731

Note: The --allow-unrelated-histories parameter only exists since git >= 2.9. See Git - git merge Documentation / --allow-unrelated-histories

Update: Added --tags as suggested by @jstadler in order to keep tags.

Solution 3

Here are two possible solutions:

Submodules

Either copy repository A into a separate directory in larger project B, or (perhaps better) clone repository A into a subdirectory in project B. Then use git submodule to make this repository a submodule of a repository B.

This is a good solution for loosely-coupled repositories, where development in repository A continues, and the major portion of development is a separate stand-alone development in A. See also SubmoduleSupport and GitSubmoduleTutorial pages on Git Wiki.

Subtree merge

You can merge repository A into a subdirectory of a project B using the subtree merge strategy. This is described in Subtree Merging and You by Markus Prinz.

git remote add -f Bproject /path/to/B
git merge -s ours --allow-unrelated-histories --no-commit Bproject/master
git read-tree --prefix=dir-B/ -u Bproject/master
git commit -m "Merge B project as our subdirectory"
git pull -s subtree Bproject master

(Option --allow-unrelated-histories is needed for Git >= 2.9.0.)

Or you can use git subtree tool (repository on GitHub) by apenwarr (Avery Pennarun), announced for example in his blog post A new alternative to Git submodules: git subtree.

---

I think in your case (A is to be part of larger project B) the correct solution would be to use subtree merge.

Solution 4

The submodule approach is good if you want to maintain the project separately. However, if you really want to merge both projects into the same repository, then you have a bit more work to do.

The first thing would be to use git filter-branch to rewrite the names of everything in the second repository to be in the subdirectory where you would like them to end up. So instead of foo.c, bar.html, you would have projb/foo.c and projb/bar.html.

Then, you should be able to do something like the following:

git remote add projb [wherever]
git pull projb

The git pull will do a git fetch followed by a git merge. There should be no conflicts, if the repository you're pulling to does not yet have a projb/ directory.

Further searching indicates that something similar was done to merge gitk into git. Junio C Hamano writes about it here: http://www.mail-archive.com/[email protected]/msg03395.html

Solution 5

git-subtree is nice, but it is probably not the one you want.

For example, if projectA is the directory created in B, after git subtree,

git log projectA

lists only one commit: the merge. The commits from the merged project are for different paths, so they don't show up.

Greg Hewgill's answer comes closest, although it doesn't actually say how to rewrite the paths.

---

The solution is surprisingly simple.

(1) In A,

PREFIX=projectA #adjust this

git filter-branch --index-filter '
    git ls-files -s |
    sed "s,\t,&'"$PREFIX"'/," |
    GIT_INDEX_FILE=$GIT_INDEX_FILE.new git update-index --index-info &&
    mv $GIT_INDEX_FILE.new $GIT_INDEX_FILE
' HEAD

Note: This rewrites history; you may want to first make a backup of A.

Note Bene: You have to modify the substitute script inside the sed command in the case that you use non-ascii characters (or white characters) in file names or path. In that case the file location inside a record produced by "ls-files -s" begins with quotation mark.

(2) Then in B, run

git pull path/to/A

Voila! You have a projectA directory in B. If you run git log projectA, you will see all commits from A.

---

In my case, I wanted two subdirectories, projectA and projectB. In that case, I did step (1) to B as well.

View more solutions