Git merge submodule into parent tree cleanly and preserving commit history

Question

I have a repository with two submodules that I want to convert into a single project. Many answers involve scripts, and some seem to be overcomplicated.

[submodule "site"]
    path = wp-content/themes/site
    url = https://[email protected]/ajf-/site.git
    fetchRecurseSubmodules = true
    ignore = all
[submodule "wpsite"]
    path = wp-content/themes/wpsite
    url = https://[email protected]/ajf-/wpsite.git
    fetchRecurseSubmodules = true
    ignore = all

Is there an officially supported / documented way to merge these submodules into the parent repository?

Solution 1

The best approach is to do subtree merging.

First, remove the submodules and related configuration from your superproject; Edit your .gitmodules file to remove the submodules affected, or delete the file entirely if you intend to merge all submodules. Delete the submodule directories as well.

Next, add the submodule repositories as proper remotes to your superproject:

git remote add site https://[email protected]/ajf-/site.git
git remote add wpsite https://[email protected]/ajf-/wpsite.git

Then, fetch the remotes:

git fetch --all

Now, check out the branches that you want to graft to your main project from each sub-project:

git checkout -b site-branch site/some_branch
git checkout -b wpsite-branch wpsite/some_other_branch

After that, return to the master branch, or to the branch where you want to create the combined superproject:

git checkout master

If you want to create a (possibly temporary) extra branch for the operation, do this instead:

git checkout -b new-superproject master

You're now ready to merge the module branches as subtrees with your main project (in the master branch in this example):

git read-tree --prefix=site/ -u site-branch
git read-tree --prefix=wpsite/ -u wpsite-branch

And you're done. Check the result with gitk --all.

Since you want to convert into a single project, you're not going to update the subprojects independently, so I'm not going to describe how that works.

You can read up on this in the chapter on subtree merging from Pro Git

Solution 2

A bit late to the party, but for those still looking for help, checkout these:

1. https://gitirc.eu/howto/using-merge-subtree.html
2. https://help.github.com/en/github/using-git/about-git-subtree-merges

Below is a near verbatim copy of the first post:

1. git remote add -f Bproject /path/to/B
2. git merge -s ours --no-commit --allow-unrelated-histories Bproject/master
3. git read-tree --prefix=dir-B/ -u Bproject/master
4. git commit -m "Merge B project as our subdirectory"

5. git pull -s subtree Bproject master

Explanation:

1. name the other project "Bproject", and fetch.
2. prepare for the later step to record the result as a merge.
3. read "master" branch of Bproject to the subdirectory "dir-B".
4. record the merge result.

5. pull in subsequent update from Bproject using "subtree" merge

As an alternative before step 4, you might want to update .gitmodules file, or just remove it:

3.1 git rm --cached .gitmodules

With this the history from the submodule is well preserved.

Comments

[Qw3ry]

Possible duplicate of un-submodule a git submodule

[Eugen Konkov]

Good article; medium.com/walkme-engineering/…

[Alain Jacomet Forte]

I had a problem with this method, see stackoverflow.com/questions/23447457/…

[Navarr]

@AlainJacometForte: The question you link to is 410 Gone. (404ing). Did you turn out to be mistaken?

[kampta]

@Navarr stackoverflow.com/questions/1759587/… maybe?

[João Andrade]

pretty elegant solution!

[newbie_android_dev]

I think you should probably add git checkout master prior to the read-tree step

[quazgar]

Note that this answer (at least for me) does not preserve the history of the submodules.

[Sigi]

@StephenHandley I have edited the post to reflect your suggestion.

[Sigi]

This is a very good answer. I should update my answer to reflect your suggestion of adding the merge step to preserve history.