Allow merging unrelated histories in git rebase

Question

When you want to rebase a branch keeping merge commits you pass the --preserve-merges flag. When you are merging unrelated history in git you need to pass the --allow-unrelated-histories flag.

If you are doing git rebase --preserve-merges when an existing merge comes from an unrelated history, it fails:

fatal: refusing to merge unrelated histories

If you try git rebase --preserve-merges --allow-unrelated-histories it fails with:

error: unknown option 'allow-unrelated-histories'

Is there some other way to tell rebase to allow the merge?

---

Edit: here is a minimal reproduction: https://github.com/vossad01/rebase-unrelated-merge-reproduction

To reproduce checkout master then execute:

git rebase --preserve-merges --onto origin/a-prime HEAD~2

Solution 1

The brute-force method is to force a common root -- since you're trying to rebase roots, with no content history, make a nonce empty commit and tell git that's the parent of the histories you're merging:

git rev-list --all --max-parents=0 \
| awk '{print $0,empty}' empty=`:|git mktree|xargs git commit-tree` \
> .git/info/grafts
git rebase here
rm .git/info/grafts

Solution 2

When git rebase fails on the merge it does not abort the rebase, so you have the opportunity to manually intervene.

If you are willing to to resolve this by hand, you can complete the merge as follows:

git merge --allow-unrelated ORIGINAL_BRANCH_THAT_WAS_MERGED --no-commit
git commit -C ORIGINAL_MERGE_COMMIT
git rebase --continue

Ideally, there would be a way for Git to handle this without manual intervention.

Solution 3

To reproduce checkout master then execute:

git rebase --preserve-merges --onto origin/a-prime HEAD~2 -i

The git-rebase docs say to not combine -i and --preserve-merges.

[--preserve-merges] uses the --interactive machinery internally, but combining it with the --interactive option explicitly is generally not a good idea unless you know what you are doing (see BUGS below).

But even without the -i it still fails with fatal: refusing to merge unrelated histories.

Part of the problem is HEAD~2 is a direct ancestor of origin/a-prime. Your test repo looks like this:

1 [master]
|
2
|\
| |  3 [origin/a-prime]
| |  |
| 4 / [origin/b]
|  /
| /
|/
5 [origin/a]

HEAD~2 of master is 5. origin/a-prime is 3. Your command is equivalent to:

git rebase -p --onto 3 5

5 is a direct ancestor of 3, so that command doesn't make much sense. If that works at all, it's going to do something weird.

---

the cases I have encountered a couple of times recently have been in moving a project's documentation from GitHub Wiki to GitHub Pages (when the website already exists).

This is an inappropriate use of rebase. Rebase turns parallel histories into linear histories, basically pretending that one set of changes was done on top of another set all along. This is good for things like keeping feature branches up to date while they're being worked on, the bookkeeping and review is easier if you don't have a bunch of intermediate merge commits that do nothing but update the branch. Those are just noise to anyone reading the code and commit history in the future.

But when you have two truly divergent histories, it's best to leave them as divergent histories. Merging them tells the correct story: the web site and docs were developed separately, but then come together into one unit.

1 - 3 - 5
         \
  2 - 4 - 6 - 7 - 8 [master]

You can look at them separately in topological order (8, 7, 6, 5, 3, 1, 4, 2) using git log --topo-order or you can look at them interleaved in date order (8, 7, 6, 5, 4, 3, 2, 1), the git log default. A history visualizer like gitk or GitX will show both orders simultaneously.

Rebasing one on top of the other tells a lie: we worked on the site, and then we worked on the documentation, and then at some point (a point you'll have to find) and for some reason we worked on the site and documentation together.

1 - 3 - 5 - 2 - 4 - 6 - 7 - 8 [master]

That loses vital information and makes puzzling out why certain changes were made more difficult in the future.

Do a merge, it's the correct thing.

Comments

[Schwern]

Could you show us some details about what problem you're trying to solve by merging unrelated history?

[vossad01]

@Schwern I have used it for various things, but the cases I have encountered a couple of times recently have been in moving a project's documentation from GitHub Wiki to GitHub Pages (when the website already exists). The reason for preserving the history (in this case) is (1) crediting contributors and (2) keeping revision history.

[Schwern]

I haven't been able to reproduce your problem using unrelated histories, my test repos might not be complex enough. For your case a rebase is probably inappropriate, that's pretending that one code base was developed on top of another one all along. They weren't, you really did have parallel development, and the history makes more sense if you preserve that. A normal merge would be more appropriate, or a subtree merge.

[vossad01]

@Schwern I have added a reproduction repository that can be cloned.

[NateW]

I'll throw in an example of why this might be useful. I currently have a repo that was once several repos. I combined them via merging the histories. Later, I decided that one of those repos actually should not have been merged in. For whatever reason, filter-branch hasn't been working out for me when I've tried to remove it from the history. So I want to do a rebase that basically excludes the commit where that was merged in. Since there are other projects merged in after that, it will involve rebasing merges of unrelated histories. Perhaps the solution is to get filter-branch to work though.

[underscore_d]

fwiw, I seem to have just gotten this to work by instead doing git rebase --rebase-merges master, which replayed the merge with an unrelated repo over master (as it originally was, at an earlier point in the latter).

[vossad01]

This blew my mind a little bit and it works! In dumbed down language, it uses grafts to artificially makes all branches related to a empty parent/root commit. You do the rebase while Git thinks everything is related. Then your remove the grafts to set things back.

[vossad01]

I agree that the two repositories history should be joined with a merge. The rebase in question is happening after that merge was done and is not trying to linearize the divergent history (hence the --preserve-merges).

[vossad01]

The expected result is that 2 would have parents 3 and 4.

[Schwern]

@vossad01 Sorry, I don't follow how your demonstration repo represents the result of merging two unrelated branches. Which commit is supposed to represent the two unrelated histories coming together? 2? Is 3 a branch that existed before the merge? I also don't follow why you're rebasing after the merge, what's that trying to accomplish? Are you trying to bring branches that existed before the merge up to date? Is that what 3 represents?

[vossad01]

Yes. 2 is the original merge of unrelated branches. 3 was not available at the time of the merge (or else the merge would have originally been done on top of it). Yes, it is trying to update the lineage so that it appears the merge occurred after 3 (which, in this case, represents the latest state of the original/base repository).

[Schwern]

@vossad01 In that case rebase origin/a-prime, and any other outstanding branches, onto the new master. git checkout origin/a-prime; git rebase -p master. You might have to specify the start and end of the branch like git rebase --onto master origin/a origin/a-prime which says to rebase all the commits starting with, but not including, origin/a up to origin/a-prime onto master.

[vossad01]

I appreciate the time and effort you spent on this. In this case a and a-prime are upstream so I cannot rewrite that history. I really need to be moving the side that I merged in and the commits I made after the merge, since those are the ones I control. I agree this is probably not an especially common case. If I weren't such a perfectionist about history (I think it can make PRs more easily accepted), I would probably just merge a-prime on top of my changes and not worry about a messier graph. @jthill's solution did accomplish what I was trying.

[Admin]

Consider using git replace --graft instead, per this answer to How do git grafts and replace differ? (Are grafts now deprecated?).

[jthill]

@sampablokuper the method git replace --graft uses is not desirable here.

[Admin]

@jthill, it would be great if you could expand your answer to explain why git replace --graft is unsuitable (or less suitable) in this case. Thanks!

[jthill]

@sampablokuper git replace --graft makes a new commit with the rewritten ancestry instead of simply specifying temporary ancestry for an existing commit, so a rebase that was meant to add a new child to the existing commit would instead add the child to the replacement commit.

[nishantjr]

Interesting to note is that this does not work with merges created via git-subtree add. For it to work, git would need to remember which merge strategy was used, and how conflict resolution happened in relation to that. Sounds aweful.

[jthill]

@sampablokuper ... which, I see, isn't really what happens, I simply found the added gyrations of actually creating a replacement object and a ref and then going through and reading a directory and then reading every file in that directory so see the replacement ids and then having to read the object database and parse each replacement object to see what the parents are, was so overengineered (you mean if I want to use grafts, now I have to actively defend my repo against other people pushing in arbitrary rewrites remotely? seriously?) I assumed, wrongly, it would fail here too.

[Sid]

Even without rebase (i.e. using merge), git doesn't preserve history of when something was "worked on", it preserves history of when something was "committed". It is at best a log of commits, not of keystrokes and brainfarts. Many of us use git as a means to communicate, and rebase helps tell that story. There's no doubt you can write a bad story, but your contention against rebase uses a strawman.

[Didier L]

For people who would like to do this with an octopus merge, this solution should be combined with Git octopus merge with unrelated repositories