Backing Out a backwards merge on Mercurial

Question

How do you reverse the effect of a merge on polarised branches without dying of agony?

This problem has been plaguing me for months and I have finally given up.

You have 1 Repository, with 2 Named Branches. A and B.

Changes that occur to A will inevitably occur on B.

Changes that occur directly on B MUST NEVER occur on A.

In such a configuration, merging "B" into "A" produces a dire problem in the repository, as all the changes to B appear in A as if they were made in A.

The only "normal" way to recover from this situation appears to be "backing out" the merge, ie:

 hg up -r A 
 hg backout -r BadMergeRev --parent BadMergerevBeforeOnA 

Which looks all fine and dandy, until you decide to merge later in the correct direction, and you end up with all sorts of nasty things happening and code that was erased / commented out on specifically branch B suddenly becomes unerased or uncommented.

There has not been a working viable solution to this so far other than "let it do its thing, and then hand fix all the problems" and that to be honest is a bit fubar.

Here is an image clarifying the problem:

[Original image lost]

Files C & E ( or changes C & E ) must appear only on branch b, and not on branch a. Revision A9 here ( branch a, revno 9 ) is the start of the problem.

Revisions A10 and A11 are the "Backout merge" and "merge the backout" phases.

And revision B12 is mercurial, erroneously repeatedly dropping a change that was intended not to be dropped.

This Dilemma has caused much frustration and blue smoke and I would like to put an end to it.

Note

It may be the obvious answer to try prohibiting the reverse merge from occurring, either with hooks or with policies, I have found the ability to muck this up is rather high and the chance of it happening so likely that even with countermeasures, you must still assume that inevitably, it will happen so that you can solve it when it does.

To Elaborate

In the model I have used Seperate files. These make the problem sound simple. These merely represent arbitrary changes which could be a separate line.

Also, to add insult to injury, there have been substantial changes on branch A which leaves the standing problem "do the changes in branch A conflict with the changes in branch B which just turned up ( and got backed out ) which looks like a change on branch A instead "

On History Rewriting Tricks:

The problem with all these retro-active solutions is as follows:

1. We have 9000 commits.
2. Cloning freshly thus takes half an hour
3. If there exists even one bad clone of the repository somewhere, there is a liklihood of it comming back in contact with the original repository, and banging it up all over again.
4. Everyone has cloned this repository already, and now several days have passed with on-going commits.
5. One such clone, happens to be a live site, so "wiping that one and starting from scratch" = "big nono"

( I admit, many of the above are a bit daft, but they are outside of my control ).

The only solutions that are viable are the ones that assume that people can and will do everything wrong, and that there is a way to 'undo' this wrongness.

Solution 1

I think I found a solution which permanently fixes the bad merge, and which does not require you to manually check any diffs. The trick is to go back in history and generate commits parallel to the bad merge.

So we have repository with separate branches per maintained version of a single product. Like the situation posed in the question, all changes made on a branch of an earlier version (ie. the bugfixes for in that version) must all eventually be merged to the branches of the later versions.

So specifically, if something is checked in on BRANCH_V8, it must be merged to BRANCH_V9.

Now one of the developers makes following mistake : he merges all changes from BRANCH_V9 into BRANCH_V8 (ie. a merge in the wrong direction). Furthermore, after that bad merge he performs some extra commits before he notices his mistake.

So the situation is as shown in the graphical log below.

o  BRANCH_V8 - 13 - important commit right after the bad merge
|
o    BRANCH_V8 - 12 - wrong merge from BRANCH_V9
|\
| o  BRANCH_V8 - 11 - adding comment on BRANCH_V8 (ie. last known good state)
| |
o |  BRANCH_V9 - 10 - last commit on BRANCH_V9
| |

We can fix this mistake as follows:

1. update your local directory to the last good state of BRANCH_V8: hg update 11
2. Create a new child of that last good state :
   1. change some file $EDITOR some/file.txt (this is necessary because Mercurial does not allow empty commits)
   2. commit these changes hg commit -m "generating commit on BRANCH_V8 to rectify wrong merge from BRANCH_V9"
      The situation now looks as follows :

      o  BRANCH_V8 - 14 - generating commit on BRANCH_V8 to rectify wrong merge from BRANCH_V9
      |
      | o  BRANCH_V8 - 13 - important commit right after the bad merge
      | |
      | o  BRANCH_V8 - 12 - wrong merge from BRANCH_V9
      |/|
      o |  BRANCH_V8 - 11 - adding comment on BRANCH_V8
      | |
      | o  BRANCH_V9 - 10 - last commit on BRANCH_V9
      

3. Merge the newly generated head with the revision in which the bad merge happened, and throw away all changes before committing. Do not simply merge the two heads, because you will then lose the important commit which happened after the merge as well!

   1. merge : hg merge 12 (ignore any conflicts)
   2. throw away all changes : hg revert -a --no-backup -r 14
   3. commit the changes : hg commit -m "throwing away wrong merge from BRANCH_V9" The situtation now looks like :

      o    BRANCH_V8 - 15 - throwing away wrong merge from BRANCH_V9
      |\
      | o  BRANCH_V8 - 14 - generating commit on BRANCH_V8 to rectify wrong merge from BRANCH_V9
      | |
      +---o  BRANCH_V8 - 13 - important commit right after the bad merge
      | |
      o |  BRANCH_V8 - 12 - wrong merge from BRANCH_V9
      |\|
      | o  BRANCH_V8 - 11 - adding comment on BRANCH_V8
      | |
      o |  BRANCH_V9 - 10 - last commit on BRANCH_V9
      | |
      

   Ie. there are two heads on BRANCH_V8: one which contains the fix of the bad merge, and the other containing the left-over important commit on BRANCH_V8 which happened right after the merge.

4. Merge the two heads on BRANCH_V8 :
   1. merge : hg merge
   2. commit : hg commit -m "merged two heads used to revert from bad merge"

The situation in the end on BRANCH_V8 is now corrected, and looks like this:

o    BRANCH_V8 - 16 - merged two heads used to revert from bad merge
|\
| o    BRANCH_V8 - 15 - throwing away wrong merge from BRANCH_V9
| |\
| | o  BRANCH_V8 - 14 - generating commit on BRANCH_V8 to rectify wrong merge from BRANCH_V9
| | |
o | |  BRANCH_V8 - 13 - important commit right after the bad merge
|/ /
o |  BRANCH_V8 - 12 - wrong merge from BRANCH_V9
|\|
| o  BRANCH_V8 - 11 - adding comment on BRANCH_V8
| |
o |  BRANCH_V9 - 10 - last commit on BRANCH_V9
| |

Now the situation on BRANCH_V8 is correct. The only problem remaining is that the next merge from BRANCH_V8 to BRANCH_V9 will be incorrect, as it will merge in the 'fix' for the bad merge as well, which we do not want on BRANCH_V9. The trick here is to merge from BRANCH_V8 to BRANCH_V9 in separate changes :

• First merge, from BRANCH_V8 to BRANCH_V9, the correct changes on BRANCH_V8 from before the bad merge.
• Second merge in the merge mistake and its fix, and, without needing to check anything, throw away all changes
• Thirdly merge in the remaining changes from BRANCH_V8.

In detail:

1. Switch your working directory to BRANCH_V9 : hg update BRANCH_V9
2. Merge in the last good state of BRANCH_V8 (ie. the commit you generated to fix the bad merge). This merge is a merge like any regular merge, ie. conflicts should be resolved as usual, and nothing needs to be thrown away.
   1. merge : hg merge 14
   2. commit : hg commit -m "Merging in last good state of BRANCH_V8" The situation is now :

      @    BRANCH_V9 - 17 - Merging in last good state of BRANCH_V8
      |\
      | | o    BRANCH_V8 - 16 - merged two heads used to revert from bad merge
      | | |\
      | +---o  BRANCH_V8 - 15 - throwing away wrong merge from BRANCH_V9
      | | | |
      | o | |  BRANCH_V8 - 14 - generating commit on BRANCH_V8 to rectify wrong merge from BRANCH_V9
      | | | |
      | | o |  BRANCH_V8 - 13 - important commit right after the bad merge
      | | |/
      +---o  BRANCH_V8 - 12 - wrong merge from BRANCH_V9
      | |/
      | o  BRANCH_V8 - 11 - adding comment on BRANCH_V8
      | |
      o |  BRANCH_V9 - 10 - last commit on BRANCH_V9
      | |
      

3. Merge in the bad merge on BRANCH_V8 + its fix, and throw away all changes :
   1. merge : hg merge 15
   2. revert all changes : hg revert -a --no-backup -r 17
   3. commit the merge : hg commit -m "Merging in bad merge from BRANCH_V8 and its fix and throwing it all away" Current situation :

      @    BRANCH_V9 - 18 - Merging in bad merge from BRANCH_V8 and its fix and throwing it all away
      |\
      | o    BRANCH_V9 - 17 - Merging in last good state of BRANCH_V8
      | |\
      +-----o  BRANCH_V8 - 16 - merged two heads used to revert from bad merge
      | | | |
      o---+ |  BRANCH_V8 - 15 - throwing away wrong merge from BRANCH_V9
      | | | |
      | | o |  BRANCH_V8 - 14 - generating commit on BRANCH_V8 to rectify wrong merge from BRANCH_V9
      | | | |
      +-----o  BRANCH_V8 - 13 - important commit right after the bad merge
      | | |
      o---+  BRANCH_V8 - 12 - wrong merge from BRANCH_V9
      |/ /
      | o  BRANCH_V8 - 11 - adding comment on BRANCH_V8
      | |
      o |  BRANCH_V9 - 10 - last commit on BRANCH_V9
      | |
      

4. Merge in the left-over changes from BRANCH_V8 :
   1. merge : hg merge BRANCH_V8
   2. commit : hg commit -m "merging changes from BRANCH_V8"

In the end the situation looks like this:

@    BRANCH_V9 - 19 - merging changes from BRANCH_V8
|\
| o    BRANCH_V9 - 18 - Merging in bad merge from BRANCH_V8 and its fix and throwing it all away
| |\
| | o    BRANCH_V9 - 17 - Merging in last good state of BRANCH_V8
| | |\
o | | |  BRANCH_V8 - 16 - merged two heads used to revert from bad merge
|\| | |
| o---+  BRANCH_V8 - 15 - throwing away wrong merge from BRANCH_V9
| | | |
| | | o  BRANCH_V8 - 14 - generating commit on BRANCH_V8 to rectify wrong merge from BRANCH_V9
| | | |
o | | |  BRANCH_V8 - 13 - important commit right after the bad merge
|/ / /
o---+  BRANCH_V8 - 12 - wrong merge from BRANCH_V9
|/ /
| o  BRANCH_V8 - 11 - adding comment on BRANCH_V8
| |
o |  BRANCH_V9 - 10 - last commit on BRANCH_V9
| |

After all these steps, in which you do not have to check any diff manually, BRANCH_V8 and BRANCH_V9 are correct, and future merges from BRANCH_V8 to BRANCH_V9 will be correct as well.

Solution 2

In a pinch, you can export the repository to a bunch diffs, edit history, and then glue back together just what you want - into a new repository, so no risk of damage. Probably not too bad for your example, but I don't know what the real history looks like.

I referenced this page while performing a simpler operation:

http://strongdynamic.blogspot.com/2007/08/expunging-problem-file-from-mercurial.html

Solution 3

OK, start by making a new empty repository in a separate directory from the broken repository (hg init). Now, pull up to and including the last known good version into the new repository; make sure you do not pull the bad merge and do pull everything before it. In the old repository, update to the last known good version of A, and do this:

hg graft r1 r2 r3

where r1-3 are changes made after the botched merge. You may get conflicts at this point; fix them.

This should produce new changes against the last known good version of A. Pull those new changes into the new repository. Just to double check that you didn't miss anything, do an hg incoming against the old repository. If you see anything other than the botched merge and r1-3, pull it.

Throw the old repository away. You're done. The merge isn't in the new repository at all and you never had to rewrite history.

Solution 4

So you want to merge just some changesets from B into A? Backing out changesets like you have been doing is a really bad idea as you have already suffered.

You should either use the transplant extension or have a third branch where you make common changes to merge into both A and B.

Solution 5

After much discussion with some of the helpful people on #mercurial on freenode, mpm has provided a partial solution that seems to work for my test case ( I generated a fake repository trying to replicate the scenario )

However, on my actual repository, for reasons I don't quite understand, it is still less than perfect.

Here is a diagram of the currently proposed way of resolving this problem:

[Original image lost]

Its now less of a problem to fix, but I'm still having to compare diffs ( ie: b46:b11 vs b46:b8 , a43:a10 vs a43:a9 ) and hand edit some changes back in.

Not closing this question/taking an answer until I get a guaranteed way that works on any repository.

Important

Anyone trying this stuff should clone their repository and play with it like a sandbox first. As you should be doing with any merge process, because that way if it goes wrong you can just throw it out and start again.

View more solutions

Comments

[Mike G.]

Ouch, that looks ugly... Did you also post this to selenic.com/mercurial/bts ? This sounds like an Hg bug that should be addressed.

[Kent Fredric]

Apologies to all, the image hosting went under :( and I can't find origianls

[Pete Duncanson]

Any news on this one just out of interest, after 2 years of development I'd like to think there might be better ways of handling this now? :)

[Sled]

Can it be understood that the 'A' branch is QA/Prod and 'B' is Main/Dev?

[Kent Fredric]

Not backing out changes because I /want/ to here ;), and the problem is no matter how many branches i create, its still possible to merge them the wrong way, causing this problem. ( trust me, been down that avenue )

[Kent Fredric]

Yeah, It would be an exhaustive process doing what you did, and it wouldn't necessarily work for my situation ( named branches work way odd ), and the history is a pigstye, when you have >4 committers all on their own schedule there is no linearity at all ;)

[oenli]

Basically the benefit of this method is that you generate a commit (in this case rev 15) which, when starting from the last good point on BRANCH_V8, contains only the bad merge and its fix. You can then separately merge this changeset into BRANCH_V9 and can safely ignore all changes that come in with that merge.

[sth]

Ahhhh there was a </pre> missing after the second-to-last code block. Took a while to find, but just when I was ready to give up... Now everything is displayed correctly, I think :)

[oenli]

Note that for Git the same technique can be used, so that you can fix the backwards merge without rewriting history, which causes headaches.

[Jesse Glick]

Beware that you might still need to discard BRANCH_V9 and recreate it (using transplant or import/export): Mercurial's merge algorithm does not really understand the intention behind backing out a change, so subsequent merges are likely to be garbled. mercurial.selenic.com/bts/issue1010 mercurial.selenic.com/bts/issue1740

[oenli]

Jesse : how is that ? If I make a change on BRANCH_V8 after all this, and then merge BRANCH_V8 to BRANCH_V9, only that one change will be merged, right ? Ie. all the other stuff (the bad merge and the fix-ups) are no longer relevant in future merges, right ?

[Harvey]

This is great. Thanks. Before doing any of this, @Kent should install a commit hook preventing backwards merges from being committed and/or pushed. Fix the root cause of the problem, then deal with the leftover symptoms.

[GuiSim]

@oenli could you elaborate why "BRANCH_V8 - 14 - generating commit on BRANCH_V8 to rectify wrong merge from BRANCH_V9" is really necessary?

[GuiSim]

I've created a Hg repository to demonstrate this solution (without the extra empty commit) bitbucket.org/janverley/wrongmerge