How do you fix a bad merge, and replay your good commits onto a fixed merge?

Question

I accidentally committed an unwanted file (filename.orig while resolving a merge) to my repository several commits ago, without me noticing it until now. I want to completely delete the file from the repository history.

Is it possible to rewrite the change history such that filename.orig was never added to the repository in the first place?

Solution 1

Please don't use this recipe if your situation is not the one described in the question. This recipe is for fixing a bad merge, and replaying your good commits onto a fixed merge.

Although filter-branch will do what you want, it is quite a complex command and I would probably choose to do this with git rebase. It's probably a personal preference. filter-branch can do it in a single, slightly more complex command, whereas the rebase solution is performing the equivalent logical operations one step at a time.

Try the following recipe:

# create and check out a temporary branch at the location of the bad merge
git checkout -b tmpfix <sha1-of-merge>

# remove the incorrectly added file
git rm somefile.orig

# commit the amended merge
git commit --amend

# go back to the master branch
git checkout master

# replant the master branch onto the corrected merge
git rebase tmpfix

# delete the temporary branch
git branch -d tmpfix

(Note that you don't actually need a temporary branch, you can do this with a 'detached HEAD', but you need to take a note of the commit id generated by the git commit --amend step to supply to the git rebase command rather than using the temporary branch name.)

Solution 2

If you haven't committed anything since, just git rm the file and git commit --amend.

If you have

git filter-branch \
--index-filter 'git rm --cached --ignore-unmatch path/to/file/filename.orig' merge-point..HEAD

will go through each change from merge-point to HEAD, delete filename.orig and rewrite the change. Using --ignore-unmatch means the command won't fail if for some reason filename.orig is missing from a change. That's the recommended way from the Examples section in the git-filter-branch man page.

Note for Windows users: The file path must use forward slashes

Solution 3

This is the best way:
http://github.com/guides/completely-remove-a-file-from-all-revisions

Just be sure to backup the copies of the files first.

EDIT

The edit by Neon got unfortunately rejected during review.
See Neons post below, it might contain useful information!

---

E.g. to remove all *.gz files accidentally committed into git repository:

$ du -sh .git ==> e.g. 100M
$ git filter-branch --index-filter 'git rm --cached --ignore-unmatch *.gz' HEAD
$ git push origin master --force
$ rm -rf .git/refs/original/
$ git reflog expire --expire=now --all
$ git gc --prune=now
$ git gc --aggressive --prune=now

That still didn't work for me? (I am currently at git version 1.7.6.1)

$ du -sh .git ==> e.g. 100M

Not sure why, since I only had ONE master branch. Anyways, I finally got my git repo truely cleaned up by pushing into a new empty and bare git repository, e.g.

$ git init --bare /path/to/newcleanrepo.git
$ git push /path/to/newcleanrepo.git master
$ du -sh /path/to/newcleanrepo.git ==> e.g. 5M 

(yes!)

Then I clone that to a new directory and moved over it's .git folder into this one. e.g.

$ mv .git ../large_dot_git
$ git clone /path/to/newcleanrepo.git ../tmpdir
$ mv ../tmpdir/.git .
$ du -sh .git ==> e.g. 5M 

(yeah! finally cleaned up!)

After verifying that all is well, then you can delete the ../large_dot_git and ../tmpdir directories (maybe in a couple weeks or month from now, just in case...)

Solution 4

Rewriting Git history demands changing all the affected commit ids, and so everyone who's working on the project will need to delete their old copies of the repo, and do a fresh clone after you've cleaned the history. The more people it inconveniences, the more you need a good reason to do it - your superfluous file isn't really causing a problem, but if only you are working on the project, you might as well clean up the Git history if you want to!

To make it as easy as possible, I'd recommend using the BFG Repo-Cleaner, a simpler, faster alternative to git-filter-branch specifically designed for removing files from Git history. One way in which it makes your life easier here is that it actually handles all refs by default (all tags, branches, etc) but it's also 10 - 50x faster.

You should carefully follow the steps here: http://rtyley.github.com/bfg-repo-cleaner/#usage - but the core bit is just this: download the BFG jar (requires Java 6 or above) and run this command:

$ java -jar bfg.jar --delete-files filename.orig my-repo.git

Your entire repository history will be scanned, and any file named filename.orig (that's not in your latest commit) will be removed. This is considerably easier than using git-filter-branch to do the same thing!

Full disclosure: I'm the author of the BFG Repo-Cleaner.

Solution 5

You should probably clone your repository first.

Remove your file from all branches history:
git filter-branch --tree-filter 'rm -f filename.orig' -- --all

Remove your file just from the current branch:
git filter-branch --tree-filter 'rm -f filename.orig' -- --HEAD    

Lastly you should run to remove empty commits:
git filter-branch -f --prune-empty -- --all

View more solutions

Comments

[CB Bailey]

If the file has been added to a commit then this doesn't even remove the file from the index, it just resets the index to the HEAD version of the file.

[atomicules]

Thanks! git filter-branch worked for me where the rebase example given as an answer didn't: The steps seemed to work, but then pushing failed. Did a pull, then pushed successfully, but the file was still around. Tried to redo the rebase steps and then it went all messy with merge conflicts. I used a slightly different filter-branch command though, the "An Improved Method" one given here: github.com/guides/completely-remove-a-file-from-all-revision‌​s git filter-branch -f --index-filter 'git update-index --remove filename' <introduction-revision-sha1>..HEAD

[Wernight]

Wouldn't a git rebase -i be faster and still as easy? $ git rebase -i <sh1-of-merge> Mark the correct one as "edit" $ git rm somefile.orig $ git commit --amend $ git rebase --continue However for some reason I still have that file somewhere the last time I did that. Probably missing something.

[Wernight]

I'm not sure which one is the improved method. Git official documentation of git-filter-branch seem to give the first one.

[CB Bailey]

git rebase -i is very useful, especially when you have multiple rebase-y operations to perform, but it's a right pain to describe accurately when you're not actually pointing over someone's shoulder and can see what they're doing with their editor. I use vim, but not everyone would be happy with: "ggjcesquash<Esc>jddjp:wq" and instructions like "Move the top line to after the current second line and change the first word on line four to 'edit' now save and quit" quickly seem more complex than the actual steps are. You normally end up with some --amend and --continue actions, as well.

[Admin]

I did this but a new commit was reapplied on top of the amended one, with the same message. Apparently git did a 3 way merge between the old, unamended commit containing the unwanted file, and the fixed commit from the other branch, and so it created a new commit on top of the old one, to re-apply the file.

[HiQ CJ]

I tried to do this when I realized an OS X .DS_Store was added but 1) the commit where it was added was duplicated by the rebase 2) Even with a rebase --onto (already complicating the recipe), of course more recent versions had a changed .DS_Store, so filter-branch ended up doing the trick for me

[CB Bailey]

@UncleCJ: Was your file added in a merge commit? This is important. This recipe is designed to cope with a bad merge commit. It's not going to work if your unwanted file was added in a normal commit in history.

[cregox]

I'm amazed how I could do all this using smartgit and no terminal at all! Thanks for the recipe!

[duma]

@CharlesBailey I used your instructions to alter the history of a local branch (i.e. my files didn't come from a merge) and it worked great. FYI.

[leontalbot]

Check out zyxware.com/articles/4027/… I find it the most complete and straight forward solution that involves filter-branch

[mikemaccana]

This is an excellent tool: a single command, it produces very clear output and provides a log file that matches every old commit to the new one. I don't like installing Java but this is worth it.

[aliteralmind]

FYI: Doing this for large files, the rebase can take quite a while. Looks frozen, but it eventually works.

[aliteralmind]

This was way more involved than I expected. I ended up having to repeatedly delete the file (in each commit) then git add --all :/ followed by git rebase --continue (as it climbed its way through each commit). If that didn't work, then git rebase --skip. Then finally, manually fix the <<<<< conflicts I found in the files. It seems to have taken, but the .git folder is still enormous. I'm not totally sure what I even did, to be honest, but it seems a step in the right direction. (I just realized I deleted the files via Windows Explorer, NOT with git rm. Not sure the consequences...)

[BingsF]

Thanks! I would suggest just one improvement: --prune-empty; this option to git filter-branch will remove any commits that only touched the file you want to remove (since after removal, they would just be empty commits)

[sol0mka]

@atomicules, if you will try to push the local repo to the remote one, git will insist on pulling from the remote first, because it has changes that you don't have locally. You can use --force flag to push to the remote - it will remove the files from there entirely. But be careful tho, make sure you won't force overwrite something other than the files only.

[sudo]

I accidentally added a HUGE file one day. I ended up just nuking my .git dir, re-adding files, and starting the git repo fresh.

[c z]

Remember to use " and not ' when using Windows, or you'll get an unhelpfully phrased "bad revision" error.

[Kevin LaBranche]

This is the only thing that worked for me but that's like because I wasn't working git filter-branch correctly. :-)

[Shadi]

This worked for me before the "That still didn't work for me?" comment

[ideasman42]

Great answer, but suggest adding --prune-empty to filter-branch command.

[Rolf]

This is pretty cool. I never fully understood what git rebase does. Now I do! Thank you.

[Cameron Lowell Palmer]

While all of the answers seem to be on the filter-branch track, this one highlights how to clean ALL branches in your history.

[avmohan]

git reflog expire --expire=now --all; git gc --prune=now is a very bad thing to do. Unless you're running out of disk space, let git garbage collect these commits after a few weeks

[clarkttfu]

Thanks for pointing that out. My repo was submitted with many large binary files and the repo is backed up entirely every night. So I just wanted every bit out of it ;)

[mbx]

"is the way to go" - It no longer is - even the docu says you should use git filter-repo instead