Detach (move) subdirectory into separate Git repository

294,656

Solution 1

Update: This process is so common, that the git team made it much simpler with a new tool, git subtree. See here: Detach (move) subdirectory into separate Git repository


You want to clone your repository and then use git filter-branch to mark everything but the subdirectory you want in your new repo to be garbage-collected.

  1. To clone your local repository:

    git clone /XYZ /ABC
    

    (Note: the repository will be cloned using hard-links, but that is not a problem since the hard-linked files will not be modified in themselves - new ones will be created.)

  2. Now, let us preserve the interesting branches which we want to rewrite as well, and then remove the origin to avoid pushing there and to make sure that old commits will not be referenced by the origin:

    cd /ABC
    for i in branch1 br2 br3; do git branch -t $i origin/$i; done
    git remote rm origin
    

    or for all remote branches:

    cd /ABC
    for i in $(git branch -r | sed "s/.*origin\///"); do git branch -t $i origin/$i; done
    git remote rm origin
    
  3. Now you might want to also remove tags which have no relation with the subproject; you can also do that later, but you might need to prune your repo again. I did not do so and got a WARNING: Ref 'refs/tags/v0.1' is unchanged for all tags (since they were all unrelated to the subproject); additionally, after removing such tags more space will be reclaimed. Apparently git filter-branch should be able to rewrite other tags, but I could not verify this. If you want to remove all tags, use git tag -l | xargs git tag -d.

  4. Then use filter-branch and reset to exclude the other files, so they can be pruned. Let's also add --tag-name-filter cat --prune-empty to remove empty commits and to rewrite tags (note that this will have to strip their signature):

    git filter-branch --tag-name-filter cat --prune-empty --subdirectory-filter ABC -- --all
    

    or alternatively, to only rewrite the HEAD branch and ignore tags and other branches:

    git filter-branch --tag-name-filter cat --prune-empty --subdirectory-filter ABC HEAD
    
  5. Then delete the backup reflogs so the space can be truly reclaimed (although now the operation is destructive)

    git reset --hard
    git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d
    git reflog expire --expire=now --all
    git gc --aggressive --prune=now
    

    and now you have a local git repository of the ABC sub-directory with all its history preserved.

Note: For most uses, git filter-branch should indeed have the added parameter -- --all. Yes that's really --space-- all. This needs to be the last parameters for the command. As Matli discovered, this keeps the project branches and tags included in the new repo.

Edit: various suggestions from comments below were incorporated to make sure, for instance, that the repository is actually shrunk (which was not always the case before).

Solution 2

The Easy Way™

It turns out that this is such a common and useful practice that the overlords of Git made it really easy, but you have to have a newer version of Git (>= 1.7.11 May 2012). See the appendix for how to install the latest Git. Also, there's a real-world example in the walkthrough below.

  1. Prepare the old repo

     cd <big-repo>
     git subtree split -P <name-of-folder> -b <name-of-new-branch>
    

Note: <name-of-folder> must NOT contain leading or trailing characters. For instance, the folder named subproject MUST be passed as subproject, NOT ./subproject/

Note for Windows users: When your folder depth is > 1, <name-of-folder> must have *nix style folder separator (/). For instance, the folder named path1\path2\subproject MUST be passed as path1/path2/subproject

  1. Create the new repo

     mkdir ~/<new-repo> && cd ~/<new-repo>
     git init
     git pull </path/to/big-repo> <name-of-new-branch>
    
  2. Link the new repo to GitHub or wherever

     git remote add origin <[email protected]:user/new-repo.git>
     git push -u origin master
    
  3. Cleanup inside <big-repo>, if desired

     git rm -rf <name-of-folder>
    

Note: This leaves all the historical references in the repository. See the Appendix below if you're actually concerned about having committed a password or you need to decreasing the file size of your .git folder.


Walkthrough

These are the same steps as above, but following my exact steps for my repository instead of using <meta-named-things>.

Here's a project I have for implementing JavaScript browser modules in node:

tree ~/node-browser-compat

node-browser-compat
├── ArrayBuffer
├── Audio
├── Blob
├── FormData
├── atob
├── btoa
├── location
└── navigator

I want to split out a single folder, btoa, into a separate Git repository

cd ~/node-browser-compat/
git subtree split -P btoa -b btoa-only

I now have a new branch, btoa-only, that only has commits for btoa and I want to create a new repository.

mkdir ~/btoa/ && cd ~/btoa/
git init
git pull ~/node-browser-compat btoa-only

Next, I create a new repo on GitHub or Bitbucket, or whatever and add it as the origin

git remote add origin [email protected]:node-browser-compat/btoa.git
git push -u origin master

Happy day!

Note: If you created a repo with a README.md, .gitignore and LICENSE, you will need to pull first:

git pull origin master
git push origin master

Lastly, I'll want to remove the folder from the bigger repo

git rm -rf btoa

Appendix

Latest Git on macOS

To get the latest version of Git using Homebrew:

brew install git

Latest Git on Ubuntu

sudo apt-get update
sudo apt-get install git
git --version

If that doesn't work (you have a very old version of Ubuntu), try

sudo add-apt-repository ppa:git-core/ppa
sudo apt-get update
sudo apt-get install git

If that still doesn't work, try

sudo chmod +x /usr/share/doc/git/contrib/subtree/git-subtree.sh
sudo ln -s \
/usr/share/doc/git/contrib/subtree/git-subtree.sh \
/usr/lib/git-core/git-subtree

Thanks to rui.araujo from the comments.

Clearing your history

By default removing files from Git doesn't actually remove them, it just commits that they aren't there anymore. If you want to actually remove the historical references (i.e. you committed a password), you need to do this:

git filter-branch --prune-empty --tree-filter 'rm -rf <name-of-folder>' HEAD

After that, you can check that your file or folder no longer shows up in the Git history at all

git log -- <name-of-folder> # should show nothing

However, you can't "push" deletes to GitHub and the like. If you try, you'll get an error and you'll have to git pull before you can git push - and then you're back to having everything in your history.

So if you want to delete history from the "origin" - meaning to delete it from GitHub, Bitbucket, etc - you'll need to delete the repo and re-push a pruned copy of the repo. But wait - there's more! - if you're really concerned about getting rid of a password or something like that you'll need to prune the backup (see below).

Making .git smaller

The aforementioned delete history command still leaves behind a bunch of backup files - because Git is all too kind in helping you to not ruin your repo by accident. It will eventually delete orphaned files over the days and months, but it leaves them there for a while in case you realize that you accidentally deleted something you didn't want to.

So if you really want to empty the trash to reduce the clone size of a repo immediately you have to do all of this really weird stuff:

rm -rf .git/refs/original/ && \
git reflog expire --all && \
git gc --aggressive --prune=now

git reflog expire --all --expire-unreachable=0
git repack -A -d
git prune

That said, I'd recommend not performing these steps unless you know that you need to - just in case you did prune the wrong subdirectory, y'know? The backup files shouldn't get cloned when you push the repo, they'll just be in your local copy.

Credit

Solution 3

Paul's answer creates a new repository containing /ABC, but does not remove /ABC from within /XYZ. The following command will remove /ABC from within /XYZ:

git filter-branch --tree-filter "rm -rf ABC" --prune-empty HEAD

Of course, test it in a 'clone --no-hardlinks' repository first, and follow it with the reset, gc and prune commands Paul lists.

Solution 4

I’ve found that in order to properly delete the old history from the new repository, you have to do a little more work after the filter-branch step.

  1. Do the clone and the filter:

    git clone --no-hardlinks foo bar; cd bar
    git filter-branch --subdirectory-filter subdir/you/want
    
  2. Remove every reference to the old history. “origin” was keeping track of your clone, and “original” is where filter-branch saves the old stuff:

    git remote rm origin
    git update-ref -d refs/original/refs/heads/master
    git reflog expire --expire=now --all
    
  3. Even now, your history might be stuck in a packfile that fsck won’t touch. Tear it to shreds, creating a new packfile and deleting the unused objects:

    git repack -ad
    

There is an explanation of this in the manual for filter-branch.

Solution 5

Edit: Bash script added.

The answers given here worked just partially for me; Lots of big files remained in the cache. What finally worked (after hours in #git on freenode):

git clone --no-hardlinks file:///SOURCE /tmp/blubb
cd blubb
git filter-branch --subdirectory-filter ./PATH_TO_EXTRACT  --prune-empty --tag-name-filter cat -- --all
git clone file:///tmp/blubb/ /tmp/blooh
cd /tmp/blooh
git reflog expire --expire=now --all
git repack -ad
git gc --prune=now

With the previous solutions, the repository size was around 100 MB. This one brought it down to 1.7 MB. Maybe it helps somebody :)


The following bash script automates the task:

!/bin/bash

if (( $# < 3 ))
then
    echo "Usage:   $0 </path/to/repo/> <directory/to/extract/> <newName>"
    echo
    echo "Example: $0 /Projects/42.git first/answer/ firstAnswer"
    exit 1
fi


clone=/tmp/${3}Clone
newN=/tmp/${3}

git clone --no-hardlinks file://$1 ${clone}
cd ${clone}

git filter-branch --subdirectory-filter $2  --prune-empty --tag-name-filter cat -- --all

git clone file://${clone} ${newN}
cd ${newN}

git reflog expire --expire=now --all
git repack -ad
git gc --prune=now
Share:
294,656
matli
Author by

matli

Updated on July 17, 2022

Comments

  • matli
    matli over 1 year

    I have a Git repository which contains a number of subdirectories. Now I have found that one of the subdirectories is unrelated to the other and should be detached to a separate repository.

    How can I do this while keeping the history of the files within the subdirectory?

    I guess I could make a clone and remove the unwanted parts of each clone, but I suppose this would give me the complete tree when checking out an older revision etc. This might be acceptable, but I would prefer to be able to pretend that the two repositories doesn't have a shared history.

    Just to make it clear, I have the following structure:

    XYZ/
        .git/
        XY1/
        ABC/
        XY2/
    

    But I would like this instead:

    XYZ/
        .git/
        XY1/
        XY2/
    ABC/
        .git/
        ABC/
    
    • jeremyjjbrown
      jeremyjjbrown over 9 years
      This is trivial now with git filter-branch see my answer below.
    • Agnel Kurian
      Agnel Kurian about 9 years
      @jeremyjjbrown is right. This is no longer difficult to do but it is difficult to find the right answer on Google because all the old answers dominate the results.
    • djvg
      djvg about 2 years
      Use of git filter-branch is discouraged. See warning in docs.
  • Paul
    Paul almost 15 years
    Good point! That would apply to most people using git filter-branch this way. In my case, I was segregating a library which had its own series of release numbers in tags, so I didn't want the project tags in my new repo. I'll edit that into the answer.
  • Osman
    Osman over 14 years
    As of today, filter-branch is not supported on Windows. It looks like it is coming soon though. Check the msysgit discussion group (at google groups) for details.
  • vdboor
    vdboor almost 14 years
    Why do you need --no-hardlinks? Removing one hardlink won't affect the other file. Git objects are immutable too. Only if you'd change owner/file permissions you need --no-hardlinks.
  • Richard Michael
    Richard Michael over 13 years
    git-filter-branch set_ident() calls LANG=C LC_ALL=C sed ... for maximum compatibility with non UTF-8 aware sed. Therefore, git-filter-branch dies on commits with UTF-8 characters in the author: or committer: fields (see git show --pretty=raw <commit SHA1>). A workaround, use a UTF-8 aware sed (e.g. GNU sed) and edit git-filter-branch: LANG=en_US.UTF-8 sed -ne ...
  • Evgeny
    Evgeny about 13 years
    in some cases messing up the history of repository XYZ is overkill ... just a simple "rm -rf ABC; git rm -r ABC; git commit -m'extracted ABC into its own repo'" would work better for most people.
  • Vi.
    Vi. almost 13 years
    Also the not-yet-included 'git filter-branch --split-submodule' can be used to preserve consistent history. vi-server.org/vi/bin/git-extract-submodule.patch
  • Brian Carlton
    Brian Carlton over 12 years
    You probably wish to use -f (force) on this command if you do it more than once, e.g., to remove two directories after they have been separated. Otherwise you will get "Cannot create a new backup."
  • jonp
    jonp over 12 years
    I find that if the /XYZ repo is large, the /ABC repo won't be as small as desirable. To further shrink things: git clone /ABC /ABC.small ; cd /ABC.small ; git repack -A -d ; git gc --prune=DATE where DATE is date +%Y-%m-%d. For some reason if I don't git clone and work on the clone, nothing shrinks. :-/
  • Malcolm Box
    Malcolm Box about 12 years
    And if you want to rewrite your tags to not reference the old structure, add --tag-name-filter cat
  • Eric Naeseth
    Eric Naeseth about 12 years
    If you're doing the --index-filter method, you may also want to make that git rm -q -r -f, so that each invocation won't print a line for each file it deletes.
  • saltycrane
    saltycrane about 12 years
    Like Paul, I did not want project tags in my new repo, so I did not use -- --all. I also ran git remote rm origin, and git tag -l | xargs git tag -d before the git filter-branch command. This shrunk my .git directory from 60M to ~300K. Note that I needed to run both of these commands to in order to get the size reduction.
  • Blaisorblade
    Blaisorblade almost 12 years
    The git man page recommends, instead of rm -rf .git/refs/original/, git for-each-ref --format="%(refname)" refs/original/ | xargs -n 1 git update-ref -d; I guess the latter is more robust if refs are not stored in the right place. Moreover, I believe that 'git remote rm origin' is also needed to shrink the repo, otherwise the refs from origin will keep objects referenced. @jonp, I think that was the problem for you. Finally, to also rewrite other branches, one must set them up manually with git branch after cloninng, -- --all and remove HEAD (which stops rewriting of other branches).
  • Blaisorblade
    Blaisorblade almost 12 years
    For many readers (including me), the answer by itself was quite imprecise. Therefore, I incorporated most of the suggestions from comments after testing them. In particular, I made sure that after following this procedure, the repository is actually shrunk (which was not the case for me and various other people). I hope that is OK.
  • KindDragon
    KindDragon over 11 years
    To remove references in Windows you can use: for /f %a IN ('git for-each-ref --format="%(refname)" refs/original/') DO git update-ref -d %a
  • pix0r
    pix0r over 11 years
    I found git-subtree to be effective as well, and much simpler. Found via this related question/answer: stackoverflow.com/a/1307969/72
  • Ole Lynge
    Ole Lynge over 11 years
    On Windows, I first tried the steps from the powershell prompt, but step 5b failed. Then I re-tried all steps from the git bash, and everything worked perfectly. Thanks!
  • Albert
    Albert over 11 years
    I think somethink like git gc --aggressive --prune=now is still missing, isn't it?
  • MaddTheSane
    MaddTheSane over 11 years
    @Albert The repack command takes care of that, and there wouldn’t be any loose objects.
  • Vladimir Grigorov
    Vladimir Grigorov about 11 years
    From the 5 or so similar questions, this is the most valuable answer for me together with this link: gbayer.com/development/…
  • Kos
    Kos almost 11 years
    @Jarl has suggested to edit and remove the --aggressive option to gc "as it makes the size of the repo explode: see also link"
  • eddiemoya
    eddiemoya almost 11 years
    One important note is that if you have multiple branches being preserved in Step #2, then you'll have to run Step #4 and Step #5 through each one. Or at least thats what happened for me.
  • Slipp D. Thompson
    Slipp D. Thompson over 10 years
    I like the style of that graph. May I ask what tool you're using?
  • Tomek Wyderka
    Tomek Wyderka over 10 years
    yeah, git gc --aggressive --prune=now reduced much of new repo
  • MM.
    MM. over 10 years
    Tower for Mac. I really like it. It's almost worth switching to Mac for in itself.
  • echristopherson
    echristopherson over 10 years
    git-subtree is now part of Git, although it's in the contrib tree, so not always installed by default. I know it is installed by the Homebrew git formula, but without its man page. apenwarr thus calls his version obsolete.
  • matbrgz
    matbrgz over 10 years
    Does this not create ABC/ instead of ABC/ABC/?
  • Jay Allen
    Jay Allen over 10 years
    Yep, though in my case, my subfoldered targetdir had been renamed at some point and git filter-branch simply called it a day, deleting all commits made prior to the rename! Shocking, considering how adept Git is at tracking such things and even migration of individual content chunks!
  • Jay Allen
    Jay Allen over 10 years
    Oh, also, if anyone finds themselves in the same boat, here's the command I used. Don't forget that git rm takes multiple args, so there's no reason to run it for each file/folder: BYEBYE="dir/subdir2 dir2 file1 dir/file2"; git filter-branch -f --index-filter "git rm -q -r -f --cached --ignore-unmatch $BYEBYE" --prune-empty -- --all
  • katyhuff
    katyhuff over 10 years
    So, I've probably used this answer 10 times now and I keep wanting to upvote it again. Tons of stuff on stackoverflow has been useful, but this answer is the most useful so far. I've even put it in evernote for super quick referencing. Thanks!
  • ThanksBro
    ThanksBro over 10 years
    How to use this if you want to take 2 directories?
  • onionjake
    onionjake over 10 years
    git subtree is still part of the 'contrib' folder and isn't installed by default on all distros. github.com/git/git/blob/master/contrib/subtree
  • krlmlr
    krlmlr over 10 years
    ...in particular, not on Ubuntu, even not when using ppa:git-core/ppa :-(
  • rui.araujo
    rui.araujo over 10 years
    @krlmlr sudo chmod +x /usr/share/doc/git/contrib/subtree/git-subtree.sh sudo ln -s /usr/share/doc/git/contrib/subtree/git-subtree.sh /usr/lib/git-core/git-subtree To activate on Ubuntu 13.04
  • krlmlr
    krlmlr over 10 years
    @rui.araujo: I ended up with creating a small shell script in my ~/bin directory that sources git-subtree.sh: gist.github.com/6338779. Of course this has to be chmod +x-d, too,
  • Miles Rout
    Miles Rout about 10 years
    If you have pushed a password to a public repository, you should change the password, not try to remove it from the public repo and hope nobody saw it.
  • erbridge
    erbridge about 10 years
    You can override the push error when pushing to GitHub by doing git push -f to force the overwriting.
  • elaRosca
    elaRosca about 10 years
    @rui.araujo your comment is very useful and makes the answer complete. It would be good if it was added to the actual answer.
  • ysth
    ysth almost 10 years
    N.B. this leaves the new repo structured as /ABC/ABC, instead of just /ABC (though someone edited the question after the fact to say that was what was wanted)
  • Tola Odejayi
    Tola Odejayi almost 10 years
    This doesn't work for me. When I call git pull </path/to/big-repo> <name-of-new-branch>, I get an error: Couldn't find remote ref <name-of-new-branch>.
  • Marcello de Sales
    Marcello de Sales almost 10 years
    This really helped me with a lot of repositories, specially using svn2git and then splitting the repository into multiple Git repos... Look at github.com/marcellodesales/svnedge-* for examples.
  • Memming
    Memming almost 10 years
    according to kernel.org/pub/software/scm/git/docs/git-filter-branch.html for git filter-branch, using git rm instead of rm is faster.
  • Erik Aronesty
    Erik Aronesty over 9 years
    I would suggest editing Paul's answer, only because Paul's is so thorough.
  • Christian Fritz
    Christian Fritz over 9 years
    on ubuntu 13.10 you can find it in the specified location: /usr/share/doc/git/contrib/subtree/git-subtree.sh (without having to use the ppa).
  • rmoestl
    rmoestl over 9 years
    When cleaning up, shouldn't the branch created with git subtree be deleted as well?
  • Adam
    Adam over 9 years
    @Paul - it should be noted that this approach fails if the ABC folder ever had a different name - in this case, the history stops at the point when it was renamed.
  • Adam
    Adam over 9 years
    What about if we have a subdirectory which was previously renamed? I tried this on a <name-of-folder> which at some point in time was renamed from <another-name-of-folder>. When I look at the history in <new-repo>, the initial commit is the one which renamed the folder - all of the history before that is erased from <new-repo>
  • Toby
    Toby over 9 years
    That didn't reclaim disk space for me, the .git folder must have a bunch of unused history still in there as the pruned folder is 10s of KB yet the .git is 1.5 GB! Any ideas?
  • orezvani
    orezvani over 9 years
    After I used filter-branch, all of the files in that directory were removed.
  • András Aszódi
    András Aszódi about 9 years
    re "latest git on OS X": there is no need to mess with brew or any other extra package manager because the XCode command line tools include Git. Available from Apple free of charge, installs/updates without any hassle.
  • woojoo666
    woojoo666 about 9 years
    this seems to make a new repo with the contents of ABC/, but the new repo doesn't contain the folder ABC/ itself, as the question asked. How would you do this?
  • coolaj86
    coolaj86 about 9 years
    In the example it looks like it's just foo names to me. xyz, xyz1, xyz2, etc. Anyway mkdir ABC/; git mv ./* ABC/. I'm sure there's some way to give a prefix - maybe git read-tree --prefix=spoon-knife/ -u spoon-knife/master. See help.github.com/articles/about-git-subtree-merges.
  • Bluu
    Bluu about 9 years
    To pull the split subtree into a folder: git subtree add --prefix <new/folder/path> </path/to/big-repo> <name-of-new-branch>
  • qAp
    qAp about 9 years
    Really useful this is. However, if /ABC used to be called /XY3 until 3 commits before HEAD, will the resulting ABC repository not only have these 3 commits? At least this is what I have found.
  • Sean
    Sean almost 9 years
    These answers are a bit confusing... Are the five steps in this answer now N/A? Is the "The Easy Way" answer below the right answer? When I first saw this answer, I figured the 'Detach subdirectory into separate Git repository' link was to more in depth documentation and was summarized by the five steps that followed. Can this be clarified? Perhaps add a header above the five steps, like "The Old Way (pre v1.7.11).
  • Umair A.
    Umair A. almost 9 years
    @Bluu I tried this on an empty repo and it didn't work. But I added my first commit and after that your solution worked for me.
  • J.T. Taylor
    J.T. Taylor over 8 years
    This worked like a charm. YOUR_SUBDIR in the above example is the subdirectory that you want to KEEP, everything else will be removed
  • jeremyjjbrown
    jeremyjjbrown over 8 years
    Updates based on you comment.
  • M.M
    M.M over 8 years
    An advantage of this method compared to "The Easy Way" is that the remote is already set up for the new repo, so you can immediately do a subtree add. In fact this way seems easier to me (even without git splits)
  • Anthony O.
    Anthony O. over 8 years
    Is there a way to apply this method in order to split multiple sub folders into a new git repository?
  • Anthony O.
    Anthony O. over 8 years
    What I finally did to split multiple sub folders into a new project was in 1. before the subtree split, move the sub folders I wanted into a unique sub-folder, rewriting history to keep it when pulling (step 2.): git filter-branch --tree-filter "mkdir <name-of-folder>; mv sub1 sub2 <name-of-folder>/" HEAD
  • Pierre Henry
    Pierre Henry over 8 years
    @AnthonyO. Could you elaborate on your method ? Or even write an answer ? I am stuck here and would upvote it for sure.
  • Pierre Henry
    Pierre Henry over 8 years
    I finally used the solution given in stackoverflow.com/questions/19954485/… it works well for me so far.
  • Jason
    Jason over 8 years
    Hey @AnthonyO. I am trying something similar to what you wrote to move multiple sub-directories, but its failing for me. The error I get is rename sub1 to folder/sub1: No such file or directory and same for the second directory. Am I missing something ?
  • Anthony O.
    Anthony O. over 8 years
    @Jason & PierreHenry : I've written what I did in a separated answer. Here it is: stackoverflow.com/a/31859803/535203
  • Jason
    Jason over 8 years
    Hey @AnthonyO. I also asked a similar question here: Splitting many subdirectories into a new, separate Git repository and got it to work. But thanks, I will try out your answer too.
  • JaviMerino
    JaviMerino over 8 years
    It's probably a good idea to pass --prune-empty to filter-branch to remove the empty commits that happen due to removing the directory
  • cowboydan
    cowboydan over 8 years
    As a corollary to this very useful answer - should anyone else use the subtree approach, be advised to manually split-out the .gitattributes file! I fell victim to this and got a CRLF/LF warning after adding a single ' ' to a file.
  • Jarl
    Jarl about 8 years
    As far as I understand with this method (git subtree) you will loose both branches and tags. Using git filter-branch-method as described in stackoverflow.com/a/359759/300632 it is possible to keep/rewrite branches and tags.
  • Cœur
    Cœur about 8 years
    This solution doesn't preserve history.
  • KrisWebDev
    KrisWebDev about 8 years
    If your objective is to link the new repository as a submodule of the original project, then you need this answer (Usage §) of "How to extract a git subdirectory and make a submodule out of it?".
  • Joe Leo
    Joe Leo almost 8 years
    If at some point in the past you ran git subtree, you may get a fatal: bad object error when running the command again. Avoid this by passing in the --ignore-joins option.
  • pglezen
    pglezen almost 8 years
    This worked for me with slight modification. Because my sub1 and sub2 folders didn't exist with the initial version, I had to modify my --tree-filter script as follows: "mkdir <name-of-folder>; if [ -d sub1 ]; then mv <sub1> <name-of-folder>/; fi". For the second filter-branch command I replaced <sub1> with <sub2>, omitted creation of <name-of-folder>, and included -f after filter-branch to override the warning of an existing backup.
  • pglezen
    pglezen almost 8 years
    The "fine and wonderful" part of subtree is that your subdirectory's history comes along for the ride. If you don't need the history, then your painfully easy method is the way to go.
  • nietras
    nietras over 7 years
    This does not work if any of the subdirs have changed during the history in git. How can this be solved?
  • Chris Dodd
    Chris Dodd over 7 years
    This method turns out to be essentially useless -- if you follow it you lose ALL branches other than master, and all the history that may be in those branches.
  • Marco Pelegrini
    Marco Pelegrini about 7 years
    Simple and elegant. Thanks!
  • rogerdpack
    rogerdpack about 7 years
    what is cat here?
  • VelLes
    VelLes about 7 years
    Github wiki with the same steps help.github.com/articles/…
  • NicBright
    NicBright over 6 years
    This does not answer the question. From the docs it says The result will contain that directory (and only that) as its project root. and indeed this is what you will get, i.e. the original project structure is not preserved.
  • Adam
    Adam about 6 years
    @NicBright Can you illustrate your issue with XYZ and ABC as in the question, to show what's wrong?
  • Adam
    Adam about 6 years
    Nice post, but I notice the first paragraph of the doc you linked says If you create a new clone of the repository, you won't lose any of your Git history or changes when you split a folder into a separate repository. Yet according to comments on all the answers here both filter-branch and the subtree script result in the loss of history wherever a subdirectory has been renamed. Is there anything that can be done to address this?
  • Adam
    Adam about 6 years
    See rogerdpack's answer if you ever did any directory renames/moves in your commits which you wish to preserve. The filter-branch and subtree methods will not keep truncate commits at directory renames.
  • Adam
    Adam about 6 years
    @nietras see rogerdpack's answer. Took me a while to find it after reading and absorbing all the info in these other answers.
  • Adam
    Adam about 6 years
    Found the needle in the git haystack! Now I can keep ALL my commit history.
  • Adam
    Adam about 6 years
    Found the solution for preserving all commits, including those preceding directory renames/moves - it's rogerdpack's answer to this very question.
  • ricardoespsanto
    ricardoespsanto almost 6 years
    Props to AndrewD for posting this solution. I have forked his repo to make it work on OSX (github.com/ricardoespsanto/git-splits) if that's useful to anyone else
  • Qiulang
    Qiulang over 5 years
    @jeremyjjbrown is it possible to reuse the cloned repo and not use a new repo, i.e. my question here stackoverflow.com/questions/49269602/…
  • Qiulang
    Qiulang over 5 years
    The only problem is that I can't use the cloned repo any more
  • Kelly Elton
    Kelly Elton over 5 years
    The special directory path requirements highlight the big problem of git being internally inconsistent sometimes, which makes it that much more obtuse.
  • Shilpa
    Shilpa over 5 years
    Also, instead of cloning a repo in the first place, you can directly add a submodule which adds the new repo to git and creates the new repo into your current repo as well !
  • jones77
    jones77 over 5 years
    The popd and pushd command make this rather implicit and harder to grok what it intends to do ...
  • ZettaCircl
    ZettaCircl over 4 years
    Nearly perfect ... but you forgot "git filter-branch --prune-empty" to remove all old commits that are now empty. To do before push to origin master !
  • ZettaCircl
    ZettaCircl over 4 years
    If you made the mistake and still want to "repush" after having removed old empty commit, perform : "git push -u origin master --force-with-lease"
  • VonC
    VonC about 4 years
    Upvoted for recommending the new filter-repo tool (which I presented last month in stackoverflow.com/a/58251653/6309)
  • Daniel Schilling
    Daniel Schilling almost 4 years
    The git subtree split command is much slower than the filter-branch approach.
  • Daniel
    Daniel almost 4 years
    On CentOS 7, do sudo yum install git-subtree to get the subtree command.
  • Jeremy Caney
    Jeremy Caney almost 4 years
    Using git-filter-repo should definitely be the preferred approach at this point. It’s much, much faster and safer than git-filter-branch, and safeguards against a lot of the gotchas one can run into when rewriting one’s git history. Hopefully this answer gets some more attention, since it’s the one to address git-filter-repo.
  • YaP
    YaP almost 4 years
    what about if your initial repository has many branches? this solution just preserves the master branch
  • jmon12
    jmon12 over 3 years
    Using the git subtree approach describe here, is there a way to preserve existing branches (and tags)? Since the whole relevant history is kept, that should be about restoring a pointer to a given commit?
  • AaA
    AaA about 3 years
    And after all this I'm still getting the same error that I was getting before. fatal: packed object xxxxxx (stored in .git/objects/pack/pack-yyyyyyyy.pack) is corrupt
  • Juan Saravia
    Juan Saravia about 3 years
    github.com/newren/git-filter-repo is the suggested way by git itself to easily do this.
  • bego
    bego over 2 years
    acutally I am currently trying to get things to work with git filter-repo but unfortunately after running it, I am missing files, which were added in a commit, containing a path that was removed by filter-repo. For example: Foo/ Foo.cs Bar/ Bar.cs All were added in the same commit. I want to move Foo and Bar in separate repos. So I cloned my repo in a folder matching the new repo name and did git filter-repo -path Foo Foo get's removed too. I am talking about a much bigger repo and it is working for every other file but not if it's a constellation like this.
  • Joel Leach
    Joel Leach over 2 years
    If files were previously moved/renamed, this will not automatically retain the history before the move/rename. However, if you include the original paths/filenames in the command, that history will not be removed. For example, git filter-repo --path CurrentPathAfterRename --path OldPathBeforeRename. git filter-repo --analyze produces a file renames.txt that can be helpful in determining these. Alternatively, you may find a script like this helpful.
  • vimterd
    vimterd over 2 years
    How would you keep the btoa folder in the new repository?
  • balu
    balu over 1 year
    For those coming here in 2022: A very good (probably better) alternative to the answer provided here, based on git filter-repo, can be found further down in stackoverflow.com/questions/359424/…