How do I clone a sub-folder of a repository in Mercurial?

Question

I have a Mercurial repository containing a handful of related projects. I want to branch just one of these projects to work on it elsewhere.

Is cloning just part of a repository possible, and is that the right way to achieve this?

Solution 1

What you want is a narrow or partial clone, but this is unfortunately not yet supported.

If you already have a big repository and you realize that it would make sense to split it into several smaller repositories, then you can use the convert extension to do a Mercurial to Mercurial conversion. Note that this creates a new repository foo and you cannot push/pull between your-big-repo and foo.

The convert extension is not enabled by default so add the following to your repo's hgrc file or your mercurial.ini file:

[extensions]
hgext.convert=

Then create a map.txt file with

include "libs/foo"
rename "libs/foo" .

(note you can use forward slashes even on Windows) and run

$ hg convert --filemap map.txt your-big-repo foo

That will make foo a repository with the full history of the libs/foo folder from your-big-repo.

If you want to delete all evidence of foo from your-big-repo you can make another conversion where you use exclude libs/foo to get rid of the directory.

When you have several repositories like that and you want to use them as a whole, then you should look at subrepositories. This feature lets you include other repositories in a checkout — similarly to how svn:externals work. Please follow the recommendations on that wiki page.

Solution 2

Instead of doing a partial clone, you can use the Convert Extension to split your repo into more than one repo by sub repository.

Specifically, see the section, Converting from Mercurial:

It's also useful to filter Mercurial repositories to get subsets of an existing one. For example to transform a subdirectory subfoo of a repository foo into a repository with its own life (while keeping its full history), do the following:

$ echo include subfoo > /tmp/myfilemap
$ echo rename subfoo . >> /tmp/myfilemap
$ hg convert --filemap /tmp/myfilemap /path/to/repo/foo /tmp/mysubfoo-repo

Solution 3

I've stumbled accross this issue and found one way to do it: Using symlinks (Linux only unfortunately)

For example, if you only need /project in the repository, on your computer clone the repo in another folder, then use ln -s /repo/location/ project. Mercurial will handle it

Solution 4

(Late 2016) Mainline Mercurial still doesn't package support for "narrow clones" but there are third party extensions that tackle the problem in different ways.

If you can cope with just a narrow checkout (aka "sparse checkout" or "partial checkout by file path") then Facebook's sparse.py extension from the hg-experimental repository (look inside the hgext3rd/ directory) may be workable. In this scenario, you still clone the full history (thus the .hg directory is no smaller) but your working directory only shows/acts on a subset of the full repository.

Alternatively Google have created a NarrowHG extension that does narrow cloning (aka "partial cloning by file path"). You will need to be in control of the server, the client and be willing to use experimental features but it really does restrict the clone's copied history in .hg to a subset of what was in the original repository.

(2019) The sparse extension was merged into Mercurial 4.3 as the experimental sparse extension. The NarrowHG extension was merged into Mercurial 4.6 as the hgext.narrow extension.

Solution 5

It is not possible, hg clone will clone the whole repository.

You can take a look a the sub-repository extension that allows you to have repositories inside a repository, which might match your needs.

View more solutions

Comments

[James Hopkin]

Thanks - I've found a workaround for what I need to do. Interesting to see the issues involved.

[Martin Geisler]

My guess is that it's because the solution doesn't really let you clone a subdirectory from a repository. Instead, it let's you convert an existing repository into a new and smaller repository that you can then clone.

[Patrick M]

1) As stated above you're not answering the question the original question asked. 2) You don't explain that what the original question asked is impossible. You just offer something different. 3) You give a link, instead of actually explaining.

[Matt]

Argh! "I have discovered a truly marvelous proof of this, which this margin is too narrow to contain."

[brandizzi]

@MartinGeisler just to note, while the convert solution can be named a bit differently from what one would call a partial clone, it allows one to do anything a partial clone would allow, including pushing to the remote repository and pulling from it AFAIK. I believe you know this but I am posting this note mostly for eventual future readers :)

[Martin Geisler]

@brandizzi: The converted repo will have new changeset IDs, so you cannot push back to the original. What you can do is to repeatedly convert the original repository and pull from the converted into your existing partial clones. That allows you to keep in sync with development in a big repo, but it is a one-way strategy where you need to manually (using, say, hg export) copy changes back to the original repo.

[brandizzi]

@MartinGeisler actually, you can! You just need to use the -f flag. See this script as an example. (Note: requires the convert and graphlog extensions.)

[Martin Geisler]

@brandizzi yes and no :) After hg push -f, you end up with not only multiple heads, but also multiple roots. This is because the root nodes of the two repositories differ and hg push -f will thus copy all data from your partial clone to the full clone. The --filemap argument should be f in your paste and after fixing that you get this output. The Created commit is duplicated and that is probably not how you want the main repository to look like.

[brandizzi]

@MartinGeisler fair enough, one has to have those shortcomings in mind! I was not aware of them, actually (well, I knew it would create multiple roots but it did not bother me). BTW, what would be the greatest practical problems on having multiple roots?

[Martin Geisler]

@brandizzi: your main repository is grossly affected by the "partial" clone and that's normally not acceptable for real repositories. You end up with duplicate data for all revisions touching the subdirectory you "partially" clone. You should write to [email protected] if you want to discuss this further, StackOverflow is not really meant for discussion.

[Bedros]

ok, if cloning of subdir is done, how would you merge changes back in the original directory?

[Anon]

@martin-geisler these days narrow and partial cloning are concepts that are part of Mercurial itself. I tried to update your comment but was told the change was "intended to address the author of the post and makes no sense as an edit" so hence this comment.