Can I clone part of a Mercurial repository?

Question

Is it possible to clone part of a Mercurial repository? Let's say the repository is quite large, or contains multiple projects, or multiple branches. Can I clone only part of the repository?

E.g. in Subversion, you might have trunk and branches. If I only want to get trunk (or one of the branches) I can just request [project]/trunk. If I clone the hg repo I'll get trunk and all of the branches. This might be a lot of information I don't want. Can I avoid getting this?

Alternatively, if I want to have multiple projects in one hg repo, how should I do this? I.e. so that I might just get one of the projects and ignore the others.

Solution 1

Yes you can. I'm sure you've moved on, but for the sake of those who will wander here later, I followed the docs at ConvertExtension, and wrote a simple batch script:

@echo off
echo Converting %1
REM Create the file map
echo include %1 > ~myfilemap               
echo rename %1 . >> ~myfilemap 
REM Run the convert process
hg convert --filemap ~myfilemap .\ ..\%1   
REM Delete the file map
del ~myfilemap                             
cd ..\%1
REM update the new repo--to create the files
hg update                                  

Name it something like split.cmd, and put it in the directory for the repo you want to split. Say for example you have C:\repos\ReallyBigProject, and a subfolder is C:\repos\ReallyBigProject\small-project. At the command prompt, run:

cd\repos\ReallyBigProject
split.cmd small-project

This will create C:\repos\small-project with a slice of the relevant history of revisions from the larger project.

The convert is not enabled by default. You'll need to make sure the following lines exist in your .hg\hgrc file (c:\repos\ReallyBigProject\.hg\hgrc in my example):

[extensions]
hgext.convert=

Solution 2

To my knowledge, that's not possible. But compared to Subversrion, cloning the whole repos may not be slower than just a branch from SVN.

Quoting from UnderstandingMercurial:

Many SVN/CVS users expect to host related projects together in one repository. This is really not what hg was made for, so you should try a different way of working. This especially means, that you cannot check out only one directory of a repository.

If you absolutely need to host multiple projects in a kind of meta-repository though, you could try the Subrepositories feature that was introduced with Mercurial 1.3 or the older ForestExtension.

Solution 3

@Nick

"E.g. in Subversion, you might have trunk and branches. If I only want to get trunk (or one of the branches) I can just request [project]/trunk. If I clone the hg repo I'll get trunk and all of the branches. This might be a lot of information I don't want. Can I avoid getting this?"

Absolutely. Just use hg clone -r <branch> and get only the branch you want. If you have lots of branches, you need a -r <branch> for each one. <branch> doesn't have to be a named branch: you can simply have multiple unnamed heads (or named heads using bookmark, though those still aren't perfect, because currently they don't show up with push/pull/clone).

Keep in mind that in DVCSes, Mercurial among them, branches are often short-lived and merged back into each other frequently. If you pull a branch you will still get the common history it has with any other branches.

Solution 4

@Nick said:

"This is a pretty big omission since a lot hosting sites only offer one repo. With svn I can effectively have as many repos as I want by only taking one branch from the main one. The subrepos sound like a hack."

Subrepos (aka submodules) are not as ideal as "narrow clones" its true. But at least for having many distinct projects in one hosting site's repository, you can have multiple code-bases in one repository. This won't allow you to slice up different sections of one repository / sub-directories of a project , but it will let you manage multiple projects. What you do is have lots of named branches each rooted at the empty (or null) changeset (i.e. they have no common root revision). It can get a little messy to track the branches but it does work.

For example:

hg init
hg branch project-1
# Changes, commits, repeated as needed
hg update null
hg branch project-2
# Changes, commits, repeated as needed

You now can see all your projects:

> hg branches
project-2                      5:42c2beffe780
project-1                      2:43fd60024328

The projects are unrelated (though you can merge them):

> hg debugancestors
-1:000000000000 

Most usefully: you can clone only the project you want, and the others won't mix in:

> hg clone <repository> -r project-1

The graph for this would look something like this (hg log -qG):

@  5 | project-2 | {tip}
|
o  4 | project-2
|
o  3 | project-2

o  2 | project-1
|
o  1 | project-1
|
o  0 | project-1

You can do this for as many projects as you need, listing each with hg branches, and jumping between them with hg update. This takes some care, because named branch support isn't perfect. It isn't always intuitive for one thing (read about hg clone -u in Mercurial 1.4 -- the pre-1.4 behavior is surprising when cloning). But it does work.

Solution 5

Mercurial and Git only permit cloning on the entire repository. Thus it is recommended that each project gets its own repository.

Mercurial has a forest extension to ease having a "forest" for project repositories. The extension keeps each project in a separate repository, but provides options to update/push/pull all the forest repositories together.

View more solutions

Comments

[quark]

For future reference: Cloning part of a repository is called, in DVCS jargon, a "narrow clone" or "partial clone". (Cloning only part of the history as opposed to the directory structure, is a "shallow clone".)

[Martin Geisler]

possible duplicate of How do I clone a sub-folder of a repository in Mercurial?

[Nick]

But even subrepos seem to be experimental in 1.3. This is a pretty big omission for me.

[Nick]

This is a pretty big omission since a lot hosting sites only offer one repo. With svn I can effectively have as many repos as I want by only taking one branch from the main one. The subrepos sound like a hack.

[Savageman]

In a centralized point-of-view, that makes sense. But Mercurial and Git are decentralized. 1 project = 1 repos, that's the way it works.

[Savageman]

Yes, the cool thing is you can create a new project whenever you want by updating to the revision 0 and commit in a new branch from here. But the clone command will still clone the whole repos, am I right?

[quark]

@Savageman: Doesn't have to. Use hg clone -r <branch> and you will get only the given branch. If you want the whole set of projects, hg clone will do what you want, yes.

[Draemon]

@Nick: No, it sounds like those hosting providers don't understand distributed version control.

[Draemon]

@Nick: sounds like you're fighting the DVCS instead of working with it. Take some time to learn how it wants to work and you'll quickly realise the benefits.

[Lee]

Just a heads up it's -b for specifying a branch to clone (you can still use multiple -b <branch>), -r is used for specifying a changeset.

[Aldo]

See in particular mercurial.selenic.com/wiki/…

[Harvey]

I don't believe that this would allow you to push back any changes. You can export your local changes to patch files that you then import into a full local copy of upstream, but then that kind of defeats the point of the partial clone in the first place. Also, the OP didn't specifically say if he wanted a shallow clone (same tree, fewer revisions) or a partial clone (sub-tree, some or all revisions). In either case, convert will change the hashes.

[saaj]

This is probably no longer true. hg help clone shows -b --branch BRANCH [+] clone only the specified branch. When cloned a repo this way when I type hg branches it show the only requested one.

[Savageman]

OP asked about a specified directory within the repo, not a specified branch.

[Anon]

The sparse.py extension only makes narrow checkouts possible (which doesn't impact the history fetched) whereas the question is talking about not fetching branches. See stackoverflow.com/a/40355673/2732969 for some further discussion.

[StayOnTarget]

Using convert will typically (or always?:) not produce a clone but a derivative copy which is "unrelated" to the original. That would totally inhibit pushing changes back to the source repo. Its useful, but not really what people may expect.