How do I work with a git repository within another repository?

Question

I have a Git media repository where I'm keeping all of my JavaScript and CSS master files and scripts that I'll use on various projects.

If I create a new project that's in its own Git repository, how do I use JavaScript files from my media repository in my new project in a way that makes it so I don't have to update both copies of the script when I make changes?

Solution 1

The key is git submodules.

Start reading the Submodules chapter of the Git Community Book or of the Users Manual

Say you have repository PROJECT1, PROJECT2, and MEDIA...

cd /path/to/PROJECT1
git submodule add ssh://path.to.repo/MEDIA
git commit -m "Added Media submodule"

Repeat on the other repo...

Now, the cool thing is, that any time you commit changes to MEDIA, you can do this:

cd /path/to/PROJECT2/MEDIA
git pull
cd ..
git add MEDIA
git commit -m "Upgraded media to version XYZ"

This just recorded the fact that the MEDIA submodule WITHIN PROJECT2 is now at version XYZ.

It gives you 100% control over what version of MEDIA each project uses. git submodules are great, but you need to experiment and learn about them.

With great power comes the great chance to get bitten in the rump.

Solution 2

Consider using subtree instead of submodules, it will make your repo users life much easier. You may find more detailed guide in Pro Git book.

Solution 3

If I understand your problem well you want the following things:

1. Have your media files stored in one single git repository, which is used by many projects
2. If you modify a media file in any of the projects in your local machine, it should immediately appear in every other project (so you don't want to commit+push+pull all the time)

Unfortunately there is no ultimate solution for what you want, but there are some things by which you can make your life easier.

First you should decide one important thing: do you want to store for every version in your project repository a reference to the version of the media files? So for example if you have a project called example.com, do you need know which style.css it used 2 weeks ago, or the latest is always (or mostly) the best?

If you don't need to know that, the solution is easy:

1. create a repository for the media files and one for each project
2. create a symbolic link in your projects which point to the locally cloned media repository. You can either create a relative symbolic link (e.g. ../media) and assume that everybody will checkout the project so that the media directory is in the same place, or write the name of the symbolic link into .gitignore, and everybody can decide where he/she puts the media files.

In most of the cases, however, you want to know this versioning information. In this case you have two choices:

1. Store every project in one big repository. The advantage of this solution is that you will have only 1 copy of the media repository. The big disadvantage is that it is much harder to switch between project versions (if you checkout to a different version you will always modify ALL projects)

2. Use submodules (as explained in answer 1). This way you will store the media files in one repository, and the projects will contain only a reference to a specific media repo version. But this way you will normally have many local copies of the media repository, and you cannot easily modify a media file in all projects.

If I were you I would probably choose the first or third solution (symbolic links or submodules). If you choose to use submodules you can still do a lot of things to make your life easier:

1. Before committing you can rename the submodule directory and put a symlink to a common media directory. When you're ready to commit, you can remove the symlink and remove the submodule back, and then commit.

2. You can add one of your copy of the media repository as a remote repository to all of your projects.

You can add local directories as a remote this way:

cd /my/project2/media
git remote add project1 /my/project1/media

If you modify a file in /my/project1/media, you can commit it and pull it from /my/project2/media without pushing it to a remote server:

cd /my/project1/media
git commit -a -m "message"
cd /my/project2/media
git pull project1 master

You are free to remove these commits later (with git reset) because you haven't shared them with other users.

Solution 4

Quite a lot of projects, modular in nature (more so since git favours not-so-big repositories), don't need a full-fledged configuration stage: you just layout a number of repositories within a given set of relative locations and you are done.

In this scenario should be easy to work with this, and this, and this repo, which is what the likes of git submodules, subtrees, etc. try to acomplish. They are up to the task, but I don't think they can be considered easy nor not error prone.

"Git repos within git repos" doesn't need to be difficult: please see my simplest-git-subrepos example.

Solution 5

I had issues with subtrees and submodules that the other answers suggest... mainly because I am using SourceTree and it seems fairly buggy.

Instead, I ended up using SymLinks and that seems to work well so I am posting it here as a possible alternative.

There is a complete guide here: http://www.howtogeek.com/howto/16226/complete-guide-to-symbolic-links-symlinks-on-windows-or-linux/

But basically you just need to mklink the two paths in an elevated command prompt. Make sure you use the /J hard link prefix. Something along these lines: mklink /J C:\projects\MainProject\plugins C:\projects\SomePlugin

You can also use relative folder paths and put it in a bat to be executed by each person when they first check out your project.

Example: mklink /J .\Assets\TaqtileTools ..\TaqtileHoloTools

Once the folder has been linked you may need to ignore the folder within your main repository that is referencing it. Otherwise you are good to go.

Note I've deleted my duplicate answer from another post as that post was marked as a duplicate question to this one.

View more solutions

Comments

[Benny Neugebauer]

Here is another informative article on subtree vs. submodule: blogs.atlassian.com/2013/05/…

[Admin]

for web related projects where you work from within Apache's www folder, you should place a .htaccess file in the root of either the www folder or your project's, with Options +FollowSymLinks in it, or better yet <IfModule mod_rewrite.c>{new line}Options +FollowSymLinks{new line}RewriteEngine on{new line}</IfModule> (replace {new line} with actual new line`)

[cyrf]

This workflow reminds me of using a private NPM module stackoverflow.com/questions/7575627/…

[jiggunjer]

If you prefer the default to be the latest version, you can add a script to propagate the MEDIA commit to all dependent projects.

[Robert Dundon]

Per that article, one of the drawbacks is: > The responsibility of not mixing super and sub-project code in commits lies with you. Nobody got time for that (IMO)

[theonlygusti]

How does this integrate with github?

[jerclarke]

I should probably test out your solution before upvoting, but I'm here to salute you on your sassy writing style and on tackling this head on. Git should just work this way! We went with submodules years ago with heavy hearts, and it's been exactly as awful as we could have predicted. What a stupid mess that gets in the way more than it helps! Thanks for trying to forge a new path.