Why can't I push this up-to-date Git subtree?

Question

I am using Git subtree with a couple of projects that I am working on, in order to share some base code between them. The base code gets updated often, and the upgrades can happen in anyone of the projects, with all of them getting updated, eventually.

I have ran into a problem where git reports that my subtree is up to date, but pushing gets rejected. For example:

#! git subtree pull --prefix=public/shared project-shared master
From github.com:****
* branch            master     -> FETCH_HEAD
Already up-to-date.

If I push, I should get a message that there is nothing to push... Right? RIGHT? :(

#! git subtree push --prefix=public/shared project-shared master
git push using:  project-shared master
To [email protected]:***
! [rejected]        72a6157733c4e0bf22f72b443e4ad3be0bc555ce -> master (non-fast-forward)
error: failed to push some refs to '[email protected]:***'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Merge the remote changes (e.g. 'git pull')
hint: before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

What could be the reason for this? Why is pushing failing?

Solution 1

I found the answer on this blog comment https://coderwall.com/p/ssxp5q

If you come across the "Updates were rejected because the tip of your current branch is behind. Merge the remote changes (e.g. 'git pull')" problem when you're pushing (due to whatever reason, esp screwing about with git history) then you'll need to nest git commands so that you can force a push to heroku. e.g, given the above example:

git push heroku `git subtree split --prefix pythonapp master`:master --force

Solution 2

On windows the nested command doesn't work:

git push heroku `git subtree split --prefix pythonapp master`:master --force

You can just run the nested bit first:

git subtree split --prefix pythonapp master

This will (after a lot of numbers) return a token, e.g.

157a66d050d7a6188f243243264c765f18bc85fb956

Use this in the containing command, e.g:

git push heroku 157a66d050d7a6188f243243264c765f18bc85fb956:master --force

Solution 3

Use the --onto flag:

# DOESN'T WORK: git subtree push --prefix=public/shared project-shared master --onto=project-shared/master

[EDIT: unfortunately subtree push doesn't forward --onto to the underlying split, so the operation has to be done in two commands! With that done, I see that my commands are identical to those in one of the other answers, but the explanation is different so I'll leave it here anyway.]

git push project-shared $(git subtree split --prefix=public/shared --onto=project-shared/master):master

Or if you're not using bash:

git subtree split --prefix=public/shared --onto=project-shared/master
# This will print an ID, say 0123456789abcdef0123456789abcdef,
# which you then use as follows:
git push project-shared 01234567:master

I spent hours poring through the git-subtree source to figure this one out, so I hope you appreciate it ;)

subtree push starts by running subtree split, which rewrites your commit history into a format which should be ready to push. The way it does this is, it strips public/shared/ off the front of any path which has it, and removes any information about files that don't have it. That means even if you pull non-squashed, all the upstream sub-repository commits are disregarded since they name the files by their bare paths. (Commits that don't touch any files under public/shared/, or merge commits that are identical to a parent, are also collapsed. [EDIT: Also, I've since found some squash detection, so now I'm thinking it's only if you pulled non-squashed, and then the simplistic merge commit collapsing described in yet another answer manages to choose the non-squashed path and discard the squashed path.]) The upshot is, the stuff it tries to push ends up containing any work someone committed to the current host repository you're pushing from, but not work people committed directly to the sub-repository or via another host repository.

However, if you use --onto, then all the upstream commits are recorded as OK to use verbatim, so when the rewriting process comes across them as one of the parents of a merge it wants to rewrite, it will keep them instead of trying to rewrite them in the usual way.

Solution 4

For a "GitHub pages" type app, where you deploy a "dist" subtree to a gh-pages branch, the solution might look something like this

git push origin `git subtree split --prefix dist master`:gh-pages --force

I mention this since it looks slightly different from the heroku examples given above. You can see that my "dist" folder exists on the master branch of my repo, and then I push it as a subtree to the gh-pages branch which is also on origin.

Solution 5

Had this issue, too. Here's what I did, based on the top answers:

Given:

#! git subtree push --prefix=public/shared project-shared master
git push using:  project-shared master
To [email protected]:***
! [rejected]        72a6157733c4e0bf22f72b443e4ad3be0bc555ce -> master (non-fast-forward)
error: failed to push some refs to '[email protected]:***'
hint: Updates were rejected because the tip of your current branch is behind
hint: its remote counterpart. Merge the remote changes (e.g. 'git pull')
hint: before pushing again.
hint: See the 'Note about fast-forwards' in 'git push --help' for details.

Enter this:

git push <origin> 72a6157733c4e0bf22f72b443e4ad3be0bc555ce:<branch> --force

Note the token that is output by 'git subtree push' is used in 'git push'.

View more solutions

Comments

[ubi]

I didn't need --force. git 1.8.2.3. I think there are some bugs around subtree right now; the original error said i needed push -f which didn't make sense to me,

[pilau]

How to run this command on Windows? I am getting error: unknown option 'prefix'

[bpipat]

Amazing, this just saved my day :)

[klimkin]

Even though it does provide a work-around, it doesn't explain the behavior.

[entheh]

Won't this solution (with --force) cause other people's changes (that they pushed earlier) to be lost? Not just recent ones, but all other changes (from any other host repository) back to the start of time? Tell me if I'm wrong about that. But see my answer at stackoverflow.com/a/30715198/1783475 for a safer solution using --onto.

[Eric Woodruff]

True but probably most uses of subtree in this manner is a one way push to heroku. Generally heroku is not considered a defacto git repo for multiple user to push and pull from.

[Kamil Wozniak]

I am recommending this answer as the one, which had helped me twice already. And description is what I needed to understand how splitting and pushing to the right place works. Thank you.

[GreenAsJade]

@klimkin I liked this answer because it was not full of complicated explanation that made my head spin. It just worked :)

[Colin D]

What is "project-shared"?

[entheh]

@ColinD "project-shared" is the name of the remote. It's a name you will have chosen when configuring the place to fetch from and push to. You can get a list of remotes with git remote -v.

[aednichols]

I had a hard time identifying what relationship heroku and pythonapp have in the answer. Translating this out of Heroku-specific language: git push <your subtree's origin> `git subtree split --prefix=Path/to/subtree master`:master --force

[jun]

@entheh the hours you spent into this have jsut made my day. Thanks!

[Cosyn]

I found this bug always happen when your sub repository has a directory whose name is the same as the prefix, e.g. you add the sub repository as "libxxx" into the main repository, and your sub repository itself also has a sub directory named "libxxx". In this case git will incorrectly treat commits in the sub repository as commits in the main repository and try to split them. The --onto option helps git to identify commits that are already in the sub repository.

[Infinity]

This is a much cleaner solution. Works for me!

[Aidas Bendoraitis]

This is exactly how I solved the same problem with remote repository out of sync with the local subtree.

[ATL_DEV]

Can people come to their senses? For whatever great stuff Git does, it is immediately mitigated by the lost productivity getting it to do what you want. Also, Git fails catastrophically for component oriented projects. It's competitors do a much better job.

[Demodave]

When I run git subtree split --prefix subtreedirectoryname master, I get fatal ambiguous argument 'master': unknown revision or path not in the working tree

[pakman198]

This is a great solution, I just kept getting git subtree is not a command before. After using your solution I was able to deploy a folder to heroku instead of all my repo

[halfmoonhalf]

is the bug fixed as of 2020-05-09 (Sat)? @klimkin

[halfmoonhalf]

the bug is fixed. see: contrib/subtree: fix "subtree split" skipped-merge bug. github.com/git/git/commit/…

[Nisharg Shah]

Great solution and time saving

[Andrew Miracle]

for some very weird reason this command deletes my subtree branch on a second push

[Andrew Miracle]

For some very weird reason this command git push heroku git subtree split --prefix pythonapp master:master --force deletes the subtree branch entirely and i don't know why

[zingi]

Why do I have to use --force here?

[Colin D]

you can of course try without force and if you get an error, you might find it helpful. I can't quite recall what my situation was

[Eric Burel]

Example with an URL: git push https://github.com/VulcanJS/vulcan-next.git git subtree split --prefix starters/next:main --force This is useful if the subtree repo is read-only but you accidentally updated it from GitHub UI for example. In my scenario, the updates go from "monorepo -> subtree repo", never the other way around so "subtree pull" won't work as usual.