Git - What is "Refspec"

Question

I've been following this guide on configuring GitLab continuous integration with Jenkins.

As part of the process, it is necessary to set the refspec as follows: +refs/heads/*:refs/remotes/origin/* +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*

Why this is necessary is not explained in the post, so I began looking online for an explanation and looked at the official documentation as well as some related StackOverflow questions like this one.

In spite of this, I'm still confused:

What exactly is refspec? And why is the above refspec necessary – what does it do?

Solution 1

A refspec tells git how to map references from a remote to the local repo.

The value you listed was +refs/heads/*:refs/remotes/origin/* +refs/merge-requests/*/head:refs/remotes/origin/merge-requests/*; so let's break that down.

You have two patterns with a space between them; this just means you're giving multiple rules. (The pro git book refers to this as two refspecs; which is probably technically more correct. However, you just about always have the ability to list multiple refspecs if you need to, so in day to day life it likely makes little difference.)

The first pattern, then, is +refs/heads/*:refs/remotes/origin/* which has three parts:

1. The + means to apply the rule without failure even if doing so would move a target ref in a non-fast-forward manner. I'll come back to that.
2. The part before the : (but after the + if there is one) is the "source" pattern. That's refs/heads/*, meaning this rule applies to any remote reference under refs/heads (meaning, branches).
3. The part after the : is the "destination" pattern. That's refs/remotes/origin/*.

So if the origin has a branch master, represented as refs/heads/master, this will create a remote branch reference origin/master represented as refs/remotes/origin/master. And so on for any branch name (*).

So back to that +... suppose the origin has

A --- B <--(master)

You fetch and, applying that refspec you get

A --- B <--(origin/master)

(If you applied typical tracking rules and did a pull you also have master pointed at B.)

A --- B <--(origin/master)(master)

Now some things happen on the remote. Someone maybe did a reset that erased B, then committed C, then forced a push. So the remote says

A --- C <--(master)

When you fetch, you get

A --- B
 \
  C

and git must decide whether to allow the move of origin/master from B to C. By default it wouldn't allow this because it's not a fast-forward (it would tell you it rejected the pull for that ref), but because the rule starts with + it will accept it.

A --- B <--(master)
 \
  C <--(origin/master)

(A pull will in this case result in a merge commit.)

The second pattern is similar, but for merge-requests refs (which I assume is related to your server's implementation of PR's; I'm not familiar with it).

More about refspecs: https://git-scm.com/book/en/v2/Git-Internals-The-Refspec

Solution 2

A refspec tells git how to map references from a remote to the local repo.

With Git 2.29 (Q4 2020), a refspec can also tell Git what reference to exclude.
"git fetch" and "git push" support negative refspecs.

So not only can you fetch selectively:

# Do not fetch any remote branch starting with 'm'
git fetch origin refs/heads/*:refs/remotes/origin/* ^refs/heads/m*

But you can even push or push --prune selectively:

# If I delete local branches, included b, 
# those same branches will be deleted in the remote 'origin' repo.
# ... except for the remote branch b!
git push --prune origin refs/heads/* ^refs/heads/b

See commit c0192df (30 Sep 2020) by Jacob Keller (jacob-keller).
^{(Merged by Junio C Hamano -- gitster -- in commit 8e3ec76, 5 Oct 2020)}

refspec: add support for negative refspecs

^{Signed-off-by: Jacob Keller}

Both fetch and push support pattern refspecs which allow fetching or pushing references that match a specific pattern.
Because these patterns are globs, they have somewhat limited ability to express more complex situations.

For example, suppose you wish to fetch all branches from a remote except for a specific one. To allow this, you must setup a set of refspecs which match only the branches you want.
Because refspecs are either explicit name matches, or simple globs, many patterns cannot be expressed.

Add support for a new type of refspec, referred to as "negative" refspecs.

These are prefixed with a '^' and mean "exclude any ref matching this refspec".
They can only have one "side" which always refers to the source.

• During a fetch, this refers to the name of the ref on the remote.
• During a push, this refers to the name of the ref on the local side.

With negative refspecs, users can express more complex patterns. For example:

git fetch origin refs/heads/*:refs/remotes/origin/* ^refs/heads/dontwant

will fetch all branches on origin into remotes/origin, but will exclude fetching the branch named dontwant.

Refspecs today are commutative, meaning that order doesn't expressly matter.
Rather than forcing an implied order, negative refspecs will always be applied last.
That is, in order to match, a ref must match at least one positive refspec, and match none of the negative refspecs.
This is similar to how negative pathspecs work.

---

The documentation now includes:

A <refspec> may contain a * in its <src> to indicate a simple pattern match.
Such a refspec functions like a glob that matches any ref with the same prefix. A pattern <refspec> must have a * in both the <src> and <dst>. It will map refs to the destination by replacing the * with the contents matched from the source.

If a refspec is prefixed by ^, it will be interpreted as a negative refspec.
Rather than specifying which refs to fetch or which local refs to update, such a refspec will instead specify refs to exclude.
A ref will be considered to match if it matches at least one positive refspec, and does not match any negative refspec.

Negative refspecs can be useful to restrict the scope of a pattern refspec so that it will not include specific refs.
Negative refspecs can themselves be pattern refspecs. However, they may only contain a <src> and do not specify a <dst>.
Fully spelled out hex object names are also not supported.

See t5582-fetch-negative-refspec.sh for more examples

Comments

[LarsH]

Thanks, this explanation is helpful. For consistency with the Pro Git book you linked to: They seem to say that +refs/heads/*:refs/remotes/origin/* +refs/merge-requests/*/head:refs/remotes/origin/merge-reques‌​ts/* is two refspecs, not one refspec with two patterns.

[Sergey Voronezhskiy]

For those who came for github solution: git config --local --add remote.origin.fetch +refs/pull/*/head:refs/remotes/origin/pr/* Don't forget to download refs from remote: git fetch origin Then you can checkout into pull-request number git checkout pr/1 You can also see all local refs by using: git show-ref

[Philip Oakley]

The documentation will be within the git help pull and git help fetch for those who just have the local man pages (v2.31 on).

[user2173353]

And what if the part before the : is blank? What does this mean?

[user2173353]

OK, found the answer online: Deleting references is done by pushing an empty local reference to it, like you can do with git's push command. So pushing [":refs/heads/i18n"] would delete the i18n branch on the remote.