Is it impossible to checkout a different branch in Jenkinsfile?

Question

I have two branches on BitBucket: master and develop. I've also got a BitBucket Team Folder job configured on my Jenkins server to build that repository. On the develop branch there's the following Jenkinsfile:

node {
    stage('Checkout') {
        checkout scm
    }

    stage('Try different branch') {
        sh "git branch -r"
        sh "git checkout master"
    }
}

When Jenkins runs it, the build fails when it attempts to checkout master:

[Pipeline] stage
[Pipeline] { (Try different branch)
[Pipeline] sh
[e_jenkinsfile-tests_develop-4R65E2H6B73J3LB52BLACQOZLBJGN2QG22IPONX3CV46B764LAXA] Running shell script
+ git branch -r
  origin/develop
[Pipeline] sh
[e_jenkinsfile-tests_develop-4R65E2H6B73J3LB52BLACQOZLBJGN2QG22IPONX3CV46B764LAXA] Running shell script
+ git checkout master
error: pathspec 'master' did not match any file(s) known to git.
[Pipeline] }

I had expected the git branch -r command to print out both origin/master and origin/develop, but for some reason it only prints the latter.

I've read around and tried to come up with any ways to do this: For example, I tried installing the SSH Agent Plugin for Jenkins and changed the Jenkinsfile to:

node {
    stage('Checkout') {
        checkout scm
    }

    stage('Try different branch') {
        sshagent(['Bitbucket']) {
            sh "git branch -r"
            sh "git checkout master"
        }
    }
}

But it still doesn't appear to find origin/master. What's worse, the SSH agent seems to be killed before it attempts to checkout master:

[Pipeline] { (Try different branch)
[Pipeline] sshagent
[ssh-agent] Using credentials ThomasKasene (Used to communicate with Bitbucket)
[ssh-agent] Looking for ssh-agent implementation...
[ssh-agent]   Exec ssh-agent (binary ssh-agent on a remote machine)
$ ssh-agent
SSH_AUTH_SOCK=/tmp/ssh-M6pIguCUpAV4/agent.11899
SSH_AGENT_PID=11902
$ ssh-add /var/jenkins_home/workspace/e_jenkinsfile-tests_develop-4R65E2H6B73J3LB52BLACQOZLBJGN2QG22IPONX3CV46B764LAXA@tmp/private_key_2394129657382526146.key
Identity added: /var/jenkins_home/workspace/e_jenkinsfile-tests_develop-4R65E2H6B73J3LB52BLACQOZLBJGN2QG22IPONX3CV46B764LAXA@tmp/private_key_2394129657382526146.key (/var/jenkins_home/workspace/e_jenkinsfile-tests_develop-4R65E2H6B73J3LB52BLACQOZLBJGN2QG22IPONX3CV46B764LAXA@tmp/private_key_2394129657382526146.key)
[ssh-agent] Started.
[Pipeline] {
[Pipeline] sh
[e_jenkinsfile-tests_develop-4R65E2H6B73J3LB52BLACQOZLBJGN2QG22IPONX3CV46B764LAXA] Running shell script
+ git branch -r
  origin/develop
[Pipeline] sh
$ ssh-agent -k
unset SSH_AUTH_SOCK;
unset SSH_AGENT_PID;
echo Agent pid 11902 killed;
[ssh-agent] Stopped.
[e_jenkinsfile-tests_develop-4R65E2H6B73J3LB52BLACQOZLBJGN2QG22IPONX3CV46B764LAXA] Running shell script
+ git checkout master
error: pathspec 'master' did not match any file(s) known to git.
[Pipeline] }

My eventual plan is to commit something to develop and then merge it into master, but so far I have had very little luck. Has anybody got a possible solution or workaround?

PS: This only seems to be a problem in Jenkinsfile; I have a freestyle job that does something similar to what I want, and it works fine.

Solution 1

After some hours of trial and error, I came up with a possible solution. It builds partly on Matt's answer, but I had to alter it to make it work.

Matt was correct in the essentials: checkout scm simply wasn't flexible enough to allow me to do what I needed, so I had to use GitSCM to customize it. The major points of interest are:

• Added extension LocalBranch to make sure I check out to an actual branch, and not just a detached HEAD.
• Added extension WipeWorkspace to delete everything in the workspace and force a complete clone. I don't think this was a part of the solution to my question, but it was still handy to have.
• Specified the SSH credentials using the credentialsId property since the repository is private.

For whatever reason, when the checkout step is executed, it only checks out the branch, but does not set it to track the remote branch. Until I find a more elegant solution, I had to do this manually.

After all that was done, I could use regular sh "git checkout master" and even sh "git push", as long as I enclosed them in an sshagent step.

I added a working example of the resulting Jenkinsfile below, but please keep in mind that it shouldn't be used for anything close to production as it's still very much in its infancy; hardcoded version numbers and no checks for which branch you're in, for example.

node {
    mvnHome = tool 'Maven'
    mvn = "${mvnHome}/bin/mvn"

    stage('Checkout') {
        checkout([
            $class: 'GitSCM',
            branches: scm.branches,
            extensions: scm.extensions + [[$class: 'LocalBranch'], [$class: 'WipeWorkspace']],
            userRemoteConfigs: [[credentialsId: 'Bitbucket', url: '[email protected]:NAVFREG/jenkinsfile-tests.git']],
            doGenerateSubmoduleConfigurations: false
        ])
    }

    stage('Release') {
        // Preparing Git
        sh "git branch -u origin/develop develop"
        sh "git config user.email \"[email protected]\""
        sh "git config user.name \"Jenkins\""

        // Making and committing new verison
        sh "${mvn} versions:set -DnewVersion=2.0.0 -DgenerateBackupPoms=false"
        sh "git commit -am \"Released version 2.0.0\""

        // Merging new version into master
        sh "git checkout master"
        sh "git merge develop"
        sh "git checkout develop"

        // Making and committing new snapshot version
        sh "${mvn} versions:set -DnewVersion=3.0.0-SNAPSHOT -DgenerateBackupPoms=false"
        sh "git commit -am \"Made new snapshot version 3.0.0-SNAPSHOT\""

        // Pushing everything to remote repository
        sshagent(['Bitbucket']) {
            sh "git push"
            sh "git checkout master"
            sh "git push"
        }
    }
}

Solution 2

I could not get the two answers above to work. I was triggering a Jenkins pipeline job by specifying a branch and was trying to checkout another branch (develop) in the job which was failing with:

error: pathspec 'develop' did not match any file(s) known to git.

I could see this in the failing job, which indicated that only the triggering branch was being fetched:

git fetch --no-tags --progress https://<github URL> +refs/heads/branch-name:refs/remotes/origin/branch-name

I got it to work by changing the remote fetch config and fetching all branches by doing the following after doing just the default checkout scm step in the triggered job's Jenkinsfile:

sh """
    git config remote.origin.fetch '+refs/heads/*:refs/remotes/origin/*'
    git fetch --all
"""

This is thanks to this answer https://stackoverflow.com/a/39986737/1988883

This also avoids having to configure Jenkins for GitSCM script approvals which I had to do for trying out the two solutions above

Solution 3

You can use the intrinsic function in Jenkins Pipeline created for Git cloning and pulling. I would also suggest cloning the branches into separate directories.

checkout([$class: 'GitSCM',
  branches: [[name: '*/branch_name']],
  doGenerateSubmoduleConfigurations: false,
  extensions: [[$class: 'RelativeTargetDirectory',
    relativeTargetDir: 'different_directory']],
  submoduleCfg: [],
  userRemoteConfigs: [[url: '[email protected]:org/repo.git']]])

Solution 4

The problem is that Jenkins is defining the origin with only the branch that is discovered.

@swoop81 answer is working but if you just want to checkout one branch, you could fetch only this one.

git config --add remote.origin.fetch +refs/heads/<branch-name>:refs/remotes/origin/<branch-name>
git fetch --no-tags https://<github-url> +refs/heads/<branch-name>:refs/remotes/origin/<branch-name>

Solution 5

You can use the git command which is the less advanced version of the checkout command

stage('Repo Checkout') {
            steps {
                deleteDir()
                dir("repo-one-master") {
                    git url: "ssh://[email protected]/folder/project",
                        branch: 'master'
                }
                dir("repo-one-feature) {
                    git url: "ssh://[email protected]/folder/project",
                        branch: 'some-branch'
                }
            }
        }

View more solutions

Comments

[Thomas Kåsene]

Is there any particular reason you suggest cloing to a separate directory, other than the potential risk of messing something up?

[Matt Schuchard]

@ThomasKåsene If you clone two repositories to the same directory, that will cause issues with your .git directory.

[Thomas Kåsene]

Assuming I don't wipe it first, I totally agree. The plan was never to clone multiple branches to the same folder, but to switch back and forth between branches in the same repostiory, which should work fine in theory. Thanks for your answer, by the way!

[Matt Schuchard]

@ThomasKåsene Got it; that does make sense.

[Matrix]

@MattSchuchard Hi, with this command, I'm wondering if we can use branch_name as variable that comes from Jenkins parameter? like this cmd but it's not working fine for me maybe I'm missing something? checkout([$class: 'GitSCM', branches: [[name: '*/$branch_name]], doGenerateSubmoduleConfigurations: false, extensions: [], submoduleCfg: [], userRemoteConfigs: [[credentialsId: 'myid', url: 'mygitrepo']]])

[Erkki Nokso-Koivisto]

thank you! this also worked in a bitbucket-pipelines.yml so that a bitbucket CI job running on a branch can also checkout master. phew. glad I found this. (upvoted!)

[Anurag Ashok]

Yes this answer worked for me and was much straightforward compared to others. Thank you!

[Jon]

If you look at the console logs, you can see that with your options, the git plugin will do git checkout -b develop 1694b133..... This is why it does not track the remote branch without an additional manual step in your pipeline. The final argument above (the commit) is the start_point and git will only track a remote automatically if set to a branch name, not a commit sha. Why git plugin does this, I have no idea, but unless you are persisting your local workspace it's rather worthless to check out to attached HEAD for a build without a tracked remote. I haven't found a solution.