Git & Jenkins: get latest green commit on branch

Question

We're just starting to push for CI-CD and as a baby step we're going to try updating a stack with the latest green develop once every couple of hours. I am fairly new to Git/Bitbucket, and can't figure out how to ensure the checkout that Jenkins makes gets the last commit to have been marked green by Jenkins, rather than just "the last commit" as a blanket statement.

We have the Bitbucket Build Status Notifier plugin installed, so Bitbucket does track which commits are green after our unit tests run. Is there a way to leverage this info to make sure the right commit is picked?

Solution 1

You don't make any mention of the scripting language you want to use, so I will talk specifically about the HTTP requests to the BitBucket API:

Assumptions

If you have a BitBucket Repository that has three commits in in it the first and the last are failing the build, the middle is passing:

• 4768815 ❌
• 49d7110 ✅
• 42d357f ❌

Get the list of commits

You can get the list of commits by calling the following API method:

https://api.bitbucket.org/2.0/repositories/{{owner}}/{{repo_slug}}/commits

• owner: RichardSlater
• repo_slug: greencommitproofofconcept

The response looks like this:

{
  "pagelen": 30,
  "values": [
    {
      "hash": "4768815fdc27abf4be17096e7c460f7f68f5d39b",
      "repository": { ... },
      "links": {
        ...
        "statuses": {
          "href": "https://api.bitbucket.org/2.0/repositories/RichardSlater/greencommitproofofconcept/commit/4768815fdc27abf4be17096e7c460f7f68f5d39b/statuses"
        }
      },
      "author": { ... },
      "parents": [ ... ],
      "date": "2017-04-10T11:38:18+00:00",
      "message": "README.md edited online with Bitbucket",
      "type": "commit"
    },
    {
      "hash": "49d7110b98616358d16055960a4abdf2926b890d",
      ...
    },
    {
      "hash": "42d357f1df7a7d7bcf1f10a9f3a5a40d85d5b11c",
      ...
    }
  ]
}

If you parse the JSON and loop over the responses you can extract out the statuses from:

values[n].links.statuses.href

Where n is the index, i.e. 0, 1 or 2 in the above example. If you were to construct this from scratch it would be in the following format.

Get the list of statuses from the commit

https://api.bitbucket.org/2.0/repositories/{{owner}}/{{repo_slug}}/commit/{{sha}}/statuses"

• owner: RichardSlater
• repo_slug: greencommitproofofconcept
• sha: 4768815fdc27abf4be17096e7c460f7f68f5d39b

Note: this is a Hypermedia API which means the urls could change so I would recommend using the links from the previous response rather than trying to generate them from scratch.

The response from the above HTTP request will be something like:

{
  "pagelen": 10,
  "values": [
    {
      "key": "POC-01",
      "name": "Build #1",
      "repository": { ... },
      "url": "http://devops.stackexchange.com/q/809/397",
      "links": { ... },
      "refname": null,
      "state": "FAILED",
      "created_on": "2017-04-10T13:04:28.261734+00:00",
      "updated_on": "2017-04-10T13:04:28.261759+00:00",
      "type": "build",
      "description": "Changes by Richard Slater"
    }
  ],
  "page": 1,
  "size": 1
}

From this response you can extract the state using:

values[n].state

Again where n is the status - there could be many of them if one commit resulted in many builds.

If the state for the build you care about is SUCCESSFUL then you have your answer and you can immediately return the sha for the commit.

Loop over all of the commits from the first phase, if you run out of commits follow the next page link that is included in the call to /commits.

Complete Flow Diagram

At a high level the flow will look like this:

Don't forget this is a Hypermedia API so wherever possible have your code follow the links in the API rather than trying to "guess them".

Solution 2

In a typical Continuous Delivery/Deployment pipeline you would have the following happen:

1. Developer pushes one or more commits, or, a pull request is merged.
2. Jenkins automatically builds and executes tests.
3. If successful Jenkins publishes a deployment package to an Artefact Repository; if failure publishes nothing and notify developers.
4. Deployment automation uses packages from the Artefact Repository and deploys them.

The goal is to avoid building the solution from source twice, you build it once and deploy it many times. You can implement approvals in Sonartype Nexus to define the environment approval process, i.e. Dev → Test → UAT → Stage → Production.

That said... if you have read all of the previous and still want to get the latest green build from source control then you can use one of two techniques:

1. Have Jenkins tag the branch with a suitably named tag, i.e. master-green then use that instead of master when you want the latest green build.
2. Use the BitBucket commits to get a list of commits and commit/{sha}/statuses over each one to find the commit with a Green status. I have expanded on this solution in another answer.

Feel free to post a follow-up question if you would like the specific detail on how to use the above approaches.

Comments

[Richard Slater]

Yes it does, it's probably my longest answer on SE ever.

[Alex]

I appreciate the time you took to explain this even if you think I'm completely insane for wanting it. Accepted

[Richard Slater]

Not completely insane, just taking your first few steps - keep my other answer in mind when you are thinking about CI/CD architectures.