Canary release strategy vs. Blue/Green

Question

My understanding of a canary release is that it's a partial release to a subset of production nodes with sticky sessions turned on. That way you can control and minimize the number of users/customers that get impacted if you end up releasing a bad bug.

My understanding of a blue/green release is that you have 2 mirrored production environments ("blue" and "green"), and you push changes out to all the nodes of either blue or green at once, and then use networking magic to control which environment users are routed to via DNS.

So, before I begin, if anything I have said so far is incorrect, please begin by correcting me!

Assuming I'm more or less on track, then a couple of questions about the two strategies:

• Are there scenarios where canary is preferred over blue/green, and vice versa?
• Are there scenarios where a deployment model can implement both strategies at the same time?

Solution 1

I have written a detailed essay on this topic here: http://blog.itaysk.com/2017/11/20/deployment-strategies-defined

In my opinion, the difference is whether or not the new 'green' version is exposed to real users. If it is, then I'd call it Canary. A common way to implement Canary is regular Blue/Green with the addition of smart routing of specific users to the new version. Read the post for a detailed comparison

Blue/Green:

Canary:

Solution 2

Blue-green releasing is simpler and faster.

You can do a blue-green release if you've tested the new version in a testing environment and are very certain that the new version will function correctly in production. Always using feature toggles is a good way to increase your confidence in a new version, since the new version functions exactly like the old until someone flips a feature toggle. Breaking your application into small, independently releaseable services is another, since there is less to test and less that can break.

You need to do a canary release if you're not completely certain that the new version will function correctly in production. Even if you are a thorough tester, the Internet is a large and complex place and is always coming up with unexpected challenges. Even if you use feature toggles, one might be implemented incorrectly.

Deployment automation takes effort, so most organizations will plan to use one strategy or the other every time.

So do blue-green deployment if you're committed to practices that allow you to be confident in doing so. Otherwise, send out the canary.

The essence of blue-green is deploying all at once and the essence of canary deployment is deploying incrementally, so given a single pool of users I can't think of a process that I would describe as doing both at the same time. If you had multiple independent pools of users, e.g. using different regional data centers, you could do blue-green within each data center and canary across data centers. Although if you didn't need canary deployment within a data center, you probably wouldn't need it across data centers.

Solution 3

Although both of these terms look quite close to each other, they have subtle differences. One put confidence in your functionality release and the other put confidence the way you release.

Canary

1. The canary release is a technique to reduce the risk of introducing a new software version in production by slowly rolling out the change to a small subset of users before rolling it out to the entire infrastructure.

2. It is about to get an idea of how new version will perform (integrate with other apps, CPU, memory, disk usage, etc).

Blue/Green:

1. It is more about the predictable release with zero downtime deployment.
2. Easy rollbacks in case of failure.
3. Completely automated deployment process

Solution 4

Here are some inline definition -

• Blue-Green Deployment - When deploying a new version of an application, a second environment is created. Once the new environment is tested, it takes over from the old version. The old environment can then be turned off.

   

• A/B Testing - Two versions of an application are running at the same time. A portion of requests go to each. Developers can then compare the versions.  
• Canary Release - A new version of a microservice is started along with the old versions. That new version can then take a portion of the requests and the team can test how this new version interacts with the overall system.

Solution 5

A good start of definitions. I think it also helpes in making a decision for your strategy if you split your "release" definition in "deploy" and "release(functionality)".

Deploy (binaries)

The action of binary deployment of your product to a (production) system.

Release (functionality)

The action of managing availability of functionality to (groups of) users.

Why? You typically have (multiple) two concerns when "releasing": 1) Bugs / backwards compatibility /etc 2) Verifying the validness/usability of new features

Then ask yourselves, before choosing a Canary or Blue/green or whatever gray/mixed mode strategy: What concern(s) do we have when releasing/deploying the new version? And only then if you know your concerns, choose your strategy.

Additionally, it is possible to do more complex Deploy/Release strategies. E.g, in some clouds/infra it is possible to have multiple production servers, and relay load in different proportions to different servers and versions of your product, and monitor soundness before scaling a release/deploy up to all users.

Feature flagging

The action of "configuring" (cold, or even hot) which functionality is (not)available for which (group) of users

If you also do something like "feature flagging" you can deploy first, measure soundness of your release in backwards compatibility/bug perspective, and release new functionality gradually to different users, or vice versa (scale down or even rollback functionality and/or binaries). Feature flagging allows for splitting availability of functionality from deployment of binaries, and gives much more fine-grained decision making then only "deploy/rollback"

View more solutions

Comments

[Patrick M]

Your understanding is sound, but I wouldn't phrase a blue-green strategy as needing to deploy to all nodes at once. You can deploy them as leisurely as you like - the only pressure is your own deadlines. Additionally, you can use blue-green to release changes to only a subset of your nodes (e.g. only modifying one of many API endpoint pools).

[kheraud]

Very nice sum up of these concepts I see everywhere without a clear definition first !

[nikli]

Very good explanation. But it would be better show user load percentage sample on canary illustration.

[kinjelom]

A few words about the meaning of colors: - the old environment could be the blue, the new the green. - In the next release, the old will be the green. Wiki: > Many languages do not distinguish between what in English are described as "blue" and "green" and instead use a cover term spanning both - "grue"

[Ligemer]

Canary isn't always faster than blue/green. It all depends on the CI and CD workflows!

[Lost]

A picture is worth thousand words!!

[Kumar Manish]

A/B & Canary both are saying the exact same thing.

[John Test]

This is really good but I think your "After" diagarm of Canary can be improved it's a bit misleading for whoever doens't read your essay. Otherwise really good work!