#canary analysis

Canary deployment is a release deployment strategy often used for Spinnaker deployments where a new release version is deployed alongside the stable running version. 

It is exposed to a small subset of the end-users and then the canary version is evaluated against the baseline (a copy of current deployment) to find out if the new release is stable and ready to be rolled out.

 Kayenta is the Spinnaker microservice that is responsible for enabling canary deployments in Spinnaker and performing automated canary analysis. 

We can integrate application monitoring (APM) tools like Prometheus with the Spinnaker to provide us comprehensive performance metrics while running the Spinnaker CI/CD pipelines. 

Kayenta can retrieve the performance metrics data from Prometheus and use them for carrying out canary judgment on whether the new release is fit to be deployed into production or it needs to be rolled back. 

Kayenta is the Spinnaker microservice that is responsible for enabling canary deployments in Spinnaker and performing automated canary analysis. Canary deployment is a release deployment strategy where a new release version is deployed alongside the stable running version.

It is exposed to a small subset of the end-users and then evaluated against the baseline (a copy of current deployment) to find out if the new release is stable and ready to be rolled out. The application config in Spinnaker has the configuration option to activate canary config and canary reports on Deck.

Kayenta supports monitoring tools like Stackdriver, Prometheus, Datadog, New Relic, e.t.c. Performing automated canary analysis with Kayenta involves two stages — Metric Retrieval and Judgement. Key metrics are retrieved from canary and baseline clusters, tagged, and stored in a time-series database.

Slack is a popular communication and collaboration platform used widely in business for quick notifications and updates. Spinnaker supports sending CI/CD pipeline notifications to Slack.

In case a Spinnaker pipeline fails, DevOps teams could simply click on the link in a Slack notification and check for the reasons behind the failure. Spinnaker needs to be aware of the Slack bot’s identity (name & token) so that Spinnaker can communicate with the Slack bots and can automatically post notifications about Spinnaker pipeline information to a channel.

Canary Analysis is a deployment strategy for ensuring stable release where any change is partially rolled out and then evaluated against the current deployment i.e. baseline.

Kayenta microservice assesses the risk of a Canary release and checks for significant degradation between the baseline and Canary. Kayenta consists of two stages — Metric Retrieval and Judgement. Kayenta supports integration with APM monitoring tools like Stackdriver, Prometheus, Datadog, New Relic e.t.c. In the Metric Retrieval stage, Kayenta retrieves performance metrics from these tools. In the Judgement stage, it compares those retrieved metrics and makes a decision to pass or fail the canary release.

Canary Analysis is a type of deployment strategy where new software releases are partially rolled out to a small subset of users before making it available to everybody.

Kayenta is the Automated Canary Analysis service for Spinnaker that uses the performance metrics to make a judgment whether the release is fit for production or it needs to be rolled back.

Read our blog “How to integrate Kayenta with Spinnaker for Automated Canary Analysis” and learn how to scrape performance metrics data using APM tools like Prometheus, evaluate a new Canary deployment and perform Automated Canary Analysis in your CI/CD pipeline using the Spinnaker pipeline.

Learn how to install Spinnaker with Canary and Prometheus enabled, add Canary configuration, and finally add the Canary stage in Spinnaker CI/CD pipeline.

Excerpt

This a two-part blog series about Canary Deployment and Canary Analysis. In the first part, we have discussed canary as a release strategy, canary deployment stages, benefits of using canary and constraints with implementation. In the second part, we discussed how canary analysis is performed after canary deployments are done, selection and evaluation criteria for choosing different metrics and smoke tests to perform the analysis and canary hypothesis testing performed with incremental traffic to the canary instances. Here we will discuss Canary Analysis.

Canary analysis is a two step process where we will assess the canary based on a selected list of metrics and logs to infer if we are going to promote or roll back the new version.

The two steps to perform Canary Analysis are:

Correct metrics selection and evaluation

Selection and evaluation of metrics

Further the selection and evaluation criteria are based on the following:

Choosing the most important business metrics

Balanced set of fast and slow metrics

Metric smoke test

Metric standard ranges

Metric correlation

The last step of canary analysis is evaluating the values of the metrics also known as Canary Hypothesis Testing which enables us to assess if the canary instance should be promoted to production or not. It is performed with incremental traffic to the canary instances.  

Today, many applications run in a dynamic environment of microservices, which makes the software integration and delivery process complex. The only way to successfully deploy updates consistently is to use solid automation with tools like Spinnaker.