#softwareprocess

Leveling up in integrating code changes

Continuous Deployment is the highest level of maturity when it comes to integrating code changes and deployment of software products. Let's see what levels we can distinguish.

At the most basic level, a team is used to manually compose releases by selecting features that have approval by the QA engineers. Development uses feature branches to code new features, and once they're done, QA is involved in testing it. Only when QA has verified the feature works correctly, it will be integrated and released on a set date.

The next step is Continuous Integration, where a part of the release process is automated by using a centralised build server. Whenever a developer pushes a series of commits into source control, the build server is triggered to run a series of automated tests to verify the code is correct. Having an extensive set of unit tests is becoming very important from this stage on. Merging of pull requests could be automated on the build server, or the team is working trunk-based and code is already merged. Either way, the QA engineer verifies the new state of the program before it is released on Production.

At the third level we have Continuous Delivery: the build server is extended by automatically deploying the next release to an Acceptance environment. In this way of working, the team has fully automated deployments. Also, it is pretty clear that the new code will in fact be in the next release, because generally, from here on, there is only one branch holding the next release. However, QA will still have to verify this new release on the Acceptance environment. Only if QA approves the release, it can be deployed to Production. This last step is semi-automated by the build server, in the sense that it is a push-button automatic deployment.

The final level is Continuous Deployment. Although it is in fact nothing more than removing the push-button part from Continuous Delivery pipeline, it changes a lot in the way of working of the team. A well-maintained delivery pipeline takes at most 10-15 minutes to automatically build, test, and deploy. If the QA engineer tests the new code on Acceptance, you're too late because it will already be on Production. A shift of the QA role needs to happen.

In Continuous Deployment, the QA engineer no longer does his/her thing at the end of the production chain. Instead, it will be more at the beginning of the chain. This may sound weird; how can you test things that haven't been built yet? Well, it is in the quality of a user story that becomes a lot more important.

Three amigos

For the creation of User Stories, a Product Owner is no longer solely responsible for delivering tickets that conform to the Definition of Ready (i.e., the promise of the Product Owner to the development team). User Stories are refined in sessions where a developer, a QA engineer, and the Product Owner (the Three Amigos) sit together to write the acceptance criteria for this user story.

Clear acceptance criteria should be stated as use case scenarios whenever possible. Following the Given...When...Then... structure, one can clearly describe the behavior of the system in terms of functionality. You set the stage (Given), perform an action (When), and expect a testable result (Then).

The Acceptance test scenarios should then be automated by the development team and run on each build to verify the functionality of the system is still working before deployment. This is automating a part of the testing work that the QA engineer used to do. Now, he/she is rather thinking of the scenarios before development takes place instead of after a feature has been delivered.

Manual testing

Yes. The QA engineer will still perform some manual testing on the system. This time, it is an added level of quality control instead of a necessity. Because we have now automated most of the functionality testing by implementing acceptance test scenarios that are run on every build, there is no real need to verify the workings again, manually.

The additional level of quality can be achieved when QA does exploratory testing on the program. This is done manually. The purpose of exploratory testing is to verify the system is still coherent. For instance, to answer questions like: Does the new feature still fit in the overall user perception of the system? How is the usability of the system? How does it feel to work with the system? Does the copy in forms and dialogs make sense?

Those questions are hard to answer by automated tests and in general can only be verified by seeing the system at work.

Acceptance Tests verify we've built the right thing, Unit Tests verify we've built the thing right. Exploratory tests verify the system is coherent.

Further read

Lean-Agile Acceptance Test Driven Development, Ken Pugh