Assembly Line Software: Agile, Waterfall and Just-in-time Software Development
I don't really think waterfall is dead, and I think it can play nicely with agile methodologies. Before you you bring out the pitchforks, I think it might be time to take another look at the evolution of delivering great software.
Let’s start at the beginning. We all start, with any luck, with the discovery of how a feature fits into our customers’ needs and how that fits into our pricing model. The age old question of cost vs. value should always be answered first.Â
Great! We have a need. Let's look at delivering it.
First we need to figure out how it is going to work. We still need an ideation phase, where we build prototypes, sometimes using paper, to evaluate our existing paradigms and how competitors have tried to solve the problem. This will probably still require some sort of buy-off from stake-holders. We may even need to do some technology feasibility testing.
By the way, this ideation phase is the best opportunity to find out the real cost of what you are about to build since estimating prototypes is easier than estimating an engineer's capability for imagining what you actually want.
After we know what the moving parts are we can branch off our main code and begin the creation of the feature. The building phase usually includes front-end designers and engineers, maybe some back end engineers. For anything that requires testing, it should always be done in a branch so it can be tested in isolation; we don't want to cause pollution of our always-deployable trunk, right?
Once we’re done with development, we wrap things up tidily and enter the first phase of isolated QA testing. I say 'isolated' because we should test the feature in the branch before we merge it. With any luck this will pass and the branch will be merged into the main release trunk for a second set of integration testing before launch.
It might fail QA for some arcane reason, either planning or personnel or technology, sometimes at the last minute. This needn’t fail our entire release, however, since if we are QA’ing in the branch we can simply decide to not merge and release it quite yet (note that this takes a lot of willpower to not-merge-until-tested, but trust me, it will earn you spades of stability).
If it passes isolated QA testing, we merge it into trunk, run everything through integration testing (with any luck with time to spare before launch to fix any regressions) and deploy it to happily paying customers who are amazed by our code stability and our ability to deliver in a timely manner.
Note that all of this is from the feature’s perspective. And it all sounds rather waterfall-y.
From a resource perspective, each stage of an organization’s kanban -- this whole market to ideation to design to development to QA to integration process -- each step can still be swapped out just-in-time for higher priority items as they appear to the organization. And teams can begin the first phases of the next feature while other teams continue forward on the current one, allowing your organization to maximize productivity in an assembly line fashion. And it is the branching strategy that gives you the opportunity and safety to make the just-in-time decisions, mitigating risk and moving everyone up in the timeline.
In the event of a swap out (due most likely to business discovery), teams can always resume where they left off. We essentially have waterfall in our agile, and agile in our waterfall.
There are risks to this process. High kickback rates from QA can cause thrashing; code can become stale in branches in fast paced environments due to neglect; bottlenecks can still develop for various reasons. I'll talk more about mitigating these risks in later posts.
On the other hand, the benefits are staggering in feature delivery, codebase stability, and resource optimization.
Most task oriented kanban boards with swim lanes are great for seeing small scale, but in most cases we miss the bigger picture and the benefits of moving our features through an assembly line process with a just-in-time mentality.
It's a wonderfully waterfall-agile-kanban-just-in-time software development world.
More to come.
