When considering Software release cycle time there are several aspect s that need to be considered.
Grouped in descending order of importance, supported by AccuRevs whitepaper Breaking the Major Release Habit I'd say we at least have these factors:
- Developer attention
- Loss of revenue
When releasing rarely - once every year or even every 6 months - the effect on developer attention is vast. Imagine you work on some code that you know is not going to be used in a year from now, compared to working on a feature that will be responsible for trading millions of $$ next week. In the first case, the attention is affected by uncertainty - both if this feature is really implemented how it should be, but also the knowledge that this will be tested thoroughly over a long time and any mistakes will most likely be found before production setting.
In the second case, the distance to the user is little, and there is not only a lot more excitement from the developer side, but also a very high feeling of importance. This is important, and I have to get it right first time on. There is a tendency to responsibility diversification when the distance between the user and the developer is large. The feeling of ownership and pride is likely more available to developers being able to work on stuff that is being used more or less immediately.
"There is a firm belief at Facebook that nothing should stand in the way of an engineer who wants to release a change or a new feature to production. There is no separate QA team. Engineers are responsible for the quality of their work. Changes are released daily, often multiple times a day. New features are released on a weekly basis."
The Facebook Case Study
Developers of a service at Amazon bear the primary responsibility for its operation throughout its life cycle. “You build it, you own it,”
Amazon CTO Werner Vogels said in the May 2006 issue of the Association for Computing Machinery’s Queue magazine.
Not releasing to production also creates the question: Why is it important that we finish the feature THIS sprint? Since this is no rush, we can continue working on the feature next sprint and maybe finish it then. The drive to complete something “Done-Done” is reduced. This in turn makes it less important to break features down into small parts. Bigger features increase the risk of implementing functionality we really do not need.
With infrequent releases the feature list usually grows bigger, and that adds to the complexity of the release. The worst part is that the risk of errors probably increases more than linearly with the number of features. Time and effort spent in finding the errors also increases rapidly with growing complexity of the features.
The famous Standish Group Survey of 2002 on feature usage in delivered software applications indicate that the minor part of what is developed actually gives value to the business:
While developers might be getting increasingly better at doing things right, there is a greater risk of not doing the right things. The earlier we are able to get feedback from real life use, the sooner we can focus on the right things to work on.
When there is a 12 month release cycle, there basically is a great risk that we have spent considerable resources on implementing features that will not be used.
In the left graph we have an up-front decided scope with no or little correction underway. The percentage of the scope we are unsure of is increasing the longer we wait getting real life feedback on what features and how the features are used. Note that it is an illusion thinking we can foresee the real value provided by the business. Only actual usage will tell. Features in the red are actually features we do not need.
In the graph to the right we have scope correction for every release. This means we are able to focus on what really gives value to the business based on actual usage information. This gives us the freedom to either deliver more of what matters or actually finish the project earlier if we wanted to.
This is also linked to conscientious Product Owners who really cares about how their product satisfies their users. When they don’t know when they will be able to release an update with a feature they have “forgot” they are more likely to add features to the release “just in case” than if they knew they would be able to furnish the users with the missing feature shortly after its detection
Loss of revenue
The less frequent we get features in production, the more completed functionality is put on hold for a long time before actually being productive. This is time wasted and basically amounts to lost revenue for the functionality that is just sitting there being unproductive.
If we were able to quantify the value of a feature into cost savings, the loss of these cost savings while the feature was done until it was actually released into production would amount to waiting time x cost saving per time unit. This will be true only if the value of the feature was constant over time, which we know is usually not true. Feature value curves vary among types of features. Sometimes the feature is more worth early than later. A feature used during a data transition from an old application to the new one will have less value the less need there is for transferring data. On other occasions the value is more or less stable, but then again, the future is difficult to predict. Maybe market changes will render the feature meaningless at some point in the lifecycle of the product?
Product Owner would prioritize features according to their value to the business. This will imply that the features waiting the longest for production also would be the features with the highest value to the business.
What is the Vicious Circle?
Delaying the release, leads to more uncertainty on hitting the target regarding user functionality. This leads to more features added to the release than you otherwise would add – just to make sure. It also increases complexity and demands even heavier release testing efforts - which in turn increases the cost of a release and the time it takes to get it out there.
The longer the code is waiting for a release, the more likely it is that developers will rely on later testing of it. Automating regression tests seems of little importance, since we are regression testing only once every release, and spending the effort automating it seems not to be worth the time. This will in turn keep the release process long (relying on manual testing) and also increase the cost of each release. The cost of a release is not in itself a reason not to do it often.
So, what to do about it?
The release frequency must be kept as high as possible to maximise value to the business. This involves amongst other things changing how we perceive each release.
In some companies the UAT is a “must-have” and unavoidable. This usually sets great limitation on the release frequency – and increasingly delays the release the bigger the release gets.
But is it really necessary?
According to Wikipedia, the definition of a User Acceptance Test is;
“In engineering and its various subdisciplines, acceptance testing is a test conducted to determine if the requirements of a specification or contract are met. Software developers often distinguish acceptance testing by the system provider from acceptance testing by the customer (the user or client) prior to accepting transfer of ownership. In the case of software, acceptance testing performed by the customer is known as user acceptance testing (UAT), end-user testing, site (acceptance) testing, or field (acceptance) testing.”
UAT’s make sense when you develop according to a clear up-front specification and are left more or less to yourself during the development cycle, but it really does not make sense when you have an iterative process with frequent releases to production with the ability to correct any misinterpreted requirements both through demonstrations and through active use of the features by the users themselves. The UAT can only verify that the implemented requirement is according to the initial specification – not according to the current need.
In an agile environment the whole process of frequent releases of small changes with immediate feedback from the real world is a more efficient way of ensuring highest possible value to the business than running a test that only verifies compliance with – sometimes – old and outdated requirements. So where is the value in keeping the UAT? Especially if it is adding cost and keeping us from releasing important functionality to value producing environments sooner?
We cannot do without testing, and thorough regression testing is needed to keep the cost of errors low. If this largely is a manual process, it will also delay the release and add cost to each cycle – in itself reducing the value of frequent releases. Automating this is therefore very important.
According to Capers Jones in Applied Software Measurement: Assuring Productivity and Quality, the cost of bugs is increasing by time since it was created. Automated tests will lower the cost of regression tests, allowing them to be run more frequently, reducing the bug-finding time.
A bug found during deployment is orders of magnitude more costly in both finding and fixing than bugs found when you still are implementing the code where the bug is created. In some complex systems, regression testing is not easily automated, and will take a fair amount of resources to implement. This investment will pay itself back many times over on increased developer attention and quality.