Agile Development Drives Better Business Results

Imagine a world where you are predictably delivering high quality products to happy customers faster than ever before and realizing more revenue and higher profits as a result. A world where compliance is automatic, audits are a breeze, and expenses are kept under control.

In this world builds don’t break, developers are happy and productively working on the right things. People are working smoothly towards common goals at multiple sites across the globe. Using parallel development, multiple projects are worked on all at the same time, customer updates are made quickly, and there are fewer regressions due to fixes that didn’t make it into a new release.

More and more folks are experiencing this world as they upgrade their development environment. By development environment I mean the combination of development tools and development methods. Just as C replaced assembler, C++ replaced C, and Java and C# are replacing C++, new development tools are replacing the old ones and Agile development is replacing traditional development methods.

Change is in the air. The terms “Agile Development,” “Extreme Programming,” “continuous integration,” and others can be heard more and more. They are in the titles of conference sessions and books, they are showing up in new publications and web sites, and more and more companies are adopting these new methods.

It has been widely stated that the benefits of Agile Development are producing value, maturity, and feedback faster; allowing feedback to inform functionality to produce a better product, and increasing flexibility. Getting these benefits is much easier using more modern software development tools.

A lot has changed in the software development world in the last 10 years. If you’ve only ever used the older tools such as Bugzilla, CVS, ClearCase, ClearQuest, PVCS, VSS, SCCS, Synergy, or Dimensions, you are missing out on a whole new world of options. There are many exciting new entrants in the world of software development including AccuRev, Ant, Cruise Control, Jira, Mojo, Perforce, Surround SCM, Subversion, Team Foundation, and Team Track. As a simple example, none of the older SCM systems provide atomic transactions, but all of the newer systems do!

There are really only four ways to increase the profitability of a business based on software development: reduce costs via outsourcing, reduce headcount, reduce other expenses, or increase revenues. In the end, reducing expenses can only go so far. By far, the most expensive part of software development is the people. Compared to your people costs, the cost of migrating to a new open source tool or purchasing a new commercial software development tool is a drop in the bucket. The most successful way to increase profits is to increase revenues or increase productivity. These new tools and development methods can help you increase your revenues by helping you get your products to market faster with higher quality.

Refactoring and the Law of Unintended Consequences

Refactoring has the advantage that over time it makes code easier to understand and maintain. This can contribute to the maturity of the code. It has the disadvantage that it requires changing the code and any change has the potential to reduce the maturity and stability of the product.

The so-called “Law of Unintended Consequences” states that any human action will always have at least one consequence that was not intended. The bad part of this is when the unintended consequence causes a problem. Another way to look at this is that each change will have an effect; one of the effects is the reason for making the change in the first place. All of the others can be classified in two different ways: expected or not and reasonably predictable or not. When making a change, there may be effects that are expected even though they were not the reason for making the change. As long as they are considered to be beneficial, that’s probably ok. If there is any reason for not wanting the effect, even if some observers might consider it beneficial, then it is probably best to think of the effect as non-beneficial.

The other classification is whether or not an effect can be reasonably predicted. Some effects might not be immediately obvious. As with chess, the effect of moving a queen into a capture square of a pawn is immediately obvious. Generally, it means that the queen is going to be captured. However, it might also be the case that the queen was moved there in order to guarantee checkmate in five moves. Unless the other player is looking ahead at all possibilities five moves in advance, there is a very good chance that they will miss this. Thus, the worst part of the law is unintended effects which are harmful and cannot be reasonably expected to be predicted.

Some changes have a non-linear effect on a system. That is, the change was very small – perhaps a single character was changed, but the impact was very large, such as destruction of large amounts of data. Because the amount of change often has no connection to the potential impact, it is always best to keep the amount of change to the absolute minimum required.

Even if a change could be reasonably expected to be predicted, sometimes the effort to do so is not invested. In any case, any effects that were not determined prior to incorporating the change are considered to be unforeseen consequences.

There are some strategies for minimizing the risk of unforeseen consequences. The first is to make a reasonable effort to determine effects other than just the intended one. The second is to do an impact analysis and either verify that the current requirements and code coverage are sufficiently high to mitigate the risk or to increase coverage to a reasonably high level.

The Agile Development Waterfall

These days, there is a lot of criticism of the “waterfall” model of development. I believe that is because it is used as a blanket term that actually has three important subcomponents. In a classic waterfall project, the duration of the project is however long it takes with the resources provided to produce the software which meets all of the requirements. There are set process stages that each work item goes through such as requirements gathering, requirements specification, design, development, integration, etc. Lastly, there is the characteristic that all of the work items go through the process stages in lock step.

I submit that all software development projects share an essential part of the “waterfall” model: the steps of the waterfall itself. This is out of necessity. When was the last time you saw code spontaneously pop into existence without being written, or code being written without any goal in mind, even if it was as vague as “make it better?”

All changes to software go through a waterfall of stages. Even if it is a fuzzy idea, change starts with a concept for what needs to be done. From this concept comes a plan. The plan may exist only as a flash of insight in the mind of the developer and they may not even be conscious of it, but there is always a plan. From the plan comes the code. Lastly, the proof of the original concept comes from trying the result. Thus, even in the least rigorous of environments, working on a single issue, there is at least a 6 step waterfall.

All of the stages described may be intermingled, have no clear start or finish, and be repeated multiple times, but they are there. Another way to describe them would be: need, requirements, specification, design, development, test.

The major differences between the classic waterfall model of development and other models of development come in the number and duration of iterations, number of steps in the waterfall, degree of detail in the steps, and the degree of synchronicity between the steps and between the work items.

For instance, the classic waterfall has a single iteration where the duration of the project is however long it takes with the resources provided to produce the software which meets all of the requirements, there is a very high degree of detail including verbose documentation every step of the way, and all of the work items progress together from one step to the next.

In the Agile model, there are frequent short iterations which consist of self-contained projects representing the highest value work that can be done during a single iteration, there is a minimum of detail and documentation, and the work items may or may not progress together from one step to the next, but those steps are still there.

I suggest that the value of the waterfall model, when applied to a single work item, is in producing a well defined series of steps which when followed is known to detect and filter out problems. For any particular development team, those steps may vary a great deal, but by refining the steps and applying them consistently, you have a higher likelihood of producing consistent results.

Next: Sustainable Pace: Supply vs. Demand

Agile Development Objection: Customers Don't Want Frequent Releases

A common objection to Agile Development is that customers don’t want frequent releases. To me, this is one of the stranger objections. It is true that very few customers want frequent releases. Nothing says that a particular customer has to take every release that you produce. In fact, they probably don’t do that no matter how frequently or infrequently you produce new releases. At AccuRev, we’ve had customers that stayed back on 2 year old and even 3 year old releases, despite the fact that bugs they had reported had been fixed and features they had asked for had been added.

Basically, the frequency of releases has nothing to do with the frequency of customer uptake of releases. The exception is when the customer has an urgent need. In that case, you had better be able to attend to that need quickly. Since that is the case anyway, again there is no connection between the frequency of regular releases and customer uptake of releases.

Customers take new releases when there is a good reason to do so. Usually, that is when the delta between where they are and the current release is big enough to warrant the disruption and they have a place in their schedule to put it. As the provider, this window is pretty much unpredictable for you. When that time comes, the customer will take your most recent release or perhaps the one before that under the assumption that more is known about the state of the older release. If you have more than one customer, the chances that your entire customer base is ready to take a release when you release it is pretty low.

One of the advantages of frequent releases is that there is a high likelihood that one of your customers will move to it fairly soon after the release, thus giving you an opportunity to gather feedback right away.

If you are just not ready to release frequently, that’s ok. Instead, use short iterations and just go on to the next iteration instead of releasing it. You can still get most of the feedback related benefits of Agile Development by having folks take a look at a demo of the new iteration and avoid the risk of disrupting your customer base.

Next: Tuning the Frequency of Your Release

The Funniest Software Development Video Ever

What happens when developers get fed up with ClearCase? Click here to find out!

Hyper Agile Q&A

I'm still working on putting together a comprehensive description of Hyper Agile. In the meantime, this recent article from Tech Target is a great overview of Hyper and how it benefited the development of AccuWorkflow.

Hyper Agile Project Wins Jolt Award

It was incredibly exciting to accept the Jolt Product Excellence award at SD West last week! I could barely speak. It was a real validation of the AccuWorkflow vision as well as the effectiveness of the Hyper Agile methodology. And behind it all was a great team. I'm very proud of the people at AccuRev that made it all happen. Thanks!!

Everything You Need to Know About Scrum

If you're interested in learning more about Scrum, but just don't have the time to read a book on the subject, you can quickly pick up the basics from this video produced by High Moon Studios.

Hyper Agile Project Now a Jolt Finalist!

AccuWorkflow, a new product from AccuRev that allows the use of a single workflow for both issue tracking and code development, has been announced as a Jolt Finalist! I think AccuWorkflow is pretty cool in its own right, but it never would have been nominated if it hadn't been for Hyper Agile. By following this methodology, we were able to do much more with less and produce a high quality product at the same time.

A Preview of the Hyper Agile Methodology

In their February 2007 issue, Dr. Dobb's has just published an article that I wrote on best practices for global development titled "Think Globally, Code Locally". It also happens to overlap with the Hyper Agile methodology. So, for a preview of Hyper Agile, check it out!