No Really, What is Agile?

[Revisions: 6/11/08 general updates, 1/9/08 responding to comments]

Giving a concise definition of Agile is far from easy, probably because Agile is actually an umbrella for a wide variety of methodologies. Some will say that "it isn't about methodologies, it is a philosophy; it is the 4 values defined by the Agile Manifesto." Philosophies and Manifestos are a fine place to start, but at the end of the day you have to get down to brass tacks in order to actually bring about change.

It may be pointless to define Agile development. It may be better to just talk about the concepts and the practices and say "these things are worth learning about regardless of what you call it or how you define the overarching concept." I still think it is worth a shot so as to establish a starting point for discussion. So, let's discuss the definition of Agile a bit and then move on to the nitty-gritty.

Because there are so many practices associated with Agile development, a simpler way to define Agile is to do it in terms of the benefits. It is not a perfect way to define something, but it is currently the best way that I know. There would be no point in doing Agile development unless it was better, so I define the benefits in relation to traditional development. I consider waterfall to be part of traditional development, but I use the phrase "traditional development" instead of waterfall because there are many shops which would say "we're not doing Agile, but we're not waterfall either."

I define Agile development as that which, in comparison to traditional development provides the benefits of more flexibility, higher ROI, faster realization of ROI, higher quality, higher visibility, and sustainable pace. Let’s take a brief look at these benefits.

More Flexibility/Options
With traditional development the software may look like a construction site until just before the release: some parts are done, others are not, but the building is not currently usable at all. Changing plans during the development cycle means ripping out work in progress and possibly work that is done but depends on other work that is not done.

With Agile development, you develop your software in short iterations where “short” means a month or less. At the end of each iteration you have software which is shippable. That means that at the end of each iteration you have the option to easily change plans before the beginning of the next iteration in order to take into account new business requirements or information gained during the previous iteration.

Higher ROI
Because you have more flexibility you can change your plans to take into account the current market conditions instead of what seemed like a good idea when you started working on the release. So, if traditional development would have released after 6 months and had features A, B, and C which had an ROI of 9 during planning at release has an ROI of 7, Agile might produce A, B, and D in the same timeframe which has an ROI of 11. The idea is that while C looked like a good idea at the beginning it turned out later to be worth less than expected and D came along which had much more ROI so we did that instead.

Faster Realization of that ROI
Since you have shippable software at the end of every iteration you can start realizing the ROI whenever you like instead of having to wait. If the Agile team is also releasing frequently, they are able to capture the value that they created sooner than they would have otherwise been able to do.

Higher Quality
In a traditional project, the elapsed time from start to finish from the perspective of any individual work item is very long. There is no consistent process applied to each and every work item. Instead, work items are fused together into “the release” and the quality of “the release” is measured.

One consequence of this is that QA gets a big dump of functionality near the end of a release and has to figure out how to best use the time remaining which is often less than originally planned. That means taking shortcuts and the "most important" new features get very thorough testing and the rest get a "spot check." Another problem with traditional projects is that since much of the test writing and execution is left to the end, problems hide out for long periods of time and thus there is a long time between introduction of a problem and the detection and fixing of that problem.

In an Agile project, the elapsed time from start to finish from the perspective of any individual work item is very short. Test plans and automated tests are created throughout the project and each work item is given the amount of QA resources that is appropriate for it. Because problems are found and fixed faster, there is less chance of the quality of a project being poor for long stretches of time. Code is written on a stable base and is more likely to be stable itself because there will be accurate and timely feedback on the results of the changes.

Higher Visibility
Time after time, with traditional development, progress is measured based on progress against a plan. The problem with that is that customers don’t buy Gantt charts, they buy shipping software and the progress that is measured by Gantt charts is not directly connected to shippable software.

Just because you have finished 100% of the requirements which is 20% of your plan, that doesn’t mean you are 20% done. In fact you are 0% done from the perspective of the customer because they can’t benefit from that progress until you ship. In a traditional project, you only know how much ahead or behind plan you seem to be based on the progress that people claim, but you don’t really know how much ahead or behind plan you actually are from an “is this shippable” perspective.

Short iterations solve the problem of false progress reports. With short iterations, you know at the end of each iteration exactly how much progress you have made. Instead of the traditional "we're still on target" all the way up to just before the end of the release, you know at the end of every iteration exactly where you are.

Whatever work was done during that iteration is done and potentially shippable. You know exactly how many work items were fully completed, how many weren’t started, how many test cases remain to be written and automated, how many test failures remain to be fixed, and how many defects were filed on work done during the iteration. You get the kind of information that is usually only available just prior to release on a monthly or even weekly basis.

Next: Sustainable Pace