Tuesday, November 27, 2007

No Really, What is Agile?

[Revisions: 6/11/08 general updates, 1/9/08 responding to comments]

Giving a concise definition of Agile is far from easy, probably because Agile is actually an umbrella for a wide variety of methodologies. Some will say that "it isn't about methodologies, it is a philosophy; it is the 4 values defined by the Agile Manifesto." Philosophies and Manifestos are a fine place to start, but at the end of the day you have to get down to brass tacks in order to actually bring about change.

It may be pointless to define Agile development. It may be better to just talk about the concepts and the practices and say "these things are worth learning about regardless of what you call it or how you define the overarching concept." I still think it is worth a shot so as to establish a starting point for discussion. So, let's discuss the definition of Agile a bit and then move on to the nitty-gritty.

Because there are so many practices associated with Agile development, a simpler way to define Agile is to do it in terms of the benefits. It is not a perfect way to define something, but it is currently the best way that I know. There would be no point in doing Agile development unless it was better, so I define the benefits in relation to traditional development. I consider waterfall to be part of traditional development, but I use the phrase "traditional development" instead of waterfall because there are many shops which would say "we're not doing Agile, but we're not waterfall either."

I define Agile development as that which, in comparison to traditional development provides the benefits of more flexibility, higher ROI, faster realization of ROI, higher quality, higher visibility, and sustainable pace. Let’s take a brief look at these benefits.

More Flexibility/Options
With traditional development the software may look like a construction site until just before the release: some parts are done, others are not, but the building is not currently usable at all. Changing plans during the development cycle means ripping out work in progress and possibly work that is done but depends on other work that is not done.

With Agile development, you develop your software in short iterations where “short” means a month or less. At the end of each iteration you have software which is shippable. That means that at the end of each iteration you have the option to easily change plans before the beginning of the next iteration in order to take into account new business requirements or information gained during the previous iteration.

Higher ROI
Because you have more flexibility you can change your plans to take into account the current market conditions instead of what seemed like a good idea when you started working on the release. So, if traditional development would have released after 6 months and had features A, B, and C which had an ROI of 9 during planning at release has an ROI of 7, Agile might produce A, B, and D in the same timeframe which has an ROI of 11. The idea is that while C looked like a good idea at the beginning it turned out later to be worth less than expected and D came along which had much more ROI so we did that instead.

Faster Realization of that ROI
Since you have shippable software at the end of every iteration you can start realizing the ROI whenever you like instead of having to wait. If the Agile team is also releasing frequently, they are able to capture the value that they created sooner than they would have otherwise been able to do.

Higher Quality
In a traditional project, the elapsed time from start to finish from the perspective of any individual work item is very long. There is no consistent process applied to each and every work item. Instead, work items are fused together into “the release” and the quality of “the release” is measured.

One consequence of this is that QA gets a big dump of functionality near the end of a release and has to figure out how to best use the time remaining which is often less than originally planned. That means taking shortcuts and the "most important" new features get very thorough testing and the rest get a "spot check." Another problem with traditional projects is that since much of the test writing and execution is left to the end, problems hide out for long periods of time and thus there is a long time between introduction of a problem and the detection and fixing of that problem.

In an Agile project, the elapsed time from start to finish from the perspective of any individual work item is very short. Test plans and automated tests are created throughout the project and each work item is given the amount of QA resources that is appropriate for it. Because problems are found and fixed faster, there is less chance of the quality of a project being poor for long stretches of time. Code is written on a stable base and is more likely to be stable itself because there will be accurate and timely feedback on the results of the changes.

Higher Visibility
Time after time, with traditional development, progress is measured based on progress against a plan. The problem with that is that customers don’t buy Gantt charts, they buy shipping software and the progress that is measured by Gantt charts is not directly connected to shippable software.

Just because you have finished 100% of the requirements which is 20% of your plan, that doesn’t mean you are 20% done. In fact you are 0% done from the perspective of the customer because they can’t benefit from that progress until you ship. In a traditional project, you only know how much ahead or behind plan you seem to be based on the progress that people claim, but you don’t really know how much ahead or behind plan you actually are from an “is this shippable” perspective.

Short iterations solve the problem of false progress reports. With short iterations, you know at the end of each iteration exactly how much progress you have made. Instead of the traditional "we're still on target" all the way up to just before the end of the release, you know at the end of every iteration exactly where you are.

Whatever work was done during that iteration is done and potentially shippable. You know exactly how many work items were fully completed, how many weren’t started, how many test cases remain to be written and automated, how many test failures remain to be fixed, and how many defects were filed on work done during the iteration. You get the kind of information that is usually only available just prior to release on a monthly or even weekly basis.

Next: Sustainable Pace

9 comments:

Mike Coon said...

I think you've got it figured out but I tossed my own two cents in too with YADFAD. I think one of the things that makes Accurev great is that the developers are the users and you guys think of stuff that helps others do their job. Kinda the eat your own dogfood thing. Thanks for sticking with this.

Mike

Anonymous said...

"Many shops say they are Agile, but that doesn't make it so...."

I think that's a good observation.

I've seen Waterfall shops that could care less about code reviews and test plan reviews, much less nailing down requirements with the customer, or changing requirements at the last minute and causing panic. They had layers and layers of bureaucracy and nobody of importance seemed to talked to each other. I call that seriously dysfunctional and a case of bad management.

On the other hand, I've seen Agile shops that could care less about story planning, much less nailing down requirements with the customer, or changing stories at the last minute and causing panic. They had layers and layers of bureaucracy and nobody of importance seemed to talked to each other. I call that seriously dysfunctional and a case of bad management.

I have read enough to know that the Agile Kool-Aid drinkers think they're immune to such concerns. Sadly, that is not the case. Instead, the Agile types keep moving the goalposts: "Oh, that's not Agile!" or "They're doing Agile wrong!"

Guess what? Eventually you'll move the goalposts so far that you'll hit the stands.

Enough big crappy companies with too much money and filled with dumb upper management will "try Agile" (they're already doing this) and where is your movement going to be when they totally bastardize it and projects fail? Well, Kent Beck and his gang started this movement with a huge failure of a project at a big company, but you always conveniently ignore that history. Perhaps in spite of that, Agile still seems somewhat immune to being looked at in a mocking manner except by Scott Adams.

Don't get me wrong there are a few good aspects to Agile, XP, Scrum, etc, but not enough to where I'd totally change philosophies and become a devotee and get baptized. But you people who are neck deep in the movement need to wake up and quit making excuses when the management of companies is turning your way of life on its head. Agile is on the verge of becoming a laughingstock, and already is in some arenas. Just try going to Slashdot and posting your nonsense. They can be quite vicious.

I suppose my point is: don't be afraid to be self-critical about Agile. Don't make excuses.

Me! said...

I think that the tendency to make out Agile as a "cult" is a very clever tactic used to discredit the actual people in the trenches who have successfully used the methodologies. If you can't successfully engage with the ideas, it is much easier to stoop to personal attacks.

I think your description is spot-on, especially the piece that Agile relies on quality. Unmaintainable software is unmaintainable software, regardless of how it's been produced. Agile just relies on it's software being maintainable to deliver value Waterfall development never promises.

I usually attempt to communicate Agile development in terms of releases and communication. Waterfall development intends to deliver once, whereas Agile attempts to deliver periodically. This allows Agile to communicate what it is possible to accomplish before the next release with more accuracy than Waterfall, especially after the first release, and to honestly and openly shape development based on other people's priorities. Most manifestations of Agile development I've encountered share these features.

But this is the other thing; there is only so far you can generalize. XP went off in a completely different direction, but since then many threads and theories have spun off of it and gone in all different directions. To paint all of them with the same brush is to do both them and the methodology a disservice.

Anonymous said...

You basically said in your opening section (I'm not reading further because what you said was so ridiculous) that if you are successful using Agile techniques then you are Agile, if you are unsuccessful using Agile techniques then you are not Agile.

Shall we define waterfall or traditional programming in the same way: they are only so if they are successful? This seems like an easy, and sleazy, way to redefine success in Agile's favour.

Anonymous said...

My thoughts exactly: You cannot define a methodology in terms of the benefits it is supposed to provide.

I guess this is a useful definition for Agile evangelists because with this definition every Agile project is successful (any project that is not successful is, by this definition, not Agile - hey presto, 100% success rates with the methodology!).

A further weakness of this approach to 'defining' Agile is its reliance on being better than other methodologies (i.e. all the points require getting better results than other methodologies would give). So, apparently you are Agile if you're getting better ROI than other methodologies.
With this definition Agile takes the high ground: 'Whatever methods you use, if its better than what you might have got using a different method, then you're on the Agile track.'. Right!

Damon Poole said...

You may not like my definition of Agile. That's fine with me. I don't like it either! I'm much more interested in discussing what Agile brings to the table than trying to nail down an exact definition. Sometimes I think that it may be pointless to define Agile development. It may be better to just talk about the concepts and the practices and say "these things are worth learning about regardless of what you call it or how you define the overarching concept." I still think it is worth a shot so as to establish a starting point for discussion. So, let's discuss the definition of Agile a bit and then move on to the nitty-gritty.

I've updated the original post to take some of the comments into account.

Anonymous said...

Seems like the key to Agile development is determining what functionality can be included with each release. This planning must include interdependencies and definition of 'core' functionality (i.e. code that other functions are built on). Maybe you have this somewhere, but criteria for deciding what functionality to introduce in each release would be helpful. Issues include: How solid is the requirement (vs subject to change), What interdependencies exist, etc..

Damon Poole said...

In response to the questions "what functionality can be included in each release?" and "criteria for deciding what functionality to introduce in each release" :

Keep in mind that iterations and releases are not synonymous. You might have 6 iterations (or more) between releases. That said, each iteration should be shippable, so I'll answer your question as though each iteration was a release.

In any case, the single criteria for what to include in a release is business value. The content of an iteration is whatever the top business value is that will fit into the next iteration.

In most cases, that business value will be determined by a variety of factors including your business strategy.

In regards to interdependencies including the need to create foundation libraries first, Agile treats this issue a bit differently using something that is called incremental design, aka emergent design or iterative design or evolutionary design. There is more information in the following links:

Agile Adoption Stage 2 See the section "The Backlog".

Iterative Design of a Lego Sports Car

Is Design Dead? Martin Fowler talking about evolutionary design.

vadi said...

I actually enjoyed reading through this posting.Many thanks.


Agile Training