Mary Poppendieck to Speak. Waltham, MA Jan 7th, 6pm

As I write this, 52 folks have already registered for Mary's talk and there are only 120 total spots available. Her talk is titled "The Leadership Team and the Software Crisis: A Cautionary Tale" . Mary's talk will tell the story of a large company as it attempts to turn theory into practice, uncover barriers to sustainable change, and look in unexpected places for the root cause of the software crisis. She will also discuss governance and the role of the leadership team in developing software-intensive systems.

If you've never heard Mary speak, now is your chance. She's a terrific speaker who always has great material. And to top it all off, this event is not only free, dinner is included!

Registration has only been open for a couple of days. If you are thinking of going, now is the time to register!

Second Edition of "Do It Yourself Agile" Available as Free Download

At the beginning of the month I released the first version of "Do It Yourself Agile." The response was tremendous and I am very grateful to the many people that blogged and tweeted about it. I am also very thankful for all of the feedback and suggestions that I have received. In total, the book has now been downloaded more than 4,000 times.

Here is the second version, which takes into account all of your feedback and also incorporates new sections based on recent blog posts. Please keep the feedback coming. If there is something you feel is missing, let me know. New material usually starts out as a blog post first, so you'll get immediate feedback.

While I believe that an Agile coach, whether recruited internally or externally, is still the best, fastest, and least expensive (in terms of ROI) path to Agile success, not everyone can do that. It may be politics, budget, or some other reason that prevents folks from using seasoned coaches. In any case, DIY Agile is here to stay, and the book "Do it Yourself Agile" is intended to be a resource you can lean on as you transition to Agile on your own.

Let me know what you think. I look forward to producing more versions incorporating your feedback.

The book is freely downloadable, no strings attached. I only ask that if you point people to the book, please point them to this blog post rather than linking directly to the pdf, copying the pdf, or providing the content in some other format.

"Do it Yourself Agile" (pdf) 180+ pages

"Do It Yourself Agile" - condensed web version

Planning Poker Reduces Risk and Waste

There are three particularly valuable things that can happen during a Planning Poker session. They may also happen during any flavor of estimation meeting, but seem to be more likely when using Planning Poker.

The Bigger They Are, The Harder They Fall

I recommend that you never use a story size greater than 13. Most stories that are estimated at 8 points or above can be split into smaller stories. You should always be looking for opportunities to break larger stories into smaller stories. If you have an 8 point or larger user story, it is probably actually two or more stories in disguise. The bigger the story, the more likely your estimates are wrong and the more likely that you have many smaller stories masquerading as one large story.

For instance, you may find that a story that originally looked like a 20 point story was really two 8 point stories, one 5 point story and 2 three point stories for a total of 27 story points. Better to break that huge story down into its five constituent stories and estimate them each individually.

But remember, a story is only a story if it provides value to the user. Splitting a story up into “As a user I want the backend for X” and “As a user I want all of the UI for X” is not the right way to go. If you can’t create smaller stories that still provide user value, then the story is already as small as you currently know how to make it.

Know When to Walk Away

You may realize while discussing a story that the story contains a big unknown, something that feels like it won’t be resolved during the implementation of that story. In that case it is best to split the story up into a research story and the story itself. For instance, “As a developer I want to know how to XYZ.” That way, if you never do figure out how to do the unknown part, you haven’t invested any effort into the overall story. This technique should only be used as a last resort, but it is much better to do this than to know going in that there is a big unknown and have to pull the whole story out near the end of the iteration.

Know When to Fold’em

Lastly, you may decide that you just don’t have enough information to estimate a story. In that case, you should have a mechanism for informing the product owner such as marking the story “need more info.” There’s no point spending time on estimation if there is insufficient information to do so. You’ll just be glossing over the problem and producing a false sense of security.

Breaking out research stories and kicking stories back to the product owner may seem like procrastinating, but it tends to build good habits. It keeps the team from committing to work that includes research projects, clearly defining some stories as research stories, and keeping the product owner on his or her toes.

Counting Your Chips

In summary, if you are mostly working on small user stories that have end user value, you are reducing the chance that you are putting something into the product that never gets used and you are also reducing the chance of starting work on something that never gets finished or has to be discontinued part of the way through. The smaller your stories, the smaller your risk and the less effort you’ve wasted when you run into problems.



See Also: "The Five Essential Ingredients of Great Agile Estimation"

The Five Essential Ingredients of Great Agile Estimation

Planning Poker is a useful Agile estimation technique (see previous post "Introduction to Planning Poker"), but it is even more useful when used in conjunction with whole teams, user stories, velocity, and story points. I refer to these five practices, used together, as the five essential ingredients of great Agile estimation.

Whole Teams

A key practice of Agile development is the use of whole teams. A whole team is a cross-functional team comprised of between 5-9 people. By cross-functional I mean the team as a whole contains all of the skills needed to accomplish the team’s goals, not that each team member is capable of doing any task. If you have more than 9 people, then you split people up into multiple whole teams.

Planning Poker reminds everybody that the estimate includes all of the work that needs to be accomplished in order for the story to be considered done. It includes development, testing, documentation, and anything else required for the story to be considered done. It is a reminder that nobody on the team gets credit for a job well done until the whole story crosses the finish line. Instead of people saying “well, I finished the development, I don’t know what is taking the tester so long” they are more likely to say “what problem are you running into and how can I help?”

User Stories

Because user stories are a simple and easy to understand description of the work, user stories allow you to focus on estimating rather than spending lots of time discussing what a particular enhancement request or requirement really is. To maximize the benefits of Planning Poker, you need to be good at creating and using user stories.

Story Points

In my experience, the best unit to use for estimates is story points. Two different people with two different skill sets or levels of ability in an area may take different amounts of time to perform a particular task. Estimating in hours mixes together the scope of the work that needs to be done with the speed at which a particular individual can do that work.

On the other hand, story points are a relative measure of the scope of a user story. Story points separates out the “what” from the “who.” For instance, if you have one individual that is stronger with .Net than with Java, they will estimate a Java story as taking more hours than somebody that is stronger with Java. But they will probably both agree that something that is twice as easy to implement will take half as long to do.

To use story points, you need to create a relative scale of scope. A simple approach is to find a simple and straightforward story that you use to represent a single story point. Then think of stories that are 2, 3, 5, and 8 times larger in scope. You should have a couple of examples for each story point value to take into account that some stories have more test than coding, more documentation than test, etc.

Story points are primarily used for planning, not for implementation. Story points are used to help determine the contents of an iteration by calculating a velocity.

Velocity

In Agile, the velocity of a team is simply the number of story points associated with stories that are finished in an iteration. For instance, if the team completed 8 stories that were each 5 points in an iteration, then their velocity for that iteration was 40 story points. In a stable team, a team that is comprised of the same individuals working full time as part of that team, the velocity is a good measure of the overall throughput of the team.

Knowing your velocity helps with planning. For example, if you know that the velocity of your team is 40 points, then you know you can expect 40 story points for each iteration. The team decides which stories to take based on the backlog which is maintained by the product owner.

More Than the Sum of The Parts

Regardless of which of these practices you are currently using – whole teams, user stories, story points, velocity – Planning Poker is an excellent tool. The more of these practices you use, the more they reinforce each other and the more value you will get out of Planning Poker. Conversely, the more you use Planning Poker, the more value you will see in implementing all of these practices.



Next: Planning Poker Reduces Risk and Waste

Introduction to Planning Poker

Many of the techniques of Agile development combine together to provide more than the sum of their parts. One technique that definitely fits this description is the Planning Poker method for doing estimation. In part 1 of this post, I will quickly describe the mechanics of Planning Poker. I will then go on to describe how user stories, whole teams, story point estimation, and velocity all work to multiply the value of Planning Poker and how Planning Poker also reinforces the use of and value of the other practices. If you are already familiar with Planning Poker, you may want to skip ahead to "The Five Essential Ingredients of Great Agile Estimation".

Planning Poker is an estimation technique first described by James Grenning in 2002 in a paper by the same name [pdf]. When I first heard about Planning Poker, I couldn’t help but chuckle. It seemed a bit silly to mix software development and poker. After reading Mike Cohn’s excellent book “Agile Estimating and Planning” I had a good grasp of the mechanics of Planning Poker, but it felt like just a variant of the Delphi method and thus nothing new.

About a year ago we had Kenny Rubin come in to do some Scrum training for us. That was just the shot in the arm that we needed to supercharge Agile adoption at AccuRev. If you are looking for an Agile trainer, I highly recommend Kenny Rubin. It was based on Kenny’s explanation and recommendation that we decided to try Planning Poker on a real project. The results were terrific and after a single session we were hooked. Planning Poker is now a standard part of our Agile process.

The Basics
In order to play Planning Poker, each participant will need a deck of Planning Poker cards. These are easy to obtain, just Google for “Planning Poker cards” or make your own. You will need the following cards: ½, 1, 2, 3, 5, 8, 13, 20 and a card that indicates “I’ve had enough.” The numbers on the cards represent estimates. You can use story points, ideal days, or some other measure, but in this point I am assuming (and advocating) story points. Typical decks contain cards for much larger numbers than 20, but I think you’ll find that 20 and higher are rarely used once you’ve been using Planning Poker for a year or two.

The whole team gets together for a set amount of time to do story estimation. An hour is usually a good amount of time, but this is completely up to you. When I say “the whole team” I am assuming that you are using small cross-functional teams of 5-9 people (see later parts that discuss the use of whole teams).

Estimation is done on a story-by-story basis in the same order as the backlog, starting with the first story that needs estimating. Somebody reads and describes the story. This is often the product owner, but could be anybody. Some teams do story point estimation without the product owner and just skip stories which they are unable to do without the product owner’s involvement. After the story has been read, participants discuss the story for a bit and then decide on an estimate by picking one of their cards and laying it face down in front of themselves. Once everybody is ready, all of the estimates are shown. If the estimates are all the same, you are done.

Usually, some of the estimates are particularly low or particularly high. A low estimate can indicate that the estimator left something out, or possibly that they know a way to reuse existing work or have some other good information that other folks weren’t aware of. A high estimate may indicate many things, but most commonly it means that the estimator is thinking of something that other folks may not be aware of. For instance, somebody may point out that there is no test framework or tooling for a particular technology and that to adequately test the story the team will need to do a lot of setup work.

In any case, after a round of discussion, everybody chooses a new estimate. This is repeated until the team comes to consensus on the estimate. The estimate is given to the story and then you move on to the next story. You end when you hit the end of the meeting time, you run out of stories to estimate, or somebody plays the “I’ve had enough” card.

The Benefits

One of the biggest benefits of Planning Poker is the sense of team that it creates. The whole team is participating in the estimation. This creates a greater sense of team ownership and team responsibility for each story. Another major benefit of Planning Poker is that you leverage the collective wisdom of the whole team. However, this really only works well when you are using whole teams in the Agile sense of the term.



Next: The Five Essential Ingredients of Great Agile Estimation

Software Development is a Search Algorithm

How we think about something can have a profound effect on how we do that thing. Thinking of the process of software development as a search algorithm rather than as the process of building software can lead to making better decisions about the products we create and the way we go about creating them.

The Search For Value
The end goal of designing software is to produce something which provides a benefit that is large enough for one or more customers to at least cover the cost of its production. Of course, the higher the value of that benefit, the more that folks will pay for it. On the other side of the equation you want to provide that benefit at the lowest cost. There is a point of diminishing returns both in looking for the highest possible value and in looking for the lowest possible cost.

As much as we might want this process to be deterministic, it isn’t. Just because you produce something doesn’t mean people will use it, and just because they use it doesn’t mean they will pay you what you want for it. The only way to truly find out is to produce something and see how much value can be realized from it, for instance by charging money for it.

The Max For the Minimum
In other words, producing software is a classic min/max search. You are searching for business opportunities with maximum value and implementations which minimize cost. You can think of it as a huge graph of opportunities and below each opportunity node is another graph of possible implementations. Complicating this is the fact that you would like the features to be close to each other from a market perspective in order to maximize the leverage of your sales and marketing departments and you would also like to have as much common implementation as possible in order to maximize the leverage of your existing implementation and engineering resources. For both you would also like to leverage your core competencies.

The faster you can traverse this graph, the faster you will unlock business value. Unfortunately, you only have a finite amount of time and resources, so you can’t run a search by producing all possible product variations and seeing how they sell. Plus, the search space is infinite. So, you have to make predictions. You look for ways to eliminate certain paths and to get more information about other paths without actually implementing them. Whatever your methods are for figuring out what to produce, they are all based around searching the solution space as fast and efficiently as possible.

Searching For Value in All The Right Places
You may think that only parts of the development process are parts of this search, but doing so will limit the effectiveness of your searching. For instance, if you arbitrarily decide that a particular feature is what your customer wants and you then spend a huge amount of time talking to your customer base about the requirements for that feature, you are limiting your search to the various ways to do that feature, but what if the perfect implementation of that feature is only worth $10? You are then searching within a low value set of possibilities. Before you dive in too deep, spend a fair amount of time at a high level so as to maximize the value of searching at the next level down.

The entire development lifecycle is part of your search. The first level of searching is to find potential high value opportunities from which to create new functionality. But don’t fall in love with a particular opportunity too quickly because implementation may cost more than the value to the customer. There are usually a variety of ways to implement something, so don’t fall in love with a particular design too quickly because there may be a cheaper implementation. And of course in order for the customer to determine whether they want to pay you or not they need to get their hands on it or at least see it in action. So, you are searching for opportunities and then within the opportunities you are searching for implementations and then you need to circle back with the customer to check the results.


Next: Specification is Planning is Design is Coding is Approximation

Specification is Planning is Design is Coding is Approximation

All Phases of Software Development are Variations on a Theme

It is easy to think of specification, estimation, planning, design, implementation, and customer feedback as completely separate and unrelated activities. However, a useful way to think about them is as variations on a theme where the theme is producing a product which provides the best return on investment as measured by the ratio of revenues to cost. Each of the phases of the development life-cycle can be thought of as a combination of discovery, prediction, and design. The prediction comes into play when you make a decision. When you decide that you will schedule a particular requirement to be implemented in your product, you are predicting that the end result is something that will provide value to the customer. You have no way of being sure until the customer has actually used it as part of their normal course of business.

What does it matter if in fact it is true that all of these activities are variations on a theme? I believe it provides a conceptual framework that can help us make better decisions about the products we create and the way we go about creating them.

Requirements Gathering is Not Design
Requirements gathering is not the process of finding out what customers need, it is the process of understanding the customer. The problem with producing elaborate requirements is not just that requirements change over time or that people change their minds, it is that users can’t tell you what they want because they don’t know. Customers cannot tell you ahead of time what they will actually spend money on. At best, requirements are an approximation of what customers need and a best guess of what will provide them value at the time that they actually use what you implement for them.

Gathering requirements from customers and the market in general is much like removing compiler errors. For any particular module that you are working on, the first error is the most accurate and accuracy goes downhill rapidly from there. After you fix the first error, the results from your compiler or IDE will probably be very different. This is the same with customer requirements. For any particular functional area, the first thing customers ask for is most likely the most accurate. Once they see the new version which incorporates that feedback, what they want next is probably very different than what they would have asked for next before they saw the new version.

The most you can hope for is to gather information and ideas that are useful in producing a successful product. If you just build whatever customers ask for, then you are ceding design to them. Customers cannot design for you. If they can then they are not just customers they are also designers. It may be that in your situation your customer is also a designer, but then you should explicitly think of them that way. Otherwise, you run the risk of making the mistake of blending the process of gathering requirements and creating a design.

When you have multiple customers, it is especially true that the customer cannot tell you what they need. It is rare that you will be able to do everything that all customers asked for, let alone everything that any one customer asked for and thus you will have to predict which combination of requests that your customers asked for will produce the best result.

Specification is Design

When you write a Marketing Requirements Document, Product Requirements Document, Specification Document, or whatever combination of documents you create along these lines and whatever you call them, let's call that "specification." A specification is a design. It is a design because you are taking all of the feedback from the customer and saying, at a high level, "this is what will provide value to the customer." You are deciding and predicting that this specification rather than any other specification is the way to go forward. It is the same as deciding that this line of code vs that line of code is the right way to go.

Specification is the First Approximation
When working with software, even if you have clear requirements/specifications that you have validated with users, it is not always clear how you will implement them.The specification is an approximation in two ways. First, it is only an approximation of what users actually want. Second, it is only an approximation of the final result. This same statement holds true at every stage of the game, from specification to design, to plans, to estimates, to the code itself. From initial customer interaction to final implementation you are building approximation on top of approximation.

I'm not advocating against specification, estimation, or design. It is useful and even necessary to produce these, and you will often hit on solutions or eliminate problems much more quickly and much more economically than jumping right to implementation, but in the end they are still only an approximation of reality, an interlocking web of educated guesses.

Not until you actually attempt to create something real will you find out how close to reality all of these approximations are and the real costs and difficulties involved. The longer you take to turn these approximations into reality, the more likely it is that they will be inaccurate and the larger those inaccuracies will be.

Estimation is Design
Estimation, which is also another word for approximation, is part of design. If somebody gives you a requirement do you just shout out a random number as the estimate? The accuracy of your estimate depends on the amount of design that you put into it. You may not think of estimation as design, but it is. Let’s say that somebody says “I need a module which will calculate the sum of an array.” If you have such a module at hand, you will give an estimate that involves integrating the existing module into the overall product. Deciding to use that module is a design decision. If you don’t have such a module at hand, you will give an answer based on your prior experience doing the exact same thing. The decision to use an approach similar to a past experience is also a design decision. It is part of creating the final design.

On the other hand, if you are given a more complicated requirement, then you will probably break it down into sub-tasks and estimate those. That forces you to think about what it will really take to implement the requirement. The process of deciding what the subtasks are is actually design. The more design you do, the more accurate you can be. The accuracy of an estimation is on a spectrum from “wild guess” to delivery of product, from complete prediction to statement of fact, from “I believe it will take 10 days” to “it took 103 hours.” The other aspect here is how much is research/design and how much is counting. When well-known tasks are what is at hand, then it is a matter of counting. If you know you need to change 20 dialogs and each change will take 10 minutes then you know the task will take 200 minutes. That’s simple counting. If you need to produce something that has never been done before, then it is a research project and you won’t have accuracy until you have removed all of the uncertainty.

Planning is Design

The specific set of features that go into a product is in fact a design. By selecting that set, you are deciding (predicting) that this is the optimal set of features to achieve a chosen goal. While it is true that you may just be choosing a set of work items that sum up to the available time for a release, that is still design, but perhaps not the optimal design.

Design is Design
Obviously, the activities that we associate with “the design process” are part of product design. The design that you have prior to implementation may be a fully fleshed out design document with lots of illustrative figures, it may be in the form of UML, or it might be entirely in your head. As with requirements and estimation, the design is also an approximation. The design that you do prior to implementation is only an approximation because during implementation you have still more design to do and you will discover information that you didn’t have during your initial design.

Implementation is Design
During implementation you are doing two kinds of design: micro-design and re-design. Micro-design is all of the little decisions you make while coding. Will you use a vector or a doubly-linked list? Will you specify an index or let the RDBMS do it for you on the fly? Re-design happens when you find out that once it came time to implement something, an unexpected problem arose. For instance, the design specifies the use of a particular third-party library, but in reality that library doesn’t provide the functionality that it claimed to provide. Another example is that the implementation works fine, but doesn’t provide the expected performance and no amount of tweaking seems to be doing the trick.

Final Validation of the Design
Finally, when you deliver new features to customers, you discover how close your approximations were. You discover what customers like and don’t like, what they use and what they don’t use, what they will pay for and what they will not pay for, what provides value and what does not provide value. This information can then be used to determine what you will put more effort into and what you will discontinue. The tighter you can make the loop from discovery to delivery, the faster you can produce the product that your customers really value.


Next: Software Development is a Search Algorithm

Management: Top Ten Things Programmers Hate About Agile

Sometimes Agile emerges from a grassroots initiative, but as Agile gains buzz, more and more managers and executives are initiating the move to Agile. Folks in management are often caught off guard by the reactions they get from their programmers. I’ve compiled a list of common gripes that I have heard and/or experienced firsthand to help folks in management avoid getting off on the wrong foot.

While this list may also apply to folks other than programmers, this post was created specifically with the good folks over on reddit/r/programming in mind. By the way, before posting there, I highly recommend reading "The Unspoken Truth About Managing Geeks."  If you are a programming subreddit regular, thanks for visiting. I look forward to your feedback and hope that folks in management reading this take your feedback to heart. Without further ado, I give you the Top Ten Programmer Gripes with Agile!

#10 Agile is Snake Oil sold by Snake Oil Salespeople
Let the buyer beware! There is definitely an element of truth to this. “Agile” can mean almost anything and there are definitely folks out there taking advantage of the fact that many people are new to Agile. Not only are they just looking to make a buck, they often either have no idea what they are doing or have simply repackaged whatever they used to sell with the “Agile” label. The good news here is that these folks are pretty easy to uncover. Just keep in mind that there are Agile snakeoil salesfolks out there, do your normal due diligence when evaluating Agile advocates and you should do fine.

#9 Agile Has Too Much Jargon
Say, how many story points did we burnup during this sprint? It seems like our velocity went up after the product owner started using user stories in the backlog. What’s up with all of this jargon? Is it really necessary?

Rather than answering that question directly, let me ask you this: doesn’t every subject have its own subject matter jargon? Isn’t OO programming “jargon rich?” Isn’t .Net “jargon rich?” Heck, programming in general is “jargon rich.” I’ll bet you didn’t even bat an eye when I used the jargon terms “object-oriented” and “.Net” ! Oh wait, I didn’t even say “object-oriented,” I only said “OO.”

#8 Agile Means The End of Freedom
This is an interesting one because when Agile is implemented well, there is actually much more freedom for all involved. I think the message here is that management needs to keep in mind that programmers are understandably wary of new initiatives that smack of yet another way to impose control. When talking to programmers about Agile, be sure to emphasize that Agile will affect the whole organization, management included. After all, “waterfall” is primarily a management method and does not include specific technical practices.

It is true though that Agile means focusing on customer value and frowns on major re-architecting projects. Agile promotes incremental architecture where the architecture is built or improved over time and directly tied to implementing customer value.

#7 Management is Only Doing This Because They Think We’re Not Very Good
If you want Agile to succeed, you need to point out, and be sincere about it, that Agile will affect the whole organization, management included. Otherwise it can come across as an “us vs them” situation where management isn’t the problem, the programmers are the problem. And lets face it, waterfall is a management method, not a programming language.

See also: "Better, Faster, Stronger Programmers" .

#6 The Agile Consultant Will be A Pain in The Neck
From experience, programmers know that if you are going to introduce some new thing there’s probably going to be a know-it-all consultant coming in to point out all of their flaws and give generic advice without even getting to know what they do.

Let the buyer beware! See #10 - "Snake Oil"

#5 Gross! Pair Programming!
You probably don’t have to worry about this one because you probably won’t introduce it anyway. On the surface it seems like paying double. While there is definitely value in this, it is far enough off the beaten path of Agile that nobody reading this needs to worry about it.

#4 Stand Up Meetings == The Inquisition
Stand up meetings are not a question and answer period. . If somebody turns them into an inquest, you have a problem and it isn’t Agile. Agile can’t help you with this, but it can make it more obvious. If you are a manager and feel tempted to turn the stand-up meeting into an interrogation, you are going to end up with a room full of unhappy and uncooperative programmers. Time to re-read "The Unspoken Truth About Managing Geeks."

#3 Drive-by Agile Transition
Some managers (not you of course) read an article about Agile, declare “we’re going Agile” and then ask at all future meetings “how are we doing on this Agile thing” and then don’t react well when they see lots of blank stares.

If you do this, Agile will fail.

One other point on this, not only will an Agile Drive-by fail, your programmers will be pretty upset that they weren’t consulted in the decision. Going Agile is a big change and you are much more likely to succeed if you involve everybody in the decision.

#2 Management Isn’t Fully Committed to the Transition
Going Agile takes sustained commitment. Not only will you need to see the transition through the tough parts, you’ll also need to provide budgetary support. That is, you need to fully fund the transition.

Depending on the size of your organization, it is likely that you will need to hire at least one Agile consultant to see you through the transition. You probably don’t have a Continuous Integration system, enough virtual machines, enough real machines, an Agile Project Management system, etc, etc. Whereas before you probably only needed one of everything, for instance an Oracle test environment, you will now probably need at least double. That said, see #10 - "Snake Oil."

#1 This is Really Just Another Way to Demand More with Less
Especially now, people are pretty sensitive to the call to “do more with less.” Agile can be perceived as a way to do more with less. But of all of the benefits that Agile provides, “doing more with less” is not one of them.

So, what is your pet peeve with Agile? Let management know!

Better, Stronger, Faster Programmers - We Have The Technology

I recently got the following comment/question via reddit:

"Can we put this effort into making programmers better faster and stronger? Agile purveyors act as if they have an army of superhuman programmers already at their disposal and the only goal is utilizing them correctly."

Interesting observation. It raises all sorts of interesting questions, such as “what is better?” Really though, what is a “better programmer?” How do I know which programmers are better than others? How can I measure that? Is it the ones that produce the most lines of code? Is it the fewest defects? What if a programmer produces no defects, but very few lines of code and the code that they do produce isn’t used by anybody? How do I really know that it has few defects if it isn’t used by anybody? Perhaps the tests that would show that it is defective weren’t written because, again, nobody cares about that code?

programmer.upgrade(infinitely);
Let’s ignore the problem of comparing programmers and just assume that I have found a training regimen which creates better, faster, stronger programmers. If the programming effort of each individual was the only thing that determined the success of a project, we’d be all set, wouldn’t we? But most projects consist of at least two or more people interacting according to some sort of process. Many projects have nebulous goals and/or requirements. Quite a few projects get cancelled or shelved, either because they were clearly headed for failure or something more interesting came along. And let’s be honest about those so-called “shelved” projects. Do they ever come off the shelf? Don’t we just start over when we get back to that project?

All of this adds up to a system which has complex and unpredictable behavioral characteristics which are different than the sum of its parts. It isn’t enough to invest in better, faster, stronger programmers if the result never sees the light of day. Before investing a penny on any improvement project, wouldn’t it be better to have a way to very quickly find out what problems exist and very quickly see whether or not a solution to a problem actually makes a difference?

Let’s face it, traditional development methods aren't very good at providing visibility into the true state of the situation, nor at providing feedback on whether anybody is actually headed in the right direction. Agile is not a silver bullet and won't actually improve anything all by itself. Simply put, Agile is an improvement framework. It will expose quite painfully where the problems are, it is up to you to recognize them and solve them.

Programmer Planet
Any time I talk to folks about Agile, I point out that there is no magical place to get an army of superhuman programmers. According to IDC, there are ~12 million programmers out there. Those programmers include you, the folks you work with, and millions of other programmers just like the ones you’ve met.

Whoever you have now is pretty much the team that you are going to have to go forward with. Sure, you can probably switch who you have now with other folks, but that's no guarantee that you will get "better" programmers, only that you will get different ones. Better figure out how to make the most of the resources you have. And while I am a big fan of training and self-improvement at the individual level, why not leverage that investment by improving the framework within which those individuals work?

The Only Person Keeping You From Going Agile is You

If you are interested in going Agile, but you are surrounded by folks that are not, don’t despair. You can still experience significant benefit from Agile on your own. Also, after reading this post you won't be able to say that somebody is preventing you from going Agile. This post is primarily directed at developers, but other folks may benefit from a similar approach.

The basic idea is to adapt the various Agile practices for a team of one. Let’s start by taking a look at scrum master, user stories, one piece flow, product owner, backlog, daily standup, iteration reviews, retrospectives, unit tests, and refactoring.

Scrum Master of One
Since there is only one person in your team, you are automatically the Scrum Master. Congratulations, but try not to abuse your new found authority over your team. Remember, the Scrum Master is a facilitator, not a boss! The key thing that the Scrum Master does is to facilitate Agile. The Scrum Master understands how Agile works, makes sure that all of the meetings are happening, and also looks for and removes impediments.

If you are going to be an effective Scrum Master, you need to understand Agile development. If you can get your company to sponsor you to take a Certified Scrum Master course, that’s a great way to get started. If not, consider sponsoring yourself. It is expensive and will probably require you to expend two vacation days, but I assure you it will pay off in the end. Short of taking a CSM course, go and get a copy of “Agile Software Development with Scrum” by Ken Schwaber. You may also be interested in checking out my freely downloadable book “Do It Yourself Agile.”

Reverse Engineered User Stories
The next step is to start working with User Stories. A user story is a simple statement in plain English (or whatever language you are primarily using) that describes the work that needs to be done from the perspective of the end user. They are generally of the form “As a user, I want X” where the “X” describes a goal that can be achieved with the software to provide value to the user. For instance, “As a user I want to be able to purchase anything that I am looking at with a single click.” The user story doesn’t describe the nitty-gritty details.

I realize that since you are not the product manager (or business analyst), you don’t get to determine what you are working on or how it is described. The way that you can benefit from user stories is to reverse-engineer whatever description you have been given of the work you need to do into user stories. The benefit to you is that you will either have a better high-level view of what you are looking to accomplish, or you will immediately start to see holes in the description of the work you have been given. User stories give you a framework for thinking about your work.

Once you start using user stories, you will start having more questions for your manager or the product manager or business analyst. At first, folks may see your questions as annoying, but over time I believe they will see your questions as reflecting your interest in their success and the success of the company.

Divide and Conquer
Now that all of your work is described in terms of user stories, think about the size of those stories. Can any of the stories be further broken down? For instance, if you had a story that was “As a user I want a simple calculator function” you could break that down into “As a user I want to be able to add a number to an existing value,” “As a user I want to be able to subtract a number from an existing value,” etc. Breaking your work down into user stories has many benefits from an Agile perspective. We’ll look at just a few of them here.

One Thing at a Time
Most Agile teams use iterations for organizing their work. As a team of one, that probably doesn’t make much sense. Instead, as a team of one, try to do One Piece Flow. The idea of One Piece Flow is that you do all of the work required for a single user story before going on to the next one. Try to resist working on two stories at once.

Product Owner For a Day – Creating the Backlog
Ok, now you have a bunch of user stories that describe everything you’ve been assigned to do and they are as small as you can make them. It is time to put on your product owner hat. As product owner, you need to rank each of these stories relative to all the rest in terms of value. If you are not sure what the value of the stories is, ask around to see if you can gather enough information to make a reasonable guess. Perhaps you know one of the actual end users. Ask them what they think about the stories you have.

Over time, you’ll get better and better at ranking your user stories. No matter what you do, having some sort of ranking is much better than none. Rank all of your stories from most value to least value and start working on the top story. This ranking is called a backlog.

The benefit of having a backlog of small user stories and working on just one story at a time comes into play when somebody taps you on the shoulder and asks you to do something other than what you are currently doing. When you are interrupted, you now have some options. You can point to your backlog and say “where do you think this new request fits in this backlog? Is it really more important than this task that I am currently working on?” If you are lucky, the person interrupting you will decide that their request has a lower priority than whatever you are currently working on. At the very least, the backlog provides a framework for discussing the relative priority of the new request.

If the new request does seem to be higher priority than what you are working on, you have one more tactic you can try. Since your stories are small, you can say that you only have one task in progress and how much work remains on the current story and suggest that you finish what you are doing prior to starting the new task. Over time, this tactic should work more and more often as you get a reputation for delivering higher quality work and being able to quickly change gears. The backlog examination exercise should also slowly train other folks on the value of user stories and the backlog. Your goal should be to have folks give you new requests and for them to say “let’s see where this belongs in your backlog.”

Even if you have to stop whatever you are currently working on and start working on a new task, it is much easier to change course if you only ever have one story in progress at a time and it is small. Don’t forget to treat this new request the same as any other task. Create one or more user stories for the request and order them relative to each other at the top of the backlog.

Once you get the hang of maintaining a backlog, consider making your backlog visible to others. One way to do this would be to print out your backlog in a large font along with an indication of which stories you are currently working on and post it near you in a place that passersby can clearly see. You may even find that the number of interruptions goes down as people can see for themselves that you are clearly working on the most important thing and leave you alone to get it done.

Talking to Yourself
It may seem silly to do a “daily standup meeting” with yourself every morning. So don’t stand up. But do go through the exercise of going through the three questions:

• What have you accomplished since the last meeting?
• What are you working on next?
• What impediments do you have?

This will help to keep you focused on your current story. If you find that you just aren’t making the kind of progress you’d like to make, it is time to get some help! Let’s face it, if you go work on something else in the meantime, you’ll eventually have to come back to face whatever it is that is blocking you now. Yes, it is possible that you will solve the problem on your own, but why not at least find somebody to describe the problem to? Maybe the simple act of explaining the problem will provide a new insight. In any case, if you do end up working on something else, try to keep the total number of stories you have in progress to a minimum.

The Moment of Truth - Did it Work For You?
When you finish a story, now it is time to do an iteration review. Normally this would be done at the end of an iteration, but since you aren’t doing iterations, you might as well do the iteration review now. It is simple. Find somebody to demo the story to. The idea here is to have an audience and get feedback. If you feel like you aren’t ready to have somebody else see a demo of your work… then your story isn’t really done, is it? Keep working on it until you feel ready for an audience.

In Retrospect, That Was a Good Idea
Once you have finished your story and done the “iteration review,” it is time for the retrospective. Find a quiet place that you can spend 30 minutes or so to reflect back on how things went, from the time that you had the story ready to the time that you finished the iteration review. What went well? What didn’t go so well? What could you do better in the future? Write down your thoughts for future reference.

Unit Tests
If you aren’t already doing unit tests, now is the time to consider it. Unit tests are something that you can decide to do on your own for your own sake. At first it may seem that it is slowing you down, but consider all of the rework you have to do when QA finds problems with your work. Why not invest that effort into writing unit tests? If you are using Java, you can use JUnit. If you are using .Net, you can use NUnit.

Refactoring
Unit tests go hand in hand with refactoring. Refactoring is simply the practice of rewriting code to make it simpler to understand and change. The more unit tests you write, the more you will refactor the code and the easier it will be to understand the code and add new functionality.

Growing Your Team
See if you can get somebody in QA interested in what you are doing. You might look for their feedback on the ranking of your backlog or the quality of your unit tests. A QA person would also be a good person to invite to an iteration review. The more that QA sees the work you are doing as helping them, the more they will want to help make you successful. A close relationship with QA is an essential ingredient to Agile success.

As you practice these Agile techniques, I think you will find that you are producing higher quality results in less time as well as keeping focused on the work that the business cares the most about. You may also find that what you are doing gets noticed and appreciated. If you can get other folks to follow your example, you may soon find yourself part of a real Agile team. Good luck, and let me know how it works out for you!

If you enjoyed this post, you may also enjoy the freely downloadable book "Do it Yourself Agile."

Scrum and Kanban - Like Chocolate and Peanutbutter

This post is primarily geared toward folks who have been doing Scrum for at least 6-9 months and are curious about what options are out there for further improving their process. One reason for looking beyond Scrum may be that you are bumping your head on the iteration boundaries. A common place to look is Extreme Programming (XP). There are many great practices in XP that complement Scrum. Even if you personally don’t want to do pair programming, XP has much more to offer than just pair programming, take a look!

Whether you’ve looked at XP or not, another great place to take a look is Kanban. Kanban is a process into itself and can be adopted whether you are doing Scrum or not, but I will be looking at Kanban as a source of good ideas to apply to an existing Scrum process.

One of the reasons that I am assuming 6-9 months of Scrum as the starting point is that Kanban requires a certain level of experience with breaking work down into small user stories and doing One Piece Flow which I believe is easier to attain via Scrum than starting with Kanban from scratch.

One of the biggest and most noticeable differences between Scrum and Kanban is that Kanban doesn’t have iterations. At first this seems outlandish, but consider that most of the impact of having iterations is at the beginning and end of the iteration. In the midst of an iteration, the iteration boundaries are immaterial; everything you are doing is story-oriented. In that respect, Scrum and Kanban are the same.

Our Old Friend: The Decoupling Principle
Now imagine that you are in the midst of a Scrum iteration… and the end of the iteration never comes. Whenever the team finishes a story, they move on to the next story, just as in Scrum. The difference here is that the “todo hopper” is being constantly filled by the product owner.

One question at this point is “but what about the per-iteration activities such as retrospectives?” And that is one of the first things about Kanban that can be applied to Scrum, with no need to remove the iterations themselves. There is an underlying principle in Kanban which is a well known and widely used principle in programming: the decoupling principle. When doing Kanban, you still need to do the equivalent of planning, assignment, estimation, retrospectives, delivery, etc. In Kanban, all of these activities are decoupled from each other whereas in Scrum they are all coupled to the iteration boundary.

How can this be applied to Scrum? Consider retrospectives. If you are just starting with Scrum, you probably have an iteration length of 1 month (or four weeks). From that it follows that you will have a retrospective once per month. If you eventually end up with an iteration length of 1 week, then it follows that you will have a retrospective every week. But this actually seems like the wrong way to set the cadence of retrospectives. Wouldn’t it be better to have the cadence of retrospectives meet the need for them? If it eventually makes sense to do a retrospective every week, doesn’t it make sense to get the benefit of them on a weekly basis when you are just starting Scrum?

Go deeper: “Applying the Decoupling Principle to Scrum”

Do One Thing and Do it Well
Kanban has something called “work in progress limits.” Limiting work in progress is a concept that comes from Lean. The basic idea is that the less you have in progress the simpler everything becomes and the more likely you are to actually get work finished. If you believe in Scrum then by extension you pretty much also believe in work in progress limits. An iteration is in effect a work in progress limit. If you have an iteration length of two weeks and a velocity of 60 story points then you are saying that you will take on no more than 60 points of work per two week period. Kanban takes this further and says that in general the smaller you can make your work in progress limits, the better.

One of the most straightforward ways to create a WIP limit is to do a headcount of the # of developers on the team. If you have three developers, that gives a WIP limit of 3. That means that you never have more than 3 stories in progress at any given time. If you have been doing Scrum for a while, are good at breaking work down into small stories, and are doing One Piece Flow well, then what you are doing now is very similar to having WIP limits.

Take it to The Limit
Once you are comfortable with having a WIP limit, the next step is to remove iterations all together. Hopefully, this doesn’t seem quite so outlandish any more. After all, you are just exchanging one constraint for another. There is a hidden danger here though, which is one of the reasons that I strongly recommend building up the discipline of One Piece Flow first.

Let’s say you have an iteration length of two weeks. That also puts a two-week limit on your stories. In Kanban, if you only have a WIP limit on the number of stories you can take on, you could end up doing things like working on a task for months and months without finishing it. Luckily, there is a simple cure for this problem: impose either a maximum elapsed time rule, a per-story story point limit, or both.

One more thing to consider here is that if you had an iteration length of 4 weeks, there's a good chance that most of the stories were no more than a week from start to finish, and probably on the order of days. You may want to consider having a per-story elapsed time limit of a week, possibly 2 weeks to start.

Kanban in Action
Now that we’ve taken a look at the various aspects of Kanban, let’s see what this would look like in practice. First of all, we’ve started with Scrum so we still have a backlog, a Scrum Master, a Product Owner, user stories, and the like. As soon as a story is completed, the team takes another story from the “todo” list and starts working on it. That then triggers the product owner to add another story from the backlog to the todo list. Stories move from backlog, to todo, to coded, to tested, to done and then on to "shipped" on a regular basis. If you have been doing Scrum well, this is essentially what you have been doing anyway, but now you no longer have the awkward iteration boundaries. Instead, you have a continual flow of stories with no need to stop artificially.

Yes You Kanban
In summary, a good approach to take when adopting some or all of Kanban is:

1. Do Scrum for 6-9 months
2. Get good at breaking work into small user stories
3. Get good at doing One Piece Flow
4. Apply the decoupling principle to your Scrum implementation
5. Impose work in progress limits
6. Drop iterations and impose a story point limit

If you liked this post, you may also be interested in my new book “Do it Yourself Agile” which is freely downloadable.

Interesting links:
Limited Work in Progress Society (Kanban resources)
Kanban vs. Scrum (Friendly comparison)
One day in Kanban Land (Kanban Cartoon!)
Scrumban (Fully mixed!)

If you are in the process of moving to Kanban from Scrum or adopting some of Kanban in your Scrum implementation, let us know how it is going! What worked well for you and what are some of your lessons learned?

One Piece Flow -- Transitioning from Scrum to Kanban Part II

If you have been doing Scrum for a while, and doing it well, it is 99% certain that you are frequently practicing something called “one piece flow”. One piece flow is a concept from Lean and is the key to succeeding with Scrum. Doing it well also happens to be one of the requirements for successfully transitioning from Scrum to Kanban.

There are four main concepts in “one piece flow.” Each story is done as if it was the only thing in the release, each aspect of developing a user story happens in rapid succession, there is as much done in parallel as possible, and each team member focuses on a single user story at a time. The result of one piece flow is that the time between when the team first starts working on a user story and when they can ship it fully developed, tested, and documented is very short. The timeframe is generally on the order of a week at most and usually days.

In one piece flow, when development starts on a story, QA should be creating test cases for that story (story testing) at the same time. When development is done, QA should then automate the test cases and make sure they all pass. At the same time, the user documentation is written. One last step that must be completed prior to considering the story done is that all of the artifacts connected to that story such as source code changes, documentation updates, new test cases, etc must be integrated into the source code mainline as part of continuous integration.

In the following diagram, there are 20 stories done over the course of three iterations. There are three developers on the team and each takes on one story at a time. In the diagram, the whole team is doing one piece flow well and you can see how the specifying (S), designing (D), coding (C), writing of tests (W), integrating (I), and testing (T) is evenly spread out but also clumped into stories.

So, from the perspective of a story, it is started, and completely finished in a short period of time. QA should not be waiting until all stories are done. Conversely, developers should not have lots of stories in progress that all finish together which keeps QA from getting involved.

You will know that you are succeeding at one piece flow when stories are being started and then are completely ready to go on an individual basis within days with all documentation written and all unit and story tests written, automated, and passing.

Once every QA person is actively engaged in a story that is part of an iteration, you are “in the flow.” Everybody is now flowing smoothly from story to story. Unfortunately, Scrum has a bad habit of disrupting one piece flow on a regular basis. As soon as one of the developers finishes their work for the last story assigned to them for the iteration you are no longer “in the flow.”

For more about how Scrum disrupts one piece flow, read about Bumping Into Scrum’s Iteration Boundaries.

Next: Scrum and Kanban -- Like Chocolate and Peanut Butter

Transitioning From Scrum to Kanban, Part 1 of 3

Bumping Into Scrum’s Boundaries
Scrum is the most popular Agile process in use today. But if you have been doing Scrum for a while, you know (or will soon experience) that it has some structural problems. Many of Scrum’s activities are tightly coupled to the cadence of the iterations but have a natural tendency towards their own cadences. In order for a story to be done-done-done, all of the work to make sure that it is done-done-done must be done within the iteration. A prerequisite of doing that work is that the code must be complete. That causes problems.

At the beginning of an iteration, there are no stories for testers to be testing until the developers get the first stories finished. At the end of an iteration, while the story validation work is being done on the last stories, the developers (and some of the testers) have nothing to do. A common response to this is that “there is always something to do.” Obviously there is always something to do, but that’s not the point. A key point of Agile is to focus on the highest value work. By definition, the highest value work is whatever is next in the backlog and not anything else. So, if you are doing something other than work required to move that iteration’s stories from todo to done you are working on the wrong things. One approach is to have the developers start on the next items in the backlog, but now you are moving outside the framework of Scrum.

Another problem is that the last stories rarely all transition neatly to done right at the end of the iteration. You may have some stories in progress at the end of the iteration which were hoped to be finished by the end of the iteration. For these folks, there is pressure to fit the work into the time remaining. Other folks may finish up early and not have any stories left to work on. While it is true that the team should be working and thinking as a team and helping each other out, it isn’t always possible to have two people work on the same story.

Just What the Doctor Ordered
There’s a commonly prescribed remedy for all of this: change the iteration length. Sometimes the recommendation is to make it longer, sometimes the recommendation is to make it shorter. In my experience, neither of these recommendations works very well. The reason is that the size of the stories that you work on tends to vary over time. The smaller, the better of course, but some stories are just naturally larger than others. No matter what you do with the iteration length, all you are really doing is changing the size of the problem, you aren’t really removing it.

The Source of the Friction
There are two simple reasons for this friction. It isn’t any easier to predict the future using Scrum. Software development is a creative endeavor. It is unpredictable. There is no way to get stories to all end smoothly at the end of the iteration, regardless of the iteration length or how often it is varied. The structure of the work itself just makes it worse.

Within each story there are two kinds of work: work that is mostly done during the first half of the story and work that is mostly done at the end of the story. The end work depends on the beginning work. As a result, it is impossible to “get the air out of the pipes.” It will always be there. This is illustrated in the following diagram.


The diagram shows two iterations of work split into three "swim lanes." I've artificially put one developer and one QA person into each swim lane to simplify the explanation. The numbers in each swim lane correspond to individual stories, the green areas represent developers and the yellow areas represent testers. The structure of the work means that there will always be areas that are either empty or filled with “other work.” You may have heard that it is good to have slack time, and I agree. However, I believe it is better for slack time to be under the control of the people in the system, not as the unpredictable result of a side-effect inherent in the process.

Long Live Scrum!
So where am I headed with this? Am I anti-Scrum? Am I about to describe a way to fix these problems with Scrum? Neither. I don’t believe these problems can be solved within Scrum. I do think that Scrum can be improved, see “Applying the Decoupling Principle to Scrum,” but I see the real solution as transitioning to Kanban. However, I also believe that it is better to do Scrum for a year or two first in order to build up the discipline of doing One Piece Flow that is necessary for succeeding with Kanban.

Next: One Piece Flow -- Transitioning from Scrum to Kanban Part II

Agile, Pizza, and Beer! Boston, Sept 24th, 5-7pm

If you are curious about Agile, you may enjoy my presentation "Is Agile Any Better?"

Download "Do It Yourself Agile" For Free!

In my copious spare time I've been working on a book about Agile development. I had originally intended to publish it as an offline paper book, but over time I realized that I was generating a bunch of content that was just getting dusty on the shelf and wasn't providing any value or getting the benefit of rapid feedback. So now I've switched gears and am moving to publishing entirely on the web.

Translating a book to the web takes time, so I’ve also decided to make the book available via pdf in the meantime. As the material has made its way to the web, it has often been rewritten. Some of those changes have made it into the book, and some haven’t.

While I believe that an Agile coach, whether recruited internally or externally, is still the best and fastest path to Agile success, not everyone can do that. "Do it Yourself Agile" is intended to be a resource you can lean on as you transition to Agile on your own.

In any case, let me know what you think. I look forward producing new versions incorporating your feedback.

[Sep 29, '09: Second version now available, link is the same.]
[Sep 14, '09: Based on the feedback so far I will be adding recent blog entries and putting out the first update to the book in a couple of weeks.]

"Do it Yourself Agile" (pdf)

For $65/pp, Send Your Whole Team to Agile Training in Boston Oct 5th

This is truly a killer deal. Alan Shalloway will be in town on October 5th doing a special 1-day Agile event for the Agile Bazaar titled "Team Technical Agility". At $65pp including lunch, I have a hunch this will fill up fast. Why not sign up your whole team? Even if you aren't in the Boston area, the airfare and hotel will probably still add up to less than you would pay for any other 1 day Agile course.

Applying the Decoupling Principle to Scrum

While working on a post about Kanban, I realized that there is a general software engineering principle at work in Kanban that can be applied to Scrum without needing to mention Kanban at all. The principle is: decoupling. That is, separating two or more things which are currently coupled together but don't need to be.

In Scrum, there are many activities which are often tightly coupled to the iteration cadence: iteration planning, having a shippable increment of work, the size of the largest story, iteration reviews, retrospectives, and releases. If you have been practicing Scrum for a while, it is likely that you have already started to decouple some activities from the iterations.

Decoupling Iteration Meetings and Story Point Estimation

Many Scrum teams have a weekly meeting to estimate story points for stories that have made it near the top of the backlog and don't yet have story point estimates. This decouples story point estimation from both the iterations and the iteration planning meeting itself. How far down into the backlog you go depends on what goal you are trying to achieve. If you just want to simplify iteration planning, you only need enough stories to cover one iteration worth of stories and perhaps a bit more just in case some stories aren't chosen for that iteration.

Decoupling Retrospectives

Let's see how decoupling might be applied to more practices. When just starting with Agile, it is much harder to use 1 week iterations than 4 week iterations. If you are doing 4 week iterations, you would naturally do a retrospective once every four weeks. If you move to 2 week iterations, you would naturally do a retrospective every two weeks.

But why not do retrospectives every week or at least every two weeks, regardless of whether you are doing 4 week iterations or two week iterations? Shouldn't the cadence of retrospectives match the cadence of their usefulness? In my experience, there is always something worth talking about every week of any project. There is always something to reflect on and improve. In any case, instead of just scheduling a retrospective at the same cadence as your iterations, schedule the retrospectives at a cadence that makes sense for your needs.

Decoupling Iteration Reviews

Another practice to consider decoupling from your iteration cadence is iteration reviews. Let's say you have 1 week iteration reviews, but it is logistically difficult to schedule iteration reviews involving multiple customers more often than once per month. Perhaps you normally have both internal and external stakeholders at the reviews. One solution would be to continue to have weekly reviews with just the internal stakeholders that are interested in being there every week, and have monthly reviews for external stakeholders. That may actually require a bit more work on your part, but if the value is there, why not consider it?

Simplification and Delegation

Another benefit of decoupling is that by breaking things apart it makes it simpler to tackle or delegate process improvement efforts because you can concentrate on smaller parts which are running at their own cadence. In the example of the iteration reviews, by separating out the external stakeholder meeting, you can now focus on ways to simplify or delegate that part of the process. Perhaps the only problem is getting a web meeting solution in place. Now you can solve that problem without impacting the internal stakeholders.

Consider how the decoupling principle will work for you. I would be interested to hear what you've already decoupled and your thoughts on applying the decoupling principle to Scrum. If you want to read about how to take decoupling to the next level, click on the next post below.

Next: Transitioning from Scrum to Kanban

Back to Work After Agile 2009

It is my first day at work after Agile 2009, and I must say I am both exhausted and re-invigorated at the same time. For me, one of the most exciting parts of Agile 2009 was that there were so many people buzzing about Kanban, a concept that I believe should be getting much more exposure and use than it currently is. By the way, if you are a fan of Kanban, check out the "Limited Work in Progress Society" web site to find all sorts of Kanban resources. Another good resource on Kanban is Henrik Kniberg's comparison of "Kanban vs Scrum". I'll write more on my thoughts about Kanban soon.

Software Development Bloggers Unite!

Do you love reading blog entries about software development? Social news sites are a great way to find posts that you are interested in. The more people know about these sites, the more people will use them and submit great posts to them. Please help to spread the word about these sites. If you know of some that aren't on the following list, let us know!!

Here are the ones I regularly visit:

Agile sub-reddit

Software development sub-reddit

Joel on Software sub-reddit (not just Joel)

Dzone Agile tag

Dzone methodology tag

What else is good??

Why Bother Going to Agile 2009 or Agile 2010??

Agile 2009 was a blast again this year. I learned a lot and met lots of incredibly interesting people. While talking to a friend about MVP for GUIs I realized that somebody that had just presented on the topic was sitting at a nearby table and we asked him about it and got a great answer.

I believe there are lots of folks out there that would get a lot out of some of the conferences that are out there, like Agile 2009, if only they went once and experienced it. I've decided to dedicate some portion of my time to getting the word out on that. As a first step I created an impromptu video which includes many great answers to the question "why go to a conference vs just surfing the web and reading books?" Next time I'll be better prepared and do an even better version.

The video was taken while the conference was in full swing, but it was the last day and some folks were tired of going to sessions and were just hanging out in the open area. Esther Derby was doing an "Open Jam" session, Alistair Cockburn was just hangin' out, Dave Anderson was having a spirited discussion on something interesting (I know not what), and others were networking, just catching up on e-mail or waiting for the next session that they wanted to attend.

I'll post more impressions of the conference soon.

Continuous Integration Sessions at Agile 2009

If you are interested in Continuous Integration, and you are going to Agile 2009, please consider attending one of my CI sessions:

Removing Integration Delays with Collocated Whole Teams and Multi-stage CI
As individuals we work in transient isolation to reduce the impact of work in progress on each other. Organizations isolate WIP by using only official versions of 3pty sources and by producing official releases for customers.

Multi-stage continuous integration (MSCI) scales CI to large distributed environments by isolating work in progress at the team level. Changes move from individual to team to mainline as fast as CI allows, but stop on failure.

MSCI is particularly important in a distributed environment where fixes to problems exposed by CI can be delayed by a full day. [Full Details]

Agile Source Code Management using Stories, Agile Workflow, and CI
You create iterations from a backlog of user stories managed via a taskboard with a simple “workflow” from “todo” to “done.” You use Continuous Integration. But in your source control system you’ve just got files and branches. You could create a branch for every story, but that’s a lot of branches to manage! How can you ask the source control system which versions/files correspond to the stories that are done in order to build the “done” version and do exploratory testing? This session will show how to manage changes using stories and how to use branches to represent your workflow. [Full Details]

Guilty Confessions of a Software Developer

I tried to look like I was still mulling over the question when Mr. J continued his interrogation, "The description of this feature is incomplete. What should happen when the -t option is combined with -V and there are no other options given?" I was sweating and nervous, my chest felt tight and I knew I was right on the edge of panic.

I couldn't remember the answer to Mr J's question. I'm not sure I ever knew it and the customer that had asked for this new feature had gone out of business 2 months after we implemented it, which was 3 months ago. I was helpless and trapped. The harsh light and the look in his eyes wasn't helping. He knew that I didn't know. I wouldn't be able to hold out for much longer, it felt like the room was starting to spin.

Finally I couldn't take it any more. I cracked under the strain and confessed what he already suspected. "I don't know. I wrote that code so long ago, I just can't remember. I'm sorry." Mr. J shook his head and walked out of my office saying, "I'll figure it out."

Ok, so this is a bit of a dramatization loosely based on a real event, but being in the situation where you are asking about or needing information about something that happened months ago in the development process is fairly typical in a traditional development project.

Perhaps you are the developer in this situation or the person trying to write the tests for the functionality. Or perhaps you were trying to write the documentation, do the demo for the user, market it, sell it, or... use it? As an industry, we've become numb to this problem. Sure, we blame specific situations or individuals and we often try to take corrective action... but it seems to keep happening again and again.

As an industry, we try to write better requirements, but it doesn't seem to help. The tough questions keep coming. We try to take a bit more time on the design, but things just get worse. The questions get more complicated. We try to do more testing and hire "better" people, but we keep running into situations where we just don't have the answer, there's no supporting documentation, and we promise we'll try harder next time.

Why does this keep happening? What can be done to prevent it? It turns out that there is a simple explanation as well as a practical solution.

Next: Traditional Development == The Game of Telephone?

Leveraging Human Strengths in the Development Process

In the previous post we saw two ways that traditional development is like the game of telephone. First, communication suffers from translation errors at each stage of the development cycle. Second, the frequent attempts to detect, prevent, and recover from translation errors require people to remember details about work they did weeks or months in the past. This is unfortunate because people are much better at remembering something they did within the past couple of days than they are at remembering something they did weeks or months ago.

One of the advantages of Agile development is that it leverages human strengths and minimizes the need for humans to do things that humans do poorly. In the case of communicating important concepts throughout the development cycle, Agile relies on four tools: user stories, one piece flow, conversations, and reduced documentation.

A user story is a simple statement in plain English of a value proposition that the end user of the software is interested in. For instance, "As a user, I want to purchase an item I am looking at with a single click." The main value of user stories for communication is that they are used by everybody involved in the development of that story, from user to tester. While the story may be translated into a design document or test cases, the original intent of the user is always available as a double-check via the user stories.

An important part of increasing communication effectiveness is a concept called "one piece flow" which comes from Lean. The idea of one piece flow is that each aspect of developing a user story happens in rapid succession, and that each team member focuses on a single user story at a time. The result of one piece flow is that the time between when the team first commits to doing a user story and they can ship it fully developed, tested, and documented is very short. The timeframe is generally on the order of a week at most and usually days.

Because user stories go from start to finish in such a short period of time, the bulk of the communication that occurs on behalf of a user story is done via conversations. Of course, Agile teams do document their work, but it is for the purpose of creating an audit trail, not for the purpose of short-term communication between team members. Conversations are a much better way to communicate technical concepts than documentation.

Conversations between Agile teams and their customers

From the perspective of a customer, the time between when they provide detailed information about a feature and when they can see a demo of the result is often as short as a month. Also, the scope of functionality is going to be much less. The reduction in timeframe and scope means that it is much more likely that a customer will remember exactly why they wanted something and that they will be much more able to provide useful feedback on the result.

That doesn't mean that all customers have to be involved every month, only that there is an opportunity to go full circle with customers over a much shorter period of time than with traditional development. Consequently, miscommunication can be caught earlier and corrected faster.

Conversations in an Agile team

From the perspective of individual team members, interaction is now focused on a specific user story at any given time and the timeframe of its development is on the order of days rather than months. Work can be initiated, rapidly completed, and then confidently considered done instead of having an ever growing list of work in progress. With Agile development, the amount of work in progress is very small and is constantly changing. There is much less of a need to rely on piles of documentation or on the long term memory of oneself or others.

Are you an Agile Do It Yourselfer? Check out the web-only book

"Do It Yourself Agile" .