Bumping Into Scrum’s Boundaries
Scrum is the most popular Agile process in use today. But if you have been doing Scrum for a while, you know (or will soon experience) that it has some structural problems. Many of Scrum’s activities are tightly coupled to the cadence of the iterations but have a natural tendency towards their own cadences. In order for a story to be done-done-done, all of the work to make sure that it is done-done-done must be done within the iteration. A prerequisite of doing that work is that the code must be complete. That causes problems.
At the beginning of an iteration, there are no stories for testers to be testing until the developers get the first stories finished. At the end of an iteration, while the story validation work is being done on the last stories, the developers (and some of the testers) have nothing to do. A common response to this is that “there is always something to do.” Obviously there is always something to do, but that’s not the point. A key point of Agile is to focus on the highest value work. By definition, the highest value work is whatever is next in the backlog and not anything else. So, if you are doing something other than work required to move that iteration’s stories from todo to done you are working on the wrong things. One approach is to have the developers start on the next items in the backlog, but now you are moving outside the framework of Scrum.
Another problem is that the last stories rarely all transition neatly to done right at the end of the iteration. You may have some stories in progress at the end of the iteration which were hoped to be finished by the end of the iteration. For these folks, there is pressure to fit the work into the time remaining. Other folks may finish up early and not have any stories left to work on. While it is true that the team should be working and thinking as a team and helping each other out, it isn’t always possible to have two people work on the same story.
Just What the Doctor Ordered
There’s a commonly prescribed remedy for all of this: change the iteration length. Sometimes the recommendation is to make it longer, sometimes the recommendation is to make it shorter. In my experience, neither of these recommendations works very well. The reason is that the size of the stories that you work on tends to vary over time. The smaller, the better of course, but some stories are just naturally larger than others. No matter what you do with the iteration length, all you are really doing is changing the size of the problem, you aren’t really removing it.
The Source of the Friction
There are two simple reasons for this friction. It isn’t any easier to predict the future using Scrum. Software development is a creative endeavor. It is unpredictable. There is no way to get stories to all end smoothly at the end of the iteration, regardless of the iteration length or how often it is varied. The structure of the work itself just makes it worse.
Within each story there are two kinds of work: work that is mostly done during the first half of the story and work that is mostly done at the end of the story. The end work depends on the beginning work. As a result, it is impossible to “get the air out of the pipes.” It will always be there. This is illustrated in the following diagram.
The diagram shows two iterations of work split into three "swim lanes." I've artificially put one developer and one QA person into each swim lane to simplify the explanation. The numbers in each swim lane correspond to individual stories, the green areas represent developers and the yellow areas represent testers. The structure of the work means that there will always be areas that are either empty or filled with “other work.” You may have heard that it is good to have slack time, and I agree. However, I believe it is better for slack time to be under the control of the people in the system, not as the unpredictable result of a side-effect inherent in the process.
Long Live Scrum!
So where am I headed with this? Am I anti-Scrum? Am I about to describe a way to fix these problems with Scrum? Neither. I don’t believe these problems can be solved within Scrum. I do think that Scrum can be improved, see “Applying the Decoupling Principle to Scrum,” but I see the real solution as transitioning to Kanban. However, I also believe that it is better to do Scrum for a year or two first in order to build up the discipline of doing One Piece Flow that is necessary for succeeding with Kanban.
Next: One Piece Flow -- Transitioning from Scrum to Kanban Part II