Scrum is the most popular Agile process in use today. But if you have been doing Scrum for a while, you know (or will soon experience) that it has some structural problems. Many of Scrum’s activities are tightly coupled to the cadence of the iterations but have a natural tendency towards their own cadences. In order for a story to be done-done-done, all of the work to make sure that it is done-done-done must be done within the iteration. A prerequisite of doing that work is that the code must be complete. That causes problems.
At the beginning of an iteration, there are no stories for testers to be testing until the developers get the first stories finished. At the end of an iteration, while the story validation work is being done on the last stories, the developers (and some of the testers) have nothing to do. A common response to this is that “there is always something to do.” Obviously there is always something to do, but that’s not the point. A key point of Agile is to focus on the highest value work. By definition, the highest value work is whatever is next in the backlog and not anything else. So, if you are doing something other than work required to move that iteration’s stories from todo to done you are working on the wrong things. One approach is to have the developers start on the next items in the backlog, but now you are moving outside the framework of Scrum.
Another problem is that the last stories rarely all transition neatly to done right at the end of the iteration. You may have some stories in progress at the end of the iteration which were hoped to be finished by the end of the iteration. For these folks, there is pressure to fit the work into the time remaining. Other folks may finish up early and not have any stories left to work on. While it is true that the team should be working and thinking as a team and helping each other out, it isn’t always possible to have two people work on the same story.
Just What the Doctor Ordered
There’s a commonly prescribed remedy for all of this: change the iteration length. Sometimes the recommendation is to make it longer, sometimes the recommendation is to make it shorter. In my experience, neither of these recommendations works very well. The reason is that the size of the stories that you work on tends to vary over time. The smaller, the better of course, but some stories are just naturally larger than others. No matter what you do with the iteration length, all you are really doing is changing the size of the problem, you aren’t really removing it.
The Source of the Friction
There are two simple reasons for this friction. It isn’t any easier to predict the future using Scrum. Software development is a creative endeavor. It is unpredictable. There is no way to get stories to all end smoothly at the end of the iteration, regardless of the iteration length or how often it is varied. The structure of the work itself just makes it worse.
Within each story there are two kinds of work: work that is mostly done during the first half of the story and work that is mostly done at the end of the story. The end work depends on the beginning work. As a result, it is impossible to “get the air out of the pipes.” It will always be there. This is illustrated in the following diagram.
The diagram shows two iterations of work split into three "swim lanes." I've artificially put one developer and one QA person into each swim lane to simplify the explanation. The numbers in each swim lane correspond to individual stories, the green areas represent developers and the yellow areas represent testers. The structure of the work means that there will always be areas that are either empty or filled with “other work.” You may have heard that it is good to have slack time, and I agree. However, I believe it is better for slack time to be under the control of the people in the system, not as the unpredictable result of a side-effect inherent in the process.
Long Live Scrum!
So where am I headed with this? Am I anti-Scrum? Am I about to describe a way to fix these problems with Scrum? Neither. I don’t believe these problems can be solved within Scrum. I do think that Scrum can be improved, see “Applying the Decoupling Principle to Scrum,” but I see the real solution as transitioning to Kanban. However, I also believe that it is better to do Scrum for a year or two first in order to build up the discipline of doing One Piece Flow that is necessary for succeeding with Kanban.
Next: One Piece Flow -- Transitioning from Scrum to Kanban Part II
11 comments:
I was hoping your post could give me good reasons to move to Kanban, but I stopped reading after:
"At the beginning of an iteration, there are no stories for testers to be testing until the developers get the first stories finished."
Does it mean you have separate people for testing and developing? If so then you are defeating one of the requirements of Scrum: everybody does everything so that nobody waits on anyone. So I don't see where it's a good reason to go to another methodology.
Hi Sarbogast,
I am not a big fan of being doctrinarian, but I am going to make an exception here. On page 37, Section 3.3.3 of Ken Schwaber's "Agile Software Development with Scrum" it says:
"Teams are cross-functional. A Scrum Team should include people with all of the skills necessary to meet the sprint goal." It goes on to talk about how silos are bad, that is development separate from QA, etc. I think you have taken the idea of a cross-functional team to the extreme with cross-functional people. Scrum does not require everybody to do everything. Look again.
If you happen to work with teams of cross-functional people, that's great and I'm happy for you and the part you quote does not apply to you. You can just ignore it.
It's funny because I learned about Scrum with the very book you're quoting, and I had completely misunderstood this part. I interpreted it with what seemed obvious to me at the time: if we want to remove all impediments and empower people to the point where they will never say again things like "I have nothing to do, I'm waiting for X to do his job", it was mandatory to have only highly versatile people on the team. And it sure was hard to get to that point sometimes, especially in old-school organizations, but I figured it was the price to pay for all the marvels of agility. And it always struck me that most teams claiming that Scrum didn't work were ones where they had different people for architecture, development, testing and so on. But how can you stick to your sprint estimations when there are so many interdependencies between people? What's the point of daily stand-ups if testers hearing about what developers do can't develop when developers are sick?
Once again, I don't question the interest of Kanban, but I have doubts about the first reason you give for moving to it. Maybe I am to extremist about how I interpret Scrum. I don't know.
Hi Sarbogast,
Thanks for your follow-up comment. I do think we are more-or-less on the same page. IMHO, the other important part of "cross-functional team" is the word "team." Successful teams don't make excuses, they work together as a team and get it done.
A better (but more difficult to write) version of what I said might be "at the beginning when there is more development work than testing, the folks who excel at testing may find that they are not as highly leveraged as they will be when there is more testing work to do..."
I invite you to read the whole post (and the related posts including the decoupling principle) and provide further feedback. I am very open to feedback and often rewrite posts based on feedback and/or create new posts based on lively discussions such as this. And of course, this will all be iterated over again when it is incorporated into the book version.
All of the recent posts I've been doing are leading up to one big post (or maybe 2-3 small ones) specifically about Kanban. Feedback like yours and Tobias' is very helpful in making it the best it can be.
Cheers!
Damon
Hi Damon
Nicely explained. I have been thinking about the same thing myself, and Kanban looks very promising, especially since it has such a body of knowledge and experience behind it in Lean and TOC.
There are a couple of advantages of the time-boxed sprint. These include:
* regular achievable goals, and a sense of closure when they are met (sprint vs. marathon)
* a regular opportunity for team members to mentally clock out and refresh.
I intentionally set our 2-week sprints to start on a Monday and finish on a Friday, so that every two weeks, team members get a weekend to play with their kids without knowing the specifics of the following week's work.
Alistair Cockburn suggested "golden tickets" to break the continual flow of work in his presentation: I Come to Bury Agile, Not to Praise It
As I understand it, it's basically a Kanban card that says 'take a certain period of time out'. It's a great idea, but has two potential downsides:
* less understanding business owners might not be so generous or respecting of the system - even if the value of it is understood, I can imagine the temptation to skimp on golden tickets would be strong
* more importantly, the team still knows full well what's coming up, and even if they can clock out consciously, I'll bet the subconscious is still working on the next work items.
So I'm still wondering if there's a way to get this mental break in Kanban, without compromising flow or a highly collaborative team environment.
Russell
Hi Russell,
Sounds like you've got a pretty smooth setup there!
I don't think I did a very good job of explaining this in my post, but I'm not actually advocating pure Kanban. I don't have anything against pure Kanban, but I prefer starting with Scrum as the basis and then modifying it to take into account some of the useful parts of Kanban as-you-go.
The reason I mention this is that if you like the two week timebox soley for the purpose that you state, you could leave that part in. That is, impose a constraint that you must have no work in progress at the end of every two week cycle. If it is two days before the end of the cycle and you are about to put something in the todo hopper that would take 3 days to complete, that would violate the constraint.
I won't tell anybody that you're not doing pure Scrum or pure Kanban if you don't. ;-)
Hi Damon,
Actually lots of food for thought there - you've identified many of the problems I've seen in running with smaller teams. The swimlanes was a bit of a light-bulb moment for me (mostly because it is a metaphor we work with a lot). I'll catch up with the rest of the posts and hope you keep blogging and sharing your thoughts.
Testers typically keep themselves busy writing test cases while the developers are busy with coding during the first few days in the Sprint. Towards the end of the Sprint, the developers who finish their own tasks early help the other team members with coding, code review, unit testing, debugging, test automation, etc.
Also, where is the teamwork if every developer works on one story at a time? In my team, the bigger stories are split among multiple developers working in close tandem.
I don't think we should worry to much about idle time of individual team members within a Sprint! We should trust the power of self adjusting team to optimize utilization of individual team members during the Sprint. If the team members realize they have undercommited during a Sprint, they will usually commit to more work in the next Sprint to increase their velocity. It takes a few iterations for every team to find the perfect balance. Most imperfections are generally sorted out during the retrospectives. I would trust the wisdom of the team to sort these things out.
Saurabh,
I understand what you are saying, and that does work pretty well. However, the iteration boundaries still end up being an artificial boundary imposed on the team. You end up with random gaps which one tries to fill with something of value. But, by definition it is not the highest value. The highest value is whatever is next in the backlog. If that's not the case, then how can one say the backlog is well ordered?
What you say about splitting stories and having folks work together is great. In my post I was only laying out a sketch of the pattern. I certainly am in support of what you are saying about teamwork.
As you say, that works well for larger stories. But if they can be split that way, then perhaps they really are separable stories? In any case, anyone reading my post should consider how it may or may not apply to their circumstances. While I definitely believe that the patterns in Kanban have general applicability, they are just patterns and each team is unique.
In my experience, the flow of Kanban is actually more natural than the regular iterations of Scrum. I believe that everything you are saying is completely compatible with and will be accentuated by one piece flow.
You elaborated it very well, Mr. Poole. I'm glad that you came up with such an observation as this one. Scrum and Kanban for me are great; friends who have worked with it told me how their work routines have changed because of these.
just linked this article on my facebook account. it’s a very interesting article for all.
Scrum Courses- cPrime
Post a Comment