Tuesday, September 15, 2009

Scrum and Kanban - Like Chocolate and Peanutbutter

This post is primarily geared toward folks who have been doing Scrum for at least 6-9 months and are curious about what options are out there for further improving their process. One reason for looking beyond Scrum may be that you are bumping your head on the iteration boundaries. A common place to look is Extreme Programming (XP). There are many great practices in XP that complement Scrum. Even if you personally don’t want to do pair programming, XP has much more to offer than just pair programming, take a look!

Whether you’ve looked at XP or not, another great place to take a look is Kanban. Kanban is a process into itself and can be adopted whether you are doing Scrum or not, but I will be looking at Kanban as a source of good ideas to apply to an existing Scrum process.

One of the reasons that I am assuming 6-9 months of Scrum as the starting point is that Kanban requires a certain level of experience with breaking work down into small user stories and doing One Piece Flow which I believe is easier to attain via Scrum than starting with Kanban from scratch.

One of the biggest and most noticeable differences between Scrum and Kanban is that Kanban doesn’t have iterations. At first this seems outlandish, but consider that most of the impact of having iterations is at the beginning and end of the iteration. In the midst of an iteration, the iteration boundaries are immaterial; everything you are doing is story-oriented. In that respect, Scrum and Kanban are the same.

Our Old Friend: The Decoupling Principle
Now imagine that you are in the midst of a Scrum iteration… and the end of the iteration never comes. Whenever the team finishes a story, they move on to the next story, just as in Scrum. The difference here is that the “todo hopper” is being constantly filled by the product owner.

One question at this point is “but what about the per-iteration activities such as retrospectives?” And that is one of the first things about Kanban that can be applied to Scrum, with no need to remove the iterations themselves. There is an underlying principle in Kanban which is a well known and widely used principle in programming: the decoupling principle. When doing Kanban, you still need to do the equivalent of planning, assignment, estimation, retrospectives, delivery, etc. In Kanban, all of these activities are decoupled from each other whereas in Scrum they are all coupled to the iteration boundary.

How can this be applied to Scrum? Consider retrospectives. If you are just starting with Scrum, you probably have an iteration length of 1 month (or four weeks). From that it follows that you will have a retrospective once per month. If you eventually end up with an iteration length of 1 week, then it follows that you will have a retrospective every week. But this actually seems like the wrong way to set the cadence of retrospectives. Wouldn’t it be better to have the cadence of retrospectives meet the need for them? If it eventually makes sense to do a retrospective every week, doesn’t it make sense to get the benefit of them on a weekly basis when you are just starting Scrum?

Go deeper: Applying the Decoupling Principle to Scrum

Do One Thing and Do it Well
Kanban has something called “work in progress limits.” Limiting work in progress is a concept that comes from Lean. The basic idea is that the less you have in progress the simpler everything becomes and the more likely you are to actually get work finished. If you believe in Scrum then by extension you pretty much also believe in work in progress limits. An iteration is in effect a work in progress limit. If you have an iteration length of two weeks and a velocity of 60 story points then you are saying that you will take on no more than 60 points of work per two week period. Kanban takes this further and says that in general the smaller you can make your work in progress limits, the better.

One of the most straightforward ways to create a WIP limit is to do a headcount of the # of developers on the team. If you have three developers, that gives a WIP limit of 3. That means that you never have more than 3 stories in progress at any given time. If you have been doing Scrum for a while, are good at breaking work down into small stories, and are doing One Piece Flow well, then what you are doing now is very similar to having WIP limits.

Take it to The Limit
Once you are comfortable with having a WIP limit, the next step is to remove iterations all together. Hopefully, this doesn’t seem quite so outlandish any more. After all, you are just exchanging one constraint for another. There is a hidden danger here though, which is one of the reasons that I strongly recommend building up the discipline of One Piece Flow first.

Let’s say you have an iteration length of two weeks. That also puts a two-week limit on your stories. In Kanban, if you only have a WIP limit on the number of stories you can take on, you could end up doing things like working on a task for months and months without finishing it. Luckily, there is a simple cure for this problem: impose either a maximum elapsed time rule, a per-story story point limit, or both.

One more thing to consider here is that if you had an iteration length of 4 weeks, there's a good chance that most of the stories were no more than a week from start to finish, and probably on the order of days. You may want to consider having a per-story elapsed time limit of a week, possibly 2 weeks to start.

Kanban in Action
Now that we’ve taken a look at the various aspects of Kanban, let’s see what this would look like in practice. First of all, we’ve started with Scrum so we still have a backlog, a Scrum Master, a Product Owner, user stories, and the like. As soon as a story is completed, the team takes another story from the “todo” list and starts working on it. That then triggers the product owner to add another story from the backlog to the todo list. Stories move from backlog, to todo, to coded, to tested, to done and then on to "shipped" on a regular basis. If you have been doing Scrum well, this is essentially what you have been doing anyway, but now you no longer have the awkward iteration boundaries. Instead, you have a continual flow of stories with no need to stop artificially.

Yes You Kanban
In summary, a good approach to take when adopting some or all of Kanban is:
  1. Do Scrum for 6-9 months
  2. Get good at breaking work into small user stories
  3. Get good at doing One Piece Flow
  4. Apply the decoupling principle to your Scrum implementation
  5. Impose work in progress limits
  6. Drop iterations and impose a story point limit
If you liked this post, you may also be interested in my new book “Do it Yourself Agile” which is freely downloadable.

Interesting links:
Limited Work in Progress Society (Kanban resources)
Kanban vs. Scrum (Friendly comparison)
One day in Kanban Land (Kanban Cartoon!)
Scrumban (Fully mixed!)

If you are in the process of moving to Kanban from Scrum or adopting some of Kanban in your Scrum implementation, let us know how it is going! What worked well for you and what are some of your lessons learned?

17 comments:

abby, the hacker chick blog said...

Thanks for the intro to kanban! However, I'm afraid I'm stuck on this one line of yours...

"Whenever you finish your part of the story you are working on, you move on to the next story, just as in Scrum. "

Are you saying that when I, as a programmer, finish coding up my story, I throw it over the wall to a tester and move onto the next story?

I'm hoping I'm misunderstanding because that sounds like it's doing the opposite of what we want with agile, which is to have the whole team working on each story together. Not doing hand offs from one role to another like mini-waterfalls...? Can you explain this part a little further?

Damon Poole said...

Hi Abby,

As it turns out, the best translation of the Japanese word "Kanban" is in fact "throw it over the wall."

Actually, the real translation is "sign board" or "signal." Tee, hee, hee.

Anyway, good question. Getting the semantics just right is always a challenge. I did not mean that a person should throw anything over the wall to another person or do mini-waterfalls (ick).

I was actually much more eloquent later when I said "As soon as a story is completed, the team takes another story from the “todo” list and starts working on it. That then triggers the product owner to add another story from the backlog to the todo list." I have now applied an Orwellian edit to the part that you referenced.

That said, how this is implemented in practice is of course up to the team. I'm not advocating, just observing, that many Scrum teams do in fact have defined roles and you may have folks working on two stories where the developer has finished "their work" (including unit tests) for one story and then "move on" to another story. Of course, hopefully they still think of completing the story as a team effort and give interaction with the tester their top priority. Of course, that then gives rise to the question: Where Does Developer Testing End and Tester Testing Begin? :-)

Does that help?

abby, the hacker chick blog said...

*grin* It does indeed!

Thanks for the explanation. I'm intrigued by the concept but concerned it would encourage silo behavior (i finished my part, now it's the tester's problem!). But I can see that there is some ideal medium in there... it's just a matter of finding it. :-)

Pawel Brodzinski said...

Personally I don't think having 6-9 months of Scrum experience is such an important thing before entering Kanban. It can make transition to Kanban easier but it doesn't have to.

For what I believer the clue here is getting good at breaking the work down. As far as the team creates good stories to put on the board the rest comes much easier.

In my case we're still adjusting size of user stories although we're pretty close to being consistent in this area. The rest is tweaking the Kanban Board - adjusting limits and changing a bit a flow of story.

Damon Poole said...

Hi Pawel,

I agree with you. I think it depends heavily on the team and the circumstances. The 6-9 months is a general recommendation based on the fact that I've seen so many teams struggling with creating an increment of shippable work in Scrum, I worry that Kanban would be too free-form for those teams.

Keep in mind that my blog is "Do It Yourself Agile." It is geared for folks who don't have an experienced coach involved in their transition. Having an experienced coach involved that understands Kanban is a different matter.

Tobias Mayer said...

@HackerChick re silo behavior, here's a picture of a typical Kanban board:

http://bit.ly/QHok0
source: http://bit.ly/sTWER

Make up your own mind.

Damon Poole said...

@tobiasmayer Many folks doing Kanban have just backlog, todo, wip, done as their states. While it is possible that folks using the board that you reference have different people in different silos doing each piece with handoffs in between, that is certainly not what I am advocating and it is not what I hear when I listen sincerely to folks like David Anderson.

Everything you hold dear about Scrum still applies just as much to Kanban. The idea is that it is a cross-functional team, working as a team, getting the work done together.

Different people have different views on how many states to have, if you want just WIP, so be it. But other folks find that it is useful to know how work on a story is progressing.

If you are involved with a team that can only think in silos and handoffs, and you find that Kanban is encouraging their habits, then for Pete's sake do something about it.

Lastly, I think that probably the best place to comment on material from another blog that may be going in a different direction than I am is on that blog. :-o

Evsy said...

I have really enjoyed reading what you have written here. It's made me think. As a result I've had a response pinging around my head ever since and so for that I am grateful. I hope you take this response as a compliment. [Seems like I've gone overboard on my response and have to break it up into two pieces as it's too long! Sorry about that. It a reflection of how inspired I was].

My experience has been in using SCRUM and introducing some XP practices specifically user stories and pair programming which have been very useful for both the product owner and the team. I have limited experience of TTD and a little more of continuous integration. I have no experience of kanban and don't know anything about it. However I have recently been reading The Toyota Way by Jeffrey Liker and have been thinking about how Toyota's philosophy and practices can be applied to software development. Therefore some of what I write may be a bit naive but here goes anyway.

It seems to me that there is a one for one substitution between one piece flow and kanban. For me the kanban element is only when the hopper gets to a certain level that a new feature request/ product backlog item is added. The need for more parts to build the product, and that's good. One piece flow is separate and what your blog post is mainly about.

I would add that part of the value of the retrospective is to improve the process at scheduled points i.e. at the end of a sprint, the Toyota Production System (TPS) equivalent is the principle of kaisen. In TPS kaisen is worked into the process, using andon i.e. stopping the production/ development to suggest and improvement. To replicate this in software development the retrospective (now an inappropriate term) becomes part of the development process. Which, for me, begs the question 'what is the software development equivalent of the andon cord?'. The answer may be as simple as ringing a bell to stop the development and discuss the proposed improvement. This could trigger a stand up type meeting with it's own parameters. This could require a much more hands on product owner than I am used to but that's possible. Also I suggest that pair programming is/ can be used in kaisen. In the TPS small teams of 4 work together and rotate jobs partly so that they can cover each other but also to have an understanding of what the team is doing and to improve this.

To go back to the feature hopper and using the TPS model the product owner becomes like the chief engineer. They direct the macro elements of the project while the team is concerned with the value added work. The product backlog has in my experience been a bit hidden, in that even if it is available in the app that is being used e.g. jira, version one, rally. but is rarely accessed by anyone other than the product owner, and even then usually only when getting close to sprint planning. So my suggestion here would be to make the mid-level view of the product backlog more visible - stick it on the wall somewhere, update it every week even if the only change is the date on it then at least it is getting some attention. Put it in the reception of the building - then the product owner is more likely to look after it. The backlog items put into the hopper then become the low-level view of the product. The high-level view being the company 'strategy' and that could/should be more visible too so that each task performed by the team can be traced back to the overall strategy. (cont.d in next comment)

Evsy said...

Sprint reviews in my experience have been used as an opportunity for the product owner and other interested parties to check in with the product and in that regard have been very useful. With continuous integration and build they become less necessary as a show and tell to benefit the product owner, especially if the integrated product is made more visible - again have it on a wall or on a prominent monitor, I don't mean a screen showing the status of the integration but the product itself. If the product owner wants to regualarly show stakeholders and management then he can schedule his own product review meeting. Without continuous integration the sprint review seems to still have a place as, again in my experience, integration has been done at a time relative to the sprint review meeting.

As for sprint planning, well this could happen by the product owner having a scheduled meeting with the tech lead or snr developer. The sprint planning meeting does not add value to the product but it does serve to offer an opportunity for communication between the product owner and the team. OK so there's breaking the backlog items into sprint tasks and the management of the expectations of the product owner but this can happen as you suggest straight from the hopper. But there is something about a higher level discussion that helps the product owner to define his product. In my experience product ownership is not about having a vision. It is a real job in a real world with an eye on cost and technical feasiblity. Keep the sprint planning but continously improve it. It's all about evolution. The TPS approach to software development (more comfortable using this term than kanban) is not revolutionary, it is what Agile is doing anyway.

In my experience SCRUM is a starting point and a very good one. It says 'here are some pracitices, start using them, oh and by the way one of them is to change the practices'. TPS is the same but it is more upfront about its aims to remove waste, this emphasis is not so clear in SCRUM - although in TDD it is more apparent, with TDD you are 'building in the quality' and the throwing over the wall ceases to happen. SCRUM is quite prescriptive in the beginning and the subtleties are not that clear in some of the early books about it, it takes some getting used to before it can be usefully refined. There's a whole new language and ethos to be learned, job titles change, attitudes have to change and this does not happen over night. To go from waterfall to a subtley refining iterative method in one step with everyone being expected to be on the same page is at the very least ambitious. And that doesn't even begin to address the need to convince management of the benefits of Agile.

Toyota do not think they have TPS right and they have been doing it for decades. It is a humane and minimum ego method, it relies on co-operation and harmony and a long term view. It is not a fad or the latest new thing nor is it done piecemeal, it is worked into the fabric, the DNA, of the company. It's an incredibly difficult thing to get right, but it works in an amazing way.

Now I can't stop thinking about if Toyota were building software how would they do it.

Damon Poole said...

Evsy,

My goodness! I think that is the largest comment I've gotten on my blog. Thanks for taking the time to share your thoughts.

First off, I think you would enjoy the Poppendieck book referenced on my book recommendations page. It is all about Lean applied to software.

Also, in regards to one piece flow if you didn't already, you may enjoy reading my post on it which is referenced at the end of the post.

I like what you have to say and where you are thinking, your team is lucky to have you!

Evsy said...

I hope you didn't mind the length of the comment - it was a timing thing as I read your post as I was nearly the end of Toyota Way. I even thought about one piece flow while assembling some Ikea furniture the other day and could see the benefits there!

I had a look at the one piece flow as applied to development post. I found it very interesting. Strikes me that bits of process can be borrowed from different methods, with the self improvement mechanism included in all of them then one would hope that each team would evolve a process for them and be empowered to do make the changes.

As for my team being lucky to have me, thank you for saying that but unfortunately I lost my job a few months ago so I will have to wait to tell my next team!

karim said...

Good one on Self Improvement and it helps a lot.

Thanks,
Karim - Positive thinking

Robert Dempsey said...

Great post Damon. We've been doing Scrum for two years and I'm looking at Kanban in a big way. Are you planning any posts on estimating projects that then use Kanban as the development method?

Damon Poole said...

Robert,

I'm not planning a post on estimating for Kanban, but I can quickly share my thought on it. I mostly look at Kanban as a place to take ideas from and apply them to a Scrum foundation. So, I would say keep on estimating the same way for a Kanban project as for a Scrum project. The way I recommend to estimate for a Scrum project is that you are continuously grooming your backlog, continually doing story point estimation, and tracking your velocity.

The only trick with Kanban is to calculate your per-iteration velocity... since there are no iterations! But you can still track your velocity on a points per time period basis. How many points are you doing per 4 weeks for instance?

Happy to continue the dialog, it is an interesting topic.

Robert Dempsey said...

Thanks Damon. If I find any other resources on this topic I'll be sure to share them.

ScrumMaster201 said...

Our organization is about to make the jump from Scrum to Kanban. I understand the concepts around Kanban but there are one issue that all the articles I have read do not address.

I am not sure how the initial user stories are created. Customers want new (big) features and unless they are properly broken into smaller components there will be months of development involved. And, most organizations I know want to know what it will cost before they authorize a start (we need an estimate). In fact, the organization needs estimates on many different features to determine which one make business sense to work on (based on cost/benefit analysis).

How is this handled in a continuous process like Kanban?

Damon Poole said...

Hello ScrumMaster 201,

I'm not sure I understand the question. You say you are looking to make the jump from Scrum to Kanban, so that would imply that you are already doing Scrum. I don't understand why these questions are only now coming up with Kanban, how did you solve them with Scrum?

Is it that they weren't really solved with Scrum either and you are taking another look at the problem in general?