Multi-Stage Continuous Integration Part II

In Part I of this topic, I said that while Continuous Integration (CI) is a great practice, it can turn into “Continuous Noise” when combined with mainline development. Also, when the mainline is broken, everybody is affected until it is stabilized. I mentioned the idea of developer integration builds to solve this problem and pointed out that this can lead to task-switching and still doesn’t remove the problem of mainline instability.

In typical mainline development, in order for you to integrate your work with anybody else’s work, they first need to make their changes available by merging their changes with the mainline. Now, everybody is forced to integrate these changes. If the process of integrating your changes with another person or team’s changes requires multiple go-rounds, then it is much worse. Now, everybody else must continually integrate with two groups of changes as you and the other team go back and forth merging your changes into the mainline until they work together.

This creates a huge opportunity for duplication of effort. If you are working on module A which interacts with B and C, and B doesn’t currently work with C, then you are blocked until B and C are integrated. You might decide to take a look at it yourself, not knowing that somebody else is already working on it. Sure, you might be aware of who is responsible for B and C, but perhaps they are in a meeting right now and you have no good way of finding out. In any case, you are blocked. It may be that you knew ahead of time that there was a problem with the mainline. In that case, you just don’t take updates and concentrate instead on your own work. In effect, your ability to easily take changes from the mainline is “down” until the mainline is stable again.

A typical coping mechanism in this case is to serialize integration and ask everybody who is not involved in the current integration effort to hold off on merging in their changes until the current integration work has finished. This turns parallel development into serial development and kills productivity.

Next: Multi-Stage CI Part III