A comment from "AMWOP" states: "Unfortunately while progressive developers may want to work in an Agile fashion the ecosystem around them is frequently fighting against being Agile. Executives, Customers, Testers, Business planners and many other stakeholders insist on detailed, committed plans up front which give them a (false) sense of confidence about viability of long range plans for software delivery. This in fact drives the developers back towards waterfall methodologies. It won't be until these functions change their mindset that Agile processes will make significant inroads in commercial environments."
I wholeheartedly agree. As with most things, adoption takes time. This is a classic example of crossing the chasm. Currently, Agile is in the visionary and early adopter phase. The software development habits that people have now took years to form. It will take many more years before they change.
In addition, the infrastructure for widespread adoption is not yet there. There are lots of books that you can buy, but they are mostly oriented towards the early adopter. On the software development tools side, most are still oriented towards waterfall methodologies. This is changing, but slowly.
This is no surprise. Change takes time. My prediction is that in 5 years time, we will drop the term "Agile Development" and just talk about software development because we'll no longer need to make the distinction between Agile and non-Agile. Instead, people will talk about how often you release because everybody will be doing short iterations. I also predict that Agile will evolve to the point that people stop saying that you might as well just throw up your hands on predictability because they will realize that Agile actually gives you the opportunity to be more predictable. More on that later.
Wednesday, October 25, 2006
Tuesday, October 17, 2006
Agile Development - It's Human Nature
The Agile Manifesto claims that the benefits of Agile Development come in part from "People over Process". That never sat quite right with me, but I could never put my finger on why. An obvious reason of course is that I work for AccuRev, a development tool vendor that specializes in process-centric SCM. So of course I must love lots and lots of complicated process that requires a tool to implement. Actually, I'm a big believer in simplicity and automation.
I believe that attitude, people, and culture have always been important to the success of any project. Therefore, I don’t agree that there is anything special about Agile when it comes to attitude, people, or culture. It may well be that current Agile projects are populated by risk takers and visionaries, but if that was what it required to succeed, then it would never become very popular.
In my opinion, all of the benefits of Agile Development can be directly attributed to the practice of short iterations. That said, simply changing your plans to release more often, say every month, won’t work. While it doesn’t matter how you get to short iterations, you do have to do things differently to get there and many of the practices associated with Agile Development are good candidates for doing it.
Over time, I’ve come to realize that there is in fact something special and people-oriented about Agile. Ironically, it is the process that makes it people-oriented. I believe that short iterations leverage human nature and thus allow us to fall into a more natural rythym in which we make fewer mistakes and we work more productively. While it is true that people can succeed on projects that have very long timeframes and involve a high degree of stress at the end, I believe that people are much more productive and much less prone to error when they work at a constant and sustainable pace. This also means that when there are unforeseen circumstances, people are more likely to be able to respond well.
Consider the following statements. I’ll bet that you’ll agree with most of them.
• When working on software for personal use, people make changes and then use the new version right away.
• One of the things that developers enjoy about software development is the fact that they can make a change and see the result right away.
• When working on a hotfix, people are more likely to cross all of the “t”s and dot all of the “i”s and take extra care to make sure that everything is done just right.
• It isn’t easy to create something that the customer thinks is exactly right the first time.
• People prefer to delay making important decisions as long as possible.
• Towards the end of a release when a new bug is found or a great idea for an enhancement comes in, it is tempting to just put it into the current release rather than the next release because the next release won’t happen for quite a while.
Each of these are a part of human nature. All of the positive parts are reinforced with short iterations, and all of the negative parts are reduced with short iterations. Conversely, long iterations work against the grain of human nature.
I believe that attitude, people, and culture have always been important to the success of any project. Therefore, I don’t agree that there is anything special about Agile when it comes to attitude, people, or culture. It may well be that current Agile projects are populated by risk takers and visionaries, but if that was what it required to succeed, then it would never become very popular.
In my opinion, all of the benefits of Agile Development can be directly attributed to the practice of short iterations. That said, simply changing your plans to release more often, say every month, won’t work. While it doesn’t matter how you get to short iterations, you do have to do things differently to get there and many of the practices associated with Agile Development are good candidates for doing it.
Over time, I’ve come to realize that there is in fact something special and people-oriented about Agile. Ironically, it is the process that makes it people-oriented. I believe that short iterations leverage human nature and thus allow us to fall into a more natural rythym in which we make fewer mistakes and we work more productively. While it is true that people can succeed on projects that have very long timeframes and involve a high degree of stress at the end, I believe that people are much more productive and much less prone to error when they work at a constant and sustainable pace. This also means that when there are unforeseen circumstances, people are more likely to be able to respond well.
Consider the following statements. I’ll bet that you’ll agree with most of them.
• When working on software for personal use, people make changes and then use the new version right away.
• One of the things that developers enjoy about software development is the fact that they can make a change and see the result right away.
• When working on a hotfix, people are more likely to cross all of the “t”s and dot all of the “i”s and take extra care to make sure that everything is done just right.
• It isn’t easy to create something that the customer thinks is exactly right the first time.
• People prefer to delay making important decisions as long as possible.
• Towards the end of a release when a new bug is found or a great idea for an enhancement comes in, it is tempting to just put it into the current release rather than the next release because the next release won’t happen for quite a while.
Each of these are a part of human nature. All of the positive parts are reinforced with short iterations, and all of the negative parts are reduced with short iterations. Conversely, long iterations work against the grain of human nature.
Wednesday, October 04, 2006
A Big Bang AccuRev Release
Some of my more recent thinking on software development, especially in regards to Agile Development, is in an article in the October issue of Queue Magazine, entitled "Breaking the Major Release Habit."
With that in mind I would like to mention that AccuRev 4.5 is now available. While it is called 4.5, it is actually one of our biggest releases ever. Here's a blog that mentions it. So why didn't we call it 5.0? Thankfully, that's not my department. :-) Anyway, my point is that while I believe very strongly in short iterations and more frequent releases, organizational change takes time and is hard to do.
To that end, I am currently working on a new project and piloting a new Agile methodology to develop it called Hyper Agile. The goal of the new project is to take AccuRev, which is already well suited to Agile projects, to the next level of Agile functionality. The aim of Hyper is to take a fresh look at Agile Development and scale it beyond small co-located teams. A preview of the Hyper methodology will appear in an upcoming issue of Dr. Dobb's, stay tuned!
With that in mind I would like to mention that AccuRev 4.5 is now available. While it is called 4.5, it is actually one of our biggest releases ever. Here's a blog that mentions it. So why didn't we call it 5.0? Thankfully, that's not my department. :-) Anyway, my point is that while I believe very strongly in short iterations and more frequent releases, organizational change takes time and is hard to do.
To that end, I am currently working on a new project and piloting a new Agile methodology to develop it called Hyper Agile. The goal of the new project is to take AccuRev, which is already well suited to Agile projects, to the next level of Agile functionality. The aim of Hyper is to take a fresh look at Agile Development and scale it beyond small co-located teams. A preview of the Hyper methodology will appear in an upcoming issue of Dr. Dobb's, stay tuned!
Thursday, June 01, 2006
Agile: Turning on a Dime
If you've read any of my previous posts, the title of this post might make you think my blog has been hacked and somebody else is now posting in my stead. But actually, it is just that I had an epiphany. I still have the same beliefs as before, but I realize now that my (let's be frank) negative reaction to Agile is not actually directed at Agile itself, but rather at specific individual practices.
I've done lots more reading and thinking about development methodologies and have come to the following conclusion: the key benefits of Agile development derive from the Agile practice of short iterations. In my opinion, everything else is simply an enabler. For instance, writing tests first is a good way to make sure that you keep the quality of the frequent releases high. But, as long as you are able to consistently do short iterations, it doesn't matter what you do to keep quality high as long as you do it.
So if like me you don't like the idea of using 3x5 cards to manage the contents of a release, fine. Use whatever you want. Use a text file, use Bugzilla, use RallyDev, it doesn't matter. The method you use is inconsequential as long as you are doing short iterations.
The vocabulary doesn't matter either. I don't happen to like the idea of creating new words when there are perfectly good words that mean the same thing. I prefer to talk about issues rather than stories. As long as I'm using short iterations, that's fine, the benefits remain the same.
As to the benefits, I believe that the key benefits of Agile development are: shorter time to value and accelerated maturation of product changes. I say shorter time to value rather than shorter time to market because realizing value (via revenues ) is much more important than availability of new features.
Product quality is an ongoing effort. No matter how much testing you do, users will find ways to use your product that you didn't think of. For any change, whether it is a bug fix or a new feature, you don't really know the impact until customers start using it. That means that the maturation of a change doesn't really start until the change ships. If you have a long release cycle, you will have changes that are made early that are basically "sitting on the shelf" until you ship. You can't really say that they are tested until you test the whole product as it will ship, so again, true maturation doesn't start until you ship.
An important aspect of maturation is customer feedback. By using short iterations, you get customer feedback sooner. Instead of releasing a major new feature all at once, if you break it up into logical pieces (assuming that it can be done for a particular feature), you can find out sooner which pieces are taking the feature in the right direction and which are not and adjust your plans accordingly. For example, let's say that the perfect implementation of a new feature has 12 sub-components. Of course, you can't know in advance what that perfect implementation is. Using a traditional approach, you implement the first version in a year and it includes 12 sub-components. It turns out that 6 are exactly what was desired and 6 are completely wrong (even though that's what was asked for). So, a year later you release the second version and now you've got 9 out of 12 correct... and so on.
Using small iterations, you could have released a component a month, getting feedback once a month instead of once a year. Logically, you will converge on the correct solution much faster using a tighter feedback loop.
Another way to look at short iterations is as the turning radius of an organization. The shorter the iterations, the smaller the turning radius and the greater the agility.
Of course, agility is not the only important measure of an organization. Small teams may feel that their agility gives them the ability to outperform a larger team. It seems that Agile is primarily used by small teams today and so that's probably true today up to a point. However, I see no reason to believe that Agile is just for small groups. I believe that as Agile sheds the fetters of small-scale practices such as using 3x5 cards, larger and larger projects will adopt it and enjoy its benefits.
For reference, here is the reading material that steered me in this direction (so to speak) in the order that I read them:
Integrating Agile Development in the Real World, Peter Schuh
An excellent survey of most of the Agile methods.
Extreme Programming Explained - 2nd Edition, Kent Beck
Whether you agree with the idea of 3x5 cards or not, Kent's book is a short and easy read that is well worth the effort. A seminal book on Agile development.
Lean Six Sigma, Michael L. George
I read this book because I was interested in learning more about Six Sigma. However, I selected this book because the idea of "Lean" was appealing. The way that the problem and solutions for finished product sitting on a factory floor is explained immediately suggested parallels to finished features sitting in the source repository waiting for release.
Lean Software Development, Mary Poppendieck and Tom Poppendieck
These folks have done an excellent job of taking the lessons of Lean manufacturing, pioneered by the Toyota Production System, and translating them to software development. I read "Lean Six Sigma" first and on my next trip to the book store "Lean Software Development" became an obvious choice. Reading this book was the tipping point for me.
Agile Project Management with Scrum, Ken Schwaber
If you are already familiar with Scrum, this is a treasure trove of examples of how it has been applied in the real world.
I've done lots more reading and thinking about development methodologies and have come to the following conclusion: the key benefits of Agile development derive from the Agile practice of short iterations. In my opinion, everything else is simply an enabler. For instance, writing tests first is a good way to make sure that you keep the quality of the frequent releases high. But, as long as you are able to consistently do short iterations, it doesn't matter what you do to keep quality high as long as you do it.
So if like me you don't like the idea of using 3x5 cards to manage the contents of a release, fine. Use whatever you want. Use a text file, use Bugzilla, use RallyDev, it doesn't matter. The method you use is inconsequential as long as you are doing short iterations.
The vocabulary doesn't matter either. I don't happen to like the idea of creating new words when there are perfectly good words that mean the same thing. I prefer to talk about issues rather than stories. As long as I'm using short iterations, that's fine, the benefits remain the same.
As to the benefits, I believe that the key benefits of Agile development are: shorter time to value and accelerated maturation of product changes. I say shorter time to value rather than shorter time to market because realizing value (via revenues ) is much more important than availability of new features.
Product quality is an ongoing effort. No matter how much testing you do, users will find ways to use your product that you didn't think of. For any change, whether it is a bug fix or a new feature, you don't really know the impact until customers start using it. That means that the maturation of a change doesn't really start until the change ships. If you have a long release cycle, you will have changes that are made early that are basically "sitting on the shelf" until you ship. You can't really say that they are tested until you test the whole product as it will ship, so again, true maturation doesn't start until you ship.
An important aspect of maturation is customer feedback. By using short iterations, you get customer feedback sooner. Instead of releasing a major new feature all at once, if you break it up into logical pieces (assuming that it can be done for a particular feature), you can find out sooner which pieces are taking the feature in the right direction and which are not and adjust your plans accordingly. For example, let's say that the perfect implementation of a new feature has 12 sub-components. Of course, you can't know in advance what that perfect implementation is. Using a traditional approach, you implement the first version in a year and it includes 12 sub-components. It turns out that 6 are exactly what was desired and 6 are completely wrong (even though that's what was asked for). So, a year later you release the second version and now you've got 9 out of 12 correct... and so on.
Using small iterations, you could have released a component a month, getting feedback once a month instead of once a year. Logically, you will converge on the correct solution much faster using a tighter feedback loop.
Another way to look at short iterations is as the turning radius of an organization. The shorter the iterations, the smaller the turning radius and the greater the agility.
Of course, agility is not the only important measure of an organization. Small teams may feel that their agility gives them the ability to outperform a larger team. It seems that Agile is primarily used by small teams today and so that's probably true today up to a point. However, I see no reason to believe that Agile is just for small groups. I believe that as Agile sheds the fetters of small-scale practices such as using 3x5 cards, larger and larger projects will adopt it and enjoy its benefits.
For reference, here is the reading material that steered me in this direction (so to speak) in the order that I read them:
Integrating Agile Development in the Real World, Peter Schuh
An excellent survey of most of the Agile methods.
Extreme Programming Explained - 2nd Edition, Kent Beck
Whether you agree with the idea of 3x5 cards or not, Kent's book is a short and easy read that is well worth the effort. A seminal book on Agile development.
Lean Six Sigma, Michael L. George
I read this book because I was interested in learning more about Six Sigma. However, I selected this book because the idea of "Lean" was appealing. The way that the problem and solutions for finished product sitting on a factory floor is explained immediately suggested parallels to finished features sitting in the source repository waiting for release.
Lean Software Development, Mary Poppendieck and Tom Poppendieck
These folks have done an excellent job of taking the lessons of Lean manufacturing, pioneered by the Toyota Production System, and translating them to software development. I read "Lean Six Sigma" first and on my next trip to the book store "Lean Software Development" became an obvious choice. Reading this book was the tipping point for me.
Agile Project Management with Scrum, Ken Schwaber
If you are already familiar with Scrum, this is a treasure trove of examples of how it has been applied in the real world.
Thursday, May 25, 2006
I Skipped 3x5 Cards and Went Straight To PostIt Notes
Previously, I wrote "After all, isn't the fundamental reason for software the automation of manual processes? Why in the world should software developers ever have to resort to keeping track of things with 3x5 cards or spreadsheets".
In response, in a comment, someone anonymously responded: "Software is used to replace repedative(sic) manual processes. In practice the use use(sic) of cards works well. To be blunt if you havn't(sic) tried XP with cards you are not qualified to comment on thier(sic) suitability."
Anonymous, your first sentence seems to be a restatement/validation of my first sentence. If you are trying to make a different point, I missed it. As for the use of cards working well, they work well in comparison to what? I will of course agree that using cards is better than nothing. I tried that prior to using issue tracking. I will also agree that using cards is better than trying to keep it all straight in my head. I also tried that. Ok, you've got me on the actual use of actual 3x5 cards.
I have to admit that I have never used 3x5 cards to keep track of stories related to software development. Though I did use them to keep track of role-playing characters at one point. Hmm. But the whole time I was wishing that they were on my computer. And I did actually try that, but storing the info on tape cassette and trying to find the exact place on tape was just too much of a manual process. These days, role-playing games are on-line, in 3D, and globally distributed. Things change fast.
But I digress. So, I admit I never used 3x5 cards in software development. I skipped that technology and went right to post-it notes. I had actually been hoping to gloss over that period in my career, but now you've forced it out of me. And though I'm smiling as I write this, it is in fact true that I used post-it notes.
That was actually a very short-lived adventure when I was working on an early prototype of what later became AccuRev. Within weeks we had moved on to storing everything in a text file. Here's a snippet:
O --- M -- - special case of merging a defuncted element 4h
C 502 E D3 - be forgiving when principal is included in ws name
I don't remember the format any more, but the terse description was the "story" and there was usually an estimate, such as "4h" for 4 hours. Many of them even had bug numbers. It was primitive, but it was a big relief getting off of the post-it notes. At least with the text file we could send it around, make changes, do diff/merge, search, create scripts to do reports, etc. We even had a script that created a web page for remote viewing.
Of course it wasn't long after that we realized that we were writing yet another issue tracking system, so we installed a free one in a couple of hours, breathed another sigh of relief, and went back to working on our real goals.
In short, I suppose I can't comment on my experience using 3x5 cards. However, based on the application of logical thought to my above experience, I think I'll take a pass. Call me crazy for liking the ability to use powerful tools to edit, search, categorize, re-organize, report, track, manage, distribute, share, and do backups. Say that I'm too demanding to want to be able to do these things from my office, from an office on the other side of the building, from home, or from a remote site. That's fine with me. Just don't expect me to go back to post-it notes (or 3x5 cards) any time soon.
Here's another way to think about it. What sort of reaction would you get if you suggested to a user of whatever software that you produce that one of the new features they wanted would be implemented using 3x5 cards? What if you also suggested that they pay for that feature?
I'm not saying you should toss the 3x5 cards and shell out dough (though if you feel like it let me know :-) ). But for goodness sake at least use a text file, or Excel (or Open Office), or Bugzilla.
In response, in a comment, someone anonymously responded: "Software is used to replace repedative(sic) manual processes. In practice the use use(sic) of cards works well. To be blunt if you havn't(sic) tried XP with cards you are not qualified to comment on thier(sic) suitability."
Anonymous, your first sentence seems to be a restatement/validation of my first sentence. If you are trying to make a different point, I missed it. As for the use of cards working well, they work well in comparison to what? I will of course agree that using cards is better than nothing. I tried that prior to using issue tracking. I will also agree that using cards is better than trying to keep it all straight in my head. I also tried that. Ok, you've got me on the actual use of actual 3x5 cards.
I have to admit that I have never used 3x5 cards to keep track of stories related to software development. Though I did use them to keep track of role-playing characters at one point. Hmm. But the whole time I was wishing that they were on my computer. And I did actually try that, but storing the info on tape cassette and trying to find the exact place on tape was just too much of a manual process. These days, role-playing games are on-line, in 3D, and globally distributed. Things change fast.
But I digress. So, I admit I never used 3x5 cards in software development. I skipped that technology and went right to post-it notes. I had actually been hoping to gloss over that period in my career, but now you've forced it out of me. And though I'm smiling as I write this, it is in fact true that I used post-it notes.
That was actually a very short-lived adventure when I was working on an early prototype of what later became AccuRev. Within weeks we had moved on to storing everything in a text file. Here's a snippet:
O --- M -- - special case of merging a defuncted element 4h
C 502 E D3 - be forgiving when principal is included in ws name
I don't remember the format any more, but the terse description was the "story" and there was usually an estimate, such as "4h" for 4 hours. Many of them even had bug numbers. It was primitive, but it was a big relief getting off of the post-it notes. At least with the text file we could send it around, make changes, do diff/merge, search, create scripts to do reports, etc. We even had a script that created a web page for remote viewing.
Of course it wasn't long after that we realized that we were writing yet another issue tracking system, so we installed a free one in a couple of hours, breathed another sigh of relief, and went back to working on our real goals.
In short, I suppose I can't comment on my experience using 3x5 cards. However, based on the application of logical thought to my above experience, I think I'll take a pass. Call me crazy for liking the ability to use powerful tools to edit, search, categorize, re-organize, report, track, manage, distribute, share, and do backups. Say that I'm too demanding to want to be able to do these things from my office, from an office on the other side of the building, from home, or from a remote site. That's fine with me. Just don't expect me to go back to post-it notes (or 3x5 cards) any time soon.
Here's another way to think about it. What sort of reaction would you get if you suggested to a user of whatever software that you produce that one of the new features they wanted would be implemented using 3x5 cards? What if you also suggested that they pay for that feature?
I'm not saying you should toss the 3x5 cards and shell out dough (though if you feel like it let me know :-) ). But for goodness sake at least use a text file, or Excel (or Open Office), or Bugzilla.
Wednesday, February 22, 2006
Leverage Your Core Competency
SD Times recently published my article on leveraging your core competency (free registration required).
In a recent posting, CollabNet is apparently attempting to reframe the article - which is fundamentally about leveraging an organization’s core competency - into a discussion of commercial software versus open source software. This is surprising because both CollabNet and AccuRev have business models that are based on a hybrid approach that involves a mix of commercial and open source tools.
AccuRev is a member of the Eclipse foundation, and a contributing member of the Eclipse ALF project which aims to provide a framework for software development tool interoperability. Furthermore, as an ardent believer in the advantages of good SCM, AccuRev salutes CollabNet’s efforts to advance the state of the art. We have a great deal of respect for the work that they have done with CollabNet Subversion and believe that CollabNet Subversion is arguably the best free SCM tool available today.
The central point of the article is that businesses should do a careful ROI analysis of the costs associated with maintaining their own homegrown SCM system (open source-based or not) in comparison with purchasing similar off-the-shelf functionality from vendors such as AccuRev and CollabNet. The subtler decision points are in the gaps between any given tool’s actual implemented capabilities and your full set of technology requirements.
In a recent posting, CollabNet is apparently attempting to reframe the article - which is fundamentally about leveraging an organization’s core competency - into a discussion of commercial software versus open source software. This is surprising because both CollabNet and AccuRev have business models that are based on a hybrid approach that involves a mix of commercial and open source tools.
AccuRev is a member of the Eclipse foundation, and a contributing member of the Eclipse ALF project which aims to provide a framework for software development tool interoperability. Furthermore, as an ardent believer in the advantages of good SCM, AccuRev salutes CollabNet’s efforts to advance the state of the art. We have a great deal of respect for the work that they have done with CollabNet Subversion and believe that CollabNet Subversion is arguably the best free SCM tool available today.
The central point of the article is that businesses should do a careful ROI analysis of the costs associated with maintaining their own homegrown SCM system (open source-based or not) in comparison with purchasing similar off-the-shelf functionality from vendors such as AccuRev and CollabNet. The subtler decision points are in the gaps between any given tool’s actual implemented capabilities and your full set of technology requirements.
Friday, January 27, 2006
What Exactly is a Stream Anyway?
From time to time, people ask "what is a Stream?" At this point pretty much anybody associated with software development has heard of branches, but streams are a relatively new concept which is similar to but at a higher level than branches.
Here's my .02 on streams, coming from the AccuRev point of view. First, I think that at some level the various SCM systems which talk about "streams" are probably all trying to achieve something fairly similar. That is, a "stream of development."
My initial exposure to "streams" was from hearing folks talk about "streams of development" independent of the SCM system that they were using (which did not have such a concept). The idea was that you had work which was towards a particular purpose, such as new development, maintenance, a team working together on a sub-project, etc. Each of these was a "stream of development".
I have also heard the terms "codeline", "development effort", and "line of development" used in the same context. At the end of the day, the folks which initiate these things (managers, business people, etc) don't really care how they are implemented, they just want to ask questions like "how is 4.0 coming along" and "are all of the fixes from maintenance in the latest release?" Somebody else then translates that into the appropriate queries, which may be in terms of branches, scripts, streams, or something else.
Prior to AccuRev, I found the need to tranlate somewhat mystifying. Why should there be any difference between the mental model of "streams" and the implementation model?
Thus, in AccuRev, the mental model and the implementation model are the same. Streams are the basic building block of the architecture. There are no branches or labels, just streams. There are streams for releases, streams for active development, streams for end users, etc. Each stream except the root stream is defined in terms of a parent stream and inherits everything from the parent (recursively).
So, if you want to do maintenance on the 4.0 release, you would create a new stream based on 4.0 . Through inheritance, it is the same as 4.0. The definition itself is all that is required. The definition is simply "4.0_maint" is everything in "4.0" plus all of the changes in "4.0_maint".
Since streams are first class objects, you can act on them directly. You can assign security attributes, lock them, define other streams in terms of them, compare them directly, do queries on them without having to understand how the streams themselves are composed, etc.
For the curious, I've written a whitepaper which describes AccuRev's stream architecture at an even deeper level:
http://accurev.com/product/docs/AccuRev_Streams.pdf
And if you want to go even deeper than that, the basis of AccuRev's streams is TimeSafe:
http://accurev.com/accurev/info/timesafe.html
Here's my .02 on streams, coming from the AccuRev point of view. First, I think that at some level the various SCM systems which talk about "streams" are probably all trying to achieve something fairly similar. That is, a "stream of development."
My initial exposure to "streams" was from hearing folks talk about "streams of development" independent of the SCM system that they were using (which did not have such a concept). The idea was that you had work which was towards a particular purpose, such as new development, maintenance, a team working together on a sub-project, etc. Each of these was a "stream of development".
I have also heard the terms "codeline", "development effort", and "line of development" used in the same context. At the end of the day, the folks which initiate these things (managers, business people, etc) don't really care how they are implemented, they just want to ask questions like "how is 4.0 coming along" and "are all of the fixes from maintenance in the latest release?" Somebody else then translates that into the appropriate queries, which may be in terms of branches, scripts, streams, or something else.
Prior to AccuRev, I found the need to tranlate somewhat mystifying. Why should there be any difference between the mental model of "streams" and the implementation model?
Thus, in AccuRev, the mental model and the implementation model are the same. Streams are the basic building block of the architecture. There are no branches or labels, just streams. There are streams for releases, streams for active development, streams for end users, etc. Each stream except the root stream is defined in terms of a parent stream and inherits everything from the parent (recursively).
So, if you want to do maintenance on the 4.0 release, you would create a new stream based on 4.0 . Through inheritance, it is the same as 4.0. The definition itself is all that is required. The definition is simply "4.0_maint" is everything in "4.0" plus all of the changes in "4.0_maint".
Since streams are first class objects, you can act on them directly. You can assign security attributes, lock them, define other streams in terms of them, compare them directly, do queries on them without having to understand how the streams themselves are composed, etc.
For the curious, I've written a whitepaper which describes AccuRev's stream architecture at an even deeper level:
http://accurev.com/product/docs/AccuRev_Streams.pdf
And if you want to go even deeper than that, the basis of AccuRev's streams is TimeSafe:
http://accurev.com/accurev/info/timesafe.html
Friday, September 30, 2005
Is Extreme Programming an Overreaction? (Part 2)
Although it may be an overreaction, what is it a reaction to? Many statistics show that even today, most software projects fail. It seems reasonable to believe that many companies are looking for a new way of doing things. From the programmer's point of view, it probably seems like it is a failure of management and a lack of proper tools to get the job done. If that's the case, it would make sense to try to minimize management and throw out the old tools.
On the other hand, there's the famous adage that managing programmers is like herding cats. From a management perspective, it can be frustrating as well: "why don't the programmers understand how important it is to get feature XYZ completed by the deadline?"
I think the truth is probably a combination of things; insufficient understanding by management of the complexities of software development, lack of reliable visibility into the plan and current status, overly complex and insufficient tools, insufficient understanding by programmers of the complexities of business realities, and lack of information in the trenches about how the current development plan is connected to the business plan.
It seems to me that part of the problem is plain and simple education. Having been in both camps, often simultaneously, I can shed some light on this subject, but my favorite part and the part I think tool vendors can help the most with is providing tools that provide high visibility of all aspects of the software development process to all stakeholders in real time.
After all, isn't the fundemental reason for software the automation of manual processes? Why in the world should software developers ever have to resort to keeping track of things with 3x5 cards or spreadsheets that provide only a modest benefit over paper and pencil compared with using a software tool that has been tailor made for tracking software development activities? What's next? A return to punch cards?!
As a software development tool vendor, I believe we are obligated to sit up, take notice, and do something about this situation. Otherwise, it seems that we can look forward to state of the art software being created with the software equivalent of sharp sticks and stone knives.
One company that has started to respond to this challenge is the Rally software company with their Rally software product. It automates many of the tasks associated with Agile development. I salute their efforts and hope that all tool vendors will respond in kind. I know that I'll be doing my utmost to make sure that AccuRev takes heed.
On the other hand, there's the famous adage that managing programmers is like herding cats. From a management perspective, it can be frustrating as well: "why don't the programmers understand how important it is to get feature XYZ completed by the deadline?"
I think the truth is probably a combination of things; insufficient understanding by management of the complexities of software development, lack of reliable visibility into the plan and current status, overly complex and insufficient tools, insufficient understanding by programmers of the complexities of business realities, and lack of information in the trenches about how the current development plan is connected to the business plan.
It seems to me that part of the problem is plain and simple education. Having been in both camps, often simultaneously, I can shed some light on this subject, but my favorite part and the part I think tool vendors can help the most with is providing tools that provide high visibility of all aspects of the software development process to all stakeholders in real time.
After all, isn't the fundemental reason for software the automation of manual processes? Why in the world should software developers ever have to resort to keeping track of things with 3x5 cards or spreadsheets that provide only a modest benefit over paper and pencil compared with using a software tool that has been tailor made for tracking software development activities? What's next? A return to punch cards?!
As a software development tool vendor, I believe we are obligated to sit up, take notice, and do something about this situation. Otherwise, it seems that we can look forward to state of the art software being created with the software equivalent of sharp sticks and stone knives.
One company that has started to respond to this challenge is the Rally software company with their Rally software product. It automates many of the tasks associated with Agile development. I salute their efforts and hope that all tool vendors will respond in kind. I know that I'll be doing my utmost to make sure that AccuRev takes heed.
Thursday, September 29, 2005
Is Extreme Programming an Overreaction? (Part 1)
I've been doing a lot of research and thinking about methodology recently. I also went to SD East this week (great show, great conference) where I was overloaded by Agile this and Extreme that. I guess I was lucky though. I joked to one speaker that I figured I'd made a mistake by leaving out the word "Agile" from my presentation title since it was in practically every other presentation title. He shrugged and said "it seems less here than at other conferences." I can only imagine which conference he recently attended. It must have been called something like "Going Beyond Extreme Programming" or "Lighting Methods for Legacy Agile Developers".
I've read up on Agile methods and Extreme Programming, read articles, talked with prospects and customers about it, and used many of the ideas over the years. For the most part I had heard about those ideas or learned about them prior to having heard of either "Agile" or XP, so I never really thought of myself as an Agile person or an XP person.
I also heard about pair programming, but outside of frequent code reviews, haven't actually done true pair programming. It doesn't strike me as my cup of tea.
In any case, I guess I had subconsciously formed an opinion about Agile methods and then began filtering out new information. After all, I "knew" what it was, so now I could think about other things.
But then I did some more in-depth research on modern methodologies in general, spent some time at a customer that is "going Agile" and followed that up with a week at SD East. I was forced to realize that I had not fully appreciated what it meant to be "Agile."
I can't help but thinking: "is this an overreaction to something?" I've heard things like "but isn't extra training contrary to the manifesto that says 'the only measure of success is creating working code?'" and "keep track of the project with a stack of 3x5 cards." You might say that I'm the one that is overreacting, that I'm only picking up on "the extremes." I don't generally react so strongly to an extreme comment here and there, so I'm pretty sure it is the steady drumbeat that I'm reacting to. By the way, how do you Google a stack of 3x5 cards?
My gut tells me that Agile is a programmer-centric reaction to a feeling that there is too much bureacracy keeping the programmer down; mangement is too heavy-handed, and the development tool stack is too hard to use. If only the programmers were in charge, everything would be better. And to top it all off, the guiding principles are set out in a Manifesto. I agree with some of the principles in the Manifesto, but to me it seems to go a bit too far.
For instance, take the principal that "Working software is the primary measure of progress." That seems far too programmer-centric to me. I agree, there's nothing quite like the rush of seeing a newly implemented feature in action, or the satisfaction of squashing a clever bug. I could spend all day implementing cool features and squashing bugs. But outside of personal hobbies and coursework, isn't the point of developing software to maximize customer happiness and new customer acquisition? At the end of the day, isn't software development a tool for achieving success in business?
Now that I have your attention, stay tuned for Part 2!
I've read up on Agile methods and Extreme Programming, read articles, talked with prospects and customers about it, and used many of the ideas over the years. For the most part I had heard about those ideas or learned about them prior to having heard of either "Agile" or XP, so I never really thought of myself as an Agile person or an XP person.
I also heard about pair programming, but outside of frequent code reviews, haven't actually done true pair programming. It doesn't strike me as my cup of tea.
In any case, I guess I had subconsciously formed an opinion about Agile methods and then began filtering out new information. After all, I "knew" what it was, so now I could think about other things.
But then I did some more in-depth research on modern methodologies in general, spent some time at a customer that is "going Agile" and followed that up with a week at SD East. I was forced to realize that I had not fully appreciated what it meant to be "Agile."
I can't help but thinking: "is this an overreaction to something?" I've heard things like "but isn't extra training contrary to the manifesto that says 'the only measure of success is creating working code?'" and "keep track of the project with a stack of 3x5 cards." You might say that I'm the one that is overreacting, that I'm only picking up on "the extremes." I don't generally react so strongly to an extreme comment here and there, so I'm pretty sure it is the steady drumbeat that I'm reacting to. By the way, how do you Google a stack of 3x5 cards?
My gut tells me that Agile is a programmer-centric reaction to a feeling that there is too much bureacracy keeping the programmer down; mangement is too heavy-handed, and the development tool stack is too hard to use. If only the programmers were in charge, everything would be better. And to top it all off, the guiding principles are set out in a Manifesto. I agree with some of the principles in the Manifesto, but to me it seems to go a bit too far.
For instance, take the principal that "Working software is the primary measure of progress." That seems far too programmer-centric to me. I agree, there's nothing quite like the rush of seeing a newly implemented feature in action, or the satisfaction of squashing a clever bug. I could spend all day implementing cool features and squashing bugs. But outside of personal hobbies and coursework, isn't the point of developing software to maximize customer happiness and new customer acquisition? At the end of the day, isn't software development a tool for achieving success in business?
Now that I have your attention, stay tuned for Part 2!
Thursday, August 18, 2005
Acquiring a New SCM Tool
Folks often ask "what is the best SCM tool" or "is there a comparison of SCM tools available." I would say that generic comparisons of SCM tools are sort of like generic comparisons of cars. Somebody looking for something that is fast and doesn't care about gas mileage probably wouldn't think much of a Prius.
Currently, IMHO, the SCM market is not a commodity market where the products are generally about the same. There is a wide range in both pricing and functionality. So, you really have to know your requirements and the priority of those requirements. A good initial priority to figure out is: which is more important to the organization; price or return on investment? There's no sense in looking at all of the tools on the market if there's not enough budget.
Generally, a good way to acquire a new SCM tool is to start from a well defined set of requirements, create a shortlist of 3-5 candidates that seem close, take a closer look and narrow that down to 2, and then do a proof of concept on those 2 and pick the 1 that most closely meets your requirements.
Depending on the size of the development group and the size of the company, acquiring a new SCM tool can be an eye-opening experience. Finding a tool that meets your requirements is perhaps the easiest part of the process. Some of the harder parts are: figuring out who all of the stakeholders are, getting all stakeholders to consensus on the requirements, getting upper-management buy-in, and securing the budget for the purchase. Getting budget and requirements to match is generally the hardest part and typically involves writing some sort of business case to justify the purchase.
Here are some typical stakeholders: CEO, CFO, Purchasing Agent, Legal, VP of Engineering, Director of QA, key developers, and Release Engineering. Some of these may seem to have nothing to do with SCM, but they do get involved in large purchases that affect the whole company. It is better to make contact with all of these folks early to get the full scope of what is required to change SCM tools so as not to get surprised later and potentially impact release schedules.
If you make a recommendation that fits the budget and best meets the other requirements, but there is still a big gap in the requirements, don't despair! If you've involved all of the stakeholders and built consensus, you have a good chance of leveraging that gap to increase the budget.
Currently, IMHO, the SCM market is not a commodity market where the products are generally about the same. There is a wide range in both pricing and functionality. So, you really have to know your requirements and the priority of those requirements. A good initial priority to figure out is: which is more important to the organization; price or return on investment? There's no sense in looking at all of the tools on the market if there's not enough budget.
Generally, a good way to acquire a new SCM tool is to start from a well defined set of requirements, create a shortlist of 3-5 candidates that seem close, take a closer look and narrow that down to 2, and then do a proof of concept on those 2 and pick the 1 that most closely meets your requirements.
Depending on the size of the development group and the size of the company, acquiring a new SCM tool can be an eye-opening experience. Finding a tool that meets your requirements is perhaps the easiest part of the process. Some of the harder parts are: figuring out who all of the stakeholders are, getting all stakeholders to consensus on the requirements, getting upper-management buy-in, and securing the budget for the purchase. Getting budget and requirements to match is generally the hardest part and typically involves writing some sort of business case to justify the purchase.
Here are some typical stakeholders: CEO, CFO, Purchasing Agent, Legal, VP of Engineering, Director of QA, key developers, and Release Engineering. Some of these may seem to have nothing to do with SCM, but they do get involved in large purchases that affect the whole company. It is better to make contact with all of these folks early to get the full scope of what is required to change SCM tools so as not to get surprised later and potentially impact release schedules.
If you make a recommendation that fits the budget and best meets the other requirements, but there is still a big gap in the requirements, don't despair! If you've involved all of the stakeholders and built consensus, you have a good chance of leveraging that gap to increase the budget.
Subscribe to:
Posts (Atom)