Featured Whitepapers

Tool Spotlight

Upcoming & Recent Webcasts

Home

CM Journal Articles

Rapid Response or Reactive CM - Responding to the Demands of Agile Development

Written by Ben Weatherall

Tuesday, 15 May 2007 14:32

A conversation with the Schizophrenic Paranoid
Para: How can CM respond faster to the needs of Development without compromising principles?
Noid: Especially since there are so few of us in any one organization!
Para: And there are not only multiple projects spanning multiple repositories, there are multiple roles we have to fill.
Noid: Roles no one even thinks about such as Build Manager and Release Manager!
Fran: Be calm. Be quiet! Do you want to let "them" know we're here?
Noid: We aren't like the Cobbler's children, we are the elves and brownies that come out at night and make the shoes. Magic! That's what "they" want. Magic

"Agile CM" and "CM for Agile Development" are two different things. The first is the application of agile principles and practices to the development work performed by CM. The second is the proactive and rapid response of CM to one or more Agile development projects. This article will look at "CM for Agile Development" since it is what most Developers and Managers are concerned with.

There are three core areas where CM is visible within Agile Development: Version Control, Defect Tracking and Build Management. Within each of these areas there are questions we as CM'ers should be asking ourselves if we purport to support Agile Development:

Version Control

Does your tool of choice support timely tracking of code changes?
Does your tool integrate into the Developers' IDE(s) of choice?
Does your tool allow for effective refactoring? In other words, does it allow for moves and renames of files and directories instead of just removing and re-adding them?
Does your tool allow for quick and safe branch creation?
Does your tool support ease of merging and the ability to identify merge contributors and results?
Does your tool support ease of backing out changes and undoing merges?

Defect Tracking

Does your tool of choice support Issue entry by Customers
Does your tool support easy reprioritization?
Does your tool allow Developers to pick the next bug/issue/feature to work on?
Does your tool integrate with the Version Control tool to at least the level of tracking the elements that change for each Defect Record?
Does your tool integrate with the Build Management tool to at least the level of tracking the build(s) that changes associated with a Defect Record occurred in?
Does your tool support the keeping of a backlog and the ability to easily prioritize it?
Does your tool facilitate communication between Development, QA/QC and the Customers/Stakeholders?

Build Management

Does your tool or tools of choice support both Continuous Integration and Clean builds?
Does your tool integrate with the Version Control tool to at least the level of being able to use and create tags and branches (or comparable technologies)?
Does your tool integrate with the Defect Tracking tool to at least the level of being able to identify the first or last build a changed element was referenced in?
Does your tool support being able to execute and report on the results of automated unit tests?

If you answered "no" to one or more of the above questions, it's OK. It just means you are using some of the same tools as the rest of us are. It does not mean that you need to go on a tool search. What it does mean is that the CM Team, be it a group of people or just one, must work harder so that Development is not impeded. It is always a balancing act between tools and resources. The "better" the tools, the fewer resources are needed to get the job done on a day-to-day basis.

Pitfall Detected!
Fran: Why is he picking only one pitfall to talk about? We seem to run into them all the time?

Noid: Do you really want him to list a bunch of them? This is going to be long enough as it is!

Para: I think he is picking this one because it is one that often gets overlooked until it comes back to bite one in...

Fran: Be nice now!

Para: What I meant to say is that occurrences tend to happen separated by years, not weeks and that one often doesn't "see the forest for the trees."

What is the major pitfall CM can fall into in its attempt to facilitate Agile Development? The biggest one is automation and integration. Why would I say this when it is the normal goal of CM to do just this? Because the more custom automation and integration you do, the harder it is to upgrade or replace tools. Let's look at a case in point...

Example: Project A is using a variation on the SCRUM methodology and are using CVS and a home grown DIET (Defect, Issue and Enhancement Tracking) tool. One of its variants is the active use of up to 7 concurrent feature-set branches. With each feature set scoped to take between 3 and 5 weeks to complete, and the simple fact that there is no way to pre-plan the exact merge order, a lot of time is spent keeping thing working as they should to guarantee a stable base branch. Time was saved and human error reduced by using triggers to update the DIET system with a list changed files, their branch and revision number every time code is committed. The build system was also modified to update the appropriate DIET records when a successful build first includes an element.

Not only was the process made smoother, but time was saved by Development, QA/QC and CM in the process. Now let's switch from the home grown DIET tool to Bugzilla. Project A is now depending on features that are not supported by Bugzilla "out of the box," so it will have to be customized before it can be rolled out. All of the triggers will need to be modified and tested. All of the historical data from the original DIET tool will need to be imported into Bugzilla. The final integration will need to be tested and the various teams trained in the way things will now work.

Then we have the planned transition from CVS to Subversion. Conversion, trigger rework and training are once again necessary. As each tool is in turn upgraded or replaced, the same process will be necessary - and the drain on CM's time will be significant. Can we avoid this pitfall? Not really. The tools are often mandated or there are cost/political considerations, as well as inertia, that must be taken into account. What we can do is to occasionally step back and see if where we are going should be addressed by an early tool change and not more customization.

Survival in the Pit
I am assuming most of us, unless one is lucky enough to have started setting up CM specifically to support Agile Development, are trapped in the pit described above. So how can we survive while supporting Development? First is by providing them with at least an acceptable level of Version Control (and hopefully DIET as well) integration into the IDE they are using. Second is be minimizing the need for Developers to create and monitor branches and/or tags - CM will need to do it for them or automate it (there's that pitfall again). If we can provide a mechanism outside of their IDE to do Continuous Integration Builds, then we should do so. Regardless, periodic builds are required as well as timely notification of their success or failure (and who is most likely responsible). If no standard backend integration exists between the VC and DIET tools, then we will have to provide one, even if it means stepping deeper into the pit!

All of this takes time. The more time it takes us to do our jobs, the more Development is impacted. In the case of Project A, the only two times things really cause Development to slow down are when CVS branches are created and when merges occur. Everyone would love to switch to a different tool, but project schedule risks say wait.

Branch creation time can be reduced to a minimum by doing it directly on the CVS server. A daemon can be created to wait for branch creation requests containing the proposed branch name, a source point (branch and timestamp, branch tip or existing tag name), requestor, email notification list and a reason for the branch to be created. The daemon can then do all of the work and let everyone in the notification list know when the branch has been created.

Branch, or even worse, partial branch, merges back to a parent branch or to a sibling branch are more difficult to handle. Each VC tool has its own mechanism for doing merges and most of the IDEs contain one as well. Often they behave differently and since it is normally the responsibility of Development to actually perform the merge, the IDE will be their first choice. The best way of minimizing merge problems is to not change the same elements on multiple related branches. Often this means architecting the system so that elements are modular instead of monolithic. Another is trying to keep concurrent changes to individual elements to a minimum - don't change it both places!

Always remember, if you keep the number of branches to a minimum, you also keep the number of potential merges to a minimum.

Worrying About Change Sets
Should we as CM worry about Change Sets? Yes! If our tools support Change Set based development in addition to linear development, then by all means take advantage of it. If our tools do not, then use the check-in comments to identify each change set. You can enforce a DIET Record ID or a standard comment string, but you need to enforce something. It will be up to CM to tie all of the changes together into a Chance Set after the fact. As others are fond of pointing out, this is error prone and should be automated whenever possible. I agree, but not everyone has the luxury of doing so. The function still remains as a necessary one, regardless of the tools in use.

The other thing to keep in mind is that initial development is rapid, chaotic and random. At best, it is a pseudo-linear activity and a Change Set, if one existed, would most likely say something like, "Implement initial requirements and refine during discussions with the Customer." If Requirement Sets are in use, then once again the initial development is often too chaotic to be able to do one requirement at a time - even if it is a high level or roll-up requirement. Once a reasonable base has been established, however, both Change and Requirement Sets make sense and some method of tracing to them must be provided by CM.

Summary
In order to respond to the needs of an Agile Development Team, CM needs to be able to minimize the impact of the VC, DIET and Build tools. Keeping distractions to a minimum are just as important as keeping the use of the CM tools from slowing things down. When the tools become invasive, such as branch creation and especially merge times, try to work it into the process where Development is taking a deep breath to regroup. If they have a kick-off meeting, create any branches or other infrastructure while they are in the meeting. If they have a port-mortem meeting or some other form of a break at the end of a sprint, do the merge at that time. If the merge is simple, have one Developer do it. If it is complex, make it a team effort where each Developer takes a different part of the source tree.

Keep the pain and distraction down and you will be able to respond to requests instead of reacting to problems and exceptions. Be proactive in trying to determine where the next roadblock will be and eliminate it before "they" get there.

Ben Weatherall is currently based in Fort Worth, Texas where he practices Practical CM on a daily basis using a combination of CVS and custom tools to support a modified Agile-SCRUM development methodology. He is a member of IEEE, ASEE (Association of Software Engineering Excellence – The SEI’s Dallas based SPIN Affiliate), NTLUG (North Texas Linux Users Group), and PLUG (Phoenix Linux Users Group).

Set as favorite

Bookmark

Email this