Featured Whitepapers

Tool Spotlight

Upcoming & Recent Webcasts

Home

Agile SCM

Product and Project SCM

PDF

| Print |

E-mail

Written by Brad Appleton, Robert Cowham, and Steve Berczuk

Monday, 18 August 2008 11:25

Products and Projects - before we get into any SCM discussions, we need to classify these beasts - what are their distinguishing marks?!

The Lean Development Mailing Listhad a pertinent discussion on this. Mary Poppendieck commented:

Products are development efforts that are (usually) funded incrementally. They start with a concept and end with a product launch. Along the way, they are funded incrementally as they progress through stages such as feasibility, test market, commercialization, etc. [...] Products are also expected to have a long and useful life over which timeframe they are generally expected to undergo constant improvement. Projects, on the other hand, due to their contract origins, tend to be fully funded at the beginning of the project. This front-end loaded funding tends to drive the people providing the funds to ask for scope and schedule plans, which are then treated as commitments - after all - the funds are committed, so they want to know what they will get for the money. In addition, on-going change after the "end" of a project is generally regarded as bad.

Alan Shalloway contributed:
Lean extends some other agile methods (e.g., Scrum, XP) which are project centric. Products are about something the company offers to its customers (internal for IT products) while projects are about building the product. Thus good products will grow, mature and continue to add value to the customer in different ways and may spawn multiple projects.

Many companies have more waste in their project selection process than in their project building process. They will spend 1 - 2 years deciding whether to approve a project. Once they have selected a project to build, Scrum and XP can be put into place. However, what they really could use to help them is the understanding of waste that Lean provides. Waste includes delay, task-switching, hand- offs and other things which don't show up on an accountants balance sheet. Value stream maps are incredibly useful tools in this area.

Internal vs. External
Then we have the difference between internal and external products:

External products (or systems or applications) are things that we sell in order to generate revenue. The more customers we have to support (even on different versions/platforms), the more revenue we generate in return.

Internal products (or systems or applications) are pure operating cost and do not directly generate revenue. The more customers we have to support (especially on different versions/platforms) the more it costs us to spend time, money and effort on activities that are not revenue generating. The people and time we spend on them means less money spent on revenue-generating activities. We only do it because we think we are getting productivity and/or cost-savings (e.g., over not having the tool/applications) in return, and we still want to minimize it.

Internal apps also generally have a harder time justifying their existence/value to the business and their impact on the bottom-line (which might well be similar to justifying "CM" :-)

Also, there is a HUGE difference between delivering to customers who don't live in your own backyard and aren't part of the same organizational culture and politics. There are typically extra layers of "customer care/service" between development engineers and direct customers/users, and the internal customers also have a lot more visibility and transparency and "closeness" into what is going on.

Even more interesting is the notion of "consultants" who are hired strictly for the duration of a project (regardless of if it is internal or external) and how much that motivates and rewards them to think and act in the best interests of the "product" and not just of the one or two projects that they will be around for. It is easier to "come in", and hack something together fast, and get a group up and running and impress management and "ride off into the sunset". But the poor folks who are stuck with the internal support and maintenance of the ill-structured, un-factored, poor-quality code are the ones that the company may end up spending a LOT more money for in the long-run.

Just because we are picking on "consultants" here, doesn't mean that organisations doing it all themselves don't suffer from similar problems. We have seen short term "project thinking" within a large programme of internal development - the teams came together, delivered the projects and then were disbanded (due to internal charging models) without paying much attention to the long- term maintenance requirements.

Deployment Models
There are many possible models of deployment which can apply, particularly to products:

local deployment & installation, with a private/independent "copy" at each installed site
centralized, shared, monolithic, such as a travel service web site, or even something like Google Docs
decentralized and distributed.

There are others too. Each one of these may or may not lend itself well to a particular maintenance and support model:

supporting multiple historical versions
supporting only current version while working on the next one
supporting a product-family of variants of the same set of components

Which model you use depends primarily on upon total number and type of distinct customer-bases and the market (i.e., how likely it is you can expect them to be willing and able to upgrade to new major versions and how much they are willing to pay you to support older ones).

Architecture/SCM/Organization
With the above definition, we can review some of the SCM considerations as to the difference between Products and Projects. We have spoken often about how many release engineering issues can be addressed by a balance of Architecture, Organizational approach, and SCM Strategy.

Also using the above definitions, we can consider project SCM needs to be a subset of product SCM.

Product Development tends to have release lines - potentially multiple independent projects under development at the same time.

As Laura Wingerd discusses in Chapter 7 of her O'Reilly Book (Channeling the Flow of Change):

The Ace Engineering story illustrates typical problems of the software life cycle:

First, at anypoint in time there is likely to be more than one supported product version available to customers. Ace must be prepared to field customer calls, diagnose problems, and fix critical bugs in all of their currently supported versions.
Second, not all development tasks have the same urgency. Some are expected to yield results immediately while others are targeted for distantly future releases.
Third, software development is not entirely predictable; some projects go according to plan, others get mired in unforeseen difficulties.

Laura presents release and project branching patterns, and also discusses the Mainline branching model.

The ITIL Service View
Many organisations undertake a variety of internal projects which deliver new or enhanced applications or systems into live usage within the organisation. A typical new system (or Product), delivered as part of a project might follow a lifecycle and branching model shown below.

Figure 1 - This is for new development which starts within the project and is promoted through to live.

From an ITIL viewpoint, the Service Transition phase (e.g. into Live usage) is very important and one of the key measurements is the ongoing documentation and support processes to allow the system to be cost effectively maintained in the future. This recognises the reality that the initial development of most systems is 30% or less of the overall life time of a system.

As mentioned above, project teams who are measured only with regard to successful delivery of their project and ride off into the sunset looking for the next project, can leave considerable maintenance problems behind them. We have seen this plenty of times as a significant problem for organisations. And yet this model is still very common.

Enhancement Projects
Many internal projects are actually enhancing already existing products, so the branching patterns should really be represented as starting from existing state and delivering back a changed (enhanced, we hope) version:

Branching structure showing "promotion" of code and baseline points. Note that this version starts with original baseline being the current application in Live use.

Multiple Parallel Projects as Part of a Product or Release
Then we get to the case where there are potentially multiple projects or programmes, each of which is delivering particular enhancements. These multiple projects may overlap in some way and both be changing a particular System (system Y below) which will then need to be integrated.

Figure 3 - Lifecycle of multiple project development. This shows multiple projects or programmes in development with their code being integrated for a particular release (e.g. 6.10) which occurs during system test. Prior to this point the projects are relatively independent of each other. The individual solutions may be concurrently updating the same application (Y in the example above) - integration and merging of the changes is done at the point of Systems Integration and Test. The diagram omits the detail of the Stage 1 (dev) branches. Also omitted are changes which may need to flow back to the projects as a result of problems during integration.

A very common model here is to have regular release cycles (e.g. several per year), where multiple projects are integrated and delivered into live usage as part of a global release.

In the IT Process Institute Change Configuration and Release Performance Study the most important two findings are:

Release trumps change - rigorous release build, testing and rollback practices have broad impact on individual performance measures and overall performance. Change tracking and change oversight practices are necessary, but not sufficient, to achieve performance improvement on their own.
Process discipline matters - there are no change, configuration and release silver bullets. Building a process-focused culture and monitoring and responding to process exceptions predicts top-levels of performance more than many of the industry recognized best practices in these areas.

Indeed, the report overview (available for free if you register) says the most important practice is:

Release scheduling and rollback-In this set of practices, IT organizations develop and maintain a fine-tuned cycle of build and test, and then release only during maintenance windows with tested rollback plans. Data about the root causes of release exceptions is then fed back to systematically improve the process.

Agile Impact
To this point, we have been discussing general SCM implications more than Agile in particular, so now let us consider some Agile impacts.

Iterative Development - Releasing Frequently
Iterative development is not restricted to Agile methods, and yet is a core part of Agile development. If you release frequently, you get better at it - you look for automation and don't ignore the many little details that can otherwise trip you up.

TDD and Test Frameworks
This is becoming increasingly common practice and one of its main benefits is that a good set of automated tests provides a safety net for future changes. They also act as documentation for the system, and documentation that is more likely to be maintained along with the code.

There are still challenges in terms of finding the right level of automated test frameworks. Initial work focussed on unit test suites during development. More progress is being made in automating tests later on in the cycle - integration and acceptance tests, as well as unit tests. This can, of course, be particularly challenging for systems which have many external interfaces.

Conclusion
Projects and Products are different, and yet related beasts, and yet how important is the difference?. As Mary Poppendieck reminds us:

Besides this direct value of thinking products instead of projects, the concept also reminds us that our customers are not really interested in software anyway. The software we deliver is a piece of a larger product that delivers value to the customer. We must always remain value-centric to the customer.

From an SCM viewpoint, the most important relevant distinctions which drive the selection of appropriate SCM patterns are:

Internal vs external
Deployment model
Degree of parallel working involved
Number of supported releases

There are plenty of good patterns to use, and agile methods which emphasize continuous integration, regular builds and regular delivery of working systems which deliver value and can be very successful in either camp.

Brad Appleton is an enterprise SCM/ALM solution architect for a Fortune 100 technology company. He is co-author of Software Configuration Management Patterns: Effective Teamwork, Practical Integration, the "Agile SCM" column in CMCrossroads.com's CM Journal, and a former section editor for The C++ Report. Since 1987, Brad has extensive experience using, developing, and supporting SCM environments for teams of all shapes and sizes. He holds an M.S. in Software Engineering and a B.S. in Computer Science and Mathematics. You can reach Brad by email at brad@bradapp.net

Robert Cowham has been in software development for over 20 years in roles ranging from programming to project management. He continues his involvement in development projects but spends most of his time on SCM Consultancy and Training. He is the Chair of the Configuration Management Specialist Group of the British Computer Society, has a BSc in Computer Science from Edinburgh University and is a Chartered Engineer (CEng MBCS CITP). You can contact him at rc@vaccaperna.co.uk

Steve Berczuk is a Technical Lead for an Agile Software Development consulting company. He has been developing software applications since 1989, often as part of geographically distributed teams. In addition to developing software he helps teams use Software Configuration Management effectively in their development process. Steve is co-author of the book Software Configuration Management Patterns: Effective Teamwork, Practical Integration and a Certified ScrumMaster. He has an M.S. in Operations Research from Stanford University and an S.B. in Electrical Engineering from MIT. You can contact him at steve@berczuk.com

Set as favorite

Bookmark

Email this