Making predictions is fraught with danger and the last thing forecasters need is somebody to actually compare their predictions with reality. In SCM there were key attempts to project the future of the discipline and the tools that support it. As a dozen years have elapsed since these predictions were made, it is time to assess their accuracy and determine where SCM has been and where it still needs to go.

Of course, we all have opinions on where SCM may be heading, but rarely do we get around to writing them down or accessing their accuracy at some point in the future. So I shall review the early work on SCM conducted at the Software Engineering Institute (SEI) with due respect. Twelve years later these papers present a future we have in many ways already lived, but the aim is not to use twenty-twenty hindsight to be critical of any deficiencies. SCM continues its evolution and some challenges identified in those early days still have to be confronted, let alone overcome. So beyond what came to pass and what did not, we may want to look back at these early works to determine what unanswered questions remain and if any unrealized futures are worthy of our renewed commitment.

The early nineties was a time when there was significant interest in SCM. The acceptance of commercial SCM solutions was growing and most of today's tools stem from research and development undertaken around that critical period. 

The problems with software development were being recognized and addressed. And best of all organizations were for the first time investing heavily in tools to support their development projects. We were still in the midst of the Computer Aided Software Engineering (CASE) revolution and there were great expectations that software itself could come to the aid of the so-called "software crisis". The industry was relatively pleased with itself and numerous techniques and methodologies abounded. It was thought to be simply a matter of adopting some combination of these to solve the problems the software industry were experiencing.

The Software Engineering Institute (SEI) was becoming influential in 1990 when it undertook a significant study of software engineering environments and SCM, even as it was promoting the Software Capability Maturity Model (CMM). In this paper we will review two key papers of that period and attempt to determine the accuracy of their predictions.

Susan Dart at the SEI undertook a survey of SCM concepts [1] to be found in the different tools and environments available at the time. The "spectrum of CM concepts" she produced captured the different approaches that the various vendors had taken to tackling SCM issues and provided a baseline (sorry, couldn't resist) by which to compare and contrast the different market offerings.

Although Dart's brief section on the future of CM systems poses many questions about the direction of the field, we can discern a number of key predictions in her summary. Therefore the first set of very high-level predictions attributable to Dart and the SEI team are as follows:

• That the CM concepts spectrum would evolve into a fundamental set of CM services that all CM system will eventually attain.
• That there are political and technical issues and concerns that will affect the future of SCM tools. Today we would more appropriately interpret the political issues as the market and commercial forces that shape the SCM landscape. A key example Dart gives of a political issue is whether CASE environments include or exclude CM capabilities - which translates to whether independent CM tools can exist separately from the environment vendors. She postulates that co-existence might be the result of "some kind of standardization would probably be needed for CM systems in relation to environments, or vice versa."
• A number of examples of technical issues are listed relating to the implementation of SCM solutions, including whether OO databases might provide the appropriate technology on which to base a CM system? And in what "layer" of the environment's architecture does CM best fit? Again the question is raised as to whether there are standard CM primitives that could be used in any environment to support all CM functionality? Is there a unified CM model?

In a subsequent paper [2] entitled "The Past, Present, and Future of Configuration Management" Dart provided a more detailed list of future challenges (which I'll loosely term predictions) expanding upon technical issues and anticipating some new directions for CM in the future.

Dart states that some CM tool vendors recognize that the CM solution for customers is only about 10% technological - "the rest entails improving management, process and user training." New CM functionality needed to improve on these figures includes: 

• Interoperability between CM systems. This is identified as necessary for very large (defense?) customers so they were not locked into a tool and instead could import data between tools.
• Turning off/on various features in CM systems. This customization could provide variable levels of control to suite different projects or phase of the life cycle, etc.
• Global perspective of CM repositories, that provides an (enterprise) overview of all the different repositories created in an organization and allowed users to work across or navigate between them.
• Distributed CM, where CM could be accessed across the network and heterogeneous environments.
• Preventative maintenance support is described as allowing users to restructure their software and involves reverse engineering or re-engineering of its design. It takes CM from being a separate, controlling capability to one of design support with a more intimate knowledge of the configuration being managed.
• Process oriented capabilities are predicted that better support CM processes that first need to be better understood and defined. Without too great a stretch these can be extended and generalized as the development or lifecycle processes that we know of today.
• Managerial issues include the (now) classic requirement for management buy-in for CM. Dart says it is necessary to give management an understanding of the complexity of the (SCM) solution and hence the costs and trade-offs. This is expected to enable a better commitment of resources including tools, people and funds to evaluate, transition, customize and integrate the SCM solution into the organization's environment.
• Political issues cover government level policies and refer to the upcoming requirement of CMM level 3 accreditation for future government contracts, in which CM is a level 2 key process area (KPA).
• Standardization and the continued of work on environments and CM is predicted to result in a model for addressing tool integration and provide a wider conceptual CM framework that can encompass all domains including Computer Aided Design (CAD). A "CM Services Model" with a plug in/plug out CM capabilities is proposed to take account of the "marketplace's need to apportion and distribute functionality" - i.e. tools provide parts of an integrated CM solution. Apparently initial work had commenced and identified fifty services that a customer could pick from and tailor to suit their particular requirements and processes.

In concluding Dart asks the question: "What is the CM boundary? It is not clear what the dividing line is between what is CM and what is not CM."  The paper closes with the statement that:  "CM is a keystone to the software engineering problem and it should not be viewed in isolation from other software engineering problems and solutions". 

Accuracy of Predictions

So how have these predictions fared in reality? We might anticipate that twelve-plus years would be enough to see the realization, or at least the emergence, of the proposed concepts.

In order to review the current status of the above predictions they have been consolidated in the table below. To remove possible repetition in accessing technical issues, standardization issues have been grouped into one point, namely the identification of standardized CM services and their interfaces. This is closely related to another technical point: the emergence of a consistent CM tool architecture that flexibly supports the (proposed) common CM primitives - even beyond the software domain.

Each of the resulting eleven "predictions" has been assigned a score between 1 and 5 to represent the accuracy of the prediction to date - remembering that some of them may simply require more time to play out. I have used a key where:

1 means the issue is still relevant; 2 means it is recognized; 3 that something is being/was done about it; 4 suggests that the issue is well in hand today and a 5 would mean that the prediction was largely accurate and the industry acted to solve the issue. 

Explanation and justification of my assessment is provided in the section below and it must be recognized that this is a personal appraisal of the predictions. I do not claim any rigor here although I have attempted to substantiate my views where possible. The good thing about a community like CM Crossroads is that you do not have to simply accept my viewpoint. You are able to present your own opinions on these predictions and I'd encourage your feedback and thoughts on this assessment!

Review of Current Status 

1.  Commonality of CM Tool Concepts The spectrum of different CM concepts has indeed been adopted to a significant degree by the majority of SCM tool vendors. Concepts like repository, workspace, change request, lifecycle model, etc that were originally introduced by individual vendors are now part of the standard CM lexicon. Tools largely share a terminology and are evolving towards a common set of capabilities as evidenced by the increasing adoption of the activity-centric model based upon change-sets [3]. With the increasing homogeneity of tool capabilities I gave this prediction a mark of 4, recognizing that there remains some unique capabilities that tools have as a result of their differing architectures and market focus. 
   
2. Standardized of CM Services Tools supporting similar features and using common terminology is one thing but Dart made a prediction that these would become standardized interfaces between tools and environments. The identification of the standard CM primitives and the definition of an interface to access them have been realized in part. Microsoft, as the environment guerilla, introduced the Source Code Control Interface (SCCI) a number of years ago, providing a de facto standard that allowed conforming SCM tools to be plugged into its development environments. The interface was however quite rudimentary and "fluid", never being endorsed by any independent body it became progressively more fuzzy as Microsoft changed the environment's architecture. Independent SCM vendors had to play catch-up as they attempted to integrate their tools and then maintain these integrations as the environment evolved - no easy task and something that constantly caused customers grief as they attempted to upgrade.  These issues have provided the impetus for the mergers between CM and environment vendors as shown by the IBM-Rational and Borland-Starbase unions. But this does little for the wider industry and while the CM services remain hidden and proprietary they limit SCM tool choice.  But partial standardization did happen, even though it was not in the "industry endorsed" manner that may have been envisaged so the prediction warrants a 3. The fact is that such standardization is more feasible today with current service-oriented architectures; it would simply take the effort to define it - maybe new tool architectures like Eclipse (http://www.eclipse.org ) will take on these challenges and succeed?  
   
3. CM Tool Architectures CM tool architectures have remained a proprietary issue and while there is some research interest in future possibilities in this area [4] the likelihood of these being adopted by vendors is uncertain. In fact it could be said that today's SCM tools have monolithic architectures that have not significantly changed since the tools entered the market. Architectures quickly become "legacy" and any changes are costly so the goal of a "common-repository" that can cater to software's needs as well as other domains like CAD and Product Data Management (PDM) which shares many similarities with SCM will take significant investment. There have been many attempts at creating such a common repository going back to the Portable Common Tool Environment (PCTE) proposal (http://www.omg.org/docs/1993/93-12-13.pdf ) during the heyday of CASE and Microsoft's more recent attempt at a repository (http://www.informationweek.com/584/84iutex.htm ) in the late 90's that morphed into a data warehousing effort before it seemingly vanished. Hence this prediction remains, tantalizingly in the future. But one might be surprised by an announcement someday soon about a CM vendor making the move - so I rate this prediction a 3 for nearly achieving reality. 
   
4. CM Interoperability The lack of interoperability between CM repositories has always been a bugbear for large customers that had different tools used across the enterprise. This prevents information being easily accessed across the different tools and tends to lock projects into a silo mentality. Migration from one tool to another is also a major barrier to upgrading CM tools and so it is usually left to the vendor to supply support utilities to aid in the migration to (definitely not from) their repository. The only initiative attempting to support CM interoperability was undertaken jointly by the Government Electronics & Information Technology Association (GEIA) that is de facto guardian of military style standards and resulted in EIA-386 (see http://www.geia.org ). This standard for Configuration Management Data Exchange and Interoperability still has a long way to go to get significant tool vendors support, unless of course a major defense program comes along with a huge (CM) budget. We will probably have to wait longer for advances in this area since the vendors themselves have little incentive to provide such interoperability and would rather propose that their solution is adopted as an enterprise standard. Therefore I have rated this prediction just a 2 for while the agenda is recognized, there is little or no progress made by CM tool vendors. 
   
5. Customizability of CM Tools The concept of a customizable CM solution that provides for the scalable and flexible incorporation of features and the modification of the tool behavior, has been a vision shared by quite a number of CM researchers [3,4]. In reality we have only seen attempts at marketing of a range of CM products such as the "lite" version of ClearCase by IBM/Rational, and Merant's purchase of Dimensions as an upgrade path for its original PVCS user base. There has been more technical progress when considering the "process" component of SCM tools discussed below. The defunct Rational ClearGuide came closest by allowing flexible creation of process modules, while Dimensions, TrueChange and Telelogic's SYNERGY have always allowed the customization of their out-of-the-box process models. The days of real CM scalability and flexibility of a single CM solution however has to await the realization of the CM services model described above. So for now I rated this prediction only a 2 (given that process support is specifically covered below). 
   
6. Enterprise CM An enterprise perspective of CM repositories remains a weak aspect of SCM tools that have not abstracted their system model much beyond that of a project. Component based development (CBD) can be addressed in some tools by applying hierarchical workspace (project) constructs but there is little support for the navigation and management of the higher-level components or sub-systems. Certainly there are other capabilities that CM tools could provide to support the entire system portfolio and navigate between repositories, but the fact is that enterprise customers continue to struggle with lower-level CM issues and so the need for enterprise visibility and navigation remains very poorly addressed by vendors. I give this a score of 2 because although vendors recognize the need it has not been a priority and there are few such capabilities in evidence. 
   
7. Distributed CM The need to support distributed teams has been a major success of CM tools, providing a level of coordination and control that has facilitated the distribution of development projects. While the original prediction included CM operations across heterogeneous platforms many international organizations simply rely on the distribution and control of workspaces across a wide area network. These capabilities have allowed software development to become (dare I say it) the international effort it is today. Whether we like it or not, SCM can provide the infrastructure to outsource development activities around the globe and still retain control over the artifacts. Without hesitation I give this prediction a big 5 for accuracy - even if the applications of these capabilities were not foreseen and standardization has largely eliminated the platform wars within organizations. 
   
8. Design Support Taking CM from a generic controlling function to a domain (and potentially language) specific capability has not been evident in today's crop of SCM tools. In fact, providing such capabilities is more likely to come from the environment where the knowledge of application specific relationships would allow CM to be applied at a level beyond the file-level as it currently is. The predicted reengineering capabilities may be seen as CM added to the refactoring support that some environments now provide. While this currently involves coding patterns that are too fine-grained, CM's contribution could become powerful when higher level modeling objects and components become involved in the refactoring. But that remains someway in the future, although it is a future also predicted by researchers [3]. For now I rate this prediction only 1, for reality has progressed little beyond the situation in the early 90's. 
   
9. Process Support While the prediction limited itself to CM processes there is little doubt that  CM tools have become the infrastructure for the entire development process. From the manner in which development task are identified and assigned, to the planning and baselining of a release, CM tools can implement what would otherwise be a paper-based process. The ability to support and automate development patterns through process workflows have made CM the core-component of any suite that purports to support the software lifecycle. Indeed, some researchers believe that CM environments are really process-centered environments [3]. So again we have a prediction that was right on the money and deserves a 4 rating since there is work remaining to be done in making process definitions precise, modular and transferable between organizations. 
   
10. Managerial Awareness The simple question is: Do managers have a better appreciation of CM now than in 1990? Dart talks bout getting managers to realize the complexity of getting CM adopted by an organization - but it was shown to be the wrong focus. Vendors do not like to paint things they sell as complicated or with long lead-times before any real pay-off. Hence (and I know this for a fact), after some initial attempts at pumping out this message, it was rejected by vendors so today you are more likely to be told just how easy and quick it will be to get real benefits from a CM offering. This may be good sales technique, but it is very dangerous. What we need is for managers to understand the enormous benefits their organization can derive from CM and accept that they have to make an investment to achieve success. But while that message comes from a vendor it will lack credibility and is unlikely to be believed - you would at least want to be cautious and verify any claims made. So, while I believe managers today do have a better appreciation of CM, they get confusing messages that do not serve our discipline well. More needs to be done - and some past work undone - so I rank the present state of this challenge a 2.
    
11. Political Endorsement Being a part of SEI, Dart had a good idea of what was coming as a result of the SW-CMM. So it is no surprise that government did set a precedent by requiring that its suppliers to be Level 3, CMM. Yet with all the KPA's in the CMM (not to mention the CMMI) CM may not get the attention it deserves. But since CM is at least recognized as a Level 2 KPA I rank this prediction a 4 - because I think CM needs yet more endorsements!

Conclusions

So do the predictions of the early 1990's represent a future past? Numerically the average score is just shy of a three, so I guess that makes them were more right than wrong. There are no zeroes so the predictions still have some relevance today, and I did get generous and give one five. But in reality there remains a lot to do.

More importantly, knowing what we do now, are these challenges identified in the early 90's still the important ones for the future? What issues today hold CM back and should we address? Contemplating these questions leads me to a couple of predictions, and one wish, of my own.

Software complexity is significantly greater today and CM itself must be reinvented to cope with the challenges of increasingly reusable, component-based architectures. Therefore I can see the following trends over the next decade:

• The continued abstraction of SCM concepts towards business and project management constructs, moving from activities to releases allowing improved management of the development lifecycle (see my previous CM Crossroads article on the convergence of PM and CM at http://www.cmcrossroads.com/newsletter/articles/ltmay03.html), and;
• The integration of software and system-level CM concepts into a comprehensive framework that enables the management of configurations at different levels of granularity.

My one wish is for a strong and focused industry-body that represents the interests of CM and can work towards the development of vendor-neutral standards and help to substantiate the real benefits of our discipline. 

Being the time for New Year's resolutions, if any of you wish to take up the latter challenge with me, we may actually be able to bring some of the, sadly, outstanding predictions to some resolution.

[1] Susan Dart: Spectrum of Functionality in Configuration Management Systems, SEI Report Number: CMU/SEI-90-TR-11 ESD-90-TR-212 http://www.sei.cmu.edu/legacy/scm/abstracts/abscm_spectrum_of_func_TR11_90.html  

[2] Susan Dart:  The Past, Present, and Future of Configuration Management, SEI Report Number: CMU/SEI-92-TR-8 ESC-TR-92--8 http://www.sei.cmu.edu/legacy/scm/abstracts/abscm_past_pres_future_TR08_92.html  

[3] Jacky Estublier: Software Configuration management: A Roadmap, International Conference of Software Engineering ICSE-2000

[4] André van der Hoek, Dennis Heimbigner, and Alexander L. Wolf: Does Configuration Management Research Have a Future? Proceedings of the 5th International Software Configuration Management Workshop, Seattle, USA, April 1995.

Trackback(0)

TrackBack URI for this entry

Comments (0)

Subscribe to this comment's feed

Write comment