Ecology verses Warehouse

In traditional reuse approaches, a reuse repository (Warehouse) is created and components are placed into the repository.  The theoretical hope is that other software development teams will reuse these stored components. Organizations adopting this approach see reuse primarily as an information retrieval task.  They focus their infrastructure efforts on search tools.  The process tends to break down whenever the warehouse does not have enough well-made, highly reusable components.   Development groups also face substantial challenges when adapting components from a warehouse, even when they are well-made.   A NASA Software Engineering Laboratory (SEL) study concluded that if you have to change more than 15 to 20 percent of a component to make it work in your program, it is more economical to build a new component.  Cultural challenges to software reuse also undermine Warehouse based reuse efforts.  These problems are inherent when reusable components are put into a repository to be picked up later for unattended reuse.  There is no good way to resolve these problems without changing and extending the approach.

A successful reuse program could be characterized by three attributes Access, Participation, and Collaboration.  The graphic below attempts to position the flow of thought for implementing a software reuse program in an enterprise. 

Management starts with a policy ("we want to promote reuse"), whereas developers start with collaboration ("who's got something I can use?"). Management may impede developer access and participation in their reuse efforts by instilling barriers such as approval processes and expensive tools.  Management's efforts reflect their desire to standardize reuse and control development.  Applying only the top-down drive often yields reuse program banners in the hallway with little actual reuse. Conversely, applying only bottom-up drive often yields of multiple reuse repositories in various formats with little actual reuse.  Solutions that address both directions of drive (top-down & bottom-up) have the greatest probability of success. 

The traditional Warehouse approach takes the easy way out to some degree and focuses on the tool issue first (access).  Going with a "tools first" solution leads to the warehouse equivalent of the "kitchen junk drawer" of software components.  Worse, it ignores the Achilles heal of all such approaches, that is, it ignores cultural change and impact.   The cultural aspects of participation and collaboration are more significant to long-term success, yet more difficult to implement. 

Such cultural barriers include:

• Managers adopting a "not-invented-here" policy to reusable components from other divisions.
• Divisions or developers refusing to allow components to be released for reuse, afraid that they will be "stuck" with maintenance issues that will seriously detract from their ongoing "real" work.
• Central IT authority, trying to ensure the quality of the reusable components, mandating a tedious approval process that inhibits contributions to the reuse program.

It seems that an appropriate approach to answering, "How to implement as successful reuse program?" is to reuse key practices from other organizations that have done better at reuse.  If one were to take this approach and go looking for successful reuse programs, you would be naturally drawn to open source projects, such as Apache (http://www.apache.org/).  Open source projects demonstrate that reuse can succeed in what appears to be a vibrant Ecology around software development, but it takes careful observation and thought to replicate that success.

Some areas where you can see reuse in process on Apache projects are:

• Apache Portable Runtime Project - (http://apr.apache.org/ )
• Jakarta  - (http://jakarta.apache.org/commons/index.html)
• XML  - (http://xml.apache.org/commons/)

Open source programs are generally characterized by community, open communication, transparency, access to source code, the ability to fork, and peer review.  The empirical evidence suggests that these characteristics are central to promoting successful reuse programs.  The lower barriers to participation and communication in open source development dynamics means that people are not only positively inclined to factor out common code and related functionality into separable components, but that they are more likely to use pre-existing components than write their own.

As typified by open source, the ecology approach to reuse is seen as an extended collaborative between the producers and consumers of code and other artifacts.  The key to reuse is not the fact that source code is available, but rather that potential consumers have broad visibility into the development process and the ability to affect it by submitting requests, bug reports, and patches.

Other cultural impacts promoting reuse are:

• Constructive feedback and incremental success set a tone of openness and collaboration among projects that reuse each other's code.  This encourages the development of open architectures within the enterprise, which ultimately creates more opportunities for reuse.
• Easy access to current and former users of the components promotes trust and support.
• Component developers - and their managers - find that a reuse community helps balance support requirements for reuse, increasing their comfort in making reusable components.
• Business users are more involved in the reuse process facilitating planning for reuse.
• Reuse access to knowledge assets such as mail-lists, discussion forums, issue tracking, and file sharing is critical to knocking down the "not invented here" aspects of traditional reuse programs.

Some Recommendations

Creating a truly successful reuse program is not easy.  Many firms have invested in reuse, but are still having problems getting the expected level of results.  Based upon this analysis, the following recommendations should be considered:

1. Focus on culture first.  This is the harder part of the problem and also the one that determines the ultimate reach and sustained success of the reuse program.

2. Think about all aspects of reuse ecology - A simple "Return on Reuse" framework to help you think about implementing a successful reuse initiative is presented in the following graphic.  The graphic shows that the potential value from reuse is typically greater than the value a company actually receives. There are two main gaps: communication (finding and using what is reusable) and maintenance (ensuring what is reusable today remains reusable tomorrow).  The Ecology approach addresses both gaps more comprehensively than the Warehouse approach, yielding both a higher return and a greater chance of sustained success.

3. Tools - Tools are an important part of the solution, but the focus should be on collaboration tools rather that database or information retrieval tools.  The Apache project achieves its collaborative development environment using web-based tools for code versioning (Concurrent Versioning System - CVS), issue tracking, mail list management, etc.  The collaboration tools help:

1. Better understand the reuse requirements by both producer and consumer
2. Share the support burden of reuse components
3. Jointly manage reuse expectation and track progress
4. Refine goals for streamlining reuse practices and solicit commitment
5. Increase transparency and encourage open communication
6.  Retain knowledge assets along with software assets
   
   
4. Don't limit reuse to the code or executables.  All development artifacts can be reused and many have higher value than the code itself.  Code tends to have limited reuse potential because it does one specific thing and cannot be reused for different purpose or in a different technology context.  In contrast, design patterns, use cases, design review checklists, etc. are not complete solutions, but they are partial solutions to a wider range of problems.  Some of the most valuable and reusable knowledge is simply "Why was this decision made?" and "Who can I ask about topic X?"
   
   
5.  Factor in Variability - There is an unpredictable and increasing amount of variability across software business systems over time.  Starting your program out with the broad based intention to be "The Enterprise Reuse Library" might be a fatal flaw.  Consider product line approaches instead.  In the product line approach, the organization realizes that it will build a series of related products, as part of a product line over a period of several years.  The goal then shifts to balancing productivity of the current project with the overall efficiency of producing the entire product line.  Unlike the reuse Warehouse repository approach, product line component developers have a fair idea of who will reuse the components and what kinds of adaptations may needed over a defined timeline.

Acknowledgements

I would like to acknowledge the contributions to this article from my colleagues and friends, including Mark Murphy, Dr. Jason Robbins, and Brian Behlendorf.

Trackback(0)

TrackBack URI for this entry

Comments (0)

Subscribe to this comment's feed

Write comment