Previous: CM Procurement Next: Standards, Process, Resources

---

Configuration Management Body of Knowledge

Chapter 10: Metrics for Configuration Management

Table of Contents -

1415

Overview

One of the major benefits coming from CM's work in monitoring processes and procedures and keeping things automated are metrics. Metrics, if used by management, can play a significant part in planning and scheduling. They can also provide a tool by which management can verify their 'perceptions' of a problem. Metrics can also alert management to potential trends and problems.

10.1 TYPES OF METRICS

The metrics listed here are but a few of the metrics that can be utilized by Configuration Management Teams.

• Life Cycle Promotions illustrate the number of promotions performed by CM on a monthly basis. This is an indicator of the work load. The quantity of life cycle promotions also serves as an indicator of merge code activity where more than one release is being worked by the team. Another variation of this particular metric is the number of promotions made per day on average in a month.

-- SmKershaw^? - 25 Feb 2003

• Average Process Time Per Month Metrics can also be helpful in monitoring work flow effectiveness. Graphing the times, for instance, that it takes to completely process a problem report from the time CM begins to track it to the time it is corrected and put into production and be of great value in providing clues concerning the overall process. The diagram below shows actual data from a real project. The X axis contains the month/year, and the Y axis shows the average number of days it took to close a Problem Report opened in that month.

-- SmKershaw^? - 24 Feb 2003

• Configuration Item Change Activity Illustrates the ratio of the number of changes per configuration item on a monthly basis. On the 'Y' axis, plot the ratio (Changes/CI). The 'X' axis shows the months. This can provide significant insight on the quantity of repeat 'visits' a developer makes against a particular CI. The greater the frequency, the greater the problem.

-- SmKershaw^? - 24 Feb 2003

• Database Changes Per Release is normally associated with Software Configuration efforts. The 'Y' axis shows the quantity of data base changes and the 'X' axis has a column for each release. Such diagrams as the one below make it readily known which releases have had the most impact on the overall product from a database change perspective. It also provides some clue as to the quality (or lack of quality) of the initial planning efforts prior to the initial release being put into production.

-- SmKershaw^? - 24 Feb 2003

• A Release Status Report should report the date that the first change was made, when the last one was made, the number of days open, the number of configuration items changed, the number of total changes, the change/CI ratio, the number of Problem reports Issued, The Problem Report/CI ratio, the Days to Complete a Problem Report, and the current status for each release.

-- SmKershaw^? - 16 Feb 2003

• Frequently Changing Files Report is a software CM specific report that can be used to analyze files that are being potentially too frequently. Such a list can provide unique insight to both CM, PM, and QA groups in that these files could then be targetted for special investigations. Such questions as these need to be answered for each file on such a list:
  • Why are these changing so often?
  • Is it because requirements are not clear?
  • Is it because testing is inaccurate?
  • Is it a training issue?
  • Is it an overall design problem?

-- SmKershaw^? - 24 Feb 2003

• Monthly Rework Metrics can be used to track the success or failure of methods applied to the project in effort to control rework. The diagram below shows the rework value from a particular project over a part of the life of that project. One has to ask:
  • Why is the trend going up so fast?
  • What is driving this so that it looks so out of control?
  • What can be done to slow the trend and reverse it?

-- SmKershaw^? - 24 Feb 2003

• Change Completion Metrics can be used to analyze handling of changes. Actual work expended can be plotted against severity, and should show a direct relationship. Calendar time to completion vs criticality should show an inverse relationship. Violation of either of these relationships may indicate incorrect classification of change requests, or non-optimal scheduling of time. Estimated time vs actual time can be used both by developers as a tool to improve their estimation capability (as in SEI's Personal Software Process(SM)), or by project managers to evaluate the accuracy of developer estimates when planning and costing future efforts.

-- CarildaAThomas^? - 18 Feb 2003

10.2 METRICS COLLECTION

The collection of data for metrics is driven by three factors. First, what information does project management require to make proper decisions in a timely manner. Second, what information does the CM Team require to monitor the work flow, processes, quality etc of their own efforts. The third factor driving metric collection is the overall direction and motivation to achieve such status as CMM Level 4.

Data for metrics comes from the tools at hand. Where tools do not provide the required information, CM is tasked with making the acquisition and retaining the data. Regardless of the nature of the data obtained, it is crucial to be conservative in applying 'meaning' to any and all metrics generated. For the saying is quite true: "Liars figure and figures lie."

It is also very beneficial to have, as part of your staff - at least part time, a statistician that can aid in more complicated metric research and development in the realm of multiple correlation analysis.

-- SmKershaw^? - 24 Feb 2003

(split and edited) -- CarildaAThomas^? - 18 Feb 2003

---

Previous: CM Procurement Next: Standards, Process, Resources

---