The old adage goes "you cannot manage what you cannot measure".
True enough, but where do you start in implementing a measurement program? What
information is really important? And, how do you decide what to measure and
what will matter to others?
No one in this audience will disagree that metrics and
ongoing analysis is a crucial element in effective software project management.
However, metrics programs do have their
challenges. Let's admit it, metrics can be difficult, time-consuming and
challenging to derive. Development managers will relate tales of woe of their
teams spinning through cycles to pull out metrics lacking in relevance to the
business. This approach to measurement is a thankless exercise that can
paralyze the entire project team.
But, there is a light at the end of the tunnel. Experts recognize
and are beginning to agree on the key components that make up an efficient and
effective measurement program. And there are methodologies available in the
market today aimed at connecting metrics directly to business benefit.
This rest of this article looks at the vital role metrics play
in effective software project management; gives some guidance on metrics that
experts in the industry consider to be fundamental to good software project
management; and touches upon a industry-accepted methodology called GQM, that can be a valuable approach in helping
project and development managers define and align measurement to business
goals.
Three
Application Development Trends
Before we talk solution, let's take a look at some emerging
trends in application development. There are three main trends that I've
noticed. The first is that project-based development is moving to a
process-based approach where automated processes and workflows are required to
be more fluid and must accommodate ongoing adjustments and changes . The
emergence of agile methodologies, compressed delivery schedules, and the
recognition that software evolves on a continuous basis are principal drivers
here.
The next and related trend is the move from waterfall
development to a more iterative development style. This methodology evolution is
really in response to the speed of business transactions today. No organization can wait a year for a
software project completion. Innovation
must be rapid and ongoing.
Finally measurement
within development organizations must make the move from ad hoc management
reporting to real-time. The logic is there; if organizations are going to move
faster and respond more quickly to business change - they cannot afford to wait
on information.
All of these trends indicate a dynamic shift in development
towards greater speed, collaboration and agility (in the broader, business
sense of the term).
Defining
Metrics
So let's look at today's rapid application development
environment and acknowledge that real time metrics should play a central role
in keeping projects on course and making incremental efficiency improvements. For
these purposes, let's define a metric in software development as the
measurement of a particular characteristic of a program's, or processes,
performance or efficiency. A metric is sometimes used directly and sometimes as
an element in an algorithm calculating a more complex metric. For instance, in
programming, a benchmark or milestone will include, or be driven from, metrics.
When you are talking measurement, it is important to realize
that context gives the data meaning. It is not enough to know what a metric is,
it is important to also know why the data is being collected in the first place
and how that data, and the metric derived from it, will be used.
In addition, software measurement programs should allow
project managers to quantify schedule, work effort, cost and quality
performance in a way that is accepted and understood by the business. For many
reasons, software projects frequently run over schedule and quality problems
persist often derailing projects and some of this is attributable to a
miscommunication or lack of appropriate and timely measures.
We attribute this challenge to the fact that most
measurement programs employed for software project management rely on manual
collection and manual analysis of data, or at the very least don't specify or
point out that automated data collection would be beneficial. Today's fast paced business reality is all
about having real-time, on-demand data representing actual activities instead
of rough estimates and manual analysis. Manual measurement is just too slow. To sum it all up - metrics give software
managers control and visibility and the more precise and real-time the metrics,
the better the decision making capability of those managers and hence the
better the project execution.
Four
Benefits of Measurement
Overall, measurement provides four fundamental benefits - to
characterize, to evaluate, to predict and to improve.
- To
characterize - Metrics allow decision-makers to better understand
processes, resources and project/work status. They also establish baseline
measures for future comparisons.
- To
Evaluate - Metrics enable project managers to see how they are performing
against plan or quality goals. The data shows when and where projects and
processes drift from plan and allow for course corrections.
- To
Predict - Metrics can help predict future performance by providing
understanding of the relationships between processes and projects. Through
reliable data, software development managers can better estimate costs,
schedules and work effort.
- To
improve - A thorough metrics program isolates roadblocks, inefficiencies
and root causes in order to improve efficiency, productivity and software
quality. Once metrics have been established, they can be used to
demonstrate alignment with business goals and can assist in communication
with senior executives.
These four benefits result in increased project efficiency,
greater collaboration, enhanced productivity and ultimately better use of
resources (time, money and people).
Aligning
IT to the Business Goals with Goal Question Metrics (GQM)
One way of establishing an overall metrics program is to
begin with the business goals and demonstrate alignment of the software
development objectives within that context. This is where a good process
framework and measurement methodology comes into play. Take the Goal Question
Metrics approach as an example. In this methodology, developed by Victor Basili,
overall business goals are defined first. The criteria of success are then
identified as questions - what do we want to achieve to satisfy our business
goals, what questions will help us plan and manage progress toward our goals. Finally,
the key criteria are broken down into an array of measures that answer those
questions. GQM is all about knowing what you want to find out, identifying the
right data to collect and why, and how that data is used to show progress
towards the overall business goals.
Regardless of the methodology used, defining metrics that
are relevant to the business needs to involve the business. By business I mean
the audience of the metrics you are collecting. This may be the CIO/CTO and LOB
owners or may be the team lead and their team members. Metrics defined in a
vacuum will not be believed by the business so involve them in their definition.
The
Role of Dashboards
When collecting data, however, there may be some pitfalls
that can hamper your efforts. Manually collecting data can increase the
overhead burden, be incomplete and subject to bias. Is the act of collecting
the data outweighing the value of the data itself? It is too time consuming to
collect metrics to promote change. You end up in a vicious circle when
implementing a manual collection paradigm.
Automating data collection to extract information is the key
to making a strong connection to aligning work efforts to meeting business
goals. A measurement system has to be analytic and able to be stored and show
trending over time. It must not be a burden to the workers who input the data
but the input of the data should be a natural by product of their day to day
responsibilities.
Once the data is in the system, how is it communicated and
presented to the business? This is where dashboards come into play. They are a
great way to consolidate the data and make it accessible, visible and meaningful
to everyone in the business. Further, if your dashboards are well designed and
active, not simple a static snapshot, individuals will be able to use them as
management tools rather than simply reports of data. Dashboards with drill-down
capabilities that derive data from a live and connected ALM system enable you
to manage what needs managing and avoid costly time delays and subjectivity.
Five
Key Metrics to consider
Typically, there are five essential metrics or more
specifically metric categories that software development should look at:
- Team
Efficiency
This metrics takes a
look at outputs/time. To maximize the value of this metric, it needs to be
trended over time and projects. Team efficiency metrics can give software
project managers insight into how time is invested and ensure that the efforts
are aligned with business goals and how those efforts can be improved.
- Process
Efficiency
Process efficiency
metrics track and report on the health of internal processes as they relate to maturity
goals (CMMI, Six Sigma, ISO etc). When working towards process maturity and
achieving various levels of process excellence, software development managers
gain knowledge about where process improvements can unlock efficiency and drive
increased productivity.
- Project
Efficiency
These metrics
demonstrate how well the project performed against the project objectives.
Access to data around these project metrics can save money, time and effort and
increase output considerably.
- Quality
Quality metrics
assess whether the project meets the standards set by the organization.
Software managers can include a pure measure of defects/ unit of output
(function, LOC, etc) by project, by phase and can also include quality
assurance metrics. Armed with this data, management can review overall
processes and remove roadblocks to enable higher quality outcomes such as a
lower number of defects per line of code.
- Value
and Effectiveness
This set of metrics
evaluates how well the project and its output align to the organization's strategic
objectives. Software development managers can engage senior management in
prioritizing projects and can align resources to the overall business goals.
Some final words of advice -
In today's
businesses, increasing competitiveness for larger wallet share means that every
department within an organization has to demonstrate efficiency, productivity
and the best use of resources. To demonstrate the value of IT within an
organization, software development managers must be able to provide meaningful measures
of the effectiveness of their efforts and demonstrate the value of every budget
dollar. With the alignment of a measurement program and metrics to overall
business goals, development and project managers are better equipped to demonstrate
their effectiveness and value at every level. And dashboards powered by automated
real-time metrics are the icing on the cake - they give managers a never before
seen insight into the workings of a project.
Increased real time visibility and
metrics allow you to make fact-based decisions to improve process, streamline
workflow and tune team performance for higher quality and efficiency. The
toughest part of establishing a metrics program within the overall business
context is bringing it to life. Good
luck and good measuring.
Doug Akers
has been in the application lifecycle management industry for the bulk of his
career after graduating from McMaster University with a BSc in computer science
and a minor in psychology. Joining MKS in 1995 Doug has worked with support,
development and marketing groups before moving into product management where he
is currently responsible for the Portfolios and Requirements feature sets within
the MKS Integrity product. In his role Doug works with global 1000 organizations
to help them implement solutions and address challenges within the software
development disciplines.
Trackback(0)
Comments 
Write comment
 |
CM Journal Articles 
TrackBack URI for this entry
Subscribe to this comment's feed