What is a "Best
Practice?"
Over the years I have found "Best Practice" to mean many
things, some of which include:
- Consensus Opinion - A large number
of people agree that whatever this is, it is a really good thing to do.
- Academic Teaching - Higher
Institutions of Learning and/or field related technical schools promote a
practice as the "correct" way to do something.
- Documented Standard - One or more
standards bodies promote a set of "Best Practices" that meet or achieve
their standards.
- Personal Experience - One or more
people determine over the course of their working life that, within a
specific technical niche, something works reliably and publishes this as a
"Best Practice."
Why do "Best
Practices" contradict each other?
In any given endeavor, if you gather together all of the
published "Best Practices" that apply to that endeavor, you will find several
that are at odds with each other. There are several reasons for this, but the
most common are:
- Evolution
- Disagreements
- Differing
Approaches to the Problem
- Not
Really the Same Niche
- Personality
Differences
- Corporate
Cultures
As an example, there are two Version Control mantras that
are both considered to be "Best Practices."
- "Check
in early; check in often."
- "Avoid
build breaks; check in only after testing."
These conflict because they are from two different
paradigms. The first assumes that there is some other mechanism that selects
code for inclusion in a build or release. Checking in early keeps people
apprised of your work if they need to know and initially reserves your element
name in the Version Control's repository namespace. It also, when performed
consistently and at specific times during the change implementation, provides
data points that can be used for root causal analysis.
The second assumes that once changes are checked in they are
immediately available for inclusion in a build. In some cases the tool dictates
which mantra is followed. In other cases it is either the corporate culture or
relative maturity (in a CMM/CMMI sense) that dictates which is followed.
Regardless, you only get to choose one. [Editor's note - the CMMi is one of
the many process improvement frameworks used by software development
professionals to improve their software development lifecycle.]
When is a "Best
Practice" too good?
It is always important that the end product is the goal, not
the process.
- If it
costs too much to make the product or takes too long to get it to market
due to following current "Best Practices," then alternate "Best Practices"
that more closely match the corporate culture, tools in use and/or
development methodology should be evaluated.
- If it
takes too many people to keep the process afloat when following "Best
Practices" or if a new "Best Practice" is evaluated and rejected just
because it violates existing
standards and practices, then there is something wrong. Either the tools,
the corporate culture or the development methodology being followed are
inconsistent with them.
- If one
or more teams are constantly threatening to revolt or start snarling
whenever someone from CM comes near, then there is something wrong as
well.
"Best Practices" are intended to reduce effort while
maximizing both personal productivity and product quality. If they do not do
this, then there is not a match. Of course, just blaming the set of "Best
Practices" being followed is not the right thing to do either. An investigation
should be performed to see if they are understood and being applied in the way
they were intended. Sometimes "Best Practices" are intended for a different
audience; sometimes they really are too good.
A Practical Example
Let's take a practical example, build tags. [Editor's note -
a tag is way to identify an exact version of the source - usually to label what
has gone into a specific release of the code.] In the following sections we
will look at a list of questions and try to determine what to do to answer
them. This will allow us to select "Best Practices" that match our both our
culture and tools in use.
- What
should be tagged? Okay, tagging the source codebase is fairly obvious, but
what about third-party components? What about the build tool chain? What
about test plans, cases and data?
- When
should the tag be applied? Should the tag be applied prior to the build or
after the build? What about when the build is unsuccessful?
- Should
the tag be immutable?
- How
will the tag be used?
- Are
some tags more important than others (and if so, how does one distinguish
between them)?
Background
The culture is fairly laid back, but developers are used to
being in the driver's seat. QA tests the results of controlled builds as
delivered to them by CM and issues defect reports based on failures found
during testing. The Version Control tool in use is cvs, the development
language is Java and the build system is based on Anthill OS and ant.
Now, let's work through the questions.
- What
should be tagged? Okay, tagging the source codebase is fairly obvious, but
what about third-party components? What about the build tool chain? What
about test plans, cases and data?
In order to determine what should
be tagged you need to understand your build procedure, or at least be willing
to modify it to suit your goals. If a Version Description Document (VDD) or
Bill of Materials (BOM) is generated, then it will capture the release levels
of the various third-party components and it is not necessary to tag them. If
the build scripts themselves reference a specific release of each third-party
component, and if the build scripts themselves are tagged along with the
codebase, then it is not necessary to tag third-party components. If
third-party components are not kept in a Version Control repository, then it is
not practical to tag them.
Tool chains, unless they are kept
as an aggregate component, are impractical to tag, though they should be
identified in a build record produced during each build. This build record may
then be versioned and tagged.
Test plans, cases and data, other
than unit level tests normally executed as a part of a build, are developed in
parallel with the codebase, but are normally targeted towards release candidate
level through release and are not build specific. That being the case, they are
not normally build tagged.
What we decided: Only source code, third-party components linked
into the release and those distributed with it are tagged. None of the build
tools, test plans, test cases or test data are included.
- When
should the tag be applied? Should the tag be applied prior to the build or
after the build? What about when the build is unsuccessful?
Whether a build tag should be applied to failed builds or
not is normally determined by what data you want to derive. If you want to do
metrics or root causal analysis on why builds fail, then they need to be tagged
in some way so they can be used in comparison to the previous and next
successful builds. If this is of no concern to your organization, and will not be in the foreseeable future,
then failed builds do not need to be tagged. If pre-build tags are applied and then the build
fails, removing those build tags may be work that just does not need to be
done. Leaving the tags alone may be acceptable and, if so, removing them would
be unnecessary work.
Whether a build tag should be
applied prior to a build and then removed if unsuccessful, or applied only
after a successful build is normally determined by the Version Control tool in
use. If it is not possible to exactly
determine the revisions of every source file used in a build after the fact,
then the tag needs to be applied prior to pulling the code for the build and
then only tagged code pulled for the build. Some example tools where pre-build
tagging is not necessary are: cvs, AccuRev and ClearCase's snapshot views.
What we decided: The builds are tagged only after the build
succeeds. Success does not include post-build report generation.
- Should
the tag be immutable?
Immutable tags can be neither
changed nor removed. If pre-build tags are to be removed for unsuccessful
builds, then they cannot be immutable. The actions of tag change or removal
should be automatically documented in some fashion however and, if possible,
limited to certain specific users and/or roles by the Version Control tool.
There is a belief, especially among
those outside of CM, that even successful build tags have a finite usefulness.
It is often felt that unless a build ends up being a release candidate or an
actual release, then there comes a time when the extra tags just "clutter up"
the system. It becomes difficult for others to determine which tags are
"important" and which ones just "happened along the road to success." This is
one reason some people will multi-tag milestone builds - so others can see
which ones are "really important." A better option is that these tags should never
be removed, even if the tool allows it, since it makes future data mining
impossible (or at least much more difficult than it needs to be). If you are
operating at a CMM/CMMI level 3 or higher, or if you are operating in a rigidly
controlled market niche (DOD, FDA, etc.), keep the tags! If you are below this
and have no plans to advance, and many organizations do not plan to ever get
beyond levels 1 or 2, then whether you should keep the tags becomes more of
tool and cultural issues.
In general, when considering more
than just build tags, there can be controlled tags and uncontrolled tags.
Examples of the former are those related to branching, merging and releases.
Examples of the latter are those applied by users to help them in selecting
files as a part of a change set and use in a promotion process. If there is no
way to restrict changes to, or removal of, tags based on role(s), then it is
better that the tags be immutable. If tags cannot be made immutable, then some
method of automatically documenting who, and when. tag change actions occurred
needs to be implemented. If that is not possible, then another tool should be
considered.
What we decided: While cvs does not allow us to prevent or reliably
log changes to tags, we treat them as immutable. Build tags are never removed
or modified.
- How
will the tag be used?
This has been partially addressed
in the previous bullets, but it deserves to be considered on its own merits.
Obviously, a build tag is applied to allow a build to be reproduced at a future
time. It also provides a method of generating a build Bill of Materials for
inclusion in Version Description Documents, Build Records, etc. It is also
possible to use two build tags to determine which elements changed between the
builds and thus support release notes.
Depending on the types of builds
being performed (ad hoc, continuous integration, periodic, etc.), it may be
possible to use information about the differences between builds to develop
trend diagrams and root causal analyses. Root causal analysis could use the
fact that the same element(s) changed between builds n-2 and n-1 and between
n-1 and n to look for patterns of why
the changes were necessary and specifically what
the changes were. Using enough of this data, one can often determine a pattern
that causes defects to be introduced and QA can work with Development to
develop a change in design and/or coding practices to avoid the problem in the
future. It is also possible to develop a pattern match scanner to look at the
rest of the codebase for similar problems and flag them for inspection.
There are other possible uses for a
build tag. Determining how your tags will be used depends on the tools you have
available and your current and future needs.
What we decided: The differences between milestone builds are used
to help determine potential merge conflicts and generate release notes. Metrics
and trend analysis are planned for the future as is data mining.
- Are
some tags more important than others (and if so, how does one distinguish
between them)?
This goes back to the discussion on
immutability. And to an agreement on the meaning of the word, "important." From
a purist CM standpoint, all tags are equally important. What makes some tags
appear more important is their visibility and use by others outside of the CM
arena. Examples are Release Candidates, Releases and Promotion States.
As mentioned earlier, it is not uncommon to have builds multiply tagged using a
pattern that is easily searchable. An example might be:
Build Tag
|
Secondary Tag(s)
|
Meaning
|
P90_58
|
R2_01_RC1
|
First release candidate for release 2.01
|
P90_72
|
R2_01_RC2
R2_01
|
Second release candidate for release 2.01 and actual release 2.01.
|
What we decided: All tags are important. The graphical interfaces
used to view the cvs repository allows for filtering of how many appear at any
one time and is adjustable on a per-user basis. No additional tags are applied
to indicate release candidate or releases; this information is maintained in a
separate system that provides appropriate traceabilty.
Summary
So how can one determine which "Best Practice" really is a "Best Practice" and which "Best
Practice" is right for you? The practical method is if a "Best Practice" works
or would work within the constraints of both the environment and culture it
needs to, or if it looks like it would work better
than the existing practice, then it is a "Best Practice." Or at least a "Better
Practice."
Ben Weatherall is currently based in Fort Worth,
Texas where he practices Practical CM on a daily basis using a
combination of CVS and custom tools to support a modified Agile-SCRUM
development methodology. He is a member of IEEE, ASEE (Association of
Software Engineering Excellence – The SEI’s Dallas based SPIN
Affiliate), NTLUG (North Texas Linux Users Group), and PLUG (Phoenix
Linux Users Group).
Trackback(0)
Comments 
Write comment
 |
TrackBack URI for this entry
Subscribe to this comment's feed