Judging by the skills enumerated on job descriptions for software configuration management (SCM) professionals, automating "the build process" is a high priority for many groups that develop software. While build automation can improve an organization's ability to deliver quality software, build automation is not a silver bullet. Build automation enables one aspect of a software development process to be repeatable. Build automation does not correct other deficiencies in a software development process. All problems involving an organization's software development process will not evaporate the moment builds are automated. Instead, the software development challenges that an organization faced prior to build automation are likely to persist even after builds are automated. To achieve benefits from build automation, an organization must first make software configuration management an integral part of its software development activities.

Examine Your View of Software Configuration Management

If you view SCM as separate and distinct from software development you should re-examine this belief before automating your build process. With the advent of modern, automated SCM tools, there is no reason why developers should not be engaged fully in software configuration management activities. However, being engaged in software configuration management activities does not mean being encumbered by these activities.

Modern SCM involves facilitating the pace of change that is inevitable when software is developed as a team activity. Modern SCM is about enabling developers with the tools and the freedom needed to perform SCM activities. Modern SCM is a team activity that does not rely on a software librarian to manage check-outs from, and check-ins to, the source code repository. Modern SCM is a team activity that does not encapsulate all build knowledge in the mind a build master.

If modern SCM does not delegate all build responsibility to a designated build master, then how can builds ever be automated? The answer begins with understanding the different types of builds and where they occur throughout the software development lifecycle.

Software professionals use the term build to describe both the act of compiling source code to create an executable software system and the instance of the executable software system created. They distinguish between the verb build and the noun build. Often, they do not distinguish between the different types of builds that are created by following a build process.

Distinguishing the different types of builds is critical to understanding how software configuration management is integrated with, and part of, software development. There is no such thing as a universal build. Instead, there are different types of builds that are created during the course of a software development lifecycle.

Maciej Zawadzki identified four types of builds in his column in the October 2003 edition of Crossroads News. The build types he described are

• local development builds,
• continuous integration builds,
• QA builds,
• and release builds.¹

Each build type described by Maciej Zawadzki is characterized by the environment in which the build will be used. A local development build is created by a developer for use in that developer's environment. The continuous integration build is created, possibly on a dedicated build machine, to pull together the changes made by different developers, each working independently on the same software system. A QA build is created so that the quality assurance team can test the software system at a point along the software development lifecycle. The release build is created with the final set of changes in preparation for the deployment of the software system.

While the names of the build types identified by Maciej Zawadzki may not be universal, the purpose of each build type is probably well understood by any developer who has worked through a complete software development lifecycle on a team project. Experienced software developers are also likely to be familiar with the build types popularized by Steve Berczuk and Brad Appleton.

Software configuration management patterns, introduced by Steve Berczuk and Brad Appleton, help explain how all parts of the software development environment interact with each other. The build patterns, and related patterns, explain how different types of builds fit in to the software development lifecycle.

In their book, Software Configuration Management Patterns: Effective Teamwork, Practical Integration, Steve Berczuk and Brad Appleton identify two build types that are central to their pattern language for software configuration management. These build types are the

• private system build and the
• integration build².

The private system build is equivalent to the local development build defined by Maciej Zawadzki. The purpose of the private system build is to compile the software system using the changes made by the developer in the developer's local environment, or private workspace.

Developers could proceed through the software development lifecycle by making code changes in their private workspaces then checking-in these changes so that they are available to other developers. Since good software development practice prescribes that code changes be tested before they are released, there needs to be a way for every developer to test changes before they are made available to the entire development team.

A private system build allows a developer to build the software system using the changes that have been made in the developer's private workspace, but not yet submitted to the version control system. Once built, the software system can be tested to verify that the changes function as expected without breaking any existing functionality. By performing a private system build then running a smoke test, a developer can validate changes before they are checked-in to the version control system for inclusion in the next integration build.

The integration build is critical to team software development. Experienced developers understand that team software development involves members of the team working, at the same time, on different changes to the software system. Eventually, the changes that each developer makes while working in isolation, in a private workspace, need to be pulled together. The integration build exists for this purpose. It is a mechanism for combining the code changes made in isolation by multiple developers. Specifically, the integration build described by Steve Berczuk and Brad Appleton is a centralized build that is performed regularly to ensure that the code base always builds reliably. For this reason, the integration build is equivalent to the continuous integration build described by Maciej Zawadzki.

¹ Zawadzki, Maciej. "Beyond Continuous Integration, A Holistic Approach to Build Management (Part 1 of 2)." October 2003. Crossroads News.

² Berczuk, Steven and Brad Appleton. Software Configuration Management Patterns: Effective Teamwork, Practical Integration. 2002.

In their column in the October 2003 edition of Crossroads News, Steve Berczuk, Brad Appleton, and Steve Konieczka extended the SCM patterns to include the release build. They identified the release build to package software either for pre-release testing or for release to the customers of the development team.³ Based on these uses, the release build defined by Steve Berczuk, Brad Appleton, and Steve Konieczka encompasses both the QA and release builds described by Maciej Zawadzki.

The different build types and build patterns illustrate that the act of creating a build occurs at different points in the software development lifecycle. Builds begin when coding starts, continue through system testing, follow up to release or deployment, and resume for post-release maintenance.

Not only do builds occur throughout the software development lifecycle, but they are central to the rhythm of that lifecycle. When an organization cannot build, it cannot integrate, it cannot test, it cannot deploy, and it cannot maintain. When an organization cannot build, it cannot create software.

When an organization loses, or never even achieves, the ability to build, it is probably the result of a software configuration management failure. Integrating good SCM practices into the organization's software development process is the first step towards correcting a build problem.

Integrating SCM with software development means SCM activities become a part of routine software development activities. The integration of SCM activities with development activities is no better exemplified than by the use of the SCM patterns.

Each SCM pattern is a solution to a typical SCM problem in a context. The patterns do not exist in isolation. Rather, each pattern completes a pattern that preceded it. Specifically, each pattern fits with other patterns to form a pattern language.

The power of the SCM patterns to integrate SCM activities with development activities is best illustrated by the following sample guidelines. These guidelines are a set of steps that each developer on a team can follow to complete the coding tasks that comprise project activities.

• 1. Start by compiling the list of high-level development activities that form the project plan. Break down each high-level activity from the project plan into small-grained development tasks. Size each small grained-development task so that it can be coded and unit tested by one developer in about a day (or by two developers who are pair programming).
• 2. Begin your small-grained development task by creating a workspace in your IDE (integrated development environment) or development environment. Populate the workspace with the code base for the software system under development. By creating a workspace you are following the private workspace pattern. The private workspace pattern isolates your development environment from outside changes so that you can work productively in an environment that is not in constant flux. By populating the workspace with the desired code base you are following the repository pattern. The repository pattern ensures that you populate the workspace with the correct versions of the files that make up the code base.
• 3. Check-out the files that you need to modify to complete the small-grained development task. Make the coding changes necessary to accomplish the task, but do not check-in these changes to the version control system.
• 4. Run unit tests after completing the code changes in your workspace, but before checking-in these changes to the version control system. By executing these tests you are following the unit test pattern to ensure that functionality works correctly after you made small-grained code changes.
• 5. Build the whole software system in an isolated environment after completing unit tests. You are following the private system build pattern to verify that the whole software system still builds after you made small-grained code changes in your workspace.
• 6. Start the software system built from the small-grained changes made in your workspace. Verify that the most basic functionality still works in this build. You are following the smoke test pattern to verify that the software system has not broken in an obvious way before you check-in the changes to the version control system.
• 7. Synchronize your workspace with the version control system to update your workspace with changes made by others while you were completing the small-grained development task.
• 8. Check-in the code changes for your small-grained development task to make them available to other members of the development team. You are following the task level commit pattern to ensure that each check-in corresponds to one small-grained task.
• 9. Each day, the SCM engineer initiates a centralized process to build the software system using the latest code changes that were checked-in to the version control system. The SCM engineer is following the integration build pattern to ensure that the code base for the software system builds reliably.
• 10. If the integration build completes successfully, the SCM engineer initiates tests on the build. The SCM engineer is following the regression test pattern to ensure that existing functionality in the software system does not break after new functionality is added.
• 11. Begin your next small-grained development task by repeating the steps above, beginning with step 3.

³ Berczuk, Steve, Brad Appleton, and Steve Konieczka. "Agile SCM - Build Management for an Agile Team." October 2003. Crossroads News.

Observe how coding, building, and testing activities fit in between the check-out and the check-in steps above. Also notice that there are different building and testing activities. Local building and testing activities are performed by the developer who completes the coding task. Integration building and testing activities are performed by an SCM engineer whose focus is on integration across the project.

By applying the SCM patterns, check-out, code, test, build, and check-in become daily activities taken to accomplish a fine-grained development task. When the development team is able to apply the SCM patterns to integrate SCM activities with daily development activities, the project is ready to automate the build process.

The first step in automating the build process is to determine who performs each build. The SCM patterns show how different types of builds are performed by different members of the whole software development organization. Some builds may be performed by developers while others may be executed by dedicated build engineers. The type of build determines who is the most appropriate person to execute it.

Clearly, a private system build is executed by a developer. The developer who makes changes while working in a private workspace is responsible for performing the private system build.

Since integration builds are a way for developers to verify that the changes they made in isolation will work together, the integration builds may be performed by a developer. Alternatively, integration builds may be executed by a dedicated build engineer. The size of the project likely determines if integration builds are managed by developers or by dedicated SCM staff.

In all but the smallest projects, builds destined for anyone outside of the development team are performed by a dedicated SCM engineer. QA builds and release builds are performed by an SCM engineer because they are deployed outside of the development environment. In fact, some organizations may require that QA builds and release builds be performed by an SCM engineer who does not report to the manager responsible for the software development team.

To automate the build process, an organization needs to execute a build from the operating system command line. While a private system build can be performed within an IDE, doing so limits the usefulness of the build. Builds created within an IDE do not use the same tools and process as builds created from the command line.

Once the build steps are established and tested they can be reproduced by means of a script. However, scripting is not the only way to automate a build process. To avoid having to maintain build scripts, an organization may elect to use a commercial build automation tool that does not require users to create and maintain scripts.

No matter what approach is used to automate the build process, the same approach should be used to create every build type used by the organization. When different technical staff perform different builds, it is important that the same procedure and tools be used by each staff member. Not only does this ensure consistency, but it reduces the chance that a build will succeed in one environment, but fail in another. Differences between environments are notorious for wreaking havoc with builds.

Automating the build process is a way to ensure that a software system can be built using a repeatable process. However, build automation is appropriate only after a software development organization integrates software configuration management with its software development process. When SCM becomes a part of routine software development activities the organization should be able to always build the code base reliably.

Trackback(0)

TrackBack URI for this entry

Comments (0)

Subscribe to this comment's feed

Write comment