 There is nothing more fun than building a large complicated software system from scratch. Build engineering has got to be one of the most enjoyable tasks because the build engineer gets to see and understand a great deal about how all of the system components get created and integrate together. In fact, the build engineer may be the only person who understands how the entire system works. CM, as a discipline, must integrate very closely with the testing function. This is not always easy and sometimes the transition from build and release phase to testing can be a bit challenging. Read on if you would like to see how your CM team can best partner with the test team and add value to the entire ALM process!
Testing and CM have an essential two-way relationship. The test function needs support from the build engineering team in order to know exactly what has changed in any release. The build engineer needs to know how to test the application or he has no reasonable way to confirm that the release is complete. CM itself must be tested. Any release that is tagged (or in some SCM tools labeled) should be independently built in what is essentially the build process itself.
What does all this have to do with process?
Any process requires a clear definition of the tasks to be completed, roles and responsibilities and, of course, a set of tests to confirm that each phase of the process is completed successfully! In fact, process engineering requires that specific milestones be put in place to confirm that the process is being followed. This is essentially "testing" the process. CM, as a discipline, includes many processes and best practices including Release Management, build and deployment, as well as source code management and integration with defect tracking systems (to associate change sets with specific changes that have been made to the code). Testing is an integral part of every aspect of CM.
When I write build and deployment systems, I build tests into every aspect of my process. This practice can include checking that a specific jar is in the build to looking for error messages in a log. Code that is deployed to QA, Integration or Production must be tested to confirm that it was successfully deployed. These tests can be as simple as a quick check to confirm that the new directory was created and the date stamp appears valid to very detailed tests which actually traverse through directory keys and create cryptographic signatures to ensure that the release arrives successfully and was never tampered with. There are a number of good automated tools on the market that make this task must easier to do.
CM'ers are members of the test team too!
Any effective development team, whether they are Agile or ad-hoc must consider testing to be an essential task which everyone must support. I always ask for a test account and a little training from the test team so that I know how to enter some orders myself and, at a minimum, do a little "smoke testing" myself after every deploy. I always ask the test team to let me know what they are doing and how well their testing effort is going. I also spend a lot of time giving the testers specific hands-on support. This may include explaining exactly what I have seen changed in a release to simply explaining how the system works from a technical perspective. I view the role of the test as being truly critical and build engineers should be very proactive at supporting and facilitating the testing process.
Early integration and testing
Continuous Integration is quickly emerging as an essential best practice. There are many good tools which support CI and every development team should spend the time to implement an effective CI initiative. From a build engineering point of view, this may not be easy. I have seen problems where the build worked just fine from the command line but ran into problems running from the CI server. Many organizations use Maven 1, 2, Ant and makefiles to build their code. While most CI tools support all of these technologies, it turns out to be tricky to support them all in the same stream. That means that you have to become very proficient in the tools or else run multiple instances of your Continuous Integration servers. There is no doubt that early and continuous integration is a best practice that is well worth the effort. In implementing these technologies - I know of instances where missing libraries were discovered and code inconsistencies uncovered. It's always better to discover these problems as early in the software development lifecycle as possible.
Unit testing
Many build tools implicitly include support for unit testing in their overall framework. Developers sometimes resist this essential practice. Build engineers should try to encourage developers to write their own unit tests which should be included in the build process. It is common practice to provide a switch to skip these tests, when necessary (e.g. rushing a build out the door). It is also common practice to include them in the nightly or hourly continuous integration frameworks. Behaviorally, we need to facilitate these best practices and highlight their value even if the test suite starts off being less than comprehensive. CM is all about overcoming resistance to change. And many developers feel a bit self conscious about setting up tools that may eventually test (and fail) their next release.
Repeatable builds
Most of us go into technology because we enjoy and see some value in performing challenging technical tasks that require a fair amount of expertise. I always say that I get paid to "play" with computers because I enjoy performing these tasks so much that I can do them for hours and still come back for more. However, many build engineers have trouble admitting that their own internal processes must be tested just as the development code must be tested. Having someone else review and comment on your build and deploy checklist is a great way to "walk the walk" instead of just "talking the talk" about quality and process improvement.
Culture of Quality
It's obvious that testing and CM are indeed intertwined and, in some ways, inseparable. Testing and CM are essential building blocks in any successful ALM (Application Lifecycle Management) effort. To get this right, we need to evolve into an environment that embraces best practices - including CM supporting the testing effort - as well as - with testing practices supporting CM!
Bob Aiello is a Senior Editor for CM Crossroads and an Associate Director at a major financial services firm in NYC, where he has company wide responsibility for Software Configuration and Release Management best practices. Bob is on the Steering Committee of the NYC Software Process Improvement Network (CitySPIN), where he is also the chair of the CM SIG which meets in Midtown NYC. Mr. Aiello has a Masters in Industrial Psychology from NYU and a BS in Computer Science from Hofstra University.
Trackback(0)
Comments 
Write comment
 |
Behaviorally Speaking 
TrackBack URI for this entry
Subscribe to this comment's feed