Best Practices for the Formal Software Testing Process
Testing is not a phase. Software developers should not simply throw software over the wall to test engineers when the developers have finished coding. A coordinated program of peer reviews and testing not only supplements a good software development process, it supports it.
A good testing life cycle begins during the requirements elucidation phase of software development, and concludes when the product is ready to install or ship following a successful system test.
Nevertheless, there is no one true way to test software; the best one can hope for is to possess a formal testing process that fits the needs of the testers as well as those of the organization and its customers.
A formal test plan is more than an early step in the software testing process—it's a vital part of your software development life cycle. This book presents a series of tasks to help you develop a formal testing process model, as well as the inputs and outputs associated with each task. These tasks include:
- review of program plans
- development of the formal test plan
- creation of test documentation (test design, test cases, test software, and test procedures)
- acquisition of automated testing tools
- test execution
- updating the test documentation
- tailoring the model for projects of all sizes
Whether you are an experienced test engineer looking for ways to improve your testing process, a new test engineer hoping to learn how to perform a good testing process, a newly assigned test manager or team leader who needs to learn more about testing, or a process improvement leader, this book will help you maximize your effectiveness.
Review By: Robin F. Goldsmith
06/23/2010In "Best Practices for the Formal Software Testing Process," Rodger Drabick presents a highly structured set of narrative-format checklists for tasks and subtasks in a formal system testing process, where separate organizational functions communicate largely via respective written plans. Each identically formatted section of the book describes tasks in terms of successively decomposed Input-Processing-Output (IPO) diagrams.
The book is structured like a set of nested menus based upon a series of IPO diagrams. Level 1's diagram describes five sub-processes: 1.1 Extract Test Information from Program Plans; 1.2 Create Test Plans; 1.3 Create Test Design, Test Cases, Test SW, and Test Procedures; 1.4 Perform Formal Test; and 1.5 Update Test Documentation. For each sub-process, Drabick describes inputs, immediate feedback, feedback from earlier process steps, process flow, outputs, and feedback from later process steps. He devotes a separate chapter with identically structured descriptions of each of the sub-sub-processes (e.g., 1.1.1-1.1.4). He also diagrams the next level down for each (e.g., 1.1.1.1-1.1.1.5) but does not repeat the input-feedback narrative descriptions. Appendix C repeats these in a checklist questionnaire format.
Drabick's formal world involves documented requirements, program management plans, software development plans, software quality assurance plans, configuration management plans, and test plans, each prepared by separate specialized organizational units. A number of the tasks involve pre- and post-testing meetings with these groups, reviewing documents, preparing testing reports for the others to review, and updating the testing documentation upon completion. Appendix B provides templates for each of these documents. Chapter 8 describes ways Drabick feels the formal tasks could be tailored to less formal test environments, such as by skipping or consolidating particular tasks. Other appendices briefly summarize the CMM method and some test execution practices.
The book's structure is its greatest strength and its greatest weakness. The identically formatted, nested IPOs will be helpful for those seeking a reference checklist of all the tasks and subtasks to do in a formal testing process. On the other hand, this orderliness means that every section looks alike. I counted twelve single-level and thirty-five multi-level IPO diagrams, each with the same set of six input-feedback narrative sections. It became a big blur. Nothing stood out as important, and I quickly lost any sense of where I was in the overall structure. The similar-appearing but uncoordinated separate numbering schemes for chapter sections and IPO diagrams further obscured any sense of "You are here."
The book is not an outline of how to do any of the prescribed tasks. Only the process flow sections give even a hint of what testers actually may encounter carrying out these numerous tasks. I sort of doubt there are a lot of people seeking such checklists, except those who already do accept and follow formal (and probably well-documented) processes. I seriously doubt even they would read through the full narrative to extract the list of tasks; and it's highly unlikely other typical testers would read through to Chapter 8's somewhat naïve suggestions for tailoring the numerous tasks to less formal environments.