Testing, Zen, and Positive Forces

Suppose an organization has been in place for eight years. All testing is manual. There are no test plans, no documented testing procedures, no requirements documents, no design documents, no comments in the code; and, over most of the organization's history, no bug-tracking system.

Sound familiar?

This is a real company. It's similar to many companies I observe. These stories are always told with an air of exasperation and disbelief. Is this a hopeless situation for a software tester? Or perhaps a fixer-upper, just waiting for you to help them recover from their slovenly ways?

I've gone through an evolution in my thinking as a tester, starting out enthusiastic and then becoming frustrated. I tried my hand at process improvement, but that didn't make me happy or successful. Since then, I've learned that it's important to achieve a Zen-like acceptance of the status quo before drawing any conclusions about how it needs to change. As Jerry Weinberg said in The Secrets of Consulting (quoting Kenneth Boulding), "Things are the way they are because they got that way."

It's not uncommon for testers to become frustrated with the processes that produce the bugs that they find. They read literature that says how software development and testing should be done, and they see that their organization isn't doing it the "right" way. These testers may have QA in their titles, but this often isn't an accurate appellation. They are taught that quality assurance is about error prevention, so they want to get involved in improving the company's processes.

Most of the companies that are the subject of these horror stories are surviving, even making a profit. Perhaps they could be more efficient, but there might be good reasons for all of the things they are (and are not) doing.

Center Yourself
Assess the current situation, and determine what it would take for you to get your job done. Challenge all of your assumptions: Must everything be tested? Does test execution have to be automated? Can tests benefit from some type of automation? Do you really need to document all of your test cases in great detail? Discover the risks, and test accordingly.

Make your plans such that you don't expect other people in the organization to change what they're doing. If there's no product documentation, do exploratory testing. If there's no unit testing, plan for additional functional testing. If there's no bug-tracking tool, keep your own records of the bugs you report. Once you have that plan laid out, then you can show how you could save a major amount of effort if someone else makes a small change to how they do their work. But when you don't get all the cooperation you ideally would like to have, you still have a workable plan.

If you're being asked to do something that you think is impossible, counteroffer with something that is possible. Find out which sides of the old scope-schedule-resources triangle are fixed, and expand the flexible sides as needed. If all three of these are overly constrained and you can't get your estimates accepted, you're better off leaving than trying the impossible and failing. But if you're creative, you can often negotiate a feasible plan.

Improvements Aren't Verboten
There's always room for improvement in an organization. Start small; concentrate first on things that directly affect your work. Using an incremental approach is crucial, because it's easy to get distracted on an improvement project and fall behind on your primary responsibilities. If some sort of process or infrastructure change is critical for your success, include it in your schedule. Also, if your company culture allows for it, you can plan for unspecified schedule overhead to handle small improvement tasks.

Of all the changes I've tried to implement, only the ones that could show short-term results have ever succeeded. See my paper "The PIT Crew" for more discussion. Also, see James Bach's "Agile Test Automation" presentation, which offers a description of a process that only endorses those test automation projects that can produce usable results with a week of effort or less.

The Long Term
So you've accepted the current state of your company's existence, and maybe you've tried to change a few things. But what if the type of work that needs to be done in your environment isn't the type of work you thought it would be? If you need more structure, there may be another company that fits you better, just as people who don't like structure might want to stay away from a company that eats and breathes the CMM. If you decide that you don't need to run away entirely, think carefully about how you can be a positive force in the organization.

Chapter 8 of the book Lessons Learned in Software Testing has lots of good advice about this subject. See especially lesson 158: "Don't try to create a control culture" and lesson 176: "Adapt your processes to the development practices that are actually in use."

If you're still fretting about whether your organization needs to make big changes to its processes, keep in mind that the development process might not be the most urgent change that your company needs to implement. Until you've done a careful assessment to find the highest-priority improvement needs, you don't know whether a major change to the development process could distract the company from more important tasks. Before you get overly frustrated with a system, you need to understand how the whole system works, not just the parts that you're directly exposed to.

Acknowledgements: I thank James Bach for helping me shape and articulate my ideas.