When Should a Test Be Automated?

[article]
Member Submitted
Summary:

The question is how to make a rational decision about which tests to automate. When I take a job as a contract tester, I typically design a series of tests for some product feature. For each of them, I need to decide whether that particular test should be automated. This paper describes how I think about the tradeoffs.

I want to automate as many tests as I can. I'm not comfortable running a test only once. What if a programmer then changes the code and introduces a bug? What if I don't catch that bug because I didn't rerun the test after the change? Wouldn't I feel horrible?

Well, yes, but I'm not paid to feel comfortable rather than horrible. I'm paid to be cost-effective. It took me a long time, but I finally realized that I was over-automating, that only some of the tests I created should be automated. Some of the tests I was automating not only did not find bugs when they were rerun, they had no significant prospect of doing so. Automating them was not a rational decision.

The question, then, is how to make a rational decision. When I take a job as a contract tester, I typically design a series of tests for some product feature. For each of them, I need to decide whether that particular test should be automated. This paper describes how I think about the tradeoffs.

About the author

CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.