Risk-Based Testing: Test Only What Matters

[article]
Summary:

Rajnish Mehta writes that test teams need to have a scientific way to support the business need of shipping a product out the door. Risk-based testing is a practical approach for test teams to utilize as it allows them to think from a business perspective. 

In today’s era, the IT industry is running on “fast mode” and the majority of stakeholders are looking for quicker solutions. The project management office is always hurrying to ship products out of the door. While there is nothing wrong in doing things this way, it does become a challenge for test teams who have to accept the fast-mode schedule and promote the product to production.

My recent experience with a retail client and its test team has shown me how desperately test teams want to meet deadlines while dealing with ongoing changes. In this instance, we had a big challenge in that the business had not prioritized its need.

In a situation like this all responsibility falls on the test team, which could easily become a scapegoat for shipping slipped defects to production. A delay in the upstream software development lifecycle (SDLC) process like requirements and development can make things worse. For my team, the stakeholder’s constant demand to make the product reachable to the customer was adding fuel to our fire; the test team was under tremendous pressure.

Being a solution provider, a test team must have a logical and scientific answer to the above challenge. Using a risk-basked testing (RBT) approach worked really well as the test team could deliver on the promise. While there could be many more factors when you assess the risk, I just chose two factors to explore in this piece for the simplicity of implementation and it’s easier to understand. With the approach we chose, our test team saved a big portion on the test execution effort; it allowed for great savings yet it delivered all that was important to the business.

All of us working in the IT industry are continuously looking for cost-effective solutions. Obviously, this means implementing the most logical solution that can bring lots of value with less effort. In my opinion, this solution is for all kinds and sizes of projects. Depending upon the industry, the risk factor may change but the concept remains the same; except, of course, if you are in the pace-maker industry and you have zero risk-taking ability.

Depending on the industry, we must always maintain medium-to-low open defects during production. For example, a company making pacemakers will have lower risk-taking ability than the financial industry, which has a lower risk-taking ability than the education industry and so on. One’s risk-taking ability depends upon the industry and then the application being designed within that company.

Some industry leaders are willing to take on known risks than delay time-to-market. I believe that in the IT industry, the QA team should find a way to support the thought of a faster time-to-market concept.

In order to achieve the goal of testing only that matters and achieving a faster time-to-market, we need to rethink testing scope. You might have some questions about doing so, which could include:

  • Do we need to test everything?
  • How do we reduce the testing effort?
  • When do we stop testing?
  • What is the best way to have a shorter test-phase cycle duration?
  • How do we be successful with more test cycles?
  • How do we get buy-in from management when we do not test certain test cases?
  • What metrics do we produce?

The answer to all of the above can be found in using a risk-based testing framework.

Risk-based testing is an approach that takes a scientific approach when accounting for risk. It is mainly based on the factors of the business impact and the likelihood of failure, although there could be more.

Using this solution will help you in many ways. For example, risk-based testing will help when expediting through the software quality assurance gate, which results in faster time-to-market. It also helps reduce schedule slippage, communicates to IT management on what exactly is at risk, helps define the severity of defects, and is a great way to flag test cases for regression testing. It’s a great tool for prioritizing test execution and selecting candidates for automation. Risk-based testing helps define the severity of defects and it can be suggestive of “what to test” when we do not have enough time for testing. When we run out of money in our budget, we can still ship the product with known risk. 

User Comments

2 comments
Joe Dwyer's picture

Risk-Based Testing is another tool for the test engineer to use. The article provides a nice explanation and analytical application (graph) which will help the stakeholders understand as well.

It is important, as you mentioned, to have the business owners and IT collaborate on the definitions of impact and priority and also categorizing. Getting the stakeholders involved will make the implementation of RBT a success.

 

To make certain that this method is working for your organization I suggest a benchmark of your escapes prior to implementing RBT. Making certain that the number of escapes remains flat or drops. And pair that up with your goal for using RBT, lowering cost, shortening time to market, being more efficient, etc.

March 4, 2014 - 11:55am
Jon Hagar's picture

ISO29119 defines a set of process to support risk based testing, FYI. 

December 2, 2014 - 9:27am

About the author

CMCrossroads is a TechWell community.

Through conferences, training, consulting, and online resources, TechWell helps you develop and deliver great software every day.