Imagine you're looking to get a used car for your teenage daughter. Ideally, a trusted friend or relative is willing to give you a great deal on a hand-me-down that you know they've treated with care. You see if your daughter likes it, then give it a test drive. If it seems like a good fit, you're all set.
Now imagine that nobody you trust is looking to unload a car right now. Are you comfortable putting your daughter in a car that you found from an independent seller on Craigslist? Or would you rest easier if you set her up with a certified preowned vehicle—something that passed a reputable dealer's one hundred fifty-point quality assurance inspection and was backed with a five-year bumper-to-bumper warranty and roadside assistance?
With service-oriented architecture (SOA), as with the hand-me-down car from a close friend or relative, the element of trust is key. Because you're familiar with the "supplier," you're not so worried about the quality of the product. You can reasonably assume that certain standards have been met, and you're comfortable moving forward without extensive vetting.
On the other hand, consuming external application programming interfaces (APIs) is more like purchasing a used car on the open market. With limited insight into its integrity (beyond API user ratings, which you might say function like Carfax reports), you can't really know what condition it's in or predict how reliably it will be able to meet your needs over time. Before you bring it into your "family," it's quite prudent to want to know how it stands up under rigorous testing.
Unfortunately for anyone using today's composite applications, most software development leaders aren't nearly as cautious about vetting the APIs their applications consume as they are about scrutinizing their personal consumer purchases. In fact, surprisingly few organizations today even think about testing the APIs they rely upon to drive their business-critical processes. This issue becomes a real challenge in organizations that embrace agile development and DevOps, where new features are developed rapidly and sometimes delivered continuously throughout the day.
SOA: "Trust but Verify"
With SOA, the services you consume are typically developed within your organization or by a close business partner. You can reasonably assume that a service was developed according to standards similar to your own and was subject to comparable quality control measures. Moreover, in the event that it fails, you know who to go running to, and you can reasonably expect that problems will be resolved in a timely manner. Also, there is at least some chance (albeit slight) that if the service producer decides to make substantial updates to the services you are relying on, they might actually let you know.
How does this translate to testing? As long as you can trust that the service meets reasonable quality expectations, you can focus on verifying the service in the context of your end-to-end business transaction.
APIs Require Substantiated Integrity
When you adopt APIs that are produced and managed outside your organization's "geopolitical control," you typically have no visibility into how solidly the exposed services were built—or when and how they are evolving. If an API fails, quite likely your hands will be tied. Unless the API was backed by a service-level agreement, you probably have little recourse. Moreover, because the failure is manifesting itself via your application, you're ultimately going to be the one to shoulder the blame and suffer the consequences.
On top of your lack of insight into the quality and evolution of external APIs, as well as your limited ability to act if there is a failure, there are also risks associated with integrating exposed APIs into your applications. To start with:
- Adopting an API that is exposed through an internal infrastructure undeniably increases your application’s attack surface area from a security perspective. By adopting an exposed API, you could open the door to API-level attacks (injections, payload-based attacks, etc.), as well as exploits that take advantage of ineffective authentication, encryption, and access control.
- The challenge of broader attack surface area is compounded if the API is hosted by a public cloud service. In traditional computing, you’re aware of (and have full control over) the parameters of the network security. With cloud services, this level of control is significantly diminished.
- An API that is reused across hundreds or thousands of applications is a much more tempting attack target than a service that impacts only one application. And once a weakness is found in a popular API, it can be shared and exploited broadly—just like attacks on widely used open source software (e.g., OpenSSL and Heartbleed).
- The performance of APIs can be extremely variable. Do you know how your application will react if an API you rely on experiences a sudden spike in usage? If an API interacts with additional layers of services, the potential for variable or unacceptable performance increases exponentially.
Because APIs don't come complete with a one hundred fifty-point inspection report, how do you properly vet an external API to determine if it has the required degree of integrity? Briefly, if an API you consume is critical to your business processes:
- It needs to be tested directly, as well as in the context of the organization's business process and distributed back-end systems. Testing from the interface (if available) is undoubtedly important, but—on its own—it's not sufficient for exposing the many types and layers of problems that could lead to outages or transaction failures.
- The end-to-end API-driven process needs to be tested continuously via a realistic and complete test environment. Otherwise, test results will not be accurate predictors of real-world behavior.
- That API, as well as its dependencies, should be regularly analyzed to ensure that the QA team is immediately alerted to changes and understands how the test assets need to be updated in response to these changes.
If You Consume It, You Own It
We're fast approaching a time when all communications between humans and machines, as well as machines and machines, will ultimately be driven via APIs. To truly protect your brand in this "API economy," it's essential to have solid processes for the identification, integration, and testing of APIs in conjunction with your business priorities. After all, if your application fails to deliver the expected business results, your customers and partners won't care if that failure stems from code you developed internally or from an external API that you've integrated. This becomes a critical factor when your organization accelerates application development with agile and DevOps, where development cycles are measured in weeks and deployments may occur continuously throughout your day.
If you consume it, you own it.
Moving beyond these myths is the first step in creating a competency center for API testing. Remember, API testing is not a singular event; it's a cross-functional strategy to ensure that the desired business expectations can be achieved.
