| 
Building Requirements Quality thru Coverage and Testability PDF Print E-mail
Tuesday, 21 November 2006
As we look to deploy or improve a requirements practice, the output should lead to a set of complete and quality requirements.  A way to get there is to ensure that requirements have been captured in all pertinent categories and are testable. 

Often times, requirements are gathered that focus on user needs and limited functional areas.  But what may be overlooked are lesser requirements categories (e.g., security, usability, etc).  It is therefore essential to capture requirements that ensure coverage in the requirements space. Once requirements cover all pertinent areas, it is critical to ensure that the requirements are testable.  Testability implies that the requirements are understood enough that a test case can be established and then run to prove that a requirement has or has not been met.  Having a requirement written at this level ensures that developers can build something that aligns with the requirement, reducing the gap of what the requirement specifies versus what gets built.          

With this in mind, this article provides a straight forward approach to requirements coverage and testability.  More advanced techniques and practices exist in this area.  The important thing to consider is what level of practice your organization is ready to address.  Once a solid step has been made, then advancements should be considered.       

Requirements Coverage
 Requirements Coverage is an approach to capturing requirements in all pertinent requirements categories.  This ensures a set of robust requirements that are encompassing for the project.  Assuming a project will test the requirements, this also helps ensure that during the testing phase, a range of testing occurs for the project that alleviates concerns around insufficient testing in key areas. 

The question becomes, "How does a person approach establishing Requirements Coverage?"  Key contributors to defining this include Analysts and Testers.  A sequence of activities would include:

Identify core Requirements Categories
The first activity is to identify the common requirements categories.  This, unfortunately, is not as easy as it sounds.  The reason is that there is no conclusive set of requirements categories.  Terminology varies among organization and what requirements categories are needed also ranges.  Searches on the internet reveal that there are various ways to articulate requirements categories.  Some examples include:

  • In "Isn't Someone Coding Yet (WISCY)? Avoiding Ineffective Requirements" by Charlene Gross (SEI), she lists the traditional requirements categories of: Business, User, Functional, Non-functional.  In Non-functional requirements categories, she lists numerous sub-categories including government regulations, platform, legacy interfaces, usability, performance, scalability, security, flexibility, and portability.

  • On the Ludwig Consulting site, Ludwig lists some requirements categories to include: Functional, Operational, Development process/Project, Performance, and Contract.  In Performance, he lists numerous sub-categories as efficiency, expandability, flexibility, integrity, interoperability, maintainability, portability, usability, reliability, throughput, response times, capacity, availability, and growth.

  • In the article, "ABCs of Requirements Engineering", by Mario Moreira on CM Crossroads, he writes, "some examples of the category of requirements are user requirements, system requirements, performance requirements, and security requirements. 

With this in mind, the important thing to do is to collaboratively determine the list of requirements categories within your organization.  Getting a small group of Analysts and Testers is a good way to begin.  Brainstorm potential requirements categories with this small team.  Then consider validating these against the common root causes of the defects that are often found in testing phase and when the product is released (customer defects).  See if they align with any existing requirements categories or that a new requirements category is needed.  For example, if you see a lot of defects relating to access (e.g., easy to access areas that are not supposed to be accessed), then this may mean that "security" should be a requirements category.  Once a robust set of requirements categories is captured, then they can either be prioritized to the top 12 core requirements categories (e.g., since it may be hard to manage too many categories) or they all can be used. 

Incorporate Requirements Categories
Once a set of requirements categories is established, build this list into the requirements process that is used within the organization.  Common places to include the requirements categories are in a:

  • Requirements Elicitation (or similar) process.  The ideal spot is the step where requirements are being gathered.  This reminds folks that there are numerous requirements categories to consider.

  • Requirements List template.  The ideal spot is to include a column or field called Requirements Categories and include the common requirements categories as a reminder. 
Note: if no process Requirements Elicitation (or similar) process and/or Requirements List template exists, then one should be established.   

Establish Requirements Coverage Metrics
Requirements coverage metrics provide the team with a view of the number of requirements in the requirements categories.  The aim is to identify if there are any categories that have few or no requirements and validate if this is reasonable. 

Considering a scenario, let us say that an organization has decided on 6 requirements categories.  They are: business, user, functional, performance, security, and usability.   What should be produced is a metric chart that illustrates the number of requirements in each requirements category.  Below is an example of the Requirements Coverage metric that can be produced (see Chart #1).  This chart highlights that both Performance category and the Security category have very few requirements.  After more work in identifying performance and security requirements, Chart #2 illustrates the increase in requirements in these areas. 

mmnov06-1smallmmnov06-2small












              Chart #1                                                           Chart #2

Initially what will be obvious in a Requirement Coverage metric is when there are few or no requirements in a given category.  However, it does not help determine the "normal" number of requirements that should be in a given requirements category to ensure you really do have coverage.  For example, would you expect to have more business requirements than usability requirements?  Over time, additional metrics can be tracked to determine, based on a project size and type, how many requirements are normal (or average) for a given requirements category.  Another enhancement to improve the robustness of this metric is to prioritize the requirements categories in general or by project type (e.g., a web project may place a higher priority on usability requirements then a back-end system processing project). 

Requirements Testability
Requirements Testability is an approach that can take many forms, two of which are considered here.  The first form is to establish a language construct for articulating requirements that ensure all elements of a requirement are available to ensure effective testing.  The second form is to establish a requirements peer review technique that ensures a methodical review of the requirements to a set of criteria. 

Requirements Language Construct
The basis for a Requirements Language Construct may depend on the requirements gathering technique that is used.  There are many good techniques.  The better techniques ask the analyst to include various core elements to a requirements string.  Even if it is not in the form of text, but in diagram form (e.g., Use Cases), a good technique will ask for certain elements within the diagram.  These elements can form the core of a language construct.  The question becomes, how does a person approach establishing a Requirements Language Construct?  Key contributors to defining this include Analysts and Testers.  A sequence of activities would include:

Identify the elements of a Requirements Language Construct
Review some of the common requirements gathering technique.  Identify the elements that are typically considered when articulating a requirement.  When identifying elements from various techniques, some common and consistent elements appear.  For example, the actor is referenced in various techniques and so are the elements of action, system, and output.  Stringing these together establish a simple but solid requirements language construct that looks like: <actor> <action> <system> <expected output>. 

Identify elements of a Test Case
Once core elements are identified via a review of the requirements gathering techniques, review the Test Case approach and identify if there are any additional elements that are expected in the test case.  For example, the test case may want to know exactly where the expected output should live.  This will make it easier to know if the test case passes.  In this example, the additional element of location can be added.  Now the language construct would read:   <actor> <action> <system> <expected output> <location of output>.   

Incorporate Language Construct
Once a language construct has been defined, consider incorporating this into several areas.  The first is to incorporate the language construct into the Requirements Gathering procedure.  Secondly, incorporate the language construct into the Requirements List template as a prompt or reminder.  Finally, incorporate the language construct into the requirements technique training materials as appropriate. 

Please note that there may be a different language construct for some of the requirements.  For example, for a user requirement, the <actor> is important to know.  However for a performance requirement relating to systems, the <actor> may not be needed. 

Requirements Peer Review
There is a danger that peer reviews get seen as a nice informal exercise when there are no guidelines used to evaluate the work product.  It becomes essential that the requirement peer review has a formal closed loop process with criteria for evaluation.  Often times, this can be known as Requirements Inspection to promote its formality.  Key contributors to defining a Requirements Peer Review Practice would include Analysts and Testers.  The following provides outline of steps that can be followed

Identify concrete criteria in performing a Peer Review
Establish a Requirements Peer Review Practice that ensures a methodical review of the requirements that meet the guidelines of the requirements technique and the requirements language construct.  In this case, when a peer review occurs, ensure that all of the core elements of the language construct appear in each requirement. 

In addition to using the language construct as criteria, there are other criteria that can be added.  In his article, "On-Track Requirements: How to evaluate requirements for testability", Rodger Drabick suggests evaluating each requirements against several quality characteristics.  These characteristics include: complete, consistent, correct, not-ambiguous, and noncompound.  If a requirement meets these characteristics, then it can be considered testable.  Consider reading his article for more details.   

Identify key personnel to perform Peer Review
There are probably three key roles that should attend a requirements peer review session.  The first are other Analysts not involved on this project.  The second are testers since testers will have to test from the requirements so they will need to ensure that all elements are present for testability.  Thirdly, consider identifying key development personnel since they have to build the product release based on the requirement. 

Deploy a Requirements Peer Review Practice
As mentioned above, it is important to document the Requirements Peer Review Practice that has a formal closed loop process with criteria for evaluation.  In this case, the evaluation criteria are two-fold: 1) whether all of the core elements of the language construct are in place and 2) whether they meet the quality characteristics (both discussed above).  It is advisable to have a peer review worksheet with the criteria to evaluate against. 

Note: Two big factors to consider in deploying a peer review practice.  It does take time so ensure time is built into the schedule for this activity.  However, time expended in the peer review should reduce time in development and testing since requirements are clearer after the peer review.  Also, ensure that personnel attending the peer review are prepared.  They should have already walked thru the list of requirements against the criteria so that the peer review is a quick exercise of identifying areas where criteria is not met.      

Summary
Introducing a Requirements Gathering practice usually includes steps to identify requirements and document them.  To ensure that complete and quality requirements are delivered, consideration should be given to specifying what elements should be in each requirement (e.g., language construct), ensuring a full range of requirements categories are articulated (e.g., coverage), and establishing a requirements peer review practice with clear criteria.  This will help a project get the whole picture of the requirements with completeness and clarity.            

References
 
Mario Moreira is a Columnist for CMCrossroads Journal, a Director/Architect of Technology, an Author of CM publications, and has worked in the SCM field since 1986.  He has experience with numerous SCM technologies and processes and has implemented SCM on over 100 applications/products, which include establishing global SCM infrastructures.  He has an MA in Mass Communication with an emphasis on communication technologies.  Mario also brings years of Release Management, Project Management, Software Quality Assurance, Requirement Engineering, IT Governance, Architecture, agile, facilitation, and team building skills and experience. 

Mario has released a new SCM book entitled, "Software Configuration Management Implementation Roadmap".  It can be found at www.wiley.com, www.wileyeurope.com, and www.amazon.com (search for Mario Moreira).  It includes step-by-step guidance for implementing SCM at the organization, application, and project level with labor-saving quick-start templates on CD. 

You may reach Mr. Moreira by email at Mario.Moreira@cmcrossroads.com
Trackback(0)
Comments (0)add comment
feedSubscribe to this comment's feed

Write comment
smaller | bigger

security image
Write the displayed characters


busy
Last Updated ( Tuesday, 21 November 2006 )
 
< Prev   Next >
[ Back ]
If you already have an account on CM Crossroads, Login Now. If you do not, register using the link below...

NOTE: Once you register you will need to activate your account by clicking the link sent to you by email.