Different Individual Review Techniques

Collating your findings

The basic review process usually requires each participant to prepare for the team review. There are various individual review techniques⁶ that help to reveal defects. These techniques can be applied to all types of review although their effectiveness can vary depending on the type of review you are conducting.

Ad hoc

No rules

As the name suggests, this technique involves no predefined rules regarding individual preparation. The reviewer usually reads the work products sequentially and notes any issues, findings, or defects that are found. The type of documentation is also arbitrary. Because they require very little (or no) preparation, ad hoc reviews take place quite frequently and, because there are no rules involved, the results depend heavily on the skills of the individual reviewer. If a review object is reviewed by more than one person, some issues are sure to be flagged multiple times.

Checklist-Based

Using checklists

Using checklists is a great way to structure your reading and thus increase the effectiveness of an individual review. You can define which checklists are used by which reviewers. A review checklist contains a list of questions based on potential defects that have been discovered in the course of earlier projects. The questions can also be based on formal guidelines or standards that need to be observed, as illustrated by the following example.

Case Study: Using checklists

Checklist for reviewing a requirements document:

• Is the document’s structure standardized?
• Is there an explanation of how the document is to be used (i.e., an explanatory introduction)?
• Is a project summary included in the requirements document?
• Is a glossary available to aid consistent comprehension?
• …

Our example shows a checklist for reviewing a requirements document, but checklists can be formulated to aid reviewing any type of work product (for example, a code review). Checklists have to be updated regularly to include questions on newly recognized issues and to remove questions related to outdated or resolved issues. Checklists should generally be kept short and should only include key questions.

The main benefit of the checklist-based technique is that it systematically covers all typical types of defects. One potential downside of this technique is that it tends to “focus” the review on specific questions, thus reducing the likelihood of discovering defects that are not related to the listed questions. Always try to keep an open mind when using this technique.

Using Scenarios and Dry Runs⁷

Use case run-throughs

A scenario-based review is based on predefined, structured guidelines that dictate how to run through the review object. The availability of specific use cases simplifies this type of review, as all you have to do is perform “dry runs” of all the use cases. For example, in our VSR-II case study, we can check whether all the “configure vehicle” and “select special edition” cases are correctly implemented.

In the case of a code review, such scenarios can also be based on classes of defects. For example, one reviewer can concentrate on error handling in the code, while another concentrates on checking that boundaries are adhered to. Scenarios help a reviewer to find specific types of defects and are often more effective than simply working through lists of questions.

However, exclusive use of scenarios makes it more difficult to identify other types of defects (such as missing attributes), and should be avoided if possible.

Role- and Viewpoint-Based

Different roles and perspectives

During a role-based review, the reviewer is tasked with checking the review object from a specialist’s point of view. To do this, the reviewer has to either already work in the specialist role, or at least empathize strongly with it. The basic idea of a role-based review is to utilize the skills each role provides. Of course, the person playing a role has to have the appropriate skills. For example, it is always useful to have testers review requirements, as they can check early on in the development process whether the requirements are sufficiently precisely defined to effectively derive test conditions and test cases from them.

Utilizing specialist skills

Other typical review roles are those of stakeholders such as end-users or the operator of the system within the customer’s organization. If end-users or the system’s operator cannot take part in review sessions (which is usually the case), these roles have to be played by others. The “end-user” role can be further refined depending on whether the person is “experienced” or “inexperienced”. If the system is to be used within an organization, a reviewer can play the role of system or user administrator.

Case Study: A role-based review

A requirements document is due for role-based review. During the review, Joe Smith is assigned the role of designer. Ideally, Joe will know about software system design and will fulfill this role within the real-world project. Joe analyzes the requirements to see what changes are necessary to the system’s architecture in order for the requirements to be adequately implemented. Jane Brown is a tester and adopts this role for the review. During the review, she will check whether it is possible to unambiguously decide (following implementation and testing) that each requirement has been fulfilled. Jane flags the requirement “The system will enable rapid operation’ as insufficiently quantified. There are no numbers that can be measured, and there are no preconditions or constraints that determine specific situations in which timing can be considered adequate. Other roles need to be filled, too, in order to further scrutinize the requirements.

Different viewpoints

All the available roles point to different defects, inconsistencies, and gaps. Role-based reviewers take on the viewpoints of various stakeholders, thus examining different aspects of the system. The (often subjective) viewpoints of the stakeholders themselves need to be included in the review. The following example illustrates the differing interests that exist in a scheduling review for the next iteration in our case study project.

Case Study: A perspective-based review

The sales department view says that a particular requirement requires high priority in the requirements document, as this feature has been requested by an important customer and needs to be implemented as soon as possible. From the tester’s viewpoint, this feature is very similar to other features that have already been implemented, so there shouldn’t be any serious issues involved in implementing and testing it. She therefore recommends timely implementation of the new feature.

However, another new feature is more critical from a testing point of view, as it is the first in this project to implement artificial intelligence (AI). So far, the team has no experience in this field. The tester therefore assumes that this feature will require a lot of implementation and testing effort, so more resources need to be assigned to it.

The basic principle that underlies a perspective-based review is that each reviewer looks at the review object from a different viewpoint and runs through different scenarios as a result. The findings can then be used not only to evaluate the document in question but also to estimate the urgency and scope of any required changes. To do this, each reviewer runs through a different scenario⁸ and uses the results to evaluate the document under investigation. Utilizing different stakeholder perspectives increases the depth of each individual review, and the distribution of roles reduces duplicate naming of defects and other issues. Using checklists is also a great way to increase the effectiveness of this kind of review.

Empirical studies (see [Sauer 00] and [Shull 00]) view perspective-based reviews as the most effective way to check requirements and technical descriptions. One of the keys to its success is the inclusion of many stakeholder viewpoints when analyzing and evaluating risk.