Test Organization

Independent Testing

A software product has to be tested throughout its lifecycle so testing activities have to be well coordinated with all other development activities. The simplest and most obvious solution is to have the developers do their own testing. However, this approach typically leads to developers being “blind to their own mistakes”, so it is usually beneficial and more efficient to have separate development and testing teams that are independently organized:

Benefits of independent testing

• Testing teams that carry the responsibility for their own actions are more objective (less biased and with other technical or specialist viewpoints) and are well qualified to find additional defects in a test object that developers overlook.
• Independent testers view assumptions that are implicit in the specifications or coding of a test object more critically, and are able to verify or disprove such assumptions from a neutral point of view.

Drawbacks of independent testing

Potential drawbacks of separate development and testing teams are:

• If the teams are too isolated from one another, this can hinder communication between them and, in extreme cases, cause unnecessary animosity.
• Independent testers might not have sufficient knowledge of the test object or might not have access to enough technical or specialist information to test efficiently and effectively.
• If there aren’t enough dedicated resources, testing can become a bottleneck in the development process and delay planned releases. Even if this isn’t the case, testing may be blamed for project delays.
• Developers’ own feeling of responsibility for product quality may decline, as they simply assume that the test team will find any errors they make.

However, these potential disadvantages can be mitigated and the advantages retained by selecting a degree of independence between the teams that is suited to the project at hand and the test level concerned. The following options for increasing independence between developers and testers are all models that have been proven in everyday practice:

Independent testing models

1. Developer testing
   Testing is performed by the developers and no separate “tester” role is established within the team. Nevertheless, “blindness to your own mistakes” can be reduced and testing effectiveness increased using “buddy” or “pair testing”—i.e., developers only test code written by a colleague, never their own.
2. Independent testers
   There is a separate “tester” role. Team members who fulfill this role are assigned exclusively to testing and perform all testing activities within the team.
3. Independent testing teams
   There are dedicated testing teams within the company or the project with their own team leaders or sub-project leaders who report to company (department) management or project management. For extra support, members of other specialist or IT departments can be assigned to these teams on a case-based or temporary basis.
4. Test specialists
   There is a pool of colleagues with training and specialist knowledge of specific aspects of testing (industry, application, or departmental know-how) or test types (for example, non-functional characteristics such as performance, usability, security, or conformity with standards and regulations). These people can be assigned to a project temporarily or as needs be. They can then perform specialized test activities or support the team with training, coaching, or help with test automation and other, similar tasks.
5. External service providers
   An external service provider takes over all or much of the testing activity for a company or individual project (for example, system testing). Such work can be carried out in-house (insourcing) or at an external site of the provider’s choice (outsourcing).

Side Note: When to use which model

Which of the models listed above is most suitable depends largely on the test level in question:

• Component testing
  This needs to be done in close cooperation with development work, and is thus often covered using developer testing (see model 1 above). Buddy testing is usually possible and helps to increase product quality. Model 2 is useful if enough dedicated testers can be assigned to the task. Both models nonetheless carry the risk of those involved seeing themselves as developers who end up neglecting the testing activities assigned to them. There are two ways to mitigate this:

Our Tip Define testing plans in advance and organize appropriate coaching

• Project or test management predefines a testing plan and demands appropriate test documentation from the developers.
• Test specialists are assigned (at least some of the time) as coaches to provide methodical support.

• Integration testing
  If components that were developed by the same team need to be integrated and tested, you can apply a technique that is analogous to conventional component testing (models 1 and 2). However, if you are integrating components built by separate teams, you need either a “mixed” integration team with members from both development teams, or an independent integration team. A single integration team would have a one-sided view of its own components and is therefore likely to overlook existing faults. Depending on the size of the project, models 3, 4, or 5 are better suited to a situation like this.
• System testing
  Here, the finished product has to be viewed from the customer and end-user points of view, so independence from the development team is essential. For this kind of situation, only models 3, 4, or 5 should be considered.

The development model used by the company or for the current project will of course influence the degree of independence of the testing process.

Independent testing in projects developed using the V-model

The V-model stipulates explicit and strong separation of development and testing. Models 3-5 listed above are typical testing methods used in projects developed according to the V-model.

Independent testing in agile projects

In contrast, agile projects emphasize close, cross-functional cooperation in small teams. Some agile teams interpret this to mean that independent testing is simply “not agile” or that it “hinders agility”. These kinds of teams use only models 1 and 2 based on the assumption that only these methods are truly agile. However, this approach automatically excludes the potential benefits of independent testing performed according to models 3, 4, and 5 (see [Linz 14]).

Not only can models 3-5 be used successfully in an agile environment, they can even increase the agility of a team:

• The short-term iterations demanded by agile development processes require automated tests that are continually maintained and expanded. Team members or service providers who specialize in test automation will be able to do this efficiently and professionally, and the same is true when you are setting up and maintaining tool-based support for continuous integration. Model 2 or model 5 (together with an external service provider) provide the best support for this approach.
• You can increase the efficiency of large-scale projects by bundling test automation and integration tasks for multiple components, and assigning the resulting workload to a service team that works according to models 3 or 5.
• Agile teams can (and should) use in-house or external test specialists (models 4 and 5) to coach them and regularly check and improve their test techniques and test design skills. Many non-functional tests aren’t executed for each iteration, and temporary test specialists can perform such tests faster, more professionally, and more cheaply.
• The product owner—perhaps supported by experts from the departments involved (model 4)—performs (exploratory) acceptance tests at the end of each iteration to validate the resulting increment compared with the original user stories.

Case Study: Test organization within the VSR-II project

Component testing for the VSR-II project is the responsibility of the development teams and is organized according to models 1 and 2 as described above. An independent testing team has been formed alongside the development teams, and is responsible for integration and system testing. Figure 6-1 illustrates the setup.

Fig. 6-1How VSR-II testing is organized

Two or three members of each department (sales, marketing, etc.) are assigned to the task of functional testing for the corresponding modules (ContractBase, DreamCar, etc.). These people know all about the business processes implemented in “their” modules and know which requirements the end-users expect. They are all experienced computer users but are not IT experts. Their task here is to help the testing specialists specify functionally meaningful test cases and to execute the resulting tests. They have all been appropriately trained in the basics of testing (processes, specification, execution, documentation).

The testing team also includes three, four, or five additional IT and testing specialists who are responsible for integration, non-functional tests, test automation, and test tool support (see “technical tests” in the chart above).

The testing team is led by a test manager who is responsible for test planning and control. The test manager’s other tasks include coaching members of the testing team, especially those testing the functional requirements.