The Differences Between Static and Dynamic Testing

Identifying different types of defects

Static and dynamic tests can be used to achieve the same objectives but complement each other by identifying different types of defects. Static tests identify faults directly in documents and other work products, whereas dynamic tests usually identify failures in the source code rather than other types of documents (with the exception of executable models). Any failures you discover have to be traced back to the faults that cause them if they are to be successfully resolved. Failures often remain undiscovered for a long time—for example, if the corresponding code is only rarely executed. It therefore requires a lot of effort to design appropriate test cases. These kinds of faults in the source code are usually easier to identify using static tests.

Checking “inner” quality

Dynamic testing verifies the visible, external behavior of the test object, whereas static tests focus on improving the internal quality of the object under investigation.

Typical defects found by reviews

The typical defects and inaccuracies that can be identified quickly and cheaply, and therefore quickly resolved, using reviews:

• Requirements defects
  Typical defects found in requirements documents are: inconsistencies, ambiguities, contradictions, gaps, inaccuracies, or redundancies.
• Software design defects
  Where defects are identified in the architecture documents that specify the system’s internal interfaces. When the degree of dependency (or “coupling”) between individual components or modules is high, it can be difficult to fully understand and test them, not least because the effort involved in creating a corresponding test environment is disproportionately to the importance of the results. The cohesion between components/modules can also be analyzed. A strongly coherent component is responsible for a single, clearly defined task.A component that performs a loose selection of tasks indicates low cohesion, which is just as disadvantageous as a high degree of coupling. Algorithms and database structures that have been inefficiently designed can be identified too.
• Coding defects
  If you have sufficient time and appropriately skilled personnel, all programming faults can be identified using code reviews. This enables you to detect variables with no value or redundant variables that are never called. You can also use a review to detect unreachable or duplicate code. These types of defects can also be detected by a compiler or by using automated static analysis tools, which is often quicker and cheaper than a human-based technique.
• Deviations from standards
  Standards and guidelines help to achieve high quality products, while insufficient adherence to programming guidelines leads to the opposite, namely: poor quality. If it is clear that adherence to certain guidelines will be checked, the motivation to stick to them from the start is much greater.
• Faulty interface specifications
  The names of the interfaces between components and their parameters (number, data type, and sequence) must be checked individually, as not all discrepancies will be found during integration.

Example: Incorrect interface definitions

The NASA Mars probe Mars Climate Orbiter is a famous and well-documented example of incorrectly specified interfaces [URL: Mars Climate Orbiter]. The system programmers used multiple measuring systems (metric and imperial), and this oversight led to the loss of the probe. A formal code review would probably have identified this inconsistency in the measurement system’s specifications.

• Security vulnerabilities¹⁴
  Alongside dynamic security tests, static tests can also help uncover these kinds of vulnerability. For example, when buffer overflows occur because constraints weren’t explicitly checked, or when input data or SQL queries are deliberately manipulated.
• Gaps or inaccuracies in test basis traceability or coverage
  We discussed the importance of traceability and degree of coverage. Gaps in traceability make traceability itself useless, as you can no longer make a connection between the requirements and the corresponding test cases. Inaccuracies in the exit criteria and the required degree of coverage make these aspects unusable too, and can lead to incorrect estimates or distorted results. A review report can, for example, point out missing tests for fulfilled exit criteria.

Maintainability

Software systems often have surprisingly long lifecycles, making maintainability an important aspect of any system. Most shortcomings in maintainability can be identified using static tests. These include incorrect modularization (see coupling and cohesion above), bad reusability of individual components, and complex code that is difficult to modify (i.e., not “clean code” as defined in [Martin 08]) and therefore increases the risk of creating new faults when changes are made.