Category: Uncategorized

Now that you have a clear picture of structural testing, the coverage criteria you can use for guidance, and how to use structural testing in combination with specification-based testing, let me discuss a few interesting points.

If code is the source of all truth, why can’t we just do structural testing? This is a very interesting question. Test suites derived only with structural testing can be reasonably effective, but they may not be strong enough. Let’s look at an example (see the “counting clumps” problem, inspired by a CodingBat assignment. The…

The most challenging part of specification-based testing is identifying boundaries. They are tricky to find, given the way we write specifications. Luckily, they are much easier to find in source code, given how precise code has to be. The idea of identifying and testing on and off points fits nicely in structural testing. For example,…

Let’s try specification-based testing and structural testing together on a real-world example: the leftPad() function from Apache Commons Lang. Left-pad a string with a specified string. Pad to a size of size. The method returns a left-padded string, the original string if no padding is necessary, or null if a null string is input. For example, if we give “abc” as the string…

You may have noticed that some of the criteria we have discussed are more rigorous than others. For example, a single test is enough to achieve 100% line coverage, but two tests are needed for 100% branch coverage. Some strategies subsume other strategies. Formally, a strategy X subsumes strategy Y if all elements that Y exercises are…

You may wonder what to do in the case of loops, such as for and while. The code block inside the loop may be executed different numbers of times, making testing more complicated. Think of a while(true) loop, which can be non-terminating. To be rigorous, we would have to test the program with the loop block executed one time, two times,…

The question is how to (mechanically) select such test cases. Let’s continue using the same if statement from the CountWords program (from listing 3.4). The statement takes three booleans as input: (1) whether the current character is a letter and whether this letter is (2) “s” or (3) “r”. Generically, this is the same as the A && (B || C) example we just discussed.…

Let’s take a simple abstract example: if(A && (B || C)), where A, B, and C all evaluate to booleans. MC/DC dictates the following: If conditions have only binary outcomes (that is, true or false), the number of tests required to achieve 100% MC/DC coverage is N + 1, where N is the number of conditions in the decision (as shown by Chilenski [2001]). Note that N…

Devising test suites that maximize the number of bugs they can identify while minimizing the effort/cost of building the test suite is part of any tester’s job. The question is, what can we do about complex, lengthy if statements? Modified condition/decision coverage (MC/DC) is a good answer. The MC/DC criterion looks at combinations of conditions, as path…

A developer aiming for path coverage covers all the possible paths of execution of the program. While ideally this is the strongest criterion, it is often impossible or too expensive to achieve. In a single program with three conditions, where each condition could be independently evaluated to true or false, we would have 23 = 8 paths to cover. In…