Category: Uncategorized

This is a popular question among practitioners and researchers. If we settle for a less-rigorous criterion, such as line coverage instead of branch coverage, we might miss something. Plus this question brings the focus back to the metric, which we do not want. Which criterion to use depends on the context: what you are testing…

I have purposefully skipped talking much about achieving 100% line coverage or branch coverage or other coverage. I do not believe that achieving a number should be the goal. Nevertheless, given how prevalent those numbers are in practice, it is important to understand them. First, let’s talk about the metrics themselves. NOTE Formulas vary among the…

I find it interesting that some people rage against code coverage. A prevalent opinion is, “If I write a test case with no assertions, I achieve 100% coverage, but I am not testing anything!” This is true. If your tests have no assertions, they do not test anything, but the production code is exercised. However,…

Now that you have a clear picture of structural testing, the coverage criteria you can use for guidance, and how to use structural testing in combination with specification-based testing, let me discuss a few interesting points.

If code is the source of all truth, why can’t we just do structural testing? This is a very interesting question. Test suites derived only with structural testing can be reasonably effective, but they may not be strong enough. Let’s look at an example (see the “counting clumps” problem, inspired by a CodingBat assignment. The…

The most challenging part of specification-based testing is identifying boundaries. They are tricky to find, given the way we write specifications. Luckily, they are much easier to find in source code, given how precise code has to be. The idea of identifying and testing on and off points fits nicely in structural testing. For example,…

Let’s try specification-based testing and structural testing together on a real-world example: the leftPad() function from Apache Commons Lang. Left-pad a string with a specified string. Pad to a size of size. The method returns a left-padded string, the original string if no padding is necessary, or null if a null string is input. For example, if we give “abc” as the string…

You may have noticed that some of the criteria we have discussed are more rigorous than others. For example, a single test is enough to achieve 100% line coverage, but two tests are needed for 100% branch coverage. Some strategies subsume other strategies. Formally, a strategy X subsumes strategy Y if all elements that Y exercises are…

You may wonder what to do in the case of loops, such as for and while. The code block inside the loop may be executed different numbers of times, making testing more complicated. Think of a while(true) loop, which can be non-terminating. To be rigorous, we would have to test the program with the loop block executed one time, two times,…

The question is how to (mechanically) select such test cases. Let’s continue using the same if statement from the CountWords program (from listing 3.4). The statement takes three booleans as input: (1) whether the current character is a letter and whether this letter is (2) “s” or (3) “r”. Generically, this is the same as the A && (B || C) example we just discussed.…