Category: Uncategorized

Now that you have a clear understanding of how to systematically devise test cases based on specifications, here are a few pragmatic tips I have learned over the years.

The developers of the Apache Commons Lang framework (the framework where I extracted the implementation of the substringsBetween method) are just too good. We did not find any bugs there. Let’s look at another example: one implemented by me, an average developer who makes mistakes from time to time. This example will show you the value of…

I propose a seven-step approach to derive systematic tests based on a specification. This approach is a mix of the category-partition method proposed by Ostrand and Balcer in their seminal 1988 work, and Kaner et al.’s Domain Testing Workbook (2013), with my own twist: see figure 2.4. Figure 2.4 The seven steps I propose to derive test cases based on specifications.…

Being systematic is good, but we should never discard our experience. In this step, we look at the partitions we’ve devised and see if we can develop interesting variations. Variation is always a good thing to have in testing. In the example, when revisiting the tests, I noticed that we never tried strings with spaces.…

It is now time to transform the test cases into automated JUnit tests. Writing those tests is mostly a mechanical task. The creative part is coming up with inputs to exercise the specific partition and understanding the correct program output for that partition. The automated test suite is shown in listings 2.3 through 2.7. They…

With the inputs, outputs, and boundaries properly dissected, we can generate concrete test cases. Ideally, we would combine all the partitions we’ve devised for each of the inputs. The example has four categories, each with four or five partitions: the str category with four partitions (null string, empty string, string of length 1, and string of length > 1), the open category with four…

Bugs in the boundaries of the input domain are common in software systems. As developers, we have all made mistakes such as using a “greater than” operator (>) where it should have been a “greater than or equal to” operator (>=). Programs with such bugs tend to work well for most provided inputs, but they…

We should find a way to prioritize and select a subset of inputs and outputs that will give us sufficient certainty about the correctness of the program. Although the number of possible program inputs and outputs is nearly infinite, some sets of inputs make the program behave the same way, regardless of the precise input…

An ad hoc exploration of what the method does may increase your understanding of it. I have noticed this when observing professional software developers writing test cases for methods they have never seen before (Aniche, Treude, and Zaidman, 2021). This step is more relevant when you did not write the code—if you wrote it, this…

Regardless of how your requirements are written (or even if they are only in your mind), they include three parts. First is what the program/method must do: its business rules. Second, the program receives data as inputs. Inputs are a fundamental part of our reasoning, as it is through them that we can test the…