Step 7: Augment the test suite with creativity and experience

Being systematic is good, but we should never discard our experience. In this step, we look at the partitions we’ve devised and see if we can develop interesting variations. Variation is always a good thing to have in testing.

In the example, when revisiting the tests, I noticed that we never tried strings with spaces. I decided to engineer two extra tests based on T15 and T20, both about “str contains both open and close tags multiple times”: one for open and close tags with lengths 1, another for open and close tags with larger lengths. These check whether the implementation works if there are whitespaces in the string. You see them in listing 2.8.

NOTE It’s possible we don’t need to test for this extra case. Maybe the implementation handles strings in a generic way. For now, we are only looking at the requirements, and testing special characters is always a good idea. If you have access to the implementation the code can help you decide whether a test is relevant.

Listing 2.8 Tests for substringsBetween using parameterized tests, part 6

@Test
void openAndCloseOfLength1() {
  // ... previous assertions here
  assertThat(substringsBetween("abcabyt byrc", "a", "c"))
    .isEqualTo(new String[] {"b", "byt byr"});
}
 
@Test
void openAndCloseTagsOfDifferentSizes() {
  // ... previous assertions here
  assertThat(substringsBetween("a abb ddc ca abbcc", "a a", "c c")).
  ➥ isEqualTo(new String[] {"bb dd"});
}

We end up with 23 test cases. Take time to revisit all the steps we have worked through, and then consider this question: are we finished?

We are finished with specification testing. However, we are not done testing. After specification testing, the next step is to bring the implementation into play and augment our test suite with what we see in the code.

Four eyes are better than two

What about a test case where the input is aabcddaabeddaab, open is aa, and close is d? “bc” and “be” are the substrings between the provided open and the close tags (aa<bc>ddaa<be>ddaab), but “bcddaabed” could also be considered a substring (aa<bcddaabed>daab).

At first, I thought I had missed this test case. But in fact, it is the same as T15 and T20.

Different people approach problems in different ways. My thought process was, “Let’s see if the program breaks if we have multiple open and close tags in the string.” The reviewer may have thought, “Let’s see if the program will incorrectly go for the longer substring.”

We want to make testing as systematic as possible, but a lot depends on how the developer models the problem. Sometimes you will not see all the test cases. When you do come up with a new test, add it to the test suite!