If two testers performed the specification-based testing technique I described earlier in the same program, would they develop the same set of tests? Ideally, but possibly not. In the substringsBetween() example, I would expect most developers to come up with similar test cases. But it is not uncommon for developers to approach a problem from completely different yet correct angles.
I am trying to reduce the role of experience and creativity by giving developers a process that everybody can follow, but in practice, experience and creativity make a difference in testing. We observed that in a small controlled experiment (Yu, Treude, and Aniche, 2019).
In the substringsBetween() example, experienced testers may see more complicated test cases, but a novice tester may have difficulty spotting those. A more experienced tester may realize that spaces in the string play no role and skip this test, whereas a novice developer may be in doubt and write an extra “useless” test. This is why I like the specification-based testing systematic approach I described in this chapter: it will help you remember what to think about. But it is still up to you to do the thinking!
Leave a Reply