Be careful with methods that are covered but not tested

Larger tests exercise more classes, methods, and behaviors together. In addition to all the trade-offs discussed with larger tests, the chances of covering a method but not testing it are much higher.

Vera-Pérez and colleagues (2019) coined the term pseudo-tested methods. These methods are tested, but if we replace their entire implementation with a simple return null, tests still pass. And believe it or not, Vera-Pérez and colleagues show that pseudo-tested methods happen in the wild, even in important open source projects. This is another reason I defend both unit tests and larger tests, used together to ensure that everything works.


Comments

Leave a Reply

Your email address will not be published. Required fields are marked *