Code Coverage – what are we talking about?

Today an interesting discussion about the definition of code coverage took place in the JMockit Users group. I hope my english was good enough to follow the details and maybe contribute something useful. 😉

There was a bit of a confusion about the kind of coverage analysis JMockIt does for its coverage reports. For me, as an ISTQB certified tester, the german terms Anweisungsüberdeckung, Zweigüberdeckung and Pfadüberdeckung have a clear meaning. It’s based on the ISTQB definitions. The german Wikipedia article Kontrollflussorientierte Testverfahren has a good explanation of all the different coverage analysis. (Including the Bedingungsüberdeckung, which you can have the most fun with. And with fun I mean sleepless nights of whitebox testing… ;-))

Unfortunatly most of the tools doing some kind of code coverage analysis try to explain their metrics without the use of control flow graphs or any other kind of graphical representation. I think this adds to the confusion.

Using real world terms like branch and path don’t help to clear things up either. That 100% branch coverage means empty branches too isn’t very clear for the beginner who thinks of a branch as a real world thing.

After all this discussion shows once again, that you really have to know the metrics you use or chance is good you’re measuring the wrong things.