Measuring your code's test coverage is important, though not definitive metric to tell that Your code is well tested. But to measure coverage properly, tests have to be properly initialized as well. And today I asked myself a question, how do I measure properly test coverage of py.test plugins?

Py.test plugins are quite tricky, because they get initialized on test session start, that is even before coverage will kick in - even though pytest-cov since version 1.7 it kicks in in exact moment, thanks to a pytest_sessionstart hook.

And just today, I was toying with the idea looking first at recent changes in pytest-cov, and it occurred to me, that to get full coverage of pytest plugin, I actually have to turn it off!

py.test -p no:pytest-plugin

But how do we use plguin's fixtures and configuration then? Well, it's enough to import everything from the plugin in test's conftest.py file!

from pytest_plugin.plugin import *

This will make your plugin's fixture to kick in, as well as any configuration option your pytest plugin might require.

Results can be seen on coveralls.io for both plugins I contribute to: pytest-dbfixtures and pytest_pyramid

Of course, code coverage can tell you only, what percentage of code has been touched by tests. Not if it is properly tested!