As anyone who's contributed to Twisted knows, we currently have a very high standard for code changes and additions. For a long time, we've relied mainly on the standard library trace module. The trace module is quite serviceable, but recently some other options have come along which are much better. In particular, coverage.py seems to provide enough additional features that it's actually worth integrating into Twisted's development process. For example, it has some basic branch coverage features (trace will show you what lines were executed; coverage.py will show you this, plus tell you whether both possible outcomes of your "if" statement were taken), and can generate fairly nice HTML reports.
So, as a first step, I've set up a coverage.py builder on Twisted's BuildBot. This means that code coverage reports are part of Twisted's continuous integration system now: for every trunk revision, new coverage reports will be generated. At the moment, these reports end up jumbled up in the builds directory along with a lot of other stuff. For example, here's the latest report as of this post. Something I still want to do is clean up how these reports are organized so they're easier to find and compare.
As you can see from this report, Twisted has 86% line coverage and 67% branch coverage (which I had to compute myself from the totals at the bottom of the page - one of a few coverage.py nits that I hope will be fixed soon :). coverage.py decides this makes overall coverage 83%, though I'm not sure how exactly it arrives at that number.
That's pretty good, but there's plenty of room for improvement. I hope these reports will become an easy way for people to find areas of Twisted which need more coverage and motivate them to contribute new tests.
Oh, and since coverage.py only reports on Python coverage, I've also set up a gcov/lcov builder which uploads its coverage reports for execution of the C code in Twisted to a similar location.
No comments:
Post a Comment