Re: Code coverage tool
On Feb 5, 9:25 pm, Ian Collins <ian-n...@hotmail.com> wrote:
James Kanze wrote:
On Feb 4, 7:22 pm, Ian Collins <ian-n...@hotmail.com> wrote:
My contention is *when* you write the tests is important, you
can only get full coverage without tools when they are written
first. The only production code that gets written is written
to pass tests.
This was, there will not be a branch unless it was required to
pass a test.
Which still doesn't begin to guarantee anything like full
coverage. Given a function which can be called with two
independent conditions, you still can easily end up with tests
that don't test all four combinations of the conditions.
It may sound like a vacuous answer, but we (my teams and I)
have found that writing test first results in a different
style of code where such cases tend not to occur.
I'm tempted to say: how do you know that? But you answer that
later.
I think we both agree that a complete set of tests is a
necessary (but obviously not sufficient) condition for quality
code. Some people may prefer writing them first, others later,
others (like myself) mixing the two. (A large part of the test
code is boring hack work, and I prefer not doing it in a single
block, but interspersing it with more interesting parts.) The
criterion for quality is not when the tests were written,
relative to the code, but that they were written, that they are
systematically run, after each modification, and that they are
relatively complete (to the degree that such a thing is
possible).
Running them systematically is trivial to enforce: almost all
version management systems have triggers which can be set to run
them every time the code is checked in. Ensuring that they are
reasonably complete, on the other hand, still requires manual
inspection, at least as far as I know. (As mentionned else
thread, my experience with coverage tools has been fairly
negative---they'd often report 100% coverage when some obvious
and important cases hadn't been tested.)
Not to mention tests that never test the "exceptional"
paths, e.g. when new throws bad_alloc, etc.
I enjoy reminding people about those! These conditions are
problematic with any form of unit test, another pair of eyes
(code reviews or pair programming with collective code
ownership) is a must to make sure they don't fall through the
cracks.
Another pair of eyes. Totally agreed. That is IMHO even more
important than the tests, and is the only way that I know of to
ensure that the tests are complete.
My own experience has been that these eyes are most effective
(by far!) if they are unjaundiced, if they haven't been involved
in the development of that particular bit of code before they
review it. Code review with at least one "independent"
reviewer, in sum.
There is no silver bullet, and writing the tests before
writing the code really doesn't buy you anything in general.
It may not be a silver bullet, but I have found writing the
tests as part of the design process does improve quality by
making a more pleasurable experience for the developers.
Other's experiences may vary, but all the teams I know who
have adopted this approach have stuck with it.
I've worked in places where that was the rule, but I can't say
that I found it very agreeable. I do believe, however, that you
should be responsible for the tests at the level you are
working. If you're writing code, you (and not someone else)
should be responsible for the unit tests. If you're doing
higher level design work, you should also be responsible for the
integration tests which are associated with it. Agreeable or
not:-).
After that, the order in which each programmer organizes his
work is up to him, as long as he does all the work he is
responsible for, and delivers it on time. (Note that in the
case of higher level design, the lower level coding can't begin
until the design has been delivered, and the associated
integration tests can't begin until the code has been delivered,
so the order design, then write the tests, is more or less
imposed.)
--
James Kanze (GABI Software) email:james.kanze@gmail.com
Conseils en informatique orient=E9e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S=E9mard, 78210 St.-Cyr-l'=C9cole, France, +33 (0)1 30 23 00 34