Re: A simple unit test framework
Ian Collins wrote:
Pete Becker wrote:
anon wrote:
Pete Becker wrote:
You can't do coverage analysis or any other form of white-box testing
on code that hasn't been written. There is a big difference between a
tester's minset and a develper's mindset, and it's very hard for one
person to do both.
The latest trends are to write tests first which demonstrates the
requirements, then code (classes+methods). In this case you will not
have to do a coverage, but it is a plus.
That's not what coverage analysis refers to. Coverage analysis takes the
test cases that you've written and measures (speaking a bit informally)
how much of the code is actually tested by the test set. You can't make
that measurement until you've written the code.
If you apply TDD correctly, you only write code to pass tests, so all of
your code is covered.
Suppose you're writing test cases for the function log, which calculates
the logarithm of its argument. Internally, it will use different
techniques for various ranges of argument values. But the specification
for log, of course, doesn't tell you this, so your test cases aren't
likely to hit each of those ranges, and certainly won't make careful
probes near their boundaries. It's only by looking at the code that you
can write these tests.
This way, the code you write will be minimal and easier to understand
and maintain.
I agree this way looks harder (and I am not using it), but I am sure
once you get use to it - your programming skills will improve drastically
When you write tests before writing code you're only doing black box
testing. Black box testing has some strengths, and it has some
weaknesses. White box testing (which includes coverage analysis)
complements black box testing. Excluding it because of some dogma about
only writing tests before writing code limits the kinds of things you
can discover through testing.
TDD advocates will (at lease they should) aways be acceptance test
advocates. These provide your white box testing.
Acceptance tests can be white box tests, but they can also be black box
tests.
The problem is that, in general, you cannot test every possible set of
input conditions to a function. So you have to select two sets of test
caes: those that check that mainline operations are correct, and those
that are most likely to find errors. That second set requires knowledge
of how the code was written, so that you can probe its likely weak spots.
Possibly, but doing so may not increase the code coverage. It may well
flush out failure modes. Once found, these should be fixed by adding a
unit test to reproduce the error.
Well, yes, but the point is that focused testing based on knowledge of
the internals of the code (i.e. white box testing) is more likely to
find some kinds of bugs than tests written without looking into the
code, which is the only kind of test you can write before you've written
the code.
--
-- Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com)
Author of "The Standard C++ Library Extensions: a Tutorial and
Reference." (www.petebecker.com/tr1book)