Re: Questions about "mismatch"
On Dec 19, 9:37 pm, Ian Collins <ian-n...@hotmail.com> wrote:
On 12/20/10 09:24 AM, jacob navia wrote:
Besides, as you know well too, the unit tests never catch all branches
and all code in complex functions. They give you only a FALSE sense
of security. Yes, they are useful (I will never deny that), but they do
not catch all errors.
If the tests are written first, the branch won't exist unless it is
there to pass a test!
Three comments:
-- Are you sure? Exceptions introduce a lot of additional
branches, that aren't in the normal program logic (but which
have to be tested). It's very easy to write a test (or
a series of tests) for a factory function that tests that it
always returns a valid object, with values corresponding to
what they should be (at least in some trivial cases). But
writing such tests first does nothing to ensure that the
case when new throws bad_alloc has been correctly handled.
(It can, of course, but it's not "automatic", just because
you write the test first.)
-- I'm not sure what he means by "all the branches". To give
a simple and somewhat artificial case:
void f(bool c1, bool c2)
{
if (c1)
doA();
else
doB();
if (c2)
doC();
else
doD();
}
"Obviously", you need four tests to test all of the branches
(unless some of the functions can throw---then you likely
need more). Regardless of whether you write the tests
before or after, it's easy to only write two or three cases;
some coverage tools I've seen report full coverage with just
f(false, false) and f(true, true).
Similar issues arrive with executing loops 0 times (which is
probably a more realistic example).
-- Finally, some problems (including most involving floating
point) aren't linear; for a lot of numerical work, for
example, just testing the end points and a couple of values
between them offers no guarantee that the implementation is
stable for all possible input. Thread safety has similar
issues: to be complete, you'd have to ensure that there was
a test for a thread switch between every machine
instruction. (This doesn't mean that you don't need the
tests. Just that they aren't sufficient.)
Thorough testing is important, and I strongly disagree with the
posting you are responding to: just because it can't be 100%
perfect is no reason to skimp on it (and you should be able to
reasonably address my first two points above---if the
development process decides to address the issues). But writing
tests first isn't a silver bullet; I'd argue, in fact, that it's
not important when they're written, as long as they *are*
written. (If requiring tests to be written first is the only
way to ensure that they are written, of course, then that's the
route you have to go. I've not found that to be the case,
however.)
--
James Kanze