Re: Dos and don'ts in C++ unit testing?
Maciej Sobczak wrote:
James Kanze wrote:
There's hyberpole to the statement, but there's a grain of truth.
Sure you need to do some testing, but it shouldn't be a substitute for
good design, careful coding, or early formal inspection.
Testing is necessary, but not sufficient.
Which immediately brings a question that if there is "something else"
that is sufficient, then maybe testing is not necessary? :-)
In theory, or in practice? In theory, good design and careful
code review can result in code so good that the tests never
reveal any errors, and so shouldn't be necessary. In practice,
how do you know that your design is that good and your code
reviews are that careful? The fact that testing reveals no
errors can be an important indication that you are doing things
right. (Of course, it can also be an important indication that
your test procedures aren't very good either:-). If testing
reveals no errors, all you know is that you are at one of the
extremes; you'll need other indicators to tell you which.)
OK, OK - the "sufficient" part can be achieved by combining various
techniques. No silver bullets, as usual.
No absolute rules, either. I once (about five years ago) did a
project with absolutely no tests; I only got my first complete
link a week before delivering it. After deliverly, the most
serious error that was found in it was a spelling error in one
of the log messages. It's not a procedure that I'd recommend,
but it happened to work that one time. (There was no code
review or design review either, which makes me think that the
fact that it worked was close on to a miracle.)
The
tests aren't there to catch errors in the code, but to keep the
developers honest in the other parts.
This is very reasonable, but also brings another question - if tests are
supposed to be the "discipline" motivators for designers and developers,
then should they write the tests by themselves?
Good question. I've worked in organizations where tests were
the responsibility of the architecture team, who specified what
the class should do. At the very least, of course, the test
suite must be reviewed with the rest of the code.
If the developer writes both the test and the code (which is what is
advocated by many), then it doesn't have this motivating effect, because
developers will know the extent to which they can be "honest enough". In
other words, by writing the test they will be able to decide how much
honesty they will require from themselves later on.
Independantly of honesty, I've generally found that if I've
forgotten to handle a case in the code, it's because I've
forgotten the case, and won't have tested it either.
Review is good here, since it is often easier to recognize that
a test case has been left out, than it is to spot an error in
handling the case in the actual code.
When working alone (as in the project mentionned above), I try
to put code I've written aside, do something else, and then come
back to it, when my "preconceptions" are no longer fixed in my
head (hopefully). I also find it useful to pretend I'm
explaining it in detail to a collegue, even if that collegue
doesn't exist. Trying to explain it in terms someone not
familiar with the code would understand often helps.
With this in mind I'd argue that some additional benefits can be reached
when developers do not know what the tests are (did I say - "black-box
tests?" ;-) ) - then they will be as honest as possible. But this brings
us back to the isolated QA teams in black suits and the "us and them"
mindsets.
I've heard of companies where the programmers weren't even
allowed to compile the code. It does make them carefuler about
typing errors, if nothing else.
Other than that: probably the most important role of management
is preventing this "us and them" mindset. The programmer should
get a positive feeling from code review---it's not personal
criticism, but a chance for him to learn something. Or even to
show off a little.
--
James Kanze (GABI Software) email:james.kanze@gmail.com
Conseils en informatique orient?e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S?mard, 78210 St.-Cyr-l'?cole, France, +33 (0)1 30 23 00 34
--
[ See http://www.gotw.ca/resources/clcm.htm for info about ]
[ comp.lang.c++.moderated. First time posters: Do this! ]