Re: A simple unit test framework
On May 7, 11:31 am, Ian Collins <ian-n...@hotmail.com> wrote:
James Kanze wrote:
On May 6, 10:01 pm, Ian Collins <ian-n...@hotmail.com> wrote:
I don't have the exact before figures,
Which is typical for most process improvements:-). At least at
the beginning; one of the problems in the initial process (or
lack of it) is that it doesn't generate figures.
but there were dozens of bugs in
the system for the previous version of the product and they took a
significant amount of developer and test time. The lack of unit tests
made the code extremely hard to fix without introducing new bugs.
Comprehensive unit tests are the only way to break out of this cycle.
Comprehensive unit tests are important for many reasons. I'm
not arguing against them. I'm saying that they aren't
sufficient, if they are the only measure. For that matter, how
do you know the tests are comprehensive? The only means I know
is to review them at the same time you review the code.
Because there were written first. We've been there before!
How does writing them first assure comprehensiveness?
The code was well reviewed there were six or eight on the team, we
rotated pairs and everyone worked on every bit of code.
Comprehensive unit tests are most important for maintenance.
New code should always be reviewed. But a review is not without
costs, and reviewing an entire module because someone has
changed just a couple of characters in just one line isn't
really cost effective. Comprehensive unit tests, on the other
hand, are very effective at catching slips of the finger in the
editor.
Again, that's where paring and continuous integration come in, the code
is under constant scrutiny from many pairs of eyes.
It's not a question of numbers. It's important that the
reviewer have a fresh view of the code.
We didn't bother tracking bugs for the replacement product, there were
so few of them and due to their minor nature, they could be fixed with=
in
a day of being reported. We had about 6 in the first year.
And why weren't they being tracked? 6 is, IMHO, a lot, and can
certainly be reduced. Find out why the bug crept in there to
begin with, and modify the process to eliminate it. (I've
released code where the only bug, for the lifetime of the
product, was a spelling error in a log message. That happened
to be the product where there weren't any unit tests, but
admittedly, it was a very small project---not even 100 KLoc in
all.)
The cause was too obvious, a couple of spelling mistakes and a couple of
bugs in a bit of code that crept in without tests. The latter was an
object lesson in following the process, the former a prime demonstration
of engineer's collective inability to spell! The remainder were in an
obscure protocol implementation we lacked the equipment to test. We had
a perfect implementation of our misunderstanding of the specification!
That, of course, is a frequent problem. Regardless of the
methodology:-).
Sounds like you weren't pairing.
We tried that, but found that it's no where near as cost
effective as good code review. The code review brings in an
"outside" view---someone who's not implicated in the code. In
practice, it turns out that it's this external viewpoint which
really finds the most bugs. And of course, code review costs
less than pairing (although not by a large amount).
Pairing is not just about review, but it brings a unique
energy into the programming process. Rotating pairs gives the
"outside" view, in our case, there wasn't anyone out side the
team to review the code.
I know what pairing is. I'm not saying that it is without
value. But it is very expensive, compared to the value it
brings.
--
James Kanze (Gabi Software) email: james.kanze@gmail.com
Conseils en informatique orient=E9e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S=E9mard, 78210 St.-Cyr-l'=C9cole, France, +33 (0)1 30 23 00 34