Re: A simple unit test framework
On May 7, 2:02 am, Branimir Maksimovic <b...@hotmail.com> wrote:
On May 6, 3:03 am, Gianni Mariani <gi3nos...@mariani.ws> wrote:
James Kanze wrote:
...
Yes, but nobody but an idiot would pay you for such a thing.
Thread safety, to site but the most obvious example, isn't
testable, so you just ignore it?
Common misconception.
1. Testability of code is a primary objective. (i.e. code that can't be
tested is unfit for purpose)
2. Any testing (MT or not) is about a level of confidence, not absolute=
ness.
I have discovered that MT test cases that push the limits of the code
using random input does provide sufficient coverage to produce a level
of confidence that makes the target "testable".
I have exactly opposite experience. Multi threading program failures
show up only with specially crafted input.
That is I have to look into code first and then make tests to
specifically target places in code .
But, such errors are usually most simplistic ones, those are common
threading errors, and can be usually catched by eye.
Even when you know exactly where the error is, it's sometimes
impossible to write code which reliably triggers it. Consider
std::string, in g++: if you have an std::string object shared by
two threads, and one thread copies it, and the other does [] or
grabs an iterator at exactly the same time, you can end up with
a dangling pointer---an std::string object whose implementation
memory has been freed. The probability of doing so, however, is
very small, and even knowing exactly where the error is, I've
yet to be able to write a program which will reliably trigger
it.
Consider some variations on DCL. In at least one case, you only
get into trouble if one of the processors has read memory in the
same cache line as the pointer just before executing the
critical code. Which means that the function can work perfectly
in one application, and fail when you link it into another
application.
As I mentionned in a response to another poster, it may just
depend on what you consider acceptable quality. I've worked on
critical systems a lot in the past. With contractual penalties
for downtime. So I tend to set my standards high.
Interestingly enough, however, it turns out that developing code
to such high standards is actually cheaper than just churning it
out. As a rough, back of the envelope figure: correcting an
error found in code review costs one tenth of correcting the
same error found in unit tests, which costs one tenth of
correcting the same error found in integration tests, which
costs one tenth of correcting the same error found in the field.
Not to mention the advantages of the transfer of knowledge which
takes place in code review. The result is that any time an
integration test finds an error, it is considered first and
foremost an error in the process; we analyse the process to find
out where it failed. And there is even a tendancy to do this
for unit tests, although perhaps not as systematically.
[...]
It does require a true multi processor system to test adequately.
Not just mp system, but *all* possible mp systems, which is
of course impossible.
It's perhaps worth pointing out that most current low-end MP
systems use a more or less synchronized memory model. This is
not true, however, for Alpha processors, nor the Itanium, nor, I
think, top of the line Sparcs, nor, probably future desktop
processors. That means that the fact that code works on a
current four processor system based on Intel 32 bits means
nothing with regards to future processors.
--
James Kanze (GABI Software) email:james.kanze@gmail.com
Conseils en informatique orient=E9e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S=E9mard, 78210 St.-Cyr-l'=C9cole, France, +33 (0)1 30 23 00 34