Re: A simple unit test framework
On May 7, 12:48 am, Gianni Mariani <gi3nos...@mariani.ws> wrote:
James Kanze wrote:
On May 6, 3:03 am, Gianni Mariani <gi3nos...@mariani.ws> wrote:
James Kanze wrote:
...
Yes, but nobody but an idiot would pay you for such a thing.
Thread safety, to site but the most obvious example, isn't
testable, so you just ignore it?
Common misconception.
1. Testability of code is a primary objective. (i.e. code that can't be
tested is unfit for purpose)
2. Any testing (MT or not) is about a level of confidence, not absolut=
eness.
I have discovered that MT test cases that push the limits of the code
using random input does provide sufficient coverage to produce a level
of confidence that makes the target "testable".
Then write it up, and publish it, because you have obviously
struck on something new, which no one else has been able to
measure.
I just did here...
I certainly didn't see it. All I saw were some wild,
unjustified claims.
... But you must mean something else by Monte Carlo
testing than has been meant in the past. Because the
probability of finding an error by just throwing random data at
a problem is pretty low for any code which has passed code
review.
For a large class of real world problems it works just fine.
For what definition of "works".
I guess it depends on the level of quality you expect.
[...]
... And in
some cases, the actual situation an only occur less often than
that: there is a threading error in the current implementation
of std::basic_string, in g++, but I've yet to see a test program
which will trigger it.
What's the error ?
Bug 21334. Found by code review; I've yet to be able to create
a test which systematically triggers it. (On the other hand, it
does cause problems in real code.)
If you are able to push randomness, how many times do you need
to reschedule one thread to hit a potential problem.
Practically an infinity.
Really. Well write better code. Again, if you can't test it, by
definition it's bad code.
So you exclude multithreading and floating point. Since neither
can really be tested.
Given cache latencies, pre-emption from other threads, program
randomness (like memory allocation variances) you can achieve
pretty close to full coverage of every possible race condition
in about 10 seconds of testing. There are some systematic
start-up effects that may not be found, but you mitigate that
by running automated testing. (In my shop, we run unit tests
on the build machine around the clock - all the time.)
So perhaps, after a couple of centuries, you can say that your
code is reliable.
In practice it usually finds problems in milliseconds.
So you write bad code to begin with. Or simply prefer to ignore
real problems that don't show up trivially.
--
James Kanze (GABI Software) email:james.kanze@gmail.com
Conseils en informatique orient=E9e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S=E9mard, 78210 St.-Cyr-l'=C9cole, France, +33 (0)1 30 23 00 34