Re: Running regression tests in batch mode
On 15 Feb, 22:07, Bruce Wheeler <bswheele...@hotmail.com> wrote:
On Mon, 15 Feb 2010 02:42:28 -0800 (PST), James Kanze
<james.ka...@gmail.com> wrote:
How do you get a memory fault to simply return a bad return
code under Windows?
I'm interested in writing regression test programs which run
in batch mode. At present, if my code has an assertion
failure, I get a pop-up window with an offer to go to the
debugger---this will be worthless on the remote machine (and
is a pain even on my machine, if I happen to hit a sequence
of tests which each trigger the same error).
The dialog box is called from abort(), which is called by
assert(). Maybe there's a way to change this behavior, but
I'm not aware of a way.
I would expect abort() to raise a signal, as specified in the C
and C++ standards, and that it was the default signal handler
which called up the dialog box, or the system, when a signal is
uncaught.
A kludge for the assert problem could be something like
void myAssert(int expression)
{
/* text output and possible other processing */
exit(EXIT_FAILURE);
}
and call myAssert() instead of assert(). Note that exit(n)
exits with status n and does not create the dialog box.
The problem isn't just assert---it's anything which would have
caused the program to crash. If a program crashes during a
regression test (which is run as a batch process, often
overnight), I would expect only a specific error status, and to
be able to go on to the next test.
(snip)
I've also tried converting the structured exception to a C++
exception, as per
http://msdn.microsoft.com/en-us/library/ms680621%28VS.85%29.aspx,
and catching the exception to return EXIT_FAILURE, with no
success either.
That should probably work. Maybe you aren't catching the
exception properly, and terminate() is being called. Note that
terminate() also calls abort() by default. You can change this
by calling set_terminate() with a function which exits instead
of aborting.
See http://msdn.microsoft.com/en-us/library/k0k9c522.aspx for
an example of this.
I know how to change the behavior in case of an uncaught
exception. I intentionally made my exception derive from
std::runtime_error, and caught std::exception, so that I'd fall
into the usual cases.
Anyway, it seems to be working except for assert, so possibly
changing the assert macro would do the trick. (If I find the
original code which caused the problems, I'll repost. And it
may depend on the compiler options.)
So how do you run a suite of regression tests under Windows?
myAssert() above would be a possibility if you are only trying
to capture asserts.
Asserts are just part of the issue. I'm concerned about
anything which might cause the program to crash.
I tend to just print an error message and
call exit(n) directly. Note that n can (nonportably) have
values other than EXIT_SUCCESS and EXIT_FAILURE under Windows,
so a batch file can branch based on different types of error
returns.
I know. I use this in my actual code (since I'm only concerned
with portability between Windows and various Unix).
--
James Kanze