Re: Unit testing of expected failures -- what do you use?
* Alf P. Steinbach:
OK, this displays my ignorance of what's out there (it's been a long
time since I developed for a living), and also my laziness not googling.
:-)
However.
I want to unit-test some library code I'm sort of extracting from some
old code I have.
The things that should work without error are easy to test, and it's
currently not so much code that I've considered a testing framework,
although the code size increases. I'm thinking that perhaps the popular
frameworks don't support my needs: there are cases where the code
/should/ assert at run time. And worse, there are cases where the could
should assert at compile time...
OK, I made a unit test driver GUI in Python 3.x, <url:
http://pastebin.com/bB1jeP5Z>.
It reports success or failure depending on whether build is expected to succeed
or fail, and depending on whether running the executable is expected to succeed
or fail.
It's very unfinished but this is a first usable version.
Tests and subtests are identified by C++ macro symbols defined in an [.ini]
file. Running a test the GUI (1) places #define's of the selected symbols in
file [_config.h], (2) invokes a 'build' script which in Windows can simply be a
batch file (building a /test/ is ordinarily very fast), and (3) if the build
succeeds and is expected to succeed, it invokes a 'run' script.
My 'build' script looks like this:
<code file="build.bat">
@echo off
devenv /nologo my_msvc_solution.sln /project my_lib_project.vcproj /build Debug
</code>
And the 'run' script is simply:
<code file="run.bat">
@Debug\the_name_of_the_executable
</code>
In the Python code the script (batch file) directory is hardcoded as
build_dir = os.path.normpath( "../build/msvc_7_1" )
but it should be easy to change.
For a main test the build is always expected to succeed.
For a subtest the build is expected to fail if the subtest id starts with "CERR_".
Cheers,
- Alf