Re: Unit testing of expected failures -- what do you use?
* Carlo Milanesi:
Alf P. Steinbach wrote:
E.g., the problem isn't to produce compile time asserts. The problem
is testing them, systematically. Preferably in an automated way.
I published the following article that addresses specifically this
problem (after a tutorial on testing C++ programs):
http://www.drdobbs.com/cpp/205801074
I published also the open source utility described in that article.
It is in the files "staticpp.txt" and "staticpp.zip" in the following
volume:
ftp://66.77.27.238/sourcecode/ddj/2008/0802.zip
Hm, it's very similar to my approach. But instead of your @LEGAL and @ILLEGAL I
just use ordinary C++ preprocessor directives. This allows testing to be
performed manually, without any preprocessing or having a script at hand.
It also allows having just a single Visual C++ project and project settings for
the test suite for a library, and almost completely decouples the scripting side
from the C++ source code side.
The script -> C++ connection is that at the scripting side one defines the
relevant preprocessor macros for each test.
Before building a test the script places the macro definitions for the selected
test in a file '_config.h', which is the only C++ file that it knows about.
The Python test driver script (not exactly finished, but working!) is available
at <url: http://pastebin.com/NK8yVcyv>.
---
The main program for my test suite for the lib I'm testing starts like this:
<code>
#include "_config.h" // Generated.
#if \
defined( USE_STATIC_ASSERT ) || \
defined( USE_STATIC_ASSERT__OVERRIDE ) || \
defined( USE_GENERAL_WARNINGS_SUPPRESSION ) || \
defined( USE_DEBUGGER_API ) || \
defined( USE_DEBUGGING ) || \
defined( USE_FIXED_SIZE_TYPES ) || \
defined( USE_TYPECHECKING ) || \
defined( USE_TYPECHECKING__DOWNCAST_OF_POINTER ) || \
defined( USE_TYPECHECKING__DOWNCAST_OF_REFERENCE ) || \
defined( USE_PRIMITIVE_TYPES__CASTBYTEPTRFROM ) || \
0
// OK
#else
# error Unknown test symbol or no test symbol defined.
#endif
</code>
I haven't gotten around to replace the "USE_" prefix with "TEST_" yet... :-)
A typical file to be tested (I just chose a short one):
<code>
// Copyright (c) Alf P. Steinbach, 2010.
#include <progrock/cppx/devsupport/static_assert.h>
#if \
defined( CPPXTEST_NO_SUBTESTS ) || \
defined( CERR_NONEXISTING ) || \
0
// OK
#else
# error Unknown or no subtest symbol defined (define CPPXTEST_NO_SUBTESTS?).
#endif
class Base
{
protected:
virtual void foo( int, char const* ) const = 0;
};
class Derived:
public Base
{
protected:
virtual void foo( int a, char const* b ) const
{
CPPX_IS_OVERRIDE_OF( Base::foo,(a, b) ); // Should compile fine.
}
virtual void bar() const
{
#if defined( CERR_NONEXISTING )
CPPX_IS_OVERRIDE_OF( Base::bar,() ); // Should yield
compilation error.
#endif
}
};
</code>
Cheers, & thanks for that article ref!,
- Alf