Re: Testing in C++
Noah Roberts wrote:
Pete Becker wrote:
Today, "regression test" seems to mean "run the tests you've run
before and see if anything got worse." I.e., run the test suite.
Formally, though, a regression test is a test you add to your test
suite in response to a user-reported defect, reproducing the user's
conditions.
I believe you're wrong on this.
I've only spent ten years as a test writer and manager, so it may be
that I don't know what I'm talking about, but I doubt it.
All definitions of regression testing I
have seen are running the full suite to make sure you didn't break
anything. This would follow from the definition of "regression":
1. the act of going back to a previous place or state; return or
reversion.
http://dictionary.reference.com/browse/regression
That's not a definition of "regression test."
I don't think this is a change either. Wikipedia quotes Fred Brooks:
"Also as a consequence of the introduction of new bugs, program
maintenance requires far more system testing per statement written than
any other programming. Theoretically, after each fix one must run the
entire batch of test cases previously run against the system, to ensure
that it has not been damaged in an obscure way. In practice, such
regression testing must indeed approximate this theoretical idea, and it
is very costly." -- Fred Brooks, The Mythical Man Month (p 122)
That book is a couple decades old at least...
And that's not a book about testing. For the true definition, see "The
Art of Software Testing," by Glenform Myers.
This is an important step to make even if expensive. Many times a fix
to a new bug can cause old bugs to reappear...for instance, sometimes a
fix introduces a new bug, which is found and reported...and then "fixed"
bringing back the old one that's fix introduced this new bug.
What you are talking about is heavily used in TDD and also hasn't gone
away or become less used. If it has a formal name I don't recall it.
Its formal name is "regression testing." Or was, until the "regression"
became vacuous.
Step one, new acceptance test for the bug...step two, find the cause,
step 3 write unit test to expose cause...step 4 fix...step 5 run new
tests...step 6 run regression.
So I don't see that anything has become meaningless here. Both new
tests for bugs and regression tests to be sure the program still passes
acceptance from previous versions are important steps to robust project
management.
As I said, "regression test" has come to mean "test." Too bad.
--
-- Pete
Roundhouse Consulting, Ltd. (www.versatilecoding.com)
Author of "The Standard C++ Library Extensions: a Tutorial and
Reference." (www.petebecker.com/tr1book)