Re: Coding Standards
On Sep 6, 4:31 am, Ian Collins <ian-n...@hotmail.com> wrote:
James Kanze wrote:
On Sep 5, 10:38 am, Ian Collins <ian-n...@hotmail.com> wrote:
But they (well written, test first unit tests) do bridge the gap betwe=
en
a client's requirements (or some formal specification) and the code.
As such, they provide an interpretation of those requirements. The
interpretation should be validated by a set of customer acceptance tes=
ts.
They are an implementation of part of the specification. But
you can't write them until you have the specification: a precise
English (or other human) language document as to what the code
is to do.
Which when provided by customers (those who pay my bills) are at a level
understandable by the customer, not precise technical language.
You have to find a middle road. The customer (including
internal customers---"customers" for individual classes are
almost always internal) must be able to specify his needs
adequately for you to understand them. You then produce a more
detailed document, but still in a language the customer can
understand.
And there are more than a few cases that they simply
cannot cover, and for the most part, they are both more verbose
than a requirement specification should be, and fail to say a
lot of things that are essential in a requirement specification.
Such as?
I'm not being difficult, just commenting based on a
development process I have been following for a number of
years.
Anything involving undefined behavior, for example. Most things
involving threading as well. And a lot of things involving
floating point. (The fact that floating point arithmetic is
non-linear means that you can't always determine the critical
values which need to be tested without a lot of additional
analysis. And that they depend on the actual algorithm used;
black box testing isn't all that effective.)
A good unit test will typically test hundreds of "limit"
cases, all of which can often be covered by one or two succinct
sentences in the requirements specification. On the other hand,
no unit test will make statements at the more abstract level:
what is the role of the class in the application. What are it's
responsibilities, and what aren't?
It's not quite the sort of thing I was thinking of, but since
it's readily available on the net: consider the guarantees (the
contract) concerning thread safety in the SGI implementation of
the STL (http://www.sgi.com/tech/stl/thread_safety.html). How
on earth could you document something like this in unit tests?
For that matter, how on earth could you even test it? And note
that the aspects which your most concerned about in testing (the
cases when an external lock isn't necessary) are precisely the
ones which are least important for the user.
Another simple example, consider a network connection class.
The "documentation" (the requirements specification, or the
contract) may simple say that an exception will be raised if the
connection is broken. My unit tests, however, will simulate all
sorts of ways the connection might break. Details that the user
doesn't care about, and probably doesn't even understand. One
the other hand, 100's of tests don't begin to express the
abstract aspect of the contract: that whatever the cause of the
connection being broken, an exception will be raised.
Some additional important points:
-- The contract is a contract. It holds for the future as
well. It doesn't just make promesses about the current
implementation; it constrains future modifications. And
tests can only do this partially: if the protocol is
modified sometime in the future, and new failure modes are
introduced, there is no way to express in the tests that
these (at present unknown) failure modes must also result in
an exception.
-- Some failure modes may be extremely difficult, if not
impossible, to simulate. If the code is well reviewed, it's
probably acceptable to not test these.
-- Most failures will in fact be detected and reported by the
system. For the user, the contract isn't: an exception if
the Posix function select returns such and such an error
code---the user doesn't care about the function select, nor
even understand it. But what my tests test, of course, is
the behavior of my code depending on the status returned by
select.
In the end, the reason why unit tests (or any tests) aren't
acceptable as a requirements specification is because they are
(or should be) oriented to a different audience. You don't use
the same language when you communicate with people as when you
communicate with a machine.
That is where unit and more importantly customer acceptance test
frameworks come in. They provide the required vocabulary to enable
developers and customers/test engineers to to specifying the assertions
in a meaningful way.
If the customer can read and write unit tests, he's perfectly
capable of implementing the code himself, and doesn't need you.
You didn't read what I wrote "and more importantly customer acceptance
test frameworks". Most of my customers can, with a bit of help, use
something like FIT for developing *acceptance* tests, not unit tests.
It's a difference of level, perhaps. Development is a
hierarchial process. The final customer or user defines the
behavior of the application, and (hopefully for him) defines an
acceptance protocol, including tests, document review, etc.
Each individual class, however, also has "customers". And unit
tests are about the only testing individual classes will
receive. Typically (at least in the places I've worked), such
unit tests aren't written by the clients; they're part of the
development. And because I tend to work in the infrastructure,
generally, my "client" programmers wouldn't necessarily be able
to understand all of the unit tests.
The way it generally happens (in well run companies) is that the
architecture team will come up with a general design---including
the interface and the contract between subsystems, and possibly
even down to the class level. (The interface will, at any rate,
be defined in terms of classes.) Then there will be a design
review, where all concerned parties "sign off" the design---they
more or less state that they can live with the interfaces as
they are specified. Depending on the organization, at this
point, the header files may or may not have been written, but no
implementation nor unit tests exist---only a contract between
the implementors (of each subsystem or class) and the clients
(who are also implementors of other subsystems or classes).
Only once the contract has been defined does anyone start to
write code. (Obviously, it is almost always necessary to revise
the contract at various times during development. Even the best
architecture team can't think of everything. But any revision
which modifies the contract must be reviewed by a representive
of all parties concerned.)
--
James Kanze (GABI Software) email:james.kanze@gmail.com
Conseils en informatique orient=E9e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S=E9mard, 78210 St.-Cyr-l'=C9cole, France, +33 (0)1 30 23 00 34