"Scott McPhillips [MVP]" <org-dot-mvps-at-scottmcp> wrote in message
news:uv2VTqJ7HHA.4612@TK2MSFTNGP03.phx.gbl...
"Peter Olcott" <NoSpam@SeeScreen.com> wrote in message
news:gf5Ci.86140$Mu5.21970@newsfe15.phx...
I have a windows console program that interfaces with a library that
interfaces with a hardware device. I run the same unmodified program with
the same input and get different results on different trials. My program
is the same, the test data is the same, yet different runs produce
different results.
There are only two sorts of things that I can think of that could cause
this:
(1) The state of the hardware device changes between runs.
(2) Some really weird memory error that causes my program to run
correctly 75% of this time and fail other times.
Does anyone have any ideas on how I can narrow this down better?
Other possible sources of inconsistent behavior are improperly
synchronized multithreading and coincidences between the timing of
hardware and software actions. Maybe floating point rounding should be
included, if your "inconsistent results" are numeric. To hopefully
narrow it down consider adding numerous TRACE statements (or a logging
file) so you can look through the flow of results to see where
differences appear.
The program is a regression testing tool that tests the hardware interface
libraries.
Sometimes when I run the tool it seems to test different sequences of
operations. It provide the wrong data to the tests. I already have two
different types of logging files.
Three out of four times it provides the correct data to the correct tests.
One out of four times it provide incorrect data to the tests. This occurs
without changing the data or the program. I store the list of data in a
set of std::vectors of std::string.