Re: Execution time of code?
On Mar 6, 10:16 pm, c...@mailvault.com wrote:
On Mar 6, 5:05 am, James Kanze <james.ka...@gmail.com> wrote:
On Mar 6, 8:50 am, c...@mailvault.com wrote:
On Mar 5, 11:55 pm, "Alf P. Steinbach" <al...@start.no> wrote:
[...]
I've done some performance testing on Windows and Linux
--www.webEbenezer.net/comparison.html. On Windows I use
clock and on Linux I use gettimeofday. From what I can
tell gettimeofday gives more accurate results than clock
on Linux. Depending on how this thread works out, I may
start using the function Victor mentioned on Windows.
On Unix based machines, clock() and gettimeofday() measure
different things. I use clock() when I want what clock()
measures, and gettimeofday() when I want what gettimeofday()
measures. For comparing algorithms to see which is more
effective, this means clock().
I've just retested the test that saves/sends a list<int> using
clock on Linux. The range of ratios from the Boost version to
my version was between 1.4 and 4.5. The thing about clock is
it returns values like 10,000, 20,000, 30,000, 50,000, 60,000,
etc.
This sounds like a defective (albeit legal) implementation.
Posix requires CLOCKS_PER_SEC to be 1000000 precisely so that
implementations can offer more precision if the system supports
it. Linux does. I'd file a bug report.
Of course, historically, a lot of systems had clocks generated
from the mains, which meant a CLOCKS_PER_SEC of 50 (in Europe)
or 60 (in North America). On such systems, better precision
simply wasn't available, and I've gotten into the habit of not
counting on values of benchmarks that run for less than about 5
minutes. So I would tend not to noticed such anomalies as you
describe.
I would be more comfortable with it if I could get it to round
its results less. The range of results with gettimeofday for
the same test is not so wide -- between 2.0 and 2.8. I don't
run other programs while I'm testing besides a shell/vi and
firefox. I definitely don't start or stop any of those
between the tests, so I'm of the opinion that the elapsed time
results are meaningful.
The relative values are probably meaningful if the actual values
are large enough (a couple of minutes, at least) and they are
reproduceable. The actual values, not really (but that's
generally not what you're interested in).
In my own tests, with clock(), under both Linux and Solaris, I
generally get differences from one run to the next of
considerably less than 10%. Which is about as accurate as
you're going to get, I think. Under Windows, I have to be more
careful about the surrounding environment, and even then, there
will be an outlier from time to time.
Victor is right about one thing: the implementation of
clock() in VC++ is broken, in the sense that it doesn't
conform to the specification of the C standard, e.g. that
"The clock function returns the implementation's best
approximation to the processor time used by the program
since the beginning of an implementation-defined era related
only to the program invocation." The last time I checked,
the clock() function in VC++ returned elapsed time, and not
processor time. (Of course, if you run enough trials, on a
quiescent machine, the functions involved are pure CPU, and
the goal is just to compare, not to obtain absolute values,
the information obtained is probably adequate anyway.)
Except for the part about the functions being purely CPU, this
describes my approach/intent.
Again, it depends on what you are trying to measure. If you
want to capture disk transfer speed, for example, then clock()
is NOT the function you want (except under Windows).
--
James Kanze (GABI Software) email:james.kanze@gmail.com
Conseils en informatique orient=E9e objet/
Beratung in objektorientierter Datenverarbeitung
9 place S=E9mard, 78210 St.-Cyr-l'=C9cole, France, +33 (0)1 30 23 00 34