Re: STL Slow - VS2005
Hi,
I'm the developer who created these tests.
"Jerry Coffin" wrote:
The first and most crucial question to ask of any benchmark is how
accurately this benchmark represents real-world workloads. Unless the
benchmark is quite close to real-world usage, it's meaningless -- and
even if it's fairly close, it can still be entirely meaningless (if the
differences between "fairly close" and "real world" make a big
difference in performance).
Yes, but I am microbenchmarking a small subset of the STL string
implementation only. Most real world apps will have bigger bottlenecks to
deal with and these benchmarks will not matter much. The benchmarks do point
to a more efficient implementation in STLPort.
string s ;
for ( int i = 0 ; i < 100000000; ++ i ) {
s += " " ;
People have done quite a few studies on string lengths as they're
actually used. Every time I've seen such a study, it's concluded that
most strings are quite short -- most people find that over 90% of
strings are 80 characters or less, and the frequency of usage drops of
quickly with length -- even 1000 characters is extremely rare. A string
of 100 megabytes is so rare that your measurement means nothing about
real use.
Test was meant to benchmark the performance when the string must do new
allocations. This was an easy way to get at that.
string s("qyweyuewunfkHBUKGYUGL ,wehbYGUW^( @T@H!BALWD:h^&@#*@(#:
JKHWJ:CND");
for (int i = 0 ; i < 10000000; ++i) {
s.find( "unfkHBUKGY" ) ;
s.find( "W^( @T@H!B" ) ;
s.find( "J:CND" ) ;
Likewise, in the real world, most strings consist primarily (if not
exclusively) of a-z, A-Z, space, and a FEW punctuation characters. I
haven't checked, but my immediate guess is that a significant difference
in string searching speed likely results from using a Boyer-Moore (or
BMH) search algorithm. Restricting the alphabet in use also
significantly reduces the advantage gained from such an algorithm.
That's not to say that such an advanced algorithm can't be useful -- but
my immediate guess is that the strings chosen above significantly
exaggerate the advantage likely to be gained in most real use.
Whatever. STLPort searched faster. Try it with other strings and see what
happens.
string s("qyweyuewunfkHBUKGYUGL ,wehbYGUW^( @T@H!BALWD:h^&@#*@(#:
JKHWJ:CND");
[ ... ]
printf("Pass and return by value function called 10 million times: %.0f
ms\n", msecs);
I can't imagine anybody passing strings by value except by accident.
Virtually every book on C++ teaches even rank beginners to pass class-
objects by ref-to-const by about chapter 2. This benchmark seems to
apply only to thoroughly inept code.
Test was meant to benchmark the copy constructor, and return value
optimization. This was an easy way to test that.
string s("1234567890");
[ ... ]
printf("Short String - Pass and return by value function called 10 million
times: %.0f ms\n", msecs);
Passing by value again?
Again, to test the copy constructor on short strings.
If I didn't know better, I'd guess you carefully
avoided the two (by far) most common situations: 1) pass a short string
by reference and return a short string by value, 2) pass a short string
by reference, modify a string whose reference was passed.
OK. Here goes with these two tests included (code at end):
VS2005
==================================================================
W:\test\string_tests\VC8\Release>string_tests.exe
Append one character to empty string 100 million times: 6791 ms
Search long string for 3 different substrings 10 million times: 3367 ms
Assignment, search, replace 10 million times: 3551 ms
Pass and return by value function called 10 million times: 8362 ms
Pass by reference return by value function called 10 million times: 4222 ms
Short String - Pass and return by value function called 10 million times:
1460 ms
Short String - Pass by reference return by value function called 10 million
times: 741 ms
Short String - Pass by reference modify reference called 10 million times:
643 ms
VS2005 & STLPORT
===================================================================
W:\test\string_tests\VC8\Release>string_tests.exe
Append one character to empty string 100 million times: 1804 ms
Search long string for 3 different substrings 10 million times: 3134 ms
Assignment, search, replace 10 million times: 2611 ms
Pass and return by value function called 10 million times: 3839 ms
Pass by reference return by value function called 10 million times: 1885 ms
Short String - Pass and return by value function called 10 million times:
829 ms
Short String - Pass by reference return by value function called 10 million
times: 421 ms
Short String - Pass by reference modify reference called 10 million times:
245 ms
{
usecTimer.Start();
string s("1234567890");
for( int i = 0 ; i < 10000000; ++ i ) {
string sx = func2( s );
}
usecTimer.End();
double msecs =
double((usecTimer.GetDifference()*1000)/usecTimer.GetFrequency());
printf("Short String - Pass by reference return by value function called 10
million times: %.0f ms\n", msecs);
}
void func3(string& par)
{
par = "1234123412";
}
{
usecTimer.Start();
string s("1234567890");
for( int i = 0 ; i < 10000000; ++ i ) {
func3( s );
}
usecTimer.End();
double msecs =
double((usecTimer.GetDifference()*1000)/usecTimer.GetFrequency());
printf("Short String - Pass by reference modify reference called 10 million
times: %.0f ms\n", msecs);
}
-ephi