Re: optimizing std::map or std::hash_map for speed
Costellj@gmail.com wrote:
Hello,
I am converting an app written in mfc to be cross platform. The goal
is for the app to be as fast or faster than it's mfc's equivalent. I
am doing some preliminary tests with std::map and std::hash_map
compared with CMap. I believe I am missing something in how I am
implementing the stl containers, or perhaps my test is bad. In general
this is my test:
get current time
for Cmap I call InitHash() with a prime number bigger than what I need
for std::map, std::hash_map I use the get allocater to allocate the
space I need in advance
I think you misunderstand what get_allocator does. It does not allow you
to reserve space in the container. Unfortunately, neither VC7.1 nor
VC8's hash_map offers a facility to reserve space, which means that
increasing the size of the map is quite expensive when compared against
an implementation that does have reservation. However, not that once the
table has expanded, it shouldn't shrink again. So, for hash_map, you could:
1. Run the test once.
2. clear() the hash_map.
3. Run the test on the same hash_map object again.
The second run should be much faster, since the container will not have
to rehash nearly as much. This reflects why synthetic benchmarks can be
a very bad idea, since in most real world applications, initial
expansion performance of a map is unlikely to be a performance bottleneck.
then I call a for loop and insert 2 million values
I then subtract the current time from the previous to get how many
seconds it takes
In my Cmap code this take between 8 and 9 seconds, but with each of the
stl it is well over 30 seconds.
I know that there must be an error in something I'm doing to get this
drastic of differences. I just can't figure it out for the life of me
Allocation performance is likely to be a second deciding factor (after
the issue above has been dealt with). std::map in particular can be
greatly speeded up by using a faster allocator than that shipped with
VC. A simple way of speeding it up is to call:
_set_sbh_threshold(1016); //1016 was used pre-Win2000.
Alternatively, you could try the boost pool allocators, which is a bit
more involved (since it involves installing and building boost):
http://www.boost.org/libs/pool/doc/index.html
or you can use one of the many fast allocator libraries available.
Finally, the standard version of a hash map, std::tr1::unordered_map,
has a rehash member function that serves the same purpose as InitHash I
think:
http://www.dinkumware.com/manuals/?manual=compleat&page=unordered_map.html#unordered_map::rehash
VC9 will probably have unordered_map, or you can get it from
www.dinkumware.com.
Tom