Re: multithreaded cache?

First off, thank you for the very professional and elegant code.

I shall study it frequently.

I had experience with CHM on an Enterprise Java project a few years back
that involved processing documents up to 1GB or so at millions of
documents per hour.

As you can imagine, concurrency was a consideration there, but due to
bureaucracy was not properly managed for a few years. I was hired around
the time they started to pay attention to such issues.

The code involved a properly but naively synchronized Map at one point.
Detailed profiling revealed that lock contention for the Map was the
number one throughput chokepoint in the whole system. Even above
database concurrency and I/O.

Boy howdy, the pundits are right to recommend hard measurement.

Lock contention has a cascade effect. In modern JVMs, like IBM's
mainframe-level ones that we used, uncontended locks process quite
quickly. Contention introduces roadblocks that delay threads, allowing
more to queue up, causing more contention, slowing things down still
more, causing more, yada. It only takes one skunk in the middle of the
main road to completely tie up rush hour everywhere.

CHM by default partitions lock space (though I'm not clear it uses locks
exactly) into sixteen independent slices. This meant far more than
sixteen times faster for us. Writes tend to happen in a solitary thread
without a whole lot of fight while reads run like greased pigs through
the other fifteen. With our mix of reads and writes, and transaction
volume, CHM pretty near eliminated lock contention. YMMV, as always, but
in this case that chokepoint went from number one to off the list.

It was still the wrong solution, since a simple, effortless,
non-concurrent better one that would also have eliminated a raft of
other problems was available, but had no political traction. However,
good enough to eliminate the throughput impact was good enough, so I
didn't raise a fuss when they decided against it.