Oliver Wong wrote:
Use less memory per record. Which may involve
simple data packing or compression.
Alternatively, assign more memory to your Java program, or don't
actually store all the data in the list. See the Proxy design pattern, for
an example of the latter.
At the risk of topic drift, does any one
know of a List implementation
which doesn't suffer from ArrayList's
"growth problem", causing (IMHO) premature
out of memory reports?
The problem is that when the current memory allocation
is exceeded, an ArrayList tries to double in size;
assuming you're currently using (e.g.) 64 Mb,
it will try and allocate 128 Mb, then copy the data
from the 64 to the 128, and only then does the 64
become available for GC.
Assuming you actual data size were 65 Mb, this
means you would require 196 Mb to "survive" the handover.
A better data structure (w.r.t growth) would be a list
of BLOCKs of some size. Additional blocks can be added,
and serial (iterator) access would be fast.
Random access is also "quite" fast, if the blocks are a
"reasonable" size.
Does anyone know an implementation of this, or something like it?
BugBear