Re: Any suggestions for handling data of huge dimension in Java?
Simon wrote:
Good day. Regarding the subject, I am doing a research simulation by
using java [sic] in eclipse [sic] galileo [sic]. My laptop is dell [sic] studio [sic] xps [sic] 1645 with
i-7 processor and 4gb ram. When running the java [sic] source codes, I have
set the Run Configurations> Arguments> VM Arguments> -Xmx1024M -
XX:MaxPermSize=128M and also assign the object to null when it is no
A false tactic. You cannot assign an object to 'null', first of all. You can
only assign 'null' to a reference. That's actually very important, because if
you have a second reference to the same object, setting the first one to
'null' is just self-delusion.
Generally speaking, setting references to 'null' is undesirable and
potentially harmful, according to the Java gurus such as Brian Goetz. Except
for the very few places where it's needed, the best it does is give you a
false sense of security while preventing you from properly scoping your
references and your object lifetimes.
longer needed. However, i [sic] keep facing java [sic] heap problem. Most of the
That's either because you aren't releasing all references to objects when you
need to (likely), or you simply have too much data for a gig of heap.
time, i [sic] am using HashMap<String,Double> and StringBuilder to hold the
http://sscce.org/
Vague descriptions mean the problem is always in the part you don't show.
data. Dimension of my data is around 5000 columns (or features) x 50
classes x 1000 files, and I need to extract that data into one file
for classification purpose. Therefore, is there any suggestions or
articles for me to cope with such problem?
If you're extracting into a file from some place presumably not originally in
Java heap RAM, why are you loading it all into memory at once?
Please construct and provide an SSCCE.
http://sscce.org/
Otherwise you're like a patient asking his doctor, "I have something wrong.
Can you prescribe me something?"
We need solid data from you.
http://sscce.org/
--
Lew
Honi soit qui mal y pense.
http://upload.wikimedia.org/wikipedia/commons/c/cf/Friz.jpg