Re: Process big Object in java
tzvika.barenholz@gmail.com wrote:
On Jan 7, 9:56 pm, Roedy Green <see_webs...@mindprod.com.invalid>
wrote:
On Mon, 7 Jan 2008 04:20:21 -0800 (PST), Colin Song
<song6...@gmail.com> wrote, quoted or indirectly quoted someone who
said :
StringBuffer request = new StringBuffer();
String str = "!@#$%^ *<>";//10Byte
for(int j=0;j<1000000;j++)
{
request.append(str); //10MB
}
That is going to be expensive. You should say
StringBuilder request = new StringBuilder( 1000000 );
Otherwise the internal buffer will start small and keep growing as you
append, each time it overflows having to allocate a new buffer. You
will create many objects in the process. When you choose the size to
start, the internal buffer allocated will be big enough to handle all
the appends.
--
Roedy Green Canadian Mind Products
The Java Glossaryhttp://mindprod.com
You guys are correct of course to suggest that the sb should be inited
with ample size, but this alone cannot explain anything like 500mb. As
it is, the sb inits itself to 16 chars, and then doubles the
allocation whenever it exceeds capacity. Even assuming none of the old
arrays get collected in the meantime, this would lead to the following
number of chars in memory:
16 + 32 + 64 + .... + 4mb + 8mb =~ 16 mb
so the answer must be in something else the original asker (or the
library code invoked) is doing.
Lasse Reichstein Nielsen showed how just the raw data could easily hit 80 MB,
given the OP's description of affairs, and was well on the way to showing
twice that just for the raw string data before jumping off. Add JVM overhead,
classes, libraries and what-all, and things can get quite large. 500 MB is
not infeasible given what we were told - on the high side, but not completely
outrageously.
However, the OP talks of doubling RAM and still seeing OutOfMemoryError (OOM).
This could indeed mean a runaway allocator (and none ever dereferences), or
some other unbounded activity, i.e., an algorithm flaw.
--
Lew