[jdom-interest] Re: jdom-interest digest, Vol 1 #1103 - 8 msgs

[Kenneth Kelley]

> From: Joerg Erdmenger <joe at woerd.com>
> Subject: [jdom-interest] OutOfMemoryError
> 
> Hi there,
> 
<snip>

> Any ideas? Clearly a 1.6M xml file can't use up more than 200M of RAM whe=
> n=20
> read in, can it?
> 
> Joerg
> 

Hahahahahahahaha!

Sorry, I shouldn't laugh at your pain.

But, yes, it is possible for a 1.6Mb file to use up more than 200Mb of
RAM when read in.  I had someone once manage to use up over 200 Mb of
RAM to read in a mere 90 kb file (2,000 lines).

He read in each line of the file and repeatedly used String += to make
one big String.  Sure, there were lots of Objects available to be
garbage collected, but it still managed to crash our server before the
garbage collector had a chance to clean up.

I'm not saying anyone at JDOM would do such a thing.  Using up
order(N^2) memory to read in a file is really bad.  However using up
order(N log(N)) memory to read in/process a file isn't so unreasonable,
if there's a lot of processing to do. 1.6 M * log(1.6M) is about 30 M,
so that's getting up there.  Maybe someone optimized for speed (at the
expense of memory footprint), thinking no one would try to read in a
file this big.

Hmm.  I guess I'm offering more of a possible explanation than a
possible solution.  Sorry.  I hope it helps, though.

-- Ken