[jdom-interest] java.lang.OutOfMemory with an 8MB xml file??

[Duffey, Kevin]

Hi all,

Ok, maybe I am doing something wrong. But this is crazy. An 8MB xml file is causing my jvm to run out of memory? I am not specifying any extra memory settings, but if it allows up to 64MB of memory, why would an 8MB file eat up that much memory?

So here is the issue. Because my app needs to work offline, and work with XML files that specify a DTD, I have to turn the feature off to use a DTD to validate the xml. Using the latest xerces parser (2 I think), this is accomplished like so:

SAXBuilder builder = new SAXBuilder(false);
builder.setFeature("http://apache.org/xml/features/nonvalidating/load-external-dtd", false);
Document doc = builder.build(new File(filename));

So, here is what my program has to do. It first loops through all selected XML files, running the above code on each, along with some code that gets the root doc, a few nodes and makes sure they exist. If they do, its a "valid" xml file of the selected format. If not, then it is not valid. None the less, the next two steps are to parse the XML again for some header info, then parse it again later for the rest of the doc that gets converted into our own format.

So the problem seems to occur if I select a file about 8MB or so in size. Now, I thought by using Sax Builder it would not load the entire document into memory. Even so, I am unclear why an 8MB xml file using a Sax Builder would run out of memory? How does the DTD validation of an xml parser handle this?

I'd appreciate any ideas or knowledge in this area. If I need to provide more info, let me know.

thanks.