[jdom-interest] How to manipulate a very large XML file? Any suggestions?

[Elliotte Rusty Harold]

At 9:34 AM -0500 2/10/04, Goswami, Raj wrote:
>If you have a need to provide a generic solution, how would you go about
>that?  If I understand correctly, even XQuery uses XPath internally.
>Doesn't that require it to create the whole document in memory?  How does
>the commercial vendors manage this (like Mercator, Vitria etc.) for document
>translation?

There's memory and then there's memory. Not all memory is RAM. 
Neither XQuery nor XPath requires an entire document to be loaded 
into RAM. It just needs to have some way of accessing all the 
different parts of the document(s) when it needs them. Whether these 
parts are stored in RAM or on disk is an implementation detail. 
Smarter implementations can be more or less efficient working with 
different sizes of files. Often there's a trade-off. Implementation 
strategies that work best for small files (stuff everything in RAM) 
often fail when approached with documents that are large relative to 
available memory.
-- 

   Elliotte Rusty Harold
   elharo at metalab.unc.edu
   Effective XML (Addison-Wesley, 2003)
   http://www.cafeconleche.org/books/effectivexml
   http://www.amazon.com/exec/obidos/ISBN%3D0321150406/ref%3Dnosim/cafeaulaitA