[jdom-interest] Announce: JDOMPlus a flexible XML framework for Java

[Brett McLaughlin]

James Strachan wrote:
> 
> ----- Original Message -----
> From: "Brett McLaughlin" <brett at newInstance.com>
> >
> >
> > James Strachan wrote:
> > >
> > > From: "Jason Hunter" <jhunter at collab.net>
> > > > Brett and I very much hope that people with design ideas continue to
> use
> > > > this list as the place for discussion.  I'm always happy to debate the
> > > > merits of different designs.
> > >
> > > Me too its fun :-)
> > >
> > > Though from Bretts responses such as...
> > >
> > >
> http://lists.denveronline.net/lists/jdom-interest/2000-November/003901.html
> > >
> http://lists.denveronline.net/lists/jdom-interest/2000-December/003954.html
> > >
> > > I think its pretty clear that interfaces, factories, multiple tree
> > > implementations ('dual' trees), read only & reusable branches and the
> like
> > > are not going to make it into JDOM. Period. JDOM has one Element
> > > implemention and one Attribute implementation end of story.
> Unfortunately, I
> > > don't see much point discussing such things on JDOM anymoreas it is
> clear
> > > JDOM will never move in the 'interface' direction.
> >
> > Well, to be blunt, you are completely wrong. See my mail this morning to
> > Guy Nirpaz.
> 
> I still haven't seen any mail from you to Guy. Maybe the list is just being
> slow?
> I waited a couple of hours from getting your mail and still nothing.

The list IS slow, unfortunately :(

> 
> > James, at the risk of offending you (which I don't want to
> > do), you simply didn't justify the design choices you wanted to make.
> > No
> > idea is arbitrarily turned away here - none.
> 
> So are you saying that, if I convince you, you may change your mind and go
> down the interface route with JDOM? Until I saw this mail from you I
> certainly didn't think that was the case.

Absolutely. And I'll take some blame as to that, as it's been a long
time since Jason or I have clearly articulated the MEANS by which
changes are accepted or rejected. So yes, with a strong use-case and
follow-ups, its possible that anything /could/ get into the API. Now
that said, it's true that some ideas (such as interfaces), we are
predisposed on a position to, so it might be "harder" to get changed
than a totally new idea. But we don't "hold the bar higher", the
arguments just have to be really sound because we might have heard them
before.

> 
> Maybe I got the wrong impression from your (and to a lesser degree, Jason &
> Elliots) responses.
> 
> I certainly formed the impression that you had pretty much decided that
> you'd been down the interface route, didn't like it and were not going back.
> I'm not the only one to get this impression too, i've had quite a few people
> mailing me directly with the same impression.

Given the requests that we have gotten so far, that is true. However,
that doesn't preclude someone from coming up with a great new argument,
or a new twist we hadn't thought of. Or a brilliant implementation that
we didn't catch onto. The impression you are getting is that we feel
that so far, nobody ahs demonstrated the amount of actual experience
concerning JDOM and interfaces that we ourselves had in a complete
interface-driven implementation, so the arguments we here are ones we
ourselves had and discarded ;-) 

> 
> Email can be a difficult medium to express complex views and be totally
> understood so if I got the wrong impression, I appologise, and let me try
> and convince you of the need for interfaces / abstract base classes ;-)

That's fair enough, on both counts ;-)

> 
> There's been alot of traffic lately about interfaces, dual tree
> implementations and the like and it certainly seemed to me you'd closed the
> door on them for now on JDOM. I hope I'm wrong.

We've closed the door; but it's not locked. We have this little hole
like in "The Princess Bride" where you can stick your head in and
convince us to open the door (as long as you don't say "Humperdink"). 

> 
> > But for any idea to be
> > accepted, you must first present a compelling use-case that affects a
> > signficant number of JDOM users.
> 
> How about if I claimed I can boost the peformance of the parsing a document
> of a specific DTD for anyone using JDOM by 5-10% in a conservative estimate.
> Would that be compelling enough? Or would it need to be more like 30% which
> I think could be possible? Or how about reduce memory consumption by 20%?
> Are any of these compelling enough?

Well, I'm a little loathe to make a performance-based decision, and
here's why. First, "premature observation is the root of all evil." We
haven't even tried to optimze JDOM, so I'm not yet convinced that the
time to make performance-based decisions is here. It would be claiming
performance improvements over an admittedly untuned API. And I'm not
convinced that you could make an argument based on "All interface
implementations would increase performance/reduce memory by x% over all
non-interface ones." Does that make sense?

> 
> I've found that just using a 'caching attribute' factory together with a
> singly linked JDOM-like tree can boost parsing peformance by about 5% over a
> fully doubly linked tree. (This obviously depends on the attribute frequency
> in your document, JVM et al). A few percent alone of this was just down to
> using the singly linked tree which totally suprised me - and the box was CPU
> bound, its nothing to do with swapping. Caching common attribute instances
> rather than creating them each time speeded things up more and used
> considerably much less memory.
> 
> A possible avenue of development is to take this concept further to using
> code generation from a DTD to generate custom, memory efficient Element and
> Attribute implementations and a special Builder. This could yield much
> bigger performance gains for 'regular' 80/20 JDOM users who parse a document
> of a particular DTD alot. (e.g. anyone doing SOAP / servlet / J2EE work
> which do alot of XML messaging or reading). BEA is doing a similar things,
> at the SAX layer, in their 6.0 WebLogic product.
> 
> Is a concrete performance increase for JDOM users who parse a particular
> kind of document a compelling enough argument to reconsider interfaces or
> abstract base classes?

Not at this point, because I'm not convinced that the other things we
are doing in JDOM won't affect performance, and perhaps be contrary to
these solutions. Again, the main point is USE-CASE. Performance isn't a
use-case, it's a justification. I'm still looking for why people would
be so aided by your doubly-linked tree.

> 
> > You have yet to do that - you have a
> > niche that you want to fill that you have not convinced Jason or me is a
> > common one (although it certainly my be YOUR niche ;-) ).
> 
> Allowing alternate implementations will only directly affect a small number
> of JDOM users directly, though the speed and memory performance effects of
> these techiques could be shared by much of the community.

So I'm reading by this that you don't have a common use-case. And that
takes us back to the beginning, where I said this didn't make sense,
because it didn't provide functional benefits for users. I'm certainly
not going to add complexity to the API (which interfaces almost
inevitably do, as well as allow misuse) purely for performance, which I
am not convinced we couldn't get otherwise.

> 
> > Second, you
> > must present a compelling design solution to that idea, one that is
> > sound in design. I have yet to see that as well.
> 
> Go to http://www.jdomplus.org or http://jdom.sourceforge.net the source code
> is all there in CVS on SourceForge. The source, examples and documentation
> was all there before I made the announcement. There's also a sample program
> there for doing performance analysis of the effect of doubly linked, singly
> linked and 'caching attribute' tree building.
> 
> Though all this will be moving someplace without "JDOM" in the URL in due
> course.
> 
> > While I salute your
> > efforts, there are many, many interface design problems that you have
> > not addressed or mentioned.
> 
> I'm yet to hear any from Jason or yourself other than 'it makes things
> complex' and that you'd been there and didn't like it. I'd love to hear some
> more, I'd like to learn from your experience.
> 
> > That indicates to Jason and I (who started
> > down this road, and went very far down it) that you may be in for some
> > unpleasant surprises down the line. While you may solve each in turn, it
> > doesn't absolve us of the responsibility to JDOM users to not trust you,
> > or anyone (no matter how good of a programmer they are ;-) ), to solve
> > those at that time, instead of up-front.
> 
> Sure. I respect and understand your position.
> 
> I just want to know, should I pursue the possibility of abstract base
> classes with no data (or interfaces) with JDOM or should I put my efforts
> into another project?

I'm going to also let Jason answer this, from his perspective, and we'll
go from there. At that point, we'll keep going along these discussion
lines...

-Brett

> 
> I got the impression it would be easier for me to start another project,
> prove the performance benefits were worth the extra 'complexity cost' then
> come back to the JDOM community and say, take a look at "the project
> formerly known as JDOM+" on website XXX and shall we move JDOM in that
> direction. Whether this research happens within JDOM or not is your &
> Jason's call.
> 
> > As with many suggestions that we don't use, people feel that we have
> > made an arbitrary decision. We haven't, and don't. Jason and I often
> > spend many hours emailing and talking these things over behind the
> > scene, and carefully weighing the merits (yeah, we're a real JDOM
> > Supreme Court ;-) ). And we were not satisfied that your ideas met a
> > common use case, and met it in a way that made sense. It is worth noting
> > that the other proponent of the doubly-linked tree also felt that the
> > implementation design you laid out was wanting. While that's no fun to
> > hear, it does indicate that perhaps Jason and I are being safe to not
> > want to include this sort of thing, at least at this point.
> 
> Sure, believe me I understand. You're right to protect the interests of your
> user base.  This is also why I thought starting another project to research
> my ideas was a better way forward than having a near-religious debate on
> jdom-interest list, trying to convince you of the value of interfaces.
> 
> > Finally, I want to ask, respectfully, that you don't state (in this mail
> > or on your API website) anywhere that we "won't" do something, because
> > it's simply not true.
> 
> OK, I'll turn down the "wont" to phrases of the "not right now" variety.
> I'll change go through the web site checking for inaccuracies, sorry if my
> rash paraphrasing caused any offence.
> 
> > Yes, it's really hard to get changes into JDOM at
> > this point of a significant nature. ;-)
> 
> I know. Its another reason for the new project.
> 
> > People know that, and it is
> > frustrating. Sometimes I want to get in changes, and can't ;-) But that
> > also results in a carefully though out, mature API, well before it's
> > time. I wish DOM and SAX had been as stringent as to what got into their
> > first versions; we might be a lot better off with them today! But
> > there's not a single suggestion we dismiss out of hand. If it looks like
> > that, ask us why - we've probably already done a lot of homework ahead
> > of time. And realize, that I completely wrote JDOM in interfaces as a
> > first pre-alpha.
> > The whole thing, completely working. And it was a pain
> > in the butt, and there were a ton of problems.
> 
> I know you did. That doesn't mean we can't revisit the problems you had in
> that approach and as a community, think more about the issues and fix them.
> 
> > And I'd submit that I'm
> > no dummy ;-)
> 
> Its clear you're not!
> 
> > So it's not blindly that we are resistant to interfaces
> > without REALLY GOOD justification.
> > Which nobody has supplied yet.
> 
> Better performance. How about that for justification in 2 words ;-)
> 
> >
> > >
> > > It is for this very reason that I started the 'project formerly known as
> > > JDOM+' ;-) project. For those people amongst us who want to freely mix
> and
> > > match different data structure implementations for different use cases /
> > > schemas within an API framework rather than relying on a 'one size fits
> all'
> > > approach to data structures.
> >
> > That's an overstatement, and hopefully it wasn't intentional.  Anyone who
> > wants to use subclasses can easily mix and match JDOM implementations.
> 
> Yes it should have said anyone who wants to mix an d match different
> implementations without inheriting all the instance data that is hardwired
> into the current Element and Attribute implementations. Also you have to
> roll your own SAXBuilder, and DOMBuilder to use your own derivations.
> 
> > Anyone who is familiar with the Collections API and facade patterns can
> > work with an immutable JDOM.
> 
> Corretion, tt is technically possible to use derivation right now in
> standard JDOM to implement immutability; however derivation with JDOM right
> now can be costly from a memory & performance point of view as you inherit
> often unnecessary instance data.
> 
> > It's really frustrating to see statements
> > like this; they are just as incorrect as the DOM-isms people slap us
> > with.
> 
> Yep sorry about that, overzealous typing fast to get something else done :-(
> 
> > > > but we believe it's best when making API
> > > > changes to "Measure twice, cut once" as they say, and the factory
> model
> > > > promoted by the forked design definitely raised some hairy unresolved
> > > > issues.
> > >
> > > What hairy unresolved issues are they? Though I like the term - hairy
> > > unresolved issue - HUI - I hope it catches on ;-)
> >
> > This is the crux of our point - it is not our job to point out or poke
> > holes in idea submissions. But it is your idea to find them,
> > nonetheless. You will have interface issues, and when you hit them, you
> > will see what we mean. That's the thing - we haven't seen a far-reaching
> > design laid out in any of your emails.
> 
> Erm, there's a web site, full CVS repository, javadoc, documentation, sample
> code and all the source available and a link to download it all as a
> tarball. What more do I need to do? Just tell me and I'll do it.
> 
> > That, in addition to not seeing a
> > common use-case, is why we would not accept your ideas. You have to do
> > more than convince yourself it's good, and then expect that to flow out
> > in your emails. You have to really justify any new concepts to us, and
> > make us believe how great they are. That just hasn't happened yet.
> 
> OK I'll try harder ;-)
> 
> > > If most of us can use interfaces for Map, List and Set quite happily I'm
> at
> > > a loss to understand what is so fundamentally wrong with using
> interfaces to
> > > represent a Java based XML tree data structure. The only problem with
> this
> > > approach as far as I can see (other than an increase in complexity which
> can
> > > be hidden from the user in many ways) is that politically it would make
> JDOM
> > > apprear more like DOM.
> >
> > Well, you'll have quite a time down the line then ;-)
> 
> ;-)
> 
> > All, it may seem like I'm picking on James here. I don't mean to be -
> > he's perfectly allowed to fork JDOM and do his own implementation
> > (although it isn't going to be called JDOM or any other derivative). In
> > fact, anyone with an idea that we don't accept could do that, from a
> > small idea to a big one. I suppose you could have
> > JDOM_with_exceptions_instead_of_null and JDOM_with_Node_base_class and
> > JDOM_with_this_idea_I_like and JDOM_with_that_idea_I_like.
> 
> Thats not very fair. I just want one JDOM that allows interfaces or abstract
> base classes so developers can roll their own tree implementations all
> working together.
> 
> > But the point
> > is that Jason and I are working as hard, and as fairly, as we can to
> > ensure that JDOM, the API, is as bulletproof and wisely designed as
> > possible. As a result, many folks (possibly including James) might feel
> > like we are dictators, that we are inflexible, that we're too
> > close-minded.
> 
> I don't think this at all. Really. I'm purely trying to understand your
> point of view and how to move forward in the direction I wish to pursue.
> 
> > I can only ask that you believe me that that is not the
> > case.
> 
> I do! :-)
> 
> > We have already completely re-written JDOM TWICE because our own
> > ideas didn't hold water. We're just as hard on ourselves as on others.
> > And lest you think we're hung up on our ideas, it was James Davidson who
> > convinced us that all our ideas on JDOM v.0.2 were wrong, and we rewrote
> > it all to NOT use interfaces! So if your idea isn't accepted, it's not
> > because we don't like you, or don't see your point, or are being
> > arbitrary. It's because we have a community that we feel responsible
> > for. There are a lot of people on this list, and a lot of people reading
> > my book, and a lot of people using JDOM, that we don't just make changes
> > that affect to. James's idea, as many ideas, didn't pass muster in our
> > opinion (based on what he wrote about the idea, not neccessarily the
> > idea itself). As a result, we have to either keep talking about the idea
> > or the submitter passes, and moves on. In this case, James has put his
> > efforts into another project. I think it would be fair to say that Jason
> > and I wish that hadn't been the case,
> 
> Me too. Hopefully one day I'll convince you and there will be only one JDOM
> ;-)
> 
> > and are unconvinced that the same
> > problems that drove us from interfaces will not haunt James at some
> > point.
> 
> Maybe. But I'll be faster and use less memory ;-)
> 
> > But we don't harbor any ill-will towards James, by any means.
> 
> Cool - and me too - we're all just trying to solve interesting problems
> better.
> 
> > Same to all others who have submitted ideas that have been rejected for
> > one reason or another. But if you are strongly in favor of something,
> > you need to let us know, and convince us. That's how open source works.
> 
> Yes. I'm trying my hardest and believe me I'll keep trying ;-)
> 
> > I hope this clears up some of the things going on, and of course
> > questions to the list are always welcome ;-)
> 
> :-)
> 
> <James/>
> 
> James Strachan
> =============
> email: james at metastuff.com
> web: http://www.metastuff.com
> 
> If you are not the addressee of this confidential e-mail and any
> attachments, please delete it and inform the sender; unauthorised
> redistribution or publication is prohibited. Views expressed are those of
> the author and do not necessarily represent those of Citria Limited.