[jdom-interest] JDOM 2 list of elements, remove nulls out entries and fails to remove?

[Rolf Lear]

Hi David.

The issue is a bug in the remove(...) method. When this method is called 
it does not update the correct 'modcount' tracking value.

As a consequence, any 'FilterList' values you may have (getChildren 
generates a Filterlist), do not re-synchronize themselves with the main 
'live' list, and produce broken values, including nulls... but may 
include non-null, but still inappropriate values....

I have found & fixed this issue. If you want you can download the fixed 
jar from github at

https://github.com/downloads/hunterhacker/jdom/jdom-2.x-issue81.zip

This is a 'hotfix' download.... and it will be 'rolled up' in to JDOM 
2.0.2 which will be released before month-end. You can do a drop-in 
replacement of your JDOM jar with the hotfix one.

Rolf

On 11/05/2012 10:47 PM, Rolf Lear wrote:
> Right, I believe I have reproduced it....
>
> Hmmm... still working it.
>
>
> package org.jdom2.test.cases;
>
> import static org.junit.Assert.*;
> import java.util.List;
> import org.junit.Test;
> import org.jdom2.Element;
>
> @SuppressWarnings("javadoc")
> public class TestMultipleFilterLists {
>
> private static final Element getRoot() {
> Element root = new Element("root");
> root.addContent(new Element("A"));
> root.addContent(new Element("B"));
> root.addContent(new Element("C"));
> root.addContent(new Element("A"));
> root.addContent(new Element("B"));
> root.addContent(new Element("C"));
> root.addContent(new Element("A"));
> root.addContent(new Element("B"));
> root.addContent(new Element("C"));
> root.addContent(new Element("A"));
> root.addContent(new Element("B"));
> root.addContent(new Element("C"));
> return root;
> }
>
> @Test
> public void testMultiLists() {
> Element root = getRoot();
> List<Element> as = root.getChildren("A");
> List<Element> bs = root.getChildren("B");
> List<Element> cs = root.getChildren("C");
>
> for (Element f : as) {
> assertTrue("A".equals(f.getName()));
> }
> for (Element f : bs) {
> assertTrue("B".equals(f.getName()));
> }
> for (Element f : cs) {
> assertTrue("C".equals(f.getName()));
> }
>
> final int bsz = bs.size();
> final int csz = cs.size();
>
> while (!as.isEmpty()) {
> final int sz = as.size() - 1;
> Element e = as.get(0);
> as.remove(e);
>
> assertTrue(sz == as.size());
> assertTrue(bsz == bs.size());
> assertTrue(csz == cs.size());
>
> for (Element f : as) {
> assertTrue("A".equals(f.getName()));
> }
> for (Element f : bs) {
> /****** THROWS NULL POINTER ******/
> assertTrue("B".equals(f.getName()));
> }
> for (Element f : cs) {
> assertTrue("C".equals(f.getName()));
> }
> }
>
>
> }
>
> }
>
>
>
> Rolf
>
>
>
> On 11/05/2012 10:17 PM, David Wall wrote:
>> Okay,
>>
>> Attached is the XML I was parsing. In particular, I noted the null
>> pointer on the elements DropDown and DropDownVersion. There should be a
>> one-to-one correspondence between these two elements, with the
>> DropDownVersion containing a dropDownId that links it to the DropDown
>> element.
>>
>> I have also attached the source code file, but it's not a nice test
>> case. I have a few areas with "JDOM:" comments that are geared for you.
>>
>> Thanks,
>> David
>>
>> On 5/11/2012 7:02 PM, Rolf Lear wrote:
>>> Please send the code.... put a comment in it where you think the
>>> failure is..... Under no condition should JDOM return a null value...
>>>
>>> Rolf
>>>
>>> On 11/05/2012 9:45 PM, David Wall wrote:
>>>> Thanks, Rolf. No, it's definitely not a threading issue. If you saw my
>>>> second posting, it all works as expected if I create my own
>>>> LinkedList<Element> from the list returned by getChildren().
>>>>
>>>> If you think it's still a possible issue, I can attached the actual
>>>> source code and the sample XML data I was parsing. I'm not familiar
>>>> with
>>>> JDOM's internals, but if I remove one Element, does that cause a shift
>>>> for all subsequent elements? I ask this because I am actually creating
>>>> two lists, one is the main object, and the second is an associated
>>>> "version" object (the main object can have multiple versions associated
>>>> with it). In the XML, they are ordered like OBJ, OBJVERSION, OBJ,
>>>> OBJVERSION, OBJ, OBJVERSION.
>>>>
>>>> When I created to the two lists (one of OBJ and the other of
>>>> OBJVERSION), I noted that as soon as I deleted the OBJ from the first
>>>> list, the second list already showed a 'null' element at the end of its
>>>> list. It made me think that the elements are shifting in the live lists
>>>> from JDOM and thus I'm no longer working on the originally retrieved
>>>> lists.
>>>>
>>>> So, if that's all it is and my usage scenario was bogus before, I'm
>>>> good
>>>> since I've changed to create my own lists now. But if you think it
>>>> could
>>>> still be an issue, I'm happy to send the code and XML to you (it's open
>>>> source code anyway).
>>>>
>>>> Thanks,
>>>> David
>>>>
>>>>
>>>> On 5/11/2012 6:14 PM, Rolf Lear wrote:
>>>>> Hi David.
>>>>>
>>>>> This problem you describe is very concerning, and should not, in
>>>>> 'supported' usage ever happen....
>>>>>
>>>>> I am wracking my brains to figure out what it could be, but, the only
>>>>> think I can think is that you are accessing the JDOM objects from
>>>>> different threads.... JDOM is *not* thread safe.
>>>>>
>>>>> If this is happening in a single-threaded mechanism, would it be
>>>>> possible to put together a failing 'test case', any code that shows
>>>>> the problem?
>>>>>
>>>>> I have to admit that I thought he iteration/remove/add/etc. type code
>>>>> is very well tested, and I would be *very* suprised if there is a
>>>>> problem like this.... in fact, I know that there are test cases that
>>>>> cover this basic condition that you describe... and they never
>>>>> fail....
>>>>>
>>>>> so, if you can confirm this is all happening in one thread.... and
>>>>> preferably if you can put together an example of the problem, I would
>>>>> look in to it immediately....
>>>>>
>>>>> Rolf
>>>>>
>>>>> On 11/05/2012 8:42 PM, David Wall wrote:
>>>>>> I believe I've run into a problem that I don't think I had with JDOM
>>>>>> 1.1.3 before I upgraded to 2.0.1.
>>>>>>
>>>>>> I create a list of elements of a selected element using:
>>>>>>
>>>>>> List<?> documentVersionElements =
>>>>>> rootElement.getChildren("DocumentVersion", ns);
>>>>>>
>>>>>> I see that I have 6 elements as expected in my list. None are null.
>>>>>>
>>>>>> I am basically then trying to find a matching related element id in
>>>>>> that list using something like:
>>>>>>
>>>>>> Element found = null;
>>>>>> ListIterator<?> iter = documentVersionElements .listIterator();
>>>>>> while( iter.hasNext() ) {
>>>>>> Element checkElement = (Element)iter.next();
>>>>>> EsfUUID evParentId = new
>>>>>> EsfUUID(checkElement.getChildText("documentId", ns));
>>>>>> if ( evParentId.equals(id) ) {
>>>>>> found = checkElement;
>>>>>> break;
>>>>>> }
>>>>>> }
>>>>>>
>>>>>> Then, assuming I find it (found != null), I process it as expected.
>>>>>> But I then want to remove the found element from the element list so
>>>>>> it cannot be found again, so I use this:
>>>>>>
>>>>>> documentVersionElements.remove(found);
>>>>>>
>>>>>> Most of the time, this seems to work as expected, and the
>>>>>> documentVersionElements list is then shorter by 1. But there are
>>>>>> times when a list of 6 elements remains 6 elements after the remove
>>>>>> (and remove() returns false), with 4 of them now null (not removed,
>>>>>> but actually a null element), so in when I return to the
>>>>>> listIterator() above and get my next() element, the element is null.
>>>>>>
>>>>>> It's as if my Element objects are not unique in the list in terms of
>>>>>> equals/hashCode as my elements should never be nulled out.
>>>>>>
>>>>>> I am not sure why an element I find by iterating cannot then be
>>>>>> removed. How could it fail to remove? Why would it change other list
>>>>>> elements to NULL instead? Is this not a valid usage pattern?
>>>>>>
>>>>>> Thanks,
>>>>>> David
>>>>>> _______________________________________________
>>>>>> To control your jdom-interest membership:
>>>>>> http://www.jdom.org/mailman/options/jdom-interest/youraddr@yourhost.com
>>>>>>
>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>
> _______________________________________________
> To control your jdom-interest membership:
> http://www.jdom.org/mailman/options/jdom-interest/youraddr@yourhost.com
>