IF YOUR MEMORY STILL SCALES LINEARLY
You need to periodically clear the root element - for example at the end of each depth one tag (zero-indexed).

Otherwise root element will keep pointers to every child which can consume quite a lot of memory if you are parsing a large file for example 50GB wikipedia dump...

In the second code you are missing one quote

Thanks for the walk through, really helped me a lot!

it is very slow((

I always use lxml. It works for my files (200-700Mb)

from lxml import etree
tree = etree.parse(self.xml_file)
root = tree.getroot()

you never defined xmL in for event, elem in etree.iterparse(xmL, events=('start', 'end', 'start-ns', 'end-ns')), what is it's definition ?

I'm new to Python, so please forgive the noob question. Is this the "preferred / recommended" method, or would something like lxml the way to go?

For reference: http://www.ibm.com/developerwo...

Thanks for the helpful info!

You might want to change the code at the "Iterate It" section. If event is "start-ns" or "end-ns", then elem will be a tupple and that code will return an error.

very useful

Good post!