python - Parse several XML declarations in a single file by means of lxml.etree.iterparse

Question

I need to parse a file that contains various XML files, i.e., <xml></xml> <xml></xml> .. and so forth. While using etree.iterparse, I get the following (correct) error:

lxml.etree.XMLSyntaxError: XML declaration allowed only at the start of the document

Now, I can preprocess the input file and produce for each contained XML file a separate file. This might be the easiest solution. But I wonder if a proper solution for this 'problem' exists.

Thanks!

Answer 1

The sample data you've provided suggests one problem, while the question and the exception you've provided suggests another. Do you have multiple XML documents concatenated together, each with its own XML declaration, or do you have an XML fragment with multiple top-level elements?

If it's the former, then the solution's going to involve breaking the input stream up into multiple streams, and parsing each one individually. This doesn't necessarily mean, as one comment suggests, implementing an XML parser. You can search a string for XML declarations without having to parse anything else in it, so long as your input doesn't include CDATA sections that contain unescaped XML declarations. You can write a file-like object that returns characters from the underlying stream until it hits an XML declaration, and then wrap it in a generator function that keeps returning streams until EOF is reached. It's not trivial, but it's not hugely difficult either.

If you have an XML fragment with multiple top-level elements, you can just wrap them an XML element and parse the whole thing.

Of course, as with most problems involving bad XML input, the easiest solution may just be to fix the thing that's producing the bad input.