Error 'failed to load external entity' when using Python lxml

python xml lxml elementtree

39,946

Solution 1

In concert with what mzjn said, if you do want to pass a string to etree.parse(), just wrap it in a StringIO object.

Example:

from lxml import etree
from StringIO import StringIO

myString = "<html><p>blah blah blah</p></html>"

tree = etree.parse(StringIO(myString))

This method is used in the lxml documentation.

Solution 2

etree.parse(source) expects source to be one of

a file name/path
a file object
a file-like object
a URL using the HTTP or FTP protocol

The problem is that you are supplying the XML content as a string.

You can also do without urllib2.urlopen(). Just use

tree = etree.parse("http://www.greenbuttondata.org/data/15MinLP_15Days.xml")

Demonstration (using lxml 2.3.4):

>>> from lxml import etree
>>> tree = etree.parse("http://www.greenbuttondata.org/data/15MinLP_15Days.xml")
>>> tree.getroot()
<Element {http://www.w3.org/2005/Atom}feed at 0xedaa08>
>>>

In a competing answer, it is suggested that lxml fails because of the stylesheet referenced by the processing instruction in the document. But that is not the problem here. lxml does not try to load the stylesheet, and the XML document is parsed just fine if you do as described above.

If you want to actually load the stylesheet, you have to be explicit about it. Something like this is needed:

from lxml import etree

tree = etree.parse("http://www.greenbuttondata.org/data/15MinLP_15Days.xml")

# Create an _XSLTProcessingInstruction object
pi = tree.xpath("//processing-instruction()")[0] 

# Parse the stylesheet and return an ElementTree
xsl = pi.parseXSL()

Solution 3

lxml docs for parse says To parse from a string, use the fromstring() function instead.

parse(...)
    parse(source, parser=None, base_url=None)

    Return an ElementTree object loaded with source elements.  If no parser
    is provided as second argument, the default parser is used.

    The ``source`` can be any of the following:

    - a file name/path
    - a file object
    - a file-like object
    - a URL using the HTTP or FTP protocol

    To parse from a string, use the ``fromstring()`` function instead.

    Note that it is generally faster to parse from a file path or URL
    than from an open file object or file-like object.  Transparent
    decompression from gzip compressed sources is supported (unless
    explicitly disabled in libxml2).

Solution 4

You're getting that error because the XML you're loading references an external resource:

<?xml-stylesheet type="text/xsl" href="GreenButtonDataStyleSheet.xslt"?>

LXML doesn't know how to resolve GreenButtonDataStyleSheet.xslt. You and I probably realize that it's going to be available relative to your original URL, http://www.greenbuttondata.org/data/15MinLP_15Days.xml...the trick is to tell lxml how to go about loading it.

The lxml documentation includes a section titled "Document loading and URL resolving", which has just about all the information you need.

View more solutions

39,946

Author by

daveeloo

Updated on August 19, 2020

Comments

daveeloo almost 4 years

I'm trying to parse an XML document I retrieve from the web, but it crashes after parsing with this error:

': failed to load external entity "<?xml version="1.0" encoding="UTF-8"?>
<?xml-stylesheet type="text/xsl" href="GreenButtonDataStyleSheet.xslt"?>

That is the second line in the XML that is downloaded. Is there a way to prevent the parser from trying to load the external entity, or another way to solve this? This is the code I have so far:

import urllib2
import lxml.etree as etree

file = urllib2.urlopen("http://www.greenbuttondata.org/data/15MinLP_15Days.xml")
data = file.read()
file.close()

tree = etree.parse(data)

daveeloo about 12 years

Do you know if it's possible to turn off loading all external resources? I looked in the documentation but couldn't find anything.
mzjn almost 12 years

"You're getting that error because the XML you're loading references an external resource". No. That is not why you get the error. Please see my answer.
mzjn almost 12 years

@Duke: Thank you! It's nice to finally get some positive feedback.
MightyPork over 10 years

I'm getting this error while parsing a URL. Any idea how to disable loading those "external entities"? I have no interest in stylesheets, just want to parse anchors from the page.
mzjn over 10 years

@MightyPork: This question is not really about "external entities"; the error message is misleading. The problem here is that the OP uses etree.parse() on a string object, which does not work. If you have a related problem, I think you should ask a new question.
MightyPork over 10 years

@mzjn I asked indeed, but all advice I got was to use different library to fetch the url. I thought you have some more insight into the problem and could help.
Iharob Al Asimi over 8 years

Yes downvoter, this is very probably the problem. It was in my case.
vault over 7 years

etree.parse('google.com'): OSError: Error reading file 'google.com': failed to load external entity "google.com"
Adversus almost 7 years

For python3: from io import StringIO
Shane Lu over 6 years

In my case, I got the error because I tried to open an https url with parse.
mzjn over 6 years

@ShaneLu: Yes, for some reason HTTPS does not work. Only HTTP and FTP protocols are supported (as stated in the lxml documentation).