How to get path of an element in lxml?

Question

I'm searching in a HTML document using XPath from lxml in python. How can I get the path to a certain element? Here's the example from ruby nokogiri:

page.xpath('//text()').each do |textnode|
    path = textnode.path
    puts path
end

print for example '/html/body/div/div[1]/div[1]/p/text()[1]' and this is the string I want to get in python.

Solution 1

Use getpath from ElementTree objects.

from lxml import etree
    
root = etree.fromstring('''
    <foo><bar>Data</bar><bar><baz>data</baz>
    <baz>data</baz></bar></foo>
    ''')
    
tree = etree.ElementTree(root)
for e in root.iter():
    print(tree.getpath(e))

Prints

/foo
/foo/bar[1]
/foo/bar[2]
/foo/bar[2]/baz[1]
/foo/bar[2]/baz[2]

Solution 2

See the Xpath and XSLT with lxml from the lxml documentation This gives the path of the element containg the text

An example would be

import cStringIO
from lxml import etree

f = cStringIO.StringIO('<foo><bar><x1>hello</x1><x1>world</x1></bar></foo>')
tree = lxml.etree.parse(f)
find_text = etree.XPath("//text()")

# and print out the required data
print [tree.getpath( text.getparent()) for text in find_text(tree)]

# answer I get is 
>>> ['/foo/bar/x1[1]', '/foo/bar/x1[2]']

Solution 3

If all you have in your section of code is the element and you want the element's xpath do then element.getroottree().getpath(element) will do the job.

from lxml import etree

xml = '''
<test>
    <a/>
    <b>
       <i/>
       <ii/>
    </b>
</test>
'''
tree = etree.fromstring(xml)

for element in tree.iter():
    print element.getroottree().getpath(element)

Solution 4

root = etree.parse(open('tmp.txt'))

for e in root.iter():
    print root.getpath(e)

View more solutions

Comments

[Jabba]

I think it should be for e in tree.iter():, i.e. tree.iter.

[nosklo]

@Jabba And why do you think that? Have you tried the code I provided the way it is? It seems to work, no? Do you have a reason to think otherwise?

[Kevin Vasko]

It might have not existed when you wrote this originally and not that it actually matters, but you can also do tree = root.getroottree() to get an ElementTree object.

[johann_ka]

Just for the sake of completion, when I use https://requests-html.kennethreitz.org/ to pull a webpage, I do the following: webpage = session.get('URL HERE') then root = etree.HTML(webpage.text) then tree = etree.ElementTree(root) and then simply the for loop as stated in the answer.