Python - Selenium and XPATH to extract all rows from a table

Question

I am using Selenium and XPATH to extract all rows from a table, but can only get the first row.

Here is what I am doing:

from selenium import webdriver

path_to_chromedriver = '/Users/me/Desktop/chromedriver'
browser = webdriver.Chrome(executable_path = path_to_chromedriver)

url = "http://www.psacard.com/smrpriceguide/SetDetail.aspx?SMRSetID=1055"

browser.get(url)
browser.implicitly_wait(10)

SMRtable = browser.find_element_by_xpath('//*[@class="set-detail-table"]/tbody')

for i in SMRtable.find_element_by_xpath('.//tr'):
    print i.get_attribute('innerHTML')

browser.close()

The SMRtable variable has all the rows in it when I convert to string and print. When I try to loop through it, it throw a not iterable error.

I also tried using browser.find_element_by_xpath('//*[@class="set-detail-table"]/tbody/tr'), but this only gives me the first row. I tried adding [position()>0] after /tr, but still got just the first row.

How can I get all of the rows?

Solution

You need find_elements_by_xpath() (watch the "s") instead:

for i in SMRtable.find_elements_by_xpath('.//tr'):
    print i.get_attribute('innerHTML')

Comments

[jdesilvio]

I hate when that happen's'. Thanks.

[Daman deep]

what is 'innerHTML ? CSS or xpath?'

[aayush_malik]

using print(i.get_attribute('innerHTML') I get the HTML Table Code. How do I render that to DataFrame or CSV? Thanks