python selenium find child elements in loop
10,175
The XPath expression in the loop has to start with a dot to be context-specific:
for article in article_elements:
title = article.find_element_by_xpath('.//article/h2').text
share_count = article.find_element_by_xpath('.//footer/div/a/span').text
poinst = article.find_element_by_xpath('.//footer/div[2]/div[1]/div[3]').text
meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst})
As a side note, you can shorten the code by using a list comprehension:
meta_info_list = [{
'title': article.find_element_by_xpath('.//article/h2').text,
'share count': article.find_element_by_xpath('.//footer/div/a/span').text,
'points': article.find_element_by_xpath('.//footer/div[2]/div[1]/div[3]').text
} for article in article_elements]
Author by
Konstantin Rusanov
Updated on June 04, 2022Comments
-
Konstantin Rusanov almost 2 years
I need to parse some child elements in all parent elements on page.
Create list of all articles on page
article_elements = driver.find_elements_by_tag_name('article')
And after tying to get child elements in for loop and append all results to list
for article in article_elements: title = article.find_element_by_xpath('//article/h2').text share_count = article.find_element_by_xpath('//footer/div/a/span').text poinst = article.find_element_by_xpath('//footer/div[2]/div[1]/div[3]').text meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst})
After loop ends i got 40 times one same article meta (of first article)
{'share count': u'66', 'points': u'53 points', 'title': u'25+ Random Acts Of Genius Vandalism'} {'share count': u'66', 'points': u'53 points', 'title': u'25+ Random Acts Of Genius Vandalism'} {'share count': u'66', 'points': u'53 points', 'title': u'25+ Random Acts Of Genius Vandalism'} {'share count': u'66', 'points': u'53 points', 'title': u'25+ Random Acts Of Genius Vandalism'} ... 40 times
My whole code
# coding: utf8 from selenium import webdriver from selenium.webdriver.common.keys import Keys import time driver = webdriver.Chrome() driver.set_window_size(1024,768) driver.get('http://www.boredpanda.com/') time.sleep(2) meta_info_list = [] article_elements = driver.find_elements_by_tag_name('article') for article in article_elements: title = article.find_element_by_xpath('//article/h2').text share_count = article.find_element_by_xpath('//footer/div/a/span').text poinst = article.find_element_by_xpath('//footer/div[2]/div[1]/div[3]').text meta_info_list.append({'title':title, 'share count':share_count, 'points':poinst}) for list in meta_info_list: print(list)