Can I find an element using regex with Python and Selenium?

Question

I need to click a dropdown list and click a hidden element with in it. the html will be generated by javascript and I won't know the id or class name but I will know it will have a phrase in it. Can I find and element by regex and then click it with selenium?

Solution 1

You cannot simply do regex-based search with built-in selenium webdriver locators, but you have multiple things that might help you:

• contains() and starts-with() XPath functions:

  //div[contains(., "Desired text")]
  //div[starts-with(., "Desired text")]
  

• preceding, preceding-sibling, following and following-sibling axis that might help you if you know the relative position of an newly generated block of elements you need to locate

---

There are also CSS selectors for partial match on element attributes:

a[href*=desiredSubstring]  # contains
a[href^=desiredSubstring]  # starts-with
a[href$=desiredSubstring]  # ends-with

---

And you can always find more elements than needed and filter them out later in Python, example:

import re

pattern = re.compile(r"^Some \w+ text.$")

elements = driver.find_elements_by_css_selector("div.some_class")
for element in elements:
    match = pattern.match(element.text)
    if match:
        print(element.text)

Solution 2

You can use import re to perform regex functions. The snippet below looks through a table and grabs the text between the <b></b> tags in the first cell if the row has 3 cells in it.

import re
from lxml import html, etree

tree = html.fromstring(browser.page_source)
party_table = tree.xpath("//table")
assert len(party_table) == 1

CURRENT_PARTIES = []
for row in party_table[0].xpath("tbody/tr"):
    cells = row.xpath("td")
    if len(cells) != 3:
        continue

    if cells[1].text == "represented by":
        match = re.search(r'<b>(.+?)</b>', etree.tostring(cells[0]), re.IGNORECASE)
        print "MATCH: ", match