how to find element by css selector using python / selenium

Question

i'm trying to pick up links of youtube channels which are located as below:

<a id="author-text" class="yt-simple-endpoint style-scope ytd-comment- 
renderer" href="/channel/UCUSy-h1fPG1L6X7KOe70asA"> <span class="style- 
scope ytd-comment-renderer">Jörgen Nilsson</span></a>

So in the example above I would want to pick up "/channel/UCUSy-h1fPG1L6X7KOe70asA". So far i have tried many options but none work:

driver = webdriver.Chrome('C:/Users/me/Chrome Web Driver/chromedriver.exe')
api_url="https://www.youtube.com/watch?v=TQG7m1BFeRc"
driver.get(api_url)
time.sleep(2) 
div = driver.find_element_by_class_name("yt-simple-endpoint style-scope ytd-comment-renderer")

but I get the following error: InvalidSelectorException: Message: invalid selector: Compound class names not permitted

I also tried other approaches:

div = driver.find_elements_by_xpath("yt-simple-endpoint style-scope ytd-comment-renderer")

div = driver.find_element_by_class_name('yt-simple-endpoint style-scope ytd-comment-renderer')

div=driver.find_element_by_css_selector('.yt-simple-endpoint style-scope ytd-comment-renderer').get_attribute('href')

but no luck.. if someone could please help it would be much appreciated. Thank you

Solution 1

Your selectors are invalid:

• driver.find_element_by_class_name("yt-simple-endpoint style-scope ytd-comment-renderer")

  you cannot pass more than one class name to find_element_by_class_name method. You can try driver.find_element_by_class_name("ytd-comment-renderer")

• driver.find_elements_by_xpath("yt-simple-endpoint style-scope ytd-comment-renderer")

  it's not a correct XPath syntax. You probably mean driver.find_elements_by_xpath("//*[@class='yt-simple-endpoint style-scope ytd-comment-renderer']")

• driver.find_element_by_css_selector('.yt-simple-endpoint style-scope ytd-comment-renderer')

  each class name should start with the dot: driver.find_element_by_css_selector('.yt-simple-endpoint.style-scope.ytd-comment-renderer')

But the best way IMHO to identify by ID:

driver.find_element_by_id("author-text")

Solution 2

You can use BeautifulSoup in python to get the links in anchor tag having specific class names like soup.find_all('a', attrs={'class':'yt-simple-endpoint'}) you can read more here find_all using css

Comments

[entropy]

Have you tried find_element_by_id('#author-text')?

[tezzaaa]

Thanks I will try as soon as I can, im travelling at present

[tezzaaa]

Hi, thank you for your help, it almost works, it picks up the user names, for example, in the following : <a id="author-text" class="yt-simple-endpoint style-scope ytd-comment- renderer" href="/channel/UCUSy-h1fPG1L6X7KOe70asA"> <span class="style- scope ytd-comment-renderer">Jörgen Nilsson</span></a> it picks up Jörgen Nilsson but i want to capture /channel/UCUSy-h1fPG1L6X7KOe70asA ... ?

[tezzaaa]

forgot to stipulate above, driver.find_elements_by_xpath("//*[@class='yt-simple-endpoin‌​t style-scope ytd-comment-renderer']") is the one that almost works

[JaSON]

@tezzaaa, add .get_attribute('href') to extract link