BeautifulSoup: object of type 'Response' has no len()

Question

Issue: when I try to execute the script, BeautifulSoup(html, ...) gives the error message "TypeError: object of type 'Response' has no len(). I tried passing the actual html as a parameter, but it still doesn't work.

import requests

url = 'http://vineoftheday.com/?order_by=rating'
response = requests.get(url)
html = response.content

soup = BeautifulSoup(html, "html.parser")

Solution 1

You are getting response.content. But it return response body as bytes (docs). But you should pass str to BeautifulSoup constructor (docs). So you need to use the response.text instead of getting content.

Solution 2

Try to pass the HTML text directly

soup = BeautifulSoup(html.text)

Solution 3

html.parser is used to ignore the warnings in the page:

soup = BeautifulSoup(html.text, "html.parser")

Solution 4

If you're using requests.get('https://example.com') to get the HTML, you should use requests.get('https://example.com').text.

Solution 5

you are getting only response code in 'response' and always use browser header for security otherwise you will face many issues

Find header in debugger console network section 'header' UserAgent

Try

import requests
from bs4 import BeautifulSoup

from fake_useragent import UserAgent

url = 'http://www.google.com'
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_13_6) 
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36'}

response = requests.get(quote_page, headers=headers).text

soup = BeautifulSoup(response, 'html.parser')
print(soup.prettify())

View more solutions

Comments

[xKobalt]

Welcome to StackOverflow! Please, provide some explanation about your solution