Http - Tunnel connection failed: 403 Forbidden error with Python web scraping
17,762
you should try to add proxy while requesting url.
proxyDict = {
'http' : "add http proxy",
'https' : "add https proxy"
}
requests.get(url, proxies=proxyDict)
you can find more information here
Author by
anonymous13
Updated on June 04, 2022Comments
-
anonymous13 almost 2 years
I am trying to web scrape a http website and I am getting below error when I am trying to read the website.
HTTPSConnectionPool(host='proxyvipecc.nb.xxxx.com', port=83): Max retries exceeded with url: http://campanulaceae.myspecies.info/ (Caused by ProxyError('Cannot connect to proxy.', OSError('Tunnel connection failed: 403 Forbidden',)))
Below is the code I have written with similar website. I tried using urllib and user-agent and still the same issue.
url = "http://campanulaceae.myspecies.info/" response = requests.get(url, headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/70.0.3538.77 Safari/537.36'}) soup = BeautifulSoup(response.text, 'html.parser')
Can anyone help me with the issue. Thanks in advance
-
Jakob F about 4 yearsHow often do you try to scrape it?
-
Andrés Quiroz Valdovinos about 3 yearsHow did you solve it?
-
-
anonymous13 about 4 yearsI tried adding proxies, it doesn't show any error. But when I trying to extract text it shows that the web page is blocked.***<html> <head> <title>Web Page Blocked</title> <meta content="text/html; charset=utf-8" http-equiv="Content-Type"/> ***