How to capture network traffic using selenium webdriver and browsermob proxy on Python?
Just stumbled across this project https://github.com/derekargueta/selenium-profiler. Spits out all network data for a URL. Shouldn't be hard to hack and integrate into whatever tests you're running.
Original source: https://www.openhub.net/p/selenium-profiler
Jose
Updated on July 22, 2022Comments
-
Jose almost 2 years
I would like to capture network traffic by using Selenium Webdriver on Python. Therefore, I must use a proxy (like BrowserMobProxy)
When I use webdriver.Chrome:
from browsermobproxy import Server server = Server("~/browsermob-proxy") server.start() proxy = server.create_proxy() from selenium import webdriver co = webdriver.ChromeOptions() co.add_argument('--proxy-server={host}:{port}'.format(host='localhost', port=proxy.port)) driver = webdriver.Chrome(executable_path = "~/chromedriver", chrome_options=co) proxy.new_har driver.get(url) proxy.har # returns a HAR for ent in proxy.har['log']['entries']: print ent['request']['url']
the webpage is loaded properly and all requests are available and accessible in the HAR file. But when I use webdriver.Firefox:
# The same as above # ... from selenium import webdriver profile = webdriver.FirefoxProfile() driver = webdriver.Firefox(firefox_profile=profile, proxy = proxy.selenium_proxy()) proxy.new_har driver.get(url) proxy.har # returns a HAR for ent in proxy.har['log']['entries']: print ent['request']['url']
The webpage cannot be loaded properly and the number of requests in the HAR file is smaller than the number of requests that should be.
Do you have any idea what the problem of proxy settings in the second code? How should I fix it to use webdriver.Firefox properly for my purpose?