How to access Network panel on google chrome developer tools with selenium?

Question

I want to get the output that is shown on the network panel of the developer tools.

[Network panel --> Name, Method, Status, Type, Initiator, Size, Time, Timeline]

I need this information.

Solution 1

This possible via Selenium WebDriver. For this you should do the following:

1. Download selenium language-specific client drivers from - http://docs.seleniumhq.org/download/ and add apropriate jar files to your project build path.

2. To run a test with Chrome/Chromium you will also need chromdriver binary which you can download from - http://chromedriver.storage.googleapis.com/index.html

3. Create a test case like this:

    // specify the path of the chromdriver binary that you have downloaded (see point 2)
    System.setProperty("webdriver.chrome.driver", "/root/Downloads/chromedriver");
    ChromeOptions options = new ChromeOptions();
    // if you like to specify another profile
    options.addArguments("user-data-dir=/root/Downloads/aaa"); 
    options.addArguments("start-maximized");
    DesiredCapabilities capabilities = DesiredCapabilities.chrome();
    capabilities.setCapability(ChromeOptions.CAPABILITY, options);
    WebDriver driver = new ChromeDriver(capabilities);
    driver.get("http://www.google.com");
    String scriptToExecute = "var performance = window.performance || window.mozPerformance || window.msPerformance || window.webkitPerformance || {}; var network = performance.getEntries() || {}; return network;";
    String netData = ((JavascriptExecutor)driver).executeScript(scriptToExecute).toString();

Executing javascript on Chrome/Chromium will help you to get the networking (not only) info. The resulting string 'netData' will contain the required data in JSONArray format.

Hope this will help.

Solution 2

From this answer.

You can use the LoggingPreferences to get the Performance logs. It returns the data in json format. Here is a sample java code. Tested this with selenium 2.53, chromedriver 2.20, Chrome 50 on Ubuntu 14.04. This should work on windows also.

    DesiredCapabilities d = DesiredCapabilities.chrome();
    LoggingPreferences logPrefs = new LoggingPreferences();
    logPrefs.enable(LogType.PERFORMANCE, Level.ALL);
    d.setCapability(CapabilityType.LOGGING_PREFS, logPrefs);
    WebDriver driver = new ChromeDriver(d);
    driver.get("http://www.google.com");
    LogEntries les = driver.manage().logs().get(LogType.PERFORMANCE);
    for (LogEntry le : les) {
        System.out.println(le.getMessage());
    }

Here is a sample output. It is formatted manually. The actual ouput is in a single line.

{
    "message": {
        "method": "Network.requestWillBeSent",
        "params": {
            "documentURL": "https://www.google.co.in/?gfe_rd=cr&ei=gpwxV4OSKMmR2ASEg6-YCg&gws_rd=ssl",
            "frameId": "31172.2",
            "initiator": {
                "stack": {
                    "callFrames": [
                        {
                            "columnNumber": 11511,
                            "functionName": "",
                            "lineNumber": 55,
                            "scriptId": "50",
                            "url": "https://www.google.co.in/?gfe_rd=cr&ei=gpwxV4OSKMmR2ASEg6-YCg&gws_rd=ssl"
                        }
                    ]
                },
                "type": "script"
            },
            "loaderId": "31172.3",
            "request": {
                "headers": {
                    "Accept": "*/*",
                    "Referer": "https://www.google.co.in/",
                    "User-Agent": "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/50.0.2661.94 Safari/537.36"
                },
                "initialPriority": "Low",
                "method": "GET",
                "mixedContentType": "none",
                "url": "https://www.google.co.in/xjs/_/js/k=xjs.s.en.VTDhrkH4c9U.O/m=sx,c,sb,cdos,cr,elog,jsa,r,hsm,qsm,j,p,d,csi/am=AJQ0CwoS8fchIGwhrCA1YGBR/rt=j/d=1/t=zcms/rs=ACT90oGi2YIjVL5cBzOc1-MD37a1NqZ1jA"
            },
            "requestId": "31172.3",
            "timestamp": 251208.074288,
            "type": "Other",
            "wallTime": 1462869123.92204
        }
    },
    "webview": "8AF4A466-8027-4340-B9E9-CFEBDA769C50"
}

Solution 3

Let's say you want to load a page (e.g. google.com) and extract array of resource timing objects (i.e. window.performance.getEntries()):

import time
from selenium import webdriver
driver = webdriver.Chrome('/path/to/chromedriver)
driver.get('https://www.google.com');
time.sleep(5)
timings = driver.execute_script("return window.performance.getEntries();")
print timings

Solution 4

As noted in other answers, you need to utilize window.performance methods.

getEntries()
getEntriesByType()
getEntriesByName()
» Entry Types

For instance, I've used the following snippet within a nodejs Selenium-WebDriver: Chromedriver test to collect google analytics network calls:

driver
  .executeScript( "return window.performance.getEntriesByType('resource');" )
  .then( (perfEntries)=> {
    let gaCalls = perfEntries.filter(function(entry){
      return /collect\?/i.test(entry.name);
    });
    console.log(gaCalls);
});

If you used getEntries() instead of getEntriesByType('resource'), it would return...all entries.

Solution 5

With Python, one way to do it is:

from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
import chromedriver_binary # If you're using conda like me.

yoururl = "www.yoururl.com"

caps = DesiredCapabilities.CHROME
caps['goog:loggingPrefs'] = {'performance': 'ALL'}
driver = webdriver.Chrome(desired_capabilities=caps)

driver.get(yoururl)
time.sleep(10) # wait for all the data to arrive. 
perf = driver.get_log('performance')

perf is a list of dictionaries and you'll be able to find the item you're looking for in this list. i.e, dictionaries are the things that you see in Chrome's dev tool network tab.

View more solutions