How to configure ChromeDriver to initiate Chrome browser in Headless mode through Selenium?

Question

I'm working on a python script to web-scrape and have gone down the path of using Chromedriver as one of the packages. I would like this to operate in the background without any pop-up windows. I'm using the option 'headless' on chromedriver and it seems to do the job in terms of not showing the browser window, however, I still see the .exe file running. See the screenshot of what I'm talking about. Screenshot

This is the code I am using to initiate ChromeDriver:

options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches",["ignore-certificate-errors"])
options.add_argument('headless')
options.add_argument('window-size=0x0')
chrome_driver_path = "C:\Python27\Scripts\chromedriver.exe"

Things I've tried to do is alter the window size in the options to 0x0 but I'm not sure that did anything as the .exe file still popped up.

Any ideas of how I can do this?

I am using Python 2.7 FYI

Solution 1

So after correcting my code to:

options = webdriver.ChromeOptions()
options.add_experimental_option("excludeSwitches",["ignore-certificate-errors"])
options.add_argument('--disable-gpu')
options.add_argument('--headless')
chrome_driver_path = "C:\Python27\Scripts\chromedriver.exe"

The .exe file still came up when running the script. Although this did get rid of some extra output telling me "Failed to launch GPU process".

What ended up working is running my Python script using a .bat file

So basically,

1. Save python script if a folder

2. Open text editor, and dump the following code (edit to your script of course)

   c:\python27\python.exe c:\SampleFolder\ThisIsMyScript.py %*

3. Save the .txt file and change the extension to .bat

4. Double click this to run the file

So this just opened the script in Command Prompt and ChromeDriver seems to be operating within this window without popping out to the front of my screen and thus solving the problem.

Solution 2

It should look like this:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.add_argument('--headless')
options.add_argument('--disable-gpu')  # Last I checked this was necessary.
driver = webdriver.Chrome(CHROMEDRIVER_PATH, chrome_options=options)

This works for me using Python 3.6, I'm sure it'll work for 2.7 too.

Update 2018-10-26: These days you can just do this:

from selenium import webdriver
from selenium.webdriver.chrome.options import Options

options = Options()
options.headless = True
driver = webdriver.Chrome(CHROMEDRIVER_PATH, options=options)

Solution 3

Answer update of 13-October-2018

To initiate a google-chrome-headless browsing context using Selenium driven ChromeDriver now you can just set the --headless property to true through an instance of Options() class as follows:

• Effective code block:

  from selenium import webdriver
  from selenium.webdriver.chrome.options import Options
  
  options = Options()
  options.headless = True
  driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
  driver.get("http://google.com/")
  print ("Headless Chrome Initialized")
  driver.quit()
  

---

Answer update of 23-April-2018

Invoking google-chrome in headless mode programmatically have become much easier with the availability of the method set_headless(headless=True) as follows :

• Documentation :

  set_headless(headless=True)
      Sets the headless argument
  
      Args:
          headless: boolean value indicating to set the headless option
  

• Sample Code :

  from selenium import webdriver
  from selenium.webdriver.chrome.options import Options
  
  options = Options()
  options.set_headless(headless=True)
  driver = webdriver.Chrome(options=options, executable_path=r'C:\path\to\chromedriver.exe')
  driver.get("http://google.com/")
  print ("Headless Chrome Initialized")
  driver.quit()
  

Note : --disable-gpu argument is implemented internally.

---

Original Answer of Mar 30 '2018

While working with Selenium Client 3.11.x, ChromeDriver v2.38 and Google Chrome v65.0.3325.181 in Headless mode you have to consider the following points :

• You need to add the argument --headless to invoke Chrome in headless mode.

• For Windows OS systems you need to add the argument --disable-gpu

• As per Headless: make --disable-gpu flag unnecessary --disable-gpu flag is not required on Linux Systems and MacOS.

• As per SwiftShader fails an assert on Windows in headless mode --disable-gpu flag will become unnecessary on Windows Systems too.

• Argument start-maximized is required for a maximized Viewport.

• Here is the link to details about Viewport.

• You may require to add the argument --no-sandbox to bypass the OS security model.

• Effective windows code block :

  from selenium import webdriver
  from selenium.webdriver.chrome.options import Options
  
  options = Options()
  options.add_argument("--headless") # Runs Chrome in headless mode.
  options.add_argument('--no-sandbox') # Bypass OS security model
  options.add_argument('--disable-gpu')  # applicable to windows os only
  options.add_argument('start-maximized') # 
  options.add_argument('disable-infobars')
  options.add_argument("--disable-extensions")
  driver = webdriver.Chrome(chrome_options=options, executable_path=r'C:\path\to\chromedriver.exe')
  driver.get("http://google.com/")
  print ("Headless Chrome Initialized on Windows OS")
  

• Effective linux code block :

  from selenium import webdriver
  from selenium.webdriver.chrome.options import Options
  
  options = Options()
  options.add_argument("--headless") # Runs Chrome in headless mode.
  options.add_argument('--no-sandbox') # # Bypass OS security model
  options.add_argument('start-maximized')
  options.add_argument('disable-infobars')
  options.add_argument("--disable-extensions")
  driver = webdriver.Chrome(chrome_options=options, executable_path='/path/to/chromedriver')
  driver.get("http://google.com/")
  print ("Headless Chrome Initialized on Linux OS")
  

---

Steps through YouTube Video

How to initialize Chrome Browser in Maximized Mode through Selenium

Outro

How to make firefox headless programmatically in Selenium with python?

---

tl; dr

Here is the link to the Sandbox story.

Solution 4

Update August 20, 2020 -- Now is simple!

chrome_options = webdriver.ChromeOptions()
chrome_options.headless = True

self.driver = webdriver.Chrome(
            executable_path=DRIVER_PATH, chrome_options=chrome_options)

Solution 5

UPDATED It works fine in my case:

from selenium import webdriver

options = webdriver.ChromeOptions()
options.headless = True
driver = webdriver.Chrome(CHROMEDRIVER_PATH, options=options)

Just changed in 2020. Works fine for me.

View more solutions

Comments

[Maz]

Thanks! This unfortunately didn't solve the issue. I have posted my answer to what did though. Appreciate your help

[TomKivy]

Latest update works when replacing: options = Options() with options = webdriver.ChromeOptions()

[Jortega]

One more edit needed here use chrome_options=options not options=options

[Jortega]

Another one: use options.add_argument("--headless") not options.headless = True

[Python_Learner_DK]

@DebanjanB, When I use your 13-Oct-18 code it executes, but throws the following errors: ` "Error parsing a meta element's content: ';' is not a valid key-value pair separator. Please use ',' instead."` and "Scripts may close only the windows that were opened by it." and "Synchronous XMLHttpRequest on the main thread is deprecated because of its detrimental effects to the end user's experience... for the website https://test.plexonline.com - browser with a 'head' executes with no errors though... any ideas?

[undetected Selenium]

@Python_Learner_DK The compete error stack trace may give us some clue. Can you raise a new question along with your binary versions please?

[Python_Learner_DK]

@DebanjanB, here is the new question: stackoverflow.com/questions/60058246/…

[Rene]

@Jortega, chrome_options is depreciated so options=options is correct.

[Red Gundu]

These are the chrome options that are needed to set in order to make chrome work in headless mode with the latest Chrome driver version 84

[x3l51]

Can you explain your answer?

[gavin]

Update: kwarg chrome_options for Chrome is deprecated in favour of options

[Andrew Anderson]

didn't work for me. Needed go through options.add_argument('headless')

[Cas]

This is duplicates existing answers and is incorrect since set_headless is a method and is actually deprecated: DeprecationWarning: use setter for headless property instead of set_headless so should use options.headless

[jaycode]

The solution does not work when the website has javascript. I added options.add_argument('user-agent=fake-useragent') and it worked.