Selenium: How to make geckodriver headless - selenium

I have completed code for geckodriver and was wondering how to make geckodriver headless now. I saw a post previously with the following text:
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
options = Options()
options.add_argument("--headless")
driver = webdriver.Firefox(firefox_options=options,
executable_path="C:\\Utility\\BrowserDrivers\\geckodriver.exe")
print("Firefox Headless Browser Invoked")
driver.get('http://google.com/')
driver.quit()
I don't understand where the download for options came from under webdriver. When I downloaded geckodriver, all that came with it was the executable file. Any help is greatly appreciated!!

Works for me. Steps I used: (1) Open a command prompt and navigate to the folder containing geckodriver.exe. (2) Start geckodriver.exe without any options from a command prompt. (3) Open another command prompt and type python and press the Return key. (4) Copy/paste the following code into your your python session.
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver import Firefox
options = Options()
options.add_argument("--headless")
# Don't put the path to geckodriver in the following. But the firefox executable
# must be in the path. If not, include the path to firefox, not geckodriver below.
driver = Firefox(firefox_options=options)
print("Firefox Headless Browser Invoked")
driver.get('http://google.com/')
# Print the first 300 characters on the page.
print(driver.page_source[:300])
driver.quit()

Related

Databricks - Selenium - Open browser tab with

I have successfully installed Selenium in Databricks and can import the Python selenium and webdriver. On my local computer once I run the Selenium get command a separate browser windows open where I can see what Selenium is doing.
However, when running the same script on Databricks there is unfortunately no window opening. I am wondering if this is at all possible. I found some options like
browser.execute_script('''window.open("http://bings.com","_blank");''')
or
add_experimental_option
but these options did not work.
I am currently using the following options on Databricks:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from webdriver_manager.utils import ChromeType
import pandas as pd
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-dev-shm-usage')
# installed driver
chrome_driver = "/tmp/chromedriver1/chromedriver"
driver = webdriver.Chrome(executable_path=chrome_driver, options=chrome_options)
Has someone an idea how this could work?
Thanks in advance!
No it won't open since you are using headless browser. See the below line from your code :-
chrome_options.add_argument('--headless')
Having said that, it will execute the Python-Selenium bindings instruction, you have. You can try to print the page title like this :
driver = webdriver.Chrome(executable_path=chrome_driver, options=chrome_options)
driver.get("https://www.google.com")
print(driver.title)
should print google.com title in the console.

Close Chromewebdriver when user close tab in Selenium

When using Selenium to create a Chrome app, chromedriver.exe still in background when user close the tab manually. Is there any way to fix this?
I don't want to use driver.quit() in my code, because user need to do something in this chrome app.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--app=https://www.jianshu.com/sign_in')
driver = webdriver.Chrome(".\\chromedriver.exe", options=chrome_options)
driver.find_element_by_xpath('//*[#id=\"session_email_or_mobile_number\"]').send_keys('aaaa')
driver.find_element_by_xpath('//*[#id=\"session_password\"]').send_keys('bbbbbbbbbb')
I think we don't have a direct solution to close the chromedriver.exe when user manually close the chrome instance launched by selenium. Alternatively you can save the below command as bat and then double click each time you close the chrome tab manually.
taskkill /IM chromedriver.exe /F
`from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--app=https://www.jianshu.com/sign_in')
driver = webdriver.Chrome(".\\chromedriver.exe", options=chrome_options)
driver.find_element_by_xpath('//*
[#id=\"session_email_or_mobile_number\"]').send_keys('aaaa')
driver.find_element_by_xpath('//*[#id=\"session_password\"]').send_keys('bbbbbbbbbb')
driver.close()
///just add driver.close() to close the chrome browser after the test

selenium.common.exceptions.SessionNotCreatedException: Message: session not created: No matching capabilities error with ChromeDriver Chrome Selenium

First, machine and package specs:
I am running:
ChromeDriver version 75.0.3770.140
Selenium: version '3.141.0'
WSL (linux subsystem) of windows 10
I am trying to run a chromebrowser through selenium. I found: these commands, to use selenium through google chrome.
I have a test directory, with only the chromedriver binary file, and the script, in it. The location of the directory is: /home/kela/test_dir/
I ran the code:
import selenium
from selenium import webdriver
from bs4 import BeautifulSoup
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
options = Options()
options.binary_location='/home/kela/test_dir/chromedriver'
driver = webdriver.Chrome(chrome_options = options,executable_path='/home/kela/test_dir/chromedriver')
The output from this code is:
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: No matching capabilities found
Can anyone explain why I need capabilities when the same script works for others without capabilities? I did try adding:
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
but I got the same error. So I'm not sure what capabilities I need to add (considering it works for others without it?)
Edit 1: Addressing DebanjanB's comments below:
Chromedriver is in the expected location. I am using windows 10. From here, the expected location is C:\Program Files (x86)\Google\Chrome\Application\chrome.exe; and this is where it is on my machine (I copied and pasted this location from the chrome Properties table).
ChromeDriver is having executable permission for non-root users.
I definitely have Google Chrome v75.0 installed (I can see that the Product version 75.0.3770.100)
I am running the script as a non-root user, as my bash command line ends with a $ and not # (i.e kela:~/test_dir$ and not kela:~/test_dir#)
Edit 2: Based on DebanjanB's answer below, I am very close to having it working, but just not quite.
The code:
import selenium
from selenium import webdriver
from bs4 import BeautifulSoup
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location='/c/Program Files (x86)/Google/Chrome/Application/chrome.exe'
driver = webdriver.Chrome(options=options)
driver.get('http://google.com/')
Produces a dialog box that reads:
Google Chrome cannot read and write to it's data directory: /tmp/.com/google.Chrom.gyw63s
So then I double checked my Chrome permissions and I should be able to write to Chrome:
Also, I can see that /tmp/ has a bunch of .com dirs in it:
.com.google.Chrome.4jnWme/ .com.google.Chrome.FdNyKP/ .com.google.Chrome.VAcWMQ/ .com.google.Chrome.ZbkRx0/ .com.google.Chrome.iRrceF/
.com.google.Chrome.A2QHHB/ .com.google.Chrome.G7Y51c/ .com.google.Chrome.WD8BtK/ .com.google.Chrome.cItmhA/ .com.google.Chrome.pm28hN/
However, since that seemed to be more of a warning than an error, I clicked 'ok' to close the dialog box, and a new tab does open in the browser; but the URL is just 'data:,'. The same thing happens if I remove the line 'driver.get('http://google.com')' from the script, so I know the warning/issue is with the line:
driver = webdriver.Chrome(chrome_options = options,executable_path='/home/kela/test_dir/chromedriver')
For example, from here, I tried adding:
options.add_argument('--profile-directory=Default')
But the same warning pops up.
Edit 3:
As edit 3 was starting to veer into a different question than specifically being addressed here, I started a new question here.
This error message...
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: No matching capabilities found
...implies that the ChromeDriver was unable to initiate/spawn a new WebBrowser i.e. Chrome Browser session.
binary_location
binary_location set/get(s) the location of the Chrome (executable) binary and is defined as:
def binary_location(self, value):
"""
Allows you to set where the chromium binary lives
:Args:
- value: path to the Chromium binary
"""
self._binary_location = value
So as per your code trials, options.binary_location='/home/kela/test_dir/chromedriver' is incorrect.
Solution
If Chrome is installed at the default location, you can safely remove this property. Incase Chrome is installed at a customized location you need to use the options.binary_location property to point to the Chrome installation.
You can find a detailed discussion in Selenium: WebDriverException:Chrome failed to start: crashed as google-chrome is no longer running so ChromeDriver is assuming that Chrome has crashed
Effectively, you code block will be:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location=r'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe'
driver = webdriver.Chrome(options=options, executable_path='/home/kela/test_dir/chromedriver.exe')
driver.get('http://google.com/')
Additionally, ensure the following:
ChromeDriver is having executable permission for non-root users.
As you are using ChromeDriver v75.0 ensure that you have the recommended version of the Google Chrome v75.0 as:
---------ChromeDriver 75.0.3770.8 (2019-04-29)---------
Supports Chrome version 75
Execute the Selenium Test as non-root user.

chromedriver can not be found in PATH, or when explicitly provided to webdriver.Chrome()

I am trying to use chromedriver with Selenium on Windows 10 but I get the following error:
Traceback (most recent call last):
File "scrape.py", line 4, in <module>
driver = webdriver.Chrome()
File "C:\Python37\lib\site-packages\selenium\webdriver\chrome\webdriver.py", line 73, in __init__
self.service.start()
File "C:\Python37\lib\site-packages\selenium\webdriver\common\service.py", line 83, in start
os.path.basename(self.path), self.start_error_message)
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
And here's my test script:
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
driver.get("http://www.python.org")
assert "Python" in driver.title
elem = driver.find_element_by_name("q")
elem.clear()
elem.send_keys("pycon")
elem.send_keys(Keys.RETURN)
assert "No results found." not in driver.page_source
driver.close()
I've tried many things. I'll detail below.
I've attempted to add the path to chromedriver to PATH. Image here:
This works fine because I can run chromedriver from the commandline:
C:\Users\KraftWurk>chromedriver
Starting ChromeDriver 74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}) on port 9515
Only local connections are allowed.
Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
But still, when I run my scripts I get the error that chromedriver needs to be in PATH ... it is, so not sure what's going on there.
I've read the following: Python Selenium Chrome Webdriver
I've attempted to provide the path as suggested using:
driver = webdriver.Chrome(executable_path=r"C:\drivers\chromedriver.exe")
I still get the same warning.
I'm not quite sure what's going on. I'm using Python 3.7 on Windows 10. Selenium 3.141.0 and Chromedriver 74.0.3729.6
To eliminates lot of manual works and incompatibility issues, I would suggest you to go for WebDriverManager as it automatically downloads required binary and we do not need to set any path.
It supports browsers such as Chrome, Firefox, PhantomJS, Microsoft Edge, or Internet Explorer.
How do we use this in our project?
Only setup required is to install this package using ‘pip’.
pip install webdriver_manager
That’s it! We are all set. Just import this module in your python project and start using it.
For Chrome:
from webdriver_manager.chrome import ChromeDriverManager
from selenium import webdriver
driver = webdriver.Chrome(executable_path=ChromeDriverManager().install())
driver.get("http://www.google.com/")
print driver.title
driver.quit()
For Firefox:
from webdriver_manager.firefox import GeckoDriverManager
from selenium import webdriver
driver = webdriver.Firefox(executable_path=GeckoDriverManager().install())
driver.get("http://www.google.com/")
print driver.title
driver.quit()
For Edge:
from webdriver_manager.microsoft import EdgeDriverManager
from selenium import webdriver
driver = webdriver.Edge(executable_path=EdgeDriverManager().install())
driver.get("http://www.google.com/")
print driver.title
driver.quit()
For IE:
from webdriver_manager.microsoft import IEDriverManager
from selenium import webdriver
driver = webdriver.Ie(executable_path=IEDriverManager().install())
driver.get("http://www.google.com/")
print driver.title
driver.quit()
webdriver_manager, by default, tries to download the latest version of a given driver binary. To use a specific version of driver, pass the driver version like below.
webdriver.Chrome(executable_path=ChromeDriverManager("2.42").install())

Is there selenium2 Chrome driver for Ruby/Python?

I want to use Selenium2 Chrome driver with Ruby/Python, but I only found those for Java and .NET on http://code.google.com/p/selenium/downloads/list . Could someone tell me is it possible to drive Chrome with Python or Ruby in Selenium2? Thanks in advance!
For python you have to download the chrome driver from http://code.google.com/p/chromium/downloads/list. Extract the zip file and set the chrome driver path in Path environment variable. Please see the below example using python -
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
driver = webdriver.Chrome()
driver.get("http://www.google.com")
elem = driver.find_element_by_name("q")
elem.send_keys("selenium")
elem.send_keys(Keys.RETURN)
assert "Google" in driver.title
driver.quit()
There definitely is for Python, you have to install the chrome driver, get it here for your appropriate OS: http://code.google.com/p/chromium/downloads/list. to start up a chrome instance do this:
self.webdriver = webdriver.Chrome(executable_path=[path to chrome driver])