Get Selenium to work with Brave browser on Linux - selenium

System:
OS: Ubuntu 20.04.2 LTS
Python: 3.8.5
selenium: 3.141.0
ChromeDriver: 90.0.4430.24
Brave: Version 1.24.86 Chromium: 90.0.4430.212 (Official Build) (64-bit)
I am trying to get Selenium to work with Brave.
I am using the script from here.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location = '/snap/bin/brave'
driver_path = '/usr/local/bin/chromedriver'
drvr = webdriver.Chrome(options=options, executable_path=driver_path)
drvr.get('https://stackoverflow.com')
I keep getting this error
selenium.common.exceptions.WebDriverException: Message: unknown error: DevToolsActivePort file doesn't exist
And I have tried all the answers in stackflow
I tried all different combinations of these arguments but nothing worked:(
options.add_argument("start-maximized")
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
options.add_argument("--disable-gpu")
options.add_argument("--disable-dev-shm-usage")
options.add_argument("--no-sandbox")
options.add_argument('--remote-debugging-port=9222')
options.add_argument('--headless')
Can anyone help me out?

I was able to get it working by changing the remote-debugging-port to a different port.
Here is my code that worked.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
#driver_path = '/usr/local/bin/operadriver'
driver_path = '/usr/local/bin/chromedriver'
brave_path = '/snap/bin/brave'
options.binary_location = brave_path
options.add_argument('--remote-debugging-port=9224') #NOT 9222
drvr = webdriver.Chrome(options=options, executable_path=driver_path)
drvr.get('https://stackoverflow.com')
drvr.quit
I did see a tip here about using an opera driver, which i found here, but it did not solve my error.
The issue was that after running the script once the process was not killed and was still running in the background so the port was taken and failed on all subsequent runs until I changed the port (or killed the process). The question is why is close() not working in brave when headless, it is working for chrome!

Related

WebDriverException (Status code 127) when running Selenium + webdriver_manager on gitlab-CI machine (linux)

I'm running a simple CI pipeline on GitLab for a Selenium script headlessly + using webdriver_manager to handle chrome driver binary.
This part is passed:
Get LATEST chromedriver version for None google-chrome
There is no [linux64] chromedriver for browser None in cache
Trying to download new driver from https://chromedriver.storage.googleapis.com/100.0.4896.60/chromedriver_linux64.zip
Driver has been saved in cache [/root/.wdm/drivers/chromedriver/linux64/100.0.4896.60]
But after that I'm getting this error:
WebDriverException: Message: Service /root/.wdm/drivers/chromedriver/linux64/100.0.4896.60/chromedriver unexpectedly exited. Status code was: 127`
What is the problem? Seems like webdriver_manager has a problem by running in CI.
Here is a simple script for reproduce:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.chrome.service import Service
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--headless")
service = Service(executable_path=ChromeDriverManager().install())
driver = webdriver.Chrome(service=service, options=chrome_options)
driver.get("http://google.com")
driver.find_element('name', 'q').send_keys("Wikipedia")
This is one of the pipelines:
https://gitlab.com/mmonfared/test/-/jobs/2350697126
This is a sample project:
https://gitlab.com/mmonfared/test
I've also opened an issue in webdriver_manager github repo, no answers yet:
https://github.com/SergeyPirogov/webdriver_manager/issues/363
This error message...
WebDriverException: Message: Service /root/.wdm/drivers/chromedriver/linux64/100.0.4896.60/chromedriver unexpectedly exited. Status code was: 127`
...implies that you are executing your tests as the root user.
Deep Dive
As per Chrome doesn't start or crashes immediately
A common cause for Chrome to crash during startup is running Chrome as
root user (administrator) on Linux. While it is possible to work
around this issue by passing --no-sandbox flag when creating your
WebDriver session, such a configuration is unsupported and highly
discouraged. Please configure your environment to run Chrome as a
regular user instead.
Solution
Execute your tests as a non-root user.

DeprecationWarning: executable_path has been deprecated, please pass in a Service object driver = webdriver.Chrome(PATH) [duplicate]

I am using sublime to code python scripts. The following code is for selenium in python to install the driver automatically by using the webdriver_manager package
# pip install webdriver-manager
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
driver = webdriver.Chrome(ChromeDriverManager().install())
driver.maximize_window()
#s=Service(path)
#driver=webdriver.Chrome(service=s)
driver.get('https://www.google.com')
driver.find_element(By.NAME, 'q').send_keys('Yasser Khalil')
The code works fine but I got a warning like that
Demo.py:7: DeprecationWarning: executable_path has been deprecated, please pass in a Service object
driver = webdriver.Chrome(ChromeDriverManager().install())
How to fix such a bug?
This error message...
DeprecationWarning: executable_path has been deprecated, please pass in a Service object
...implies that the key executable_path will be deprecated in the upcoming releases.
This change is inline with the Selenium 4.0 Beta 1 changelog which mentions:
Deprecate all but Options and Service arguments in driver instantiation. (#9125,#9128)
Solution
With selenium4 as the key executable_path is deprecated you have to use an instance of the Service() class along with ChromeDriverManager().install() command as discussed below.
Pre-requisites
Ensure that:
Selenium is upgraded to v4.0.0
pip3 install -U selenium
Webdriver Manager for Python is installed
pip3 install webdriver-manager
You can find a detailed discussion on installing Webdriver Manager for Python in ModuleNotFoundError: No module named 'webdriver_manager' error even after installing webdrivermanager
Selenium v4 compatible Code Block
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()))
driver.get("https://www.google.com")
Console Output:
[WDM] - ====== WebDriver manager ======
[WDM] - Current google-chrome version is 96.0.4664
[WDM] - Get LATEST driver version for 96.0.4664
[WDM] - Driver [C:\Users\Admin\.wdm\drivers\chromedriver\win32\96.0.4664.45\chromedriver.exe] found in cache
You can find a detailed discussion on installing Webdriver Manager for Python in Selenium ChromeDriver issue using Webdriver Manager for Python
Incase you want to pass the Options() object you can use:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
options = Options()
options.add_argument("start-maximized")
driver = webdriver.Chrome(service=Service(ChromeDriverManager().install()), options=options)
driver.get("https://www.google.com")
TL; DR
You can find the relevant Bug Report/Pull Request in:
Bug Report: deprecate all but Options and Service arguments in driver instantiation
Pull Request: deprecate all but Options and Service arguments in driver instantiation
This works for me
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
ser = Service(r"C:\chromedriver.exe")
op = webdriver.ChromeOptions()
s = webdriver.Chrome(service=ser, options=op)
Extending on the accepted answer, the Service class allows to explicitly specify a ChromeDriver executable in the same way as previously using the executable_path parameter. In this way existing code is easily migrated (clearly you need to replace C:\chromedriver.exe above by your path).
I could figure it out
# pip install webdriver-manager
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
from webdriver_manager.chrome import ChromeDriverManager
from selenium.webdriver.common.by import By
s=Service(ChromeDriverManager().install())
driver = webdriver.Chrome(service=s)
driver.maximize_window()
driver.get('https://www.google.com')
driver.find_element(By.NAME, 'q').send_keys('Yasser Khalil')
I found this deprecation issue is appearing on Selenium, Pip and Python updates. so simply just change :
before:
from selenium import webdriver
chrome_driver_path = 'C:/Users/Morteza/Documents/Dev/chromedriver.exe'
driver = webdriver.Chrome(executable_path=chrome_driver_path)
url = "https://www.google.com"
driver.get(url)
after:
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
s=Service('C:/Users/Morteza/Documents/Dev/chromedriver.exe')
browser = webdriver.Chrome(service=s)
url='https://www.google.com'
browser.get(url)
All the above answers refer to Chrome, adding the one for Firefox
Install:
pip install webdriver-manager
Code:
from selenium.webdriver.firefox.service import Service
from webdriver_manager.firefox import GeckoDriverManager
driver = webdriver.Firefox(service=Service(executable_path=GeckoDriverManager().install()))
Reference: https://github.com/SergeyPirogov/webdriver_manager/issues/262#issuecomment-955197860
from selenium import webdriver
from selenium.webdriver.chrome.service import Service
service_obj = Service("WebDrivers_path\chromedriver.exe")
driver = webdriver.Chrome(service=service_obj)
driver.get("https://www.google.com")
Simplest option with Chrome auto-installer:
from selenium import webdriver
import chromedriver_autoinstaller
from selenium.webdriver.chrome.service import Service
chromedriver_autoinstaller.install()
driver = webdriver.Chrome(service=Service())
Have a look at the new definition in the Service object here.
My solution
from selenium.webdriver.chrome.service import Service
chrome_executable = Service(executable_path='chromedriver.exe', log_path='NUL')
driver = webdriver.Chrome(service=chrome_executable)
if you are using any IDE like PyCharm install webdriver-manager package of that IDE as how do install for selenium package
You can create an instance of ChromeOptions, which has convenient methods for setting ChromeDriver-specific capabilities. You can then pass the ChromeOptions object into the ChromeDriver constructor:
ChromeOptions options = new ChromeOptions();
options.addExtensions(new File("/path/to/extension.crx"));
ChromeDriver driver = new ChromeDriver(options);
Since Selenium version 3.6.0, the ChromeOptions class in Java also implements the Capabilities interface, allowing you to specify other WebDriver capabilities not specific to ChromeDriver.
ChromeOptions options = new ChromeOptions();
// Add the WebDriver proxy capability.
Proxy proxy = new Proxy();
proxy.setHttpProxy("myhttpproxy:3337");
options.setCapability("proxy", proxy);
// Add a ChromeDriver-specific capability.
options.addExtensions(new File("/path/to/extension.crx"));
ChromeDriver driver = new ChromeDriver(options);

Debugging Selenium and Chromedriver on AWS Lambda (chrome not reachable)

I've got a Lambda Function for headless chrome + python selenium deployed with Serverless framework that runs fine locally but crashes on lambda.
Some basic details:
(Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 4.14.231-180.360.amzn2.x86_64 x86_64)
Chromium Version: 89xx
selenium==3.141.0
Here is how i'm invoking it with selenium:
options = Options()
options.binary_location = '/opt/headless-chromium'
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--single-process')
options.add_argument("--remote-debugging-port=9222")
options.add_argument('--disable-dev-shm-usage')
#'/opt/chromedriver' not found
driver = webdriver.Chrome('/opt/chromedriver', chrome_options=options)
driver.get('https://www.neaminational.org.au/')
body = f"Headless Chrome Initialized, Page title: {driver.title}"
driver.close();
driver.quit();
response = {
"statusCode": 200,
"body": body
}
I'm getting the cryptic Message: unknown error: Chrome failed to start: exited abnormally
(chrome not reachable)
(The process started from chrome location /opt/headless-chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.).
Now i've tested this on my ubuntu 18 (same chromium binary, same chrome driver, same install selenium version) and it's working fine... so my issue must be with compatibility with the lambda amz linux environment.
Can anyone give me some idea on how i could troubleshoot this? Seems silly to stumble around trying different versions when they all seem compatible with eachother locally.
Any insight appreciated greatly!
I found this to be really helpful:
https://www.youtube.com/watch?v=jWqbYiHudt8
https://github.com/soumilshah1995/Selenium-on-AWS-Lambda-Python3.7
The versions are the following:
RUNTIME=python3.7
SELENIUM_VER=3.141.0
CHROME_BINARY_VER=v1.0.0-55 # based on Chromium 69.0.3497.81
CHROMEDRIVER_VER=2.43 # supports Chrome v69-71
Credits go to Soumil Nitin Shah.
Best,
Ramón

WebDriverException: Message: chrome not reachable - chromedriver 2.30

In [12]: from selenium import webdriver
In [13]: chrome_options = webdriver.ChromeOptions()
In [14]: chrome_options.add_argument('--no_sandbox')
In [15]: chrome_options.add_argument('--privileged')
In [16]: browser = webdriver.Chrome('/home/jeremie/Downloads/chromedriver', chrome_o
...: ptions=chrome_options)
In [17]: browser.get('http://localhost:8000')
When I ran the last line, I got
WebDriverException: Message: chrome not reachable
(Session info: chrome=58.0.3029.81)
(Driver info: chromedriver=2.30.477691 (6ee44a7247c639c0703f291d320bdf05c1531b57),platform=Linux 4.8.0-32-generic x86_64)
I tried to change another version of the chromedriver, but I got other issues. I tried to fix my problem with other question from SE, but nothing solved my problem. What could I do to fix that problem?
check you have the execution right on the chrome bin. Is possbile the chrome browser installed not by your account?
Give a try not to add arguments to chromeOptions
Give a try with lower down your chromedriver version
I guess you are not setProperty()
driver = webdriver.Chrome('C:\\Users\\Downloads\\chromedriver_win32\\chromedriver.exe')
driver.get("https://stackoverflow.com/")
There can be multiple reasons for seeing the WebDriverException as chrome not reachable.
First and foremost, we have to ensure that our Selenium version, chromedriver version and Chrome version are compatible. You can find the compatibility information on the Downloads page of ChromeDriver individually for each releases.
Consider removing all the dangling chromedriver instances from you system. If possible make a system restart.
Periodically run CCleaner to wipe away all the previous execution leftovers.
When you initialized the webdriver instance, you mentioned:
browser = webdriver.Chrome('/home/jeremie/Downloads/chromedriver', chrome_options=chrome_options)
Instead, while we mention the absolute path of the chromedriver binary we must also provide the argument executable_path. So we may need to change to:
driver = webdriver.Chrome(chrome_options=chrome_options, executable_path='/home/jeremie/Downloads/chromedriver')
driver.get("https://stackoverflow.com/")

Unable to find a matching set of capabilities with selenium 3.4.3, firefox 54.0 and gecko driver 0.17

if __name__ == '__main__':
driver=webdriver.Firefox(executable_path=r'/home/saurabh/Saurabh/LearnPython/Automation/geckodriver');
After running the above code I am getting an error as :
selenium.common.exceptions.WebDriverException: Message: Unable to find a matching set of capabilities
I don't see any significant error in your code as such.
It is to be noted that the current Selenium-Python binding is unstable with geckodriver and looks to be Architecture specific. You can find the github discussion and merge here. So you may additionally need to pass the absolute path of the firefox binary as firefox_binary argument while initializing the webdriver
Here is your own code with a simple tweak which opens the Mozilla Firefox browser:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
if __name__ == '__main__':
binary = FirefoxBinary('C:\\Program Files\\Mozilla Firefox\\firefox.exe')
driver = webdriver.Firefox(firefox_binary=binary, executable_path="C:\\path\\to\\geckodriver.exe")
Make sur you are pointing to \path\to\FirefoxPortable\App\Firefox64\firefox.exe and not just \path\to\FirefoxPortable\FirefoxPortable.exe