Selenium Chrome Driver - Exception when trying headless option - selenium

I am trying to use chrome but I need it to be hidden from the user.
So I tried to use "headless" with ChromeOptions
ChromeOptions options = new ChromeOptions();
options.addExtensions(new File("C://temp//1.14.12_0.crx"));
options.addArguments("disable-infobars");
options.addArguments("headless");
but then I crashed with the following exception:
[1524948093.974][WARNING]: chrome quit unexpectedly, leaving behind temporary directories for debugging:
[1524948093.974][SEVERE]: Port leaked: 12545
Exception in thread "main" org.openqa.selenium.WebDriverException: unknown error: failed to wait for extension background page to load: chrome-extension://cjpalhdlnbpafiamejdnhcphjbkeiagm/background.html
from unknown error: page could not be found: chrome-extension://cjpalhdlnbpafiamejdnhcphjbkeiagm/background.html
(Driver info: chromedriver=2.37.544315 (730aa6a5fdba159ac9f4c1e8cbc59bf1b5ce12b7),platform=Windows NT 10.0.16299 x86_6
I made some searching and found out that it might be related to the fact that I also load an ad blocker extension, so I temporarily commented the extension loading and the chrome was loaded HIDDEN.
But I still need this extension to work for me when chrome is hidden.
Is there any solution?
Chrome Version - 66
Chrome Driver version - 2.37
Thanks!

So, this one made the trick:
options.addArguments("load-extension=.//resources//1.14.12_0.crx");
instead of
options.addExtensions(new File(".//resources//1.14.12_0.crx"));
Enjoy!

Related

Debugging Selenium and Chromedriver on AWS Lambda (chrome not reachable)

I've got a Lambda Function for headless chrome + python selenium deployed with Serverless framework that runs fine locally but crashes on lambda.
Some basic details:
(Driver info: chromedriver=2.41.578700 (2f1ed5f9343c13f73144538f15c00b370eda6706),platform=Linux 4.14.231-180.360.amzn2.x86_64 x86_64)
Chromium Version: 89xx
selenium==3.141.0
Here is how i'm invoking it with selenium:
options = Options()
options.binary_location = '/opt/headless-chromium'
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--single-process')
options.add_argument("--remote-debugging-port=9222")
options.add_argument('--disable-dev-shm-usage')
#'/opt/chromedriver' not found
driver = webdriver.Chrome('/opt/chromedriver', chrome_options=options)
driver.get('https://www.neaminational.org.au/')
body = f"Headless Chrome Initialized, Page title: {driver.title}"
driver.close();
driver.quit();
response = {
"statusCode": 200,
"body": body
}
I'm getting the cryptic Message: unknown error: Chrome failed to start: exited abnormally
(chrome not reachable)
(The process started from chrome location /opt/headless-chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.).
Now i've tested this on my ubuntu 18 (same chromium binary, same chrome driver, same install selenium version) and it's working fine... so my issue must be with compatibility with the lambda amz linux environment.
Can anyone give me some idea on how i could troubleshoot this? Seems silly to stumble around trying different versions when they all seem compatible with eachother locally.
Any insight appreciated greatly!
I found this to be really helpful:
https://www.youtube.com/watch?v=jWqbYiHudt8
https://github.com/soumilshah1995/Selenium-on-AWS-Lambda-Python3.7
The versions are the following:
RUNTIME=python3.7
SELENIUM_VER=3.141.0
CHROME_BINARY_VER=v1.0.0-55 # based on Chromium 69.0.3497.81
CHROMEDRIVER_VER=2.43 # supports Chrome v69-71
Credits go to Soumil Nitin Shah.
Best,
Ramón

WebDriverException: unknown error: DevToolsActivePort file doesn't exist

I have an issue when trying a simple test with selenium avec chromedriver. I get the error org.openqa.selenium.WebDriverException: unknown error: DevToolsActivePort file doesn't exist
This has been reported a couple of time on stackoveflow, i've read near all the answers without success at this time.
I'm using a corporate managed computer without admin rights, using chrome 86.0.4240 with same version of chromedriver, selenium-standalone-server 3.9.1.
The code used is below, with the trace of different tries according to different stackoverflow topics found.
public static void main(String[] args) {
System.setProperty("webdriver.chrome.driver","C:\\Users\\" + System.getProperty("user.name") + "\\Downloads\\chromedriver_win32\\chromedriver.exe");
ChromeOptions options = new ChromeOptions();
Proxy proxy = new Proxy();
proxy.setHttpProxy("127.0.0.1:9000");
options.setCapability("proxy", proxy);
// Exception in thread "main" org.openqa.selenium.WebDriverException: unknown error: DevToolsActivePort file doesn't exist
// https://stackoverflow.com/questions/50642308/webdriverexception-unknown-error-devtoolsactiveport-file-doesnt-exist-while-t/51326137#51326137
options.addArguments("--disable-dev-shm-usage"); // overcome limited resource problems
options.addArguments("start-maximized"); // open Browser in maximized mode
options.addArguments("disable-infobars"); // disabling infobars
options.addArguments("--disable-extensions"); // disabling extensions
options.addArguments("--disable-gpu"); // applicable to windows os only
options.addArguments("--no-sandbox"); // Bypass OS security model
// All chrome process are previously killed : Get-Process -Name chrome* | Stop-Process
// https://stackoverflow.com/questions/59987080/invalidargumentexception-message-invalid-argument-user-data-directory-is-alre
//options.addArguments("--user-data-dir=C:\\Users\\" + System.getProperty("user.name") + "\\AppData\\Local\\Google\\Chrome\\Automation\\");
WebDriver driver = new ChromeDriver();
driver.get("https://www.google.com"); }
When i run the test, Chrome Open, display the message tell it's controlled by an automated tool, display "data:" in the URL bar and hang. After a couple of seconds (minutes), the script ends with the above error.
I'm runing on a plain windows 10 plateform, with no docker.
I cannot create a new chrome profile, so i take care to close all chrome process before starting the test.
Any help will be greatly appreciated.

Even if I run headless, I get an "unknown error: cannot find Chrome binary" error

I'm trying to run Selenium tests in the headless mode in my Jenkins build.
Here is the part of the code where I configure the Chromedriver (yup, many options hahaha):
WebDriverManager.chromedriver().setup();
ChromeOptions chromeOptions = new ChromeOptions();
chromeOptions.setPageLoadStrategy(PageLoadStrategy.NONE);
chromeOptions.addArguments("--headless");
chromeOptions.addArguments("--window-size=1920x1080");
chromeOptions.addArguments("start-maximised");
chromeOptions.addArguments("enable-automation");
chromeOptions.addArguments("--no-sandbox");
chromeOptions.addArguments("--disable-infobars");
chromeOptions.addArguments("--disable-dev-shm-usage");
chromeOptions.addArguments("--disable-browser-side-navigation");
chromeOptions.addArguments("--disable-gpu");
driver = new ChromeDriver(chromeOptions); // I get the error at this line
Since I run in headless mode, I expect Jenkins not to go find a Chrome binary. But... it does.
org.openqa.selenium.WebDriverException: unknown error: cannot find Chrome binary
[... nothing relevant...]
Driver info: driver.version: ChromeDriver
Any idea of what I can do?
Thanks so much in advance

Chrome works in headless mode on linux Server but shows "DevToolsActivePort file doesn't exist" error in gui mode through Selenium and Python

selenium code to automate on chrome browser.
chrome version: 79.0.3945.117 (same for local and server machine)
chrome driver: tried all latest & below till chrome version driver.
Execution Status - On Local machine:
Works fine in headless and gui.
Execution Status - On server Centros 7 machine.
works fine in headless.In GUI giving error:
options.addArguments("--no-sandbox"); // Bypass OS security model
options.addArguments("--disable-dev-shm-usage"); // overcome limited resource problems
Error log
SLF4J: Failed to load class "org.slf4j.impl.StaticLoggerBinder".
SLF4J: Defaulting to no-operation (NOP) logger implementation
SLF4J: See http://www.slf4j.org/codes.html#StaticLoggerBinder for further details.
>>>>> Initializing the webdriver: CHROME on OS: linux64
Jan 11, 2020 12:27:52 PM org.openqa.selenium.remote.DesiredCapabilities chrome
INFO: Using `new ChromeOptions()` is preferred to `DesiredCapabilities.chrome()`
Starting ChromeDriver 79.0.3945.36 (3582db32b33893869b8c1339e8f4d9ed1816f143-refs/branch-heads/3945#{#614}) on port 16949
Only local connections are allowed.
Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
>>>>> Initializing the webdriver: CHROME on OS: linux64
unknown error: Chrome failed to start: exited abnormally
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
Build info: version: '3.141.59', revision: 'e82be7d358', time: '2018-11-14T08:17:03'
System info: host: 'qa9', ip: '', os.name: 'Linux', os.arch: 'amd64', os.version: '3.10.0-1062.9.1.el7.x86_64', java.version: '1.8.0_232'
Driver info: driver.version: ChromeDriver
remote stacktrace: #0 0x564eeb93d479 <unknown>
Jan 11, 2020 12:27:52 PM org.openqa.selenium.remote.DesiredCapabilities chrome
INFO: Using `new ChromeOptions()` is preferred to `DesiredCapabilities.chrome()`
null
2020-01-11 12:27:52 INFO BaseClass:23 - Test Completed
Analysis
1- Chrome is opening fine on server machine.
2- Using google-chrome --no-sandbox on server machine launches chrome
3- google-chrome is available at location /usr/bin/google-chrome
Hence tried all steps available at other answers and using various chrome options but still unable to run selenium on chrome gui.
options.addArguments("--no-sandbox"); // Bypass OS security model
options.setBinary("/usr/bin/google-chrome");
options.addArguments("start-maximized"); // open Browser in maximized mode
options.addArguments("disable-infobars"); // disabling infobars
options.addArguments("--disable-dev-shm-usage"); // overcome limited resource problems
options.addArguments("--test-type");
options.addArguments("--window-size=1420,1080");
options.addArguments("--disable-extensions"); //to disable browser extension popup
options.addArguments("--headless");
options.addArguments("--disable-gpu"); // applicable to windows os only
options.setExperimentalOption("useAutomationExtension", false);
options.addArguments("--disable-dev-shm-usage"); // overcome limited resource problems
ChromeDriver logs
[1578754796.203][INFO]: Launching chrome: /usr/bin/google-chrome --disable-background-networking --disable-client-side-phishing-detection --disable-default-apps --disable-dev-shm-usage --disable-hang-monitor --disable-popup-blocking --disable-prompt-on-repost --disable-sync --enable-blink-features=ShadowDOMV0 --enable-logging --log-level=0 --no-first-run --no-sandbox --password-store=basic --remote-debugging-port=0 --start-maximized --test-type --use-mock-keychain --user-data-dir=/tmp/.com.google.Chrome.CKtLXZ --window-size=1420,1080 data:,
(google-chrome:29029): Gtk-WARNING **: 15:59:56.269: cannot open display:
[0111/155956.272402:ERROR:nacl_helper_linux.cc(311)] NaCl helper process running without a sandbox!
Most likely you need to configure your SUID sandbox correctly
[1578754796.304][INFO]: [e36d644ac30ae2e028b2ee00ba18b335] RESPONSE InitSession ERROR unknown error: Chrome failed to start: exited abnormally
(unknown error: DevToolsActivePort file doesn't exist)
(The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
[1578754796.304][DEBUG]: Log type 'driver' lost 0 entries on destruction
[1578754796.304][DEBUG]: Log type 'browser' lost 0 entries on destruction
This error message...
unknown error: Chrome failed to start: exited abnormally (unknown error: DevToolsActivePort file doesn't exist) (The process started from chrome location /usr/bin/google-chrome is no longer running, so ChromeDriver is assuming that Chrome has crashed.)
...implies that the ChromeDriver was unable to initiate/spawn a new Browsing Context i.e. Chrome Browser session.
You need to take care of a couple of things:
The purpose of the argument --disable-gpu was to enable google-chrome-headless on windows platform. It was needed as SwiftShader fails an assert on Windows in headless mode earlier. This issue was resolved through Headless: make --disable-gpu flag unnecessary. As you are on centos you need to remove the line of code:
options.addArguments("--disable-gpu"); // applicable to windows os only
As per your question as you are using chrome=79.0 you need to ensure:
ChromeDriver is updated to current ChromeDriver v79.0.3945.36 level.
As you are adding the ExperimentalOption:
options.setExperimentalOption("useAutomationExtension", false);
You also need to add the ExperimentalOption:
options.setExperimentalOption("excludeSwitches", Collections.singletonList("enable-automation"));
But you need to remove:
options.addArguments("disable-infobars"); // disabling infobars
options.addArguments("--disable-extensions"); //to disable browser extension popup
Reference
You can find a relevant detailed discussion in:
Chrome Options in Python Selenium : Disable GPU vs Headless
ERROR:gpu_process_transport_factory.cc(1007)-Lost UI shared context : while initializing Chrome browser through ChromeDriver in Headless mode
An answer provided here indicts Chrome version > 78 as the cause. Once we back-leveled, problem subsided:
I had this problem. I installed the latest version of the "webdrivers" Rubygem (~> 4.2.0). Then, on my server, I ran:
whereis google-chrome
And got "/usr/bin/google-chrome"
I use ruby, and can now instantiate chrome with
options = Selenium::WebDriver::Chrome::Options.new(args: ['headless'], binary: '/usr/bin/google-chrome')
driver = Selenium::WebDriver.for(:chrome, options: options)

Unable to find/open Firefox Binary - webdriver/robot framework

Unable to find/open Firefox Binary - webdriver/robot framework
My tests run fine in java and fitnesse. They also run fine when executing them through robot framework with Internet Explorer and Chrome. However when I execute them through Firefox, using 'new FirefoxDriver()', I receive the following error:
DEBUG java.lang.ExceptionInInitializerError
at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java: 81)
Caused by: java.lang.NullPointerException
org.openqa.selenium.firefox.FirefoxBinary.<clinit>(FirefoxBinary.java: 42)
... 183 more
In the FirefoxBinary and FirefoxDriver classes these lines correspond to the following code:
FirefoxBinary ln42-43
private static final String PATH_PREFIX = "/" +
FirefoxBinary.class.getPackage().getName().replace(".", "/") + "/";
and FirefoxDriver ln 80-82
public FirefoxDriver(FirefoxProfile profile) {
this(new FirefoxBinary(), profile);
}
I have tried setting the path to the Firefox binary in my classpath, pythonpath (used by robotframework) and path. I have also written the following lines of code to try to force the binary to be found:
System.setProperty("webdriver.firefox.bin", "C:\\Program Files (x86)\\Mozilla Firefox\\firefox.exe");
DesiredCapabilities cap = new DesiredCapabilities();
cap.setCapability(FirefoxDriver.BINARY, "C:\\Program Files (x86)\\Mozilla Firefox\\firefox.exe");
I have tried to execute the tests on two computers, my work and home machines. Further I have tried to use a firefox profile created using firefox.exe –p and also by creating one in the java code. I have tried Firefox 6-8. Unfortunately none of these things have worked.
I am also using/have used:
Java 1.6
Selenium 2.9.0/2.13.0
Windows 7
I am unsure if this is related but as a work around I have been trying to get Firefox to run through a remote browser. I have been trying the following code:
rcc = new RemoteControlConfiguration();
rcc.setPort(4447);
rcc.setPortDriversShouldContact(4447);
rcc.setInteractive(true);
rcc.setSingleWindow(true);
rcc.setTimeoutInSeconds(30);
ss = new SeleniumServer(rcc);
ss.start();
DesiredCapabilities cap = new DesiredCapabilities();
cap.setJavascriptEnabled(true);
cap.setBrowserName("firefox");
URL url = new URL ("http://www.google.com/");
driver = new RemoteWebDriver(url,cap);
However when I run the above I get the following error message:
Exception in thread "main" org.openqa.selenium.WebDriverException: Error communicating with the remote browser. It may have died.
Build info: version: '2.13.0', revision: '14794', time: '2011-11-18 17:49:47'
System info: os.name: 'Windows 7', os.arch: 'amd64', os.version: '6.1', java.version: '1.7.0'
Driver info: driver.version: Selenium2Driver
at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:412)
Does anyone have any idea on how to fix either of my problems?
Any help would be greatly appreciated, I feel very stuck on this issue atm. Two days of trying to get Firefox to work when Internet Explorer already does….. It feels as if the world is about to end.
Thanks,
James
EDIT:1
It is possible for me to run Firefox by using selenium-server.
James, FYI, URL for RemoteWebDriver appears incorrect in post above. Should be something more like "localhost:4444/wd/hub";? Interestingly, I'm having the opposite problem with Web Driver, having issues starting Firefox via RemoteWebDriver but Firefox runs fine via native FirefoxDriver. IE works fine over remote. – David Dec 4 '11 at 4:51
Thanks David!
I am not understanding why you have not configured your Firefox binary in your remote grids config.json file? That's how I would do it. Then, your DesiredCapabilities object would not need to define it. A hint can be found here.
If it works, the line in the JSON file might look like:
"binary": "C:/Program Files/Mozilla Firefox/firefox.exe",
I guess it doesn't allow you to dynamically set the binary location from your code, but perhaps you can try it that way to prove if it should work or not as a troubleshooting step.
FirefoxProfile profile = new FirefoxProfile();
FirefoxBinary binary = new FirefoxBinary(new File("C:\\path to firefox\\firefox.exe"));
driver = new FirefoxDriver(binary, profile);
try this
This kind of issue obtained because of selenium web driver fail to find the .exe files of Firefox. Please check whether C:\Program Files (x86)\Mozilla Firefox you have exe file in the location and don’t forget to set environment variable having the java jdk path.
Source:- read [Solved Cannot find firefox binary in PATH Selenium][1]http://www.tech4crack.com/solved-cannot-find-firefox-binary-in-path/