Use custom firefox browser version capability for geckodriver - selenium

I build selenoid docker images for firefox myself and have them configured in the browsers.json as follows:
"firefox": {
"default": "66.0",
"versions": {
"beta": {
"image": "my/path/to/firefox:beta",
"port": "4444"
},
"66.0": {
"image": "selenoid/vnc:firefox_66.0",
"port": "4444"
}
}
}
Sending the version=beta capability causes the webdriver to throw
selenium.WebDriverException: Requested environment is not available
In the logs I found
2019/04/03 08:17:29 [3] [PROXY_TO] [90ab834d22aa3bbe2731eeb550497eec7ef9fb11c1e7f4609d617cf6a25124e7] [http://172.17.0.4:4444]
2019/04/03 08:17:29 [3] [SESSION_ATTEMPTED] [http://172.17.0.4:4444] [1]
2019/04/03 08:17:29 [3] [SESSION_ATTEMPTED] [http://172.17.0.4:4444/wd/hub] [2]
2019/04/03 08:17:29 [3] [SESSION_FAILED] [http://172.17.0.4:4444/wd/hub] [400 Bad Request]
Since the exact same thing works for chrome and it also works if I set "default": "beta" in the browsers.json and do not set the version capability, I assume this happens because geckodrivers matches the version from the capabilites against the actual browser version. (as presumed here).
I have several ideas how to work around this, but do not know how to technically implement these:
Prevent Selenoid from passing the version capability to the selenium driver (geckodriver in this case)
Prevent geckodriver from checking version capability and browser version
Add another flag to selenoid to get the docker image from browsers.json such as browser_version instead of version
Add another fake browser to browsers.json and set the beta to default, then just set the browserName capability. The problem here is, selenoid checks for the browser names and if it does not match chrome, firefox or opera the selenoid container cannot be started. E.g.:
browserName=firefox-beta
"firefox-beta": {
"default": "beta",
"versions": {
"beta": {
"image": "my/path/to/firefox:beta",
"port": "4444"
}
}
}
Any help or further information will be greatly appreciated

For w3c compliant browsers (as of today, only non-Chrome), they require the browser version to be in the browserVersion capability instead of version. I see you tried browser_version, but did you try browserVersion?
https://www.w3.org/TR/webdriver1/#capabilities

Related

Selenium 4.x trying to POST CDP: "UnsupportedCommandException"

I`m trying to execute some commands via CDP, however no matter what combination of Selenium/Driver/Chrome I use it's always the same result.
Last tested with:
Selenium 4.1.1
Chrome + Driver 96.0.4664.110
The project is made in C so I am posting manually to Selenium via CURL. Every other command besides CDP works fine.
I have checked Selenium, Chrome Driver; they both have the CDP support built in.
The URL's I tried to post to are:
- /session/id/goog/cdp/execute
- /session/id/{}/cdp/execute
The posted data format is: "cmd" + "params" (json object).
Both end in the same result: org.openqa.selenium.UnsupportedCommandException.
I also tried to run Selenium in different modes, standalone, hub/node, same result.
Can someone please advise what I am doing wrong? Or maybe I have misunderstood the usage?
Using chromedriver executable
This worked for me (Windows + Postman), but should also work with CURL Linux/Mac.
1 Download chromedriver: https://chromedriver.chromium.org/downloads for your chrome version.
2 Start chromedriver
start chromedriver.exe
output:
Starting ChromeDriver 97.0.4692.71 on port 9515...
3 Send requests to localhost:9515/
3.1 Create Session:
POST localhost:9515/session
request json body:
{"capabilities":{"goog:chromeOptions": {}}}
status 200
response:
"value": {
"capabilities": {
...
},
"sessionId": "b8ac49ce2203739fa0d32dfe8d1a23b5"
3.2 Navigate some url (optional, just check request by sessionId works):
POST localhost:9515/session/b8ac49ce2203739fa0d32dfe8d1a23b5/url
request json body:
{"url": "https://example.com"}
status 200
3.3 Execute CDP command (take screenshot):
POST localhost:9515/session/b8ac49ce2203739fa0d32dfe8d1a23b5/goog/cdp/execute
request json body:
{"cmd":"Page.captureScreenshot", "params":{}}
status 200
response:
{
"value": {
"data": "iVBORw0KGgoAAAANSUhEUgA...."
}
}
Allow remote connections
By default chromedriver allows only local connections.
To allow some remote IPs:
start chromedriver.exe --allowed-ips="some-remote-ip"
Reference: https://sites.google.com/a/chromium.org/chromedriver/security-considerations
Run CDP commands with Selenium Grid
Eventually, it started to work for me with
ChromeDriver 97.0.4692.71
selenium-server-4.1.1
Chrome 97.0.4692.71 (Official Build) (64-bit)
Note: Content-Type header should have charset=utf-8
Content-Type:application/json;charset=utf-8 for Selenium Grid HTTP requests.
Prerequisites
1 Download and run selenium server according to
https://www.selenium.dev/documentation/grid/getting_started/
java -jar selenium-server-<version>.jar standalone --driver-configuration display-name='Chrome' stereotype='{"browserName":"chrome"}'
2 Create Session:
POST localhost:4444/wd/hub/session
request json body:
{
"desiredCapabilities": {
"browserName": "chrome",
"goog:chromeOptions": {
"args": [
],
"extensions": [
]
}
},
"capabilities": {
"firstMatch": [
{
"browserName": "chrome",
"goog:chromeOptions": {
"args": [
],
"extensions": [
]
}
}
]
}
}
status 200
response:
{
"status": 0,
"sessionId": "69ac1c82306f72c7aaf53cfbb28a30e7",
...
}
}
3 Execute CDP command (take screenshot):
POST localhost:4444/wd/hub/session/69ac1c82306f72c7aaf53cfbb28a30e7/goog/cdp/execute
request json body:
{"cmd":"Page.captureScreenshot", "params":{}}
status 200
response:
{
"value": {
"data": "iVBORw0KGgoAAAANSUhEUgA...."
}
}

How to use Edge Chromium webdriver "unknown error: cannot find MSEdge binary"

I am trying to use the WebDriver for the Edge Chromium version with PostMan, but I cannot make it work.
WebDriver Download: https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/
Doc: https://learn.microsoft.com/en-us/microsoft-edge/webdriver
I try to add the Edge Chromium and Webdriver in the system PATH environment variable and no difference. I think the selenium implementation in java work with the Edge Chromium Webdriver
msedgedriver.exe --port 9515
POST localhost:9515/session
{
"capabilities":{
"firstMatch":[
{
}
],
"alwaysMatch":{
"browserName":"MicrosoftEdge",
"platformName":"windows",
"goog:chromeOptions":{
"extensions":[
],
"args":[
]
}
}
},
"desiredCapabilities":{
"browserName":"MicrosoftEdge",
"version":"",
"platform":"WINDOWS",
"goog:chromeOptions":{
"extensions":[
],
"args":[
]
}
}
}
{
"value": {
"error": "unknown error",
"message": "unknown error: cannot find MSEdge binary",
"stacktrace":
"Backtrace:\n\tOrdinal0 [0x00007FF6678D7C52+1932370]\n\tOrdinal0
[0x00007FF66783CDA2+1297826]\n\tOrdinal0
[0x00007FF6677A0A51+658001]\n\tOrdinal0 [0x00007FF667710F1F+69407]\n\tOrdinal0
[0x00007FF66770EF02+61186]\n\tOrdinal0 [0x00007FF667737DBD+228797]\n\tOrdinal0
[0x00007FF6677351AF+217519]\n\tOrdinal0 [0x00007FF66771706F+94319]\n\tOrdinal0
[0x00007FF66771822E+98862]\n\tOrdinal0
[0x00007FF66785B531+1422641]\n\tGetHandleVerifier
[0x00007FF6679991E9+656297]\n\tGetHandleVerifier
[0x00007FF667998F81+655681]\n\tGetHandleVerifier
[0x00007FF6679A104C+688652]\n\tGetHandleVerifier
[0x00007FF6679999C3+658307]\n\tOrdinal0
[0x00007FF66785177E+1382270]\n\tOrdinal0
[0x00007FF66785D9D6+1432022]\n\tOrdinal0
[0x00007FF66785C84D+1427533]\n\tBaseThreadInitThunk
[0x00007FF909056FD4+20]\n\tRtlUserThreadStart [0x00007FF90AB5B1F1+33]\n"
}
}
The WebDriver should open and be able to execute other commands
I suggest you to refer steps below may help to solve the issue.
(1) First try to set the environment variable for your Edge (chromium) application.
(2) Try to open PowerShell window and try to launch the Edge web driver.
(3) Launch the Postman app and try to use http://localhost:9515/session as POST request.
(4) Add code below as body of the request.
{
"desiredCapabilities": {
"nativeEvents": false,
"browserName": "edg",
"version": "",
"platform": "ANY",
"javascriptEnabled": true,
"takesScreenshot": true,
"handlesAlerts": true,
"databaseEnabled": true,
"locationContextEnabled": true,
"applicationCacheEnabled": false,
"browserConnectionEnabled": true,
"cssSelectorsEnabled": true,
"webStorageEnabled": true,
"rotatable": true
}
}
(5) Try to send the request.
Reference:
SeleniumHQ/selenium
Notes:
Your environment variable should properly set and referenced to your
Edge (chromium) correctly.
Use the supported version of Edge web driver with your Edge
(chromium) browser.
This should really be a comment and not a full answer (ridiculous reputation system tbh); Anyway, to add to Deepak-MSFT's answer - after adding the environment variables be sure to restart your IDE.
I added my MSEdge directory to the PATH and Eclipse would still complain. Everything worked after I closed and re-opened Eclipse.
Also check if you are pointing to MSEdge.exe or the directory where MSEdge.exe is located. I noticed that it only works if you point to the dir and not the .exe.
So it would be something like: C:\Program Files (x86)\Microsoft\Edge Dev\Application\
I did face the same issue, and trust me I tried doing every possible thing, but you know what helped me, though it might sound lame to you, the Edge browser version it was old, when I updated it, it worked for me, the error was resolved, do give it a shot as well
Unlike Chrome browser it does not throws error lie the browser driver version and browser are incompatible.
Yours's sincerely,
A frustrated Automation tester

My nightwatch.js tests not runs in Chrome headless of CentOS

I run nightwatch.js tests using Nightwatch version 1.0.18 and It's working in windows environment but when I run it in centOS after installment of Xvfb I found below error.
Error while running .navigateTo() protocol action: invalid session id
Error while running .locateMultipleElements() protocol action: invalid session id
Error while running .locateMultipleElements() protocol action: invalid session id
Here is my nightwatch.json file code:
{
"src_folders": [
"./tests"
],
"output_folder": "./reports",
"custom_commands_path": "./custom_commands",
"custom_assertions_path": "",
"test_workers": false,
"webdriver": {
"start_process": true
},
"test_settings": {
"default": {
"webdriver": {
"port": 9515,
"server_path": "./node_modules/chromedriver/lib/chromedriver/chromedriver",
"cli_args": [
"--log",
"debug"
]
},
"skip_testcases_on_fail": true,
"desiredCapabilities": {
"browserName": "chrome",
"javascriptEnabled": true,
"acceptSslCerts": true,
"chromeOptions": {
"args": [
"headless",
"no-sandbox",
"disable-gpu"
]
}
}
}
}
}
am I missing something to run my tests in the centOS environment because it is running in the windows environment?
I had the same issue with Nightwatchjs and the npm chomedriver setup.
Background:
Everything was working until I just recently updated Chromium on my system. In addition to the errors in the original post, verbose logging also showed:
{
message: 'unknown error: Chrome failed to start: exited abnormally',
error: [
"(unknown error: DevToolsActivePort file doesn't exist)",
'(The process started from chrome location /usr/bin/chromium is no longer running, so ChromeDriver is assuming that Chrome has crashed.)',
'(Driver info: chromedriver=2.46.628388 (4a34a70827ac54148e092aafb70504c4ea7ae926),platform=Linux 4.9.0-8-amd64 x86_64)'
],
}
After downloading the standalone chromedriver (2.46.628388) to match my Chromium version (72.0.3626.69) it was still showing the same errors.
Solution:
I ended up downloading an older version of Chromium (71.0.3578.127) and setting chromeOptions.binary to the new path of the chromium 71 binary. I also had to include 'no-sandbox' with chromeOptions.args.
Here is the snippet from the site mentioned above:
Downloading old builds of Chrome / Chromium
Let's say you want a build of Chrome 44 for debugging purposes. Google does not offer old builds as they do not have up-to-date security fixes.
However, you can get a build of Chromium 44.x which should mostly match the stable release. Here's how you find it:
Look in https://googlechromereleases.blogspot.com/search/label/Stable%20updates for the last time "44." was mentioned.
Loop up that version history ("44.0.2403.157") in the Position Lookup
In this case it returns a base position of "330231". This is the commit of where the 44 release was branched, back in May 2015.*
Open the continuous builds archive
Click through on your platform (Linux/Mac/Win)
Paste "330231" into the filter field at the top and wait for all the results to XHR in.
Eventually I get a perfect hit: https://commondatastorage.googleapis.com/chromium-browser-snapshots/index.html?prefix=Mac/330231/
Sometimes you may have to decrement the commit number until you find one.
Download and run!
Upgrading to the latest version of chromedriver solved the issue for me. You can find the latest version here; https://www.npmjs.com/package/chromedriver
In my situation, when that error occurs:
Error while running .navigateTo() protocol action: invalid session id
I added the following code into .travis.yml:
addons:
chrome: stable

Nightwatch vs. self signed certificates

My nightwatch.js setup for Geckodriver is as follows:
"firefox": {
"launch_url": "...",
"selenium_port": 4444,
"selenium_host": "localhost",
"silent": true,
...
"desiredCapabilities": {
"browserName": "gecko",
"marionette": true,
"acceptSslCerts": true
}
}
When running, all tests fail because my certificate is insecure testing on a local installation and enforced ssl. Chromedriver (with basically identical setup) seems to accept the "acceptSslCerts" property and ignores the wrong cert. Geckodriver does not. Is the config wrong or does Nightwatch or Selenium have a problem with "acceptSslCerts"?
I am using the latest version of Geckodriver and Selenium 3.8.1
It looks like the correct capability for the firefox driver is acceptInsecureCerts.
You can see it listed here.

Setting the Firefox profile when using Selenium and Firefox Portable

I'm in a setup, in which I need to use Firefox Portable 38.7.1 for my Selenium tests (version 2.53.0). Everything works fine, but now I need to configure a proxy.
I configured it in the default profile (it gets saved in ${FF_PORTABLE_PATH}/Data/profile/prefs.js)
user_pref("network.proxy.http", "proxyHost");
user_pref("network.proxy.http_port", proxyPort);
user_pref("network.proxy.share_proxy_settings", true);
user_pref("network.proxy.ssl", "proxyHost");
user_pref("network.proxy.ssl_port", 51854);
user_pref("network.proxy.type", 1);
...
When starting the browser manually, this works fine. However, when triggered by Selenium an anonymous profile is created and used, which doesn't have my proxy settings.
I tried to specify the profile when starting the node.
At first I tried using -Dwebdriver.firefox.profile:
java -jar selenium-server-standalone-2.53.0.jar -role node -Dwebdriver.firefox.profile=default
Then I tried to use the default profile as a template using -firefoxProfileTemplate:
java -jar selenium-server-standalone-2.53.0.jar -role node -firefoxProfileTemplate "${FF_PORTABLE_PATH}/Data" -nodeConfig ...
I also created a new profile (using the ProfilistPortable plugin) and specified it on startup of the node (with the webdriver.firefox.profile-parameter).
In all cases the Selenium node opens up Firefox Portable with a "clean" anonymous profile without my proxy settings.
Can anyone help me how to get this setup working with Firefox Portable? I don't really need separate profiles. As long as I can force Selenium to use a profile, which has the proxy configured, I'm fine.
Here's my nodeConfig:
{
"capabilities": [
{
"browserName": "firefox",
"version": "38.7.1",
"firefox_binary": "${FF_PORTABLE_PATH}\\FirefoxPortable.exe",
"platform": "WINDOWS",
"maxInstances": 1,
"seleniumProtocol": "WebDriver"
}
],
"configuration": {
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"maxSession": 1,
"port": 5550,
"host": "ip",
"register": true,
"registerCycle": 5000,
"hubHost": "localhost",
"hubPort": 4440
}
}
I use ${FF_PORTABLE_PATH} in the examples above. In reality this (fully qualified) path is hardcoded in all my settings.
I had the same issue. Digging into selenium code I was able to find a solution.
FirefoxBinary binary = new FirefoxBinary(new File(portablePath.trim()));
File profile_dir = new File(portablePath.trim().replace("FirefoxPortable.exe", "/Data/profile"));
fp = new FirefoxProfile(profile_dir);
dc.setCapability(FirefoxDriver.PROFILE, fp);
driver = new FirefoxDriver( binary , fp, dc);