Setting the Firefox profile when using Selenium and Firefox Portable - selenium

I'm in a setup, in which I need to use Firefox Portable 38.7.1 for my Selenium tests (version 2.53.0). Everything works fine, but now I need to configure a proxy.
I configured it in the default profile (it gets saved in ${FF_PORTABLE_PATH}/Data/profile/prefs.js)
user_pref("network.proxy.http", "proxyHost");
user_pref("network.proxy.http_port", proxyPort);
user_pref("network.proxy.share_proxy_settings", true);
user_pref("network.proxy.ssl", "proxyHost");
user_pref("network.proxy.ssl_port", 51854);
user_pref("network.proxy.type", 1);
...
When starting the browser manually, this works fine. However, when triggered by Selenium an anonymous profile is created and used, which doesn't have my proxy settings.
I tried to specify the profile when starting the node.
At first I tried using -Dwebdriver.firefox.profile:
java -jar selenium-server-standalone-2.53.0.jar -role node -Dwebdriver.firefox.profile=default
Then I tried to use the default profile as a template using -firefoxProfileTemplate:
java -jar selenium-server-standalone-2.53.0.jar -role node -firefoxProfileTemplate "${FF_PORTABLE_PATH}/Data" -nodeConfig ...
I also created a new profile (using the ProfilistPortable plugin) and specified it on startup of the node (with the webdriver.firefox.profile-parameter).
In all cases the Selenium node opens up Firefox Portable with a "clean" anonymous profile without my proxy settings.
Can anyone help me how to get this setup working with Firefox Portable? I don't really need separate profiles. As long as I can force Selenium to use a profile, which has the proxy configured, I'm fine.
Here's my nodeConfig:
{
"capabilities": [
{
"browserName": "firefox",
"version": "38.7.1",
"firefox_binary": "${FF_PORTABLE_PATH}\\FirefoxPortable.exe",
"platform": "WINDOWS",
"maxInstances": 1,
"seleniumProtocol": "WebDriver"
}
],
"configuration": {
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"maxSession": 1,
"port": 5550,
"host": "ip",
"register": true,
"registerCycle": 5000,
"hubHost": "localhost",
"hubPort": 4440
}
}
I use ${FF_PORTABLE_PATH} in the examples above. In reality this (fully qualified) path is hardcoded in all my settings.

I had the same issue. Digging into selenium code I was able to find a solution.
FirefoxBinary binary = new FirefoxBinary(new File(portablePath.trim()));
File profile_dir = new File(portablePath.trim().replace("FirefoxPortable.exe", "/Data/profile"));
fp = new FirefoxProfile(profile_dir);
dc.setCapability(FirefoxDriver.PROFILE, fp);
driver = new FirefoxDriver( binary , fp, dc);

Related

How to setup custom browser versions for selenium grid?

Setup
We have a selenium grid where all nodes are completely identical except one. All the identical nodes run windows 10, have basic chrome, IE, edge browsers, and run on MST. The exception runs IE, chrome, and edge, but they are set to look like it is in the UK, and runs GMT+0 most of the year (except DST of course then it runs BST). This changes date formats and such and is used to test that dates are formatted properly and that documents on our system reflect accurate times in their own timezone. The browsers on the UK node all have custom browser versions declared in their node_XXXX.json file for selenium grid.
Issue
We lost our hub server a while back, and after rebuilding we can no longer get the IE, EDGE, and Firefox browsers to work on the UK server. We used to use custom browser versions for the Uk browsers eg: instead of "version":11 it is "version":"11.eu" to denote that we wanted to run on the special server. Chrome works fine with this model. However, IE is giving an error:
OpenQA.Selenium.WebDriverException : Error forwarding the new session cannot find : Capabilities {browserName: internet explorer, browserVersion: 11.eu, platformName: windows, se:ieOptions: {enablePersistentHover: true, ie.ensureCleanSession: true, nativeEvents: true}}
Attempted Solutions
We have already tried to change the version number (It used to say 9.eu, legacy from when we were running IE9). We have also tried "updating" IE to version 11 again. Nothing has worked or allowed the driver to connect.
Question
How do we get our UK IE browser to work on the selenium grid again? We have many tests failing simply because they cannot connect to a driver instance. Any help would be appreciated, thanks!
Reference
For reference, here is our node_XXXX.json file:
{
"capabilities": [
{
"seleniumProtocol": "WebDriver",
"browserName": "firefox",
"maxInstances": 1,
"version": "17.eu",
"platform": "WINDOWS"
},
{
"seleniumProtocol": "WebDriver",
"browserName": "internet explorer",
"maxInstances": 1,
"version": "11.eu",
"platform": "windows"
},
{
"seleniumProtocol": "WebDriver",
"browserName": "chrome",
"maxInstances": 1,
"version": "77.eu",
"platform": "WINDOWS"
}
],
"loadedFromFile":
"node_5555.json",
"proxy": "com.groupon.seleniumgridextras.grid.proxies.SetupTeardownProxy",
"servlets": [],
"maxSession": 3,
"port": 5555,
"register": true,
"unregisterIfStillDownAfter": 10000,
"hubPort": 4444,
"hubHost": "test-slum01.ndtest.local",
"registerCycle": 5000,
"nodeStatusCheckTimeout": 10000,
"custom": {},
"downPollingLimit": 0
}
And this is the code that calls the remote web driver:
case "iehta9UK":
var ieUkOptions = new InternetExplorerOptions();
ieUkOptions.EnsureCleanSession = true;
ieUkOptions.BrowserVersion = "11.eu";
return IS_REMOTE_RUN ? CreateRemoteWebDriver(ieUkOptions) : new InternetExplorerDriver(ieUkOptions);
This is the strictly US IE code that works fine:
case "IE":
case "ie":
case "InternetExplorer":
case "iehta11":
var ieOptions = new InternetExplorerOptions();
ieOptions.EnsureCleanSession = true;
return IS_REMOTE_RUN ? CreateRemoteWebDriver(ieOptions) : new InternetExplorerDriver(ieOptions);
Aha! We figured it out. We had made the changes to the node_XXXX.json file shown above, but had not restarted the selenium grid service.
Make sure you restart the server after making changes to the json file for them to take affect.

How do you setup an HtmlUnit node with Selenium Grid?

I'm having trouble setting up an HtmlUnit node with Selenium Grid.
I'm launching my Hub with:
java -jar selenium-server-standalone.jar -role hub -port 4444
And HtmlUnitDriver nodes with:
java -jar selenium-server-standalone.jar -role node -browser browserName=htmlunit,maxInstances=5 -hub http://localhost:4444/grid/register -port 5564
My code to connect to the node, which is getting the exception, looks something like:
DesiredCapabilities cap = new DesiredCapabilities();
cap.setBrowserName("htmlunit");
cap.setJavascriptEnabled(false);
driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), cap);
The exception I'm getting is:
Unable to create session from {
"desiredCapabilities": {
"browserName": "htmlunit",
"server:CONFIG_UUID": "93a0486d-c9f6-46da-8065-603ab07c0294",
"javascriptEnabled": false
},
"capabilities": {
"firstMatch": [
{
"browserName": "htmlunit",
"server:CONFIG_UUID": "93a0486d-c9f6-46da-8065-603ab07c0294"
}
]
}
}
To set up a Selenium Grid instance, I use the Selenium Foundation library. The "standalone" JAR brings in scads of transitive dependencies you don't need that end up creating conflicts. I created a sample project (local-grid-utility) that launches a Selenium 2 Grid that dispenses PhantomJS sessions, which could be easily altered to launch a Selenium 3 Grid dispensing HtmlUnit sessions.
The key to getting HtmlUnit sessions stood up in Grid is providing a complete class path that includes all of the dependencies. Here's what the library assembles for HtmlUnitDriver 2.40.0:
${JAVA_HOME}/jre/bin/java -cp ${M2_ROOT}/repository/org/seleniumhq/selenium/selenium-support/3.141.59/selenium-support-3.141.59.jar:${M2_ROOT}/repository/org/seleniumhq/selenium/htmlunit-driver/2.40.0/htmlunit-driver-2.40.0.jar:${M2_ROOT}/repository/org/apache/commons/commons-lang3/3.8.1/commons-lang3-3.8.1.jar:${M2_ROOT}/repository/xerces/xercesImpl/2.12.0/xercesImpl-2.12.0.jar:${M2_ROOT}/repository/org/seleniumhq/selenium/selenium-api/3.141.59/selenium-api-3.141.59.jar:${M2_ROOT}/repository/net/sourceforge/htmlunit/htmlunit-core-js/2.40.0/htmlunit-core-js-2.40.0.jar:${M2_ROOT}/repository/org/eclipse/jetty/websocket/websocket-api/9.4.28.v20200408/websocket-api-9.4.28.v20200408.jar:${M2_ROOT}/repository/xalan/xalan/2.7.2/xalan-2.7.2.jar:${M2_ROOT}/repository/org/eclipse/jetty/jetty-xml/9.4.28.v20200408/jetty-xml-9.4.28.v20200408.jar:${M2_ROOT}/repository/org/apache/httpcomponents/httpclient/4.5.12/httpclient-4.5.12.jar:${M2_ROOT}/repository/org/eclipse/jetty/jetty-http/9.4.28.v20200408/jetty-http-9.4.28.v20200408.jar:/Applications/Eclipse.app/Contents/Eclipse/plugins/com.beust.jcommander_1.72.0.jar:${M2_ROOT}/repository/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar:${M2_ROOT}/repository/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar:${M2_ROOT}/repository/commons-codec/commons-codec/1.11/commons-codec-1.11.jar:${M2_ROOT}/repository/net/sourceforge/htmlunit/neko-htmlunit/2.40.0/neko-htmlunit-2.40.0.jar:${M2_ROOT}/repository/com/squareup/okhttp3/okhttp/3.11.0/okhttp-3.11.0.jar:${M2_ROOT}/repository/commons-net/commons-net/3.6/commons-net-3.6.jar:${M2_ROOT}/repository/org/seleniumhq/selenium/selenium-server/3.141.59/selenium-server-3.141.59.jar:${M2_ROOT}/repository/xalan/serializer/2.7.2/serializer-2.7.2.jar:${M2_ROOT}/repository/org/eclipse/jetty/jetty-client/9.4.28.v20200408/jetty-client-9.4.28.v20200408.jar:${M2_ROOT}/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar:${M2_ROOT}/repository/org/apache/commons/commons-text/1.8/commons-text-1.8.jar:${M2_ROOT}/repository/org/eclipse/jetty/jetty-io/9.4.28.v20200408/jetty-io-9.4.28.v20200408.jar:${M2_ROOT}/repository/net/sourceforge/htmlunit/htmlunit-cssparser/1.5.0/htmlunit-cssparser-1.5.0.jar:${M2_ROOT}/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:${M2_ROOT}/repository/org/apache/httpcomponents/httpmime/4.5.12/httpmime-4.5.12.jar:${M2_ROOT}/repository/org/seleniumhq/selenium/jetty-repacked/9.4.12.v20180830/jetty-repacked-9.4.12.v20180830.jar:${M2_ROOT}/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:${M2_ROOT}/repository/org/eclipse/jetty/websocket/websocket-client/9.4.28.v20200408/websocket-client-9.4.28.v20200408.jar:${M2_ROOT}/repository/org/brotli/dec/0.1.2/dec-0.1.2.jar:${M2_ROOT}/repository/com/google/guava/guava/28.1-jre/guava-28.1-jre.jar:${M2_ROOT}/repository/com/squareup/okio/okio/1.14.0/okio-1.14.0.jar:${M2_ROOT}/repository/net/sourceforge/htmlunit/htmlunit/2.40.0/htmlunit-2.40.0.jar:${M2_ROOT}/repository/org/seleniumhq/selenium/selenium-remote-driver/3.141.59/selenium-remote-driver-3.141.59.jar:${M2_ROOT}/repository/commons-io/commons-io/2.6/commons-io-2.6.jar:${M2_ROOT}/repository/org/eclipse/jetty/jetty-util/9.4.28.v20200408/jetty-util-9.4.28.v20200408.jar:${M2_ROOT}/repository/org/eclipse/jetty/websocket/websocket-common/9.4.28.v20200408/websocket-common-9.4.28.v20200408.jar org.openqa.grid.selenium.GridLauncherV3 -role node -servlets org.openqa.grid.web.servlet.LifecycleServlet -host 192.168.1.6 -port 34464 -nodeConfig /Users/bb8d/github/Selenium-Foundation/target-s3/classes/nodeConfig-s3-9CF40153.json
You could create your own standalone package to bundle up the dependencies if that suits your fancy. The node server is provided by the GridLauncherV3 class. I also add the LifecycleServlet plugin to enable remote shutdown. The host specification corresponds to the local machine, and the port is auto-selected from whatever is currently available.
The JSON configuration file looks like this:
{
"capabilities": [
{
"browserName": "htmlunit",
"javascriptEnabled": true,
"maxInstances": 5,
"seleniumProtocol": "WebDriver"
}
],
"custom": {
},
"debug": false,
"downPollingLimit": 2,
"enablePlatformVerification": true,
"host": "0.0.0.0",
"hub": "http:\u002f\u002f192.168.1.6:4445\u002fwd\u002fhub",
"maxSession": 5,
"nodePolling": 5000,
"nodeStatusCheckTimeout": 5000,
"port": -1,
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"register": true,
"registerCycle": 5000,
"role": "node",
"servlets": [
],
"unregisterIfStillDownAfter": 60000,
"withoutServlets": [
]
}
The critical settings can be specified in the command line, but I think the config file is more manageable.

Nightwatch vs. self signed certificates

My nightwatch.js setup for Geckodriver is as follows:
"firefox": {
"launch_url": "...",
"selenium_port": 4444,
"selenium_host": "localhost",
"silent": true,
...
"desiredCapabilities": {
"browserName": "gecko",
"marionette": true,
"acceptSslCerts": true
}
}
When running, all tests fail because my certificate is insecure testing on a local installation and enforced ssl. Chromedriver (with basically identical setup) seems to accept the "acceptSslCerts" property and ignores the wrong cert. Geckodriver does not. Is the config wrong or does Nightwatch or Selenium have a problem with "acceptSslCerts"?
I am using the latest version of Geckodriver and Selenium 3.8.1
It looks like the correct capability for the firefox driver is acceptInsecureCerts.
You can see it listed here.

Selenium Grid 3 node startup ignoring -nodeconfig node.json

I'm just setting up a new Selenium Grid, and have just one node to register to my hub at the moment. Both hub and node running on CentOS 7. I have two different versions of Firefox on my node, and no other browser, as I will be setting up different nodes for different browser types.
If I start the node completely from the command line, specifying the different Firefox versions and locations, it all registers and starts properly. On the Grid console page I see my node with the two versions of Firefox, 3 instances of each, correctly detailed with the version numbers I'm specifying. This is the command line I use:
java -jar selenium-server-standalone.jar -role node -hub http://xx.xx.xx.xx:4444/grid/register -browser browserName=firefox,version=50.1.0,maxInsfirefox_binary=/opt/firefox-50.1.0/firefox,maxInstances=3,platform=LINUX -browser browserName=firefox,version=51.0.1,firefox_binary=/opt/firefox-51.0.1/firefox,maxInstances=3,platform=LINUX
If however I try using a .json config file to specify my parameters, the java startup completely ignores the config file and starts up using defaults; on the Grid console page it shows 5 Firefox sessions, 1 IE and 5 Chrome, and none of the Firefox sessions show the versions I'm specifying. This is the command line invoking the json file:
java -jar selenium-server-standalone.jar -role node -hub http://xx.xx.xx.xx:4444/grid/register -nodeConfig /opt/selenium/node.json
... and this is the node.json file itself:
{
“capabilities”:
[
{
"browserName": firefox,
"firefox_binary": "/opt/firefox-50.1.0/firefox",
"maxInstances": 3,
"platform": LINUX,
"version": 50,
"seleniumProtocol": WebDriver
},
{
"browserName": firefox,
"firefox_binary": "/opt/firefox-51.0.1/firefox",
"maxInstances": 3,
"platform": LINUX,
"version": 51,
"seleniumProtocol": WebDriver
},
],
“maxSession”: 5,
“port”: 5555,
"hub": "http://xx.xx.xx.xx:4444",
“register”: true,
"registerCycle": 5000,
"nodeStatusCheckTimeout": 5000,
"role": node,
"cleanUpCycle": 2000
}
I have tried putting all parameters encased in double quotation marks, in case that was the issue; made no difference.
Any clues why the parameters in my node.json file are being ignored, please?

Launch Safari on Mac node from Windows PC with Selenium RemoteWebDriver and Grid

I'm trying to fire off a test to my iMac from my Windows PC.
I have downloaded and installed the webdriver addon for Safari and I have established a connection to my Windows based Selenium Grid hub.
When I try to run my test I receive an error for an OperaDriver:
org.openqa.selenium.WebDriverException: The best matching driver provider org.openqa.selenium.opera.OperaDriver can't create a new driver instance for Capabilities [{browserName=safari, safari.options={port=0, cleanSession=true}, version=9, platform=MAC}]
Current Setup:
Windows PC:
java -jar selenium-server-standalone-2.53.0.jar -role hub -port 4445
Mac:
java -jar selenium-server-standalone-2.53.0.jar -role node -nodeConfig node1Config.json
node1Config:
{
"capabilities": [
{
"browserName": "safari",
"acceptSslCerts": true,
"javascriptEnabled": true,
"takeScreenshot": false,
"browser-version": "9",
"platform": "MAC",
"maxInstances": 5,
"cleanSession": true
}
],
"configuration": {
"_comment": "Configuration for Node",
"cleanUpCycle": 2000,
"timeout": 30000,
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"port": 5568,
"hubHost": "MyNetworkIpWasHere",
"register": true,
"hubPort": 4445,
"maxSessions": 5
}
}
Java to launch test:
DesiredCapabilities capabilities = DesiredCapabilities.safari();
capabilities.setPlatform(Platform.MAC);
capabilities.setBrowserName("safari");
capabilities.setVersion("9");
webDriver = new RemoteWebDriver(new URL("http://myipwashere:4445/wd/hub"), capabilities);
Edit: There are 5 safari nodes available on my grid, none are being used.
I must be overlooking something, any help would be greatly appreciated!
Thanks in advance.
After much trial and error, the URL being passed to the remotewebdriver was incorrect only for Safari. Hopefully this will help someone that has a similar problem in the future.
Thanks RemcoW for all of your help.