How to Run Parallel Appium Test With Selenium Grid Only By Specifying Platform - selenium

I am running parallel robot framework Appium tests using pabot. Below are my config files:
configDevice1.json
{
"capabilities":[
{
"platformName": "Android",
"platformVersion": "9.0",
"browserName": "Android-Huawei-Matepad",
"deviceName": "Huawei Matepad",
"udid": "902xxxxxxx",
"appActivity":"com.app.MainActivity",
"appPackage":"com.app.preview",
"noReset":false,
"automationName":"uiautomator2",
"groups":"device",
"maxInstances": 1
}
],
"configuration":
{
"url":"http://127.0.0.1:4724/wd/hub",
"cleanUpCycle":2500,
"timeout":30000,
"maxSession": 1,
"register": true,
"registerCycle": 1000,
"hubPort": 4444,
"hubHost": "localhost"
}
}
configDevice2.json
{
"capabilities":[
{
"platformName": "Android",
"platformVersion": "9.0",
"browserName": "Android-Samsung",
"deviceName": "Android Samsung",
"udid": "111xxxxxxx",
"appActivity":"com.app.MainActivity",
"appPackage":"com.app.preview",
"noReset":false,
"automationName":"uiautomator2",
"groups":"device",
"maxInstances": 1
}
],
"configuration":
{
"url":"http://127.0.0.1:4724/wd/hub",
"cleanUpCycle":2500,
"timeout":30000,
"maxSession": 1,
"register": true,
"registerCycle": 1000,
"hubPort": 4444,
"hubHost": "localhost"
}
}
I started the two Appium nodes in separate process:
appium --nodeconfig configDevice1.json -p 4723 -bp 5723
appium --nodeconfig configDevice2.json -p 4724 -bp 5724
The parallel test works fine if I define the udid on the desired capabilities. However, I am looking for a way to run parallel test without having to specify any udid. For instance, I want to run parallel tests on all devices with platform of Android.
When I run the parallel tests only with capabilities of platformName=Android, all tests run only on one device, conflicting each session from another.
I can see Appium is retrieving connected devices, but then only proceed selecting the first device from that list. This happens in both Appium sessions (Hence making tests failure because it tries to override an existing session)
[debug] [ADB] Connected devices: [{"udid":"111xxxxxxx","state":"device"},{"udid":"902xxxxxxx","state":"device"}]
[AndroidDriver] Using device: 111xxxxxxx
Is there anything one can do to make tests parallel on platform information available on Selenium grid nodes?
Thanks!

Silly me. The config.json value set should be platform and version. It works well now!

Related

How to setup custom browser versions for selenium grid?

Setup
We have a selenium grid where all nodes are completely identical except one. All the identical nodes run windows 10, have basic chrome, IE, edge browsers, and run on MST. The exception runs IE, chrome, and edge, but they are set to look like it is in the UK, and runs GMT+0 most of the year (except DST of course then it runs BST). This changes date formats and such and is used to test that dates are formatted properly and that documents on our system reflect accurate times in their own timezone. The browsers on the UK node all have custom browser versions declared in their node_XXXX.json file for selenium grid.
Issue
We lost our hub server a while back, and after rebuilding we can no longer get the IE, EDGE, and Firefox browsers to work on the UK server. We used to use custom browser versions for the Uk browsers eg: instead of "version":11 it is "version":"11.eu" to denote that we wanted to run on the special server. Chrome works fine with this model. However, IE is giving an error:
OpenQA.Selenium.WebDriverException : Error forwarding the new session cannot find : Capabilities {browserName: internet explorer, browserVersion: 11.eu, platformName: windows, se:ieOptions: {enablePersistentHover: true, ie.ensureCleanSession: true, nativeEvents: true}}
Attempted Solutions
We have already tried to change the version number (It used to say 9.eu, legacy from when we were running IE9). We have also tried "updating" IE to version 11 again. Nothing has worked or allowed the driver to connect.
Question
How do we get our UK IE browser to work on the selenium grid again? We have many tests failing simply because they cannot connect to a driver instance. Any help would be appreciated, thanks!
Reference
For reference, here is our node_XXXX.json file:
{
"capabilities": [
{
"seleniumProtocol": "WebDriver",
"browserName": "firefox",
"maxInstances": 1,
"version": "17.eu",
"platform": "WINDOWS"
},
{
"seleniumProtocol": "WebDriver",
"browserName": "internet explorer",
"maxInstances": 1,
"version": "11.eu",
"platform": "windows"
},
{
"seleniumProtocol": "WebDriver",
"browserName": "chrome",
"maxInstances": 1,
"version": "77.eu",
"platform": "WINDOWS"
}
],
"loadedFromFile":
"node_5555.json",
"proxy": "com.groupon.seleniumgridextras.grid.proxies.SetupTeardownProxy",
"servlets": [],
"maxSession": 3,
"port": 5555,
"register": true,
"unregisterIfStillDownAfter": 10000,
"hubPort": 4444,
"hubHost": "test-slum01.ndtest.local",
"registerCycle": 5000,
"nodeStatusCheckTimeout": 10000,
"custom": {},
"downPollingLimit": 0
}
And this is the code that calls the remote web driver:
case "iehta9UK":
var ieUkOptions = new InternetExplorerOptions();
ieUkOptions.EnsureCleanSession = true;
ieUkOptions.BrowserVersion = "11.eu";
return IS_REMOTE_RUN ? CreateRemoteWebDriver(ieUkOptions) : new InternetExplorerDriver(ieUkOptions);
This is the strictly US IE code that works fine:
case "IE":
case "ie":
case "InternetExplorer":
case "iehta11":
var ieOptions = new InternetExplorerOptions();
ieOptions.EnsureCleanSession = true;
return IS_REMOTE_RUN ? CreateRemoteWebDriver(ieOptions) : new InternetExplorerDriver(ieOptions);
Aha! We figured it out. We had made the changes to the node_XXXX.json file shown above, but had not restarted the selenium grid service.
Make sure you restart the server after making changes to the json file for them to take affect.

How do you setup an HtmlUnit node with Selenium Grid?

I'm having trouble setting up an HtmlUnit node with Selenium Grid.
I'm launching my Hub with:
java -jar selenium-server-standalone.jar -role hub -port 4444
And HtmlUnitDriver nodes with:
java -jar selenium-server-standalone.jar -role node -browser browserName=htmlunit,maxInstances=5 -hub http://localhost:4444/grid/register -port 5564
My code to connect to the node, which is getting the exception, looks something like:
DesiredCapabilities cap = new DesiredCapabilities();
cap.setBrowserName("htmlunit");
cap.setJavascriptEnabled(false);
driver = new RemoteWebDriver(new URL("http://localhost:4444/wd/hub"), cap);
The exception I'm getting is:
Unable to create session from {
"desiredCapabilities": {
"browserName": "htmlunit",
"server:CONFIG_UUID": "93a0486d-c9f6-46da-8065-603ab07c0294",
"javascriptEnabled": false
},
"capabilities": {
"firstMatch": [
{
"browserName": "htmlunit",
"server:CONFIG_UUID": "93a0486d-c9f6-46da-8065-603ab07c0294"
}
]
}
}
To set up a Selenium Grid instance, I use the Selenium Foundation library. The "standalone" JAR brings in scads of transitive dependencies you don't need that end up creating conflicts. I created a sample project (local-grid-utility) that launches a Selenium 2 Grid that dispenses PhantomJS sessions, which could be easily altered to launch a Selenium 3 Grid dispensing HtmlUnit sessions.
The key to getting HtmlUnit sessions stood up in Grid is providing a complete class path that includes all of the dependencies. Here's what the library assembles for HtmlUnitDriver 2.40.0:
${JAVA_HOME}/jre/bin/java -cp ${M2_ROOT}/repository/org/seleniumhq/selenium/selenium-support/3.141.59/selenium-support-3.141.59.jar:${M2_ROOT}/repository/org/seleniumhq/selenium/htmlunit-driver/2.40.0/htmlunit-driver-2.40.0.jar:${M2_ROOT}/repository/org/apache/commons/commons-lang3/3.8.1/commons-lang3-3.8.1.jar:${M2_ROOT}/repository/xerces/xercesImpl/2.12.0/xercesImpl-2.12.0.jar:${M2_ROOT}/repository/org/seleniumhq/selenium/selenium-api/3.141.59/selenium-api-3.141.59.jar:${M2_ROOT}/repository/net/sourceforge/htmlunit/htmlunit-core-js/2.40.0/htmlunit-core-js-2.40.0.jar:${M2_ROOT}/repository/org/eclipse/jetty/websocket/websocket-api/9.4.28.v20200408/websocket-api-9.4.28.v20200408.jar:${M2_ROOT}/repository/xalan/xalan/2.7.2/xalan-2.7.2.jar:${M2_ROOT}/repository/org/eclipse/jetty/jetty-xml/9.4.28.v20200408/jetty-xml-9.4.28.v20200408.jar:${M2_ROOT}/repository/org/apache/httpcomponents/httpclient/4.5.12/httpclient-4.5.12.jar:${M2_ROOT}/repository/org/eclipse/jetty/jetty-http/9.4.28.v20200408/jetty-http-9.4.28.v20200408.jar:/Applications/Eclipse.app/Contents/Eclipse/plugins/com.beust.jcommander_1.72.0.jar:${M2_ROOT}/repository/org/apache/httpcomponents/httpcore/4.4.13/httpcore-4.4.13.jar:${M2_ROOT}/repository/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar:${M2_ROOT}/repository/commons-codec/commons-codec/1.11/commons-codec-1.11.jar:${M2_ROOT}/repository/net/sourceforge/htmlunit/neko-htmlunit/2.40.0/neko-htmlunit-2.40.0.jar:${M2_ROOT}/repository/com/squareup/okhttp3/okhttp/3.11.0/okhttp-3.11.0.jar:${M2_ROOT}/repository/commons-net/commons-net/3.6/commons-net-3.6.jar:${M2_ROOT}/repository/org/seleniumhq/selenium/selenium-server/3.141.59/selenium-server-3.141.59.jar:${M2_ROOT}/repository/xalan/serializer/2.7.2/serializer-2.7.2.jar:${M2_ROOT}/repository/org/eclipse/jetty/jetty-client/9.4.28.v20200408/jetty-client-9.4.28.v20200408.jar:${M2_ROOT}/repository/ch/qos/logback/logback-classic/1.2.3/logback-classic-1.2.3.jar:${M2_ROOT}/repository/org/apache/commons/commons-text/1.8/commons-text-1.8.jar:${M2_ROOT}/repository/org/eclipse/jetty/jetty-io/9.4.28.v20200408/jetty-io-9.4.28.v20200408.jar:${M2_ROOT}/repository/net/sourceforge/htmlunit/htmlunit-cssparser/1.5.0/htmlunit-cssparser-1.5.0.jar:${M2_ROOT}/repository/commons-logging/commons-logging/1.2/commons-logging-1.2.jar:${M2_ROOT}/repository/org/apache/httpcomponents/httpmime/4.5.12/httpmime-4.5.12.jar:${M2_ROOT}/repository/org/seleniumhq/selenium/jetty-repacked/9.4.12.v20180830/jetty-repacked-9.4.12.v20180830.jar:${M2_ROOT}/repository/javax/servlet/javax.servlet-api/3.1.0/javax.servlet-api-3.1.0.jar:${M2_ROOT}/repository/org/eclipse/jetty/websocket/websocket-client/9.4.28.v20200408/websocket-client-9.4.28.v20200408.jar:${M2_ROOT}/repository/org/brotli/dec/0.1.2/dec-0.1.2.jar:${M2_ROOT}/repository/com/google/guava/guava/28.1-jre/guava-28.1-jre.jar:${M2_ROOT}/repository/com/squareup/okio/okio/1.14.0/okio-1.14.0.jar:${M2_ROOT}/repository/net/sourceforge/htmlunit/htmlunit/2.40.0/htmlunit-2.40.0.jar:${M2_ROOT}/repository/org/seleniumhq/selenium/selenium-remote-driver/3.141.59/selenium-remote-driver-3.141.59.jar:${M2_ROOT}/repository/commons-io/commons-io/2.6/commons-io-2.6.jar:${M2_ROOT}/repository/org/eclipse/jetty/jetty-util/9.4.28.v20200408/jetty-util-9.4.28.v20200408.jar:${M2_ROOT}/repository/org/eclipse/jetty/websocket/websocket-common/9.4.28.v20200408/websocket-common-9.4.28.v20200408.jar org.openqa.grid.selenium.GridLauncherV3 -role node -servlets org.openqa.grid.web.servlet.LifecycleServlet -host 192.168.1.6 -port 34464 -nodeConfig /Users/bb8d/github/Selenium-Foundation/target-s3/classes/nodeConfig-s3-9CF40153.json
You could create your own standalone package to bundle up the dependencies if that suits your fancy. The node server is provided by the GridLauncherV3 class. I also add the LifecycleServlet plugin to enable remote shutdown. The host specification corresponds to the local machine, and the port is auto-selected from whatever is currently available.
The JSON configuration file looks like this:
{
"capabilities": [
{
"browserName": "htmlunit",
"javascriptEnabled": true,
"maxInstances": 5,
"seleniumProtocol": "WebDriver"
}
],
"custom": {
},
"debug": false,
"downPollingLimit": 2,
"enablePlatformVerification": true,
"host": "0.0.0.0",
"hub": "http:\u002f\u002f192.168.1.6:4445\u002fwd\u002fhub",
"maxSession": 5,
"nodePolling": 5000,
"nodeStatusCheckTimeout": 5000,
"port": -1,
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"register": true,
"registerCycle": 5000,
"role": "node",
"servlets": [
],
"unregisterIfStillDownAfter": 60000,
"withoutServlets": [
]
}
The critical settings can be specified in the command line, but I think the config file is more manageable.

Selenium grid - node configuration file for Firefox not reading moz:firefoxOptions

I have setup my selenium grid (currently 1 hub and 1 node).
The hub is working fine but for my node I setup the below configuration file (nodeconfig.json)
I wanted to start the firefox in headless mode but it looks like the "moz:firefoxOptions" configuration did not catch up.
I tried setting the "moz:firefoxOptions" in a variety of location in side the json file without any luck :( .
I are using :
windows server 2016 64bit ,
selenium-server-standalone 3.9.1
geckodriver v0.19.1
firefox v59.0
we are running the node using the following command :
java -Dwebdriver.gecko.driver="geckodriver.exe" -Dwebdriver.chrome.driver="chromedriver.exe" -jar selenium-server-standalone-3.9.1.jar -role node -nodeConfig nodeconfig.json
I can tell that the configuration file was read as I can see it in the selenium grid console.
nodeconfig.json file:
{
"capabilities": [
{
"browserName": "firefox",
"maxInstances": 5,
"seleniumProtocol": "WebDriver",
"alwaysMatch": {
"moz:firefoxOptions": {
"args": [
"-headless"
]
}
}
},
{
"browserName": "chrome",
"maxInstances": 5,
"seleniumProtocol": "WebDriver"
}
],
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"maxSession": 5,
"port": 5555,
"register": true,
"registerCycle": 5000,
"hub": "http://localhost:4444",
"nodeStatusCheckTimeout": 5000,
"nodePolling": 5000,
"role": "node",
"unregisterIfStillDownAfter": 60000,
"downPollingLimit": 2,
"debug": false,
"servlets": [],
"withoutServlets": [],
"custom": {}
}
In addition from the logs I can tell that the argument is not added.
The relevant line is :
1520854660258 mozrunner::runner INFO Running command: "C:\Program Files\Mozilla Firefox\firefox.exe" "-marionette" "-profile" "C:\Users\avi\AppData\Local\Temp\rust_mozprofile.03tQm3hduVDR"
As per the documentation the argument is --headless not -headless. So possibly you have to change to :
"moz:firefoxOptions": {
"args": [
"--headless"
]
}
You can find a detailed discussion in How to make firefox headless programatically in Selenium with python?

Selenium Grid - limited on running tests in parallel

Let me describe the issue that I am currently facing:
I have setup a selenium grid with 11 VMs (1 hub and 10 nodes)
I have a test suite containing 10 tests. I am trying to run all these tests in parallel at the same time across the 10 available nodes. I am running the tests only on Chrome and so one test per node is good. I am using Gallio to run my tests in parallel.
I have setup a jenkins job to kick off my test suite. As soon as the jenkins job is kicked off, the hub gets the request and is passing them on to the nodes. There are no issues here. The only problem is, initially the hub is only sending 5 tests requests to the nodes. So, only 5 nodes are running the first 5 tests and the other 5 nodes are ideal. Only after one of the test finishes, the next test is sent to node 6. Once another test completes, the hub sends the next test to node 7 and so on.
I would ideally want all the 10 tests to be run in parallel on all the 10 available nodes. Can anyone help me here?
Here is the json config files that I am using:
hubconfig.json (I tried changing maxSession and that did not do anything)
{
"host": localhost,
"port": 4444,
"newSessionWaitTimeout": -1,
"servlets" : [],
"prioritizer": null,
"capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher",
"throwOnCapabilityNotPresent": true,
"nodePolling": 5000,
"cleanUpCycle": 5000,
"timeout": 240000,
"browserTimeout": 0,
"maxSession": 5
}
nodeconfig.json
{
"capabilities":
[
{
"browserName": "chrome",
"platform": "WINDOWS",
"maxInstances": 1
}
],
"configuration":
{
"proxy":"org.openqa.apt.AptWebDriverProxy",
"hubHost": localhost ,
"hubPort":4444,
"maxSession":5,
"nodePolling": 2000,
}
}

Selenium Grid 2 : Local hub and 2 local nodes

Currently, I am setting up a Selenium Grid for running test suites on a hub and distribute these tests over the nodes.
The idea is to get the test executed on the hub and distributed over 10 nodes to execute them further. For the starters and evaluate the grid, I have set up a local hub and 2 local nodes.
In my test class, I have 4 tests, on running the test using RemoteDriver and passing the hub as URL and configured capabilities, it executes all four tests on Node1 and does not distribute it over to Node2. Also, it runs all 4 tests serially on Node1. Does any one know what could be wrong here. Please find the setup below.
Hub Configuration :
C:\Proto\Selserversidedjars>java
-jar selenium-server-standalone-2.44.0.jar
-role hub -hubConfig DefaultHub.json
Node 1 Configuration :
C:\Proto\Selserversidedjars>java
-jar selenium-server-standalone-2.44.0.jar
-role node
-hub http://localhost:4444/grid/register
Node 2 Configuration :
C:\Proto\Selserversidedjars>java
-jar selenium-server-standalone-2.44.0.jar
-role node
-nodeConfig DefaultNode1.json
-port 6666
Defaulthub.json :
{
"host": null,
"port": 4444,
"newSessionWaitTimeout": -1,
"servlets" : [],
"prioritizer": null,
"capabilityMatcher": "org.openqa.grid.internal.utils.DefaultCapabilityMatcher",
"throwOnCapabilityNotPresent": true,
"nodePolling": 5000,
"cleanUpCycle": 5000,
"timeout": 300000,
"browserTimeout": 0,
"maxSession": 5,
"jettyMaxThreads":-1
}
DefaultNode1.json:
{
"capabilities":
[
{
"browserName": "*firefox",
"maxInstances": 5,
"seleniumProtocol": "Selenium"
},
{
"browserName": "*googlechrome",
"maxInstances": 5,
"seleniumProtocol": "Selenium"
},
{
"browserName": "*iexplore",
"maxInstances": 1,
"seleniumProtocol": "Selenium"
},
{
"browserName": "firefox",
"maxInstances": 5,
"seleniumProtocol": "WebDriver"
},
{
"browserName": "chrome",
"maxInstances": 5,
"seleniumProtocol": "WebDriver"
},
{
"browserName": "internet explorer",
"maxInstances": 1,
"seleniumProtocol": "WebDriver"
}
],
"configuration":
{
"proxy": "org.openqa.grid.selenium.proxy.DefaultRemoteProxy",
"maxSession": 5,
"port": 6666,
"host": ip,
"register": true,
"registerCycle": 5000,
"hubPort": 4444,
"hubHost": ip
}
}
Now after these configuration setup, my hub is started on port 4444, node 1 is started at 5555, node 2 is started at 6666. In my TestNG tests, I am trying to execute 4 tests. Given that I have multiple tests to run in a single class, is it possible to distribute those tests over multiple nodes or multiple instances on a single node?
Any help on "how to execute multiple instances on Node1 and distribute tests(let's say more than 10 tests) over to the Node2 in this situation" will be highly appreciated.
Thanks in advance!
Running serially or parallely depends on how you are triggering your tests and just doesn't happen automatically just by using the grid. Grid just helps in distributing tests over various nodes - if tests are sent serially it would distribute serially, if sent parallely, it would distribute on various nodes.
You might consider using TestNG to run your tests parallely or you can venture to implement your own parallelism.
You mention your single class has multiple tests - with testng, you can set the parallel attribute to methods which would trigger each test in individual threads which would be sent to the grid and the hub would take care of picking up a free node and executing your test.
Take care to write threadsafe driver launch code so that each thread has it's own driver instance.