I'm having a hard time trying to grasp some concepts on selenium Grid/RC. What I need is to provide specific environments (ie6-on-xp, ie7-on-xp, etc) to the tests. For what I've been reading, the browser line in grid_configuration.yml do not make any reference of what version of MSIE or Firefox I'm running. So I can't get my head around in which form I can tell Grid/RC that I want some specific browsers and the path to run them (how RC knows which exe to run?)
Second, I'd like to run portable versions of those browsers. I've only seen that specified in the tests, and not in the RC's command line to run them. That is the way to do it, per test?
I will answer your question by breaking up the info that you need
What I need is to provide specific
environments (ie6-on-xp, ie7-on-xp,
etc) to the tests.
Well since you can't have multiple IE instances on the same machine, I know there are apps that allow you to do that but in my experience they cause more issues than solving them. Ideally you want different machines to run the tests. By doing this you are also setting up a selenium farm for your devs to use because they can target a test at a specific instance. So setting up Grid as an Infrastructure is a good step.
For what I've been reading, the
browser line in grid_configuration.yml
do not make any reference of what
version of MSIE or Firefox I'm
running. So I can't get my head around
in which form I can tell Grid/RC that
I want some specific browsers and the
path to run them (how RC knows which
exe to run?)
The YAML just lets you know what the grid can handle. When starting up the grid you need to make sure that you pass in similar configurations. Think of Se:GRID like you would Se:RC except you don't care where the RC server is because you speak to a central place that works the rest out for you.
If you need it to run tests against a specific items then you will need to handle this in your test setup. There is a common misconception that all tests will run the same in every single browser. This will happen if you never rely on XPath or CSS selectors in your tests because browsers always handle this slightly differently and the slight differences can lead to flaky tests which should always be avoided.
One way to specify which browser to use for a test is to have a central configuration file. In C# this would be using the app.config that has a collection for each browser and doing
Config
<Firefox>
<addKey browserVersion='3.5.6' OS='WindowsXP'>
</Firefox>
Central Config Class looking inside 1 element
public class BoothElement : ConfigurationElement
{
[ConfigurationProperty("browserVersion", DefaultValue = "", IsKey = true, IsRequired = true)]
public string browserVersion
{
get
{
return ((string)(base["browserVersion"]));
}
set
{
base["browserVersion"] = value;
}
}
Tests
selenium = new DefaultSelenium(HubPort, HubPort, browserVersion, SUTServer);
selenium.Open("/test.htm");
//Rest of the test
In python you could create an Array in a module that you include on all your tests
include.py
hubServer = 'hub'
hubPort = 5555
sut = 'http://serverUnderTest'
firefox = [hubServer,hubPort,"\*chrome",sut]
iexplore = [hubServer,hubPort,"\*iehta",sut]
test.py
sel = selenium(firefox)
sel.open("/test.html")
#rest of the test
When using Selenium Grid try thinking of it more as a test infrastructure help framework and hopefully that will help you a little more.
Second, I'd like to run portable
versions of those browsers. I've only
seen that specified in the tests, and
not in the RC's command line to run
them. That is the way to do it, per
test?
I have never tried to get Selenium to work on mobile browsers and don't think it would work to well, however with Selenium 2 which is currently in alpha there is android support for testing apps.
EDIT FROM COMMENT
- name: "Firefox on OS X"
browser: "*firefox"
- name: "Firefox on Linux"
browser: "*firefox"
- name: "IE on Windows"
browser: "*iehta"
- name: "Safari on OS X"
browser: "*safari"
So say we have the above setup, according to the YAML file we have a number of different *firefox instances. So to call those different ones in our tests our browser setup command would look like
selenium.Start(hubHost, hubPort, "Firefox on Linux", "http://serverUnderTest");
or selenium.Start(hubHost, hubPort, "Firefox on OS X", "http://serverUnderTest");
The hub will translate that into *firefox for you. I prefer having very granular names for my environment instead of the usual *firefox so that if there is a failure its easier to spot where it was and on which specific browser.
Virtual machines can be very handy for setting up "inexpensive" mules in the Selenium Grid farm.
Related
I am using the Karate framework to do the API testing. As part of CI efforts, we send an email at the end of test execution listing the summary of test results. There is a need to include the screeshot of the test execution counts from 'overview-feature.html' file.
I did so through the TestRunner.java file - launched Chrome using Chrome.start() and then using it to take screenshot. It all works well locally on Windows.
However when executing on CI server which is a Unix box, the chrome executable is not present in the default location (usr/bin/google-chrome) and hence the connection for the localhost fails.
Is there a way we can change the default location of the chrome executable?
PS: Apologies if this was too trivial to be asked.
Yes Chrome on CI is hard to get right, refer: https://stackoverflow.com/a/62325328/143475 - note that CI boxes typically are "headless" a browser may not be even installed.
I think the best thing for you is to ZIP the HTML and send it. But I really think you need to work with some CI experts, because the report generation and e-mailing business is normally done by things like Jenkins. What you are doing is certainly not normal or best-practice.
If you really want, there is a Karate Docker container that can give you a proper Chrome instance (see docs) but that is overkill for what you need.
EDIT: The Chrome Java API allows for customization of the executable path and this is in the docs: https://github.com/intuit/karate/tree/master/karate-core#chrome-java-api
It should be something like this:
Chrome.start("/opt/blah/chrome");
We have our Health Check automated wherein all of our applications are logged onto, all via Selenium and only on Internet Explorer.
The code runs well when it was executed directly from a batch file. However, when Jenkins calls this batch (.bat) file, it doesn't execute it completely.
BTW the platform is Windows Server 2008 R2 Standard
This is the .bat file code
#echo off
set path="";
set path="E:\XXXX\jre1.8.0_141\bin";
pushd E:\Jenkins_Softwares\SeleniumCode\HealthCheck_jar
SET JAVA_OPTS=-Xmx4g -Xms512m -XX:MaxPermSize=256m -XX:ReservedCodeCacheSize=128m -XX:MaxHeapSize=512m
java -jar HealthCheck_JenkinsNG.jar
I've added these additional IE options in the JAVA code before launching the IE driver.
InternetExplorerOptions options = new InternetExplorerOptions();
options.introduceFlakinessByIgnoringSecurityDomains();
options.enableNativeEvents();
options.destructivelyEnsureCleanSession();
When Jenkins executes the batch file, the IE browser opens into the Login page. There's something odd when this page is displayed - the entire page alignment is disrupted and all elements get aligned to the left. (I'd like to stress that when the batch file is instead executed directly, there is no such page alignment disruption. The elements retain their original centre position. For some reason, Jenkins sets all of this to the left). The alignment is not exactly a deal breaker for me.
However, when username and password is entered via the Selenium code, it types into the perfect text boxes; but when the submit button is hit, the content in these textboxes turn blank and I'm unable to login. (When this same piece of code is executed via running the batch file directly, I'm able to login and The homepage of my application is displayed)
I doubt if there's something wrong with the selenium java code. Since, it executes properly, when run from the .bat file or even command line or even as a Java Application from an IDE.
For some reason, when this is executed from Jenkins it does not work.
Is there any options or settings that needs to be set when Jenkins works with Selenium on IE 11? Because I've tried tweaking the selenium code so much, they all yield the same result - The elements on the Login page get cleared after the submit button is clicked.
Also, just to mention, all of this is run on one Master node of Jenkins only. There are no slave nodes.
You need to take care of a couple of things as follows:
For the build process Jenkins would need the path of jdk. Simply jre may not suffice.
JDK 8u141 is ancient now and you need to upgrade to latest JDK 8u202
introduceFlakinessByIgnoringSecurityDomains() (in Java) and IntroduceInstabilityByIgnoringProtectedModeSettings() (in DotNet) is not an ideal solution to address the issues croping out of Protected Mode settings.
Here you can find a detailed discussion in Internet Explorer Protective mode setting and Zoom levels
To work with Selenium, InternetExplorerDriver and InternetExplorer you need to fulfill the Required Configuration
I am trying to setup a test project that uses serenity and jbehave
I am noticing that all examples use serenity.properties that define a browser in it
I would like to structure my tests in a way so that same test can be executed in IE/firefox/chrome etc
How do I do this?
You can pass in properties as command line properties, so you can rerun the same tests with different browsers by passing in different settings for webdriver.driver, e.g.
$ mvn verify -Dwebdriver.driver=firefox
$ mvn verify -Dwebdriver.driver=chrome
etc.
I think you are able to get this to work by creating multiple Junit test classes with each its own driver and execute them all in a single run.
Every test class should be able to assign a specific 'managed' driver (e.g. PhantomJS, Chrome, Firefox). This is documented here: http://www.thucydides.info/docs/serenity/#_serenity_webdriver_support_in_junit
I don't know what the impact this would have on the generated report, hopefully you are still able to identify the feature/driver combination.
I have a configuration and tests written that don't work when I pass them through Selenium grid (but work without).
Selenium Tests / JUnit / Eclipse
One Jenkins job that clones the repository and runs the tests
Selenium Grid plugin installed on Jenkins, and all the nodes (Jenkins slaves on Linux with FF) connected to the hub.
Question 1: The Jenkins job for my test suite starts on a slave. Is this correct? Should I start it on master (I tried and got other issues, but at least I would focus on the real problems if I knew what is the correct way to go).
Question 2. I initialize the driver in #Before test. Is this correct?
driver = new RemoteWebDriver(new URL("http://......"), DesiredCapabilities.firefox());
This is where I initialize the firefox driver when not using grid, and it is working.
Question 3. How do I tell Selenium grid to offer me some logs. In all fairness, I have no idea if my tests even attempt to visit the hub.
The last question is regarding the error that I get only when using grid (again, all my tests work fine without).
org.openqa.selenium.WebDriverException: Specified firefox binary location does not exist or is not a real file: /usr/bin/firefox
There are quite a few solutions for the last error that I have googled and tried, without any luck. Being a beginner, I now doubt my configuration.
Thanks for your time.
Ann
Answers:
Question 1
It's not incorrect to run it on a slave. That's perfectly acceptable.
Question 2
You can initialize the driver wherever you want. You can put it in a #Before, a constructor, or even the method. It's all dependent on what works best for you.
Question 3
i'm not sure if you can do this with firefox, but with the chromedriver you can pass arguments like this: --verbose --log-path=/tmp/chromedriver.log \$*
Last Question
I will answer this question by first asking you a question. your grid and nodes are different servers than your jenkins server? If so, then this most likely means that you need to make sure that your firefox executables are under /usr/bin/firefox. If they are, then make sure that they are executable! e.g.: sudo chmod u+x /usr/bin/firefox
Also, you have your RemoteWebDriver initialization masked so i can't see, but make sure that you have the url of your grid, with /wd/hub concat'ed at the end. e.g.: http://selenium-grid:4444/wd/hub
I have the following set-up: an integration tests project which has a suite of tests written in Groovy/Geb + Spock, which are running perfectly both using Selenium WebDriver and Selenium Grid (RemoteWebDriver).
The problem is that no matter how much I try to tweak the "system", I can't get the tests to run in parallel (i.e. although I have 3 slaves [nodes] registered to the hub, only one of the slaves actually receives the requests). I've enforced maxSession=1 to the Selenium nodes and tried different combinations of parallel=classes|methods, threadCount and fork settings in failsafe plugin configuration (pom.xml file).
I have the feeling that the problem lies somewhere between the maven configuration and selenium grid, probably in relation to Geb/Spock config.
Does any of you have any insight on this issue?
PS: someone suggested that running tests in parallel using Geb / Spock is not possible - because for some reason ?Geb? locks the JUnitRunner (not sure what this means).
Add following configuration to your build.gradle file:
tasks.withType(Test) {
maxParallelForks = 3 // here three forks shall open in parallel
forkEvery = 1
include '**/*TestName*.class' // name of your test class
}
There are test frameworks, TestNG for example, that support parallel testing on the method level out of the box.
Spock, as an example to the contrary, does not support it.
But you do not have to have multithreading implemented by your test framework to make this work.
You can use your build tool to run test classes in parallel, both Maven and Gradle support this.
If you are using Maven, this documentation page and examples might help you:
https://maven.apache.org/surefire/maven-surefire-plugin/examples/fork-options-and-parallel-execution.html
Specifically have a look at "Forked Test Execution".
You can run it for sure, The point is you have to put them (your tests) in threads. Here is the link.