I am using the Karate framework to do the API testing. As part of CI efforts, we send an email at the end of test execution listing the summary of test results. There is a need to include the screeshot of the test execution counts from 'overview-feature.html' file.
I did so through the TestRunner.java file - launched Chrome using Chrome.start() and then using it to take screenshot. It all works well locally on Windows.
However when executing on CI server which is a Unix box, the chrome executable is not present in the default location (usr/bin/google-chrome) and hence the connection for the localhost fails.
Is there a way we can change the default location of the chrome executable?
PS: Apologies if this was too trivial to be asked.
Yes Chrome on CI is hard to get right, refer: https://stackoverflow.com/a/62325328/143475 - note that CI boxes typically are "headless" a browser may not be even installed.
I think the best thing for you is to ZIP the HTML and send it. But I really think you need to work with some CI experts, because the report generation and e-mailing business is normally done by things like Jenkins. What you are doing is certainly not normal or best-practice.
If you really want, there is a Karate Docker container that can give you a proper Chrome instance (see docs) but that is overkill for what you need.
EDIT: The Chrome Java API allows for customization of the executable path and this is in the docs: https://github.com/intuit/karate/tree/master/karate-core#chrome-java-api
It should be something like this:
Chrome.start("/opt/blah/chrome");
Related
As I am implementing an automated way to GUI test our webapplication with selenium I ran into some issues.
I am using selenese-runner to execute our Selenium test suites, created with Selenium IDE as a post build action in Jenkins.
This works perfeclty fine, as the build fails when something is wrong, and the build succeeds if all tests are passed. And the results are stored on a per build basis as HTML files, generated be selenese-runner.
My problem is however, that I seem to be unable to find a way, how to display these results in the respective jenkins build.
Does anyone have an idea how to solve this issue. Or maybe I am on the wrong path at all?
Your help is highly appreciated!
I believe the JUnit plugin should do what you want, but it doesn't work for me.
My config uses this shell script to run the tests (you can see the names of all my test suites):
/usr/bin/Xvfb &
export DISPLAY=localhost:0.0
cd ${WORKSPACE}
java -jar ./test/selenium/bin/selenese-runner.jar --baseurl http://${testenvironment} --screenshot-on-fail ./seleniumResults/ --html-result ./seleniumResults/ ./test/selenium/Search_TestSuite.html ./test/selenium/Admin_RegisteredUser_Suite.html ./test/selenium/Admin_InternalUser_Suite.html ./test/selenium/PortfolioAgency_Suite.html ./test/selenium/FOAdmin_Suite.html ./test/selenium/PublicWebsite_Suite.html ./test/selenium/SystemAdmin_Content_Suite.html ./test/selenium/SystemAdmin_MetaData_Suite.html
killall Xvfb
And I can see the result of the most recent test (you can see the name of my jenkins task folder)
http://<JENKINS.MY.COMPANY>/job/seleniumRegressionTest/ws/seleniumResults/index.html
Earlier tests are all saved on the Jenkins server, so I can view them if I need to.
I am trying to setup a test project that uses serenity and jbehave
I am noticing that all examples use serenity.properties that define a browser in it
I would like to structure my tests in a way so that same test can be executed in IE/firefox/chrome etc
How do I do this?
You can pass in properties as command line properties, so you can rerun the same tests with different browsers by passing in different settings for webdriver.driver, e.g.
$ mvn verify -Dwebdriver.driver=firefox
$ mvn verify -Dwebdriver.driver=chrome
etc.
I think you are able to get this to work by creating multiple Junit test classes with each its own driver and execute them all in a single run.
Every test class should be able to assign a specific 'managed' driver (e.g. PhantomJS, Chrome, Firefox). This is documented here: http://www.thucydides.info/docs/serenity/#_serenity_webdriver_support_in_junit
I don't know what the impact this would have on the generated report, hopefully you are still able to identify the feature/driver combination.
I see that Codeception takes screenshots if a test fails, is there anyway to make it take screenshots if the tests pass too?
For info, we're using the phantomjs WebDriver.
The best documentation for this I've found is for the WebDriver module
In there there's a description of makeScreenshot() which allows you to take a screenshot at any given point of your tests. It appears that it will put all of the screenshots in one directory so better think about a good naming convention.
Alternatively, we've now got the Recorder extension working which automates the process of taking screenshots at every test step and presents them in a useful report.
You can use Recorder extension with option delete_successful in configuration acceptance.suite.yml (in extensions: part, not modules:)
extensions:
enabled:
- Codeception\Extension\Recorder:
delete_successful: false # keep screenshots of successful tests
Quote from documentation about Recorder and PhantomJS:
Since PhantomJS doesn’t give you any visual feedback, it’s probably a good idea to install Codeception\Extension\Recorder which gives you screenshots of how PhantomJS “sees” your pages.
More information you can find in documentation for Recorder extension: http://codeception.com/extensions#Recorder
How to find screenchots
Results of the last test running you can find in a file _output/records.html. The file contains links to slideshows with screenshots for each test. Just reload the page after the test.
I'm trying to follow the tutorial here to setup a headless selenium test-run with jenkins. I'm running CentOS 5.6, and I've followed the instructions. Now, when I run this:
export DISPLAY=":99" && java -jar /var/lib/selenium/selenium-server.jar -browserSessionReuse -htmlSuite *firefox http://www.google.com ./test/selenium/html/TestSuite.html ./target/selenium/html/TestSuiteResults.html
Selenium hangs on INFO - Checking Resource Aliases. I can run the TestSuite.html file manually, and the path is correct.
How can I even begin to try and figure out what's going on? Is there a way I could connect to the display to see what's happening? I am behind a corporate proxy, but with or without -Dhttp.proxyHost arguments, I get the same hung result.
Well, after pointing at an internal server, I get right on past the INFO - Checking Resource Aliases step, so clearly the proxy was the issue.
By trying to hit a site that required the proxy, I was doing too much at once. Confounding variables confounded me.
Selenium is not hanging on INFO - Checking Resource Aliases. Its waiting for a command to execute. You need to trigger your tests using ANT or some other build tool in Jenkins. That should get you going
I'm having a hard time trying to grasp some concepts on selenium Grid/RC. What I need is to provide specific environments (ie6-on-xp, ie7-on-xp, etc) to the tests. For what I've been reading, the browser line in grid_configuration.yml do not make any reference of what version of MSIE or Firefox I'm running. So I can't get my head around in which form I can tell Grid/RC that I want some specific browsers and the path to run them (how RC knows which exe to run?)
Second, I'd like to run portable versions of those browsers. I've only seen that specified in the tests, and not in the RC's command line to run them. That is the way to do it, per test?
I will answer your question by breaking up the info that you need
What I need is to provide specific
environments (ie6-on-xp, ie7-on-xp,
etc) to the tests.
Well since you can't have multiple IE instances on the same machine, I know there are apps that allow you to do that but in my experience they cause more issues than solving them. Ideally you want different machines to run the tests. By doing this you are also setting up a selenium farm for your devs to use because they can target a test at a specific instance. So setting up Grid as an Infrastructure is a good step.
For what I've been reading, the
browser line in grid_configuration.yml
do not make any reference of what
version of MSIE or Firefox I'm
running. So I can't get my head around
in which form I can tell Grid/RC that
I want some specific browsers and the
path to run them (how RC knows which
exe to run?)
The YAML just lets you know what the grid can handle. When starting up the grid you need to make sure that you pass in similar configurations. Think of Se:GRID like you would Se:RC except you don't care where the RC server is because you speak to a central place that works the rest out for you.
If you need it to run tests against a specific items then you will need to handle this in your test setup. There is a common misconception that all tests will run the same in every single browser. This will happen if you never rely on XPath or CSS selectors in your tests because browsers always handle this slightly differently and the slight differences can lead to flaky tests which should always be avoided.
One way to specify which browser to use for a test is to have a central configuration file. In C# this would be using the app.config that has a collection for each browser and doing
Config
<Firefox>
<addKey browserVersion='3.5.6' OS='WindowsXP'>
</Firefox>
Central Config Class looking inside 1 element
public class BoothElement : ConfigurationElement
{
[ConfigurationProperty("browserVersion", DefaultValue = "", IsKey = true, IsRequired = true)]
public string browserVersion
{
get
{
return ((string)(base["browserVersion"]));
}
set
{
base["browserVersion"] = value;
}
}
Tests
selenium = new DefaultSelenium(HubPort, HubPort, browserVersion, SUTServer);
selenium.Open("/test.htm");
//Rest of the test
In python you could create an Array in a module that you include on all your tests
include.py
hubServer = 'hub'
hubPort = 5555
sut = 'http://serverUnderTest'
firefox = [hubServer,hubPort,"\*chrome",sut]
iexplore = [hubServer,hubPort,"\*iehta",sut]
test.py
sel = selenium(firefox)
sel.open("/test.html")
#rest of the test
When using Selenium Grid try thinking of it more as a test infrastructure help framework and hopefully that will help you a little more.
Second, I'd like to run portable
versions of those browsers. I've only
seen that specified in the tests, and
not in the RC's command line to run
them. That is the way to do it, per
test?
I have never tried to get Selenium to work on mobile browsers and don't think it would work to well, however with Selenium 2 which is currently in alpha there is android support for testing apps.
EDIT FROM COMMENT
- name: "Firefox on OS X"
browser: "*firefox"
- name: "Firefox on Linux"
browser: "*firefox"
- name: "IE on Windows"
browser: "*iehta"
- name: "Safari on OS X"
browser: "*safari"
So say we have the above setup, according to the YAML file we have a number of different *firefox instances. So to call those different ones in our tests our browser setup command would look like
selenium.Start(hubHost, hubPort, "Firefox on Linux", "http://serverUnderTest");
or selenium.Start(hubHost, hubPort, "Firefox on OS X", "http://serverUnderTest");
The hub will translate that into *firefox for you. I prefer having very granular names for my environment instead of the usual *firefox so that if there is a failure its easier to spot where it was and on which specific browser.
Virtual machines can be very handy for setting up "inexpensive" mules in the Selenium Grid farm.