I have a .side file generated by the Selenium IDE, which I need to run on CI using Jenkins.
I am running it as a build step with the following shell command:
selenium-side-runner /path/to/file.ide
The problem arises due to the fact that no matter if the selenium test fails, Jenkins always shows is as success.
In this thread it's suggested to upload the file as generic, but still, the commands to execute it are missing
How to upload a generic file into a Jenkins job?
I've found a possible solution to it on this posts, but I would appreciate having a cleaner way to solve this instead of parsing the results checking for errors.
How to mark a build unstable in Jenkins when running shell scripts
Is there a plugin able to run selenium .side files on Jenkins and this one showing the success/failures of the test?
You can generate a Junit test report file and then use the Jenkins Junit plugin after your tests execution.
selenium-side-runner --output-directory=results --output-format=junit
# Outputs results in `junit` frormat in `./results/projectName.xml'
Check the official documentation for more details.
Related
As I am implementing an automated way to GUI test our webapplication with selenium I ran into some issues.
I am using selenese-runner to execute our Selenium test suites, created with Selenium IDE as a post build action in Jenkins.
This works perfeclty fine, as the build fails when something is wrong, and the build succeeds if all tests are passed. And the results are stored on a per build basis as HTML files, generated be selenese-runner.
My problem is however, that I seem to be unable to find a way, how to display these results in the respective jenkins build.
Does anyone have an idea how to solve this issue. Or maybe I am on the wrong path at all?
Your help is highly appreciated!
I believe the JUnit plugin should do what you want, but it doesn't work for me.
My config uses this shell script to run the tests (you can see the names of all my test suites):
/usr/bin/Xvfb &
export DISPLAY=localhost:0.0
cd ${WORKSPACE}
java -jar ./test/selenium/bin/selenese-runner.jar --baseurl http://${testenvironment} --screenshot-on-fail ./seleniumResults/ --html-result ./seleniumResults/ ./test/selenium/Search_TestSuite.html ./test/selenium/Admin_RegisteredUser_Suite.html ./test/selenium/Admin_InternalUser_Suite.html ./test/selenium/PortfolioAgency_Suite.html ./test/selenium/FOAdmin_Suite.html ./test/selenium/PublicWebsite_Suite.html ./test/selenium/SystemAdmin_Content_Suite.html ./test/selenium/SystemAdmin_MetaData_Suite.html
killall Xvfb
And I can see the result of the most recent test (you can see the name of my jenkins task folder)
http://<JENKINS.MY.COMPANY>/job/seleniumRegressionTest/ws/seleniumResults/index.html
Earlier tests are all saved on the Jenkins server, so I can view them if I need to.
I would like to add a command line start for Selenium IDE test scripts to our build process so some of the tests are run immediately after the latest code is built for our products.
I have been looking and everything I find seems to relate to older versions. I can't find anything that looks up to date.
Can I run tests that I have recorded in Selenium IDE 2.9.1 from the command line - and by extension, a batch file to run multiple tests.
Is there documentation that I did not find? Can someone direct me to it?
Thanks.
You didnt specify an operating system but i think you mean windows, and yes, you can run selenium IDE test suites from powershell or CMD on windows.
You will need Java installed on your machine, and also the Selenium Server.jar file which should have come with your installation of Selenium, if not you can download from the Selenium Site.
Then, create a test suite in IDE and save it somewhere on your machine. (you can only run test suites, not testcases - i wont explain the difference between the two here)
Then in powershell or CMD you run this command, you will need to change it depending on your installation directories.
C:\Java\JDK1.8.0_66-X64\bin\java -jar C:\Selenium\Selenium-Server\selenium-server.jar -htmlSuite *firefox "baseurl" "C:\temp\testsuitename" "C:\TEMP\logs" -timeout 0060 -trustAllSSLCertificates
I will break this down for you -
C:\JDK1.8.0_66-X64\bin\java -jar - this is your Java installation directory (mine is Java JDK 1.8)
C:\Selenium\Selenium-Server\selenium-server.jar - This is the location of your Selenium .jar file.
-htmlSuite *firefox These are .jar file options to tell the jar file you are running it from a command line and to use firefox
"baseurl" In here, you should enter your base url which would normally be at the top of the IDE gui. eg, www.google.com
"C:\temp\testsuitename" The full path where you have saved your test suite
"C:\TEMP\logs" Running from command line produces a html log, advise where you want this placed
-timeout 0060 -trustAllSSLCertificates There are additional options, of which there are many. In this case the total time the command will run before it times out is 60 seconds, and it will ignore all those untrusted certificate errors that firefox often gets (where you need to keep adding exceptions etc).
Hope i could help!
I have recorded some simple selenium tests by Selenium IDE. Now I want to run those tests in Jenkins.
Which plugin to Jenkins do I need to do that? And how to run the tests step by step? Help is appreciated.
you can use recorded selenium IDE script and selenium-server.jar file to run it from Jenkins
Here is steps:
Go to SeleniumHQ page and download Selenium Server file
Eg: selenium-server-standalone-2.33.0.jar
Repair html test suite Use Selenium IDE to record then save as
html test case and test suite then put them in a same folder
eg: TestCase.html, TestSuite.html
In jenkins
Plugin Seleniumhq
Configure Selenium runner file
Manage Jenkins > Configure System > Selenium Remote Control: htmlSuite Runner = path to file u have download in step 1
Configure Job to run
In Build field click " Add build step" then select "seleniumhq htmlSuite Run"
browser: *firefox or *iexploer ....
startURL: http://www.google.com or ...
suiteFile: Input absolutely path to TestSuite.html file saved in step 2
resultFile: Input absolutely path to a file that results will be saved
Hope this help!
I did the same but the following error occurred:
Unable to find the HTML runner. This is normally because you have not downloaded
or made available the 'selenium-leg-rc' jar on the CLASSPATH. Your test will
not be run.
Download the Selenium HTML Runner from http://www.seleniumhq.org/download/ and
use that in place of the selenium-server-standalone.jar for the simplest way of
running your HTML suite.
We have a Play (1.2.4) application which is auto-tested via our Teamcity server. The regular (JUnit) tests return a Test-{classname}.xml datafile in the /test-result/ directory when using
play.bat auto-test
but the selenium tests do not. They only generate a /test-result/{testname}.test.html.failed.html or ...succes.html
This is annoying because i can not easily show which selenium test has failed from within Teamcity. I have to look at the logfiles/artifacts and see which failed.html file is present, open it and look at it which testcase has failed. Annoying!
How can i have play 1.2.4 generate a datafile/xml something which i can parse to see which selenium test has failed?
Since i could find no information what so ever and no clue's how to get it done i wrote my own parsers of the Selenium HTML testresult files. I have released it on github with sourcecode under GPL: https://github.com/crunchie84/play-seleniumresults-parser
I want to run my Selenium HTML Test Suite through Jenkins (a continuous integration). The following shows, how the build is configured for the current project:
And here's the console output after commiting a new test for example:
ERROR: The suiteFile is not a file or an url ! Check your build configuration.
Build step 'SeleniumHQ htmlSuite Run' changed build result to FAILURE
Build step 'SeleniumHQ htmlSuite Run' marked build as failure
Publishing Selenium report...
Finished: FAILURE
In fact, I get these log issues even after committing both extensionless test files AND .html files.
SeleniumHQ Jenkins plugin supports only ONE suite file per build step. Try out Selunit to run Selenese suites in batch and across multiple browsers. This tutorial shows hot to setup the test execution in Jenkins/Hudson.
Your suiteFile is written with wildcard as: tests/selenium/*.html. I think it is wrong.
You need to provide the exact/absolute path to your suite without the wildcard as below:
tests/selenium/suite.html