As I am implementing an automated way to GUI test our webapplication with selenium I ran into some issues.
I am using selenese-runner to execute our Selenium test suites, created with Selenium IDE as a post build action in Jenkins.
This works perfeclty fine, as the build fails when something is wrong, and the build succeeds if all tests are passed. And the results are stored on a per build basis as HTML files, generated be selenese-runner.
My problem is however, that I seem to be unable to find a way, how to display these results in the respective jenkins build.
Does anyone have an idea how to solve this issue. Or maybe I am on the wrong path at all?
Your help is highly appreciated!
I believe the JUnit plugin should do what you want, but it doesn't work for me.
My config uses this shell script to run the tests (you can see the names of all my test suites):
/usr/bin/Xvfb &
export DISPLAY=localhost:0.0
cd ${WORKSPACE}
java -jar ./test/selenium/bin/selenese-runner.jar --baseurl http://${testenvironment} --screenshot-on-fail ./seleniumResults/ --html-result ./seleniumResults/ ./test/selenium/Search_TestSuite.html ./test/selenium/Admin_RegisteredUser_Suite.html ./test/selenium/Admin_InternalUser_Suite.html ./test/selenium/PortfolioAgency_Suite.html ./test/selenium/FOAdmin_Suite.html ./test/selenium/PublicWebsite_Suite.html ./test/selenium/SystemAdmin_Content_Suite.html ./test/selenium/SystemAdmin_MetaData_Suite.html
killall Xvfb
And I can see the result of the most recent test (you can see the name of my jenkins task folder)
http://<JENKINS.MY.COMPANY>/job/seleniumRegressionTest/ws/seleniumResults/index.html
Earlier tests are all saved on the Jenkins server, so I can view them if I need to.
Related
I'm trying to run some TestCafe tests from our build server, but getting the following error...
"Could not find test files at the following location: "C:\Testing\TestCafe".
Check patterns for errors:
tests/my-test.ts
or launch TestCafe from a different directory."
I did have them running or able to be found on this machine previously, but others have taken over the test coding and changed the structure a bit when moving it to a Git repository. Now when I grab the tests from Git and try to run, the problem presents itself. I'm not sure if there is something in a config file that needs adjustment but don't know where to start looking.
The intention is to have it part of our CI process, but the problem is also seen when I attempt to run the tests from the command line. The build process does install TestCafe, but there is something strange around this as well.
When the build failes with the can't find tests error, if I try to run the following command in the proper location...
tescafe chrome tests/my-test.ts
... I get, 'testcafe' is not recognized as an internal or external command,
operable program or batch file.
Just can't understand why I can't get these tests running. TestCafe setup was pretty much easy previously.
ADDENDUM: I've added a screenshot of the working directory where I cd to and run the testcafe command as well as the tests subdirectory containing the test I'm trying to run.
Any help is appreciated!!
testcafe chrome tests/my-test.ts is just a template; it isn't a real path to your tests. This error means that the path that you set in CLI is wrong, and there aren't any tests. You need to:
Find out where you start CLI. Please attach a screenshot to your question.
Define an absolute path to tests or a path relative to the place where CLI was started. Please share a screenshot of your project tree where the directory with tests is open.
Also, you missed t in the tescafe chrome tests/my-test.ts command. It should be tesTcafe chrome tests/my-test.ts. That is why you get the "'tescafe' is not recognized as an internal or external command" error.
I was able to get things working by starting from scratch. I uninstalled TestCafe and cleaned the working folder. During next build it was fine. I'm sure I've tried this several times, but it just started working.
One positive that came out of it was that I discovered a typo in a test file name, which was also causing issues finding the test I was using to check testing setup.
Thanks for helping!!
I have a .side file generated by the Selenium IDE, which I need to run on CI using Jenkins.
I am running it as a build step with the following shell command:
selenium-side-runner /path/to/file.ide
The problem arises due to the fact that no matter if the selenium test fails, Jenkins always shows is as success.
In this thread it's suggested to upload the file as generic, but still, the commands to execute it are missing
How to upload a generic file into a Jenkins job?
I've found a possible solution to it on this posts, but I would appreciate having a cleaner way to solve this instead of parsing the results checking for errors.
How to mark a build unstable in Jenkins when running shell scripts
Is there a plugin able to run selenium .side files on Jenkins and this one showing the success/failures of the test?
You can generate a Junit test report file and then use the Jenkins Junit plugin after your tests execution.
selenium-side-runner --output-directory=results --output-format=junit
# Outputs results in `junit` frormat in `./results/projectName.xml'
Check the official documentation for more details.
I am trying to setup a test project that uses serenity and jbehave
I am noticing that all examples use serenity.properties that define a browser in it
I would like to structure my tests in a way so that same test can be executed in IE/firefox/chrome etc
How do I do this?
You can pass in properties as command line properties, so you can rerun the same tests with different browsers by passing in different settings for webdriver.driver, e.g.
$ mvn verify -Dwebdriver.driver=firefox
$ mvn verify -Dwebdriver.driver=chrome
etc.
I think you are able to get this to work by creating multiple Junit test classes with each its own driver and execute them all in a single run.
Every test class should be able to assign a specific 'managed' driver (e.g. PhantomJS, Chrome, Firefox). This is documented here: http://www.thucydides.info/docs/serenity/#_serenity_webdriver_support_in_junit
I don't know what the impact this would have on the generated report, hopefully you are still able to identify the feature/driver combination.
we are considering to use FitNesse/Slim.
But is there a way to start all written Tests without browsing the webpage and starting each manually ?
It would be sufficient if there is a one-start-all kind of button somewhere to click.
So either starting all tests from command line (with a report of course) or with on button from the webpage.
Is this doable ?
Thanks in advance
There is a very easy way to do that. You can run FitNesse tests from the command line. You do this by using the following command line:
java -jar lib/fitnesse.jar -c "FrontPage?suite&format=text"
This will run all tests under the FrontPage and show the results as they happen in a command line friendly format. If you change FrontPage to FrontPage.MainSuite, it will run only the tests under that page.
If you have tests that are in different states. Maybe some of them are started but are not done yet. You can add a Suite Tag to the tests that must run, then you can filter the tests that are run. that would look like the following:
java -jar lib/fitnesse.jar -c "FrontPage?suite&suiteFilter=MustBeGreen&format=text"
This is also possible to do using an ANT java task.
Assuming you have a current version of fitnesse and have it running on port 8080, the following link will take you to a page with more details: http://localhost:8080/FitNesse.UserGuide.ControllingFitnesseFromTheCommandLine
Dan has it correct; but the User Guide is also posted online at:
http://fitnesse.org/FitNesse.UserGuide.ControllingFitNesseFromTheCommandLine
So i have been writing Selenium tests for a while now and i have been running the only from the IDE untill now.
i would like to be able to write a script that runs all my tests sequentialy.
I am using the testng framework with eclipse and selenium-2.0b3 jar.
What i'd like is eventually to have a file like "runSeleniumTests.bat" witch just runs them and gives me some sort of report when in finishes.
if anyone has an idea, it will be very greatly appreciated, thanks :)
This has 4 options for how to run testng tests the xml file should include all your tests
http://testng.org/doc/documentation-main.html#running-testng
You could also try to use maven and as long as your tests are annotated correctly and in /src/test/java they will all be run for you with mvn integration-test (as long as you have the proper dependencies defined and I believe maven-surefire-plugin). This might be more difficult to setup initially but usually pays off.