I'm trying to get tests results while the job is building.
When we run tests suite by eclipse we get tests results from TestNG viewer while running the suite, I want to get the same viewer or similar in Jenkins to know the current status of the build before finish.
I mean this in TestNG Viewer:
Results of running suite TestNG viewer
Thanks All :)
AFAIK it is not inbuilt as part of any plugin. But there are couple of options that you can try.
Write results to a database in the IInvokedMethodListener after implementation. Build a ui over the database.
Maintain a datastructure of results , do console out of summary(if you are the only one who needs to know the results) on jenkins in test listener or method listener of the results based on the frequency at which you need to know results. Or you can start of a parallel script which parses the consoleText either as a shell or a separate utility doing curl on the consoletext.
Related
As I am implementing an automated way to GUI test our webapplication with selenium I ran into some issues.
I am using selenese-runner to execute our Selenium test suites, created with Selenium IDE as a post build action in Jenkins.
This works perfeclty fine, as the build fails when something is wrong, and the build succeeds if all tests are passed. And the results are stored on a per build basis as HTML files, generated be selenese-runner.
My problem is however, that I seem to be unable to find a way, how to display these results in the respective jenkins build.
Does anyone have an idea how to solve this issue. Or maybe I am on the wrong path at all?
Your help is highly appreciated!
I believe the JUnit plugin should do what you want, but it doesn't work for me.
My config uses this shell script to run the tests (you can see the names of all my test suites):
/usr/bin/Xvfb &
export DISPLAY=localhost:0.0
cd ${WORKSPACE}
java -jar ./test/selenium/bin/selenese-runner.jar --baseurl http://${testenvironment} --screenshot-on-fail ./seleniumResults/ --html-result ./seleniumResults/ ./test/selenium/Search_TestSuite.html ./test/selenium/Admin_RegisteredUser_Suite.html ./test/selenium/Admin_InternalUser_Suite.html ./test/selenium/PortfolioAgency_Suite.html ./test/selenium/FOAdmin_Suite.html ./test/selenium/PublicWebsite_Suite.html ./test/selenium/SystemAdmin_Content_Suite.html ./test/selenium/SystemAdmin_MetaData_Suite.html
killall Xvfb
And I can see the result of the most recent test (you can see the name of my jenkins task folder)
http://<JENKINS.MY.COMPANY>/job/seleniumRegressionTest/ws/seleniumResults/index.html
Earlier tests are all saved on the Jenkins server, so I can view them if I need to.
I am trying to setup a test project that uses serenity and jbehave
I am noticing that all examples use serenity.properties that define a browser in it
I would like to structure my tests in a way so that same test can be executed in IE/firefox/chrome etc
How do I do this?
You can pass in properties as command line properties, so you can rerun the same tests with different browsers by passing in different settings for webdriver.driver, e.g.
$ mvn verify -Dwebdriver.driver=firefox
$ mvn verify -Dwebdriver.driver=chrome
etc.
I think you are able to get this to work by creating multiple Junit test classes with each its own driver and execute them all in a single run.
Every test class should be able to assign a specific 'managed' driver (e.g. PhantomJS, Chrome, Firefox). This is documented here: http://www.thucydides.info/docs/serenity/#_serenity_webdriver_support_in_junit
I don't know what the impact this would have on the generated report, hopefully you are still able to identify the feature/driver combination.
I have Groovy Script in my Test Case in SOAP UI.
There are one big script and a lot of test steps which are run by this script. I know about functionality of TestRunner but it run all steps in test case, but i need to run only my groovy script. How?
UPDATE
I disable all Test steps and left active only groovy script. Then I Launch TestRunner on Test Case, it return exception :
java.lang.Exception: TestCase [Case1] failed without assertions
at com.eviware.soapui.tools.SoapUITestCaseRunner.throwFailureException(SoapUITestCaseRunner.java:535)
at com.eviware.soapui.tools.SoapUITestCaseRunner.runRunner(SoapUITestCaseRunner.java:437)
at com.eviware.soapui.tools.AbstractSoapUIRunner.run(AbstractSoapUIRunner.java:162)
at com.eviware.soapui.tools.AbstractSoapUIRunner.runFromCommandLine(AbstractSoapUIRunner.java:93)
at com.eviware.soapui.tools.SoapUITestCaseRunner.main(SoapUITestCaseRunner.java:119)
Just disable the rest of the steps and run the test case. The disabled test steps are not run automatically by soapui, you can call those conditionally :)
we are considering to use FitNesse/Slim.
But is there a way to start all written Tests without browsing the webpage and starting each manually ?
It would be sufficient if there is a one-start-all kind of button somewhere to click.
So either starting all tests from command line (with a report of course) or with on button from the webpage.
Is this doable ?
Thanks in advance
There is a very easy way to do that. You can run FitNesse tests from the command line. You do this by using the following command line:
java -jar lib/fitnesse.jar -c "FrontPage?suite&format=text"
This will run all tests under the FrontPage and show the results as they happen in a command line friendly format. If you change FrontPage to FrontPage.MainSuite, it will run only the tests under that page.
If you have tests that are in different states. Maybe some of them are started but are not done yet. You can add a Suite Tag to the tests that must run, then you can filter the tests that are run. that would look like the following:
java -jar lib/fitnesse.jar -c "FrontPage?suite&suiteFilter=MustBeGreen&format=text"
This is also possible to do using an ANT java task.
Assuming you have a current version of fitnesse and have it running on port 8080, the following link will take you to a page with more details: http://localhost:8080/FitNesse.UserGuide.ControllingFitnesseFromTheCommandLine
Dan has it correct; but the User Guide is also posted online at:
http://fitnesse.org/FitNesse.UserGuide.ControllingFitNesseFromTheCommandLine
So i have been writing Selenium tests for a while now and i have been running the only from the IDE untill now.
i would like to be able to write a script that runs all my tests sequentialy.
I am using the testng framework with eclipse and selenium-2.0b3 jar.
What i'd like is eventually to have a file like "runSeleniumTests.bat" witch just runs them and gives me some sort of report when in finishes.
if anyone has an idea, it will be very greatly appreciated, thanks :)
This has 4 options for how to run testng tests the xml file should include all your tests
http://testng.org/doc/documentation-main.html#running-testng
You could also try to use maven and as long as your tests are annotated correctly and in /src/test/java they will all be run for you with mvn integration-test (as long as you have the proper dependencies defined and I believe maven-surefire-plugin). This might be more difficult to setup initially but usually pays off.