For Test Automation of web project we use Hudson, PHPUnit, and Selenium. The results of the build are stored in the JUnit XML format.
When I try to include generation of reports using the Hudson Publish JUnit test result report option, the build finishes with Failed status.
Below is my Hudson configuration to run the tests.
sudo -u apache phpunit - log-junit /var/lib/hudson/jobs/Work-stars-Tests/builds/${BUILD_ID}/ seleniumReports/seleniumTests.xml + path to test php files
Generation of reports is enabled through the Hudson configuration option «Publish JUnit test result report», where I specify the path to the folder with PHP tests.
The user we use to run Hudson has permission to write/read files in the folder with the reports. As for the path we've tried to specify both full and relative.
The error No test report files were found. Configuration error? is displayed in the console after the build.
How do we resolve this issue?
Found my own solution to this problem :)
In Hudson project configuration settings modified execute shell command to
> #!/bin/sh -x phpunit --log-junit ${WORKSPACE}/zf/tests/_tmp/reports/seleniumTests.xml
> ${WORKSPACE}/zf/tests/selenium/; sed
> -i '/<testsuite name=".*\/"/D;/^ <\/testsuite>$/D'
> ${WORKSPACE}/zf/tests/_tmp/reports/seleniumTests.xml
and set following path to Junit reports
**/zf/tests/_tmp/reports/*.xml
Problem is solved. Yahoo!
Related
I have a .side file generated by the Selenium IDE, which I need to run on CI using Jenkins.
I am running it as a build step with the following shell command:
selenium-side-runner /path/to/file.ide
The problem arises due to the fact that no matter if the selenium test fails, Jenkins always shows is as success.
In this thread it's suggested to upload the file as generic, but still, the commands to execute it are missing
How to upload a generic file into a Jenkins job?
I've found a possible solution to it on this posts, but I would appreciate having a cleaner way to solve this instead of parsing the results checking for errors.
How to mark a build unstable in Jenkins when running shell scripts
Is there a plugin able to run selenium .side files on Jenkins and this one showing the success/failures of the test?
You can generate a Junit test report file and then use the Jenkins Junit plugin after your tests execution.
selenium-side-runner --output-directory=results --output-format=junit
# Outputs results in `junit` frormat in `./results/projectName.xml'
Check the official documentation for more details.
I'm successfully generating 2 .exec files by Jacoco within "build/jacoco" folder after running a Gradle based build and integration tests.
Gradle command:
"gradle clean build integrationTest"
Once done, it generates the following .exec files under build/jacoco folder.
test.exec
integrationTest.exec
Following is my sonar-project.properties file. When, I run "sonar-runner" from Linux prompt it completes but on SonarQube dashboard for this project, I see Unit test says some 34.5% but integration tests says 0.0%. Both .exec files have valid size. I also did "cat" on the .exec files and piped the output to "strings" command in Linux and saw that integrationTest.exec did hit the Tests functions - I have only 1 .java file.
When I run "gradle clean build integrationTest sonarRunner -Dxxx.xxx=yyy -Dyyy.xx=zzz" i.e. by passing all the sonar variable as mentioned in the sonar-project.properties file using -D option, it works but same result on SonarQube project's dashboard. Project's sonar dashboard has both widgets configured for Unit / Integration Tests and I'm including IT tests for showing Overall coverage. Overall coverage is showing 34.5% (which is Unit test % value). Sonar does see test.exec, integrationTest.exec and also auto generates overall-xxx.exec file as well during this operation.
NOTE: I'm no where - while starting tomcat on a separate putty / linux console -OR within Gradle build script, providing any value or setting JAVA Agent for Jacoco. I'm getting integrationTest.exec file and test.exec file already so not sure if JVM needs to be stopped once IT tests are complete running. I don't think I need those as i have valid file size for .exec files.
My ?:
- Why sonar is not getting IT coverage on the dashboard even though I'm setting / passing the following variable correctly:
sonar.jacoco.itReportPath=build/jacoco/integrationTest.exec
-bash-3.2$ cat sonar-project.properties
# Root project information
sonar.projectKey=com:company:product:ProjectA
sonar.projectName=ProjectA
sonar.projectVersion=1.0
# optional description
sonar.projectDescription=ProjectA Service
#Tells SonarQube that the code coverage tool by unit tests is JaCoCo
sonar.java.coveragePlugin=jacoco
#Tells SonarQube to reuse existing reports for unit tests execution and coverage reports
sonar.dynamicAnalysis=reuseReports
# Some properties that will be inherited by the modules
sonar.sources=src/java,test/java,src/java-test
# Sonar Unit Test Report path
sonar.jacoco.reportPath=build/jacoco/test.exec
# Sonar Integration Test Report Path
sonar.jacoco.itReportPath=build/jacoco/integrationTest.exec
sonar.junit.reportsPath=build/UT/results
# Sonar Binaries
sonar.binaries=build/classes/main
Narrowing down the cause: I think it's due to the .exec file for Integration test. To proove it: I passed UT exex file to both reportsPaths in Sonar variables i.e. the following and SonarQube picked both UT/IT test coverage. This prooves that if .exec file for IT tests is good (which I think it's But I need to double check) then Sonar will pick the .exec file and show a valid coverage % instead of 0.0%. Note: the following is just to proove if Sonar is picking the values or not. itReportPath variable should use the .exe file which is generated by Integration tests by Jacoco.
sonar.jacoco.reportPath=build/jacoco/test.exec
# Sonar Integration Test Report Path
#sonar.jacoco.itReportPath=build/jacoco/testintegrationTest.exec
sonar.jacoco.itReportPath=build/jacoco/test.exec
OK Found the issue. I was running integrationTest task in Gradle and was NOT attaching the jacocoagent.jar (as per Jacoco documentation) to the target JVM (Tomcat's instance) scope. Once I did that, I removed jacoco { ... } section from integrationTest task in Gradle (build.gradle or GRADLE_HOME/init.d/some.common.gradle file as this attach jacoco agent to the Java JVM in which Gradle is running). Now, once jacocoagent.jar was attached to Tomcat's JVM (as per the line below which I added in Tomcat's startup.sh script and added the variable to the command which starts Tomcat), then I ran Gradle (integrationTest) task for running IT tests.
PROJ_EXTRA_JVM_OPTS=-javaagent:tomcat/jacocoagent.jar=destfile=build/jacoco/IT/jacocoIT.exec,append=false
Then while Gradle was in progress, tests ran and I got a file (jacocoIT.exec at the given location) with some file size BUT this is not yet the final one. I had to stop the Tomcat session/JVM instance by running Tomcat's stop.sh script. Once Tomcat was stopped, I saw jacocoIT.exec file size increased significantly and this was the valid final jacocoIT.exec file (which I needed for sonarRunner Gradle task OR sonar-runner exectuable to pick and successfully push IT code coverage data to project's sonar dashboard). Once done, I got both UT + IT and it's combined code coverage.
sonar.jacoco.reportPath=build/jacoco/UT/jacocoUT.exec
sonar.jacoco.itReportPath=build/jacoco/IT/jacocoIT.exec
I have recorded some simple selenium tests by Selenium IDE. Now I want to run those tests in Jenkins.
Which plugin to Jenkins do I need to do that? And how to run the tests step by step? Help is appreciated.
you can use recorded selenium IDE script and selenium-server.jar file to run it from Jenkins
Here is steps:
Go to SeleniumHQ page and download Selenium Server file
Eg: selenium-server-standalone-2.33.0.jar
Repair html test suite Use Selenium IDE to record then save as
html test case and test suite then put them in a same folder
eg: TestCase.html, TestSuite.html
In jenkins
Plugin Seleniumhq
Configure Selenium runner file
Manage Jenkins > Configure System > Selenium Remote Control: htmlSuite Runner = path to file u have download in step 1
Configure Job to run
In Build field click " Add build step" then select "seleniumhq htmlSuite Run"
browser: *firefox or *iexploer ....
startURL: http://www.google.com or ...
suiteFile: Input absolutely path to TestSuite.html file saved in step 2
resultFile: Input absolutely path to a file that results will be saved
Hope this help!
I did the same but the following error occurred:
Unable to find the HTML runner. This is normally because you have not downloaded
or made available the 'selenium-leg-rc' jar on the CLASSPATH. Your test will
not be run.
Download the Selenium HTML Runner from http://www.seleniumhq.org/download/ and
use that in place of the selenium-server-standalone.jar for the simplest way of
running your HTML suite.
I am using Jenkins and Ant to run my selenium tests. It works fine when I run them on my local machine.
Now I have a Jenkins server which is on a different location and I have to run my tests on that server.
when I tried to run the tests on that server Its is failing because it couldn't find the build.xml file. can you please let me know how can I change the home directory path?
Started by user :**********
Building in workspace /var/lib/jenkins/jobs/moveon4 java tests/workspace
FATAL: Unable to find build script at C:\selenium tests for move\movetests1\build.xml
Build step 'Invoke Ant' marked build as failure
Finished: FAILURE
Are you using "Invoke Ant" in your Jenkins job configuration for the build step that executes your Ant build ? If so, the build file location is hidden away in the Advanced section - it's an annoying quirk of Jenkins.
I want to run my Selenium HTML Test Suite through Jenkins (a continuous integration). The following shows, how the build is configured for the current project:
And here's the console output after commiting a new test for example:
ERROR: The suiteFile is not a file or an url ! Check your build configuration.
Build step 'SeleniumHQ htmlSuite Run' changed build result to FAILURE
Build step 'SeleniumHQ htmlSuite Run' marked build as failure
Publishing Selenium report...
Finished: FAILURE
In fact, I get these log issues even after committing both extensionless test files AND .html files.
SeleniumHQ Jenkins plugin supports only ONE suite file per build step. Try out Selunit to run Selenese suites in batch and across multiple browsers. This tutorial shows hot to setup the test execution in Jenkins/Hudson.
Your suiteFile is written with wildcard as: tests/selenium/*.html. I think it is wrong.
You need to provide the exact/absolute path to your suite without the wildcard as below:
tests/selenium/suite.html