None of the plugins in the reporting section of Maven execute if there is a unit test failure.
I found that I can set maven.test.failure.ignore=true here - http://jira.codehaus.org/browse/SUREFIRE-247 The problem to this approach is now our hudson builds are successful even if there are unit test failures.
What I would really like to do is set the reporting plugin maven-surefire-report-plugin to run with the build plugins on a phase but I can't get this to work.
Any idea on how to get the Maven reporting plugins to execute if a unit test failure occurs?
Firstly run: mvn test OR mvn install. Then, if the tests fail, please run the following target to generate the reports for the test results executed above: mvn -Dmaven.test.skip=true surefire-report:report
In the link you posted:
With the latest version (2.1.2), I get
a message saying that "There are some
test failure," but I get no reports
anywhere whether or not I specify that
variable, or whether or not I specify
"testFailureIgnore" in the plugin
config. I got the reports fine with
2.0, but not with 2.1.2.
Do you need version 2,1 or can you work with a 2.0 version of Maven?
The error you see with 2.1.2 is because of forkmode settings which you need to perform in the plugin.
set forkmode=never and try it (I susppect there might problem in your useSystemclassloader property).
Otherwise make use of maven-surefire plugin version 2.5 which should definitel work and generate surefire rpeorts even though few test fails.
Please make use of surefire-report:report-only plugin if the reports are already generated after execution.
I had the same issue and it is due to a wrong call to the report plugin.
The correct execution of maven command is: mvn surefire-report:report
This will run the test phase by itself and if it fails it will generate the report anyway.
Check the documentation:
http://maven.apache.org/surefire/maven-surefire-report-plugin/report-mojo.html
Hope this helps!! :D
Related
I am using VSTest to run tests via Azure DevOps. Tests run successfully, but using the option rerun failed tests: true, an error appears during the restart phase.
NUnit 3.12.0 ;
NUnit Adapter 3.16.1.0 (Checked with 4.0.0.0)
vstest.console.exe "C:\agent2.172.2\_work\r1\a\UITest\drop\Tests.Web\bin\Release\netcoreapp3.1\Tests.Web.dll"
/Settings:"C:\agent2.172.2\_work\_temp\3utv233tymm.runsettings"
/Logger:"trx"
/TestAdapterPath:"C:\agent2.172.2\_work\r1\a\UITest\drop\Tests.Web\bin\Release\netcoreapp3.1"
/TestCaseFilter:"FullyQualifiedName=Tests.Web.Tests._5.CourierModuleTest.N1_SendingTest.Id_5_1_01_TransferToCourierModule(Chrome)|FullyQualifiedName=Tests.Web.Tests._3.IssuanceOfDocuments.InformationOnTheApplicationIdentificationOfRecipient.N2_RecipientIdentificationTest.Id_3_2_13_RegisterAddressByFiasTest(Chrome)"
NUnit Adapter 3.16.1.0: Test execution started
An exception occurred while invoking executor 'executor://nunit3testexecutor/': Incorrect format for TestCaseFilter Missing Operator '|' or '&'. Specify the correct format and try again. Note that the incorrect format can lead to no test getting executed.
How can I fix this error and successfully restart the tests in DevOps?
I tried googling for a similar issue but didn't find anything that would work. Any help is really appreciated
Here's similar discussion, according to comments from the Contributors of azure-pipelines-task repo:
1.You should update your VS and VSTest component within it to latest version since we don't support data driven tests for Rerun scenario. It will be available with VS 15.8 release and higher.
2.You should configure your VSTest task following this solution:
Recommended to give total number of your tests as input (Number of tests per batch).
I have a Bamboo job for mulesoft code, with steps as code checkout, build, test which generates coverage report, deploy.
Test step is failing with message 'Failing task since test cases were expected but none were found.' and coverage report is generated in specific path.
When I remove the specific path, job is success but coverage is not generated.
I tried enabling 'maven return code', which makes the job success, but can't see coverage report.
Give the path **/target/*, enable maven return code - job success, coverage not generated
Give the path **/target/*, disable maven return code - job failed, coverage generated
I found the fix, I had to upgrade munit plugin to higher version ! This fixed the issue !
This should be very late answer but I got same issue with Java + Maven + TestNG + Bamboo and resolved with the respective bamboo task
The below option should be unchecked
The build will produce test results - Uncheck
Maven task configuration
As I am implementing an automated way to GUI test our webapplication with selenium I ran into some issues.
I am using selenese-runner to execute our Selenium test suites, created with Selenium IDE as a post build action in Jenkins.
This works perfeclty fine, as the build fails when something is wrong, and the build succeeds if all tests are passed. And the results are stored on a per build basis as HTML files, generated be selenese-runner.
My problem is however, that I seem to be unable to find a way, how to display these results in the respective jenkins build.
Does anyone have an idea how to solve this issue. Or maybe I am on the wrong path at all?
Your help is highly appreciated!
I believe the JUnit plugin should do what you want, but it doesn't work for me.
My config uses this shell script to run the tests (you can see the names of all my test suites):
/usr/bin/Xvfb &
export DISPLAY=localhost:0.0
cd ${WORKSPACE}
java -jar ./test/selenium/bin/selenese-runner.jar --baseurl http://${testenvironment} --screenshot-on-fail ./seleniumResults/ --html-result ./seleniumResults/ ./test/selenium/Search_TestSuite.html ./test/selenium/Admin_RegisteredUser_Suite.html ./test/selenium/Admin_InternalUser_Suite.html ./test/selenium/PortfolioAgency_Suite.html ./test/selenium/FOAdmin_Suite.html ./test/selenium/PublicWebsite_Suite.html ./test/selenium/SystemAdmin_Content_Suite.html ./test/selenium/SystemAdmin_MetaData_Suite.html
killall Xvfb
And I can see the result of the most recent test (you can see the name of my jenkins task folder)
http://<JENKINS.MY.COMPANY>/job/seleniumRegressionTest/ws/seleniumResults/index.html
Earlier tests are all saved on the Jenkins server, so I can view them if I need to.
I tried running my code analysis against different sonnar runners:
with ANT I tried:
sonar-ant-task-2.2.jar
sonar-ant-task-2.0.jar
with maven I tried:
sonarqube-eclipse plugin
sonar:sonar maven task
Why are they giving me different violations count even though they all use the same quality profile?
Wild guess here given the few information available (please share the configuration of your analysis) but it can be due to the fact that bytecode is not provided when running with ant whereas this is done automatically when running with maven.
I am fairly new to Griffon and have some experience with Grails.
I have a problem loading a file from the resources directory.
I am using Griffon version 1.4.0.
When I run griffon run-app the following code (inside a Service) to get the location of an XML file works fine:
URL resource = getResourceAsURL('schema.xsd')
assert resource != null : "schema cannot be located"
When I run griffon test-app however the same code produces an assertion error because the returned URL is null. Same behaviour with getResourceAsStream().
This happens in the unit test of said service.
I put the file in ./griffon-app/resources.
What am I doing wrong? Do I have to copy all resources from production to some test resources folder, do I have to edit the build-configuration?
Thanks in advance!
Edit as suggested below I filed a bug report in the griffon-projects issue tracker.
araxn1d is correct, running the tests in integration mode will give you the right answer because the full application gets bootstrapped before tests are run. Now, running this kind of test (a unit test that depends on resources being available in the classpath) encounters a problem because the classpath is not setup correctly. Executing the following command
griffon -Dgriffon.cli.verbose=true test-app --unit --compileTrace=true
will output all classpaths. There you can see that the resources directory points to $USER_HOME/.griffon/1.4.0/projects/<project_name>/resources. If you inspect that directory you'll find the file you're looking for inside griffon-app/resources. This means the test classpath is not accurately configured as it should be $USER_HOME/.griffon/1.4.0/projects/<project_name>/resources/griffon-app/resources instead. This is clearly a bug, most likely found in the $GRIFFON_HOME/scripts/_GriffonClasspath.groovy script. Could you please file a JIRA http://jira.codehaus.org/browse/griffon ticket for it? Thanks!
You should run test-app to run your unit tests. In this case you should mock any refers to real files, otherwise you should implement Integration Tests. Pls see Griffon Testing. Integration tests differ from unit tests in that you have full access to the Griffon application within the test.