I am trying to automate running some tests (in my own test infrastructure) with TeamCity.
We are not using java or ant but I can generate junit test results from a perl script:
run_tests.pl
Which generates junit result files (junit1.xml, junit2.xml etc.).
I have integrated this into Jenkins before by running it as a command line and giving it the path to the junit files.
In TeamCity it appears that due to the closer integration with ant and junit I need to run run_tests.pl from a ant junit task.
So how can I run a command from ant to do this? Or is there an easier way to achieve what I want (junit or other test results reported in teamcity)?
Okay, found a good way to do this. The key thing you need is XML Report processing so you don't need ant or junit tasks http://confluence.jetbrains.net/display/TCD7/XML+Report+Processing.
Summary:
Create your junit1.xml files using a command line build step
Add a build feature to your build configurations of type XML report
processing
Set report type to be ANT JUnit
Add a monitoring rule of +:junit*.xml
Related
I am using jenkins pipeline, does junit command once executed during build, stores the test results even after deleting the junit reports directory? If yes, where does Jenkins reads the deleted junit reports?
Jenkins does archive the JUnit test reports. So after your pipeline finishes (e.g. on an ephemeral agent), you'll still be able to access those reports. The same is true if you remove the report folder after you captured them with the plugin.
You can tell the JUnit plugin where to find your .xml Surefire/Failsafe reports. In a declarative pipeline, this can look like the following:
post {
always {
junit '**/target/surefire-reports/*.xml'
}
}
The source code of this plugin is available on GitHub, so you can take a look at how it's achieved under the hood.
Karate suggests that to run all tests in a CI environment, a *Test.java file should be added above the feature files (in hierarchy) and then run using - mvn test command.
I am using my Runner.java file to create test data before the tests are run and then do a clean up. I run this runner file in IDE and everything runs fine - the data is created, all feature files in the same package run and then clean-up is performed. The reason i used the Runner file to create data is because i using karate itself to create test data and the Runner file passes some information on created data to the feature files to run api tests. I had earlier posted a question regarding how to achieve this, please refer to this answer - https://stackoverflow.com/a/55931786/4741035
So now I have a *Test.java file in my project which i run using - mvn test. This runs all the feature files and tests fails as the Runner.java is not executed at all.
Why doesn't karate run the Runner file, if it is present first that the feature files?
Help is much appreciated.
If you are trying to run something "once" before all your tests, use karate.callSingle() documented here: https://github.com/intuit/karate#hooks
var result = karate.callSingle('classpath:demo/headers/common-noheaders.feature', config);
And in the above feature (or JS) you can call Java code using Java interop.
By the way I don't agree with the answer you linked because of the above approaches.
Is it possible to target specific runner classes or feature files within Jenkins?
Let's say for example I have the following files.
Runner classes:
RunnerClass1.java
RunnerClass1.java
Feature Files:
Login.feature
SignUp.feature
Is there a way to trigger specific runner classes or feature files within the Jenkins UI, I know you can use specific plugins such as: 'Parameterised / String Parameters', has anyone else found a solution to target specific tests from Jenkins?
thanks for your help
You can tag your test cases and maven will be able to run them by these tags.
For example, When I have Login cases with #Login tags and I want to run them with Maven, I am using the following terminal script :
mvn clean test -Dcucumber.options="--tags #Login"
How to create XSLT reports in Selenium project bases on Java and Ant.
I have already added that to my project but the result is not as per my expectation. All Testcases are passing but some are showing failed in reports
XSLT reports can be generated based on the results of your TestNG.XML file. if you are using TestNG as unit testing framework make sure to place -output directory of the TestNG should be available in home directory.
Read this (http://learn-automation.com/generate-xslt-report-in-selenium/) post you will get better Idea.
Upgraded to IntelliJ 14.0.1 One of the big new features I was looking for:
"If you run tests via a Gradle task, the IDE offers you the standard Test Runner instead of the console output." (Source: https://www.jetbrains.com/idea/whatsnew/#buildTools)
I right click on the Gradle Task to run our Integration Tests:
However, I see the results of the test still going to console output, not to the Test Runner:
Has anyone been able to get this new feature in IntelliJ IDEA 14 to work?
Thank you in advance,
Philip
Looks like IntelliJ looks for a task named "test" rather than a task of type Test.
https://github.com/JetBrains/intellij-community/blob/master/plugins/gradle/src/org/jetbrains/plugins/gradle/execution/test/runner/GradleTestsExecutionConsoleManager.java#L191
Rename the test task to unitTest and then create a wrapper that runs both:
// Rename test to unitTest
tasks.test.name = "unitTest"
// Wrap and run both
task test(dependsOn:['unitTest', 'integrationTest'])
If you only want to run integration tests, just overwrite it:
task test(overwrite: true, dependsOn: ['integrationTest'])
This allows me to run the integration tests in the test runner successfully (at least it works in IDEA 15 EAP but it should work in 14 as well I would think).
I still get this in IntelliJ 2017.1, but specifically when running tests in the gradle buildSrc directory. After digging a while, it turns out that the Gradle API doesn’t expose the test tasks in the special buildSrc directory to Intellij, so IntelliJ doesn’t recognize it as a test.
Workaround: Open another IntellIJ project for the buildSrc directory directory instead of trying to run tests cleanly inside the root project.