Currently I can hard code the header.xsl in cruisecontrol to show anything I want (e.g. browser type), but I'd like to also see the version of my app testng is running against. The version number is situated on the first page of my app under a div class. Is there a way to have cruisecontrol set the value into the header.xsl from grabbing the testng report?
Expected result:
I want to see on the cruisecontrol report info about my app like version #
Thanks.
Is that version number ever output during the build process? Can you find it in the build log? If so it should be rather easy to figure out the xsl necessary to get it. If it isn't you would have to do a lot of customizing because they system expects to only display info inside the build log, statistics.csv, and report.xml files.
Related
We have QA and DEV environment in our automation repo. We are using karate as our framework. We have TestParallel class and integrated allure report.
How could we run all tests in QA first then in DEV back to back using TestParallel Class and see the results in the same report?
Thanks for such a great tool btw.
We are going to try and make this easier in the next version.
For now, you have to aggregate the reports yourself. Can you try this and let us know how it goes.
use the Runner class 2 times to run your tests with different settings and karate.env set for QA and then DEV
the important part is using a different value for the workingDir, e.g. target/reports/qa and then target/reports/dev - else the second run will overwrite the first
now when generating the HTML report, you can provide target/reports as the source folder. this should work for the Maven Cucumber Reports, for Allure, please figure this out on your own
if the above approach does not work well enough for your needs, please figure out a way to manually aggregate the Results object you get from each instance of the Runner, this should not be too complicated as Java code
I understand that nunit console 3 can write to TestResult.xml after running the tests, where the TestResult.xml is located inside the directory specified by --work parameter.
But from what I can tell, TestResult.xml contains too many (irrelevant) details, that I don't need to fix my unit tests errors. All I need is just the failed or ignored test cases, just like what is displayed in command line prompt when I am running nunit console in it.
How to configure the parameters for nunit console 3 so that it only gives me failed or ignored test cases?
Sorry, I'm a few months late (unfortunately I came across this problem only now). But as far as I searched, the only possibility is to use XSL transformation. There are some applications that can convert the XML report file, but... I tested a few, but unfortunately the output was not that was I need.
Therefore I created a simple NUnit3summary application that can transform the standard XML output file to text file. I was surprised that nobody until now made something like this (or at least did not publish it). It were only two hours of work (first working version) and a few more to finish it to stare ready for publishing.
It is only a simple application that was aimed for my needs. You can use filtration for now only with another application, e.g.:
NUnit3summary.exe TestResult.xml | grep Failed >FailedTests.txt
You can see a practical application here (this is also the project where the application was needed, because of too many errors in unit tests).
Currently our Jenkins server only displays a history/graph for the overall number of passed/skipped/failed tests - I'm assuming that's the behavior out of the box.
If you select a single test, you'll get information for how long the test was failing (assuming it did fail).
However, we'd like to see is a history for that single test across the different builds to identify whether the test has been failing in the past (and when) even though it just passed. If you find a build where it failed, you could click on it, and investigate what might have caused the failure; if it passes again, you could check whether something actually fixed the test, or whether it was failing randomly all along.
Is this something that can be done somehow through the config, or do we need an additional plugin for this? If yes, which one?
Not sure if this makes much difference, but we're using Java (Maven) & TestNG (Surefire).
Both the TestNG plugin and the JUnit plugin will actually display history of the test results.
You just need to pick a given result and then:
For JUnit click on "History" on the left side, and
For TestNG click you will see the history in the graph above the result. You can just click on the bars in the bars to see the older results, and also if you click closer to the edge, the scope of the test results will adjust
The Test Results Analyzer plugin does the job for me. There appears to be other suitable plugins out there as well.
https://wiki.jenkins-ci.org/display/JENKINS/Test+Results+Analyzer+Plugin
Does the Static Code Analysis plugin help?
So my typical workflow is
I write a data driven test using TestNG in IntelliJ.
I supply hundreds of data items
Run the test and one or two of them fail
I see the list of passed/failed tests in the "Run" pane.
I would like the ability to just right click that "instance" of the test and run that test alone (with breakpoints). Currently IntelliJ does not seem to have that feature. I would have to right click the test and when I run, it runs the whole set of tests with hundreds of data points.
Is this possible?
TestNG supports this at the testng.xml level, where you can specify which indices of your data provider should be used. It's called "invocation-numbers" and you can see what it looks like by running a test with a data provider, failing some of its invocation numbers and looking at the testng-failed.xml that gets generated.
Back to your question: your IDE needs to support this feature in order to make it available in the UI, so I suggest you ask on the IDEA forums
The feature has been added as of Intellij 142.1217: https://youtrack.jetbrains.com/issue/IDEA-57906
I would like to be able to, as part of a maven build, set the build number (doesn't matter exactly what) in a properties file/class (so I can show it in a UI). Any ideas?
We used the Build Number Plugin now available from Codehaus. It can generate a sequential build number or allows you to use the time stamp.
The Maven build number plugin should do what you want.
I use the maven-property-plugin to store the CruiseControl build label in a properties file (the build label is is available as a system property named 'label').
A post on how to do this with Hudson.