During automatic test execution in IntelliJ IDEA the Run tab of the IDE is dedicated to test results and is possible to recall old test execution results using the "Clock"/Test history button.
But how can I recall these results if I then run another "activity" (like Maven build or application launch)?
Is there a way to access the test data when Run tab is no longer in "test mode"?
NOTE I'm using IntelliJ IDEA 2022.2.3 (Ultimate Edition)
You should be able to retrieve old test results under the Run | Test History menu item.
Related
I find that when I'm running a failing Gradle task from within IDEA, the "Run" tool window automatically selects the most recent "node" in the tool window, which filters out all the console logs except from that "node" - IDEA tries help by showing only the logs relating to the failing task.
The problem is that the console output of just the last task never shows useful output.
It doesn't show the exception that caused the failure, and it doesn't show any logs leading up to the failure.
For example, when running a JavaExec task, by default IDEA shows this when the task ends:
In order to see the info that's actually relevant to debugging the issue, I need to manually select a node higher in the tree:
The question:
Is there any way to tell IDEA to automatically select the "task node" for a failing Gradle task (which usually contains the usable information), instead of the "command" node which almost never shows useful information?
IntelliJ: 2022.1.3
Gradle: 7.4.1
As per comment from Andrey - there is no such option for IDEA 2022.1.
We have QA and DEV environment in our automation repo. We are using karate as our framework. We have TestParallel class and integrated allure report.
How could we run all tests in QA first then in DEV back to back using TestParallel Class and see the results in the same report?
Thanks for such a great tool btw.
We are going to try and make this easier in the next version.
For now, you have to aggregate the reports yourself. Can you try this and let us know how it goes.
use the Runner class 2 times to run your tests with different settings and karate.env set for QA and then DEV
the important part is using a different value for the workingDir, e.g. target/reports/qa and then target/reports/dev - else the second run will overwrite the first
now when generating the HTML report, you can provide target/reports as the source folder. this should work for the Maven Cucumber Reports, for Allure, please figure this out on your own
if the above approach does not work well enough for your needs, please figure out a way to manually aggregate the Results object you get from each instance of the Runner, this should not be too complicated as Java code
I have many builds with failed tests.
I learn one test and want to find last build, where this test was succesfully.
How I can find this build or how I can get test results history?
If as a post-build step you are publishing your test results, you should be able to go to the job page and see the test results graph. If you are looking for the last build where all tests passed, you can click on "enlarge" under the graph. You will then see a high level view of all the builds from which you should be able to see the last build where all tests passed.
If you are looking for the last time a particular test passed, you can click on the latest build in the test results graph, drill down to the particular test you are interested in, and then click the previous build button until you find a build where that test passed.
If you aren't publishing your test results, you may be able to write some groovy code to scan through your build results, but beyond that I don't know of a way to find what your looking for without publishing test results in your post-build.
Currently our Jenkins server only displays a history/graph for the overall number of passed/skipped/failed tests - I'm assuming that's the behavior out of the box.
If you select a single test, you'll get information for how long the test was failing (assuming it did fail).
However, we'd like to see is a history for that single test across the different builds to identify whether the test has been failing in the past (and when) even though it just passed. If you find a build where it failed, you could click on it, and investigate what might have caused the failure; if it passes again, you could check whether something actually fixed the test, or whether it was failing randomly all along.
Is this something that can be done somehow through the config, or do we need an additional plugin for this? If yes, which one?
Not sure if this makes much difference, but we're using Java (Maven) & TestNG (Surefire).
Both the TestNG plugin and the JUnit plugin will actually display history of the test results.
You just need to pick a given result and then:
For JUnit click on "History" on the left side, and
For TestNG click you will see the history in the graph above the result. You can just click on the bars in the bars to see the older results, and also if you click closer to the edge, the scope of the test results will adjust
The Test Results Analyzer plugin does the job for me. There appears to be other suitable plugins out there as well.
https://wiki.jenkins-ci.org/display/JENKINS/Test+Results+Analyzer+Plugin
Does the Static Code Analysis plugin help?
So my typical workflow is
I write a data driven test using TestNG in IntelliJ.
I supply hundreds of data items
Run the test and one or two of them fail
I see the list of passed/failed tests in the "Run" pane.
I would like the ability to just right click that "instance" of the test and run that test alone (with breakpoints). Currently IntelliJ does not seem to have that feature. I would have to right click the test and when I run, it runs the whole set of tests with hundreds of data points.
Is this possible?
TestNG supports this at the testng.xml level, where you can specify which indices of your data provider should be used. It's called "invocation-numbers" and you can see what it looks like by running a test with a data provider, failing some of its invocation numbers and looking at the testng-failed.xml that gets generated.
Back to your question: your IDE needs to support this feature in order to make it available in the UI, so I suggest you ask on the IDEA forums
The feature has been added as of Intellij 142.1217: https://youtrack.jetbrains.com/issue/IDEA-57906