I have a multi module Gradle script with 3 modules: common, services and web.
If I run gradle check all the test suite is performed correctly, but I don't have a single test report instead I found a test report for each subproject. How can I merge the test results?
If useful this is the script I'm using https://github.com/CarloMicieli/trenako/blob/master/build.gradle
Thanks
Carlo
The Gradle source contains a Task class used to accomplish this, which unfortunately isn't available in the public API. Basically all it does is copy all of the test reports from various projects into one directory and then run the standard test report task to get an aggregate. Check out the source code at https://github.com/gradle/gradle/blob/master/buildSrc/src/main/groovy/org/gradle/build/TestReportAggregator.groovy
Related
Is it possible to generate auto API test report using Datadog? :)
I'm creating a project in Java using Rest Assured and Hamcrest. I have the project on gitlab and I would like the tests to be automatically run. Based on them, I would like to receive a report :).
At first I through I could use a allure but my supervisor asked me if I could do it in Datadog.
I tried to find background material on this subject, but failed :(
I will be grateful for every link with supporting materials :)
What do you use for build tool maven or gradle?
I think this should work for you:
https://docs.datadoghq.com/continuous_integration/tests/java
We have QA and DEV environment in our automation repo. We are using karate as our framework. We have TestParallel class and integrated allure report.
How could we run all tests in QA first then in DEV back to back using TestParallel Class and see the results in the same report?
Thanks for such a great tool btw.
We are going to try and make this easier in the next version.
For now, you have to aggregate the reports yourself. Can you try this and let us know how it goes.
use the Runner class 2 times to run your tests with different settings and karate.env set for QA and then DEV
the important part is using a different value for the workingDir, e.g. target/reports/qa and then target/reports/dev - else the second run will overwrite the first
now when generating the HTML report, you can provide target/reports as the source folder. this should work for the Maven Cucumber Reports, for Allure, please figure this out on your own
if the above approach does not work well enough for your needs, please figure out a way to manually aggregate the Results object you get from each instance of the Runner, this should not be too complicated as Java code
I would like to merge an existing MTP (Master Test Plan) defined as a Test Suite with subfolders and TestCases, into another MTP (same structured), but located in a different project in TFS2017.
The idea is to have both project sharing the same MTP, so any change can be visible in both projects.
By now, I've just seen the existing possibilities about cloning/copying test cases between different test suites, but all of them should be part of the same project.
I guess there must be an easy way of doing that merge instead of manually, but I can't find the way to do it.
We can use an Existing Test Case (by reference) cross projects, but cannot share the same Test Plan cross projects.
If you wish to have a discrete copy of a test case, test cases, or test suites across projects (to a test plan that resides in a different project), you can perform a “clone” across projects via the command line (tcm.exe).
You could do this in MTM, could also use Tcm.exe command to copy test suites.
Please reference below articles :
Copying and cloning test suites and test cases.
Microsoft Test Manager – Working Across Projects: Adding an Existing
Test Case vs.Cloning/Copying
Actually, there has been a related uservoice for your requirement: Export test plan tree between projects and collections in MTM
We have enabled both Test Plan and Test Suite clone capabilities
within the product. See
https://msdn.microsoft.com/en-us/library/hh543843.aspx
This allows you to clone test plans/test suites across projects within
a collection. For moving artifacts (including test artifacts) across
collections, you can use the “TFS Integration Platform toolkit”.
Marking this item as “Completed”.
UPDATE:
To copy test cases, or test suites across projects, you can use “clone” via the command line (tcm.exe).
Below sample for your reference:
Run cmd
cd C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE
tcm suites /clone /collection:http://server:8080/tfs/CollectionLC /teamproject:0418Scrum /suiteid:271 /destinationteamproject:TFVC-Scrum /destinationsuiteid:274 /overridefield:"System.IterationPath"="TFVC-Scrum\Sprint 2" /overridefield:"System.AreaPath"=TFVC-Scrum
So my typical workflow is
I write a data driven test using TestNG in IntelliJ.
I supply hundreds of data items
Run the test and one or two of them fail
I see the list of passed/failed tests in the "Run" pane.
I would like the ability to just right click that "instance" of the test and run that test alone (with breakpoints). Currently IntelliJ does not seem to have that feature. I would have to right click the test and when I run, it runs the whole set of tests with hundreds of data points.
Is this possible?
TestNG supports this at the testng.xml level, where you can specify which indices of your data provider should be used. It's called "invocation-numbers" and you can see what it looks like by running a test with a data provider, failing some of its invocation numbers and looking at the testng-failed.xml that gets generated.
Back to your question: your IDE needs to support this feature in order to make it available in the UI, so I suggest you ask on the IDEA forums
The feature has been added as of Intellij 142.1217: https://youtrack.jetbrains.com/issue/IDEA-57906
In TFS 2012, we have several build definitions - CIs, Deployments and nightly.
Our CI builds run all of the (n)unit tests from our solution, however, we need to get it to ignore certain tests.
This is because we have some long running integration tests, and these only need to be run nightly.
Things I've tried:
Using the TestCategoryAttribute (from MSTest) and setting the Test Case Filter property try and exclude 'Integration'.
Using the CategoryAttribute (from NUnit) and setting the Test Case Filter property try and exclude 'Integration'.
A combination of the above.
The tests that need to be ignored are all in separate assemblies with the word IntegrationTests or Integration.Tests in the name.
Thanks,
Kieron
I've been using a combination of the MSTest TestCategory attribute on my unit tests, and the Test category filter setting for my build process definition in TFS 2012.
According to the Microsoft MSDN article found here you can specify which categories to use by setting the Test category filter to
TestCategory=CategoryName
According to your original post, you'd need to use the following filter:
TestCategory!=Integration
and decorate your tests with this attribute:
[TestCategory("Integration")]
Do this on all of your unit tests that you want ignored during your build. The test lists have been deprecated in Visual Studio, and it took a while to convert everything to the categories, but in the end it's worth it.
Hope that helps!