Where can i save my testNG reports for analysis later.
My regression suite will be running daily and I want to store the reports for anaysis and comparitive study later.
What can i use for the same?
Allure, ReportPortal or maybe some other tools could help you.
There is no such repository. You can go ahead and build a customized org.testng.IReporter which persists the test results to a database which you can fetch and run your analysis.
As suggested you can go database..
Generally i used custom HTML reports for suite and links to Tests HTML reports. I will create dir with data and time stamp for every run and move to shared location to keep and share.
Related
Hello everyone I would like to ask for your assistance if there's a way I could create an HTML/GUI Report upon running "dbt test" command. Your response is highly appreciated.
This is the result I got upon running dbt test
There are some 3rd party libraries that will generate reports like the one you're looking for. One you can try is called Elementary Data (https://www.elementary-data.com/). When you run DBT tasks, it creates a schema with all of the stats on model execution timings, test results, etc. It then has a command to generate an HTML page using those results to publish some simple graphs and aggregate test results.
Is it possible to generate auto API test report using Datadog? :)
I'm creating a project in Java using Rest Assured and Hamcrest. I have the project on gitlab and I would like the tests to be automatically run. Based on them, I would like to receive a report :).
At first I through I could use a allure but my supervisor asked me if I could do it in Datadog.
I tried to find background material on this subject, but failed :(
I will be grateful for every link with supporting materials :)
What do you use for build tool maven or gradle?
I think this should work for you:
https://docs.datadoghq.com/continuous_integration/tests/java
We have QA and DEV environment in our automation repo. We are using karate as our framework. We have TestParallel class and integrated allure report.
How could we run all tests in QA first then in DEV back to back using TestParallel Class and see the results in the same report?
Thanks for such a great tool btw.
We are going to try and make this easier in the next version.
For now, you have to aggregate the reports yourself. Can you try this and let us know how it goes.
use the Runner class 2 times to run your tests with different settings and karate.env set for QA and then DEV
the important part is using a different value for the workingDir, e.g. target/reports/qa and then target/reports/dev - else the second run will overwrite the first
now when generating the HTML report, you can provide target/reports as the source folder. this should work for the Maven Cucumber Reports, for Allure, please figure this out on your own
if the above approach does not work well enough for your needs, please figure out a way to manually aggregate the Results object you get from each instance of the Runner, this should not be too complicated as Java code
So my typical workflow is
I write a data driven test using TestNG in IntelliJ.
I supply hundreds of data items
Run the test and one or two of them fail
I see the list of passed/failed tests in the "Run" pane.
I would like the ability to just right click that "instance" of the test and run that test alone (with breakpoints). Currently IntelliJ does not seem to have that feature. I would have to right click the test and when I run, it runs the whole set of tests with hundreds of data points.
Is this possible?
TestNG supports this at the testng.xml level, where you can specify which indices of your data provider should be used. It's called "invocation-numbers" and you can see what it looks like by running a test with a data provider, failing some of its invocation numbers and looking at the testng-failed.xml that gets generated.
Back to your question: your IDE needs to support this feature in order to make it available in the UI, so I suggest you ask on the IDEA forums
The feature has been added as of Intellij 142.1217: https://youtrack.jetbrains.com/issue/IDEA-57906
I'm looking for a some sort of management/reporting tool that collects the results of tests, stores them for reporting, then lets users generate reports based on those tests.
We have numerous test running tools that run on a variety of platforms, but all output test results in the JUnit format. The test are not specific to hardware or platform, but rather generic. What we would like to do is have an automated (or manual) test run be able to submit it to a central location along with additional information, like platform, OS, hardware configuration and maybe user defined data. The management/reporting tool would store this data.
Then, a manager would be able to go to the tool and request (or more likely, access a dashboard that developers have setup) an update on the current status. This could be a list of test results that were run in a particular configuration, or a hardware status, or just the results of specific test(s).
Any suggestions?
We built a test management tool Enterprise Tester (www.enterprisetester.com) that allows users to pull automated test results in nUnit, nUnit, XSLT, Selenium etc and report off the results.
In addition to pulling the results and reporting you are able to trace these tests back to requirements that have been created giving you the ability to see test coverage and the status of this coverage on dashboards. If you are using JIRA (or google) or anything that uses open social gadgets you can pass these gadgets to other tools also.
Feel free to contact me directly if you would like to talk further about it
Regards
Bryce
You can also try an open source project called Allure Framework. It was created specially for showing test execution results in a nice form. A set of popular test frameworks such as JUnit, PHPUnit, TestNG, py.test, RSpec, Scalatest are already supported. Other ones such as NUnit will be supported soon.
Check out Hudson. it's very useful and configurable