I have got to create one common dashboard that can be used to display the execution results along with execution status and pass percentage from various test automation tools like
api - mochasome reports,
seleium- extent reports,
selenium - cucumber reports.
selenium - testng reports
pitest reports
etc..
Which tool can be used for this requirement?
Datadog might work if you can upload the reports: https://docs.datadoghq.com/continuous_integration/setup_tests/junit_upload/?tab=linux
Still looking for a solution myself too.
Related
We have QA and DEV environment in our automation repo. We are using karate as our framework. We have TestParallel class and integrated allure report.
How could we run all tests in QA first then in DEV back to back using TestParallel Class and see the results in the same report?
Thanks for such a great tool btw.
We are going to try and make this easier in the next version.
For now, you have to aggregate the reports yourself. Can you try this and let us know how it goes.
use the Runner class 2 times to run your tests with different settings and karate.env set for QA and then DEV
the important part is using a different value for the workingDir, e.g. target/reports/qa and then target/reports/dev - else the second run will overwrite the first
now when generating the HTML report, you can provide target/reports as the source folder. this should work for the Maven Cucumber Reports, for Allure, please figure this out on your own
if the above approach does not work well enough for your needs, please figure out a way to manually aggregate the Results object you get from each instance of the Runner, this should not be too complicated as Java code
I am using this github project as a baseline to extend the capabilities in my existing testcafe project and to introduce BDD capabilities.
Also, I am trying multiple-cucumber-html-reporter reporter to beautify end reports
I am using the Map object to generate runtime test data for each scenario and facing issues in posting the test data from map objects and screenshots in the report to show the test data and UI error used at each step or collectively.
I don't know how to achieve this and looking for some help in the right direction or working example would be a great help.
The following shows cases of how to add data to the cucumber json report which I'm guessing your custom report pulls from https://github.com/cucumber/cucumber-js/blob/master/docs/support_files/attachments.md
So my typical workflow is
I write a data driven test using TestNG in IntelliJ.
I supply hundreds of data items
Run the test and one or two of them fail
I see the list of passed/failed tests in the "Run" pane.
I would like the ability to just right click that "instance" of the test and run that test alone (with breakpoints). Currently IntelliJ does not seem to have that feature. I would have to right click the test and when I run, it runs the whole set of tests with hundreds of data points.
Is this possible?
TestNG supports this at the testng.xml level, where you can specify which indices of your data provider should be used. It's called "invocation-numbers" and you can see what it looks like by running a test with a data provider, failing some of its invocation numbers and looking at the testng-failed.xml that gets generated.
Back to your question: your IDE needs to support this feature in order to make it available in the UI, so I suggest you ask on the IDEA forums
The feature has been added as of Intellij 142.1217: https://youtrack.jetbrains.com/issue/IDEA-57906
I’m new to Selenium WebDriver, Currently working on reporting testcases & test suits using TestNG (import org.testng.Reporter)on eclipse, At present TestNG produces high level reporting features for test cases ,methods & suites on test- output folder, similar to this --> http://testng.org/test-output/index.html.
I'm looking for a way to have following in the report created by testNG,
All selenium commands executed in my test method(s) should be displayed in index.html &
Failed screenshots for the relevant failure command with the reports, something similar to loggingSelenium ---> http://loggingselenium.sourceforge.net/samples/sampleResultSuccess.html
The reason I prefer this way is its easier, because this way I know exactly what selenium commands which were executed , I would have correct information what goes wrong, and when this occurs and the input parameters belong to specific selenium commands.
Is it possible in testNG?
I'm looking for a some sort of management/reporting tool that collects the results of tests, stores them for reporting, then lets users generate reports based on those tests.
We have numerous test running tools that run on a variety of platforms, but all output test results in the JUnit format. The test are not specific to hardware or platform, but rather generic. What we would like to do is have an automated (or manual) test run be able to submit it to a central location along with additional information, like platform, OS, hardware configuration and maybe user defined data. The management/reporting tool would store this data.
Then, a manager would be able to go to the tool and request (or more likely, access a dashboard that developers have setup) an update on the current status. This could be a list of test results that were run in a particular configuration, or a hardware status, or just the results of specific test(s).
Any suggestions?
We built a test management tool Enterprise Tester (www.enterprisetester.com) that allows users to pull automated test results in nUnit, nUnit, XSLT, Selenium etc and report off the results.
In addition to pulling the results and reporting you are able to trace these tests back to requirements that have been created giving you the ability to see test coverage and the status of this coverage on dashboards. If you are using JIRA (or google) or anything that uses open social gadgets you can pass these gadgets to other tools also.
Feel free to contact me directly if you would like to talk further about it
Regards
Bryce
You can also try an open source project called Allure Framework. It was created specially for showing test execution results in a nice form. A set of popular test frameworks such as JUnit, PHPUnit, TestNG, py.test, RSpec, Scalatest are already supported. Other ones such as NUnit will be supported soon.
Check out Hudson. it's very useful and configurable