I would like to generate a legacy JUnit 5 report with failsafe which includes manually published entries to the TestReporter which are listed in the events XML file like:
<e:reported id="106" time="2023-01-11T13:46:41.746443Z"><attachments><data time="2023-01-11T14:46:41.746347"><entry key="bla">test</entry></data></attachments></e:reported>
I cannot find them in the legacy report XML TEST-MyIT.xml which only contains a list of test method names:
<testcase name="xx()[30] bla" classname="MyIT" time="0.393"/>
How can I addd all entries to legacy test reports?
Related
I have the xUnitFileImport scheduled job configured in my polarion project (as described in Polarion documentation) to import e2e test results (formatted to JUnit test results)
<job cronExpression="0 0/5 * * * ? *" id="xUnitFileImport" name="Import e2e Tests Results" scope="system">
<path>D:\myProject\data\import-test-results\e2e-gitlab</path>
<project>myProject</project>
<userAccountVaultKey>myKey</userAccountVaultKey>
<maxCreatedDefects>10</maxCreatedDefects>
<maxCreatedDefectsPercent>5</maxCreatedDefectsPercent>
<templateTestRunId>xUnit Build Test</templateTestRunId>
<idRegex>(.*).xml</idRegex>
<groupIdRegex>(.*)_.*.xml</groupIdRegex>
</job>
This works and I get my test results imported into a new test run and new test cases are created. But if I run the import job multiple times (for each test run) it creates duplicate test case work items even though they have the same name, which leads to this situation:
Is there some way to tell the import job to reference the existing testcases to the
newly created test run, instead of creating new ones?
What i have done so far:
yes I checked that the "custom field for test case id" in the "testing > configuration" is configured
yes I checked that the field value is really set in the created test case
The current value in this field is e.g. ".Login" as i don't want the classnames in the report.
YES I still get the same behaviour with the classname set
In the scheduler I have changed the job parameter for the group id because it wasn't filled. New value is: <groupIdRegex>e2e-results-(.*).xml</groupIdRegex>
I checked that no other custom fields are interfering, only the standard fields are set
I checked that no readonly fields are present
I do use a template for the testcases as supported by the xUnitFileImport. The testcases are successfully created and i don't see anything that would interfere
However I do have a hyperlink set in the template (I'll try removing this soon™)
I changed the test run template from "xUnit Build test" to "xUnit Manual Test Upload" this however did not lead to any visible change
I changed the template status from draft to active. Had no change in behaviour.
I tripple checked all the fields in the created test cases. They are literally the same, which leads to the conclusion that no fields in the testcases interfere with referencing to them
After all this time i have invested now, researching on my own and asking on different forums, I am ready to call this a polarion bug unless someone proves me this functionality is working.
I believe you have to set a custom field that identifies the testcase with the xUnit file you're importing, for the importer to identify the testcase.
Try adding a custom field to the TestCase workitem and selecting it here.
Custom Field for Test Case ID option in settings
If you're planning on creating test cases beforehand, note that the ID is formatted form the {classname}.{name} for a given case.
We are trying to generate reports for our tests using serenity BDD. But we couldn't find anything to help with report generation.If anyone is familiar with this,Please suggest any simple way to achieve this.
To generate a report you need to use apply plugin: 'net.serenity-bdd.aggregator' plugin in your build.gradle file. Also while executing your project use gradlew clean test aggregate command from your command line.
After execution you will find index.html report under \target\site\serenity
You can use the Groovy MarkUpBuilder and create customized reports for your use case. Basically, you'll need to create a markup builder instance in Groovy as below :
def xmlWriter = new FileWriter(file("${project.buildDir}/index.html"))
def xmlMarkup = new MarkupBuilder(xmlWriter)
Create custom tags using below sytax :
xmlMarkup.myCustomTag("Lorem Ipsum")
which will produce :
<myCustomTag>Lorem Ipsum</myCustomTag>
So, for a syntax like xmlMarkup.h1("Lorem Ipsum") you'll get the output as <h1>Lorem Ipsum</h1>
Then you can just create a gradle task which parses all the test outputs (xml or json) into HTML.
I had written an article about that in the past which you can find here
I have suppose 10 test cases in test suite in which 2 test cases are disabled.I want to get those two test cases in test result of jenkins job like pass = 7 ,fail = 1 and disabled/notrun= 2.
By default, TestNG generates report for your test suite and you may refer to index.html file under the test-output folder. If you click on "Ignored Methods" hyperlink, it will show you all the ignored test cases and its class name and count of ignored methods.
All test cases annotated with #Test(enabled = false) will be showing in "Ignored Methods" link.
I have attached a sample image. Refer below.
If your test generates JUnit XML reports, you can use the JUnit plugin to parse these reports after the build (as a post-build action). Then, you can go into your build and click 'Test Result'. You should see a breakdown of how the execution went (including passed, failed, and skipped tests).
I am generating a SOAtest report after running the parasoft SOAtest from command line, I am seeing results at Test case level where for one test there are multiple validations from Datasource. How can I generate that gives me report on all validation I am doing rather than just the Tests in suite
(SOATest 9.8)Below gives you Complete Validation report in test suite hierarchy structure in HTML or XML.
Right click on .tst Structure report, Select your configuration and file format you need the report in.
I am using PHPUnit Selenium for functional testing of my project.
I am using junit for logging and using the log file to gnerate the report. Following is the log tag in phpunit.xml
<phpunit>
<logging>
<log type="junit" target="reports/logfile.xml" logIncompleteSkipped="false" />
</logging>
</phpunit>
Then I use the logfile.xml to generate the report.
What I am looking for is the ability to log additional information (information telling what exactly is getting tested in assertion, in both cases i.e. in both pass/fail of assertion).
Basically in reports I want to tell what is being asserted. And that information will be written by the test writer in the test case manually along with assertion.
assert functions comes with the third optional parameter as message but that is shown only on failure.
Eg:
<?php
// $accountExists is the dummy variable which wil probably checking in database for the existence of the record
$this->assertEquals(true, $accountExists, 'Expecting for accountExists to be true');
?>
Above will return message on failure but not when test is passed.
you must use the
--printer command line argument to point to a custom printer class
http://www.phpunit.de/manual/3.6/en/extending-phpunit.html#extending-phpunit.PHPUnit_Framework_TestListener
in your endTest function whatever you put in printf will show up in your log file.