I have generated the allure report from my webdriverio framework and configured the report in jenkins.
How can I share the allure report from jenkins to stakeholders?
Could you please help me?
Thank you for the question.
There are several ways to make this happen:
You may provide a link to the Jenkins job which contains the report but, usually, stakeholders don't want to dive into CI. Or they don't have the necessary credentials.
You may download and archive and send it. In case there are many tests in the suite, the archive might be massive. That's also an issue.
In case you need to share reports often, I would recommend trying Allure TestOps: it's an Allure Report-based management tool providing an extremely flexible report and dashboard sharing opportunities.
Related
As part of testing in the cloud, I plan to copy my test results (I happen to be using gradle) as xml out of a short lived container to an s3 bucket.
I am wondering what is the best approach for making use of the bucket full of test results and if anyone has ideas for how to get the most of these results with the least custom code, etc.
In the past I have handled this with XSLT to make the XML into HTML but that approach feels like more work than I'd like to have to do.
Has anyone solved this issue for their own purposes that can share your approach?
As I side note... I could also copy out to my s3 bucket the HTML reports from gradle if someone has an idea about how to use the HTML instead.
Ideally, I would be able to see a sorted list of results and click on the most recent to see the test run details.
Disclaimer: I'm not a Devops guy, I'll present a "developer's" point of view here
The answer IMO depends on what exactly would like to do with all these reports.
In a nutshell, you can have text/xml reports from the test execution (I can't say for gradle but in maven surefire plugin creates these reports, I believe gradle does the same).
You can also generate a good looking site with the reports information, there are tools for this as well, for example Allure Reports. Bottom line, some stuff to be shown can be generated in the build directory.
Copying the reports is as simple as copying a file, but the question is what would like to do with them when they're in S3?
In my understanding, its a Job for CI tools (like Jenkins) to run the tests during the build, or maybe a suit of automation tests against a deployed environment (in the cloud).
These CI Tools run gradle to build the project, and run tests, then they show test results per build memorizing last N builds. It can also be integrated with tools like aforementioned allure and show the HTML report per build as long as the build is stored in CI tool.
So I'm not sure I understand why would you like to "run tests in the cloud" (from which I assume you're running tests suite when the artifact is being deployed).
As for possible ideas of what can be done with all these results:
You can create a site that will show the results across builds (again, this is
already solved by CI tool like Jenkins)
You can store the results of running the tests in some database and provide some
clever statistics (I personally don't see much benefit if usually the build is
green and you're not in the process of "recuperation" of project from flaky tests or
something)
I am trying to integrate Allure2 reports with Jira plugin called Xray. So far i managed to generate those reports on local machine and on Jenkins server in pipeline job with Allure plugin.
Also i see that there is an actual Allure plugin called xray-plugin on Allure Framework GitHub, however instructions how to use this plugin are not clear enough for me (for example, where one should put those Jira env variables?).
Can someone here explain in more clear manner how to integrate Allure2 with x-ray?
I'm trying to get allure reporting working as part of an NUnit project in C# using Selenium WebDriver. Following the documentation for installing allure seems to work fine on a local machine but I'm trying to get the Team City plugin to also work this the project. I've uploaded the allure team city plugin and added a build step that will generate a report even if the build fails or is stopped. The project has all the required allure nunit tags added and again this generates results locally that a report can be generated from. However, when running tests from Team City the build hangs and does not even begin to run tests. At the step that it should start running tests it just sits there. The Team City build logs do not show anything wrong. I'm using NUnit 3.6, Team City 2018.1, allure 2,7, allure team city plugin 2.9.
Does anyone have any experience with allure reporting? The documentation is a little out of date but I've done as much as I can with it.
I'm not sure if this is allowed, answering your own question, but in light of little on the ground of existing responses given the Allure developers actually suggest posting to this site as opposed to posting to their own Git hub site, I find it rather frustrating that there has been little response.
Anyway, having spent further hours looking into this I believe that the hanging of the builds is purely down to the allure-config.json file and the path to the allure-results setting within it. Any attempt to change that setting from its default appears to instantly cause the TC builds containing Allure Reports to hang.
hopefully this may help others having similar issues in future.
i am using Selenium Web driver with Cucumber for test automation. To see the reports i am integrating with Jenkins. I am able to see the report from jenkins. It is providing the status as Passed or failed. I need to add the reasons for these failures in the report. A new column next to status has to be included for the reasons. Can anyone please help me on this as i am new to these topics?
It might be hard to include the reason of the feature failure because reason may be different for different scenarios.
What I recommend is to publish the report over Jenkins using plusgin such as Cucumber-JVM Reports
I'm working with a Jenkins system, and I've recently started working to optimize the tests. There are almost a thousand Selenium tests and twice as many unit and integration tests.
I'm wondering if there is some way to find which tests are the most prone to failure so we can attack the worst offenders first. It would also be nice if there was an integrated way to track who is working on which test, and if a test has been fixed, what was done to fix it. I'm new to Jenkins, so please point me to some documentation or a plugin I can install that would help me with this.
I think what you are describing is JIRA which can tie SCMs to issues to releases to developers.
There is Jenkins to JIRA integration to create Jira issues out of Jenkins