We are going with XUnit to run Selenium Functional Tests. In Microsoft Azure Devops, we have a pipeline stage which runs our functional tests.
If the test fails, I want to take a screenshot using the Web Driver and attach it to the test results output.
I know it can be done with NUnit (using TestContext), but how can I achieve this using XUnit? Not sure if this is correct, but is there a way using REST APIs that I could somehow achieve this?
XUnit still doesn't support TestContext, check here: https://github.com/xunit/xunit/issues/2133
After taking the screenshots, you may consider using Attachments - Create Test Result Attachment API to attach the screenshot to a test result:
POST https://dev.azure.com/{organization}/{project}/_apis/test/Runs/{runId}/Results/{testCaseResultId}/attachments?api-version=6.0-preview.1
Looks like xUnit will support it in version 3.0. Currently the teams I work on use the API listed by Cece to push screenshots per-test.
https://github.com/xunit/xunit/issues/621
Related
I am using the Karate framework to do the API testing. As part of CI efforts, we send an email at the end of test execution listing the summary of test results. There is a need to include the screeshot of the test execution counts from 'overview-feature.html' file.
I did so through the TestRunner.java file - launched Chrome using Chrome.start() and then using it to take screenshot. It all works well locally on Windows.
However when executing on CI server which is a Unix box, the chrome executable is not present in the default location (usr/bin/google-chrome) and hence the connection for the localhost fails.
Is there a way we can change the default location of the chrome executable?
PS: Apologies if this was too trivial to be asked.
Yes Chrome on CI is hard to get right, refer: https://stackoverflow.com/a/62325328/143475 - note that CI boxes typically are "headless" a browser may not be even installed.
I think the best thing for you is to ZIP the HTML and send it. But I really think you need to work with some CI experts, because the report generation and e-mailing business is normally done by things like Jenkins. What you are doing is certainly not normal or best-practice.
If you really want, there is a Karate Docker container that can give you a proper Chrome instance (see docs) but that is overkill for what you need.
EDIT: The Chrome Java API allows for customization of the executable path and this is in the docs: https://github.com/intuit/karate/tree/master/karate-core#chrome-java-api
It should be something like this:
Chrome.start("/opt/blah/chrome");
I'm running Jest integration tests on Jenkins and I want to integrate them with TestRail in order to automatically put test results to TestRail. In this way I will know easily how many tests are passed/failed?
Does anyone tried that?
I guess you are looking for something like this one https://github.com/zeljkosimic95/Jest-2-Testrail . Although your question is too old but it might help someone else. This is not official plugin but it may help you.
Adding a suggestion here, we should choose the tool after analyzing all the requirements in your software testing services and product. Because there is no official plugin for this except this https://gitlab.com/craydent/jest-testrail (haven't tried). But you still can do this without plugin with this library/code https://github.com/zeljkosimic95/Jest-2-Testrail
You can probably have a look at Agiletestware Pangolin solution which allows you to export results of your tests into TestRail automatically from popular CI systems.
In order to be able to upload test results, Pangolin requires you to create report in JUnit format which can be done by using https://github.com/jest-community/jest-junit
Disclaimer: I'm a developer of Agiletestware Pangolin
I'm running nunit tests externally via Nunit3-console.
I'm not able to see any console.logs/console.Writeln
I find it mandatory to be able to track in real time, every step of the test.
I've read that nunit3 framework created a parallel test run feature, which is why the real time test output logs have been taken off.
But if I want to enjoy both worlds?
How can I trigger console logs during a test run?
Thanks in advance.
Nunit3 framework doesn't support live hard coded console outputs when running tests from Nunit3-console, because nunit3 framework was meant to bring the parallel testing methodology, they found it pointless to allow such outputs when more than 1 test is running in parallel.
I solved this by downgrading to Nunit 2.6.4 which isnt supporting parallel testing and allows me to fire console outputs from my test.
I know there is an existing post asking if it's possible to generate an HTML report of code coverage analysis for tests written and run with Intern, and it's been answered:
Generate HTML code coverage reports with intern
However, the post doesn't mention what type of environment that the OP runs in; i.e., are the tests running in a Node.js client? I ask because I am running my unittests using the Intern framework in a browser [edit: invoking tests comparably to http://path/to/intern-tutorial/node_modules/intern/client.html?config=tests/intern]. The article here:
https://github.com/theintern/intern/wiki/Using-and-Writing-Reporters#custom-reporters
outlines that HTML is the only reporter available for the browser platform; LCOV and LCOVHTML are not. But has that changed at all? This limited array of reporters
for browsers isn't very convenient, and I was hoping to take advantage of Istanbul built into Intern, rather than try to plug in another code coverage analysis tool (or hack my own thing :( ).
Code coverage information will be correctly retrieved from code running in browsers if you run your tests with intern-runner. The actual collation and output of the coverage results occurs on the server (Node.js) side.
I'm running tests using Siesta from command line.
Is that possible to generate screenshots on each test fail while running Siesta test runner?
Edit:
Need to wait for the feature
http://www.bryntum.com/products/siesta/changelog/
It's not possible in current version 2.0.8, but they are planning to add this in future.
More info can be found on their forum http://www.bryntum.com/forum/
Also Siesta can be runned in cloud on http://www.browserstack.com/ or https://saucelabs.com/. These services offers evene whole screencasts of the tests.