I'm trying to set up a GitHub action workflow where I run unit test cases and if it is successful, I will deploy my app.
It prints the success or failed then continues to next step. When I tried to get the output from the test in GitHub action it returns an empty object.
What should I set up in my Karma config to get the output i.e SUCCESS or FAILED?
Related
I'm using TestCafe Studio to create my tests and executing the tests written in .testcafe format using the testcafe docker container. Further I'm using 'Drone' as CI environment.
Below is the command that I use to execute my tests
`- /opt/testcafe/docker/testcafe-docker.sh -c 3 chromium -q --skip-js-errors --ass`ertion-timeout 60000 --selector-timeout 60000 CommonScenarios/*.testcafe
When a test failure occur I will not get enough information about the test failure. For example below is an error log printed.
1) AssertionError: expected false to be truthy
Browser: Chrome 91.0.4472.124 / Linux 0.0
7
Is there any way that I can get enough details about which step is actually failing when the tests are executed in .testcafe format?
(When I ran .js format of the test it gives which line is failing)
This looks like a bug in TestCafe Framework. I opened an issue in the GitHub repository: https://github.com/DevExpress/testcafe/issues/6424. Subscribe it to be notified of updates.
As a simple solution, you can convert your *.testcafe fixture file to *.js. Also, there is a more complex workaround - it allows you to determine which step is failing:
Open the *.testcafe file in VS Code or some other editor. You will see that it looks like a JSON file.
Find an object with the "name" property whose value corresponds to your test name: "name": "Your-failed-test-name"
Look at the "commands" array. Find an object with the "callsite" property whose value is equal to the number that you can see in the error console output. This object specifies the failed step.
Note that the format of this file is intended for internal use, and it is not recommend to modify it manually.
We have a running api test suite on our CI environment.
Today I had a case where something broke in the code in the _beforeSuite function (in the response of some endpoint, I receive an empty JSON object).
Now, I am investigating any options, to include _beforeSuite in the test report, so in such cases, we could see what exactly went wrong on "CI test report" level.
Expected behavior:
When I launch my tests for example with
codecept run tests/api --env stage1 --html my_report.html
I want "my_report" to include steps from _beforeSute function
I'm setting up an Azure-DevOps pipeline in which I want to include automated tests via the Newman CLI.
Imagine a pipeline like this.
Build Project
Copy build to test folder
Run the application => (API-Server)
Run Newman
Kill API Server Process
On Success Copy Build to another folder.
My Problem is that my server application is in a waiting state after it's initialization.
The next Task in my build pipeline won't start.
Is there a way to run multiple command lines asynchronously in Azure-DevOps?
Starting the process in via "start" won't work since it throws me an
ERROR: Input redirection is not supported, exiting the process immediately.
start "%TESTDIR%\foo\bar.exe"
timeout 10
I've got some tests setup to run via Taurus and kicking them off in a Jenkins stage like so:
...previous stages...
stage('Load Tests'){
dir('./tests'){
bat "bat _testFile.yml"
}
}
...stages to execute if Load Tests stage succeeds
I want to bail out of the whole build if any one of the iterations in any of my tests fails. But this setup, as well as wrapping in a try/catch don't work.
My fault. The fail criteria were not properly implemented. Now that they are, the above 'successfully' fails the jenkins build when the criteria are not met.
I am using bamboo to run E2E Tests (using Selenium). After the build finishes I am using the Bamboo API to get the build result.
The problem is that the build result doesn't contain any stack trace of the failed tests. I have tried use the URI param ?expand=testResults.failed.testResult.errors but it only includes an exception message or an assert message.
I need the stacktrace of the exeception too.