Even after my expect assertion failed my script is executing next 'it' block. How can I stop execution of script if any test case is failed in my webdriverIO
code:
it('6. Confirm flight IATA code and Airline name',async()=>{
await expect(cargodamagePage.flightIATA).toHaveValue('ERV')
console.log("The flight IATA code is",await cargodamagePage.flightIATA.getValue());
})
There is a bail option you can set in your wdio.conf.js file. If you set it to 1, it will stop all tests if after a single test failure:
https://webdriver.io/docs/options#bail
Related
In Robot Automation, how to re-run the failed test case immediately if it is failed, before going to another test case execution.
For instance,
*** Test Cases ***
Login User And Create Another User
Login User ....
Create Another User ...
Login With New User
Login User..
Test Function ABC
.....
.....
Since one test has a dependency on another test, I need to re-run the failed case immediately after it is failed. Before executing another test.
In one word, you can't, and you shouldn't; a case is a case, with binary outcome. And if you have dependencies between tests, that's a smelly design; try to change it to a pre-condition (env setup) for the second case, so it is atomic.
Disclaimer: this rant is for the automatic re-execution in a single run. After a run has finished, RF has baked-in functionality to re-execute just the failed ones (so flaky tests are given the chance to succeed); but as I understood your question, you are not asking for the latter.
In two words, if you really need to do it, you can; extract the whole test case in a keyword, and call it inside Wait Until Keyword Succeeds, giving it 2 (or more?) attempts:
*** Test Cases ***
Test Function ABC
Wait Until Keyword Succeeds 2 times 100ms The Actual Test For Function ABC
*** Keywords ***
The Actual Test For Function ABC
.....
.....
When running the following scenario, the tests finish running but execution hangs immediately after and the gradle test command never finishes. The cucumber report isn't built, so it hangs before that point.
It seems to be caused by having 2 call read() to different scenarios, that both call a third scenario. That third scenario references the parent context to inspect the current request.
When that parent request is stored in a variable the tests hang. When that variable is cleared before leaving that third scenario, the test finishes as normal. So something about having a reference to that context hangs the tests at the end.
Is there a reason this doesn't complete? Am I missing some important code that lets the tests finish?
I've added * def currentRequest = {} at the end of the special-request scenario and that allows the tests to complete, but that seems like a hack.
This is the top-level test scenario:
Scenario: Updates user id
* def user = call read('utils.feature#endpoint=create-user')
* set user.clientAccountId = user.accountNumber + '-test-client-account-id'
* call read('utils.feature#endpoint=update-user') user
* print 'the test is done!'
The test scenario calls 2 different scenarios in the same utls.feature file
utils.feature:
#ignore
Feature: /users
Background:
* url baseUrl
#endpoint=create-user
Scenario: create a standard user for a test
Given path '/create'
* def restMethod = 'post'
* call read('special-request.feature')
When method restMethod
Then status 201
#endpoint=update-user
Scenario: set a user's client account ID
Given path '/update'
* def restMethod = 'put'
* call read('special-request.feature')
When method restMethod
Then status 201
And match response == {"status":"Success", "message":"Update complete"}
Both of the util scenarios call the special-request feature with different parameters/requests.
special-request.feature:
#ignore
Feature: Builds a special
Scenario: special-request
# The next line causes the test to sit for a long time
* def currentRequest = karate.context.parentContext.getRequest()
# Without the below clear of currentRequest, the test never finishes
# We are de-referencing the parent context's request allows test to finish
* def currentRequest = {}
without currentRequest = {} these are the last lines of output I get before the tests seem to stop.
12:21:38.816 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - response time in milliseconds: 8.48
1 < 201
1 < Content-Type: application/json
{
"status": "Success",
"message": "Update complete"
}
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.818 [ForkJoinPool-1-worker-1] INFO com.intuit.karate - [print] the test is done!
12:21:38.818 [pool-1-thread-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
<==========---> 81% EXECUTING [39s]
With currentRequest = {}, the test completes and the cucumber report generates successfully which is what I would expect to happen even without that line.
Two comments:
* karate.context.parentContext.getRequest()
Wow, these are internal API-s not intended for users to use, I would strongly advise passing values around as variables instead. So all bets are off if you have trouble with that.
It does sound like you have a null-pointer in the above (no surprises here).
There is a bug in 0.9.4 that causes test failures in some edge cases such as the things you are doing, pre-test life-cycle or failures in karate-config.js to hang the parallel runner. You should see something in the logs that indicates a failure, if not - do try help us replicate this problem.
This should be fixed in the develop branch, so you could help if you can build from source and test locally. Instructions are here: https://github.com/intuit/karate/wiki/Developer-Guide
And if you still see a problem, please do this: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
I have a keyword called "debug teardown" which prints the test status and then runs the debuglibrary Debug keyword, if the test has failed.
I would like to be able to log to console which keyword has caused the failure, so I can more effectively debug my test.
Is it possible to get the stack trace or most recent test keyword, and log it to the console?
Here is my Debug Teardown keyword:
Debug Teardown
Run Keyword If Test Failed Log ${TEST STATUS}: ${TEST MESSAGE} ERROR
Run Keyword If Test Failed Debug
You can get a bit more information if you create a listener and also set the log level to DEBUG. Inside the listener you can save the results of log commands, and then when a keyword fails you can print it out or do whatever else you want.
For example, here's a listener that will print to stdout the last log message when a keyword fails:
from __future__ import print_function
class my_listener():
ROBOT_LISTENER_API_VERSION = 2
def __init__(self):
self.ROBOT_LIBRARY_LISTENER = self
self.last_log = None
def _log_message(self, message):
self.last_log = message
def _end_keyword(self, name, attrs):
if attrs['status'] == 'FAIL':
print("\n******\n", self.last_log['message'])
You would use it by importing the listener like a normal library, and also setting the log level to DEBUG (otherwise you'll get the error but no stack trace).
Example:
*** Settings ***
Library my_listener
Suite Setup set log level DEBUG
*** Test cases ***
Example
some keyword
You might be able to utilize set suite variable to update a "global" variable as you go. The last set variable value would be the value that failed.
Karate step execution stops when any one of the step fails.
Example:
Scenario : verify user details.
Given url "this is my webservice"
When method post
Then status 200
*assert 1==2
Then response
Then match XXXXXXX
The match XXXX
The steps fails Assert , remain steps does not execute. Is there any way even my assert fails remaining steps can continue the process
This is the expected behavior.
But you can use the karate.match() function to perform the assert manually. Then you can use conditional logic to decide if you want to continue next steps or not. But I totally don't recommend this.
For example:
* def temp = karate.match(actual, expected)
* print 'some step'
* assert temp.pass
I am using Cucumber JVM with Groovy and using try catch block in assertions like:
try {
assert response.status == 200
tr.pass("Status is 200")
} catch(AssertionError err) {
tr.fail(err.message)
}
Now, if somehow assertion fails then it will catch the exception and not stop the scenario at that time and start executing next step in same scenario. I have 2 choice:
1. after try catch I put my assertion again
2. Add some code in catch block which abort my scenario and proceed with next scenario.
I am preferring option 2. Please help me out.
Not tr.pass and tr.fail are user defined function to write status,
i think it is doing already if you do not use try catch in step definition. You could use assertion in step definition.
assertTrue(response.status == 200);
If it fails than stop scenario and skip other step (When,And and Then) and move on other scenario.