This question already has an answer here:
Attaching screenshots to json report
(1 answer)
Closed 1 year ago.
I have a UI test where I want to capture the screenshot when the UI test fails. I have explored the driver.screenshot() functionality and it works well whenever I want to take a screenshot during the test. However I just want to capture the screenshot at the point when the test fails.
I had a look at the solution in the afterScenario hook mentioned in:
Karate UI standalone - can there be screenshots attached to reports on failure?
However this expects INFO.errorMessage to be present in the log files.
Here is the sample of what we see in my karate.log file and I cannot find the error being logged in under INFO.errorMessage. The structure is slightly different:
09:49:22.192 [ForkJoinPool-1-worker-3] DEBUG c.intuit.karate.driver.DriverOptions - >> {"method":"Runtime.evaluate","params":{"expression":"document.evaluate(\"\/\/div[#role='option']\", document, null, 9, null).singleNodeValue.click()","returnByValue":true},"id":361}
09:49:22.200 [nioEventLoopGroup-2-1] DEBUG c.intuit.karate.driver.DriverOptions - << {"id":361,"result":{"result":{"type":"object","subtype":"error","className":"TypeError","description":"TypeError: Cannot read property 'click' of null\n at <anonymous>:1:84","objectId":"{\"injectedScriptId\":2,\"id\":3}"},"exceptionDetails":{"exceptionId":2,"text":"Uncaught","lineNumber":0,"columnNumber":83,"scriptId":"19","stackTrace":{"callFrames":[{"functionName":"","scriptId":"19","url":"","lineNumber":0,"columnNumber":83}]},"exception":{"type":"object","subtype":"error","className":"TypeError","description":"TypeError: Cannot read property 'click' of null\n at <anonymous>:1:84","objectId":"{\"injectedScriptId\":2,\"id\":4}"}}}}
09:49:22.201 [nioEventLoopGroup-2-1] WARN c.intuit.karate.driver.DriverOptions - devtools error: [id: 361, result: [type: MAP, value: {type=object, subtype=error, className=TypeError, description=TypeError: Cannot read property 'click' of null
at <anonymous>:1:84, objectId={"injectedScriptId":2,"id":3}}]]
09:49:22.202 [ForkJoinPool-1-worker-3] ERROR c.intuit.karate.driver.DriverOptions - js eval failed twice:document.evaluate("//div[#role='option']", document, null, 9, null).singleNodeValue.click(), error: {"type":"object","subtype":"error","className":"TypeError","description":"TypeError: Cannot read property 'click' of null\n at <anonymous>:1:84","objectId":"{\"injectedScriptId\":2,\"id\":3}"}
**09:49:22.207 [ForkJoinPool-1-worker-3] ERROR com.intuit.karate - feature call failed: classpath:AMUI/EndToEndTests/CreateAgreement.feature
arg: {"shared_agrname":"TEST_57059","local_agrname":"TEST_57059","agr_type":"CSA"}
CreateAgreement.feature:76 - evaluation (js) failed: click("//div[#role='option']"), java.lang.RuntimeException: js eval failed twice:document.evaluate("//div[#role='option']", document, null, 9, null).singleNodeValue.click(), error: {"type":"object","subtype":"error","className":"TypeError","description":"TypeError: Cannot read property 'click' of null\n at <anonymous>:1:84","objectId":"{\"injectedScriptId\":2,\"id\":3}"}
stack trace: com.intuit.karate.driver.DevToolsDriver.eval(DevToolsDriver.java:300)**
09:49:22.596 [pool-1-thread-1] INFO com.intuit.karate.Runner - <<fail>> feature 41 of 42: classpath:AMUI/EndToEndTests/CSA.feature
Looking at the above, I modified the afterScenario hook to:
* configure afterScenario = function(){ if (karate.ERROR.arg) driver.screenshot() }
but that does not help. It does not take the screenshot.
Any suggestions here would be very helpful.
Also is this expected in the log file? Not having UI errors captured under the info.errorMessage structure? I have not seen any errors captured in this structure so far in my tests. Does the framework capture it this way or is it the application under test that is responsible for it?
karate.info.errorMessage has nothing to do with log-levels, read the example please: hooks.info
Related
I am investigating exponential increase in JAVA heap size when executing complex scenarios especially with multiple reusable scenarios. This is my attempt to troubleshoot the issue with simple example and possible explanation to JVM heap usage.
Environment: Karate 1.1.0.RC4 | JDK 14 | Maven 3.6.3
Example: Download project, extract and execute maven command as per READEME
Observation: As per following example, if we call same scenario multiple times, response object grows exponentially since it includes response from previous called scenario along with copies of global variables.
#unexpected
Scenario: Not over-writing nested variable
* def response = call read('classpath:examples/library.feature#getLibraryData')
* string out = response
* def resp1 = response.randomTag
* karate.log('FIRST RESPONSE SIZE = ', out.length)
* def response = call read('classpath:examples/library.feature#getLibraryData')
* string out = response
* def resp2 = response.randomTag
* karate.log('SECOND RESPONSE SIZE = ', out.length)
Output:
10:26:23.863 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 9 by tag: getLibraryData
10:26:23.875 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 14 by tag: libraryData
10:26:23.885 [main] INFO com.intuit.karate - FIRST RESPONSE SIZE = 331
10:26:23.885 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 9 by tag: getLibraryData
10:26:23.894 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 14 by tag: libraryData
10:26:23.974 [main] INFO com.intuit.karate - SECOND RESPONSE SIZE = 1783
10:26:23.974 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 9 by tag: getLibraryData
10:26:23.974 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 14 by tag: libraryData
10:26:23.988 [main] INFO com.intuit.karate - THIRD RESPONSE SIZE = 8009
Do we really need to include response and global variables in the response of called feature file (non-shared scope)?
When we read large json file and call multiple reusable scenario files, each time copy of read json data gets added to response object. Is there way to avoid this behavior?
Is there a better way to script complex test using reusable scenarios without having multiple copies of same variables?
Okay, can you look at this issue:
https://github.com/intuit/karate/issues/1675
I agree we can optimize the response and global variables. Would be great if you can contribute code.
This question already has an answer here:
KarateDSL UI Testing - Friendly Locators not working
(1 answer)
Closed 2 years ago.
I am studying KarateUI possibilities. And I tried to use drag and drop functionality of framework.
I used a page with draggable elements https://www.seleniumeasy.com/test/drag-and-drop-demo.html and my script does not work on it. What is wrong with my script? Here it is:
mouse().move('{div/span}Draggable 1').down().move('#mydropzone').up()
And i also see in console of IDE next log
16:11:40.196 [ForkJoinPool-1-worker-1] DEBUG c.intuit.karate.driver.DriverOptions - >> {"method":"Input.dispatchMouseEvent","params":{"type":"mouseMoved","x":31,"y":820},"id":16}
16:11:40.200 [nioEventLoopGroup-2-1] DEBUG c.intuit.karate.driver.DriverOptions - << {"id":16,"result":{}}
16:11:40.203 [ForkJoinPool-1-worker-1] DEBUG c.intuit.karate.driver.DriverOptions - >> {"method":"Input.dispatchMouseEvent","params":{"type":"mousePressed","x":31,"y":820,"button":"left","clickCount":1},"id":17}
16:11:40.234 [nioEventLoopGroup-2-1] DEBUG c.intuit.karate.driver.DriverOptions - << {"id":17,"result":{}}
16:11:40.234 [ForkJoinPool-1-worker-1] DEBUG c.intuit.karate.driver.DriverOptions - >> {"method":"Input.dispatchMouseEvent","params":{"type":"mouseMoved","x":231,"y":827},"id":18}
16:11:40.242 [nioEventLoopGroup-2-1] DEBUG c.intuit.karate.driver.DriverOptions - << {"id":18,"result":{}}
16:11:40.242 [ForkJoinPool-1-worker-1] DEBUG c.intuit.karate.driver.DriverOptions - >> {"method":"Input.dispatchMouseEvent","params":{"type":"mouseReleased","x":231,"y":827,"button":"left","clickCount":1},"id":19}
16:11:40.250 [nioEventLoopGroup-2-1] DEBUG c.intuit.karate.driver.DriverOptions - << {"id":19,"result":{}}
Drag and drop is actually quite hard to get right, so I recommend doing this via JavaScript. Executing JS is actually quite easy using Karate:
* driver 'https://www.seleniumeasy.com/test/drag-and-drop-demo.html'
* script("var myDragEvent = new Event('dragstart'); myDragEvent.dataTransfer = new DataTransfer()")
* waitFor('{}Draggable 1').script("_.dispatchEvent(myDragEvent)")
* script("var myDropEvent = new Event('drop'); myDropEvent.dataTransfer = myDragEvent.dataTransfer")
* script('#mydropzone', "_.dispatchEvent(myDropEvent)")
* screenshot()
So with a little bit of awareness of some of the internals - e.g. the HTML5 DataTransfer API - you can do pretty much anything. I think "bending the rules" in cases like this is fine when it comes to automating complex E2E user-interactions in a browser.
You can of course wrap the drag-and-drop into a re-usable function in Karate, just keep in mind that "DOM JS" is sent to the browser as plain-text.
Refer the docs: https://github.com/intuit/karate/tree/master/karate-core#function-composition
EDIT: for those looking for other examples of using JS on the DOM:
https://stackoverflow.com/a/60618233/143475
https://stackoverflow.com/a/61478834/143475
https://stackoverflow.com/a/66677401/143475
https://stackoverflow.com/a/67701399/143475
https://stackoverflow.com/a/67629911/143475
When running the following scenario, the tests finish running but execution hangs immediately after and the gradle test command never finishes. The cucumber report isn't built, so it hangs before that point.
It seems to be caused by having 2 call read() to different scenarios, that both call a third scenario. That third scenario references the parent context to inspect the current request.
When that parent request is stored in a variable the tests hang. When that variable is cleared before leaving that third scenario, the test finishes as normal. So something about having a reference to that context hangs the tests at the end.
Is there a reason this doesn't complete? Am I missing some important code that lets the tests finish?
I've added * def currentRequest = {} at the end of the special-request scenario and that allows the tests to complete, but that seems like a hack.
This is the top-level test scenario:
Scenario: Updates user id
* def user = call read('utils.feature#endpoint=create-user')
* set user.clientAccountId = user.accountNumber + '-test-client-account-id'
* call read('utils.feature#endpoint=update-user') user
* print 'the test is done!'
The test scenario calls 2 different scenarios in the same utls.feature file
utils.feature:
#ignore
Feature: /users
Background:
* url baseUrl
#endpoint=create-user
Scenario: create a standard user for a test
Given path '/create'
* def restMethod = 'post'
* call read('special-request.feature')
When method restMethod
Then status 201
#endpoint=update-user
Scenario: set a user's client account ID
Given path '/update'
* def restMethod = 'put'
* call read('special-request.feature')
When method restMethod
Then status 201
And match response == {"status":"Success", "message":"Update complete"}
Both of the util scenarios call the special-request feature with different parameters/requests.
special-request.feature:
#ignore
Feature: Builds a special
Scenario: special-request
# The next line causes the test to sit for a long time
* def currentRequest = karate.context.parentContext.getRequest()
# Without the below clear of currentRequest, the test never finishes
# We are de-referencing the parent context's request allows test to finish
* def currentRequest = {}
without currentRequest = {} these are the last lines of output I get before the tests seem to stop.
12:21:38.816 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - response time in milliseconds: 8.48
1 < 201
1 < Content-Type: application/json
{
"status": "Success",
"message": "Update complete"
}
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.818 [ForkJoinPool-1-worker-1] INFO com.intuit.karate - [print] the test is done!
12:21:38.818 [pool-1-thread-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
<==========---> 81% EXECUTING [39s]
With currentRequest = {}, the test completes and the cucumber report generates successfully which is what I would expect to happen even without that line.
Two comments:
* karate.context.parentContext.getRequest()
Wow, these are internal API-s not intended for users to use, I would strongly advise passing values around as variables instead. So all bets are off if you have trouble with that.
It does sound like you have a null-pointer in the above (no surprises here).
There is a bug in 0.9.4 that causes test failures in some edge cases such as the things you are doing, pre-test life-cycle or failures in karate-config.js to hang the parallel runner. You should see something in the logs that indicates a failure, if not - do try help us replicate this problem.
This should be fixed in the develop branch, so you could help if you can build from source and test locally. Instructions are here: https://github.com/intuit/karate/wiki/Developer-Guide
And if you still see a problem, please do this: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
I have a keyword called "debug teardown" which prints the test status and then runs the debuglibrary Debug keyword, if the test has failed.
I would like to be able to log to console which keyword has caused the failure, so I can more effectively debug my test.
Is it possible to get the stack trace or most recent test keyword, and log it to the console?
Here is my Debug Teardown keyword:
Debug Teardown
Run Keyword If Test Failed Log ${TEST STATUS}: ${TEST MESSAGE} ERROR
Run Keyword If Test Failed Debug
You can get a bit more information if you create a listener and also set the log level to DEBUG. Inside the listener you can save the results of log commands, and then when a keyword fails you can print it out or do whatever else you want.
For example, here's a listener that will print to stdout the last log message when a keyword fails:
from __future__ import print_function
class my_listener():
ROBOT_LISTENER_API_VERSION = 2
def __init__(self):
self.ROBOT_LIBRARY_LISTENER = self
self.last_log = None
def _log_message(self, message):
self.last_log = message
def _end_keyword(self, name, attrs):
if attrs['status'] == 'FAIL':
print("\n******\n", self.last_log['message'])
You would use it by importing the listener like a normal library, and also setting the log level to DEBUG (otherwise you'll get the error but no stack trace).
Example:
*** Settings ***
Library my_listener
Suite Setup set log level DEBUG
*** Test cases ***
Example
some keyword
You might be able to utilize set suite variable to update a "global" variable as you go. The last set variable value would be the value that failed.
I am having an issue with NSURLSessionDataTask , while trying to upload a JSON object , I am getting the following message in console .
[] __tcp_connection_write_eof_block_invoke Write close callback received
[error: [89] Operation canceled]
How to solve this issue ?
May be it is related to OS_ACTIVITY_MODE settings for Schemes.
disable OS_ACTIVITY_MODE mode by following steps and check:
--- Go in Product ---> Scheme ---> Edit Scheme
--- in Run Section on the left, select Argument Tab and in Environment Variable Change value as below:
Name: OS_ACTIVITY_MODE
Value: disable