Feature: Principal feature
Background:
* url 'http://example.com'
Scenario: Feature calling
* def inputTable = call read('input_table.feature')
* call read('my_call.feature') inputTable.inputTestData
where the data table file is:
//input_table.feature
Feature:TABLE_TEST
Scenario:TABLE_TEST
* table inputTestData
|inputName|outputName|
|requestA|responseA|
this throw me an error of:
ERROR com.intuit.karate - feature call failed: .../input_table.feature
arg: null
input_table.feature:3 - evaluation (js) failed: requestA, javax.script.ScriptException: ReferenceError: "requestA" is not defined in <eval> at line number 1
stack trace: jdk.nashorn.api.scripting.NashornScriptEngine.throwAsScriptException(NashornScriptEngine.java:470)
but instead if i call the my_call feature with the data table defined inside in the Examples field it works correctly.
Any help?
There's a subtle difference between Examples and table. Maybe you meant to do this:
* table inputTestData
|inputName|outputName|
|'requestA'|'responseA'|
Read the docs: https://github.com/intuit/karate#table
Related
I am investigating exponential increase in JAVA heap size when executing complex scenarios especially with multiple reusable scenarios. This is my attempt to troubleshoot the issue with simple example and possible explanation to JVM heap usage.
Environment: Karate 1.1.0.RC4 | JDK 14 | Maven 3.6.3
Example: Download project, extract and execute maven command as per READEME
Observation: As per following example, if we call same scenario multiple times, response object grows exponentially since it includes response from previous called scenario along with copies of global variables.
#unexpected
Scenario: Not over-writing nested variable
* def response = call read('classpath:examples/library.feature#getLibraryData')
* string out = response
* def resp1 = response.randomTag
* karate.log('FIRST RESPONSE SIZE = ', out.length)
* def response = call read('classpath:examples/library.feature#getLibraryData')
* string out = response
* def resp2 = response.randomTag
* karate.log('SECOND RESPONSE SIZE = ', out.length)
Output:
10:26:23.863 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 9 by tag: getLibraryData
10:26:23.875 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 14 by tag: libraryData
10:26:23.885 [main] INFO com.intuit.karate - FIRST RESPONSE SIZE = 331
10:26:23.885 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 9 by tag: getLibraryData
10:26:23.894 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 14 by tag: libraryData
10:26:23.974 [main] INFO com.intuit.karate - SECOND RESPONSE SIZE = 1783
10:26:23.974 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 9 by tag: getLibraryData
10:26:23.974 [main] INFO c.intuit.karate.core.FeatureRuntime - scenario called at line: 14 by tag: libraryData
10:26:23.988 [main] INFO com.intuit.karate - THIRD RESPONSE SIZE = 8009
Do we really need to include response and global variables in the response of called feature file (non-shared scope)?
When we read large json file and call multiple reusable scenario files, each time copy of read json data gets added to response object. Is there way to avoid this behavior?
Is there a better way to script complex test using reusable scenarios without having multiple copies of same variables?
Okay, can you look at this issue:
https://github.com/intuit/karate/issues/1675
I agree we can optimize the response and global variables. Would be great if you can contribute code.
This question already has an answer here:
Attaching screenshots to json report
(1 answer)
Closed 1 year ago.
I have a UI test where I want to capture the screenshot when the UI test fails. I have explored the driver.screenshot() functionality and it works well whenever I want to take a screenshot during the test. However I just want to capture the screenshot at the point when the test fails.
I had a look at the solution in the afterScenario hook mentioned in:
Karate UI standalone - can there be screenshots attached to reports on failure?
However this expects INFO.errorMessage to be present in the log files.
Here is the sample of what we see in my karate.log file and I cannot find the error being logged in under INFO.errorMessage. The structure is slightly different:
09:49:22.192 [ForkJoinPool-1-worker-3] DEBUG c.intuit.karate.driver.DriverOptions - >> {"method":"Runtime.evaluate","params":{"expression":"document.evaluate(\"\/\/div[#role='option']\", document, null, 9, null).singleNodeValue.click()","returnByValue":true},"id":361}
09:49:22.200 [nioEventLoopGroup-2-1] DEBUG c.intuit.karate.driver.DriverOptions - << {"id":361,"result":{"result":{"type":"object","subtype":"error","className":"TypeError","description":"TypeError: Cannot read property 'click' of null\n at <anonymous>:1:84","objectId":"{\"injectedScriptId\":2,\"id\":3}"},"exceptionDetails":{"exceptionId":2,"text":"Uncaught","lineNumber":0,"columnNumber":83,"scriptId":"19","stackTrace":{"callFrames":[{"functionName":"","scriptId":"19","url":"","lineNumber":0,"columnNumber":83}]},"exception":{"type":"object","subtype":"error","className":"TypeError","description":"TypeError: Cannot read property 'click' of null\n at <anonymous>:1:84","objectId":"{\"injectedScriptId\":2,\"id\":4}"}}}}
09:49:22.201 [nioEventLoopGroup-2-1] WARN c.intuit.karate.driver.DriverOptions - devtools error: [id: 361, result: [type: MAP, value: {type=object, subtype=error, className=TypeError, description=TypeError: Cannot read property 'click' of null
at <anonymous>:1:84, objectId={"injectedScriptId":2,"id":3}}]]
09:49:22.202 [ForkJoinPool-1-worker-3] ERROR c.intuit.karate.driver.DriverOptions - js eval failed twice:document.evaluate("//div[#role='option']", document, null, 9, null).singleNodeValue.click(), error: {"type":"object","subtype":"error","className":"TypeError","description":"TypeError: Cannot read property 'click' of null\n at <anonymous>:1:84","objectId":"{\"injectedScriptId\":2,\"id\":3}"}
**09:49:22.207 [ForkJoinPool-1-worker-3] ERROR com.intuit.karate - feature call failed: classpath:AMUI/EndToEndTests/CreateAgreement.feature
arg: {"shared_agrname":"TEST_57059","local_agrname":"TEST_57059","agr_type":"CSA"}
CreateAgreement.feature:76 - evaluation (js) failed: click("//div[#role='option']"), java.lang.RuntimeException: js eval failed twice:document.evaluate("//div[#role='option']", document, null, 9, null).singleNodeValue.click(), error: {"type":"object","subtype":"error","className":"TypeError","description":"TypeError: Cannot read property 'click' of null\n at <anonymous>:1:84","objectId":"{\"injectedScriptId\":2,\"id\":3}"}
stack trace: com.intuit.karate.driver.DevToolsDriver.eval(DevToolsDriver.java:300)**
09:49:22.596 [pool-1-thread-1] INFO com.intuit.karate.Runner - <<fail>> feature 41 of 42: classpath:AMUI/EndToEndTests/CSA.feature
Looking at the above, I modified the afterScenario hook to:
* configure afterScenario = function(){ if (karate.ERROR.arg) driver.screenshot() }
but that does not help. It does not take the screenshot.
Any suggestions here would be very helpful.
Also is this expected in the log file? Not having UI errors captured under the info.errorMessage structure? I have not seen any errors captured in this structure so far in my tests. Does the framework capture it this way or is it the application under test that is responsible for it?
karate.info.errorMessage has nothing to do with log-levels, read the example please: hooks.info
When running the following scenario, the tests finish running but execution hangs immediately after and the gradle test command never finishes. The cucumber report isn't built, so it hangs before that point.
It seems to be caused by having 2 call read() to different scenarios, that both call a third scenario. That third scenario references the parent context to inspect the current request.
When that parent request is stored in a variable the tests hang. When that variable is cleared before leaving that third scenario, the test finishes as normal. So something about having a reference to that context hangs the tests at the end.
Is there a reason this doesn't complete? Am I missing some important code that lets the tests finish?
I've added * def currentRequest = {} at the end of the special-request scenario and that allows the tests to complete, but that seems like a hack.
This is the top-level test scenario:
Scenario: Updates user id
* def user = call read('utils.feature#endpoint=create-user')
* set user.clientAccountId = user.accountNumber + '-test-client-account-id'
* call read('utils.feature#endpoint=update-user') user
* print 'the test is done!'
The test scenario calls 2 different scenarios in the same utls.feature file
utils.feature:
#ignore
Feature: /users
Background:
* url baseUrl
#endpoint=create-user
Scenario: create a standard user for a test
Given path '/create'
* def restMethod = 'post'
* call read('special-request.feature')
When method restMethod
Then status 201
#endpoint=update-user
Scenario: set a user's client account ID
Given path '/update'
* def restMethod = 'put'
* call read('special-request.feature')
When method restMethod
Then status 201
And match response == {"status":"Success", "message":"Update complete"}
Both of the util scenarios call the special-request feature with different parameters/requests.
special-request.feature:
#ignore
Feature: Builds a special
Scenario: special-request
# The next line causes the test to sit for a long time
* def currentRequest = karate.context.parentContext.getRequest()
# Without the below clear of currentRequest, the test never finishes
# We are de-referencing the parent context's request allows test to finish
* def currentRequest = {}
without currentRequest = {} these are the last lines of output I get before the tests seem to stop.
12:21:38.816 [ForkJoinPool-1-worker-1] DEBUG com.intuit.karate - response time in milliseconds: 8.48
1 < 201
1 < Content-Type: application/json
{
"status": "Success",
"message": "Update complete"
}
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.817 [ForkJoinPool-1-worker-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
12:21:38.818 [ForkJoinPool-1-worker-1] INFO com.intuit.karate - [print] the test is done!
12:21:38.818 [pool-1-thread-1] DEBUG com.jayway.jsonpath.internal.path.CompiledPath - Evaluating path: $
<==========---> 81% EXECUTING [39s]
With currentRequest = {}, the test completes and the cucumber report generates successfully which is what I would expect to happen even without that line.
Two comments:
* karate.context.parentContext.getRequest()
Wow, these are internal API-s not intended for users to use, I would strongly advise passing values around as variables instead. So all bets are off if you have trouble with that.
It does sound like you have a null-pointer in the above (no surprises here).
There is a bug in 0.9.4 that causes test failures in some edge cases such as the things you are doing, pre-test life-cycle or failures in karate-config.js to hang the parallel runner. You should see something in the logs that indicates a failure, if not - do try help us replicate this problem.
This should be fixed in the develop branch, so you could help if you can build from source and test locally. Instructions are here: https://github.com/intuit/karate/wiki/Developer-Guide
And if you still see a problem, please do this: https://github.com/intuit/karate/wiki/How-to-Submit-an-Issue
Let's say if I am having a scenario like
Scenario: Call a Get API and validate the response
Given path 'myteam'
When method get
Then status 201
And print response
And match response contains { teamFeature: 'pick any feature'}
And my API response is
{
"id": "6c0377cd-96c9-4651-bcc8-0c9a7d962bc3",
"teamFeature": "pick any feature"
}
Then I am getting the error like
example.feature:19 - javascript evaluation failed: feature'}, :1:9 Missing close quote
feature'}
^ in at line number 1 at column number 9
If my API response does not contain the word 'any' and I change the match statement then it is working fine. Looks like I need to escape the the word 'any' somehow.
May I know how can I escape the word 'any'?
Not sure if this is a bug in Karate.
Tried to call
com.intuit.karate.Match match = new com.intuit.karate.Match("pick any feature");
System.out.println(match.contains("pick any feature"));
And received following error
Exception in thread "main" java.lang.RuntimeException: javascript
evaluation failed: pick any feature, :1:5 Expected ; but found
any pick any feature
^ in at line number 1 at column number 5 at com.intuit.karate.ScriptBindings.eval(ScriptBindings.java:152) at
com.intuit.karate.ScriptBindings.updateBindingsAndEval(ScriptBindings.java:142)
at
com.intuit.karate.ScriptBindings.evalInNashorn(ScriptBindings.java:127)
at com.intuit.karate.Script.evalJsExpression(Script.java:423) at
com.intuit.karate.Script.evalKarateExpression(Script.java:337) at
com.intuit.karate.Script.evalKarateExpression(Script.java:203) at
com.intuit.karate.Match.(Match.java:67) at
com.intuit.karate.Match.(Match.java:53)
Yes this is a bug in Karate, we've opened an issue: https://github.com/intuit/karate/issues/678
The workaround suggested by #BabuSekaran will work:
* def response = { foo: 'a any b' }
* def temp = { foo: 'a any b' }
* match response contains temp
I have a keyword called "debug teardown" which prints the test status and then runs the debuglibrary Debug keyword, if the test has failed.
I would like to be able to log to console which keyword has caused the failure, so I can more effectively debug my test.
Is it possible to get the stack trace or most recent test keyword, and log it to the console?
Here is my Debug Teardown keyword:
Debug Teardown
Run Keyword If Test Failed Log ${TEST STATUS}: ${TEST MESSAGE} ERROR
Run Keyword If Test Failed Debug
You can get a bit more information if you create a listener and also set the log level to DEBUG. Inside the listener you can save the results of log commands, and then when a keyword fails you can print it out or do whatever else you want.
For example, here's a listener that will print to stdout the last log message when a keyword fails:
from __future__ import print_function
class my_listener():
ROBOT_LISTENER_API_VERSION = 2
def __init__(self):
self.ROBOT_LIBRARY_LISTENER = self
self.last_log = None
def _log_message(self, message):
self.last_log = message
def _end_keyword(self, name, attrs):
if attrs['status'] == 'FAIL':
print("\n******\n", self.last_log['message'])
You would use it by importing the listener like a normal library, and also setting the log level to DEBUG (otherwise you'll get the error but no stack trace).
Example:
*** Settings ***
Library my_listener
Suite Setup set log level DEBUG
*** Test cases ***
Example
some keyword
You might be able to utilize set suite variable to update a "global" variable as you go. The last set variable value would be the value that failed.