We have a system that we use in order to make some PUT requests during tests. The issue is that this system has, at the moment, some issues with its resources and sometimes it can't answer in time. We can't increase the read timeout to a value greater than 60 seconds for 2 reasons: if the system doesn't answer fast it most likely won't answer at all and secondly we're using a proxy which times out at 60 seconds.
I'm aware of this question , but waitForHttp or waitForPort don't fit our requirements as the port is available and there is no health endpoint telling us if the PUT request will succeed or just be lost.
Is there anything else we can do?
Example feature:
Background:
* url dep_url
* configure headers = read('classpath:package/headers/headers.json')
* retry until responseStatus !== 429
Scenario: Create entity
Given request read('classpath:package/body/' + file)
When path '/entity/' + entity[index]
And method put
Then assert responseStatus == 200 || responseStatus == 201
We need retries when a SocketTimeoutException is thrown, for example:
java.net.SocketTimeoutException: Read timed out, http call failed after 253 milliseconds for url
Well, try polling then: https://stackoverflow.com/a/56799845/143475
And combine that approach with the fact that you can use a JS try-catch in Karate: https://stackoverflow.com/a/67024149/143475
So maybe something like this:
* def failed = false
* eval try { karate.call('called.feature') } catch (e) { karate.set('failed', true) }
You should be able to figure out a solution now. Do post what you come up with.
EDIT: for those who feel bad about calling a second feature file, you can do everything in one file like this:
Feature:
Scenario:
* call read('#called')
#ignore #called
Scenario:
* print 'called'
Related
I am using Karate to test other people's implementations of an API. When checking response status codes I often need to accept more than one response. For example in response to a PUT I may see 200, 201, 205 - they are equally valid. I know I can use
Then assert responseStatus >= 200 && responseStatus < 300
to check for success but the shortcut really helps readability of the tests.
Would you consider an enhancement to the language to support response classes such as:
success (meaning 200-299)
redirect (meaning 300-399)
fail (meaning 400-499)
2xx
3xx
4xx
If I were to look at submitting a PR for this would you agree it is useful and would you have a preferred mechanism? Would these classes be best as parsed symbols or strings that force a different match to be implemented when it detects the status is not a number?
Yes, my first reaction is not to add a new keyword. Also to be honest, this seems to be a rare requirement - never had this ask before, I guess API testing would generally mean predictable responses.
My proposal is that you can write a custom function:
* def statusSuccess = function(){ var status = karate.get('responseStatus'); return status >= 200 && status < 300 }
* url 'https://httpbin.org'
* path 'status', 200
* method get
* assert statusSuccess()
EDIT - also see this: https://twitter.com/KarateDSL/status/1364433453412851714
We have a system that we use in order to make some PUT requests during tests. The issue is that this system has, at the moment, some issues with its resources and sometimes it can't answer in time. We can't increase the read timeout to a value greater than 60 seconds for 2 reasons: if the system doesn't answer fast it most likely won't answer at all and secondly we're using a proxy which times out at 60 seconds.
I'm aware of this question , but waitForHttp or waitForPort don't fit our requirements as the port is available and there is no health endpoint telling us if the PUT request will succeed or just be lost.
Is there anything else we can do?
Example feature:
Background:
* url dep_url
* configure headers = read('classpath:package/headers/headers.json')
* retry until responseStatus !== 429
Scenario: Create entity
Given request read('classpath:package/body/' + file)
When path '/entity/' + entity[index]
And method put
Then assert responseStatus == 200 || responseStatus == 201
We need retries when a SocketTimeoutException is thrown, for example:
java.net.SocketTimeoutException: Read timed out, http call failed after 253 milliseconds for url
Well, try polling then: https://stackoverflow.com/a/56799845/143475
And combine that approach with the fact that you can use a JS try-catch in Karate: https://stackoverflow.com/a/67024149/143475
So maybe something like this:
* def failed = false
* eval try { karate.call('called.feature') } catch (e) { karate.set('failed', true) }
You should be able to figure out a solution now. Do post what you come up with.
EDIT: for those who feel bad about calling a second feature file, you can do everything in one file like this:
Feature:
Scenario:
* call read('#called')
#ignore #called
Scenario:
* print 'called'
We have a system that we use in order to make some PUT requests during tests. The issue is that this system has, at the moment, some issues with its resources and sometimes it can't answer in time. We can't increase the read timeout to a value greater than 60 seconds for 2 reasons: if the system doesn't answer fast it most likely won't answer at all and secondly we're using a proxy which times out at 60 seconds.
I'm aware of this question , but waitForHttp or waitForPort don't fit our requirements as the port is available and there is no health endpoint telling us if the PUT request will succeed or just be lost.
Is there anything else we can do?
Example feature:
Background:
* url dep_url
* configure headers = read('classpath:package/headers/headers.json')
* retry until responseStatus !== 429
Scenario: Create entity
Given request read('classpath:package/body/' + file)
When path '/entity/' + entity[index]
And method put
Then assert responseStatus == 200 || responseStatus == 201
We need retries when a SocketTimeoutException is thrown, for example:
java.net.SocketTimeoutException: Read timed out, http call failed after 253 milliseconds for url
Well, try polling then: https://stackoverflow.com/a/56799845/143475
And combine that approach with the fact that you can use a JS try-catch in Karate: https://stackoverflow.com/a/67024149/143475
So maybe something like this:
* def failed = false
* eval try { karate.call('called.feature') } catch (e) { karate.set('failed', true) }
You should be able to figure out a solution now. Do post what you come up with.
EDIT: for those who feel bad about calling a second feature file, you can do everything in one file like this:
Feature:
Scenario:
* call read('#called')
#ignore #called
Scenario:
* print 'called'
I am using Karate to test other people's implementations of an API. When checking response status codes I often need to accept more than one response. For example in response to a PUT I may see 200, 201, 205 - they are equally valid. I know I can use
Then assert responseStatus >= 200 && responseStatus < 300
to check for success but the shortcut really helps readability of the tests.
Would you consider an enhancement to the language to support response classes such as:
success (meaning 200-299)
redirect (meaning 300-399)
fail (meaning 400-499)
2xx
3xx
4xx
If I were to look at submitting a PR for this would you agree it is useful and would you have a preferred mechanism? Would these classes be best as parsed symbols or strings that force a different match to be implemented when it detects the status is not a number?
Yes, my first reaction is not to add a new keyword. Also to be honest, this seems to be a rare requirement - never had this ask before, I guess API testing would generally mean predictable responses.
My proposal is that you can write a custom function:
* def statusSuccess = function(){ var status = karate.get('responseStatus'); return status >= 200 && status < 300 }
* url 'https://httpbin.org'
* path 'status', 200
* method get
* assert statusSuccess()
EDIT - also see this: https://twitter.com/KarateDSL/status/1364433453412851714
Having some stunning success with Karate. Working on end-to-end tests that use 'retry until' timeout on a GETs in order to wait for a specific parameter value within the response body. The parameter is expected to change state from A to B as data processing completes in the system under test. Interested to learn about patterns due to having standardised on a model of one .feature per API route. However this is only possible if we can parameterise the retry until terms. Otherwise this would mean writing multiple features to support different retry combinations.
---Example of reusing retry until from comments---
To maintain a single get_notification_ref.feature rather than one for each until combination, provide external until parameters in the call that can be used by the retry within the .feature.
Implementation that relies on specifying the until parameters inside the .feature file. End up with GET Notification feature file for each retry combination :
Scenario: Get notification & wait for status
* call read('classpath:NotifyV1/get_notification_ref_wait_status.feature')
.
.
Scenario: Get notification & wait for status indicator colour
* def expectedColour = 'GREEN'
* call read('classpath:NotifyV1/get_notification_ref_wait_colour.feature')
get_notification_ref_wait_status.feature
Scenario: Get notification and wait on response status = 200
Given path 'notification', notificationTypeReference
And retry until responseStatus == 200
When method get
* def notificationResponse = $
get_notification_ref_wait_colour.feature
Scenario: Get notification and wait on response status = 200 and colour
Given path 'notification', notificationTypeReference
And retry until responseStatus == 200 && response.statusColour == expectedColour
When method get
* def notificationResponse = $
Implementation of the above that can handle parameterised retry until's would look like this - note there is now only one GET Notification feature file:
Scenario: Get notification & wait for status 200
* call read('classpath:NotifyV1/get_notification_ref.feature')
.
.
Scenario: Get notification & wait for status 200 and indicator colour
* def UntilTerm = function(response){ return karate.match(response, '{statusColour: "GREEN"}').pass }
* call read('classpath:NotifyV1/get_notification_ref.feature')
get_notification_ref.feature
Scenario: Get notification
* def untilTerm = karate.get('UntilTerm') ? UntilTerm : function(response){ return true }
* def untilStatus = karate.get('UntilStatus') ? UntilStatus : 200
Given path 'notification', notificationTypeReference
And retry until responseStatus == untilStatus && untilTerm(response)
When method get
* def notificationResponse = $
* karate.set('UntilTerm',null)
* karate.set('UntilStatus',null)
I would say retry until may be sufficient. Since you can tweak the default time and interval, you can set this differently even at the point of need, i.e. a specific HTTP call: https://github.com/intuit/karate#retry-until
Unless you really truly have a way for the external process to call back - and in that case you can look at karate.signal() and friends. Otherwise I think you are better off sticking to retry until.