Is it possible to evaluate an error response from a load balancer to replay a previous step using a conditional goto? For example the request may come back as below:
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
</body>
</html>
It is not possible to use namespace in the above example. Could anyone please advise if this is possible I.e. by searching a certain phrase in the message, without having to write a custom groovy script?
A Groovy test step may be your only option. If you have assertions on your test step checking for valid responses, receiving a 502 or other error will short-circuit your test case and you won't get to the conditional step.
The following is a simple Groovy test step that looks for "502 Bad Gateway" anywhere in the response of the test step SOAP Request and executes the test step again:
def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
def holder = groovyUtils.getXmlHolder( "SOAP Request#Response" )
// Check for a 502 Bad Gateway error
if (holder["//text()[contains(.,'502 Bad Gateway')]"]) {
log.info('Got 502 error... retrying the test step')
def testCase = testRunner.testCase
def testStep = testCase.getTestStepByName("SOAP Request");
testStepContext = new com.eviware.soapui.impl.wsdl.testcase.WsdlTestRunContext(testStep);
testStep.run(testRunner, testStepContext);
} else {
log.info('Running normal assertions...')
}
But, there's a bit of thinking that needs to go into retries: if you retry a request immediately you'll probably get the same response, so you'd likely need to build in a delay. And, how many times should you retry before giving up, and what about other error types? One advantage of using a Groovy test step is you can call on lots of Java libraries that handle retries, such as Guava.
Related
Consider a simple application where a user fills a form to divide two numbers, in the routes the form data is proceeded [made into float] and then passed as parameters to a python script's function that has the division logic.
The logic fails due to division by 0 is handled as a custom message in the terminal. How does one send this custom message back to the front end UI along with a 500 error message? Trying to make a restful flask app here.
So far I can abort and show a custom message but not the one that propagated from the backend. Also looked into custom error handling but I want to writer of the external python script to be able to write the custom message.
You can Flask errorhandler(errorcode) to manage your errors and display those on the frontend.
#app.errorhandler(500)
def code_500(error):
return render_template("errors/500.html", error=error), 500
You can put whatever else you want in the html template.
You can also call the code_500(error) func directly.
Same principle applies for any other HTTP error code if you want to customize the page and the message (401, 403, 404, etc...).
If you're inside a blueprint, you can use app_errorhandler instead.
You could use the abort() function. From the docs:
When using Flask for web APIs, you can use the same techniques as above to return JSON responses to API errors. abort() is called with a description parameter. The errorhandler() will use that as the JSON error message, and set the status code to 404.
You could implement it like this
#app.route("/div")
def divide():
x, y = request.form['x'], request.form['y']
try:
result = x / y
except ZeroDivisionError:
abort(400, description="Your message here")
else:
# Proper response
From there, the important step is properly catching that message on your frontend.
Say for example, I created a flow for scatter-gather and I want to check if all endpoints are returning same result status code 200 or throw an error if not.
Configure the Response Validator (General > Response > Response Validator) for each HTTP Request so only 200..299 responses are considered valid.
You can use try block for every HTTP request on wrap whole scatter gather. If one fails, capture HTTP status code in on error propogate and log the results.
I suggest you wrap each request into try block, if you already have a global error handler defined, it should pick up status code 500 etc. Otherwise, capture response code into dataweave
I run some load tests (all endpoints) and we do have a known issue in our code: if multiple POST requests are sent in the same time we do get a duplicate error based on a timestamp field in our database.
All I want to do is to count timeouts (based on the message received "Service is not available. Request timeout") in a variable and accept this as a normal behavior (don't fail the tests).
For now I've added a Response Assertion for this (in order to keep the tests running) but I cannot tell if or how many timeout actually happen.
How can I count this?
Thank you
I would recommend doing this as follows:
Add JSR223 Listener to your Test Plan
Put the following code into "Script" area:
if (prev.getResponseDataAsString().contains('Service is not available. Request timeout')) {
prev.setSampleLabel('False negative')
}
That's it, if sampler will contain Service is not available. Request timeout in the response body - JMeter will change its title to False negative.
You can even mark it as passed by adding prev.setSuccessful(false) line to your script. See Apache Groovy - Why and How You Should Use It article fore more information on what else you can do with Groovy in JMeter tests
If you just need to find out the count based on the response message then you can save the performance results in a csv file using simple data writer (configure for csv only) and then filter csv based on the response message to get the required count. Or you can use Display only "errors" option to get all the errors and then filter out based on the expected error message.
If you need to find out at the runtime then you can use aggregate report listener and use "Errors" checkbox to get the count of failure but this will include other failures also.
But, if you need to get the count at the run time to use it later then it is a different case. I am assuming that it is not the case.
Thanks,
I have a Java EE application that processes a bunch of messages from different interfaces. (Some of the) functional tests are performed with SoapUI.
In one of the cases I have created a SOAP VirtResponse step that receives the output of my application and checks the values in the received message. The test has the following steps:
Datasource step to load input and expected output (multiple scenarios)
JMS step to send input to application on a specific interface.
SOAP step to receive application output on another interface (no validation).
Groovy script to check results (e.g. no message received or message received with specific values). See below for the script, couldn't get it to work in between the list items.
Datasource loop to step 2.
There is a scenario (well, there are more scenarios) in which the input should not generate an output. I want to check if my application did not send an output in such a scenario.
Strategy 1:
I have added a fourth groovy step in which I validate that the result of step 3 is an empty string. To make the test pass, I had to disable the checkbox in TestCase Options that says "Fail TestCase on Error". This works in cases of happy execution of tests. However if an error does occur (e.g. the application did send a response when it was not supposed to or the application send a wrong response), the entire TestCase is set to passed (because of the checkbox) and only the specific step deep down in the logs is failed. This makes it hard to see the results of the entire test suite.
Attempted strategy 2:
Started out by adding a conditional test step that will skip step 3 based on the input. However that way I no longer validate if my application does not send a message when it is not supposed to.
What is the best way to check these kinds of scenarios?
EDITS:
The entire testcase should fail if one of the scenarios from the datasource fails. (It is not a problem if this means that some scenarios were not evaluated yet)
Groovy script:
// Get flag from datasource that indicates if a message should be received
def soapBerichtOntvangen = context.expand('${DataSourceUISBerichten#SoapBerichtOntvangen}' );
// Get the message from the previous step.
def receivedSoapRequest = context.expand( '${SOAPVirtResponse#Request#declare namespace out=\'http://application/messageprocessing/outbound/out:SendOutboundMessage[1]/Message[1]}' )
// If we don't expect a message the flag is set to "N"
if(soapBerichtOntvangen=="N"){
assert(receivedSoapRequest=="")
} else if(receivedSoapRequest!=null && receivedSoapRequest!=""){
def slurpedMessage = new XmlSlurper().parseText(receivedSoapRequest)
def messageType=slurpedMessage.MessageHeader.MessageReference.MessageType
// Get expected values from context
def verwachtMessageType = context.expand('${DataSourceOutboundCIBerichten#messageType}' )
assert(String.valueOf(messageType)==verwachtMessageType)
} else {
// Should have received a message, but none came up.
assert(false)
}
I am new to jmeter and am working on putting together a test plan. The hurdle I've encountered is as follows:
First, a POST is made to processForm.aspx
Then, the user is redirected to pleaseWait.aspx
This page either redirects immediately to results.aspx OR loads, with a META REFRESH tag set to refresh in 5 seconds (and this step is repeated).
Now -- I can get this to execute by doing the following:
HTTP Sampler POST to processForm.aspx
Assert Response contains "<something on pleaseWait.aspx>"
While LAST
HTTP Sampler GET to pleaseWait.aspx
Assert Response contains "<something on results.aspx>"
However -- I don't care for this method, because it results in failed assertions (even though things are working as expected). I am sure there must be some other way to do this? Anyone more familiar with JMeter than I?
UPDATE:
Got it going by using Regular Expression Extractor instead of Assertions.
1) Add a User Defined Variables section at Test Plan Root
2) Add a variable to it "LoginWait" and "false"
HTTP Sampler POST to processForm.aspx
RegEx Extract Response Body contains "<something on pleaseWait.aspx>" into LoginWait
While ${LoginWait}
HTTP Sampler GET to pleaseWait.aspx
RegEx Extract Response Body contains "<something on pleaseWait.aspx>" into LoginWait
...
You could try using "follow redirects" on your HTTP Request. It would eliminate the logic you need, and still get you to the page you're going.