I have a Java EE application that processes a bunch of messages from different interfaces. (Some of the) functional tests are performed with SoapUI.
In one of the cases I have created a SOAP VirtResponse step that receives the output of my application and checks the values in the received message. The test has the following steps:
Datasource step to load input and expected output (multiple scenarios)
JMS step to send input to application on a specific interface.
SOAP step to receive application output on another interface (no validation).
Groovy script to check results (e.g. no message received or message received with specific values). See below for the script, couldn't get it to work in between the list items.
Datasource loop to step 2.
There is a scenario (well, there are more scenarios) in which the input should not generate an output. I want to check if my application did not send an output in such a scenario.
Strategy 1:
I have added a fourth groovy step in which I validate that the result of step 3 is an empty string. To make the test pass, I had to disable the checkbox in TestCase Options that says "Fail TestCase on Error". This works in cases of happy execution of tests. However if an error does occur (e.g. the application did send a response when it was not supposed to or the application send a wrong response), the entire TestCase is set to passed (because of the checkbox) and only the specific step deep down in the logs is failed. This makes it hard to see the results of the entire test suite.
Attempted strategy 2:
Started out by adding a conditional test step that will skip step 3 based on the input. However that way I no longer validate if my application does not send a message when it is not supposed to.
What is the best way to check these kinds of scenarios?
EDITS:
The entire testcase should fail if one of the scenarios from the datasource fails. (It is not a problem if this means that some scenarios were not evaluated yet)
Groovy script:
// Get flag from datasource that indicates if a message should be received
def soapBerichtOntvangen = context.expand('${DataSourceUISBerichten#SoapBerichtOntvangen}' );
// Get the message from the previous step.
def receivedSoapRequest = context.expand( '${SOAPVirtResponse#Request#declare namespace out=\'http://application/messageprocessing/outbound/out:SendOutboundMessage[1]/Message[1]}' )
// If we don't expect a message the flag is set to "N"
if(soapBerichtOntvangen=="N"){
assert(receivedSoapRequest=="")
} else if(receivedSoapRequest!=null && receivedSoapRequest!=""){
def slurpedMessage = new XmlSlurper().parseText(receivedSoapRequest)
def messageType=slurpedMessage.MessageHeader.MessageReference.MessageType
// Get expected values from context
def verwachtMessageType = context.expand('${DataSourceOutboundCIBerichten#messageType}' )
assert(String.valueOf(messageType)==verwachtMessageType)
} else {
// Should have received a message, but none came up.
assert(false)
}
Related
I am trying to reuse a scenario in a feature file with call feature in the main feature file
the steps are as follows
main feature file calling the reusable features file with parameter input to the called scenario
ex:
def outputdata = call read('file:features/commonFeatures/commonFunctions.feature#customer') ({ 'customerId' : partyId })
Note: tried with karate.call as well, perhaps it does not provide the required outcome
and the customer id is passed as input to the called scenario of the feature file.
when i used the outputdata variable to get the output of the call scenario, not getting the response of API returned. getting polygot exception
could you please suggest me a way to get the response returned from a called scenario with parameterization
Is it possible to evaluate an error response from a load balancer to replay a previous step using a conditional goto? For example the request may come back as below:
<html>
<head><title>502 Bad Gateway</title></head>
<body bgcolor="white">
<center><h1>502 Bad Gateway</h1></center>
</body>
</html>
It is not possible to use namespace in the above example. Could anyone please advise if this is possible I.e. by searching a certain phrase in the message, without having to write a custom groovy script?
A Groovy test step may be your only option. If you have assertions on your test step checking for valid responses, receiving a 502 or other error will short-circuit your test case and you won't get to the conditional step.
The following is a simple Groovy test step that looks for "502 Bad Gateway" anywhere in the response of the test step SOAP Request and executes the test step again:
def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
def holder = groovyUtils.getXmlHolder( "SOAP Request#Response" )
// Check for a 502 Bad Gateway error
if (holder["//text()[contains(.,'502 Bad Gateway')]"]) {
log.info('Got 502 error... retrying the test step')
def testCase = testRunner.testCase
def testStep = testCase.getTestStepByName("SOAP Request");
testStepContext = new com.eviware.soapui.impl.wsdl.testcase.WsdlTestRunContext(testStep);
testStep.run(testRunner, testStepContext);
} else {
log.info('Running normal assertions...')
}
But, there's a bit of thinking that needs to go into retries: if you retry a request immediately you'll probably get the same response, so you'd likely need to build in a delay. And, how many times should you retry before giving up, and what about other error types? One advantage of using a Groovy test step is you can call on lots of Java libraries that handle retries, such as Guava.
I'd like to perform a while loop using jmeter. Within the loop I'm using xpath extract to pull information from the server response, and storing it in a variable. I'd like to quit the loop if that variable has any data in it (if the request has been successful) - otherwise I'd like to fail if it doesn't respond correctly in x number of attempts. Is this something that JMeter can do?
I found a solution for this
If you know the response that you are trying to extract from Xpath extractor,
With the help of response assertions and while loop its possible..
here is my answer
First of all add a beanshell sampler to the test plan before while loop.
In the beanshell sampler add the following 2 lines
vars.put("counter","1");
vars.put("txtFound","FALSE")
Next Add a While controller with the following condition
${__javaScript("${txtFound}" == "FALSE" && parseInt(${counter})<=3,)}
Above expression evaluates to true if both conditions are true.here 3 represents the number of attempts.
Now in the while loop add your request .
To the same request add a response assertion and add the pattern(the text you are trying to extract using Xpath)
to the same request add a beanshell post processor and copy the following code to it
int counter = Integer.parseInt(vars.get("counter"));
if(counter==3)
vars.put("txtFound","TRUE");
counter++;
vars.put("counter",Integer.toString(counter));
in above code 3 represents number of attempts.
The code will increment number of attempts by one for each iteration and if it reaches max attempts it sets txtFound to TRUE to stop the test.
Add an if condition below the request as shown below
In if loop add a bean shell sampler and set the txtFound value to TRUE as shown below
When response assertion fails if condition will not be executed and if the response assertion passes if condition is set to true and the elements in If will be executed
The test stops if it finds the correct response ant time or it will stop if it reaches max number of attempts
In my case i kept 3 as response assertion so if it finds 3 it will stop or if it reaches max number of 3 attempts
Please follow this link for more information on
while controller
I run some load tests (all endpoints) and we do have a known issue in our code: if multiple POST requests are sent in the same time we do get a duplicate error based on a timestamp field in our database.
All I want to do is to count timeouts (based on the message received "Service is not available. Request timeout") in a variable and accept this as a normal behavior (don't fail the tests).
For now I've added a Response Assertion for this (in order to keep the tests running) but I cannot tell if or how many timeout actually happen.
How can I count this?
Thank you
I would recommend doing this as follows:
Add JSR223 Listener to your Test Plan
Put the following code into "Script" area:
if (prev.getResponseDataAsString().contains('Service is not available. Request timeout')) {
prev.setSampleLabel('False negative')
}
That's it, if sampler will contain Service is not available. Request timeout in the response body - JMeter will change its title to False negative.
You can even mark it as passed by adding prev.setSuccessful(false) line to your script. See Apache Groovy - Why and How You Should Use It article fore more information on what else you can do with Groovy in JMeter tests
If you just need to find out the count based on the response message then you can save the performance results in a csv file using simple data writer (configure for csv only) and then filter csv based on the response message to get the required count. Or you can use Display only "errors" option to get all the errors and then filter out based on the expected error message.
If you need to find out at the runtime then you can use aggregate report listener and use "Errors" checkbox to get the count of failure but this will include other failures also.
But, if you need to get the count at the run time to use it later then it is a different case. I am assuming that it is not the case.
Thanks,
I'm running a test in Jmeter. that test sends in the end the status of the test - "success" or "fail".
I've created a 'user defined variable' that is named 'subject' and assigned it with value 'success'.
within the http requests I've put 'BeanShell Assertion' that assigns the 'subject' variable with 'failure' if the test failed:
if( (ResponseCode != null) && (ResponseCode.equals ("200") == false))
{
//Failure=true;
vars.put("subject","failure");
}
now, in the SMTP sampler I'm sending ${subject} as the subject of the mail.
the sampler doesn't recognise the variable (it is empty).
any ideas?
Can you show the screenshot of your Test Plan? I'm particularly interested in where Beanshell Assertion lives.
JMeter Assertions have their scope, i.e. given the following test structure:
Sampler 1
Assertion
Sampler 2
Assertion will be applied to Sampler 1
In case of the following test plan:
Sampler 1
Sampler 2
Assertion
Assertion will be applied to both Sampler 1 and Sampler 2
Nothing is wrong with your code, it should be setting "subject" variable in case of non "200" response code.
By the way, there is a pre-defined JMeter variable - ${JMeterThread.last_sample_ok} - which returns "true" if previous sampler was successful and "false" in the other case. It is better to use it in combination with the If Controller as Beanshell has known performance issues and can become a bottleneck in case of heavy loads.
The problem was that when using the 'Beanshell Assertion' I didn't pass it the variable - ${subject}, so when the test succeeded it was like the variable was never assigned.