What's the best strategy to use when writing JMeters tests against a web application where the values of certain query-string and post variables are going to change for each run.
Quick, common, example
You go to a Web Page
Enter some information into a form
Click Save
Behind the scenes, a new record is entered in the database
You want to edit the record you just entered, so you go to another web page. Behind the scenes it's passing the page a parameter with the Database ID of the row you just created
When you're running step 5 of the above test, the page parameter/Database ID is going to change each time.
The workflow/strategy I'm currently using is
Record a test using the above actions
Make a note of each place where a query string variable may change from run to run
Use a XPath or Regular Expression Extractor to pull the value out of a response and into a JMeter variable
Replace all appropriate instances of the hard-coded parameter with the above variable.
This works and can be automated to an extent. However, it can get tedious, is error prone, and fragile. Is there a better/commonly accepted way of handling this situation? (Or is this why most people just use JMeter to play back logs? (-;)
Sounds to me like your on the right track. The best that can be achieved by JMeter is to extract page variables with a regular expression or xpath post processor. However your absolutely correct in that this is not a scalable solution and becomes increasingly tricky to maintain or grow.
If you've reached is point then you may want to consider a tool which is more specialised for this sort of problem. Have a look web testing tool such as Watir, it will automatically handle changing post parameters; but you would still need to extract parameters if you need to do a database update but using Watir allows for better code reuse making the problem less painful.
We have had great success in testing similar scenarios with JMeter by storing parameters in JMeter Variables within a JDBC assertion. We then do our http get/post and use a BSF Assertion and javascript do complex validation of the response. Hope it helps
Related
This is not a code/case specific question.
I am new to Apex, and I'm trying to test methods that do Callouts to external APIs. I understand that in order to test this method, I have to create a class that implements HttpCalloutMock and use it in my test.
However, I want to know: in the Test, when I call the actual method I'm testing, does a call go out to the API behind the scenes? Or is the data I'm putting in the mock the only data that gets passed around?
(I'm asking because, if the latter, wouldn't that mean these tests are extremely counterproductive and unnecessary?)
The dummy data you provided in the mock class will be dutifully returned. And yes, it's annoying, double work.
But how else could it be done? Really calling an external API might have bad consequences (sending "My Awesome Test Order!!!1one!eleven" to production fulfilment system would be a disaster, especially if you do it few times because deployment kept failing). And when such API would be down and you really, really need to deploy something to production - you shouldn't be a hostage of 3rd party server, even test one.
Instead of grumbling try to embrace it. Yes, it's rubbish. But this is your opportunity to test how your code handles different outputs. How it reacts when the API response is "HTTP 500 Internal Server Error", HTML instead of JSON or even there's no response, just timeout. The more solid you make it, the more confident you'll be.
Is it really that hard? Capture couple real messages & errors, remove sensitive data, implement some switch statement "if account number = 123 return this else return that" and you're done.
And yes, it essentially means implementing 3rd party's logic yourself. But well, with test-driven development you ideally would start with a dummy representation of their service anyway, something that's close enough to the API "contract" you have. And as a bonus - you get to shout at them when something suddenly breaks and you can prove it wasn't a change on your end.
In the end it's not too different from splitting work with another SF developer. "OK, I'll do the UI bit, you do the apex bit, here's the data interface we promise to use, see you in 1 week's time". How far can you trust the guy, eh? ;)
Does karate provide any listener support where I can intercept any specific things like rest calls?
This is more like added customization we want to perform apart from what the karate provides. There will be always something or other we need to customize based on the need.
Say that I have 10000 test cases running in parallel and using karate parallel runner I get a nice report with the time it takes for each step and test cases. One of my service is getting called multiple times and I wanted to know what is the average time the service takes out of all the calls. What is the maximum or minimum time it takes.
I think Karate Hooks will get you what you need - if you write a function to aggregate the responseTime.
I'm willing to look at introducing this feature if needed, but you'll have to make a proposal on what the syntax should look like. Feel free to open a feature request. Today we do have configure headers that is like a "before" for all requests. Maybe something along those lines.
I'm using Selenium from Java with a remote grid. When I find an element on a page I would like to retrieve its text, multiple attributes from this element, check whether it is displayed and whether it is enabled.
As far as I can see each thing I retrieve triggers a new remote call (to http endpoint of the webdriver). Since I know beforehand which values I'm interested in I would like to combine them in a single http call (as the call can be quite slow). Is this possible in Selenium with Java? Or even with the webdriver protocol?
To be clear: my problem is not finding an element based on multiple criteria in one go, I know how to do that. But after I find the element I want to know the values of multiple properties, and I want to gather these efficiently.
As far as I can see the protocol requires a separate call for each attribute value, the text, whether the element is displayed and whether it enabled. For me this means for instance 6 round trips to the server, where one could suffice if I were able to 'multiplex' all data I would like to retrieve in a single call.
Is there a way to optimize retrieving multiple details/properties of an element once I found it?
On solution to have less calls between the driver and server could be to use some javascript in the context of the client side/window.
You can write something like
combinedObject = driver.executeScript("function(domelement) {
return { abc: domelement.getAttribute('abc'), efg: domelement.getAttribute('efg'), hij: domelement.getAttribute('hij') };
}",foundedElement);
This can reduce the number of calls between driver and server.
If it makes sense to mix some javascript functions with your java code is your decision.
I'm trying to benchmark/ do performance testing of API's at my work. So the client facing is REST format while the backend data is retrieved by SOAP messages. So my question is can some of you share your thoughts on how you implement it (if you have done so in the past/doing it now), am basically interested in avg response time it takes for API to return results for the client
Please let me know if you need any additional information to answer the question
Could not say it any better than Mark, really: http://www.mnot.net/blog/2011/05/18/http_benchmark_rules
Maybe you should give JMeter a try.
You can try using Apache Benchmark.This is simple and quick
Jmeter gives you additional flexibility like adding functional cases along with performance details. Results will be almost similar to Apache Benchmark tool.
The detailed one which gives Functional Test Result, performance counters settings, Call response time details, CPU and Memory changes along with Load/Stress results, with different bandwidth and browser settings - Visual Studio Team system
I used VSTS2010 for performance testing. Also GET and POST are straight forward. PUT and DELETE need coded version of webtest.
Thanks,
Madhusudanan
Tesco
If you are trying to test the REST -> SOAP calls. One more thing you can consider is to have some stubs created (for backend). This way you can perf test REST -> Stub performance followed by Stub -> SOAP perfomance. This will help in analyzing the individual components.
How to pass struts form parameters in Jmeter. I got in formation by using Regular Expression Extractor we can solve it but, i am not getting any information how?
I think the best way is NOT to create the requests yourself ( if that's what your doing ), but instead use jmeter as a proxy to record your web app flow. The proxy recording session will take take of boring head parameteres that, in my opinion, are too tedious to add by hand. You can find how to set up the jmeter proxy mode here and here. You can't miss it.
good luck!