Which one's better in terms of testing properly as well as speed? I've read somewhere that Xpath Assertion takes more memory and CPU
It depends on the nature of your response, its size, the criteria you need to assert and their complexity.
For example if you're looking for the presence of a single word - the Response Assertion would work just fine.
But in case if you need to validate an attribute of a node which has parent attribute equal to something - coming up with a proper regular expression would be either impossible or at least very hard.
So if you can accomplish your requirement using Response Assertion - go for it.
You can additionally measure execution speed and resource consumption by using a Dummy Sampler and see how many requests JMeter can handle and how much CPU it takes like it's described in The Boundary Extractor vs. the Regular Expression Extractor in JMeter article
XPath2 Assertion is better than XPath Assertion given your response is a valid XML.
Related
I have an api project which requires the payload to be validated against a set of rules. For this, I have written a karate feature file with all the required assertions for my incoming request json payload. The feature file then returns true or false depending upon the satisfied conditions. This mechanism is working perfectly in my local currently.
Is this approach recommended for production use case? Will the karate framework in this format be capable of handling large volumes of requests coming in a very short span of time?
Developer of Karate here, I really like your question because it validates what I personally believe, that Karate just makes it easy to work with JSON.
Why I won't recommend this for production use is because Karate embeds a JS engine that can be targeted using a "script injection" attack. Karate is designed for users running "locally" and has no safeguards built-in to prevent any malicious attacks coming in via JSON payloads.
The other question is performance, personally I am quite confident, because Karate is being used in conjunction with Gatling and some work has gone into improving performance over the years. But at the end of the day, Karate does use a JS engine in interpreted mode. So you need to run a performance test or load test yourself to validate if Karate can handle the volume you expect.
Maybe you can contribute to Karate to address both the above concerns !
I am learning the spring 5 webflux and reactive streams. And there are new HandlerFunctions and RouterFunctions to implement the Http requests and response.
and as per the documentations:
The annotation counterpart to a handler function would be a method with #RequestMapping.
As #RequestMapping is quite easy to handle, implement and understand, then why is there a need of more complex and difficult way to handle Http request and response via this HandlerFunctions and RouterFunction utility?
Please suggest.
Spring WebFlux gives you two different styles: annotations and functional.
Depending on the type of application that you'd like to build, the constraints that you have to deal with - one or the other might be more relevant. You can even mix both in the same application.
The annotation-based model is very successful, but it also comes with a few limitations, mostly because of Java annotations themselves:
the code path is not always clear, unless you know the internals of Spring Framework (do you know where handler mappings are detected? matched against incoming requests?)
it's using reflection, which has a cost
it can be hard to debug and extend
The functional variant tries to fix those issues and embrace a functional style (with the JDK8 function API) and immutability. It's got a "more library; less framework" touch to it, meaning that you're more in control of things. Here's an example: with RouterFunction, you can chain RequestPredicates and they are executed in order, so you're in full control of what ultimately handles the incoming request. With the annotations model, the most specific handler will be selected, by looking at the annotations on the method and the incoming request.
If you're perfectly happy with the annotations model, there's no reason to switch. But again, you can mix both and maybe you'll find the functional model handy. In my opinion, trying it even if you don't plan on adopting it won't hurt - worst case scenario this will broaden a bit your perspective as a developer and show you a different way of doing things.
For more on that, you can check out Arjen Poutsma's talk on the functional web framework in Spring WebFlux.
It's not needed, and doesn't break webflux. I'd use #RequestMapping, if you don't have special needs making HandlerFunction neccessary.
For RouterFunctions: If you don't want to use JSON parsing, and want to modify the ServerRequest directly (e.g. have the raw InputStream), you'd have to use the RouterFunctions (AFAIK). You'd then return a raw stream (Mono), too. I had a case, where I needed to play proxy with a little bit extra, and thus needed to avoid the JSON parsing, that you'd usually have with #RequestMapping
I recently transferred to an existing (but new) project, still in beta, that already has a fair amount of unit, integration, and system / e2e tests. Presently, a large portion of integration and system tests are asserting equality between an expected JSON payload and the actual JSON payload.
Considering the fast moving nature of this project (and that it's in beta), changes are often causing many tests to flip red, as they're changing the JSON payload. It seems redundant to frequently change expected JSON payload to match the new output, but I'll do it without complaining if I know that's the ideal way to test.
My question (TLDR): In the case of a JSON API, if I am only testing "Happy Path" e2e / system test scenarios; what would be my ideal assertion statement? Am I looking to test the entire payload against an expected payload or would it make more sense to compare status codes, and maybe some high level JSON keys?
This question is similar to this one but regarding functional tests rather than unit tests.
I'm currently testing a UI using Selenium and I was wondering if only one assertion statement is needed, or if it depends on the test.
For example if I wanna test a basic Facebook login, would it suffice just use an assertion statement for the end case (ex: finding an element that only exists when logged in) or should the test be more detailed and include more than one assertion statement (check if you're on the correct site, check inputs, check for an element that only exists when logged in, etc).
Let me try to answer your Questions one by one:
if only one assertion statement is needed, or if it depends on the test - Let us speak of a manual Testcase. A Testcase consists of several steps but at the end we do cross check the Actual Result against the Expected Result. The same ideology is implemented in Automation through assertions. So ideally as per best practices, an assertion statement is a must but it's not mandatory.
should the test be more detailed and include more than one assertion statement - You can always have multiple assertions in a Testcase. No issues in that. But you must remember, if one Assertions fails the rest of the assertions won't be executed. Which gives you a single result either Pass or Fail. Now if you want to keep multiple validation points then you have to take help of if/else block so all your validations gets executed irrespective of each of them Pass/Fail.
Let me know if this Answers your query.
The one assertion per testcase rule is bit over the top for functional tests. For functional end-to-end tests I think as a general guide it should be test only ONE behavior. Use as many asserts you need to verify this ONE behavior.
If a test is failing you want to understand what is not working without reading the actual test-code. Having multiple assertions in a single test could lead to testing multiple behaviors and multiple reasons to fail. This is sub-optimal.
Do be practical as functional end-to-end tend to be slow. Multiplying the same steps to test a slightly different assert seems a waste of run time. Your test-suites should also be fast if you want to run them on each check-in. Therefor don't write too many tests on this level. Keep a good balance as the test-pyramid suggests.
For your example I think one assert would be enough. The behavior is to check if the user is now logged in, not if all the elements are on the page. Also keep in mind if the implementation of the site changes you need to update all your tests. The less you assert to more maintainable your tests will be.
Is using Assert in Selenium a good thing?
Someone once suggested that using asserts slows down the test and I shouldn't use them.
Assert is to verify test output. More you limit asserts better it can get overload during test cases Read this
Asserts are same overload for testing as comparisons in programming terms. Example is to use asserts to test values validations or filled in Form input that gets populated from grid to Edit/Update fields Just a simple example i can think of.
I don't think that this would be the bottleneck of your tests. Waiting for the browser to render the web page, executing JavaScript and parsing and accessing the page via Selenium will outweigh the time needed for assertions by far.
I am using Asserts for my automated testcases in Serenity, Selenium and JBehave and in my opinion it doesn't matter if you use Asserts, Exceptions or something else...
As Sebastian posted already, the time for opening, rendering pages and accessing the objects will outweigh the time needed for assertion way more. The only thing you can optimize is the loading speed of your page and how you access the web elements.
If you search for an XPath expression you should see that you can optimize your XPath expression for faster searching (e.g., not using //* to look into every node) or even better you use CSS or id selectors.