I have an api project which requires the payload to be validated against a set of rules. For this, I have written a karate feature file with all the required assertions for my incoming request json payload. The feature file then returns true or false depending upon the satisfied conditions. This mechanism is working perfectly in my local currently.
Is this approach recommended for production use case? Will the karate framework in this format be capable of handling large volumes of requests coming in a very short span of time?
Developer of Karate here, I really like your question because it validates what I personally believe, that Karate just makes it easy to work with JSON.
Why I won't recommend this for production use is because Karate embeds a JS engine that can be targeted using a "script injection" attack. Karate is designed for users running "locally" and has no safeguards built-in to prevent any malicious attacks coming in via JSON payloads.
The other question is performance, personally I am quite confident, because Karate is being used in conjunction with Gatling and some work has gone into improving performance over the years. But at the end of the day, Karate does use a JS engine in interpreted mode. So you need to run a performance test or load test yourself to validate if Karate can handle the volume you expect.
Maybe you can contribute to Karate to address both the above concerns !
Related
Does Karate API tests supports integration with TM4J (Test management for Jira - Adaptavist)?
We have been using Karate for our GraphQL layer and it has been quite successful so far. we have a requirement to integrate TM4J across multiple frameworks across the company and hence wanted to know if it is possible?
TM4J supports cucumber.options
Any help on this is much appreciated.
Thanks!
Since Karate exports the cucumber-json, this may work without any change.
Also refer this thread: https://github.com/intuit/karate/issues/619
And note that Karate does one thing slightly differently, so you may need to be aware of that: https://stackoverflow.com/a/56516016/143475
Anything beyond this we consider the responsibility of the external tool side, or you have to write a custom-hook: https://stackoverflow.com/a/59080128/143475
Does Karate API tests supports integration with TM4J (Test management for Jira - Adaptavist)?
We have been using Karate for our GraphQL layer and it has been quite successful so far. we have a requirement to integrate TM4J across multiple frameworks across the company and hence wanted to know if it is possible?
TM4J supports cucumber.options
Any help on this is much appreciated.
Thanks!
Since Karate exports the cucumber-json, this may work without any change.
Also refer this thread: https://github.com/intuit/karate/issues/619
And note that Karate does one thing slightly differently, so you may need to be aware of that: https://stackoverflow.com/a/56516016/143475
Anything beyond this we consider the responsibility of the external tool side, or you have to write a custom-hook: https://stackoverflow.com/a/59080128/143475
I'd like to scale an infrastructure for automating user actions. I've found tools like InstaPy and InstaBot and have seen that one uses a browser automation tool like Selenium whereas the other uses only http requests, etc.
What are the pros/cons of these various approaches? Which would one be simpler and more cost effective to scale? I assume the request approach would be more lean but perhaps a headless approach with testing frameworks could also be as lean?
User action in the sense means Functional Testing. For Functional Testing, one must go for the real time browser simulation.
On the other hand, there is Performance Testing. To which you need to go with API's(HTTP Requests). Since, API testing is faster than Selenium, your first step is to check with API's to verify whether backend gets broken.
Once API's are passed, we can go ahead with UI/Functional Testing. Both are cost effective, but I would say, Functional Testing is much better, as UI rendering fails often when it comes to Web applications. Functional Testing also tells you whether it is API failure or not for which you can use network traffic logs.
Currently we use soap-ui/manual posting of xmls etc for load testing our spring boot webapplications.
Looking for any free load testing tools that others have been using and want to recommend?
Thanks for your help!
The specific load testing or in general testing tool is the one you will "like" more by lots of personal/company needs. There are plenty of them, here is the short list of them I used:
Blazemeter
Gatling
JMeter
I presonally spent most of load testing time in Gatling, first of all it is using scala development language and quite easy to include in your Java project via maven/gradle, secondaly apart from other benefits it have direct JDBC connection possibilty which let's you to have your test data directly in database. Lots of other pros. But one more time it is strongly my opinion and my preferences. E.g. if you are big fan of XML you will most probably like JMetter, in general Blazemter is kind of a next level of JMeter.
I usually code the test using my current favorite language, Python in recent times. For example in python this would be some code around the requests library, and possibly some multi-process code- nothing to heavy or complex.
I find it more flexible to code myself, on average having more control over the load (there are pros and cons to using a prebuilt tool in this sense) and it usually integrates better with other code in my automation suite.
But the answer is somewhat context dependent, is there someone with the knowledge and resource to develop a tool ? do you have to document the results or make them comparable with other systems?
I am exploring individual microservice testing & came across a good guide https://martinfowler.com/articles/microservice-testing/#testing-component-in-process-diagram. This post suggest there is a way to test the individual microservices "in-process".
Which means the test & service will execute in same process, it also provides the lib which can be used. However I am not able to understand following things
1) How exactly "in-process" testing will work in microservice architecture
2) Any pointers on how to use "inproctester" lib.
Thanks
This can be accomplished by hosting your API in process, interacting with it - in process - whilst using test doubles at the edges of you API, or (put another way) using test doubles for external collaborators.
This allows an API (and its core behaviour) to be tested in a test (unit/integration depending on opinion) without ever hitting a network or database or filesystem.
This testing will provide coverage but obviously not all the coverage you need
In .NET core you can use TestServer included in the Microsoft.AspNetCore.TestHost package.
If your API is simple enough you can perhaps reduce your types of testing (N.B. I am not suggesting reducing coverage)
This approach is alluded to in the slide deck you mention on page 8/25
Paraphrasing. If the logic in the API is small, component tests may be favourable to isolated and sociable unit tests
This approach is also recommended in the Spotify "Honeycomb" style testing. They call component testing of an API Integration testing (which coincidentally Microsoft also call it when they are using TestServer in their tests (second link for concrete examples)
https://labs.spotify.com/2018/01/11/testing-of-microservices/
https://learn.microsoft.com/en-us/aspnet/core/testing/integration-testing
I think that the thrust of the approach (regardless of its specific name) is that in process testing is fast and external collaborators are replaed with test doubles.
One thing worth noting is that "out of process" component tests are basically end to end tests but with all external collaborators swapped out with test doubles.
These may be a bit slower than in process component tests but exercise a bit more of the "stack". I'd suggest that if you are going to end-end test anyway, I'd try and do in process component testing as much as possible.