We're required to use the API of an external partner. The API is in a good shape and we got access to a sandbox environment we can use for automatic testing.
We already test every single call of the external API using unit tests but are unsure regarding best practices for integration tests when it comes to complex operations on the side of the external partner.
Example: Every user of our service also got a user object at our external partner. When performing external API call X on this user object, we expect object Y to appear inside collection Z of this user (which we have to query using a different call).
What are best practices for testing cases like this?
Mock the external API as much as possible and rely on the unit tests to do their job? Advantages: Tests run fast and independent from an internet connection. Disadvantages: Mistakes is in our mocks could lead to false positives.
Integrate the external API sandbox and run every integration test against it. Advantages: Close to real life API interactions. Disadvantages: Tests can only be run with an open internet connection and take more time.
Use a hybrid of mocked and sandbox data, set a boolean to switch between the internal (=mocked) and external (=sandbox) environment when required. Advantages: Reliable tests. Disadvantages: Could be a pain to set up.
Other best practices?
Thanks!
Related: How are integration tests written for interacting with external API? However, the answer "You don't. You have to actually trust that the actual API actually works." is not sufficient in our opinion.
[EDIT] We fear that integration testing only against our assumptions how the external API should work (even if they are based on unit tests) – and not against the actual API – will leave us with false positives. What we'd need is a test that verifies that our assumptions (mocks) are actually correct – not only in the context of unit tests but also in the context of complex operations with several steps.
Validation might be a good example: What if we mess up the integration code and send malformed data or data that does not make any sense in the context we send it in because we missed a step? Our mock API, which does not validate (or only in very limited range) would still return valid data instead of passing the error we would receive from the real API.
I believe there should be 2 level of verifications we need to do when we interface with an external API:
API verification: verify that the API works according to its specs and/or our understanding
App functionality verification: verify that our business logic works according to the expectation to the API that passes API verification
In our case, we use a mock API together with real and mock API verification.
Mock API allows us to isolate any runtime errors/exceptions to app functionality only, so we don't blame any external party for issues
The same API verification is executed against both real and mock APIs, to make sure that the real one works the way we expect, as well as the mock one should mimic the real one correctly
If along the way, external API changes, API verification may turn red, triggering changes in mock API. Changes in mock API may make app verification turn red, triggering changes in app implementation. This way you never miss any gap between external API and app implementation (ideally).
Another extra benefit of having a mock API + API verification is that your developers can use it as a documentation/specification of how the API is supposed to work.
Related
I am a new to REST API automation project, as part of that I have learned using jayway rest assured instead of jersy client. Now, the problem is I am able to use protocol methods and getting response to parse and checking required data is not.
Now,
I want to explore more to implement project setup like a pro, by using structured java classes or by using any class designs.
I want to use this project for load testing
I want to learn parameterization, i.e., First request's response may be input to 4th request (ex: login token id used for subsequent requests)
I also want to know how to feed data as input from external files
Note: I have searched for sample projects on other channels but they are not as per my requirement and I spent time to understand those project but going over my head, couldn't able to understand their style of implementation :(
Well, there are several things that you need to learn/understand:
It is not about design patterns, them you should learn in any case to consider yourself as a good developer or software engineer in test
REST Assured - is indeed for API testing, yes, but not the best
choice, or even, is the last choice for load/performance testing
Haveing request based dependencies, more sound like component testing or end-to-end testing which is usually also the last choice
and have to be as minimum as possible
Load/performance tools that you can choose and learn by your preferences are (not a full list, but ones that I used to use)
Blazemeter
Gatling
Jmeter
A bit of context: I started implementing payment in my project using Stripe, and I started thinking about how to write the tests. After learning a bit about how it should be done and having into account the "don't mock what you don't own" philosophy, I decided to implement a wrapper for stripe API and I now have two kinds of tests: Tests for my application mocking the API wrapper, and tests for the API wrapper (which I don't run as often as the ones for my app). This second tests call stripe for real, so I need to provide a Stripe API Key. And here's where my question comes in.
Question: Should I use my account's test api key for unit testing, or can I use a generic Stripe API key? Being them unit tests, I don't want them to leave any persistent logs to my account, not even on the test dashboard.
Searching in google I found the following api key: tGN0bIwXnHdwOa85VABjPdSn8nWY7G7I and unlike the test api key from my account, it doesn't follow the pattern "pk_test_*", but it does work with stripe and returns correct responses. If you are wondering where did I get that key (and if I should be publishing it), it has been officially published by stripe in blog posts as well as repositories, but I couldn't find any explanation in the documentation or anywhere else, hence this question.
Old Stripe API keys did not follow the [sk|pk]_[test|live]_... pattern. This is such a key.
Honestly though, I would recommend using either your own test API keys, or creating a different account (you can do so with the same email address: https://stripe.com/blog/manage-multiple-accounts). If a test fails, being able to see the log entry in the dashboard will probably be very helpful.
I don't know about testing but I would like to have a clear picture on how is API testing different from other testing methods.
API testing will not include UI as regular testing have
API testing requires basic networking knowledge such as what is the use of GET, POST, PUT, etc commands used.
API testing includes having knowledge of how various html elements work. For example, If I press a button, what will be the next function call. We need to know how 'button' element works
In API only API functions are tested, but in regular testing all the elements are tested
There are different tools used in API testing. POSTMAN is one of them
In API Testing we test Backend functionality while in Regular testing we check UI + Functional testing.
API Testing is helpful in testing Core Functionality.It helps us to reduce the risks.
Steps to Test API Manually:-
To use API manually, we can use browser based REST API plugins.
a)Install POSTMAN(Chrome) / REST(Firefox) plugin
b)Enter the API URL
c)Select the REST method
d)Select content-Header
e)Enter Request JSON (POST)
f)Click on send
g)It will return output response
Steps to start API Automation using REST
in general API testing is made for not doing many similar actions, when we can easily measure the result.
For example if you app button is not on the right place, it is hard to measure using code.
Another example is when you write a library for collections you can say just once:
CheckIntersect method:
result = mylibrary.getIntersection([1,2,3,4], [3,4,5,6])
if result != [3,4]
postTestError("CheckIntersect [1,2,3,4], [3,4,5,6]" + result.ToString() )
In this case you can easily measure the result, and not have a fear of you can't even find the problem in code.
I would like differentiate API vs. Other Testing instead looking into technical details.
API: Testing point of view the API is so important because, we can prepare independent test cases that are separate files. This makes our test approach relatively simple.
For better understanding, "Web API is typically done as HTTP/REST, nothing is defined, output can be eg. JSON/XML, input can be XML/JSON/or plain data. There are no standards for anything => no automatic calling and discovery."
API is a simple interface using HTTP protocol.
Other Testing:
Other testing like GUI, Regression, Unit, etc. Testing is absolutely essential for any application has to be in user friendly. The end user should be comfortable while using all the components should also perform their functionality with utmost clarity. Different Functional and GUI Testing can refer to just ensuring that the look and feel and Functional usability of the application is acceptable to the user.
Conclusion:
API can be:
Developed by one company, used by another company, and hosted by a third company Such involvement of several companies is a business cases for independent testing of API.
Example: Weather information API Developed and Tested by One & Accessed by many.
General Testing is testing each and every feature of the application from UI like web or mobile .
But,
API Testing is to verify the JSON Request to Server and Response from the Server .
If the application is using API all the content and Features based on the API Response from the Server .
For Example In FB app Profile screen, if the name is wrong ,
you can check from UI that general Testing, The same thing you can
Check it from API Response from the server , like below.
{"name":"Dharma","friends":450}
On some websites that provide API service, they always mention that "Respect their API". I think when a website provide their API, they allow users to use them in a convenient way. We cannot really change the code or the output that we retrieve from the API.
I'm just wondering what do we need to do to actually "respect their API" and what kinds of action are counted toward "disrespect their API"?
An action that compromises the API's security, speed, or use can be construed as "disrespecting the API." From CAPEC.org:
An attacker manipulates the processing of Application Programming Interface (API) resulting in the API's function having an adverse impact upon the security of the system or application implementing the API. This can allow the attacker to execute functionality not intended by the API implementation, possibly compromising the system or application which integrates the API. API Abuse can take on a number of forms. For example, the API may trust that the calling function properly validates its data and thus it may be manipulated by supplying metacharacters or alternate encodings as input, resulting in any number of injection flaws, including SQL injection, cross-site scripting, or command execution. Another example could be API methods that should be disabled in a production application but were not, thus exposing dangerous functionality within a production environment.
Our application is a API based one wherein the frontend relies on REST API calls to the back end. This sometimes creates a problem wherein the frontend team can't move forward unless the backend API's have been implemented since they invariably progress at different speeds. Is there a way to set up a server so that the front-ent can work independently regardless of the backend status ?
I know this is a bit of an old post but I created a tool just for this purpose and I thought I should share it for anyone who stumbles across it.
It's called Interfake and you can find it at https://github.com/basicallydan/interfake. I frequently use it for prototyping APIs which haven't been built yet, in fact that is my main use of it. I hope that helps.
A common solution that we work with is as follows :
FE and BE contracts/APIs are agreed upon and the back end apis are mocked.
The BE rest APIs use a filter that we configured.
For all the apis that are ready, the filter redirects to the correct api and for all the apis that are mocked the filter redirects to the mock api.
So transparent to the FE team as the BE team is building and completing more apis, they just update the map that the filter looks and and automatically the back end apis get invoked as soon as the BE team is ready to open it up.
So the flow is as follows :
FE ->BE Rest API Server
|API Filter->(for apis updated in the map as complete)-->server/port with actual api
|------------->(for apis that are still being mocked) --> server/port with mock
Hope that helps.