I have an API that I've written and now I'm in the middle of writing an SDK for 3rd parties to more easily interact with my API.
When writing tests for my SDK, it's my understanding that it's best not to simply call all of the API endpoints because:
The tests in the API will be responsible for making sure that the API works.
If the SDK tests did directly call the API, my tests would be really slow.
As an example, let's say my API has this endpoint:
/account
In my API test suite I actually call this endpoint to verify that it returns the proper data.
What approach do I take to test this in my SDK? Should I be mocking a request to /account? What else would I need to do to give my SDK good coverage?
I've looked to other SDKs to see how they're handling this (Stripe, Algolia, AWS), but in some cases it does look like they're calling a sandbox version of the actual API.
(I'm currently working with PHPUnit, but I'll be writing SDKs in other languages as well.)
I ended up taking this approach:
I have both unit tests AND integration tests.
My integration tests call the actual API. I usually run this much less frequently — like before I push code to a remote. (Anyone who consumes my code will have to supply their own API credentials)
My unit tests — which I run very frequently — just make sure that the responses from my code are what I expect them to look like. I trust the 3rd party API is going to give me good data (and I still have the integration tests to back that up).
I've accomplished this by mocking Guzzle, using Reflection to replace the client instance in my SDK code, and then using Mock Handlers to mock the actual response I expect.
Here's an example:
/** #test */
public function it_retrieves_an_account()
{
$account = $this->mockClient()->retrieve();
$this->assertEquals(json_decode('{"id": "9876543210"}'), $account);
}
protected function mockClient()
{
$stream = Psr7\stream_for('{"id": "9876543210"}');
$mock = new MockHandler([new Response(
200,
['Content-Type' => 'application/json'],
Psr7\stream_for($stream)
)]);
$handler = HandlerStack::create($mock);
$mockClient = new Client(['handler' => $handler]);
$account = new SparklyAppsAccount(new SparklyApps('0123456789'));
$reflection = new \ReflectionClass($account);
$reflection_property = $reflection->getProperty('client');
$reflection_property->setAccessible(true);
$reflection_property->setValue($account, $mockClient);
return $account;
}
When writing tests for the SDK you assume that your api DOES work exactly like it should (and you write tests for your api to assure that).
So using some kind of sandbox or even a complete mock of your api is sufficient.
I would recommend to mock your API using something like wiremock and then write your unit tests around that mock API to make sure everything works as desired.
This way when your production app with break, you can at-least make sure (by running unit tests) that nothing broken in your application side but there can be problem with actual API (i.e response format being changed).
Related
We're successfully using Karate to automate tests for REST and SOAP webservices. In addition, we're having some legacy webservices, which are based on the Hessian Web Service protocol (http://hessian.caucho.com/).
Hessian calls are HTTP requests as well, so we would like to add them to our Karate test suites.
My first attempt was to use the Java Interop feature, so the tests are implemeted as Java code and the Java classes are getting called within the Feature files.
Example:
Scenario: Test offer purchase order
* def OfferPurchaseClient = Java.type('com.xyz.OfferPurchaseClient')
* def orderId = OfferPurchaseClient.createOrder('12345', 'xyz', 'max.mustermann#test.de')
* match orderId == '#number'
This approach is working, but I'm wondering if there is a more elegant way which would also use some more features of the Karate DSL.
I'm thinking about something like this (Dummy Code):
Scenario: Test offer purchase order
Given url orderManagementEndpoint
And path offerPurchase
And request serializeHessian(offerPurchase.json)
When method post
Then status 200
And match deserializeHessian(response).orderId == '#number'
Any recommendations/tips about how to implement such an approach?
I actually think you are already on the right track with the Java interop approach. The HTTP client is in my honest opinion a very small sub-set of the capabilities of Karate, there are so many other aspects such as assertions, reports, parallel-execution, even load-testing etc. So I wouldn't worry about not using request, method, status etc.
Maybe OfferPurchaseClient.createOrder() can return a JSON.
Or what I would do is consider an approach like OfferPurchaseClient.request(Map<String, Object> payload) - where payload can include all the parameters including the path and expected response code etc.
To put this another way, I'm suggesting that you write your own mini-framework and custom DSL around some Java code. If you want inspiration and ideas, please look at this discussion, it is an example of building a CLI testing tool around Karate: https://stackoverflow.com/a/62911366/143475
I'm dealing with a problem I have in a Symfony 4 API functional tests. My functional tests consists in making requests to the API and analyze the response given. I've been working like this and works fine.
The problem comes with a new API method I'm implementing which needs the perform a request to an external service. I want to mock during my tests, but I don't know how can I create a mock that persists when the API receives the request from the functional test.
I've been thinking about something like create mocks which are always used in the test environment but I haven't found anything...
you can check in http-client service called url and if it compare your external api url return certain response, it will be look something like this:
$guzzleServiceMock = $this
->getMockBuilder(GuzzleHttp\Client::class)->disableOriginalConstructor()
->setMethods(['get'])
->getMock();
$guzzleServiceMock
->expects($this->any())
->method('get')
->with(
$this->stringContains('/external/api/route')
)
->willReturnCallback(
function ($uri, $options = []) {
return new Response(
200,
[],
'{"result": {
"status": "success",
"data": "fake data",
}}'
);
}
);
next step you will need to inject service into container, with this question you can look this repo, there are good examples of how this can be done: https://github.com/peakle/symfony-4-service-mock-examples/blob/master/tests/Util/BaseServiceTest.php
I'm not sure if I understand correctly, but if you'd like to mock an external API (meaning your application is connecting to another application on another server), one solution would be to actually lunch a mock server to replace your actual external server.
Either you implement such a mock server yourself or you use an existing solution such as http://wiremock.org/
I'm designing a web service running on Google App Engine that scrapes a number of websites and presents their data via a RESTful interface. Based on some background reading, I think I'd like to attempt Test Driven Development (TDD) and develop my tests before I write any business code.
My problem is caused by the fact that my list of scraped elements includes timetables and other records that change quite frequently. The limit of my knowledge on TDD is that you write tests that examine the results of code execution and compare these results to a hardcoded result set. Seeing as the data set changes frequently, this method seems impossible. Assuming that this is true, what would be the best approach to test such an API? How would a large-scale web API be tested (Twitter, Google, Netflix etc.)?
You have to choose the type of test:
Unit tests just test proper operation of your modules (units). You provide input data and test that code outputs proper results. If there are system dependent classes you try to mock them or in case of GAE services, you use google provided local services. Unit tests can be run locally on your machine or on CI servers. There are two popular unit test libs for java: Junit & TestNG.
Integration tests check that various modules (internal & external) work together - they basically check that APIs between modules are working. They are usually run on real servers and call real external services. They are technology specific and are harder to run.
In your case, I'd go with unit tests and provide sets of different input data which you logic should parse and act upon. Since your flow is pretty simple (load data from fixed Url, parse it) you could also embed loading of real data into unit tests (we do this when we parse external sources).
From what you are describing you could easily find yourself writing integration tests. If your aim is to test the logic for processing what is returned from the scraped data (e.g. you know that you are going to get a timetable in a specific format coming in and you now have logic to process that data) you will need to create a SEAM between your web services logic and your processing logic. Once you have done this you should be able to mock the data that is returned from the web service call to always return the same table data and then you can write consistent unit tests against it.
public class ScrapingService : IScrapingService
{
public string Scrape(string url)
{// scraping logic}
}
public interface IScrapingService
{
string Scrape(string url);
}
public class ScrapingProcessor
{
private IScrapingService _scrapingService
// inject the dependency
pubilc ScrapingProcessor(IScrapingService scrapingService)
{
_scrapingService = scrapingService;
}
public void Process(string url)
{
var scrapedData = _scrapingService.Scrape(url)
// now process the scrapedData
}
}
To test you can now create a FakeScrapingService that implements the IScrapingService interface and then return whatever data you like from the Scrape method. There are some very good Mocking frameworks out there that make this type of thing easy. My personal favorite is NSubstitue.
I hope this explanation helps.
The following is the method I am trying to use for getting the API to make the calls to Google calendar. I am not sure what the stub should return. Should I capture the normal response and use it as is or is there a reference with minimum set of parameters?
api = client.discovered_api('calendar', 'v3')
result = client.execute!(:api_method => api.calendar_list.list)
I can see that Omniauth provides it's own mock support and I can see that Google provides Python mock libraries, but I'm not aware of any direct Google support for mocking from Ruby.
That said, given your example, you would need test doubles for client and api. It's not clear where client is coming from, but assuming that's established as a double somehow, you'd have at a minimum:
api = double('api')
client.should_receive(:discovered_api).and_return(api)
api.stub_chain(:calendar_list, :list)
client.should_receive(:execute!).and_return(... whatever result you want ...)
If in addition you want to confirm that your code is passing the right parameters to the Google API, then you'd need to augment the above with message expectations and, in the case of the api stub_chain, a return value which would then have to feed into the message expectations for the execute! call.
I'm still not sure that answers your question, but if not, I'll look forward to reading any additional comments.
I see several other questions about load testing web services. But as far as I can tell those are all synchronous load testing tools. (Meaning they send a ton of requests but the go one at a time.)
I am looking for a tool where I can say, "I want 100 requests to be launched at the exact same time".
Now, I am new to the whole load testing thing, so it is possible that those tools are asynchronous and I am just missing it.
Anyway, in short my question is: Is there a good tool for load testing WCF Web Services asynchronously (ie lots of threads).
In general, I recommend you look at soapUI, for anything to do with testing web services. They do have load testing features in the Professional edition (I haven't used these yet).
In addition, they've just entered beta with a loadUI product. If it's anywhere near as good as the parent product, then it's worth a hard look.
you can use the Visual Studio load testing agent components to run on multiple client machines and that will allow you to run as asynchronously as you have machines to load.
There is a licence requirement for using this feature.
There are no tools that will allow you to apply a load at exactly the same instant (i.e. within milliseconds), but this is not necessary to load test an application correctly.
For most needs a single load test server running Visual Studio Ultimate edition will be more than enough to get an understand of how your webservice performs under load.
Visual Studio and most other tools I imagine will apply load in an asynchronous manner, but I think in your view you want to apply a set load all at once.
This is not really necessary as in practice load is not applied to a service in this manner.
The best bet for services expecting high load is to load your service until a given number of "requests per second" is reached. Finding what level your application should expect is a bit trickier, but involves figuring out roughly how many users you would expect and the amount they will be using it over a given period.
The other test to do is to setup a load test harness and run the load up until either the webservice starts to perform badly or the test harness runs out of "oomph" and cannot create any more load.
For development time you can use NLoad (http://nload.github.io)
to run load tests on your development machine or testing environment.
For example
public class MyTest : ITest
{
public void Initialize()
{
// Initialize your test, e.g., create a WCF client, load files, etc.
}
public void Execute()
{
// Send http request, invoke a WCF service or whatever you want to load test.
}
}
Then create, configure and run a load test:
var loadTest = NLoad.Test<MyTest>()
.WithNumberOfThreads(100)
.WithDurationOf(TimeSpan.FromMinutes(5))
.WithDeleyBetweenThreadStart(TimeSpan.Zero)
.OnHeartbeat((s, e) => Console.WriteLine(e.Throughput))
.Build();
var result = loadTest.Run();