I started experiencing pact a while a ago, and I'm wondering if any of you guys has any idea if pact support Json schemas!
I'll describe the flow. suppose you have two microservices:
Microservice A - the consumer
Microservice B - the Provider
The provider exposes an Api - basically a POST request and expects an explicit json body payload (some field are required).
Let's say:
{
"id": "123",
"name": "Bob"
}
both id and name are required properties.
Suppose now that the provider changed it's behaviour to expect the same json payload but with an additional property:
{
"id": "123",
"name": "Bob"
"extraProperty": "newProperty"
}
My question is is there anyway to detect such breaking contract using pact framework? if not what do think is the best way to test such scenario.
Thanks for the help.
My question is is there anyway to detect such breaking contract using pact framework?
Of course - this is what Pact is designed to do!
If extraProperty is a new required field (in addition to id and name) then when the provider tests run against the contracts generated by the consumer(s) that don't know about the new field, then your API will not behave as they expected and your pact tests will fail.
As to your other question:
I'm wondering if any of you guys has any idea if pact support Json schemas!
I'm not sure what you mean - we certainly support JSON formats. If you mean JSON Schema then yes, but you still need to write the tests (see https://docs.pact.io/faq#why-doesnt-pact-use-json-schema).
Related
As an example, let's say that I'm building a simple social network. I currently have two services:
Identity, managing the users, their personal data (e-mail, password hashes, etc.) and their public profiles (username) and authentication
Social, managing the users' posts, their friends and their feed
The Identity service can give the public profile of an user using its API at /api/users/{id}:
// GET /api/users/1 HTTP/1.1
// Host: my-identity-service
{
"id": 1,
"username": "cat_sun_dog"
}
The Social service can give a post with its API at /api/posts/{id}:
// GET /api/posts/5 HTTP/1.1
// Host: my-social-service
{
"id": 5,
"content": "Cats are great, dogs are too. But, to be fair, the sun is much better.",
"authorId": 1
}
That's great, but my client, a web app, would like to show the post with the author's name, and it would preferably receive the following JSON data in one single REST request.
{
"id": 5,
"content": "Cats are great, dogs are too. But, to be fair, the sun is much better.",
"author": {
"id": 1,
"username": "cat_sun_dog"
}
}
I found two main ways to approach this.
Data replication
As described in Microsoft's guide for data and Microsoft's guide for communication between microservices, it's possible for a microservice to replicate the data it needs by setting up an event bus (such as RabbitMQ) and consuming events from other services:
And finally (and this is where most of the issues arise when building microservices), if your initial microservice needs data that's originally owned by other microservices, do not rely on making synchronous requests for that data. Instead, replicate or propagate that data (only the attributes you need) into the initial service's database by using eventual consistency (typically by using integration events, as explained in upcoming sections).
Therefore, the Social service can consume events produced by the Identity service such as UserCreatedEvent and UserUpdatedEvent. Then, the Social service can have in its very own database a copy of all the users, but only the required data (their Id and Username, nothing more).
With this eventual consistent approach, the Social service now has all the required data for the UI, all in one request!
// GET /api/posts/5 HTTP/1.1
// Host: my-social-service
{
"id": 5,
"content": "Cats are great, dogs are too. But, to be fair, the sun is much better.",
"author": {
"id": 1,
"username": "cat_sun_dog"
}
}
Benefits:
Makes the Social service totally independent from the Identity service; it can work totally fine without it
Retrieving the data requires less network roundtrips
Provides data for cross-service validation (e.g. check if the given user exists)
Drawbacks and questions:
Takes some time for a change to propagate
The system is absolutely RUINED for some users if some messages fail to get through due to a disaster that fried all your replicated queues!
What if, one day, I need more data from the user, like their ProfilePicture?
What to do if I want to add a new service with the same replicated data?
API Gateway aggregation
As described in Microsoft's guide for data, it's possible to create an API gateway that aggregates data from two requests: one to the Social service, and another to the Identity service.
Therefore, we can have an API gateway action (/api/posts/{id}) implemented as such, in pseudo-code for ASP.NET Core:
[HttpGet("/api/posts/{id}")]
public async Task<IActionResult> GetPost(int id)
{
var post = await _postService.GetPostById(id);
if (post is null)
{
return NotFound();
}
var author = await _userService.GetUserById(post.AuthorId);
return Ok(new
{
Id = post.Id,
Content = post.Content,
Author = new
{
Id = author.Id,
Username = author.Username
}
});
}
Then, a client just uses the API gateway and gets all the data in one query, without any client-side overhead:
// GET /api/posts/5 HTTP/1.1
// Host: my-api-gateway
{
"id": 5,
"content": "Cats are great, dogs are too. But, to be fair, the sun is much better.",
"author": {
"id": 1,
"username": "cat_sun_dog"
}
}
Benefits:
Very easy to implement
Always gives the up-to-date data
Gives a centralized place to cache API queries
Drawbacks and questions:
Increased latency: in this case, it's due to two sequential network roundtrips
The action breaks if the Identity service is down, although this can be mitigated using the circuit breaker pattern, the client won't see the author's name anyway
Unused data might get still queried and waste resources (but that's marginal most of the time)
Having those two options: aggregation on the API gateway and data replication on individual microservices using events, which one to use for which situation, and how to implement them correctly?
In general, I strongly favor state replication via events in durable log-structured storage over services making synchronous (in the logical sense, even if executed in a non-blocking fashion) queries.
Note that all systems are, at a sufficiently high level, eventually consistent: because we don't stop the world to allow an update to a service to happen, there's always a delay from update to visibility elsewhere (including in a user's mind).
In general, if you lose your datastores, things get ruined. However, logs of immutable events give you active-passive replication for nearly free (you have a consumer of that log which replicates events to another datacenter): in a disaster you can make the passive side active.
If you need more events than you are already publishing, you just add a log. You can seed the log with a backfilled dump of synthesized events from the state before the log existed (e.g. dump out all the current ProfilePictures).
When you think of your event bus as a replicated log (e.g. by implementing it using Kafka), consumption of an event doesn't prevent arbitrarily many other consumers from coming along later (it's just incrementing your read-position in the log). So that allows for other consumers to come along and consume the log for doing their own remix. One of those consumers could be simply replicating the log to another datacenter (enabling that active-passive).
Note that once you allow services to maintain their own views of the important bits of data from other services, you are in practice doing Command Query Responsibility Segregation (CQRS); it's thus a good idea to familiarize yourself with CQRS patterns.
Can the HAL+JSON hypermedia type be used in a way that creates a RESTful service?
From what I have read, the client of a RESTful API should not need to treat different resources with special cases. The media type should instead be used to describe what resources are expected to look like.
The HAL spec gives this example:
GET /orders
{
...
"shippedToday": 20,
...
}
```
As a client of this sample HAL+JSON-serving API, I seem to need to know that an "order" has an attribute of shippedToday. That seems to go against the constraint that the client should not need to understand the representation's syntax.
This is not a critique of HAL. The question is to help my (and others') understanding of RESTful API design.
Can the HAL+JSON hypermedia type be used in a way that creates a RESTful service?
Yes, definitely.
The API should have a billboard URL, which in your case could be /.
This is the entry point from which humans and ideally even machines can start to discover your API.
According to the HAL specification a resources representation contains an optional property called "_links" which is described here:
It is an object whose property names are link relation types (as
defined by RFC5988) and values are either a Link Object or an array
of Link Objects.
So these links represent the hypermedia part of your API. The relations can be IANA-registered relations or you can use your own extension relations.
Relations should not be ambiguous. Their names should be unique. That's why it is recommended to use URIs from your own domain as names for your own relations. These URIs identify a resource that represents the relation and contains an API documentation, a human or machine readable documentation of your relation.
In your case this would be a relation that describes the state transition to the /orders resource. This should also include a description and explanation of the response and therefore document that e.g. the /orders resource represents a list of orders and has a property called "shippedToday" with a value of type number.
Here is an example response for a GET / HTTP/1.1 request:
HTTP/1.1 200 OK
Content-Type: application/hal+json
{
"_links": {
"self": { "href": "/" },
"http://yourdomain.com/docs/rels/orders": { "href": "/orders" },
}
}
Under http://yourdomain.com/docs/rels/orders there should be the API docs.
I am using Ember.
The model being posted by ember is
{
"user": {
"firstName": "Vivek",
"lastName": "Muthal"
}
}
Ember had wrapped the data into "user" object.
But the service I have written accepts only {"firstName":"string","lastName":"string"}.
So my question is does REST standards specifies to send / receive data in wrapped Object only?
Any references please so I can change the service accordingly.
Or else I will modify the ember to use my current service as it is. Thanks.
I suppose that the resource is User, so the JSON should represent a User. Lets say you have this URI schema.
GET /host/users/{userId}
PUT /host/users/{userId}
POST /host/users
When we do GET we expect a JSON that represents A User
{
"firstName": "Vivek",
"lastName": "Muthal"
}
There is no need to specify the resource name because we already mentioned that in our GET request, The same for POST, there is no need to mention the resource name in the request body because it is specified in the request URI.So no there, is no need for user key.
No. There is no predefined format for the data you send in the body of your HTTP requests. Well ok, the HTTP RFCs do put technical limits on the data you send, but the formatting is entirely up to you.
As such, you can format your data however you want. You just need to represent the resource. You do need to consider if the JSON for a user should clearly mark the object as a 'user' or not, I would consider it mostly redundant to do so.
REST defines an uniform interface constraint. This constraint states that you have to use standards solutions to create an uniform interface between the service and the clients. This way the clients will be decoupled by the implementation details of the service, because the standards we use are not maintained by the server...
To make it short you can use any standard MIME type, or define a vendor specific MIME type. I strongly suggest you to use an existing hypermedia format like JSON-LD+Hydra or HAL. I guess this kind of serialization is an ember specific thing.
We are now designing our RESTful API and I have a question for how to expose the pagination information.
It seems some famous services like Github or Firefox Market Place having something like below in their API:
{
"meta": {
"limit": 3,
"next": "/api/v1/apps/category/?limit=3&offset=6",
"offset": 3,
"previous": "/api/v1/apps/category/?limit=3&offset=0",
"total_count": 16
}
}
My question is:
Why should the server give the complete next/previous url in the response?
It seem to me that the client is making the first request. So it knows what parameters it used to call (offset/limit/api version). It is easy for the client to figure out what is the next/previous url to call. Why bother to calculate the redundant urls and give it to the client?
It's RESTful! That is specifically part of HATEOAS, or Hypermedia as the Engine of Application State.
Except for simple fixed entry points to the application, a client does not assume that any particular action is available for any particular resources beyond those described in representations previously received from the server.
and:
[HATEOAS] is a constraint of the REST application architecture that distinguishes it from most other network application architectures. The principle is that a client interacts with a network application entirely through hypermedia provided dynamically by application servers. A REST client needs no prior knowledge about how to interact with any particular application or server beyond a generic understanding of hypermedia.
...
A REST client enters a REST application through a simple fixed URL. All future actions the client may take are discovered within resource representations returned from the server.
(Emphasis added)
It seem to me that the client is making the first request. So it knows what parameters it used to call (offset/limit/api version).
Yes, the client makes the first request, but that doesn't mean it knows anything about URL discovery, pagination, limit/offset parameters, etc.
At work (bank) we are re-designing our MW / Web services. We are using bottom-up approach to build those services. We are using Java, jax-ws. So I need to create rules to be followed. So I have 2 questions so far:
Should we create types to our objects fields, i.e. in class Client, should we create a CellPhone object or use simply string for that. I can see the pros & cons , object will be become heavy weight, but easy to validate & control.
Any other ideas ?
Should we be using SOAP built-in fault or create our own error status code (maybe in the SOAP header). I really like the SOAP fault because of the direct mapping to Java Exception.
Thanks in advance
Some answers:
1. Bare in mind that Web Services (I assume you're talking about SOAP-based WS, as you mentioned jax-ws and not jax-rs) use SOAP which is an XML based protocol.
2. For every class you create, you will have a type in your WSDL file.
3. The SOAP envelope (which holds the "body" of your message will also hold additional XML element to denote the cellphone - you're creating more traffic.
To conclude 1 - 3 and the fact you're talking about CellPhone, I don't understand why you need to have class for this.
Are you taking about a CellPhoen class that actually models a CellPhone (i.e - the cell-phone device, with properties like "vendor", "operator" ,etc..) or are you talking about the cell phone number? If this is just Cell-Phone number, then my recommendation, based on 1-3 is still valid.
To handle validation:
You can use many validator fameworks in order to validate phone number.
There is even a JSR in Java for validation.
I recommend you to look at here to read about the Hibernate-Validator framework which conforms to JSR 303.
You can also download the source of oVirt open source project ,
and take a look at oVirt-engine (look at ovirt-engine/backend/manager/modules/common) at our BusinessEntiies and see some "real life" examples on how to use these validators.
Regarding fault - you can create your own Faults, and map them to Java exceptions, I see no harm there.