Can I compose two JSON Schemas in a third one? - jsonschema

I want to describe the JSON my API will return using JSON Schema, referencing the schemas in my OpenAPI configuration file.
I will need to have a different schema for each API method. Let’s say I support GET /people and GET /people/{id}. I know how to define the schema of a "person" once and reference it in both /people and /people/{id} using $ref.
[EDIT: See a (hopefully) clearer example at the end of the post]
What I don’t get is how to define and reuse the structure of my response, that is:
{
"success": true,
"result" : [results]
}
or
{
"success": false,
"message": [string]
}
Using anyOf (both for the success/error format check, and for the results, referencing various schemas (people-multi.json, people-single.json), I can define a "root schema" api-response.json, and I can check the general validity of the JSON response, but it doesn’t allow me to check that the /people call returns an array of people and not a single person, for instance.
How can I define an api-method-people.json that would include the general structure of the response (from an external schema of course, to keep it DRY) and inject another schema in result?
EDIT: A more concrete example (hopefully presented in a clearer way)
I have two JSON schemas describing the response format of my two API methods: method-1.json and method-2.json.
I could define them like this (not a schema here, I’m too lazy):
method-1.json :
{
success: (boolean),
result: { id: (integer), name: (string) }
}
method-2.json :
{
success: (boolean),
result: [ (integer), (integer), ... ]
}
But I don’t want to repeat the structure (first level of the JSON), so I want to extract it in a response-base.json that would be somehow (?) referenced in both method-1.json and method-2.json, instead of defining the success and result properties for every method.
In short, I guess I want some kind of composition or inheritance, as opposed to inclusion (permitted by $ref).

So JSON Schema doesn’t allow this kind of composition, at least in a simple way or before draft 2019-09 (thanks #Relequestual!).
However, I managed to make it work in my case. I first separated the two main cases ("result" vs. "error") in two base schemas api-result.json and api-error.json. (If I want to return an error, I just point to the api-error.json schema.)
In the case of a proper API result, I define a schema for a given operation using allOf and $ref to extend the base result schema, and then redefine the result property:
{
"$schema: "…",
"$id": "…/api-result-get-people.json",
"allOf": [{ "$ref": "api-result.json" }],
"properties": {
"result": {
…
}
}
}
(Edit: I was previously using just $ref at the top level, but it doesn’t seem to work)
This way I can point to this api-result-get-people.json, and check the general structure (success key with a value of true, and a required result key) as well as the specific form of the result for this "get people" API method.

Related

Validating JSON schema with intersections and unions classes in Kotlin

I'm thinking about rewriting small web server project that I have from TypeScript to Kotlin. The problem that I have is server accepts a JSON with unions and intersections. For the purpose of this example:
I want incoming JSON to have a string field with either the name one or name two, but not both or none of them, e.g. {"one": "some text"} or {"two": "some other text"}
I want format of incoming request be dependent on one of its fields, e.g.:
{
"someFlag": false, // or can be missing
"ids": {
"user": 123 // mandatory field for this structure
}
}
or if someFlag exists and true, it makes couple more extra id fields mandatory
{
"someFlag": true
"ids": {
"user": 123,
"company": 456, // mandatory because someFlag is true
"device": 789 // mandatory because someFlag is true
}
}
Unions and intersections within same type is quite trivial in TS, but I have no idea how to achieve same thing in Kotlin and then utilize validated request in a typesafe manner. I'm pretty sure I can try to figure out some "dirty" approach with bunch of optional fields and manual checks for field existence on multiple program levels, but if there's a way to make it typesafe, I'd rather take it.
So the question is, can I make this validation typesafe?

JSON Schema validator compatibility

I'm trying to understand how a single JSON Schema behaves when used in different validators. Some validators define custom keywords. For example ajv validator ajv-keywords package defines a prohibited keyword that is not part of the JSON Schema standard. JSON Schema on the other hand defines the required keyword that would seem to be the polar opposite of prohibited. JSON Schema also defines a oneOf combinator that can be used to validate that the input should match one and only one of several schema definitions.
Consider the following schema example. By reading the json schema specification, I get the impression that the example json schema should validate any json object when used in ajv. However, according to the unknown keyword rules, validators are supposed to ignore any keywords they do not support. So, I imagine that another validator would ignore the custom prohibited keyword, causing the schema to reject an input with property foo. Is this correct or am I failing to read the json schema specification?
{
"oneOf": [
{
"type": "object",
"required": ["foo"]
},
{
"type": "object",
"prohibited": ["foo"]
}
]
}
You are correct. A standard JSON Schema validator will fail validation for an object that has a property "foo". You should be very careful using non-standard keywords if you expect your schemas to be used by standard validators.
It should be okay to use custom keywords as long as you follow the principle of progressive enhancement. Effectively, that means the behavior should degrade as gracefully as possible if the custom keyword is ignored. Your example violates this principle because you end up with a false negative result if prohibited is ignored.
An simple example that does follow progressive enhancement might look like this...
{
"type": "object",
"properties": {
"foo": {}
},
"required": ["foo"],
"prohibited": ["bar"]
}
If I run this through a standard validator, all assertions work as expected except prohibited which is ignored. Assuming a client-server architecture, this allows clients to mostly validate their requests before sending them to the server. The server then does it's own validation with a validator that understands the custom keywords and can respond with an error if "bar" is present.

Use of type : object and properties in JSON schema

I'm new to JSON.
I see in various examples of JSON, like the following, where complex values are prefixed with "type":"object", properties { }
{
"$schema": "http://json-schema.org/draft-06/schema#",
"motor" : {
"type" : "object",
"properties" : {
"class" : "string",
"voltage" : "number",
"amperage" : "number"
}
}
}
I have written JSON without type, object, and properties, like the following.
{
"$schema": "http://json-schema.org/draft-06/schema#",
"motor" : {
"class" : "string",
"voltage" : "number",
"amperage" : "number"
}
}
and submitted to an on-line JSON schema validator with no errors.
What is the purpose of type:object, properties { }? Is it optional?
Yes it is optional, try removing it and use your validator.
{
"$schema": "http://json-schema.org/draft-06/schema#",
"foo": "bar"
}
You actually don't even need to use the $schema keyword i.e. {} is valid json
I would start by understanding what json is, https://www.json.org/ is the best place to start but you may prefer something easier to read like https://www.w3schools.com/js/js_json_intro.asp.
A schema is just a template (or definition) to make sure you're producing valid json for the consumer
As an example let's say you have an application that parses some json and looks for a key named test_score and saves the value (the score) in a database in some table/column. For this example we'll call the table tests and the column score. Since a database column requires a type we'll choose a numeric type, i.e. integer for our score column.
A valid json example for this may look like
{
"test_score": 100
}
Following this example the application would parse the key test_score and save the value 100 to the tests.score database table/column.
But let's say a score is absent so you put in a string i.e "NA"
{
"test_score": "NA"
}
When the application attempts to save NA to the database it will error because NA is a string not an integer which the database expects.
If you put each of those examples into any online json validator they are valid json example. However, while it's valid json to use "NA" or 100 it is not valid for the actual application that needs to consume the json.
So now you may understand that the author of the json may wonder
What are the different valid types I can use as values for my test
score?
The responsibility then falls on the writers of the application to provide some sort of definition (i.e a schema) that the clients (authors) can reference so the author knows exactly how to structure the json so the application can process it accordingly. Having a schema also allows you to validate/test your json so you know it can be processed by the application without actually having to send your json through the application.
So putting it altogether let's say in the schema you see
"$test_score": {
"type": "integer",
"format": "tinyint"
},
The writer of the json now knows that they must pass an integer and the range is 0 to 255 because it's a tinyint. They no longer have to trial by error different values and see which ones the application process. This is a big benefit to having a schema.

Unable to use Ember data with JSONAPI and fragments to support nested JSON data

Overview
I'm using Ember data and have a JSONAPI. Everything works fine until I have a more complex object (let's say an invoice for a generic concept) with an array of items called lineEntries. The line entries are not mapped directly to a table so need to be stored as raw JSON object data. The line entry model also contains default and computed values. I wish to store the list data as a JSON object and then when loaded back from the store that I can manipulate it as normal in Ember as an array of my model.
What I've tried
I've looked at and tried several approaches, the best appear to be (open to suggestions here!):
Fragments
Replace problem models with fragments
I've tried making the line entry model a fragment and then referencing the fragment on the invoice model as a fragmentArray. Line entries add to the array as normal but default values don't work (should they?). It creates the object and I can store it in the backend but when I return it, it fails with either a normalisation issue or a serialiser issue. Can anyone state the format the data be returned in? It's confusing as normalising the data seems to require JSONAPI but the fragment requires JSON serialiser. I've tried several combinations but no luck so far. My line entries don't have actual ids as the data is saved and loaded as a block. Is this an issue?
DS.EmbeddedRecordsMixin
Although not supported in JSONAPI, it sounds possible to use JSONAPI and then switch to JSONSerializer or RESTSerializer for the problem models. If this is possible could someone give me a working example and the JSON format that should be returned by the API? I have header authorisation and other such data so would I still be able to set this at the application level for all request not using my JSONAPI?
Ember-data-save-relationships
I found an add on here that provides an add on to do this. It seems more involved than the other approaches but when I've tried this I can send the data up by setting a the data as embedded. Great! But although it saves it doesn't unwrap it correct and I'm back with the same issues.
Custom serialiser
Replace the models serialiser with something that takes the data and sends it as plain JSON data and then deserialises back into something Ember can use. This sounds similar to the above but I do the heavy lifting. The only reason to do this is because all examples for the above solutions are quite light and don't really show how to set this up with an actual JSONAPI set up that would need it.
Where I am and what I need
Basically all approaches lead to saving the JSON fine but the return JSON from the server not being the correct format or the deserialisation failing but it's unclear what it should be or what needs to change without breaking the existing JSONAPI models that work fine.
If anyone know the format for return API data it may resolve this. I've tried JSONAPI with lineEntries returning the same format as it saved. I've tried placing relationship sections like the add on suggested and I've also tried placing relationship only data against the entries and an include section with all the references. Any help on this would be great as I've learned a lot through this but deadlines a looming and I can't see a viable solution that doesn't break as much as it fixes.
If you are looking for return format for relational data from the API server you need to make sure of the following:
Make sure the relationship is defined in the ember model
Return all successes with a status code of 200
From there you need to make sure you return relational data correctly. If you've set the ember model for the relationship to {async: true} you need only return the id of the relational model - which should also be defined in ember. If you do not set {async: true}, ember expects all relational data to be included.
return data with relationships in JSON API specification
Example:
models\unicorn.js in ember:
import DS from 'ember-data';
export default DS.Model.extend({
user: DS.belongsTo('user', {async: true}),
staticrace: DS.belongsTo('staticrace',{async: true}),
unicornName: DS.attr('string'),
unicornLevel: DS.attr('number'),
experience: DS.attr('number'),
hatchesAt: DS.attr('number'),
isHatched: DS.attr('boolean'),
raceEndsAt: DS.attr('number'),
isRacing: DS.attr('boolean'),
});
in routes\unicorns.js on the api server on GET/:id:
var jsonObject = {
"data": {
"type": "unicorn",
"id": unicorn.dataValues.id,
"attributes": {
"unicorn-name" : unicorn.dataValues.unicornName,
"unicorn-level" : unicorn.dataValues.unicornLevel,
"experience" : unicorn.dataValues.experience,
"hatches-at" : unicorn.dataValues.hatchesAt,
"is-hatched" : unicorn.dataValues.isHatched,
"raceEndsAt" : unicorn.dataValues.raceEndsAt,
"isRacing" : unicorn.dataValues.isRacing
},
"relationships": {
"staticrace": {
"data": {"type": "staticrace", "id" : unicorn.dataValues.staticRaceId}
},
"user":{
"data": {"type": "user", "id" : unicorn.dataValues.userId}
}
}
}
}
res.status(200).json(jsonObject);
In ember, you can call this by chaining model functions. For example when this unicorn goes to race in controllers\unicornracer.js:
raceUnicorn() {
if (this.get('unicornId') === '') {return false}
else {
return this.store.findRecord('unicorn', this.get('unicornId', { backgroundReload: false})).then(unicorn => {
return this.store.findRecord('staticrace', this.get('raceId')).then(staticrace => {
if (unicorn.getProperties('unicornLevel').unicornLevel >= staticrace.getProperties('raceMinimumLevel').raceMinimumLevel) {
unicorn.set('isRacing', true);
unicorn.set('staticrace', staticrace);
unicorn.set('raceEndsAt', Math.floor(Date.now()/1000) + staticrace.get('duration'))
this.set('unicornId', '');
return unicorn.save();
}
else {return false;}
});
});
}
}
The above code sends a PATCH to the api server route unicorns/:id
Final note about GET,POST,DELETE,PATCH:
GET assumes you are getting ALL of the information associated with a model (the example above shows a GET response). This is associated with model.findRecord (GET/:id)(expects one record), model.findAll(GET/)(expects an array of records), model.query(GET/?query=&string=)(expects an array of records), model.queryRecord(GET/?query=&string=)(expects one record)
POST assumes you at least return at least what you POST to the api server from ember , but can also return additional information you created on the apiServer side such as createdAt dates. If the data returned is different from what you used to create the model, it'll update the created model with the returned information. This is associated with model.createRecord(POST/)(expects one record).
DELETE assumes you return the type, and the id of the deleted object, not data or relationships. This is associated with model.deleteRecord(DELETE/:id)(expects one record).
PATCH assumes you return at least what information was changed. If you only change one field, for instance in my unicorn model, the unicornName, it would only PATCH the following:
{
data: {
"type":"unicorn",
"id": req.params.id,
"attributes": {
"unicorn-name" : "This is a new name!"
}
}
}
So it only expects a returned response of at least that, but like POST, you can return other changed items!
I hope this answers your questions about the JSON API adapter. Most of this information was originally gleamed by reading over the specification at http://jsonapi.org/format/ and the ember implementation documentation at https://emberjs.com/api/data/classes/DS.JSONAPIAdapter.html

RESTfully handling sub-resources

I've been creating a RESTful application and am undecided over how I should handle requests that don't return all entities of a resource or return multiple resources (a GET /resource/all request). Please allow me a few moments to setup the situation (I'll try to generalize this as much as possible so it can apply to others besides me):
Let's say I'm creating a product API. For simplicity, let's say it returns JSON (after the proper accept headers are sent). Products can be accessed at /product/[id]. Products have reviews which can be accessed at /products/[id]/review/[id].
My first question lies in this sub-resource pattern. Since you may not always want the reviews when you GET a product, they are accessible by another URI. From what I read I should include the URI of the request that will return all review URI's for a product in the response for a product request. How should I go about this so that it abides to RESTful standards? Should it be a header like Reviews-URI: /product/123/review/all or should I include the URL in the response body like so:
{ 'name': 'Shamwow',
'price': '$14.99',
'reviews': '/product/123/review/all'
}
My second question is about how the /product/[id]/review/all request should function. I've heard that I should just send the URL's of all of the reviews and make the user GET each of them instead of packaging all of them into one request. How should I indicate this array of review URIs according to RESTful standards? Should I use a header or list the URIs in the response body like so:
{ 'reviews': [ '/product/123/review/1',
'/product/123/review/2',
'/product/123/review/3'
]
}
Your problem is you're not using Hypermedia. Hypermedia specifically has elements that hold links to other things.
You should consider HAL, as this is a Hypermedia content type that happens to also be in JSON.
Then you can leverage the links within HAL to provide references to your reviews.
As to your first question (header or body), definitely do not invent your own custom header. Some here will argue that you should use the Link header, but I think you'll find plenty of need for nested links and should keep them in the body.
How you indicate either the URI to the reviews/ resource, or the list of URI's within that, is entirely up to the media type you select to represent each resource. If you're using HTML, for example, you can use an anchor tag. If you're using plain JSON, which has no hypermedia syntax, you'll have to spend some time in the documentation for your API describing which values are URI's, either by nominating them with special keys, or wrapping them in special syntax like {"link": "reviews/123"}, or with a related schema document.
Take a look at Shoji, a JSON-based media type which was designed explicitly for this pattern of subresources.
The JSON Schema standard might help you here, in particular Hyper-Schemas.
It lets you define how to extract link URIs from your data, and what their "rel"s are - essentially turning your JSON data into hyper-media. So for your first bit of data, you might write a schema like:
{
"title": "Product",
"type": "object",
"properties": {...},
"links": [
{"rel": "reviews", "href": "{reviews}"}
]
}
The value of href is a URI Template - so for example, if your data included productId, then you could replace the value of href with "/product/{productId}/review/all".
For the second bit of example data (the list of reviews) you might have a schema like this:
{
"type": "object",
"properties": {
"reviews": {
"type": "array",
"items": {
"links": [
{"rel": "full", "href": "{$}"}
]
}
}
}
}
In the URI Template of href, the special value of {$} means "the value of the JSON node itself". So that Hyper-Schema specifies that each item in the reviews array should be replaced with the data at the specified URL (rel="full").