I'm building a REST API powered SPA application and I'm trying to decide on the best way to deliver "global options" via the API. By global options I mean an assortment of random fields that relate to the application as a whole rather than being associated with one specific model, for example brand logos and contact details that need to be accessible from multiple locations within the app.
Something like Wordpress would store these in an options table and access them via a php function using the option name, however since this is a REST API I'm not sure how I would go about accessing/updating multiple options without making lots of requests for each one.
I know a lot of projects just use a json file to store this data, but it specifcally needs to be editable via a CMS and served via the API. The following are two methods that have come to mind, but none of them feel like complete solutions:
1: An options table with one generic endpoint that takes a query string specifying which fields you want to access. This works for getting data, however updating data seems to get a bit messy and the only way I can think to do this is by sending an object of key/values pairs to bulk create or update the options:
GET: example.com/api/options?pick=logo,contact_phone,contact_email
POST: example.com/api/options
{
contact_email: "info#example.com",
contact_address: "123 Test St"
}
PUT: example.com/api/options
{
contact_email: "info#example.com",
contact_address: "123 Test St"
}
2: Breaking fields into groups and storing them as a json field in a "pages" table, this solves the creating and updating issues but breaks down when you have fields are used in multiple locations, and they aren't really pages so it's not very REST like.
GET: example.com/api/pages/contact
POST: example.com/api/pages
{
name: "contact",
values: {
email: "info#example.com",
address: "123 Test St",
}
}
PUT: example.com/api/pages/contact
{
values: {
email: "info#example.com",
address: "123 Test St",
}
}
I also need to take into account the issue of access permissions, for example the logo field would be accessible to the public, but the user support contact number would only be accessible to logged in users. With the first example you could have an extra permission column for each option, but this wouldn't work for the second option.
I've been googling but have failed to find any good information about this topic as REST schemas/documention generally only deal with concreate entities, so any insight into how this is achieved in real world applciations would be great!
Cheers,
Cam
Related
I have an API with endpoint GET /users/{id} which returns a User object. The User object can contain sensitive fields such as cardLast4, cardBrand, etc.
{
firstName: ...,
lastName: ...,
cardLast4: ...,
cardBrand: ...
}
If the user calls that endpoint with their own ID, all fields should be visible. However, if it is someone elses ID then cardLast4 and cardBrand should be hidden.
I want to know what are the best practices here for designing my response. I see three options:
Option 1. Two DTOs, one with all fields and one without the hidden fields:
// OtherUserDTO
{
firstName: ...,
lastName: ..., // cardLast4 and cardBrand hidden
}
I can see this becoming out of hand with DTOs based on role, what if now I have UserDTOForAdminRole, UserDTOForAccountingRole, etc... It looks like it quickly gets out of hand with the number of potential DTOs.
Option 2. One response object being the User, but null out the values that the user should not be able to see.
{
firstName: ...,
lastName: ...,
cardLast4: null, // hidden
cardBrand: null // hidden
}
Option 3. Create another endpoint such as /payment-methods?userId={userId} even though PaymentMethod is not an entity in my database. This will now require 2 api calls to get all the data. If the userId is not their own, it will return 403 forbidden.
{
cardLast4: ...,
cardBrand: ...
}
What are the best practices here?
You're gonna get different opinions about this, but I feel that doing a GET request on some endpoint, and getting a different shape of data depending on the authorization status can be confusing.
So I would be tempted, if it's reasonable to do this, to expose the privileged data via a secondary endpoint. Either by just exposing the private properties there, or by having 2 distinct endpoints, one with the unprivileged data and a second that repeats the data + the new private properties.
I tend to go for option 1 here, because an API endpoint is not just a means to get data. The URI is an identity, so I would want /users/123 to mean the same thing everywhere, and have a second /users/123/secret-properties
I have an API with endpoint GET /users/{id} which returns a User object.
In general, it may help to reframe your thinking -- resources in REST are generalizations of documents (think "web pages"), not generalizations of objects. "HTTP is an application protocol whose application domain is the transfer of documents over a network" -- Jim Webber, 2011
If the user calls that endpoint with their own ID, all fields should be visible. However, if it is someone elses ID then cardLast4 and cardBrand should be hidden.
Big picture view: in HTTP, you've got a bit of tension between privacy (only show documents with sensitive information to people allowed access) and caching (save bandwidth and server pressure by using copies of documents to satisfy more than one request).
Cache is an important architectural constraint in the REST architectural style; that's the bit that puts the "web scale" in the world wide web.
OK, good news first -- HTTP has special rules for caching web requests with Authorization headers. Unless you deliberately opt-in to allowing the responses to be re-used, you don't have to worry the caching.
Treating the two different views as two different documents, with different identifiers, makes almost everything easier -- the public documents are available to the public, the sensitive documents are locked down, operators looking at traffic in the log can distinguish the two different views because the logged identifier is different, and so on.
The thing that isn't easier: the case where someone is editing (POST/PUT/PATCH) one document and expecting to see the changes appear in the other. Cache-invalidation is one of the two hard problems in computer science. HTTP doesn't have a general purpose mechanism that allows the origin server to mark arbitrary documents for invalidation - successful unsafe requests will invalidate the effective-target-uri, the Location, the Content-Location, and that's it... and all three of those values have other important uses, making them more challenging to game.
Documents with different absolute-uri are different documents, and those documents, once copied from the origin server, can get out of sync.
This is the option I would normally choose - a client looking at cached copies of a document isn't seeing changes made by the server
OK, you decide that you don't like those trade offs. Can we do it with just one resource identifier? You immediately lose some clarity in your general purpose logs, but perhaps a bespoke logging system will get you past that.
You probably also have to dump public caching at this point. The only general purpose header that changes between the user allowed to look at the sensitive information and the user who isn't? That's the authorization header, and there's no "Vary" mechanism on authorization.
You've also got something of a challenge for the user who is making changes to the sensitive copy, but wants to now review the public copy (to make sure nothing leaked? or to make sure that the publicly visible changes took hold?)
There's no general purpose header for "show me the public version", so either you need to use a non standard header (which general purpose components will ignore), or you need to try standardizing something and then driving adoption by the implementors of general purpose components. It's doable (PATCH happened, after all) but it's a lot of work.
The other trick you can try is to play games with Content-Type and the Accept header -- perhaps clients use something normal for the public version (ex application/json), and a specialized type for the sensitive version (application/prs.example-sensitive+json).
That would allow the origin server to use the Vary header to indicate that the response is only suitable if the same accept headers are used.
Once again, general purpose components aren't going to know about your bespoke content-type, and are never going to ask for it.
The standardization route really isn't going to help you here, because the thing you really need is that clients discriminate between the two modes, where general purpose components today are trying to use that channel to advertise all of the standardized representations that they can handle.
I don't think this actually gets you anywhere that you can't fake more easily with a bespoke header.
REST leans heavily into the idea of using readily standardizable forms; if you think this is a general problem that could potentially apply to all resources in the world, then a header is the right way to go. So a reasonable approach would be to try a custom header, and get a bunch of experience with it, then try writing something up and getting everybody to buy in.
If you want something that just works with the out of the box web that we have today, use two different URI and move on to solving important problems.
I've been using Netlify for storing 100% of my app (both frontend and backend) for the last three months. So far, so good.
The only problem now is that I need to store a custom property for each user (say, the phone number), and apparently Netlify Identity doesn't support this (only email, name and roles https://www.netlify.com/docs/identity/).
I don't want to change the whole app to migrate to another hosting provider just for this detail (actually, I can't, it's for a client and I just don't have time), because it works great, but at the same time I need it.
Can you think of any workaround to this? The less "hackish", the better, but I understand that I'm going beyond the intended use of Netlify Identity.
So it actually does look like Netlify's GoTrue API has a specific endpoint for updating custom user data. After a user is created, you can update metadata by including it as "data" within an authenticated PUT request to /user.
PUT /user
{
"data" {
"custom_key": "value",
}
}
See https://github.com/netlify/gotrue for more info.
There are dozens of ways to do this, so I'll talk about two generally applicable ways now:
the most "generally capable" one is probably using lambda functions: https://www.netlify.com/docs/functions . This lets you run dynamic code, such as "store to database hosted elsewhere" or "email to our office manager to update a spreadsheet" or even "commit to our closed git repo so it's available in-code" (last one is probably a worst practice, but is possible). You can similarly use a function to read that data back out without exposing API tokens (code example: https://github.com/netlify/code-examples/tree/master/function_examples/token-hider)
you could have the data gathered via a form submission (https://www.netlify.com/docs/form-handling). I'd probably use zapier.com to receive a notification of the form submission (https://www.netlify.com/docs/form-handling/#notifications). Zapier can of course connect to just about anything on the planet :) . Getting the data back out if you want to show it in your UI is a bit more of a challenge, but you could use the above mentioned functions if you need to connect to some private data store to pull it out. Or for an MVP, just not show it, only let people enter/update it ;)
im building a specific book reader like app.
Main page call api/booksList and receive the json array containing each book info like:
[ { id: server_db_id, title: "title test", sum: 10 , date: ... }
]
ans its cached after the request, so im not saving the book list into indexedDB, localStorage or other storage. If i need one specific book, i just call the api book list again and filter it. Is that bad design? (book will be over 200 items)
Whe user open the book, it call the /api/book/book_id and its cached too, the opened book response is a json list of the lines of the book, eg:
[
{
id: ...
content: "This is line...lore ipsum..."
....
}
]
I put the api response inside vue data variable and the component is rendered correclty
Im not using any kind of handler for keeping this offline by my hand. To detect if user already opened this book, i just call the api, check if errors happened or the responde body has content.
Is that a wrong, bad or stupid decision? Will this hit the quota limit api or other kind of limitation? The "gods" of pwa will raise the finger to me and say: WAAAT. (im not using indexedDB at first because it need some models handling and i want to make things easier if possible)
I my self was just researching this and concluded, at the moment I am going to go with this method, where I use cache for assets, js, css, html etc based on their matching routes.
Then when it comes to data e.g. json requests etc. Its best to store them in indexedDB (or an equivalent), which really does not require a model or schema as such.
See Jake Archibald's IndexedDB-Promise library https://github.com/jakearchibald/idb its really simple to get your head round.
Though both Jake and Addy say it's not a defacto rule, so you can decide ultimately what is best for you.
Read this for better clarification
https://developers.google.com/web/ilt/pwa/live-data-in-the-service-worker
https://medium.com/dev-channel/offline-storage-for-progressive-web-apps-70d52695513c
It helped me to make a better decision on how to go about moving forward.
Recommendations Also
Check out PWA Training: https://developers.google.com/web/ilt/pwa
Workbox: https://developers.google.com/web/tools/workbox (This has sped up my development massively!)
Codelabs: https://codelabs.developers.google.com/ (Search PWA)
The guides on here are really good at taking you through everything you need.
Good Luck with your PWA
Random thought (edit)
One thing that makes me question this though is based on some of the examples and guides I have seen is that, data storage is handled in a more ad-hoc manner. For example, if the PWA calls out an API, there are two methods I have come across where you can either manage cached data in the application or in the service worker, e.g. if your API calls to get JSON fails in the app, it can revert to getting data in the indexedDB which hopefully was pre-cached the first time your app called the API.
Or you can use self.addEventListener('fetch', (event) => { ad-hoc stuff here }) this is where you can match either an asset, or data request and hijack the response with either a cache or indexedDB response. Which prevents the need handle offline data in your app.
The first method makes me feel uneasy so i'm gonna go with the addEventListener approach both in the service worker cause thats what it is there for plus my app does not then have to worry about that.
Let's assume that I have a resource called group with the following representation:
{
"id": 1,
"name": "Superheroes"
"_links": {
"self": {
"href": "http://my.api.com/groups/1"
}
}
}
Now let's say I want to create a new person instance by POSTing to /persons/1. Which of the following should I use for the request body:
Using ID
{
"name": "Batman",
"groupId": 1
}
Using link
{
"name": "Batman",
"group": "http://my.api.com/groups/1"
}
With the first method I access the id directly either to look up the related resource or eventually store the id in the database, when I persist the person instance. But with the other method, I either have to extract the id from the URI or, follow the link to load the related resource, and then find out its id. I really don't want to store the URI in the database.
With the latter option, seeing that the server controls the structure of the URI, is it fine for me to parse the id out of the link? Following the link back to the server itself seems odd, seeing that at this point we already have access to the information directly (we just need the id).
So to sum up, which of these options is best?
Use the id directly.
Use the link, but parse out the id.
Use the link, but access the link to get the resource instance, and then get the id.
TL;DR: Use simple ids.
More detailed explanation:
A straightforward approach is to create a person by POSTing to /groups/1/persons with a payload {"name": "Batman"}.
However, while such approach works for simple cases, the situation gets complicated if there are 2 resources that need to be referenced. Let's assume that a person also needs to belong to exactly one company:
GET /persons/1
{
"name": "Batman",
"group": 1, // Superheros, available at /groups/1
"company": 5 // Wayne Enterprises, available at /companies/5
}
Since there is no relationship between companies and groups, it is not semantically correct to create a person via POSTing to /groups/1/companies/5/persons or to /companies/5/groups/1/persons.
So let's assume you want to create a person with a request looking like this:
POST /persons
{
"name": "Batman"
"group": ???, // <--- What to put here?
"company": ??? // <--- What to put here?
}
Which brings us to the answer to your question:
Ease of use. Your API should be primarily designed for the ease of use. This is especially true, if you design a public API. Therefore, Option 2 (Use the link, but parse out the id) is out, since it imposes additional work for clients of your API.
Constructing search queries. If you want to be able to query persons which belong to the company 10 and the group 42, simple ids lead to more readable and less error-prone urls. Which of the following do you consider more readable?
URL with a simple id:
GET /groups/42?company=10
or URL with a url-encoded link:
GET /groups/42?company=http%3A%2F%2Fmy.api.com%2Fcompanies%2F10
I wouldn't underestimate the point of readability. How many times do you need to debug your API in various curls, logs, postmans, etc.
Development Links need to be parsed in the backend, while simple ids can be used directly. It's not about performance, but rather about additional work/tests you have to put in.
Endpoint maintenance. Imagine that your API endpoint evolves. You decide one day to switch to https or to include versioning in the url. This might break API clients, if they for some reason rely on structure of the links. Also, you might want to checkout if link parsing on your backend is done properly.
Argumentum ab auctoritate I know this is not a proper argument, but if you checkout APIs of large players, e.g. Twitter, Github or Stripe, they all use simple ids.
HATEOAS. One common argument in favour of links is that it is aligned with HATEOAS. However, as far as I know, this relates to additional links in API responses rather than using links in payloads of POST requests.
All in all, I would go for simple ids, since I haven't yet heard a compelling argument favouring links, which would beat the aforementioned.
You are missing two important things here.
You need a standard way to describe forms in the response, in this case your POST form.
The information about the group ids / uris, or how to get them has to be described in the form in a standard way.
For example a HTML FORM with a SELECT INPUT would be RESTful. The closest thing we got in json to do the same is json-ld and hydra. But if you are obsessed with hal, then use hyperagent forms or something like that. It will never be a standard, but if compatibility is not an issue, then it is good enough.
To answer your question, you should use the id, because the server knows how to interpret it. The client needs the resource identifiers, the server needs it only in the uri part of the request, not in the body.
From my experience, it is always best to go with the simplest solution for making requests.
The process of generating a new url and parsing it seems excessive to get a resource, whereas sending the id of the item you want seems much simpler.
Thus, I would send a request in the form:
{
"name": "Batman",
"group": 1
}
A few general design questions:
Give the example here:
https://developers.google.com/+/api/latest/activities/list#nextPageToken
Why would the server return a token to retreive the next paginated result? Doesn't this break the idea of being stateless?
Why not just pass a MySQL like LIMIT name=value as the parameters? The server now has to return the number of pages I suppose...what am I missing?
I read many but this one was of interest:
REST Web Services API Design
The second reply, offers the following examples.
GET http://api.domain.com/user/<id>
GET http://api.domain.com/users
PUT http://api.domain.com/user/<id>
POST http://api.domain.com/users
DELETE http://api.domain.com/user/<id>
Makes sense but why are there two plural resources? Could one not assume that if "user" is queried and was NULL or not provided that "all" was intended? Likewise for POST? If plural is for improved readability - why is there not a "users" resource for DELETE?
Ultimately, I understand REST to mean...representation of a single resource - using HTTP verbs (GET, PUT, POST, DELETE) to essentially manage that resource - similar to CRUD.
EDIT | Lastly I also wanted to ask why Google API sends the API version in the URI instead of using HTTP headers? Is there a reason? For backwards compat with older clients?
Comments?
Why would the server return a token to retrieve the next paginated result? Doesn't this break the idea of being stateless?
Using this kind of mechanism for paginated result sets is completely standard and does not break the idea of being stateless. Consider the following example.
Suppose GET /users?after=<after> (where after is optional) is supposed to return the list of all users in a paginated fashion, say <= 4 per page.
The first request a client makes is GET /users with a response that might look like the following (formatted as JSON).
{
"users": [ "alex", "bob", "carter", "dan" ]
"more_after": "dan"
}
In this example, the more_after property designates there may be more users left in the user list. So the client then requests GET /users?after=dan and gets a second response that looks like the following.
{
"users": [ "edward", "frank" ]
}
The absence of the more_after property designates that this is the last page of users.
Now the question is: was the "dan" token used as the page separator something that breaks the "statelessness" property we want? Clearly the answer is no. The server doesn't have to remember anything between the two GET requests. There's no concept of a session. Any state that needs to persist between the two GET requests exists only client-side - that's the important distinction. It's completely acceptable - and often required - to have the client persist state between calls to the service.