Single endpoint + huge payload - api

Let's say we have to display (in the UI) a list of invoices and we know each invoice has a list of articles.
I could design my API like this
One endpoint to retrieve the list of invoices and their articles
/endpoints
Two separate endpoints
/endpoints
/endpoints/{invoice_id}/articles
I know each has its own pros and cons and that this is kind of subjective.
This API will be called solely by a frontend application written in React.
I am going with one endpoint that fetches all the information rather than having separate endpoints with separate calls.
My concern is that the payload might be too huge. Is there a limit in the size of the payload transmitted over the wire from the backend to the frontend?

Related

API design pattern to be integrated both by own web app and other systems

So this backend will be consumed by an ad-hoc front end application. But will also be integrated by other systems and we will expose API for them.
When designing the rest I see that there is ONE database table (we call it for table A) that can join many other tables, lets say about 10 to 20 other tables.
Now, my strategy would be to build routes in my backend that will "reason" according to the ad-hoc frontend we have.
So if there is a page in the frontend (let's call this page for page1) that requires to get rows from the table A but also fields from let's say 3 other join tables, then I want to create a route in the backend called maybe "page1" which will return rows from table A and also from the other 3 tables.
This is of course an ordinary way to build a backend. But as it will also be used by other systems then somebody could argue that these systems maybe don't have any need for the route "page1". Their frontend will maybe never build a "page1".
So according to people here, it would better to build the API more agnostically. And instead of creating the route "page1" I should build it according to "hateoas". And if I understand that principle, instead of letting my ad-hoc frontend to request the resource "page1" it would request "pageForTableA". And then, the resource "pageForTableA" should return which are the possible table to be joined.
In this case, for my frontend's page1, I would need to make 4 subsequent request to the server, instead of one like I would like to do if there was a "page1" resource in the backend.
What do you think?
I also see a thirt strategy. I don't know if there is a name for this pattern but it would be this way:
A resource in backend that returns only rows from table A. BUT, the route also takes arguments. And the argument is an array with the name of all the other tables someone want to include.
So if frontend calls:
getTableA(array('tableB', 'tableD', 'tableF'))
Then the resource would include/join the tables B, D and F. In short: API resource let's the frontend decide what it want to get delivered.
Which of these 3 strategies are best do you think? Or there is some more that could be taken in consideration?
You need to architect your API in a way that consumers shouldn't know about how the data is stored in the underlying data store.
Furthermore, if you want to allow consumers to decide which fields you want to project in the response, you could give them using some query string format.
BTW, maybe you should avoid re-inventing the wheel. There's a standard called Open Data (OData) which already defines a lot of things like you already require in your API, and since it has been made by Microsoft, it has deep support on .NET.
Check this tutorial (Create an OData v4 Endpoint Using ASP.NET Web API 2.2) to get more in touch with OData.

How can I represent a model with multiple different serializations in Ember?

I have a Client model. When viewing /clients, I want to return a simple serialization of my clients, with just a few aggregate values (lets say total_unbilled and total_owing). When viewing /clients/1, I want to return a full serialization of the client, including all it's nested tasks and expenses. The back-end has already been configured to do this.
I don't want to return the full serialization of all clients when the user views /clients, as there can be a lot of data under potentially hundreds of clients. I'd like to load that extra information only when needed, when the user views a particular client.
What's the best way to handle this use-case, where models can be serialized in multiple ways, using Ember Data? I know it will cache the initial representation of the client, so if the user visits /clients first, it won't ever try to fetch the full serialization of the client if the user then visits /clients/1. Is there a sensible way to override this? Or would I have to have two different Ember Data models client-side (eg. Client and MiniClient)?
Honestly the easiest is to use two different models, or just don't use Ember Data for the mini clients. It sounds like they won't be used for much more than just an info.
I'd possibly just do POJOs for the mini client, and Ember Data for the full client (since caching would be most useful at that point) Ember without Ember Data

Bigcommerce - request products based on a list of IDs

I am using the Bigcommerce API to develop a small standalone application for a client. I store product information in a local database anytime I fetch products from Bigcommerce, to reduce latency and network load. However, products can change on Bigcommerce, and while it is acceptable for my application to show mildly outdated information, I will need to update my local cache at some point. My current plan is to do this by storing the original date I requested the product, after which I will need to perform another request to refresh the cache.
My question is, given a list of products (including their Bigcommerce IDs), is there a way to request updates to all of them through a single call to the Products Resource? I can make a request for each individual product by calling:
GET {api}/v2/products/{id}
I can also request all products within an unbroken ID range by calling:
GET {api}/v2/products?min_id={value}&max_id={value}
I am able to successfully call both of the above methods, and I can chain them together in loops to fetch all products. What I want to do is request multiple products with unrelated IDs in a single call. So, something like this:
//THIS IS NOT A REAL METHOD!
GET {api}/v2/products?id[]={value1}&id[]={value2}
Is there any way I can do this? Or is there another approach to solving this that I haven't considered? My main requirements are:
Minimal API requests. My application is small but my client's bigcommerce store is not, and I will be processing tens of thousands of products. I have limited CPU and network resources available, and I simply cannot process that many requests.
Scalable. As I said, my client's store is large, and growing. I need a solution whose overhead scales at a manageable rate with number of products.
Note: my application is a small web application written in PHP running on a Linux shared hosting environment. It is a back of house system which will likely only be used by single user at a time, during standard business hours. I haven't tagged the question with PHP because my question is about the API, which is language agnostic.
One approch can be.
First get all products from BigCommerce using simple products call.
Set some interval time to get updated product list.
You can use min_date_modified and max_date_modified OR min_date_created and max_date_created in products API call to get updated products details.

In SOA architecture should single API do everything or API should be split as multiple action

We have an app which is exposing a RESTful API to a UI for purchasing an item. I have a question regarding API design. Lets say the following action should be taken in order
Item to be chosen for purchase
Next give the address to be delivered to
My question is: should we design a single API which gets both data perform both? Or should we design two API calls - one that creates a purchase record and second that update the address to be delivered to?
The recommended SOA approach is to choose coarse-grained services, which would seem to argue the minimum amount of API calls.
However, from a business point of view, item selection and purchase and item delivery are two very different concerns that should be separate components in your architecture. For item selection, you need to consider things like inventory and pricing. For delivery address, you need to consider user address lists, address validation, shipping, and taxation.
Those two components are not likely to interact much except maybe some external association between an item id and address id. For this reason, I'd recommend two API calls. Functionally, this would also allow your API users do things like update the delivery address without re-purchasing an item, send the bill to one address and the item to another address, etc.
As you state that you design a RESTful API you usually start by designing resources instead of the intended calls. Later on, resource representations can be chosen that include other resources for optimised HTTP request counts.
You might want to choose to proceed the following way:
Model the item list resource (GET - lists all items, POST - allows item creation) /items/
Model the address list resource /addresses/
Model the item instance resource /items/item/resourceId
Model the address instance resource /addresses/address/resourceId
Now all of your resources are available you can think about usage patterns. All your resources are generically available and can be mashed up.
Possible approaches to answer your questions would be:
Extend the item instance resource with the needed address details (coarse grained as lreeder stated)
Model a resource /deliverydetails/ as list and instance resource containing item(s) and address, make the deliverydetails to be queried by item id or whatever your use case fits best
Hope that helps!
Btw. you are automatically following SOA approaches with a Resource Oriented Design. Interfaces will be naturally fitting your business requirements and generic enough to support more advanced use cases.
Here is a good example

How best to notify clients of changes in data via API

I have an internal API for my company that contains a large amount of factual data (80MM records as of right now). I have four clients that connect to me on a regular basis. The main API call adds a new item to the database, verifies its authenticity, and then returns structured, analyzed data based on the item submitted.
Over time, as we identify more data to be associated with an item, I need to be able to let my clients know that records have changed.
Right now I have a /recent endpoint, which returns all of the records that have changed since $timestamp. This is fine for small data sets, but given the large number of transactions, one could easily wind up with a /recent dataset of over a million items, especially if there's a large data import.
Another idea I had was to use web hooks to push data to the clients, but then the problem becomes pushing too much data. My clients don't necessarily need updates for every single item that changed -- maybe they only need ones they've already submitted.
The question is less about code and more about design patterns or code strategies:
What are some optimal strategies for notifying my clients of updated records without flooding my clients with unnecessary requests or providing millions of records on a poll?
I've used 3rd party APIs (such as Amazon) that paginate large requests. If the data set exceeds the page limit the client needs to make another a request for the next page. This would be in combination with the /recent endpoint.
The actual implementation would be something like
{
requestId: "foobar",
page: 0,
pages: 10,
data: {
...
}
}
The client makes the request and gets the first page of data, then sends to an endpoint the requstId and the page number. Somehow you'd want to persist a reference to what data corresponds to a requestId.