Do you plan to allow the creation of multiple objects in one only call? For example, currently if I want to create 50 devices (by import), I need to call the API 50 times.
I think it can load the server more unnecessarily that if all objects are contained in the same call.
For a project we don't want to communicate the measurements in real time (every seconds) but postpone the storage in cumulocity. So potentially we need to create ~4000 measurements in one time every hours. Is this approach realistic?
sure, there's no problem with this approach. It also permits you to optimise your mobile bandwidth, if you send the data over a mobile data channel. POST a measurement collection instead of a single measurement, i.e., use
Content-Type: application/vnd.com.nsn.cumulocity.measurementCollection+json
and in the body, use
{ "measurements": [ { ... first measurement ... }, { ... second measurement ... }, ... ] }
If you plan to create a large number of measurements at the same time and on a regular base on our public production system, we appreciate an advance note for capacity provisioning.
There's currently no bulk API for creating multiple managed objects in the same call. It's not been a bottleneck for our customers in practical roll-out scenarios.
However, there's an API for bulk registration of devices. Maybe that helps? It's used by the upload button on the device registration page, and is described here: https://cumulocity.com/guides/reference/device-credentials/ ("Bulk device credentials")
Cheers,
André
Related
For testing API, I need to set some data.
Is it better (cheaper and faster) to import data through the app like regular user. Using frontend which will set and store the data into data base, and I will test that data and have it stored forever, until I decide that data needs to be changed.
Or is it better to do a POST method with API and set data through the API.
I assume that programing API POST method is more expensive and time consuming then using regular app. And also data is stored only while test is runing (that is less memory on other hand, but assume that few MB of DB is not giving higher advantage).
Maybe I am missing something.
Uploading anything through UI is more costlier and slower than uploading through API its not the other way:
Thats why we have the inverted pyramid tobe anti pattern in testing:
Ideal pattern is the testing pyramid or inverted cone
We(producer for the API) have an endpoint
/users/:{id}/name
which is used to retrieve name for the user 'id'
Now as a consumer I want to display the list of names for users like:
user1: id1, name1
uder2: id2, name2
where I have the ids in input.
In such a case should I make 2(here the list is dependent on UI pagination example 50) separate calls to the API to fetch information or else create/ask the producer to create a bulk endpoint like:
POST /users/name
body: { ids: []}
If later, then am I not loosing the REST principle here to fetch information using POST but not GET? If former, then I am not putting too much network overhead in the system?
Also since this seems to be a very common usecase, if we choose the POST method is there really a need of the GET endpoint since the POST endpoint can handle a single user as well?
Which approach should be chosen?
A GET API call should be used for fetching data. Since browser knows GET calls are idempotent, it can handle some situations on its own, like make another call if previous fails.
Since REST calls are lightweight, we tend to overuse API call repeatedly. In your case, since you want all name v/s id mapping at once, one call is sufficient. Or have a Aggregator endpoint in backend API gateway to reduce network traffic and make repeated calls nearer to actual service.
Keeping GET /users/:{id}/name , is also not a bad idea alongside this. It helps to abstract business functionality. A particular scenario can only allow single fetch.
Also using GET /users/name with pagination and returning list of names is complex for single use so keeping both are fine.
Is there any way of filtering the events in a projection associated with a read model by the aggregateId?
In the tests carried out we always receive all registered events. Is it possible to apply filters in a previous stage?
We have 100,000 aggregateId and each id has associated 15,000 events. Unable to filter by aggregateId, our projections have to iterate over all events.
So you have 100.000 aggregates with 15.000 events each.
You can use ReadModel or ViewModel:
Read Model:
Read model can be seen as a read database for your app. So if you want to store some data about each aggregate, you should insert/update row or entry in some table for each aggregate, see Hacker News example read model code.
It is important to understand that resolve read models are built on demand - on the first query. If you have a lot of events, it may take some time.
Another thing to consider - a newly created resolve app is configured to use in-memory database for read models, so on each app start you will have it rebuilt.
If you have a lot of events, and don't want to wait to wait for read models to build each time you start the app, you have to configure a real database storage for your read models.
Configiuring adapters is not well documented, we'll fix this. Here is what you need to write in the relevant config file for mongoDB:
readModelAdapters: [
{
name: 'default',
module: 'resolve-readmodel-mongo',
options: {
url: 'mongodb://127.0.0.1:27017/MyDatabaseName',
}
}
]
Since you have a database engine, you can use it for an event store too:
storageAdapter: {
module: 'resolve-storage-mongo',
options: {
url: 'mongodb://127.0.0.1:27017/MyDatabaseName',
collectionName: 'Events'
}
}
ViewModel
ViewModel is built on the fly during the query. It does not require a storage, but it reads all events for the given aggregateId.
reSolve view models are using snapshots. So if you have 15.000 events for a give aggregate, then on the first request all those events will be applied to calculate a vies state for the first time. After this, this state will be saved, and all subsequent requests will read a snapshot and all later events. By default snapshot is done per 100 events. So on the second query reSolve would read a snapshot for this view model, and apply not more than 100 events to it.
Again, keep in mind, that if you want snapshot storage to be persistent, you should configure a snapshot adapter:
snapshotAdapter: {
module: 'resolve-snapshot-lite',
options: {
pathToFile: 'path/to/file',
bucketSize: 100
}
}
ViewModel has one more benefit - if you use resolve-redux middleware on the client, it will be kept up-to-date there, reactively applying events that app is receiving via websockets.
I'm currently creating an application (let's say, notes app, for instance - webapplication + mobile app). I wanted to use RESTful API here, so I read a lot about this topic and I found out there's a lot of ambiguity over there.
So let's start at the beginning. And the beginning in REST is that we have to first request the / (root), then it returns list of paths we can further retrieve, etc, etc. Isn't this the the first part where REST is completely wasteful? Instead of rigid paths, we have to obtain them each time we want to do something. Nah.
The second problem I encountered was bulk operations. How to implement them in REST? Let's say, user didn't have access to the internet for a while, made a few changes and they all have to be made on server as well. So, let's say user modified 50 notes, added 30 and removed 20. We have to make 100 separate requests now. A way to make bulk operations would be very helpful - I saw this stackoverflow topic: Patterns for handling batch operations in REST web services? but I didn't found anything interesting here actually. Everything is okay as long as you want to do one type of operation on one type of resources.
Last, but not least - retrieving whole collection of items. When writing an example app I mentioned - notes app - you probably want to retrieve all collection of items (notes, tags, available notes colors, etc...) at once. With REST, you have to first retrieve list of item links, then fetch the items one by one. 100 notes = over 100 requests.
Since I'm currently learning all this REST stuff, I may be completely wrong at what I said here. Anyway, the more I read about it, the more gruesome it looks like for me. So my question finally is: where am I wrong and/or how to solve problems I mentioned?
It's all about resources. Resources that are obtained through a uniform interface (usually via URI and HTTP methods).
You do not have to navigate through root every time. A good interface keeps their URIs alive forever (if they go stale, they should return HTTP Moved or similar). Root offering pathways to navigate is a part of HATEOAS, one of Roy Fieldings defined architectural elements of REST.
Bulk operations are a thing the architectural style is not strong on. Basically nothing is stopping you to POST a payload containing multiple items to a specific resource. Again, it's all up to what resource you are using/offering and ultimately, how your server implementation handles requests. Your case of 100 requests: I would probably stick with 100 requests. It is clean to write and the overhead is not that huge.
Retrieving a collection: It's about resources what the API decides to offer. GET bookCollection/ vs GET book/1 , GET/book/2 ... or even GET book/all. Or maybe GET book/?includeDetails=true to return all books with same detail as GETting them one-by-one.
I think that this link could give you interesting hints to design a RESTful service: https://templth.wordpress.com/2014/12/15/designing-a-web-api/.
That said, here are my answers to your questions:
There is no need to implement a resource for the root path. With this, I think that you refered to HATEOS. In addition, no link within the payload is also required. Otherwise you can use available formats like Swagger or RAML to document your RESTful service. This show to your end users what is available.
Regarding bulk operations, you are free to use methods POST or PATCH to implement this on the list resource. I think that these two answers could be helpful to you:
REST API - Bulk Create or Update in single request - REST API - Bulk Create or Update in single request
How to Update a REST Resource Collection - How to Update a REST Resource Collection
In fact, you are free to regarding the content you want for your methods GET. This means the root element managed by the resources (list resource and element resource) can contain its hints and also the ones of dependencies. So you can have several levels in the returned content. For example, you can have something like this for an element Book that references a list of Author:
GET /books
{
"title": "the title",
(...)
"authors": [
{
"firstName": "first name",
"lastName": last name"
}, {
(...)
},
(...)
]
}
You can notice that you can leverage query parameters to ask the RESTful service to get back the expected level. For example, if you want only book hints or book hints with corresponding authors:
GET /books
{
"title": "the title",
(...)
}
GET /books?include=authors
{
"title": "the title",
(...)
"authors": [
{
"firstName": "first name",
"lastName": last name"
}, {
(...)
},
(...)
]
}
You can notice that you can distinguish two concepts here:
The inner data (complex types, inner objects): data that are specific to the element and are embedded in the element itself
The referenced data: data that reference and correspond other elements. In this case, you can have a link or the data itself embedded in the root element.
The OData specification addresses such issue with its feature "navigation links" and its query parameter expand. See the following links for more details:
http://www.odata.org/getting-started/basic-tutorial/ - Section "System Query Option $expand"
https://msdn.microsoft.com/en-us/library/ff478141.aspx - Section "Query and navigation"
Hope it helps you,
Thierry
I am a PHP dev looking to port my API over to the Parse platform.
Am I right in thinking that you only need cloud code for complex operations? For example, consider the following methods:
// Simple function to fetch a user by id
function getUser($userid) {
return (SELECT * FROM users WHERE userid=$userid LIMIT 1)
}
// another simple function, fetches all of a user's allergies (by their user id)
function getAllergies($userid) {
return (SELECT * FROM allergies WHERE userid=$userid)
}
// Creates a script (story?) about the user using their user id
// Uses their name and allergies to create the story
function getScript($userid) {
$user = getUser($userid)
$allergies = getAllergies($userid).
return "My name is {$user->getName()}. I am allergic to {$allergies}"
}
Would I need to implement getUser()/getAllergies() endpoints in Cloud Code? Or can I simply use Parse.Query("User")... thus leaving me with only the getScript() endpoint to implement in cloud code?
Cloud code is for computation heavy operations that should not be performed on the client, i.e. handling a large dataset.
It is also for performing beforeSave/afterSave and similar hooks.
In your example, providing you have set up a reasonable data model, none of the operations require cloud code.
Your approach sounds reasonable. I tend to put simply queries that will most likely not change on the client side, but it all depends on your scenario. When developing mobile apps I tend to put a lot of code in cloud code. I've found that it speeds up my development cycle. For example, if someone finds a bug and it's in cloud code, make the fix, run parse deploy, done! The change is available to all mobile environments instantly!!! If that same code is in my mobile app, it really sucks, cause now I have to fix the bug, rebuild, push it to the app store/google play, wait x number of days for it to be approved, have the users download it... you see where I'm going here.
Take for example your
SELECT * FROM allergies WHERE userid=$userid query.
Even though this is a simple query, what if you want to sort it? maybe add some additional filtering?
These are the kinds of things I think of when deciding where to put the code. Hope this helps!
As a side note, I have also found cloud code very handy when needing to add extra security to my apps.