According to API documentation here https://backendless.com/documentation/data/rest/data_search_and_query.htm , backendless shall provide paging info in response body like this
{
"nextPage":null,
"data":[
{
"updated":null,
"created":"02/05/2014 18:13:40 GMT+0000",
"ownerId":null,
"objectId":"6FAF3CE5-6F55-1B32-FF83-D333252D0300",
"name":"Bob",
"age":20
},
{
"updated":null,
"created":"02/04/2014 19:40:10 GMT+0000",
"ownerId":null,
"objectId":"28325E9F-2DED-D3CA-FFC6-C76911AFBB00",
"name":"Frank",
"age":26
}],
"offset":0,
"totalObjects":2
}
However, when I send request to get data from table like this:
https://api.backendless.com/<version>/data/<table-name>
it returns only collection of objects (which should be in "data") and now paging info.
Adding page requests:
https://api.backendless.com/<version>/data/<table-name>?pageSize=10&offset=10
returns correct data, but still without paging info.
What I'm doing wrong? How can I access paging info?
The docs link you mention is for Backendless version which is being deprecated (there's a warning on the top of the page). Here is a link to the docs for the current version: https://backendless.com/docs/rest/doc.html#data_basic_search
Indeed, in version 4 we got rid of that additional information in the response to simplify the response object in non-REST clients. But still you can easily determine current params since you're specifying them in the request (or they're zeros in case you're not).
Related
Overview
I'm using Ember data and have a JSONAPI. Everything works fine until I have a more complex object (let's say an invoice for a generic concept) with an array of items called lineEntries. The line entries are not mapped directly to a table so need to be stored as raw JSON object data. The line entry model also contains default and computed values. I wish to store the list data as a JSON object and then when loaded back from the store that I can manipulate it as normal in Ember as an array of my model.
What I've tried
I've looked at and tried several approaches, the best appear to be (open to suggestions here!):
Fragments
Replace problem models with fragments
I've tried making the line entry model a fragment and then referencing the fragment on the invoice model as a fragmentArray. Line entries add to the array as normal but default values don't work (should they?). It creates the object and I can store it in the backend but when I return it, it fails with either a normalisation issue or a serialiser issue. Can anyone state the format the data be returned in? It's confusing as normalising the data seems to require JSONAPI but the fragment requires JSON serialiser. I've tried several combinations but no luck so far. My line entries don't have actual ids as the data is saved and loaded as a block. Is this an issue?
DS.EmbeddedRecordsMixin
Although not supported in JSONAPI, it sounds possible to use JSONAPI and then switch to JSONSerializer or RESTSerializer for the problem models. If this is possible could someone give me a working example and the JSON format that should be returned by the API? I have header authorisation and other such data so would I still be able to set this at the application level for all request not using my JSONAPI?
Ember-data-save-relationships
I found an add on here that provides an add on to do this. It seems more involved than the other approaches but when I've tried this I can send the data up by setting a the data as embedded. Great! But although it saves it doesn't unwrap it correct and I'm back with the same issues.
Custom serialiser
Replace the models serialiser with something that takes the data and sends it as plain JSON data and then deserialises back into something Ember can use. This sounds similar to the above but I do the heavy lifting. The only reason to do this is because all examples for the above solutions are quite light and don't really show how to set this up with an actual JSONAPI set up that would need it.
Where I am and what I need
Basically all approaches lead to saving the JSON fine but the return JSON from the server not being the correct format or the deserialisation failing but it's unclear what it should be or what needs to change without breaking the existing JSONAPI models that work fine.
If anyone know the format for return API data it may resolve this. I've tried JSONAPI with lineEntries returning the same format as it saved. I've tried placing relationship sections like the add on suggested and I've also tried placing relationship only data against the entries and an include section with all the references. Any help on this would be great as I've learned a lot through this but deadlines a looming and I can't see a viable solution that doesn't break as much as it fixes.
If you are looking for return format for relational data from the API server you need to make sure of the following:
Make sure the relationship is defined in the ember model
Return all successes with a status code of 200
From there you need to make sure you return relational data correctly. If you've set the ember model for the relationship to {async: true} you need only return the id of the relational model - which should also be defined in ember. If you do not set {async: true}, ember expects all relational data to be included.
return data with relationships in JSON API specification
Example:
models\unicorn.js in ember:
import DS from 'ember-data';
export default DS.Model.extend({
user: DS.belongsTo('user', {async: true}),
staticrace: DS.belongsTo('staticrace',{async: true}),
unicornName: DS.attr('string'),
unicornLevel: DS.attr('number'),
experience: DS.attr('number'),
hatchesAt: DS.attr('number'),
isHatched: DS.attr('boolean'),
raceEndsAt: DS.attr('number'),
isRacing: DS.attr('boolean'),
});
in routes\unicorns.js on the api server on GET/:id:
var jsonObject = {
"data": {
"type": "unicorn",
"id": unicorn.dataValues.id,
"attributes": {
"unicorn-name" : unicorn.dataValues.unicornName,
"unicorn-level" : unicorn.dataValues.unicornLevel,
"experience" : unicorn.dataValues.experience,
"hatches-at" : unicorn.dataValues.hatchesAt,
"is-hatched" : unicorn.dataValues.isHatched,
"raceEndsAt" : unicorn.dataValues.raceEndsAt,
"isRacing" : unicorn.dataValues.isRacing
},
"relationships": {
"staticrace": {
"data": {"type": "staticrace", "id" : unicorn.dataValues.staticRaceId}
},
"user":{
"data": {"type": "user", "id" : unicorn.dataValues.userId}
}
}
}
}
res.status(200).json(jsonObject);
In ember, you can call this by chaining model functions. For example when this unicorn goes to race in controllers\unicornracer.js:
raceUnicorn() {
if (this.get('unicornId') === '') {return false}
else {
return this.store.findRecord('unicorn', this.get('unicornId', { backgroundReload: false})).then(unicorn => {
return this.store.findRecord('staticrace', this.get('raceId')).then(staticrace => {
if (unicorn.getProperties('unicornLevel').unicornLevel >= staticrace.getProperties('raceMinimumLevel').raceMinimumLevel) {
unicorn.set('isRacing', true);
unicorn.set('staticrace', staticrace);
unicorn.set('raceEndsAt', Math.floor(Date.now()/1000) + staticrace.get('duration'))
this.set('unicornId', '');
return unicorn.save();
}
else {return false;}
});
});
}
}
The above code sends a PATCH to the api server route unicorns/:id
Final note about GET,POST,DELETE,PATCH:
GET assumes you are getting ALL of the information associated with a model (the example above shows a GET response). This is associated with model.findRecord (GET/:id)(expects one record), model.findAll(GET/)(expects an array of records), model.query(GET/?query=&string=)(expects an array of records), model.queryRecord(GET/?query=&string=)(expects one record)
POST assumes you at least return at least what you POST to the api server from ember , but can also return additional information you created on the apiServer side such as createdAt dates. If the data returned is different from what you used to create the model, it'll update the created model with the returned information. This is associated with model.createRecord(POST/)(expects one record).
DELETE assumes you return the type, and the id of the deleted object, not data or relationships. This is associated with model.deleteRecord(DELETE/:id)(expects one record).
PATCH assumes you return at least what information was changed. If you only change one field, for instance in my unicorn model, the unicornName, it would only PATCH the following:
{
data: {
"type":"unicorn",
"id": req.params.id,
"attributes": {
"unicorn-name" : "This is a new name!"
}
}
}
So it only expects a returned response of at least that, but like POST, you can return other changed items!
I hope this answers your questions about the JSON API adapter. Most of this information was originally gleamed by reading over the specification at http://jsonapi.org/format/ and the ember implementation documentation at https://emberjs.com/api/data/classes/DS.JSONAPIAdapter.html
For designing and creating a RESTful API the following question occurs:
The API supports GET (for queries), POST (for creating), PUT (for updates) and DELETE (for deleting).
Lets assume in the database we have an article and a shop both already existing.
Now we need a rest call to link the article instance to the shop instance. Which of the following solutions is the best / most clean REST design:
/shop/id/article/id/ --> with POST
/shop/id/article/id/ --> with PUT
/shoparticlerelation/ --> with POST (object with ids in body)
/shoparticlerelation/ --> with PUT (object with ids in body)
If there is no clear answer or all solutions are equally good this may also be a valid answer if there is a clear argumentation why.
I presume in this situation you already have a collection of shops and a collection of articles, and you just wish to link two together.
One option is to expose a more db like 'resource' that presents this link, and have operations like
POST /shopArticleLinks HTTP/1.1
{ "shop" : xxx,
"article: YYY
}
I would personally look to expose it as a property of the shops and/or articles in a more natural manor, like
PUT /shop/<ID> HTTP/1.1
{ /* existing details */
"articles": [ /* list of articles */ ]
}
I've used JSON there, but of course use what ever format you want to use. I've also stuck with using PUT as you stated, but keep in mind that with PUT you should send a full replacement for the new modified version, PATCH can be used to send partial updates, but then you need to consider how you want do that, may something like
PATCH /shops/<ID>/articleLinks HTTP/1.1
{ "add" : [],
"remove : []
}
Don't forget that server side you can look at what articles are being refereed to and ensure they have a proper back pointer.
Additional thoughts
Regarding the second method, where you expose the link as a property of the shop and/or article resources. Keep in mind that it is perfectly acceptable (and in this case rather appropriate) that when you update the links in a given shop that the links in the corresponding articles are also updated.
/shop/id/article/id/
You cannot use this because at the moment you want to link them, this endpoint doesn't (or at least shouldn't) yet exist. It is the action of linking them together that should define this endpoint.
/shoparticlerelation/
You should not use this because a shoparticlerelation is not a resource / entity. Usually with rest, every named url segment represents a resource that can be CRUD-ed. /shops is a good example and so is /articles but this one isn't.
I suggest the following:
Define the following endpoints
/shops for POSTing new shops
/shops/id for operating on a single shop
/articles for POSTing new articles
/articles/id for operating on a single article
Then to link them together you can do a so called PATCH request, to update a shop's articles, or an article's shops:
PATCH /shops/1 HTTP/1.1
{
"attribute": "articles",
"operation": "add",
"value": "8" // the article id
}
and
PATCH /articles/9 HTTP/1.1
{
"attribute": "shops",
"operation": "add",
"value": "1" // the shop id
}
Based on your comments I made the assumption that an Article model has a list of Shops as attribute, and vice-versa, making this approach valid.
A PATCH request is used to modify an existing resource by specifying how and what to update. This is different from a PUT because a PUT replaces the entire resource with values from the request, however PATCH is only used to modify (not replace) a resource.
I am looking for a more detailed list of possible API responses when using Yodlee's REST API. Think of it as an XSD response but for a JSON string. I want to know if there are possible data elements that are not listed Yodlee's JSON response examples.
The only info I can really find so far is here.
When I review these examples, it appears that the example JSON responses do not fully describe every field.
Here is part of the getItemSummaryForItem1 JSON example for maturityDate element
"maturityDate":{
},
It looks like there is an array, but the possible data elements for that maturityDate array are undeclared. Then later on maturityDate is shown to be:
"maturityDate":{
"date":"0014-02-01T00:00:00-0800",
"localFormat":"dd/MM/yyyy"
},
And then in another example from getUserTransactionCategories
{
"categoryId":31,
"categoryName":"Retirement Income",
"transactionCategoryTypeId":2,
"isBudgetable":1,
"localizedCategoryName":"Retirement Income",
"isHidden":false,
"categoryLevelId":3
},
Based on that I would think all possible data elements are there.
But then there is another one which introduces the childCategory data element
{
"categoryId":2,
"categoryName":"Automotive Expenses",
"isDeleted":0,
"transactionCategoryTypeId":4,
"isBudgetable":1,
"localizedCategoryName":"Automotive Expenses",
"isHidden":false,
"categoryLevelId":3,
"childCategory":[
{
"categoryId":5641,
"categoryName":"1_SubCategory1",
"categoryDescription":"Subcategory desc1",
"isDeleted":0,
"isBudgetable":0,
"localizedCategoryName":"1_SubCategory1",
"isHidden":false,
"parentCategoryId":2,
"categoryLevelId":4
}
}
Thanks!
Yodlee team is working on to get this details documented, this is a time taking process and will be soon available over their portal. Meanwhile, is there any specific field or API response for which you are looking to get all the child elements which will help you out without blocking your integration?
I using Instagram API to get user info
api = InstagramAPI(access_token=access_token)
profile = api.user(user_id="kallaucyahoocojp") # I try to put output data to profile variable here
And I get the below error:
DownloadError: Unable to fetch URL: https://api.instagram.com/v1/users/kallaucyahoocojp.json?access_token=(u'1191812153.f78cd79.d2d99595c79d4c23a7994d85ea0d412c', {u'username': u'kallaucyahoocojp', u'bio': u'\u30c4\u30a4\u30c3\u30bf\u30d5\u30a9\u30ed\u30ef\u30fc\u5897\u52a0\u30b5\u30fc\u30d3\u30b9', u'website': u'http://twitter\u30d5\u30a9\u30ed\u30ef\u30fc.jp', u'profile_picture': u'http://images.ak.instagram.com/profiles/anonymousUser.jpg', u'full_name': u'Kallauc', u'id': u'1191812153'})
Can anybody help me to fix it?
You need to pass the numeric-based user id, rather than the username. For example, instead of passing kallaucyahoocojp, you might pass 1234 if t
Here's how to get the ID if you don't have it:
Search for the instagram user id using this endpoint. In the python api:
api.user_search(q="kallaucyahoocojp", count=100)
Check the results for an exact string match on each user name while iterating through the results (calling .lower() to be sure to ignore potential case issues).
If you don't find the user in the first page of results, call to the next page using the max id returned.
Get the user id object from the returned from the matching users search result, then call your original function again with the numeric id.
A couple of very important notes:
Notice that I called the search function for users with a count of 100. You can pick any number, but contrary to other SO posts, the first user is not always the user you want in a search. The search can and will match partials, and not always according to an exact match first. How do I know? I have production instagram apps. I will qualify and say that usually the results are in the first 2-3 matches. Decide what is cheaper; repeated API calls that bring you closer to the limit, or 1 large bulk call where you are certain to get all the results.
The python Instagram API last I checked does a terrible job returning paging information. You actually get the paging URL which defeats the purpose of the python API itself to get additional pages. Your options are extract the next id parameter from the URL using urlparse or something similar, or fix the API to return the paging data as an object per the json (I've done both). What happens is the API itself is discarding part of the json and only giving you the URL which normally you don't want/need.
In your example, here's the search response:
{
"meta": {
"code": 200
},
"data": [
{
"username": "kallaucyahoocojp",
"bio": "ツイッタフォロワー増加サービス",
"website": "http://twitterフォロワー.jp",
"profile_picture": "http://images.ak.instagram.com/profiles/anonymousUser.jpg",
"full_name": "Kallauc",
"id": "1191812153"
}
]
}
Revising your call:
api = InstagramAPI(access_token=access_token)
profile = api.user(user_id="1191812153")
I should note that you may not need to call the user call if you did a search because you may simply have all the info you need. It will depend on what you are doing of course, so I am giving you the general method to use the rest of the user api.
For extracting profile info using Instagram API, userid is required.
The endpoint for extracting userID:
https://api.instagram.com/v1/users/search?q=[username]&access_token=[HERE]
The endpoint for extracting profile info:
https://api.instagram.com/v1/users/[userid]/?access_token=[HERE]
Note that before extracting information, check the login permissions for your access token.
Suppose my model is:
User:
id
nickname
I have a collection /users/
I want the Users to be retrieved by /users/{id} and not /users/${nickname}, because in some more complex cases, there could be no "logical unique constraint".
So the basic JSON payload I could use is for exemple:
{
id: 123,
nickname: 'someUserName'
}
Nothing fancy here.
POST on /users/
As far as I know, an user as an identifier. It is part of the resource representation, so it should be in the payload (?).
Put what if I want to generate the ID myself on the backend, using a DB sequence for exemple?
Then my payload becomes:
{
nickname: 'someUserName'
}
Is this appropriate?
What is supposed to be the output of this POST? Nothing? Just a header referencing the resource location, including the ID?
GET on /users/id
When we get the resource, we load its content as JSON:
{
id: 123,
nickname: 'someUserName'
}
PUT on /users/id
As far as I know, the payload used on this method is supposed to "override" the resource content. If we wanted partial updates, we would have used PATCH.
But what if I do:
PUT /users/123
{
id: 456,
nickname: 'someUserName'
}
Does this mean that we want to update the id of a resource?
Isn't it kind of redundant to use the id in both the URI and the payload?
Actually I don't really know how to handle the id.
I don't know if I am supposed to use the same resource representation in all POST / PUT / DELETE operations.
I don't know if the id should be part of the unique(?) resource representation.
But if the id is not part of the representation, then when I list the users, using GET /users/, if the ids are not returned, then I don't know how the client can get the user ids...
Can someone help me? :)
First of all
It is not REST if you don't use HATEOAS
I hope you understand this, I'll come back to that at the very end.
POST on /users/
It perfectly ok to not use an ID in the POST payload. If an ID is present react with an error message, so developers understand they are doing wrong.
Therefore only the nickname as a payload is perfectly valid if you don't have anything else in your user resource
The output of your server should include three important things:
HEADER: A status code indicating success or failure (usually 201 Created)
HEADER: The location of the newly created resource (just Location: /path/to/resource)
BODY: A representation of the created resource. Give back a complete payload like on a GET!
GET
perfectly valid
PUT
your analysis regarding PUT/PATCH matchs the spec, the new resource should be identical to the payload meaning the user wishes to change the id if it differs. if a payload contains values which shouldn't be changed (like the ID) you have two possibilities:
Ignore the ID in the payload
Return an error
In both cases inform the user about what you did and what went wrong. I prefer to send/get a 400 Bad Request. If a privileged user could change the ID but the particular user can't an 403 Forbidden may be more appropriate.
Also make sure to document your APIs behaviour. You may allow the ID to be omitted in your API. Don't forget to treat IDs given in a POST payload in a consistent way!
Overall questions
REST operates over Resources.
/users/ is an example for an collection of resources
/users/{id} is an example for a single resource
You should always use the exact same representation in each and every response. If for some reason it is more appropriate to give only a snippet of the information add metadata (link) pointing to the full resource representation.
The ID is always present except in the first POST request of an user.
POST implies that the future location of the resource is not known and has to be provided by the server.
This also means that GET /users/ should return the IDs for each resource.
As always in APIs return strict and be forgiving in requests. document your behaviour so users can learn.
HATEOAS
The true beauty of REST comes to daylight if you implement HATEOAS (Hypermedia As The Engine Of Application State). Part of this means that you should sugar your representations with useful tag/link combinations. This way clients never have to construct an url anymore.
An Example using HAL for your user representation would be:
{
"_links:" {
"self": { "href": "http://yourrest/users/123" }
},
"id": "123"
"nickname": "someUserName"
}
A nice wrapup of using HAL was written by Matthew Weier O'Phinney in his blog when he developed a ZF2 REST Module (first entry is completly zf free, only explaining HAL).
I'm interpreting your descriptions as saying that the id is not part of the resource, it's a unique identifier of the resource. In that case, it should not be part of the payload for any operation.
POST /users with payload {"nickname": "somebody"} would create a new resource with a URL returned in the Location header. That URL would presumably look like /users/123 but from the client's point of view there's no reason to expect that. It could look like /something/else/entirely.
GET /users/123 would (assuming that URL was returned by an earlier POST) return the payload {"nickname": "somebody"}.
PUT /users/123 would (with the same assumption as above) replace the resource with the payload you send with the PUT, say {"nickname": "somebody else"}.
If you want the client to be able to name a resource, then you'd also let PUT /users/123 create a new resource with that URL.
I know of no common RESTful way to rename a resource. I suppose a POST with the old URL as part of the query part or the body would make sense.
Now, suppose I'm wrong and you do want id to be part of the resource itself. Then every payload would include it. But from the client's point of view, there should be no assumption that "id": 123 implies that the URL would be /users/123.
Finally, all of this is from a fairly purist point of view. There is value to thinking of URLs as the only real identifier of a resource, but it's not awful to break that rule and have the client use logic to create the URLs.