How do I return filtering meta data in a REST API search query - api

I'm currently designing and implementing a RESTful API in PHP.
The API allows users to search for hotels.
A simplified example of the search request is:
GET hotels/searchresults?location=<location> #collection of hotels within location
The response also contains some meta information about the returned collection.
The basic structure of the response is:
“meta": {
“totalNrOfHotels": 100,
"totalNrAvailable": 80
},
“hotels": [
{
“id": 123,
“name": "Hotel A"
},
{
“id": 135,
“name": "Hotel B"
},
...
]
This resource also supports pagination:
GET hotels/searchresults?location=<location>&offset=0&limit=20
Now, there are a few filters that can be applied to the search results, e.g. stars, rating score.
For example, if I want just 2 star hotels, I can query:
GET hotels/searchresults?location=<location>&offset=0&limit=20&stars=2
Now, in the user interface for filtering, it is common to display the number of options available per filter setting:
In my opinion, these numbers can be seen as meta data about the search query. So, we could add an extra field to the meta in the response:
“meta": {
“totalNrOfHotels": 100,
"totalNrAvailable": 80
“filterNrs": {
"stars”: {
“1": 1,
“2”: 9,
“3”: 39,
“4”: 12,
“5”: 11,
“none”: 9
}
}
},
“hotels": [
{“id": 123,
“name": "Hotel A"
},
{“id": 135,
“name": "Hotel B"
},
...
]
So, I have two questions:
Should this “filterNrs” property sit in the meta section, as proposed above? To me, it doesn’t make sense to be a separate resource/request
How can we deal with the fact that this can slow down the query? I’d prefer to make the “filterNrs” field optional. We are thinking of using a “metaFields" parameter to allow the user to specify which fields in the meta she would like to recieve. We already support this for the hotels returned, with a “fields” parameter. (Similar to: https://developers.google.com/youtube/2.0/developers_guide_protocol_partial). Alternatively, we put this field filterNrs (or the full meta info) in a separate resource, something like hotels/searchresults/meta. From a developers perspective would you prefer to have this split into multiple resources or have a single resource with the option to show full or partial meta information?

Does the number rated per star count varies? For example, do I get different "filterNrs" for the queries below?
GET hotels/searchresults?location=1
GET hotels/searchresults?location=2
I would expect such filters to be contextual, so different locations would return different numbers per star count, which indicates this is some form of contextual information related to the query.
Otherwise if the results are global this indicates it's a separate resource. If it's a separate resource scenario, you can use links to access the numbers and other details about it:
“meta": {
“totalNrOfHotels": 100,
"totalNrAvailable": 80
“filterNrs": {
"stars”: {
"options" : ["1", "2", "3", "4", "5", "none"],
"details" : "http://example.com/stars"
}
}
},

Related

Square Connect API: Retrieving all items within a category

I have been reading over the Square Connect API and messing around with the catalog portion.
I am unable to find how to retrieve all items and their data associated with a particular category. Can someone please point me in the right direction.
I thought it was the
BatchRetrieveCatalogObjects endpoint
I was using the category ID but it was only returning the catalog's data. I need each of the IDs of the items to retrieve their individual data.
I was looking to propagate a list of all the items and their data in one request in JSON.
JSON data to be passed to endpoint:
data = {
"object_ids": [
"category id"
],
"include_related_objects": True
}
My connection to the API:
category_item_endpoint = self.connection.post('/v2/catalog/batch-retrieve', data)
I am using python3 and the requests library.
In order to list items in a category I found it easiest to use the /v2/catalog/search endpoint. Simply follow the documentation on what parameters are accepted. Below are the search parameters that I used to list items by category id.
let sParams: JSON = [
"object_types": [
"ITEM"
],
"include_related_objects": true,
"include_deleted_objects": false,
"query": [
"exact_query": [
"attribute_name": "category_id",
"attribute_value": id
]
],
"limit": 1000
]
You'd probably have the most luck listing your entire catalog GET /v2/catalog/list and then applying filtering (in this case specific catagory_ids ) after you get the data. Based on the documentation doing what you desire doesn't seem possible with an endpoint/query combitionation.

How to construct intersection in REST Hypermedia API?

This question is language independent. Let's not worry about frameworks or implementation, let's just say everything can be implemented and let's look at REST API in an abstract way. In other words: I'm building a framework right now and I didn't see any solution to this problem anywhere.
Question
How one can construct REST URL endpoint for intersection of two independent REST paths which return collections? Short example: How to intersect /users/1/comments and /companies/6/comments?
Constraint
All endpoints should return single data model entity or collection of entities.
Imho this is a very reasonable constraint and all examples of Hypermedia APIs look like this, even in draft-kelly-json-hal-07.
If you think this is an invalid constraint or you know a better way please let me know.
Example
So let's say we have an application which has three data types: products, categories and companies. Each company can add some products to their profile page. While adding the product they must attach a category to the product. For example we can access this kind of data like this:
GET /categories will return collection of all categories
GET /categories/9 will return category of id 9
GET /categories/9/products will return all products inside category of id 9
GET /companies/7/products will return all products added to profile page of company of id 7
I've omitted _links hypermedia part on purpose because it is straightforward, for example / gives _links to /categories and /companies etc. We just need to remember that by using hypermedia we are traversing relations graph.
How to write URL that will return: all products that are from company(7) and are of category(9)? In otherwords how to intersect /categories/9/products and /companies/7/products?
Assuming that all endpoints should represent data model resource or collection of them I believe this is a fundamental problem of REST Hypermedia API, because in traversing hypermedia api we are traversing relational graph going down one path so it is impossible to describe such intersection because it is a cross-section of two independent graph paths.
In other words I think we cannot represent two independent paths with only one path. Normally we traverse one path like A->B->C, but if we have X->Y and Z->Y and we want all Ys that come from X and Z then we have a problem.
So far my proposition is to use query strings: /categories/9/products?intersect=/companies/9 but can we do better?
Why do I want this?
Because I'm building a framework which will auto-generate REST Hypermedia API based on SQL database relations. You could think of it as a trans compiler of URLs to SELECT ... JOIN ... WHERE queries, but the client of the API only sees Hypermedia and the client would like to have a nice way of doing intersections, like in the example.
I don't think you should always look at REST as database representation, this case looks more of a kind of specific functionality to me. I think I'd go with something like this:
/intersection/comments?company=9&product=5
I've been digging after I wrote it and this is what I've found (http://www.vinaysahni.com/best-practices-for-a-pragmatic-restful-api):
Sometimes you really have no way to map the action to a sensible RESTful structure. For example, a multi-resource search doesn't really make sense to be applied to a specific resource's endpoint. In this case, /search would make the most sense even though it isn't a resource. This is OK - just do what's right from the perspective of the API consumer and make sure it's documented clearly to avoid confusion.
What You want to do is to filter products in one of the categories ... so following Your example if we have:
GET /categories/9/products
Above will return all products in category 9, so to filter out products for company 7 I would use something like this
GET /categories/9/products?company=7
You should treat URI as link to fetch all data (just like simple select query in SQL) and query parameters as where, limit, desc etc.
Using this approach You can build complex and readable queries fe.
GET /categories/9/products?company=7&order=name,asc&offset=10&limit=20
All endpoints should return single data model entity or collection of
entities.
This is NOT a REST constraint. If you want to read about REST constraints, then read the Fielding dissertation.
Because I'm building a framework which will auto-generate REST
Hypermedia API based on SQL database relations.
This is a wrong approach and has nothing to do with REST.
By REST you describe possible resource state transitions (or operation call templates) by sending hyperlinks in the response. These hyperlinks consist of a HTTP methods and URIs (and other data which is not relevant now) if you build the uniform interface using the HTTP and URI standards, and we usually do so. The URIs are not (necessarily) database entity and collection identifiers and if you apply such a constraint you will end up with a CRUD API, not with a REST API.
If you cannot describe an operation with the combination of HTTP methods and already existing resources, then you need a new resource.
In your case you want to aggregate the GET /users/1/comments and GET /companies/6/comments responses, so you need to define a link with GET and a third resource:
GET /comments/?users=1&companies=6
GET /intersection/users:1/companies:6/comments
GET /intersection/users/1/companies/6/comments
etc...
RESTful architecture is about returning resources that contain hypermedia controls that offer state transitions. What i see here is a multistep process of state transitions. Let's assume you have a root resource and somehow navigate over to /categories/9/products using the available hypermedia controls. I'd bet the results would look something like this in hal:
{
_links : {
self : { href : "/categories/9/products"}
},
_embedded : {
item : [
{json of prod 1},
{json of prod 2}
]
}
}
If you want your client to be able to intersect this with another collection you need to provide to them the mechanism to perform this. You have to give them a hypermedia control. HAL only has links, templated links, and embedded as control types. let's go with links..change the response to:
{
_links : {
self : { href : "/categories/9/products"},
x:intersect-with : [
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 1",
title : "Company 6 products"
},
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 2",
title : "Company 5 products"
},
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 3",
title : "Company 7 products"
}
]
},
_embedded : {
item : [
{json of prod 1},
{json of prod 2}
]
}
}
Now the client just picks the right hypermedia control (aka link) based on the title field of the link.
That's the simplest solution. But you'll probably say there's 1000's of companies i don't want 1000's of links...well ok if that;s REALLY the case...you just offer a state transition in the middle of the two we have:
{
_links : {
self : { href : "/categories/9/products"},
x:intersect-options : { href : "URL to a Paged collection of all intersect options"},
x:intersect-with : [
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 1",
title : "Company 6 products"
},
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 2",
title : "Company 5 products"
},
{
href : "URL IS ABSOLUTELY IRRELEVANT!!! but unique 3",
title : "Company 7 products"
}
]
},
_embedded : {
item : [
{json of prod 1},
{json of prod 2}
]
}
}
See what i did there? an extra control for an extra state transition. JUST LIKE YOU WOULD DO IF YOU HAD A WEBPAGE. You'd probably put it in a pop up, well that's what the client of your app can do too with the result of that control.
It's really that simple...just think how you'd do it in HTML and do the same.
The big benefit here is that the client NEVER EVER needed to know a company or category id or ever plug that in to some template. The id's are implementation details, the client never knows they exist, they just executed Hypermedia controls..and that is RESTful.

REST API Design: Subcollection inheriting an editable variation of a resource

Bear with me, as I'm new to API Design and stackoverflow.
For my API design I have three main components:
key/value pairs (a singleton resource)
group (a collection of KVPs)
subgroup (a variation of the corresponding group)
In my API the user should be able to:
create, retrieve, update, and delete KVPs (basic CRUD actions)
organize KVPs into groups and access a listing of those groups
create subgroup variations of the groups that automatically inherit the parent groups' KVPs, and in those subgroups they can edit each KVPs data independently
For example, my Create a New Group request in my documentation looks like this:
{
"group": "age",
"kvps" : [
{
"key": "height",
"value": 5.2
}, {
"key": "weight",
"value": 150
}
],
"subgroups": [
"adult",
"teen",
"baby"
]
}
Using this as an example, I would like the subgroups "adult", "teen", and "baby" to have their own versions of the KVPs "height" and "weight" each with a different value (e.g. adult with a height of 6 and weight of 200, teen with a height of 5.2 and weight 140, etc.) that can be edited independently from each other.
My question: How should I structure my api so that a user could either:
have just editable kvps
have any number of editable kvps organized into any number of groups
or
have any number of editable kvps organized into groups and then further into subgroups
WITHOUT duplicating any of the data?
So far I have a uri structure like this:
/settings/kvps/{kvp_id} (CRUD on one single KVP)
/settings/groups/{group_id} (CRUD on one single group)
/settings/groups/{group_id}/kvps/{kvp_id}
/settings/groups/{group_id}/subgroups/{subgroup_id}
/settings/groups/{group_id}/subgroups/{subgroup_id}/kvps/{kvp_id}
So the kvps have 3 possible URIs, which seems messy to me, and from what I know editing the value of a kvp at
/settings/groups/1/subgroups/1/kvps/1
would also change its value at
/settings/groups/1/subgroups/2/kvps/1
/settings/groups/1/subgroups/3/kvps/1
/settings/groups/1/kvps/1
/settings/kvps/1
and so on, which is the opposite of what I want.
Any ideas?
Once again, I apologize if this is a very simple question that I am way off of, this is my first time designing an API.

Without JOINs, what is the right way to handle data in document databases?

I understand that JOINs are either not possible or frowned upon in document databases. I'm coming from a relational database background and trying to understand how to handle such scenarios.
Let's say I have an Employees collection where I store all employee related information. The following is a typical employee document:
{
"id": 1234,
"firstName": "John",
"lastName": "Smith",
"gender": "Male",
"dateOfBirth": "3/21/1967",
"emailAddresses":[
{ "email": "johnsmith#mydomain.com", "isPrimary": "true" },
{ "email": "jsmith#someotherdomain.com", "isPrimary": "false" }
]
}
Let's also say, I have a separate Projects collection where I store project data that looks something like that:
{
"id": 444,
"projectName": "My Construction Project",
"projectType": "Construction",
"projectTeam":[
{ "_id": 2345, "position": "Engineer" },
{ "_id": 1234, "position": "Project Manager" }
]
}
If I want to return a list of all my projects along with project teams, how do I handle making sure that I return all the pertinent information about individuals in the team i.e. full names, email addresses, etc?
Is it two separate queries? One for projects and the other for people whose ID's appear in the projects collection?
If so, how do I then insert the data about people i.e. full names, email addresses? Do I then do a foreach loop in my app to update the data?
If I'm relying on my application to handle populating all the pertinent data, is this not a performance hit that would offset the performance benefits of document databases such as MongoDB?
Thanks for your help.
"...how do I handle making sure that I return all the pertinent information about individuals in the team i.e. full names, email addresses, etc? Is it two separate queries?"
It is either 2 separate queries OR you denormalize into the Project document. In our applications we do the 2nd query and keep the data as normalized as possible in the documents.
It is actually NOT common to see the "_id" key anywhere but on the top-level document. Further, for collections that you are going to have millions of documents in, you save storage by keeping the keys "terse". Consider "name" rather than "projectName", "type" rather than "projectType", "pos" rather than "position". It seems trivial but it adds up. You'll also want to put an index on "team.empId" so the query "how many projects has Joe Average worked on" runs well.
{
"_id": 444,
"name": "My Construction Project",
"type": "Construction",
"team":[
{ "empId": 2345, "pos": "Engineer" },
{ "empId": 1234, "pos": "Project Manager" }
]
}
Another thing to get used to is that you don't have to write the whole document every time you want to update an individual field or, say, add a new member to the team. You can do targeted updates that uniquely identify the document but only update an individual field or array element.
db.projects.update(
{ _id : 444 },
{ $addToSet : "team" : { "empId": 666, "position": "Minion" } }
);
The 2 queries to get one thing done hurts at first, but you'll get past it.
Mongo DB is a document storage database.
It supports High Availability, and Scalability.
For returning a list of all your projects along with project team(details),
according to my understanding, you will have to run 2 queries.
Since mongoDb do not have FK constraints, we need to maintain it at the program level.
Instead of FK constraints,
1) if the data is less, then we can embed the data as a sub document.
2) rather than normalized way of designing the db, in MongoDb we need to design according to the access pattern. i.e. the way we need to query the data more likely. (However time for update is more(slow), but at the user end the performance mainly depends on read activity, which will be better than RDBMS)
The following link provides a certificate course on mongo Db, free of cost.
Mongo DB University
They also have a forum, which is pretty good.

ElasticSearch mapping for nested enumerable objects (i18n)

I'm at a loss as to how to map a document for search with the following structure:
{
"_id": "007ff234cb2248",
"ids": {
"source1": "123",
"source2": "456",
"source3": "789"
}
"names": [
{"en":"Example"},
{"fr":"exemple"},
{"es":"ejemplo"},
{"de":"Beispiel"}
],
"children" : [
{
"ids": {
"source1": "CXXIII",
"source2": "CDLVI",
"source3": "DCCLXXXIX",
}
names: [
{"en":"Example Child"},
{"fr":"exemple enfant"},
{"es":"Ejemplo niño"},
{"de":"Beispiel Kindes"}
]
}
],
"relatives": {
// Typically no "ids" at this level.
"relation": 'uncle',
"children": [
{
"ids": {
"source1": "0x7B",
"source2": "0x1C8",
"source3": "0x315"
},
"names": [
{"en":"Example Cousin"},
{"fr":"exemple cousine"},
{"es":"Ejemplo primo"},
{"de":"Beispiel Cousin"}
]
}
]
}
}
The child object may appear in the children section directly, or further nested in my document as uncle.children (cousins, in this case). The IDs field is common to levels one (the root), level two (the children and the uncle), and to level three (the cousins), the naming structure is also common to levels one and three.
My use-case is to be able to search for IDs (nested objects) by prefix, and by the whole ID. And also to be able to search for child names, following an (as yet undefined) set of analyzer rules.
I haven't been able to find a way to map these in any useful way. I don't believe I'll have much success using the same technique for ids and names, as there's an extra level of mapping between names and the document root.
I'm not even certain that it is even mappable. I believe at least in principle that the ids should be mappable as terms, and perhaps that if I index the names as terms in some way, too.
I'm simply at a loss, and the documentation doesn't seem to cover anything like this level of complex mapping.
I have limited (read: no) control of the document as it's coming from the CouchDB river, and the upstream application already relies on this format, so I can't really change it.
I'm looking for being able to search by the following pseudo conditions, all of which should match:
ID: "123"
ID by source (I don't know how best to mark this up in pseudo language)
ID prefix: "CDL"
Name: "Example", "Example Child"
Localized name (I don't even know how best to pseudo-mark this up!
The specifics of tokenising and analysis I can figure out for myself, when I at least know how to map
Objects when both the key and the value of the object properties are important
Enumerable objects when the key and value are important.
If the mapping from an ID to its children is 1-to-many, then you could store the children's names in a child field, as a field can have multiple values. Each document would then have an ID field, possibly a relation field, and zero or more child fields.