REST API Design: Subcollection inheriting an editable variation of a resource - api

Bear with me, as I'm new to API Design and stackoverflow.
For my API design I have three main components:
key/value pairs (a singleton resource)
group (a collection of KVPs)
subgroup (a variation of the corresponding group)
In my API the user should be able to:
create, retrieve, update, and delete KVPs (basic CRUD actions)
organize KVPs into groups and access a listing of those groups
create subgroup variations of the groups that automatically inherit the parent groups' KVPs, and in those subgroups they can edit each KVPs data independently
For example, my Create a New Group request in my documentation looks like this:
{
"group": "age",
"kvps" : [
{
"key": "height",
"value": 5.2
}, {
"key": "weight",
"value": 150
}
],
"subgroups": [
"adult",
"teen",
"baby"
]
}
Using this as an example, I would like the subgroups "adult", "teen", and "baby" to have their own versions of the KVPs "height" and "weight" each with a different value (e.g. adult with a height of 6 and weight of 200, teen with a height of 5.2 and weight 140, etc.) that can be edited independently from each other.
My question: How should I structure my api so that a user could either:
have just editable kvps
have any number of editable kvps organized into any number of groups
or
have any number of editable kvps organized into groups and then further into subgroups
WITHOUT duplicating any of the data?
So far I have a uri structure like this:
/settings/kvps/{kvp_id} (CRUD on one single KVP)
/settings/groups/{group_id} (CRUD on one single group)
/settings/groups/{group_id}/kvps/{kvp_id}
/settings/groups/{group_id}/subgroups/{subgroup_id}
/settings/groups/{group_id}/subgroups/{subgroup_id}/kvps/{kvp_id}
So the kvps have 3 possible URIs, which seems messy to me, and from what I know editing the value of a kvp at
/settings/groups/1/subgroups/1/kvps/1
would also change its value at
/settings/groups/1/subgroups/2/kvps/1
/settings/groups/1/subgroups/3/kvps/1
/settings/groups/1/kvps/1
/settings/kvps/1
and so on, which is the opposite of what I want.
Any ideas?
Once again, I apologize if this is a very simple question that I am way off of, this is my first time designing an API.

Related

How to update(add/remove) all Groups in ONE Grouping, without affecting OTHER Groupings?

I need help with an API call to update a Members Groups in a Grouping in my MailChimp List.
I have a few "Interest Groupings", each with several "Groups". For example, the first two are....
Grouping: Purchased
Groups: P_SPA3ASX, P_SPA3CFD, P_SPA3ETF, .....
Grouping: Member Stattus
Groups: Lead, Active, Inactive, Staff
Using an API call, I would like to update the Groups in the Purchased Grouping, without affecting ANY of the other Groupings. I have had some success but one scenario eludes me.
My API Call looks like this:
POST to: https://us11.api.mailchimp.com/2.0/lists/update-member.json
POST Body is:
{
"apikey": "myapikey",
"id": "mtlistid",
"email": {
"leid": "165320973"
},
"double_optin": false,
"update_existing": false,
"send_welcome": false,
"replace_interests": false,
"merge_vars": {
"groupings": [
{
"name": "Purchased",
"groups": ["P_SPA3ASX","","","",""]
}
]
}
}
When I change the "replace_interests" setting and pass different "groups" to the API, this is what happens.
Scenario 1: replace_interests = false
Result:
Good. Groups are added to "Purchased".
Bad. Groups are NOT removed from "Purchased".
Good. Other Groupings are not affected.
Scenario 2: replace_interests = true
Result:
Good. Groups are added to "Purchased".
Good. Groups are removed from "Purchased".
Bad. Other Groupings ARE affected. They are all cleared!
But how do I achieve all three (Add Groups, Remove Groups and Not affect other Groupings).
This isn't possible via API v2.0. In order to update a list subscribers interests, you have to supply all of them due to the use of an array to describe that data. In API v3.0, interests can be modified individually without affecting other interests.

Without JOINs, what is the right way to handle data in document databases?

I understand that JOINs are either not possible or frowned upon in document databases. I'm coming from a relational database background and trying to understand how to handle such scenarios.
Let's say I have an Employees collection where I store all employee related information. The following is a typical employee document:
{
"id": 1234,
"firstName": "John",
"lastName": "Smith",
"gender": "Male",
"dateOfBirth": "3/21/1967",
"emailAddresses":[
{ "email": "johnsmith#mydomain.com", "isPrimary": "true" },
{ "email": "jsmith#someotherdomain.com", "isPrimary": "false" }
]
}
Let's also say, I have a separate Projects collection where I store project data that looks something like that:
{
"id": 444,
"projectName": "My Construction Project",
"projectType": "Construction",
"projectTeam":[
{ "_id": 2345, "position": "Engineer" },
{ "_id": 1234, "position": "Project Manager" }
]
}
If I want to return a list of all my projects along with project teams, how do I handle making sure that I return all the pertinent information about individuals in the team i.e. full names, email addresses, etc?
Is it two separate queries? One for projects and the other for people whose ID's appear in the projects collection?
If so, how do I then insert the data about people i.e. full names, email addresses? Do I then do a foreach loop in my app to update the data?
If I'm relying on my application to handle populating all the pertinent data, is this not a performance hit that would offset the performance benefits of document databases such as MongoDB?
Thanks for your help.
"...how do I handle making sure that I return all the pertinent information about individuals in the team i.e. full names, email addresses, etc? Is it two separate queries?"
It is either 2 separate queries OR you denormalize into the Project document. In our applications we do the 2nd query and keep the data as normalized as possible in the documents.
It is actually NOT common to see the "_id" key anywhere but on the top-level document. Further, for collections that you are going to have millions of documents in, you save storage by keeping the keys "terse". Consider "name" rather than "projectName", "type" rather than "projectType", "pos" rather than "position". It seems trivial but it adds up. You'll also want to put an index on "team.empId" so the query "how many projects has Joe Average worked on" runs well.
{
"_id": 444,
"name": "My Construction Project",
"type": "Construction",
"team":[
{ "empId": 2345, "pos": "Engineer" },
{ "empId": 1234, "pos": "Project Manager" }
]
}
Another thing to get used to is that you don't have to write the whole document every time you want to update an individual field or, say, add a new member to the team. You can do targeted updates that uniquely identify the document but only update an individual field or array element.
db.projects.update(
{ _id : 444 },
{ $addToSet : "team" : { "empId": 666, "position": "Minion" } }
);
The 2 queries to get one thing done hurts at first, but you'll get past it.
Mongo DB is a document storage database.
It supports High Availability, and Scalability.
For returning a list of all your projects along with project team(details),
according to my understanding, you will have to run 2 queries.
Since mongoDb do not have FK constraints, we need to maintain it at the program level.
Instead of FK constraints,
1) if the data is less, then we can embed the data as a sub document.
2) rather than normalized way of designing the db, in MongoDb we need to design according to the access pattern. i.e. the way we need to query the data more likely. (However time for update is more(slow), but at the user end the performance mainly depends on read activity, which will be better than RDBMS)
The following link provides a certificate course on mongo Db, free of cost.
Mongo DB University
They also have a forum, which is pretty good.

Creating Mandatory User Filters with multiple element IDs

Mandatory User Filters
I am working on a tool to allow customers to apply Mandatory User Filters. When attributes are loaded like "Year" or "Age", each can have hundreds of elements with the subsequent ids. In the POST request to create a filter (documented here: https://developer.gooddata.com/article/lets-get-started-with-mandatory-user-filters), looks like this:
{
"userFilter": {
"content": {
"expression": "[/gdc/md/{project-id}/obj/{object-id}]=[/gdc/md/{project-id}/obj/{object-id}/elements?id={element-id}]"
},
"meta": {
"category": "userFilter",
"title": "My User Filter Name"
}
}
}
In the "expression" property, it notes how one ID could be set. What I want is to have multiple ids associated with the object-id set with the post. For example, if I user wanted to add a filter to all of the elements in "Year" (there are 150) in the demo project, it seems odd to make 150 post requests.
Is there a better way?
UPDATE
Tomas thank you for your help.
I am not having trouble assigning multiple userfilters to a user. I can easily apply a singular filter to a user with the method outlined in the documentation. However, this overwrites the userfilter field. What is the syntax for this?
Here is my demo POST data:
{ "userFilters":
{ "items": [
{ "user": "/gdc/account/profile/decd0b2e3077cf9c47f8cfbc32f6460e",
"userFilters":["/gdc/md/a1nc4jfa14wey1bnfs1vh9dljaf8ejuq/obj/808728","/gdc/md/a1nc4jfa14wey1bnfs1vh9dljaf8ejuq/obj/808729","/gdc/md/a1nc4jfa14wey1bnfs1vh9dljaf8ejuq/obj/808728"]
}
]
}
}
This receives a BAD REQUEST.
I'm not sure what you mean by "have multiple ids associated with the object-id" exactly, but I'll try to tell you all I know about it. :-)
If you indeed made multiple POST requests, created multiple userFilters and set them all for one user, the user wouldn't see anything at all. That's because the system combines separate userFilters using logical AND, and a Year cannot be 2013 and 2014 at the same time. So for the rest of my answer, I'll assume that you want OR instead.
There are several ways to do this. As you may have guessed by now, you can use AND/OR explicitly, using an expression like this:
[/…/obj/{object-id}]=[/…/obj/{object-id}/elements?id={element-id}] OR [/…/obj/{object-id}]=[/…/obj/{object-id}/elements?id={element-id}]
This can often be further simplified to:
[/…/obj/{object-id}] IN ( [/…/obj/{object-id}/elements?id={element-id}], [/…/obj/{object-id}/elements?id={element-id}], … )
If the attribute is a date (year, month, …) attribute, you could, in theory, also specify ranges using BETWEEN instead of listing all elements:
[/…/obj/{object-id}] BETWEEN [/…/obj/{object-id}/elements?id={element-id}] AND [/…/obj/{object-id}/elements?id={element-id}]
It seems, though, that this only works in metrics MAQL and is not allowed in the implementation of user filters. I have no idea why.
Also, for your own attribute like Age, you can't do that since user-defined numeric attributes aren't supported. You could, in theory, add a fact that holds the numeric value, and construct a BETWEEN filter based on that fact. It seems that this is not allowed in the implementation of user filters either. :-(
Hope this helps.

How do I return filtering meta data in a REST API search query

I'm currently designing and implementing a RESTful API in PHP.
The API allows users to search for hotels.
A simplified example of the search request is:
GET hotels/searchresults?location=<location> #collection of hotels within location
The response also contains some meta information about the returned collection.
The basic structure of the response is:
“meta": {
“totalNrOfHotels": 100,
"totalNrAvailable": 80
},
“hotels": [
{
“id": 123,
“name": "Hotel A"
},
{
“id": 135,
“name": "Hotel B"
},
...
]
This resource also supports pagination:
GET hotels/searchresults?location=<location>&offset=0&limit=20
Now, there are a few filters that can be applied to the search results, e.g. stars, rating score.
For example, if I want just 2 star hotels, I can query:
GET hotels/searchresults?location=<location>&offset=0&limit=20&stars=2
Now, in the user interface for filtering, it is common to display the number of options available per filter setting:
In my opinion, these numbers can be seen as meta data about the search query. So, we could add an extra field to the meta in the response:
“meta": {
“totalNrOfHotels": 100,
"totalNrAvailable": 80
“filterNrs": {
"stars”: {
“1": 1,
“2”: 9,
“3”: 39,
“4”: 12,
“5”: 11,
“none”: 9
}
}
},
“hotels": [
{“id": 123,
“name": "Hotel A"
},
{“id": 135,
“name": "Hotel B"
},
...
]
So, I have two questions:
Should this “filterNrs” property sit in the meta section, as proposed above? To me, it doesn’t make sense to be a separate resource/request
How can we deal with the fact that this can slow down the query? I’d prefer to make the “filterNrs” field optional. We are thinking of using a “metaFields" parameter to allow the user to specify which fields in the meta she would like to recieve. We already support this for the hotels returned, with a “fields” parameter. (Similar to: https://developers.google.com/youtube/2.0/developers_guide_protocol_partial). Alternatively, we put this field filterNrs (or the full meta info) in a separate resource, something like hotels/searchresults/meta. From a developers perspective would you prefer to have this split into multiple resources or have a single resource with the option to show full or partial meta information?
Does the number rated per star count varies? For example, do I get different "filterNrs" for the queries below?
GET hotels/searchresults?location=1
GET hotels/searchresults?location=2
I would expect such filters to be contextual, so different locations would return different numbers per star count, which indicates this is some form of contextual information related to the query.
Otherwise if the results are global this indicates it's a separate resource. If it's a separate resource scenario, you can use links to access the numbers and other details about it:
“meta": {
“totalNrOfHotels": 100,
"totalNrAvailable": 80
“filterNrs": {
"stars”: {
"options" : ["1", "2", "3", "4", "5", "none"],
"details" : "http://example.com/stars"
}
}
},

ElasticSearch mapping for nested enumerable objects (i18n)

I'm at a loss as to how to map a document for search with the following structure:
{
"_id": "007ff234cb2248",
"ids": {
"source1": "123",
"source2": "456",
"source3": "789"
}
"names": [
{"en":"Example"},
{"fr":"exemple"},
{"es":"ejemplo"},
{"de":"Beispiel"}
],
"children" : [
{
"ids": {
"source1": "CXXIII",
"source2": "CDLVI",
"source3": "DCCLXXXIX",
}
names: [
{"en":"Example Child"},
{"fr":"exemple enfant"},
{"es":"Ejemplo niño"},
{"de":"Beispiel Kindes"}
]
}
],
"relatives": {
// Typically no "ids" at this level.
"relation": 'uncle',
"children": [
{
"ids": {
"source1": "0x7B",
"source2": "0x1C8",
"source3": "0x315"
},
"names": [
{"en":"Example Cousin"},
{"fr":"exemple cousine"},
{"es":"Ejemplo primo"},
{"de":"Beispiel Cousin"}
]
}
]
}
}
The child object may appear in the children section directly, or further nested in my document as uncle.children (cousins, in this case). The IDs field is common to levels one (the root), level two (the children and the uncle), and to level three (the cousins), the naming structure is also common to levels one and three.
My use-case is to be able to search for IDs (nested objects) by prefix, and by the whole ID. And also to be able to search for child names, following an (as yet undefined) set of analyzer rules.
I haven't been able to find a way to map these in any useful way. I don't believe I'll have much success using the same technique for ids and names, as there's an extra level of mapping between names and the document root.
I'm not even certain that it is even mappable. I believe at least in principle that the ids should be mappable as terms, and perhaps that if I index the names as terms in some way, too.
I'm simply at a loss, and the documentation doesn't seem to cover anything like this level of complex mapping.
I have limited (read: no) control of the document as it's coming from the CouchDB river, and the upstream application already relies on this format, so I can't really change it.
I'm looking for being able to search by the following pseudo conditions, all of which should match:
ID: "123"
ID by source (I don't know how best to mark this up in pseudo language)
ID prefix: "CDL"
Name: "Example", "Example Child"
Localized name (I don't even know how best to pseudo-mark this up!
The specifics of tokenising and analysis I can figure out for myself, when I at least know how to map
Objects when both the key and the value of the object properties are important
Enumerable objects when the key and value are important.
If the mapping from an ID to its children is 1-to-many, then you could store the children's names in a child field, as a field can have multiple values. Each document would then have an ID field, possibly a relation field, and zero or more child fields.