Background
In a project I am working on, we are using AWS as the cloud provider and Amplify with AppSync GraphQL to help rapidly build the back end. AppSync in Amplify defaults to having a DynamoDB database so that is what we are using with it.
We are using data-level security so users can only see records they are meant to see in a shared database.
There is a hierarchy whereby a Project object can contain many other objects and we need to ensure that anyone who is authorised to view the Project, can also view everything it contains.
The Problem
To do this, it was assumed that we could have an Asset object with a projectId attribute and project connection to populate the project details on reading the record. The Project table/object contains a groups attribute which we use dynamic group security in the #auth directive.
here's an example of the schema:
type Project
#model
#auth(rules: [
{ allow: groups, groupsField: "groups" }
])
{
id: ID!
title: String!
reference: String
clientId: ID!
client: Organisation!
#connection(fields: ["clientId"])
modifiedById: ID
changeComment: String
groups: [String!]
}
type Asset
#model
#key(name: "byProjectId", fields: ["projectId"])
{
id: ID!
name: String!
projectId: ID!
project: Project!
#connection(fields: ["projectId"])
completedAt: AWSDateTime
}
It is simplified so does not make a lot of sense as written. Ideally, we would use the #auth directive to specify the field for the Asset type as well like this
#auth( allow: groups, groupsField: "project.groups" )
We have also attempted to do it manually with a custom resolver written in Velocity (.vtl) which turned out to be very much the same thing:
#set( $allowedGroups = $util.defaultIfNull($item.project.groups, []) )
#set( $userGroups = $util.defaultIfNull($ctx.identity.claims.get("cognito:groups"), []) )
#foreach( $userGroup in $userGroups )
#if( $util.isList($allowedGroups) )
#if( $allowedGroups.contains($userGroup) )
#set( $isLocalDynamicGroupAuthorized = true )
#end
#end
#if( $util.isString($allowedGroups) )
#if( $allowedGroups == $userGroup )
#set( $isLocalDynamicGroupAuthorized = true )
#end
#end
#end
After checking it, I found at the time this resolver is run, the project field is null which leads me to think it has not been joined in yet so can't be used as originally planned.
The Answer?
Is there a way to do this or a more suitable alternative? My concern is that the obvious solution of having all objects having their own groups attribute which contains the parent project's user group name could get very unwieldy as there are upwards of 10k assets to a project and more than just assets to consider.
Am I wrestling with my urge to normalise the data since this is a NoSQL db and we should just duplicate the values and handle that in the code?
Would a User Permission Matrix table be more sensible and would it be easy enough to build into AppSync with Amplify?
Any hints as to how to do this or implement a more suitable solution would be very much appreciated.
Related
I'm using KeystoneJS v6. I'm trying to enable functionality which allow me to reorder the placement of images when used in another list. Currently i'm setting up the image list below, however I'm unable to set the defaultIsOrderable to true due to the error pasted.
KeystoneJS list:
Image: list({
fields: {
title: text({
validation: { isRequired: true },
isIndexed: 'unique',
isFilterable: true,
isOrderable: true,
}),
images: cloudinaryImage({
cloudinary: {
cloudName: process.env.CLOUDINARY_CLOUD_NAME,
apiKey: process.env.CLOUDINARY_API_KEY,
apiSecret: process.env.CLOUDINARY_API_SECRET,
folder: process.env.CLOUDINARY_API_FOLDER,
},
}),
},
defaultIsOrderable: true
}),
Error message:
The expected type comes from property 'defaultIsOrderable' which is declared here on type 'ListConfig<BaseListTypeInfo, BaseFields<BaseListTypeInfo>>'
Peeking at the definition of the field shows
defaultIsOrderable?: false | ((args: FilterOrderArgs<ListTypeInfo>) => MaybePromise<boolean>);
Looking at the schema API docs, the defaultIsOrderable lets you set:
[...] the default value to use for isOrderable for fields on this list.
You're trying to set this to true but, according to the relevant section of the field docs, the isOrderable field option already defaults to true.
I believe this is why the defaultIsOrderable type doesn't allow you to supply the true literal – doing so would be redundant.
So that explains the specific error your getting but I think you also may have misunderstood the purpose of the orderBy option.
The OrderBy Option
The field docs mention the two effects the field OrderBy option has:
If true (default), the GraphQL API and Admin UI will support ordering by this field.
Take, for example, your Image list above.
As the title field is "orderable", it is included in the list's orderBy GraphQL type (ImageOrderByInput).
When querying the list, you can order the results by the values in this field, like this:
query {
images (orderBy: [{ title: desc }]) {
id
title
images { publicUrl }
}
}
The GraphQL API docs have some details on this.
You can also use the field to order items when listing them in the Admin UI, either by clicking the column heading or selecting the field from the "sort" dropdown:
Note though, these features order items at runtime, by the values stored in orderable fields.
They don't allow an admin to "re-order" items in the Admin UI (unless you did so by changing the image titles in this case).
Specifying an Order
If you want to set the order of items within a list you'd need to store separate values in, for example, a displayOrder field like this:
Image: list({
fields: {
title: text({
validation: { isRequired: true },
isIndexed: 'unique',
isFilterable: true,
}),
displayOrder: integer(),
// ...
},
}),
Unfortunately Keystone doesn't yet give you a great way to manage this the Admin UI (ie. you can't "drag and drop" in the list view or anything like that). You need to edit each item individually to set the displayOrder values.
Ordering Within a Relationship
I notice your question says you're trying to "reorder the placement of images when used in another list" (emphasis mine).
In this case you're talking about relationships, which changes the problem somewhat. Some approaches are..
If the relationship is one-to-many, you can use the displayOrder: integer() solution shown above but the UX is worse again. You're still setting the order values against each item but not in the context of the relationship. However, querying based on these order values and setting them via the GraphQL API should be fairly straight forward.
If the relationship is many-to-many, it's similar but you can't store the "displayOrder" value in the Image list as any one image may be linked to multiple other items. You need to store the order info "with" the relationship itself. It's not trivial but my recent answer on storing additional values on a many-to-many relationship may point you in the right direction.
A third option is to not use the relationship field at all but to link items using the inline relationships functionality of the document field. This is a bit different to work with - easier to manage from the Admin UI but less powerful in GraphQL as you can't traverse the relationship as easily. However it does give you a way to manage a small, ordered set of related items in a many-to-many relationship.
You can save an ordered set of ids to a json field. This is similar to using a document field but a more manual.
Hopefully that clears up what's possible with the current "orderBy" functionality and relationship options. Which of these solutions is most appropriate depends heavily on the specifics of your project and use case.
Note too, there are plans to extend Keystone's functionality for sorting and reordering lists from both the DX and UX perspectives.
See "Sortable lists" on the Keystone roadmap.
Overview
I'm using Ember data and have a JSONAPI. Everything works fine until I have a more complex object (let's say an invoice for a generic concept) with an array of items called lineEntries. The line entries are not mapped directly to a table so need to be stored as raw JSON object data. The line entry model also contains default and computed values. I wish to store the list data as a JSON object and then when loaded back from the store that I can manipulate it as normal in Ember as an array of my model.
What I've tried
I've looked at and tried several approaches, the best appear to be (open to suggestions here!):
Fragments
Replace problem models with fragments
I've tried making the line entry model a fragment and then referencing the fragment on the invoice model as a fragmentArray. Line entries add to the array as normal but default values don't work (should they?). It creates the object and I can store it in the backend but when I return it, it fails with either a normalisation issue or a serialiser issue. Can anyone state the format the data be returned in? It's confusing as normalising the data seems to require JSONAPI but the fragment requires JSON serialiser. I've tried several combinations but no luck so far. My line entries don't have actual ids as the data is saved and loaded as a block. Is this an issue?
DS.EmbeddedRecordsMixin
Although not supported in JSONAPI, it sounds possible to use JSONAPI and then switch to JSONSerializer or RESTSerializer for the problem models. If this is possible could someone give me a working example and the JSON format that should be returned by the API? I have header authorisation and other such data so would I still be able to set this at the application level for all request not using my JSONAPI?
Ember-data-save-relationships
I found an add on here that provides an add on to do this. It seems more involved than the other approaches but when I've tried this I can send the data up by setting a the data as embedded. Great! But although it saves it doesn't unwrap it correct and I'm back with the same issues.
Custom serialiser
Replace the models serialiser with something that takes the data and sends it as plain JSON data and then deserialises back into something Ember can use. This sounds similar to the above but I do the heavy lifting. The only reason to do this is because all examples for the above solutions are quite light and don't really show how to set this up with an actual JSONAPI set up that would need it.
Where I am and what I need
Basically all approaches lead to saving the JSON fine but the return JSON from the server not being the correct format or the deserialisation failing but it's unclear what it should be or what needs to change without breaking the existing JSONAPI models that work fine.
If anyone know the format for return API data it may resolve this. I've tried JSONAPI with lineEntries returning the same format as it saved. I've tried placing relationship sections like the add on suggested and I've also tried placing relationship only data against the entries and an include section with all the references. Any help on this would be great as I've learned a lot through this but deadlines a looming and I can't see a viable solution that doesn't break as much as it fixes.
If you are looking for return format for relational data from the API server you need to make sure of the following:
Make sure the relationship is defined in the ember model
Return all successes with a status code of 200
From there you need to make sure you return relational data correctly. If you've set the ember model for the relationship to {async: true} you need only return the id of the relational model - which should also be defined in ember. If you do not set {async: true}, ember expects all relational data to be included.
return data with relationships in JSON API specification
Example:
models\unicorn.js in ember:
import DS from 'ember-data';
export default DS.Model.extend({
user: DS.belongsTo('user', {async: true}),
staticrace: DS.belongsTo('staticrace',{async: true}),
unicornName: DS.attr('string'),
unicornLevel: DS.attr('number'),
experience: DS.attr('number'),
hatchesAt: DS.attr('number'),
isHatched: DS.attr('boolean'),
raceEndsAt: DS.attr('number'),
isRacing: DS.attr('boolean'),
});
in routes\unicorns.js on the api server on GET/:id:
var jsonObject = {
"data": {
"type": "unicorn",
"id": unicorn.dataValues.id,
"attributes": {
"unicorn-name" : unicorn.dataValues.unicornName,
"unicorn-level" : unicorn.dataValues.unicornLevel,
"experience" : unicorn.dataValues.experience,
"hatches-at" : unicorn.dataValues.hatchesAt,
"is-hatched" : unicorn.dataValues.isHatched,
"raceEndsAt" : unicorn.dataValues.raceEndsAt,
"isRacing" : unicorn.dataValues.isRacing
},
"relationships": {
"staticrace": {
"data": {"type": "staticrace", "id" : unicorn.dataValues.staticRaceId}
},
"user":{
"data": {"type": "user", "id" : unicorn.dataValues.userId}
}
}
}
}
res.status(200).json(jsonObject);
In ember, you can call this by chaining model functions. For example when this unicorn goes to race in controllers\unicornracer.js:
raceUnicorn() {
if (this.get('unicornId') === '') {return false}
else {
return this.store.findRecord('unicorn', this.get('unicornId', { backgroundReload: false})).then(unicorn => {
return this.store.findRecord('staticrace', this.get('raceId')).then(staticrace => {
if (unicorn.getProperties('unicornLevel').unicornLevel >= staticrace.getProperties('raceMinimumLevel').raceMinimumLevel) {
unicorn.set('isRacing', true);
unicorn.set('staticrace', staticrace);
unicorn.set('raceEndsAt', Math.floor(Date.now()/1000) + staticrace.get('duration'))
this.set('unicornId', '');
return unicorn.save();
}
else {return false;}
});
});
}
}
The above code sends a PATCH to the api server route unicorns/:id
Final note about GET,POST,DELETE,PATCH:
GET assumes you are getting ALL of the information associated with a model (the example above shows a GET response). This is associated with model.findRecord (GET/:id)(expects one record), model.findAll(GET/)(expects an array of records), model.query(GET/?query=&string=)(expects an array of records), model.queryRecord(GET/?query=&string=)(expects one record)
POST assumes you at least return at least what you POST to the api server from ember , but can also return additional information you created on the apiServer side such as createdAt dates. If the data returned is different from what you used to create the model, it'll update the created model with the returned information. This is associated with model.createRecord(POST/)(expects one record).
DELETE assumes you return the type, and the id of the deleted object, not data or relationships. This is associated with model.deleteRecord(DELETE/:id)(expects one record).
PATCH assumes you return at least what information was changed. If you only change one field, for instance in my unicorn model, the unicornName, it would only PATCH the following:
{
data: {
"type":"unicorn",
"id": req.params.id,
"attributes": {
"unicorn-name" : "This is a new name!"
}
}
}
So it only expects a returned response of at least that, but like POST, you can return other changed items!
I hope this answers your questions about the JSON API adapter. Most of this information was originally gleamed by reading over the specification at http://jsonapi.org/format/ and the ember implementation documentation at https://emberjs.com/api/data/classes/DS.JSONAPIAdapter.html
Is there a way to store custom application configurations on the Cumulocity backend through the c8y.sdk? I would like to store a JSON with configuration information specific to an application created using the the smart app toolkit.
You can save settings at the tenant by using c8ySettings,
BUT you have a limitation of 256 characters per value.
I'm struggling with the same problem of storing larger plugin configurations without any success.
I asked about this from cumulocity support and they said it is possible to store custom JSON under managedobjects because at the moment they dont support storing JSON to database otherwise.
So you will need "create" or "admin" rights to inventory to be able to create new managedobject. You can store values like this:
var userSettings = {
type: 'userDashboardSettings',
yourSetting: somesetting,
id: settingsId
};
c8yInventory.save(userSettings).then(function(){
//do something
});
then you can search this newly created managedobject like this:
c8yInventory.list({
type: 'userDashboardSettings',
owner: id,
pageSize: 2
}).then(function (settings) { //do something }
So this way I was for example able to save user specific settings.
To find user dashboard settings do GET to: https://yourdomain.com/inventory/managedObjects/?type=userDashboardSettings&owner=IDHERE
To Delete managedObject do DELETE to: https://yourdomain.com/inventory/managedObjects/IDHERE
To see all managedObjects do GET: https://yourdomain.com/inventory/managedObjects
Type and user are important, this is how you find the managedobject.
I am looking to get all projects under a selected project (i.e the entire child project branch ) using Wsapi data store query in Rally SDK 2.0rc1. Is it possible using a query to recursively get all child project names? or will I have to write a separate recursive function to get that information? If a separate recursive function is required, how should I populate that data into for example, a combo box? Do I need to create a separate data store and push the data from my recursive function in it and then link the Combobox's store to it?
Also, how to get the "current workspace name" (workspace that I am working in, inside Rally), in Rally SDK 2.0rc1 ?
Use the 'context' config option to specify which project level to start at and add 'projectScopeDown' to make sure child projects are returned. That would look something like this:
Ext.create('Rally.data.WsapiDataStore', {
limit : Infinity,
model : 'Project',
fetch : ['Name','ObjectID'],
context : {
project : '/project/' + PROJECT_OID,
projectScopeDown : true
}
}).load({
callback: function(store) {
//Use project store data here
}
});
To get your current context data, use: this.getContext().
var workspace = this.getContext().getWorkspace();
var project = this.getContext().getProject();
If you try exposing with console.log the this.getContext().getWorkspace() and this.getContext().getProject() you may understand better what is returned and what is required. In one of my cases I had to use this.getContext().getProject().project.
Using console debug statement is best way to figure what you need based on its usage.
I'm trying to code a class handling serialization of documents by reading their metadata. I got inspired by this implementation for entities with Doctrine ORM and modified it to match how Doctrine ODM handles documents. Unfortunatly something is not working correctly as one document is never serialized more than once even if it is refered a 2nd time thus resulting on incomplete serialization.
For example, it outputs this (in json) for a user1 (see User document) that belongs to some place1 (see Place document). Then it outputs the place and the users belonging to it where we should see the user1 again but we don't :
{
id: "505cac0d6803fa1e15000004",
login: "user1",
places: [
{
id: "505cac0d6803fa1e15000005",
code: "place1",
users: [
{
id: "505c862c6803fa6812000000",
login: "user2"
}
]
}
]
}
I guess it could be related to something preventing circular references but is there a way around it ?
Also, i'm using this in a ZF2 application, would there be a better way to implement this using the ZF2 Serializer ?
Thanks for your help.
I have a serializer already written for DoctrineODM. You can find it in http://github.com/superdweebie/DoctrineExtensions - look in lib/Sds/DoctrineExtensions/Serializer.
If you are are using zf2, then you might also like http://github.com/superdweebie/DoctrineExtensionsModule, which configures DoctrineExtensions for use in zf2.
To use the Module, install it with composer, as you would any other module. Then add the following to your zf2 config:
'sds' => [
'doctrineExtensions' => [
'extensionConfigs' => [
'Sds\DoctrineExtensions\Serializer' => null,
),
),
),
To get the serializer use:
$serializer = $serivceLocator->get('Sds\DoctrineExtensions\Serializer');
To use the serializer:
$array = $serializer->toArray($document)
$json = $serializer->toJson($document)
$document = $serializer->fromArray($array)
$document = $serializer->fromJson($json)
There are also some extra annotations available to control serialization, if you want to use them:
#Sds\Setter - specify a non standard setter for a property
#Sds\Getter - specify a non standard getter fora property
#Sds\Serializer(#Sds\Ignore) - ignore a property when serializing
It's all still a work in progress, so any comments/improvements would be much appreciated. As you come across issues with these libs, just log them on github and they will get addressed promptly.
Finally a note on serializing embedded documents and referenced documents - embedded documents should be serialized with their parent, while referenced documents should not. This reflects the way data is saved in the db. It also means circular references are not a problem.
Update
I've pushed updates to Sds/DoctrineExtensions/Serializer so that it can now handle references properly. The following three (five) methods have been updated:
toArray/toJson
fromArray/fromJson
applySerializeMetadataToArray
The first two are self explainitory - the last is to allow serialization rules to be applied without having to hydrate db results into documents.
By default references will be serialized to an array like this:
[$ref: 'CollectionName/DocumentId']
The $ref style of referencing is what Mongo uses internally, so it seemed appropriate. The format of the reference is given with the expectation it could be used as a URL to a REST API.
The default behaviour can be overridden by defineing an alternative ReferenceSerializer like this:
/**
* #ODM\ReferenceMany(targetDocument="MyTargetDocument")
* #Sds\Serializer(#Sds\ReferenceSerializer('MyAlternativeSerializer'))
*/
protected $myDocumentProperty;
One alternate ReferenceSerializer is already included with the lib. It is the eager serializer - it will serialize references as if they were embedded documents. It can be used like this:
/**
* #ODM\ReferenceMany(targetDocument="MyTargetDocument")
* #Sds\Serializer(#Sds\ReferenceSerializer('Sds\DoctrineExtensions\Serializer\Reference\Eager'))
*/
protected $myDocumentProperty;
Or an alternate shorthand annotation is provided:
/**
* #ODM\ReferenceMany(targetDocument="MyTargetDocument")
* #Sds\Serializer(#Sds\Eager))
*/
protected $myDocumentProperty;
Alternate ReferenceSerializers must implement Sds\DoctrineExtensions\Serializer\Reference\ReferenceSerializerInterface
Also, I cleaned up the ignore annotation, so the following annotations can be added to properties to give more fine grained control of serialization:
#Sds\Serializer(#Sds\Ignore('ignore_when_serializing'))
#Sds\Serializer(#Sds\Ignore('ignore_when_unserializing'))
#Sds\Serializer(#Sds\Ignore('ignore_always'))
#Sds\Serializer(#Sds\Ignore('ignore_never'))
For example, put #Sds\Serializer(#Sds\Ignore('ignore_when_serializing')) on an email property - it means that the email can be sent upto the server for update, but can never be serialized down to the client for security.
And lastly, if you hadn't noticed, sds annotations support inheritance and overriding, so they play nice with complex document structures.
Another very simple, framework independent way to transforming Doctrine ODM Document to Array or JSON - http://ajaxray.com/blog/converting-doctrine-mongodb-document-tojson-or-toarray
This solution gives you a Trait that provides toArray() and toJSON() functions for your ODM Documents. After useing the trait in your Document, you can do -
<?php
// Assuming in a Symfony2 Controller
// If you're not, then make your DocmentManager as you want
$dm = $this->get('doctrine_mongodb')->getManager();
$report = $dm->getRepository('YourCoreBundle:Report')->find($id);
// Will return simple PHP array
$docArray = $report->toArray();
// Will return JSON string
$docJSON = $report->toJSON();
BTW, it will work only on PHP 5.4 and above.