I have this scenario at hand:
I have an API server holding enum value and sending it using Restfull API.
The client will receive it and needs to act according to the value.
My main question: Which is better - sending the int value or the string? I can see benefits for both approaches.
Is there any way to avoid holding the enum in both sides? I am not familiar with one that actually can be useful.
Thanks!
If the API server maintains the enum, the client could fetch it by:
GET /enums
... which will return a list of enum values:
[
{ "id" : "1001", "value" : "enum-item-1" },
{ "id" : "1002", "value" : "enum-item-2" },
...
{ "id" : "100N", "value" : "enum-item-3" },
]
Thus allowing the client to fetch one of the enum items:
GET /enums/1017
Or perhaps perform an operation on it:
POST /enums/1017/disable
Normally, one would only refer to the enum items by their unique ID - and if the client always starts by querying the server for the enum list, you will not need to maintain the enum on the client and server.
But - if the values in your business case are permanently unique and there is a compelling reason to have 'nicer' human-readable URLs, one could use:
GET /enums/enum-item-26
Generally, this is not best practice, as the enum item value probably has business meaning and thus might change. Even though that currently seems unlikely.
Related
I have appsettings.json with code:
"Serilog": {
"WriteTo": [
{
"Name": "RollingFile",
"Args": {
"pathFormat": "/home/www-data/aissubject/storage/logs/log-{Date}.txt"
}
}
]
}
How can I read value of "pathFormat" key?
What you're referring to is a JSON array. How you access that varies depending on what you're doing, but I'm assuming that since you're asking this, you're trying to get it directly out of IConfiguration, rather than using the options pattern (as you likely should be).
IConfiguration is basically a dictionary. In order to create the keys of that dictionary from something like JSON, the JSON is "flattened" using certain conventions. Each level will be separated by a colon. Arrays will be flattened by adding a colon-delimited component containing the index. In other words, to get at pathFormat in this particular example, you'd need:
Configuration["Serilog:WriteTo:0:Args:pathFormat"]
Where the 0 portion denotes that you're getting the first item in the array. Again, it's much better and more appropriate to use the options pattern to map the configuration values onto an actual object, which would let you actually access this as an array rather than a magic string like this.
I'm building a generic API with content and a schema that can be user-defined. I want to add filtering logic to API responses, so that users can query for specific objects they've stored in the API. For example, if a user is storing event objects, they could do things like filter on:
Array contains: Whether properties.categories contains Engineering
Greater than: Whether properties.created_at is older than 2016-10-02
Not equal: Whether properties.address.city is not Washington
Equal: Whether properties.name is Meetup
etc.
I'm trying to design filtering into the query string of API responses, and coming up with a few options, but I'm not sure which syntax for it is best...
1. Operator as Nested Key
/events?properties.name=Harry&properties.address.city.neq=Washington
This example is uses just a nested object to specific the operators (like neq as shown). This is nice in that it is very simple, and easy to read.
But in cases where the properties of an event can be defined by the user, it runs into an issue where there is a potential clash between a property named address.city.neq using a normal equal operator, and a property named address.city using a not equal operator.
Example: Stripe's API
2. Operator as Key Suffix
/events?properties.name=Harry&properties.address.city+neq=Washington
This example is similar to the first one, except it uses a + delimiter (which is equivalent to a space) for operations, instead of . so that there is no confusion, since keys in my domain can't contain spaces.
One downside is that it is slightly harder to read, although that's arguable since it might be construed as more clear. Another might be that it is slightly harder to parse, but not that much.
3. Operator as Value Prefix
/events?properties.name=Harry&properties.address.city=neq:Washington
This example is very similar to the previous one, except that it moves the operator syntax into the value of the parameter instead of the key. This has the benefit of eliminating a bit of the complexity in parsing the query string.
But this comes at the cost of no longer being able to differentiate between an equal operator checking for the literal string neq:Washington and a not equal operator checking for the string Washington.
Example: Sparkpay's API
4. Custom Filter Parameter
/events?filter=properties.name==Harry;properties.address.city!=Washington
This example uses a single top-level query paramter, filter, to namespace all of the filtering logic under. This is nice in that you never have to worry about the top-level namespace colliding. (Although in my case, everything custom is nested under properties. so this isn't an issue in the first place.)
But this comes at a cost of having a harder query string to type out when you want to do basic equality filtering, which will probably result in having to check the documentation most of the time. And relying on symbols for the operators might lead to confusion for non-obvious operations like "near" or "within" or "contains".
Example: Google Analytics's API
5. Custom Verbose Filter Parameter
/events?filter=properties.name eq Harry; properties.address.city neq Washington
This example uses a similar top-level filter parameter as the previous one, but it spells out the operators with word instead of defining them with symbols, and has spaces between them. This might be slightly more readable.
But this comes at a cost of having a longer URL, and a lot of spaces that will need to be encoded?
Example: OData's API
6. Object Filter Parameter
/events?filter[1][key]=properties.name&filter[1][eq]=Harry&filter[2][key]=properties.address.city&filter[2][neq]=Washington
This example also uses a top-level filter parameter, but instead of creating a completely custom syntax for it that mimics programming, it instead builds up an object definition of filters using a more standard query string syntax. This has the benefit of bring slightly more "standard".
But it comes at the cost of being very verbose to type and hard to parse.
Example Magento's API
Given all of those examples, or a different approach, which syntax is best? Ideally it would be easy to construct the query parameter, so that playing around in the URL bar is doable, but also not pose problems for future interoperability.
I'm leaning towards #2 since it seems like it is legible, but also doesn't have some of the downsides of other schemes.
I might not answer the "which one is best" question, but I can at least give you some insights and other examples to consider.
First, you are talking about "generic API with content and a schema that can be user-defined".
That sound a lot like solr / elasticsearch which are both hi level wrappers over Apache Lucene which basically indexes and aggregates documents.
Those two took totally different approaches to their rest API, I happened to work with both of them.
Elasticsearch :
They made entire JSON based Query DSL, which currently looks like this :
GET /_search
{
"query": {
"bool": {
"must": [
{ "match": { "title": "Search" }},
{ "match": { "content": "Elasticsearch" }}
],
"filter": [
{ "term": { "status": "published" }},
{ "range": { "publish_date": { "gte": "2015-01-01" }}}
]
}
}
}
Taken from their current doc. I was surprised that you can actually put data in GET...
It actually looks better now, in earlier versions it was much more hierarchical.
From my personal experience, this DSL was powerful, but rather hard to learn and use fluently (especially older versions). And to actually get some result you need more than just play with URL. Starting with the fact that many clients don't even support data in GET request.
SOLR :
They put everything into query params, which basically looks like this (taken from the doc) :
q=*:*&fq={!cache=false cost=5}inStock:true&fq={!frange l=1 u=4 cache=false cost=50}sqrt(popularity)
Working with that was more straightforward. But that's just my personal taste.
Now about my experiences. We were implementing another layer above those two and we took approach number #4. Actually, I think #4 and #5 should be supported at the same time. Why? Because whatever you pick people will be complaining, and since you will be having your own "micro-DSL" anyway, you might as well support few more aliases for your keywords.
Why not #2? Having single filter param and query inside gives you total control over DSL. Half a year after we made our resource, we got "simple" feature request - logical OR and parenthesis (). Query parameters are basically a list of AND operations and logical OR like city=London OR age>25 don't really fit there. On the other hand parenthesis introduced nesting into DSL structure, which would also be a problem in flat query string structure.
Well, those were the problems we stumbled upon, your case might be different. But it is still worth to consider, what future expectations from this API will be.
Matomo Analytics has an other approach to deal with segment filter and its syntaxe seems to be more readable and intuitive, e.g:
developer.matomo.org/api-reference/reporting-api-segmentation
Operator
Behavior
Example
==
Equals
&segment=countryCode==IN Return results where the country is India
!=
Not equals
&segment=actions!=1 Return results where the number of actions (page views, downloads, etc.) is not 1
<=
Less than or equal to
&segment=actions<=4 Return results where the number of actions (page views, downloads, etc.) is 4 or less
<
Less than
&segment=visitServerHour<12 Return results where the Server time (hour) is before midday.
=#
Contains
&segment=referrerName=#piwik Return results where the Referer name (website domain or search engine name) contains the word "piwik".
!#
Does not contain
&segment=referrerKeyword!#yourBrand Return results where the keyword used to access the website does not contain word "yourBrand".
=^
Starts with
&segment=referrerKeyword=^yourBrand Return results where the keyword used to access the website starts with "yourBrand" (requires at least Matomo 2.15.1).
=$
Ends with
&segment=referrerKeyword=$yourBrand Return results where the keyword used to access the website ends with "yourBrand" (requires at least Matomo 2.15.1).
and you can have a close look at how they parse the segment filter here: https://github.com/matomo-org/matomo/blob/4.x-dev/core/Segment/SegmentExpression.php
#4
I like how Google Analytics filter API looks like, easy to use and easy to understand from a client's point of view.
They use a URL encoded form, for example:
Equals: %3D%3D filters=ga:timeOnPage%3D%3D10
Not equals: !%3D filters=ga:timeOnPage!%3D10
Although you need to check documentation but it still has its own advantages. IF you think that the users can get accustomed to this then go for it.
#2
Using operators as key suffixes also seems like a good idea (according to your requirements).
However I would recommend to encode the + sign so that it isn't parsed as a space. Also it might be slightly harder to parse as mentioned but I think you can write a custom parser for this one. I stumbled across this gist by jlong some time back. Perhaps you'll find it useful to write your parser.
You could also try Spring Expression Language (SpEL)
All you need to do is to stick to the said format in the document, the SpEL engine would take care of parsing the query and executing it on a given object. Similar to your requirement of filtering a list of objects, you could write the query as:
properties.address.city == 'Washington' and properties.name == 'Harry'
It supports all kind of relational and logical operators that you would need. The rest api could just take this query as the filter string and pass it to SpEL engine to run on an object.
Benefits: it's readable, easy to write, and execution is well taken care of.
So, the URL would look like:
/events?filter="properties.address.city == 'Washington' and properties.name == 'Harry'"
Sample code using org.springframework:spring-core:4.3.4.RELEASE :
The main function of interest:
/**
* Filter the list of objects based on the given query
*
* #param query
* #param objects
* #return
*/
private static <T> List<T> filter(String query, List<T> objects) {
ExpressionParser parser = new SpelExpressionParser();
Expression exp = parser.parseExpression(query);
return objects.stream().filter(obj -> {
return exp.getValue(obj, Boolean.class);
}).collect(Collectors.toList());
}
Complete example with helper classes and other non-interesting code:
import java.util.Arrays;
import java.util.List;
import java.util.stream.Collectors;
import org.springframework.expression.Expression;
import org.springframework.expression.ExpressionParser;
import org.springframework.expression.spel.standard.SpelExpressionParser;
public class SpELTest {
public static void main(String[] args) {
String query = "address.city == 'Washington' and name == 'Harry'";
Event event1 = new Event(new Address("Washington"), "Harry");
Event event2 = new Event(new Address("XYZ"), "Harry");
List<Event> events = Arrays.asList(event1, event2);
List<Event> filteredEvents = filter(query, events);
System.out.println(filteredEvents.size()); // 1
}
/**
* Filter the list of objects based on the query
*
* #param query
* #param objects
* #return
*/
private static <T> List<T> filter(String query, List<T> objects) {
ExpressionParser parser = new SpelExpressionParser();
Expression exp = parser.parseExpression(query);
return objects.stream().filter(obj -> {
return exp.getValue(obj, Boolean.class);
}).collect(Collectors.toList());
}
public static class Event {
private Address address;
private String name;
public Event(Address address, String name) {
this.address = address;
this.name = name;
}
public Address getAddress() {
return address;
}
public void setAddress(Address address) {
this.address = address;
}
public String getName() {
return name;
}
public void setName(String name) {
this.name = name;
}
}
public static class Address {
private String city;
public Address(String city) {
this.city = city;
}
public String getCity() {
return city;
}
public void setCity(String city) {
this.city = city;
}
}
}
I decided to compare the approaches #1/#2 (1) and #3 (2) and concluded that (1) is preferred (at least, for Java server side).
Assume, some parameter a must be equal 10 or 20. Our URL query in this case must look like ?a.eq=10&a.eq=20 for (1) and ?a=eq:10&a=eq:20 for (2). In Java HttpServletRequest#getParameterMap() will return the next values: { a.eq: [10, 20] } for (1) and { a: [eq:10, eq:20] } for (2). Later we must convert returned maps, for example, to SQL where clause. And we should get: where a = 10 or a = 20 for both (1) and (2). Briefly, it looks something like that:
1) ?a=eq:10&a=eq:20 -> { a: [eq:10, eq:20] } -> where a = 10 or a = 20
2) ?a.eq=10&a.eq=20 -> { a.eq: [10, 20] } -> where a = 10 or a = 20
So, we got the next rule: when we pass through URL query two parameters with the same name we must use OR operand in SQL.
But let's assume another case. The parameter a must be greater than 10 and less than
20. Applying the rule above we will have the next conversion:
1) ?a.gt=10&a.ls=20 -> { a.gt: 10, a.lt: 20 } -> where a > 10 and a < 20
2) ?a=gt:10&a=ls:20 -> { a: [gt.10, lt.20] } -> where a > 10 or(?!) a < 20
As you can see, in (1) we have two parameters with different names: a.gt and a.ls. This means our SQL query will have AND operand. But for (2) we still have the same names and it must be converted to the SQL with OR operand!
This means that for (2) instead of using #getParameterMap() we must directly parse the URL query and analyze repeated parameter names.
I know this is old school, but how about a sort of operator overloading?
It would make the query parsing a lot harder (and not standard CGI), but would resemble the contents of an SQL WHERE clause.
/events?properties.name=Harry&properties.address.city+neq=Washington
would become
/events?properties.name=='Harry'&&properties.address.city!='Washington'||properties.name=='Jack'&&properties.address.city!=('Paris','New Orleans')
paranthesis would start a list. Keeping strings in quotes would simplify parsing.
So the above query would be for events for Harry's not in Washington or for Jacks not in Paris or in New Orleans.
It would be a ton of work to implement... and the database optimization to run those queries would be a nightmare, but if you're looking for a simple and powerful query language, just imitate SQL :)
-k
We have several aggregate roots that have two primary means of identification:
an integer "key", which is used as a primary key in the database (and is used as a foreign key by referencing aggregates), and internally within the application, and is not accessible by the public web API.
a string-based "id", which also uniquely identifies the aggregate root and is accessible by the public web API.
There are several reasons for having an integer-based private identifiers and a string-based public identifier - for example, the database performs better (8-byte integers as opposed to variable-length strings) and the public identifiers are difficult to guess.
However, the classes internally reference each other using the integer-based identifiers and if an integer-based identifier is 0, this signifies that the object hasn't yet been stored to the database. This creates a problem, in that entities are not able to reference other aggregate roots until after they have been saved.
How does one get around this problem, or is there a flaw in my understanding of persistence ignorance?
EDIT regarding string-based identifiers
The string-based identifiers are generated by the repository, connected to a PostgreSQL database, which generates the identifier to ensure that it does not clash with anything currently in the database. For example:
class Customer {
public function __construct($customerKey, $customerId, $name) {
$this->customerKey = $customerKey;
$this->customerId = $customerId;
$this->name = $name;
}
}
function test(Repository $repository, UnitOfWork $unitOfWork) {
$customer = new Customer(0, $repository->generateCustomerId(), "John Doe");
// $customer->customerKey == 0
$unitOfWork->saveCustomer($customer);
// $customer->customerKey != 0
}
I assume that the same concept could be used to create an entity with an integer-based key of non-0, and the Unit of Work could use the fact that it doesn't exist in the database as a reason to INSERT rather than UPDATE. The test() function above would then become:
function test(Repository $repository, UnitOfWork $unitOfWork) {
$customer = new Customer($repository->generateCustomerKey(), $repository->generateCustomerId(), "John Doe");
// $customer->customerKey != 0
$unitOfWork->saveCustomer($customer);
// $customer->customerKey still != 0
}
However, given the above, errors may occur if the Unit of Work does not save the database objects in the correct order. Is the way to get around this to ensure that the Unit of Work saves entities in the correct order?
I hope the above edit clarifies my situation.
It's a good approach to look at Aggregates as consistency boundaries. In other words, two different aggregates have separate lifecycles and you should refrain from tying their fates together inside the same transaction. From that axiom you can safely state that no aggregate A will ever have an ID of 0 when looked at from another aggregate B's perspective, because either the transaction that creates A has not finished yet and it is not visible by B, or it has completed and A has an ID.
Regarding the double identity, I'd rather have the string ID generated by the language than the database because I suppose coming up with a unique ID would imply a transaction, possibly across multiple tables. Languages can usually generate unique strings with a good entropy.
I'm trying to find a data schema which can be used for different scenaries and a promissing format I found so far is the collection+json format (http://amundsen.com/media-types/collection/ ).
So far it has a lot of the functionallity I need and is very flexible, however I don't get why it uses anonymous objects ( example: {"name" : "full-name", "value" : "J. Doe", "prompt" : "Full Name"}, ) instead of simple key value pairs. (example: "full-name": "J. Doe", ).
I see how you can transfer more information like the prompt, etc. but the parsing is much slower and it is harder to create a client for it since he has to access the fields by searching in an array. When binding the data to a spezific view, it has to be know which fields exists, so the anonymous objects have to be converted into a key value map again.
So is there a real advante using this anonymous objects instead of a key value map?
I think that the main reason is because a consumer client does not need to know in advance the format of the data.
As it is proposed now in collection+json, you know that in the data object you will find stuff about data simply by parsing through it, 'name' is always the identifying name for the field, 'value' is the value and so on, your client can be agnostic about how many fields or their names:
{
"name" : "full-name",
"value" : "J. Doe",
"prompt" : "Full Name"
},
{
"name" : "age",
"value" : "42",
"prompt" : "Age"
}
if you had instead
{
"full-name" : "J. Doe",
"age" : "42"
}
the client needs to have previous knowledge about your representation, so it should expect and understand 'full-name', 'age, and all the application specific fields.
I wrote this question and then forgott about it, but found the answer I looked for here:
https://groups.google.com/forum/#!searchin/collectionjson/key/collectionjson/_xaXs2Q7N_0/GQkg2mvPjqMJ
From Mike Amundsen the creator of collection+JSON
I understand your desire to add object serialization patterns to CJ.
However, one of the primary goals of my CJ design is to not support
object serialization. I know that serialization is an often-requested
feature for Web messages. It's a great way to optimize both the code
and the programmer experience. But it's just not what I am aiming for
in this particular design.
I think the extension Kevin ref'd is a good option. Not sure if anyone
is really using it, tho. If you'd like to design one (based on your
"body" idea), that's cool. If you'd like to start w/ a gist instead of
doing the whole "pull" thing, that's cool. Post it and let's talk
about it.
On another note, I've written a very basic client-side parser for CJ
(it's in the basic folder of the repo) to show how to hide the
"costly" parts of navigating CJ's state model cleanly. I actually
have a work item to create a client-side lib that can convert the
state representation into an arbitrary local object model, but haven't
had time to do the work yet. Maybe this is something you'd like help
me with?
At a deeper (and possibly more boring) level, this state-model
approach is part of a bigger pattern I have in mind to treat messages
as "type-less" containers and to allow clients and servers to utilize
whatever local object models they prefer - without the need for
cross-web agreement on that object model. This is an "opinionated
design model" that I am working toward.
Finally, as you rightly point out at the top of the thread, HAL and
Siren are much better able to support object serialization style
messages. I think that's very cool and I encourage folks (including my
clients) to use these other formats when object serialization is the
preferred pattern and to use CJ when state transfer is the preferred
pattern.
This question already has an answer here:
Newtonsoft JSON.net deserialization error where fields in JSON change order
(1 answer)
Closed 6 years ago.
I have the following method in my web api
public void Put(string id, [FromBody]IContent value) {
//Do stuff
}
I'm using backbone js to send the following JSON to the server using fiddler the value is null:
{
"id": "articles/1",
"heading": "Bar",
"$type": "BrickPile.Samples.Models.Article, BrickPile.Samples"
}
but if I add the $type property first in the JSON object the deserialization works fine, see:
{
"$type": "BrickPile.Samples.Models.Article, BrickPile.Samples",
"id": "articles/1",
"heading": "Bar"
}
is it possible to configure newtonsoft to check for the $type property anywhere in the object instead of the first property or can I configure backbone so it always adds the $type property first in the JSON object?
I would very strongly recommend against configuring any serializer (including JSON.NET) to read the object type from the incoming payload. This has historically been the cause of a large number of vulnerabilities in web applications. Instead, change the public entry point to your action to take the actual type as a bound parameter, then delegate to an internal testable method if desired.
First, AFAIK, the code of Json.NET is optimized to avoid holding the whole object in memory just to read its type. So it's better to place $type as the first property.
Second, you can write your own JsonConverter which reads first to JObject (using Load method), manually reads $type property, gets type from serializer's SerializationBinder, creates the value and populates it from JObject.
Third, regarding security. While Json.NET's $type may sound like a good idea, it's often not. It allows Json.NET to create any object type from any assembly just by writing its type in JSON file. It's better to use custom SerializationBinder with a dictionary which allows only types which you specify. You can find an example in my private framework (it also supports getting values for $type from JsonObjectAttribute):
https://github.com/Athari/Alba.Framework/blob/742ff1aeeb114179a16ca42667781944b26f3815/Alba.Framework/Serialization/DictionarySerializationBinder.cs
(This version uses some methods from other classes, but they're trivial. Later commits made the class more complex.)
I had kind of the same question apparently, and someone found an answer. Not sure whats the appropriate way to share an answer and give him ALL the credit, but this is the link:
Newtonsoft JSON.net deserialization error where fields in JSON change order
and this is the guy:
https://stackoverflow.com/users/3744182/dbc
This will work in backbone, but I don't know if every browser will behave the same. There's no guarantee, basically, that every browser will keep the items in the order which they are added.
MyModel = Backbone.Model.extend({
// ...
toJSON: function(){
// build the "$type" as the first parameter
var json = {"$type": "BrickPile.Samples.Models.Article, BrickPile.Samples"};
// get the rest of the data
_.extend(json, Backbone.Model.prototype.toJSON.call(this));
// send it back, and hope it's in the right order
return json;
}
});
You're better of getting NewtonSoft's JSON deserializer to work without needing it in a specific position, though. Hopefully that will be possible.