I'm designing a RESTful API and I'm trying to work out how I could represent a predicate with OR an operator when querying for a resource.
For example if I had a resource Foo with a property Name, how would you search for all Foo resources with a name matching "Name1" OR "Name2"?
This is straight forward when it's an AND operator as I could do the following:
http://www.website.com/Foo?Name=Name1&Age=19
The other approach I've seen is to post the search in the body.
You will need to pick your own approach, but I can name few that seem to be pretty logical (although not without disadvantages):
Option 1.: Using | operator:
http://www.website.com/Foo?Name=Name1|Name2
Option 2.: Using modified query param to allow selection by one of the values from the set (list of possible comma-separated values):
http://www.website.com/Foo?Name_in=Name1,Name2
Option 3.: Using PHP-like notation to provide list instead of single string:
http://www.website.com/Foo?Name[]=Name1&Name[]=Name2
All of the above mentioned options have one huge advantage: they do not interfere with other query params.
But as I mentioned, pick your own approach and be consistent about it across your API.
Well one quick way to fixing that is to add an additional parameter that is identifying the relationship between your parameters wether they're an and or an or for example:
http://www.website.com/Foo?Name=Name1&Age=19&or=true
Or for much more complex queries just keep a single parameter and in it include your whole query by making up your own little query language and on the server side you would parse the whole string and extract the information and the statement.
Related
I need an REST API endpoint which will return all the records having name starting with ABC.
The SQL query would be something like:
SELECT * FROM MyResource WHERE Name LIKE 'ABC%'
But how should I define the query string in the REST endpoint?
Using equal sign in the query string would not be appropriate, I think.
{Base URL}/myresource?name=ABC
Sofar I found following specification that can be used as base for the definition of the query: https://datatracker.ietf.org/doc/html/draft-ietf-scim-api-12#section-3.2.2.2
Equals is fine provided you use correct naming. For this request you can define i.e. 'namePrefix' parameter that is self-descriptive.
But how should I define the query string in the REST endpoint? Using equal sign in the query string would not be appropriate, I think.
Using an equal sign in the query string is fine -- the world wide web has been catastrophically successful, and you'll find query strings with encoded key/value pairs being used for all sorts of things.
There's no particular reason to assume that the spelling of a resource identifier should necessarily match the internal implementation details -- in fact, the opposite is the case: we're supposed to be able to change how a resource is implemented without necessarily needing to introduce a new identifier.
There's a tremendous advantage to using application/x-www-form-urlencoded data as your query string: that's how HTML GET forms do it, which in turn means that pretty much everyone has access to at least one general purpose library that knows how to construct resource identifiers that use that convention.
But if you would rather forego that advantage in favor of some other concern, that's OK too; REST/HTTP don't care what spelling conventions you use for your resource identifiers, so long as the result is consistent with the production rules described in RFC 3986.
My client is making database searches using a django webapp that I've written. The query sends a regex search to the database and outputs the results.
Because the regex searches can be pretty long and unintuitive, the client has asked for certain custom "wildcards" to be created for the regex searches. For example.
Ω := [^aeiou] (all non-vowels)
etc.
This could be achieved with a simple permanent string substitution in the query, something like
query = query.replace("Ω", "[^aeiou]")
for all the elements in the substitution list. This seems like it should be safe, but I'm not really sure.
He has also asked that it be possible for the user to define custom wildcards for their searches on the fly. So that there would be some other input box where a user could define
∫ := some other regex
And to store them you might create a model
class RegexWildcard(models.Model):
symbol = ...
replacement = ...
I'm personally a bit wary of this, because it does not seem to add a whole lot of functionality, but does seem to add a lot of complexity and potential problems to the code. Clients can now write their queries to a db. Can they overwrite each other's symbols?
That I haven't seen this done anywhere before also makes me kind of wary of the idea.
Is this possible? Desirable? A great idea? A terrible idea? Resources and any guidance appreciated.
Well, you're getting paid by the hour....
I don't see how involving the Greek alphabet is to anyone's advantage. If the queries are stored anywhere, everyone approaching the system would have to learn the new syntax to understand them. Plus, there's the problem of how to type the special symbols.
If the client creates complex regular expressions they'd like to be able to reuse, that's understandable. Your application could maintain a list of such expressions that the user could add to and choose from. Notionally, the user would "click on" an expression, and it would be inserted into the query.
The saved expressions could have user-defined names, to make them easier to remember and refer to. And you could define a syntax that referenced them, something otherwise invalid in SQL, such as ::name. Before submitting the query to the DBMS, you substitute the regex for the name.
You still have the problem of choosing good names, and training.
To prevent malformed SQL, I imagine you'll want to ensure the regex is valid. You wouldn't want your system to store a ; drop table CUSTOMERS; as a "regular expression"! You'll either have to validate the expression or, if you can, treat the regex as data in a parameterized query.
The real question to me, though, is why you're in the vicinity of standardized regex queries. That need suggests a database design issue: it suggests the column being queried is composed of composite data, and should be represented as multiple columns that can be queried directly, without using regular expressions.
My requirement is to implement advanced search Rest API for searching the phones. The URI for the search API is http://myservice/api/v1/phones/search?q=${query_expression}
Where q is the complex query expression. Have the following questions
1) Since advanced search involves a lengthy query expression, the URI will not fit in a GET call. Is it alright to implement the search API via POST request and still maintain the RESTfulness?
2) I have come across the following implementations for the advanced search:
1st approach - Send the complete infix expression for the query expression.
eg.
PHONENAME STARTSWITH 'AR' AND ( PHONETYPE = '4G' OR PHONECOLOR = 'RED')
2nd approach - Constructing entire query expression in the form of a json.
eg.
{"criteria":[
{"index":1,"field":"PHONENAME","value":"AR","comparator":"STARTSWITH"},
{"index":2,"field":"PHONETYPE","value":"4G","comparator":"EQUALS"},
{"index":3,"field":"PHONECOLOR","value":"RED","comparator":"EQUALS"}
],"criteria":"( 1 AND (2 OR 3) )"}
3rd approach - Alternative way to implement the query expression as a json.
eg.
{"and":[
{"field":"PHONENAME","value":"AR","comparator":"STARTSWITH"},
"or":[
{"field":"PHONETYPE","value":"4G","comparator":"EQUALS"},
{"field":"PHONECOLOR","value":"RED","comparator":"EQUALS"}]
]}
Which approach would be considered more RESTful out of the three? Suggestions for any other approaches are welcome :)
You could follow the approach taken by ElasticSearch, which out of the examples you had given is the third one.
See https://www.elastic.co/guide/en/elasticsearch/reference/current/search.html
The third approach is also easier to understand and easier to maintain.
For example if in the future you would need to add "fuzzy" query operator and it would have a completely different model, that would be an easy thing to do.
Yes, POST is a catch-all. It's preferable to use it for resource creation, but according to the spec it may be used in this way also. However, you should consider changing the endpoint to be /search-results. This gives you the flexibility to start storing search results later, and you can return a Location header pointing to the results of a particular complex query. Another alternative is to let users POST their search criteria, and then do a GET /search-results?criteria={id}.
Don't do the second one. It's hard to read and more complex than it should be. Either the first or the third are fine. The first is more compact but probably harder to handle on the back end. For the third, you really don't need the index.
First of all, I have read RESTful URL design for search and How to design RESTful search/filtering? questions. I am trying to design more advanced options for searching in a simple and RESTful way.
The answers to those questions have given me some insight and clues on how to design my previous application url pattern for search/filter functionality.
First, I came up with quite nice and simple solution for basic filtering options, using pattern:
Equality search: key = val
IN search: key = val1 & key = val2
But as application has grown, so were the search requirements. And I ended up with some rather unpleasant and complex url pattern for advanced searching options which included:
Negation search: key-N = val
Like search: key-L = val
OR search: key1-O = val1 & key2 = val2
Range search: key1-RS = val1 & key1-RE = val2
Whats more, beside filters, query has to get information about pagination and order by, so the filter parameter has F- suffix, order by fields has O- suffix, and pagination has P- suffix.
I hope that at this point I do not have to add that parsing such request is rather malicious task, with the possibility of ambiguity if key will contain '-'. I have created some regexp to parse it, and it works quite well as for now, but...
Now I am starting to write a new web app and I have the chance to redesign this piece from scratch.
I am wondering about creating object in a browser containing all information in structured and self-explanatory way and send it to server as as JSON string, like:
filter = {{'type':'like','field':key,'value':val1,'operator':'and','negation':false},..}
But I get strange feeling that this is not good idea - I really don't know why.
So, this would be the definition of my context. Now the question:
I am searching for simpler and safer pattern for implementing advanced search including options I mentioned above as RESTful GET parameters - can you share some ideas?
Or maybe some insights on not doing this in a RESTful way?
Also, if you see some pitfalls in JSON way, please share them.
EDIT:
I know what makes sending json as get parameter, not so good idea. Encoding it - it makes it ugly and hard to read.
Info provided by links sended by thierry templier, gave me something to think about and I managed to design more consistient and safe filter handling in GET parameters. Below is definition of syntax.
For filters - multiple F parameters (one for each search criterium):
F = OPERATOR:NEGATION:TYPE:FIELD:VAL[:VAL1,:VAL2...]
allowed values:
[AND|OR]:[T|F]:[EQ|LI|IN|RA]:FIELD_NAME:VALUE1:VALUE2...
For order by - multiple O parameters (one for each ordered field):
O = ODINAL_NO:DIRECTION:FIELD
allowed values:
[0-9]+:[ASC|DESC]:FIELD_NAME
Pagination - single P parameter:
P = ITEMS_PER_PAGE:FROM_PAGE:TO_PAGE
I think this will be good solution - it meets all my requirements, it is easy to parse and write, it is readable and I do not see how that syntax can become ambiguous.
I wloud appreciate any thoughts on that idea - do you see any pitfalls?
There are several options here. But it's clear that if your queries tend to be complex with operators, and so on... you can't use a set of query parameters. I see two approaches:
Provide the query as JSON content to a method POST
Provide the query in a query parameter with a specific format / grammar to a method GET
I think that you could have a look at what ElasticSearch for their queries. They are able to describe very complex queries with JSON contents (using several levels). Here is a link to their query DSL: http://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl.html.
You can also have a look at what OData does for queries. They choose another approach with a single query parameter $filter. Here are some links that can give you some examples: https://msdn.microsoft.com/en-us/library/hh169248(v=nav.70) and http://www.odata.org/documentation/odata-version-3-0/url-conventions/. This option requires to have a grammar on the server side to parse your query.
In general, this link could also give you some hints at this level in its section "Filtering data": https://templth.wordpress.com/2014/12/15/designing-a-web-api/.
Hope it gives you some helpful hints to design your queries within your RESTful services ;-)
Thierry
In the android Notes demo, it accepts the URI:
sUriMatcher.addURI(NotePad.AUTHORITY, "notes", NOTES);
sUriMatcher.addURI(NotePad.AUTHORITY, "notes/#", NOTE_ID);
Where the difference between notes and notes/# is that notes/# returns the note who's ID matches #.
However, the managedQuery() method that is used to get data from the content provider has the following parameters:
Parameters
uri The URI of the content provider to query.
projection List of columns to return.
selection SQL WHERE clause.
selectionArgs The arguments to selection, if any ?s are pesent
sortOrder SQL ORDER BY clause.
So, is there any particular cause for the design decision of providing a URI for that, rather than just using the selection parameter? Or is it just a matter of taste?
Thank you.
I thinks its so you can do more complex lookups without having to complicate your selections and arguments. For example in my project I have multiple tables but use the same selection and arguments. To filter content. By using the URI I don't have interpret the query, I can just switch on the URI. It.might be personal taste to begin with. But in more complex scenarios you appreciate the URI. You can also use * to match strings in the same.way you can with#.
I think it's mostly a matter of taste. IMHO, putting the id in the Uri is a little cleaner since you can make the id opaque rather than require the client to know that it actually represents a specific row id. For instance, you can pass a lookup key (like in the the Contacts API) rather than a specific row id.