I am developing a web application (a dashboard of some sort), and on some pages I have a couple of data tables. Each data table has its own page number, filter and sort. Although the data is fetched asynchronously in the background (hence the page does not need to reload), but I need to store these information (page, filter, sort) so it can be persistent for example during a page refresh. As far as I know, there are two ways to store these information:
In the session storage (local storage is not an option, because I don't want these information to be persistent between sessions)
In the query string
Up until now I used to store these information in the query string because I thought it has some advantages: e.g. the user can copy, bookmark, or share the URL with a colleague to discuss some data which they find interesting (for example on the 10th page of a table after performing some filtering and sorting). With the session storage I cannot do this
But now I am extending the abilities of the filter function, and now the information is quite long to use in the query string, and may even exceed the limit (I think 2048 characters, right?). And also there may be more than a couple of tables on each page, and therefore the query string would even become longer.
So first I wanted to know what is the best practice in this situation
And second, is that feature (being able to copy/bookmark/share the page as is) really that important, or not?
Note: please note that the information that I'm talking about is nothing secret or sensitive. It's just table page number, table filter and table sort
The 2048 limit seems that's not really an actual limit, see What is the maximum possible length of a query string?
About the best approach - I've personally always hated that most "new websites" do not support the feature you care about, so I'd personally encourage you to support it!
And finally about the exact mechanism, firstly the query string approach will keep working for a lot longer than 2048 characters, but I can see that copy&pasting it might be unwieldy and depending on the media to share the URL it can introduce mistakes.
So, from the user's perspective, I think the best experience would be given by storing those searches on the backend side and enabling a shorter URL for permalinking/sharing/bookmarking.
This new URL could be obtained by the user via a specific UI button (Share/Permalink) so you save the search in that moment and return the URL, or (best experience but harder and costlier to implement) you can be saving it continuously and sending back to the UI the generated URL and use Javascript to replace the URL for the nicer version (either always or just when people copy it).
Also consider: it may be good enough to just keep using the query string :-)
Related
I have an API with endpoint GET /users/{id} which returns a User object. The User object can contain sensitive fields such as cardLast4, cardBrand, etc.
{
firstName: ...,
lastName: ...,
cardLast4: ...,
cardBrand: ...
}
If the user calls that endpoint with their own ID, all fields should be visible. However, if it is someone elses ID then cardLast4 and cardBrand should be hidden.
I want to know what are the best practices here for designing my response. I see three options:
Option 1. Two DTOs, one with all fields and one without the hidden fields:
// OtherUserDTO
{
firstName: ...,
lastName: ..., // cardLast4 and cardBrand hidden
}
I can see this becoming out of hand with DTOs based on role, what if now I have UserDTOForAdminRole, UserDTOForAccountingRole, etc... It looks like it quickly gets out of hand with the number of potential DTOs.
Option 2. One response object being the User, but null out the values that the user should not be able to see.
{
firstName: ...,
lastName: ...,
cardLast4: null, // hidden
cardBrand: null // hidden
}
Option 3. Create another endpoint such as /payment-methods?userId={userId} even though PaymentMethod is not an entity in my database. This will now require 2 api calls to get all the data. If the userId is not their own, it will return 403 forbidden.
{
cardLast4: ...,
cardBrand: ...
}
What are the best practices here?
You're gonna get different opinions about this, but I feel that doing a GET request on some endpoint, and getting a different shape of data depending on the authorization status can be confusing.
So I would be tempted, if it's reasonable to do this, to expose the privileged data via a secondary endpoint. Either by just exposing the private properties there, or by having 2 distinct endpoints, one with the unprivileged data and a second that repeats the data + the new private properties.
I tend to go for option 1 here, because an API endpoint is not just a means to get data. The URI is an identity, so I would want /users/123 to mean the same thing everywhere, and have a second /users/123/secret-properties
I have an API with endpoint GET /users/{id} which returns a User object.
In general, it may help to reframe your thinking -- resources in REST are generalizations of documents (think "web pages"), not generalizations of objects. "HTTP is an application protocol whose application domain is the transfer of documents over a network" -- Jim Webber, 2011
If the user calls that endpoint with their own ID, all fields should be visible. However, if it is someone elses ID then cardLast4 and cardBrand should be hidden.
Big picture view: in HTTP, you've got a bit of tension between privacy (only show documents with sensitive information to people allowed access) and caching (save bandwidth and server pressure by using copies of documents to satisfy more than one request).
Cache is an important architectural constraint in the REST architectural style; that's the bit that puts the "web scale" in the world wide web.
OK, good news first -- HTTP has special rules for caching web requests with Authorization headers. Unless you deliberately opt-in to allowing the responses to be re-used, you don't have to worry the caching.
Treating the two different views as two different documents, with different identifiers, makes almost everything easier -- the public documents are available to the public, the sensitive documents are locked down, operators looking at traffic in the log can distinguish the two different views because the logged identifier is different, and so on.
The thing that isn't easier: the case where someone is editing (POST/PUT/PATCH) one document and expecting to see the changes appear in the other. Cache-invalidation is one of the two hard problems in computer science. HTTP doesn't have a general purpose mechanism that allows the origin server to mark arbitrary documents for invalidation - successful unsafe requests will invalidate the effective-target-uri, the Location, the Content-Location, and that's it... and all three of those values have other important uses, making them more challenging to game.
Documents with different absolute-uri are different documents, and those documents, once copied from the origin server, can get out of sync.
This is the option I would normally choose - a client looking at cached copies of a document isn't seeing changes made by the server
OK, you decide that you don't like those trade offs. Can we do it with just one resource identifier? You immediately lose some clarity in your general purpose logs, but perhaps a bespoke logging system will get you past that.
You probably also have to dump public caching at this point. The only general purpose header that changes between the user allowed to look at the sensitive information and the user who isn't? That's the authorization header, and there's no "Vary" mechanism on authorization.
You've also got something of a challenge for the user who is making changes to the sensitive copy, but wants to now review the public copy (to make sure nothing leaked? or to make sure that the publicly visible changes took hold?)
There's no general purpose header for "show me the public version", so either you need to use a non standard header (which general purpose components will ignore), or you need to try standardizing something and then driving adoption by the implementors of general purpose components. It's doable (PATCH happened, after all) but it's a lot of work.
The other trick you can try is to play games with Content-Type and the Accept header -- perhaps clients use something normal for the public version (ex application/json), and a specialized type for the sensitive version (application/prs.example-sensitive+json).
That would allow the origin server to use the Vary header to indicate that the response is only suitable if the same accept headers are used.
Once again, general purpose components aren't going to know about your bespoke content-type, and are never going to ask for it.
The standardization route really isn't going to help you here, because the thing you really need is that clients discriminate between the two modes, where general purpose components today are trying to use that channel to advertise all of the standardized representations that they can handle.
I don't think this actually gets you anywhere that you can't fake more easily with a bespoke header.
REST leans heavily into the idea of using readily standardizable forms; if you think this is a general problem that could potentially apply to all resources in the world, then a header is the right way to go. So a reasonable approach would be to try a custom header, and get a bunch of experience with it, then try writing something up and getting everybody to buy in.
If you want something that just works with the out of the box web that we have today, use two different URI and move on to solving important problems.
We currently trying to design some REST api for our webservices. We have two resources, 'record' and 'expedition'. As far as we know, a record can be associated with multiple expeditions, and an expedition can be associated with one record (but not necessarily).
When we create an expedition, and we want to "attach" it to a record, we have come to two solutions :
POST /expeditions?recordId=xxx
POST /records/xxx/expeditions
and a POST /expeditions WS to create expeditions independently.
My colleague suggested the first approach, but I found the second the most usual way to do so. I have not found articles on the web presenting the first approach as a good or bad design.
So, which solution is the good one for you ? Which kind of consideration can help us to choose ?
Thank you.
Which kind of consideration can help us to choose ?
Think about cache-invalidation.
HTTP is about document transfer. We obtain information from the server by asking for a copy of a document; that request might be handled by the server itself, or it might be handled by a cache that has a valid copy of the document.
We send information to a server by proposing edits to documents - POST being the most common method used to do that (via HTML forms).
When an edit is successful, it follows that the previously cached copies of the document are out of date, and we would really prefer that they be replaced by the updated copy.
General purpose cache invalidation is kind of limited; in particular, it doesn't support arbitrary invalidation of documents. Only the target-uri, Location, and Content-Location are invalidated.
Therefore, when we are designing our resource interactions, we want to consider this limitation.
That usually means that the request that we use to change a document should have the same target-uri as the request to read that same document.
(Yes, that means that if we are going to have different kinds of edits to the document, all of the different edits share the same target-uri, and we disambiguate the edit by looking at other parts of the request -- for instance by parsing the body.)
POST /records/xxx/expeditions and a POST /expeditions WS to create expeditions independently.
That's not required - the server is permitted to apply changes to more than one document; HTTP constrains the meaning of the request, but does not constrain the effects.
That said, general purpose caches won't magically know that both documents have been edited. To some degree, part of what you are choosing in your design is which document needs to be refreshed now, and which ones can be out of date for a time (typically until the cached representation reaches its max age).
For the special case where your response to the successful edit is going to be a copy of the updated representation of the resource, you have a little bit more freedom, because use can use the Content-Location header to identify which document we are returning in the response, and that header is automatically invalidated.
POST /foo/bar
...
200 OK
Content-Location: /foo
In this sequence, general purpose headers will invalidated their cached copies of both /foo and /foo/bar.
(of course, there are still issues, in so far as we don't have a mechanism to return both the updated copy of /foo and the updated copy of /bar in a single response. So instead we need to look into other ideas, like server push).
Design the URL paths in a way that the resources can easily be retrieved.
Query string/parameter present in the URL mentioned in the first approach is typically used to locate a resource and perhaps a little counter intuitive to me.
The second approach, perhaps this would work as you are creating an expedition under an associated record xxx i.e. /records/xxx/expeditions. But it could get challenging in a scenario where an expedition is not related to any record.
Another alternative thought here is to link the expedition and record through the payload i.e. have the record id XXX within the POST payload during the "expedition" resource creation. POST /expedition => This operation would return you an expedition id in response as the resource newly gets created. To retrieve the data, you could then use GET /expedition/XXX/record where XXX is the expedition id and you retrieve the record corresponding to XXX. You don't need to mention a record id in this case. You either get a associated record or you don't(in case there is no record tied to the expedition). To retrieve the expedition itself, the URL could be GET /expedition/XXX.
I made a program that gets the data from the clipboard and saves it in a string variable. Then it looks for specific words in that string and generates several URLs. Afterwards it open the browser and shows each URL in an own tab.
Some of my friends already use this program frequently and I want to have some statistics about how often. I simple counter variable would be enough but I need to get access to it.
I came up with two options that could work:
I could send an email to a specific adress every time my app is executed. Then I can track the amount of uses by manually or automaticly counting the amount of emails in the postbox. I think this would be a Vers dirty solution.
I could create and publish a website containing a counter. This counter could be refreshed by my application. This solution is a bit better I think but a lot more work for just one single counter.
Do you have better ideas to solve my problem or is one of mine already a good one?
Thank you in advace!
You can use Measurement Protocol Overview. This provides you statistics of usage your application compared with Google Analytics. You can see even a geo statistic, version distribution, crash reports. It is easy to use it from .net. It is just about requesting http request to google.
I'm a new REST convert and I'm trying to design my first RESTful (hopefully) api and here is my question about addressing resources
Some notes first:
The data described here are 3d render
jobs
A user (graphics company) has multiple projects.
A project has multiple render jobs.
A render job has multiple frames.
There is a hierarchy enforced in the data (1 render job
belongs to one project, to one user)
How's this for naming my resourses...?
https:/api.myrenderjobsite.com/
/users/graphicscompany/projects
/users/graphicscompany/projects/112233
/users/graphicscompany/projects/112233/renders/
/users/graphicscompany/projects/112233/renders/889900
/users/graphicscompany/projects/112233/renders/889900/frames/0004
OR a shortened address for renders?
/users/graphicscompany/renders/889900
/users/graphicscompany/renders/889900/frames/0004
OR should I shorten (even more) the address if possible, omitting the user when not needed...?
/projects/112233/
/renders/889900/
/renders/889900/frames/0004
THANK YOU!
Instead of thinking about your api in terms of URLs, try thinking of it more like pages and links
between those pages.
Consider the following:
Will it be reasonable to create a resource for users? Do you have 10, 20 or 50 users? Or do you have 10,000 users? If it is the latter then obviously creating a single resource that represents all users is probably not going too work to well when you do a GET on it.
Is the list of Users a reasonable root url? i.e. The entry point into your service. Should the list of projects that belong to a GraphicsCompany be a separate resource, or should it just be embedded into the Graphics Company resource? You can ask the same question of each of the 1-to-many relationships that exist. Even if you do decide to merge the list of projects into the GraphicsCompany resource, you may still want a distinct resource to exist simple for the purpose of being able to POST to it in order to create a new project for that company.
Using this approach you should be able get a good idea of most of the resources in your API and how they are connected without having to worry about what your URLs look like. In fact if you do the design right, then any client application you right will not need to know anything about the URLs that you create. The only part of the system that cares what the URL looks like is your server, so that it can dispatch the request to the right controller.
The other significant question you need to ask yourself is what media type are you going to use for these resources. How many different clients will need to access these resources? Are you writing the clients, or is someone else? Should you attempt to reuse an existing standard like XHTML and classes/microformats? Could you squeeze most of the information into Atom? Maybe Atom with some extra namespaces like GDATA does it? Or is this only going to be used internally so you can just create your own media types, like application/vnd.YourCompany.Project+xml, application/vnd.YourCompany.Render+xml, etc.
There are many things to think about when designing a REST api, don't get hung up on what your URLs look like and you should really try to avoid doing "design by URL".
Presuming that you authenticate to the service, I would use the 1st option, but remove the user, particularly if the user is the currently logged in user.
If user actually represents something else (like client), I would include it, but not if it simply designates the currently logged in user. Agree with StaxMan, though, don't worry too much about squeezing the paths, as readability is key in RESTful APIs.
Personally I would not try to squeeze path too much, that is, some amount of redundant information is helpful both to quickly see what resource is, and for future expansion.
Generally users won't be typing paths anyway, so verbosity is not all that bad.
Context:
I'm in the design phase of what I'm hoping will be a big website (lots of traffic, lots of users reading and writing to database).
I want to offer this website in the three languages I speak myself (English, French, and by the time I finish the website, I will hopefully have learned enough Spanish to offer that too)
Dilemma:
I'm wondering how I should go about offering these various languages (and perhaps more in the future).
Criteria:
Many methods exist for designing multi-language websites. I'm looking for the technique that will result in a faster browsing experience for the user.
Choices:
Currently, I can think of (and have read about) the following choices. They are sorted in order of preference up to now.
Store all language-specific strings
in a database and fetch the good one
depending on prefered-language
(members can choose which language
they prefer),
browser-default-language and which
language is selected during the
current session, in that order.
Pros:
Most of the time, a single
test at the beggining of the session
confirms which language to use for
the remainder of the session (stored
in a SESSION variable). Otherwise, a
user logging in also fetches the
right language and keeps it until
he/she logs out (no further tests). So the testing part should be
pretty fast.
Cons:
I'm afraid that accessing the
database all the time would be quite
time-consuming (longer page load for
the user), especially considering
that lots of users could also be
accessing the database at the same
time for the same reason (getting the website text in the correct language), but also
for posting comments and the such.
Strings which include variables
(e.g. "Hello " + user.name + ", how
are you?") are harder to
store because the variable (e.g.
user name) changes for each user.
A direct link to a portal for a specific language would be ugly (e.g. www.site.com?lang=es)
Store all language-specific strings
in a text file and fetch the good one
depending on prefered-language
(members can choose which language
they prefer),
browser-default-language and which
language is selected during the
current session, in that order.
Pros:
Most of the time, a single
test at the beggining of the session
confirms which language to use for
the remainder of the session (stored
in a SESSION variable). Otherwise, a
user logging in also fetches the
right language and keeps it until
he/she logs out (no further tests). So the testing part should be
pretty fast.
Cons:
I'm afraid that accessing the
text file all the time would be quite
time-consuming (longer page load for
the user), especially considering
that lots of users could also be
accessing the file at the same
time for the same reason (getting the website text in the correct language).
Strings which include variables
(e.g. "Hello " + user.name + ", how
are you?") are harder to
store because the variable (e.g.
user name) changes for each user.
I don't think multiple users could access the text file concurrently, though I may be wrong. If that's the case though, every user loading a page would have to wait for his/her turn to access the text file.
Fetching the very last string of the text file could be pretty long...
A direct link to a portal for a specific language would be ugly (e.g. www.site.com?lang=es)
Creating multiple versions of the website in seperate folders, where each version is in a different language.
Pros:
No extra-treatment is needed for handling languages, so no extra waiting time.
Cons:
Maintaining the website will be like going to school: painfull, long, makes you stupid after doing the same thing over and over again.
ugly url (e.g. www.site.com/es/ instead of www.site.com)
Additionnaly, the coices above could be combined with one or more of the following techniques:
Caching certain frequently requested pages (in a singleton or static PHP function?). Certain sentences could also be cached for every language.
Pros
Quicker access for frequently-requested pages.
Which pages need caching can be determined dynamically, with time.
Cons
I'm not sure about this one, but would this end up bloating the server's RAM?
Rewritting the url could be used for many things.
A user looking for direct access to one language could do so using www.site.com/fr/somefile and would be redirected to www.site.com/somefile, but with the language selected beign stored in a session variable.
Pros
Search engines like this because they have two different pages to show for two different languages
Cons
Bookmarking a page doesn't mean you'll en up with the right language when you come back, unless I put the language information in the url (www.site.com/somefile?lang=fr)
A little more info
I usually user the following technologies to make a website:
PHP
SQL
XHTML
CSS
Javascript (and AJAX)
This being said, if a solution requires that I learn a new language or something, I'm very open to doing so. I have no deadline for this project and I do intend to learn a lot from doing it!
Conclusion:
What I'm looking for is a method that allows me to offer multiple languages while not increasing page load time and not going crazy when trying to maintain the website. If you guys/gals have other ideas I should consider, I will try adding them to my list. Another possibility is that I'm overdoing this. Maybe I won't gain enough time with these methods for this all to be worth it, I just don't know how to verify if I need to worry about this or not.. so if you have any ideas for that, it would also help me.
Whether you use a database or a filesystem to store the translations, you should be loading the text all at once and then serving it from memory. Most applications will typically not have so much text that this becomes a problem. In Java or .Net this could be accomplished by storing the text in a singleton or static object. Then all the strings are in RAM and do not need to be loaded or parsed. If your platform does not have a convenient way to store data in ram, you could run a separate caching application such as memcached.
The rest of your concerns can be mitigated by hiding the details. Build or find a framework that lets you load your translations and then look them up by some key. If you decide to switch to files or a database later, the rest of your code is unaffected. In the short term do whichever is easier for you. I've found that it's best to have a mix: it's easier to manage application text along with the source code in a version control system. But some text changes often, or needs to change without requiring a build+deployment cycle, and that text should be in the DB.
Finally, don't build strings with substitutions in them. Use some kind of format string, because otherwise your translators will go crazy trying to translate sentence fragments.
(Warning: Java code sample)
//WRONG
String msg = "Hello, " + username + ", welcome back.";
//RIGHT
String fmt = "Hello, %s, welcome back."; // in real code: load this string from a file or the db
String msg = fmt.format(username);
Another person mentioned encoding the language in the URL. This is the preferred way to do it if you care what a search engine thinks of your site. Google recommends using different hostnames or a different subdirectory. This means that the language headers sent by the user can't be used for anything, except perhaps initially sending them to one landing page or another. You will need to determine the language for each request based on the incoming URL (this actually simplifies your code a lot later on). In Java I'd store the language code in the Request and just grab it whenever I need it.
The easiest way to handle language codes in the URL is to use re-writing. A client sends a request for www.yoursite.com/de/somepage and internally you re-write the request to www.yoursite.com/somepage and store the language identifier somewhere. In Java each request has an HttpServletRequest object where you can store attributes for the lifecycle of the request. If your framework doesn't have anything like that you can just add a parameter to the url: www.yoursite.com/de/somepage => www.yoursite.com/somepage?lang=de. If you are using hostname-based languages you can use hostnames such as de.yoursite.com or www.yoursite.de. There are pros and cons to using this approach. For one thing, using country-code TLDs means registering new TLDs and trying to figure out whether a country code is appropriate to represent a language (it's often not). Using differnet hostnames/domains means you have to consider under what domains cookies are stored. If you want a cookie-free subdomain you need to plan this carefully. But from the coding side a language-based hostname doesn't need any additional re-writing; you can read the hostname (it's the Host header in the HTTP request) and parse that to determine the language.
Offer the initial page in a language depending on the Accept-Language HTTP header.
Let the user set the language in the current session and, if they're authenticated, in their user profile.
In your code and templates, mark strings as "translatable." You should have tools that gather all the strings from your codebase and let your translaters translate them.
Have a layer which loads the translations from the database either individually or as a bundle, and apply them to the page which is loading. Cache these parts to make them fast -- every page load shouldn't make a hundred calls to the database for every translatable string.
Checkout how Django does it -- it should be enlightening.
"I'm afraid that accessing [the database/text file] all the time would be quite time-consuming"
It would be, but that's why you'd likely be using caching to some extent. Nearly all large sites are accessing data stored outside the HTML page itself and, as such, utilize caching techniques as needed.
Your question regarding speed really is irrelevant to having multiple languages. It's an issue of storing data (content) so it's easy to maintain and present to the user. Whether it's one language or 10 the problem is the same.
Create the most generic form of the site as you can. Import the translation from a database, with fall back (i.e. an order of languages, if a translation does not exist then use the next best langauge (For German: German, Dutch, English etc).
You would solve performance issues by keeping caches of the dynamically created pages. [Check the dependent data and update if necessary]
The perfered language that a user would like is passed along in the HTTP request headers. Having a select language+query string would often be unnecessary.
Resource files would be one way to go. It is easier to send to translators. However it can be difficult to resuse amongst multiple websites.
Databases are convient because it is the first thing that should be backed up on a website. It also has the benefit of being fast. However, if you have an extremely database focused project, you may not want to add additional strain on your database.
For my solutions I want this:
The language should be indicated in the URL, it works better with google indexing the page and people following the links in google's search result.
As much pre-generated translations as possible, for faster page-serving.
The first is quite easily done by having an URL like http://example.com/fr/and-so-on. URL rewriting can turn that into http://example.com/and-so-on?lang=fr which is potentially easier to handle.
For pre-generating translations, it is good to use a html template framework so you can generate translated templates from one set of source templates. A blunt approach is to generate a sed-script from a language key-value files, and run that sed script on each template to get a translated version.
What remains then is to translate the dynamically generated parts of the pages. There are a few tools for that java has bundles, gnu gettext is a quite nice tool.