I have been trying to wrap my brain around authentication on a REST API.
I've tried to think of a way to successfully authenticate users, keeping in mind that users can access all data on the client, and I've come up with this idea.
Client sends username and password to the server
Server checks if they match a user.
If it does, we create a hashed string with user_id+e-mail+currentTime+salt
and stores this in a database-table with an expiration date.
Server returns hashed string to client
Client sends random request to server including key
Server checks if key is correct and if it's expired
Is this a proper way to do it, and do you see any security flaws?
You're effectively storing session state on the server, which is something you shouldn't be doing on a RESTful API.
Authentication on a RESTful API should simply follow whatever is the standardized authentication method for the underlying protocol. Instead of reinventing HTTP authentication, you should simply require clients to authenticate through HTTP Basic Auth on every request, using the Authorization header. Obviously, all your client-server interactions should be done over SSL.
If you really need some authentication token with an expire date, you can have a resource that provides it once the client is authenticated with basic (like a signed timestamp) but clients should still send that in the Authorization header, with a custom realm, and no state should be stored on the server.
Related
I am developing two linux programs, a CLI client and a server communicating via gRPC, and I now would like to authenticate users against a given private authorization server such as LDAP, Active Directory, etc.
I am confused regarding the various possible authentication flows. I think I can't use any classical flow including HTTP redirects since I shouldn't rely on a browser being installed or having internet access. I can't even define an endpoint I could redirect to (servers don't have internet access, and both are behind NATs).
So I was thinking of trying to store user's credentials as a JWT token file in the user's computer and then load it from my CLI client program to include it in my RPC requests and then validate it on the server-side. But, supposing I'm right, then what would be the best standard way of getting this token file?
If you had a browser you could use OAuth and the 'oob' (out of band) method where the CLI opens the browser and after the user authenticates it displays a number which the user copy/pastes into the CLI. This how my flickr backup CLI works. The number they copy/paste is because the CLI has no OAuth endpoint and the number is their access token to allow me to call the flickr api on their behalf.
If you can't use a browser the CLI can just accept a username/password from the user, send it to the server and receive a token in return. You don't really need anything fancy like JWT. A simple UUID would be enough. The UUID 'asserts' that the user is allowed to access the server's other RPC methods. The server would validate the UUID token to make sure it's still valid. If you need user information from the token, the server could do that. Keeps the user information off the client's disk and only the CLI can access that information, if the token is still valid.
So in effect, you need a new server RPC method, perhaps, authenticate, that accepts a username and password and returns a UUID token. All other RPC methods then need to accept that token and validate it before performing the requested function. As part of the server-side authentication process, the server could associate that token with the user information it got from the LDAP server so you don't need to store that information on the client. Lets you encrypt it on the server too and if the client needs it, it asks for it using the UUID token if it's still valid (time to live?). If it's no longer valid, the client just needs to ask for username/password again and the server can re-authenticate the user via LDAP and refresh the token and user information.
gRPC has authentication protocols but the SSL/TLS doesn't seem to match your needs and the OAuth won't work as you don't have a browser. So perhaps rolling your own simple token service (authenticate) combined with LDAP authentication might be a workable option.
Just found out that the basic workflow for token-based authentication is as follows:
User requests access by providing username and password
The application validates the credentials and returns a token to the client
The token is then stored on the client and sent with every request henceforth
The server then validates the token and returns private data as a response
Now, I understand the flow more or less, however, I'm having issues with the terms application, client and server. I understand the term server to mean where the API is stored... which is also part of the application. But the application could also be anything from a web app to a mobile app on various platforms... a client in other words.
So isn't it true that the application includes both the server and the client. So what does it mean by each term exactly, in the above context?
On second thoughts... I guess the original token is being generated on the server side, and this is then being returned to the client. Is this true?
Those terms terms are pretty overloaded in software development, so it's always difficult to nail down the exact meaning without focusing in a very specific context. Bear in mind that even authentication can be seen as a very broad context.
I would rephrase your proposed workflow to the following:
User requests access by providing a set of user credentials (we don't have to use passwords all the time, see passwordless authentication out of curiosity).
The authorization server validates the user identity and, if valid, issues an access token.
The client application from which the user started the process receives and stores the issued access token.
The client application calls into a resource server using the access token in order to obtain user associated resources.
Damn, now we have even more terms, but let's try to fix that by providing some definitions.
First, the ones more generic:
Client: An application that obtains information from a server for local use.
Credentials: Usernames, passwords, email addresses—any of a variety of means for communicating parties to generate or obtain security tokens.
(source: Auth0 Identity Glossary)
Then definitions within the context of OAuth 2.0 and/or OpenID Connect:
Authorization server: The server issuing access tokens to the client after successfully authenticating the resource owner and obtaining authorization.
Resource server: The server hosting the protected resources, capable of accepting and responding to protected resource requests using access tokens.
Client: An application making protected resource requests on behalf of the resource owner and with its authorization. The term "client" does not imply any particular implementation characteristics (e.g., whether the application executes on a server, a desktop, or other devices).
(source: RFC 6749)
It's not even useful trying to define application, but what we should conclude from the other definitions is the following:
As you said, a client can range from web, to mobile to event server-side applications.
The role of the authorization server and resource server can be played by the same component, for example, a Web API that has an endpoint protected by HTTP basic authentication that can be used to exchange a pair of username/password credentials with an access token and then all the remaining API endpoints are protected in such way that only allow access if you provide that access token.
Finally, one final note to clarify your last question, yes, the creation of the access tokens need to happen on the server side because the creation of the token will be accompanied by some kind of mechanism that will ensure that the token cannot be tampered with and was in fact created by a very well-know entity. For the case of JWT's this mechanism consists of signing the token which is accomplished by having the server know a secret that no one else knows.
I want to design a web application which guarantees secure authentication and gives API access only to the authorised users. The basic idea is simply sending username and password to get the user authenticated. And user can make request to server with the session_id without authenticating himself again.
Definitely, it is very insecure. But as far as I could understand now, in order not to expose the user's credentials, we can apply TLS(https) to get it encrypted.
However, as I research around, I get acquainted with a lot of concepts, like Base64, HMAC_SHA1, API keys, OAuth1.0. But I could not understand why do we need those mechanism other than TLS. Can anyone help explain why TSL is not enough to ensure authentication and API access to be secure?
Secure sessions work fine if your web application authenticates the user, issues the session id and validates the id on each call. You can store the session id in a secure cookie that gets sent back on each request.
Things get more complicated when you have your API on a different domain. Now your cookies are not automatically sent to the service (same-origin policy). You could of course stick the session id in an Authorization header when you call your API. But now your API needs to talk to the same backend store maintaining your session state to verify the authorization. This backend store becomes a bottleneck in scalability and a single point of failure.
To get around this, modern protocols (like OAuth2) issue security tokens. These tokens are digitally signed (using HMAC) and the receiver trusts the token if the signature is validated successfully. No backend calls are needed to validate the token, just a simple cryptographic operation.
API keys are used to allow applications to get a security token without relying on a user to authenticate. Think of them as a password for an application.
Using security tokens also allows you to use a 3rd party authorization server (like Facebook or Google etc) and completely get out of the business of authenticating users, storing passwords, issuing tokens etc.
I'm having some trouble deciding how to implement authentication for a RESTful API that will be secure for consumption by both a web app and a mobile app.
Firstly, I thought to investigate HTTP Basic Authentication over HTTPS as an option. It would work well for a mobile app, where the username and password could be stored in the OS keychain securely and couldn't be intercepted in transit since the request would be over HTTPS. It's also elegant for the API since it'll be completely stateless. The problem with this is for the web app. There won't be access to such a keychain for storing the username and password, so I would need to use a cookie or localStorage, but then I'm storing the user's private details in a readily accessible place.
After more research, I found a lot of talk about HMAC authentication. The problem I see with this approach is there needs to be a shared secret that only the client and server knows. How can I get this per-user secret to a particular user in the web app, unless I have an api/login endpoint which takes username/password and gives the secret back to store in a cookie? to use in future requests. This is introducing state to the API however.
To throw another spanner into the works, I'd like to be able to restrict the API to certain applications (or, to be able to block certain apps from using the API). I can't see how this would be possible with the web app being completely public.
I don't really want to implement OAuth. It's probably overkill for my needs.
I feel as though I might not be understanding HMAC fully, so I'd welcome an explanation and how I could implement it securely with a web app and a mobile app.
Update
I ended up using HTTP Basic Auth, however instead of providing the actual username and password every request, an endpoint was implemented to exchange the username and password for an access key which is then provided for every authenticated request. Eliminates the problem of storing the username and password in the browser, but of course you could still fish out the token if you had access to the machine and use it. In hindsight, I would probably have looked at OAuth further, but it's pretty complicated for beginners.
You should use OAuth2. Here is how:
1) Mobile App
The mobile app store client credentials as you state yourself. It then uses "Resource Owner Password Credentials Grant" (see https://www.rfc-editor.org/rfc/rfc6749#section-4.3) to send those credentials. In turn it gets a (bearer) token it can use in the following requests.
2) Web site
The website uses "Authorization Code Grant" (see https://www.rfc-editor.org/rfc/rfc6749#section-4.1):
Website sees unauthorized request and redirects browser to HTML-enabled autorization endpoint in the REST api.
User authenticates with REST service
REST site redirects user back to website with access token in URL.
Website calls REST site and swaps access token to authorization token.
Here after the website uses the authorization token for accessing the REST service (on behalf of the end-user) - usually by including the token as a "bearer" token in the HTTP Authorization header.
It is not rocket science but it does take some time to understand completely.
3) Restricting API access for certain applications
In OAuth2 each client is issued a client ID and client secret (here "client" is your mobile app or website). The client must send these credentials when authorizing. Your REST service can use this to validate the calling client
I resolved this for my own API quite easily and securely without the need to expose any client credentials.
I also split the problem into 2 parts. API authentication - is this a valid request from a recognised entity (website or native app). API authorisation, is that entity allowed to use this particular endpoint and HTTP verb.
Authorisation is coded into the API using an access control list and user permissions and settings that are set up within the API code, configuration and database as required. A simple if statement in the API can test for authorisation and return the appropriate response (not authorised or the results of processing the API call).
Authentication is now just about checking to see if the call is genuine. To do this I issue self signed certificates to clients. A call to the API is made from their server whenever they want - typically when they generate their first page (or when they are performing their own app login checks). This call uses the certificates I have previously provided. If on my side I am happy the certificate is valid I can return a nonce and a time limited generated API key. This key is used in all subsequent calls to other API endpoints, in the bearer header for example, and it can be stored quite openly in an HTML form field or javascript variable or a variable within an app.
The nonce will prevent replay attacks and the API key can be stolen if someone wants - they will not be able to continue using after it expires or if the nonce changes before they make the next call.
Each API response will contain the next nonce of if the nonce doesn't match it will return an authentication error. In fact of the nonce doesn't match I kill the API key too. This will then force a genuine API user to reauthenticate using the certificates.
As long as the end user keeps those certificates safe and doesn't expose the method they use to make the initial authentication call (like making it an ajax request that can be replayed) then the API's are nice and secure.
One way of addressing the issue of user authentication to the API is by requesting an authentication token from the API when the user logs in. This token can then be used for subsequent requests. You've already touched on this approach - it's pretty sound.
With respect to restricting certain web apps. You'll want to have each web app identify itself with each request and have this authentication carried out inside your API implementation. Pretty straight forward.
Correct me if I am wrong: In a traditional web application, the browser automatically appends session information into a request to the server, so the server can know who the request comes from. What exactly is appended actually?
However, in a API based app, this information is not sent automatically, so when developing an API, I must check myself if the request comes from an authenticated user for example? How is this normally done?
HTTP Protocol is stateless by design, each request is done separately and is executed in a separate context.
The idea behind session management is to put requests from the same client in the same context. This is done by issuing an identifier by the server and sending it to the client, then the client would save this identifier and resend it in subsequent requests so the server can identify it.
Cookies
In a typical browser-server case; the browser manages a list of key/value pairs, known as cookies, for each domain:
Cookies can be managed by the server (created/modified/deleted) using the Set-Cookie HTTP response header.
Cookies can be accessed by the server (read) by parsing the Cookie HTTP request header.
Web-targeted programming languages/frameworks provide functions to deal with cookies on a higher level, for example, PHP provides setcookie/$_COOKIE to write/read cookies.
Sessions
Back to sessions, In a typical browser-server case (again), server-side session management takes advantage of client-side cookie management. PHP's session management sets a session id cookie and use it to identify subsequent requests.
Web applications API?
Now back to your question; since you'd be the one responsible for designing the API and documenting it, the implementation would be your decision. You basically have to
give the client an identifier, be it via a Set-Cookie HTTP response header, inside the response body (XML/JSON auth response).
have a mechanism to maintain identifier/client association. for example a database table that associates identifier 00112233445566778899aabbccddeeff with client/user #1337.
have the client resend the identifier sent to it at (1.) in all subsequent requests, be it in an HTTP Cookie request header, a ?sid=00112233445566778899aabbccddeeff param(*).
lookup the received identifier, using the mechanism at (2.), check if a valid authentication, and is authorized to do requested operation, and then proceed with the operation on behalf on the auth'd user.
Of course you can build upon existing infrastructure, you can use PHP's session management (that would take care of 1./2. and the authentication part of 4.) in your app, and require that client-side implementation do cookie management(that would take care of 3.), and then you do the rest of your app logic upon that.
(*) Each approach has cons and pros, for example, using a GET request param is easier to implement, but may have security implications, since GET requests are logged. You should use https for critical (all?) applications.
The session management is server responsibility. When session is created, a session token is generated and sent to the client (and stored in a cookie). After that, in the next requests between client and server, the client sends the token (usually) as an HTTP cookie. All session data is stored on the server, the client only stores the token. For example, to start a session in PHP you just need to:
session_start(); // Will create a cookie named PHPSESSID with the session token
After the session is created you can save data on it. For example, if you want to keep a user logged:
// If username and password match, you can just save the user id on the session
$_SESSION['userID'] = 123;
Now you are able to check whether a user is authenticated or not:
if ($_SESSION['userID'])
echo 'user is authenticated';
else
echo 'user isn't authenticated';
If you want, you can create a session only for an authenticated user:
if (verifyAccountInformation($user,$pass)){ // Check user credentials
// Will create a cookie named PHPSESSID with the session token
session_start();
$_SESSION['userID'] = 123;
}
There are numerous way for authentic users, both for Web applications and APIs. There are couple of standards, or you can write your own custom authorization / and or authentication. I would like to point out difference between authorization and authentication. First, application needs to authenticate user(or api client) that request is coming from. Once user has been authenticated, based on user's identity application needs to determine whatever authenticated user has permission to perform certain application (authorization). For the most of traditional web applications, there is no fine granularity in security model, so once the user is authenticated, it's in most cases also and authorized to perform certain action. However, this two concepts (authentication and authorization) should be as two different logical operations.
Further more, in classical web applications, after user has been authenticated and authorized
(mostly by looking up username/password pair in database), authorization and identity info is written in session storage. Session storage does not have to be server side, as most of the answers above suggest, it could also be stored in cookie on client side, encrypted in most cases. For an example, PHP CodeIgniter framework does this by default. There is number of mechanism for protecting session on client side, and I don't see this way of storing session data any less secure than storing sessionId, which is then looked up in session storage on server-side. Also, storing session client-side is quite convenient in distributed environment, because it eliminates need for designing solution (or using already existing one) for central session management on server side.
Further more, authenticating with simple user-password pair does not have to be in all case done trough custom code which looks up matching user-record in database. There is, for example basic authentication protocol , or digest authentication. On proprietary software like Windows platform, there are also ways of authenticating user trough, for an example,ActiveDirectory
Providing username/password pair is not only way to authenticate, if using HTTPS protocol, you can also consider authentication using digital certificates.
In specific use case, if designing web service, which uses SOAP as protocol, there is also WS-Security extension for SOAP protocol.
With all these said, I would say that answers to following question enter decision procedure for choice of authorization/authentication mechanism for WebApi:
1) What's the targeted audience, is it publicly available, or for registered(paying) members only?
2) Is it run or *NIX, or MS platform
3) What number of users is expected
4) How much sensitive data API deals with (stronger vs weaker authentication mechanisms)
5) Is there any SSO service that you could use
.. and many more.
Hope that this clears things bit, as there are many variables in equation.
If the API based APP is a Client, then the API must have option to retrieve/read the cookies from server response stream and store it. For automatic appending of cookies while preparing request object for same server/url. If it is not available, session id cannot be retrieved.
You are right, well the reason things are 'automatic' in a standard environment is because cookies are preferred over URL propagation to keep things pretty for the users. That said, the browser (client software) manages storing and sending the session cookie along with every request.
In the API world, simple systems often just have authentication credentials passed along with every request (at least in my line of work). Client authors are typically (again in my experience) reluctant to implement cookie storage, and transmission with every request and generally anything more than the bare minimum...
There are plenty of other authentication mechanisms out there for HTTP-based APIs, HTTP basic / digest to name a couple, and of course the ubiquitous o-auth which is designed specifically for these things if I'm not mistaken. No cookies are maintained, credentials are part of every exchange (fairly sure on that).
The other thing to consider is what you're going to do w/ the session on the server in an API. The session on a website provides storage for the current user, and typically stores small amounts of data to take load off the db from page to page. In an API context this is less of a need as things are more-or-less stateless, speaking generally of course; it really depends what the service is doing.
I would suggest you send some kind of token with each request.
Dependent on the server and service those can be a JSESSIONID parameter in your GET/POST request or something mature like SAML in SOAP over HTTP in your Web Service request.