After creating a basic REST service, I've have come to the point where it would be appropriate to add some sort of password protection, as I need to verify that my users are both properly logged and have sufficient permissions to execute whatever action they are going to.
The REST service will mainly be accessed from a Javascript-heavy frontend and with that in mind, I have come up with the two following alternatives to solve this:
Make users login by first sending credentials to a /login page with POST. The page sets a session cookie wherein the user is
marked as logged in, along with the permission level. On each
following request, I verify that the user is logged in and his/her
permission level. When the session expires, automatically or
manually (logout, the user will have to re-logon).
Temporarily save the credentials hashed locally and send the users credentials along every single request made by the user to verify the credentials & permissions backend on a per-request basis.
Are there more ways to solve this and is there something else that I should be concerned with?
I'm currently developing a REST API along with a client (written in javascript), below I'll try to explain the methods used to protect the API against unauthorized access.
Make your REST API to require a Auth-Key header upon every request to the API, besides /api/authenticate.
/api/authenticate will take a username and a password (sent using POST), and return user information along side with the Auth-Key.
This Auth-Key is randomly generated after a call to /api/authenticate and stored in the backend users table with the specific user entry, a md5 hash of the remote ip + the user agent provided by the client.
On every request the value of Auth-Key, and the md5 sum mentioned, is searched for in users . If a valid user is found that has been active during the past N minutes the user will be granted access, if not: http return code 401.
In the REST client, first get the Auth-Key by posting to /api/authenticate, then store this value in a variable and send in on every future request.
If you want to stay true to the definition of a REST service then it should be stateless and not store any login (or other context specific) data on the server: http://en.wikipedia.org/wiki/Representational_state_transfer#Constraints
Your 2nd approach would fit this model
First decide what it is that you're protecting against:
Authentication? (Knowing who is requesting your service?)
Authorization? (Whether a given person can properly request a given service or not?)
I recommend that you provide hashed keys for your service. That way you can manage the key issue separately from the services. Or a client key and a secret, Amazon does this.
It is always easier for the client if you have a stateless protocol. And send everything through the parameters, cookies are a bother for the client too.
Remember that it is in your interest to make it as easy as possible for potential developers to use your service. A super secure service that no one uses is boring.
You can let clients choose the security level by giving them the choice of HTTP or SSL/HTTP endpoints to connect to. Client choice is a good thing.
Make users login by first sending credentials to a /login page with POST. The page sets a session cookie wherein the user is marked
as logged in, along with the permission level. On each following
request, I verify that the user is logged in and his/her permission
level. When the session expires, automatically or manually (logout,
the user will have to re-logon).
Temporarily save the credentials hashed locally and send the users credentials along every single request made by the user to verify the
credentials & permissions backend on a per-request basis.
Your first approach does not meat the statelessness constraint of REST. You cannot maintain client sessions on server side. This constraint makes REST highly scalable...
Your second solution is appropriate. The simplest way to use HTTP basic auth. You don't have to hash the password on client side. What you need is an encrypted connection. On server side you can have an [username, password] -> [identity, permissions] cache, so this solution is much faster and superior in every other way than having server side sessions.
By 3rd party (non-trusted) clients the authentication is more complex, I guess you don't need that part.
I'm no security-expert. I use the RESTful Play!-webframework and they do the following things to authenticate users.
The cookie is protected against manipulation. It is signed with a long secret key and is checked for each request. Just hashing it is not enough!
They recommend to set unique information the identify the user in the cookie. As the server should be the only one to manipulate the cookie, that's enough.
Don't put the password as credential in the cookie. If someone sniffs the cookie, not only the session can be hijacked, but also the complete account or even worse, other accounts with the same credentials.
If you want to protect the cookie against hijacking using https.
Related
I have built a Web API and now I am trying to determine the best approach to secure it.
I would like to use tokens along with credentials and thus, once the user is validated, on future requests a token can be passed with the http request. This API will always be called by one particular account and the username/password will always remain the same.
I am working with an already existing site backend, which has its own login implemented and stores user data. So I would like to stay away from creating new database tables to store user records. For that reason, I think implementing .Net Identity is maybe a overkill.
One of the options I am thinking of is grabbing the credentials from the http request and attempting the SQL connection with it. If the connection passes, then the user is legit. If it does not, it means I have to return access denied. Is this a good way of going about it? If yes, what can I use for token generation and validation?
Check out this guide which is specific for Oauth tokens with .NET:
OAuth with JSON Web Tokens In .NET
Also, make sure to follow the guideliness, because tokens must expire and be renewed after a while, for security reasons. You shoudn't use a permanent token, of course.
I'm working on a cloud service authentication system and I'm not entirely sure what the optimal way to handle authenticating requests is. We're planning to run our image server as a separate process from our API server so that we can scale them independently of each other. Handling request authentication with API keys is fairly simple, because we can just have the image server store its own API key and check that requests provide it in a header (over HTTPS obviously), same with the API server. For users though it gets more complex.
Right now we have it setup so that the API server will handle generating a session token and storing users in its database, however what we'd like to do is use 3 servers:
authentication server
API server
image server
and have the image and API servers authenticate requests against the authentication server. How exactly should this be done though? It seems like a bad idea performance-wise to hit the authentication server for every request that the API and image servers make. Can/should a token be verified from a different server than it was created on?
So for example: can/should I pass the token received from the authentication server to the image server, verify that the token came from "my.auth.server" and check that the user ID is the right one? Would JWTs be a good type of token for this?
Here's an alternative approach.
Your authentication issues a JWT token that is signed using a secret that is also available in your API and server images. The reason they need to be there too is that you will need to verify the tokens received to make sure you created them. The nice thing about JWTs is that their payload can hold claims as to what the user is authorised to access should different users have different access control levels.
That architecture renders authentication stateless: No need to store any tokens in a database unless you would like to handle token blacklisting (think banning users). Being stateless is crucial if you ever need to scale. That also frees up your API and image servers from having to call the authentication server at all as all the information they need for both authentication and authorisation are in the issued token.
Flow (no refresh tokens):
User authenticates with the authentication server (eg: POST /auth/login) and receives a JWT token generated and signed by the auth server.
User uses that token to talk to your API and image servers and assuming user is authorised), gets and posts the necessary resources.
There are a couple of issues here. Namely, that auth token in the wrong hands provides unlimited access to a malicious user to pretend they are the affected user and call your APIs indefinitely. To handle that, tokens have an expiry date and clients are forced to request new tokens whenever expiry happens. That expiry is part of the token's payload. But if tokens are short-lived, do we require users to authenticate with their usernames and password every time? No. We do not want to ask a user for their password every 30min to an hour, and we do not want to persist that password anywhere in the client. To get around that issue, we introduce the concept of refresh tokens. They are longer lived tokens that serve one purpose: act as a user's password, authenticate them to get a new token. Downside is that with this architecture your authentication server needs to persist these refresh token in a database to make them revokable before they expire (think "revoked" column in tokens table).
New flow (with refresh tokens):
User authenticates with the authentication server (eg: POST /auth/login) and receives a JWT token generated and signed by the auth server, alongside a long lived (eg: 6 months) refresh token that they store securely
Whenever the user needs to make an API request, the token's expiry is checked. Assuming it has not yet expired, user uses that token to talk to your API and image servers and assuming user is authorised), gets and posts the necessary resources.
If the token has indeed expired, there is a need to refresh your token, user calls authentication server (EG: POST / auth/token) and passes the securely stored refresh token. Response is a new access token issued.
Use that new token to talk to your API image servers.
OPTIONAL (banning users)
How do we ban users? Using that model there is no easy way to do so. Enhancement: Every persisted refresh token includes a blacklisted field and only issue new tokens if the refresh token isn't black listed.
Things to consider:
You may want to rotate refresh token. To do so, blacklist the refresh token each time your user needs a new access token. That way refresh tokens can only be used once. Downside you will end up with a lot more refresh tokens but that can easily be solved with a job that clears blacklisted refresh tokens (eg: once a day)
You may want to consider setting a maximum number of allowed refresh tokens issued per user (say 10 or 20) as you issue a new one every time they login (with username and password). This number depends on your flow, how many clients a user may use (web, mobile, etc) and other factors.
You can store some additional metadata (ip, geolocation, device, browser cookie, etc.) alongside refresh tokens. That way, you can be smart about when to reject malicious usages of refresh tokens in case it's compromised.
Common Q: Why store all refresh tokens, and not just revoked ones? You could do that. Ask yourself the following: Will I, at any point, need to have a functionality where I can dynamically revoke valid refresh tokens, based on some arbitrary, regulatory, legal, integrity, security etc. criteria? If so, the least you will need is a list of all issued tokens, plus any data required to implement the criterion logic. Example: Due to regulation, I need to ban all EU users equates to a delete from refresh_tokens were user_ip in <... eu logic ...>
one of the best ways to use is a JWT Token , you can generate and share it between all your servers and validate it on the server side .
https://jwt.io
also I think the best architecture to use in this case is the micro service architecture
I'm working on an app which will use CouchDB to store some data for users. But I don't want users to log into CouchDB directly.
I will have an app client (mobile/web), an app server, and the CouchDB server. The client app will authenticate to the app server, then myy ideal scenario would be for my app server to authenticate the users programmatically, then send just the 10-minute cookie to the client app.
That is, I would like the app server to request a Cookie from the CouchDB server on behalf of the user of the app client, then send only the cookie to the app client.
The app server could just POST to _session on behalf of the authenticated user, but this requires:
maintaining a list of users' passwords in the app server
using a single, known, password for all users
resetting the password to something random for each authentication request
For security reasons, #3 seems the best, but this seems like extra work, and is an extra round-trip to the DB (albeit, not an expensive one). So my question is: Is there any, as an administrator, way to generate a cookie on behalf of a user, without using the users' password at all?
This would also potentially allow me to entirely reject requests to _session except from my app server, as an added security measure.
And for the sake of completeness, I'll also mention that I've looked at these other options, and found them wanting:
Proxy Auth
The fact that the x_auth_token never expires is worrisome to me. It means a compromised token would grant forever access to the user's data. And AFAICT, the token can't even be invalidated without changing the user name or the server secret (which would in effect invalidate everyone else's auth tokens as well). But maybe I'm missing something here?
OAuth auth
This seems to just move the problem. Now rather than storing users' passwords in my server app, I have to store OAuth secrets. Plus, now my server and client code must be more complicated.
I don't follow your exact goals. You seem to imply users might have passwords ("app server authenticating the users programmatically") but you don't want the users to "ever need to know their CouchDB password". What sort of authentication do you want?
There's two (and a half) general approaches I've taken to authentication with CouchDB:
"Man-in-the-middle[ware]" approach, where I have thin middleware in front of CouchDB. This middleware forwards username/password to the "/_session" which yields a cookie or error codes based on the CouchDB _users database. The middleware copies this cookie from CouchDB onto its own HTTP response back to the client (or displays a message in case of error). Then on subsequent requests, that need database access, it forwards the cookie (now from the client request) back again to the database.
The traditional approach, where you just use CouchDB as a data store and maintain your own "user" entries/indexes. Make sure you use current best practices for password storage/handling or use a library that takes care of those details for you. The middleware connects to the database as "itself" and handles read/write permissions with its own logic based on its own session handling.
Or — sort of a hybrid approach — you can use the "/_session" API only to see if CouchDB accepts the username+password as valid. If it does, create a separate middleware-handled session for that user. (Basically you're only using CouchDB's _user database as the "password handling library" and the rest is the traditional approach where the access control is implemented all in the middleware rather than at the database.)
For real-world production stuff, I've tended to use only the latter two (or one-and-a-half given the earlier numbering…) — the first method is kind of fun, but CouchDB's lack of document-level read permissions usually means that giving users nearly-direct access to the database server is untenable in practice.
UPDATE: your question now makes it clear that you want the client app to talk directly to both servers: the app (formerly "middleware") server and the CouchDB (database) server. I'm leaving the content above because I think it's still somewhat useful and provides a bit of background/context for this update.
You are right in your suspicions that Proxy Authentication is the wrong solution: it is not intended for end-user usage, but really to replace the cookie-forwarding "trick" portion of #1 above. That is, proxy authentication is when you fully trust one party (i.e. your middleware) to provide the user information as it works on behalf of a user. But you want the users to talk to the database directly, and you cannot trust them with the X-Auth-CouchDB-Token.
I will defer to your judgement on the OAuth option. I do think it is closer to want you want but it is clear that somehow you are authenticating users against a different service and don't need to store per-user keys in CouchDB itself. The request signing required by OAuth 1.0 does mean you'd need support in your client app's HTTP library too.
I see a few options, without building a custom CouchDB plugin, that could let your app server hand out a token to authenticated users which your database server will accept:
Proxy after all! That is, hide your database server behind your app server or another lightweight custom reverse-proxy. All this middleware needs to do is check your existing client app session (cookie or other authentication header) and if it's valid, set the internal proxy auth headers that CouchDB will accept — then it forwards the rest of the request/response verbatim.
Deterministic password, per-user if it makes you feel better. Configure your app server with a secret known only to it, then set each user password to something like HMAC(username, app_server_secret). Now when you want to generate a token for a user, your app server can generate the password on a per-user basis. Note that this really isn't any more secure than just using the app_server_secret as the password for every user — CouchDB already salts and hashes each user password independently so if someone gets a hold of the database but not your app's configuration values the attacker couldn't tell the two apart. In both cases, preventing unauthorized database usage hinges entirely on keeping app_server_secret secret.
Re-implement CouchDB's current cookie generation algorithm. CouchDB's cookie algorithm (view source) is basically data = username + ':' + timestamp; base64(data + ':' + sha_mac(data, secret)). Where secret is the couch_httpd_auth.secret value plus the user's salt value. You can tell your app server the couchdb_httpd_auth/secret value and it can follow the same steps to generate a valid cookie which you provide to the client app, and CouchDB will accept it as its own. This cookie will be valid until the timestamp + the configured couch_httpd_auth/timeout. As "hacky" as it seems, this is probably the closest to what you are asking for, although you still need to set/disable the users' actual passwords somehow.
Expanding on natevw's brilliant answer. I was having similar problems, and never would have realized option 3 was possible without having stumbled across that answer.
Here is my python3 implementation for generating a cookie (uses pycouchdb to interface with couch):
def generate_couchdb_cookie(couchAddress, couchSecret, username):
timestamp = format(int(time.time()), 'X')
data = username + ":" + timestamp
server = pycouchdb.Server(couchAddress)
db = server.database("_users")
doc = db.get("org.couchdb.user:" + username)
salt = doc["salt"]
secret = couchSecret + salt
hashed = hmac.new(secret.encode(), data.encode(), hashlib.sha1).digest()
inbytes = data.encode() + ":".encode() + hashed
result = base64.urlsafe_b64encode(inbytes)
return "AuthSession=" + (result.decode("utf-8")).rstrip('=')
What would be a step-by-step description of how cookie-based authentication work?
I've never done anything involving either authentication or cookies. What does the browser need to do? What does the server need to do? In what order? How do we keep things secure?
I've been reading about different types of authentication and about cookies, but I would like a basic description of how to use the two together. I've only read that they are often used together, but I could not find a description of how.
To expand on Conor's answer and add a little bit more to the discussion...
Can someone give me a step by step description of how cookie based authentication works? I've never done anything involving either authentication or cookies. What does the browser need to do? What does the server need to do? In what order? How do we keep things secure?
Step 1: Client > Signing up
Before anything else, the user has to sign up. The client posts a HTTP request to the server containing his/her username and password.
Step 2: Server > Handling sign up
The server receives this request and hashes the password before storing the username and password in your database. This way, if someone gains access to your database they won't see your users' actual passwords.
Step 3: Client > User login
Now your user logs in. He/she provides their username/password and again, this is posted as a HTTP request to the server.
Step 4: Server > Validating login
The server looks up the username in the database, hashes the supplied login password, and compares it to the previously hashed password in the database. If it doesn't check out, we may deny them access by sending a 401 status code and ending the request.
Step 5: Server > Generating access token
If everything checks out, we're going to create an access token, which uniquely identifies the user's session. Still in the server, we do two things with the access token:
Store it in the database associated with that user
Attach it to a response cookie to be returned to the client. Be sure to set an expiration date/time to limit the user's session
Henceforth, the cookies will be attached to every request (and response) made between the client and server.
Step 6: Client > Making page requests
Back on the client side, we are now logged in. Every time the client makes a request for a page that requires authorization (i.e. they need to be logged in), the server obtains the access token from the cookie and checks it against the one in the database associated with that user. If it checks out, access is granted.
This should get you started. Be sure to clear the cookies upon logout!
A cookie is basically just an item in a dictionary. Each item has a key and a value. For authentication, the key could be something like 'username' and the value would be the username. Each time you make a request to a website, your browser will include the cookies in the request, and the host server will check the cookies. So authentication can be done automatically like that.
To set a cookie, you just have to add it to the response the server sends back after requests. The browser will then add the cookie upon receiving the response.
There are different options you can configure for the cookie server side, like expiration times or encryption. An encrypted cookie is often referred to as a signed cookie. Basically the server encrypts the key and value in the dictionary item, so only the server can make use of the information. So then cookie would be secure.
A browser will save the cookies set by the server. In the HTTP header of every request the browser makes to that server, it will add the cookies. It will only add cookies for the domains that set them. Example.com can set a cookie and also add options in the HTTP header for the browsers to send the cookie back to subdomains, like sub.example.com. It would be unacceptable for a browser to ever sends cookies to a different domain.
Cookie-Based Authentication
Cookie-based authentication normally works in these four steps:
The user provides a username and password in the login form and the client/browser sends a login request.
After the request is made, the server validates the user on the backend by querying the database. If the request is valid, it will create a session by using the user information fetched from the database and store them. For each session a unique ID called the session ID is created. By default, the session ID will be given to the client through the browser.
The browser will submit this session ID on each subsequent request. The session ID is verified against the database. Based on this session ID, the server will identify the session belonging to which client and then give the request access.
Once a user logs out of the app, the session is destroyed both client-side and server-side.
Correct me if I am wrong: In a traditional web application, the browser automatically appends session information into a request to the server, so the server can know who the request comes from. What exactly is appended actually?
However, in a API based app, this information is not sent automatically, so when developing an API, I must check myself if the request comes from an authenticated user for example? How is this normally done?
HTTP Protocol is stateless by design, each request is done separately and is executed in a separate context.
The idea behind session management is to put requests from the same client in the same context. This is done by issuing an identifier by the server and sending it to the client, then the client would save this identifier and resend it in subsequent requests so the server can identify it.
Cookies
In a typical browser-server case; the browser manages a list of key/value pairs, known as cookies, for each domain:
Cookies can be managed by the server (created/modified/deleted) using the Set-Cookie HTTP response header.
Cookies can be accessed by the server (read) by parsing the Cookie HTTP request header.
Web-targeted programming languages/frameworks provide functions to deal with cookies on a higher level, for example, PHP provides setcookie/$_COOKIE to write/read cookies.
Sessions
Back to sessions, In a typical browser-server case (again), server-side session management takes advantage of client-side cookie management. PHP's session management sets a session id cookie and use it to identify subsequent requests.
Web applications API?
Now back to your question; since you'd be the one responsible for designing the API and documenting it, the implementation would be your decision. You basically have to
give the client an identifier, be it via a Set-Cookie HTTP response header, inside the response body (XML/JSON auth response).
have a mechanism to maintain identifier/client association. for example a database table that associates identifier 00112233445566778899aabbccddeeff with client/user #1337.
have the client resend the identifier sent to it at (1.) in all subsequent requests, be it in an HTTP Cookie request header, a ?sid=00112233445566778899aabbccddeeff param(*).
lookup the received identifier, using the mechanism at (2.), check if a valid authentication, and is authorized to do requested operation, and then proceed with the operation on behalf on the auth'd user.
Of course you can build upon existing infrastructure, you can use PHP's session management (that would take care of 1./2. and the authentication part of 4.) in your app, and require that client-side implementation do cookie management(that would take care of 3.), and then you do the rest of your app logic upon that.
(*) Each approach has cons and pros, for example, using a GET request param is easier to implement, but may have security implications, since GET requests are logged. You should use https for critical (all?) applications.
The session management is server responsibility. When session is created, a session token is generated and sent to the client (and stored in a cookie). After that, in the next requests between client and server, the client sends the token (usually) as an HTTP cookie. All session data is stored on the server, the client only stores the token. For example, to start a session in PHP you just need to:
session_start(); // Will create a cookie named PHPSESSID with the session token
After the session is created you can save data on it. For example, if you want to keep a user logged:
// If username and password match, you can just save the user id on the session
$_SESSION['userID'] = 123;
Now you are able to check whether a user is authenticated or not:
if ($_SESSION['userID'])
echo 'user is authenticated';
else
echo 'user isn't authenticated';
If you want, you can create a session only for an authenticated user:
if (verifyAccountInformation($user,$pass)){ // Check user credentials
// Will create a cookie named PHPSESSID with the session token
session_start();
$_SESSION['userID'] = 123;
}
There are numerous way for authentic users, both for Web applications and APIs. There are couple of standards, or you can write your own custom authorization / and or authentication. I would like to point out difference between authorization and authentication. First, application needs to authenticate user(or api client) that request is coming from. Once user has been authenticated, based on user's identity application needs to determine whatever authenticated user has permission to perform certain application (authorization). For the most of traditional web applications, there is no fine granularity in security model, so once the user is authenticated, it's in most cases also and authorized to perform certain action. However, this two concepts (authentication and authorization) should be as two different logical operations.
Further more, in classical web applications, after user has been authenticated and authorized
(mostly by looking up username/password pair in database), authorization and identity info is written in session storage. Session storage does not have to be server side, as most of the answers above suggest, it could also be stored in cookie on client side, encrypted in most cases. For an example, PHP CodeIgniter framework does this by default. There is number of mechanism for protecting session on client side, and I don't see this way of storing session data any less secure than storing sessionId, which is then looked up in session storage on server-side. Also, storing session client-side is quite convenient in distributed environment, because it eliminates need for designing solution (or using already existing one) for central session management on server side.
Further more, authenticating with simple user-password pair does not have to be in all case done trough custom code which looks up matching user-record in database. There is, for example basic authentication protocol , or digest authentication. On proprietary software like Windows platform, there are also ways of authenticating user trough, for an example,ActiveDirectory
Providing username/password pair is not only way to authenticate, if using HTTPS protocol, you can also consider authentication using digital certificates.
In specific use case, if designing web service, which uses SOAP as protocol, there is also WS-Security extension for SOAP protocol.
With all these said, I would say that answers to following question enter decision procedure for choice of authorization/authentication mechanism for WebApi:
1) What's the targeted audience, is it publicly available, or for registered(paying) members only?
2) Is it run or *NIX, or MS platform
3) What number of users is expected
4) How much sensitive data API deals with (stronger vs weaker authentication mechanisms)
5) Is there any SSO service that you could use
.. and many more.
Hope that this clears things bit, as there are many variables in equation.
If the API based APP is a Client, then the API must have option to retrieve/read the cookies from server response stream and store it. For automatic appending of cookies while preparing request object for same server/url. If it is not available, session id cannot be retrieved.
You are right, well the reason things are 'automatic' in a standard environment is because cookies are preferred over URL propagation to keep things pretty for the users. That said, the browser (client software) manages storing and sending the session cookie along with every request.
In the API world, simple systems often just have authentication credentials passed along with every request (at least in my line of work). Client authors are typically (again in my experience) reluctant to implement cookie storage, and transmission with every request and generally anything more than the bare minimum...
There are plenty of other authentication mechanisms out there for HTTP-based APIs, HTTP basic / digest to name a couple, and of course the ubiquitous o-auth which is designed specifically for these things if I'm not mistaken. No cookies are maintained, credentials are part of every exchange (fairly sure on that).
The other thing to consider is what you're going to do w/ the session on the server in an API. The session on a website provides storage for the current user, and typically stores small amounts of data to take load off the db from page to page. In an API context this is less of a need as things are more-or-less stateless, speaking generally of course; it really depends what the service is doing.
I would suggest you send some kind of token with each request.
Dependent on the server and service those can be a JSESSIONID parameter in your GET/POST request or something mature like SAML in SOAP over HTTP in your Web Service request.