Folks:
This is a REST design question, not specific to any programming language. I am creating an application backend that is accessed via REST APIs. I would like to use the same APIs for both UI and API-based access. I am trying to figure out the best way to authenticate users so that I can reuse the same methods.
My current thinking on authentication is as follows:
API Users
These users get a user GUID and a pre-shared symmetric key. On each API request they include additional headers or request parameters that contain:
Their GUID
A security token that contains the user GUID, the current timestamp and another GUI (token GUID) concatenated together and encrypted using the shared key
Upon receiving the request, the server looks at the claimed GUID, retrieves the shared key, attempts to decrypt and verifies the token.
UI Users
These users will make a login request, supplying human credentials (userid/password). Once authenticated, a session is established backed by cookies and further REST calls are secured based on this session.
The Problem
What is the best way to write one REST endpoint that secures both ways: API access and UI access cleanly without too much duplication? I am looking to do the equivalent of the following, but perhaps more cleanly:
#app.route('/')
def hello():
user = None
if session:
user = get_authenticated_user()
else:
user = process_auth_headers()
# Do something with user
I am looking to code the server in Flask, but I am sure the solution will apply as easily to any REST-based server-side framework.
Looking forward to some insights from the community.
Thanks.
We use node for our server, but I think the method we use is pretty common. There are session libraries that express can use, and they can utilize pretty well any database to store session information. They use a cookie with a key that does a lookup on the database when the client comes in. The session data is created when the client authenticates, and the cookie with the client key is added to the browser. The clients GUID is stored in the session, it never leaves the server. We use that info when they hit the server to check if they are logged in, who they are, and what they can do. We have used both FB, (client checks FB, then sends the FB id and token down to the server, which then rechecks and sets up the session or rejects it,) or the old classic, email and password. This works well when you have to scale across multiple app servers, as session is independent of the server, it works for both mobile clients and the web.
Related
My service allow any HTML documents to be converted to PDF using a POST request.
It is mostly used on the backend of my client's server and thus, the API key used for the communication is kept private.
Now, I'm thinking of a way to let my client's visitors be able to call my service on behalf of my client API key, without exposing this secure API Key.
My main issue here is security. If my client add an XHR POST requests that contains the API key, someone can take that API key and use it for their own purpose and abusing my client's account.
I could filter by domain, but this is easily spoofed so it's not possible.
I was wondering if there was a way to call a private service and be identified without risking its identity to be stolen, from the client ('s client) side?
If you're providing this sublet for authenticated users, then it's fairly trivial to give them unique keys (something that hashes their user ID or session against the API key and an initial timestamp, and checks it / logs it / looks for brutes before accessing the API). If you're doing it on the open web, without any kind of user authentication, then rate limiting gets very tricky indeed. Generally you'd want to use a combination of session hashes, IP address, operating system and browser data to create an anonymous profile that gets a temporary key on the frontend. One fairly solid way to do this is to force users through a CAPTCHA before serving a temporary key that allows them a limited number of uses of the permanent key. Any user whose ip/browser/session matches the existing attributes of a known client key is shunted to that one (and gets to skip the CAPTCHA); anyone who doesn't match an existing profile gets the CAPTCHA. That makes you a less attractive target for spoofing. On top of that, you should always rate-limit the entire thing, within a reasonable number of hits per day based on what kind of traffic you expect (or can afford), just so you don't have any surprises. This is the minimal security you'd want if your client's money is on the line every time their API key is used. It will require a simple database to store these "profiles", track usage, check for brutes and maintain the currently valid client keys. Client keys should always be expired regularly - either with a time diff against when they were created, or a regular cron process, or a maximum number of uses, etc.
One other thing I frequently do is rate-limit based on a curve. If I think 5 uses per minute is reasonable, for example, then after 5 uses in a minute from a session, each usage adds a delay of a fraction of a second * the number of uses in the last minute, squared, before the data is served.
The best answer would be to put this all behind a login system and secure that.
Assuming that you are using OAuth kind of system, In that case, make use of Access Token Mechanism that provides access to private API/User's data on behalf of User(Client) without exposing his/her credentials or API Key(Authentication key), also the access token can be expired based on the time/usage.
Example: The access token is generated against a single endpoint that can be the Html Conversion endpoint and will be expired once the action completion.
https://auth0.com/docs/tokens/access-token
And following blog post would be helpful to architect your authentication system
https://templth.wordpress.com/2015/01/05/implementing-authentication-with-tokens-for-restful-applications/
there is no good way to do front-end secure storage but my recommendation is :
is an API that used HMAC signing of requests in combination with OAuth authentication. The API key is actually a signing key. they key does not get transferred. The API key can still get found on the front-end but it becomes useless because you still need the OAuth token to send a valid request.
i know users will have to login in, but you can see this as an advantage because atleast you can log who is using the app by getting information from oauth.
please consider back-end secure storage!
You can use JWT tokens in my opinion. On the basis of username, password or any other info you can generate unique jwt tokens for different users.
Anyone can decipher these jwt tokens but not he unique security token.
If you want to add more more security to tokens, use JWE, encrypted web tokens.
More about these schemes can be found at https://medium.facilelogin.com/jwt-jws-and-jwe-for-not-so-dummies-b63310d201a3
Hashing is a decent option and should be done anyway, but for a fully secure method that wouldn't add too much complexity, you could simply abstract away from the authorization/API key by building your own API to interface with the API. This way you could both limit the kinds of things that can be done with the API key and also completely obscure the API key from the user
I don't think you should always go for user auth or JWT, it just doesn't fit all use cases. The idea of using a Captcha is interesting but also somewhat complex.
If complexity is not an issue I would rather use an infrastructure approach, I'm most familiar with AWS so I'll focus on that. Assuming you can change the host of your front end you can have your site hosted on an S3 bucket, served through a CDN, and create a proxy Lambda function that will hold the logic to call your API and store the API key as an encrypted environment variable. This Lambda you call through an API Gateway that can only be called by a specific IAM role which the S3 bucket also uses. You can also use a Cognito User Pool without authentication.
Going back to a simpler alternative the Captcha approach can be implemented as an attestation provider. I know of two services that do this, Firebase and KOR Connect. Due to Firebase using this approach only for their own resources as of the time of this writing I much rather use KOR Connect as it’s a very simple middleware that basically solves this issue. I won't go into detail about these services as it’s not the main concern of this topic but you can check the documentation their respective links.
I'm working on a client-server application using WCF. The first client will be a desktop app (WPF) but I plan to add Xamarin (Android and iOS) and web client. I want this to be an internet service, and to potentially support a large number of clients, so I'm using http, a stateless service to conserve server resources.
I am having difficulties implementing authentication logic.
I found this great example that implements forms authentication:
http://www.dotnetspeak.com/wcf/securing-wcf-with-forms-authentication/
It allows me to handle authentication how I want - compare username and password against the database, create an authentication cookie and return it. Subsequent calls using this cookie will be authenticated automatically.
But the problem is, I don't know which user called the service. If GetMyData() is called by user1, I want to make sure he only gets his own data. I obviously don't want to have the client send their ID separately with each request, because that can be easily tampered with - just switch "user1" for "user2" and hey presto, you're getting someone else's data.
I can get to the authentication cookie inside the service method by calling
WebOperationContext.Current.IncomingRequest.Headers[HttpRequestHeader.Cookie]
(I can also see there's one other header called "Host")
The cookie is generated from a FormsAuthenticationTicket, which contains the username and password, but it's encrypted. I'm not sure whether it's possible for me to decrypt the cookie in my code, but I'm pretty sure it wouldn't be the correct approach. My method was called AFTER the underlying system authenticated the caller, so presumably the cookie was decrypted and the ticket was found to be valid. So why can't I get to the data?
Sadly, every article I've found only deals with authenticating the user, but nobody seems to care about which user is using the service afterwards, as long as he's authorized.
I suppose I could store the cookies server-side, along with mapping to the specific user, and find the user that way. But I want the service to be as stateless as possible for performance reasons. Also, this would involve doing fulltext matching on a 300 character long string - for every single request! Yikes!
It seems to me that what I need is a very common use case, so there must be a proper way to do it. I just can't seem to find it. Can anyone help me out?
If you have Forms authentication setup correctly then you can get the logged-in username via Thread.CurrentPrincipal.Identity.Name and send it to your service method for data access validation. Get the user id from username and validate ownership.
Thread.CurrentPrincipal.Identity.Name decrypts the cookie ticket and returns the logged-in username.
I have been reading about securing REST APIs and have read about oAuth and JWTs. Both are really great approaches, but from what I have understood, they both work after a user is authenticated or in other words "logged in". That is based on user credentials oAuth and JWTs are generated and once the oAuth token or JWT is obtained the user can perform all actions it is authorized for.
But my question is, what about the login and sign up apis? How does one secure them? If somebody reads my javascript files to see my ajax calls, they can easily find out the end points and the parameters passed, and they could hit it multiple times through some REST Client, more severely they could code a program that hits my sign up api say a thousand times, which would be create a thousand spam users, or they could even brute force the login api. So how does one secures them?
I am writing my API in yii2.
The Yii 2.0 framework has a buil-in filter called yii\filters\RateLimiter that implements a rate limiting algorithm based on the leaky bucket algorithm. It will allow you to limit the maximum number of accepted requests in a certain interval of time. As example you may limit both login and signup endpoints to accept at most 100 API calls within a 10 minutes interval of time. When that limit is exceeded a yii\web\TooManyRequestsHttpException exception (429 status code) will be thrown.
You can read more about it in the Yii2 RESTful API related documentation or within this SO post.
I didn't use it myself so far but from what I did read about it in official docs, and I mean this:
Note that RateLimiter requires
$user
to implement the
yii\filters\RateLimitInterface.
RateLimiter will do nothing if
$user
is not set or does not implement
yii\filters\RateLimitInterface.
I guess it was designed to work with logged in users only maybe by using the user related database table, the default one introduced within the advanced template. I'm not sure about it but I know it needs to store the number of allowed requests and the related timestamp to some persistent storage within the saveAllowance method that you'll need to define in the user class. So I think you will have to track your guest users by IP addresses as #LajosArpad did suggest then maybe redesigning your user class to hold their identities so you can enable it.
A quick google search let me to this extension:yii2-ip-ratelimiter to which you may also have a look.
Your URLs will easily be determined. You should have a black list of IP addresses and when an IP address acts suspiciously, just add it to the black list. You define what suspicious is, but if you are not sure, you can start with the following:
Create something like a database table with this schema:
ip_addresses(ip, is_suspicious, login_attempts, register_attempts)
Where is_suspicious means it is blacklisted. login_attemtps and register_attempts should be json values, showing the history of that ip address trying to log in/register. If the last 20 attempts were unsuccessful and were within a minute, then the ip address should be blacklisted. Blacklisted ip addresses should receive a response that they are blacklisted whatever their request was. So if they deny your services or try to hack things, then you deny your services from them.
Secure passwords using sha1, for example. That algorithm is secure-enough and it is quicker than sha256, for instance, which might be an overkill. If your API involves bank accounts or something extremely important like that, important-enough for the bad guys to use server parks to hack it, then force the users to create very long passwords, including numbers, special characters, big and small letters.
For javascript you should use OAuth 2.0 Implicit Grant flow like Google or Facebook.
Login and Signup use 2 basic web page. Don't forget add captcha for them.
For some special client such as mobile app or webServer:
If you sure that your binary file is secure, You can create a custom login API for it. In this API you must try to verify your client.
A simple solution, you can refer:
use an encryption algorithm such as AES or 3DES to encrypt password
from client use a secret key (only client and server knows about it)
use a hash algorithm such as sha256 to hash (username + client time + an other
secret key). Client will send both client time and hash string to
server. Server will reject request if client time is too different
from server or hash string is not correct.
Eg:
api/login?user=user1&password=AES('password',$secret_key1)&time=1449570208&hash=sha256('user1'+'|'+'1449570208'+'|'+$secret_key2)
Note: In any case, server should use captcha to avoid brute force attack, Do not believe in any other filter
About captcha for REST APIs, we can create captcha base on token.
Eg.
For sign up action: you must call 2 api
/getSignupToken : to get image captcha url and a signup token
respectively.
/signup : to post sign up data (include signup token and
captcha typed by user)
For login action: we can require captcha by count failed logins base on username
Folow my api module here for reference. I manager user authentication by access token. When login, i generate token, then access again, client need send token and server will check.
Yii2 Starter Kit lite
I am trying to design REST APIs to support various mobile clients (iOS and Android apps). These apps will let user login using facebook login along with our own email authentication. You can refer to the diagram below to understand my design
There are two levels of authorization take place:
First one is "Client (or App) Authorization" that uses OAuth2. So when user install our app on mobile device, and starts app, then very first thing, app makes "Client (App) Authorization" as shown in above diagram (1st image). And server sends back an long-lived access_token to client to use for all subsequent calls. Here my question are:
Q1) You can see client is sending client_key and client_secret and I am storing them in client_info table. Should this secret be in plain text or it should be in decryt-able format? If I encrypt it, I still need to keep encryption key somewhere in my system. So how it will make it secure? Also in every call, decryption will be an overhead.
Q2) Is it ok to cache access_token for the client in plain text format in redis and use that cache first?
Q3) In order to be extra safe, I am asking clients to send appsecret_proof to make sure the access_token, they are sending belongs to this client only. It uses the same concept as Facebook https://developers.facebook.com/docs/graph-api/securing-requests#appsecret_proof. And it is hash_hmac('sha256', access_token, client_secret)
Q4) We will only have our own 2 mobile app (each for iOS and Android) and not providing third party to use our API to develop other apps. That means, our client_info table will only have two rows one for each type of apps. So is it okay, that in app code, we keep client_key and client_secret hardcoded? If yes, then in future when we have to invalidate and use new secret then how will we achieve replacing those info?
Q5) Since it is our own apps for couple of years, so there would be multiple access_token will get created against same client_key and client_secret. In order to save all of them, is it a good idea to store client_key as key and an array of all access_tokens as value in redis. In future, when we will open our API to third party, then this redis storage design can still scale?
=================
Later on, user decides to perform some actions on my app, for that we need user to login to his account. For that user click on "facebook login". My app gets facebook access_token and fb user's id from facebook and pass those info to API server (as shown in 2nd diagram). API server takes that token and call facebook API to validate its access_token. Once token is validated, server uses some metadata related to that user along with FB access token to generate our own user_access_token, lets say utoken. And pass that utoken back to client to pass back in every subsequent user specific API calls. Here my questions are:
Q1) Is it ok to save that utoken in database, user_token table. Should this utoken be in plain text or it should be in decryt-able format? If I encrypt it, I still need to keep encryption key somewhere in my system. So how it will make it secure? Also in every call, decryption will be an overhead.
Q2) In every user specific API calls, should I call facebook every time to check facebook access_token is still valid? I believe I should not, as that is not going to get anything to me. Please note, Facebook is ONLY used for "facebook login".
Q3) What are the information I should encrypt to generate utoken? I am thinking to have a hash or associative array of user's email, user id, role and facebook token and then serialize that data structure and finally encrypt it. Do you think that would be good enough. I understand its per my requirement, but as a standard or common app, are they good enough? Or is there any best practice?
Q4) Should client store utoken in its cookie/cache? Isn't that scary?
Q5) Please note user may have multiple devices, logged in with same user credential. That means, in user_token table, we would have to store multiple utokens for those logged-in session, while all of them will belong to the same user. Does that sound right?
A design proposal somewhat smiliar to mine REST API for website which uses Facebook for authentication
Q1.1: No!. Client credentials is not intended to be used that way. If your client is a Single Page App or a Mobile App, you will be forced to store your client credentials in an insecure environment, the user's machine. You should be using OAuth's Implicit flow
Q1.2: Assuming the token is short lived, no problem caching it. The key of OAuth, apart from ensuring that you can rely on other application to authenticate your users, is that you effectively substitute user or application credentials, which are long lived, with a short lived token. So if someone gains access to the token,at least, their access to the system will be limited in time.
Q1.3: Check out that facebook documentation:
Graph API calls can be made from clients or from your server on behalf of clients. Calls from a server can be better secured by adding a parameter called appsecret_proof.
It states that appsecret_proof is to be used for calls from the server on behalf of the user. The point here has to do with Q1.1. If you were storing your client_secret in your user's device, they could generate the appsecret_proof.
Q1.4: Again, No! You should have a good read of OAuth Specification and understand the different flow types and when to use each. Also bear in mind, that if you provide an API for your App the API is public for any one to abuse. The only difference is that it is not documented. The same will happen with a Web App. Once it is in the internet, I could write a scraper and abuse the Web App. This is perfectly normal, just bear in mind that anything on the internet is not private, it is just undocumented.
Q1.5: Again, tokens should be short lived. If their lifespan is the same of the credentials, which live until the user change them, then tokens lose their purpose.
Q2.1: You should save that token A ReST architecture uses a client cache constraint.
Q2.2: I don't think so. Facebook is just telling you that the user that obtained that token has some identity (an email, for example) that you can associate to a user in your system. Once you know that association you should't care much about the Facebook token but to make calls to Facebook API. But as you say, you are using it just for login.
Q2.3: Seems not bad but check again the Oauth Specification as you seem to be building an Implicit flow and using JWT tokens. As per what you want to store in your token, seems fine.
Q2.4: JWT tokens must be cached by the client. Nothing scary, because they are opaque to the client as they are encrypted. The client sends the JWT token with each request and the API server decrypts the token with a private key (that has never been exposed outside the server) and can check the identity of the user.
Q2.5: Remember short lived tokens. Tokens must expire!.
I have built a simple test API for a CakePHP application that will let a user login from a mobile device (or any device for that matter) and get a JSON response. This API could be used for a mobile app built in PhoneGap.
The login method looks like so:
public function login()
{
if($this->request->is('post'))
{
// Use custom method in Model to find record with password params
$findUser = $this->User->findUser(
$_POST['username_or_email'],
AuthComponent::password($_POST['password'])
);
// If a user exists and matches params
if($findUser)
{
$this->User->id = $findUser['User']['id'];
$this->autoRender = false;
$this->response->type('json');
$this->response->body(json_encode(array('authenticated'=>true,'message'=>__('You have been logged in successfully'))));
}
else
{
$this->autoRender = false;
$this->response->type('json');
$this->response->body(json_encode(array('authenticated'=>false,'message'=>__('Username or password is incorrect'))));
}
}
else
{
$this->autoRender = false;
$this->response->type('json');
$this->response->body(json_encode(array('message'=>'GET request not allowed!')));
}
}
The mobile device (or any API user) can send their login details and then they get the request as JSON as true or false for authenticated. This boolean is NOT used to give the user access, it instead tells the mobile app if they can see certain screens and they ONLY get the data or can send data if the session exists!
As just stated, they are also actually logged into the API itself on the device so if they visit the website directly (from that device) they will have a session and see the same response for the JSON.
So essentially a user remains logged in for the duration of the session on the device they communicated with the server on. This is different to a token which would need to be passed for every request, where as in this example they have a session.
Now the questions...
Is it bad practice for the user to be 'actually' logged into the API
with a session like shown above? It seems like the most secure way to handle authentication for a device as it's using the same logic as the direct web root.
I've seen some APIs use access tokens instead which I've also
implemented (user gets their token returned instead of the boolean
and no session is created). But from what I can tell, this seems
like more work as then I need to check for the access token against
a user record every time a request is made.
edit
For the sake of clarity, I am not a supporter of REST, I AM a supporter of RESTful/RESTlike services. If you look at all of the API's on the internet, very few actually stick to one standard. Whatever scheme you choose will depend on your specific problem-space. Just try to be secure and use intuitive design choices (ie dont name a service "cats" if it returns info about "dogs")
end edit
It is good practice in RESTful API's to manage some form of session/tokenizing scheme. Really the ideal (at least in my opinion, there are many schools of thought on this problem) setup involves rolling tokens.
If you are at all concerned with the security of your API, then permissions should be managed out of your database layer. Yes, this creates a bottleneck, BUT THAT IS ACTUALLY A GOOD THING. Needing to hit the database every single time to validate a client's token adds an extra step in the entire process. This slows down the API, which is actually desireable in a secure system. You don't want a malicious individual to be able to hit your API 3000 times a second, you want their requests to hang for a (somewhat) sizeable fraction of a second.
This is similar to MD5 hashing algorithms. Many of them recalculate the hash a few hundred times, with random pauses in between. This helps to keep a malicious client from attempting to brute force a password (by making it take more time to test each variation of the password string). The same applies to your API.
The other benefit, is that if you DO have a malicious user trying to log in over and over again, if you are managing them from the database layer, then you can red flag their IP Address/username/what-have-you and just drop their requests at step 1.
Anyway, for a suggested process (with rolling tokens, you can cut out parts of this if it seems overkill, but this is hella secure):
User hits a 'login' service, this requires a username/password, and returns two tokens, a Private Access Token and a Public Request Token (the server stores these tokens in the db).
The client stores these Tokens in a secure place
User accesses another endpoint to push/pull some data
Request includes a timestamp
Request includes the Public Request Token
Request includes an Access Token=> This token should be a MD5 hash of the string resulting from concatenating the timestamp string to the end of the Private Access Token string
The server takes the Public Request Token, uses that to lookup the Private Access Token that was stored
The server takes that Private Access Token, and concatenates on the Timestamp String, it then takes the MD5 of this string
If the new Access Token matches the one that the client sent the server, HURRAY, this client is validated, so push/pull the data
(Optional) The server generates new tokens on every request, and returns them to the client. This way every transaction invalidates the old tokens, and if there was some kind of man-in-the-middle attack occurring, if the VALID user has already completed their request, the malicious user now has invalid tokens and can't start messing with your API. This scheme tries to ensure that a malicious user can not expect to intercept a single communication between the server and the client, and still gain access to the system. If they do, then the REAL user should immediately get invalidated tokens. Which should then trigger their API client to hit the 'login' service AGAIN, getting new valid tokens. This once again kicks the malicious user out of the system.
This scheme is not 100% secure, no user access system ever will be. It can be made more secure by adding expiration dates on tokens. This scheme also has the added benefit that you can assign specific permissions to users/tokens (ie Read-Only access, only certain End-Points can be seen, etc)
This is not the only way you can do things, I would look up other Authentication Schemes and take what you want from each of them (OAUTH is a good place to start, then I'd look at Facebook/Twitter/Instagram)
Make your app login everytime, but not with login-pass pair as Swayok lastly suggested. When you login, server generates a token and returns it back to the client. Client then uses this token whenever it makes a request. On each request, server checks whether the token is valid and if so, executes the request.
This is very similar to how sessions work in that, server side frameworks manage it internally and these tokens expire from time to time. However, as Swayok rightuflly pointed out, you don't want session mainly because you're RESTful API should have no state. You get the same utility without storing any user specific data regarding user and logging user in with every request.
Here's a good article on this, or you can try the Facebook Graph API explorer to see it in action
Restful API restricts using sessions and saving system state at all. Each request must log-in user.
Access tokes are great but also require additional handling.
The easiest way is to send authorisation data via HTTP Basic Auth ("Authorization" HTTP header)
http://www.httpwatch.com/httpgallery/authentication/
Mobile Applications can easily do that and it is easy to add this header for each request to API.
On server side:
$username = env('PHP_AUTH_USER');
$password = env('PHP_AUTH_PW');
And process user log-in with this data in ApiAppController->beforeFilter()
To answer your questions
Its not a bad practice as long as you close their session on app close and recreate it when needed. it is same as if they were logged in on a browser they would know and have facility to log out however the same should be available on the app as well otherwise they might have closed the app but not actually ended their session. You can handle this in many ways by asking them to log out automatic checking when they close app
Tokens are an enhanced way of doing the above however you have to consider how secure the token is when transmitted and server need to verify the token on each request. You have said that it seems like more work so yes its more work and if you have time or money constrains and looking for an answer to say if the session style would harm your application in future it wont as long as you are in control of session and not leaving user without ending the session. If you have time then implement tokens and you would like that.