I have created an API that I only want certain clients having access to. After a bit of research, I found that API Keys and API Secrets are pretty good way of controlling that.
I want to basically generate my own secrets using information I control. For example, if I create the secret 1500315177265-8005550000-System, the secret itself has information I can reference and validate. From what I understand, it doesn't matter what the secret is, as long as its not shared, and that's what makes it a secret. I can append some long salt values to make it much harder to guess. Can I use secrets securely this way?
Along the same lines, I am thinking what the need for the API Key is needed if the secret itself has identifying information. More than likely, I'm missing something here or more APIs would do it this way.
You don't need to suffer creating this mechanism by your own, you could just use OAuth for this. Actually by reading how OAuth works, I'm pretty sure that you'll get an overall idea of what the "API Key" is doing.
Long story short. OAuth allows you to create access tokens that will be requested by your users, once the user has asked for an access token, he'll be able to use it to get authenticated on the API.
You can also configure the duration for these access tokens, for example, let's say that you only want them to be valid for one hour. Then the user will use a new access token every hour, helping you to minimize risks against compromised access tokens.
If you want to take a look at the pipeline used by OAuth, it will be something like this:
The user sends an identifier and a shared secret (see them as username and password). If the identifier and the secret are correct, he'll receive a "refresh token", this token will be used to send requests to the token server to receive new "access tokens". The client sends a request to the tokens server to receive an access token, and now he'll be able to send requests to your API, where he'll use his access token as an identifier, and only if the token is valid, he'll get access to your application.
Maybe you could be curious about expired tokens, for example, What happens when the token has expired? Now our user needs to repeat all process again? No, because if the token has expired, he can just send a new request to the tokens server, sending his refresh token, and he'll get a new token to be used with your API.
I'm pretty sure that you already noticed that your API key is the "access token" equivalent, and your shared secret is the equivalent to the credentials used at the beginning of the OAuth process.
You can create your own mechanism, but you will need to take a lot of considerations that are already covered by OAuth. For example, Will you expire your API keys? After all, you should not trust a single key to be sent over every request for an undetermined amount of time. How will you handle the process to request new keys? Do you really want your users to send their secret every time they want a new key? Refresh tokens are useful for this.
Related
When a user is authenticated by the server, the server sends, according to the most common implementation I found(which is were my question stems from), sends BOTH access token and the refresh token to the client.
-- My first questions is:
What is the point of sending both tokens to the client if, to my understanding, we have the refresh token in the first place to help us mitigate the consequences of an access token being leaked?
If both tokens are sent to the client and are stored together (in the same place, localStorage, sessionStorage, one in former, one in latter - I don't think it matters to the questions how we decide to store them on the client), if an attacker manages to steal the access token, then it is safe to assume he would also find a way to steal the refresh token from the same client since they are most likely stored together. I don't understand why most implementations and answers I find online mention sending both tokens to the client since its basically like putting all eggs in the same basket.
(A big part of my confusion comes also from the fact that those answers and implementations don't mention anything about storing the tokens in HttpOnly cookies, which I'd think would be a common practice if we want to prevent tokens getting leaked, so I'm getting a feeling I'm missing something)
-- My second question would be:
Would it be a correct solution then to persist the access and refresh tokens upon issue in an "accessT <-> refreshT"(like a key=value pair) table on a server, so that only one token is ever sent to the client - the access token? And when that access token expires the process of "refreshing" would be the following:
ResServer = resource server
AuthServer = authorization server
The request with the expired token is sent to the ResServer.
ResServer checks and sees that the access token has expired.
ResServer then sends a request to the AuthServer to look up the refresh token by the access token in the table mention above.
If AuthServer finds a refresh token by using the expired access token as a key - good, AuthServer generates a new access token.
AuthServer then responds to the ResServer with that new access token.
Having received a positive response, ResServer proceeds on with it's usual flow for authorized users.
^^ This seems to me to be a more rational way of keeping the refresh token safe - making it serve its primary function, which is why I don't understand why most other implementations on the web always mention sending both tokens to the client.
-- And my third question is:
Not having much experience implementing microservices, I am confused about why we would ever prefer to implement this token back and forth approach, when, from what I understand, the entire point of tokens is that they are supposed to be a stateless solution(well, the refresh token is always kept on the server regardless of implementation, which confuses me even more about that "stateless" part). Doesn't it make more sense to just use cookies that basically serve the same function, but are easier to implement, secure by default("Secure", "HttpOnly" flags), and, as it appears to me, are much easier to revoke in can they are leaked?
Thanks you for taking your time to read this entire post, I appreciate your help.
Regarding your first question:
The advantage of having two tokens is not that you can store one of the tokens more securely. The advantage is that if the Resource Server gets compromised, it does not allow lateral escalation. For this to make sense, your access tokens must be bound to specific resource servers.
As an example, assume you have two resource servers ResA and ResB. Then your Authentication server would allow you to obtain one access token for each of these servers, and one refresh token (that allows refreshing both access tokens). If resource server A gets compromised (or the network path between the client and that server), then the access token to that server will expire at some point, and the attacker will not be able to access resource server B with the access token issued for resource server A. At the same time, only the legitimate user is able to refresh their tokens, since they only provide the refresh token to the AuthServer, such that a compromised or malicious resource server never sees at.
That leads directly to your second question:
Your proposed model prevents key rollover. If a resource server gets compromised, all its tokens are eternally compromised, unless the Authentication Server voids all refresh tokens. Voiding refresh tokens would lead to users being logged out of all resource servers (even those who are not compromised).
As an example, consider the following scenario: Tour authentication provider is a large public service, such as "Sign in with Google", with thousands of resource servers using this service. Now one of those resource servers gets compromised, and attackers steal the authentication tokens.
In the commonly employed auth-refresh token pattern, the auth tokens will expire, which means that the service will be inaccessible for attackers as soon as the used vulnerability is fixed + the time it takes for tokens to expire (since the attackers could not get hold of the resource tokens), without the need for the AuthServer to do anything. Additionally, the AuthServer does not need to keep track of the issued tokens, if it uses cryptography to sign the tokens with a private key only known to the auth server, as it (and all resource servers) can trivially verify token authenticity by validating against a public key (please also see the end of my answer).
In your proposed approach, the AuthServer (which is Google in this example) would have to void all refresh tokens OR they would have to selectively delete all auth-refresh token pairs for the affected resource server from the key-value store. That key-value store will be gigantic, since it has to contain EVERY auth token ever issued, for any service that uses "Sign in with Google", and apart from storage costs, this delete operation will take quite some time.
Regarding your last question:
You can store access tokens in Cookies. There is a bunch of advantages to that (including the Secure and HttpOnly flags that you mentioned). It has two disadvantages: If you want to implement something like a CLI client that runs outside of a browser, you would have to implement Cookie handling. The other disadvantage is the CSRF risk as Cookies are automatically sent on every request. Using local storage prevents this, as an external website cannot access the local storage of your website to extract the token.
In the end it comes down to the concrete scenario, if Cookies or Local Storage + Custom Header are the preferable solution.
However, I think you have a misconception regarding statefulness:
Access and Refresh tokens do not need to be stored on the server. Instead, the Auth Server signs the Token Contents with a cryptographic private key, before sending them to a client. In doing so, the AuthServer includes an expiration date in the token, which is included in the signed content.
To validate the token, the Resource Server (or the AuthServer) checks if the signature comes from the AuthServer's private key, and then checks if the expiration date has passed. In particular, the resource server does not need to communicate with the AuthServer, apart from obtaining its Public Key from time to time. This makes this solution scale really well for very large deployments. See the Microsoft Azure docs, which do a great job on explaining the difference between Auth and Refresh tokens.
What you are alluding to are traditional session ids, which are stored on the server until they expire. Thats obviously a much simpler solution, and works well for smaller, more monolithic applications. It just does not work for hyper scalers.
I'm just learning JWT in nodejs, and I found out about refresh tokens.
As far as I understand, a user gets an access token and a refresh token. After the access token expires, a request containing the refresh token is made to get a new access token. To get a new access token, the server checks if the received refresh token is contained in a database. If the refresh token is stolen, it can easily be deleted from the DB and prevent further refreshes.
My question is: Why don't we just make the access tokens behave like refresh tokens? i.e. We store them in a database and check if they are there when making a request, and when compromised we just delete them?
The key element to answering your question is: You need to add an expiration date on access tokens you deliver to clients. This is the main purpose with refresh token.
Imagine someone steels your access_token, and you didn't make it expirable: It means that as long as you didn't discover that your access_token has been stolen, you're giving literaly a lifetime free pass to whoever has it.
With refresh tokens and expirable access_tokens, you know that the window of vulnerability is really small.
Now your second question: Why don't we make access_tokens behave like refresh_tokens ?
The key idea here is to keep your refresh_token in a safe spot, and only expose access_tokens.
And by the way, refresh_tokens have one job: Carry information to generate new access_tokens, access_tokens on the other hand have their own job: Carry information necessary to give you direct access to resources.
If you pay attention to most serious websites, they have a centralized auth server that serves access_tokens.
Answering comments:
Key difference between them is: refresh token key is like a master key, it stays on the authorization server and is never ever shared with any other server, unlike access token key, it can be passed to another server to authenticate users
in other words:
auth.yourapp.com: stores access_token_key and refresh_token_key
api.yourapp.com: stores access_token_key ONLY, to make sure that users did actualy authenticate on your auth.yourapp.com domain, and api.yourapp.com can easily confirm that. if access_token_key has been compromised, vulnerability has a shorter lifetime, and you can easily isolate the attacked server.
if one of your servers is compromised, the rest is safer.
In my project there's a requirement to invalidate all jwt tokens of a user when the user changes his password. I was thinking of giving each user a different signing key, and simply reset the key when password is changed. Then I googled around and found Redis is a good place to store those per-user keys. Everything seems to work just fine.
But there one thing I cannot get my head around. Since it has to hit Redis once per request, is it any different than issuing the user an opaque token instead of JWT, and store the token -> JWT payload mapping in Redis?Isn't that defeats the purpose of using JWT?
To invalidate tokens you need to revoke them. OAuth spec also does not require getting secret key from remote server every time you need to validate JWT (as you said it kind of defeats the purpose). The key can be stored locally at resource site.
You have two options here:
1) Introspect the JWT token from resource side against OAuth server every time it validates it. Seems like overkill to me. The best approach is to give short expiration time to JWT token and let the already issued tokens to just expire.
2) Have the resource store the secret key locally and when it fails to validate go and get the key and re-validate it again.
From the point of view of invalidating the token, there's no particular need to store the JWT in Redis - anything that you can check and later invalidate should do the trick.
That said, presumably you're using a JWT for other reasons. For example, it's what the AuthN/Identity service provides. Or perhaps you use it to store claims or other metadata that you validate as part of the AuthN/AuthZ logic. In that case, since it's handy, storing the JWT seems very reasonable.
I am planning to secure my rest API in django with a ACCESS_TOKEN.
When ever user is logged in using their username and password, once they are authenticated, I generate a ACCESS_TOKEN and passed to frontend be it Website or Native application. and then later used that ACCESS_TOKEN for further communication.
I am generating this token based on some user data and then encrypting this with public key. Later when application send this for in any request, I decrypt the ACCESS_TOKEN with private key and extract user data data and process the request. This is something similar to session where session data is in encrypted form in ACCESS_TOKEN and only private key and decrypt the ACCESS_TOKEN. This is what I am planning to do.
Please suggest me for following questions:-
1. Is is the best way to secure my REST API? I want to use my API in same way from Web-application(AJAX calling) and NATIVE application(Android/IOS etc) ?
2. What is the best way to expire the token? Do I need to keep track of access token at my end in order to expire them?
Also I do want to use the Oauth in my API.
Most people I see use JWT's that are signed but not encrypted so they store non-PI data like user_id or session_id. I guess you could encrypt it if needing to store personal information but I don't see any other reason. Assuming you are using HTTPS, then only the end client would have access to the information. Sounds like asking for trouble if the secret gets leaked so you would want a really good key rotation scheme since you may not even know its leaked until too late.
Many people using JWTs do so because they don't want a centralized auth server, thus the token is short lived like a few hours or days. If you need really tight control on expiring tokens, you can take a blacklist approach where a blacklist of JTI's (JWT Ids) are stored in a K/V to be checked against. https://www.moesif.com/blog/technical/restful-apis/Authorization-on-RESTful-APIs/
I'm referencing another SO post that discusses using refresh tokens with JWT.
JWT (JSON Web Token) automatic prolongation of expiration
I have an application with a very common architecture where my clients (web and mobile) talk to a REST API which then talks to a service layer and data layer.
I understand JWT token authentication, but I am a little confused at how I should use refresh tokens.
I want my JWT authentication to have the following properties:
JWT Token has an expiration of 2 hours.
The token is refreshed every hour by the client.
If the user token is not refreshed (user is inactive and the app is not open) and expires, they will need to log in whenever they want to resume.
I see a lot of people claiming to make this a better experience using the concept of a refresh token, however, I don't see the benefit of this. It seems like an added complexity having to manage it.
My questions are the following:
If I WERE to use a refresh token, wouldn't it still be beneficial to have a long term expiration for good practice on that token as well?
If I WERE to use a refresh token, would that token be persisted with the userId and/or JWT token?
When I update my token every 1 hour, how does this work? Will I want to create an endpoint that takes in my JWT token or my refresh token? Will this update the expiration date of my original JWT token, or create a new token?
Is there the need for a refresh token given these details? It seems that If the user is just using a JWT token to grab a new token (per the link above) then the refresh token is obsolete.
Let me come to your questions a little later down the line and start by actually discussing the whole purpose of a refresh token.
So the situation is:
The user opens the app and provides his login credentials. Now, most probably the app is interacting with a REST backend service. REST is stateless, there isn't a way to authorize access to the APIs. Hence, so far in the discussion, there is no way to check if an authorized user is accessing the APIs or is just some random requests coming through.
Now to be able to solve this problem, we need a way to know that the requests are coming from an authorized user. So, what we did was to introduce something called an access token. So now once the user is authenticated successfully, he is issued an access token. This token is supposed to be a long and highly random token (to ensure that it can not be guessed). This is where the JWT comes into the picture. Now you may/may not want to store any user-specific details in a JWT token. Ideally, you would want to just store very simple, extremely non-sensitive details in the JWT. The manipulation of the JWT hash to retrieve other user's details (IDOR etc.) is taken care of by JWT (the library being used) itself.
So, for now, our problem with authorized access is solved.
Now we talk of an attack scenario. Let's say using all of the above user Alice, using the app, has the authorized access token and now her app can make requests to all the APIs and retrieve the data as per her authorization.
Assume that SOMEHOW Alice loses the Access Token or put another way, an adversary, Bob, gets access to Alice's access token. Now Bob, despite being unauthorized, can make requests to all the APIs that Alice was authorized to.
SOMETHING WE IDEALLY DON'T WANT.
Now the solution to this problem is :
Either detect that there is something of this sort happening.
Reduce the attack window itself.
Using just the access token alone, it is hard to achieve condition 1 above, because be it Alice or Bob, it's the same authorized token being used and hence requests form the two users are not distinguishable.
So we try achieving 2 above and hence we add an expiration to the validity of the access token, say the access token is valid for 't' (short-lived) time.
How does it help? Well, even if Bob has the access token, he can use it only while it is valid. As soon as it expires, he will have to retrieve it again. Now, of course, you could say that he can get it the same way he got it the first time. But then again there's nothing like 100% security!
The above approach still has a problem and in some cases an unacceptable one. When the access token expires, it would require the user to enter his login credentials and obtain an authorized access token again, which at least in case of mobile apps, is a bad (not acceptable) user experience.
Solution: This is where the refresh token comes in. It is again a random unpredictable token that is also issued to the app along with the access token in the first place. This refresh token is a very long-lived special token, which makes sure that as soon as the access token expires, it requests the server for a new access token, thus removing the need for the user to re-enter his login credentials to retrieve a new authorized access token, once an existing one has expired.
Now you may ask, Bob can have access to the refresh token as well, similar to the way he compromised the access token. YES. He can. However, now it becomes easy to identify such an incidence, which was not possible in the case of an access token alone, and take the necessary action to reduce the damage done.
How?
For every authenticated user (in case of a mobile app, generally), a one to one mapped refresh token and access token pair is issued to the app. So at any given point in time, for a single authenticated user, there will be only one access token corresponding to a refresh token. Now assume that if Bob has compromised the refresh token, he would be using it to generate an access token (because access token is the only thing which is authorized to access resources through the APIs). As soon as Bob (attacker) requests with the newly generated access token because Alice's (genuine user) access token is still valid, the server would see this as an anomaly, because for a single refresh token there can be only one authorized access token at a time. Identifying the anomaly, the server would destroy the refresh token in question and along with it all, it's associated access tokens will also get invalidated. Thus preventing any further access, genuine or malicious, to any authorization requiring resources.
The user, Alice, would be required to once again authenticate with her credentials and fetch a valid pair of a refresh and access tokens.
Of course, you could still argue that Bob could once again get access to both refresh and access tokens and repeat the entire story above, potentially leading to a DoS on Alice, the actual genuine customer, but then again there is nothing like 100% security.
Also as a good practice, the refresh token should have an expiry, although a pretty long one.
I believe for this scenario you could work with the access token alone, making
life easier for your clients but keeping the security benefits of a refresh token.
This is how it would work:
When your user logs in with credentials (username/password) you return a
short-lived JWT. You also create a db record where you store:
JWT id
user id
IP address
user agent
a valid flag (defaults to TRUE)
createdAt
updatedAt
Your client submits the JWT in every request. As long as the JWT hasn't expired,
it has access to the resources. If the JWT expired, you refresh it
behind the scenes and return both the resource and an additional X-JWT header
with the new JWT.
When the client receives a response with an X-JWT header, it discards the
old JWT and uses the new one for future requests.
How refreshing the JWT works on the server
Look for the matching db record using the JWT id.
Check if the valid flag is still true, otherwise reject.
Optionally, you can compare the request IP address and user agent against
the stored IP address and user agent, and decide to reject if something looks
fishy.
Optionally, you can check the db record's createdAt or updatedAt fields, and
decide not to refresh if too much time has passed.
Update the updatedAt field in the db record.
Return the new JWT (which is basically a copy of the expired JWT, but with an extended expiration time).
This design would also give you the option to revoke all tokens for a user (for
example, if the user loses his phone or updates his password).
Benefits:
Your client never has to check expiration times or make refresh token
requests, all it does is check for an X-JWT header on responses.
You can add custom refresh logic based on IP address, user agent, max-token
age, or a combination of those.
You can revoke some or all tokens for a user.
If I WERE to use a refresh token, wouldn't it still be beneficial to have a long term expiration for good practice on that token as well?
Refresh Tokens are long-lived, Access Tokens are short-lived.
If I WERE to use a refresh token, would that token be persisted with the userId and/or JWT token?
It would be persisted as a separate token on the client, alongside JWT but not inside JWT. UserID/UID can be stored inside the JWT token itself.
When I update my token every 1 hour, how does this work? Will I want to create an endpoint that takes in my JWT token or my refresh token? Will this update the expiration date of my original JWT token, or create a new token?
Yes, you need a separate service that issues and refreshes token. It won't update the expiration of the existing JWT Token. A token is simply JSON field-value pairs that are base64 encoded. So changing the data, changes the output. The token also has the issue date, which will at the very least change on every fresh issue (refresh). So every token will be unique and new. The old tokens will auto-expire, hence you need expiration on all Access Tokens, otherwise they will linger around forever.
The other answer here states that old tokens get destroyed when you issue a new token. That's simply not the case. Tokens cannot be destroyed. In fact, you can harvest hundreds of tokens by constantly contacting the auth server and asking for new fresh tokens using your Refresh Token. Each of those Access Tokens will be valid till their expiry. So expiry is imperative, and it should be short.
Is there really the need for a refresh token given these details? It seems that If the user is just using a JWT token to grab a new token (per the link above) then the refresh token is obsolete.
JWT tokens have client claims. For example is_manager:true claim on a JWT token might allow access to manager-level features. Now if you decide to demote the user from manager to contractor, that won't take effect immediately. The user may still be using the old token. Finally when that expires, he hits the auth server to refresh his token. The auth server issues a new token without the managerial claim and the user won't be able to access managerial features any more. This creates a window during which the user's claims are not in sync with the server. This again explains why Access Tokens should be short-lived so sync'ing can happen often.
Essentially you are updating the authorization checks every 15 minutes, instead of checking them on every single request (which is how typical session-based auth works). If you want real-time permissions instead of every-15-minute refreshes, then JWT may not be a good fit.