I know this question has been asked a lot of times in varying shapes and forms, but I'm still quite unclear about a few things. It's confusing how resources about this topic around the Web refer to "user" without clear context. API keys are issued for users, and so are access tokens, but I don't think API key users, are the same as access token users.
Do you need to have different API keys for different instances of end user clients? For example if I build a mobile app for a third party API, does each instance keep their own API key? I don't think they do, but how do I tie API keys, and access tokens together to say that a certain request comes from this particular instance of an app authorized by a known user? If I were the auth provider, do I have to keep track of each of those?
API keys, and access tokens are usually represented by a pair of public, and shared keys. As a service provider (server side), which one do I use to verify the message I receive, the API key, or the access token? If I understand correctly, the idea is that each request should come with a signature derived from the secret part of the API key so that the server can check that it comes from a trusted client. Now what use do I have for access token secret? I know the access token is used to verify that a system user has authorized the app to carry operations on their behalf, but which part of the message does the access token secret be useful for?
Is a hash generated from a (secured) random number, and a time stamp salt a good API key generation strategy?
Are there (preferably open source, Java-based) frameworks that do most of these?
Let me try to answer as many of your queries as I can.
Apikey vs Access Token usage
First of all, apikeys are not used per user. Apikeys are assigned per
application (of a developer). A developer of a service signs up their application and obtains a
pair of keys.
On the other hand access tokens are issued for each
end-user in context of the usage (exception is Client Credential
grant).
Service providers can identify the application from the
apikey in use.
Service providers can identify the end-users using
access token attributes.
You should have any end-user APIs, that is an API that has end-user resources (data or context) associated, protected by 3 legged Oauth. So access token should be necessary for accessing those resources.
Developer-only resources can be protected by apikey or two-legged Oauth. Here I am referring to Oauth2 standards.
Oauth1 is preferred when there is no HTTPS is supported. This way the shared secret is not sent over unprotected channel. Instead it is used to generate a signature. I strongly suggest Oauth2 over HTTPs and avoid Oauth1 for ease of use. You and your API consumers would find Oauth2 to be much more simpler to implement and work with. Unless you have a specific reason to use Oauth v1
As a service provider you can use Apigee's Edge platform that provides Oauth 1 and 2. It is not opensource. However you can use it for free, until you need some high TPS or higher SLAs.
Related
My team and I have been working on a web application for our clients that uses JSON web tokens for authentication and authorization. Using Azure AD as our identity provider, we verify a user's identity and generate a signed JWT with the user's permissions in it. The JWT then gets included in the authorization header of all subsequent requests to the APIs. Pretty standard stuff as far as JWTs go.
We're now being asked to provide the capability to link directly into our system from another third-party web application without forcing the user to reauthenticate. I'm trying to figure out if there's a way to do so without creating a massive security loophole.
The way I picture this working would be to implement an endpoint for programmatic authentication in our system that accepts a cryptographically signed payload with an API key and the user's ID or email address. The third-party system would have a private key with which to sign the payload, and we'd have a public one to verify the signature. If the request is legitimate, we'd issue a token for the specified user, and they could use that to link to whatever they like.
I'm already getting yelled at by at least one person that this is a complete joke from a security standpoint because, among other things, it completely bypasses AAD authentication. I believe the third-party system in question does use AAD for authentication, but that's not really relevant either way because we're trusting them implicitly whether they've authenticated their users or not. Either way I take his point.
I'm not a security expert and I don't claim to know whether there even is a proper way to do this kind of thing, but from my vantage it doesn't really seem all that much less secure than any other mechanism of authentication and authorization using JWTs. Is that true? Are we nuts for even trying? Is there a way to do it that's more secure? What should I know about this that I demonstrably don't already?
Thanks in advance for the help. At the very least I hope this spurs some helpful conversation.
Single Sign-On (SSO) enables users to enter their credentials once to sign in and establish a session which can be reused across multiple applications without requiring to authenticate again. This provides a seamless experience to the user and reduces the repeated prompts for credentials.
Azure AD provides SSO capabilities to applications by setting a session cookie when the user authenticates the first time. The MSAL.js library allows applications to leverage this in a few ways.
MSAL relies on the session cookie to provide SSO for the user between different applications.
Read more in this documentation.
Consider an API that a client accesses directly (machine to machine) and that doesn't require user-specific authentication. The way I understand it, in client_credentials, the client must store a client_id and client_secret that it uses to acquire and refresh tokens. With an API key, the client just stores the key. What makes OAuth more secure in this case? It would appear to me that if the API key is never compromised, no attacker could pose as the intended client. And if the API key is compromised, it is effectively the same as compromising the client_id and client_secret, which an attacker would be able to use to obtain tokens and access the data in the API, posing as the client.
edit: clarified this is a machine-to-machine call
TLDR;
The difference comes down to direct access vs. delegated access.
OAuth allows you to make delegated access. The benefits of delegated access don't change if there is a user involved or not. The same arguments that make the OAuth Authorization code flow attractive for user-to-machine access, apply to the OAuth Client credentials flow for machine-to-machine access.
Ask yourself, do you want the resource server to handle client credentials or not?
On confidential clients for machine-to-machine access, the cost of delegated access vs. direct access may very well outweigh the benefits. That's why so many APIs still use API keys. You'll have to decide that for your individual use case.
Differences
In the OAuth client credentials flow, the client sends an access token to the resource server, which it got beforehand by the authorization server after presenting its client ID and secret. The resource server never sees the client secret. With an API key, the client sends the key with every request.
OAuth adds an additional layer of indirection with the authorization server, such that the credentials themselves never get transmitted to the resource server. This allows the authorization server to give the client only access for a limited amount of time or with limited permissions, without ever needing to change the actual client credentials. It also allows to revoke access tokens without revoking the credentials themselves. For multiple instances of a client this allows you to revoke access for some but not all.
Of course this all comes at the cost of a more complex implementation, and an additional roundtrip from the client to the authorization server.
I won't touch on transmission (URL, header, body, etc.) or format (random string, signed JWT, etc.), since these can be the same for access tokens just as for API keys.
Another, maybe not so obvious, advantage of OAuth is having a clear spec that libraries, documentation and discussions can be based on. With direct access there is no single best practice and different people may understand different things when referring to direct access methods like API keys.
With client credential flow your Client Id and Client Secret are sent to the authorization server to get back an access token. For all subsequent request to the API/resource servers, you pass the access token and not the client credentials themselves. The access token is usually a JWT, which is a set of encoded claims including the token expiry (exp), not before (nbf), token issuer (iss), authorized party (azp), roles, permissions, etc.
This has a number of advantages over a simple API Key approach. e.g.
If the access token (which is included in requests to the API/resource server) is compromised, it's only valid until it expires (which is typically ~1 day for M2M tokens). If an API Key is compromised, it can be used indefinitely or until it's explicitly blocked by the API/resource server.
JWT access tokens are encoded JSON objects that contains a number of fields (a.k.a. claims) that can be used for fine grained authorization e.g. roles, permissions, grant type, authorized party etc. An API Key is generally opaque and is all or nothing when it comes to auth.
You machine tokens can get validated and authorized on the API/resource servers the same way as your user tokens, so you don't end up with multiple auth implementations on the back-end.
OAuth Client Credentials Flow
What is the security difference between API Keys and the client credentials flow of OAuth?
OAuth client credentials flow is not meant to be used by public clients, just between machines.
From auth0.com/docs:
Client Credentials Flow
With machine-to-machine (M2M) applications, such as CLIs, daemons, or services running on your back-end, the system authenticates and authorizes the app rather than a user. For this scenario, typical authentication schemes like username + password or social logins don't make sense. Instead, M2M apps use the Client Credentials Flow (defined in OAuth 2.0 RFC 6749, section 4.4), in which they pass along their Client ID and Client Secret to authenticate themselves and get a token.
So, I am not sure what is your scenario, but I will assume in my reply that you are referring to public clients.
If it is in the public client code, then it is public
The way I understand it, in client_credentials, the client must store a client_id and client_secret that it uses to acquire and refresh tokens.
Yes, it needs to be stored in the client code for the client to be able to obtain the OAuth token.
If you use the client_secret from a web app or mobile app you are making it public, therefore not a secret anymore.
Extracting secrets from public clients
For example, in a web app all it takes to extract the client_secret is to hit F12 in the browser and search for it, thus how much time can this take?
Now, in a mobile app, some may think it's secure because they are compiled into a binary but is almost as easy as it is in the browser, because we have several open-source tools that can help us with this task, like the MobSF framework, and on Linux, you can even achieve this with the strings command. Using the MobSF to perform static binary analysis on the mobile app binary allows for anyone without hacking knowledge to easily extract the client_secret in minutes, just like I show in my article How to Extract an API key from a Mobile App with Static Binary Analysis:
The range of open source tools available for reverse engineering is huge, and we really can't scratch the surface of this topic in this article, but instead, we will focus in using the Mobile Security Framework(MobSF) to demonstrate how to reverse engineer the APK of our mobile app. MobSF is a collection of open-source tools that present their results in an attractive dashboard, but the same tools used under the hood within MobSF and elsewhere can be used individually to achieve the same results.
So, the process of extracting the api-key in my article is the same you will use to extract the client_secret or any other string of your interest in the mobile app binary.
OAuth or API Key?
What makes OAuth more secure in this case? It would appear to me that if the API key is never compromised, no attacker could pose as the intended client. And if the API key is compromised, it is effectively the same as compromising the client_id and client_secret, which an attacker would be able to use to obtain tokens and access the data in the API, posing as the client.
If used from a public client neither are secure, because if read my linked article, you understand by now how easy is to bypass an API Key or extract the client_secret and client_id.
So, if your client is public you should not use the OAuth client credential flow, thus you need to go with the insecure API key approach or you can be more diligent and try to apply defence-in-depth approaches, but this will depend if the API clients are only web apps or mobile apps or both.
If your API clients are only web apps I invite you to read my answer to the question Secure API data from calls out of the app, especially the section dedicated to Defending the API Server.
In the case the API clients are only mobile apps then I recommend you to read this answer I gave to the question How to secure an API REST for mobile app?, especially the sections Securing the API Server and A Possible Better Solution.
On the other hand, if your API clients are both a web app and a mobile app I recommend you to apply the security measures more relevant to you from both answers linked above.
Remember that security is always about adding as many layers of defences as you can afford or it's required by law. Even in the past century, the castles were built with a lot of different security defence layers, thus this is nothing new to the digital era.
Do You Want To Go The Extra Mile?
In any response to a security question I always like to reference the excellent work from the OWASP foundation.
For APIS
OWASP API Security Top 10
The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.
For Mobile Apps
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.
For Web Apps
The Web Security Testing Guide:
The OWASP Web Security Testing Guide includes a "best practice" penetration testing framework which users can implement in their own organizations and a "low level" penetration testing guide that describes techniques for testing most common web application and web service security issues.
There are multiple partied involved in OAuth2 conversation. Consider the
following diagram from the article here
Consider that I have an application that has data for restaurants and has APIs related to it. Let's call is restaurants APIs. Let us assign some role to each party in context of this example
User - our chefs, who have some recipes in restaurant
Application - Web client written in HTML5, JS, CSS that our Users use to interact with APIs
OAuth Endpoint - Google (who acts as Authorization Server)
API - My application API keeping all data for chefs
The workflow for Implicit (as per above diagram in the link) states the Application gets the access token and then the Application(browser) calls API (my application with chefs recipes) and gets the data back.
Questions
Shouldn't I secure my application endpoints or rather just believe the accesssTokens? Yes, the trust is established between Application and OAuth Endpoint (Google), but there is no trust developed API and Application by confirming the validity of accessToken with OAuth Endpoint (Google)?
If I should secure my application API endpoints, shall I have a /login endpoint for my APIs where my application accepts accessTokens, validate and create a JWT based headers for clients to use for further communication with protected resources like /recipes.
Looking forward to your ideas here.
Thanks in advance
TL;DR - don't blindly trust the access tokens. Ask Google to reveal the user/email associated with them and the client ID that was used when generating them. You can still provide a /login endpoint for scalability purposes mostly.
Let's deal with the core security first
OAuth is a delegation protocol, not an authentication protocol. To quote from the OAuth website:
The OAuth 2.0 specification defines a delegation protocol [...] OAuth is used in a wide variety of applications, including providing mechanisms for user authentication. [...] Let's say that again, to be clear:
OAuth 2.0 is not an authentication protocol.
Because it's not an authentication protocol, your app/API never learns who the user is. It just gets a token. Delegation in this context means that OAuth lets App A request access to resources in App B that belong to a User, by having the User authenticate to App B and then passing the token back to App A. In your example, it can provide your web app with access to Google resources (email, photos, etc. - depending on the required scopes) that are owned by the Users (chefs).
Note that this isn't what you are doing here, since you're accessing resources managed by your app, not by Google. In particular, as you correctly identified, the access token means nothing to your API. I could just as well give it a random string.
You might be tempted to use the following scheme:
Implement the implicit scheme as described in your question.
Have the API server validate the access token with Google, and ask Google for the name or email associated with the token. This will be the identity of the user who actually logged in to Google, and you can then decide whether or not to grant permission to that user.
The problem with this approach is that many apps use OAuth with Google, and so many apps will have Google access tokens that don't belong you app. How can you tell the difference?
You can ask Google, when you present it with the access token, to also provide you with the client ID that was provided when this token was generated (see how your diagram indicates that the client ID is sent?). Since that client ID uniquely identifies your app, then your API can tell that it's been given tokens that only came from your app. Note that this critical part of the OAuth flow is very different in mobile apps which is why the implicit flow should not be used with mobile apps (but it's fine with web apps).
Note that your client ID should be considered common knowledge (e.g. it's found in the .js files on the machines performing this flow), but it cannot be spoofed because as part of the OAuth flow, the user's browser will be redirected to a URL that is pre-configured in Google and belongs to your app. So even if a malicious app uses your client ID, Google will still send the token to your app.
Other practicalities
The above requires you to issue a call to Google on every API call, or at least cache the valid access tokens (which means you keep state, which is a bummer for scalability). If you want to avoid this, you can create a /login endpoint which generates a JWT. Note that you'll still need to validate the access tokens upon login.
I have a question regarding how I should architecture a REST API using access token and API keys.
I have an API that needs authentication. I want to enable two use cases:
The user logs into the interface using OAuth2 (password grant), and is granted a temporary access token. This token is used to authenticate the user. Therefore, the UI, that itself using the API, can fetch data and display it.
I also want the user to have an API key to do the same calls, but in its application. Obviously, contrary to the access token, I want the API key to be long lived. Also, contrary to the access token that is tied to a given user (if we introduce a team mechanism, each user will have different access token, although they access the same resources), the API key should be unique to the project.
While similar, I'm not sure about how should I architecture that. I think that, internally, both API keys and access tokens should be stored in the same table, but API keys having no expiration time. Am I right?
One thing I'm not sure also is the concept of client. It seems that in the spec, the client is more like an external application. However may I actually use this concept here?
For instance, each "project" is actually a different client (although the client here is the same application, not an application created by a third-party developer).
Therefore, if user A creates an account on the system, a client A will be automatically created, with an access token tied to the client A, with a long-lived access token (aka API key). This can be used to perform API calls directly on his code, for instance.
Then, if user A logs into the dashboard, a temporary access token will be created, but this time with no application, but tied to the user, with a short life.
Does this sound sane? Have anyone already implemented such a thing?
Thanks!
I think you should not consider the "API keys" a substitute of the access token.
You will have to use an access token anyway to bear the authentication between requests, so what you're actually modelling with your "API keys" is not a replacement of the usual bearer token, but rather a different client that provides other grant types to request a token with.
The flow I'd personally implement is the following:
The user authenticates with the password grant type with a common client for every user (i.e. your "web app" client, which is public, i.e. it doesn't have a client_secret).
The user can then create its own client. As per OAuth2 specs, these are not public, so they will consists of a client_id and a client_secret. These are what you call "API keys".
A user will then be able to request an access token via their client, with any given grant type you want to support (e.g. direct client credentials, authorization code, implicit, third parties, etc.). You will have to stress quite a bit about the due safety practices on how to handle the client credentials.
Obviously, you will have to implement your OAuth2 server in such a way that clients can belong specific users, and have different acceptable grant types (i.e. you may not want to allow the password grant usage with a user client, while you may want to disallow any grant type other than the password one for your web app client).
You will then be able to define tokens TTLs, or lack thereof, on a per client or per grant type basis (e.g. access token requested via password grant, only usable by web app client, will have a short TTL, while authorization code grant will provide long lived tokens).
I would advise against complete lack of TTL, though, and rather use the refresh_token grant type to renew expired access tokens.
Furthermore, you'll probably have to define an authorization system of some some sort (ACL, RBAC, whatever), to define which client can do what. This means each access token should contain a reference to the client used for its creation.
So, to sum it up, here are the relations:
User has a Client.
Client has a User.
Client has many Token.
Token has a Client.
Token has a User.
YMMV on bidirectionals.
You should be able to implement everything I described with the most common OAuth2 servers implementations of any given platform.
TL;DR: "API keys" are actually OAuth2 clients.
I wrote a post about the way to use access tokens for RESTful applications: https://templth.wordpress.com/2015/01/05/implementing-authentication-with-tokens-for-restful-applications/. Perhaps can this give some hints.
To answer your questions, I think that we need to have something homogeneous. I mean all your authentication mechanisms should be based on access tokens. Your API keys would allow you to get an access token that would be actually used for authentication.
As far as I understand, you have two kinds of users of your applications:
End-users using the Web UI (login with password through OAuth2)
Applications (login with API keys)
So I would implement these two kinds of users and make them the ability to get access tokens. Access tokens will be used in both cases to access the RESTful services.
In addition, I think that this answer can give you some other hints: Securing my REST API with OAuth while still allowing authentication via third party OAuth providers (using DotNetOpenAuth).
Hope it answers your question.
Thierry
Thank you for your answer.
I'm actually quite experience with OAuth2 itself, my question was more targeted to API keys. I like the idea of an API key exchanging an access token but I think that does not work. The API key is fixed and does not change, while the access token can expires.
The question is: how the app can know if this is an access token or API keys. I mean, ok, let's say that in my database, each user has an "api_key" column in their database.
Contrary to an access token, the api_key does not expires (although the user can eventually rotate it). What I want, as I told, is homogeneous handling of authentication.
Case 1: my own web app do API calls
The workflow is as follow, using OAuth2:
User enters his mail/password.
Authorization server returns a temporary access token (eg.: "abc").
In the web app, all API calls are done using this token. For instance: "/payments/1" with Authorization header: "Bearer abc".
Nice and simple.
Case 2: the user has an API key, that does not expire and can be used privately in their own app
Obviously, the authorization mechanism must stay the same. So:
User goes into his account, and read that his API key is "def".
In their server code, they can do the same call, with same authentication mechanism. So he can call "/payments/1" with Authorization: "Bearer def".
And it must work. As you can see, nothing has changed in both examples. They access the same resource, same authorization mechanism... but in one case we have an access token and in other case we have an API key. And I have no idea how I should implement that both from a database point of view and in the authorization code.
One potential idea I had is using different auth mechanism. For OAuth, it would be "Authorization: Bearer accessToken", while for API it would be a Basic authentication: "Authorization: Basic apiKey".
Does this sound good?
A typical recommendation for securing a REST API is to use HTTP Basic Authentication over SSL. My question is, should HTTP Basic Authentication only be used to authenticate the client (ie. the app accessing the API), or can it also be used to authenticate the user (the consumer of the app)?
It seems most APIs have to deal with both, as almost all web services employ some sort of user accounts. Just consider Twitter or Vimeo—there are public resources, and there are private (user specific) resources.
It seems logical that a simple REST API could do both client and user authentication at the same time using using HTTP Basic Authentication (over SSL).
Is this a good design?
By authenticate the client you probably mean the usage of API Key, this mechanism is used to track the concrete application/client. The second thing is that it gives you the possibility to disable the application by disabling the key, for example when client's author removes his account from the service. If you want to make your API public then it is a good idea.
But you need to remember that it gives you no real protection, everybody can download the client and extract that key.
I would not recommend to use Basic Authentication for API authentication. When it comes to authentication then you should consider that the application (client) developer has to implement its side of the authentication, too. Part of that is not only authentication itself but also how to get credentials and even much more than that.
I recommend to make use of an established authentication standard that ships with client libraries for the most popular programming languages. Those libraries make it much more likely that developers are going to adapt your API, because they reduce implementation effort on the client side.
Another important reason for using authentication standards is that they make developers (and others) more confident in the security of your authentication system. Those standards have been audited by experts and their weaknesses and strengths are well known and documented. It is unlikely that you are going to develop a nearly as solid authentication flow unless you are a security expert :-).
The most established standard in this field is OAuth but you can find alternatives by searching for "oauth alternatives".
How does OAuth help you with your problem setting?
In OAuth 2, the application client has to obtain an access token for a user before accessing any protected resource. To get an access token, the application must authenticate itself with its application credentials. Depending on the use-case (e.g. 3rd party, mobile) this is done in different ways that are defined by the OAuth standard.
An access token should not only represent a user but also which operations may be used on what resources (permissions). A user may grant different permissions to different applications so this information must somehow be linked to the token.
How to achieve such a semantic for access tokens however is not part of OAuth - it just defines the flow of how to obtain access tokens. Therefor, the implementation of the access token semantic is usually application specific.
You can implement such token semantic by storing a link between an access tokens and its permissions in your backend when you create the access token. The permissions may either be stored for every user-application combination or just for every application, depending on how fine-granular you want things to be.
Then, each time that an access token is processed by the API, you fetch this information and check whether the user has sufficient permissions to access the resource and to perform the desired operation.
Another option is to put the permission information into the access token and to sign or encrypt the token. When you receive the access token, you verify or decrypt it and use the permissions that are stored in the access token to make your decision. You may want to have a look on Json Web Tokens (JWT) on how to accomplish that.
The benefit of the later solution is better scalability and less effort during backend implementation. The downside of it are potentially larger requests (especially with RSA encryption) and less control over tokens.