Our current APIs and password protected websites currently use access tokens to retrieve information such as storage URLs and GA tracking keys. This works really well for our password protected websites and APIs where we utilize Authorization code flow with PKCE and client credentials flow.
However, in the case of our non-password protected websites(public), which also require access tokens to get the correct information from our APIs, we are unsure which OAuth flow to use.
Since the website(SPA) would authenticate as an application, rather than a user, it would feel natural to employ the client credentials flow here as well. However, since the site is purely front-end, it wouldn't feel right to simply expose the client id and client secret in the browser. We are aware that the access token would have to be easy to get(since it would have to be done in the browser), and thus wouldn't be "safe", but we are planning to heavily limit the access(specific read operations) which would be granted by such a token.
Any tips or recommendations going forward would be appreciated.
If the web page is public then any data shown is public, so you should not care if a hacker accesses the information. Personally I would use anonymous API routes for public data, that do not require credentials, from either secured or unsecured clients.
If at an API level you want to require tokens, the SPA could call a utility API that implements the client credentials flow, in order to keep secrets out of the browser. This could return an access token or cookie to the browser that is then used by the SPA to call APIs. But any hacker could then use HTTP tools to grab the token or cookie, then play it against APIs directly, so there is zero security here.
OAuth scopes are likely to be essential for protecting the data correctly. If your public info is a list of products then unsecured clients might access it with the first scope. and secured clients with the second. APIs then need to ensure that the first scope can never be used to access secured resources:
scope=public
scope=products_read
To prevent unexpected behaviour you need to design an end-to-end solution, and think through the angles. This must ensure that introducing low privilege tokens (which have zero security) does not introduce its own vulnerabilities.
Consider an API that a client accesses directly (machine to machine) and that doesn't require user-specific authentication. The way I understand it, in client_credentials, the client must store a client_id and client_secret that it uses to acquire and refresh tokens. With an API key, the client just stores the key. What makes OAuth more secure in this case? It would appear to me that if the API key is never compromised, no attacker could pose as the intended client. And if the API key is compromised, it is effectively the same as compromising the client_id and client_secret, which an attacker would be able to use to obtain tokens and access the data in the API, posing as the client.
edit: clarified this is a machine-to-machine call
TLDR;
The difference comes down to direct access vs. delegated access.
OAuth allows you to make delegated access. The benefits of delegated access don't change if there is a user involved or not. The same arguments that make the OAuth Authorization code flow attractive for user-to-machine access, apply to the OAuth Client credentials flow for machine-to-machine access.
Ask yourself, do you want the resource server to handle client credentials or not?
On confidential clients for machine-to-machine access, the cost of delegated access vs. direct access may very well outweigh the benefits. That's why so many APIs still use API keys. You'll have to decide that for your individual use case.
Differences
In the OAuth client credentials flow, the client sends an access token to the resource server, which it got beforehand by the authorization server after presenting its client ID and secret. The resource server never sees the client secret. With an API key, the client sends the key with every request.
OAuth adds an additional layer of indirection with the authorization server, such that the credentials themselves never get transmitted to the resource server. This allows the authorization server to give the client only access for a limited amount of time or with limited permissions, without ever needing to change the actual client credentials. It also allows to revoke access tokens without revoking the credentials themselves. For multiple instances of a client this allows you to revoke access for some but not all.
Of course this all comes at the cost of a more complex implementation, and an additional roundtrip from the client to the authorization server.
I won't touch on transmission (URL, header, body, etc.) or format (random string, signed JWT, etc.), since these can be the same for access tokens just as for API keys.
Another, maybe not so obvious, advantage of OAuth is having a clear spec that libraries, documentation and discussions can be based on. With direct access there is no single best practice and different people may understand different things when referring to direct access methods like API keys.
With client credential flow your Client Id and Client Secret are sent to the authorization server to get back an access token. For all subsequent request to the API/resource servers, you pass the access token and not the client credentials themselves. The access token is usually a JWT, which is a set of encoded claims including the token expiry (exp), not before (nbf), token issuer (iss), authorized party (azp), roles, permissions, etc.
This has a number of advantages over a simple API Key approach. e.g.
If the access token (which is included in requests to the API/resource server) is compromised, it's only valid until it expires (which is typically ~1 day for M2M tokens). If an API Key is compromised, it can be used indefinitely or until it's explicitly blocked by the API/resource server.
JWT access tokens are encoded JSON objects that contains a number of fields (a.k.a. claims) that can be used for fine grained authorization e.g. roles, permissions, grant type, authorized party etc. An API Key is generally opaque and is all or nothing when it comes to auth.
You machine tokens can get validated and authorized on the API/resource servers the same way as your user tokens, so you don't end up with multiple auth implementations on the back-end.
OAuth Client Credentials Flow
What is the security difference between API Keys and the client credentials flow of OAuth?
OAuth client credentials flow is not meant to be used by public clients, just between machines.
From auth0.com/docs:
Client Credentials Flow
With machine-to-machine (M2M) applications, such as CLIs, daemons, or services running on your back-end, the system authenticates and authorizes the app rather than a user. For this scenario, typical authentication schemes like username + password or social logins don't make sense. Instead, M2M apps use the Client Credentials Flow (defined in OAuth 2.0 RFC 6749, section 4.4), in which they pass along their Client ID and Client Secret to authenticate themselves and get a token.
So, I am not sure what is your scenario, but I will assume in my reply that you are referring to public clients.
If it is in the public client code, then it is public
The way I understand it, in client_credentials, the client must store a client_id and client_secret that it uses to acquire and refresh tokens.
Yes, it needs to be stored in the client code for the client to be able to obtain the OAuth token.
If you use the client_secret from a web app or mobile app you are making it public, therefore not a secret anymore.
Extracting secrets from public clients
For example, in a web app all it takes to extract the client_secret is to hit F12 in the browser and search for it, thus how much time can this take?
Now, in a mobile app, some may think it's secure because they are compiled into a binary but is almost as easy as it is in the browser, because we have several open-source tools that can help us with this task, like the MobSF framework, and on Linux, you can even achieve this with the strings command. Using the MobSF to perform static binary analysis on the mobile app binary allows for anyone without hacking knowledge to easily extract the client_secret in minutes, just like I show in my article How to Extract an API key from a Mobile App with Static Binary Analysis:
The range of open source tools available for reverse engineering is huge, and we really can't scratch the surface of this topic in this article, but instead, we will focus in using the Mobile Security Framework(MobSF) to demonstrate how to reverse engineer the APK of our mobile app. MobSF is a collection of open-source tools that present their results in an attractive dashboard, but the same tools used under the hood within MobSF and elsewhere can be used individually to achieve the same results.
So, the process of extracting the api-key in my article is the same you will use to extract the client_secret or any other string of your interest in the mobile app binary.
OAuth or API Key?
What makes OAuth more secure in this case? It would appear to me that if the API key is never compromised, no attacker could pose as the intended client. And if the API key is compromised, it is effectively the same as compromising the client_id and client_secret, which an attacker would be able to use to obtain tokens and access the data in the API, posing as the client.
If used from a public client neither are secure, because if read my linked article, you understand by now how easy is to bypass an API Key or extract the client_secret and client_id.
So, if your client is public you should not use the OAuth client credential flow, thus you need to go with the insecure API key approach or you can be more diligent and try to apply defence-in-depth approaches, but this will depend if the API clients are only web apps or mobile apps or both.
If your API clients are only web apps I invite you to read my answer to the question Secure API data from calls out of the app, especially the section dedicated to Defending the API Server.
In the case the API clients are only mobile apps then I recommend you to read this answer I gave to the question How to secure an API REST for mobile app?, especially the sections Securing the API Server and A Possible Better Solution.
On the other hand, if your API clients are both a web app and a mobile app I recommend you to apply the security measures more relevant to you from both answers linked above.
Remember that security is always about adding as many layers of defences as you can afford or it's required by law. Even in the past century, the castles were built with a lot of different security defence layers, thus this is nothing new to the digital era.
Do You Want To Go The Extra Mile?
In any response to a security question I always like to reference the excellent work from the OWASP foundation.
For APIS
OWASP API Security Top 10
The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.
For Mobile Apps
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.
For Web Apps
The Web Security Testing Guide:
The OWASP Web Security Testing Guide includes a "best practice" penetration testing framework which users can implement in their own organizations and a "low level" penetration testing guide that describes techniques for testing most common web application and web service security issues.
I found one interesting article that has this illustration:
It says that:
API Gateway verifies access token for all incoming requests via introspection
But what does this mean?
it says that gateway goes to authorization server and validates token (JWT).
why is that needed?
if gateway has authorization server's public key, it can check token validity using signature just like every backend service, why is introspection needed and how is it done?
Depending on your Identity provider it can be done either way but there are trade offs.
If you validate the token locally then yes it can use public keys to do that and that's very efficient way, however downside is that if the token or signing keys are revoked then your token is still valid. With Remote check you have to bear the http overhead but that is more reliable.
Normally tokens are kept short lived and validated locally. But if your access token are long lived, your application require strict access controls or library doesn't support local validation then it's a good idea to check them remotely
I believe you are looking at this document.
What I understood from this Secure API Gateway is that the gateway is responsible for introspection and the back-end services will only check the token signature, which is less secure than introspection, but still a layer of security.
Introspection is necessary to validate the token information against the Authorization Server.
This is more secure, because the system can ensure that the token received is not malicious, expired and it is from an known source.
The details on how it is done are explained in RFC 7662.
Yes, the gateway could validate the token signature if it has access to the certificate.
I can't really tell why they choose the back-end server to do it, probably a project decision.
API Gateway primarily meant for routing the incoming calls to the corresponding MicroService Clusters.
In addition to that,it can also play a role to validate the token, so that only valid traffic is routed to the downstream servers (filtering malicious traffic).
The level of validation could be up to the product owner/architect decision. It could be as simple as validating against the list of in memory cached token or in depth validation on set of claims, digital signature verification, expiry time, audience claim etc.
You can view the set of claims inside the token using JWT decoder like https://devtoolzone.com/decoder/jwt
I would like to develop RESTful API for my web application. Client have to be clear JS + HTML and user have to somehow authenticate in system. I've read few articles about security and RESTful API but some point still are not clear for me. As I've understood at the first step user have to provide his credentials to server. After that server have to validate user credentials and if they are valid, sent some encoded token to user (assume it will be [user key]:[user IP]:[token creation time]). After user authentication client have to sent this this key with each API call. That's how I've understood RESTful API authentication principes.
With such solution I assume that token can be stolen and another user can access to secured user data (even if IP is included to access token and there will be validation on each request). For this purpose I plan to include token creation time but as I understand with such solution I have to renew access token each time when it expiring - in this case it's not clear for me how to implement "remember me" functionality.
It's not clear for me how to implement 100% safe authentication for my RESTful API. Maybe I'm missing something. Or maybe my understanding of authentication principes is wrong.
It depends from what authentication scenario you are using. For example when dealing with in ASP.NET MVC + REST with Basic Authentication it will produce for you token which is in the fact Base64 encoded string '{username}:{password}'. And you are right it could be stolen, that's why for Basic Auth HTTPS is must, as token goes throw Authentication header with Basic schema.
For REST security most suitable and secure are OpenId and OAuth. Just don't reinvent wheel and use already existing standards. OAuth in compare to OpenID includes not only authentication but authorization as well. OAuth already describes all nuances with token renew and token creation time and so on.
Now practical how to implement OAuth in REST. First of all read standard. For your case read with attention Implicit Grant flow, because standard has multiple flows for different client with different trust level and security.
https://www.rfc-editor.org/rfc/rfc6749
And after that you can try some already implemented library in technological stack you are using either Java or .NET. For client it is not so important to use library in compare but for server implementation
About potential security problem read here https://www.rfc-editor.org/rfc/rfc6749#section-10.
Some think that OAuth 2.0 is less secure that OAuth 1.0, and it is also dependant from token format. Anyway access token should be passed in HTTP Header and through HTTPS as well as clientid should be stored and passed securely.
A typical recommendation for securing a REST API is to use HTTP Basic Authentication over SSL. My question is, should HTTP Basic Authentication only be used to authenticate the client (ie. the app accessing the API), or can it also be used to authenticate the user (the consumer of the app)?
It seems most APIs have to deal with both, as almost all web services employ some sort of user accounts. Just consider Twitter or Vimeo—there are public resources, and there are private (user specific) resources.
It seems logical that a simple REST API could do both client and user authentication at the same time using using HTTP Basic Authentication (over SSL).
Is this a good design?
By authenticate the client you probably mean the usage of API Key, this mechanism is used to track the concrete application/client. The second thing is that it gives you the possibility to disable the application by disabling the key, for example when client's author removes his account from the service. If you want to make your API public then it is a good idea.
But you need to remember that it gives you no real protection, everybody can download the client and extract that key.
I would not recommend to use Basic Authentication for API authentication. When it comes to authentication then you should consider that the application (client) developer has to implement its side of the authentication, too. Part of that is not only authentication itself but also how to get credentials and even much more than that.
I recommend to make use of an established authentication standard that ships with client libraries for the most popular programming languages. Those libraries make it much more likely that developers are going to adapt your API, because they reduce implementation effort on the client side.
Another important reason for using authentication standards is that they make developers (and others) more confident in the security of your authentication system. Those standards have been audited by experts and their weaknesses and strengths are well known and documented. It is unlikely that you are going to develop a nearly as solid authentication flow unless you are a security expert :-).
The most established standard in this field is OAuth but you can find alternatives by searching for "oauth alternatives".
How does OAuth help you with your problem setting?
In OAuth 2, the application client has to obtain an access token for a user before accessing any protected resource. To get an access token, the application must authenticate itself with its application credentials. Depending on the use-case (e.g. 3rd party, mobile) this is done in different ways that are defined by the OAuth standard.
An access token should not only represent a user but also which operations may be used on what resources (permissions). A user may grant different permissions to different applications so this information must somehow be linked to the token.
How to achieve such a semantic for access tokens however is not part of OAuth - it just defines the flow of how to obtain access tokens. Therefor, the implementation of the access token semantic is usually application specific.
You can implement such token semantic by storing a link between an access tokens and its permissions in your backend when you create the access token. The permissions may either be stored for every user-application combination or just for every application, depending on how fine-granular you want things to be.
Then, each time that an access token is processed by the API, you fetch this information and check whether the user has sufficient permissions to access the resource and to perform the desired operation.
Another option is to put the permission information into the access token and to sign or encrypt the token. When you receive the access token, you verify or decrypt it and use the permissions that are stored in the access token to make your decision. You may want to have a look on Json Web Tokens (JWT) on how to accomplish that.
The benefit of the later solution is better scalability and less effort during backend implementation. The downside of it are potentially larger requests (especially with RSA encryption) and less control over tokens.