I have a Single Page App application which is working based on RESTful APIs. Generally, all APIs have a route access which can be found while inspecting web application.
Although I have authentication mechanism based on user tokens, a hacker can find the API routes and use his given token to send many requests to APIs directly.
What is the best solution to prevent such behavior? I was thinking about CSRF, but as APIs are based on REST, and the project is a SPA, I think I should have another mechanism.
May you help me please?
You cannot authenticate the client application, it is not possible. If a user can send a request from an spa, because they have the credentials and the endpoints to send them to, they can use whatever client from Burp through ZAP or Postman to curl or whatever else to send the request.
Your API must be resilient, you should have rate limiting, user quotas, monitoring and secure operation practices in general on the server side based on your threat model to mitigate this risk.
In practice this might mean hosting the API in a way that's resilient to DoS on the network level, having a per-user request rate limit, identifying functionality that is a burden for the server for some reason (calls external services, sends email and so on) and protect/monitor those even more carefully. There is no one size fits all solution.
Related
All of the React Native Twitter Login Clients that I'm finding seem to be hard-coding the TWITTER_CONSUMER_KEY and TWITTER_CONSUMER_SECRET into the the client code, rather than relying on a server to generate tokens and/or a twitter redirect URL.
Is this safe? (e.g. couldn't a consumer then DOS the API with the TWITTER_CONSUMER_KEY, causing the app to be rate limited?)
Is this the correct way to do it?
Is there a better / more secure way?
According to twitter's documentation, it seems like this is NOT the correct way to do this:
"In the event that you believe that your API keys has been exposed, you should regenerate your API keys by following these steps" - Authentication best practices
Examples which specify that the consumer key/secret should be hardcoded:
https://rnfirebase.io/docs/v5.x.x/auth/social-auth#Twitter
https://github.com/GoldenOwlAsia/react-native-twitter-signin/blob/master/Example/TwitterButton.js#L14
Related questions:
Twitter consumer secret really a secret?
Is it a security vulnerability
Yes.
Your app can be rate limited or flagged as malware/spam etc.
Is there a better / more secure way?
Basically only to have your own site auth (oauth2) done correctly and proxy specific requests from your clients, after validation or a simplified locked down site API that is then translated to the Twitter API.
Why is this, Twitter app-only auth supports OAuth2, allows a secure negotiated handshake and then requests made using a Bearer token. In this mode you can make requests on behalf of your App, but without a logged in user. So can't post tweets or see private accounts or read DMs.
For user-auth, Twitter only support OAuth1 and both the App and User are authenticated, but using a model that assumed plaintext http, so can't share a single token. Every single request needs to be made using consumer key/secret and signing the request. So there isn't a way to do this from a javascript client safely.
Is this safe?
Absolutely not. A bad actor can get users to authenticate via Twitter to receive their token credentials and then use your app's consumer key/secret (which would be available in plain text) to masquerade as your app to do all kinds of nasty stuff.
Is this the correct way to do it?
Given the security vulnerability described above, no.
Is there a better / more secure way?
I'm currently in the process of trying to figure out how to securely achieve authentication with Twitter. This involved a lot of reading, but it appears as though it's simply not possible without your own backend. I'll try and explain why:
Your goal is to receive the user's email/Twitter-ID
To achieve (1), you need to send a request to the GET account/verify_credentials endpoint (https://developer.twitter.com/en/docs/twitter-api/v1/accounts-and-users/manage-account-settings/api-reference/get-account-verify_credentials).
To do (2), you need to provide an authorisation header, which is constructed out of several items, including the user's OAuth tokens as well as your app's consumer key/secret. More info here: https://developer.twitter.com/en/docs/authentication/oauth-1-0a/authorizing-a-request.
You retrieve the user's OAuth tokens using the 3-legged OAuth flow
described here: https://developer.twitter.com/en/docs/authentication/oauth-1-0a/obtaining-user-access-tokens. The first step of this process is to send a POST request to the oauth/request_token endpoint (https://developer.twitter.com/en/docs/authentication/api-reference/request_token).
This endpoint itself requires an authorisation header constructed using
your app's consumer key/secret.
Obviously you can't perform step (4) because that implies you would have your consumer secret available in the client; even if it's not hardcoded, it would have to be in memory at runtime, at some point
Once you have your own backend service, one option would be for your client app to open a browser and direct to an endpoint (let's call it /auth/twitter) on this service which will perform all the steps mentioned above.
This same service could also implement another endpoint (/auth/twitter/token) which handles requests to the callback URL, which you set in your Twitter app settings. This callback URL is used as part of the same 3-legged flow. This endpoint would have all the information needed to then go ahead and retrieve the user's email/Twitter-ID.
Finally, /auth/twitter/token can redirect to a custom URL which your client app would need to handle as part of its URL schemes. It can include enough information by way of parameters for your app to continue as needed post-auth.
I am using HERE api in both frontend and backend. If I try to put my app_id and app_code into the frontend code, it will be available to anyone seeing my site.
I can try to create a domain whitelist and put my domain in this. But still, if I set the HTTP header "Referer" to my domain, I am able to access the API from any IP.
So, what do I do?
The Difference Between WHO and WHAT is Accessing the API Server
Before I dive into your problem I would like to first clear a misconception about WHO and WHAT is accessing an API server.
To better understand the differences between the WHO and the WHAT are accessing an API server, let’s use this picture:
So replace the mobile app by web app, and keep following my analogy around this picture.
The Intended Communication Channel represents the web app being used as you expected, by a legit user without any malicious intentions, communicating with the API server from the browser, not using Postman or using any other tool to perform a man in the middle(MitM) attack.
The actual channel may represent several different scenarios, like a legit user with malicious intentions that may be using Curl or a tool like Postman to perform the requests, a hacker using a MitM attack tool, like MitmProxy, to understand how the communication between the web app and the API server is being done in order to be able to replay the requests or even automate attacks against the API server. Many other scenarios are possible, but we will not enumerate each one here.
I hope that by now you may already have a clue why the WHO and the WHAT are not the same, but if not it will become clear in a moment.
The WHO is the user of the web app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
OAUTH
Generally, OAuth provides to clients a "secure delegated access" to server resources on behalf of a resource owner. It specifies a process for resource owners to authorize third-party access to their server resources without sharing their credentials. Designed specifically to work with Hypertext Transfer Protocol (HTTP), OAuth essentially allows access tokens to be issued to third-party clients by an authorization server, with the approval of the resource owner. The third party then uses the access token to access the protected resources hosted by the resource server.
OpenID Connect
OpenID Connect 1.0 is a simple identity layer on top of the OAuth 2.0 protocol. It allows Clients to verify the identity of the End-User based on the authentication performed by an Authorization Server, as well as to obtain basic profile information about the End-User in an interoperable and REST-like manner.
While user authentication may let the API server know WHO is using the API, it cannot guarantee that the requests have originated from WHAT you expect, the browser were your web app should be running from, with a real user.
Now we need a way to identify WHAT is calling the API server, and here things become more tricky than most developers may think. The WHAT is the thing making the request to the API server. Is it really a genuine instance of the web app, or is a bot, an automated script or an attacker manually poking around with the API server, using a tool like Postman?
For your surprise, you may end up discovering that It can be one of the legit users manipulating manually the requests or an automated script that is trying to gamify and take advantage of the service provided by the web app.
Well, to identify the WHAT, developers tend to resort to an API key that usually is sent in the headers of the web app. Some developers go the extra mile and compute the key at run-time in the web app, inside obfuscated javascript, thus it becomes a runtime secret, that can be reverse engineered by deobusfaction tools, and by inspecting the traffic between the web app and API server with the F12 or MitM tools.
The above write-up was extracted from an article I wrote, entitled WHY DOES YOUR MOBILE APP NEED AN API KEY?. While in the context of a Mobile App, the overall idea is still valid in the context of a web app. You wish you can read the article in full here, that is the first article in a series of articles about API keys.
Your Problem
I can try to create a domain whitelist and put my domain in this. But still, if I set the HTTP header "Referer" to my domain, I am able to access the API from any IP.
So this seems to be related with using the HERE admin interface, and I cannot help you here...
So, what do I do?
I am using HERE API in both frontend and backend.
The frontend MUST always delegate access to third part APIs into a backend that is under the control of the owner of the frontend, this way you don't expose access credentials to access this third part services in your frontend.
So the difference is that now is under your direct control how you will protect against abuse of HERE API access, because you are no longer exposing to the public the HERE api_id and api_code, and access to it must be processed through your backend, where your access secrets are hidden from public pry eyes, and where you can easily monitor and throttle usage, before your bill skyrockets in the HERE API.
If I try to put my app_id and app_code into the frontend code, it will be available to anyone seeing my site.
So to recap, the only credentials you SHOULD expose in your frontend is the ones to access your backend, the usual api-key and Authorization tokens, or whatsoever you want to name them, not the api_id or api_code to access the HERE API. This approach leaves you only with one access to protect, instead of multiple ones.
Defending an API Server
As I already said, but want to reinforce a web app should only communicate with an API server that is under your control and any access to third part APIs services must be done by this same API server you control. This way you limit the attack surface to only one place, where you will employ as many layers of defence as what you are protecting is worth.
For an API serving a web app, you can employ several layers of dense, starting with reCaptcha V3, followed by Web Application Firewall(WAF) and finally if you can afford it a User Behavior Analytics(UBA) solution.
Google reCAPTCHA V3:
reCAPTCHA is a free service that protects your website from spam and abuse. reCAPTCHA uses an advanced risk analysis engine and adaptive challenges to keep automated software from engaging in abusive activities on your site. It does this while letting your valid users pass through with ease.
...helps you detect abusive traffic on your website without any user friction. It returns a score based on the interactions with your website and provides you more flexibility to take appropriate actions.
WAF - Web Application Firewall:
A web application firewall (or WAF) filters, monitors, and blocks HTTP traffic to and from a web application. A WAF is differentiated from a regular firewall in that a WAF is able to filter the content of specific web applications while regular firewalls serve as a safety gate between servers. By inspecting HTTP traffic, it can prevent attacks stemming from web application security flaws, such as SQL injection, cross-site scripting (XSS), file inclusion, and security misconfigurations.
UBA - User Behavior Analytics:
User behavior analytics (UBA) as defined by Gartner is a cybersecurity process about the detection of insider threats, targeted attacks, and financial fraud. UBA solutions look at patterns of human behavior, and then apply algorithms and statistical analysis to detect meaningful anomalies from those patterns—anomalies that indicate potential threats. Instead of tracking devices or security events, UBA tracks a system's users. Big data platforms like Apache Hadoop are increasing UBA functionality by allowing them to analyze petabytes worth of data to detect insider threats and advanced persistent threats.
All these solutions work based on a negative identification model, by other words they try their best to differentiate the bad from the good by identifying what is bad, not what is good, thus they are prone to false positives, despite the advanced technology used by some of them, like machine learning and artificial intelligence.
So you may find yourself more often than not in having to relax how you block the access to the API server in order to not affect the good users. This also means that these solutions require constant monitoring to validate that the false positives are not blocking your legit users and that at the same time they are properly keeping at bay the unauthorized ones.
Summary
Anything that runs on the client side and needs some secret to access an API can be abused in different ways and you must delegate the access to all third part APIs to a backend under your control, so that you reduce the attack surface, and at the same time protect their secrets from public pry eyes.
In the end, the solution to use in order to protect your API server must be chosen in accordance with the value of what you are trying to protect and the legal requirements for that type of data, like the GDPR regulations in Europe.
So using API keys may sound like locking the door of your home and leave the key under the mat, but not using them is liking leaving your car parked with the door closed, but the key in the ignition.
Going the Extra Mile
OWASP Web Top 10 Risks
The OWASP Top 10 is a powerful awareness document for web application security. It represents a broad consensus about the most critical security risks to web applications. Project members include a variety of security experts from around the world who have shared their expertise to produce this list.
Goal: My server needs to direct non-users to a landing/home page, and logged in users to the actual app. When the app is loaded, it will make authenticated HTTP requests to a RESTful API (via Ajax).
I have a RESTful API that needs authentication. On another server I have my website, which also needs authentication, so I can determine whether to display the landing/home page for non-users or the app for logged in users.
Initially I thought it would be enough to implement HTTP Basic Auth for the RESTful API. However, in order to get authentication running for my website too, I would also need to setup authentication there, which would mean duplicating the low-level code to check the credentials in the database in both the REST API and the website servers.
Alternatively, I wondered if the website could authenticate via the RESTful API. For example, in my request handler for POST /login, I could make a GET request to my API, passing along the user credentials from the request body. If the request returns 200 OK, I could sign the user’s session, thus authenticating them. From there onwards, the Ajax requests to the REST API need to be authenticated with the same credentials, so I could:
set a cookie containing the credentials, thus allowing the JavaScript to retrieve the credentials before doing the request (OK with SSL?)
dump the credentials in the served HTML for the web app thus allowing the JavaScript to retrieve the credentials before doing the request (OK with SSL?)
proxy the API through the web app server, where I could retrieve the credentials from the session and add them to the Authorization header of the proxied request?
Alternatively, I imagine I could just share a session between the two servers, although I’ve heard that’s bad practice for RESTful design.
What would be wrong with doing it like this? Is there a better way to meet my goal?
I recently implemented something similar to this (assuming I understand you correctly), and there seemed to be a few viable options.
Have the server side of your web-app always authenticate with a specific username/password when accessing the REST API, ensuring that your web-app is always trusted and assuming that users are properly logged in on the web-app if a request is authenticated as the app.
Pros: Easy to implement, easy to understand, easy to extend for other applications as well (we had a CLI that accessed the same REST API as well).
Cons: It's impossible for the REST API to know which user is actually accessing it. If a trusted client is compromised the whole system is compromised.
Have the server side of your web-app keep user details in the session and authenticate using the users credentials every time you access the REST API.
Pros: Fairly easy to implement (although some authentication mechanisms make it hard to keep hold of the user password - for good reason). The whole procedure is transparent to the REST API.
Cons: You're now storing (for all intents and purposes in clear-text) the username and password of a user in the session of the web-server - one of the most prime targets for attack in the system.
Create an authentication system on the REST API that authenticates a request with a username/password authorization and returns a token that is valid for a limited time.
Pros: More secure, if your web-app is compromised you're not providing the attacker with your users username/passwords, but instead only allowing them a limited time access.
Cons: Much harder to implement. You might need to deal with token timeouts specifically. For purists it also means that your REST implementation (or at least the authentication system) will be arguably "stateful".
What you should implement would depend on your situation. Personally I'd definitely go with the more secure option (the last one), but due to external constraints we were forced to implement the first option in our specific case (with the promise we'd revisit it and upgrade later - unfortunately later never comes).
I think your approach with using Basic HTTP Authentication in REST service and having your app authenticate with the service is perfectly fine. The only caveat here (which I am sure you are aware of), is that your REST service should run over SSL, as Basic HTTP authentication is not very secure - username and password are just Base64 encoded.
I'm creating a mobile app for customers that need to access an api that I use.
The api requires authentication and the app needs to call the api to receive some data that is specific to each individual customer(mobile app).
I just want to make sure that the right way to do this is for the mobile app to send the query to my server which will then make the authenticated api call and return the response to the mobile client?
or is it possible to have the mobile make the api calls directly, presumably using the same authorisation key?
This is primarily an opinion-based question, however I'll give it a go:
[paraphrased] Can my server act as an API proxy to make authenticated calls to another API on behalf of my unauthenticated users?
This is a common occurrence in the API world, but some things you need to consider:
It's an extra layer in between the user and the service, which adds time to the data transport. Make sure you build your proxy to be scalable or use a 3rd party service that can manage that on your behalf. Either way, don't forget to factor in cost.
Usually service providers require authentication for a reason. Are you violating any license agreements by opening up their API like this?
Is the authentication per-application, or per-user? If it's per-user (e.g. each user logs in and retrieves a unique access_token) then you're going to be making calls to the back-end API as a user instead of an application.
Is the destination API rate-limited? Instagram's API, for example, only allows 5000 requests per hour. If you have 10,000 users that use it once per hour, you'll have already hit that limit.
Are there security concerns opening up the destination API like this? Is there sensitive information that you need to protect? If so, opening it up like you do are you creating security holes?
Is it possible to have the mobile make API calls directly to the target API, presumably using the same authorization key?
Absolutely this is possible - provided that you follow the authentication flow established by the target API. You'll want to consider the same list of concerns listed above though, in addition to:
If you're using an auth flow like OAuth2, the standard dictates that each user should authenticate as themselves and make API calls using a unique access_token. Does your target API provider offer this service? If so, that's the way to go, that way if an access_token is compromised, only that user's data/account/etc. is at risk.
If you're using app-level authentication (e.g. your app's client_id and client_secret) directly in your mobile app, be warned that it can be obtained and compromised with little effort, and thus an attacker could gain access to the entire target API this way.
I'm developing a REST API that requires authentication. Because the authentication itself occurs via an external webservice over HTTP, I reasoned that we would dispense tokens to avoid repeatedly calling the authentication service. Which brings me neatly to my first question:
Is this really any better than just requiring clients to use HTTP Basic Auth on each request and caching calls to the authentication service server-side?
The Basic Auth solution has the advantage of not requiring a full round-trip to the server before requests for content can begin. Tokens can potentially be more flexible in scope (i.e. only grant rights to particular resources or actions), but that seems more appropriate to the OAuth context than my simpler use case.
Currently tokens are acquired like this:
curl -X POST localhost/token --data "api_key=81169d80...
&verifier=2f5ae51a...
×tamp=1234567
&user=foo
&pass=bar"
The api_key, timestamp and verifier are required by all requests. The "verifier" is returned by:
sha1(timestamp + api_key + shared_secret)
My intention is to only allow calls from known parties, and to prevent calls from being reused verbatim.
Is this good enough? Underkill? Overkill?
With a token in hand, clients can acquire resources:
curl localhost/posts?api_key=81169d80...
&verifier=81169d80...
&token=9fUyas64...
×tamp=1234567
For the simplest call possible, this seems kind of horribly verbose. Considering the shared_secret will wind up being embedded in (at minimum) an iOS application, from which I would assume it can be extracted, is this even offering anything beyond a false sense of security?
Let me seperate up everything and solve approach each problem in isolation:
Authentication
For authentication, baseauth has the advantage that it is a mature solution on the protocol level. This means a lot of "might crop up later" problems are already solved for you. For example, with BaseAuth, user agents know the password is a password so they don't cache it.
Auth server load
If you dispense a token to the user instead of caching the authentication on your server, you are still doing the same thing: Caching authentication information. The only difference is that you are turning the responsibility for the caching to the user. This seems like unnecessary labor for the user with no gains, so I recommend to handle this transparently on your server as you suggested.
Transmission Security
If can use an SSL connection, that's all there is to it, the connection is secure*. To prevent accidental multiple execution, you can filter multiple urls or ask users to include a random component ("nonce") in the URL.
url = username:key#myhost.com/api/call/nonce
If that is not possible, and the transmitted information is not secret, I recommend securing the request with a hash, as you suggested in the token approach. Since the hash provides the security, you could instruct your users to provide the hash as the baseauth password. For improved robustness, I recommend using a random string instead of the timestamp as a "nonce" to prevent replay attacks (two legit requests could be made during the same second). Instead of providing seperate "shared secret" and "api key" fields, you can simply use the api key as shared secret, and then use a salt that doesn't change to prevent rainbow table attacks. The username field seems like a good place to put the nonce too, since it is part of the auth. So now you have a clean call like this:
nonce = generate_secure_password(length: 16);
one_time_key = nonce + '-' + sha1(nonce+salt+shared_key);
url = username:one_time_key#myhost.com/api/call
It is true that this is a bit laborious. This is because you aren't using a protocol level solution (like SSL). So it might be a good idea to provide some kind of SDK to users so at least they don't have to go through it themselves. If you need to do it this way, I find the security level appropriate (just-right-kill).
Secure secret storage
It depends who you are trying to thwart. If you are preventing people with access to the user's phone from using your REST service in the user's name, then it would be a good idea to find some kind of keyring API on the target OS and have the SDK (or the implementor) store the key there. If that's not possible, you can at least make it a bit harder to get the secret by encrypting it, and storing the encrypted data and the encryption key in seperate places.
If you are trying to keep other software vendors from getting your API key to prevent the development of alternate clients, only the encrypt-and-store-seperately approach almost works. This is whitebox crypto, and to date, no one has come up with a truly secure solution to problems of this class. The least you can do is still issue a single key for each user so you can ban abused keys.
(*) EDIT: SSL connections should no longer be considered secure without taking additional steps to verify them.
A pure RESTful API should use the underlying protocol standard features:
For HTTP, the RESTful API should comply with existing HTTP standard headers. Adding a new HTTP header violates the REST principles. Do not re-invent the wheel, use all the standard features in HTTP/1.1 standards - including status response codes, headers, and so on. RESTFul web services should leverage and rely upon the HTTP standards.
RESTful services MUST be STATELESS. Any tricks, such as token based authentication that attempts to remember the state of previous REST requests on the server violates the REST principles. Again, this is a MUST; that is, if you web server saves any request/response context related information on the server in attempt to establish any sort of session on the server, then your web service is NOT Stateless. And if it is NOT stateless it is NOT RESTFul.
Bottom-line: For authentication/authorization purposes you should use HTTP standard authorization header. That is, you should add the HTTP authorization / authentication header in each subsequent request that needs to be authenticated. The REST API should follow the HTTP Authentication Scheme standards.The specifics of how this header should be formatted are defined in the RFC 2616 HTTP 1.1 standards – section 14.8 Authorization of RFC 2616, and in the RFC 2617 HTTP Authentication: Basic and Digest Access Authentication.
I have developed a RESTful service for the Cisco Prime Performance Manager application. Search Google for the REST API document that I wrote for that application for more details about RESTFul API compliance here. In that implementation, I have chosen to use HTTP "Basic" Authorization scheme. - check out version 1.5 or above of that REST API document, and search for authorization in the document.
In the web a stateful protocol is based on having a temporary token that is exchanged between a browser and a server (via cookie header or URI rewriting) on every request. That token is usually created on the server end, and it is a piece of opaque data that has a certain time-to-live, and it has the sole purpose of identifying a specific web user agent. That is, the token is temporary, and becomes a STATE that the web server has to maintain on behalf of a client user agent during the duration of that conversation. Therefore, the communication using a token in this way is STATEFUL. And if the conversation between client and server is STATEFUL it is not RESTful.
The username/password (sent on the Authorization header) is usually persisted on the database with the intent of identifying a user. Sometimes the user could mean another application; however, the username/password is NEVER intended to identify a specific web client user agent. The conversation between a web agent and server based on using the username/password in the Authorization header (following the HTTP Basic Authorization) is STATELESS because the web server front-end is not creating or maintaining any STATE information whatsoever on behalf of a specific web client user agent. And based on my understanding of REST, the protocol states clearly that the conversation between clients and server should be STATELESS. Therefore, if we want to have a true RESTful service we should use username/password (Refer to RFC mentioned in my previous post) in the Authorization header for every single call, NOT a sension kind of token (e.g. Session tokens created in web servers, OAuth tokens created in authorization servers, and so on).
I understand that several called REST providers are using tokens like OAuth1 or OAuth2 accept-tokens to be be passed as "Authorization: Bearer " in HTTP headers. However, it appears to me that using those tokens for RESTful services would violate the true STATELESS meaning that REST embraces; because those tokens are temporary piece of data created/maintained on the server side to identify a specific web client user agent for the valid duration of a that web client/server conversation. Therefore, any service that is using those OAuth1/2 tokens should not be called REST if we want to stick to the TRUE meaning of a STATELESS protocol.
Rubens