I know this is not a programming question, but a valid questions to decide the architecture.
I am working on creating APIs would be used by third-party developers. This means the developers needs to sign-up for a plan (possibly using Stripe), get API key and start making requests.
I am leaning more towards using https://aws.amazon.com/api-gateway/ for the benefits it provides around API management, security (via Cognito) and having a developer portal (https://docs.aws.amazon.com/apigateway/latest/developerguide/api-gateway-documenting-api.html). In my observation so far, Cloudflare API gateway does not provide these benefits. Another benefit is using CDK to manage the entire stack programmatically.
When comparing serverless functions, I am more interested in leveraging Cloudflare workers because of
No cold start issue.
Better pricing.
However, I am unsure about a few things
If a request comes to API gateway and authenticates perfectly, how do I securely invoke Cloudflare worker?
I am sure there would be some latency added between the 2 systems. Are there any ways to minimize the latency?
The guidance is very much appreciated.
Thank you
Related
I recently read a lot of post and article about securing sensitive info into a React Native app. From what I understand, you can't fully protect your sensitive info but only make hacker's life harder to get them.
So, from that point of view, I would like to know if it wouldn't be "safer" to get those sensitive info (i.e. API keys) from an external server (i.e. Rest API).
I explain:
I know about MitM attacks, but would it be safer (and more flexible) to have my mobile app calling my API to get API keys on request thru HTTPS? This way, no sensitive info remains in the app binary files.
And to secure MitM attacks, I could frequently change those API key values so they would remains valid only on a short period of time.
I would like to hear anyone about PROS and CONS of such a system.
APIs Misconceptions
To prepare you for my answer I will first clear out some usual misconceptions around public/private APIs and about who vs what is really accessing your backend.
Public and Private APIs
I often see that developers think that their APIs are private, because they have no docs for it, have not advertise it anywhere, and many other reasons.
The truth is that when you release a mobile app all the APIs it communicates with are now belonging to the public domain and if this APIs don't have an authentication and authorization mechanism in place then all data behind it can be accessed by anyone in the internet that reverse engineers how your mobile app works. Even when APIs have authentication in place they may be vulnerable to bad implementations of it and some have a total lack of authorization mechanisms or buggy ones as per OWASP API Security Top 10 vulnerability list.
The Difference Between WHO and WHAT is Accessing the API Server
I wrote a series of articles around API and Mobile security, and in the article Why Does Your Mobile App Need An Api Key? you can read in detail the difference between who and what is accessing your API server, but I will extract here the main takes from it:
The what is the thing making the request to the API server. Is it really a genuine instance of your mobile app, or is it a bot, an automated script or an attacker manually poking around your API server with a tool like Postman?
The who is the user of the mobile app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
So think about the who as the user your API server will be able to Authenticate and Authorize access to the data, and think about the what as the software making that request in behalf of the user.
API Keys Service
I know about MitM attacks, but would it be safer (and more flexible) to have my mobile app calling my API to get API keys on request thru HTTPS? This way, no sensitive info remains in the app binary files.
While you indeed don't have any sensitive info in the app binary files you haven't solved the problem. In my opinion you are more exposed, because you are now getting the API keys from an public and open API endpoint.
I say it's open because you don't have any safeguard that what is making the request to it are indeed a genuine and untampered version of your mobile app.
So, now all an attacker needs to do is to MitM attack your mobile app or decompile it to see from which API endpoint you grab the API keys to make the requests, and then replicate the procedure from their automated scripts/bots, therefore doesn't really matter that you don't have them hardcoded in the app binary any more.
API Keys Rotation
And to secure MitM attacks, I could frequently change those API key values so they would remains valid only on a short period of time.
In light of the above explanation , on the API Keys Service section, you can even make the API keys restricted to be used only for one single request that the attacker will still succeed, because the attacker will be able to query the API endpoint to obtain API keys as if he was what the backend expects, a genuine and untampered version of your mobile app.
So, to be clear I am in favour of API keys rotation but only if you can get them into your mobile app from a secured external source, but your approach is open to be accessed by anyone on the internet.
I would like to hear anyone about PROS and CONS of such a system.
The system you are describing is not advisable to implement, because without being secured it's just a security disaster asking to occur. Securing it with an API key it's just going back to the initial problem with the disadvantage that your giving back to the mobile the sensitive info you want to keep away from hackers.
The best approach for you is to use a Reverse Proxy to keep the API keys private and secured from prying eyes.
The Reverse Proxy Approach
So, from that point of view, I would like to know if it wouldn't be "safer" to get those sensitive info (i.e. API keys) from an external server (i.e. Rest API).
What you are looking for is to implement a Reverse Proxy, that is usually used to protect access to third party APIs and your own APIs, by having the mobile app delegating the API requests to the Reverse Proxy, instead of asking for the API keys to make them from inside the mobile app.
The Reverse Proxy approach will avoid to have several API keys harcoded in the mobile app, but you still need one API key to protect access to the Reverse Proxy, therefore you are still vulnerable to the MitM attacks and to static reverse engineering of your mobile app.
The advantage now is that all your sensitive API keys are private and in an environment you can control and employ as many security measures you need to ensure that the request are indeed from what your backend expects, a genuine and untampered version of your mobile app.
Learn more about using a Reverse Proxy by reading the article I wrote Using a Reverse Proxy to Protect Third Party APIs:
In this article you will start by learning what Third Party APIs are, and why you shouldn’t access them directly from within your mobile app. Next you will learn what a Reverse Proxy is, followed by when and why you should use it to protect the access to the Third Party APIs used in your mobile app.
While the article focus on third party APIs the principle also applies to use with your own APIs.
Preventing MitM Attacks
When certificate pinning is implemented in a mobile app to secure the https channel then the sensitive data on the API requests is more safeguarded from being extracted.
I recommend you to read the section Preventing MitM Attacks in this answer I gave to another question where you will learn how to implement static certificate pinning and how to bypass it.
Despite being possible to bypass certificate pinning I still strongly recommend it to be implemented, because it reduces the attack surface on your mobile app.
A Possible Better Solution
I recommend you to read this answer I gave to the question How to secure an API REST for mobile app?, especially the sections Hardening and Shielding the Mobile App, Securing the API Server and A Possible Better Solution.
The solution will be the use of a Mobile App Attestation solution that will allow your backend to have an high degree of confidence that the request is from what it expects, a genuine and untampered version of your mobile app.
Do You Want To Go The Extra Mile?
In any response to a security question I always like to reference the excellent work from the OWASP foundation.
For APIS
OWASP API Security Top 10
The OWASP API Security Project seeks to provide value to software developers and security assessors by underscoring the potential risks in insecure APIs, and illustrating how these risks may be mitigated. In order to facilitate this goal, the OWASP API Security Project will create and maintain a Top 10 API Security Risks document, as well as a documentation portal for best practices when creating or assessing APIs.
For Mobile Apps
OWASP Mobile Security Project - Top 10 risks
The OWASP Mobile Security Project is a centralized resource intended to give developers and security teams the resources they need to build and maintain secure mobile applications. Through the project, our goal is to classify mobile security risks and provide developmental controls to reduce their impact or likelihood of exploitation.
OWASP - Mobile Security Testing Guide:
The Mobile Security Testing Guide (MSTG) is a comprehensive manual for mobile app security development, testing and reverse engineering.
I'm writing a service to be deployed on Kubernetes. Clients will be other services, not people, and those services may be in other namespaces or even clusters. My goals are:
Authenticate the calling services
Authorize the calling services
Apply some policies based on the identity of the calling service (like quota)
I understand that Kubernetes doesn't provide services that really help with any of these, and I'll need to build something explicit into my service. I'd like to understand what the current best practice is and how to maximize what's available in Kubernetes or in the ecosystem to make these goals achievable while minimizing the coding and administrative burden. A few options that I've considered:
Custom username / shared-secret. I could just pass out shared secrets to all of the calling services, and write my own custom code to verify that the shared secret matches. I assume passing these around as Bearer tokens would be the right move. Would using Kubernetes serviceaccount and role objects be reasonable containers for these shared secrets? If so, are there libraries that make the lookups, associations, and policy work easier?
JWT. JWT seems more intended for passing around claims, like end-user identity, and would seem to require that all of the participating components share the same JWT secret. Since I don't want calling-service-foo to be able to authenticate as calling-service-bar, it's not clear that JWT is the right move. Thoughts?
mTLS. I could issue TLS certificates for all of the participating services. Are there components I can use to automate the issuance of these certificates? Should I try to use Kubernetes serviceaccount or role objects to manage these, or maybe roll my own CRDs?
Istio. It seems like Istio can do a lot of this transparently, but so far all of the resources I've found that explain this seem to assume transparency is a goal. Since I will need the identities of the calling services, though, is it possible to get that out of Istio? Can this work if my callers aren't in my cluster?
SPIRE (spiffe.io). This looks like it matches well for my use cases, but it seems new and I don't know how much experience people have with it.
Do any of these options (and please correct my understanding of any of them) stand out as best practices, or are there others I should be considering?
Thank you!
What you need is a component that acts as the gateway to the microservices API endpoints. That kind of component belongs to a category of software called "API management" (Wikipedia page) and its usage is not limited to Kubernetes only.
There are many choices of API management software such as listed in the Wikipedia page but my project uses Gravitee and so far we are loving it due to its simple administration UI. Feel free to explore it at https://gravitee.io/.
N.B. I'm not related in any way to Gravitee.io apart from being one of the users (although I did contribute to one PR)
I know I'm late now but if anyone else is looking:
Istio has included Multicluster support and it makes the communication painless.
reference: https://istio.io/latest/docs/setup/install/multicluster
I am using HERE api in both frontend and backend. If I try to put my app_id and app_code into the frontend code, it will be available to anyone seeing my site.
I can try to create a domain whitelist and put my domain in this. But still, if I set the HTTP header "Referer" to my domain, I am able to access the API from any IP.
So, what do I do?
The Difference Between WHO and WHAT is Accessing the API Server
Before I dive into your problem I would like to first clear a misconception about WHO and WHAT is accessing an API server.
To better understand the differences between the WHO and the WHAT are accessing an API server, let’s use this picture:
So replace the mobile app by web app, and keep following my analogy around this picture.
The Intended Communication Channel represents the web app being used as you expected, by a legit user without any malicious intentions, communicating with the API server from the browser, not using Postman or using any other tool to perform a man in the middle(MitM) attack.
The actual channel may represent several different scenarios, like a legit user with malicious intentions that may be using Curl or a tool like Postman to perform the requests, a hacker using a MitM attack tool, like MitmProxy, to understand how the communication between the web app and the API server is being done in order to be able to replay the requests or even automate attacks against the API server. Many other scenarios are possible, but we will not enumerate each one here.
I hope that by now you may already have a clue why the WHO and the WHAT are not the same, but if not it will become clear in a moment.
The WHO is the user of the web app that we can authenticate, authorize and identify in several ways, like using OpenID Connect or OAUTH2 flows.
OAUTH
Generally, OAuth provides to clients a "secure delegated access" to server resources on behalf of a resource owner. It specifies a process for resource owners to authorize third-party access to their server resources without sharing their credentials. Designed specifically to work with Hypertext Transfer Protocol (HTTP), OAuth essentially allows access tokens to be issued to third-party clients by an authorization server, with the approval of the resource owner. The third party then uses the access token to access the protected resources hosted by the resource server.
OpenID Connect
OpenID Connect 1.0 is a simple identity layer on top of the OAuth 2.0 protocol. It allows Clients to verify the identity of the End-User based on the authentication performed by an Authorization Server, as well as to obtain basic profile information about the End-User in an interoperable and REST-like manner.
While user authentication may let the API server know WHO is using the API, it cannot guarantee that the requests have originated from WHAT you expect, the browser were your web app should be running from, with a real user.
Now we need a way to identify WHAT is calling the API server, and here things become more tricky than most developers may think. The WHAT is the thing making the request to the API server. Is it really a genuine instance of the web app, or is a bot, an automated script or an attacker manually poking around with the API server, using a tool like Postman?
For your surprise, you may end up discovering that It can be one of the legit users manipulating manually the requests or an automated script that is trying to gamify and take advantage of the service provided by the web app.
Well, to identify the WHAT, developers tend to resort to an API key that usually is sent in the headers of the web app. Some developers go the extra mile and compute the key at run-time in the web app, inside obfuscated javascript, thus it becomes a runtime secret, that can be reverse engineered by deobusfaction tools, and by inspecting the traffic between the web app and API server with the F12 or MitM tools.
The above write-up was extracted from an article I wrote, entitled WHY DOES YOUR MOBILE APP NEED AN API KEY?. While in the context of a Mobile App, the overall idea is still valid in the context of a web app. You wish you can read the article in full here, that is the first article in a series of articles about API keys.
Your Problem
I can try to create a domain whitelist and put my domain in this. But still, if I set the HTTP header "Referer" to my domain, I am able to access the API from any IP.
So this seems to be related with using the HERE admin interface, and I cannot help you here...
So, what do I do?
I am using HERE API in both frontend and backend.
The frontend MUST always delegate access to third part APIs into a backend that is under the control of the owner of the frontend, this way you don't expose access credentials to access this third part services in your frontend.
So the difference is that now is under your direct control how you will protect against abuse of HERE API access, because you are no longer exposing to the public the HERE api_id and api_code, and access to it must be processed through your backend, where your access secrets are hidden from public pry eyes, and where you can easily monitor and throttle usage, before your bill skyrockets in the HERE API.
If I try to put my app_id and app_code into the frontend code, it will be available to anyone seeing my site.
So to recap, the only credentials you SHOULD expose in your frontend is the ones to access your backend, the usual api-key and Authorization tokens, or whatsoever you want to name them, not the api_id or api_code to access the HERE API. This approach leaves you only with one access to protect, instead of multiple ones.
Defending an API Server
As I already said, but want to reinforce a web app should only communicate with an API server that is under your control and any access to third part APIs services must be done by this same API server you control. This way you limit the attack surface to only one place, where you will employ as many layers of defence as what you are protecting is worth.
For an API serving a web app, you can employ several layers of dense, starting with reCaptcha V3, followed by Web Application Firewall(WAF) and finally if you can afford it a User Behavior Analytics(UBA) solution.
Google reCAPTCHA V3:
reCAPTCHA is a free service that protects your website from spam and abuse. reCAPTCHA uses an advanced risk analysis engine and adaptive challenges to keep automated software from engaging in abusive activities on your site. It does this while letting your valid users pass through with ease.
...helps you detect abusive traffic on your website without any user friction. It returns a score based on the interactions with your website and provides you more flexibility to take appropriate actions.
WAF - Web Application Firewall:
A web application firewall (or WAF) filters, monitors, and blocks HTTP traffic to and from a web application. A WAF is differentiated from a regular firewall in that a WAF is able to filter the content of specific web applications while regular firewalls serve as a safety gate between servers. By inspecting HTTP traffic, it can prevent attacks stemming from web application security flaws, such as SQL injection, cross-site scripting (XSS), file inclusion, and security misconfigurations.
UBA - User Behavior Analytics:
User behavior analytics (UBA) as defined by Gartner is a cybersecurity process about the detection of insider threats, targeted attacks, and financial fraud. UBA solutions look at patterns of human behavior, and then apply algorithms and statistical analysis to detect meaningful anomalies from those patterns—anomalies that indicate potential threats. Instead of tracking devices or security events, UBA tracks a system's users. Big data platforms like Apache Hadoop are increasing UBA functionality by allowing them to analyze petabytes worth of data to detect insider threats and advanced persistent threats.
All these solutions work based on a negative identification model, by other words they try their best to differentiate the bad from the good by identifying what is bad, not what is good, thus they are prone to false positives, despite the advanced technology used by some of them, like machine learning and artificial intelligence.
So you may find yourself more often than not in having to relax how you block the access to the API server in order to not affect the good users. This also means that these solutions require constant monitoring to validate that the false positives are not blocking your legit users and that at the same time they are properly keeping at bay the unauthorized ones.
Summary
Anything that runs on the client side and needs some secret to access an API can be abused in different ways and you must delegate the access to all third part APIs to a backend under your control, so that you reduce the attack surface, and at the same time protect their secrets from public pry eyes.
In the end, the solution to use in order to protect your API server must be chosen in accordance with the value of what you are trying to protect and the legal requirements for that type of data, like the GDPR regulations in Europe.
So using API keys may sound like locking the door of your home and leave the key under the mat, but not using them is liking leaving your car parked with the door closed, but the key in the ignition.
Going the Extra Mile
OWASP Web Top 10 Risks
The OWASP Top 10 is a powerful awareness document for web application security. It represents a broad consensus about the most critical security risks to web applications. Project members include a variety of security experts from around the world who have shared their expertise to produce this list.
I am using Nuxt (with SSR/ PWA/ Vuejs/ Node.js/ Vuex/ Firestore) and would like to have a general idea or have an example for the following:
How can I secure an API key. For example to call MailChimp API
I am not familiar with how a hacker would see this if a poor solution is implemented. How can I verify it is not accessible to them?
I have found a number of "solutions" that recommend using environment Variables, but for every solution someone indicates it wont be secure. See:
https://github.com/nuxt-community/dotenv-module/issues/7
https://github.com/nuxt/nuxt.js/issues/2033
Perhaps server middleware is the answer? https://blog.lichter.io/posts/sending-emails-through-nuxtjs and https://www.youtube.com/watch?v=j-3RwvWZoaU (#11:30). I just need to add an email to a mail chimp account once entered, seems like a lot of overhead.
Also I see I store my Firestore api key as an environment variable already. Is this secure? When I open chrome dev tools-> sources-> page-> app.js i can see the api key right there (only tested in dev mode)!
You could use either a server middleware or https://github.com/nuxt-community/separate-env-module
Middleware itself wont work because it can be executed on client too, and code that is used in middleware will be available on client
For #2 you can check whether its included in client js sources. There way more other way hacker to get anything e.g. xss, but its general things and not much related to your code.
How can I secure an API key. For example to call MailChimp API
The cruel truth here is NO... In the client side you cannot secure any kind of secret, at least in a web app.
Just for you to have an idea of the techniques that can be used to protect an API and how they can be bypassed you can read this series of articles. While it is in the context of an Api serving a mobile app, the majority of it also applies for an API serving a web app. You will learn how api-keys, ouath tokens, hmac and certificate pinning can be used and bypassed.
Access to third part services must be always done in the back-end, never on the client side. With this approach you only have one place to protected, that is under your control.
For example in your case of accessing the Mailchimp API... If your back-end is the one in charge of doing it in behalf of your web app, then you can put security measures in place to detect and mitigate the usage of Mailchimp by your web app, like a User Behaviour Analytics (UBA) solution, but leaving for the web app the access to the Mailchimp API means that you only know that someone is abusing it when Mailchimp alerts your or you see it in their dashboards.
I am not familiar with how a hacker would see this if a poor solution is implemented. How can I verify it is not accessible to them?
As you may already know F12 to access the developers tools is one of the ways.
Another ways id to use the OWASP security tool Zed Attack Proxy (ZAP) , and using their words:
The OWASP Zed Attack Proxy (ZAP) is one of the world’s most popular free security tools and is actively maintained by hundreds of international volunteers*. It can help you automatically find security vulnerabilities in your web applications while you are developing and testing your applications. Its also a great tool for experienced pentesters to use for manual security testing.
Storing secrets in the front-end is a big no no in terms of security.
If your website is using server-side rendering (aka SSG or static website) and is hosted on Netlify it sound like a perfect job for the Netlify functions (server side logic) and environnement variables.
You can find some documentations here : Netlify functions.
Netlify functions are powered by AWS Lambda.
You would typically create a function folder into your project directory and write your functions there. Functions are built after each deploy but you can test your functions locally with Netlify Dev
Here is an example of function using Mailchimp service wit injected secrets :
https://github.com/tobilg/netlify-functions-landingpage/blob/169de175d04b165b5d4801b09cb250cd9a740da5/src/lambda/signup.js
I think privateRuntimeConfig, by which secrets are only available on the server side is another workable solution here, if you're in a situation where you only need to access an API during Server Side Rendering.
https://nuxtjs.org/tutorials/moving-from-nuxtjs-dotenv-to-runtime-config/#misconceptions:~:text=privateRuntimeConfig%20should%20hold%20all%20env%20variables%20that%20are%20private%20and%20that%20should%20not%20be%20exposed%20on%20the%20frontend.%20This%20could%20include%20a%20reference%20to%20your%20API%20secret%20tokens%20for%20example.
I have my Rest API's configured in API Gateway. I want to implement OAuth on top of it so that the access to the url's are authenticated. I know that Lambda functions will be required here. Following are my questions:
1. Any good third party oauth providers that I can link with my project?
Scaling needs will be pretty high, possibly around 1M/second. Will these providers be scalable enough?
I would want to be doing everything programmatically. Like mostly these providers will require registration in order to acquire client id's and secrets. I want that also to happen programmatically as the no of users signing up will be dynamic.
i find Authlete good and also provide a guide for Amazon API Gateway
Yes, they can scale pretty well (check the pricing section) if you are willing to pay enough