I have a firebase cloud function that uses express middleware to generate an authToken which is passed on up through the routes.
This token lasts 24 hr's so I don't want to spam the service with excessive requests. As such I want to cache the result for a while before regenerating it.
I considered two approaches.
See how to cache the axios request when generating the token.
Use Firebase Secret Manager to store the value.
Whilst researching the axios caching, I already had secret manager in use so have implemented both in parallel.
Secret Manager
Currently I have implemented secret manager to store and update the token, when I need the token for the external API, I can just grab it from secret manager..
I have a scheduled job that then creates a new token, and subsequently disables all prior secret versions before adding in the new token.
Creating the token is achieved by using axios to call the auth end point of the external API.
Axios Caching
When it comes to finding out how to enable caching for express running on cloud functions. I'm currently getting the impression that it involves adding in another layer of complexity, such as either a CDN, or Redis solution.
I've also seen intermediate cache solutions such as https://github.com/AlbinoDrought/cachios which looks like it leverages local memory of some sort. Which I assume would mean that cloud functions distributed across different regions/instances would have there own unique local cache. I wouldn't know if this is a problem or not. But would I think be more lightweight in configuration and setup than say redis?
My Problem is, and I hope this isn't too opinion based for SO. but.
Is this use of secret manager a really bad idea. Given secret manager is generally I think geared around long lived values that don't change that often. Or is it perfectly fine, that if I have 2 requests a day, that could be 2x365 versions a year logged against the secret manager, which would have to be tidied up everynow and then. Are the implications I'm not aware of?
Or is enabling some sort of CDN or express caching layer a much more standard approach? And if so is there any guidance on a simple express caching layer that's geared specifically for google Firebase cloud functions?
2 requests per day is fine. See the documentation on rotating keys here: https://cloud.google.com/secret-manager/docs/rotation-recommendations
Note that it suggests, in approach 2 & 3, that valid use cases include grabbing secrets on each launch, and grabbing secrets continuously. 2x/day is less frequent than both of these.
Related
Why would one use iron-session instead of next-auth? Doesn't next-auth do normal username/password log-in in addition to Social (while iron-session only does the former)?
next-auth does a lot of things like you said but it also makes you do things their way. So you have to model your database a certain way to make next-auth work but it does come with a lot of helper functions and makes you write a lot less code.
iron-session only does session management. You have to write all the auth logic with iron-session which next-auth handles for you automatically.
As far as which one is better: if you'd like complete control over your database then use iron-session because sometimes next-auth might not work and it's annoying. I faced an issue with it so ditched it but it just might work for you. iron-session gives full control but you have to write the correct code. Personally, I'm facing issues with iron-session as well with useUser hook because I have to use react-query and not swr but it's probably my issue.
iron-session is lightweight library. It is simply used for session management. session is not related only to authentication. For example, maybe you want to sign some data on one API endpoint, attached the signed data to the req.session and you still need to reach this data on a different API endpoint iron-session store data on the encrypted cookie and save it to the browser. From here:
Node.js stateless session utility using signed and encrypted cookies
to store data. Works with Next.js, Express, and Node.js HTTP servers.
The session data is stored in encrypted cookies ("seals"). And only
your server can decode the session data. There are no session ids,
making iron sessions "stateless" from the server point of view.
On the other hand, next-auth is a more robust authentication package. as the name says next-auth, it is dedicated to authentication. It has too many authentication methods. It creates a session specifically for the user's login
There is a lot of information on the web about OAuth 2, its different types of flows and where/how to use them. I find that most of these resources discuss authenticating a user for an application, but I am struggling to understand what the best/correct approach would be when consuming third party APIs (i.e. when our own API is the "middleman" between a user and their data in a third party API).
With the help of an example scenario and some diagrams, I would be really grateful for advice/opinions on how I should properly implement integration with third party APIs and what the pros & cons of each approach are.
Starting point
As a starting point, suppose we have a web app, structured as follows:
Frontend - SPA (Angular), hosted with AWS S3 + Cloudfront
Backend - Node, running as stateless AWS lambda functions with AWS API Gateway
Auth0 for handling auth/signin etc. The frontend uses an Implicit OAuth2 flow to obtain access_tokens, which are stored in local storage and included as a header in all requests to the backend.
Potentially also native mobile app(s), consuming the same backend API.
Goal
Now suppose that we wish to add integration with Google Sheets. The new feature would allow users to use their own Google Sheets (i.e. stored in their own Google account) as a data source, with the app having read&write access to the sheet. There may be other integrations in the future, so I am assuming that other APIs would require a similar process.
Problem statement
In addition to the existing OAuth process (which allows users to sign in to the "MyApp" frontend and communicate with the "MyApp API"), there needs to be an additional OAuth process for users to connect MyApp to the third party Google Sheets API.
The documentation has two Quickstart examples, but neither seems to quite fit my needs:
Browser - https://developers.google.com/sheets/api/quickstart/js
Node.js (console app) - https://developers.google.com/sheets/api/quickstart/nodejs
The data from the third-party (Google) API is one of potentially several integration points, so intuitively it seems more logical (and more secure) that all communication with the Google Sheets API should happen in the MyApp API, and not on the frontend/client side. The MyApp API would fetch data, process/manipulate/format it in some way and then present it for display in the frontend or mobile apps.
We require access to each user's own data, so the Client Credentials flow is not suitable. I am focussing on the Implicit or Authorization Grant workflows.
Important note: The trickiness seems to come from the fact that the MyApp API is stateless, so there is no long-lived session in which to store tokens. On that basis, it seems like tokens need to be stored either in the frontend (e.g. local storage/cookies etc) or in a backend database.
Below is my interpretation of two possible approaches. I'd appreciate thoughts/corrections.
Option 1: Implicit flow - tokens stored FE, passed along to BE which then makes requests to Google
Pros:
Allows access to user's own data
Simpler flow, access_token retrieved immediately without needing the code step
Less steps to implement between initial sign-in process and actually obtaining data
No need for a backend database, can resend the token with each request
Cons:
Frontend (browser) has access to Google access_token which seems unnecessary and is a potential security concern
It seems like a strange process to pass the access_token from FE to BE, purely to allow BE to then use that token to make another request
I'm not sure how we would refresh/renew tokens since I understand that storing refresh_tokens on the client is bad practice. It would not be a good user experience if the user had to frequently sign in to reconnect their account
Option 2: Authorization Code Flow - all communication with Google via BE, tokens stored in BE database
Pros:
Allows access to user's own data
Other than the code-request / consent page, all communication with Google is implemented backend, so the tokens are not accessible on the client
Client secret can be used from the BE
Cons:
More complex flow, requires extra steps
Given that the BE is stateless, it's not clear how best to store the tokens. It seems like it would require storing them in a database which is extra complication and seems like it would have security implications - how would you properly secure / encrypt the access_token/refresh_tokens in said database?
Conclusion
Given that the data processing is to happen on the backend, option 2 seems slightly more suitable because the sensitive tokens can be hidden from the frontend app, and several clients (web frontend, mobile apps) have less obligation to be involved in the process with the exception of the initial sign in / user consent. However I’m not sure whether having a database full of user auth tokens is a good idea or how I could properly protect this database.
The good news is that both options are perfectly valid and equally secure. The concern about a short-lived Access Token being in the browser isn't an issue. Equally, if you only held the tokens on the BE, then you would need to implement your own client authentiation/session/JWT blah blah, which presents the same attack surface.
I've done both, and am currently migrating from BE to FE. In my case the reason is that everything I need to do, I can do on the FE, so I end up with no BE at all. This isn't strictly true since I do some onboarding/payment with the BE, but that's about all.
So the best approach depends on factors beyond those in your question, such as the nature of app, what the BE cost is and how important that is, what your devops skillsets look like for maintaining two environments, to what extent a BE is required anyway, vs being completely optional.
My service allow any HTML documents to be converted to PDF using a POST request.
It is mostly used on the backend of my client's server and thus, the API key used for the communication is kept private.
Now, I'm thinking of a way to let my client's visitors be able to call my service on behalf of my client API key, without exposing this secure API Key.
My main issue here is security. If my client add an XHR POST requests that contains the API key, someone can take that API key and use it for their own purpose and abusing my client's account.
I could filter by domain, but this is easily spoofed so it's not possible.
I was wondering if there was a way to call a private service and be identified without risking its identity to be stolen, from the client ('s client) side?
If you're providing this sublet for authenticated users, then it's fairly trivial to give them unique keys (something that hashes their user ID or session against the API key and an initial timestamp, and checks it / logs it / looks for brutes before accessing the API). If you're doing it on the open web, without any kind of user authentication, then rate limiting gets very tricky indeed. Generally you'd want to use a combination of session hashes, IP address, operating system and browser data to create an anonymous profile that gets a temporary key on the frontend. One fairly solid way to do this is to force users through a CAPTCHA before serving a temporary key that allows them a limited number of uses of the permanent key. Any user whose ip/browser/session matches the existing attributes of a known client key is shunted to that one (and gets to skip the CAPTCHA); anyone who doesn't match an existing profile gets the CAPTCHA. That makes you a less attractive target for spoofing. On top of that, you should always rate-limit the entire thing, within a reasonable number of hits per day based on what kind of traffic you expect (or can afford), just so you don't have any surprises. This is the minimal security you'd want if your client's money is on the line every time their API key is used. It will require a simple database to store these "profiles", track usage, check for brutes and maintain the currently valid client keys. Client keys should always be expired regularly - either with a time diff against when they were created, or a regular cron process, or a maximum number of uses, etc.
One other thing I frequently do is rate-limit based on a curve. If I think 5 uses per minute is reasonable, for example, then after 5 uses in a minute from a session, each usage adds a delay of a fraction of a second * the number of uses in the last minute, squared, before the data is served.
The best answer would be to put this all behind a login system and secure that.
Assuming that you are using OAuth kind of system, In that case, make use of Access Token Mechanism that provides access to private API/User's data on behalf of User(Client) without exposing his/her credentials or API Key(Authentication key), also the access token can be expired based on the time/usage.
Example: The access token is generated against a single endpoint that can be the Html Conversion endpoint and will be expired once the action completion.
https://auth0.com/docs/tokens/access-token
And following blog post would be helpful to architect your authentication system
https://templth.wordpress.com/2015/01/05/implementing-authentication-with-tokens-for-restful-applications/
there is no good way to do front-end secure storage but my recommendation is :
is an API that used HMAC signing of requests in combination with OAuth authentication. The API key is actually a signing key. they key does not get transferred. The API key can still get found on the front-end but it becomes useless because you still need the OAuth token to send a valid request.
i know users will have to login in, but you can see this as an advantage because atleast you can log who is using the app by getting information from oauth.
please consider back-end secure storage!
You can use JWT tokens in my opinion. On the basis of username, password or any other info you can generate unique jwt tokens for different users.
Anyone can decipher these jwt tokens but not he unique security token.
If you want to add more more security to tokens, use JWE, encrypted web tokens.
More about these schemes can be found at https://medium.facilelogin.com/jwt-jws-and-jwe-for-not-so-dummies-b63310d201a3
Hashing is a decent option and should be done anyway, but for a fully secure method that wouldn't add too much complexity, you could simply abstract away from the authorization/API key by building your own API to interface with the API. This way you could both limit the kinds of things that can be done with the API key and also completely obscure the API key from the user
I don't think you should always go for user auth or JWT, it just doesn't fit all use cases. The idea of using a Captcha is interesting but also somewhat complex.
If complexity is not an issue I would rather use an infrastructure approach, I'm most familiar with AWS so I'll focus on that. Assuming you can change the host of your front end you can have your site hosted on an S3 bucket, served through a CDN, and create a proxy Lambda function that will hold the logic to call your API and store the API key as an encrypted environment variable. This Lambda you call through an API Gateway that can only be called by a specific IAM role which the S3 bucket also uses. You can also use a Cognito User Pool without authentication.
Going back to a simpler alternative the Captcha approach can be implemented as an attestation provider. I know of two services that do this, Firebase and KOR Connect. Due to Firebase using this approach only for their own resources as of the time of this writing I much rather use KOR Connect as it’s a very simple middleware that basically solves this issue. I won't go into detail about these services as it’s not the main concern of this topic but you can check the documentation their respective links.
What is the advantage of using JWTs over sessions in situations like authentication?
Is it used as a standalone approach or is it used in the session?
JWT doesn't have a benefit over using "sessions" per se. JWTs provide a means of maintaining session state on the client instead of doing it on the server.
What people often mean when asking this is "What are the benefits of using JWTs over using Server-side sessions".
With server-side sessions, you will either have to store the session identifier in a database, or else keep it in memory and make sure that the client always hits the same server. Both of these have drawbacks. In the case of the database (or other centralised storage), this becomes a bottleneck and a thing to maintain - essentially an extra query to be done with every request.
With an in-memory solution, you limit your horizontal scaling, and sessions will be affected by network issues (clients roaming between Wifi and mobile data, servers rebooting, etc).
Moving the session to the client means that you remove the dependency on a server-side session, but it imposes its own set of challenges.
Storing the token securely.
Transporting it securely.
JWT sessions can sometimes be hard to invalidate.
Trusting the client's claim.
These issues are shared by JWTs and other client-side session mechanisms alike.
JWT, in particular, addresses the last of these. It may help to understand what a JWT is:
It is a bit of information. For user sessions, you could include the username and the time when the token expires. But it could conceivably be anything, even the session ID or the user's entire profile (please don't do that though).
It has got a secure signature that prevents malicious parties from generating fake tokens (you need access to the server's private key to sign them and you can verify that they were not modified after they were signed).
You send them with every request, just like a cookie or Authorization Header would be sent. In fact, they are commonly sent in the HTTP Authorization header but using a cookie is fine too.
The token is signed and so the server can verify its origin. We will assume that the server trusts its own ability to sign securely (you should use a standard library: don't try to do it yourself, and secure the server properly).
On the issue with securely transporting the token, the answer is commonly to send it via an encrypted channel, usually httpS.
Regarding securely storing the token in the client, you need to ensure that the bad guys can't get to it. This (mostly) means preventing JS from bad web sites from reading the token to send it back to them. This is mitigated using the same strategies used to mitigate other kinds of XSS attacks.
If you have a need to invalidate JWTs, there are definitely ways this can be achieved. Storing a per-user epoch for only users who have requested to have their "other sessions terminated" is a very efficient method that will probably be good enough. If an application needs per-session invalidation, then a session ID can be maintained in the same way and the "killed tokens" table can still be maintained to be much smaller than the full user table (you only need to retain records newer than the longest allowed token lifetime). So the ability to invalidate the token partially negates the benefit of client-side sessions in that you would have to maintain this session killed state. This will more than likely be a much smaller table than the original session state table, so the lookups are still more efficient though.
One other benefit of using JWT tokens is that it is reasonably easy to implement using libraries available in probably every language you can expect to have it. It is also completely divorced from your initial user authentication scheme - if you move to a fingerprint-based system, you do not need to make any changes to the session management scheme.
A more subtle benefit: Because the JWT can carry "information" and this can be accessed by the client, you can now start doing some smart things. For example, remind the user that their session will be expiring a few days before they are logged out, giving them the option to re-authenticate, based on the expiry date in the token. Whatever you can imagine.
So in short: JWTs answers some of the questions and shortcomings of other session techniques.
"Cheaper" authentication because you can eliminate a DB round trip (or at least have a much smaller table to query!), which in turns enable horizontal scalability.
Tamper-proof client-side claims.
While JWTs does not answer the other issues like secure storage or transport, it does not introduce any new security issues.
A lot of negativity exists around JWTs, but if you implement the same security that you would for other types of authentication, you will be fine.
One final note: It is also not Cookies vs Tokens. Cookies is a mechanism for storing and transporting bits of information and can be used to store and transport JWT tokens too.
The short answer is: None.
A longer version is:
I implemented JWTs for session management after reading this recommendation in the GraphQL docs:
If you aren't familiar with any of these authentication mechanisms, we
recommend using express-jwt because it's simple without sacrificing
any future flexibility.
Implementation was indeed simple as it only added a little bit of complexity. After a while however, I (like you) started wondering what the benefits were. It turns out there are very few (or possibly none) for JWT as far as session management goes, as this blog post explains in detail:
Stop using JWT for sessions
I had a similar question choosing between JWT and token + cache for user authentication.
After reading these articles, it's clear to me the benefits JWT promises do not outpace the problems it brings. So token + cache(Redis/Memcached) is the way to go for me.
Auth Headers vs JWT vs Sessions — How to Choose the Right Auth Technique for APIs
Authentication Techniques for APIs
Stop using jwt for sessions
My two cents, which on the way add some contrast to joepie91's famous blog post.
Considering that today's (and tomorrow's) applications are (mostly) cloud native
There's an economic benefit to Stateless JWT Authentication,
which scales as the application scales:
Cloud applications incur cost with every passing second.
This cost is reduced when users no longer have to authenticate "against" a session store.
Detailed below are some factors which add to the cost of an application when not using JWT:
Database Server
Running a session store 24/7 costs money.
You can not get away with local storage / memory based solutions in the world of K8S, as pods are ephemeral.
Sticky sessions will not fare well for the exact same reason.
Storage
Storing data costs money. storing data in a SSD costs even more.
Session related operations need to be resolved quickly, so an optical drive is not an option.
I/O
Some cloud providers charge money for Disc related I/O.
Download
Circa 2022, it is safe to assume that the API and session store are separate server instances.
Some cloud providers charge for downloading information from one instance to another.
Scaling the session store
This affects all aforementioned factors.
Yet another slightly different perspective that may be useful if you are on AWS.
We had implemented PHP5.x session storage on AWS ElastiCache to centralise session storage across multiple servers.
It worked perfected until we moved to PHP7. It was difficult to configure for PHP7 and we were plagued with intermittent issues where it seemed that the session "failed/mismatched/got a bit confused" for a particular user and then they could not log in on that device until the old session expired.
We moved to using DynamoDb to store the session and no more issues. It is marginally slower but only noticeable at login (session storage) stage.
While this was going on, we implemented AWS cognito to replace our authentication and started to use the API-Gateway to deliver content via lambda python functions.
We use the PHP SDK to authenticate with Cognito and then we store the JWT in a cookie but still also use the PHP session to keep our legacy code working.
Now we have two stacks and the best of both worlds: PHP7 does it's bit and gets the main content to the user (very quickly). Then JS takes over and provides additional content using the JWT.
What I think is great about JWT is the fact that it can be passed around between these two stacks and used to authenticate the user in both cases.
Now we are wondering if it is worthwhile taking the plunge and switching over to the new JWT system entirely?
In PHP, we still use our legacy session but we also pass the token to cognito to authenticate it. It's a bit of extra security that is probably not necessary but it gives a warm cozy feeling. Then again, there are costs and maintenance with dynamoDb that could be saved.
In session authentication (or stateful authentication) you can store more data than token. But you have to store the session object somewhere and this makes the authentication centralized. Also, you need to have extra memory to store the sessions and this will give extra work to manage this memory. If your application grows, now you have to implement different designs based on your needs, implementing a session cache service, and storing the session on web application or database.
In Jwt or in general Stateless authentication, you do not store anything. You send the token with the request header. This makes it decentralized authentication. The drawback of this authentication is token revocation. Each token has an expiry time and if your token is stolen, it will be valid till it expires. You have to implement logic to minimize the risk.
I have started to design a RESTful API and I'm thinking about how to handle authentication. I want to use some kind of authentication token but I can't use OAuth o similar infrastructures so I have to handle it myself.
One of the requirements for this API is that it must have good performance, enough to handle a high volume of requests before there is the need to scale; my concern is how to make on each request the time needed to verify the token (integrity, expiration, IP Address, etc...) as little as possibile.
I suppose the token should some kind of hash and not an encrypted string containing the user information because the decryption time would be to heavy.
I've read that I could store the tokens in an in-memory hashtable where the key is the token and the value is the user info needed to process the request, but how can I make this work in a clustered environment where there will be an hashtable on each "node"?
Should I put tokens on a DB table an hit the DB every time also Handling manually the retention of expired tickets?
Probably it's not that important for the question but I'm using Spring MVC for the RESTfull API.
Thanks in advance.
I solved my problem by using both an in-memory cache and a db cache. Here is a summary of my solution that may help anyone with the same task.
the user logs in and in that moment a unique key is generated and sent back to the user.
that login token (which is basically a GUID with some processing) is also store in a db table with additional info like exipiration and with the user's info and roles. the same pieces of information are also store in memory (google guava hashtable where the token is the key)
the token must be passed along with every api call in the authorization token as #ipa suggested
the server code checks if the token is in its memory cache the user info are already available otherwise (e.g. the api call is done on another node in the cluster) the token is search in the token db
once the token is found you can check expiration, roles, etc...
This grants a good level of performance and security, the token can be generated with any arbitrary algorithm even a relative slow one since you don't have to recalculate it on every api call. Also this works with a stateless service wich can be scaled up horizontally.
I assume you use https and therefore all the traffic is encrypted. I'd suggest one of the following principles.
Basic Authentication
You can add the credentials in the Authorization header of the request. This credentials are encoded with Base64 (see below). This credentials could be sent on every request and then checked with your DB. To get this faster and less IO intensive you can still use a cache. Once I implemented an API like this without a cache and was able to handle thousands of requests per second.
Authorization: Basic QWxhZGRpbjpvcGVuIHNlc2FtZQ==
Authorization Token
There are different ways to implement your idea with a token. A common one is that every API user has his own token usually called api key which never expires. Another one is that you first have to authorize (Basic Authentication) and then get a token back which expires. This one is then used as api key for a certain time.
Either way you have to decide whether to use a cache or not. I would keep it simple and go for basic authentication and check the db every time. Almost every framework has very good support for this approach because it's simple http. If this causes performance issues (I'd recommend performance tests anyway) try to add the table with your credentials to the JPA cache. If you want to implement something with expiring tokens have a look at Infinispan.
You can store token in Redis. If you are going to store it in DB, make sure you optimise server (if you are managing it) for read operations. I have couple of implementation where folks have used key value store as well. Hashtable is also good idea.