I have a mobile app which authenticates users on my server. I'd like to store images of authenticated users in Google Cloud Storage bucket but I'd like to avoid uploading images via my server to google bucket, they should be directly uploaded (or downloaded) from the bucket.
(I also don't want to display another Google login to users to grant access to their bucket)
So my best case scenario would be that when user authenticates to my server, my server also generates short lived access token to specific Google storage bucket with read and write access.
I know that service accounts can generate accessTokens but I couldn't find any documentation if it is a good practice top pass these access tokens from server to client app and if it is possible to limit scope of the access token to specific bucket.
I found authorization documentation quite confusing and asking here what would be best practice approach to achieve access to the cloud storage for my case?
I think you are looking for signed urls.
A signed URL is a URL that provides limited permission and time to
make a request. Signed URLs contain authentication information in
their query string, allowing users without credentials to perform
specific actions on a resource.
Here you can see more about them in GCP. Here you have an explanation of how you can adapt them for your program.
Related
I have a simple and free Google user account like this: my.name#gmail.com.
Im working with SomeCompany with a billable Google account. This company exposed a bucket to which im supposed to upload someFile.txt. The bucket url looks like this: https://console.cloud.google.com/storage/browser/SomeCompany-multi-44444
or, alternatively gsutil:
gs://SomeCompany-multi-44444
I can access and use this bucket (after auth prompt) from my browser.
Question: Can i access this bucket using API (preferably using Python oauth2client or gcloud) without creating (billable) Service account of my own? How? I fail to understand how to create an API authentication to this bucket without creating a service account which requires credit card. Is there something that SomeCompany have to do in order for me to succeed?
Yes, it's possible and reasonable.
Service accounts and user accounts are all Google identities (as are Groups).
The difference is that service accounts use two-legged auth and have a simpler flow. But, a user account is a valid identity and yours has been authorized to use the bucket.
The difference is that you need to use three-legged auth and exchange your credentials for an access token that you may use to authenticate to the service.
Here's a link to the Python Cloud Client Library section on using 3-legged (User) auth.
I am currently developing a client-side app where users login using e-mail/password against MongoDB Atlas. The backend runs completely serverless.
All logged in users should be able to upload and retrieve images from GCP - Storage bucket without a visible login, which means the application should authenticate for every user on the background.
I was thinking about using Google Service Accounts in combination with auth0, but I don't know where to start...
If someone could help me tell where to start, that would be great :)
The question is difficult to answer. However, here some insights.
The prefered way is to have a serverless backend, AppEngine standard, Cloud Run or Cloud Function for doing this. The user performs its authentication and then exchange security token between the frontend and the backend. When the user want to reach a GCP resource, it asks the backend, which request the request thanks to its own service account.
By the way, it's easy to trace the user request and to serve him only the resources related to it. And you have only 1 service account, for the backend
If you grant access to a bucket to a user, it could download all the files (But maybe there is one bucket per user?). If you chose to limit object access with ACL, the management is complex.
You don't need to have a service account per user (and in any case, you have a quota to 100 service accounts per project), you can use Cloud Identity Platform (CIP) instead of your MongoDB database for authentication (CIP don't perform authorization, you should use MongoDB for authorization and other stuffs related to authenticated user). CIP is Firebase Auth rebranded
O.k. my fellow devops and coders. I have spent the last week trying to figure this out with Google (GCP) Cloud Storage objects. Here is my objective.
The solution needs to be light weight as it will be used to download images inside a docker image, hence the curl requirement.
The GCP bucket and object needs to be secure and not public.
I need a "long" lived ticket/key/client_ID.
I have tried the OAuth2.0 setup that Google's documentation mentions but everytime I want to setup an OAuth2.0 key it I do not get the option to have the "offline" access. AND to top it off it requires you to put in source URL's that will be accessing the auth request.
Also Google Cloud Storage does not support the key= like some of their other services. So here I have a an API KEY for my project as well as an OAuth JSON file for my service user and they are useless.
I can get a curl command to work with the temp OAuth bearer key but I need a long term solution for this.
RUN curl -X GET \
-H "Authorization: Bearer ya29.GlsoB-ck37IIrXkvYVZLIr3u_oGB8e60UyUgiP74l4UZ4UkT2aki2TI1ZtROKs6GKB6ZMeYSZWRTjoHQSMA1R0Q9wW9ZSP003MsAnFSVx5FkRd9-XhCu4MIWYTHX" \
-o "/home/shmac/test.tar.gz" \
"https://www.googleapis.com/storage/v1/b/mybucket/o/my.tar.gz?alt=media"
A long term key/ID/secret that will allow me to download a GCP bucket object from any location.
The solution needs to be lightweight as it will be used to download
images inside a docker image, hence the curl requirement.
This is a vague requirement. What is lightweight? No external libraries, everything written in assembly language, must fit in 1 KB, etc.
The GCP bucket and object needs to be secure and not public.
This normal requirement. With some exceptions (static file storage for websites, etc) you want your buckets to be private.
I need a "long" lived ticket/key/client_ID.
My advice is to stop thinking "long-term keys". The trend in security is to implement short-term keys. In Google Cloud Storage, seven-days is considered long-term. 3600 seconds (one hour) is the norm almost everywhere in Google Cloud.
For Google Cloud Storage you have several options. You did not specify the environment so I will include both user credentials, service account, and presigned-url based access.
User Credentials
You can authenticate with User Credentials (eg username#gmail.com) and save the Refresh Token. Then when an Access Token is required, you can generate one from the Refresh Token. In my website article about learning the Go language, I wrote a program on Day #8 which implements Google OAuth, saves the necessary credentials and creates Access Tokens and ID Tokens as required with no further "login" required. The comments in the source code should help you understand how this is done. https://www.jhanley.com/google-cloud-and-go-my-journey-to-learn-a-new-language-in-30-days/#day_08
This is the choice if you need to use User Credentials. This technique is more complicated, requires protecting the secrets file but will give you refreshable long term tokens.
Service Account Credentials
Service Account JSON key files are the standard method for service-to-service authentication and authorization. Using these keys, Access Tokens valid for one hour are generated. When they expire new ones are created. The max time is 3600 seconds.
This is the choice if you are programmatically accessing Cloud Storage with programs under your control (the service account JSON file must be protected).
Presigned-URLs
This is the standard method of providing access to private Google Cloud Storage objects. This method requires the URL and generates a signature with an expiration so that objects can be accessed for a defined period of time. One of your requirements (which is unrealistic) is that you don't want to use source URLs. The max time is seven-days.
This is the choice if you need to provide access to third-parties to access your Cloud Storage Objects.
IAM Based Access
This method does not use Access Tokens, instead, it uses Identity Tokens. Permissions are assigned to Cloud Storage buckets and objects and not to the IAM member account. This method requires a solid understanding of how Identities work in Google Cloud Storage and is the future direction for Google security - meaning for many services access will be controlled on a service/object basis and not via roles that grant wide access to an entire service in a project. I talk about this in my article on Identity Based Access Control
Summary
You have not clearly defined what will be accessing Cloud Storage, how secrets are stored, if the secrets need to be protected from users (public URL access), etc. The choice depends on a number of factors.
If you read the latest articles on my website I discuss a number of advanced techniques on Identity Based Access Control. These features are starting to appear on a number of Google Services in the beta level commands. This includes Cloud Scheduler, Cloud Pub/Sub, Cloud Functions, Cloud Run, Cloud KMS and soon more. Cloud Storage supports Identity Based Access which requires no permissions at all - the identity is used to control access.
I'm hosting a static website on S3 that uses an API. My auth token for the API is stored in a JS file, but I want to keep that obscured from public users, but NOT from my application.
At the moment, it looks like you need to make S3 buckets (and all of their files) publicly accessible by everyone, but I want to mask my config file. Is this possible, and if so, what is the best way to do it?
Thanks!
Amazon provides a service called Lambda. It is a serverless computing. You use can can be solved using this.
You can write an auth function in Lambda where you can place the api auth token.
You are not really going to be able to completely hide your token, no matter what you do by masking it etc, ultimately your browser is issuing an API call and passing along the credentials which anyone that cares to look for it can see it.
What you want to do is use something like aws cognito to generate temporary, restricted tokens for each user, even anonymous users.
Cognito Identity supports the creation and token vending process for
unauthenticated users as well as authenticated users. This removes the
friction of an additional login screen in your app, but still enables
you to use temporary, limited privilege credentials to access AWS
resources.
https://aws.amazon.com/cognito/faqs/
If you do this, someone can still see the token being used, but it is time and permission limited - not the keys to the kingdom, so they can't do much with it.
In development, we have successfully written an image to S3 bucket and then get the url back so we can store the url.
Now that we're moving into production, we need to not include the access and secret keys.
Everything is saying to use Cognito, but we don't want to authenticate users. We just want images that are stored in the app to be backed up online and store the url. Every user can dump images in the same bucket because they will never access the images, just download via url.
Does anyone know, is there an invisible way to establish this connection securely to only read and write from the app without forcing users to login?
You may want to checkout CognitoIdentityService. CognitoIdentity allows developers to get temporary credentials to call other AWS service. So, developers need not put access and secret keys within the application. They can simply use the credentials provided by Cognito.
With Cognito, developers can configure if they wish the users to be authenticated or not. With CognitoIdentity authentication is optional. If user not authenticated, user will be given a new identityId every time. For authenticated users, identityId remains same. But it can easily be used for either case to get temporary credentials.