would love to hear your ideas.
In this project, multiple users (let's say 1000 users) will upload files into the same storage account (AWS S3, Azure Blob Storage or DigitalOcean Spaces) using a Windows desktop app C#
The desktop app does have user authentication from a Web API
Questions
Is it correct that each user will have his/her own bucket?
What is the best way to securely introduce API key and bucket information into the desktop app so that files will be uploaded to the correct bucket and storage account?
Think about the structure of your S3 bucket and how you would later identify each object, which a user uploaded.
I would create for each user an initial key, which a user is able to upload the files, e.g.
username1/object1
/object2
/objectx
username2/object1
username3/object1
usernamex/objectx
This will give you the possibility, if a user is deleted, that you can just delete all objects with that username too. If you are using a generated key to identify the user, than you also can use the keyID instead of username.
The most interesting question is on how you will secure this, so that no other user will be able to see objects from others. If you have a underlying API, than it's "easy"... give the API the access to the S3 bucket and secure the requests, that only those objects will be listed for which the username or keyID matches.
If you are using IAM users (or roles), than you have automatically generate a policy for each base key (username1 or keyID) for the specific actions.
If you set up something like that, please be really sure to harden your security and also try to enable logging of this bucket to be sure, that user1 can't access objects from user2.
Related
I am looking for a solution on the best practice that needs to be followed in AWS S3 access by third party who do not have account in S3.
In my case there are REST interface which would need to provide the link of images .This images resides on AWS S3. Based on the identity of the caller is there a way we can give access to the user. I would not want to make the access level of the bucket to public.
Say if we get a call from user X ( may be we ask them to set a new header ) we allow them the access to the bucket.
As this API is enterprise and we have partners using this API we would want only some of the identified callers to have access to the images.
Any pointers will help a lot.
Signed S3 URL's, make the bucket private, only accessible to your API via an IAM role, if the API is running on EC2, lambda etc.
Your API would do the authentication and authorization, then provide the caller a signed s3 url to download the image.
When you create a pre-signed URL for your object, you must provide your security credentials, specify a bucket name, an object key, specify the HTTP method (GET to download the object) and expiration date and time. The pre-signed URLs are valid only for the specified duration.
Anyone who receives the pre-signed URL can then access the object. For
example, if you have a video in your bucket and both the bucket and
the object are private, you can share the video with others by
generating a pre-signed URL.
I'm trying to arrive at a situation, where
one class of users can upload files that are subsequently not publicly available
another class of users can upload files that are publicly available.
I think I need to use two IAM users:
the first which has putObject permissions only and where I bake the secret key into javascript. (I use the AWS SDK putObject here, and bake in the first secret key)
the other where I keep the secret key on the server, and provide signatures for uploading to signed-in users of the right category. (I ended up using a POST command for this with multipart form-data, as I could not understand how to do it with the SDK other than baking in the second secret key, which would be bad as files can be uploaded and downloaded)
But I'm struggling to set up bucket permissions that support some files being publicly available while others are not at all.
Is there a way, or do I need to use separate buckets?
Update
Based on the first comment, I tried to add "acl": "public-read" to my policy and POST form data fields. The signatures are matching correctly, but I am now getting a forbidden response from AWS, which I don't get when this field is absent (but then the uploads are not publicly visible)
What is the proper way of delegating file access authentication from S3 to our authentiation service?
For example: web site's user(he have our session id in headers) sending request to S3 to get file by url. S3 sends request to our authentication service asking if user with such headers can access that file, and if our auth service allow getting that file it will be downloaded.
There are a lot of information about presigned requests but absolutely nothing about s3 quering with "hidden" authentication.
If a file has been made public on S3, then of course anyone can download it, using a direct link to the file.
If the file is not public, then there needs to be some type of authentication. There are really only two ways a file from S3 can be obtained if it is not public, one is via a pre-signed url, and the other is to be an Amazon user who has access to S3. Obviously this is how it works when you yourself want to access an object on S3, you must provide your access key and a signature in the header of the GET request. You can grant other users access to S3 via Amazon IAM, which is more like the 'hidden' authentication you mentioned. Via the IAM route, there are different ways of providing access including Federated Users. Visit this link to learn more:
http://docs.aws.amazon.com/AmazonS3/latest/dev/MakingAuthenticatedRequests.html
If you are simply trying to provide a authenticated user access to a file, the best and easiest way to do that would be to create a pre-signed url with an expiration time. The expiration time can be something short, like 10 minutes or even 1 minute, to prevent the user from passing the link to others.
For example, I have a website with User A and B.
Both of them can login to my website using my own login system.
How do I make certain files from S3 accessible only to User A once he login to my website?
Note: I saw "Permission" in AWS Management Console with "Authenticated Users" option but it seems that it's meant for other S3 users only, is it something I can use to achieve my goal?
You need to use Amazon IAM - you can define what part of any S3 bucket A can see, as well as B and each will not have access to do 'anything'. In general you should never use the account ID and secret for anything, always make an IAM user have just whats needed to run your stuff. The admin user likely does not need EC2 or SQS, or SimpleDB, etc.
Federated access is great for allowing arbitrary users to sign into your website and only be granted access for say 12 hours. They get special AWSIDs for that access that will work only on the section of S3 you let them look at.
I'm new to Amazon S3, and do not want to give more information than is necessary for the team to whom I'm outsourcing a project. They are building an image hosting site, and would need access to my S3 credentials - what exactly would the devs need to have access to? Just my Access Key ID?
Thanks.
They'll need an Access Key ID & corresponding Secret Access Key.
You can generate a unique one for them to use via the Security Credentials Page in the Account section of the website.
When they're done, you can delete there key and make a different one to use for the live site. Just make sure that when they develop the app they put the key information in a configuration file so you can change it when they're done.