AWS S3 Bucket Policy Deny + Open With Signed URL - amazon-s3

I have a public S3 bucket and I want to make all the files within folderA to be access via signed URL but remain folderB access publicly.
Below is the folder structure:
- Bucket (public)
- folderA <-- want to be access via Signed URL
- file1
- file2
- folderB <-- remain public
- file3
- file4
I tried insert the policy below into the bucket. However, after I inserted the policy, the files is totally unable to access publicly and with signed URL too.
{
"Version": "2012-10-17",
"Id": "Policy1596533716384",
"Statement": [
{
"Sid": "Stmt1596533705733",
"Effect": "Deny",
"Principal": "*",
"Action": [
"s3:GetObject",
"s3:GetObjectAcl"
],
"Resource": "arn:aws:s3:::bucket/folderA/*"
}
]
}
What configure is missing from the bucket policy ?
Any opinion is appreciate. Thanks.

All objects in Amazon S3 are private by default.
To make the contents of folderB public, you would need to add a Bucket Policy similar to:
{
"Version": "2012-10-17",
"Id": "Policy1596533716384",
"Statement": [
{
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject"
],
"Resource": "arn:aws:s3:::bucket/folderB/*"
}
]
}
This makes everything in folderB public.
As for making everything in folderA accessible via pre-signed URLs, your application would be responsible for generating the pre-signed URLs. Typically, the flow would be:
A user authenticates to your application
The user requests access to a private object (in your case, this would be something in folderA)
The application would check whether the user is entitled to access this object
If they are entitled to access the object, then the application would generate a pre-signed URL, which provides time-limited access to the private object
The application would give this signed URL to the user, either as a link or as part of an HTML page (eg in a <img> tag)
The user's browser would access the object via the provided URL. S3 would validate that the URL is correctly signed and that the time period has not expired. If so, it grants access to the private object.
Note: When you click the Open button in the Amazon S3 console, the console is generating a pre-signed URL to allow you to access a private object. This pre-signed URL is generated when you click the button and it will only work if your IAM User has been granted access to the object. That is, it uses the permissions of the person using the console to generate a pre-signed URL.
Your policy is saying "Deny everybody access to folderA". This includes the IAM User that is using the console, which is why they are unable to access the file, even when using a pre-signed URL. The pre-signed URL only has access to objects that are available to the IAM User that generated the pre-signed URL.

Related

Prevent access of S3 bucket for admin via console

I have some very highly confidential data that i want to store in s3 bucket.
I want to make policies ( bucket or iam whatever required) in such a way that no one ( not even admin) can read the contents of files in that bucket from aws console.
But i will have a program running on my host that needs to put and get data from that s3 bucket.
Also i will be using server side encryption of s3 but i can't use client side encryption of s3.
You are looking for something like this;
{
"Id": "bucketPolicy",
"Statement": [
{
"Action": "s3:*",
"Effect": "Deny",
"NotPrincipal": {
"AWS": [
"arn:aws:iam::111111111111:user/USERNAME",
"arn:aws:iam::111111111111:role/ROLENAME"
]
},
"Resource": [
"arn:aws:s3:::examplebucket",
"arn:aws:s3:::examplebucket/*"
]
}
],
"Version": "2012-10-17"
}
For test purposes make sure you replace arn:aws:iam::111111111111:user/USERNAME with your user arn. So in case you lock out everybody you can at least perform actions on the bucket.
arn:aws:iam::111111111111:role/ROLENAME should be replaced by the role arn which is attached to your EC2 instance (I am assuming that is what you mean by host).

Make Files Uploaded by SMTP to S3 Public

I'm using cloudmailin.com to upload attachments from emails to an S3 bucket. The bucket itself has public settings, but I am unable to access the uploaded data from the data's link as S3 apparently doesn't make files uploaded by other users public.
Currently, I'm using the bucket policy below, which is supposed to make the bucket's contents public:
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "AllowPublicRead",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::mjkaufer_cloudmailin/*"
}
]
}
This only works if I manually upload something to S3. If I manually upload something, I can view the link and it can open in a new window. But if it's uploaded from cloudmailin, I'm unable to view the file.
Does anybody know a workaround? I just need to make it such that files uploaded to a bucket by a 3rd party (cloudmailin) are visible.
Thanks.
The problem is that as CloudMailin uploads your attachments to AWS S3 it needs to set the ACL permissions on them. AWS offers a couple of ‘canned permissions' for this. The only two that are really of interest to CloudMailin are the following:
bucket-owner-full-control - this is what you’re currently set to, it grants the bucket owner full control over the uploaded attachment.
public-read - this will grant you full control as the bucket owner but will also set read only privileges for everyone.
If you log into CloudMailin head to your address list and click ‘manage' on the address you need to edit. Then head to ‘Edit Attachment Store’ and select ‘Public Read’ then CloudMailin will set this canned policy as the email attachments are uploaded to S3.

Amazon S3 Server Side Encryption Bucket Policy problems

I am using a bucket policy that denies any non-SSL communications and UnEncryptedObjectUploads.
{
"Id": "Policy1361300844915",
"Version": "2012-10-17",
"Statement": [
{
"Sid": "DenyUnSecureCommunications",
"Action": "s3:*",
"Effect": "Deny",
"Resource": "arn:aws:s3:::my-bucket",
"Condition": {
"Bool": {
"aws:SecureTransport": false
}
},
"Principal": {
"AWS": "*"
}
},
{
"Sid": "DenyUnEncryptedObjectUploads",
"Action": "s3:PutObject",
"Effect": "Deny",
"Resource": "arn:aws:s3:::my-bucket/*",
"Condition": {
"StringNotEquals": {
"s3:x-amz-server-side-encryption": "AES256"
}
},
"Principal": {
"AWS": "*"
}
}
]
}
This policy works for applications that support SSL and SSE settings but only for the objects being uploaded.
I ran into these issues:
CloudBerry Explorer and S3 Browser failed during folders and files RENAME in the bucket with that Bucket Policy. After I applied only SSL requirement in the bucket policy, those browsers successfully completed file/folder renaming.
CloudBerry Explorer was able to RENAME objects with the full SSL/SSE bucket policy only after I enabled in Options – Amazon S3 Copy/Move through the local computer (slower and costs money).
All copy/move inside Amazon S3 failed due to that restrictive policy.
That means that we cannot control copy/move process that is not originated from the application that manipulates local objects. At least above mentioned CloudBerry Options proved that.
But I might be wrong, that is why I am posting this question.
In my case, with that bucket policy enabled, S3 Management Console becomes useless. Users cannot create folders, delete them, what they can is only upload files.
Is there something wrong with my bucket policy? I do not know those Amazon S3 mechanisms that used for objects manipulating.
Does Amazon S3 treat external requests (API/http headers) and internal requests differently?
Is it possible to apply this policy only to the uploads and not to internal Amazon S3 GET/PUT etc..? I have tried http referer with the bucket URL to no avail.
The bucket policy with SSL/SSE requirements is a mandatory for my implementation.
Any ideas would be appreciated.
Thank you in advance.
IMHO There is no way to automatically tell Amazon S3 to turn on SSE for every PUT requests.
So, what I would investigate is the following :
write a script that list your bucket
for each object, get the meta data
if SSE is not enabled, use the PUT COPY API (http://docs.aws.amazon.com/AmazonS3/latest/API/RESTObjectCOPY.html) to add SSE
"(...) When copying an object, you can preserve most of the metadata (default) or specify new metadata (...)"
If the PUT operation succeeded, use the DELETE object API to delete the original object
Then run that script on an hourly or daily basis, depending on your business requirements.
You can use S3 API in Python (http://boto.readthedocs.org/en/latest/ref/s3.html) to make it easier to write the script.
If this "change-after-write" solution is not valid for you business wise, you can work at different level
use a proxy between your API client and S3 API (like a reverse proxy on your site), and configure it to add the SSE HTTP header for every PUT / POST requests.
Developer must go through the proxy and not be authorised to issue requests against S3 API endpoints
write a wrapper library to add the SSE meta data automatically and oblige developer to use your library on top of the SDK.
The later today are a matter of discipline in the organisation, as it is not easy to enforce them at a technical level.
Seb

How can I make a S3 bucket public (the amazon example policy doesn't work)?

Amazon provides an example for Granting Permission to an Anonymous User as follows (see Example Cases for Amazon S3 Bucket Policies):
{
"Version": "2008-10-17",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::bucket/*"
}
]
}
Within my policy I've changed "bucket" in ""arn:aws:s3:::bucket/" to "my-bucket".
However, once I try to access an image within a folder of that bucket, I get the following Access denied error:
This XML file does not appear to have any style information associated
with it. The document tree is shown below.
(if I explicitly change the properties of that image to public, then reload its url, the image loads perfectly)
What am I doing wrong?
Update #1: Apparently it has something to do with a third party site that I've given access to. Although it has all of the permissions as the main user (me), and its objects are in the same folder, with the exact same permissions, it still won't let me make them publicly viewable. No idea why.
Update #2: Bucket policies do not apply to objects "owned" by others, even though they are within your bucket, see my answer for details.
Update
As per GoodGets' comment, the real issue has been that bucket policies to do not apply to objects "owned" by someone else, even though they are in your bucket, see GoodGets' own answer for details (+1).
Is this a new bucket/object setup or are you trying to add a bucket policy to a pre-existing setup?
In the latter case you might have stumbled over a related pitfall due to the interaction between the meanwhile three different S3 access control mechanisms available, which can be rather confusing indeed. This is addressed e.g. in Using ACLs and Bucket Policies Together:
When you have ACLs and bucket policies assigned to buckets, Amazon S3
evaluates the existing Amazon S3 ACLs as well as the bucket policy
when determining an account’s access permissions to an Amazon S3
resource. If an account has access to resources that an ACL or policy
specifies, they are able to access the requested resource.
While this sounds easy enough, unintentional interferences may result from the subtle different defaults between ACLs an policies:
With existing Amazon S3 ACLs, a grant always provides access to a
bucket or object. When using policies, a deny always overrides a
grant. [emphasis mine]
This explains why adding an ACL grant always guarantees access, however, this does not apply to adding a policy grant, because an explicit policy deny provided elsewhere in your setup would still be enforced, as further illustrated in e.g. IAM and Bucket Policies Together and Evaluation Logic.
Consequently I recommend to start with a fresh bucket/object setup to test the desired configuration before applying it to a production scenario (which might still interfere of course, but identifying/debugging the difference will be easier in case).
Good luck!
Bucket policies do not apply files with other owners. So although I've given write access to a third party, the ownership remains them, and my bucket policy will not apply to those objects.
I wasted hours on this, the root cause was stupid, and the solutions mentioned here didn't help (I tried them all), and the AWS s3 permissions docs didn't emphasize this point.
If you have Requester Pays setting ON, you cannot enable Anonymous access (either by bucket policy or ACL 'Everyone'). You can sure write the policies and ACL and apply them and even use the console to explicitly set a file to public, but a non signed url will still get a 403 access denied 100% of the time on that file, until you uncheck requester pays setting in the console for the entire bucket (properties tab when bucket is selected). Or, I assume, via some API REST call.
Unchecked Requester Pays and now anonymous access is working, with referrer restrictions, ect. In fairness, the AWS console does tell us:
While Requester Pays is enabled, anonymous access to this bucket is disabled.
The issue is with your Action it should be in array format
Try this:
{
"Version":"2012-10-17",
"Statement":[
{
"Sid":"AddPerm",
"Effect":"Allow",
"Principal": "*",
"Action":["s3:GetObject"],
"Resource":["arn:aws:s3:::examplebucket/*"]
}
]
}
Pass your Bucket name in 'Resource'
If you're having this problem with Zencoder uploads, checkout this page: https://app.zencoder.com/docs/api/encoding/s3-settings/public
The following policy will make the entire bucket public :
{
"Version":"2012-10-17",
"Statement":[
{
"Sid":"AddPerm",
"Effect":"Allow",
"Principal": "*",
"Action":["s3:GetObject"],
"Resource":["arn:aws:s3:::examplebucket/*"]
}
]
}
If you want a specific folder under that bucket to be public using Bucket policies , then you have to explicitly make that folder/prefix as public and then apply the bucket policy as follows :
{
"Version":"2012-10-17",
"Statement":[
{
"Sid":"AddPerm",
"Effect":"Allow",
"Principal": "*",
"Action":["s3:GetObject"],
"Resource":["arn:aws:s3:::examplebucket/images/*"]
}
]
}
The above policy will allow public read to all of the objects under images , but you will not be able to access other objects inside the bucket.
I know it is an old question but I would like to add information that may still be relevant today.
I believe that this bucket should be a static site. Because of this, you must use a specific URL for your rules to be accepted. To do this, you must add a "website" to your URL. Otherwise, it will treat it just like an object repository.
Example:
With the problem pointed out:
https://name-your-bucket.sa-east-1.amazonaws.com/home
Without the problem pointed out:
http://name-your-bucket.s3-website-sa-east-1.amazonaws.com/home
Hope this helps :)
This works.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "PublicRead",
"Effect": "Allow",
"Principal": "*",
"Action": [
"s3:GetObject",
"s3:GetObjectVersion"
],
"Resource": "arn:aws:s3:::example-bucket/*"
}
]
}

How to set a bucket's ACL on S3?

I tried a couple of things: S3Browse, the RightAws Ruby gem and other tools. All allow granting access on an individual key basis, but I wasn't able to set the ACL on buckets. Actually, I set the ACL on the bucket, no errors are returned. But when I refresh or check in another tool, the bucket's ACL is reset to owner only.
I want to give read and write access to FlixCloud for an application I'm developing. They need the access to write the output files.
I was struggling with the ACL vs. Bucket Policy and found the following useful.
ACL
The ACL defines the permissions attached to a single file in your bucket. The Bucket Policy is a script that explains the permissions for any folder or file in a bucket. Use the bucket polcies to restrict hot linking, grant or deny access to specific or all files, restrict IP address, etc.
Edit the S3 Bucket Policy
Log into Amazon Web Services, click to S3 and click on the bucket name in the left column. View the bucket Properties panel at the bottom of the page. Click the button on the lower right corner that says "Edit bucket policy". This brings up a lightbox that you can paste the policy script into. If the script fails validation it will not save.
Sample Policy that enabled read access to everyone (useful if the bucket is being used as a content delivery network)
{
"Version": "2008-10-17",
"Id": "",
"Statement": [
{
"Sid": "AddPerm",
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": "s3:GetObject",
"Resource": "arn:aws:s3:::my_bucket_name/*"
}
]
}
Sample policy to prevent unauthorized hotlinking (third party sites linking to it) but allow anybody to download the files:
{
"Version":"2008-10-17",
"Id":"preventHotLinking",
"Statement":[ {
"Sid":"1",
"Effect":"Allow",
"Principal": {
"AWS":"*"
},
"Action":"s3:GetObject",
"Resource":"arn:aws:s3:::your.bucket.name/*",
"Condition":{
"StringLike": {
"aws:Referer": [
"http://yourwebsitename.com/*",
"http://www.yourwebsitename.com/*"
]
}
}
}]
}
Generate a Policy
http://awspolicygen.s3.amazonaws.com/policygen.html
Sample Bucket Policies
http://docs.amazonwebservices.com/AmazonS3/latest/dev/index.html?AccessPolicyLanguage_UseCases_s3_a.html
Yup, just checked it again after 10 min. ACL remains as configured. I guess this is something at your end then. Try different account/workstation.
I have just double checked that for you - S3fm was able to change the ACL successfully. I used their email s3#flixcloud.com as userid. You can see the user in the list afterwords as flixclouds3.