Granting read access to the Authenticated Users group for a file - amazon-s3

How do I grant read access to the Authenticated Users group for a file? I'm using s3cmd and want to do it while uploading but I'm just focusing directly on changing the acl. What should I put in for http://acs.amazonaws.com/groups/global/AuthenticatedUsers? I have tried every combination of AuthenticatedUsers possible.
./s3cmd setacl
--acl-grant=read:http://acs.amazonaws.com/groups/global/AuthenticatedUsers
s3://BUCKET/FILE
./s3cmd setacl
--acl-grant=read:AuthenticatedUsers
s3://BUCKET/FILE

This doesn't seem to be possible with s3cmd. Instead I had to switch to the aws cli tools.
Here are the directions to install them:
http://docs.aws.amazon.com/cli/latest/userguide/installing.html
It's possible to set the acl to read by authenticated users during upload with the command:
aws s3 cp <file-to-upload> s3://<bucket>/ --acl authenticated-read
Plus a whole load of other combinations you can check out here:
http://docs.aws.amazon.com/cli/latest/reference/s3/index.html#cli-aws-s3

The following command works for me with s3cmd version 1.6.0:
s3cmd setacl s3://<bucket>/<file-name> --acl-grant='read:http://acs.amazonaws.com/groups/global/AuthenticatedUsers' for an individual file.
s3cmd setacl s3://<bucket>/<dir-name> --acl-grant='read:http://acs.amazonaws.com/groups/global/AuthenticatedUsers' --recursive
for all files in a directory.

This is from http://s3tools.org/s3cmd:
Upload a file into the bucket ~$ s3cmd put addressbook.xml
s3://logix.cz-test/addrbook.xml File 'addressbook.xml' stored as
s3://logix.cz-test/addrbook.xml (123456 bytes) Note about ACL
(Access control lists) — a file uploaded to Amazon S3 bucket can
either be private, that is readable only by you, possessor of the
access and secret keys, or public, readable by anyone. Each file
uploaded as public is not only accessible using s3cmd but also has a
HTTP address, URL, that can be used just like any other URL and
accessed for instance by web browsers.
~$ s3cmd put --acl-public --guess-mime-type storage.jpg
s3://logix.cz-test/storage.jpg File 'storage.jpg' stored as
s3://logix.cz-test/storage.jpg (33045 bytes) Public URL of the
object is: http://logix.cz-test.s3.amazonaws.com/storage.jpg
Now anyone can display the storage.jpg file in their browser. Cool, eh?
try changing public to authenticated and that should work.
see http://docs.amazonwebservices.com/AmazonS3/latest/dev/ACLOverview.html#CannedACL
it explains on amazon side how to use their ACLs, supposedly if you use public in s3cmd - this would translate to public-read in amazon, so authenticated should translate to authenticated-read.

If you're willing to use Python, the boto library provides all the functionality to get and set an ACL; from the boto S3 documentation:
b.set_acl('public-read')
Where b is a bucket. Of course in your case you should change 'public-read' to 'authenticated-read'. You can do something similar for keys (files).

If you want to do it at bucket level you can do -
aws s3api put-bucket-acl --bucket bucketname --grant-full-control uri=http://acs.amazonaws.com/groups/global/AuthenticatedUsers
Docs - http://docs.aws.amazon.com/cli/latest/reference/s3api/put-bucket-acl.html

Here is an example command that will set the ACL on an S3 object to authenticated-read.
aws s3api put-object-acl --acl authenticated-read --bucket mybucket --key myfile.txt
.

Related

Updating files for my static website on S3

I'm trying to update a static website I'm hosting on amazon AWS S3 - just need to put a new version of my resume up there. I've gone through the documentation and it seems as though I need to 'invalidate' the file - but all the guides I'm finding only talk about using cloudfront, which is a service I don't use.
So for a static website where I need to update a single file, how do I do that without cloudfront?
You can upload the file directly to S3 through the AWS S3 Console, programmatically using a package for python, ruby, etc., or using the AWS Command Line.
If you are using the AWS Command Line, you can upload a file to s3 using these commands:
$ aws configure
AWS Access Key ID [None]: AKIAIOSFODNN7EXAMPLE
AWS Secret Access Key [None]: wJalrXUtnFEMI/K7MDENG/bPxRfiCYEXAMPLEKEY
Default region name [None]: us-east-1
Default output format [None]: json
$ aws s3 cp myvideo.mp4 s3://mybucket/

How to use S3 adapter cli for snowball

I'm using s3 adapter to copy files from a snowball device to local machine.
Everything appears to be in order as I was able to run this command and see the bucket name:
aws s3 ls --endpoint http://snowballip:8080
But besides this, aws doesn't offer any examples for calling cp command. How do I provide the bucket name and the key with this --endpoint flag.
Further, when I ran this:
aws s3 ls --endpoint http://snowballip:8080/bucketname
It returned 'Bucket'... Not sure what that means because I expect to see the files.
I can confirm the following is correct for snowball and snowball edge, as #sqlbot says in the comment
aws s3 ls --endpoint http://snowballip:8080 s3://bucketname/[optionalprefix]
References:
http://docs.aws.amazon.com/cli/latest/reference/
http://docs.aws.amazon.com/snowball/latest/ug/using-adapter-cli.html
Just got one in the post

How do I modify object permissions using aws s3 command?

It seems to me that aws s3 does not have a dedicated command to modify object permissions. I have some files that are uploaded via s3fuse. Afterwards, I would like to make them public. Is there any way to make those files public using aws s3 command?
Thanks.
I found out how to do this. It seems there is another cli called aws s3api, that mimics the underlying s3 api. Using aws s3api put-object-acl http://docs.aws.amazon.com/cli/latest/reference/s3api/put-object-acl.html command, I can change object permissions directly.
aws s3api put-object-acl --acl public-read --bucket mybucket --key targets/my_binary

migration from s3 to google cloud storage and ACL

I am currently planning a possible migration from s3 to google cloud storage(g-c-s). I have decided to spin up a gce instance and use gsutil to rsync several millions of files. I would like to know if the permission will be preserved or not.
for example if a file has public read on amazon s3 what will be the acl on g-c-s.
thanks
If you use the gsutil cp command you can specify a canned ACL on the command line, like this:
gsutil cp -R -a public-read s3://your-s3-bucket gs://your-gs-bucket
The rsync command doesn't have a way to do that. However, the other option is you can set a default object ACL on the destination bucket, using the gsutil defacl command. Then you can use gsutil cp without specifying the canned ACL, or you could use gsutil rsync.

How can I download a file from an S3 bucket with wget?

I can push some content to an S3 bucket with my credentials through S3cmd tool with s3cmd put contentfile S3://test_bucket/test_file
I am required to download the content from this bucket in other computers that don't have s3cmd installed on them, BUT they have wget installed.
when I try to download some content from my bucket with wget I get this:
https://s3.amazonaws.com/test_bucket/test_file
--2013-08-14 18:17:40-- `https`://s3.amazonaws.com/test_bucket/test_file
Resolving s3.amazonaws.com (s3.amazonaws.com)... [ip_here]
Connecting to s3.amazonaws.com (s3.amazonaws.com)|ip_here|:port... connected.
HTTP request sent, awaiting response... 403 Forbidden
`2013`-08-14 18:17:40 ERROR 403: Forbidden.
I have manually made this bucket public through the Amazon AWS web console.
How can I download content from an S3 bucket with wget into a local txt file?
You should be able to access it from a url created as follows:
http://{bucket-name}.s3.amazonaws.com/<path-to-file>
Now, say your s3 file path is:
s3://test-bucket/test-folder/test-file.txt
You should be able to wget this file with following url:
http://test-bucket.s3.amazonaws.com/test-folder/test-file.txt
Go to S3 console
Select your object
Click 'Object Actions'
Choose 'Download As'
Use your mouse right-click to 'Copy Link Address'
Then use the command:
wget --no-check-certificate --no-proxy 'http://your_bucket.s3.amazonaws.com/your-copied-link-address.jpg'
AWS cli has a 'presign' command that one can use to get a temporary public URL to a private s3 resource.
aws s3 presign s3://private_resource
You can then use wget to download the resource using the presigned URL.
Got it ... If you upload a file in an S3 bucket with S3CMD with the --acl public flag then one shall be able to download the file from S3 with wget easily ...
Conclusion: In order to download with wget, first of one needs to upload the content in S3 with s3cmd put --acl public --guess-mime-type <test_file> s3://test_bucket/test_file
alternatively you can try:
s3cmd setacl --acl-public --guess-mime-type s3://test_bucket/test_file
notice the setacl flag above. THAT WILL set the file in s3 accessible publicly
then you can execute the wget http://s3.amazonaws.com/test_bucket/test_file
I had the same situation for couple of times. It’s the fastest and the easiest way to download any file from AWS using CLI is next command:
aws s3 cp s3://bucket/dump.zip dump.zip
File downloaded way faster than via wget, at least if you are outside of US.
I had the same error and I solved it by adding a Security Groups Inbound rule:
HTTPS type at port 443 to my IP address ( as I'm the only one accessing it ) for the subnet my instance was in.
Hope it helps anyone who forgot to include this
Please make sure that the read permission has been given correctly.
If you do not want to enter any account/password, just by wget command without any password, make sure the permission is like the following setting shows.
By Amazon S3 -> Buckets -> Permisions - Edit
Check the Object for "Everyone (public access)" and save changes.permission setting like this - screenshot
or choose the objest and go to "Actions" -> "Make public", would do the same thing under permission settings.
incase you do not have access to install aws client on ur Linux machine try below method.
got to the bucket and click on download as button. copy the link generated.
execute command below
wget --no-check-certificate --no-proxy --user=username --ask-password -O "download url"
Thanks
you have made the bucket public, you need to also make the object public.
also, the wget command doesn't work with the S3:// address, you need to find the object's URL in AWS web console.
I know I'm too late to this post. But thought I'll add something no one mentioned here.
If you're creating a presigned s3 URL for wget, make sure you're running aws cli v2.
I ran into the same issue and realized s3 had this problem
Requests specifying Server Side Encryption with AWS KMS managed keys require AWS Signature Version 4
This gets resolved once you presign on aws cli v2
The simplest way to do that is to disable Block all public firstly.
Hit your bucket name >> go to Permissions >> Block public access (bucket settings)
If it is on >> hit Edit >> Uncheck the box, then click on Save changes
Now hit the object name >> Object action >> Make public using ACL >> then confirm Make public
After that, copy the Object URL, and proceed to download
I hope it helps the future askers. Cheers
I had the same mistake
I did the following :
created IAM role > AWS Service type > AmazonS3FullAccess policy inside
applied this role to the EC2 instance
in the the Security Groups opened inbound HTTP and HTTPS to Anywhere-IPv4
made the S3 bucket public
profit! wget works! ✅