I have uploaded a number of directories that contain images, what is the correct way to modify the content-type to "image/jpeg" without having to re-upload the images?
I tried:
s3cmd modify s3://ccc-public/catalog/cropped/EP01L.jpg --add-header="Cache-Control:max-age=1296000" --mime-type="image/jpeg" --recursive -vvv
but the Content-Type always comes as binary/octet-stream
any advise much appreciated
These days, it is recommended to use the official Amazon Command Line Interface (CLI).
The aws s3 cp command includes options for specifying mime-type, but by default the mime type of a file is guessed when it is uploaded.
s3cmd has recently (as in, this past weekend) fixed this, but the code is not yet upstream. Please try with this branch:
https://github.com/mdomsch/s3cmd/tree/bug/content-type
With a little more testing, this will get merged into upstream.
Then, your command should work exactly as you expect.
-mdomsch, s3cmd maintainer
Ran into this recently as well and used some of the info here How can i change AWS S3 content type only for audio files
For us, it came down to ensuring that you change the content-type with the following
--add-header='content-type':'image/png'
I want to emphasize using the lower case content-type vs Content-Type here. That's what ended up making a difference.
--add-header did not help for me. But the below worked.
s3cmd put style.css -m 'text/css' s3://mybucket
Related
I have a zip file with the size 1 GB on S3 bucket. After downloading, I can't seem to unzip it. It always says
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
Later, I download it again, using s3cmd this time. It says
WARNING: MD5 signatures do not match: computed=384c9a702c2730a6b46d21606137265d, received="b42099447c7a1a390d8e7e06a988804b-18"
Is there any s3 limitation I need to know or this is a bug?
This question seems dead, but I'll ask it for anyone landing here:
Amazon S3's multipart uploads (those suitable for big files) produce ETag values which no longer matches file's MD5, so if you're using this as checksum (as it seems looking at your received MD5) it won't work.
Best you can do for validation is ensuring ContentMD5 header is added to every part's header on your multipart upload ensuring file does not get corrupted during upload, and adding your own MD5 metadata field for checking data after download.
Thanks #ergoithz for reminding me that I had this question :)The problem is already fixed, with AWS SDK for nodejs being the problem. Apparently it cannot upload large files using stream data fs.createReadStream(), so I switched to using Knox where it worked perfectly
I want to synchronise files from a location in Perforce to an S3 bucket. I've set up a Jenkins job that uses s3cmd to sync files from Perforce to S3.
The problem I'm having is that the auto-detection of mime-types is not working as I would like. Is there a relatively straightforward way to override the detection with my own mime-type mapping? Let's say I want all .xml.gz files to be mime-typed as application/x-gzip.
How do I do this without rolling my own equivalent of s3cmd sync? Is there a way to do this with s3cmd, or is there another tool that's good for syncing a folder to S3 that has this functionality?
EDIT:
This isn't what I was looking for, but if anyone else has the problem it at least gets past the issue. I modified S3.py and after the snippet that looks like this:
if not content_type:
content_type = self.config.default_mime_type
I added:
# JUN-DAI'S HACK TO GET .gz mimetypes correct.
# I couldn't find another way to do this as the mimetypes library strips out the ".gz" suffix and determines the mimetype of the underlying file without it.
if filename.endswith(".gz"):
content_type = "application/x-gzip"
print "Setting content-type of {0} to {1}".format(filename, content_type)
Now you can use the options --no-guess-mime-type --default-mime-type "application/x-gzip" with s3cmd sync or put.
I working with mediawiki.
I want to change upload directory path to aws s3, i tried these two extensions but i getting some warning message.
I dont know these extension are working correctly.
https://www.mediawiki.org/wiki/Extension:LocalS3Repo and
https://www.mediawiki.org/wiki/Extension:AWS
If anybody is working with these extension or if you achieved these in any other ways
please explain me
I have been succesfully using the method described here, though in step 6, rather than using an apache rewrite, I changed the image paths in LocalSettings.php.
(It was quite a lot of work though, and I never figured out a way to the the cache-control and expires headers on the files, which was the real reason why I wanted to do it to begin with.)
I'm trying to figure out right now how to backup some data to S3.
We have a local backup system implemented using rsnapshot and that works perfectly. We're trying to use s3cmd with the --sync option to mimic rsync to transfer the files.
The problem we're having is that symlinks aren't created as symlinks, it seems to be resolving them to the physical file and uploading that instead. Does anyone have any suggestions as to why this would happen?
Am I missing something obvious? Or is it that S3 just isn't suited to this sort of operation? I could setup an EC2 instance and attach some EBS, but it'd be preferable to use S3.
Amazon S3 itself doesn't have the concept of symlinks, which is why I suspect s3cmd uploads the physical file. Its a limitation of S3, not s3cmd.
I'm assuming that you need the symlink itself copied though? If that's the case, can you gzip/tar your directory with symlink and upload that?
There is no symlinks available on S3 but what you can use is google's solution which creates a based file system using S3 (FUSE) . More information here :
https://code.google.com/p/s3fs/wiki/FuseOverAmazon
and here:
http://tjstein.com/articles/mounting-s3-buckets-using-fuse/
I hope it helps
Try using the -F, --follow-symlinks OPTiON when using sync. This worked for me.
I have added the "Content-Encoding: gzip" header to my S3 files and now when I try to access them, it returns me a "Error 330 (net::ERR_CONTENT_DECODING_FAILED)".
Note that my files are simply images, js and css.
How do I solve that issue?
You're going to have to manually gzip them and then upload them to S3. S3 doesn't have the ability to gzip on the fly like your web server does.
EDIT: Images are already compressed so don't gzip them.
Don't know if you are using Grunt as deployment tool but, use this to compress your files:
https://github.com/gruntjs/grunt-contrib-compress
Then:
https://github.com/MathieuLoutre/grunt-aws-s3
To upload compressed files to Amazon S3. Et voila!