FineUploader - Error on multi-part upload to S3 - amazon-s3

I am using FineUploader to upload to S3. I have everything working including deletes. However, when I upload larger files that get broken into multi-part uploads, I get the following error in the console (debugging turned on):
Specific problem detected initiating multipart upload request for 0: 'The request signature we calculated does not match the signature you provided. Check your key and signing method.'.
Can someone point me in the right direction as what I should check for settings, or what additional info you might need?

Since you haven't included anything really specific to your setup, code, or the failing request, my best guess is that your server isn't returning a proper signature response for uploads made to the S3 REST API (which is used for larger files). You'll need to review that procedure for generating a response to this type of signature request.
Here's the relevant section from Fine Uploader's S3 documentation:
Fine Uploader S3 uses Amazon S3’s REST API to initiate, upload,
complete, and abort multipart uploads. The REST API handles
authentication by signing canonically formatted headers. This signing
is something you need to implement server-side. All your server needs
to do to authenticate and supported chunked uploads direct to Amazon
S3 is sign a string representing the headers of the request that Fine
Uploader sends to S3. This string is found in the payload of the
signature request:
{ "headers": /* string to sign */ }
The presence of this property indicates to your sever that this is, in
fact, a request to sign a REST/multipart request and not a policy
document.
This signature for the headers string differs slightly from the policy
document signature. You should NOT base64 encode the headers string
before signing it. All you must do, server-side, is generate an HMAC
SHA1 signature of the string using your AWS secret key and then base64
encode the result. Your server should respond with the following in
the body of an ‘application/json’ response:
{ "signature": /* signed headers string */ }

Related

SignatureDoesNotMatch The request signature we calculated does not match the signature you provided. Check your key and signing method

I got this error when I sent an HTTP post request for uploading files directly from the browser to the AWS s3 bucket. I think the problem is with calculating signature and I am not sure what data/parameters should be used to calculate signature and how signature should be calculated and I am using Django.
SignatureDoesNotMatch
The request signature we calculated does not match the signature you provided. Check your key and signing method.
In my case was the aws credentials and secret. If you using localstack try to put your key and secret 'test'/'test', in my case it works. If not check if your credentials are ok

Cannot do multipart upload to S3 bucket with SSE-KMS encryption (using .NET SDK)

I can successfully send the InitiateMultipartUploadRequest and get InitiateMultipartUploadResponse back, but then get Access Denied error when sending the 1st UploadPartRequest.
Note that all of the below cases upload the document successfully:
Exactly the same code (i.e. using multipart upload), but to a different bucket that uses SSE-S3 encryption.
Using low-level API and uploading the document in one go, i.e. creating PutObjectRequest and then calling amazonS3Client.PutObjectAsync(putObjectRequest).
Using high-level API TransferUtility class.
Maybe the encryption key was not forwarded in the call properly.

Couldn't call aws api gateway from postman

I have created an api in amazon api gateway service with s3 proxy, and created a method post to upload a file to s3 using the document. Deployed the API and then using that url i tried to call the api from postman. But i couldn't post the file and it returns an error 'missing authentication token'.
I set authorization as NONE.
Then it returns an Unexpected "<" error.
Ah, okay. S3 only supports POST from an HTML form, which is why you see the error where it is expecting multipart formdata.
The method you need to use is PUT, instead of POST. PUT requires an item path, so you'll need to change the resource path to have a bucket and key, or get those from other places.
I have some more info on how to set this up in upload binary from api gateway to S3 bucket
It sounds like the document you're uploading isn't JSON. By default, API Gateway expects UTF-8 encoded JSON.
What content type are you sending with your Postman request?

Initiating Multipart upload to amazon s3 expecting x-amz-content-sha256 header

I am experimenting with libcurl for a multipart upload to s3. My initiate multipart request looks like this
POST /my_new_file.mbi?uploads HTTP/1.1 Accept: / Host:
test_bucket.s3.amazonaws.com Date: Thu, 01 May 2014 13:35:17 GMT
Authorization: AWS4-HMAC-SHA256
Credential=XXXXXXX/20140501/us-east-1/s3/aws4_request,SignedHeaders=host,Signature=1a3fd6195040494dd95507455a3b1eefef40346485e3fdafbe6cc136192365a2
I get the following response
The provided 'x-amz-content-sha256' header must be a valid SHA256.
s3 documentation says we do not need any other headers for Initiate multipart upload call(POST). Have tried with various combinations of signed empty content, but no luck.
What am I missing here? Any suggestions here will be very helpful.
Thanks
I haven't used version 4 auth, yet, for multipart uploads (my code uses v2), but I did find this:
x-amz-content-sha256
When using signature version 4 to authenticate request, this header provides a hash of the request payload. For more information see Authenticating Requests by Using the Authorization Header (Compute Checksum of the Entire Payload Prior to Transmission) - Signature Version 4. When uploading object in chunks, you set the value to STREAMING-AWS4-HMAC-SHA256-PAYLOAD to indicate that the signature covers only headers and that there is no payload. For more information, see Authenticating Requests Using HTTP Authorization Header (Chunked Upload).
http://docs.aws.amazon.com/AmazonS3/latest/API/RESTCommonRequestHeaders.html
Obviously, chunked and multipart are not the same thing, but perhaps this value is also appropriate for a multipart upload request, or will generate a new and more helpful error message. The documentation seems unfortunately sparse in this case.
For Googlers who got this error:
Missing required header for this request: x-amz-content-sha256
While using awscli, what worked for me was setting the region correctly in the file ~/.aws/config (I'm using Ubuntu) to us-east-1. US only doesn't work nor does US Standard. The returned error doesn't really indicate that.
STREAMING-AWS4-HMAC-SHA256-PAYLOAD appears to no longer work. I was able to make it work by passing the SHA256 hash of the empty string, e3b0c44298fc1c149afbf4c8996fb92427ae41e4649b934ca495991b7852b855

Amazon S3 & Checksum

I try to verify the integrity of a file that was uploaded to a bucket but I don't find any information of this.
In the file's headers, there is a "E-tag" but I think its not a md5 checksum.
So, how can I check if the file that I uploaded on Amazon S3 is the same that I have on my computer ?
Thanks. :)
If you are using the REST API to upload an object (up to 5GB) in a single operation, then you can add the Content-MD5 header in your PUT request. According the the S3 documentation for PUT, the Content-MD5 header is:
The base64 encoded 128-bit MD5 digest of the message (without the headers) according to RFC 1864. This header can be used as a message integrity check to verify that the data is the same data that was originally sent. Although it is optional, we recommend using the Content-MD5 mechanism as an end-to-end integrity check.
Check this answer on how to compute a base64 encoded 128-bit MD5 digest. If you are using s3curl, you can include the computed digest in your request headers using the --contentMd5 option.
If the md5 digest computed by Amazon upon upload completion does not match the md5 digest you provided in the Content-MD5 header, Amazon will respond with a BadDigest error code.
If you are using multipart upload, the Content-MD5 header serves as an integrity check for each part individually. Once the multipart upload is finalized, Amazon does not currently provide a way to verify the integrity of the assembled file.