slingshot meteor s3 error - amazon-s3

I’m afraid I don’t understand how this is supposed to work at all. How does slingshot know the address to find my s3 bucket? Is this completely determined by the access keys?
This is the code I have in my server/files.js:
var imageDetails = new Mongo.Collection('images');
Slingshot.fileRestrictions("myImageUploads", {
allowedFileTypes: ["image/png", "image/jpeg", "image/gif"],
maxSize: 2 * 1024 * 1024,
});
Slingshot.createDirective("myImageUploads", Slingshot.S3Storage, {
AWSAccessKeyId: "AWSAccessKeyId",
AWSSecretAccessKey: "AWSSecretAccessKey",
bucket: "mybucketname",
acl: "public-read",
region: "us-west-1",
authorize: function () {
if (!this.userId) {
var message = "Please login before posting images";
throw new Meteor.Error("Login Required", message);
}
return true;
},
key: function (file) {
var currentUserId = Meteor.user().emails[0].address;
return currentUserId + "/" + file.name;
}
});
And this is my settings.json file
{
"AWSAccessKeyId" : "my access key",
"AWSSecretAccessKey" : "my secret access key",
"AWSBucket" : "mybucketname"
}
I get this error in my browser:
Cross-Origin Request Blocked: The Same Origin Policy disallows reading the remote resource at https://mybucketname.s3-us-west-1.amazonaws.com/. (Reason: CORS header ‘Access-Control-Allow-Origin’ missing).
But I have a CORS configuration in my theportdata bucket.
The first step, I guess, is there any way to check if my application is making contact at all with my s3 bucket? Like I said, I don’t really understand how slingshot finds the bucket.

SOLVED
Changed "region: us-west-1" to "region: us-west-2" and it works.
There is also no need for the AWSAccessKeyId and AWSSecretAccessKey, since slingshot finds this automatically from settings.json.
Apparently all that's needed for an address is the bucket name and the region.
https://docs.aws.amazon.com/AmazonS3/latest/dev/WebsiteHosting.html

Related

AWS S3 getBucketLogging fails when called from lambda function

I am trying in an AWS lambda to get the bucket logging settings for my buckets. For this I enumerate the buckets with S3.listBuckets() - which works just fine. I then iterate over the bucket names like this (Typescript):
const bucketNames = await getBucketNames() // <- works without problems
for (const bucketName of bucketNames) {
try {
console.log(`get logging for bucket ${bucketName}`) // <-- getting to this log
const bucketLogging: GetBucketLoggingOutput = await s3.getBucketLogging({
Bucket: bucketName,
ExpectedBucketOwner: accountId
}).promise()
// check logging setup and adjust if necessary
} catch (error) {
console.log(JSON.stringify(error))
}
}
The call to getBucketLogging() fails
{
"message": "Access Denied",
"code": "AccessDenied",
"region": null,
"time": "2022-07-19T11:16:26.671Z",
"requestId": "****",
"extendedRequestId": "****",
"statusCode": 403,
"retryable": false,
"retryDelay": 70.19937788683632
}
The accountId that is passed in is definitely right (it's optional anyway); the lambda is in the same account as the bucket owner (which is the sole condition described in the docs at https://docs.aws.amazon.com/AWSJavaScriptSDK/latest/AWS/S3.html#getBucketLogging-property).
When doing this call from a terminal CLI I have no problems to get results, only when running from a lambda.
What am I missing or overseeing?
You should make sure to attach the respective IAM permissions to your lambda function. Just because you have the s3:ListBuckets role doesn't mean that it is also permitted to perform the same for the BucketLogging information. Please refer to the following docs for more details on S3 IAM actions: https://docs.aws.amazon.com/service-authorization/latest/reference/list_amazons3.html

"Access key does not exist" when generating pre-signed S3 URL from Lambda function

I'm trying to generate a presigned URL from within a Lambda function, to get an existing S3 object .
(The Lambda function runs an ExpressJS app, and the code to generate the URL is called on one of its routes.)
I'm getting an error "The AWS Access Key Id you provided does not exist in our records." when I visit the generated URL, though, and Google isn't helping me:
<Error>
<Code>InvalidAccessKeyId</Code>
<Message>The AWS Access Key Id you provided does not exist in our records.</Message>
<AWSAccessKeyId>AKIAJ4LNLEBHJ5LTJZ5A</AWSAccessKeyId>
<RequestId>DKQ55DK3XJBYGKQ6</RequestId>
<HostId>IempRjLRk8iK66ncWcNdiTV0FW1WpGuNv1Eg4Fcq0mqqWUATujYxmXqEMAFHAPyNyQQ5tRxto2U=</HostId>
</Error>
The Lambda function is defined via AWS SAM and given bucket access via the predefined S3CrudPolicy template:
ExpressLambdaFunction:
Type: AWS::Serverless::Function
Properties:
FunctionName: ExpressJSApp
Description: Main website request handler
CodeUri: ../lambda.zip
Handler: lambda.handler
[SNIP]
Policies:
- S3CrudPolicy:
BucketName: my-bucket-name
The URL is generated via the AWS SDK:
const router = require('express').Router();
const AWS = require('aws-sdk');
router.get('/', (req, res) => {
const s3 = new AWS.S3({
region: 'eu-west-1',
signatureVersion: 'v4'
});
const params = {
'Bucket': 'my-bucket-name',
'Key': 'my-file-name'
};
s3.getSignedUrl('getObject', params, (error, url) => {
res.send(`<p>${url}</p>`)
});
});
What's going wrong? Do I need to pass credentials explicitly when calling getSignedUrl() from within a Lambda function? Doesn't the function's execute role supply those? Am I barking up the wrong tree?
tldr; Go sure, to have the correct order of signature_v4 headers/formdata, in your request.
I had the same exact issue.
I am not sure if this is the solution for everyone who is encountering the problem, but I learned the following:
The error message, and other misleading error messages can occur, if you don't use the correct order of security headers. In my case I was using the endpoint to create a presigned url, for posting a file, to upload it. In this case, you need to go sure, that you are having the correct order of security relevant data in your form-data. For signatureVersion 's3v3' it is:
key
x-amz-algorithm
x-amz-credential
x-amz-date
policy
x-amz-security-token
x-amz-signature
In the special case of a POST-Request to a presigned url, to upload a file, it's important to have your file, AFTER the security data.
After that, the request works as expected.
I can't say for certain but I'm guessing this may have something to do with you using the old SDK. Here it is w/ v3 of the SDK. You may need to massage it a little more.
const { getSignedUrl } = require("#aws-sdk/s3-request-presigner");
const { S3Client, GetObjectCommand } = require("#aws-sdk/client-s3");
// ...
const client = new S3Client({ region: 'eu-west-1' });
const params = {
'Bucket': 'my-bucket-name',
'Key': 'my-file-name'
};
const command = new GetObjectCommand(params);
getSignedUrl(client, command(error, url) => {
res.send(`<p>${url}</p>`)
});

node S3 Object Storage Linode

Im trying to use the aws-sdk to acces my linode S3 compatible bucket, but everything I try doesn't work. Not sure what the correct endpoint should be? For testing purposes is my bucket set to public read/write.
const s3 = new S3({
endpoint: "https://linodeobjects.com",
region: eu-central-1,
accesKeyId: <accesKey>,
secretAccessKey: <secretKey>,
});
const params = {
Bucket: bucketName,
Key: "someKey",
Expires: 60,
};
const uploadURL = await s3.getSignedUrlPromise("putObject", params);
The error im getting
code: 'CredentialsError',
time: 2021-07-15T08:29:50.000Z,
retryable: true,
originalError: {
message: 'Could not load credentials from any providers',
code: 'CredentialsError',
time: 2021-07-15T08:29:50.000Z,
retryable: true,
originalError: {
message: 'EC2 Metadata roleName request returned error',
code: 'TimeoutError',
time: 2021-07-15T08:29:49.999Z,
retryable: true,
originalError: [Object]
}
}
}
It seems like a problem with the credentials of the environment that this code is executed in and not with the bucket permissions themselves.
The pre-signing of the URL is an operation that is done entirely locally. It uses local credentials (i.e., access key ID and secret access key) to create a sigv4 signature for the URL. This also means that whether or not the credentials used for signing the URL are valid is only checked at the moment the URL is used, and not at the moment of signing the URL itself.
The error simply indicates that from all the ways the SDK is trying to find credentials (more info here) it cannot find credentials it can use to sign the URL.
This might be unrelated, but according to the documentation, the endpoint should be the following: The endpoint URI to send requests to. The default endpoint is built from the configured region. The endpoint should be a string like 'https://{service}.{region}.amazonaws.com' or an Endpoint object. Which, in the code example above, is not the case.
You should set the endpoint to be eu-central-1.linodeobjects.com. When using Linode object storage the region is not determined by the endpoint that you use.

500 Internal Server Error when uploading to an AWS S3 bucket from Nuxt

I am trying to upload a file to AWS S3 using aws-sdk v3 from a Nuxt app's Vue Component.
Here's how I upload it.
<script>
export default {
...
methods: {
onSubmit(event) {
event.preventDefault()
this.addPhoto()
},
addPhoto() {
// Load the required clients and packages
const { CognitoIdentityClient } = require('#aws-sdk/client-cognito-identity')
const { fromCognitoIdentityPool } = require('#aws-sdk/credential-provider-cognito-identity')
const {
S3Client,
PutObjectCommand,
ListObjectsCommand,
DeleteObjectCommand,
} = require('#aws-sdk/client-s3')
const REGION = 'us-east-1' // REGION
const albumBucketName = 'samyojya-1'
const IdentityPoolId = 'XXXXXXX'
const s3 = new S3Client({
region: REGION,
credentials: {
accessKeyId: this.$config.CLIENT_ID,
secretAccessKey: this.$config.CLIENT_SECRET,
sessionToken: localStorage.getItem('accessToken'),
},
})
var file = this.formFields[0].fieldName
var fileName = this.formFields[0].fieldName.name
var photoKey = 'user-dp/' + fileName
var s3Response = s3.send(
new PutObjectCommand({
Bucket: albumBucketName,
Key: photoKey,
Body: file,
}),
)
s3Response
.then((response) => {
console.log('Successfully uploaded photo.' + JSON.stringify(response))
})
.catch((error) => {
console.log(
'There was an error uploading your photo: Error stacktrace' + JSON.stringify(error.message),
)
const { requestId, cfId, extendedRequestId } = error.$metadata
console.log({ requestId, cfId, extendedRequestId })
})
},
...
}
</script>
The issue now is that the browser complains about CORS.
This is my CORS configuration on AWS S3
I'm suspecting something while creating the upload request using SDK. (I'm open to use an API that is better than what I'm using).
Nuxt setting that allows CORS.
Something else on S3 CORS config at permissions
Network tab on chrome dev tools shows Internal Server Error (500) for prefetch. (Don't know why we see 2 entries here)
Appreciate any pointers on how to debug this.
I was having the same issue today. The S3 logs were saying it returned a 200 code response, but Chrome was seeing a 500 response. In Safari, the error showed up as:
received 'us-west-1'; expected 'eu-west-1'
Adding region: 'eu-west-1' (i.e. the region where the bucked was created)to the parameters when creating the S3 service solved the issue for me.
https://docs.aws.amazon.com/sdk-for-javascript/v2/developer-guide/setting-region.html#setting-region-constructor
In the bucket policy use this
{
"Version": "2008-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"s3:GetObjectAcl",
"s3:GetObject",
"s3:PutObject",
"s3:PutObjectAcl",
"s3:ListMultipartUploadParts"
],
"Resource": "arn:aws:s3:::YOUR_BUCKET_NAME/*",
"Condition": {
"StringLike": {
"aws:Referer": "https://example/*"
}
}
}
]}
and use the region of your bucket
const s3 = new aws.S3({
apiVersion: 'latest',
accessKeyId: process.env.AWS_ACCESS_KEY_ID_CUSTOM,
secretAccessKey: process.env.AWS_SECRET_ACCESS_KEY_CUSTOM,
region: 'us-west-1',
})
I am having the same problem, but according to the docs you should be using Cognito Identity to access the bucket. Only in V3 for clients to be able to access the buckets from the browser you must use Cognito Identity to authenticate users in order to have access to bucket/object commands. Currently trying to implement, so I am not 100% how to do it just the process. Feel free to take a look. I hope this helps. ~~~~~~~~~~~~~~~~~~~~~~~~~~
| Cognito SDK Link: | https://docs.aws.amazon.com/cognito/latest/developerguide/what-is-amazon-cognito.html
| Example: | https://docs.aws.amazon.com/sdk-for-javascript/v3/developer-guide/loading-browser-credentials-cognito.html
The error needs to be fixed on the backend, since it's CORS. It's clearly states a missing header of Access-Control-Allow-Origin.
So, checking it in the official AWS docs gives you the answer: https://docs.aws.amazon.com/apigateway/latest/developerguide/how-to-cors.html
I was doing multiple things wrongly here. Every answer on this post helped me make a little progress while debugging. Can't thank you enough!
My bucket policy was not using role-based ALLOW/DENY that has to correspond to authenticated role on my cognito identity pool.
Needed to rightly configure the Authentication provider as Cognito Userpool.
Making sure the region is right. Cognito region could be different from S3 region.
Make sure CORS policy includes relevant information like "Access-Control-Allow-Origin".
Double check the token includes the right credentials. This comes very handy cognito decode-verify
Was stand-alone testing from the browser. But this is not a good approach. Use an API server to take the file and push to S3 from there.

Fine Uploader S3 - Return Uploaded URL on Complete callback

How can i get the full image url of the uploaded image on my amazon s3 bucket with fine-uploader?
My javascript code is:
jQuery(document).ready(function () {
jQuery("#fine-uploader").fineUploaderS3({
debug: true,
request: {
endpoint: 'bucket.s3.amazonaws.com',
accessKey: 'xxxxxxxx'
},
signature: {
endpoint: 'end.php '
},
uploadSuccess: {
endpoint: 'success.php '
},
iframeSupport: {
localBlankPagePath: 'success.html '
},
retry: {
enableAuto: true
},
validation: {
allowedExtensions: ['
jpeg ', '
jpg ', '
png '],
sizeLimit: 1048576
}
}).on('
complete ', function (event, id, name, response) {
console.log(response.tempLink);
});
});
UPDATE
Following the s3 demo i am using the response.tempLink and just trying to log it to the console and then i will use it later on. The upload always works fine but my console returns an undefined response everytime.
From finding this q&a: having trouble displaying an image uploaded to Amazon s3 by fine-uploader
It seems like my IAM user/policy settings and $serverPublicKey and $serverPrivateKey might be the cause? My setup is:
Exact copy of this file for my end.php file:
https://github.com/Widen/fine-uploader-server/blob/master/php/s3/s3demo-cors.php
with the following changes:
// changed to match the secret access key for the FIRST IAM user as discussed in the docs
$clientPrivateKey = 'user_secret_key...';
// bucket name
$expectedBucketName = "my.bucket.name";
// changed to match the access and secret of the SECOND IAM 'server' user
$serverPublicKey = 'server_user_access_key...';
$serverPrivateKey = 'server user secret key...';
// updated to my website
function handlePreflightedRequest() {
header('Access-Control-Allow-Origin: http://www.mywebsite.com');
}
In my Amazon IAM console I have mt SECOND IAM 'server' user setup as:
Group:
grp-server
Group Policy: (is GetObject the correct action?)
{
"Version":"2012-10-17",
"Statement":[{
"Effect":"Allow",
"Action":"s3:GetObject",
"Resource":"arn:aws:s3:::my.bucket.name/*"
}]
}
or i've tried the following which gives full admin access just to check
{
"Version":"2012-10-17",
"Statement":[{
"Effect":"Allow",
"Resource":"*"
}]
}
User:
user-server
added to grp-server (which inherits group policy)
user-server access key becomes $serverPublicKey in end.php
user-server secret key becomes $serverPrivateKey in end.php
Am i missing anything from this?
Have your uploadSuccess.endpoint return a pre-signed URL that you then handle in the onComplete handler. Note that pre-signed URLs can only be generated server-side. See this PHP server for details.