Due to some compliance requirements, whatever we put into the S3 bucket we have to enforce bucket encryption and object encryption. How we can attain this after we do "sls deploy" it automatically does the whole thing. For the bucket-level encryption, we are using "serverless-deployment-bucket" but what to do for object-level encryption?
Related
Trying to send records to Amazon S3 with Flink: however these records need to be sent with an AES256 SSE header to request server side encryption
see aws documentation:
If you need server-side encryption for all of the objects that are stored in a bucket, use a bucket policy. For example, the following bucket policy denies permissions to upload an object unless the request includes the x-amz-server-side-encryption header to request server-side encryption:
Is this something that can be set for specific file sinks? have not found any documentation on the matter and beginning to think a forwarding lambda will be needed to transform the data.
I'm setting up a Kinesis Firehose delivery stream to S3, and I noticed you can set a custom KMS key to be used for encrypting the files on S3.
However, if the S3 bucket already has KMS encryption enabled, files would be encrypted anyway. The difference is of course that the default AWS manager S3 KMS key will be used instead of the customer managed custom KMS key supplied to Firehose.
What reason is there typically to use a custom KMS key for the encryption of Firehose data on S3, as opposed to relying on the default S3 KMS key? Is there any point at all in doing so if you're also the owner of the S3 bucket and in control of its settings, or is the primary use to enable using encryption also when you're not in control of the settings of the target bucket?
Or is the Firehose associated KMS key also used for encrypting data in transit, as opposed to the S3 provided KMS key used to encrypt data at rest?
Kinesis Firehose will use the KMS key you specify to encrypt the objects when landing in S3. You may not have control over the S3 bucket's encryption settings, and you may want to use a different KMS key (with different permissions) than the S3 default KMS encryption key for whatever reason. There can be many different objects in that S3 bucket at different hierarchies, requiring different KMS encryption, or not.
S3 should not "double encrypt" your data. The KMS encryption from Kinesis Firehose will be specified in the S3 put header, so S3 will know which encryption settings to use when it does the actual write. If there are default KMS settings on the S3 bucket, and it does not find an encryption setting in the put header (whether SSE or KMS), then S3 should apply the default encryption specified in the bucket settings.
Is it possible to crawl S3 file encrypted using CSE-KMS in AWS Glue? I know that Athena can do that, but haven't found similar functionality in Glue crawler
I do not think AWS Glue supports reading from client-side encryption. They have just added server-side encryption support, which is much simpler to support compared client-side encryption.
Glue Does not support Client Side Encrypted data. It only supports AWS KMS-managed keys (SSE-KMS) or Amazon S3-managed encryption keys (SSE-S3). these are the only two currently available in encryption models in Security Configuration in Glue [1].
[1] https://docs.aws.amazon.com/glue/latest/dg/console-security-configurations.html
if I'm using the AmazonS3Client to put and fetch files, is my connection encrypted? This seems basic, but my googling seems to return things about encrypting the S3 storage and not whether the transmission from this client is secure. If it's not secure is there a setting to make it secure?
Amazon S3 endpoints support both HTTP and HTTPS. It is recommended that you communicate via HTTPS to ensure your data is encrypted in transit.
You can also create a Bucket Policy that enforces communication via HTTPS. See:
Stackoverflow: Force SSL on Amazon S3
Sample policy: s3BucketPolicyEncryptionSSL.json
I want to offer presigned urls to my s3 buckets with an expiry date.
The .net sdk nicely generates these urls, but looking at them makes me worry a little:
https://s3.amazonaws.com/upload_youtipit/myfile?AWSAccessKeyId=**MYACCESSKEY**&Expires=1317924047&response-content-disposition=attachment;filename=feedback.jpg&response-content-type=image/jpeg&Signature=podKJSrNeFel6%2B0aOneb342H5GA%3D
Why does it need to put my (public) AWSAccessKey in the Url? Shouldn't this be kept a bit more confidential? I know its not the secret, but I still don't feel comfortable exposing it in public..
How likely is it that, somebody who has my public key, can guess/bruteforce my private key?
The Access Key ID is not a secret and does not need protecting.
In fact, you can give expiring URLs to random strangers if you want them to access an S3 object. They can see the access key in that URL, but can't do anything with it that you have not authorized.
reference: http://docs.amazonwebservices.com/AWSSecurityCredentials/1.0/AboutAWSCredentials.html#AccessKeys
I kind of agree with the accepted answer, but there is an easy way to do what you want.
You need to use Amazon IAM to create a user that can only read files (it does not really matter, but they at least need read only to the bucket that you are dealing with). Then use THAT users AWS ID and secret to generate a download link.
This does not open up your whole bucket, as to see the whole bucket the person needs the AWSID of this IAM 'reader' user, plus their secret key. You still need to construct time limited URLs.
You can use the IAM console to create a user like that in a few mins. You only get one chance to get the secret key - at the time you make the user (or if you rotate his keys).
That should do it. This AWSID has no access to anything, and is not linked to you.
In general Amazon now recommends that your primary AWSID is not used 'for anything'. You create users with permissions in IAM, then use those codes. This allows for a lot of security flexibility. You can turn off your read only user with a simple action in the IAM console.
your private key is a one-way computation of some unknown parameters, it may not even use the public key itself as part of the algorithm. knowing your AWSAccessKeyId should have no effect on the complexity of brute-forcing your private key whatsoever.
https ensures the only thing someone can tell about your connection (if they are sniffing network traffic) is that it's between your IP and the IP of s3.amazonaws.com on port 443. even the fact that you're connecting to amazon would have to be deduced from known ip-address ranges assigned to them (which is probably well-known anyhow)
the only way to get at your AWSAccessKeyId in the address bar is to physically see it on your screen, installing some key-logger or trojan on your computer or MIM (man in the middle-ing) amazon's SSL certificate, which is extremely hard even if someone has access to the needed upstream network nodes.
leon
You can use AWS Security Token Service (AWS STS) if you concern of more security.
Temporary security credentials work almost identically to the long-term access key credentials that your IAM users can use, with the following differences:
Temporary security credentials are short-term, as the name implies. They can be configured to last for anywhere from a few minutes to several hours. After the credentials expire, AWS no longer recognizes them or allows any kind of access from API requests made with them.
Temporary security credentials are not stored with the user but are generated dynamically and provided to the user when requested. When (or even before) the temporary security credentials expire, the user can request new credentials, as long as the user requesting them still has permissions to do so.
Ref: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp.html