I'm looking to redirect rules for an AWS S3 site. I don't see anywhere in the documentation that states which rules take priority when multiple rules might match.
Let's say I have a configuration like:
[
{
"Condition": {
"KeyPrefixEquals": "docs/foo"
},
"Redirect": {
"ReplaceKeyPrefixWith": "foo/"
}
},
{
"Condition": {
"KeyPrefixEquals": "docs/foo/bar"
},
"Redirect": {
"ReplaceKeyPrefixWith": "bar/"
}
}
]
Would a request for docs/foo/bar/baz, which object would the static site rout to and why?
Good question. It is indeed not mentioned in the doc, but after some experiments, it appears that the rule with a smaller index in the array will take the priority. I guess the S3 checks the configuration array by iterating through it in order when a request comes.
In the case of a request with the path docs/foo/bar/baz, the first routing rule matches and you are redirected to foo//bar/baz.
Hope this helps.
Related
I'm unable to read a file from encrypted s3 bucket in a lambda.
Below is my policy document where i'm giving access to s3 as well as kms. I've attached this policy to lambda.
When i try to read a file from the bucket, I get Access Denied error.
I'm adding kms:RequestAlias condition to kms statement so that the lambda will only have access to keys which has mytoken in the alias.
I suspect this is where i'm making mistake because if i remove the condition, the lambda gets access to all keys and read the encrypted file without any issues.
Can someone help me restrict access to only keys which has mytoken in the alias
data "aws_iam_policy_document" "lambda_s3_policy_doc" {
statement {
sid = ""
effect = "Allow"
resources = [
"arn:aws:s3:::mybucket*",
"arn:aws:s3:::mybucket*/*"
]
actions = [
"s3:AbortMultipartUpload",
"s3:CreateBucket",
"s3:DeleteObject",
"s3:GetBucketLocation",
"s3:GetObject",
"s3:ListBucket",
"s3:ListBucketMultipartUploads",
"s3:ListMultipartUploadParts",
"s3:PutObject"
]
}
statement {
effect = "Allow"
actions = [
"kms:Decrypt",
"kms:DescribeKey",
"kms:Encrypt",
"kms:GenerateDataKey"
]
resources = ["*"]
condition {
test = "StringLike"
variable = "kms:RequestAlias"
values = [
"alias/*mytoken*"
]
}
}
}
What worked for me (I was trying to download the files from a couple of encrypted buckets directly from the AWS console and your case is in fact slightly different) was replacing kms:RequestAlias with kms:ResourceAliases.
statement {
sid = "AllowKMSAccessUsingAliases"
effect = "Allow"
actions = [
"kms:Decrypt",
]
resources = [
"arn:aws:kms:eu-central-1:111111111111:key/*",
]
condition {
test = "ForAnyValue:StringEquals"
variable = "kms:ResourceAliases"
values = [
"alias/alias-bucket-1",
"alias/alias-bucket-2",
]
}
}
According to what the AWS documentation says, it makes sense at least for me: it seems that you should use kms:RequestAlias when you are including the alias as part of your KMS request like in the image below:
When you use kms:ResourceAliases what gets checked is the alias associated to the KMS resource involved in the operation regardless of whether the alias was explicitly included in the request or not
So probably, your lambda function, when asking for the un-encryption of a file in a bucket, is using the KMS id in the request instead of the KMS alias and if that is the case kms:RequestAlias won't work because there is no alias in the request to be checked.
Im using terraform 0.12.4 to attempt tor write some code to enable the ‘access logs’ for my load balancer to write logs to an s3 bucket.
So far the buckets been created and the load balancers have been created by someone else but the bit where the ‘access_logs’ were supposeed to be configured was commented out and a TODO comment was placed there also. Hmmm methinks.
Theres too much code to place here but i keep receving access denied errors when setting them up. Ive found a couple of resources detailing what to do but none work. Has anyone managed to do this in TF?
According to the documentation on the Access Logs, you need to add permissions on your bucket for the ALB to write to S3.
data "aws_elb_service_account" "main" {}
data "aws_caller_identity" "current" {}
data "aws_iam_policy_document" "allow_load_balancer_write" {
statement {
principals {
type = "AWS"
identifiers = ["${data.aws_elb_service_account.main.arn}"]
}
actions = [
"s3:PutObject"
]
resources = [
"${aws_s3_bucket.access_logs.arn}/<YOUR_PREFIX_HERE>/AWSLogs/${data.aws_caller_identity.current.account_id}/*",
]
}
}
resource "aws_s3_bucket_policy" "access_logs" {
bucket = "${aws_s3_bucket.<YOUR_BUCKET>.id}"
policy = data.aws_iam_policy_document.allow_load_balancer_write.json
}
Also, it seems server side encryption needs to be enabled on the bucket.
resource "aws_s3_bucket" "access_logs" {
bucket_prefix = "<YOUR_BUCKET_NAME>-"
}
resource "aws_s3_bucket_server_side_encryption_configuration" "access_logs_encryption" {
bucket = "${aws_s3_bucket.access_logs.bucket}"
rule {
apply_server_side_encryption_by_default {
sse_algorithm = "AES256"
}
}
}
I have some API like https://api-******y?apikey=sX4&q=london, Where q=london is a parameter that will keep changing, So how can i add this URL in the list of urls in ngsw-config.json file
"dataGroups": [
{
"name": "api-performance",
"urls": [
"https://api-***************y?apikey=sX******4&q=london",
"https://api-***************y?apikey=sX******4&q=manchester",
"https://api-***************y?apikey=sX******4&q=birmingham"
],
"cacheConfig": {
"strategy": "performance",
"maxSize": 100,
"maxAge": "3d"
}
},
I have two doubts:
Instead of adding same url again and again for all different parameters like london/manchester/birmingham is there a way that i can just add the base URL without parameter and it will store the responses for all different parameters when user will search for it?
Instead of adding the URL's manually, the URL's response should be stored dynamically when the user has searched for it when he was online. I mean i will initially not add any URL in list in the ngsw-config.json file, BUT when user has requested for that url, its response automatically get stored in cache.
I got this one, Its actually a little stupid of me to think that i have to give the complete url in the ngsw-config.json file,
Actually we just can simply give the base url, we neither have to give the apiKey parameter nor any other parameter like london/manchester/birmingham, ngsw.json file in dist folder after the build, stores the url as patterns, it just matches the base url and stores all the queries with all different parameters made to that url by user. above code can re-written like below. it will work fine by caching the data for all the stores london/manchester/birmingham, whenever user searches for them.
"dataGroups": [
{
"name": "api-performance",
"urls": [
"https://api-***************y
],
"cacheConfig": {
"strategy": "performance",
"maxSize": 100,
"maxAge": "3d"
}
},
I am having trouble trying to create s3 event notifications. Does anyone know the resolutions to this?
Error is:
*Error applying plan:
1 error(s) occurred:
* module.Test-S3-Bucket.aws_s3_bucket_notification.s3-notification: 1 error(s) occurred:
* aws_s3_bucket_notification.s3-notification: Error putting S3 notification configuration: InvalidArgument: Unable to validate the following destination configurations
status code: 400, request id: AD9B5BF2FF84A6CB, host id: ShUVJ+TdkpqAZfpeDM3grkF9Vue3Q/AF0LydchperKTF6XdQyDM6BisZi/38pGAh/ZqS+gNyrSM=*
Below is the code that gives me the error:
resource "aws_s3_bucket" "s3-bucket" {
bucket = "${var.bucket_name}"
acl = ""
lifecycle_rule {
enabled = true
prefix = ""
expiration {
days = 45
}
}
tags {
CostC = "${var.tag}"
}
}
resource "aws_s3_bucket_notification" "s3-notification" {
bucket = "${var.bucket_name}"
topic {
topic_arn = "arn:aws:sns:us-east-1:1223445555:Test"
events = [ "s3:ObjectCreated:*", "s3:ObjectRemoved:*" ]
filter_prefix = "test1/"
}
}
If you haven't done it already, you need to specify a policy on the topic that grants the SNS:Publish permission to S3 (only from the bucket specified in the Condition attribute) - if you are also provisioning the topic via Terraform then something like this should do it (we know, as it caught us out just a few days ago too!):
resource "aws_sns_topic" "my-sns-topic" {
name = "Test"
policy = <<POLICY
{
"Version":"2012-10-17",
"Statement":[{
"Effect": "Allow",
"Principal": {
"Service": "s3.amazonaws.com"
},
"Action": "SNS:Publish",
"Resource": "arn:aws:sns:us-east-1:1223445555:Test",
"Condition":{
"ArnLike":{"aws:SourceArn":"${aws_s3_bucket.s3-bucket.arn}"}
}
}]
}
POLICY
}
Hope that helps.
Well, I know that this is not your exact case, but I had the same error and I didn't manage to find an answer here, and because this post is the first that Google gave me, I will leave the answer to my case here in the hope that it will help someone else.
So, I notice that after Terraform apply I had this error and I went to the UI to see what happened and found this message:
The Lambda console can't validate one or more event sources for this trigger. The most common cause is when a source ARN includes a wildcard (*) character. You can manage unvalidated triggers using the AWS CLI or AWS SDK.
And guess what? I really had a wildcard (*) character in ARN like this:
source_arn = "{aws_s3_bucket.bucket.arn}/*"
So I changed it to:
source_arn = aws_s3_bucket.bucket.arn
And it worked. So, if you read this - there might be the same mistake in your case.
I took a look at this question that seeks to address the issue of REST media-type explosion. One of the suggestions was to have a media-type that describes a collection of anything. So for example, we could have an application/vnd.collection+json which is a collection with well-defined semantics that can hold a list of references to other resources:
{
"size": "3"
"elements": [
{ "href" : "http://my.api.com/resource/1" },
{ "href" : "http://my.api.com/resource/2" },
{ "href" : "http://my.api.com/resource/3" }
]
}
I know an option to alleviate chattiness is to include embedded representations of resources. How would a "generic" media-type for lists accomplish that? Don't the semantics of the list change based on which embedded resource is inside it? This is especially relevant if embedded resources have different processing-rules (which would ideally be conveyed by the media type). Would be alright in this case to allow in-band information that describes the media type of the embedded resource? For example we could have application/vnd.collection+json for both links and embedded resources that do something like this:
{
"size": "3"
"element-content-type": "application/vnd.link+json"
"elements": [
{ "href" : "http://my.api.com/resource/1" },
{ "href" : "http://my.api.com/resource/2" },
{ "href" : "http://my.api.com/resource/3" }
]
}
and if it contains an embedded resource:
{
"size": "3"
"element-content-type": "application/vnd.resource+json"
"elements": [
{
"id": "1"
"name": "Cool Resource"
},
{
"id": "2"
"name": "Awesome Resource"
},
{
"id": "3"
"name": "Super Awesome Resource"
}
]
}
The assumption is that application/vnd.link+json and application/vnd.resource+json have been documented as well.
I thought about this a little bit more, and I think it is actually OK to include the content-type like that. The reason is, we already do this. In HTML the script tag has a type attribute that can be application/javascript or application/vbscript (for example). The provides the browser a hint as to how to process the content of that tag. Similarly, I think the content-type in the above example achieves the same purpose: it tells the client how to process the elements in the collection.
I wanted to update this answer some more. It appears that another way to do this is to use a rel. At least, this is how HAL does it. You can create a namespaced rel with a curie so that you end up resolving the rel to a URL that points to documentation about that resource. This way you have access to the documentation and that should tell you all you need to know about the resource(s).