CREATE_FAILED Bucketpolicy - Unknown field Fn::Join - amazon-s3

My Cloudformation stack fails and keeps getting rolled back because of the following S3 bucket policy. The referenced S3 bucket is a separate bucket meant for CloudTrail logs (as I read that such a thing is best practice when using CloudTrail). The bucket gets created along with the rest of the stack during the cloudFormation process: [stackname]-cloudtraillogs-[randomstring]
I tried not using any functions to specify the bucket, but that doesn't seem to work. My guess is because it then goes looking for a bucket 'cloudtraillogs' and can't find any bucket with that name. Using a Fn::Join with a reference might solve that(?), but then CloudFormation gives 'Unknown field Fn::Join' when evaluating bucket policy.
Anyone who can spot what I might be doing wrong here?
Bucketpolicy
{
"Resources": {
"policycloudtraillogs": {
"Type": "AWS::S3::BucketPolicy",
"Properties": {
"Bucket": {
"Ref": "cloudtraillogs"
},
"PolicyDocument": {
"Statement": [
{
"Sid": "AWSCloudTrailAclCheck20160224",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:GetBucketAcl",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "cloudtraillogs"
},
"/*"
]
]
},
{
"Sid": "AWSCloudTrailWrite20160224",
"Effect": "Allow",
"Principal": {
"Service": "cloudtrail.amazonaws.com"
},
"Action": "s3:PutObject",
"Resource": {
"Fn::Join": [
"",
[
"arn:aws:s3:::",
{
"Ref": "cloudtraillogs"
},
"/AWSLogs/myAccountID/*"
]
]
},
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control"
}
}
}
]
}
}
}
}
}

Your template does not appear to be valid JSON. Your first policy statement (AWSCloudTrailAclCheck20160224) is missing a closing bracket } for its Resource object.

Related

How to write VPC flow logs to an S3 bucket on another AWS account?

I am trying to write VPC Flow logs (from account 1) to an S3 bucket (on account 2), using terraform:
resource "aws_flow_log" "security_logs" {
log_destination = "arn:aws:s3:::my_vpcflowlogs_bucket"
log_destination_type = "s3"
vpc_id = var.vpc_id
traffic_type = "ALL"
}
resource "aws_iam_role" "vpc_flow_logs" {
name = "vpc_flow_logs"
assume_role_policy = <<EOF
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "",
"Effect": "Allow",
"Principal": {
"Service": "vpc-flow-logs.amazonaws.com"
},
"Action": "sts:AssumeRole"
}
]
}
EOF
}
resource "aws_iam_role_policy" "write_vpc_flow_logs" {
name = "write_vpc_flow_logs"
role = aws_iam_role.vpc_flow_logs.id
policy = jsonencode({
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"logs:CreateLogDelivery",
"logs:DeleteLogDelivery"
],
"Resource": "arn:aws:s3:::my_vpcflowlogs_bucket"
}
]
})
}
Account 1 & 2 belong to the same organisation.
I am getting the following response:
Error creating Flow Log for (vpc-xxxxxxxxxxxx), error: Access Denied for LogDestination: my_vpcflowlogs_bucket. Please check LogDestination permission
How can I make this work? This bucket contains sensitive information, therefore i have restricted every kind of public access.
I am guessing that there is a way to allow certain principals to write into the bucket even from different accounts, but I am unaware how.
S3 Policy
{
"Version" : "2012-10-17",
"Statement" : [
{
"Sid": "AWSLogDeliveryWrite",
"Effect": "Allow",
"Principal": {"Service": "delivery.logs.amazonaws.com"},
"Action": "s3:PutObject",
"Resource": "arn:aws:s3:::my_vpcflowlogs_bucket/*",
"Condition": {
"StringEquals": {
"s3:x-amz-acl": "bucket-owner-full-control",
"aws:SourceAccount": <account-1-id>
},
"ArnLike": {
"aws:SourceArn": "arn:aws:logs::<account-1-id>:*"
}
}
},
{
"Sid": "AWSLogDeliveryCheck",
"Effect": "Allow",
"Principal": {"Service": "delivery.logs.amazonaws.com"},
"Action": ["s3:GetBucketAcl", "s3:ListBucket"],
"Resource": "arn:aws:s3:::my_vpcflowlogs_bucket",
"Condition": {
"StringEquals": {
"aws:SourceAccount": <account-1-id>
},
"ArnLike": {
"aws:SourceArn": "arn:aws:logs::<account-1-id>:*"
}
}
}
]
}

Need help understanding inconsistent behaviour and KMS error when deploying changes via CDK

We are using CDK to create s3 buckets in different regions and to manage a codepipeline which deploys to these cross region and cross account buckets.
We have seen some inconsistent behavior lately when we deployed changes to the pipeline (Add a permission to Codebuild role).
We were getting the following error:
Policy contains a statement with one or more invalid principals. (Service: AWSKMS; Status Code: 400; Error Code: MalformedPolicyDocumentException;
We do not explicitly create KMS keys. The KMS keys and its policy were auto generated during CDK deploy.
To resolve this error, we destroyed the existing S3 buckets and the pipeline and re-deployed it with a different name. The 1st time the deploy was successful in deploying the S3 buckets in all regions except for ap-southeast-2 (With the same error). The second time we rinsed and repeated, it deployed fine.
We are not sure what is causing this behavior. We wanted to confirm if this is a known bug or something we need to change in our code to resolve this inconsistency.
Following is the KMS policy auto-generated for ap-southeast-2 using cdk synth when it was failing to deploy :
"KeyPolicy": {
"Statement": [
{
"Action": [
<Truncated>
],
"Effect": "Allow",
"Principal": {
"AWS": {
"Fn::Join": [
"",
[
"arn:",
{
"Ref": "AWS::Partition"
},
":iam::<Bucket_ACCOUNT_ID>:root"
]
]
}
},
"Resource": "*"
},
{
"Action": [
<Truncated>
],
"Effect": "Allow",
"Principal": {
"AWS": {
"Fn::Join": [
"",
[
"arn:",
{
"Ref": "AWS::Partition"
},
":iam::<Bucket_ACCOUNT_ID>:root"
]
]
}
},
"Resource": "*"
},
{
"Action": [
<Truncated>
],
"Effect": "Allow",
"Principal": {
"AWS": {
"Fn::Join": [
"",
[
"arn:",
{
"Ref": "AWS::Partition"
},
":iam::<Pipeline_Account_ID>:role/<role_name>"
]
]
}
},
"Resource": "*"
},
{
"Action": [
<Truncated>
],
"Effect": "Allow",
"Principal": {
"AWS": {
"Fn::Join": [
"",
[
"arn:",
{
"Ref": "AWS::Partition"
},
":iam::<Pipeline_Account_ID>:role/<role_name>"
]
]
}
},

S3 bucket policy multiple conditions

I'm looking to grant access to a bucket that will allow instances in my VPC full access to it along with machines via our Data Center. Without the aws:SouceIp line, I can restrict access to VPC online machines.
I need the policy to work so that the bucket can only be accessible from machines within the VPC AND from my office.
{
"Version": "2012-10-17",
"Id": "Policy1496253408968",
"Statement": [
{
"Sid": "Stmt1496253402061",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::xyz-sam-test/*",
"arn:aws:s3:::xyz-sam-test"
],
"Condition": {
"StringLike": {
"aws:sourceVpc": "vpc-dcb634bf",
"aws:SourceIp": "<MY PUBLIC IP>"
}
}
}
]
}
You can generate a policy whose Effect is to Deny access to the bucket when StringNotLike Condition for both keys matches those specific wildcards.
{
"Version": "2012-10-17",
"Id": "Policy1496253408968",
"Statement": [
{
"Sid": "Stmt1496253402061",
"Effect": "Deny",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::xyz-sam-test/*",
"arn:aws:s3:::xyz-sam-test"
],
"Condition": {
"StringNotLike": {
"aws:sourceVpc": "vpc-dcb634bf",
"aws:SourceIp": "<MY PUBLIC IP>"
}
}
}
]
}
The second condition could also be separated to its own statement. AWS applies a logical OR across the statements. 1
{
"Version": "2012-10-17",
"Id": "Policy1496253408968",
"Statement": [
{
"Sid": "Stmt1496253402061",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::xyz-sam-test/*",
"arn:aws:s3:::xyz-sam-test"
],
"Condition": {
"StringLike": {
"aws:sourceVpc": "vpc-dcb634bf",
}
}
},
{
"Sid": "Stmt1496253402062",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::xyz-sam-test/*",
"arn:aws:s3:::xyz-sam-test"
],
"Condition": {
"StringLike": {
"aws:SourceIp": "<MY PUBLIC IP>"
}
}
}
]
}
AWS has predefined condition operators and keys (like aws:CurrentTime). Individual AWS services also define service-specific keys.
As an example, assume that you want to let user John access your Amazon SQS queue under the following conditions:
The time is after 12:00 p.m. on 7/16/2019
The time is before 3:00 p.m. on 7/16/2019
The request comes from an IP address within the range 192.0.2.0 to 192.0.2.255 or 203.0.113.0 to 203.0.113.255.
Your condition block has three separate condition operators, and all three of them must be met for John to have access to your queue, topic, or resource.
The following shows what the condition block looks like in your policy. The two values for aws:SourceIp are evaluated using OR. The three separate condition operators are evaluated using AND.
"Condition" : {
"DateGreaterThan" : {
"aws:CurrentTime" : "2019-07-16T12:00:00Z"
},
"DateLessThan": {
"aws:CurrentTime" : "2019-07-16T15:00:00Z"
},
"IpAddress" : {
"aws:SourceIp" : ["192.0.2.0/24", "203.0.113.0/24"]
}
}
reference: https://docs.aws.amazon.com/IAM/latest/UserGuide/reference_policies_multi-value-conditions.html
this is an old question, but I think that there is a better solution with AWS new capabilities. Especially, I don't really like the deny / StringNotLike combination, because denying on an s3 policy can have unexpected effects such as locking your own S3 bucket down, by denying yourself (this could only be fixed by using the root account, which you may not have easily accessible in a professional context)
So the solution I have in mind is to use ForAnyValue in your condition (source). e.g something like this:
{
"Version": "2012-10-17",
"Id": "Policy1496253408968",
"Statement": [
{
"Sid": "Stmt1496253402061",
"Effect": "Allow",
"Principal": "*",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::xyz-sam-test/*",
"arn:aws:s3:::xyz-sam-test"
],
"Condition": {
"ForAnyValue:StringEquals": {
"aws:sourceVpc": [
"vpc-dcb634bf",
"<MY PUBLIC IP>"
]
}
}
}
]
}

Unable to access S3 from EC2 Instance in Cloudformation -- A client error (301) occurred when calling the HeadObject operation: Moved Permanently

I'm trying to download a file from an S3 bucket to an instance through the userdata property of the instance. However, I get the error:
A client error (301) occurred when calling the HeadObject operation:
Moved Permanently.
I use an IAM Role, Managed Policy, and Instance Profile to give the instance accessibility to the s3 bucket:
"Role": {
"Type": "AWS::IAM::Role",
"Properties": {
"AssumeRolePolicyDocument": {
"Statement": [
{
"Effect": "Allow",
"Principal": {
"Service": [
"ec2.amazonaws.com",
"s3.amazonaws.com"
]
},
"Action": [
"sts:AssumeRole"
]
}
]
},
"Path": "/",
"ManagedPolicyArns": [
{
"Ref": "ManagedPolicy"
}
]
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "069d4411-2718-400f-98dd-529bb95fd531"
}
}
},
"RolePolicy": {
"Type": "AWS::IAM::Policy",
"Properties": {
"PolicyName": "S3Download",
"PolicyDocument": {
"Statement": [
{
"Action": [
"s3:*"
],
"Effect": "Allow",
"Resource": "arn:aws:s3:::mybucket/*"
}
]
},
"Roles": [
{
"Ref": "Role"
}
]
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "babd8869-948c-4b8a-958d-b1bff9d3063b"
}
}
},
"InstanceProfile": {
"Type": "AWS::IAM::InstanceProfile",
"Properties": {
"Path": "/",
"Roles": [
{
"Ref": "Role"
}
]
},
"Metadata": {
"AWS::CloudFormation::Designer": {
"id": "890c4df0-5d25-4f2c-b81e-05a8b8ab37c4"
}
}
},
And I attempt to download the file using this line in the userdata property:
aws s3 cp s3://mybucket/login.keytab
destination_directory/
Any thoughts as to what is going wrong? I can download the file successfully if I make it public then use wget from the command line, but for some reason the bucket/file can't be found when using cp and the file isn't publicly accessible.
Moved Permanently normally indicates that you are being redirected to the location of the object. This is normally because the request is being sent to an endpoint that is in a different region.
Add a --region parameter where the region matches the bucket's region. For example:
aws s3 cp s3://mybucket/login.keytab destination_directory/ --region ap-southeast-2
you can modify /root/.aws/credentials file and add region like region = ap-southeast-2

s3 bucket policy to add exception

Hi I am trying to write the permissions policy to access my bucket.
I want to deny access to a particular user-agent and allow access to all other user agents. With the below policy the access is getting denied to all.
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Stmt1456658595000",
"Effect": "Deny",
"Action": [
"s3:*"
],
"Condition": {
"StringLike": {
"aws:UserAgent": "NSPlayer"
}
},
"Resource": [
"arn:aws:s3:::bucket/"
]
},
{
"Effect": "Allow",
"Action": [
"s3:*"
],
"Resource": [
"arn:aws:s3:::bucket/*"
]
}
]
}
Please let me know how should I write such policy so that except one user agent all others are able to access the same.
It has to be written this way!
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "SID",
"Effect": "Allow",
"Action": [
"s3:*"
],
"Condition": {
"StringNotLike": {
"aws:UserAgent": "NSPlayer"
}
},
"Resource": [
"*"
]
}
]
}
This solution works if bucket objects are not public read/write. One related answer is here: Deny access to user agent to access a bucket in AWS S3