How to refer the environment variable value for s3 bucket name in sam template? - amazon-s3

I am trying to build s3 event application by referring the environment variable defined.
Below the template i am referring
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
CreateThumbnail:
Type: AWS::Serverless::Function
Properties:
Handler: handler
Runtime: runtime
Timeout: 60
Policies: AWSLambdaExecute
Environment:
Variables:
BUCKET_NAME: !Sub "${S3}"
Events:
CreateThumbnailEvent:
Type: S3
Properties:
Bucket: ""How can refer the s3 bucket form the environment here"
Events: s3:ObjectCreated:*
i actually need to refer the 'BUCKET NAME' from the 'Environment variable' and also need to add event trigger from the same s3.
can anyone help on this?
Thanks

There are two options,
Either you have defined the S3 bucket in the same template, and then you can reference it using the logical CFN ID:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Resources:
S3Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: my-bucket-name
CreateThumbnail:
Type: AWS::Serverless::Function
Properties:
Handler: handler
Runtime: runtime
Timeout: 60
Policies: AWSLambdaExecute
Environment:
Variables:
BUCKET_NAME: !Ref S3Bucket
Events:
CreateThumbnailEvent:
Type: S3
Properties:
Bucket: !Ref S3Bucket
Events: s3:ObjectCreated:*
Or you do not have it defined in the same template, in which case you have to pass it to the stack as a parameter during deployment like this:
AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Parameters:
S3Bucket:
Description: Name of the S3 Bucket
Type: String
Resources:
CreateThumbnail:
Type: AWS::Serverless::Function
Properties:
Handler: handler
Runtime: runtime
Timeout: 60
Policies: AWSLambdaExecute
Environment:
Variables:
BUCKET_NAME: !Ref S3Bucket
Events:
CreateThumbnailEvent:
Type: S3
Properties:
Bucket: !Ref S3Bucket
Events: s3:ObjectCreated:*

Related

Unable to create S3 Bucket with Events via SAM: The event is not supported notifications (Service: Amazon Status Code: 400; Error InvalidArgument;

Transform: AWS::Serverless-2016-10-31
Description: >
patientcheckout
Sample SAM Template for patientcheckout
# More info about Globals: https://github.com/awslabs/serverless-application-model/blob/master/docs/globals.rst
Globals:
Function:
Timeout: 20
Runtime: java11
#Architectures:
# - x86_64
MemorySize: 512
Resources:
PatientCheckoutBucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub ${AWS::StackName}-${AWS::AccountId}-${AWS::Region}
PatientCheckoutFunction:
Type: AWS::Serverless::Function
Properties:
CodeUri: patientcheckout
Handler: com.harsha.aws.lambda.patientcheckout.PatientCheckoutLambda::handler
Policies:
- S3ReadPolicy:
BucketName: !Sub ${AWS::StackName}-${AWS::AccountId}-${AWS::Region}
Events:
S3Event:
Type: S3
Properties:
Bucket: !Ref PatientCheckoutBucket
Events: s3:ObjectCreate:*
The event is not supported notifications (Service: Amazon S3 Status Code: 400; Error InvalidArgument;
I get this when error I run the code with sam cli. sam build is success, but s3 bucket creation fails

SNS topic not receiving CodeBuild notifications

Trying to get CodeBuild to push notifications to an SNS topic (bound to a Lambda), via a CloudWatch Events Rule.
Cloudformation template (see below) deploys fine.
CodeBuild process works fine (have tested).
SNS topic and bound Lambda work fine - I can push a message to the topic via AWS CLI and see Lambda dump that message to Cloudwatch logs.
The Cloudwatch Event Rule seems to be configured fine - I can see it in the cosole, it looks well formed, seems to be bound to SNS topic.
In addition I have been careful to give the Event Rule a role with permissions to sns:Publish, and also defined an AWS::SNS::TopicPolicy for the SNS topic -
Unable to successfully set up SNS on CodeBuild project through CFT but works manually
But still nothing - CodeBuild successfully completes but I don't receive any notifications.
Any thoughts as to what might be wrong ?
TIA :)
---
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
AppName:
Type: String
StagingBucket:
Type: String
RepoOwner:
Type: String
RepoName:
Type: String
RepoBranch:
Type: String
RepoPAT:
Type: String
CodeBuildBuildSpec:
Type: String
CodeBuildType:
Type: String
Default: LINUX_CONTAINER
CodeBuildComputeType:
Type: String
Default: BUILD_GENERAL1_SMALL
CodeBuildImage:
Type: String
Default: aws/codebuild/standard:4.0
LambdaHandler:
Type: String
Default: "index.handler"
LambdaMemory:
Type: Number
Default: 128
LambdaTimeout:
Type: Number
Default: 30
LambdaRuntime:
Type: String
Default: python3.8
Resources:
CodeBuildProject:
DependsOn:
- CodeBuildSourceCredential
Properties:
Environment:
ComputeType:
Ref: CodeBuildComputeType
Image:
Ref: CodeBuildImage
Type:
Ref: CodeBuildType
Name:
Ref: AppName
ServiceRole:
Fn::GetAtt:
- CodeBuildRole
- Arn
Source:
Location:
Fn::Sub:
- "https://github.com/${repo_owner}/${repo_name}.git"
- repo_owner:
Ref: RepoOwner
repo_name:
Ref: RepoName
Type: GITHUB
BuildSpec:
Fn::Sub:
- "${build_spec}"
- build_spec:
Ref: CodeBuildBuildSpec
Artifacts:
Type: S3
Location:
Ref: StagingBucket
SourceVersion:
Ref: RepoBranch
Triggers:
Webhook: true
FilterGroups:
- - Type: EVENT
Pattern: PUSH
ExcludeMatchedPattern: false
- Type: HEAD_REF
Pattern: "refs/tags/.*"
ExcludeMatchedPattern: false
Type: AWS::CodeBuild::Project
CodeBuildSourceCredential:
Type: AWS::CodeBuild::SourceCredential
Properties:
Token:
Ref: RepoPAT
ServerType: GITHUB
AuthType: PERSONAL_ACCESS_TOKEN
CodeBuildRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: codebuild.amazonaws.com
Version: '2012-10-17'
ManagedPolicyArns:
- arn:aws:iam::aws:policy/AdministratorAccess
Path: /
Type: AWS::IAM::Role
CodeBuildNotificationFunction:
Properties:
Code:
ZipFile: "def handler(event, context):\n print (event)"
Handler:
Ref: LambdaHandler
MemorySize:
Ref: LambdaMemory
Role:
Fn::GetAtt:
- CodeBuildNotificationFunctionRole
- Arn
Runtime:
Ref: LambdaRuntime
Timeout:
Ref: LambdaTimeout
Type: AWS::Lambda::Function
CodeBuildNotificationFunctionRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Version: '2012-10-17'
Policies:
- PolicyDocument:
Statement:
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName: code-build-notification-role-policy
Type: AWS::IAM::Role
CodeBuildNotificationTopic:
Properties:
Subscription:
- Protocol: lambda
Endpoint:
Fn::GetAtt:
- CodeBuildNotificationFunction
- Arn
Type: AWS::SNS::Topic
CodeBuildNotificationTopicPolicy:
Properties:
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: "events.amazonaws.com"
Action:
- "sns:Publish"
Resource:
Ref: CodeBuildNotificationTopic
Topics:
- Ref: CodeBuildNotificationTopic
Type: AWS::SNS::TopicPolicy
CodeBuildNotificationLambdaInvokePermission:
Properties:
Action: "lambda:InvokeFunction"
FunctionName:
Ref: CodeBuildNotificationFunction
Principal: "sns.amazonaws.com"
SourceArn:
Ref: CodeBuildNotificationTopic
Type: AWS::Lambda::Permission
SampleNotificationRule:
Type: AWS::Events::Rule
Properties:
EventPattern:
Fn::Sub:
- '{"source": ["aws.codebuild"], "detail-type": ["Codebuild Build Phase Change"], "detail": {"completed-phase": ["SUBMITTED", "PROVISIONING", "DOWNLOAD_SOURCE", "INSTALL", "PRE_BUILD", "BUILD", "POST_BUILD", "UPLOAD_ARTIFACTS", "FINALIZING"], "completed-phase-status": ["TIMED_OUT", "STOPPED", "FAILED", "SUCCEEDED", "FAULT", "CLIENT_ERROR"], "project-name": ["${project_name}"]}}'
- project_name:
Ref: CodeBuildProject
State: ENABLED
RoleArn:
Fn::GetAtt:
- SampleNotificationRuleRole
- Arn
Targets:
- Arn:
Ref: CodeBuildNotificationTopic
Id: sample-notification
SampleNotificationRuleRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: events.amazonaws.com
Version: '2012-10-17'
Policies:
- PolicyDocument:
Statement:
- Action:
- "sns:Publish"
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName: sample-notification-rule-role-policy
Type: AWS::IAM::Role
Fixed - AWS::Events::Rule target was missing an InputTransformer block like so
Targets:
- Arn:
Ref: CodeBuildNotificationTopic
Id: sample-notification
InputTransformer:
InputPathsMap:
build-id: "$.detail.build-id"
project-name: "$.detail.project-name"
completed-phase: "$.detail.completed-phase"
completed-phase-status: "$.detail.completed-phase-status"
InputTemplate: |
"Build '<build-id>' for build project '<project-name>' has completed the build phase of '<completed-phase>' with a status of '<completed-phase-status>'."

InvalidCondition in s3 Bucket Policy

I am creating a cloudformation stack where the templates creates Cloud trail and then S3 bucket and pushes all logs to S3 bucket.
I have tried creating Cloud trail,s3 bucket and tried attaching the s3 Bucket Policy to the bucket
Parameters:
loggroupname:
Type: String
trailname:
Type: String
s3bucketname:
Type: String
Resources:
createloggroup:
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: !Sub ${loggroupname}
creates3bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Sub ${s3bucketname}
s3bucketpolicy:
Type: AWS::S3::BucketPolicy
Properties:
Bucket: !Sub ${s3bucketname}
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: 'AWSCloudTrailAclCheck20150319'
Effect: 'Allow'
Principal:
Service: 'cloudtrail.amazonaws.com'
Action: 's3:GetBucketAcl'
Resource:
!Sub 'arn:aws:s3:::${s3bucketname}'
PolicyDocument:
Version: '2012-10-17'
Statement:
- Sid: AWSCloudTrailWrite20150319
Effect: 'Allow'
Principal:
Service: 'cloudtrail.amazonaws.com'
Action: 's3:PutObject'
Resource:
!Sub 'arn:aws:s3:::${s3bucketname}/AWSLogs/${AWS::AccountId}/*'
Condition:
StringsEquals:
s3:x-amz-acl: 'bucket-owner-full-control'
myvpctrail:
DependsOn:
- s3bucketpolicy
Type: AWS::CloudTrail::Trail
Properties:
IsLogging: true
IsMultiRegionTrail: true
IncludeGlobalServiceEvents: true
S3BucketName: !Ref creates3bucket
Invalid Condition type : StringsEquals (Service: Amazon S3; Status Code: 400; Error Code: MalformedPolicy; Request ID: F7439B111E82A3FA; S3 Extended Request ID: IGU1L7BB77WcrhPtmydd5j6viQdMK0vqA3Qo4RTS209FAvjT3q6wBIsyabdt5B7pBFvdr2MT+sM=)
Simple typo.
It's StringEquals not StringsEquals.
Source: https://docs.aws.amazon.com/AmazonS3/latest/dev/amazon-s3-policy-keys.html

unable to add notificationconfiguration to s3 bucket

Created cloud formation template to create bucket with notification.
Following is code:
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
CBRS3ToS3IADelay:
Description: Number of days before an S3 object is transitioned from S3 to S3-IA
Type: Number
Default: 365
CBRS3ToGlacierDelay:
Description: Number of days before an S3-IA object is transitioned from S3-IA to Glacier.
Type: Number
Default: 1460
CBRBucketName:
Description: S3 bucket name
Type: String
Default: "my-bucket-test0011"
Resources:
CBRS3Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName:
Ref: CBRBucketName
AccessControl: Private
LifecycleConfiguration:
Rules:
- Id: CbrCertReportGlacierArchiveRule
Status: Enabled
Transitions:
- StorageClass: STANDARD_IA
TransitionInDays: !Ref CBRS3ToS3IADelay
- StorageClass: GLACIER
TransitionInDays: !Ref CBRS3ToGlacierDelay
NotificationConfiguration:
LambdaConfigurations:
-
Function: "arn:aws:lambda:xxxx:xxxx:function:xxxx"
Event: "s3:ObjectCreated:Put"
Filter:
S3Key:
Rules:
-
Name: suffix
Value: ".gz"
Tags:
- Key: PRODUCT
Value: CRAWS
VersioningConfiguration:
Status: Enabled
Code working with notification block.
But above template is not working with notification.
Getting following error:
Unable to validate the following destination configurations (Service: Amazon S3; Status Code: 400; Error Code: InvalidArgument
I able to do from console.
Anyone help me to fix this issue?
this is late, so more of answering myself for this question (just managed to fix the same problem): it fails due to a preliminary check on s3 to invoke that lambda function, we will need this:
CBRS3BucketCanInvokeFunctionX:
Type: 'AWS::Lambda::Permission'
Properties:
FunctionName: ARN_OF_FUNCTION_X
Action: 'lambda:InvokeFunction'
Principal: s3.amazonaws.com
SourceAccount: !Ref 'AWS::AccountId'
SourceArn: !Sub 'arn:aws:s3:::${CBRBucketName}'
your CBRS3Bucket will also need to let above resource run first:
CBRS3Bucket:
Type: AWS::S3::Bucket
DependsOn: CBRS3BucketCanInvokeFunctionX
Try taking the .gz and put in just gz.

CloudFormation BucketPolicy stuck at CREATE. Never finished CREATE

Cloud forming an S3 bucket with user, user access keys, and policy. It should create the stack and output the user access keys needed to work with the created S3 bucket via the SDK. Bucket Policy gets stuck in the CREATING phase forever when trying to reference the BucketUser ARN in the BucketPolicy Principal.
CloudFormation is successful with
BucketPolicy: ... Principal: "*"
But BucketPolicy resource is stuck in CREATE forever with
BucketPolicy: ... Principal: !GetAtt BucketUser.Arn
This successfully returns the BucketUser.Arn when BucketPolicy: ... Principal: "*"
Outputs:
BucketUserArn:
Value: !GetAtt BucketUser.Arn
Desired Template:
AWSTemplateFormatVersion: "2010-09-09"
Description: "Creates bucket with bucket policy"
#Metadata:
Parameters:
app:
Type: String
Description: (required) Application name (Also used for bucket name. Follow S3 bucket name conventions)
Default: ymessage-bucket-test
Resources:
BucketUser:
Type: "AWS::IAM::User"
Properties:
UserName: !Ref app
UserAccessKey:
Type: "AWS::IAM::AccessKey"
Properties:
Status: Active
UserName: !Ref app
DependsOn: BucketUser
Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref app
BucketPolicy:
Type: "AWS::S3::BucketPolicy"
Properties:
Bucket: !Ref app
PolicyDocument:
Statement:
-
Action:
- "s3:*"
Effect: "Allow"
Resource:
Fn::Join:
- ""
-
- "arn:aws:s3:::"
- !Ref app
- "/*"
Principal: !GetAtt BucketUser.Arn
DependsOn: BucketUser
Outputs:
AccessKeyId:
Value: !Ref UserAccessKey
AccessKeySecret:
Value: !GetAtt UserAccessKey.SecretAccessKey
BucketURL:
Value: !GetAtt Bucket.WebsiteURL
BucketUserArn:
Value: !GetAtt BucketUser.Arn
Working Template:
AWSTemplateFormatVersion: "2010-09-09"
Description: "Creates bucket with bucket policy"
#Metadata:
Parameters:
app:
Type: String
Description: (required) Application name (Also used for bucket name. Follow S3 bucket name conventions)
Default: ymessage-bucket-test
Resources:
BucketUser:
Type: "AWS::IAM::User"
Properties:
UserName: !Ref app
UserAccessKey:
Type: "AWS::IAM::AccessKey"
Properties:
Status: Active
UserName: !Ref app
DependsOn: BucketUser
Bucket:
Type: AWS::S3::Bucket
Properties:
BucketName: !Ref app
BucketPolicy:
Type: "AWS::S3::BucketPolicy"
Properties:
Bucket: !Ref app
PolicyDocument:
Statement:
-
Action:
- "s3:*"
Effect: "Allow"
Resource:
Fn::Join:
- ""
-
- "arn:aws:s3:::"
- !Ref app
- "/*"
Principal: "*"
DependsOn: BucketUser
Outputs:
AccessKeyId:
Value: !Ref UserAccessKey
AccessKeySecret:
Value: !GetAtt UserAccessKey.SecretAccessKey
BucketURL:
Value: !GetAtt Bucket.WebsiteURL
BucketUserArn:
Value: !GetAtt BucketUser.Arn
Found the issue: In the BucketPolicy it can directly accept Principal: "*" but if you want to use an arn do this:
Principal:
AWS:
- !GetAtt BucketUser.Arn