Serverless: TypeError: Cannot read property 'stage' of undefined - serverless-framework

frameworkVersion: '2'
plugins:
- serverless-step-functions
- serverless-python-requirements
- serverless-parameters
- serverless-pseudo-parameters
provider:
name: aws
region: us-east-2
stage: ${opt:stage, 'dev'}
runtime: python3.7
versionFunctions: false
iam:
role: arn:aws:iam::#{AWS::AccountId}:role/AWSLambdaVPCAccessExecutionRole
apiGateway:
shouldStartNameWithService: true
lambdaHashingVersion: 20201221
package:
exclude:
- node_modules/**
- venv/**
# Lambda functions
functions:
generateAlert:
handler: handler.generateAlert
generateData:
handler: handler.generateDataHandler
timeout: 600
approveDenied:
handler: handler.approveDenied
timeout: 600
stepFunctions:
stateMachines:
"claims-etl-and-insight-generation-${self:provider.stage}":
loggingConfig:
level: ALL
includeExecutionData: true
destinations:
- Fn::GetAtt: ["ETLStepFunctionLogGroup", Arn]
name: "claims-etl-and-insight-generation-${self:provider.stage}"
definition:
Comment: "${self:provider.stage} ETL Workflow"
StartAt: RawQualityJob
States:
# Raw Data Quality Check Job Start
RawQualityJob:
Type: Task
Resource: arn:aws:states:::glue:startJobRun.sync
Parameters:
JobName: "data_quality_v2_${self:provider.stage}"
Arguments:
"--workflow-name": "${self:provider.stage}-Workflow"
"--dataset_id.$": "$.datasetId"
"--client_id.$": "$.clientId"
Next: DataQualityChoice
Retry:
- ErrorEquals: [States.ALL]
MaxAttempts: 2
IntervalSeconds: 10
BackoffRate: 5
Catch:
- ErrorEquals: [States.ALL]
Next: GenerateErrorAlertDataQuality
# End Raw Data Quality Check Job
DataQualityChoice:
Type: Task
Resource:
Fn::GetAtt: [approveDenied, Arn]
Next: Is Approved ?
Is Approved ?:
Type: Choice
Choices:
- Variable: "$.quality_status"
StringEquals: "Denied"
Next: FailState
Default: HeaderLineJob
FailState:
Type: Fail
Cause: "Denied status"
# Header Line Job Start
HeaderLineJob:
Type: Parallel
Branches:
- StartAt: HeaderLineIngestion
States:
HeaderLineIngestion:
Type: Task
Resource: arn:aws:states:::glue:startJobRun.sync
Parameters:
JobName: headers_lines_etl_rs_v2
Arguments:
"--workflow-name.$": "$.Arguments.--workflow-name"
"--dataset_id.$": "$.Arguments.--dataset_id"
"--client_id.$": "$.Arguments.--client_id"
End: True
Retry:
- ErrorEquals: [States.ALL]
MaxAttempts: 2
IntervalSeconds: 10
BackoffRate: 5
Catch:
- ErrorEquals: [States.ALL]
Next: GenerateErrorAlertHeaderLine
End: True
# Header Line Job End
GenerateErrorAlertDataQuality:
Type: Task
Resource:
Fn::GetAtt: [generateAlert, Arn]
End: true
resources:
Resources:
# Cloudwatch Log
"ETLStepFunctionLogGroup":
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: "ETLStepFunctionLogGroup_${self:provider.stage}"
This is what my serverless.yml file looks like.
When I run the command:
sls deploy --stage staging
It show
Type Error ----------------------------------------------
TypeError: Cannot read property 'stage' of undefined
at Variables.getValueFromOptions (/snapshot/serverless/lib/classes/Variables.js:648:37)
at Variables.getValueFromSource (/snapshot/serverless/lib/classes/Variables.js:579:17)
at /snapshot/serverless/lib/classes/Variables.js:539:12
Your Environment Information ---------------------------
Operating System: linux
Node Version: 14.4.0
Framework Version: 2.30.3 (standalone)
Plugin Version: 4.5.1
SDK Version: 4.2.0
Components Version: 3.7.4
How I can fix this? I tried with different version of serverless.
There is error in yamlParser file, which is provided by serverless-step-functions.
Above is my serverless config file.

It looks like a $ sign is missing from your provider -> stage?
provider:
name: aws
region: us-east-2
stage: ${opt:stage, 'dev'} # $ sign is missing?
runtime: python3.7
versionFunctions: false
iam:
role: arn:aws:iam::#{AWS::AccountId}:role/AWSLambdaVPCAccessExecutionRole
apiGateway:
shouldStartNameWithService: true
lambdaHashingVersion: 20201221

Related

EventRule matched by default EventBridge EventBus ignored by custom EventBus

I have a Cloudformation template which implements a CI/CD process for Lambda functions (at bottom)
Essentially it -
watches a Github repo
pulls source code on new git tag creation
starts a CodeBuild process which runs unit tests, zips source code (on test success) and pushes archive to S3
enables CodeBuild notifications
implements a CloudWatch EventRule to pattern match raw CodeBuild notifications, format and push them to SNS
binds a Lambda function to SNS, which pushes the notifications to Slack via a webhook
This works fine with the default EventBridge EventBus, but the pattern matching seems to fail if I switch to a custom EventBus.
(see resources EventBus and EventRule in the stack; it's currently set up to use the custom EventBus, and fails / ignores new git tags; if you comment out the reference to property EventBusName in EventBus, it defaults to using the default EventBus and works)
Why would a custom EventBus behave differently to the default EventBus in this situation ?
---
AWSTemplateFormatVersion: '2010-09-09'
Parameters:
AppName:
Type: String
RepoOwner:
Type: String
RepoName:
Type: String
RepoBranch:
Type: String
Default: master
RepoAuth:
Type: String
Default: master
WebhookUrl:
Type: String
WebhookLambda:
Type: String
CodeBuildBuildSpec:
Type: String
CodeBuildType:
Type: String
Default: LINUX_CONTAINER
CodeBuildComputeType:
Type: String
Default: BUILD_GENERAL1_SMALL
CodeBuildImage:
Type: String
Default: aws/codebuild/standard:4.0
LambdaHandler:
Type: String
Default: "index.handler"
LambdaMemory:
Type: Number
Default: 128
LambdaTimeout:
Type: Number
Default: 30
LambdaRuntime:
Type: String
Default: python3.8
Resources:
ArtifactsBucket:
Type: AWS::S3::Bucket
Properties:
BucketName:
Fn::Sub:
- ${app_name}-lambda-artifacts
- app_name:
Ref: AppName
CodeBuildProject:
Properties:
Environment:
ComputeType:
Ref: CodeBuildComputeType
Image:
Ref: CodeBuildImage
Type:
Ref: CodeBuildType
Name:
Fn::Sub:
- ${app_name}-lambda-ci
- app_name:
Ref: AppName
ServiceRole:
Fn::GetAtt:
- CodeBuildRole
- Arn
Source:
Auth:
Resource:
Ref: RepoAuth
Type: OAUTH
Location:
Fn::Sub:
- "https://github.com/${repo_owner}/${repo_name}.git"
- repo_owner:
Ref: RepoOwner
repo_name:
Ref: RepoName
Type: GITHUB
BuildSpec:
Fn::Sub:
- "${build_spec}"
- build_spec:
Ref: CodeBuildBuildSpec
Artifacts:
Type: NO_ARTIFACTS
SourceVersion:
Ref: RepoBranch
Triggers:
Webhook: true
FilterGroups:
- - Type: EVENT
Pattern: PUSH
ExcludeMatchedPattern: false
- Type: HEAD_REF
Pattern: "refs/tags/.*"
ExcludeMatchedPattern: false
Type: AWS::CodeBuild::Project
CodeBuildRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: codebuild.amazonaws.com
Version: '2012-10-17'
Policies:
- PolicyDocument:
Statement:
- Action:
- codebuild:*
- events:*
- s3:PutObject
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName: code-build-role-policy
Type: AWS::IAM::Role
WebhookFunction:
Properties:
FunctionName:
Fn::Sub:
- ${app_name}-lambda-webhook
- app_name:
Ref: AppName
Code:
ZipFile:
Ref: WebhookLambda
Environment:
Variables:
WEBHOOK_URL:
Ref: WebhookUrl
Handler:
Ref: LambdaHandler
MemorySize:
Ref: LambdaMemory
Role:
Fn::GetAtt:
- WebhookFunctionRole
- Arn
Runtime:
Ref: LambdaRuntime
Timeout:
Ref: LambdaTimeout
Type: AWS::Lambda::Function
WebhookFunctionRole:
Properties:
AssumeRolePolicyDocument:
Statement:
- Action: sts:AssumeRole
Effect: Allow
Principal:
Service: lambda.amazonaws.com
Version: '2012-10-17'
Policies:
- PolicyDocument:
Statement:
- Action:
- logs:CreateLogGroup
- logs:CreateLogStream
- logs:PutLogEvents
Effect: Allow
Resource: '*'
Version: '2012-10-17'
PolicyName: webhook-role-policy
Type: AWS::IAM::Role
WebhookFunctionPermission:
Properties:
Action: "lambda:InvokeFunction"
FunctionName:
Ref: WebhookFunction
Principal: "sns.amazonaws.com"
SourceArn:
Ref: WebhookTopic
Type: AWS::Lambda::Permission
WebhookTopic:
Properties:
Subscription:
- Protocol: lambda
Endpoint:
Fn::GetAtt:
- WebhookFunction
- Arn
Type: AWS::SNS::Topic
WebhookTopicPolicy:
Properties:
PolicyDocument:
Version: "2012-10-17"
Statement:
- Effect: Allow
Principal:
Service: "events.amazonaws.com"
Action:
- "sns:Publish"
Resource:
Ref: WebhookTopic
Topics:
- Ref: WebhookTopic
Type: AWS::SNS::TopicPolicy
EventBus:
Type: AWS::Events::EventBus
Properties:
Name:
Fn::Sub:
- ${app_name}-bus
- app_name:
Ref: AppName
EventRule:
Type: AWS::Events::Rule
Properties:
EventBusName: # CURRENTLY USING CUSTOM EVENT BUS (PATTERN MATCHING FAILS); REMOVE THIS PROPERTY TO SWITCH TO DEFAULT EVENT BUS (PATTERN MATCHING WORKS)
Ref: EventBus
EventPattern:
source:
- "aws.codebuild"
detail-type:
- "CodeBuild Build Phase Change"
detail:
completed-phase:
- SUBMITTED
- PROVISIONING
- DOWNLOAD_SOURCE
- INSTALL
- PRE_BUILD
- BUILD
- POST_BUILD
- UPLOAD_ARTIFACTS
- FINALIZING
completed-phase-status:
- TIMED_OUT
- STOPPED
- FAILED
- SUCCEEDED
- FAULT
- CLIENT_ERROR
project-name:
- Ref: CodeBuildProject
State: ENABLED
Targets:
- Arn:
Ref: WebhookTopic
Id:
Fn::Sub:
- "${project_name}-codebuild-notifications"
- project_name:
Ref: CodeBuildProject
InputTransformer:
InputPathsMap:
build-id: "$.detail.build-id"
project-name: "$.detail.project-name"
completed-phase: "$.detail.completed-phase"
completed-phase-status: "$.detail.completed-phase-status"
InputTemplate: |
"{'build-id': '<build-id>', 'project-name': '<project-name>', 'completed-phase': '<completed-phase>', 'completed-phase-status': '<completed-phase-status>'}"
"Custom event buses serve a use case of receiving events from your custom applications and services. Unfortunately, it is not possible for AWS services to push events to a custom event bus." (AWS Support)

How to eliminate serverless framework error Template format error

Testing, learning serverless framework. I'm trying to deploy simple/basic state machine with two simple lambda functions.
Serverless definition as follows:
frameworkVersion: '2'
app: state-machine
org: macdrorepo
service: state-machine
plugins:
- serverless-python-requirements
- serverless-iam-roles-per-function
- serverless-step-functions
- serverless-pseudo-parameters
custom:
pythonRequirements:
dockerizePip: non-linux
slim: true
zip: true
provider:
name: aws
runtime: python3.8
region: eu-central-1
stage: ${opt:stage, 'testing'}
timeout: 30
package:
individually: true
exclude:
- node_modules/**
- .git/**
- .venv/**
functions:
processpurchase:
module: state-machine
memorySize: 128
stages:
- testing
- dev
handler: ProcessPurchase.process_purchase
processrefund:
module: state-machine
memorySize: 128
stages:
- testing
- dev
handler: ProcessRefund.process_refund
stepFunctions:
validate: true
stateMachines:
TransactionChoiceMachine:
name: ChoiceMachineTest-${self:provider.stage}
dependsOn: CustomIamRole
definition:
Comment: "Purchase refund choice"
StartAt: ProcessTransaction
States:
ProcessTransaction:
Type: Choice
Choices:
- Variable: "$.TransactionType"
StringEquals: PURCHASE
Next: PurchaseState
- Variable: "$.TransactionType"
StringEquals: REFUND
Next: RefundState
PurchaseState:
Type: Task
Resource:
Fn::GetAtt: [processpurchase, Arn]
End: true
RefundState:
Type: Task
Resource:
Fn::GetAtt: [processrefund, Arn]
End: true
During deploy, sls is saying my state machine definition is ok: State machine "TransactionChoiceMachine" definition is valid
My environment information:
Your Environment Information ---------------------------
Operating System: linux
Node Version: 12.20.0
Framework Version: 2.14.0
Plugin Version: 4.1.2
SDK Version: 2.3.2
Components Version: 3.4.3
Setup SLS_DEBUG=* is not helping me much as I do not know js unfortunately.
After serverless deploy command, I'm getting error:
Error: The CloudFormation template is invalid: Template format error: Unresolved resource dependencies [CustomIamRole] in the Resources block of the template
Looks like you are referencing something called CustomIamRole in your state machine creation but I cannot see it being created anywhere in the yaml file. Either create the role and use it in creation or remove the depends on part.

Serverless Enterprise deployment

I recently updated to v1.44.0 and used the #serverless/enterprise-plugin and am now unable to deploy. I’m simply trying to create a User Pool, but keep getting an error.
An error occurred: EnterpriseLogAccessIamRole - Policy statement must contain resources. (Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument; Request ID: dc158686-378c-4d01-97fb-1414d55a735d)
serverless.yml
tenant: [omitted]
app: [omitted]
service: auth
frameworkVersion: ">=1.44.0"
plugins:
- '#serverless/enterprise-plugin'
provider:
name: aws
runtime: nodejs8.10
region: us-east-1
custom:
stage: ${opt:stage, self:provider.stage}
cognito:
app:
userPool: ${self:service}-app-user-pool-${self:custom.stage}
identityPool: AppIdentityPoolDev
resources:
Resources:
AppUserPool:
Type: AWS::Cognito::UserPool
Properties:
UserPoolName: ${self:custom.cognito.app.userPool}
UsernameAttributes:
- email
AutoVerifiedAttributes:
- email
MobileAppClient:
Type: AWS::Cognito::UserPoolClient
Properties:
ClientName: ${self:service}-mobile-app-client-${self:custom.stage}
UserPoolId:
Ref: AppUserPool
GenerateSecret: true
Outputs:
AppUserPool:
Value:
Ref: AppUserPool
MobileAppClient:
Value:
Ref: MobileAppClient

Unable to deploy application on EC2 instance using AWS CloudFormation template through cfn-init and UserData script

I am trying to deploy sample.war application on EC2 instance at the time of launch. That is when an instance is launched the application should be deployed automatically on it using cfn-init and Metadata. I added a user with policy and authentication with no luck. If I wget with the S3 path, the file is being downloaded. Below is my script. What am I missing in this, or is there any other way to do this?
---
AWSTemplateFormatVersion: 2010-09-09
Description: Test QA Template
Resources:
MyInstance:
Type: AWS::EC2::Instance
Properties:
ImageId: !Ref AMIIdParam
InstanceType: !Ref InstanceType
Metadata:
AWS::CloudFormation::Init:
config:
packages:
yum:
java-1.8.0-openjdk.x86_64: []
tomcat: []
httpd.x86_64: []
services:
sysvinit:
httpd:
enabled: true
ensureRunning: true
files:
/usr/share/tomcat/webapps/sample.zip:
source: https://s3.amazonaws.com/mybucket/sample.zip
mode: '000500'
owner: tomcat
group: tomcat
authentication: S3AccessCreds
AWS::CloudFormation::Authentication:
S3AccessCreds:
type: 'S3'
accessKeyId: !Ref HostKeys
secretKey: Fn::GetAtt:
- HostKeys
- SecretAccessKey
buckets: !Ref BucketName
CfnUser:
Type: AWS::IAM::User
Properties:
Path: '/'
Policies:
- PolicyName: 'S3Access'
PolicyDocument:
Statement:
- Effect: 'Allow'
Action: s3:*
Resource: '*'
HostKeys:
Type: AWS::IAM::AccessKey
Properties:
UserName: !Ref CfnUser
I was unable to reproduce this using the following template:
---
AWSTemplateFormatVersion: 2010-09-09
Description: Test QA Template
Resources:
MyInstance:
Type: AWS::EC2::Instance
Properties:
ImageId: ami-08589eca6dcc9b39c
InstanceType: t2.micro
KeyName: default
UserData:
Fn::Base64: !Sub |
#!/bin/bash -xe
/opt/aws/bin/cfn-init -s ${AWS::StackId} --resource MyInstance --region ${AWS::Region}
Metadata:
AWS::CloudFormation::Init:
config:
packages:
yum:
java-1.8.0-openjdk.x86_64: []
tomcat: []
httpd.x86_64: []
services:
sysvinit:
httpd:
enabled: true
ensureRunning: true
files:
/usr/share/tomcat/webapps/sample.zip:
source: https://s3.amazonaws.com/mybucket/sample.zip
mode: '000500'
owner: tomcat
group: tomcat
(In other words, use of the above template allowed me to install a sample.zip file using cfn-init.)
Thus there is something permissions-related in the way you're accessing the S3 bucket.
Suffice to say it is a bad practice to use Access Keys. Have a look at this document on best practices of assigning an IAM Role to an EC2 instance and then adding a Bucket Policy that grants appropriate access to that Role.

Serverless Framework: Request must be smaller than 69905067 bytes for the UpdateFunctionCode operation

I am using package uploading zipped file like
frameworkVersion: "=1.27.3"
service: recipes
provider:
name: aws
endpointType: REGIONAL
runtime: python3.6
stage: dev
region: eu-central-1
memorySize: 512
deploymentBucket:
name: dfki-meta
versionFunctions: false
stackTags:
Project: DFKIAPP
# Allows updates to all resources except deleting/replacing EC2 instances
stackPolicy:
- Effect: Allow
Principal: "*"
Action: "Update:*"
Resource: "*"
- Effect: Deny
Principal: "*"
Action:
- Update: Replace
- Update: Delete
Resource: "*"
Condition:
StringEquals:
ResourceType:
- AWS::EC2::Instance
# Access to RDS and S3 Bucket
iamRoleStatements:
- Effect: "Allow"
Action: "s3:ListBucket"
Resource: "*"
package:
individually: true
functions:
# get_recipes:
# handler: handler.get_recipes
# module: recipes_crud
# package:
# individually: true
# timeout: 30
# events:
# - http:
# path: recipes
# method: get
# request:
# parameters:
# querystring:
# persona: true
get_recommendation:
handler: handler.get_recommendation
module: recipes_ml
package:
artifact: zipped_dir.zip
timeout: 30
events:
- http:
path: recipes/{id}
method: get
request:
parameters:
paths:
id: true
querystring:
schaerfe_def: true
saettig_def: true
erfahrung_def: true
schaerfe_wunsch: true
saettig_wunsch: true
erfahrung_wunsch: true
gericht_wunsch: true
stimmung_wunsch: true
Can not understand this error, isn't 52.18 under 69905067 bytes ?
(node:50928) ExperimentalWarning: The fs.promises API is experimental
Serverless: Packaging function: get_recommendation...
Serverless: Uploading function: get_recommendation (52.18 MB)...
Serverless Error ---------------------------------------
Request must be smaller than 69905067 bytes for the UpdateFunctionCode operation
Get Support --------------------------------------------
Docs: docs.serverless.com
Bugs: github.com/serverless/serverless/issues
Issues: forum.serverless.com
Your Environment Information -----------------------------
OS: darwin
Node Version: 10.1.0
Serverless Version: 1.27.3
The package size should be lower than 50MB according to the docs
https://docs.aws.amazon.com/lambda/latest/dg/limits.html
from this blog post
The 20 MB addition presumably is there there to account for request
overhead involved with the AWS API (e.g. base64 encoding of the zip
file data). So far the 50 MB limit holds true-ish. But, we’re not
defeated yet.
This seems to be an issue only while uploading individual lambda function using serverless but if you don't give --function parameter and deploy full stack then it works fine!!!