How to eliminate serverless framework error Template format error - serverless-framework

Testing, learning serverless framework. I'm trying to deploy simple/basic state machine with two simple lambda functions.
Serverless definition as follows:
frameworkVersion: '2'
app: state-machine
org: macdrorepo
service: state-machine
plugins:
- serverless-python-requirements
- serverless-iam-roles-per-function
- serverless-step-functions
- serverless-pseudo-parameters
custom:
pythonRequirements:
dockerizePip: non-linux
slim: true
zip: true
provider:
name: aws
runtime: python3.8
region: eu-central-1
stage: ${opt:stage, 'testing'}
timeout: 30
package:
individually: true
exclude:
- node_modules/**
- .git/**
- .venv/**
functions:
processpurchase:
module: state-machine
memorySize: 128
stages:
- testing
- dev
handler: ProcessPurchase.process_purchase
processrefund:
module: state-machine
memorySize: 128
stages:
- testing
- dev
handler: ProcessRefund.process_refund
stepFunctions:
validate: true
stateMachines:
TransactionChoiceMachine:
name: ChoiceMachineTest-${self:provider.stage}
dependsOn: CustomIamRole
definition:
Comment: "Purchase refund choice"
StartAt: ProcessTransaction
States:
ProcessTransaction:
Type: Choice
Choices:
- Variable: "$.TransactionType"
StringEquals: PURCHASE
Next: PurchaseState
- Variable: "$.TransactionType"
StringEquals: REFUND
Next: RefundState
PurchaseState:
Type: Task
Resource:
Fn::GetAtt: [processpurchase, Arn]
End: true
RefundState:
Type: Task
Resource:
Fn::GetAtt: [processrefund, Arn]
End: true
During deploy, sls is saying my state machine definition is ok: State machine "TransactionChoiceMachine" definition is valid
My environment information:
Your Environment Information ---------------------------
Operating System: linux
Node Version: 12.20.0
Framework Version: 2.14.0
Plugin Version: 4.1.2
SDK Version: 2.3.2
Components Version: 3.4.3
Setup SLS_DEBUG=* is not helping me much as I do not know js unfortunately.
After serverless deploy command, I'm getting error:
Error: The CloudFormation template is invalid: Template format error: Unresolved resource dependencies [CustomIamRole] in the Resources block of the template

Looks like you are referencing something called CustomIamRole in your state machine creation but I cannot see it being created anywhere in the yaml file. Either create the role and use it in creation or remove the depends on part.

Related

Serverless deploy fails - Error: Write EPIPE

I have used Serverless Framework previously but lately I am having an error I cannot resolve. Serverless Framework version 3.8.0. was installed using npm. Node version 17.7.1. Running on Linux(Manjaro)
Running sls deploy
āœ” Service deployed to stack test-api-dev (0s)
But nothing actually changes on AWS, nothing is actually deployed.
Running sls deploy | echo outputs
Running "serverless" from node_modules
āœ– Uncaught exception
/home/mascs/api/node_modules/serverless/scripts/serverless.js:49
throw error;
^
Error: write EPIPE
at afterWriteDispatched (node:internal/stream_base_commons:160:15)
at writeGeneric (node:internal/stream_base_commons:151:3)
at Socket._writeGeneric (node:net:817:11)
at Socket._write (node:net:829:8)
at writeOrBuffer (node:internal/streams/writable:390:12)
at _write (node:internal/streams/writable:331:10)
at Socket.Writable.write (node:internal/streams/writable:335:10)
at Object.<anonymous> (/home/mascs/node_modules/#serverless/utils/log-reporters/node.js:50:39)
at Object.emit (/home/mascs/api/node_modules/event-emitter/index.js:97:9)
at /home/mascs/api/node_modules/#serverless/utils/lib/log/get-output-reporter.js:17:27
at module.exports (/home/mascs/api/node_modules/serverless/lib/cli/handle-error.js:81:3)
at finalize (/home/mascs/api/node_modules/serverless/scripts/serverless.js:57:41)
at /home/mascs/api/node_modules/serverless/scripts/serverless.js:772:11
Emitted 'error' event on Socket instance at:
at emitErrorNT (node:internal/streams/destroy:164:8)
at emitErrorCloseNT (node:internal/streams/destroy:129:3)
at processTicksAndRejections (node:internal/process/task_queues:83:21) {
errno: -32,
code: 'EPIPE',
syscall: 'write'
}
Node.js v17.7.1
Running as sudo doesn't change things, so it doesn't seem to be a permissions issue.
Here is the serverless.yml I am using. Can't see any syntax errors.
service: test-api
frameworkVersion: ">=3.0.0 <4.0.0"
provider:
name: aws
runtime: python3.7
region: eu-west-1
stage: dev
environment:
DYN_TABLE: ${self:provider.stage}-test
iamRoleStatements:
- Effect: Allow
Action:
- dynamodb:Query
- dynamodb:Scan
- dynamodb:GetItem
- dynamodb:BatchGetItem
- dynamodb:UpdateItem
- dynamodb:PutItem
Resource: "arn:aws:dynamodb:${opt:region, self:provider.region}:*:table/${self:provider.environment.DYN_TABLE}"
plugins:
- serverless-python-requirements
- serverless-ignore
package:
individually: false
custom:
pythonRequirements:
dockerizePip: true
slim: true
functions:
lambda-test:
handler: api/main.get
memorySize: 256
timeout: 6
events:
- http:
path: v1/
method: get
cors: true
I am at a bit of a loss here, would appreciate if anyone has any ideas.
Have tried uninstalling and then reinstalling Serverless Framework also but it has made no difference.

How to run a Lambda Docker with serverless offline

I would like to run serverless offline using a Lambda function that points to a Docker image.
When I try to run serverless offline, I am just receiving:
Offline [http for lambda] listening on http://localhost:3002
Function names exposed for local invocation by aws-sdk:
* hello-function: sample-app3-dev-hello-function
If I try to access http://localhost:3002/hello, a 404 error is returned
serverless.yml
service: sample-app3
frameworkVersion: '3'
plugins:
- serverless-offline
provider:
name: aws
ecr:
images:
sampleapp3image:
path: ./app/
platform: linux/amd64
functions:
hello-function:
image:
name: sampleapp3image
events:
- httpApi:
path: /hello
method: GET
app/myfunction.py
def lambda_handler(event, context):
return {
'statusCode': 200,
'body': 'Hello World!'
}
app/Dockerfile
FROM public.ecr.aws/lambda/python:3.9
COPY myfunction.py ./
CMD ["myfunction.lambda_handler"]
at the moment such functionality is not supported in serverless-offline plugin. There's an issue open where the discussion started around supporting this use case: https://github.com/dherault/serverless-offline/issues/1324

Serverless: TypeError: Cannot read property 'stage' of undefined

frameworkVersion: '2'
plugins:
- serverless-step-functions
- serverless-python-requirements
- serverless-parameters
- serverless-pseudo-parameters
provider:
name: aws
region: us-east-2
stage: ${opt:stage, 'dev'}
runtime: python3.7
versionFunctions: false
iam:
role: arn:aws:iam::#{AWS::AccountId}:role/AWSLambdaVPCAccessExecutionRole
apiGateway:
shouldStartNameWithService: true
lambdaHashingVersion: 20201221
package:
exclude:
- node_modules/**
- venv/**
# Lambda functions
functions:
generateAlert:
handler: handler.generateAlert
generateData:
handler: handler.generateDataHandler
timeout: 600
approveDenied:
handler: handler.approveDenied
timeout: 600
stepFunctions:
stateMachines:
"claims-etl-and-insight-generation-${self:provider.stage}":
loggingConfig:
level: ALL
includeExecutionData: true
destinations:
- Fn::GetAtt: ["ETLStepFunctionLogGroup", Arn]
name: "claims-etl-and-insight-generation-${self:provider.stage}"
definition:
Comment: "${self:provider.stage} ETL Workflow"
StartAt: RawQualityJob
States:
# Raw Data Quality Check Job Start
RawQualityJob:
Type: Task
Resource: arn:aws:states:::glue:startJobRun.sync
Parameters:
JobName: "data_quality_v2_${self:provider.stage}"
Arguments:
"--workflow-name": "${self:provider.stage}-Workflow"
"--dataset_id.$": "$.datasetId"
"--client_id.$": "$.clientId"
Next: DataQualityChoice
Retry:
- ErrorEquals: [States.ALL]
MaxAttempts: 2
IntervalSeconds: 10
BackoffRate: 5
Catch:
- ErrorEquals: [States.ALL]
Next: GenerateErrorAlertDataQuality
# End Raw Data Quality Check Job
DataQualityChoice:
Type: Task
Resource:
Fn::GetAtt: [approveDenied, Arn]
Next: Is Approved ?
Is Approved ?:
Type: Choice
Choices:
- Variable: "$.quality_status"
StringEquals: "Denied"
Next: FailState
Default: HeaderLineJob
FailState:
Type: Fail
Cause: "Denied status"
# Header Line Job Start
HeaderLineJob:
Type: Parallel
Branches:
- StartAt: HeaderLineIngestion
States:
HeaderLineIngestion:
Type: Task
Resource: arn:aws:states:::glue:startJobRun.sync
Parameters:
JobName: headers_lines_etl_rs_v2
Arguments:
"--workflow-name.$": "$.Arguments.--workflow-name"
"--dataset_id.$": "$.Arguments.--dataset_id"
"--client_id.$": "$.Arguments.--client_id"
End: True
Retry:
- ErrorEquals: [States.ALL]
MaxAttempts: 2
IntervalSeconds: 10
BackoffRate: 5
Catch:
- ErrorEquals: [States.ALL]
Next: GenerateErrorAlertHeaderLine
End: True
# Header Line Job End
GenerateErrorAlertDataQuality:
Type: Task
Resource:
Fn::GetAtt: [generateAlert, Arn]
End: true
resources:
Resources:
# Cloudwatch Log
"ETLStepFunctionLogGroup":
Type: AWS::Logs::LogGroup
Properties:
LogGroupName: "ETLStepFunctionLogGroup_${self:provider.stage}"
This is what my serverless.yml file looks like.
When I run the command:
sls deploy --stage staging
It show
Type Error ----------------------------------------------
TypeError: Cannot read property 'stage' of undefined
at Variables.getValueFromOptions (/snapshot/serverless/lib/classes/Variables.js:648:37)
at Variables.getValueFromSource (/snapshot/serverless/lib/classes/Variables.js:579:17)
at /snapshot/serverless/lib/classes/Variables.js:539:12
Your Environment Information ---------------------------
Operating System: linux
Node Version: 14.4.0
Framework Version: 2.30.3 (standalone)
Plugin Version: 4.5.1
SDK Version: 4.2.0
Components Version: 3.7.4
How I can fix this? I tried with different version of serverless.
There is error in yamlParser file, which is provided by serverless-step-functions.
Above is my serverless config file.
It looks like a $ sign is missing from your provider -> stage?
provider:
name: aws
region: us-east-2
stage: ${opt:stage, 'dev'} # $ sign is missing?
runtime: python3.7
versionFunctions: false
iam:
role: arn:aws:iam::#{AWS::AccountId}:role/AWSLambdaVPCAccessExecutionRole
apiGateway:
shouldStartNameWithService: true
lambdaHashingVersion: 20201221

Deploying cube.js using serverless framework results in an error

I am trying to deploy cube.js project using serverless framework on aws and when I access the endpoint produced by serverless, it results in the following error on the browser
Cannot GET /
Here is my serverless.yml file
service: cloud-analytics
provider:
name: aws
stage: production
runtime: nodejs8.10
iamRoleStatements:
- Effect: "Allow"
Action:
- "sns:*"
- "athena:*"
- "s3:*"
- "glue:*"
Resource:
- "*"
vpc:
securityGroupIds:
- sg-xxxxxxxxx # Your DB and Redis security groups here
subnetIds:
- subnet-xxxxxxxxx
environment:
CUBEJS_AWS_KEY: ${opt:awsKey}
CUBEJS_AWS_SECRET: ${opt:awsSecret}
CUBEJS_AWS_REGION: us-east-1
CUBEJS_AWS_S3_OUTPUT_LOCATION: ${opt:location}
REDIS_URL: ${opt:redis_url_with_port}
CUBEJS_DB_TYPE: athena
CUBEJS_API_SECRET:XXXXXX
CUBEJS_APP: "${self:service.name}-${self:provider.stage}"
NODE_ENV: ${self:provider.stage}
AWS_ACCOUNT_ID:
Fn::Join:
- ""
- - Ref: "AWS::AccountId"
functions:
cubejs:
handler: cube.api
timeout: 30
events:
- http:
path: /
method: GET
- http:
path: /{proxy+}
method: ANY
cubejsProcess:
handler: cube.process
timeout: 630
events:
- sns: "${self:service.name}-${self:provider.stage}-process"
plugins:
- serverless-express
I have followed this steps in this blog to set up NAT https://medium.com/#philippholly/aws-lambda-enable-outgoing-internet-access-within-vpc-8dd250e11e12
Cube.js file is as follows with server core options
const AWSHandlers = require('#cubejs-backend/serverless-aws');
const AthenaDriver = require('#cubejs-backend/athena-driver');
module.exports = new AWSHandlers({
externalDbType: 'athena',
externalDriverFactory: () => new AthenaDriver({
accessKeyId: process.env.CUBEJS_AWS_KEY,
secretAccessKey: process.env.CUBEJS_AWS_SECRET,
region: process.env.CUBEJS_AWS_REGION,
S3OutputLocation: process.env.CUBEJS_AWS_S3_OUTPUT_LOCATION
})
});
When I run the endpoint
https://xxxxx.execute-api.us-east-1.amazonaws.com/production/
which is produced by the serverless api gateway I get the error
Cannot GET /
On Cloudwatch I see the cubejs lambda being invoked and see logs for start and end request id. I dont see any logs on cubejsProcess lambda.
Where/How can I debug this to see where the issue is?
By default in production mode Cube.js disables dev server capability and it's why you don't see any Playground working at / path: https://cube.dev/docs/deployment#production-mode. Please use REST API to test your deployment: https://cube.dev/docs/rest-api.

Serverless Enterprise deployment

I recently updated to v1.44.0 and used the #serverless/enterprise-plugin and am now unable to deploy. Iā€™m simply trying to create a User Pool, but keep getting an error.
An error occurred: EnterpriseLogAccessIamRole - Policy statement must contain resources. (Service: AmazonIdentityManagement; Status Code: 400; Error Code: MalformedPolicyDocument; Request ID: dc158686-378c-4d01-97fb-1414d55a735d)
serverless.yml
tenant: [omitted]
app: [omitted]
service: auth
frameworkVersion: ">=1.44.0"
plugins:
- '#serverless/enterprise-plugin'
provider:
name: aws
runtime: nodejs8.10
region: us-east-1
custom:
stage: ${opt:stage, self:provider.stage}
cognito:
app:
userPool: ${self:service}-app-user-pool-${self:custom.stage}
identityPool: AppIdentityPoolDev
resources:
Resources:
AppUserPool:
Type: AWS::Cognito::UserPool
Properties:
UserPoolName: ${self:custom.cognito.app.userPool}
UsernameAttributes:
- email
AutoVerifiedAttributes:
- email
MobileAppClient:
Type: AWS::Cognito::UserPoolClient
Properties:
ClientName: ${self:service}-mobile-app-client-${self:custom.stage}
UserPoolId:
Ref: AppUserPool
GenerateSecret: true
Outputs:
AppUserPool:
Value:
Ref: AppUserPool
MobileAppClient:
Value:
Ref: MobileAppClient