Frequent timeout with app using Serverless Framework (AWS Lambda/Gateway), Express, Mongoose/MongoDB Atlas - express

Question

Trigger warning : Beginner question.
I built an api using Express and Mongoose with a MongoDB Atlas DB.
Most of the time, it works normally, but often I get timeout errors. This seems to happen very randomly and concerns all routes, etc... Precisely, I get :
`502 Internal server error via POSTMAN`
and in the Serverless Dashboard, I get :
invocation
time invoked 1 day ago, mar 08 at 1:38pm
fatal error Function execution duration going to exceeded configured timeout limit.
cold start
duration 48.9 s
memory used na
request
endpoint /{proxy+}
method POST
status 502
message Internal server error
latency 27 ms
and span & log :
I used this tutorial to wrap my express app to deploy it with serverless framework : https://dev.to/adnanrahic/a-crash-course-on-serverless-apis-with-express-and-mongodb-193k
Serverless.yml file :
service: serviceName
app: appName
org: orgName
provider:
name: aws
runtime: nodejs12.x
stage: ${env:NODE_ENV}
region: eu-central-1
environment:
NODE_ENV: ${env:NODE_ENV}
DB: ${env:DB}
functions:
app:
handler: server.run
events:
- http:
path: /
method: ANY
cors: true
- http:
path: /{proxy+}
method: ANY
cors: true
plugins:
- serverless-offline # Utiliser pour tester localement
- serverless-dotenv-plugin
server.js file :
const sls = require('serverless-http')
const app = require('./app')
module.exports.run = sls(app)
app.js file :
const express = require('express')
const cors = require('cors')
const bodyParser = require('body-parser')
const newRoutes = require('./routes/file')
const app = express()
app.use(bodyParser.json())
const helmet = require('helmet')
app.use(helmet())
app.options('*', cors())
app.use(cors({ allowedHeaders: 'Content-Type, Authorization' }))
app.use('/new-route', newRoutes)
app.use((error, req, res, next) => {
console.log(error)
const status = error.status || 500
const message = error.message
res.status(status).json({
status: status,
message: message
})
})
// Gère la connexion à la base de donnée :
require('./db')
module.exports = app
and finally db.js file :
const mongoose = require('mongoose')
mongoose
.connect(
process.env.DB, {
useNewUrlParser: true,
useUnifiedTopology: true
})
.then(() => {
console.log('connected')
})
.catch(err => console.log(err))
From what I have read, it is related to cold start in Lambda and the way API Gateway handles timeouts (!?). I have read this on mongoose documentation (https://mongoosejs.com/docs/lambda.html), and read also other tutorials, but I don't how exaclty I should adapt it to my situation.
Thank you for your help

Answer 1

Under your provider add timeout, maximum value of timeout in lambda is
900 seconds, place it according to your execution time like 30 seconds
and see what happens
provider:
timeout: 30
The error is clearly saying that it's execution exceeded timeout, since you have not configured timeout so it was using default timeout of 3 seconds, hopefully it will solve the issue

Answer 2

The issue is likely due to your open database connection. While this connection is established any calls to callback won't be returned to the client and your function will timeout.
You need to set context.callbackWaitsForEmptyEventLoop to false.
Here is the explanation from the docs:
callbackWaitsForEmptyEventLoop – Set to false to send the response right away when the callback executes, instead of waiting for the Node.js event loop to be empty. If this is false, any outstanding events continue to run during the next invocation.
With serverless-http you can set this option quite easily within your server.js file:
const sls = require('serverless-http')
const app = require('./app')
module.exports.run = sls(app, { callbackWaitsForEmptyEventLoop: false })