Axios get request working locally but timing out on serverless - express

I'm trying to scrape some websites, but for some reason it works locally (localhost) with express, but not when I've deployed it to lambda. Tried w/ the ff serverless-http and aws-serverless-express and serverless-express plugin. Also tried switching between axios and superagent.
Routes work fine, and after hrs of investigating, I've narrowed the problem down to the fetch/axios bit. When i don't add a timeout to axios/superagent/etc, the app just keeps running and timing out at 15/30 sec, whichever is set and get an error 50*.
service: scrape
provider:
name: aws
runtime: nodejs10.x
stage: dev
region: us-east-2
memorySize: 128
timeout: 15
plugins:
- serverless-plugin-typescript
- serverless-express
functions:
app:
handler: src/server.handler
events:
- http:
path: /
method: ANY
cors: true
- http:
path: /{proxy+}
method: ANY
cors: true
protected async fetchHtml(uri: string): Promise<CheerioStatic | null> {
const htmlElement = await Axios.get(uri, { timeout: 5000 });
if(htmlElement.status === 200) {
const $ = Cheerio.load(htmlElement && htmlElement.data || '');
$('script').remove();
return $;
}
return null;
}

As far as i know, the default timeout of axios is indefinite. Remember, API gateway has hard limit of 29 sec timeout.
I had the same issue recently, sometimes the timeouts are due to cold starts. So I basically had to add a retry logic for the api call in my frontend react application.

Related

Nuxt.js on Google App Engine Standard Environment - 500 errors

I am serving Nuxt.js SSR application from Google App Engine, from F2 instance. All is deployed ok and works well, but sometimes I hit a weird 500 server error. I think the problem is because GAE shuts down my instance because it is rarely used. When the instance is created again, my website fails to load (timeouts), resulting in 500 error. I am using serverMiddleware with Express.
/server/middleware.js
const express = require('express')
const app = express()
// this fetches a CMS token and creates a CMS object used in the request below
const CMS = require('../cms')
app.use(express.json())
app.use(express.urlencoded({ extended: true }))
// get Page from the CMS
app.post('/page', async (req, res, next) => {
try {
let result = await CMS.getPage(req.body.route)
res.status(200).json(result)
} catch (error) {
console.log('Error', error)
}
})
app.yaml
runtime: nodejs12
instance_class: F2
handlers:
- url: /_nuxt
static_dir: .nuxt/dist/client
secure: always
- url: /(.*\.(gif|png|jpg|ico|txt))$
static_files: static/\1
upload: static/.*\.(gif|png|jpg|ico|txt)$
secure: always
- url: /.*
secure: always
redirect_http_response_code: 301
script: auto
env_variables:
HOST: "0.0.0.0"
NODE_ENV: "production"
automatic_scaling:
min_idle_instances: 1
max_idle_instances: 1
min_instances: 1
max_instances: 1
target_cpu_utilization: 0.9
target_throughput_utilization: 0.9
max_concurrent_requests: 50
And here is the relevant log screenshot:
0: {
logMessage: "Request was aborted after waiting too long to attempt to service your request."
severity: "ERROR"
time: "2022-01-14T08:55:22.057499Z"
}
Am I doing something wrong here? I would not mind keeping the costs down and scaling back to 0 instances when they are not needed, but the website would still need to be served as fast as possible.

Infinispan java.lang.SecurityException: ISPN006017: Unauthorized 'PUT' operation

I am trying to put a value in Infinispan cache using Hotrod nodeJS client. The code runs fine if the server is installed locally. However, when I run the same code with Infinispan server hosted on docker container I get the following error
java.lang.SecurityException: ISPN006017: Unauthorized 'PUT' operation
try {
client = await infinispan.client({
port: 11222,
host: '127.0.0.1'
}, {
cacheName: 'testcache'
});
console.log(`Connected to cache`);
await client.put('test', 'hello 1');
await client.disconnect();
} catch (e) {
console.log(e);
await client.disconnect();
}
I have tried setting CORS Allow all option on the server as well
Need to provide custom config.yaml to docker with following configurations
endpoints:
hotrod:
auth: false
enabled: false
qop: auth
serverName: infinispan
Unfortunately the nodejs client doesn't support authentication yet. The issue to implement this is https://issues.redhat.com/projects/HRJS/issues/HRJS-36

"Execution failed" when setting up API Gateway and Fargate with AWS CDK

I am trying to setup AWS API Gateway to access a fargate container in a private VPC as described here. For this I am using AWS CDK as described below. But when I curl the endpoint after successful cdk deploy I get "Internal Server Error" as a response. I can't find any additional information. For some reason API GW can't reach the container.
So when I curl the endpoint like this:
curl - i https://xxx.execute-api.eu-central-1.amazonaws.com/prod/MyResource
... I get the following log output in cloud watch:
Extended Request Id: NpuEPFWHliAFm_w=
Verifying Usage Plan for request: 757c6b9e-c4af-4dab-a5b1-542b15a1ba21. API Key: API Stage: ...
PI Key authorized because method 'ANY /MyResource/{proxy+}' does not require API Key. Request will not contribute to throttle or quota limits
Usage Plan check succeeded for API Key and API Stage ...
Starting execution for request: 757c6b9e-c4af-4dab-a5b1-542b15a1ba21
HTTP Method: GET, Resource Path: /MyResource/test
Execution failed due to configuration error: There was an internal error while executing your request
CDK Code
First I create a network load balanced fargate service:
private setupService(): NetworkLoadBalancedFargateService {
const vpc = new Vpc(this, 'MyVpc');
const cluster = new Cluster(this, 'MyCluster', {
vpc: vpc,
});
cluster.connections.allowFromAnyIpv4(Port.tcp(5050));
const taskDefinition = new FargateTaskDefinition(this, 'MyTaskDefinition');
const container = taskDefinition.addContainer('MyContainer', {
image: ContainerImage.fromRegistry('vad1mo/hello-world-rest'),
});
container.addPortMappings({
containerPort: 5050,
hostPort: 5050,
});
const service = new NetworkLoadBalancedFargateService(this, 'MyFargateServie', {
cluster,
taskDefinition,
assignPublicIp: true,
});
service.service.connections.allowFromAnyIpv4(Port.tcp(5050));
return service;
}
Next I create the VpcLink and the API Gateway:
private setupApiGw(service: NetworkLoadBalancedFargateService) {
const api = new RestApi(this, `MyApi`, {
restApiName: `MyApi`,
deployOptions: {
loggingLevel: MethodLoggingLevel.INFO,
},
});
// setup api resource which forwards to container
const resource = api.root.addResource('MyResource');
resource.addProxy({
anyMethod: true,
defaultIntegration: new HttpIntegration('http://localhost.com:5050', {
httpMethod: 'ANY',
options: {
connectionType: ConnectionType.VPC_LINK,
vpcLink: new VpcLink(this, 'MyVpcLink', {
targets: [service.loadBalancer],
vpcLinkName: 'MyVpcLink',
}),
},
proxy: true,
}),
defaultMethodOptions: {
authorizationType: AuthorizationType.NONE,
},
});
resource.addMethod('ANY');
this.addCorsOptions(resource);
}
Anyone has a clue what is wrong with this config?
After hours of trying I finally figured out that the security groups do not seem to be updated correctly when setting up the VpcLink with CDK. Broadening the allowed connection with
service.service.connections.allowFromAnyIpv4(Port.allTraffic())
solved it. Still need to figure out which minimum set needs to be set instead of allTrafic()
Additionally I replaced localhost in the HttpIntegration by the endpoint of the load balancer like this:
resource.addMethod("ANY", new HttpIntegration(
'http://' + service.loadBalancer.loadBalancerDnsName,
{
httpMethod: 'ANY',
options: {
connectionType: ConnectionType.VPC_LINK,
vpcLink: new VpcLink(this, 'MyVpcLink', {
targets: [service.loadBalancer],
vpcLinkName: 'MyVpcLink',
})
},
}
))

Frequent timeout with app using Serverless Framework (AWS Lambda/Gateway), Express, Mongoose/MongoDB Atlas

Trigger warning : Beginner question.
I built an api using Express and Mongoose with a MongoDB Atlas DB.
Most of the time, it works normally, but often I get timeout errors. This seems to happen very randomly and concerns all routes, etc... Precisely, I get :
`502 Internal server error via POSTMAN`
and in the Serverless Dashboard, I get :
invocation
time invoked 1 day ago, mar 08 at 1:38pm
fatal error Function execution duration going to exceeded configured timeout limit.
cold start
duration 48.9 s
memory used na
request
endpoint /{proxy+}
method POST
status 502
message Internal server error
latency 27 ms
and span & log :
I used this tutorial to wrap my express app to deploy it with serverless framework : https://dev.to/adnanrahic/a-crash-course-on-serverless-apis-with-express-and-mongodb-193k
Serverless.yml file :
service: serviceName
app: appName
org: orgName
provider:
name: aws
runtime: nodejs12.x
stage: ${env:NODE_ENV}
region: eu-central-1
environment:
NODE_ENV: ${env:NODE_ENV}
DB: ${env:DB}
functions:
app:
handler: server.run
events:
- http:
path: /
method: ANY
cors: true
- http:
path: /{proxy+}
method: ANY
cors: true
plugins:
- serverless-offline # Utiliser pour tester localement
- serverless-dotenv-plugin
server.js file :
const sls = require('serverless-http')
const app = require('./app')
module.exports.run = sls(app)
app.js file :
const express = require('express')
const cors = require('cors')
const bodyParser = require('body-parser')
const newRoutes = require('./routes/file')
const app = express()
app.use(bodyParser.json())
const helmet = require('helmet')
app.use(helmet())
app.options('*', cors())
app.use(cors({ allowedHeaders: 'Content-Type, Authorization' }))
app.use('/new-route', newRoutes)
app.use((error, req, res, next) => {
console.log(error)
const status = error.status || 500
const message = error.message
res.status(status).json({
status: status,
message: message
})
})
// Gère la connexion à la base de donnée :
require('./db')
module.exports = app
and finally db.js file :
const mongoose = require('mongoose')
mongoose
.connect(
process.env.DB, {
useNewUrlParser: true,
useUnifiedTopology: true
})
.then(() => {
console.log('connected')
})
.catch(err => console.log(err))
From what I have read, it is related to cold start in Lambda and the way API Gateway handles timeouts (!?). I have read this on mongoose documentation (https://mongoosejs.com/docs/lambda.html), and read also other tutorials, but I don't how exaclty I should adapt it to my situation.
Thank you for your help
Under your provider add timeout, maximum value of timeout in lambda is
900 seconds, place it according to your execution time like 30 seconds
and see what happens
provider:
timeout: 30
The error is clearly saying that it's execution exceeded timeout, since you have not configured timeout so it was using default timeout of 3 seconds, hopefully it will solve the issue
The issue is likely due to your open database connection. While this connection is established any calls to callback won't be returned to the client and your function will timeout.
You need to set context.callbackWaitsForEmptyEventLoop to false.
Here is the explanation from the docs:
callbackWaitsForEmptyEventLoop – Set to false to send the response right away when the callback executes, instead of waiting for the Node.js event loop to be empty. If this is false, any outstanding events continue to run during the next invocation.
With serverless-http you can set this option quite easily within your server.js file:
const sls = require('serverless-http')
const app = require('./app')
module.exports.run = sls(app, { callbackWaitsForEmptyEventLoop: false })

Serverless function with authorizer arn provided returns 401

I am using serverless.
When I setup one of my functions as the following, which includes authorizer, on the client, I receive 401.
However when I remove it, there are no problems.
provider:
name: aws
runtime: nodejs8.10
region: eu-west-1
environment:
USER_POOL_ARN: "arn:aws:cognito-idp:eu-west-1:974280.....:userpool/eu-west-1........"
functions:
create:
handler: handlers/create.main
events:
- http:
path: create
method: post
cors: true
authorizer:
type: COGNITO_USER_POOLS
name: serviceBAuthFunc
arn: ${self:provider.environment.USER_POOL_ARN}
On the client, I expect a logged in user of the same user pool could get expected response. However it returns 401.
Any help is appreciated. Thanks.
After desperate hours spent, I have come up with the solution.
For anyone who comes across the same issue, here is a solution that worked for me.
Add integration: lambda after cors: true (though the order doesn't matter).
Below is just demonstrating that.
functions:
create:
handler: handlers/create.main
events:
- http:
path: create
method: post
cors: true
integration: lambda // this solves the problem
authorizer:
type: COGNITO_USER_POOLS
name: serviceBAuthFunc
arn: ${self:provider.environment.USER_POOL_ARN}
Send Authorization header with the value of Auth.currentSession().idToken.jwtToken while making the request.
Below is an example for sending headers using API of #aws-amplify/api and Auth of #aws-amplify/auth.
const currentSession = await Auth.currentSession()
await API.post(
'your-endpoint-name',
"/your-endpoint-path/..",
{
headers: {
'Authorization': currentSession.idToken.jwtToken
}
}
)