recursive lambda function never seems to run - amazon-s3

Question

I'm not familiar enough with node.js or lambda to see an obvious solution to a dilemma I have. I'm writing some utilities on lambda to manipulate images in an S3 bucket and make them accessible via the GatewayAPI to rest calls.
BACKGROUND DETAILS:
One of the utilities I have retrieves the headObject information such as the mtime, size and metadata. The images themselves will likely be coming in from various means and I won't always have control over adding metadata to them when they arrive/are-created. But I don't really need it until it's necessary to view details about the image from a web interface. And when I do that, I use a thumbnail instead so I created a lambda create-event triggered script (and also have a fall back variation of it via the gatewayAPI) that will create a thumbnail (either when the image is first uploaded to S3 or whenever I make the gateway CreateThumbbnail call) at which time it adds metadata to the thumbnail for the image with things like the original image mimetype, pixel width and height.
What I would like to be able to do, is to create a 'GetObjectInfo' that firsts pulls the headObject data, then checks to see if the bucket specified is or is not the bucket with the associated thumbnail files. (e.g. if it is or is not a thumbnail object) If it is 'not' a thumbnail, I want to then go retrieve -- or at least attempt to retrieve -- the headObject for the associated thumbnail file and attach the thumbnail file's metadata (if the thumbnail exists) onto the data from the original head request before returning the information.
The problem is, that when I set up an async callback scheme, the first headObject request completes, the second never seems to get out of the starting gate.
The method in my class is:
getHeadObject(bucket,object,callback) {
console.log(bucket, "CLASS-head#1")
this.s3.headObject({"Bucket":bucket,"Key":object}, function(err,data){
console.log(bucket, "CLASS-head#2")
callback(err,data)
})
}
getObjectInfo(bucket,object,callback) {
let scope = this
console.log(bucket,"CLASS-object#1")
this.getHeadObject(bucket,object,function(err,data) {
console.log(bucket,"CLASS-object#2")
if(err)
callback(err,data)
else
callback(null,data)
})
}
The lambda code that calls it recursively is:
var cInst = new myClass()
cInst.getObjectInfo(srcBucket,filePath,function(err,data) {
if(data.status == 1) { // if parent request success
// if parent is not thumbnail
if(srcBucket != THUMB_BUCKET) { // see if a thumbnail exists
let thumbPath = myClass.getThumbPath(srcBucket,userId,directory,targetObject)
console.log('---- thumbPath', thumbPath)
cInst.getObjectInfo(THUMB_BUCKET,thumbPath, function(err,thumbData) {
console.log("thumbData #1",thumbData)
if(thumbData.status == 1) { // thumbnail exists
console.log("thumbData")
}
})
}
context.succeed(myClass.createResponse(1, data, api))
} else {
context.fail(myClass.createResponse(data.status, data, api))
}
})
First call on the parent is see
{bucket} "CLASS-object#1"
{bucket} "CLASS-head#1"
{bucket} "CLASS-head#2"
{bucket} "CLASS-object#2"
on the second I only see:
image-thumbnails "CLASS-object#1"
image-thumbnails "CLASS-head#1"
(getThumbPath is just a static utility function that builds the thumbnail path based on the parameters related to the original file. It is already tested as working and produces something like {original-bucket-name}/{userid}/{subdirectory}/{file-basename_150x150.jpg} for any given image - I confirmed that in this instance, the thumbnail exists and matches the path returned by getThumbPath and the acl appears to have permission to read the bucket and the object)
UPDATE: More weirdness
I tried setting the permissions to publicly readable on the thumbnail and it worked. So I started messing with the acl. For the time being since I am still testing, I just gave the role for the scripts full S3 permissions.
But I noticed now that it's working and not working intermittently. One time it completes, the next time it doesn't. WTF is going on here?

Answer 1

I would bet that this is the most common problem that people see when using Node.js with Lambda.
When a Node.js Lambda reaches the end of the main thread, it ends all other threads. When it reaches the end of the handler, it stops all concurrent promises or async calls that are running.
To make sure that the lambda does not prematurely terminate those threads, wait until those promises are complete by using await.
In your case, the following will work: wrap any async calls in a promise and then await them.
await new Promise(async (resolve, reject) => {
cInst.getObjectInfo(srcBucket,filePath,function(err,data) {
if(data.status == 1) {
if(srcBucket != THUMB_BUCKET) {
...
...
await new Promise((resolve2, reject2) => {
cInst.getObjectInfo(THUMB_BUCKET,thumbPath, function(err,thumbData) {
...
...
resolve2();
})
})
}
context.succeed(myClass.createResponse(1, data, api))
resolve();
} else {
context.fail(myClass.createResponse(data.status, data, api))
reject();
}
})
})