Cloud function stop executing when Batch object is being used - google-bigquery

Question

I'm using Google Cloud Function to execute a query on bigQuery and store the result if firestore.
My problem is that as soon as I try to use the firestore batch object, the cloud function stop executing.
Using dichotomy, I think it's when I inclue the batch object code that the function suddenly stop working.
I've tried to increase the memory of the function to 1GB without luck. (currently it's using 128mb)
const {BigQuery} = require('#google-cloud/bigquery');
const {Firestore} = require('#google-cloud/firestore');
const bigquery = new BigQuery ();
const firestore = new Firestore ();
const fsCollectionName = 'ul_queteur_stats_per_year';
const queryStr = "the bigquery query";
function handleError(err){
//skipped
}
/**
* Triggered from a message on a Cloud Pub/Sub topic.
*
* #param {!Object} event Event payload.
* #param {!Object} context Metadata for the event.
*/
exports.ULQueteurStatsPerYear = (event, context) => {
const pubsubMessage = event.data;
const parsedObject = JSON.parse(Buffer.from(pubsubMessage, 'base64').toString());
console.log("Recieved Message : "+JSON.stringify(parsedObject));
//{ ul_id:parsedObject.ul_id }
const queryObj = {
query: queryStr,
params: {
ul_id: parsedObject.ul_id
}
};
bigquery
.query(queryObj)
.then((data) => {
console.log("Query Successful, # rows : "+data.length+" data[0].length:"+data[0].length);
//rows : [{"amount":367.63,"weight":2399.3,"time_spent_in_minutes":420}]
const rows = data[0];
console.log("Query Successful");
const batch = firestore.batch();
console.log("Batch Created ");
console.log("Getting Collection");
const collection = firestore.collection(fsCollectionName);
console.log("Getting Collection '"+fsCollectionName+"' retrieved");
//#####################################
for(let i=0;i<rows.length;i++)
{
console.log("getting a new DocId");
const docRef = collection.doc();
console.log("Adding to docRef='"+docRef.id+"' : "+JSON.stringify(rows[i]));
batch.set(docRef, rows[i]);
console.log("Added to batch");
}
console.log("Commiting batch insert");
batch.commit().then(() => {
console.log('Successfully executed batch');
});
//#####################################
})
.catch(err => {
handleError(err);
});
};
Expected:
data inserted in Firestore
Actual result :
If I remove the code between the
//#####################################
Then I get each log in stackdriver.
(The first one saying there's 420 rows)
If I let the code between
//#####################################
(or just the batch.commit() part, or just the for loop part)
I only get the first log, and then nothing.
Query Successful, # rows : 1 data[0].length:420
Even if I put the whole code in a try/catch block with a console.log of the exception, I see no error in stack driver.
Solution
the solution is to return the bigquery promise.
So the above code should be changed to :
return bigquery
.query(queryObj)
.then(...);
Thanks Doug for the help !

Answer 1

You need to return a promise that resolves when all the asynchronous work is complete. Right now, you're returning nothing, which means the function will terminate and shut down almost immediately, before your query is done.
You'll need to pay attention to all the promises that your code is using, including the query, and all the batch commits. You can't ignore any promise returned by any API, else the work will be terminated before it's done.