Sending response in async function - express

I need to return an array of labels, but I can only return 1 of the labels so far. The error which I get is "Cannot set headers after they are sent to the client". So I tried res.write and placed res.end after my for loop then I get the obvious error of doing a res.end before a res.write. How do I solve this?
for(let i=0;i<arr.length;i++){
request.get(arr[i], function (error, response, body) {
if (!error && response.statusCode == 200) {
myfunction();
async function myfunction(){
const Labels = await Somefunctioncallwhoseresponseigetlater(body)
res.send(Labels);
}
}
});}
New code-
async function getDataSendResponse(res) {
let allLabels = [];
for (let url of arr) {
let body = await got(url).buffer();
var imgbuff= Buffer.from(body,'base64')
const imageLabels = await rekognition.detectLabels(imgbuff);
allLabels.push(...imageLabels);
}
res.send(allLabels);
}
The error I have with this code is
"Resolver: AsyncResolver
TypeError: Cannot destructure property Resolver of 'undefined' or 'null'."

You are trying to call res.send() inside a for loop. That means you'll be trying to call it more than once. You can't do that. You get to send one response for any given http request and res.send() sends an entire response. So, when you try to call it again inside the loop, you can the warning you see.
If what you're trying to do is to send an array of labels, then you need to accumulate the array of labels first and then make one call to res.send() to send the final array.
You don't show the whole calling context here, but making the following assumptions:
Somefunctioncallwhoseresponseigetlater() returns a promise that resolves when it is done
You want to accumulate all the labels you collected in your loop
Your Labels variable is an array
Your http request returns a text response. If it returns something else like JSON, then .text() would need to be changed to .json().
then you can do it like this:
const got = require('got');
async function getDataSendResponse(res) {
let allLabels = [];
for (let url of arr) {
let body = await got(url).buffer();
const labels = await Somefunctioncallwhoseresponseigetlater(body);
allLabels.push(...labels);
}
res.send(allLabels);
}
Note, I'm using the got() library instead of the deprecated request() library both because request() is not deprecated and because this type of code is way easier when you have an http library that supports promises (like got() does).

Related

_.forEach out of order

I have some code within an async function:
let listOfResults = [];
_.forEach(listOfProducts, async function (product) {
const result = await calc.calculateProductScore(product);
const response = await mutateDB.sendProductScoreData(result.productId, result);
await listOfResults.push(response);
})
if (listOfResults.length >= 1) {
return "yay";
} else {
return "boo";
}
Given that there's three items in the listOfProducts array, stepping through the debugger in VS Code when this executes this happens in the following order:
the line const result = await calc.calculateProductScore(product); is executed three times
the if statement (which returns boo as nothing has been pushed onto the listOfResults array)
the line const response = await mutateDB.sendProductScoreData(result.productId, result); is executed three times
the line await listOfResults.push(response); is executed three times
I can see in lodash forEach documentation https://lodash.com/docs/4.17.15#forEach it mentions that "iteration order is not guaranteed" - should I not be using lodash then for this type of code block? How can I use Lodash and guarantee that order of code execution?
I understand if I want to keep things in order I need to use a for (const product of listOfProducts) loop....
Please check this good article, it's talking about the Array.forEach, but I think it should help.
https://codeburst.io/javascript-async-await-with-foreach-b6ba62bbf404

HapiJS reply with readable stream

For one call, I am replying with a huge JSON object which sometimes causes the Node event loop to become blocked. As such, I'm using Big Friendly JSON package to stream JSON instead. My issue is I cannot figure out how to actually reply with the stream
My original code was simply
let searchResults = s3Access.getSavedSearch(guid)).Body;
searchResults = JSON.parse(searchResults.toString());
return reply(searchResults);
Works great but bogs down on huge payloads
I've tried things like, using the Big Friendly JSON package https://gitlab.com/philbooth/bfj
const stream = bfj.streamify(searchResults);
return reply(stream); // according to docs it's a readable stream
But then my browser complained about an empty response. I then tried to add the below to the reply, same result.
.header('content-encoding', 'json')
.header('Content-Length', stream.length);
I also tried return reply(null, stream); but that produced a ton of node errors
Is there some other way I need to organize this? My understanding was I could just reply a readable stream and Hapi would take care of it, but the response keeps showing up as empty.
Did you try to use h.response, here h is reply.
Example:
handler: async (request, h) => {
const { limit, sortBy, order } = request.query;
const queryString = {
where: { status: 1 },
limit,
order: [[sortBy, order]],
};
let userList = {};
try {
userList = await _getList(User, queryString);
} catch (e) {
// throw new Boom(e);
Boom.badRequest(i18n.__('controllers.user.fetchUser'), e);
}
return h.response(userList);
}

ExpressJS - res.status(500) returning undefined so I can't call send on it

I have some code that usually works, but twice now has produced the following error:
TypeError: Cannot read property 'send' of undefined
The code is:
app.user.get('/status.json', mGatewayTimeout, function (req, res) {
var user = req.user
var qs = cu.querystring.parseUrl(req.url)
if (user.apps && user.apps.beeminder && user.apps.beeminder.access_token) {
if (bsCache[user.username] && !qs.force) {
res.send(bsCache[user.username])
} else {
var bee = new Bee({access_token: req.user.apps.beeminder.access_token})
bee.getUserSkinny(function (err, bm_user) {
if (err) {
bsCache[user.username] = null
return res.status(500).send('error: ' + err.toString())
So that last line produces the TypeError when it tries to call .send on res.status(500).
I've left in a whole bunch of stuff that is certainly irrelevant, because if I tried to take out everything that I thought was irrelevant, we'd be left with nothing.
Bee is an API client, but one that I wrote, so I'm not sure it isn't doing something weird, except that I can't see how it could be affecting the response object.
Try
res.status(500).send(`error ${err.message}`),
new Error's are objects made from a constructor and one of the properties is message.
Oh man, I'm a fool. I made my own problem here. This was on an endpoint that accesses an external API, and since I'm on heroku I need all requests to take under 30s, so I wrote some timeout code that would send an error then turn various functions like res.send into no-op functions. Problem, of course, was that I didn't proceed to return the res object in those no-ops.
🤦‍♀️

FETCH API return undefined

I want to use Fetch API but i don' t really understand it's mecanism.
I have an in my HTML and i want to assign the result of my fetch with this code :
const weather = "http://api.apixu.com/v1/current.json?key=cba287f271e44f88a60143926172803&q=Paris";
const array = [];
fetch(weather)
.then(blob => blob.json())
.then(data => {
array.push(data.current.humidity)
console.log(array[0])
}
);
document.querySelector('h1').innerHTML = array[0];
i have the result with the console.log but the returns "undefined". can you explain why ?
thanks a lot
This is because the call to the API is asynchronous, meaning that the code is not executed just line by line as you write it. The callback only runs as soon as the call to the API has finished, basically meaning that
data => {
array.push(data.current.humidity)
console.log(array[0])
}
runs after
document.querySelector('h1').innerHTML = array[0];
So when you try to set your h1, array is still empty. If you want to set it as soon the data is available, you have to do it within the callback function:
data => {
array.push(data.current.humidity)
document.querySelector('h1').innerHTML = array[0];
}
This might seem weird at first, but keep in mind that you're only registering an anonymous function but not running it yet. You just define the function that you want to trigger as soon as something happens, in this case: when your API call has finished.

Dojo datagrid jsonrest response headers

I'd like to use custom headers to provide some more information about the response data. Is it possible to get the headers in a response from a dojo datagrid hooked up to a jsonRest object via an object store (dojo 1.7)? I see this is possible when you are making the XHR request, but in this case it is being made by the grid.
The API provides an event for a response error which returns the response object:
on(this.grid, 'FetchError', function (response, req) {
var header = response.xhr.getAllResponseHeaders();
});
using this I am successfully able to access my custom response headers. However, there doesn't appear to be a way to get the response object when the request is successful. I have been using the undocumented private event _onFetchComplete with aspect after, however, this does not allow access to the response object, just the response values
aspect.after(this.grid, '_onFetchComplete', function (response, request)
{
///unable to get headers, response is the returned values
}, true);
Edit:
I managed to get something working, but I suspect it is very over engineered and someone with a better understanding could come up with a simpler solution. I ended up adding aspect around to allow me to get hold of the deferred object in the rest store which is returned to the object store. Here I added a new function to the deffered to return the headers. I then hooked in to the onFetch of the object store using dojo hitch (because I needed the results in the current scope). It seems messy to me
aspect.around(restStore, "query", function (original) {
return function (method, args) {
var def = original.call(this, method, args);
def.headers = deferred1.then(function () {
var hd = def.ioArgs.xhr.getResponseHeader("myHeader");
return hd;
});
return def;
};
});
aspect.after(objectStore, 'onFetch', lang.hitch(this, function (response) {
response.headers.then(lang.hitch(this, function (evt) {
var headerResult = evt;
}));
}), true);
Is there a better way?
I solved this today after reading this post, thought I'd feed back.
dojo/store/JsonRest solves it also but my code ended up slightly different.
var MyStore = declare(JsonRest, {
query: function () {
var results = this.inherited(arguments);
console.log('Results: ', results);
results.response.then(function (res) {
var myheader = res.xhr.getResponseHeader('My-Header');
doSomethingWith(myheader);
});
return results;
}
});
So you override the normal query() function, let it execute and return its promise, and attach your own listener to its 'response' member resolving, in which you can access the xhr object that has the headers. This ought to let you interpret the JsonRest result while fitting nicely into the chain of the query() all invokers.
One word of warning, this code is modified for posting here, and actually inherited from another intermediary class that also overrode query(), but the basics here are pretty sound.
If what you want is to get info from the server, also a custom key-value in the cookie can be a solution, that was my case, first I was looking for a custom response header but I couldn't make it work so I did the cookie way getting the info after the grid data is fetched:
dojo.connect(grid, "_onFetchComplete", function (){
doSomethingWith(dojo.cookie("My-Key"));
});
This is useful for example to present a SUM(field) for all rows in a paginated datagrid, and not only those included in the current page. In the server you can fetch the COUNT and the SUM, the COUNT will be sent in the Content-Range header and the SUM can be sent in the cookie.