I was using this piece of code to send some http requests and get data, and it was working fine. I recently updated apache and php to latest versions, as well as node.
And 'close' event stopped firing. I also tried 'end' and 'finish' none of this seems to be working.
I need to know when response is ended so I can start processing data, usually it comes in several chunks. Can you guys help?
var req = http.request(options, function(res) {
res.on('data', function (chunk) {
if(chunk != null && chunk != "") {
dataString += chunk; c
}
});
});
req.on('close', function () {
//tadaa it is finished, so we can process dataString
});
req.write(post_data);
req.end();
Current versions: Apache 2.4, PHP 5.4 node 0.10.9
Maybe there is some particular config settings of Apache that prevents it from closing connection?
P.S. I do not think it is Apache though.. I tried google.com with same result.. pretty strange... Anyone have a working code example? (load big data, and know when it ended)
You should be waiting for the end event of the response, not the request.
e.g.
res.on('end', function () {
// now I can process the data
});
Related
Try to archive:
I try to use the HTML5 EventSourcing API https://developer.mozilla.org/de/docs/Web/API/EventSource to push events to my client application (javascript).
working example code with plain node http:
With a plain example node implementation it works perfectly and as expected. Example code: https://www.html5rocks.com/en/tutorials/eventsource/basics/
Problem:
When i try to integrate EventSourcing (or SSE) into my API endpoint which is based on hapi (currently using latest - 18.1.0) it does not work.
My route handler code mixed with some code i found:
const Stream = require('stream');
class ResponseStream extends Stream.PassThrough {
setCompressor (compressor) {
this._compressor = compressor;
}
}
const stream = new ResponseStream();
let data = 0;
setInterval(() => {
data++;
stream.write('event: message\n');
stream.write('data:' + data + '\n\n');
console.log('write data...', data);
// stream.end();
}, 1000);
return h
.response(stream)
.type('text/event-stream')
.header('Connection', 'keep-alive')
.header('Cache-Control', 'no-cache')
Findings:
I already searched and it seems since hapi 17.x there they exposed the flush method for the compressor < https://github.com/hapijs/hapi/issues/3658 >, section features.
But it still does not working.
They only way it sends a message is to uncomment the stream.end() line after sending the data. The problem obviously is that i cant send further data if i close the stream :/.
If i kill the server (with stream.end() line commented) the data gets transmitted to the client in a "single transmission". I think the problem is is still somewhere with the gzip buffering even when flushing the stream.
There are some code examples in the hapi github but i got none working with hapi 17 or 18 (all exmaples where hapi =< 16) :/
Someone know how to solve the problem or has a working EventSource example with latest hapi? I would kindly appreciate any help or suggestions.
Edit - Solution
The solution from the post below does work but i had also an nginx reverse proxy in front of my api endpoint it seems the main problem was not my code it was the nginx which had also buffered the eventsource messages.
To avoid this sort of problem add in your hapi: X-Accel-Buffering: no; and it works flawless
Well I just tested with Hapi 18.1.0 and managed to create a working example.
This is my handler code:
handler: async (request, h) => {
class ResponseStream extends Stream.PassThrough {
setCompressor(compressor) {
this._compressor = compressor;
}
}
const stream = new ResponseStream();
let data = 0;
setInterval(() => {
data++;
stream.write('event: message\n');
stream.write('data:' + data + '\n\n');
console.log('write data...', data);
stream._compressor.flush();
}, 1000);
return h.response(stream)
.type('text/event-stream')
}
and this is client code just to test
var evtSource = new EventSource("http://localhost/");
evtSource.onmessage = function(e) {
console.log("Data", + e.data);
};
evtSource.onerror = function(e) {
console.log("EventSource failed.", e);
};
These are the resources that where I found my way to working example
https://github.com/hapijs/hapi/blob/70f777bd2fbe6e2462847f05ee10a7206571e280/test/transmit.js#L1816
https://github.com/hapijs/hapi/issues/3599#issuecomment-485190525
I am trying to write a simple electron app to interface with a REST server. The server doesn't have the appropriate certificates. When I try to make a 'GET' request (using fetch()), I get the following error message:
Failed to load resource: net::ERR_BAD_SSL_CLIENT_AUTH_CERT
Fixing the certs is not currently an option. I tried to use the 'ignore-certificates-error' flag (see below). It seems like it should allow me to skip over this error, but it doesn't.
var electron = require('electron');
var app = electron.app
app.commandLine.appendSwitch('ignore-certificate-errors');
...
The result is the same error.
Questions:
I am correct in assuming this options is supposed to help here?
If so, any ideas what I am doing wrong?
Electron version: 1.2.8
Thanks!
You can update your version of electron and use this callback:
app.on('certificate-error', (event, webContents, link, error, certificate, callback) => {
if ('yourURL/api/'.indexOf(link) !== -1) {
// Verification logic.
event.preventDefault();
callback(true);
} else {
callback(false);
}
});
That you going do the fetch to your api with https.
I'm trying to write a front end to an API service with Node JS.
I'd like to be able to have a user point their browser at my node server and make a request. The node script would modify the input to the request, call the api service, then modify the output and pass back to the user.
I like the solution here (with Express JS and node-http-proxy) as it passes the cookies and headers directly from the user through my site to the api server.
proxy request in node.js / express
I see how to modify the input to the request, but i can't figure out how to modify the response. Any suggestions?
transformer-proxy could be useful here. I'm the author of this plugin and I'm answering here because I found this page when looking for the same question and wasn't satisfied with harmon as I don't want to manipulate HTML.
Maybe someone else is looking for this and finds it useful.
Harmon is designed to plug into node-http-proxy https://github.com/No9/harmon
It uses trumpet and so is stream based to work around any buffering problems.
It uses an element and attribute selector to enable manipulation of a response.
This can be used to modify output response.
See here: https://github.com/nodejitsu/node-http-proxy/issues/382#issuecomment-14895039
http-proxy-interceptor is a middleware I wrote for this very purpose. It allows you to modify the http response using one or more transform streams. There are tons of stream-based packages available (like trumpet, which harmon uses), and by using streams you can avoid buffering the entire response.
var httpProxy = require('http-proxy');
var modifyResponse = require('http-proxy-response-rewrite');
var proxy = httpProxy.createServer({
target:'target server IP here',
});
proxy.listen(8001);
proxy.on('error', function (err, req, res) {
res.writeHead(500, {
'Content-Type': 'text/plain'
});
res.end('Something went wrong. And we are reporting a custom error message.');
});
proxy.on('proxyRes', function (proxyRes, req, res) {
modifyResponse(res, proxyRes.headers['content-encoding'], function (body) {
if (body && (body.indexOf("<process-order-response>")!= -1)) {
var beforeTag = "</receipt-text>"; //tag after which u can add data to
// response
var beforeTagBody = body.substring(0,(body.indexOf(beforeTag) + beforeTag.length));
var requiredXml = " <ga-loyalty-rewards>\n"+
"<previousBalance>0</previousBalance>\n"+
"<availableBalance>0</availableBalance>\n"+
"<accuruedAmount>0</accuruedAmount>\n"+
"<redeemedAmount>0</redeemedAmount>\n"+
"</ga-loyalty-rewards>";
var afterTagBody = body.substring(body.indexOf(beforeTag)+ beforeTag.length)+
var res = [];
res.push(beforeTagBody, requiredXml, afterTagBody);
console.log(res.join(""));
return res.join("");
}
return body;
});
});
When using https.request with node.js v04.7, I get the following error:
Error: socket hang up
at CleartextStream.<anonymous> (http.js:1272:45)
at CleartextStream.emit (events.js:61:17)
at Array.<anonymous> (tls.js:617:22)
at EventEmitter._tickCallback (node.js:126:26)
Simplified code that will generate the error:
var https = require('https')
, fs = require('fs')
var options = {
host: 'localhost'
, port: 8000
, key: fs.readFileSync('../../test-key.pem')
, cert: fs.readFileSync('../../test-cert.pem')
}
// Set up server and start listening
https.createServer(function (req, res) {
res.writeHead(200, {'Content-Type': 'text/plain'})
res.end('success')
}).listen(options.port, options.host)
// Wait a second to let the server start up
setTimeout(function() {
var clientRequest = https.request(options, function(res) {
res.on('data', function (chunk) {
console.log('Called')
})
})
clientRequest.write('')
clientRequest.end()
}, 1000)
I get the error even with the server and client running on different node instances and have tested with port 8000, 3000, and 443 and with and without the SSL certificates. I do have libssl and libssl-dev on my Ubuntu machine.
Any ideas on what could be the cause?
In
https.createServer(function (req, res) {
you are missing options when you create the server, should be:
https.createServer(options, function (req, res) {
with your key and cert inside
I had a very similar problem where the response's end event never fired.
Adding this line fixed the problem:
// Hack to emit end on close because of a core bug that never fires end
response.on('close', function () {response.emit('end')});
I found an example of this in the request library mentioned in the previous answer.
Short answer: Use the the latest source code instead of the one you have. Store it where you will and then require it, you are good to go.
In the request 1.2.0 source code, main.js line 76, I see
http.createClient(options.uri.port, options.uri.hostname, options.uri.protocol === 'https:');
Looking at the http.js source code, I see
exports.createClient = function(port, host) {
var c = new Client();
c.port = port;
c.host = host;
return c;
};
It is requesting with 3 params but the actual function only has 2. The functionality is replaced with a separate module for https.
Looking at the latest main.js source code, I see dramatic changes. The most important is the addition of require('https').
It appears that request has been fixed but never re-released. Fortunately, the fix seems to work if you just copy manually from the raw view of the latest main.js source code and use it instead.
I had a similar problem and i think i got a fix. but then I have another socket problem.
See my solution here: http://groups.google.com/group/nodejs/browse_thread/thread/9189df2597aa199e/b83b16c08a051706?lnk=gst&q=hang+up#b83b16c08a051706
key point: use 0.4.8, http.request instead of http.createClient.
However, the new problem is, if I let the program running for long time, (I actually left the program running but no activity during weekend), then I will get socket hang up error when I send a request to http Server. (not even reach the http.request). I don't know if it is because of my code, or it is different problem with http Server
I am trying to use the following to do a cross-domain get:
dojo.io.script.get({
url: myUrl,
callbackParamName: "callback",
preventCache: true,
load: dojo.hitch( this, loadFunction ),
error: dojo.hitch( this, function() {
console.log('Error!!!');
})
});
The load function runs fine, however, when the server returns a 404, the error function does not run. Can anyone tell me why?
EDIT
After some investigation, I found that a timeout and handler could be implemented in the following way:
dojo.io.script.get({
url: myUrl,
callbackParamName: "callback",
timeout: 2000
}).then(function(data){
console.log(data);
}, function(error){
alert(error);
});
This uses functionality provided by the dojo.Deferred object.
When accessing server with script tags (that what dojo.io.script.get does), status code and headers are not available.
You may try some other ways to detect a problem, like using a timeout and analyzing a content of a script. The latter is problematic for JSONP calls (like in your example).
I realize this is old but I thought I'd share a solution in case others, like I had, come across this thread.
dojo.io.script is essentially adding a <script/> to your html page. So you can try this:
var script = document.createElement('script');
script.setAttribute('type', 'text/javascript');
script.setAttribute('src', myUrl);
script.onerror = function() {
debugger
}
script.onload = function() {
debugger
}
document.getElementsByTagName('body')[0].appendChild(script);
That way if the script fails to load the onerror event is called.
*This may not work in every instance but is a good start