Intercept Requests With Custom Responses in PhantomJS? - phantomjs

Is there a way to intercept a resource request and give it a response directly from the handler? Something like this:
page.onRequest(function(request){
request.reply({data: 123});
});
My use case is for using PhantomJS to render a page that makes calls to my API. In order to avoid authentication issues, I'd like to intercept all http requests to the API and return the responses manually, without making the actual http request.
onResourceRequest almost does this, but doesn't have any modification capabilities.
Possibilities that I see:
I could store the page as a Handlebars template, and render the data into the page and pass it off as the raw html to PhantomJS (instead of a URL). While this would work, it would make changes difficult since I'd have to write the data layer for each webpage, and the webpages couldn't stand alone.
I could redirect to localhost, and have a server there that listens and responds to the requests. This assumes that it would be ok to have an open, un-authenticated version of the API on localhost.
Add the data via page.evaluate to the page's global window object. This has the same problems as #1: I'd need to know a-priori what data the page needs, and write server side code unique to each page.

I recently needed to do this when generating pdfs with phantom js.
It's slightly hacky, but seems to work.
var page = require('webpage').create(),
server = require('webserver').create(),
totallyRandomPortnumber = 29522,
...
//in my actual code, totallyRandomPortnumber is created by a java application,
//because phantomjs will report the port in use as '0' when listening to a random port
//thereby preventing its reuse in page.onResourceRequested...
server.listen(totallyRandomPortnumber, function(request, response) {
response.statusCode = 200;
response.setHeader('Content-Type', 'application/json;charset=UTF-8');
response.write(JSON.stringify({data: 'somevalue'}));
response.close();
});
page.onResourceRequested = function(requestData, networkRequest) {
if(requestData.url.indexOf('interceptme') != -1) {
networkRequest.changeUrl('http://localhost:' + totallyRandomPortnumber);
}
};
In my actual application I'm sending some data to phantomjs to overwrite request/responses, so I'm doing more checking on urls both in server.listen and page.onResourceRequested.
This feels like a poor-mans-interceptor, but it should get you (or whoever this may concern) going.

Related

Expressjs send content after listen

I want to make a route in express js to send some content after 1000 ms.
Note: I cant use res.sendFile, it has to be a plain route.
This is the code for the route:
app.get('/r', (req,res)=>{
res.send("a")
setTimeout(()=>{
res.send("fter")
}, 1000)
}
app.listen(8080)
But I get the error: ERR_HTTP_HEADERS_SENT, I assume because the page has already been loaded.
I need my node program to send it after it has already been loaded, so I cant like send a html,js,css script to do it. Is it possible? I cant seem to find how.
Well, if that is not possible, what I am really trying to do is after the page has loaded, execute js or send a message that the page can receive from the node program, like if there was res.execute_js('postMessage(1)')
EDIT based on your edit: So as I understand you want a way to send different values from a node app endpoint without using socketio. I've managed to replicate a similar experimental behavior using readable streams. Starting off, instead of returning response to the request with res.send() you should be using res.write() In my case I did something like this:
app.post('/api', (req, res) => {
res.write("First");
setTimeout(() => {
res.write("Second");
res.end();
}, 1000);
});
This will write to a stream "First" then after 1000ms it'll write another "Second" chunk then end the stream, thus completing the POST request.
Now in the client, you'll make the fetch response callback async, get the ReadableStream from the request like so
const reader = response.body.getReader();
now we should be reading this stream, we'll first initialize an array to collect all what we're reading,
const output = [];
now to actually read the stream,
let finished, current;
while (!finished) {
({ current, finished} = await reader.read());
if (finished) break;
output.push(current);
}
if you read current in the loop, it'll contain each value we passed from res.write() and it should read twice, "First" and after 1000ms "Second".
EDIT: This is very experimental however, and I wouldn't recommend this in a production codebase. I'd suggest trying out socketio or a publish/subscribe mechanism instead.
Old answer: You're already sending "a" back, you should remove the first res.send() invocation at the top of the callback.
So, this is for all the people wondering. No you cannot do this with pure express (there is a workaround, so keep reading).
The reason you cant do this is because, when the user requests to the url, it sends them a response, and the browser renders it. You cant then tell it to change the response, as the browser has already received a response. Even if you send multiple, like with res.write, rather then res.send, the browser will just wait until it receives all the data.
Here are two workarounds:
    1. Use socket.io, cscnode, or another library to have events for updating text,
    2. Send hardcoded html, that updates text (1 was probably better)
That is all I think you can do.
More clarification on the socketio one is basically have an event for changing text that you can fire from node, and the browser will understand, and change the text.

What exactly does the expected HTTP response for a Reverse-AJAX request look like?

I'm trying to implement a simple Web Service (running on an Arduino board using an Ethernet shield) that can provide (push) information to a subscribed client by means of Reverse-AJAX.
The web service hosts a single web page that presents information from a (2D-LIDAR) sensor connected to that server board. Whenever the sensor output changes (very frequently and rapidly) the clients viewing that page should be instantly updated. For this application Reverse-AJAX / AJAX Push seems to be the option of choice, however I'm struggling to get the server part working.
This is what's in my aforementioned web page to "listen" for updates:
var xhr = new XMLHttpRequest();
xhr.multipart = true;
xhr.open( 'GET', 'push', true) ;
xhr.onreadystatechange = function() {
if (xhr.readyState == 4) {
processEvents( window.JSON.parse( xhr.responseText ) );
}
}
xhr.send( null );
I'd like to keep the XmlHttpRequest running forever and have it call the processEvents function whenever a chunk of (JSON) data comes in from the server side. However I'm not sure what the server response, especially the HTTP response header should look like to make this work as expected.
Whenever I have the server send a HTTP response header like this
HTTP/1.1 200 OK\r\n
Connection: keep-alive\r\n
Content-Length: 100\r\n
Content-Type: text/json\r\n
\r\n
the XmlHttpRequest finishes after receiving exactly one "chunk" of data. I also tried without "Content-Length" header, but "Content-Type: multipart/mixed; boundary=..." or "Transfer-Encoding: chunked" but both never happened to fire processEvents, supposedly because the browser was waiting for the response to complete, whatever that means.
I'm therefore looking for a working example of such a HTTP Response to an AJAX-Push request. What does a HTTP Response generally need to look like to be accepted by the indefinitely running XmlHttpRequest and to fire processEvents whenever new data arrives?
Btw. I tried those things using Firefox 64.0.
if you a looking for a low latency http based comm you should take a look at websockets. Your xmlhttp with Post method will have a high latency of about 50 milliseconds , trust me, I tried to develop a rgb controller based on a rainbow color picker before, either sync and asynchronous post methods wasn’t working well for me, the script just hold waiting for a response.
Now to answer you question specifically, download postman, a software that allows you to simulate all the http methods requests and headers you wish to. Also gives you the code to implement in many languages, and don’t forget f12 on chrome > network tab, this way you can check how the output of or http requests are being handled

How to distinguish between GET and POST

I'm writing a simple api for training using express. Here's my testing code:
var express = require('express');
var app = express();
app.post("/api/:var_name", function(req, res) {
res.send(req.params.var_name);
});
is simply testing to see if POST is working. When I call http://localhost:3000/api/1 I get Cannot GET /api/1, so the server is obviously interpreting the POST request as GET, what do I need to do to call POST instead?
Anything you call in the address bar of your browser will be sent via get. This is due to the fact that post-messages (and almost all other methods) do have a body-part. But there is no way for your browser to send additional information inside the body of the http packet.
If you want to test your routes for any method other than GET I would suggest you download a tool like postman.
https://www.getpostman.com/
BEWARE: This is my preference. You can of curse also use text based browsers like curl to test it.
The server interprets the request according to the verb you set in the HTTP request. If no method/verb is specified it is interpreted as GET(not sure about this part).
When you call that URL, you need to use the method as well. For example if you use the fetch API, you can call it like:
fetch(url, {method:"POST"})
If you're entering it in your browser and expect it to be interpreted as a post request, it's not. All browser url requests are GET. Use a tool like Postman to call different HTTP verbs. It's really useful when creating such APIs.
You can check out this answer on details of how to add body and headers to a post request: Fetch: POST json data

Passing javascript variable to velocity variable templete

I have installed xwiki successfully and able to generate wiki pages using velocity template language.
Could anyone please tell me that how can I pass javascript varible to velocity templete. I have gone through few forums that I need to pass the parameter to server to get this but I have no idea. Please find the files below.
<script type="text/javascript">
function generateFunction()
{
var variable = document.getElementById('text').value;
}
</script>
#set($test = "variable")
$test
You have to make an ajax call from the client to the server.If you're using jquery, you would have something like:
$.post('/send/my/var', { 'variable' : value });
Without jquery, see this XmlHttpRequest documentation.
And then, on the server side, the /send/my/var URL should reach a template where you can do:
#set($test = $params.variable)
And you would do something useful with it on the server-side, like store it in the session, in the database, etc.
If you need to send back something from Velocity to Javascript, then you'll typically have to format JSON code, and add an asynchronous completion callback parameter to the ajax call:
$.post('/send/my/var', { 'variable' : value },
function(data)
{
// do something with data sent back from the server
});
It's also possible to have synchronous calls, that is to have javascript wait for the server response, but it's generally a bad idea to do so and I won't extrapolate on it here.
As a final note, you should also implement a proper error handling. With jQuery for instance, the syntax would be:
$(document).ajaxError(function(event, jqxhr, settings, message)
{
console.log(message);
});
It can't be done,
Apache Velocity template is a server side engine,
Meaning that on the server, Velocity will get the template and try to render, only after it finished to render the template, it will be returned to client which will execute client code as Javascript
Velocity alternative is freemarker, which I found similar question and answer , Question:
How to call freemarker function with param from javascript
Answer:
There's no way for the client side web browser code to call a server side Freemarker function

Still confused about using XMLHTTPRequest cross domain

I need to POST data to a server in a different domain. That server is using SSL and expects the data to be in the form of a JSON string. I am attempting to do this from javascript.
I create the data and use JSON.stringify() to get it into the correct format. Then I send it as follows:
var url = "https://api.postageapp.com/v.1.0/send_message.json";
http=new XMLHttpRequest();
http.open("POST", url, true);
http.setRequestHeader("Content-type", "application/json");
http.setRequestHeader("Connection", "close");
// create the data in a data structure named post_data
var JSONText = JSON.stringify(post_data);
http.send(JSONText);
Doing a packet trace I see my client do a handshake with the server but then twice the server replies with "Encrypted alert" including the last time it sends a packet back. The browser debugger always shows a 405 - Method Now Allowed error.
What am I missing to get this to work? When they try it within their domain it runs fine.
You need server to return a HTTP Header like that:
header('Access-Control-Allow-Origin: *');
Live example:
Making cross domain JavaScript requests using XMLHttpRequest or XDomainRequest
You cannot do a cross domain post like that.
Alternative is to use Server side proxy (read this link for a nice explanation as to why you can't do that) or iframe approach.
Strictly speaking it should not be possible (due to security issues) however using a workaround called JSONP you can achieve this with a RESTful web service.
See the link below.
http://en.wikipedia.org/wiki/JSONP
MS has some code you can download somewhere on the internet with specific bindings the code is called.
JSONPBehaviour.cs
JSONPBindingElement.cs
JSONPBindingExtension.cs
JSONPEncoderFactory.cs