According to these: https://gist.github.com/2266544, Session support in Now.js.
This code should return the NowJS clientId, cookie and session:
nowjs.on('connect', function() { console.log(this.user); })
However, what I got was:
{ clientId: 'something', cookie: { } }
The cookie field is empty. There is no session entry (which I read is an official problem).
Additional Information
I am also using ExpressJS, PassportJS, Mongoose, session-mongoose. I initialized NowJS after configuring Express.
Question
Why is my cookie field empty and how can I get it populated?
After many hours of frustration I discovered that the cookie was not being sent because I was listening to port 3000, which was considered a cross domain request. The port that is being listened to has to be the http server that set the cookie.
I am using Dreamhost, which has Apache listening to port 80 and we're not allowed to unbind it, or bind to port 80. But I was able to resolve this by listening to port 8080.
The cookie was successfully sent when I set it to listen to port 8080.
Edit
This only worked for a short time. Not sure why. Cookies stopped being sent after a while.
Edit 2
This worked again after I set up a proxy forwarding 80 to 8080 (google for "dreamhost proxy"). Essentially putting Apache in front of Node.
Related
I'm having a hard time with this setup. I have a node.js box serving HTTP on 3000, websockets on 3001, and secure websockets on 3002. Out in front of that I have a remote Hitch/Varnish caching proxy on its own server that's listening on 443/80 and connecting the first server as its default backend via 3000. A user who visits the site URL https://foo.tld hits the varnish proxy and sees the site, where some javascript on the site tells their browser to connect to wss://foo.tld:3002 for secure websockets.
My problem is getting websockets to pass transparently through to the backend. In the VCL I have the standard
if (req.http.upgrade ~ "(?i)websocket") {
return (pipe);
}
and
sub vcl_pipe {
#Declare pipe handler for websockets
if (req.http.upgrade) {
set bereq.http.upgrade = req.http.upgrade;
set bereq.http.connection = req.http.connection;
}
}
Which doesn't work in this case. To list what I have tried so far with no success:
1: Creating a second backend in VCL named "websockets" that is the same backend IP but on either port 3001 or 3002 and adding "set req.backend_hint = websockets;" before the pipe summon in the first snippet above.
2: Turning off HTTPS and trying to connect it over pure HTTP.
3: Modifying varnish.service to try and make varnish listen on ports other than, or in addition to, -a :80 and -a :8443,proxy, in which cases Varnish simply refuses to start. One attempt was to simply use HTTP only and attempt to run varnish on 3001 to get ws:// working without SSL but varnish refuses to start.
4: Most recently I attempted the following in VCL to try and pick up client connections coming in on 3001:
if (std.port(server.ip) == 3001) {
set req.backend_hint = websockets;
}
My goal is for the Varnish box to pick up secure websocket traffic (wss://) on 3002 (via hitch at 443 using the normal secure websocket connection protocol) and have that passed transparently to the backend websocket server, whether SSL encrypted across that leg of the connection or not. I have set up other, smaller servers like this before and getting websockets working is trivial if Varnish and the backend service are either on the same machine or behind a regulating CDN like Cloudflare, so it has been extra frustrating trying to figure out just what this remote proxy setup needs. I feel like part of the solution is having Varnish or Hitch (not sure) listening on 3002 to accept the connections at which point the normal req.http.upgrade and pipe functions would come into play, but the software refuses to cooperate.
--------Update--
I have broken down the problem into the simplest form I can. The main server (backend) is now serving plain HTTP on 8080 and WS:// on 6081. I have removed hitch and TLS from the equation entirely, but even in this simplified form it is impossible to connect to websockets through Varnish. I can verify that the Websocket server is working correctly on the backend. Connecting to the backend IP address with a browser shows websockets functioning perfectly there. It's Varnish that's the problem.
My current hitch.conf (not relevant here but provided per request):
frontend = "[*]:443"
frontend = "[*]:3001"
backend = "[127.0.0.1]:8443" # 6086 is the default Varnish PROXY port.
workers = 4 # number of CPU cores
daemon = on
# We strongly recommend you create a separate non-privileged hitch
# user and group
user = "redacted"
group = "redacted"
# Enable to let clients negotiate HTTP/2 with ALPN. (default off)
# alpn-protos = "h2, http/1.1"
# run Varnish as backend over PROXY; varnishd -a :80 -a localhost:6086,PROXY ..
write-proxy-v2 = on # Write PROXY header
syslog = on
log-level = 1
# Add pem files to this directory
# pem-dir = "/etc/pki/tls/private"
pem-file = "/redacted/hitch-bundle.pem"
Current default.vcl (stripped down to almost nothing just for testing this. The backend is NOT running on the same machine, it is remote):
# Marker to tell the VCL compiler that this VCL has been adapted to the
# new 4.0 format.
vcl 4.0;
# Default backend definition. Set this to point to your content server.
backend default {
.host = "remote.server.ip";
.port = "8080";
}
backend websockets {
.host = "remote.server.ip";
.port = "6081";
}
sub vcl_recv {
# Happens before we check if we have this in cache already.
#
# Typically you clean up the request here, removing cookies you don't need,
# rewriting the request, etc.
#Allow websockets to pass through the cache (summons pipe handler below)
if (req.http.Upgrade ~ "(?i)websocket") {
set req.backend_hint = websockets;
return (pipe);
} else {
set req.backend_hint = default;
}
}
sub vcl_pipe {
if (req.http.upgrade) {
set bereq.http.upgrade = req.http.upgrade;
set bereq.http.connection = req.http.connection;
}
return (pipe);
}
Varnish's systemd exec parameters:
ExecStart=/usr/sbin/varnishd \
-a http=:80 \
-a proxy=localhost:8443,PROXY \
-a ws=:6081 \
-p feature=+http2 \
-f /etc/varnish/default.vcl \
-s malloc,256m \
-p pipe_timeout=1800
Working in plain HTTP and insecure websockets like this, it should be very simple to get a working model. I don't understand what could possibly be going wrong.
Varnish Cache, the open source version of Varnish, doesn't support backend connections over TLS.
While you can offload TLS using Hitch, the connection to your websocket server will not be encrypted.
Basic VCL example
Here's a very basic VCL example where web & websocket requests are split and sent to separate backends:
vcl 4.1;
backend web {
.port = "3000";
}
backend ws {
.port = "3001";
}
sub vcl_recv {
if (req.http.Upgrade ~ "(?i)websocket") {
set req.backend_hint = ws;
return (pipe);
} else {
set req.backend_hint = web;
}
}
sub vcl_pipe {
if (req.http.upgrade) {
set bereq.http.upgrade = req.http.upgrade;
}
return (pipe);
}
Need more input
However, I'm probably missing a lot of context. I also didn't specify a .host parameter in the backends, so the assumption is that all services are hosted locally.
Please add your full VCL, your Hitch config and the varnishd runtime parameters to your question. This will add context and allows me to come up with a better solution.
What about Hitch?
If you terminate TLS in Hitch, both HTTPS & secure websockets will be handled by Hitch where the plain-text HTTP & websockets will still be directly handeled by Varnish.
See https://www.varnish-software.com/developers/tutorials/terminate-tls-varnish-hitch for a Hitch tutorial that also explains how Varnish should be configured.
I'm a big advocate of using the PROXY protocol in Varnish. The hitch tutorial has a specific section about this: https://www.varnish-software.com/developers/tutorials/terminate-tls-varnish-hitch/#enable-the-proxy-protocol-in-varnish
Custom ports
The standard ports to access the service are 80 for HTTP and insecure websockets and 443 for HTTPS and secure websockets.
If you want to use custom ports for the websockets, it is possible to configure them in Hitch and Varnish.
Let's say you want to main ports 3001 and 3002 for your websockets. This means you need 2 frontends in Hitch:
One for HTTPs on 443
One for secure WS on 3002
See https://www.varnish-software.com/developers/tutorials/terminate-tls-varnish-hitch/#listening-address for more information about the frontend config.
Varnish on the other hand needs to have 3 listening addresses:
One for HTTP on port 80 (-a http=:80)
One for offloaded HTTPS & secure WS with PROXY support on port 8443 (-a proxy=:8443,PROXY)
One for insecure WS on port 3001 (-a ws=:3001)
Next steps
Please use the information and see if this helps to find a solution. If not, please share your VCL file, your Hitch config and varnishd runtime.
Update
Now that you provided more input, the picture starts to become more clear. The fact that you eliminated the TLS part for now will make it a lot easier to debug.
Assuming the names of your listening interfaces for varnishd are http and ws (as mentioned in your systemd unit file), we can use the following varnishlog commands to debug:
varnishlog -g request -q "ReqStart[3] eq 'http'"
This command will show logs for all log transactions where the http listening interface is used.
If you want to make it more granular, you can also add the request URL as a filtering criterium. This will narrow down the number of transactions:
varnishlog -g request -q "ReqStart[3] eq 'http' and ReqUrl eq '/'"
Please add a complete log transaction for one of the failed requests. This will help us understand why requests are failing.
You can do the same for requests on the ws listening interface by using the commands below:
varnishlog -g request -q "ReqStart[3] eq 'ws'"
varnishlog -g request -q "ReqStart[3] eq 'ws' and ReqUrl eq '/'"
I'm assuming you're successful at starting the varnishd program but unsuccessful at getting decent output out of Varnish. The varnishlog program will provide the insight we need. Please add the logging output to your question so I can look into it.
I have a functional app running in a docker on port 3000. I have selenium tests that works when I set my host to http://localhost:3000. I created a container to launch the selenium tests and it fails with the following error:
WebDriverError:Reachederrorpage:about:neterror?e=nssFailure2&u=https://app:3000/&c=UTF-8&f=regular&d=An error occurred during a connection to app:3000.
SSL received a record that exceeded the maximum permissible length.
Error code: <a id="errorCode" title="SSL_ERROR_RX_RECORD_TOO_LONG">SSL_ERROR_RX_RECORD_TOO_LONG</a>
Snippet of my docker-compose.yml
app:
build:
context: .
dockerfile: Dockerfile.dev
volumes:
- ./:/usr/src/app/
ports:
- "3000:3000"
- "3001:3001"
networks:
tests:
selenium-tester:
build:
context: .
dockerfile: Dockerfile.selenium.tests
volumes:
- ./:/usr/src/app/
- /dev/shm:/dev/shm
depends_on:
- app
networks:
tests:
I replaced the host by http://app:3000 but firefox seems to want to redirect this http to https (which is not working). And finally I build my driver like this:
const ffoptions = new firefox.Options()
.headless()
.setPreference('browser.urlbar.autoFill', 'false'); // test to disable auto https redirect… not working obviously
const driver = Builder()
.setFirefoxOptions(ffoptions)
.forBrowser('firefox')
.build();
When manually contacting the http://app:3000 using curl inside the selenium-tester container it works as expected, I get my homepage.
I'm short on ideas now and even decomposing my problem to write this question didn't get me new ones
I had exactly the same problem - couldn't successfully make request on HTTP to app from Selenium-controlled browsers (Chrome or Firefox) in other Docker container on same network. cURL from that container though worked fine! Connect on HTTP, but something seemed to be trying to force HTTPS. Identical situation right down to the name of the container "app".
The answer is... it's the name of the container!
"app" is a top level domain on the HSTS preloaded list - that is, browsers will force access through HTTPS.
Fix is to use a container name that isn't on HSTS preloaded lists.
HSTS - more reading
As you mentioned manually contacting the http://app:3000 using curl inside the selenium-tester container it works as expected
This error message...
WebDriverError:Reachederrorpage:about:neterror?e=nssFailure2&u=https://app:3000/&c=UTF-8&f=regular&d=An error occurred during a connection to app:3000.
SSL received a record that exceeded the maximum permissible length.
Error code: <a id="errorCode" title="SSL_ERROR_RX_RECORD_TOO_LONG">SSL_ERROR_RX_RECORD_TOO_LONG</a>
...implies that SSL layer in curl or one of its dependencies seems broken.
#RussellFulton in this discussion mentioned:
This seems to be the result you see from Firefox when the server is not configured properly for SSL. Possibly Chrome would have just gave a generic ssl failed error.
This can happen when the browser sends a SSL handshake when the server is expecting an HTTP request. Server responds with a 400 code and an error message that is much bigger that the handshake message that the browser expects. Hence you see the message.
Reasons and Solution
When the error prone code tries to redirect to HTTPS on port 80 (port 3000 in your case).
Solution: Removing the port 80 (port 3000 in your case) from the url, the redirect works.
HTTPS by default runs over port 443.
This error also occurs when you have enabled the SSL module.
Solution: You have run a2enmod ssl.
a2enmod ssl
//or
a2ensite default-ssl
Provided a wrong IP in the ssl config.
Solution: Changed IP to what it should be.
Remove the IP if not needed in the ssl config.
Solution: Change
VirtualHost your.domain.com:443
//to
VirtualHost default:443
curl: (35) SSL received a record that exceeded the maximum permissible length. issue was discussed at length.
As per Curl Support HTTPS proxy and SOCKS+HTTP(s) there was another attempt to get the HTTPS proxy support into Curl.
This curl commit should have addressed your issue.
I am new to node.js and am trying to get into the hang of actually using it. I am very familiar with JavaScript so the language itself is self-explanatory but the use of Node.js is quite different from the browser implementation.
I have my own remote virtual server and have installed Node and the Package Manager and everything works as expected. I am not exactly a server extraordinaire and have limited experience with the Terminal and Apache Configurations.
I can run my server using:
nodejs index.js
Which gives me: listening on *:3300 as expected.
I can then access my localhost from the terminal using: curl http://localhost:3300/ which gives me the response I expect.
Given that the website that links to my server is https://example.com, how do I allow this link to access: http://localhost:3300/ so that I can actually use my node server in production? For example, http://localhost:3300/ runs a Socket Server that I would like to use using Socket.io on https://example.com/chat.html with the JavaScript:
var socket = io.connect('http://localhost:3300/', {transports: ['websocket'], upgrade: false});
Ok, this question has nothing to do with nodeJS.
localhost is a hostname that means this computer. it's equivalent to 127.0.0.1 or whatever IP address you can refer to your computer.
After the double colon (:) you enter the port number.
So if you want to make an HTTP call to a web-server running on your server, you have to know what is the IP address of your server, or the domain name, and then you call it with the port number where the server is running.
For Instance, you would call https://example.com:3300/chat.html to make an HTTP call to a server running on example.com with port 3300.
Keep in mind, that you have to make sure with your firewall configuration, that the specific port is open for incoming HTTP requests.
I have the following model that i drew below:
I have a number of processes running on the server. I want nginx or apache to direct the incoming clients through port 80 to one of the server processes to handle the requests. However each connection also establishes a websocket connection to the same process. This is currently initiated from the client side within javascript. At the moment for testing purposes I pass the port within the html rendered on the client. The client then takes this port and estabilishes a websocket connection to the same port that handled its request.
Moving forward to an nginx or apache envionment would it be possible not to pass the port value to the client and have nginx or apache know where it directed the incoming client and use the same port for the websocket connection?
This would have the benefit on not opening all the server ports 8000, 8001, 8002 in the diagram below to the public.
I have the following situation: 2 hosts, one is a client and the other an HTTPS server.
Client (:<brwsr-port>) <=============> Web server (:443)
I installed Fiddler on the server so that I now have Fiddler running on my server on port 8888.
The situation i would like to reach is the following:
|Client (:<brwsr-port>)| <===> |Fiddler (:8888) <===> Web server (:443)|
|-Me-------------------| |-Server--------------------------------|
From my computer I want to contact Fiddler which will redirect traffic to the web server. The web server however uses HTTPS.
On The server I set up Fiddler to handle HTTPS sessions and decrypt them. I was asked to install on the server Fiddler's fake CA's certificate and I did it! I also inserted the script suggested by the Fiddler wiki page to redirect HTTPS traffic
// HTTPS redirect -----------------------
FiddlerObject.log("Connect received...");
if (oSession.HTTPMethodIs("CONNECT") && (oSession.PathAndQuery == "<server-addr>:8888")) {
oSession.PathAndQuery = "<server-addr>:443";
}
// --------------------------------------
However when I try https://myserver:8888/index.html I fail!
Failure details
When using Fiddler on the client, I can see that the CONNECT request starts but the session fails because response is HTTP error 502. Looks like no one is listening on port 8888. In fact, If I stop Fiddler on the server I get the same situation: 502 bad gateway.
Please note that when I try https://myserver/index.html and https://myserver:443/index.html everything works!
Question
What am I doing wrong?
Is it possible that...?
I thought that since maybe TLS/SSL works on port 443, I should have Fiddler listen there and move my web server to another port, like 444 (I should probably set on IIS an https binding on port 444 then). Is it correct?
If Fiddler isn't configured as the client's proxy and is instead running as a reverse proxy on the Server, then things get a bit more complicated.
Running Fiddler as a Reverse Proxy for HTTPS
Move your existing HTTPS server to a new port (e.g. 444)
Inside Tools > Fiddler Options > Connections, tick Allow Remote Clients to Connect. Restart Fiddler.
Inside Fiddler's QuickExec box, type !listen 443 ServerName where ServerName is whatever the server's hostname is; for instance, for https://Fuzzle/ you would use fuzzle for the server name.
Inside your OnBeforeRequest method, add:
if ((oSession.HostnameIs("fuzzle")) &&
(oSession.oRequest.pipeClient.LocalPort == 443) )
{
oSession.host = "fuzzle:444";
}
Why do you need to do it this way?
The !listen command instructs Fiddler to create a new endpoint that will perform a HTTPS handshake with the client upon connection; the default proxy endpoint doesn't do that because when a proxy receives a connection for HTTPS traffic it gets a HTTP CONNECT request instead of a handshake.
I just ran into a similar situation where I have VS2013 (IISExpress) running a web application on HTTPS (port 44300) and I wanted to browse the application from a mobile device.
I configured Fiddler to "act as a reverse proxy" and "allow remote clients to connect" but it would only work on port 80 (HTTP).
Following on from EricLaw's suggestion, I changed the listening port from 8888 to 8889 and ran the command "!listen 8889 [host_machine_name]" and bingo I was able to browse my application on HTTPS on port 8889.
Note: I had previously entered the forwarding port number into the registry (as described here) so Fiddler already knew what port to forward the requests on to.