I'd like to figure out the value of 'keepalive_requests' for a given 'Nginx' or 'Apache' server from the client side. The default for 'Nginx' is 100 (http://nginx.org/en/docs/http/ngx_http_core_module.html) but I'd like to analyse this for www.example.com where I don't have access to the config.
Obviously I could start a Wireshark and do it manually. I was hoping on some sort of easy shell (e.g. 'wget' like) command.
From client side, I use ab test, wireshark, and an editor like notepad++ to count number of http requests in a socket.
First I use apache ab test to send request, something as bellow:
ab -n 100 -c 100 http://www.example.com/index.php
Before execution, start my wireshark, and set the display filter:
ip.dst == && tcp.port == 80 && !http && tcp.flags.fin==1
After the ab test finished, the result list of wireshark shows the total sockets used during the requests. Right click one packet, and click follow TCP stream, the opened windows shows all the message send and receive in this socket.
If the last FIN direction was sent by remote nginx server, which was influenced by the value of keepalive_requests. We can copy the all requests in this TCP connection into the notepad++, search for the key word and then count the requests in this socket. the number presents the value of keepalive_requests setting in the remote nginx server.
BTW, I wish a better solution, my solution is not so good.
Related
Trying to configure a syslog-ng server to send all of the logs that it receives, to another syslog-ng server over TLS. Both running RHEL 7. Everything seems to be working from an encryption and cert perspective. Not seeing any error messages in the logs, an openssl s_client test connection works successfully, I can see the packets coming in over the port that I'm using for TLS, but nothing is being written to disk on the second syslog-ng server. Here's the summary of the config on the syslog server that I'm trying to send the logs to:
source:
source s_encrypted_syslog {
syslog(ip(0.0.0.0) port(1470) transport("tls")
tls(key-file("/etc/syslog-ng/key.d/privkey.pem")
certfile("/etc/syslog-ng/cert.d/servercert.pem")
peer-verify(optional-untrusted)
}
#changing to trusted once issue is fixed
destination:
destination d_syslog_facility_f {
file("/mnt/syslog/$LOGHOST/log/$R_YEAR-$R_MONTH-$R_DAY/$HOST_FROM/$HOST/$FACILITY.log" dir-owner ("syslogng") dir-group("syslogng") owner("syslogng") group("syslogng"));
log setting:
log { source (s_encrypted_syslog); destination (d_syslog_facility_f); };
syslog-ng is currently running as root to rule out permission issues. selinux is currently set to permissive. Tried increasing the verbosity on syslog-ng logs and turned on debugging, but not seeing anything jumping out at me as far as errors or issues go. Also the odd thing is, I have very similar config on the first syslog-ng server and it's receiving and storing logs just fine.
Also, I should note that there could be some small typo's in the config above as I'm not able to copy and paste it. Syslog-ng allows me to start up the service with no errors with the config that I have loaded currently. It's simply not writing the data that it's receiving to the destination that I have specified.
It happens quite often that the packet filter prevents a connection to the syslog port, or in your case port 1470. In that case the server starts up successfully, you might even be able to connect using openssl s_client on the same host, but the client will not be able to establish a connection to the server.
Please check that you can actually connect to the server from the client computer (e.g. via openssl s_client, or at least with something like netcat or telnet).
If the connection works, another issue might be that the client is not routing messages to this encrypted destination. syslog-ng only performs the SSL handshake as messages are being sent. No messages would result in the connection being open but not really exchanging packets on the TCP level.
Couple of troubleshooting tips:
You can check if there is a connection between the client and the server with "netstat -antp | grep syslog-ng" on the server or the client. You should see connections in the ESTABLISHED state on both sides of the connection (with local/remote addresses switched of course).
Check that your packet filter lets port 1470 connections through. You are most likely using iptables, try reviewing your ruleset and see if port 1470 on TCP is allowed to pass in the INPUT chain. You could try adding a "LOG" rule right before the default rule to see if the packets are dropped at that level. If you already have LOG rules, you might check the kernel logs of the server to see if that LOG rule produced any messages.
You can also confirm if there's traffic with tcpdump on the server (e.g. tcpdump -pen port 1470). If you write the traffic dump to a file (e.g. the -w argument to tcpdump, along with -s 0 to avoid truncation), then this dump file can be analyzed with wireshark to see if the negotiation takes place. You should at the very least see a "Client Hello" and a "Server Hello" packet which are not encrypted at the beginning of the handshake.
I am creating a 'firewall' type device (i.e. sitting in the middle of a communication) that in some cases need to intercept a HTTPS request and return a message to the client browser (like e.g. : sorry this is blocked).
I can do this for HTTP by redirecting (with iptables DNAT) to another port on the device where netcat is listening:
while true; do echo -e "HTTP/1.1 200 OK\n\nsorry this is blocked"|nc -l -p 8000; done
(so nc is listening on port 8000 and returning a normal code 200 reply. Could of course also be some other return code like 403 Forbidden etc.)
But what to do for HTTPS?
The whole thing is encapsulated in SSL/TLS and if intercepted the browser will just display a message that the secure connection failed.
I tried responding with a HTTP 307 Temporary Redirect with a Location pointing to http://127.0.0.1 (which would then give the above message). But the browser doesn't like this.
I need to display some sort of customized message (not necessarily HTML).
I realize that it would be a huge security issue if a HTTPS request could be changed to HTTP, thus stripping the security without the client noticing, but can a popup message or something not be forced in the client? Or at least a standard code like '403 Forbidden'..?
Is there something in the SSL or TLS protocols that I can (ab)use?
Thanks.
So you are developing a transparent proxy. When it comes to HTTPS traffic every proxy has the choice:
Pass it without decryption
Block it completely
Perform a man-in-the-middle attack for getting access to the content
If you performing the man-in-the-middle attack and the client does not trust the certificate used by the proxy it will get a certificate warning. You can not send anything HTTP related to the client because SSL/TLS already fails to establish the tunnel. No tunnel means that you will not be able to transmit a single "HTTP byte" (this also means that you can not redirect the client somewhere else).
And on SSL/TLS level there is AFAIK no way to send a custom message. The "TLS alert message" only allows pre-defined constant values.
It's not that much of a question, rather a confirmation that what I did is right or not and if it is safe or not.
Until now what I have found googling around is that you cannot run rtorrent through a proxy. You can either put the http request through a proxy, or tsocks, in both cases either the actual transfers are done directly or not done at all. Therefore until now the only proposed viable solution is a VPN which I wanted to avoid.
What I did was use an http proxy for the http part and a port forwarding for the actual download part. For example, lets assume the following:
192.168.1.10 --> Local machine with the actual rtorrent
remote.machine.com --> The remote machine used as a proxy
Procedure:
I created 2 ssh tunnels
ssh -N -D 9090 user#remote.machine.com
ssh -R 9091:localhost:9091 user#remote.machine.com
From the local machine I installed polipo as the html proxy and configured it to use a socks proxy in the remote.machine.com.
I edited the following lines in /etc/polipo/config so that I can get the socks proxy.
socksParentProxy = "localhost:9090"
socksProxyType = socks5
I also changed the html proxy port for extra security, again in /etc/polipo/config
proxyPort = 9080
On the local machine I changed the ~/.rtorrent.rc as following:
#Proxy of the http requests through polipo
http_proxy=localhost:9080
# The ip address reported to the tracker.
#Really important, in order to get connections for downloads
ip = remote.machine.com
# The ip address the listening socket and outgoing connections is
# bound to.
bind = 192.168.1.10
# Port range to use for listening.
port_range = 9091-9091
# Start opening ports at a random position within the port range.
port_random = no
The system seems to work. I connect to the trackers and I have up and down traffic. So the questions are:
Am I safe that all the traffic concerning rtorrent is done through the remote.machine.com?
Did I miss something?
Are there any problems or concerns regarding this method?
As far as I see, you have covered inbound connections, as well as outgoing HTTP traffic, but any outbound peer-to-peer connections will be created directly, not through any tunnel. Currently, rtorrent does not appear to support passing outbound P2P connections through a tunnel or proxy of any kind, so in order to handle these, you'll need some other mechanism.
You mentioned tsocks and that it does not work – not even in addition to the rtorrent configuration you have set up above? (Although with tsocks you should be able to drop the HTTP proxy part.)
If that fails, there are alternatives to tsocks mentioned on the tsocks project page. A slightly more involved alternative would be to create a new loopback interface (lo:1 with IP 127.0.0.2), bind your rtorrent to that one and use something like sshuttle to direct all traffic originating on that interface through an SSH tunnel. Unfortunately, sshuttle doesn't let you restrict its operation to a specific interface at the moment, though, so you'd have to fiddle with the iptables rules it creates to make them match your needs. I assume a patch adding this feature to sshuttle would be welcome.
As a side note, you can create multiple port forwards and SOCKS proxies in a single SSH connection, like this:
ssh -N -D 9090 -R 0.0.0.0:9091:localhost:9091 myself#my.example.com
In my Rails 3.2 app, I'm using embedded javascript charts from TradingView. However, it seems that port 443 must be opened for it to show. Therefore, I'm trying to do a check on if the port is open for the user, and if not, show something aside from the chart.
I found this post, "Ruby - See if a port is open" and have been working of this answer:
require 'socket'
require 'timeout'
def port_open?(ip, port, seconds=1)
Timeout::timeout(seconds) do
begin
TCPSocket.new(ip, port).close
true
rescue Errno::ECONNREFUSED, Errno::EHOSTUNREACH
false
end
end
rescue Timeout::Error
false
end
However, I modified it to be:
def port_open?(ip, port=443, seconds=1)
since the port to check will always be the same for me.
I'm calling the method like this:
#ip = request.remote_ip
test = port_open?(#ip)
I discovered the issue when I was behind a firewall.
What I don't understand is that I can see the chart now that I'm not behind a firewall, yet if I run the code below locally, I can see the chart, but I receive false.
However, if I change the port=3000 I receive true. What am I doing wrong? Thanks!
Ports under 1000 are protected by the system and need to be run with root privileges. At first glance, I'd guess this is what's wrong.
Port 443 is the normal channel for SSL (https) connections -- it would be quite surprising if a company set outbound restrictions on this port on their firewall. More likely, it's the IP address of the site that may be restricted.
Get to a command line and try with telnet. First try something you know works, like google
> telnet google.com 443
> Trying 74.125.226.231...
> Connected to google.com.
> Escape character is '^]'.
The Connected to google.com response demonstrates that your machine has made a successful connection to the port and IP address. Otherwise, it will hang at the Trying x.x.x.x... line.
Another possibility is that your intranet is set up to go through a proxy server -- you make a connection internally to a machine in your network, and it in turn makes the actual connection. This can screw things up -- not sure what you would need to do to handle that, but something special -- browsers have special settings for handling proxy servers, so your Rails code would need to, as well.
I've built a self hosted wcf server, using wsHttpBinding. I'm running Win 2003 server R2 SP2.
If I configure it to listen on http://localhost:443/MyService, everything works fine. I can connect to http://localhost:443/MyService with Internet Explorer, and I get the standard "Bad Request" message
Now, if I try to switch to HTTPS, I'm witnessing a strange phenomenon.
Here's what I've done :
I've changed my wcf config file from http://localhost to https://localhost and from Security=None to Security=Transport (as explained in numerous wcf tutorials)
I've registered my HTTP port like this :
httpcfg delete ssl -i 0.0.0.0:443
httpcfg set ssl -i 0.0.0.0:443 -h ea2e450ef9d4...
Note that the certificate I've used is a "real certificate" (i.e. issued by a trusted CA, namely Comodo). The server responds to ping on the NS mentioned in the certificate.
Now, the following will timeout :
Microsoft Telnet> open localhost 443
Here's the output from netstat (The Pid '4' is the 'System' process):
netstat -nao
Proto Local Adress Remote Adress State Pid
TCP 0.0.0.0:443 0.0.0.0:0 Listening 4
And here's a screenshot from TCPView captured when I issued the open command in telnet :
alt text http://img26.imageshack.us/img26/3376/tcpview2si6.jpg
I'm a bit puzzled. To me, if netstat says the server is listening on 443, the telnet connection to 443 shouldn't timeout, and I should have at least a blank prompt, expecting me to type some encrypted stuff :)
So far I've tried to :
Redo all the steps from scratch following exactly the MSDN tutorial
Used port 10443 instead of 443
Disable the firewall
Use a self signed certificate
I don't know what to try next .. any ideas?
The telnet client is not going to know to send a properly constructed request to initiate an https handshake, so I imagine the ssl secured server is just waiting for more data.
The telnet client is certainly not going to know what to do with the response from a ssl secured server (it's certainly not going to prompt you for data to send along). Communication can only happen once the https handshake has completed.
You need to use a client that knows how to do a handshake. The openssl binary can do this out of the box.
Telnet cannot be used to comunicate with encrited webs.
Checkout this microsfot note. It says "NOTE: This example assumes that the Web server is configured to use the default HTTP port (TCP 80). If the Web server is listening on a different port, substitute that port number in the first line of the example. Also, this example does not work properly over an HTTPS/SSL connection (TCP 443, by default), because the telnet client cannot negotiate the necessary encryption commands to establish the SSL session. Although an initial connection is possible over the HTTPS/SSL port, no data is returned when you issue a GET request."
Update: Checkout this other note HOW TO: Determine If SSL Connectivity Is Not Working on the Web Server or on an Intermediate Device
As FerrariB said, telnet does not perform the negotiations necessary to open an SSL connection. Telnet knows nothing about certificates, nor encryption. Thus, you are guaranteed to not be able to communicate with HTTPS port 443 via telnet. You will have to find another way to do whatever you are trying to do.
Check out the Wikipedia page on TLS for example, where it says directly:
If any one of the above steps fails, the TLS handshake
fails, and the connection is not created.
This is precisely what you are seeing by trying to use telnet to communicate with an SSL endpoint.
in command prompt: netstat -nao |find "443"
the last columns show a number:
pic no.1
Now open task manager.find result number in 1st section in pid column (if pid wasn't enabled, choose it from view tab) program name show the program which uses the port.
disable the program that uses the port /in my case I stopped it from services