I am looking for a batch script file which will automatically telnet to a user input website on port 80 and 443 and get us the Server: value (Info).
Below script:
telnet -f ip.txt %ip% 80
GET / HTTP/1.1
Host: %ip%
for /F "delims=" %%c in ('type ip.txt ^| findstr Server:') do set server=%%c
This script does the job but need to enter Ctrl+C to close the session, I need something which will automatically close the connection and get me the output.
I also require help with openssl.exe with the same requirement to check TCP 443 (HTTPS) websites.
Using openssl.exe, I need loop or nested for checking sslv3, tls1.0 - 1.2. Any help with this is highly appreciated.
Related
I've been setting up an AWS EC2 server this week, and I'm almost there with what I want to do. But opening up as a web server is proving to be a stumbling block.
MY SETUP
I have an AWS EC2 instance running Red Hat EL7.
I have an Apache server running on my instance:
[ec2-user#ip-172-xx-xx-xx ~]$ ps -ef | grep -i httpd
root 18162 1 0 18:02 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 18163 18162 0 18:02 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 18164 18162 0 18:02 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 18165 18162 0 18:02 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 18166 18162 0 18:02 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
apache 18167 18162 0 18:02 ? 00:00:00 /usr/sbin/httpd -DFOREGROUND
ec2-user 21345 20507 0 19:03 pts/1 00:00:00 grep --color=auto -i httpd
It seems to be listening on port 80:
[root#ip-172-xx-xx-xx ~]# netstat -lntp | grep 80
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 18162/httpd
I added inbound rules to the "launch-wizard-1" security group (which is shown as the security group for the instance) for port 80 (HTTP) and 443 (HTTPS) with sources of "0.0.0.0/0" and "::/0"
And finally, for testing my setup, I created an index.html file in my document root (/var/www/html):
<html>
<h1>TEST!</h1>
</html>
THE PROBLEM
From my chrome browser on my computer, when I try to hit:
http://ec2-18-xxx-xxx-xx.us-east-2.compute.amazonaws.com/index.html
I just get:
This page isn’t working
ec2-18-xxx-xxx-xx.us-east-2.compute.amazonaws.com didn’t send any data.
ERR_EMPTY_RESPONSE
(I get the same when I hit one of my domain names which I've set up on there, which is what I'm really trying to do of course!)
I've tried connecting from Chrome on 2 different computers, and from Safari on my phone ("Safari cannot open the page because it could not connect to the server")
CHECKS I'VE PERFORMED
I don't believe I have any server firewall preventing this:
[root#ip-xx-xx-xx-xx conf]# /sbin/iptables -L -v -n
Chain INPUT (policy ACCEPT 3575 packets, 275K bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 2215 packets, 350K bytes)
pkts bytes target prot opt in out source destination
Testing with telnet from a terminal session on my mac, port 80 appears to be open. Firstly using the IPv2 Public IP:
telnet 18.xxx.xxx.xx 80
Trying 18.xxx.xxx.xx...
Connected to ec2-18-xxx-xxx-xx.us-east-2.compute.amazonaws.com.
Escape character is '^]'.
Connection closed by foreign host.
and using the Public DNS (IPv4):
telnet ec2-18-xxx-xxx-xx.us-east-2.compute.amazonaws.com 80
Trying 18.xxx.xxx.xx...
Connected to ec2-18-xxx-xxx-xx.us-east-2.compute.amazonaws.com.
Escape character is '^]'.
Connection closed by foreign host.
And again, the same goes for my domain names - telnet to port 80 shows "Connected".
- Is the fact that the "foreign host" closes the connection immediately significant? Should it stay open if everything is working as it should?
Running curl on the host correctly returns my simple index.html file:
[ec2-user#ip-172-xx-xx-xx ~]$ curl localhost
<html>
<h1>TEST!</h1>
</html>
However, running a curl on my local computer - to the server - returns:
curl -v http://ec2-18-xxx-xxx-xx.us-east-2.compute.amazonaws.com:80
* Rebuilt URL to: http://ec2-18-xxx-xxx-xx.us-east-2.compute.amazonaws.com:80/
* Trying 18.xxx.xxx.xx...
* Connected to ec2-18-xxx-xxx-xx.us-east-2.compute.amazonaws.com (18.xxx.xxx.xx) port 80 (#0)
> GET / HTTP/1.1
> Host: ec2-18-xxx-xxx-xx.us-east-2.compute.amazonaws.com
> User-Agent: curl/7.43.0
> Accept: */*
>
* Empty reply from server
* Connection #0 to host ec2-18-xxx-xxx-xx.us-east-2.compute.amazonaws.com left intact
curl: (52) Empty reply from server
I also tested the webserver "internally" by running google chrome (headless) on the server to create a screenshot, downloaded to my local computer and it shows TEST! (i.e. its working):
google-chrome-stable --headless --disable-gpu --screenshot http://localhost
One more thing to add - when I attempt the hit the webserver from my local machine, nothing shows in the webserver logs (error_log or access_log) on the server.
So, my opinion is that the web server is up and running, works locally, but is not working correctly for anything coming from "outside". I'm stumped now though.
Doh! I rebooted the instance and.. all working now!
22 years working with computers and it took me 22 hrs to resort to a reboot. Fool!
Connect to your EC2 instance using ssh on terminal
Install python if not installed
Start a python server using nohup to continuously use the server
nohup python -m http.server &
This usually open port 8000, goto EC2 Security Group Make source anywhere or as needed.
Navigate to the folder having index.html, file path will look like below
http://ec2---.compute-1.amazonaws.com:8000/folder/website/
You will be able to develop and see your changes as needed.
I am trying to test my configuration using filebeat test ouput -e -c filebeat.yml i see only the help message with command list.
I am actually trying to output the data file to verify. Though i have tested filebeat test config -e -c filebeat.yml with ok.
Assuming you're using filebeat 6.x (these tests were done with filebeat 6.5.0 in a CentOS 7.5 system)
To test your filebeat configuration (syntax), you can do:
[root#localhost ~]# filebeat test config
Config OK
If you just downloaded the tarball, it uses by default the filebeat.yml in the untared filebeat directory. If you installed the RPM, it uses /etc/filebeat/filebeat.yml.
If you want to define a different configuration file, you can do:
[root#localhost ~]# filebeat test config -c /etc/filebeat/filebeat2.yml
Config OK
To test the output block (i.e: if you have connectivity to elasticsearch instance or kafka broker), you can do:
[root#localhost ~]# filebeat test output
elasticsearch: http://localhost:9200...
parse url... OK
connection...
parse host... OK
dns lookup... OK
addresses: ::1, 127.0.0.1
dial up... ERROR dial tcp [::1]:9200: connect: connection refused
In this case my localhost elasticsearch is down, so filebeat throws an error saying it cannot connect to my output block.
The same way as syntax validation (test config), you can provide a different configuration file for output connection test:
[root#localhost ~]# filebeat test output -c /etc/filebeat/filebeat2.yml
logstash: localhost:5044...
connection...
parse host... OK
dns lookup... OK
addresses: ::1, 127.0.0.1
dial up... ERROR dial tcp [::1]:5044: connect: connection refused
In this alternative configuration file, my output block also fails to connect to a logstash instance.
-e -c flags must be used before the command "test"
filebeat -e -c filebeat.yml test ouput
-c, --c argList Configuration file, relative to path.config (default beat.yml)
I'm currently running a bunch of:
sudo ssh -L PORT:IP:PORT root#IP
where IP is the target of a secured machine, and PORT represents the ports I'm forwarding.
This is because I use a lot of applications which I cannot access without this forwarding. After performing this, I can access through localhost:PORT.
The main problem occured now that I actually have 4 of these ports that I have to forward.
My solution is to open 4 shells and constantly search my history backwards to look for exactly which ports need to be forwarded etc, and then run this command - one in each shell (having to fill in passwords etc).
If only I could do something like:
sudo ssh -L PORT1+PORT2+PORT+3:IP:PORT+PORT2+PORT3 root#IP
then that would already really help.
Is there a way to make it easier to do this?
The -L option can be specified multiple times within the same command. Every time with different ports. I.e. ssh -L localPort0:ip:remotePort0 -L localPort1:ip:remotePort1 ...
Exactly what NaN answered, you specify multiple -L arguments. I do this all the time. Here is an example of multi port forwarding:
ssh remote-host -L 8822:REMOTE_IP_1:22 -L 9922:REMOTE_IP_2:22
Note: This is same as -L localhost:8822:REMOTE_IP_1:22 if you don't specify localhost.
Now with this, you can now (from another terminal) do:
ssh localhost -p 8822
to connect to REMOTE_IP_1 on port 22
and similarly
ssh localhost -p 9922
to connect to REMOTE_IP_2 on port 22
Of course, there is nothing stopping you from wrapping this into a script or automate it if you have many different host/ports to forward and to certain specific ones.
For people who are forwarding multiple port through the same host can setup something like this in their ~/.ssh/config
Host all-port-forwards
Hostname 10.122.0.3
User username
LocalForward PORT_1 IP:PORT_1
LocalForward PORT_2 IP:PORT_2
LocalForward PORT_3 IP:PORT_3
LocalForward PORT_4 IP:PORT_4
and it becomes a simple ssh all-port-forwards away.
You can use the following bash function (just add it to your ~/.bashrc):
function pfwd {
for i in ${#:2}
do
echo Forwarding port $i
ssh -N -L $i:localhost:$i $1 &
done
}
Usage example:
pfwd hostname {6000..6009}
jbchichoko and yuval have given viable solutions. But jbchichoko's answer isn't a flexible answer as a function, and the opened tunnels by yuval's answer cannot be shut down by ctrl+c because it runs in the background. I give my solution below solving both the two flaws:
Defing a function in ~/.bashrc or ~/.zshrc:
# fsshmap multiple ports
function fsshmap() {
echo -n "-L 1$1:127.0.0.1:$1 " > $HOME/sh/sshports.txt
for ((i=($1+1);i<$2;i++))
do
echo -n "-L 1$i:127.0.0.1:$i " >> $HOME/sh/sshports.txt
done
line=$(head -n 1 $HOME/sh/sshports.txt)
cline="ssh "$3" "$line
echo $cline
eval $cline
}
A example of running the function:
fsshmap 6000 6010 hostname
Result of this example:
You can access 127.0.0.1:16000~16009 the same as hostname:6000~6009
In my company both me and my team members need access to 3 ports of a non-reachable "target" server so I created a permanent tunnel (that is a tunnel that can run in background indefinitely, see params -f and -N) from a reachable server to the target one. On the command line of the reachable server I executed:
ssh root#reachableIP -f -N -L *:8822:targetIP:22 -L *:9006:targetIP:9006 -L *:9100:targetIP:9100
I used user root but your own user will work. You will have to enter the password of the chosen user (even if you are already connected to the reachable server with that user).
Now port 8822 of the reachable machine corresponds to port 22 of the target one (for ssh/PuTTY/WinSCP) and ports 9006 and 9100 on the reachable machine correspond to the same ports of the target one (they host two web services in my case).
Another one liner that I use and works on debian:
ssh user#192.168.1.10 $(for j in $(seq 20000 1 20100 ) ; do echo " -L$j:127.0.0.1:$j " ; done | tr -d "\n")
One of the benefits of logging into a server with port forwarding is facilitating the use of Jupyter Notebook. This link provides an excellent description of how to it. Here I would like to do some summary and expansion for all of you guys to refer.
Situation 1. Login from a local machine named Host-A (e.g. your own laptop) to a remote work machine named Host-B.
ssh user#Host-B -L port_A:localhost:port_B
jupyter notebook --NotebookApp.token='' --no-browser --port=port_B
Then you can open a browser and enter: http://localhost:port_A/ to do your work on Host-B but see it in Host-A.
Situation 2. Login from a local machine named Host-A (e.g. your own laptop) to a remote login machine named Host-B and from there login to the remote work machine named Host-C. This is usually the case for most analytical servers within universities and can be achieved by using two ssh -L connected with -t.
ssh -L port_A:localhost:port_B user#Host-B -t ssh -L port_B:localhost:port_C user#Host-C
jupyter notebook --NotebookApp.token='' --no-browser --port=port_C
Then you can open a browser and enter: http://localhost:port_A/ to do your work on Host-C but see it in Host-A.
Situation 3. Login from a local machine named Host-A (e.g. your own laptop) to a remote login machine named Host-B and from there login to the remote work machine named Host-C and finally login to the remote work machine Host-D. This is not usually the case but might happen sometime. It's an extension of Situation 2 and the same logic can be applied on more machines.
ssh -L port_A:localhost:port_B user#Host-B -t ssh -L port_B:localhost:port_C user#Host-C -t ssh -L port_C:localhost:port_D user#Host-D
jupyter notebook --NotebookApp.token='' --no-browser --port=port_D
Then you can open a browser and enter: http://localhost:port_A/ to do your work on Host-D but see it in Host-A.
Note that port_A, port_B, port_C, port_D can be random numbers except common port numbers listed here. In Situation 1, port_A and port_B can be the same to simplify the procedure.
Here is a solution inspired from the one from Yuval Atzmon.
It has a few benefits over the initial solution:
first it creates a single background process and not one per port
it generates the alias that allows you to kill your tunnels
it binds only to 127.0.0.1 which is a little more secure
You may use it as:
tnl your.remote.com 1234
tnl your.remote.com {1234,1235}
tnl your.remote.com {1234..1236}
And finally kill them all with tnlkill.
function tnl {
TUNNEL="ssh -N "
echo Port forwarding for ports:
for i in ${#:2}
do
echo " - $i"
TUNNEL="$TUNNEL -L 127.0.0.1:$i:localhost:$i"
done
TUNNEL="$TUNNEL $1"
$TUNNEL &
PID=$!
alias tnlkill="kill $PID && unalias tnlkill"
}
An alternative approach is to tell ssh to work as a SOCKS proxy using the -D flag.
That way you would be able to connect to any remote network address/port accesible through the ssh server as long as the client applications are able to go through a SOCKS proxy (or work with something like socksify).
If you want a simple solution that runs in the background and is easy to kill - use a control socket
# start
$ ssh -f -N -M -S $SOCKET -L localhost:9200:localhost:9200 $HOST
# stop
$ ssh -S $SOCKET -O exit $HOST
I've developed loco for help with ssh forwarding. It can be used to share ports 5000 and 7000 on remote locally at the same ports:
pip install loco
loco listen SSHINFO -r 5000 -r 7000
First It can be done using Parallel Execution by xargs -P 0.
Create a file for binding the ports e.g.
localhost:8080:localhost:8080
localhost:9090:localhost:8080
then run
xargs -P 0 -I xxx ssh -vNTCL xxx <REMOTE> < port-forward
or you can do a one-liner
echo localhost:{8080,9090} | tr ' ' '\n' | sed 's/.*/&:&/' | xargs -P 0 -I xxx ssh -vNTCL xxx <REMOTE>
pros independent ssh port-forwarding, they are independent == avoiding Single Point of Failure
cons each ssh port-forwarding is forked separately, somehow not efficient
second it can be done using curly brackets expansion feature in bash
echo "ssh -vNTC $(echo localhost:{10,20,30,40,50} | perl -lpe 's/[^ ]+/-L $&:$&/g') <REMOTE>"
# output
ssh -vNTC -L localhost:10:localhost:10 -L localhost:20:localhost:20 -L localhost:30:localhost:30 -L localhost:40:localhost:40 -L localhost:50:localhost:50 <REMOTE>
real example
echo "-vNTC $(echo localhost:{8080,9090} | perl -lpe 's/[^ ]+/-L $&:$&/g') gitlab" | xargs ssh
Forwarding 8080 and 9090 to gitlab server.
pros one single fork == efficient
cons by closing this process (ssh) all forwarding are closed == Single Point of Failure
You can use this zsh function (probably works with bash, too)(Put it in ~/.zshrc):
ashL () {
local a=() i
for i in "$#[2,-1]"
do
a+=(-L "${i}:localhost:${i}")
done
autossh -M 0 -o "ServerAliveInterval 30" -o "ServerAliveCountMax 3" -NT "$1" "$a[#]"
}
Examples:
ashL db#114.39.161.24 6480 7690 7477
ashL db#114.39.161.24 {6000..6050} # Forwards the whole range. This is simply shell syntax sugar.
I have a server who is forwarding connections to a set of other servers.
Here I forward all incomming connections on:
my.tunnel.com:33199 to my.server2.com:52222
And..
my.tunnel.com:33200 to my.server3.com:52222
.. until
my.tunnel.com:XXXXX to my.serverN.com:52222
I'm initiating this by the following command on each server, except the tunnel my.tunnel.com:
ssh -o StrictHostKeyChecking=no -l root -i /etc/ssh/id_rsa -R *:33199:127.0.0.1:22 -p 443 my.tunnel.com 0 33199
...
ssh -o StrictHostKeyChecking=no -l root -i /etc/ssh/id_rsa -R *:XXXXX:127.0.0.1:22 -p 443 my.tunnel.com 0 XXXXX
Well, this works fine!
But!
At the point of the launching of each of these commands I'd like to check on my.tunnel.com that my.server2.com wants my.tunnel.com to forward exactly from port 33199, but not another port! So at this point I'd like to get this port number.
Please let me know if the problem is still not enough clearly exposed.
Thanks!
To get the forwarded port
There is no such information in the environmental variables, so you must pass it yourself:
ssh -R 33199:127.0.0.1:22 my.tunnel.com "export MY_FWD_PORT=33199; my_command"
(my_command is the script you want to run on the server). More information about passing variables - https://superuser.com/q/163167/93604
To get the source port
Look at the environment variable SSH_CONNECTION in man ssh(1). Its meaning is:
source_ip source_port dest_ip dest_port
You probably want source_port, so just get the second part of it:
echo $SSH_CONNECTION | awk '{ print $2 }'
or
echo $SSH_CONNECTION | cut -d" " -f 2
I have written a small bash script which needs an ssh tunnel to draw data from a remote server, so it prompts the user:
echo "Please open an ssh tunnel using 'ssh -L 6000:localhost:5432 example.com'"
I would like to check whether the user had opened this tunnel, and exit with an error message if no tunnel exist. Is there any way to query the ssh tunnel, i.e. check if the local port 6000 is really tunneled to that server?
Netcat is your friend:
nc -z localhost 6000 || echo "no tunnel open"
This is my test. Hope it is useful.
# $COMMAND is the command used to create the reverse ssh tunnel
COMMAND="ssh -p $SSH_PORT -q -N -R $REMOTE_HOST:$REMOTE_HTTP_PORT:localhost:80 $USER_NAME#$REMOTE_HOST"
# Is the tunnel up? Perform two tests:
# 1. Check for relevant process ($COMMAND)
pgrep -f -x "$COMMAND" > /dev/null 2>&1 || $COMMAND
# 2. Test tunnel by looking at "netstat" output on $REMOTE_HOST
ssh -p $SSH_PORT $USER_NAME#$REMOTE_HOST netstat -an | egrep "tcp.*:$REMOTE_HTTP_PORT.*LISTEN" \
> /dev/null 2>&1
if [ $? -ne 0 ] ; then
pkill -f -x "$COMMAND"
$COMMAND
fi
Autossh is best option - checking process is not working in all cases (e.g. zombie process, network related problems)
example:
autossh -M 2323 -c arcfour -f -N -L 8088:localhost:80 host2
This is really more of a serverfault-type question, but you can use netstat.
something like:
# netstat -lpnt | grep 6000 | grep ssh
This will tell you if there's an ssh process listening on the specified port. it will also tell you the PID of the process.
If you really want to double-check that the ssh process was started with the right options, you can then look up the process by PID in something like
# ps aux | grep PID
Use autossh. It's the tool that's meant for monitoring the ssh connection.
We can check using ps command
# ps -aux | grep ssh
Will show all shh service running and we can find the tunnel service listed
These are more detailed steps to test or troubleshoot an SSH tunnel. You can use some of them in a script. I'm adding this answer because I had to troubleshoot the link between two applications after they stopped working. Just grepping for the ssh process wasn't enough, as it was still there. And I couldn't use nc -z because that option wasn't available on my incantation of netcat.
Let's start from the beginning. Assume there is a machine, which will be called local with IP address 10.0.0.1 and another, called remote, at 10.0.3.12. I will prepend these hostnames, to the commands below, so it's obvious where they're being executed.
The goal is to create a tunnel that will forward TCP traffic from the loopback address on the remote machine on port 123 to the local machine on port 456. This can be done with the following command, on the local machine:
local:~# ssh -N -R 123:127.0.0.1:456 10.0.3.12
To check that the process is running, we can do:
local:~# ps aux | grep ssh
If you see the command in the output, we can proceed. Otherwise, check that the SSH key is installed in the remote. Note that excluding the username before the remote IP, makes ssh use the current username.
Next, we want to check that the tunnel is open on the remote:
remote:~# netstat | grep 10.0.0.1
We should get an output similar to this:
tcp 0 0 10.0.3.12:ssh 10.0.0.1:45988 ESTABLISHED
Would be nice to actually see some data going through from the remote to the host. This is where netcat comes in. On CentOS it can be installed with yum install nc.
First, open a listening port on the local machine:
local:~# nc -l 127.0.0.1:456
Then make a connection on the remote:
remote:~# nc 127.0.0.1 123
If you open a second terminal to the local machine, you can see the connection. Something like this:
local:~# netstat | grep 456
tcp 0 0 localhost.localdom:456 localhost.localdo:33826 ESTABLISHED
tcp 0 0 localhost.localdo:33826 localhost.localdom:456 ESTABLISHED
Better still, go ahead and type something on the remote:
remote:~# nc 127.0.0.1 8888
Hallo?
anyone there?
You should see this being mirrored on the local terminal:
local:~# nc -l 127.0.0.1:456
Hallo?
anyone there?
The tunnel is working! But what if you have an application, called appname, which is supposed to be listening on port 456 on the local machine? Terminate nc on both sides then run your application. You can check that it's listening on the correct port with this:
local:~# netstat -tulpn | grep LISTEN | grep appname
tcp 0 0 127.0.0.1:456 0.0.0.0:* LISTEN 2964/appname
By the way, running the same command on the remote should show sshd listening on port 127.0.0.1:123.
#!/bin/bash
# Check do we have tunnel to example.com server
lsof -i tcp#localhost:6000 > /dev/null
# If exit code wasn't 0 then tunnel doesn't exist.
if [ $? -eq 1 ]
then
echo ' > You missing ssh tunnel. Creating one..'
ssh -L 6000:localhost:5432 example.com
fi
echo ' > DO YOUR STUFF < '
stunnel is a good tool to make semi-permanent connections between hosts.
http://www.stunnel.org/
If you are using ssh in background, use this:
sudo lsof -i -n | egrep '\<ssh\>'