at the first day when i created the instance, i was able to SSH no problem, but after yesterday, i just couldnt connect to my instance. when i checked the console i get something like this
Nov 5 15:30:49 my-app kernel: [79738.555434] [UFW BLOCK] IN=ens4 OUT= MAC=42:01:0a:94:00:02:42:01:0a:94:00:01:08:00 SRC=71.15.27.115 DST=10.121.0.7 LEN=60 TOS=0x00 PREC=0x00 TTL=50 ID=38049 PROTO=TCP SPT=37344 DPT=22 WINDOW=60720 RES=0x00 SYN URGP=0
i figured its a firewall issue, but my firewall rule seems okay (assuming i did not change anything since first i created the instance). i wonder what else could be the problem? here's my fw config
default-allow-http
http-server
IP ranges: 0.0.0.0/0
tcp:80
Allow
1000
default
default-allow-https
https-server
IP ranges: 0.0.0.0/0
tcp:443
Allow
1000
default
default-beego-http
http-server
IP ranges: 0.0.0.0/0
tcp:8080
Allow
1000
default
default-jenkins-app
http-server
IP ranges: 0.0.0.0/0
tcp:8989
Allow
1000
default
default-allow-icmp
Apply to all
IP ranges: 0.0.0.0/0
icmp
Allow
65534
default
default-allow-internal
Apply to all
IP ranges: 10.128.0.0/9
tcp:0-65535, udp:0-65535, 1 more
Allow
65534
default
default-allow-rdp
Apply to all
IP ranges: 0.0.0.0/0
tcp:3389
Allow
65534
default
default-allow-ssh
Apply to all
IP ranges: 0.0.0.0/0
tcp:22
Allow
65534
default
Looking at the output you’ve provided following your attempt to SSH into your instance, it looks like you’re being blocked by UFW (Uncomplicated Firewall) which is installed/enabled on the actual instance, rather than the GCP project wide firewall rules you have set (these look okay).
In order to SSH into your VM you will need to open port 22 in UFW on the instance. There are a couple of possible methods that will allow you to do this.
Firstly, see Google Compute Engine - alternative log in to VM instance if ssh port is disabled , specifically the answer by Adrián which explains how to open port 22 using a startup script. This method requires you to reboot your instance before the firewall rule is applied.
Another method which doesn’t require a reboot of the machine makes use of the Serial Console. However, in order to use this method a password for the VM is required. This method is therefore only possible if you previously set a password on the VM (before losing access).
To connect via the Serial Console the following metadata must be added, either to the instance you are trying to connect to, or to the entire project:
serial-port-enable=1
You can apply the metadata to a specific instance like so:
gcloud compute instances add-metadata [INSTANCE_NAME] \
--metadata=serial-port-enable=1
Or alternatively, to the entire project by running:
gcloud compute project-info add-metadata --metadata=serial-port-enable=1
After setting this metadata you can attempt to connect to the instance via the Serial Console by running the following command from the Cloud Shell:
gcloud compute connect-to-serial-port [INSTANCE_NAME]
When you have accessed the instance you will be able to manage the UFW rules. To open port 22 you can run:
sudo /usr/sbin/ufw allow 22/tcp
Once UFW port 22 is open, you should then be able to SSH into your instance from Cloud Shell or from the Console.
There is some additional info about connecting to instances via the Serial Console here:
https://cloud.google.com/compute/docs/instances/interacting-with-serial-console
Related
Currently, I have built a small datacenter environment in OTC with Terraform. based on Ubuntu 20.04 images.
The idea is to have a jump host in the setup phase and for operational purposes that allows spontaneous access to service frontends via ssh proxy jumps without permanently routing them to the public net.
Basic setup works fine so far - I can access the jump host with ssh, and can access the internal machines from there with ssh when I put the private key onto the jump host. So, cloudwise the security seems to be fine. Key pair is generated with ed25519, I use the same key for jump host and internal servers (for now).
What I cannot achieve is the proxy jump as a chained command from my outside machine.
On the jump host, I set AllowTcpForwarding to "yes" in /etc/ssh/sshd_config and restarted ssh and sshd services.
My current local ssh config looks like this:
Host otc
User ubuntu
Hostname <FloatingIP-Address>
Port 22
StrictHostKeyChecking=no
UserKnownHostsFile=/dev/null
IdentityFile= ~/.ssh/ssh_access
ControlPath ~/.ssh/cm-%r#%h:%p
ControlMaster auto
ControlPersist 10m
Host 10.*
User ubuntu
Port 22
IdentityFile=~/.ssh/ssh_access
ProxyJump otc
StrictHostKeyChecking=no
UserKnownHostsFile=/dev/null
I can use this to ssh otc to the jump host.
What I would expect is that I could use e.g. ssh 10.0.0.56 to reach an internal host without further ado. As well I should be able to use commands like ssh -L 8080:10.0.0.56:8080 10.0.0.56 -N to map an internal server's port to a localhost port on my external machine. This is how I managed that successfully on other hosting scenarios in the public cloud.
All I get is:
Stdio forwarding request failed: Session open refused by peer
kex_exchange_identification: Connection closed by remote host
Journal on the Jump host says:
Jul 30 07:19:04 dev-nc-o-bastion sshd[2176]: refused local port forward: originator 127.0.0.1 port 65535, target 10.0.0.56 port 22
What I checked as well:
ufw is off on the Jump Host.
replaced ProxyJump configuration with ProxyCommand
So I am at the end of my knowledge. Has anyone a hint what else could be the reason? Any help welcome!
Ok, cause is found (but not yet fully explained).
My local ssh setting was allowing multiplexed forwards (ControlMaster auto ) which caused the creation of a unix socket file for the Controlpath in ~/.ssh.
I had to login to the jump host to AllowTcpForwarding in the first place.
After rebooting the sshd, I returned to the local machine and the failure occured when trying to forward to the remote internal machine.
After deleting the socket file in ~/.ssh, the connection can now be established as needed. Obviously, the persistent tunnel was not impacted by the restarted daemon on the jump host and simply refused to follow the new directive.
This cost me two days. On the bright side, I learned a lot about ssh :o
I am trying to connect to tensorboard on my google compute engine instance but it is not working.
I have an anacondo distribution and use:
tensorboard --logdir=/logs
to create my tensorboard at default port 6006.
I also allowed HTTP/HTTPS traffic at my instance and also edited my firewall rules to allow traffic at:
IP ranges: 0.0.0.0/0
tcp:6006
udp:6006
But, when I try to acess my tensorboard at
http://EXTERNAL_IP:6006
I get a timeout loading.
Can anybody help me?
Normally this type of configuration is related to port communication issues. Go ahead and get all the available ports with nmap, and you should see something as following:
$ nmap -Pn [YOUR IP ADDRESS]
PORT STATE SERVICE
22/tcp open ssh
80/tcp closed http
443/tcp closed https
3389/tcp closed ms-wbt-server
Once, you confirm if the port "6006" is open, check if it can connect to your server with a telnet:
$ telnet [YOUR IP ADDRESS] [YOUR PORT]
telnet: Unable to connect to remote host: Connection refused
If you get "connection refused" make sure not only that this port is "open" but that it's "listening" as well (remember this needs to be configured on your application in your web server). You can check that with a netstat as following:
$ netstat -an | egrep -w “6006”
And you should see something like this (example for port 22):
$ netstat -an | grep 22
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN
If it says 127.0.0.1 on the Local Address column, it means that port is ONLY listening for connections from your PC itself, not from the Internet or network. If it says 0.0.0.0, it means that port is listening on all 'network interfaces' (i.e. your computer, your modem(s) and your network card(s)).
Thus, the IP you need is the one as the example (0.0.0.0), since this means all IPs can reach that specific port. Plus, you must see the “Listen” status.
In addition, make sure to set up properly the Firewall rules in GCP and your software running on the instance itself to allow traffic to/from this port “6006” in specific, either to any instance or to a specific one using network tags.
I've been connecting to a Google Cloud VM instance via gcloud ssh from my macOS:
$ gcloud compute ssh [username]#[instance]
Starting from a week ago, the connection will just drop after ~60 seconds of idle connection and returns:
Connection to [my_external_ip] closed by remote host.
Connection to [my_external_ip] closed.
ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255].
I configured the TCP keepalive time to 30 seconds on both my macbook and the VM. But that did not solve the problem.
Any idea how do I extend the connection duration?
This is unlikely an issue with your timeout setting, but more likely an issue with your firewall rules or routes.
Firstly I would suggest checking your firewall rules and ensure you have an ingress firewall rule opening port 22. If you have, check the configuration of this rule, in particular:
Check the IP range in 'Source filters'. Does the range include the IP address of your home computer? For testing purposes, to ensure it does, you could temporarily set this to 0.0.0.0/0 to include all IP addresses.
Check the 'Targets' drop-down. Is this set to apply to 'All instances in the network' or is it set to 'Specified target tags'? If you have set it to 'Specified target tags', make sure that the same tag is added to the 'Network tags' section of the instance, otherwise the firewall rule will not apply to the instance and allow SSH traffic.
Ensure this rule has a higher priority than any other rules that could counteract it (when I say higher priority I mean lower number, for example, a a rule with a priority of 1000 is a higher priority than a rule with a priority of 20000).
If the above doesn't resolve the issue, run the following command to check the routes:
gcloud compute routes list
Ensure there is an entry which contains the following:
default 0.0.0.0/0 default-internet-gateway
EDIT
If you are able to sometimes SSH into the instance but then the connection drops, there may be some useful information in the logs, or the serial console.
You can access the serial console by clicking on the instance name in the GCP Console, then clicking on "Serial port 1 ".
When you SSH into the instance, information about the SSH session populates the serial console output (this can be refreshed by hitting the 'Refresh' at the top of the page.) Information about the session ending also populates the serial console. There may be some useful information/clues about why the session ends in this output.
It might also be worth checking the status of SSH daemon on the instance and giving it a restart to see if that makes a difference:
Check status of sshd:
systemctl status sshd
Restart sshd:
sudo systemctl restart sshd
I'm installing Phabricator on AWS and the usual proxy/web/database setup works for HTTP/S. Now I want to add SSH access to the repos. How can I configure SSH access to the repositories? My usage is small and I'd rather not create complex or obscure setups.
If the elastic IP is associated with the proxy server, the proxy would have to proxy SSH requests to Phabricator. Will a SOCKS proxy work? Is there an easy way (i.e. package) to have a socks proxy connect to the web server whenever either one is rebooted?
Without the SOCKS proxy, it seems the alternative is to put the everything on one server, except the database of course. This means the web server (running Phabricator) will need to be in the public VPS with an elastic IP associated with it. That way HTTPS and SSH can connect to the same hostname.
Are there alternatives? Is my only option to setup everything on one server?
This is accomplished with IP and port forwarding. My initial search for port forwarding was overwhelmed by port forwarding in routers and firewalls.
Add the following to /etc/sysctl.d/50-ip_forward.conf. This works on openSUSE Leap and CentOS.
# Enable IP_FOWARD
net.ipv4.ip_forward = 1
Apply the file with sysctl.
sudo sysctl -p /etc/sysctl.d/50-ip_forward.conf
Then forward the SSH port and test. If you lose access to the server, reboot it from the AWS console. I already changed the admin port to another port before making this change. It's easier for me to add the Port to my ~/.ssh/config than it is for users to specify a different port in their git client.
sudo firewall-cmd --zone external --add-forward-port port=22:proto=tcp:toaddr=1.2.3.4
If everything checks out, make the change permanent.
sudo firewall-cmd --zone external --add-forward-port port=22:proto=tcp:toaddr=1.2.3.4 --permanent
Don't forget to add the necessary ports in the AWS security group. Reboot the server to make sure everything is kept across reboots.
NAT instance
As an added bonus let's turn the proxy into a NAT instance! The external zone already has masquerade enabled but is not used. Assign the eth0 interface to the external zone.
sudo firewall-cmd --zone external --add-interface eth0
In your VPC, add a default route for 0.0.0.0/0 with a target of the instance above to the private subnet. Now test internet connectivity from an internal server in the private subnet. Performance isn't an issue because this route is only for server updates, not production traffic.
ping 8.8.8.8
If everything works, make the change permanent.
sudo firewall-cmd --zone external --add-interface eth0 --permanent
I like to reboot after making network/firewall changes to make sure everything is kept across reboots.
I have a GCE Instance a Debian 1v CPU & 1.7GB. Then I followed the below tutorial and installed webmin on it.
https://www.howtoforge.com/tutorial/how-to-install-webmin-on-ubuntu-15-04/
The installation went successfully. Then I Created a Firewall exception on using the UFW and allowed port 10000.
sudo ufw allow 10000/tcp
But I was not able to access Webmin through the browser.
https://my-gce-instance-ip-address:10000.
Then i created firewall exception using the Google Cloud Console. Again tried the url it didnt work.
Then i thought this might be because of webmin is https mode. So i open the /etc/webmin/miniserv.conf and changed ssl=0. After that i restarted the webmin.
/etc/init.d/webmin restart
Then I tried the the url with Http, still I can't access.
I tried below command and checked the output. Accordingly Webmin is correctly running and listening on port 10000.
netstat -tulpn | grep :10000.
I can't seem to think what I am doing wrong. I have now spent several days on this without and solution in sight. Hope someone can kindly help me?
try this ... it's working for me
iptables -I INPUT 1 -p tcp --dport 10000 -j ACCEPT
service iptables save
/etc/init.d/iptables restart
open both link in Browser
https://your-IP:10000
and
http://your-IP:10000
you need to allow port 10000 from iptables
sudo iptables -A INPUT -p tcp -m tcp --dport 10000 -j ACCEPT
this work for me
i'm using ubuntu 16.04
You don't need to do any firewall configuration in the instance itself. All firewall configuration is done in the Google Cloud console.
The steps I typically follow, as you show to have figured out in your comment, are:
Create the firewall rule, in it opening the particular port you need (10000 in the case of Webmin) for ingress TCP traffic, accepting connections from some IP range (e.g. 0.0.0.0/0), and specifying target tags to be later assigned to instances to which that rule shall apply.
Add one of those tags to the "network tags" section of some particular instance.
This alone should work, opening the port for your instance in the firewall.
I was almost creating another question here on SO when yours was suggested as a possible duplicate. I had followed the steps above on my Webmin machine, and yet the machine refused to connect on port 10000. As I kept writing the question, I figured out my particular problem: in the firewall rule, in the source IP range filter, I set the single meta-address 0.0.0.0 instead of the range 0.0.0.0/0. So, to anyone who has followed the steps above and still can't connect to their webmin installation, do check if your source range filter is correctly set.