Can't SSH connect to Google Cloud Compute Engine after rebooting - ssh

I just restarted my google cloud compute engine instance and now I cannot SSH login to it. Before restarting, it worked fine. I tried fully stopping it and then starting again. There was a webserver on it and if I try to access any API method, I get timeout.
If I try to login from console website:
If I try to login from terminal:
ssh: connect to host 104.197.20.65 port 22: Operation timed out
ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]. See https://cloud.google.com/compute/docs/troubleshooting#ssherrors for troubleshooting hints.
What is wrong?

As the error message suggest the first thing to do is check the firewall rules. There must be a rule allowing connections from any ip (0.0.0.0/0) through the port 22. I assume that your instance's ssh was working before, so it can be two things.
The rule is being applied to a certain network, and the network configuration of the VM was changed when it was shutted down.
The rule is not being applied to the VM, maybe it is in a subnetwork, closing the communication.
To simplify the solution:
A- Add a network tag to your VM. (Ex. ssh-conn)
https://cloud.google.com/compute/docs/vpc/add-remove-network-tags
B- Create a rule to allow connections from any ip and apply it to the network tag previously created.
https://cloud.google.com/compute/docs/vpc/using-firewalls

I had the same problem, and tried lots of way but none of them work.
But the way below is work for me:
Connect to instance by serial console.
vi /etc/ssh/sshd_config, delete the hash tag before Port 22, like this:
Port 22
#AddressFamily any
#ListenAddress 0.0.0.0
#ListenAddress ::
Hope it work for you too.

Related

How to connect to OCI machine if the sshd is not responding?

I was updating the ssh port of an Oracle Cloud Infrastructure machine
I changed /etc/ssh/sshd_config
The port was
#Port 22
I changed it to
Port 40531
Then
restarted the sshd service systemctl restart sshd
open the port on the OCI Web
however, now I cannot connect.
ssh -vvv -p 40531 -i ~/.ssh/vm.key opc#129.xxx.xxx.xxx
OpenSSH_8.2p1, OpenSSL 1.1.1e 17 Mar 2020
debug1: Reading configuration data /etc/ssh/ssh_config
debug2: resolve_canonicalize: hostname 129.xxx.xxx.xxx is address
debug2: ssh_connect_direct
debug1: Connecting to 129.xxx.xxx.xxx [129.xxx.xxx.xxx] port 40531.
debug1: connect to address 129.xxx.xxx.xxx port 40531: Connection timed out
ssh: connect to host 129.xxx.xxx.xxx port 40531: Connection timed out
I saw a Cloud Shell but I'm not sure if it can be used to connect to the machine to perform maintenance tasks
Is there a way to connect to the VM from the web oci interface to fix the ssh issues?
I used to use a VPS service that has a web console from which you can enter to fix problems like this
is there something like this in OCI?
Note:
SELinux was disabled on the machine
if you are about to do this on your machine, remember to update the SELinux configuration prior restart the sshd service or you will be locked out, another option is to disable SELinux totally (this is what I did)
The changes above described worked well, the only thing that was causing issues on my side
(I don't really know why) is that I was connected from a VPN
After I disconnected the VPN and tried to connected again it worked
Update:
I figured out why the ssh using a different port was not working. The VPN I use is a corporate VPN which has very strict inbound and outbound rules, The VPN outbound rules were blocked by the TCP on port 40xxx.
Update:
if you are struggling with a VM you can connect using the below instruction
Creating the Instance Console Connection
Before you can connect to the serial console or VNC console, you need to create the instance console connection.
To create the console connection for an instance
Open the navigation menu. Under Core Infrastructure, go to Compute and click Instances.
Click the instance that you're interested in.
Under Resources, click Console Connection.
Click Create Console Connection.
Upload the public key (.pub) portion for the SSH key. You can browse to a public key file on your computer or paste your public key into the text box.
Click Create Console Connection.
When the console connection has been created and is available, the state changes to Active.
Thanks to #bmuthuv for the info
You can connect to Serial Console of the VM where you could get access to GRUB Menu during a Reboot operation. You can subsequently use typical Linux commands to get to Shell from Grub. You can subsequently undo anything you would like to.
Serial Console connection can be created on OCI Web Console in the Instance's page.

How do I establish ssh connection

I've installed ansible on my master vm, created and copied my key to my worker vm, but I'm unable to establish connection with the worker vm. When I try, I don't get any error message; it just stalls. I'm using ubuntu 18 on gcp.
root#mastervm-project:~# ssh ubuntu#10.128.0.11
There is no firewall rule that allows ingress on 22 on your target instance.
I just reproduced. This will do the job:
Step 1:
Add a network tag on your VMs (allow-ssh in my example)
Step 2:
Go to VPC-FirewallRules and create a firewall rule that allows Ingress traffic incoming from instances that have the allow-ssh tag targeting other instances that have the same tag assigned, on tcp 22
Of course this will allow ssh both ways, but I suppose this is not a problem. If you need it one way only, add a comment.
For SSH Database connection, Please check following solution:
Create SSH DB general connection and call it from some other file

TURN server broken - Coturn

Just a bit of background first.
I have installed coturn server in my local machine (Behind a firewall and with local IP).
I have created a port forwarding for cotrun server lets say my local ip is: 192.168.0.11:3478 is mapped with my public ip 10.1.10.212:3478.
When I use some online utility to check my public IP has this port open it? that utility shows that it is open.
I am trying to test this server using this fiddle: Code to test TURN server
I removed the stun configuration from it, and added my live ip and port. And when I try to test it, it says your TURN server is broken in console on web browser. And I see failed as output.
I have also used following commands to create a user from this link.
Wiki to Coturn Configuration
// created test user
sudo turnadmin -a -u test -r test -p test
// enabling admin support in web (not sure how to access it in browser)
sudo turnadmin -A -u test -p test
I start the turn server using following command:
turnserver -L 192.168.0.11 -a -f -v -X -r test
I have enabled the "TURNSERVER_ENABLED"
/etc/default/coturn
TURNSERVER_ENABLED=1
Screenshot of my about:webrtc:
Please find the below attached Screen shot for my about:webrtc. I am not sure what is missing.
EDIT-1
I can also see the incoming packet message processed, error 401: Unauthorized in the console of TURN server
EDIT-2
I resolved the TURN server related issue, added the turn server in client code as well, still facing issue, call is not going through,
Update
Here is what I found, the reason why it wasn't working earlier was due to a defective router in the network. This thing has harassed me for almost a month (however the internet used to work fine). But I found that there is a loss of packets and I switch to LAN cable I even avoided the firewall of the organization and there it was working just fine. Then I configured the rules in firewall and it started working as well.
A few issues that might be causing this:
The -X option requires an argument - the external IP if the TURN server is behind a NAT.
If you are setting all your parameters on the command line, you should include -n to ensure that it does not load an unrelated configuration file.
TURNSERVER_ENABLED=1 is to start turnserver with default configuration at system boot. You do not want this if you are starting turnserver manually from the command line.
https://github.com/coturn/coturn/blob/master/README.turnserver
" I have created a port forwarding for CoTurn server lets say my local ip is: 192.168.0.11:3478 is mapped with my public ip 10.1.10.212:3478."
Now surely you are aware that the private address-spaces in IPV4 are (private IP ranges):
10.0.0.0 – 10.255.255.255
172.16.0.0 – 172.31.255.255
192.168.0.0 – 192.168.255.255
So, your first IP is in the last private IP range, and your so-called "public IP" is in the first private IP-range. Now you need to find your TRUE REAL public-IP address to try to connect to it from outside. I would say that it looks like your ISP has you behind a NAT. So your TURN server might be behind two NATs. Which is kind of ironic situation for a server designed to help penetrate NATs of webRTC-clients. You might want to try to put CoTurn somewhere where it is possible for it to work like AWS (special case of CoTurn friendly NAT).
Just recently got (well with lots of help -- that even did the most of the work) CoTurn to work in cloud and on local Ubuntu ;-) (never stopped me from bragging though). Though there is troubling few srflx-call-successes yet (mostly local network or relay(TURN) calls). :-D

HTTP access on GCE instance after firewall rule added

I'm trying to get Apache working on a GCE instance.
Following GCE's Quickstart guide, I did the following:
Created instance "my-instance" in "my-project" (CentOS image)
Installed httpd, verified it's running
Added the following firewall rule:
gcutil addfirewall http2 --description="Incoming http allowed." --allowed="tcp:http"
and did the same for HTTPS and ICMP
Verified through gce gui that these rules were added to default network
I can ping my instance's IP address but I can't get an HTTP response. I've tried through the browser, from a curl command - no dice. And it works fine when on localhost so I know Apache is returning the index.html page.
When I use curl from a remote host, the error is:
curl: (7) Failed connect to (instance ip addr):80; Connection refused
Thoughts?
I did some experiments to replicate this. In short, I believe HTTP port 80 may be blocked by iptables firewall rules on the local Centos instance. This appears to be the default behavior.
I have a GCE firewall rule setup to allow port 80 traffic to all instances. I created a centos based image via the Cloud Console (which is indeed using the v1 API). Logged in via SSH and started a web server on port 80. I was not able to hit the web server from my laptop. However I was also not able to hit it from another instance in my project. This lead me to suspect a firewall local to the instance rather than Compute Engine's firewall.
I ran this command (which drops the default reject of all ports for testing - this is unsafe to do for machines which are directly exposed to the internet):
$ sudo iptables -D INPUT -j REJECT --reject-with icmp-host-prohibited
After running that, I was able to hit my webserver from both another instance and my laptop. Note that this change is lost after restarting the instance. I don't know the correct procedure for changing the default firewall rules on Centos.
Please try a similar experiment on your instances, especially try to hit the web server from another Compute Engine instance, since service level firewalls do not block traffic between instances on the same network.

AWS ssh access 'port 22: Operation timed out' issue

I can't access to AWS EC2 instance from one day.
(AMI: ubuntu/images/ebs/ubuntu-precise-12.04-amd64-server-20121001 (ami-22ad1223))
$ ssh -v -i mykey.pem ubuntu#XXX.XXX.XXX.XXX
OpenSSH_5.9p1, OpenSSL 0.9.8x 10 May 2012
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: Connecting to xxx.xxx.xxx.xxx [xxx.xxx.xxx.xxx] port 22.
debug1: connect to address xxx.xxx.xxx.xxx port 22: Operation timed out
ssh: connect to host xxx.xxx.xxx.xxx port 22: Operation timed out
This is my "Security Groups" setting in EC2.
I did not change the setting from the time had a good connection.
Ports Protocol Source
22 tcp 0.0.0.0/0
80 tcp 0.0.0.0/0
3000 tcp 0.0.0.0/0
3006 tcp 0.0.0.0/0
I've tried many times to restart the server.
Web server is going well. However SSH connection is not.
What could be problem and how to make it work?
My usual checklist:
On AWS console: is the Instance up and healthy?
Is it in a public Subnet?
Does it have a public ip?
Does the VPC have an associated Internet Gateway?
Does it have the Routing Table to the Internet Gateway? (Attached to the subnet?)
Are the Network ACL rules default?
Does the Security group allow ping? If yes, does the ping work?
Does the Security group allow SSH inbound?
If there is still no clue, then fire up a new instance (from a base AMI) in the same VPC. Connect to it via SSH. If it was successful, try to ssh from that instance.
I too faced the same issue. Actually, by mistake, I deleted the default Internet Gateway.
Go to VPC and click "Internet Gateways" from the left menu.
Click "Create internet gateway" button and provide Name tag (any name - optional) and click create.
By default, it is detached. So click the Actions drop-down and select "Attach to VPC" and attach it with default VPC
Now go to "Route Table" and select default route table and edit the route by clicking "Edit routes" button under Routes tab
Then in the Destination text box provide "0.0.0.0/0" and in target select the newly created Internet gateway (starts with igw-alphanumeric) and save the route.
Now you should be able to SSH EC2 instance.
For newbies to AWS, like me, remember the hostname can change if you reboot or stop/start your instances. So remember to use the right hostname - visible in the description of your instance each time you ssh.
If this happens "from one day", the IP your AWS EC2 instance associated with may be blocked from this day.
If the IP is blocked, you need to add a new dynamic IP and associate this new dynamic IP with your AWS EC2 instance.
Steps:
1.Go to "Elastic IPs".
2.Allocate new address.
3.Choose this new address. Click "Actions" and "Associate address".
4.Select your instance and Click "Associate".
In my case, adding new dynamic IP to my AWS EC2 instance fix the problem.(My problem was I can't access to AWS EC2 instance from one day too)
Kindly create a new security group and select type SSH
SSH
TCP
22
0.0.0.0/0
In addition to Adam's answer, also check if your public subnet's RT table is using the IGW and the private Subnets' RT has 0.0.0.0/0 -> NAT instance Id.
Check that you are connecting to the public dynamic IP or associate an ElasticIP and connect to it.
I was using public wifi in the library and that was not letting me connect, which I came to know when I switched to my mobile hotspot wifi (password protected). Try switching to a protected network.
Even I also faced this same problem, good to know i have not allowed from route table.
Check EC2 Instance associated
And try these steps
subnet,Route table and allowed CIDR blocks
key pair associated with EC2 Instance
security group ssh port 22 allowed or not.
If you are accessing from a new machine, then make sure the IP of the machine you are accessing from is included in the inbound rules. If not add a rule
SSH | TCP | Port:22 | Source: MY IP
For me, I had to delete all my rules for the security group for the particular instance and create new rules for the same ssh, http and https
For some reason after stopping the instance and starting it later, my IP changed... probably because I switched my wifi connectivity device.
But putting new rules with the new IP address worked! You can check the ip by googling "myip"
Well, if this happened all of a sudden, try disconnecting and connecting back to your VPN (if accessing through a VPN). It might work!
I was able to fixed it simply by following this instruction
https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/get-set-up-for-amazon-ec2.html
It sets up your private key pair as well as security group. The issue I think mainly because the default security group doesn't has a ssh inbound for your local IP setup.
If none of the troubleshooting steps above work for you, make sure that your EC2 container meets all system requirements for the application(s) you're running on the container. SSH will sometimes not be able to start if the memory runs out before getting to the SSH service.
Example: I was perfectly able to SSH into my EC2 container when I first launched it. I then proceeded to install Mailcow. My issue with SSH arose after restarting my container because the application I had installed required heavy services -- Docker, for example. After reading the system requirements from Mailcow, I realized a t2.micro wasn't even close to what I needed to run everything. I changed to a t3.large, and all worked perfectly.
Even after doing this for awhile, you can sometimes forget the most basic steps and requirements.
Try stopping the ec2 instance and then restarting. It worked for me!