Cannot ssh into Google-Engine, connecting in a loop - ssh

I am unable to connect through SSH to my GCE instance. I was connecting without any problem, the only think I changes was my user name through top right corner of the browser then selected Change Linux Username.
When I try to ssh into my google engine via browser, I keep having following message in a endless loop:
When I try to ssh via cloud shell I also get following error message, (serial console output):
Permission denied (publickey).
ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255].
[Q] Is there any way to fix this problem? Since I have no access to the engine now, I don't know what to do.

However you could always get back access through serial console then from there you could internally y troubleshoot user/ssh issue.
1) $ gcloud compute instances add-metadata [INSTANCE_NAME] --metadata=serial-port-enable=1
You can then connect to the instance through the serial port
NOTE:The root password have must been already set in order to use the serial port
2)
$ gcloud compute connect-to-serial-port [INSTANCE_NAME]
If you never set the root password you could set it by adding a startup-script to your instance that will set a password as root by running the below command :
NOTE: the instance must be rebooted in order to run the startup script.
3) $ gcloud compute instances add-metadata [instance name] --metadata startup-script='echo "root:YourPasswdHere" | chpasswd'
Reboot the instance run the command on the step "2)" authenticate your self as root with the password that you set on the startup script in the step "3)" .

I had the same problem, It took me several days to figure out what was happening in my case.
To find out, I created a new instance from scratch and started making all modifications I've done to those that eventually couldn't connect to, one by one, exiting the ssh connection and re entering so as to test it.
I've tried it a couple of times, in both cases, the connection was impossible after uninstalling python (I only needed 3.7 version so I was uninstalling all others and installing that one I needed).
My command for uninstalling it was
sudo apt purge python2.7-minimal
and
sudo apt purge python3.5-minimal
I don't know if it was specifically because of deleting python, or because of using purge (in which case this problem might reproduce if using purge with another program).
I don't even know why would this affect ssh connection.
Could it be that google cloud is somehow using destination python for the ssh web?
In any case, if you are experiencing this problem try to avoid uninstalling anything from the base VM.

Related

SSH login error - "libintl.so.9" not found

I'm logging in to vSphere VM (FreeBSD) using SSH and getting the following error-
Shared object "libintl.so.9" not found, required by "bash"
Connection to xxx closed.
I mistakenly changed root user's shell to bash. I was able to login earlier using csh. I can't run chsh or any other commands because I am not able to login to the server.
Is there a way to revert the shell change or specify shell while SSH login? I have tried rebooting the VM using vSphere but still getting the same error.
I have also tried sftp using FileZilla but since it uses SSH, I am getting the following error-
Status: Connected to xxx
Error: FATAL ERROR: Received unexpected end-of-file from SFTP server
Error: Could not connect to server
To fix this, you will need to shutdown the VM from vsphere, reboot, then choose "Single User" mode. Once at the single user shell, change root's shell to /bin/sh or /bin/csh. Don't use 3rd part shells for root.
Also, you get the error because your bash binary is out of date and not ABI compatible with the OS it's installed. Using pkg update should help once you get access again.

Zeek cluster fails with pcap_error: socket: Operation not permitted (pcap_activate)

I'm trying to setting up a Zeek IDS cluster (v.3.2.0-dev.271) on 3 Ubuntu 18.04 LTS hosts to no avail - running zeek deploy command fails with the following output:
fatal error: problem with interface ens3 (pcap_error: socket: Operation not permitted (pcap_activate))
I have followed the official documentation (which is pretty generic at best) and set up passwordless SSH authentication between the zeek nodes.
I also preemptively created the /usr/local/zeek path on all hosts and gave the zeek user full permissions on that directory. The documentation says The Zeek user must be able to either create this directory or, where it already exists, must have write permission inside this directory on all hosts.
The documentation also says that on the worker nodes this user must have access to the target network interface in promiscuous mode.
My zeek user is a sudoer AND a member of netdev group on all 3 nodes. Yet, the cluster deployment fails. Apparently, when zeekctl establishes the SSH connection to the workers it cannot get a hold of the network interfaces and set caps.
Eventually I was able to successfully run the cluster by following this article - however it requires you to set up the entire cluster as root, which I would like to avoid if at all possible.
So my question is, is there anything blatantly obvious that I am missing? To the best of my knowledge this setup should work, otherwise I don't know how to force zeekctl to run 'sudo' in front of every SSH command it is supposed to run on the workers, or how to satisfy this requirement.
Any guidance will be greatly appreciated, thanks!
I was experiencing the same error for my standalone setup. Found this question from googling it. More googling the error brought me to a few blogs including one in which the comments mentioned the same error. The author mentioned giving the binaries permissions using setcap:
$sudo setcap cap_net_raw,cap_net_admin=eip /usr/local/zeek/bin/zeek
$sudo setcap cap_net_raw,cap_net_admin=eip /usr/local/zeek/bin/zeekctl
After running them both, my instance of zeek is now running successfully.
Source: https://www.ericooi.com/zeekurity-zen-part-i-how-to-install-zeek-on-centos-8/#comment-1586
So, just in case someone else stumbles upon the same issue - I figured out what was happening.
I streamlined the cluster deployment with Ansible (using 'become' directive at task level) and did not elevate when running the handlers responsible for issuing the zeekctl deploy command.
Once I did, the Zeek Cluster deployment succeeded.

Desired port for google cloudSQL connection is not able to be used

I am following the steps here, to setup a CloudSQL DB in Google Cloud Platform. I'm stuck at the step with:
./cloud_sql_proxy -instances="[YOUR_INSTANCE_CONNECTION_NAME]"=tcp:3306
I get the message below:
2018/02/07 19:44:10 listen tcp 127.0.0.1:3306: bind: address already in use
I've tried: lsof -i tcp:3306 but nothing shows up. Alternatively, I am able to start a connection to tcp:3307, but that's not what's required in the tutorial, and may prevent the rest of the tutorial from working. When I do lsof -i tcp:3307 however, I am able to see the PID, and kill the SQL connection.
How is the port address 3306 already in use?? Even rebooted my computer.
My Steps I took to fix
I stop Mysql on my local machine
brew services stop mysql
But I had a problem of giving a directory for
Directory to use for placing Unix sockets representing database instances as seen by the console error
Then I did
sudo mkdir /cloudsql; sudo chmod 777 /cloudsql
My Final Script
./cloud_sql_proxy -instances=MyInstanceConnName=tcp:3306 -projects=myproject -dir=/cloudsql/
UPDATE: After trying to dig through many methods to kill the sql process, find out whats actually running on it, joining the gcloud slack group to ask around, etc etc, I ended up uninstalling mysql, and reinstalling it. Fixed it. :shrug:

ERROR: (gcloud.compute.ssh) [/usr/bin/ssh] exited with return code [255]

I kept getting kicked out of my compute engine instance after a few seconds of idle with the indicated error (255).
I used 'gcloud compute ssh' to log in.
I am using the default firewall setting, which I believe would be good enough for ssh.
But if I am missing something, please so indicate and suggest the fix for this error.
Basically I can't get any efficient work done at this point having to ssh in so many times.
gcloud denies an ssh connection if there was a change in the setup, e.g.
after you changed your default zone or region, or you created another instance.
Then, you must update the ssh keys in your metadata by
sudo gcloud compute config-ssh
If this complains about different entries in your config file where your ssh key entries are stored, ~/.ssh/config, delete this file and execute the above command again.
If you have installed gcloud without sudo, you can omit sudo.
255 is the interactive ssh exit code for ssh failure - otherwise interactive ssh exits with the exit code of the last command executed in the ssh session.
The next time you get exit code 255 from ssh try running with --ssh-flag="-vvv" (more v's => more debugging output) and see if it helps track down connection problems.
For those who stop by this page. This helped me to solve the problem.
Try to the following:
Go to your Google and remove the SSH key for the server
Go to your google cloud console -> compute engine -> Metadata -> "SSH
keys" tab and click on edit. Here you can delete the ssh keys.
Run the gcloud command again
Click on the "Instances" link on the left side of your google cloud account, which will list down all the instances on the right side. Under
connect column, you will see "SSH" drop-down, click on "View cloud
Command" and this will bring a new dialog. Copy that command and run on your PC's terminal. This will let you SSH into the google compute engine.
It seems a feature/issue from Google Cloud Platform itself, we are going to continue checking it.
If the default network was edited, or if not using the default network, you may need to explicitly enable ssh access by adding a firewall-rule:
$ gcloud compute firewall-rules create --network=YOUR_NETWORK \
default-allow-ssh --allow tcp:22
After that, retry the 'gcloud compute ssh' command.
This is a real problem with very little documentation to dealing with it.
Sometime after creating the instance using the gcloud sdk ssh snippet provided via GCP console stopped working and continually errors with 255 making connecting to ssh on the instance only available through browser via GCP console for the compute instance in question. Not to mention this has happened to me on many different instances some without touching the default account permissions after initial setup and deployment which is overly frustrating. Cause for no reason it just stops working...works, then doesn't...
The only thing that worked for me was creating a new user to connect with through gcloud sdk! Be it Windows/PowerShell or Linux locally, using the following snippet:
gcloud compute ssh newuser-name#instance-name
That all per GCP documentation here: https://cloud.google.com/compute/docs/troubleshooting/troubleshooting-ssh
Everything else passed per suggestions in documentation - port 22 open with access meaning it has to be a a problem with the default users authorization_keys WHICH they provide absolutely no documentation on how to fix that - at least nothing I could find on fixing (not creating or deleting)
I've tried updating the account, tried deleting the user and credentials from the instance, nothing appears to work. using:
gcloud compute --project "project-name" ssh --zone "us-east4-a" "instance-name"
Just doesn't work...
- even tried 'gcloud compute config-ssh --force-key-file-overwrite' NOTHING WORKS...
But creating a new user works every time, and once the user is created you can keep using that user via gcloud sdk
It's a work around, and I hate work around's for things like this but for my sanity this works at least until I can figure out how to reset the default account permissions, so if anyone has any ideas there or can point me in a direction for that I'd more than appreciate it!
IT was my mistake stating that the default firewall would allow all connections into an instance. The contrary turned out to be true. Please refer to an appropriate firewall rule must be set up to allow connection into an instance
Anh-
If you have Identity-Aware Proxy (IAP) enabled for your setup, try adding the --tunnel-through-iap option to the gcloud compute ssh command.
$ gcloud compute ssh --zone <zone> --project <project> --tunnel-through-iap <instance-name>
More information for people landing on this page, if you're using preemptible instances to save some compute costs, that could also be the reason for getting kicked out like this. Your instance may have just randomly stopped.
In my case, the I had created a bootable disk for the VM without adding the information of what source-image it needs to have. Because of this, even though the instance was coming up alright and ssh-allow rule was there, the VM was not booting up.
Finally added the source image to the disk and I was able to ssh into the VM.
Hope this helps for someone.
I had the same error . i restarted the VM instance and ssh workis fine
I had the problem where after clicking on the SSH button it would keep trying to establish a connection and fail. After long struggle I resolved it by adding Service Account User role to myself. If your account was created after the VM instance was created, it might result in this situation.
I know this was opened a long time ago, but for a more recent update on this topic. I had the same trouble connecting via ssh. It was giving the error code 225. Obviously there was a connectivity issue. There was already a firewall rule set under VPC network-> Firewall to allow ssh. However, to fix this problem I had to go to the specific network and create a rule under the network Firewall Rules. VPC network details -> FIREWALL RULES and create an inbound TCP rule for port 22.
if you are having a problem trying to access you g-cloud VM instance from your computer terminal remotely, and are getting the error code 255,the problem is that the ssh protocols in your computer are wrong or not updated.
In this case the best way to fix it is to go to your home directory (in your computer) check the hidden files and find the folder ".ssh" .Just delete this folder and re-open your bash terminal. Then run again your gcloud vm command.
Example:
you#your_computer:~$ gcloud beta compute ssh --zone "us-central1-a" "your_VM_name" --project "your_project_name"
You should this time instead of getting the error 255 code, the messages below:
WARNING: The private SSH key file for gcloud does not exist.
WARNING: The public SSH key file for gcloud does not exist.
WARNING: You do not have an SSH key for gcloud.
WARNING: SSH keygen will be executed to generate a key.
This tool needs to create the directory [/home/your_name/.ssh] before being able to
generate SSH keys.
Do you want to continue (Y/n)?
Type "Y" and gcloud will setup the new protocols by creating a brand new updated .ssh file.
After that you should be able to access your VM with your gcloud command without any problem.
That should solve the problem
Cheers
https://blackpearlmatrix.com
had the exact same symptoms - in my case the reason appeared to be the following. I was using root user + ssh key whereas root login is by default disabled in /etc/ssh/sshd_config (PermitRootLogin property).
I eventually had to delete my instance and make a new one with the same disk. See https://cloud.google.com/compute/docs/troubleshooting/troubleshooting-ssh#use_your_disk_on_a_new_instance for details.
For me, my other teammates were able to login into the machine, but not me. So I asked them to create a user of my name with sudo rights, logged into serial console and changed passwordAuthentication to yes followed by sudo service ssh restart (for few this could be sudo service sshd restart.)
Post this I was able to login with
ssh -o PreferredAuthentications=password username#publicIP -p 22
This trick worked fine for me.
Reinitializing the gcloud with "gcloud init" and generating new ssh keys resolved the problem for me.
I had same issue.
I had connected the serial control and had checked logs. and there was some error log like "there is no disk space". Then I had resized disk as written in this document.
Now I am able to connect to instance with ssh.
Try switching to a different Internet connection
So, I was getting the same error but in my case I was not able to log in to the instance at all.
(base) girish#girish:~$ gcloud beta compute ssh --zone "asia-east1-b" "fp-1" --project "fp-public"
ssh: connect to host 12.345.678.90 port 22: Resource temporarily unavailable
ERROR: (gcloud.beta.compute.ssh) [/usr/bin/ssh] exited with return code [255].
(base) girish#girish:~$ gcloud beta compute ssh --ssh-flag='-vvv' --zone "asia-east1-b" "fp-1" --project "fp-public"
OpenSSH_7.6p1 Ubuntu-4ubuntu0.3, OpenSSL 1.0.2n 7 Dec 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug2: resolving "12.345.678.90" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to 12.345.678.90 [12.345.678.90] port 22.
[debug1: connect to address 12.345.678.90 port 22: Resource temporarily unavailable
ssh: connect to host 12.345.678.903 port 22: Resource temporarily unavailable
ERROR: (gcloud.beta.compute.ssh) [/usr/bin/ssh] exited with return code [255].
What worked for me:
I tried reinstalling lots of things and re-initializing various config and then landed on a thread which suggest to change the Internet network you are using and it worked!!
It's possible you have a rule that only allows whiltelisted IPs to ssh into a gcloud VM. So you may have forgotten to enable your work VPN or out of your work's office IP.
Try restarting your computer.
I got the same error and tried gcloud config ssh as mentioned previously to no avail. I then checked that the IDs and roles of serviceaccount and developer had 'editor' permissions, and that was fine. I started a new instance and logged out of all of my other google accounts and it still threw the error. Then, I restarted my computer and did not log back into my other google accounts. That fixed it.
When using IAP, GCP stores the key in instance metadata and then propagate
that to the ~/.ssh/authorized_keys file.
You might get the error OP talks about when you remove the key from the ~/.ssh/authorized_keys file and it's still in the instance metadata. Reason being:
GCP check that the user, key combo that you are using to ssh is already in the instance metadata.
It assumes that the exists in the ~/.ssh/authorized_keys file for that user and doesn't propagate the key.
As the key doesn't exist in ~/.ssh/authorized_keys file for whatever reason (you deleted it, someone else deleted it etc. etc.) - you get access denied.
If this is the case with you, then fix is simple: remove the instance metadata entry for that user, key combo (have attached an image for ref, just click X and remove your faulty key) and try ssh again
What worked for me was turning my firewall on. (On a Mac, ssh'ing into a gcp instance).
In another instance of the error, my connection worked fine when I was on ethernet, but not when I was on wifi. Switching back to ethernet allowed me to connect again.
In my case sorted out the issue after restarting the VM.
if you are able to access the VM previously and suddenly giving SSH issues, give it a try by restarting.
Permission wise check whether you have IAP-secured Tunnel User
gcloud compute ssh --zone "your_zone" "instance_name" --tunnel-through-iap --project "project_name"
If this not works check with the GCP built-in SSH client, and click open in browser window.
Hope this help !!!

Access Denied when executing through cygwin openssh

When I execute the command "iisreset" through an ssh terminal on a remote windows machine, I get the following error:
Attempting stop...
Restart attempt failed.
Access denied, you must be an administrator of the remote computer to use this
command. Either have your account added to the administrator local group of
the remote computer or to the domain administrator global group.
When I type whoami, it shows that I am the administrator. My cygwin ssh session is running as the "cyg_server" user who has admin privileges.
My ssh server is configured with privilege separation and allows me to login as administrator.
When I run the command locally, it works fine. The problem is execution through ssh.
I've also used process monitor to see what's going on, but it does not indicate the problem.
That is pretty strange because I am able to do admin-only operations in remote ssh such as:
echo "hi">/cygdrive/c/x.txt
rm /cygdrive/c/x.txt
Turning off UAC did not make a difference.
Any ideas?
I had a similar problem: unable to start/stop services using net start/net stop from a remote password-less (public/private key) SSH user. Attempting to start/stop the service was resulting in a "System Error 5 has occurred. Access is denied." error).
I had to install Cygwin's LSA authentication package (see http://cygwin.com/cygwin-ug-net/ntsec.html#ntsec-setuid-overview) in order for (I presume) setuid to work properly for password-less logins.
The problem should go away once LSA is installed on the Cygwin/SSH host and the machine has been rebooted.
I got scared of the LSA package mentioned in #user3609241's answer because of this sentence in the LSA docs:
as soon as the LSA encounters serious problems (for instance, one of
the protected LSA processes died), it triggers a system reboot.
But, those same docs point to a very easy way to "runas" SYSTEM - just use the at command:
$ date
Mon, Jan 12, 2015 8:17:35 PM
$ at 20:18 iisreset
Added a new job with job ID = 1
$ at
Status ID Day Time Command Line
-------------------------------------------------------------------------------
1 Today 8:18 PM iisreset
It works, at the cost of having to wait up to 59 seconds.
(wrapping the above sequence of commands in a simple-to-call script is left as an exercise to the reader; our management util is written in Perl so it was pretty straightforward).
Run the Cygwin terminal as administrator