Cannot ssh to compute engine after suggested reboot - ssh

The last time I ssh into my compute engine instance, it suggested a system reboot. I rebooted it. After the reboot, I can no longer ssh into it. The instance is still running, the web server is still working. I can even get authentication succeeded. It than it close the connection. The log is a below.
ssh -v -i "privatekey" username#hostname
OpenSSH_7.2p2 Ubuntu-4ubuntu2.4, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to hostname [ip] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file [key location] type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/keyfile-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to hostname:22 as 'username'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: Server host key
debug1: Host 'hostname' is known and matches the ECDSA host key.
debug1: Found key in /home/username/.ssh/known_hosts:2
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: privatekeyfile
Enter passphrase for key 'privatekeyfile':
debug1: Authentication succeeded (publickey).
Authenticated to hostname ([ip]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions#openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: channel 0: free: client-session, nchannels 1
**Connection to hostname closed by remote host.**
**Connection to hostname closed.**
Transferred: sent 2256, received 1380 bytes, in 0.0 seconds
Bytes per second: sent 155684.5, received 95232.6
debug1: Exit status -1

Based on the debug output you have provided, I have found two resolved issues with similar output. Here are the two links found 1st link and 2nd link. They were essentially resolved by changing the ~/.ssh permissions.

Related

Google Cloud Platform VM unable to SSH from local (Permission denied (publickey))

I tried to ssh to Google Cloud VM from My MacBook terminal, which worked just fine yesterday.
I have my public keys and private keys the way it should be from my understandings.
When I try to connect as bellow,
XXXXXXXX-MacBook-Pro:~ xxxxxxxx$ ssh -i .ssh/id_rsa xxxxxxxx#12.345.678.910 -v
The error debug goes as bellow. (User names, IP Address etc. are masked)
OpenSSH_8.1p1, LibreSSL 2.7.3
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 47: Applying options for *
debug1: Connecting to 12.345.678.910 [12.345.678.910] port 22.
debug1: Connection established.
debug1: identity file .ssh/id_rsa type 0
debug1: identity file .ssh/id_rsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4p1 Debian-10+deb9u7
debug1: match: OpenSSH_7.4p1 Debian-10+deb9u7 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug1: Authenticating to 12.345.678.910:22 as 'xxxxxxxx'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:yyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyyy
debug1: Host '12.345.678.910' is known and matches the ECDSA host key.
debug1: Found key in /Users/xxxxxxxx/.ssh/known_hosts:1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: .ssh/id_rsa RSA SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx explicit
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-xxxxx,ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-xxxxxx,ecdsa-sha2-xxxxxx>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering public key: .ssh/id_rsa RSA SHA256:xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx explicit
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
xxxxxxxxxx#12.345.678.910: Permission denied (publickey).
The "Server host key" and the "Offering public key: .ssh/id_rsa RSA" was different. However I copy & pasted .ssh/id_rsa.pub from my local and the authorized_keys in Google Cloud VM had the same public key attached.
What causes this and how can I fix this?

git clone via ssh from gitlab leads to "Permission denied (publickey)"

If created a new ssh-key, copied and pasted the public-key to the gitlab project and created a conifg-file in ~/.ssh ...
Still I can't clone the project (I'm not the owner, but I was added as user)
That's what I try (of course I'm using the correct path):
git clone -v git#gitlab.foo.bar.com:bla/blub.git
this gives me
cloning to 'blub' ...
Permission denied (publickey).
fatal: couldn't read from Remote-Repository
Then I tried the following
ssh -vT -i ~/.ssh/gitlab git#gitlab.foo.bar.com:bla
which gives me
OpenSSH_7.2p2, OpenSSL 1.0.2j-fips 26 Sep 2016
debug1: Reading configuration data /user/me/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: Connecting to gitlab.com [52.167.219.168] port 22.
debug1: Connection established.
debug1: identity file /user/me/.ssh/gitlab type 1
debug1: key_load_public: No such file or directory
debug1: identity file /user/me/.ssh/gitlab-cert type -1
debug1: identity file /user/me/.ssh/gitlab type 1
debug1: key_load_public: No such file or directory
debug1: identity file /user/me/.ssh/gitlab-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to gitlab.com:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256#libssh.org need=64 dh_need=64
debug1: kex: curve25519-sha256#libssh.org need=64 dh_need=64
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw
debug1: Host 'gitlab.com' is known and matches the ECDSA host key.
debug1: Found key in /user/me/.ssh/known_hosts:26
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /user/me/.ssh/gitlab
debug1: Authentications that can continue: publickey
debug1: Offering RSA public key: /user/me/.ssh/gitlab
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).
I really don't get it...Cloning and pulling via http works pefectly fine.
SSH doesn't.
Any hints?
EDIT: the config-file in ~/.ssh looks like this (as suggested here)
Host gitlab.foo.bar.com
RSAAuthentication yes
IdentityFile ~/.ssh/gitlab
the private key is named gitlab and the public one is gitlab.pub
EDIT2:
Permission for ~/.ssh is set to 700
and for ~/.ssh/* set to 600
Try the following. This has worked for me.
eval "$(ssh-agent -s)"
ssh-add

SSH Key authorization for user

Scenario :
There are multiple users who need SSH into a server. What I'm following is
Create user1 , user2 in the server (Ubuntu 14.04 Server) running in AWS EC2.
Add each user public key (id_rsa.pub) into server /home/$USER/.ssh/authorized_keys file
Problem is I only can SSH to ubuntu (Default user) with the private key of the user. But I can not SSH into user1 user.
Sample
[root#rubi-pc Downloads]# ssh -i ~/.ssh/id_rsa ruwan#DummyIP -v
OpenSSH_7.5p1, OpenSSL 1.1.0e 16 Feb 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to DummyIP [DummyIP] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.5
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.8
debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.8 pat OpenSSH_6.6.1* compat 0x04000000
debug1: Authenticating to DummyIP:22 as 'ruwan'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:0kqNAt37ILO4cCP8ioiLN9j9x9badLOGL+etIvSYzfs
debug1: Host 'DummyIP' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:2
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /root/.ssh/id_rsa
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).
Please do let me know if any details requeired .
You have got the filename wrong. It is not
/home/$USER/.ssh/authorize_key
but
/home/$USER/.ssh/authorized_keys
Problem Occurred because of .ssh/authorized_keys not chowned to $USER and groups.
also chmod -rw-rw-r-- ( 664 ) .

ssh slow before Authentications can continue

I've been googling on this problem for a while now, but I have not found a solution that works.
The problem is SSH login to my server is very slow all of a sudden. I can see it takes about 10 seconds before Authentications than can continue, here is my ssh verbose log:
OpenSSH_7.3p1, LibreSSL 2.4.1
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: Connecting to 54.86.29.78 [54.86.29.78] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file .ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file .ssh/id_rsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.3
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.4
debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2.4 pat OpenSSH_6.6.1* compat 0x04000000
debug1: Authenticating to 54.86.29.78:22 as 'perfeyhe'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:g/REeuiZutP7YHv6WJR/vhm4kROJQ+9e0ODdd1Tuvpc
debug1: Host '54.86.29.78' is known and matches the ECDSA host key.
debug1: Found key in /Users/admin/.ssh/known_hosts:3
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_SERVICE_ACCEPT received
----------------------------------
takes about 10s
----------------------------------
debug1: Authentications that can continue: publickey,password,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Trying private key: .ssh/id_rsa
debug1: Authentication succeeded (publickey).
Authenticated to 54.86.29.78 ([54.86.29.78]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions#openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: Sending environment.
debug1: Sending env LANG = zh_CN.UTF-8
Welcome to Ubuntu 14.04.3 LTS (GNU/Linux 3.13.0-74-generic x86_64)
and attachment is what I saw from wireshark:
enter image description here
The delay is on server so you can not see anything important in the client log nor in client side wireshark dump.
There are several possibilities which can cause this, but most probably there is UseDNS enabled in server sshd_config. Try to disable it (write UseDNS no to /etc/ssh/sshd_config), restart sshd service (service sshd restart) and try again.
If it will not help, set DEBUG3 LogLevel in the same file, restart and see the server logs where it "hangs" and what does it wait for.
I solved this issue as follows:
systemctl restart systemd-logind
Found this in an answer by #HBruijn in this thread: https://serverfault.com/questions/707377/slow-ssh-login-activation-of-org-freedesktop-login1-timed-out.

Launched new instance on EC2 with same keypair, can't ssh

This is on AWS. I tried to launch a new Ubuntu instance (#2) with the same keypair that is associated with another Ubuntu instance (#1).
AWS instances are associated with keypairs at the time they are launched and cannot be changed once launched.
I can connect to instance #1 but can't connect to instance #2 using the same ssh command. I've checked my permissions and everything seems to be ok.
The key is in ~/AMI.pem.
I have chmod 600 on my home directory and chmod 400 on ~/AMI.pem, so it's only readable/writeable by me.
The instances are nearly identical, with the same security groups and keypairs, same version of Ubuntu, etc. Instance #1 has some additional packages installed. I verified both are associated with the same keypair called "AMI".
Connecting to #2:
USAU9900:ScrapeEdgar reedn$ ssh -i ~/AMI.pem -v ubuntu#54.67.18.96
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/reedn/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: Connecting to 54.67.18.96 [54.67.18.96] port 22.
debug1: Connection established.
debug1: identity file /Users/reedn/AMI.pem type -1
debug1: identity file /Users/reedn/AMI.pem-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.2
debug1: match: OpenSSH_6.2 pat OpenSSH*
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5-etm#openssh.com none
debug1: kex: client->server aes128-ctr hmac-md5-etm#openssh.com none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA 36:69:f6:95:b4:3c:f0:77:02:ab:02:01:af:c7:cc:6a
debug1: Host '54.67.18.96' is known and matches the RSA host key.
debug1: Found key in /Users/reedn/.ssh/known_hosts:1
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/reedn/AMI.pem
debug1: read PEM private key done: type RSA
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
*Permission denied (publickey).*
Connecting to instance #1:
USAU9900:ScrapeEdgar reedn$ ssh -i ~/AMI.pem -v ubuntu#54.183.77.100
OpenSSH_6.2p2, OSSLShim 0.9.8r 8 Dec 2011
debug1: Reading configuration data /Users/reedn/.ssh/config
debug1: Reading configuration data /etc/ssh_config
debug1: /etc/ssh_config line 20: Applying options for *
debug1: Connecting to 54.183.77.100 [54.183.77.100] port 22.
debug1: Connection established.
debug1: identity file /Users/reedn/AMI.pem type -1
debug1: identity file /Users/reedn/AMI.pem-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_6.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.6.1p1 Ubuntu-2ubuntu2
debug1: match: OpenSSH_6.6.1p1 Ubuntu-2ubuntu2 pat OpenSSH*
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: server->client aes128-ctr hmac-md5-etm#openssh.com none
debug1: kex: client->server aes128-ctr hmac-md5-etm#openssh.com none
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<1024<8192) sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_GROUP
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug1: expecting SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: RSA a6:53:a0:99:b8:61:ef:fe:05:d3:fe:47:90:ea:de:b6
The authenticity of host '54.183.77.100 (54.183.77.100)' can't be established.
RSA key fingerprint is a6:53:a0:99:b8:61:ef:fe:05:d3:fe:47:90:ea:de:b6.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '54.183.77.100' (RSA) to the list of known hosts.
debug1: ssh_rsa_verify: signature correct
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: Roaming not allowed by server
debug1: SSH2_MSG_SERVICE_REQUEST sent
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/reedn/AMI.pem
debug1: read PEM private key done: type RSA
debug1: Authentication succeeded (publickey).
Authenticated to 54.183.77.100 ([54.183.77.100]:22)
First you need make sure the keypair has been applied to instance #2. Something like below in aws console on this instance:
Key pair name: AMI (should be same as instance #1)
Then try below steps:
If this ec2 instance is not in VPC, could you stop and start the instance to refresh with new public IP address and try again?
clean content in .ssh/known_hosts in your local machine, and try again.
One of them should work for you.