ssh, scp works, rsync over ssh doesn't - ssh

I try to run an rsync sync on a Synology from a linux machine.
I have access to the Synology with SSH with key authentication. SSH SCP works, permissions are correct.
~# scp /tmp/foo.bar backupch:/tmp
foo.bar 100% 9 0.5KB/s 00:00
Rsync fails:
rsync /tmp/foo.bar backupch:/tmp -e 'ssh -v'
Rsync output with debug:
OpenSSH_8.9p1 Ubuntu-3, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /root/.ssh/config
debug1: /root/.ssh/config line 14: Applying options for backupch
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to myhost.mydomain [*.*.*.*] port 2222.
debug1: Connection established.
debug1: identity file /root/.ssh/ipdch.backup.key type -1
debug1: identity file /root/.ssh/ipdch.backup.key-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.9p1 Ubuntu-3
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2
debug1: compat_banner: match: OpenSSH_8.2 pat OpenSSH* compat 0x04000000
debug1: Authenticating to myhost.mydomain:2222 as 'backup'
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:bbl6asZ3Qc+DnOV8hKPDWdlnJtE2yXMQxb1jyInlc+c
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '[myhost.mydomain]:2222' is known and matches the ED25519 host key.
debug1: Found key in /root/.ssh/known_hosts:1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /root/.ssh/ipdch.backup.key explicit
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519#openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256#openssh.com>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/ipdch.backup.key
Authenticated to myhost.mydomain ([*.*.*.*]:2222) using "publickey".
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions#openssh.com
debug1: Entering interactive session.
debug1: pledge: filesystem
debug1: client_input_global_request: rtype hostkeys-00#openssh.com want_reply 0
debug1: client_input_hostkeys: searching /root/.ssh/known_hosts for [myhost.mydomain]:2222 / (none)
debug1: client_input_hostkeys: searching /root/.ssh/known_hosts2 for [myhost.mydomain]:2222 / (none)
debug1: client_input_hostkeys: hostkeys file /root/.ssh/known_hosts2 does not exist
debug1: Remote: /var/services/homes/backup/.ssh/authorized_keys:1: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug1: Sending environment.
debug1: channel 0: setting env LANG = "en_US.UTF-8"
debug1: Sending command: rsync --server -e.LsfxCIvu . /tmp
debug1: client_global_hostkeys_private_confirm: server used untrusted RSA signature algorithm ssh-rsa for key 0, disregarding
debug1: update_known_hosts: known hosts file /root/.ssh/known_hosts2 does not exist
Permission denied, please try again.
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow#openssh.com reply 0
debug1: channel 0: free: client-session, nchannels 1
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
Transferred: sent 3412, received 2716 bytes, in 0.1 seconds
Bytes per second: sent 56899.8, received 45293.1
debug1: Exit status 1
rsync error: error in rsync protocol data stream (code 12) at io.c(228) [sender=3.2.3]
Any idea what is going wrong here??

It seems that there is a problem with newer versions of rsync on Synology.
https://community.synology.com/enu/forum/17/post/114547
The solution is to specify the --rsync-path=/bin/rsync with the rsync command.
rsync --rsync-path=/bin/rsync /tmp/foo.bar backupch:/tmp -e 'ssh -v'

Related

SSH to BAstion ec2-user#localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic)

I am trying to connect to my AWS RDS cluster via an EC2 bastion. I first exceute this line to establish an ssm-session to the bastion:
jameseaston#Jamess-MacBook-Pro code % aws ssm start-session \
--target i-09d38c1******* \
--document-name AWS-StartPortForwardingSession \
--parameters '{"portNumber":["22"], "localPortNumber":["9995"]}'
Starting session with SessionId: qa-prod-0149f4e1c8aaaa10b
Port 9995 opened for sessionId qa-prod-0149f4e1c8aaaa10b.
Waiting for connections...
Connection accepted for session [qa-prod-0149f4e1c8aaaa10b]
This works fine.
I then need to create an ssh connection over this ssm session:
jameseaston#Jamess-MacBook-Pro code % ssh -i ~/.ssh/notifications_bastion_production.pem ec2-user#localhost \
-p 9995 \
-N \
-L 3389:qa-portal-production-notifications-rds.cluster-crb*********.eu-west-2.rds.amazonaws.com:5432 -v
This part is failing with the error message:
ec2-user#localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
The output is below.
Having read quite a few posts its worth noting:
I have checked the sshd_config on the bastion and this is configured the same as our test and staging boxes and these work
There is no password on the ssh key
Any help much appreciated!
OpenSSH_8.6p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/jameseaston/.ssh/config
debug1: /Users/jameseaston/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to localhost port 9995.
debug1: Connection established.
debug1: identity file /Users/jameseaston/.ssh/notifications_bastion_production.pem type -1
debug1: identity file /Users/jameseaston/.ssh/notifications_bastion_production.pem-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: compat_banner: match: OpenSSH_7.4 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug1: Authenticating to localhost:9995 as 'ec2-user'
debug1: load_hostkeys: fopen /Users/jameseaston/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:TJZV4J1RmIzEwIwvDNm3mjll1rrP/f0QxiUD7kb3NeI
debug1: load_hostkeys: fopen /Users/jameseaston/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: checking without port identifier
debug1: load_hostkeys: fopen /Users/jameseaston/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: hostkeys_find_by_key_hostfile: hostkeys file /Users/jameseaston/.ssh/known_hosts2 does not exist
debug1: hostkeys_find_by_key_hostfile: hostkeys file /etc/ssh/ssh_known_hosts does not exist
debug1: hostkeys_find_by_key_hostfile: hostkeys file /etc/ssh/ssh_known_hosts2 does not exist
The authenticity of host '[localhost]:9995 ([127.0.0.1]:9995)' can't be established.
ED25519 key fingerprint is SHA256:TJZV4J1RmIzEwIwvDNm3mjll1rrP/f0QxiUD7kb3NeI.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[localhost]:9995' (ED25519) to the list of known hosts.
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /Users/jameseaston/.ssh/notifications_bastion_production.pem explicit
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/jameseaston/.ssh/notifications_bastion_production.pem
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: No more authentication methods to try.
ec2-user#localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

install arch gitlab and can not git clone using ssh method

I followed, https://wiki.archlinux.org/index.php/GitLab#Configuration, and finally brought gitlab it up. But, I could not use ssh to clone the project.
$ ssh -T gitlab#192.168.48.3
Failed to get username: who='' is invalid
What is this message who='' and how to debug???
Using ssh -Tv, I got
$ ssh -T gitlab#192.168.48.3
Failed to get username: who='' is invalid
[alexlai#N2eMMc128G ~]$ ssh -Tv gitlab#192.168.48.3
OpenSSH_8.1p1, OpenSSL 1.1.1d 10 Sep 2019
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: Connecting to 192.168.48.3 [192.168.48.3] port 22.
debug1: Connection established.
debug1: identity file /home/alexlai/.ssh/id_rsa type 0
debug1: identity file /home/alexlai/.ssh/id_rsa-cert type -1
debug1: identity file /home/alexlai/.ssh/id_dsa type -1
debug1: identity file /home/alexlai/.ssh/id_dsa-cert type -1
debug1: identity file /home/alexlai/.ssh/id_ecdsa type -1
debug1: identity file /home/alexlai/.ssh/id_ecdsa-cert type -1
debug1: identity file /home/alexlai/.ssh/id_ed25519 type -1
debug1: identity file /home/alexlai/.ssh/id_ed25519-cert type -1
debug1: identity file /home/alexlai/.ssh/id_xmss type -1
debug1: identity file /home/alexlai/.ssh/id_xmss-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.1
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.1
debug1: match: OpenSSH_8.1 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 192.168.48.3:22 as 'gitlab'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:TbXi96Gynm1xqcTglDvS4ajP7hPHMHf2hnOiJ3w5e74
debug1: Host '192.168.48.3' is known and matches the ECDSA host key.
debug1: Found key in /home/alexlai/.ssh/known_hosts:5
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /home/alexlai/.ssh/id_rsa RSA SHA256:tdOYKwO8CDOzJeJo/6WYbtStpsMg8kuSGLCie5JOY1g
debug1: Will attempt key: /home/alexlai/.ssh/id_dsa
debug1: Will attempt key: /home/alexlai/.ssh/id_ecdsa
debug1: Will attempt key: /home/alexlai/.ssh/id_ed25519
debug1: Will attempt key: /home/alexlai/.ssh/id_xmss
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Offering public key: /home/alexlai/.ssh/id_rsa RSA SHA256:tdOYKwO8CDOzJeJo/6WYbtStpsMg8kuSGLCie5JOY1g
debug1: Server accepts key: /home/alexlai/.ssh/id_rsa RSA SHA256:tdOYKwO8CDOzJeJo/6WYbtStpsMg8kuSGLCie5JOY1g
debug1: Authentication succeeded (publickey).
Authenticated to 192.168.48.3 ([192.168.48.3]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions#openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype hostkeys-00#openssh.com want_reply 0
debug1: Remote: /var/lib/gitlab/.ssh/authorized_keys:2: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug1: Remote: /var/lib/gitlab/.ssh/authorized_keys:2: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
Failed to get username: who='' is invalid
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow#openssh.com reply 0
debug1: channel 0: free: client-session, nchannels 1
Transferred: sent 3468, received 3288 bytes, in 0.1 seconds
Bytes per second: sent 39481.4, received 37432.2
debug1: Exit status 1

GitLab SSH Keys - Permission denied (publickey)

I followed this document to setup my SSH authentication https://kgcoe-git.rit.edu/help/ssh/README#generating-a-new-ssh-key-pair
It has worked before but this time on a new setup I get Permission denied (publickey) error
Below is the debug output from the ssh command
ankushko#node-0:~/Code$ sudo ssh -vT git#pc3.instageni.utc.edu
OpenSSH_7.2p2 Ubuntu-4ubuntu2.4, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to pc3.instageni.utc.edu [150.182.135.23] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to pc3.instageni.utc.edu:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:BUPmhA1nWbH8Zq0cg4uohS3L/Vo4uOZ8PLrdUYf3ano
debug1: Host 'pc3.instageni.utc.edu' is known and matches the RSA host key.
debug1: Found key in /root/.ssh/known_hosts:1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Trying private key: /root/.ssh/id_ecdsa
debug1: Trying private key: /root/.ssh/id_ed25519
debug1: No more authentication methods to try.
Permission denied (publickey).
Keys
ankushko#node-0:~$ sudo ls ~/.ssh/
authorized_keys id_rsa id_rsa.pub known_hosts
Update: Tried without sudo
ankushko#node-0:~$ ssh -vT git#pc3.instageni.utc.edu
OpenSSH_7.2p2 Ubuntu-4ubuntu2.4, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to pc3.instageni.utc.edu [150.182.135.23] port 22.
debug1: Connection established.
debug1: identity file /users/ankushko/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to pc3.instageni.utc.edu:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:BUPmhA1nWbH8Zq0cg4uohS3L/Vo4uOZ8PLrdUYf3ano
debug1: Host 'pc3.instageni.utc.edu' is known and matches the RSA host key.
debug1: Found key in /users/ankushko/.ssh/known_hosts:1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /users/ankushko/.ssh/id_rsa
debug1: Authentications that can continue: publickey
debug1: Trying private key: /users/ankushko/.ssh/id_dsa
debug1: Trying private key: /users/ankushko/.ssh/id_ecdsa
debug1: Trying private key: /users/ankushko/.ssh/id_ed25519
debug1: No more authentication methods to try.
Permission denied (publickey).
Thanks in advance.
I solved git#gitlab.com: Permission denied (publickey) issue by following these steps:
RUN cat ~/.ssh/id_rsa.pub
Copy id_rsa.pub (public key) to your getlab `Setting -> SSH Keys
RUN cat ~/.ssh/id_rsa
Copy id_rsa (private key) to `Code_repo->git_auth->id_rsa (private keys local file)
NOTE: Take care of the machine user if you are using root user in your DockerFile or anywhere else then use sudo su before running the above commands to get root user public and private keys.
The command:
sudo ssh -vT git#pc3.instageni.utc.edu
is running as root and will use /root/.ssh/config and a private ssh key from that same directory, e.g. /root/.ssh/id_rsa. In the command:
sudo ls ~/.ssh/
the shell, probably bash, will expand ~ to your home directory before it even calls sudo (prove this with sudo echo ~), so it will display files from your home directory, probably /home/ankushko.
Can I ask why you are running these command with sudo? Would it work to just connect as a normal user? That would be more secure, but sometimes there is a need to run things as root.
Either way, you need to make sure the private ssh key that you are actually using matches the public ssh key on the server.

Cannot ssh to compute engine after suggested reboot

The last time I ssh into my compute engine instance, it suggested a system reboot. I rebooted it. After the reboot, I can no longer ssh into it. The instance is still running, the web server is still working. I can even get authentication succeeded. It than it close the connection. The log is a below.
ssh -v -i "privatekey" username#hostname
OpenSSH_7.2p2 Ubuntu-4ubuntu2.4, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to hostname [ip] port 22.
debug1: Connection established.
debug1: key_load_public: No such file or directory
debug1: identity file [key location] type -1
debug1: key_load_public: No such file or directory
debug1: identity file /home/keyfile-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to hostname:22 as 'username'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: Server host key
debug1: Host 'hostname' is known and matches the ECDSA host key.
debug1: Found key in /home/username/.ssh/known_hosts:2
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: privatekeyfile
Enter passphrase for key 'privatekeyfile':
debug1: Authentication succeeded (publickey).
Authenticated to hostname ([ip]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions#openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: channel 0: free: client-session, nchannels 1
**Connection to hostname closed by remote host.**
**Connection to hostname closed.**
Transferred: sent 2256, received 1380 bytes, in 0.0 seconds
Bytes per second: sent 155684.5, received 95232.6
debug1: Exit status -1
Based on the debug output you have provided, I have found two resolved issues with similar output. Here are the two links found 1st link and 2nd link. They were essentially resolved by changing the ~/.ssh permissions.

ssh connect do not use identity from config

I am trying to setup ssh connection to gitlab.com. I have corresponding identity file and it works if I use it explicitly. But failed when I expect it read identity file from config.
Here explicit usage:
ssh -v -i /root/.ssh/reshop-deploy -T git#gitlab.com
OpenSSH_7.4p1, OpenSSL 1.0.2k 26 Jan 2017
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 1: Applying options for *
debug1: Connecting to gitlab.com [52.167.219.168] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/reshop-deploy type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/reshop-deploy-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.1
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.1 pat OpenSSH* compat 0x04000000
debug1: Authenticating to gitlab.com:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw
debug1: Host 'gitlab.com' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:36
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/reshop-deploy
debug1: Authentication succeeded (publickey).
Authenticated to gitlab.com ([52.167.219.168]:22).
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions#openssh.com
debug1: Entering interactive session.
debug1: pledge: network
debug1: client_input_global_request: rtype hostkeys-00#openssh.com want_reply 0
debug1: Remote: Forced command.
debug1: Remote: Port forwarding disabled.
debug1: Remote: X11 forwarding disabled.
debug1: Remote: Agent forwarding disabled.
debug1: Remote: PTY allocation disabled.
Welcome to GitLab, XXXX!
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow#openssh.com reply 0
debug1: channel 0: free: client-session, nchannels 1
Transferred: sent 2880, received 2776 bytes, in 0.4 seconds
Bytes per second: sent 6778.9, received 6534.1
debug1: Exit status 0
But if I run it with hope on /root/.ssh/config I got:
ssh -v -T git#gitlab.com
OpenSSH_7.4p1, OpenSSL 1.0.2k 26 Jan 2017
debug1: Reading configuration data /root/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 1: Applying options for *
debug1: Connecting to gitlab.com [52.167.219.168] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.1
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.1 pat OpenSSH* compat 0x04000000
debug1: Authenticating to gitlab.com:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw
debug1: Host 'gitlab.com' is known and matches the ECDSA host key.
debug1: Found key in /root/.ssh/known_hosts:36
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
debug1: Authentications that can continue: publickey
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Trying private key: /root/.ssh/id_ecdsa
debug1: Trying private key: /root/.ssh/id_ed25519
debug1: No more authentication methods to try.
Permission denied (publickey).
.ssh folder list:
ls -al /root/.ssh/
total 60
drwxr-x--- 2 root root 4096 Apr 25 17:54 .
drwx------ 11 root root 4096 Apr 25 18:01 ..
-rw------- 1 root root 5024 Jun 13 2016 authorized_keys
-rw------- 1 root root 193 Apr 25 17:48 config
-rw------- 1 root root 1675 Dec 4 2015 id_rsa
-rw-r--r-- 1 root root 13991 Apr 25 16:59 known_hosts
-rw------- 1 root root 3243 Apr 25 14:40 reshop-deploy
Here it is config content:
cat /root/.ssh/config
Host gitlab
Hostname gitlab.com
User git
IdentityFile ~/.ssh/reshop-deploy
IdentitiesOnly yes
Seems it ignores Identity file, but I don't understand why.
Your ~/.ssh/config configuration begins with Host gitlab. That means that all of the configuration directives after it, up to the next Host line, will only take effect if you refer to the server exactly as "gitlab" on the command line, e.g., ssh gitlab. If you also want your configuration to apply when doing ssh gitlab.com or ssh git#gitlab.com, then you need to change your Host line to Host gitlab gitlab.com.