SSH to BAstion ec2-user#localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic) - ssh

I am trying to connect to my AWS RDS cluster via an EC2 bastion. I first exceute this line to establish an ssm-session to the bastion:
jameseaston#Jamess-MacBook-Pro code % aws ssm start-session \
--target i-09d38c1******* \
--document-name AWS-StartPortForwardingSession \
--parameters '{"portNumber":["22"], "localPortNumber":["9995"]}'
Starting session with SessionId: qa-prod-0149f4e1c8aaaa10b
Port 9995 opened for sessionId qa-prod-0149f4e1c8aaaa10b.
Waiting for connections...
Connection accepted for session [qa-prod-0149f4e1c8aaaa10b]
This works fine.
I then need to create an ssh connection over this ssm session:
jameseaston#Jamess-MacBook-Pro code % ssh -i ~/.ssh/notifications_bastion_production.pem ec2-user#localhost \
-p 9995 \
-N \
-L 3389:qa-portal-production-notifications-rds.cluster-crb*********.eu-west-2.rds.amazonaws.com:5432 -v
This part is failing with the error message:
ec2-user#localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).
The output is below.
Having read quite a few posts its worth noting:
I have checked the sshd_config on the bastion and this is configured the same as our test and staging boxes and these work
There is no password on the ssh key
Any help much appreciated!
OpenSSH_8.6p1, LibreSSL 3.3.6
debug1: Reading configuration data /Users/jameseaston/.ssh/config
debug1: /Users/jameseaston/.ssh/config line 1: Applying options for *
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 21: include /etc/ssh/ssh_config.d/* matched no files
debug1: /etc/ssh/ssh_config line 54: Applying options for *
debug1: Authenticator provider $SSH_SK_PROVIDER did not resolve; disabling
debug1: Connecting to localhost port 9995.
debug1: Connection established.
debug1: identity file /Users/jameseaston/.ssh/notifications_bastion_production.pem type -1
debug1: identity file /Users/jameseaston/.ssh/notifications_bastion_production.pem-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.6
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.4
debug1: compat_banner: match: OpenSSH_7.4 pat OpenSSH_7.0*,OpenSSH_7.1*,OpenSSH_7.2*,OpenSSH_7.3*,OpenSSH_7.4*,OpenSSH_7.5*,OpenSSH_7.6*,OpenSSH_7.7* compat 0x04000002
debug1: Authenticating to localhost:9995 as 'ec2-user'
debug1: load_hostkeys: fopen /Users/jameseaston/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:TJZV4J1RmIzEwIwvDNm3mjll1rrP/f0QxiUD7kb3NeI
debug1: load_hostkeys: fopen /Users/jameseaston/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: checking without port identifier
debug1: load_hostkeys: fopen /Users/jameseaston/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: hostkeys_find_by_key_hostfile: hostkeys file /Users/jameseaston/.ssh/known_hosts2 does not exist
debug1: hostkeys_find_by_key_hostfile: hostkeys file /etc/ssh/ssh_known_hosts does not exist
debug1: hostkeys_find_by_key_hostfile: hostkeys file /etc/ssh/ssh_known_hosts2 does not exist
The authenticity of host '[localhost]:9995 ([127.0.0.1]:9995)' can't be established.
ED25519 key fingerprint is SHA256:TJZV4J1RmIzEwIwvDNm3mjll1rrP/f0QxiUD7kb3NeI.
This key is not known by any other names
Are you sure you want to continue connecting (yes/no/[fingerprint])? yes
Warning: Permanently added '[localhost]:9995' (ED25519) to the list of known hosts.
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /Users/jameseaston/.ssh/notifications_bastion_production.pem explicit
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: Next authentication method: publickey
debug1: Trying private key: /Users/jameseaston/.ssh/notifications_bastion_production.pem
debug1: Authentications that can continue: publickey,gssapi-keyex,gssapi-with-mic
debug1: No more authentication methods to try.
ec2-user#localhost: Permission denied (publickey,gssapi-keyex,gssapi-with-mic).

Related

ssh, scp works, rsync over ssh doesn't

I try to run an rsync sync on a Synology from a linux machine.
I have access to the Synology with SSH with key authentication. SSH SCP works, permissions are correct.
~# scp /tmp/foo.bar backupch:/tmp
foo.bar 100% 9 0.5KB/s 00:00
Rsync fails:
rsync /tmp/foo.bar backupch:/tmp -e 'ssh -v'
Rsync output with debug:
OpenSSH_8.9p1 Ubuntu-3, OpenSSL 3.0.2 15 Mar 2022
debug1: Reading configuration data /root/.ssh/config
debug1: /root/.ssh/config line 14: Applying options for backupch
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: include /etc/ssh/ssh_config.d/*.conf matched no files
debug1: /etc/ssh/ssh_config line 21: Applying options for *
debug1: Connecting to myhost.mydomain [*.*.*.*] port 2222.
debug1: Connection established.
debug1: identity file /root/.ssh/ipdch.backup.key type -1
debug1: identity file /root/.ssh/ipdch.backup.key-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.9p1 Ubuntu-3
debug1: Remote protocol version 2.0, remote software version OpenSSH_8.2
debug1: compat_banner: match: OpenSSH_8.2 pat OpenSSH* compat 0x04000000
debug1: Authenticating to myhost.mydomain:2222 as 'backup'
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256
debug1: kex: host key algorithm: ssh-ed25519
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ssh-ed25519 SHA256:bbl6asZ3Qc+DnOV8hKPDWdlnJtE2yXMQxb1jyInlc+c
debug1: load_hostkeys: fopen /root/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh/ssh_known_hosts2: No such file or directory
debug1: Host '[myhost.mydomain]:2222' is known and matches the ED25519 host key.
debug1: Found key in /root/.ssh/known_hosts:1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /root/.ssh/ipdch.backup.key explicit
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<ssh-ed25519,sk-ssh-ed25519#openssh.com,ssh-rsa,rsa-sha2-256,rsa-sha2-512,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,sk-ecdsa-sha2-nistp256#openssh.com>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,password
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/ipdch.backup.key
Authenticated to myhost.mydomain ([*.*.*.*]:2222) using "publickey".
debug1: channel 0: new [client-session]
debug1: Requesting no-more-sessions#openssh.com
debug1: Entering interactive session.
debug1: pledge: filesystem
debug1: client_input_global_request: rtype hostkeys-00#openssh.com want_reply 0
debug1: client_input_hostkeys: searching /root/.ssh/known_hosts for [myhost.mydomain]:2222 / (none)
debug1: client_input_hostkeys: searching /root/.ssh/known_hosts2 for [myhost.mydomain]:2222 / (none)
debug1: client_input_hostkeys: hostkeys file /root/.ssh/known_hosts2 does not exist
debug1: Remote: /var/services/homes/backup/.ssh/authorized_keys:1: key options: agent-forwarding port-forwarding pty user-rc x11-forwarding
debug1: Sending environment.
debug1: channel 0: setting env LANG = "en_US.UTF-8"
debug1: Sending command: rsync --server -e.LsfxCIvu . /tmp
debug1: client_global_hostkeys_private_confirm: server used untrusted RSA signature algorithm ssh-rsa for key 0, disregarding
debug1: update_known_hosts: known hosts file /root/.ssh/known_hosts2 does not exist
Permission denied, please try again.
debug1: client_input_channel_req: channel 0 rtype exit-status reply 0
debug1: client_input_channel_req: channel 0 rtype eow#openssh.com reply 0
debug1: channel 0: free: client-session, nchannels 1
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
Transferred: sent 3412, received 2716 bytes, in 0.1 seconds
Bytes per second: sent 56899.8, received 45293.1
debug1: Exit status 1
rsync error: error in rsync protocol data stream (code 12) at io.c(228) [sender=3.2.3]
Any idea what is going wrong here??
It seems that there is a problem with newer versions of rsync on Synology.
https://community.synology.com/enu/forum/17/post/114547
The solution is to specify the --rsync-path=/bin/rsync with the rsync command.
rsync --rsync-path=/bin/rsync /tmp/foo.bar backupch:/tmp -e 'ssh -v'

SSH using RSA keys - Permission denied

From MACHINE_2, as USER1, I need to SSH into MACHINE_1 without having to provide a password.
I created a pair of RSA keys (private_key, public_key) using the command C:\cygwin64\bin\ssh-keygen -t rsa.
I have put the two files private_key and public_key in the folder C:\cygwin64\home\USER1\.ssh\machines_1_2\.
On MACHINE_1, I copied the contents of the public_key file into the file D:\cygwin\home\USER1\.ssh\authorized_keys.
On MACHINE_2, I created the file C:\cygwin64\home\USER1\.ssh\config in which I put the following lines:
Host 192.168.77.45
Port 22
IdentityFile ~/.ssh/machines_1_2/private_key
When I try to SSH into MACHINE_1, I get a "Permission denied" message (see the output below).
USER1#MACHINE_2$ ssh -v 192.168.77.45
OpenSSH_8.9p1, OpenSSL 1.1.1m 14 Dec 2021
debug1: Reading configuration data /etc/ssh_config
debug1: Connecting to 192.168.77.45 [192.168.77.45] port 22.
debug1: Connection established.
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_rsa type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_rsa-cert type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_ecdsa type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_ecdsa-cert type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_ecdsa_sk type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_ecdsa_sk-cert type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_ed25519 type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_ed25519-cert type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_ed25519_sk type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_ed25519_sk-cert type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_xmss type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_xmss-cert type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_dsa type -1
debug1: identity file /cygdrive/c/Users/USER1/.ssh/id_dsa-cert type -1
debug1: Local version string SSH-2.0-OpenSSH_8.9
debug1: Remote protocol version 2.0, remote software version OpenSSH_6.7
debug1: compat_banner: match: OpenSSH_6.7 pat OpenSSH* compat 0x04000000
debug1: Authenticating to 192.168.77.45:22 as 'MACHINE_1+USER1'
debug1: load_hostkeys: fopen /cygdrive/c/Users/USER1/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh_known_hosts2: No such file or directory
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: SSH2_MSG_KEX_ECDH_REPLY received
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:qSH/p+CitL49lw4RPcwmVd8P+e02Ot1yjTSagmyiZhA
debug1: load_hostkeys: fopen /cygdrive/c/Users/USER1/.ssh/known_hosts2: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh_known_hosts: No such file or directory
debug1: load_hostkeys: fopen /etc/ssh_known_hosts2: No such file or directory
debug1: Host '192.168.77.45' is known and matches the ECDSA host key.
debug1: Found key in /cygdrive/c/Users/USER1/.ssh/known_hosts:1
debug1: rekey out after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey in after 134217728 blocks
debug1: Will attempt key: /cygdrive/c/Users/USER1/.ssh/id_rsa
debug1: Will attempt key: /cygdrive/c/Users/USER1/.ssh/id_ecdsa
debug1: Will attempt key: /cygdrive/c/Users/USER1/.ssh/id_ecdsa_sk
debug1: Will attempt key: /cygdrive/c/Users/USER1/.ssh/id_ed25519
debug1: Will attempt key: /cygdrive/c/Users/USER1/.ssh/id_ed25519_sk
debug1: Will attempt key: /cygdrive/c/Users/USER1/.ssh/id_xmss
debug1: Will attempt key: /cygdrive/c/Users/USER1/.ssh/id_dsa
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: Next authentication method: publickey
debug1: Trying private key: /cygdrive/c/Users/USER1/.ssh/id_rsa
debug1: Trying private key: /cygdrive/c/Users/USER1/.ssh/id_ecdsa
debug1: Trying private key: /cygdrive/c/Users/USER1/.ssh/id_ecdsa_sk
debug1: Trying private key: /cygdrive/c/Users/USER1/.ssh/id_ed25519
debug1: Trying private key: /cygdrive/c/Users/USER1/.ssh/id_ed25519_sk
debug1: Trying private key: /cygdrive/c/Users/USER1/.ssh/id_xmss
debug1: Trying private key: /cygdrive/c/Users/USER1/.ssh/id_dsa
debug1: Next authentication method: keyboard-interactive
debug1: Authentications that can continue: publickey,keyboard-interactive
debug1: No more authentication methods to try.
MACHINE_1+USER1#192.168.77.45: Permission denied (publickey,keyboard-interactive).
I was wondering if there was not a confusion between the two ".ssh" directories:
C:\cygwin64\home\USER1\.ssh\ and C:\Users\USER1\.ssh\
Personally, I always usually deal with the C:\cygwin64\home\USER1\.ssh\ directory... not the other one.
Also, MACHINE_1 and MACHINE_2 are two "Windows" machines with "Cygwin" installed.
On MACHINE_1, the cygwin version is 3.3.4.
On MACHINE_2, the cygwin version is 1.7.33.
Can you help me investigate what's going wrong?
============= Edit 1 ================
I installed a new Cygwin (v. 3.3.4) on MACHINE_2.
I still have the same problem.
============= Edit 2 ================
It looks like the config file is not been read.
============= Edit 3 ================
I put the config file and the machines_1_2 directory in the C:\Users\USER1\.ssh\ directory instead of the C:\cygwin64\home\USER1\.ssh\ directory.
It allowed me to SSH into MACHINE_1 without having to enter a password.
Until this day, I always used the C:\cygwin64\home\USER1\.ssh\ directory...
As I could understand:
$ set | grep HOME
HOME=/cygdrive/c/Users/USER1
/etc/passwd and /etc/group are no longer automatically generated since version 1.7.34.
In my older Cygwin configuration (the version 1.7.33), the user's home directory was set to /home/USER1 that is to say C:\cygwin64\home\USER1
I guess developers have good reasons for making such changes but for people like me it takes precious time and nerves to fix the problems and that does not make my work more fun or interesting. It makes me feel stupid. My two cents.
Recent version openssh remove rsa support by default, so you should enable it in your ssh config:
$ cat ~/.ssh/config
Host *
HostkeyAlgorithms +ssh-rsa
PubkeyAcceptedKeyTypes +ssh-rsa

GitLab SSH Keys - Permission denied (publickey)

I followed this document to setup my SSH authentication https://kgcoe-git.rit.edu/help/ssh/README#generating-a-new-ssh-key-pair
It has worked before but this time on a new setup I get Permission denied (publickey) error
Below is the debug output from the ssh command
ankushko#node-0:~/Code$ sudo ssh -vT git#pc3.instageni.utc.edu
OpenSSH_7.2p2 Ubuntu-4ubuntu2.4, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to pc3.instageni.utc.edu [150.182.135.23] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to pc3.instageni.utc.edu:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:BUPmhA1nWbH8Zq0cg4uohS3L/Vo4uOZ8PLrdUYf3ano
debug1: Host 'pc3.instageni.utc.edu' is known and matches the RSA host key.
debug1: Found key in /root/.ssh/known_hosts:1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Trying private key: /root/.ssh/id_rsa
debug1: Trying private key: /root/.ssh/id_dsa
debug1: Trying private key: /root/.ssh/id_ecdsa
debug1: Trying private key: /root/.ssh/id_ed25519
debug1: No more authentication methods to try.
Permission denied (publickey).
Keys
ankushko#node-0:~$ sudo ls ~/.ssh/
authorized_keys id_rsa id_rsa.pub known_hosts
Update: Tried without sudo
ankushko#node-0:~$ ssh -vT git#pc3.instageni.utc.edu
OpenSSH_7.2p2 Ubuntu-4ubuntu2.4, OpenSSL 1.0.2g 1 Mar 2016
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to pc3.instageni.utc.edu [150.182.135.23] port 22.
debug1: Connection established.
debug1: identity file /users/ankushko/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /users/ankushko/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to pc3.instageni.utc.edu:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: rsa-sha2-512
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:BUPmhA1nWbH8Zq0cg4uohS3L/Vo4uOZ8PLrdUYf3ano
debug1: Host 'pc3.instageni.utc.edu' is known and matches the RSA host key.
debug1: Found key in /users/ankushko/.ssh/known_hosts:1
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /users/ankushko/.ssh/id_rsa
debug1: Authentications that can continue: publickey
debug1: Trying private key: /users/ankushko/.ssh/id_dsa
debug1: Trying private key: /users/ankushko/.ssh/id_ecdsa
debug1: Trying private key: /users/ankushko/.ssh/id_ed25519
debug1: No more authentication methods to try.
Permission denied (publickey).
Thanks in advance.
I solved git#gitlab.com: Permission denied (publickey) issue by following these steps:
RUN cat ~/.ssh/id_rsa.pub
Copy id_rsa.pub (public key) to your getlab `Setting -> SSH Keys
RUN cat ~/.ssh/id_rsa
Copy id_rsa (private key) to `Code_repo->git_auth->id_rsa (private keys local file)
NOTE: Take care of the machine user if you are using root user in your DockerFile or anywhere else then use sudo su before running the above commands to get root user public and private keys.
The command:
sudo ssh -vT git#pc3.instageni.utc.edu
is running as root and will use /root/.ssh/config and a private ssh key from that same directory, e.g. /root/.ssh/id_rsa. In the command:
sudo ls ~/.ssh/
the shell, probably bash, will expand ~ to your home directory before it even calls sudo (prove this with sudo echo ~), so it will display files from your home directory, probably /home/ankushko.
Can I ask why you are running these command with sudo? Would it work to just connect as a normal user? That would be more secure, but sometimes there is a need to run things as root.
Either way, you need to make sure the private ssh key that you are actually using matches the public ssh key on the server.

Can not ssh login to bitbucket.org

I can't push to bitbucket.org anymore.
I have checked the public key is still at bitbucket.
As suggested elsewhere I tried GIT_SSH_COMMAND='ssh -v' git push
and got this result.
I wonder if some options have changed with the latest update to ssh
Everything seems fine (to me), yet git push fails with Authentication failed.
OpenSSH_7.4p1 Debian-10+deb9u4, OpenSSL 1.0.2l 25 May 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 19: Applying options for *
debug1: Connecting to bitbucket.org [2406:da00:ff00::22cd:e0db] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_dsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ecdsa-cert type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519 type -1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_ed25519-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4p1 Debian-10+deb9u4
debug1: Remote protocol version 2.0, remote software version conker_1.1.15 49a70a8 app-155
debug1: no match: conker_1.1.15-49a70a8 app-155
debug1: Authenticating to bitbucket.org:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ssh-rsa SHA256:zzXQOXSRBEiUtuE8AikJYKwbHaxvSc0ojez9YXaGp1A
debug1: Host 'bitbucket.org' is known and matches the RSA host key.
debug1: Found key in /root/.ssh/known_hosts:13
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: SSH2_MSG_NEWKEYS received
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /root/.ssh/id_rsa
debug1: Server accepts key: pkalg ssh-rsa blen 279
Authentication failed.
fatal: Could not read from remote repository.
Please make sure you have the correct access rights and the repository exists.
```
Even though the public key was already on bitbucket, and the public and private keys seemed to match, using some suggestions here, I made a new keypair, and made a special key for git#bitbucket.org in /etc/.ssh/config
Now I have my access back.

git clone via ssh from gitlab leads to "Permission denied (publickey)"

If created a new ssh-key, copied and pasted the public-key to the gitlab project and created a conifg-file in ~/.ssh ...
Still I can't clone the project (I'm not the owner, but I was added as user)
That's what I try (of course I'm using the correct path):
git clone -v git#gitlab.foo.bar.com:bla/blub.git
this gives me
cloning to 'blub' ...
Permission denied (publickey).
fatal: couldn't read from Remote-Repository
Then I tried the following
ssh -vT -i ~/.ssh/gitlab git#gitlab.foo.bar.com:bla
which gives me
OpenSSH_7.2p2, OpenSSL 1.0.2j-fips 26 Sep 2016
debug1: Reading configuration data /user/me/.ssh/config
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 20: Applying options for *
debug1: Connecting to gitlab.com [52.167.219.168] port 22.
debug1: Connection established.
debug1: identity file /user/me/.ssh/gitlab type 1
debug1: key_load_public: No such file or directory
debug1: identity file /user/me/.ssh/gitlab-cert type -1
debug1: identity file /user/me/.ssh/gitlab type 1
debug1: key_load_public: No such file or directory
debug1: identity file /user/me/.ssh/gitlab-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.2
debug1: Remote protocol version 2.0, remote software version OpenSSH_7.2p2 Ubuntu-4ubuntu2.4
debug1: match: OpenSSH_7.2p2 Ubuntu-4ubuntu2.4 pat OpenSSH* compat 0x04000000
debug1: Authenticating to gitlab.com:22 as 'git'
debug1: SSH2_MSG_KEXINIT sent
debug1: SSH2_MSG_KEXINIT received
debug1: kex: algorithm: curve25519-sha256#libssh.org
debug1: kex: host key algorithm: ecdsa-sha2-nistp256
debug1: kex: server->client cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: client->server cipher: chacha20-poly1305#openssh.com MAC: <implicit> compression: none
debug1: kex: curve25519-sha256#libssh.org need=64 dh_need=64
debug1: kex: curve25519-sha256#libssh.org need=64 dh_need=64
debug1: expecting SSH2_MSG_KEX_ECDH_REPLY
debug1: Server host key: ecdsa-sha2-nistp256 SHA256:HbW3g8zUjNSksFbqTiUWPWg2Bq1x8xdGUrliXFzSnUw
debug1: Host 'gitlab.com' is known and matches the ECDSA host key.
debug1: Found key in /user/me/.ssh/known_hosts:26
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug1: rekey after 134217728 blocks
debug1: SSH2_MSG_NEWKEYS received
debug1: SSH2_MSG_EXT_INFO received
debug1: kex_input_ext_info: server-sig-algs=<rsa-sha2-256,rsa-sha2-512>
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug1: Authentications that can continue: publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: /user/me/.ssh/gitlab
debug1: Authentications that can continue: publickey
debug1: Offering RSA public key: /user/me/.ssh/gitlab
debug1: Authentications that can continue: publickey
debug1: No more authentication methods to try.
Permission denied (publickey).
I really don't get it...Cloning and pulling via http works pefectly fine.
SSH doesn't.
Any hints?
EDIT: the config-file in ~/.ssh looks like this (as suggested here)
Host gitlab.foo.bar.com
RSAAuthentication yes
IdentityFile ~/.ssh/gitlab
the private key is named gitlab and the public one is gitlab.pub
EDIT2:
Permission for ~/.ssh is set to 700
and for ~/.ssh/* set to 600
Try the following. This has worked for me.
eval "$(ssh-agent -s)"
ssh-add