Java with Jsch: SSH using public key authentication - ssh

Question

So I had asked this question before but it was closed as I could not include code in time. So here it is again.
Required:
I am using Jsch v 0.1.55 for SFTP operation. I want to connect with remote server without passing password or server's key.
Setup
I setup the environment using ssh-copy-id for the remote server and providing password once. I can then login from shell using SSH without providing password or key. I wanted to do the same from Jsch but I keep getting com.jcraft.jsch.JSchException: Auth fail error.
Code
The latest code that I am using not after looking through other questions is:
jsch.addIdentity("/root/.ssh/id_rsa");
session = jsch.getSession(username, remoteserver, 22);
session.setServerAliveInterval(30000);
session.setConfig("PreferredAuthentications", "publickey,password");
session.setConfig("StrictHostKeyChecking", "no");
session.connect(30000);
channel = (ChannelSftp) session.openChannel("sftp");
channel.connect();
I get the exception com.jcraft.jsch.JSchException: Auth fail when doing session.connect()
I also enabled Jsch logging and get following output (a part of it is copied here)
kex: server->client aes128-ctr hmac-md5 none
kex: client->server aes128-ctr hmac-md5 none
SSH_MSG_KEXDH_INIT sent
expecting SSH_MSG_KEXDH_REPLY
ssh_rsa_verify: signature true
Permanently added '<IP>' (RSA) to the list of known hosts.
SSH_MSG_NEWKEYS sent
SSH_MSG_NEWKEYS received
SSH_MSG_SERVICE_REQUEST sent
SSH_MSG_SERVICE_ACCEPT received
Authentications that can continue: publickey,password
Next authentication method: publickey
Authentications that can continue: password
Next authentication method: password
Disconnecting from <IP> port 22
I also tried to print some additional information from session before calling session.connect():
getHost = <IP>
getUserName = <USER>
getHostKey = null
getHostKeyAlias = null
getHostKeyRepository = []
Maybe things are not loaded correctly?
What I have tried
I had tried different ways using the jsch.addIdentity() (here I provide path to local host's private key) as well as jsch.setKnownHosts() (here I provide the path to known hosts file on local host)
When I added the known hosts using the jsch.setKnownHosts() I atleast get some different output for Jsch logs:
Host <IP> is known and matches the RSA host key
SSH_MSG_NEWKEYS sent
SSH_MSG_NEWKEYS received
SSH_MSG_SERVICE_REQUEST sent
SSH_MSG_SERVICE_ACCEPT received
Authentications that can continue: publickey,password
Next authentication method: publickey
Authentications that can continue: password
Next authentication method: password
Disconnecting from <IP> port 22
Failed to connect session towards <IP>: Auth fail
This alteast matches somewhat when I tried to use SSH from shell with verbose output until the part where it fails.
I have also changed files permission to 400 but not working.
Is there anything specific required or any additional steps required here?
SSH verbose output
ssh -vvv -i /root/.ssh/id_rsa root#<IP>
OpenSSH_7.4p1, OpenSSL 1.0.2k-fips 26 Jan 2017
debug1: Reading configuration data /etc/ssh/ssh_config
debug1: /etc/ssh/ssh_config line 58: Applying options for *
debug2: resolving "<IP>" port 22
debug2: ssh_connect_direct: needpriv 0
debug1: Connecting to <IP> [<IP>] port 22.
debug1: Connection established.
debug1: permanently_set_uid: 0/0
debug1: identity file /root/.ssh/id_rsa type 1
debug1: key_load_public: No such file or directory
debug1: identity file /root/.ssh/id_rsa-cert type -1
debug1: Enabling compatibility mode for protocol 2.0
debug1: Local version string SSH-2.0-OpenSSH_7.4
debug1: Remote protocol version 2.0, remote software version Sun_SSH_2.4
debug1: no match: Sun_SSH_2.4
debug2: fd 3 setting O_NONBLOCK
debug1: Authenticating to <IP>:22 as 'root'
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /root/.ssh/known_hosts:3
debug3: load_hostkeys: loaded 1 keys from <IP>
debug3: order_hostkeyalgs: prefer hostkeyalgs: ssh-rsa-cert-v01#openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa
debug3: send packet: type 20
debug1: SSH2_MSG_KEXINIT sent
debug3: receive packet: type 20
debug1: SSH2_MSG_KEXINIT received
debug2: local client KEXINIT proposal
debug2: KEX algorithms: curve25519-sha256,curve25519-sha256#libssh.org,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group-exchange-sha256,diffie-hellman-group16-sha512,diffie-hellman-group18-sha512,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha256,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1,ext-info-c
debug2: host key algorithms: ssh-rsa-cert-v01#openssh.com,rsa-sha2-512,rsa-sha2-256,ssh-rsa,ecdsa-sha2-nistp256-cert-v01#openssh.com,ecdsa-sha2-nistp384-cert-v01#openssh.com,ecdsa-sha2-nistp521-cert-v01#openssh.com,ssh-ed25519-cert-v01#openssh.com,ssh-dss-cert-v01#openssh.com,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521,ssh-ed25519,ssh-dss
debug2: ciphers ctos: chacha20-poly1305#openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm#openssh.com,aes256-gcm#openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: ciphers stoc: chacha20-poly1305#openssh.com,aes128-ctr,aes192-ctr,aes256-ctr,aes128-gcm#openssh.com,aes256-gcm#openssh.com,aes128-cbc,aes192-cbc,aes256-cbc
debug2: MACs ctos: umac-64-etm#openssh.com,umac-128-etm#openssh.com,hmac-sha2-256-etm#openssh.com,hmac-sha2-512-etm#openssh.com,hmac-sha1-etm#openssh.com,umac-64#openssh.com,umac-128#openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: MACs stoc: umac-64-etm#openssh.com,umac-128-etm#openssh.com,hmac-sha2-256-etm#openssh.com,hmac-sha2-512-etm#openssh.com,hmac-sha1-etm#openssh.com,umac-64#openssh.com,umac-128#openssh.com,hmac-sha2-256,hmac-sha2-512,hmac-sha1
debug2: compression ctos: none,zlib#openssh.com,zlib
debug2: compression stoc: none,zlib#openssh.com,zlib
debug2: languages ctos:
debug2: languages stoc:
debug2: first_kex_follows 0
debug2: reserved 0
debug2: peer server KEXINIT proposal
debug2: KEX algorithms: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
debug2: host key algorithms: ssh-rsa,ssh-dss
debug2: ciphers ctos: aes128-ctr,aes192-ctr,aes256-ctr,arcfour128,arcfour256,arcfour
debug2: ciphers stoc: aes128-ctr,aes192-ctr,aes256-ctr,arcfour128,arcfour256,arcfour
debug2: MACs ctos: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha2-256-96,hmac-sha2-512-96,hmac-sha1-96,hmac-md5,hmac-md5-96
debug2: MACs stoc: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha2-256-96,hmac-sha2-512-96,hmac-sha1-96,hmac-md5,hmac-md5-96
debug2: compression ctos: none,zlib#openssh.com,zlib
debug2: compression stoc: none,zlib#openssh.com,zlib
debug2: languages ctos: de-DE,en-US,es-ES,fr-FR,it-IT,ja-JP,ko-KR,pt-BR,zh-CN,zh-TW,i-default
debug2: languages stoc: de-DE,en-US,es-ES,fr-FR,it-IT,ja-JP,ko-KR,pt-BR,zh-CN,zh-TW,i-default
debug2: first_kex_follows 0
debug2: reserved 0
debug1: kex: algorithm: diffie-hellman-group-exchange-sha256
debug1: kex: host key algorithm: ssh-rsa
debug1: kex: server->client cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug1: kex: client->server cipher: aes128-ctr MAC: hmac-sha2-256 compression: none
debug1: kex: diffie-hellman-group-exchange-sha256 need=32 dh_need=32
debug1: kex: diffie-hellman-group-exchange-sha256 need=32 dh_need=32
debug3: send packet: type 34
debug1: SSH2_MSG_KEX_DH_GEX_REQUEST(1024<8192<8192) sent
debug3: receive packet: type 31
debug1: got SSH2_MSG_KEX_DH_GEX_GROUP
debug2: bits set: 4155/8192
debug3: send packet: type 32
debug1: SSH2_MSG_KEX_DH_GEX_INIT sent
debug3: receive packet: type 33
debug1: got SSH2_MSG_KEX_DH_GEX_REPLY
debug1: Server host key: ssh-rsa SHA256:<KEY>
debug3: hostkeys_foreach: reading file "/root/.ssh/known_hosts"
debug3: record_hostkey: found key type RSA in file /root/.ssh/known_hosts:3
debug3: load_hostkeys: loaded 1 keys from <IP>
debug1: Host '<IP>' is known and matches the RSA host key.
debug1: Found key in /root/.ssh/known_hosts:3
debug2: bits set: 4090/8192
debug3: send packet: type 21
debug2: set_newkeys: mode 1
debug1: rekey after 4294967296 blocks
debug1: SSH2_MSG_NEWKEYS sent
debug1: expecting SSH2_MSG_NEWKEYS
debug3: receive packet: type 21
debug1: SSH2_MSG_NEWKEYS received
debug2: set_newkeys: mode 0
debug1: rekey after 4294967296 blocks
debug2: key: imported-openssh-key (0x557eb3021580), agent
debug2: key: /root/.ssh/id_rsa (0x557eb301b710), explicit
debug3: send packet: type 5
debug3: receive packet: type 6
debug2: service_accept: ssh-userauth
debug1: SSH2_MSG_SERVICE_ACCEPT received
debug3: send packet: type 50
debug3: receive packet: type 51
debug1: Authentications that can continue: gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive
debug3: start over, passed a different list gssapi-keyex,gssapi-with-mic,publickey,password,keyboard-interactive
debug3: preferred gssapi-keyex,gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_lookup gssapi-keyex
debug3: remaining preferred: gssapi-with-mic,publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-keyex
debug1: Next authentication method: gssapi-keyex
debug1: No valid Key exchange context
debug2: we did not send a packet, disable method
debug3: authmethod_lookup gssapi-with-mic
debug3: remaining preferred: publickey,keyboard-interactive,password
debug3: authmethod_is_enabled gssapi-with-mic
debug1: Next authentication method: gssapi-with-mic
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available (default cache: KEYRING:persistent:0)
debug1: Unspecified GSS failure. Minor code may provide more information
No Kerberos credentials available (default cache: KEYRING:persistent:0)
debug2: we did not send a packet, disable method
debug3: authmethod_lookup publickey
debug3: remaining preferred: keyboard-interactive,password
debug3: authmethod_is_enabled publickey
debug1: Next authentication method: publickey
debug1: Offering RSA public key: imported-openssh-key
debug3: send_pubkey_test
debug3: send packet: type 50
debug2: we sent a publickey packet, wait for reply
debug3: receive packet: type 60
debug1: Server accepts key: pkalg ssh-rsa blen 279
debug2: input_userauth_pk_ok: fp SHA256:<PUBKEY>
debug3: sign_and_send_pubkey: RSA SHA256:<PUBKEY>
debug3: send packet: type 50
debug3: receive packet: type 52
debug1: Authentication succeeded (publickey).
Authenticated to <IP> ([<IP>]:22).
debug1: channel 0: new [client-session]
debug3: ssh_session2_open: channel_new: 0
debug2: channel 0: send open
debug3: send packet: type 90
debug1: Entering interactive session.
debug1: pledge: network
debug3: receive packet: type 91
debug2: callback start
debug2: fd 3 setting TCP_NODELAY
debug3: ssh_packet_set_tos: set IP_TOS 0x10
debug2: client_session2_setup: id 0
debug2: channel 0: request pty-req confirm 1
debug3: send packet: type 98
debug1: Sending environment.
debug3: Ignored env XDG_SESSION_ID
debug3: Ignored env HOSTNAME
debug3: Ignored env SELINUX_ROLE_REQUESTED
debug3: Ignored env TERM
debug3: Ignored env SHELL
debug3: Ignored env HISTSIZE
debug3: Ignored env SSH_CLIENT
debug3: Ignored env SELINUX_USE_CURRENT_RANGE
debug3: Ignored env OLDPWD
debug3: Ignored env SSH_TTY
debug3: Ignored env USER
debug3: Ignored env LS_COLORS
debug3: Ignored env SSH_AUTH_SOCK
debug3: Ignored env MAIL
debug3: Ignored env PATH
debug3: Ignored env PWD
debug3: Ignored env EDITOR
debug1: Sending env LANG = en_US.UTF-8
debug2: channel 0: request env confirm 0
debug3: send packet: type 98
debug3: Ignored env PS1
debug3: Ignored env SELINUX_LEVEL_REQUESTED
debug3: Ignored env HISTCONTROL
debug3: Ignored env SHLVL
debug3: Ignored env HOME
debug3: Ignored env LOGNAME
debug3: Ignored env SSH_CONNECTION
debug3: Ignored env LESSOPEN
debug3: Ignored env XDG_RUNTIME_DIR
debug3: Ignored env _
debug2: channel 0: request shell confirm 1
debug3: send packet: type 98
debug2: callback done
debug2: channel 0: open confirm rwindow 0 rmax 32768
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: PTY allocation request accepted on channel 0
debug3: receive packet: type 4
debug1: Remote: Channel 0 set: LANG=en_US.UTF-8
debug2: channel 0: rcvd adjust 1025184
debug3: receive packet: type 99
debug2: channel_input_status_confirm: type 99 id 0
debug2: shell request accepted on channel 0
Last login: Thu Nov 4 13:15:05 2021 from <LOCALHOST>
Oracle Corporation SunOS 5.11 11.3 September 2017
You have new mail.
Jsch log output
Connecting to <IP> port 22
Connection established
Remote version string: SSH-2.0-Sun_SSH_2.4
Local version string: SSH-2.0-JSCH-0.1.54
CheckCiphers: aes256-ctr,aes192-ctr,aes128-ctr,aes256-cbc,aes192-cbc,aes128-cbc,3des-ctr,arcfour,arcfour128,arcfour256
CheckKexes: diffie-hellman-group14-sha1,ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521
CheckSignatures: ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
SSH_MSG_KEXINIT sent
SSH_MSG_KEXINIT received
kex: server: diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group14-sha1,diffie-hellman-group1-sha1
kex: server: ssh-rsa,ssh-dss
kex: server: aes128-ctr,aes192-ctr,aes256-ctr,arcfour128,arcfour256,arcfour
kex: server: aes128-ctr,aes192-ctr,aes256-ctr,arcfour128,arcfour256,arcfour
kex: server: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha2-256-96,hmac-sha2-512-96,hmac-sha1-96,hmac-md5,hmac-md5-96
kex: server: hmac-sha2-256,hmac-sha2-512,hmac-sha1,hmac-sha2-256-96,hmac-sha2-512-96,hmac-sha1-96,hmac-md5,hmac-md5-96
kex: server: none,zlib#openssh.com,zlib
kex: server: none,zlib#openssh.com,zlib
kex: server: de-DE,en-US,es-ES,fr-FR,it-IT,ja-JP,ko-KR,pt-BR,zh-CN,zh-TW,i-default
kex: server: de-DE,en-US,es-ES,fr-FR,it-IT,ja-JP,ko-KR,pt-BR,zh-CN,zh-TW,i-default
kex: client: ecdh-sha2-nistp256,ecdh-sha2-nistp384,ecdh-sha2-nistp521,diffie-hellman-group14-sha1,diffie-hellman-group-exchange-sha256,diffie-hellman-group-exchange-sha1,diffie-hellman-group1-sha1
kex: client: ssh-rsa,ssh-dss,ecdsa-sha2-nistp256,ecdsa-sha2-nistp384,ecdsa-sha2-nistp521
kex: client: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-ctr,aes192-cbc,aes256-ctr,aes256-cbc
kex: client: aes128-ctr,aes128-cbc,3des-ctr,3des-cbc,blowfish-cbc,aes192-ctr,aes192-cbc,aes256-ctr,aes256-cbc
kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96
kex: client: hmac-md5,hmac-sha1,hmac-sha2-256,hmac-sha1-96,hmac-md5-96
kex: client: none
kex: client: none
kex: client:
kex: client:
kex: server->client aes128-ctr hmac-md5 none
kex: client->server aes128-ctr hmac-md5 none
SSH_MSG_KEXDH_INIT sent
expecting SSH_MSG_KEXDH_REPLY
ssh_rsa_verify: signature true
Permanently added '<IP>' (RSA) to the list of known hosts.
SSH_MSG_NEWKEYS sent
SSH_MSG_NEWKEYS received
SSH_MSG_SERVICE_REQUEST sent
SSH_MSG_SERVICE_ACCEPT received
Authentications that can continue: publickey,password
Next authentication method: publickey
Authentications that can continue: password
Next authentication method: password
Disconnecting from <IP> port 22

Answer 1

Ok I found the issue.
The issue was how the keys were copied. The copy-ssh-key should be executed with -i flag to use the key in identity file.
When looking at authorized key on remote server, I saw that the key copied was different from what I had for the user in local server. So after copying the key again with -i flag, I was able to login as required through Jsch.