when I'm trying to start jenkins slave (windows) I get this error:
[07/07/15 12:54:15] [SSH] Opening SSH connection to pcskala:22105.
[07/07/15 12:54:15] [SSH] Authentication successful.
[07/07/15 12:54:15] [SSH] The remote users environment is:
Unable to execute command or shell on remote system: Failed to Execute process.
[07/07/15 12:54:15] [SSH] Starting sftp client.
[07/07/15 12:54:15] [SSH] SFTP failed. Copying via SCP.
hudson.util.IOException2: Could not copy slave.jar into 'c:\Users\jenkins' on slave
at hudson.plugins.sshslaves.SSHLauncher.copySlaveJarUsingSCP(SSHLauncher.java:1065)
at hudson.plugins.sshslaves.SSHLauncher.copySlaveJar(SSHLauncher.java:1024)
at hudson.plugins.sshslaves.SSHLauncher.access$300(SSHLauncher.java:133)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:709)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:696)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.io.IOException: Sorry, this connection is closed.
at com.trilead.ssh2.transport.TransportManager.ensureConnected(TransportManager.java:587)
at com.trilead.ssh2.transport.TransportManager.sendMessage(TransportManager.java:660)
at com.trilead.ssh2.channel.ChannelManager.openSessionChannel(ChannelManager.java:572)
at com.trilead.ssh2.Session.<init>(Session.java:42)
at com.trilead.ssh2.Connection.openSession(Connection.java:1129)
at com.trilead.ssh2.Connection.exec(Connection.java:1551)
at hudson.plugins.sshslaves.SSHLauncher.copySlaveJarUsingSCP(SSHLauncher.java:1048)
... 8 more
Caused by: java.net.SocketException: Socket closed
at java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:121)
at java.net.SocketOutputStream.write(SocketOutputStream.java:159)
at com.trilead.ssh2.crypto.cipher.CipherOutputStream.flush(CipherOutputStream.java:75)
at com.trilead.ssh2.transport.TransportConnection.sendMessage(TransportConnection.java:193)
at com.trilead.ssh2.transport.TransportConnection.sendMessage(TransportConnection.java:107)
at com.trilead.ssh2.transport.TransportManager.sendMessage(TransportManager.java:677)
at com.trilead.ssh2.channel.ChannelManager.closeChannel(ChannelManager.java:304)
at com.trilead.ssh2.Session.close(Session.java:565)
at com.trilead.ssh2.Connection.exec(Connection.java:1568)
at hudson.plugins.sshslaves.SSHLauncher.reportEnvironment(SSHLauncher.java:1071)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:704)
... 5 more
[07/07/15 12:54:15] Launch failed - cleaning up connection
[07/07/15 12:54:15] [SSH] Connection closed.
Windows slave is running SSH and SFTP servers. Keys should be set fine, because I can connect to that machine via ssh and copy files via scp and sftp as well without entering password (from Unix system). I apologize that I don't post any more info, but I do not have direct access to slaves configuration in jenkins.
I don't know if this is important, but slave.jar file is located in home folder: C:\Users\jenkins.
I have found a thread in jenkins issues, where one had the similar issue and he resolved it with changing permission to /dev/null, but I don't think (just a thought) this would be the reason.
Do you have any ideas how to fix this?
Thank you
Related
Connecting with CLI in default way towards new Jenkins instance hangs.
java -jar $JENKINS_HOME/war/WEB-INF/jenkins-cli.jar -s http://git-dapp902.XXXXXXXX:17801 -auth admin:aaaaaaa who-am-i
(aaaaaa = initial password for admin account)
Following actions were performed
- startup new Jenkins instance
- log on as admin with initial password
- enable global security JLNP on fixed port (without this, I would get a Connection Refused)
- restart the jenkins server
- launch the above java command (on the same Linux VM)
- client does not return. No logging on Jenkins server
- When client is canceled below logging is shown
java -jar $JENKINS_HOME/war/WEB-INF/jenkins-cli.jar -s http://git-dapp902.XXXXXXXX:17801 -auth admin:aaaaaaa who-am-i
WARNING: Connection #4 failed
java.io.EOFException
at java.base/java.io.DataInputStream.readFully (DataInputStream.java:202)
at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:614)
at java.base/java.io.DataInputStream.readUTF(DataInputStream.java:569)
at hudson.TcpSlaveAgentListener$ConnectionHandler.run(TcpSlaveAgentListener.java:254)
In the meantime I found the solution.
The Jenkins URL does not need to point to the port in JLNP setting, but to the "plain" port.
Luc
I get a permissions denied but don't know why. From my jenkins master I was able to run the following command using ssh-rsa-key
scp /var/cache/jenkins/war/WEB-INF/slave.jar jenkins#<my_slave_host>:/var/jenkins/
Note: I did manually create /var/jenkins/ on the my slave host when i saw it didn't exist and made it owned by the jenkins user. My master jenkins is configured to use the jenkins#mySlaveHost using .ssh keys.
Any ideas why I'm getting a permissions denied? What is it trying to do?
Here's the log from master jenkins after clicking [Lauch slave agent] button:
[02/27/15 15:18:01] [SSH] Opening SSH connection to <my_slave_host>:22.
[02/27/15 15:18:02] [SSH] Authentication successful.
[02/27/15 15:18:03] [SSH] The remote users environment is:
BASH=/bin/bash
BASHOPTS=cmdhist:complete_fullquote:extquote:force_fignore:hostcomplete:interactive_comments:progcomp:promptvars:sourcepath
BASH_ALIASES=()
BASH_ARGC=()
BASH_ARGV=()
BASH_CMDS=()
BASH_EXECUTION_STRING=set
BASH_LINENO=()
BASH_SOURCE=()
BASH_VERSINFO=([0]="4" [1]="3" [2]="11" [3]="1" [4]="release" [5]="x86_64-pc-linux-gnu")
BASH_VERSION='4.3.11(1)-release'
CATALINA_HOME=/opt/tomcat/current
DIRSTACK=()
EUID=107
GROUPS=()
HOME=/var/lib/jenkins
HOSTNAME=*********** REMOVED***********
HOSTTYPE=x86_64
IFS=$' \t\n'
JAVA_HOME=/usr/lib/jvm/java-7-oracle
LANG=en_US.UTF-8
LOGNAME=jenkins
MACHTYPE=x86_64-pc-linux-gnu
MAIL=/var/mail/jenkins
OPTERR=1
OPTIND=1
OSTYPE=linux-gnu
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games
PIPESTATUS=([0]="0")
PPID=10592
PS4='+ '
PWD=/var/lib/jenkins
SHELL=/bin/bash
SHELLOPTS=braceexpand:hashall:interactive-comments
SHLVL=1
SSH_CLIENT='10.57.13.32 34436 22'
SSH_CONNECTION='10.57.13.32 34436 10.57.6.42 22'
TERM=dumb
UID=107
USER=jenkins
XDG_RUNTIME_DIR=/run/user/107
XDG_SESSION_ID=42
_=']'
[02/27/15 15:18:03] [SSH] Checking java version of java
[02/27/15 15:18:04] [SSH] java -version returned 1.7.0_76.
[02/27/15 15:18:04] [SSH] Starting sftp client.
[02/27/15 15:18:04] [SSH] Copying latest slave.jar...
hudson.util.IOException2: Could not copy slave.jar into '/var/jenkins' on slave
at hudson.plugins.sshslaves.SSHLauncher.copySlaveJar(SSHLauncher.java:1019)
at hudson.plugins.sshslaves.SSHLauncher.access$300(SSHLauncher.java:133)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:709)
at hudson.plugins.sshslaves.SSHLauncher$2.call(SSHLauncher.java:696)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: hudson.util.IOException2: Could not copy slave.jar to '/var/jenkins/slave.jar' on slave
at hudson.plugins.sshslaves.SSHLauncher.copySlaveJar(SSHLauncher.java:1016)
... 7 more
Caused by: com.trilead.ssh2.SFTPException: Permission denied (SSH_FX_PERMISSION_DENIED: The user does not have sufficient permissions to perform the operation.)
at com.trilead.ssh2.SFTPv3Client.openFile(SFTPv3Client.java:1201)
at com.trilead.ssh2.SFTPv3Client.createFile(SFTPv3Client.java:1074)
at com.trilead.ssh2.SFTPv3Client.createFile(SFTPv3Client.java:1055)
at hudson.plugins.sshslaves.SFTPClient.writeToFile(SFTPClient.java:93)
at hudson.plugins.sshslaves.SSHLauncher.copySlaveJar(SSHLauncher.java:1008)
... 7 more
[02/27/15 15:18:04] Launch failed - cleaning up connection
[02/27/15 15:18:04] [SSH] Connection closed.
Edit:
Here's /var/jenkins on the slave:
$ ls -al
total 436
drwxr-xr-x 2 jenkins jenkins 22 Feb 27 15:17 .
drwxr-xr-x 14 root root 4096 Feb 27 15:12 ..
-rw-r--r-- 1 jenkins jenkins 439584 Feb 27 15:17 slave.jar
As for SFTP, I do not think it is enabled, can you point me to any docs that says SFTP is a pre-requisite for a slave? All the pages I've seen do not mention SFTP.
It looks like the problem is tied to your Remote root directory setting. That needs to be the location of the slave.jar, as Jenkins will try to execute it from there.
As for the permissions, the Remote Root Directory (whatever you set it to) needs to be configured to allow Jenkins to access it.
Therefore, if you change your Remote root directory setting to be /var/jenkins/ in your case, it should launch the Jenkins slave successfully.
Granting
sudo chmod -R 777 /var/lib/jenkins
works for me
sudo chmod -R 777 /var/jenkins
Make sure that the location's permissions where the jar needs to be copied is as the logged in user (i.e. jenkins).
check for permisison using:
ls -l directory_name
Most probably you'll find another owner, so change the owner with:
chown -R username:username directory_name
That worked for me !
For anyone with an external drive, check that it's mounted correct:
drwxrwxrwx+ 2 App admin 68 Aug 25 19:33 Jenkins_Support
drwxrwxr-x 19 App staff 714 Sep 25 10:46 Jenkins_Support 1
This might be a problem
If you changed a user to connect to slave, please also make sure slave destination directory is empty (not containing slave.jar copied there by previous user).
This is kind of stupid, but costed me a time.
In ubuntu terminal check
service ufw status
if active
service ufw stop
In redhat terminal check
service iptables status
if active
service iptables stop
service ip6tables status
if active
service ip6tables stop
then check jenkins-slave-node status
I have jenkins running on master.com, and want to have a slave running on slave.com. However, to ssh to the slave I need to go through gateway.com. Generally to ssh to this machine from my normal account, I just use ~/.ssh/config to set up a ProxyCommand.
I have replicated this setup in my /var/lib/jenkins/.ssh/config file:
Host slave.com
User felix
ProxyCommand ssh felix#gateway.com nc %h %p
I have public key authentication set up for both the gateway and the slave, such that from the command line I can ssh directly from jenkins#master.com to felix#slave.com simply by doing ssh slave.com.
Unfortunately Jenkins does not seem to respect my .ssh/config setup, and the connection times out (the slave is not reachable directly). The Jenkins slave log file is:
java.io.IOException: There was a problem while connecting to slave.com:22
....
Caused by: java.net.ConnectException: Connection timed out
How can I figure out whether or not jenkins is respecting my .ssh/config file? Am I missing a step in configuring the master jenkins account or the .ssh/config file for jenkins?
Instead of using Jenkin's built in SSH implementation, use the "Launch slave by execution of command on the Master". You can then use the regular ssh command, and take advantage of .ssh/config like you're used to. If you click the ? button next to the option, it should give you all the details you need.
The jenkins-ssh-slaves plugin uses the trilead SSH2 implementation written in Java.
~/.ssh/config is only read by OpenSSH, and won't affect how Jenkins connects to your slaves.
Summary
Connecting to ActiveMQ via TCP works fine with or without netlib.
Connecting via SSL works fine without netlib but fails with netlib.
Note: I'm using apache-activemq-5.6.0 and silvertunnel.org_netlib-0.14-beta.
This problem occurs whether using netlib's tcpipNetLayer or torNetLayer. However, only the tcpipNetLayer case is considered here to keep things simple.
BTW, SSL + netlib worked fine with the HornetQ messaging server (which implements SSL authentication differently) but HornetQ had problems that led me to try using ActiveMQ.
For convenience I have created a simple test case with a consumer and two different producers.
producer connects without netlib - works fine.
producer-tcpip connects with neltib using the tcpipNetLayer - the following errors occur:
With SSL debugging turned on for ActiveMQ the producer-tcpip program outputs to the console the following error message and then hangs:
ActiveMQ Transport: ssl://null:0, handling exception: java.lang.NullPointerException
This is midway through an SSL handshake because the producer-tcpip program generates the log4j message:
DEBUG org.apache.activemq.transport.failover.FailoverTransport - Attempting 0th connect to: ssl://localhost:61616
DEBUG org.apache.activemq.transport.WireFormatNegotiator - Sending: WireFormatInfo { version=9, properties={MaxFrameSize=104857600, CacheSize=1024, CacheEnabled=true, SizePrefixDisabled=false, MaxInactivityDurationInitalDelay=10000, TcpNoDelayEnabled=true, MaxInactivityDuration=30000, TightEncodingEnabled=true, StackTraceEnabled=true}, magic=[A,c,t,i,v,e,M,Q]}
Then when the producer-tcpip program is killed (e.g. with a ctrl+C) then the activemq.log reports that:
WARN | Transport Connection to: tcp://127.0.0.1:49463 failed: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: ssl:///127.0.0.1:49463
ERROR | Could not accept connection from tcp://127.0.0.1:49463: javax.net.ssl.SSLHandshakeException: Remote host closed connection during handshake | org.apache.activemq.broker.TransportConnector | ActiveMQ Task-3
I have no idea what could be causing this or how it can be resolved. Can anyone help?
Steps to reproduce the problem (takes under 15 minutes):
download the test case
wget http://anandavala.info/miscl/testing-AMQ-Netlib-SSL.zip
unzip testing-AMQ-Netlib-SSL.zip
rm testing-AMQ-Netlib-SSL.zip
cd testing-AMQ-Netlib-SSL
Download latest ActiveMQ into the testing directory
wget https://www.apache.org/dist/activemq/apache-activemq/5.6.0/apache-activemq-5.6.0-bin.tar.gz
tar -xzf apache-activemq-5.6.0-bin.tar.gz
rm apache-activemq-5.6.0-bin.tar.gz
Edit conf/activemq.xml to accept ssl
gedit apache-activemq-5.6.0/conf/activemq.xml
replace the transportConnectors entry with the following lines (to enable SSL)
<transportConnectors>
<transportConnector name="ssl" uri="ssl://0.0.0.0:61616?needClientAuth=true"/>
</transportConnectors>
<sslContext>
<sslContext
keyStore="broker.ks" keyStorePassword="password"
trustStore="client.ks" trustStorePassword="password"/>
</sslContext>
start server
cd apache-activemq-5.6.0
bin/activemq start
view logging output
tail -f data/activemq.log
open another console then cd into the testing-AMQ-Netlib-SSL directory
Download latest SilverTunnel netlib into the testing directory
wget http://sourceforge.net/projects/silvertunnel/files/silvertunnel_Netlib/silvertunnel.org_netlib-0.14-beta.zip
unzip silvertunnel.org_netlib-0.14-beta.zip
rm silvertunnel.org_netlib-0.14-beta.zip
compile the source code into three runnable jar files in separate directories
ant
run the consumer
cd consumer
./consumer
open another console then cd into the testing-AMQ-Netlib-SSL/producer directory
run the producer
./producer
let it run for a bit to satisfy yourself that it works then kill the producer by hitting ctrl+C
run the producer-tcpip
cd ../producer-tcpip
./producer-tcpip
let it run for a bit to satisfy yourself that it has stalled at a NullPointerException then kill the producer-tcpip by hitting ctrl+C
notice the debug message in producer-tcpip's output
tail -f logs/output.log
also notice the error message that appears at the bottom of activemq.log (see the tail output from the first console window) or type at the current prompt
tail -f ../apache-activemq-5.6.0/data/activemq.log
Thanks for your help!
Trying to setup replication from gerrit to my github repo. Getting this stacktrace:
[2011-09-20 00:11:58,470] ERROR com.google.gerrit.server.git.PushReplication : Cannot replicate to git#github.com:myuser/myrepo.git
org.eclipse.jgit.errors.TransportException: git#github.com:myuser/myrepo.git: Session.connect: java.net.SocketTimeoutException: Read timed out
at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:138)
at org.eclipse.jgit.transport.SshTransport.getSession(SshTransport.java:121)
at org.eclipse.jgit.transport.TransportGitSsh$SshFetchConnection.(TransportGitSsh.java:248)
at org.eclipse.jgit.transport.TransportGitSsh.openFetch(TransportGitSsh.java:147)
at com.google.gerrit.server.git.PushOp.listRemote(PushOp.java:358)
at com.google.gerrit.server.git.PushOp.generateUpdates(PushOp.java:312)
at com.google.gerrit.server.git.PushOp.pushVia(PushOp.java:258)
at com.google.gerrit.server.git.PushOp.runImpl(PushOp.java:213)
at com.google.gerrit.server.git.PushOp.run(PushOp.java:166)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
at java.util.concurrent.FutureTask.run(FutureTask.java:138)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:98)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:206)
at com.google.gerrit.server.git.WorkQueue$Task.run(WorkQueue.java:324)
at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
at java.lang.Thread.run(Thread.java:662)
Caused by: com.jcraft.jsch.JSchException: Session.connect: java.net.SocketTimeoutException: Read timed out
at com.jcraft.jsch.Session.connect(Session.java:504)
at org.eclipse.jgit.transport.JschConfigSessionFactory.getSession(JschConfigSessionFactory.java:128)
My config files are as follows:
replication.config:
[remote "github"]
url = git#github.com:myuser/${name}.git
push = +refs/heads/*:refs/heads/*
push = +refs/tags/*:refs/tags/*
timeout = 5
replicationDelay = 0
authGroup = Administrators
secure.config:
[database]
password = secret
[ssh]
file = /home/gerrit2/.ssh/config
.ssh/config:
Host github.com:
IdentityFile ~/.ssh/id_rsa
PreferredAuthentications publickey
I tried running
ssh -i etc/ssh_host_dsa_key git#github.com
ssh -i etc/ssh_host_rsa_key git#github.com
ssh -i ~/.ssh/id_rsa git#github.com
ssh -i ~/.ssh/id_dsa git#github.com
and was able to "hit" github so the keys are imported OK, not sure what the problem is,
any suggestions appreciated.
As the name of the exception SocketTimeoutException says (and its documentation details):
Signals that a timeout has occurred on a socket read or accept.
Looking in the documentation of Gerrit's replication.config:
remote.<name>.timeout
Number of seconds to wait for a network read or write to complete before giving up and declaring the remote side
is not responding. If 0, there is no timeout, and the push client waits indefinitely.
A timeout should be large enough to mostly transfer the objects to the other side. 1 second may be too small
for larger projects, especially over a WAN link, while 10-30 seconds is a much more reasonable timeout value.
Defaults to 0 seconds, wait indefinitely.
Your value is 5 seconds, you could try it with a larger value instead.
(The problem is only indirectly related to JSch - JSch's Session.connect(int) will simply take the timeout parameter and pass it on to the Socket it creates. That reminds me that I should document which unit this timeout is for JSch - seconds or milliseconds.)