I am trying to execute a simple .bat file in jenkins - selserver.bat (The bat file executes the selenium standalone server) using 'Execute windows batch command'. Under that, in Execute shell, I have my automation test cases that will use the selenium server. But when i run the job, I get this error
New build name is '#Sel_job'
[#Sel_job] $ cmd /c call /tmp/hudson1913007337035643872.bat
FATAL: command execution failed
java.io.IOException: Cannot run program "cmd" (in directory "/var/lib/jenkins/workspace/#Sel_job"): error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1047)
at hudson.Proc$LocalProc.<init>(Proc.java:243)
at hudson.Proc$LocalProc.<init>(Proc.java:212)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:815)
at hudson.Launcher$ProcStarter.start(Launcher.java:381)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:108)
at hudson.tasks.CommandInterpreter.perform(CommandInterpreter.java:65)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:779)
at hudson.model.Build$BuildExecution.build(Build.java:205)
at hudson.model.Build$BuildExecution.doRun(Build.java:162)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:534)
at hudson.model.Run.execute(Run.java:1720)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:43)
at hudson.model.ResourceController.execute(ResourceController.java:98)
at hudson.model.Executor.run(Executor.java:401)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:187)
at java.lang.ProcessImpl.start(ProcessImpl.java:130)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1028)
... 15 more```
What I dont understand is why is jenkins trying to get some hudsonXXXX.bat in the tmp location? because I have only selserver.bat in my config. How should I fix this. Appreciate all answers in advance!
I am using Apache Pig. I am trying to load a comma separated file as a Pig table. It does not throw any error while loading the file.
But when I try to print that table using "dump" command, it gives error.
File I loaded
Error,fdgdf
Error,dfgdf
Error,dfgdf
Info,dfgdf
Info,dfgdf
Info,dfgdf
Info,dfgdf
Info,dfgdf
Info,dfgdf
Debug,dfgdf
Debug,dfgdf
Debug,dfgdf
Debug,dfgdf
Debug,dfgdf
Debug,dfgdf
Command to load
logFile1 = LOAD 'PigTestFile' using PigStorage();
Command to print table
dump logFile1
Error I get
led Jobs:
JobId Alias Feature Message Outputs
job_1454617624671_0152 logFile1 MAP_ONLY Message: org.apache.pig.backend.executionengine.ExecException: ERROR 2118: Input path does not exist: hdfs:
//ip-172-31-53-48.ec2.internal:8020/user/e1681fe26eed362777aabca1682510/PigTestFile
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:279)
at org.apache.hadoop.mapreduce.JobSubmitter.writeNewSplits(JobSubmitter.java:301)
at org.apache.hadoop.mapreduce.JobSubmitter.writeSplits(JobSubmitter.java:318)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:196)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1290)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1287)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1287)
at org.apache.hadoop.mapreduce.lib.jobcontrol.ControlledJob.submit(ControlledJob.java:335)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at org.apache.pig.backend.hadoop23.PigJobControl.submit(PigJobControl.java:128)
at org.apache.pig.backend.hadoop23.PigJobControl.run(PigJobControl.java:194)
at java.lang.Thread.run(Thread.java:745)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher$1.run(MapReduceLauncher.java:276)
Caused by: org.apache.hadoop.mapreduce.lib.input.InvalidInputException: Input path does not exist: hdfs://ip-172-31-53-48.ec2.internal:8020/user/e1681fe26eed362777aabca1682510/PigTestFile
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.singleThreadedListStatus(FileInputFormat.java:323)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.listStatus(FileInputFormat.java:265)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigTextInputFormat.listStatus(PigTextInputFormat.java:36)
at org.apache.hadoop.mapreduce.lib.input.FileInputFormat.getSplits(FileInputFormat.java:387)
at org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.PigInputFormat.getSplits(PigInputFormat.java:265)
... 18 more
hdfs://ip-172-31-53-48.ec2.internal:8020/tmp/temp1258481141/tmp-1928081547,
:
:
2016-02-07 06:31:20,100 [main] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MapReduceLauncher - Failed!
2016-02-07 06:31:20,107 [main] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 1066: Unable to open iterator for alias logFile1. Backend error : java.lang.IllegalStateException: Job in state DEFINE instead of RUNNING
[EDIT]
When I closely read the log I found that it is not able to find the file which was used to load the table. It is expecting it to be in HDFS. Where as my file was on local box.
I then moved the file into HDFS and then ran same commands. It worked well.
But then why did it not give error while executing "Load" command itself ??
As explained by Murali in his answer (which I have accepted) Map/ Reduce jobs for a script will get triggered only when STORE/ DUMP is encountered.
Here is more explanation about it from Apache Pig documentation
In general, Pig processes Pig Latin statements as follows:
First, Pig validates the syntax and semantics of all statements.
Next, if Pig encounters a DUMP or STORE, Pig will execute the statements.
In this example Pig will validate, but not execute, the LOAD and FOREACH statements.
A = LOAD 'student' USING PigStorage() AS (name:chararray, age:int, gpa:float);
B = FOREACH A GENERATE name;
In this example, Pig will validate and then execute the LOAD, FOREACH, and DUMP statements.
A = LOAD 'student' USING PigStorage() AS (name:chararray, age:int, gpa:float);
B = FOREACH A GENERATE name;
DUMP B;
(John)
(Mary)
(Bill)
(Joe)
Map/ Reduce jobs for a script will get triggered only when STORE/ DUMP is encountered.
In this case, Map phase for LOAD command will start only when STORE/ DUMP is encountered in script.
Default execution mode is map reduce. If the file is in local path you have use local mode for execution.
pig -x local {pigfilename.pig}
Refer : https://pig.apache.org/docs/r0.9.1/start.html#execution-modes
Extract from above link :
Pig has two execution modes or exectypes:
Local Mode - To run Pig in local mode, you need access to a single
machine; all files are installed and run using your local host and
file system. Specify local mode using the -x flag (pig -x local).
Mapreduce Mode - To run Pig in mapreduce mode, you need access to a
Hadoop cluster and HDFS installation. Mapreduce mode is the default
mode; you can, but don't need to, specify it using the -x flag (pig OR
pig -x mapreduce).
I have configured a single node cluster and its is running fine unexpectedly I got the below error and it stopped working. All the hadoop paths are configured in their paths but still I'm getting this error how to come out of this.
/opt/hadoop/hadoop/bin/../libexec/hadoop-config.sh: line 24: /bin/basename: cannot execute binary file
/opt/hadoop/hadoop/bin/../libexec/hadoop-config.sh: line 29: /bin/basename: cannot execute binary file
Warning: $HADOOP_HOME is deprecated.
Error: Could not find or load main class org.apache.hadoop.util.PlatformName
Error: Could not find or load main class org.apache.hadoop.fs.FsShell
We are deploying on one of our server and we have below error.
ERROR: tooltwist.fip.FipException Unknown response from server: 500: Internal Server Error
Exception: tooltwist.fip.FipException: tooltwist.fip.FipException: Unknown response from server: 500: Internal Server Error
Looking at the FIP log, it shows:
Error installing batch: tooltwist.fip.FipException: Pre-commit command failed: protected/pre_commit.sh
tooltwist.fip.FipException: Pre-commit command failed: protected/pre_commit.sh
at tooltwist.fip.FipServer_updateExecuter.commitTransaction(FipServer_updateExecuter.java:309)
at tooltwist.fip.FipServer_updateExecuter.prepareUpdates_1_3(FipServer_updateExecuter.java:250)
at tooltwist.fip.FipServer_updateExecuter.executeUpdates(FipServer_updateExecuter.java:142)
at tooltwist.fip.FipServer.destination_installBatchOfFiles(FipServer.java:199)
at tooltwist.fip.jetty.InstallBatchServlet.doPost(InstallBatchServlet.java:134)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
at org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:530)
at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:426)
at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:227)
at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:931)
at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:361)
at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:186)
at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:867)
at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:117)
at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:113)
at org.eclipse.jetty.server.Server.handle(Server.java:337)
at org.eclipse.jetty.server.HttpConnection.handleRequest(HttpConnection.java:581)
at org.eclipse.jetty.server.HttpConnection$RequestHandler.content(HttpConnection.java:1020)
at org.eclipse.jetty.http.HttpParser.parseNext(HttpParser.java:775)
at org.eclipse.jetty.http.HttpParser.parseAvailable(HttpParser.java:228)
at org.eclipse.jetty.server.HttpConnection.handle(HttpConnection.java:417)
at org.eclipse.jetty.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:474)
at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:437)
at java.lang.Thread.run(Thread.java:662)
Any idea about the error?
Couple of suggestions:
1. Does the pre_commit.sh shell script exist on the server.
2. Does it have +x permissions?
If FIP was installed using the normal way, it should not cause an issue.
The fipserver initially saves files it receives on the destination server, but into temporary locations. Once all of the files have been received and saved it runs a three step process to complete the installation:
1. Run a script named protected/pre-commit.sh. The normal operation of this script is to shut down the web server.
2. For each new file:
a) move any existing file to .fip-rollback-xxxxxx/filename.
b) move the new file from it's temporary location to the correct location.
3. Run a script named protected/post-commit.sh. This most commonly restarts the server.
The pre and post commit scripts are user provided. They should should normally exit with a status of zero, as any other status indicates that an error has occurred.
As suggested in the previous answer, check that these scripts exist, and that they are executable. If this fails to solve your problem, insert debug into the scripts to determine where and why they are failing.
I have a problem with Java 1.5.0 for AIX. The error happens just when I log on with specific user on AIX (myuser). When I log on with other user java works ok.
The error come up even when I executed just "java -version" or simply "java" (of course, without quoting). I've tried executing it with the full path: /usr/java5/jre/bin/java but still fails.
There was installed the version 1.4 of java on system too. So the $PATH variable for the user contained /usr/java14/jre/bin, but I removed that value, I even uninstalled that version of java (1.4) so that just java 5 exists on the system, but the error continues.
If I execute "java -fullversion" it doesn't crash.
This is part of the error (the full output is very long):
JVMJ9VM011W Unable to load j9dmp23: No such file or directory
JVMJ9VM011W Unable to load j9jit23: No such file or directory
JVMJ9VM011W Unable to load j9gc23: No such file or directory
JVMJ9VM011W Unable to load j9vrb23: No such file or directory
Unhandled exception
Type=Illegal instruction vmState=0x00000000
J9Generic_Signal_Number=00000010 Signal_Number=00000004 Error_Value=00000000
Signal_Code=0000001e
Handler1=F0719CC8 Handler2=F0714F5C
.....
Target=2_30_20091103_45935_bHdSMr (AIX 5.3)
CPU=ppc (4 logical CPUs) (0x7d0000000 RAM)
JavaVMInitArgs.nOptions=14:
-Xjcl:jclscar_23
-Dcom.ibm.oti.vm.bootstrap.library.path=/usr/java5/jre/bin
-Dsun.boot.library.path=/usr/java5/jre/bin
-Djava.library.path=/usr/java5/jre/bin:/usr/java5/jre/bin:/usr/java5/jre/bin/classic:/usr/java5/jre/bin:/sqllib/lib:/home/myuser/comm:/home/myuser/sys:/home/myuser/bin:/db2util/db2adm/sqllib/lib64:/usr/java5/jre/bin/j9vm:/usr/lib
-Djava.home=/usr/java5/jre
-Djava.ext.dirs=/usr/java5/jre/lib/ext
-Duser.dir=/home/myuser
_j2se_j9=70912 (extra info: F070EA2C)
-Xdump
vfprintf (extra info: 300017A4)
-Dinvokedviajava
-Djava.class.path=/db2util/db2adm/sqllib/java/db2java.zip:/db2util/db2adm/sqllib/java/db2jcc.jar:/db2util/db2adm/sqllib/java/sqlj.zip:/db2util/db2adm/sqllib/function:/db2util/db2adm/sqllib/java/db2jcc_license_cu.jar:.
vfprintf
_port_library (extra info: F070EE30)
Note: "Enable full CORE dump" in smit is set to FALSE and as a result there will be limited threading information in core file.
Note: dump may be truncated if "ulimit -c" is set too low
Generated system dump: {default OS core name}
(no Thread object associated with thread)
(no Thread object associated with thread)
Unhandled exception in signal handler
ksh: 2179192 IOT/Abort trap(coredump)
I found the error. The problem is a line on the .profile which sets the environment variable LIBPATH:
export LIBPATH=/home/myuser/sys
I deleted that line in the .profile and java worked.