I want to create a Windows script that runs a Kettle transformation (.ktr).
First, I tried to run my script but it does nothing.
After that, I decided to try it by cmd, but it still is not working. It outputs a ClassNotFoundException:
Exception in thread "main" java.lang.ClassNotFoundException: org.pentaho.di.pan.Pan/file:C:\data-integration\UpdateDBUrgencies.ktr
at java.net.URLClassLoader.findClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at java.lang.ClassLoader.loadClass(Unknown Source)
at org.pentaho.commons.launcher.Launcher.main(Launcher.java:87)
Java HotSpot(TM) 64-Bit Server VM warning: ignoring option MaxPermSize=256m; support was removed in 8.0
The cmd line I run is:
pan.bat /file:"C:\data-integration\UpdateDBUrgencies.ktr" /level:Basic
All I want is to have an script that executes UpdateDBUrgencies.ktr, so it can be run everyday using cron of Windows. How do I have to do it? Where is the problem in my command line?
Thanks for your help.
As a personal experience with .BAT for running .KJB's on windows, i had several issues when using /file or /level, try changing your commandline for:
Pan.bat -file:C:/data-integration/UpdateDBUrgencies.ktr "-level:Basic"
I have begun using this format of command line for all my batches and it's been working without erros, i'm using version 5.3 though, i don't know if there are major differences in command line paramaters and such on newer versions.
Related
What doesn't work
I'm trying to setup WSL2 GUI environment under the latest version Windows Insider Preview. Gedit and Thunar worked fine but when I tried to run GoLand it gave me the following error
Start Failed
Failed to initialize graphics environment
java.awt.AWTError: Can't connect to X11 window server using ':0' as the value of the DISPLAY variable.
at java.desktop/sun.awt.X11GraphicsEnvironment.initDisplay(Native Method)
at java.desktop/sun.awt.X11GraphicsEnvironment$1.run(X11GraphicsEnvironment.java:102)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.desktop/sun.awt.X11GraphicsEnvironment.<clinit>(X11GraphicsEnvironment.java:61)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at java.desktop/java.awt.GraphicsEnvironment$LocalGE.createGE(GraphicsEnvironment.java:101)
at java.desktop/java.awt.GraphicsEnvironment$LocalGE.<clinit>(GraphicsEnvironment.java:83)
at java.desktop/java.awt.GraphicsEnvironment.getLocalGraphicsEnvironment(GraphicsEnvironment.java:129)
at java.desktop/sun.awt.X11.XToolkit.<clinit>(XToolkit.java:233)
at java.base/java.lang.Class.forName0(Native Method)
at java.base/java.lang.Class.forName(Class.java:315)
at java.desktop/java.awt.Toolkit$2.run(Toolkit.java:588)
at java.desktop/java.awt.Toolkit$2.run(Toolkit.java:583)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.desktop/java.awt.Toolkit.getDefaultToolkit(Toolkit.java:582)
at java.desktop/java.awt.Toolkit.getEventQueue(Toolkit.java:1499)
at java.desktop/java.awt.EventQueue.invokeLater(EventQueue.java:1318)
at com.intellij.idea.StartupUtil.lambda$scheduleInitUi$9(StartupUtil.java:347)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:668)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1$1.run(Executors.java:665)
at java.base/java.security.AccessController.doPrivileged(Native Method)
at java.base/java.util.concurrent.Executors$PrivilegedThreadFactory$1.run(Executors.java:665)
at java.base/java.lang.Thread.run(Thread.java:829)
-----
Your JRE: 11.0.11+9-b1341.57 amd64 (JetBrains s.r.o.)
/home/ubuntu/executables/GoLand-2021.1.3/jbr
I'm using WSL2 - Ubuntu-20.04, Windows 11 build 22000.100
What works
start VcXSrv in Windows
$DISPLAY is set to $WINDOWS_IPADDR:0, startxfce4 in wsl2
start GoLand in VcXSrv window
Everything works fine
p.s. I defined a environment variable WINDOWS_IPADDR=$(cat /etc/resolv.conf | grep nameserver | cut -d ' ' -f 2).
What I have tried
I've tried Pycharm and Idea and both of them displayed the same error.
export DISPLAY=localhost:0 && goland.sh gives the same error.
export DISPLAY=0:0 && goland.sh gives the same error.
export DISPLAY=$WINDOWS_IPADDR:0 && goland.sh will not show this error, but nothing happened.
I tried to install a new openjdk-11 which might be different from openjdk-11-jdk-headless but it didn't change the JRE path used by IntelliJ-idea/GoLand/PyCharm, which is always IDE_INSTALL_PATH/jbr, e.g. goland alwayws use jre in /home/ubuntu/executables/GoLand-2021.1.3/jbr
I'm not familiar with X11 or X-Server so I tried searching for similar issues but few is related. I'm think it might be some kind of compatibility issue with WSLg.
Does anyone know how to fix this error? Any and all help/direction is appreciated.
UPDATE:
I install a new distro and everything just magically work now. However, I mistakenly deleted the original distro so I cannot test it anymore. I guess WSLg cannot work very well with gnome & xfce4 as I've installed both desktop environments in WSL. Think twice before you want to install multiple desktops!
Pentaho PDI version 8.3.0 CE if it matters
When I try to run a job or transformation commandline using kitchen or pan respectively I get a nullpointer exception. This happens only when trying to run something from a repository.
When I try to run the same transformation or job from spoon, all is fine and the job runs great.
I use the following commands, which both provide the same error:
./pan.sh -trans=get_clusters -rep=myrepo -user=admin -pass=mypass -dir=/Transformations
and
./kitchen.sh -job=scheduled_update_job -rep=myrepo -user=admin -pass=mypass -dir=/Jobs
NOTE: This error also happens when I try to run the job or transformation from a docker container.
The error I receive is as follows and identical for PAN and Kitchen:
020/02/05 09:07:56 - Pan - Start of run.
Processing has stopped because of an error: null
java.lang.NullPointerException
at org.pentaho.di.core.plugins.PluginRegistry.getPluginId(PluginRegistry.java:689)
at org.pentaho.di.core.plugins.PluginRegistry.getPlugin(PluginRegistry.java:715)
at org.pentaho.di.core.plugins.PluginRegistry.loadClass(PluginRegistry.java:370)
at org.pentaho.di.base.AbstractBaseCommandExecutor.establishRepositoryConnection(AbstractBaseCommandExecutor.java:195)
at org.pentaho.di.pan.PanCommandExecutor.execute(PanCommandExecutor.java:119)
at org.pentaho.di.pan.Pan.main(Pan.java:270)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)
Any help would be appreciated.
Run the job from your home directory (as working directory) using the full path of pan.sh or kitchen.sh.
I'm not sure what exactly causes the trouble. Likely causes:
Your KETTLE_HOME is not valid, causing Pentaho to look for .kettle in the working directory. (Do not include .kettle in the HOME)
A variant of this is that you don't have permissions on the files if you copied/moved them as root.
Your user does not have write access to the data-integration directory, causing some failure writing a configuration that would normally go into the working dir. It is normal to run Pentaho with an account that does not have write access here, that is not the problem, just that it doesn't like a non-writable working dir.
I am unable to execute the below command from pyspark windows
schemaPeople = spark.createDataFrame(people)
I have set HADOOP_HOME to winutils
I have provide 77 permission to C:/tmp/hive
Still I am getting the below error -
Py4JJavaError: An error occurred while calling o23.applySchemaToPythonRDD.
: java.lang.RuntimeException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
at org.apache.spark.sql.hive.client.HiveClientImpl.<init>(HiveClientImpl.scala:189)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at org.apache.spark.sql.hive.client.IsolatedClientLoader.createClient(IsolatedClientLoader.scala:258)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:359)
at org.apache.spark.sql.hive.HiveUtils$.newClientForMetadata(HiveUtils.scala:263)
at org.apache.spark.sql.hive.HiveSharedState.metadataHive$lzycompute(HiveSharedState.scala:39)
at org.apache.spark.sql.hive.HiveSharedState.metadataHive(HiveSharedState.scala:38)
at org.apache.spark.sql.hive.HiveSharedState.externalCatalog$lzycompute(HiveSharedState.scala:46)
I have gone through a lot of similar questions before posting this , appreciate any help here
I got this error a bunch when trying to setup Spark on windows using the winutils file. I had to setup Spark differently to get around this.
I ended up downloading the Hadoop binary for my version of spark and going from there. I documented the whole thing with a walkthrough if you are interested. Spark on windows
The gist is that the official Hadoop release from Apache does not include a Windows binary and compiling from sources can be tedious so really helpful people have made compiled distributions available. If you want to use Spark 2.0.2 download the binaries from steve loughran's github for 2.1.0 you can download from here from there you should be able to set it up as expected.
I am stuck at square one of trying to profile my app using JProfiler. I am trying to use "jpenable" on RHEL6, but when I select my VM, it simply crashes to the desktop as follows:
Please select the profiling mode:
GUI mode (attach with JProfiler GUI) [1, Enter]
Offline mode (use config file to set profiling settings) [2]
1
Please enter a profiling port
[31757]
java.lang.NoSuchMethodError: javax.xml.parsers.SAXParserFactory.setSchema(Ljavax/xml/validation/Schema;)V
at org.jdom2.input.sax.XMLReaders.<init>(XMLReaders.java:124)
at org.jdom2.input.sax.XMLReaders.<clinit>(XMLReaders.java:95)
at org.jdom2.input.SAXBuilder.<init>(SAXBuilder.java:338)
at org.jdom2.input.SAXBuilder.<init>(SAXBuilder.java:221)
at com.jprofiler.a.h.a(ejt:500)
<snip so StackOverflow allows the post>
at com.jprofiler.a.i.g.a(ejt:38)
at com.jprofiler.frontend.attach.c.a(ejt:243)
at com.jprofiler.frontend.EnableApplication.a(ejt:118)
at com.jprofiler.frontend.EnableApplication.g(ejt:81)
at com.jprofiler.frontend.EnableApplication.main(ejt:238)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:601)
at com.exe4j.runtime.LauncherEngine.launch(Unknown Source)
at com.install4j.runtime.launcher.Launcher.main(Unknown Source)
Any suggestions on what to do? I installed JProfiler 8.0.5 (the current version) from the RPM and simply entered the jpenable command. Everything else is shown above. This is a rather old RHEL6 image, but I cannot upgrade to a more recent one.
I am at a loss for what to try next.
Any help would be greatly appreciated,
Mike
This is because of a conflicting version of one of the following files
xml-apis.jar
xmlParserAPIs.jar
xercesImpl-x.x.x.jar
jaxp-api.jar
in the extension directory ($JAVA_HOME/lib/ext) of the JRE. You would have to remove that file.
My code is compiled on OSX and it works fine(compiler version 1.6). But when I copy it to a remote ubuntu. I can not run it.
When I run
java TPCC.class
I got exception like this.(By the way TPCC.class is in default package)
Exception in thread "main" java.lang.NoClassDefFoundError: TPCC/class
Caused by: java.lang.ClassNotFoundException: TPCC.class
at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
Could not find the main class: TPCC.class. Program will exit.
And my java version on ubuntu:
java version "1.6.0_21"
Java(TM) SE Runtime Environment (build 1.6.0_21-b06)
Java HotSpot(TM) Client VM (build 17.0-b16, mixed mode, sharing)
It's also 1.6 and I think it fits the compiled binary. How can I fix this problem?
The problem is in the way you run the program. Compiled java called
ClassName.java
will be ClassName.class. You should run it like
"java ClassName"
. You shouldn't run it as
"java ClassName.class"
What happens here is this. When you run it as java ClassName.class, JVM try to look for ClassName called "class" in the package called "ClassName" that does not exist. So it will throw NoClassDefFoundError.
This is an invocation problem as the error shows: java should be used and not the filename of the class.
The error message tries to say you this by stating that "TPCC/class" is not found.
(sorry for first adding comment, used the wrong field)
Specify the classpath when invoking java:
java -cp <path-to-jar> TPCC