zeppelin interpreter error even after giving correct details - apache

getting the below error
and i have given mysql settings in the interpreter:
com.mysql.jdbc.Driver
jdbc:mysql://:3306/
username and password
restarted interpreter and binded it, but still get the error
using commands: use and select commands
enter code herejava.lang.NullPointerException
at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:201)
at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:288)
at org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
at org.apache.zeppelin.scheduler.Job.run(Job.java:169)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:134)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)

Adding the jar to $ZEPPELIN_HOME/lib folder like user3921855 didn't work for me.
I got it working adding MYSQL connector to dependency section in the interpreter config (ex: Artifact : mysql:mysql-connector-java:5.1.38)
IMPORTANT :
You need to restart Zeppelin daemon for the interpreter to pick the new jar. Don't know why because it said it had restarted the sub-process. Might be a bug.
Stop / Start Reminder :
$ZEPPELIN_HOME/bin/zeppelin-daemon.sh stop
$ZEPPELIN_HOME/bin/zeppelin-daemon.sh start

Related

Pentaho commandline nullpointer exception

Pentaho PDI version 8.3.0 CE if it matters
When I try to run a job or transformation commandline using kitchen or pan respectively I get a nullpointer exception. This happens only when trying to run something from a repository.
When I try to run the same transformation or job from spoon, all is fine and the job runs great.
I use the following commands, which both provide the same error:
./pan.sh -trans=get_clusters -rep=myrepo -user=admin -pass=mypass -dir=/Transformations
and
./kitchen.sh -job=scheduled_update_job -rep=myrepo -user=admin -pass=mypass -dir=/Jobs
NOTE: This error also happens when I try to run the job or transformation from a docker container.
The error I receive is as follows and identical for PAN and Kitchen:
020/02/05 09:07:56 - Pan - Start of run.
Processing has stopped because of an error: null
java.lang.NullPointerException
at org.pentaho.di.core.plugins.PluginRegistry.getPluginId(PluginRegistry.java:689)
at org.pentaho.di.core.plugins.PluginRegistry.getPlugin(PluginRegistry.java:715)
at org.pentaho.di.core.plugins.PluginRegistry.loadClass(PluginRegistry.java:370)
at org.pentaho.di.base.AbstractBaseCommandExecutor.establishRepositoryConnection(AbstractBaseCommandExecutor.java:195)
at org.pentaho.di.pan.PanCommandExecutor.execute(PanCommandExecutor.java:119)
at org.pentaho.di.pan.Pan.main(Pan.java:270)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.pentaho.commons.launcher.Launcher.main(Launcher.java:92)
Any help would be appreciated.
Run the job from your home directory (as working directory) using the full path of pan.sh or kitchen.sh.
I'm not sure what exactly causes the trouble. Likely causes:
Your KETTLE_HOME is not valid, causing Pentaho to look for .kettle in the working directory. (Do not include .kettle in the HOME)
A variant of this is that you don't have permissions on the files if you copied/moved them as root.
Your user does not have write access to the data-integration directory, causing some failure writing a configuration that would normally go into the working dir. It is normal to run Pentaho with an account that does not have write access here, that is not the problem, just that it doesn't like a non-writable working dir.

Jrebel is not working with weblogic 12.X

Issue Description:
Unable to bounce the weblogic server with JRebel
Error:
I am getting the follwoing error when i try to bounce the server
JRebel: ERROR Class 'java.lang.ClassLoader' could not be processed by com.zeroturnaround.javarebel.bv#null: org.zeroturnaround.bundled.javassist.bytecode.DuplicateMemberException: duplicate method: _jr$defineClass in java.lang.ClassLoader
at org.zeroturnaround.bundled.javassist.bytecode.ClassFile.testExistingMethod(SourceFile:721)
at org.zeroturnaround.bundled.javassist.bytecode.ClassFile.addMethod(SourceFile:696)
at org.zeroturnaround.bundled.javassist.CtClassType.addMethod(SourceFile:1411)
at com.zeroturnaround.javarebel.bv.process(SourceFile:40)
at org.zeroturnaround.javarebel.integration.support.JavassistClassBytecodeProcessor.process(SourceFile:79)
at com.zeroturnaround.javarebel.vu.a(SourceFile:376)
at com.zeroturnaround.javarebel.vu.a(SourceFile:365)
at com.zeroturnaround.javarebel.vu.a(SourceFile:350)
at com.zeroturnaround.javarebel.f.runBootClassProcessors(SourceFile:245)
at com.zeroturnaround.javarebel.bl.a(SourceFile:115)
at com.zeroturnaround.javarebel.gib.a(SourceFile:63)
at com.zeroturnaround.javarebel.gfc.a(SourceFile:59)
at com.zeroturnaround.javarebel.gfc.doTransform(SourceFile:39)
at com.zeroturnaround.jrebelbase.reorder.a.transform(SourceFile:182)
at com.zeroturnaround.jrebelbase.reorder.a.transform(SourceFile:148)
at sun.instrument.InstrumentationImpl.transform(InstrumentationImpl.java)
at sun.instrument.InstrumentationImpl.redefineClasses0(Native Method)
at sun.instrument.InstrumentationImpl.redefineClasses(InstrumentationImpl.java:170)
at com.mercury.opal.capture.jdk15.agent.ProbeClassFileTransformer.instrumentAndReplace(ProbeClassFileTransformer.java:369)
at com.mercury.opal.capture.jdk15.agent.ProbeClassFileTransformer.reinstrumentClass(ProbeClassFileTransformer.java:331)
at com.mercury.opal.capture.jdk15.agent.ProbeClassFileTransformer.patchClassLoaders(ProbeClassFileTransformer.java:137)
at com.mercury.opal.capture.jdk15.agent.ProbeClassFileTransformer.<init>(ProbeClassFileTransformer.java:98)
at com.mercury.opal.capture.jdk15.agent.InstrumentationAgent.premain(InstrumentationAgent.java:66)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.__invoke(DelegatingMethodAccessorImpl.java:43)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java)
at java.lang.reflect.Method.invoke(Method.java:483)
at sun.instrument.InstrumentationImpl._jrLoadClassAndStartAgent(InstrumentationImpl.java:386)
at com.zeroturnaround.jrebelbase.reorder.b.a(SourceFile:31)
at com.zeroturnaround.jrebelbase.reorder.a.c(SourceFile:129)
at com.zeroturnaround.jrebelbase.reorder.a.a(SourceFile:118)
at com.zeroturnaround.javarebel.gec.a(SourceFile:309)
at com.zeroturnaround.javarebel.gec.deferredInitHook(SourceFile:149)
at sun.launcher.LauncherHelper.checkAndLoadMain(LauncherHelper.java)
Config Details :
following lines in startup script based on the Jrebel root folder
export REBEL_HOME=[JRebel root folder]
export JAVA_OPTIONS="-agentpath:$REBEL_HOME/lib/libjrebel64.so -Drebel.remoting_plugin=true $JAVA_OPTIONS"
Version:
Weblogic : 12.2.1.2.0
Jrebel : 7.1.2
Can anyone help me to come over?
Please try using the latest version of JRebel instead of 7.1.2. Download it at https://zeroturnaround.com/software/jrebel/download/prev-releases/.
If for some reason the result is still the same, also delete REBEL_HOME/bootcache and retry. Should you still see the exception, please write to support#zeroturnaround.com along with the jrebel.log from REBEL_HOME and they'll be able to help you out.

Apache zeppelin error

I have started apache zeppelin and running successfully in the configured port.
While I am executing simple spark commands like
sc.version
println(zeppelin)
I can just see "ERROR" string near run button without any error output on my console.
My error log :
ERROR [2016-10-21 22:38:05,837] ({pool-2-thread-6} Job.java[run]:189) - Job failed
org.apache.spark.SparkException: Found both spark.driver.extraClassPath and SPARK_CLASSPATH. Use only the former.
at org.apache.spark.SparkConf$$anonfun$validateSettings$7$$anonfun$apply$8.apply(SparkConf.scala:492)
at org.apache.spark.SparkConf$$anonfun$validateSettings$7$$anonfun$apply$8.apply(SparkConf.scala:490)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.apache.spark.SparkConf$$anonfun$validateSettings$7.apply(SparkConf.scala:490)
at org.apache.spark.SparkConf$$anonfun$validateSettings$7.apply(SparkConf.scala:478)
at scala.Option.foreach(Option.scala:236)
at org.apache.spark.SparkConf.validateSettings(SparkConf.scala:478)
at org.apache.spark.SparkContext.(SparkContext.scala:398)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_1(SparkInterpreter.java:440)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:354)
at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:137)
at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:743)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Also I have mentioned the spark and java path in _zeppelin-env.sh_ file.
I know, this is quite a late answer, but for any one who faces similar issue.
Even I had similar issue, I had added some external dependency {com.twitter:algebird-core_2.11:0.11.0} in interpreter Mode for Spark. If any dependency added for any component, there might be a case that dependency is not compatible with Zeppelin Version or does not exist on remote. Once I removed that dependency, I started seeing other errors on Zeppelin console.
Just to make sure, "zeppelin.spark.printREPLOutput" is set to true, generally by default it is true only, but just to make sure!!

Exception trying to create a new Weblogic Domain

I have installed Oracle WebLogic Server 12.1.2.0 per the instructions. When I run the ./configure.sh it asks if I want to create a domain, I said no. Further down the instructions it asks if I want to create a new domain and start WLS. The following commands are listed:
$ mkdir /home/myhome/mydomain
$ cd /home/myhome/mydomain
$ $JAVA_HOME/bin/java $JAVA_OPTIONS -Xmx1024m -XX:MaxPermSize=256m
weblogic.Server
when the command is executed the following exception is generated:
[tester#kohls-enterprise-dev gravityDomain]$ $JAVA_HOME/bin/java
$JAVA_OPTIONS -Xmx1024m -XX:MaxPermSize=256m weblogic.Server
Exception in thread "main" java.lang.NoClassDefFoundError:
weblogic/Server
Caused by: java.lang.ClassNotFoundException: weblogic.Server
at java.net.URLClassLoader$1.run(URLClassLoader.java:217)
at java.security.AccessController.doPrivileged(Native Method)
at java.net.URLClassLoader.findClass(URLClassLoader.java:205)
at java.lang.ClassLoader.loadClass(ClassLoader.java:323)
at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:294)
at java.lang.ClassLoader.loadClass(ClassLoader.java:268)
Could not find the main class: weblogic.Server. Program will exit.
I have ensured that the MW_HOME is set per the instructions and have run the setWLSEnv.sh
Any suggestions?
Thank you in Advanced
O. Frank
You need to have weblogic.jar and many other jar files in the CLASSPATH. This is not the way to run a WebLogic Domain. You should create the domain and do it the prescribed way. If you really want to get low-key, include the following in your CLASSPATH
ant-all.jar
ant-contrib.jar
config-launch.jar
derbyclient.jar
derbynet.jar
tools.jar
weblogic.jar
weblogic_patch.jar
weblogic.server.modules_10.3.6.0.jar
weblogic_sp.jar
webservices.jar
xqrl.jar
Plus you would need a config.xml and additional files. Best would be to create a domain and scale down from there if you want.
You need to run configuration wizard: (config.sh)
a domain contains several files/directories, not only a few jar files.
the wizard will help you creating the domain (and the servers as well).
follow the steps from Oracle's documentation:
https://docs.oracle.com/middleware/1212/wls/WLDCW/newdom.htm#WLDCW109

Nutch problems executing crawl

I am trying to get nutch 1.11 to execute a crawl. I am using cygwin to run these commands in windows 7.
Nutch is running, I am getting results from running bin/nutch, but I keep getting error messages when I try to run a crawl.
I am getting the following error when I try to run a crawl execute with nutch:
Error running: /cygdrive/c/Users/User5/Documents/Nutch/apache-nutch-1.11/runtime/local/bin/nutch inject TestCrawl/crawldb C:/Users/User5/Documents/Nutch/apache-nutch-1.11/runtime/local/urls/seed.txt
Failed with exit value 127.
I have my JAVA_HOME classpath set, and I have altered the host file to include the 127.0.0.1 as the localhost.
I am curious if I am calling the write directory correctly, if maybe that is the problem.
The full printout looks like:
User5#User5-PC /cygdrive/c/Users/User5/Documents/Nutch/apache-nutch-1.11/runtime/local
$ bin/crawl -i -D solr.server.url=http://localhost:8983/solr/ C:/Users/User5/Documents/Nutch/apache-nutch-1.11/runtime/local/urls/ TestCrawl/ 2
Injecting seed URLs
/cygdrive/c/Users/User5/Documents/Nutch/apache-nutch-1.11/runtime/local/bin/nutch inject TestCrawl//crawldb C:/Users/User5/Documents/Nutch/apache-nutch-1.11/runtime/local/urls/
Injector: starting at 2015-12-23 17:48:21
Injector: crawlDb: TestCrawl/crawldb
Injector: urlDir: C:/Users/User5/Documents/Nutch/apache-nutch-1.11/runtime/local/urls
Injector: Converting injected urls to crawl db entries.
Injector: java.lang.NullPointerException
at java.lang.ProcessBuilder.start(ProcessBuilder.java:1012)
at org.apache.hadoop.util.Shell.runCommand(Shell.java:445)
at org.apache.hadoop.util.Shell.run(Shell.java:418)
at org.apache.hadoop.util.Shell$ShellCommandExecutor.execute(Shell.java:650)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:739)
at org.apache.hadoop.util.Shell.execCommand(Shell.java:722)
at org.apache.hadoop.fs.RawLocalFileSystem.setPermission(RawLocalFileSystem.java:633)
at org.apache.hadoop.fs.RawLocalFileSystem.mkdirs(RawLocalFileSystem.java:421)
at org.apache.hadoop.fs.FilterFileSystem.mkdirs(FilterFileSystem.java:281)
at org.apache.hadoop.mapreduce.JobSubmissionFiles.getStagingDir(JobSubmissionFiles.java:125)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:348)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1285)
at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1282)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1282)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:833)
at org.apache.nutch.crawl.Injector.inject(Injector.java:323)
at org.apache.nutch.crawl.Injector.run(Injector.java:379)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.crawl.Injector.main(Injector.java:369)
Error running:
/cygdrive/c/Users/User5/Documents/Nutch/apache-nutch-1.11/runtime/local/bin/nutch inject TestCrawl//crawldb C:/Users/User5/Documents/Nutch/apache-nutch-1.11/runtime/local/urls/
Failed with exit value 127.
The hadoop log that I think may have something to do with the error I am getting is:
2016-01-07 12:24:40,360 ERROR util.Shell - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
at org.apache.hadoop.util.Shell.getQualifiedBinPath(Shell.java:318)
at org.apache.hadoop.util.Shell.getWinUtilsPath(Shell.java:333)
at org.apache.hadoop.util.Shell.<clinit>(Shell.java:326)
at org.apache.hadoop.util.GenericOptionsParser.preProcessForWindows(GenericOptionsParser.java:432)
at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:478)
at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:170)
at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:64)
at org.apache.nutch.crawl.Injector.main(Injector.java:369)
2016-01-07 12:24:40,450 ERROR crawl.Injector - Injector: java.lang.IllegalArgumentException: java.net.URISyntaxException: Illegal character in scheme name at index 15: solr.server.url=http://localhost:8983/solr
at org.apache.hadoop.fs.Path.initialize(Path.java:206)
at org.apache.hadoop.fs.Path.<init>(Path.java:172)
at org.apache.nutch.crawl.Injector.run(Injector.java:379)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.nutch.crawl.Injector.main(Injector.java:369)
Caused by: java.net.URISyntaxException: Illegal character in scheme name at index 15: solr.server.url=http://localhost:8983/solr
at java.net.URI$Parser.fail(URI.java:2848)
at java.net.URI$Parser.checkChars(URI.java:3021)
at java.net.URI$Parser.parse(URI.java:3048)
at java.net.URI.<init>(URI.java:746)
at org.apache.hadoop.fs.Path.initialize(Path.java:203)
... 4 more
You are running linux commands from Cygwin and there is no C:\ path in linux systems. Correct command should be something like
/cygdrive/c/Users/User5/Documents/Nutch/apache-nutch1.11/runtime/local/bin/nutch inject TestCrawl/crawldb /cygdrive/c/Users/User5/Documents/Nutch/apache-nutch1.11/runtime/local/urls/seed.txt
You have answer to your problem in this message:
2016-01-07 12:24:40,360 ERROR util.Shell - Failed to locate the winutils binary in the hadoop binary path
java.io.IOException: Could not locate executable null\bin\winutils.exe in the Hadoop binaries.
This is happening because hadoop version included with nutch 1.11 is designed to work in linux out of the box and not on windows.
I had same situation and I ended up using nutch1.11 in ubuntu virtual box.
hadoop-core jar file is needed when you are working with nutch
with nutch 1.11 compatible hadoop-core jar is 0.20.0
please download jar from this link :
http://www.java2s.com/Code/Jar/h/Downloadhadoop0200corejar.htm
paste that jar into "C:\cygwin64\home\apache-nutch-1.11\lib" folder
and it will run successfully.
The problem is pretty clear. According to your hadoop log, it cannnot find the winutils.exe file. Include winutils.exe in %HADOOP_HOME%/bin folder