Apache zeppelin error - apache

I have started apache zeppelin and running successfully in the configured port.
While I am executing simple spark commands like
sc.version
println(zeppelin)
I can just see "ERROR" string near run button without any error output on my console.
My error log :
ERROR [2016-10-21 22:38:05,837] ({pool-2-thread-6} Job.java[run]:189) - Job failed
org.apache.spark.SparkException: Found both spark.driver.extraClassPath and SPARK_CLASSPATH. Use only the former.
at org.apache.spark.SparkConf$$anonfun$validateSettings$7$$anonfun$apply$8.apply(SparkConf.scala:492)
at org.apache.spark.SparkConf$$anonfun$validateSettings$7$$anonfun$apply$8.apply(SparkConf.scala:490)
at scala.collection.immutable.List.foreach(List.scala:318)
at org.apache.spark.SparkConf$$anonfun$validateSettings$7.apply(SparkConf.scala:490)
at org.apache.spark.SparkConf$$anonfun$validateSettings$7.apply(SparkConf.scala:478)
at scala.Option.foreach(Option.scala:236)
at org.apache.spark.SparkConf.validateSettings(SparkConf.scala:478)
at org.apache.spark.SparkContext.(SparkContext.scala:398)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext_1(SparkInterpreter.java:440)
at org.apache.zeppelin.spark.SparkInterpreter.createSparkContext(SparkInterpreter.java:354)
at org.apache.zeppelin.spark.SparkInterpreter.getSparkContext(SparkInterpreter.java:137)
at org.apache.zeppelin.spark.SparkInterpreter.open(SparkInterpreter.java:743)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.open(LazyOpenInterpreter.java:69)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:341)
at org.apache.zeppelin.scheduler.Job.run(Job.java:176)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:139)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Also I have mentioned the spark and java path in _zeppelin-env.sh_ file.

I know, this is quite a late answer, but for any one who faces similar issue.
Even I had similar issue, I had added some external dependency {com.twitter:algebird-core_2.11:0.11.0} in interpreter Mode for Spark. If any dependency added for any component, there might be a case that dependency is not compatible with Zeppelin Version or does not exist on remote. Once I removed that dependency, I started seeing other errors on Zeppelin console.
Just to make sure, "zeppelin.spark.printREPLOutput" is set to true, generally by default it is true only, but just to make sure!!

Related

Supported JVM targets for Ktor

I've been running a Ktor server application locally and deploying to Heroku for the last several months without any issues. However after deploying the latest version, I started to see the following error on any POST requests trying to process the request body on Heroku:
java.lang.NoSuchMethodError: java.nio.ByteBuffer.limit(I)Ljava/nio/ByteBuffer;
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$3.invoke(ExceptionUtilsJvm.kt:103)
at io.ktor.utils.io.ExceptionUtilsJvmKt$createConstructor$$inlined$safeCtor$3.invoke(ExceptionUtilsJvm.kt:90)
at io.ktor.utils.io.ExceptionUtilsJvmKt.tryCopyException(ExceptionUtilsJvm.kt:66)
at io.ktor.utils.io.ByteBufferChannelKt.rethrowClosed(ByteBufferChannel.kt:2456)
at io.ktor.utils.io.ByteBufferChannelKt.access$rethrowClosed(ByteBufferChannel.kt:1)
at io.ktor.utils.io.ByteBufferChannel.readRemaining$suspendImpl(ByteBufferChannel.kt:2126)
at io.ktor.utils.io.ByteBufferChannel.readRemaining(ByteBufferChannel.kt)
at io.ktor.utils.io.ByteReadChannelKt.readRemaining(ByteReadChannel.kt:217)
at io.ktor.serialization.SerializationConverter.convertForReceive(SerializationConverter.kt:158)
at io.ktor.features.ContentNegotiation$Feature$install$3.invokeSuspend(ContentNegotiation.kt:224)
at io.ktor.features.ContentNegotiation$Feature$install$3.invoke(ContentNegotiation.kt)
at io.ktor.features.ContentNegotiation$Feature$install$3.invoke(ContentNegotiation.kt)
at io.ktor.util.pipeline.SuspendFunctionGun.loop(SuspendFunctionGun.kt:248)
at io.ktor.util.pipeline.SuspendFunctionGun.proceed(SuspendFunctionGun.kt:116)
at io.ktor.util.pipeline.SuspendFunctionGun.execute(SuspendFunctionGun.kt:136)
at io.ktor.util.pipeline.Pipeline.execute(Pipeline.kt:78)
at io.ktor.request.ApplicationReceiveFunctionsKt.receive(ApplicationReceiveFunctions.kt:116)
at io.ktor.request.ApplicationReceiveFunctionsKt.receiveOrNull(ApplicationReceiveFunctions.kt:135)
Thinking that there might be something unexpected with the request body, I first tried changing call.receive<MyObject>() to call.receiveOrNull<MyObject>(), yet the issue persisted.
I got tipped off that this might be a JVM mismatch issues, so I checked Heroku's Java documentation and learned that the default JVM is 1.8. I then ran my application locally with Java 1.8 and was able to re-create the same behavior.
After changing the target JVM on Heroku for my application to version 11, this has apparently fixed the problem. While I did recently add the Ktor Authentication "feature", I can't see any other changes that might have caused this. I tried to find in the Ktor documentation what the minimum required JVM target is and was not able to find it. It's great that it's working again, but I'd like to have more confidence moving forward.
In order to ensure proper execution of the server and to avoid this kind of issue in the future, I'd like to ask what the supported JVM target(s) are for Ktor Server (and if possible, have this added to the Ktor documentation).
I solved by upgrding to ktor_version:1.6.7.

IntelliJ Fails Deploying exploded war to tomcat

I have a Java web application I am developing in IntelliJ and deploying using Apache Tomcat. I have some run configurations defined that are shared between everyone on the team. These run configurations are defined to deploy an exploded war artifact into a local tomcat. Other team members are able to deploy and run just fine, but I get the following error:
Artifact Gradle : project : project.war (exploded): Error during artifact deployment. See server log for details.
Artifact Gradle : project : project.war (exploded): com.intellij.javaee.oss.admin.jmx.JmxAdminException: com.intellij.execution.ExecutionException: /IdeaProjects/project/build/libs/exploded/project.war not found for the web module.
It seems that IntelliJ doesn't want to create the exploded/project.war directories. If I create those directories manually I don't get that error message, but none of the WAR contents get placed into the directory.
I thought that it could be a permission error so I even tried giving my project directory 777 just to try to eliminate possible problems.
Has anyone ever run into this problem?
I am running MacOS 10.12, and have been able to deploy in the past, but this problem just started.
Thanks
Update
The IntelliJ log doesn't really show any additional errors, just the full StackTrace for the errors above.
2017-11-02 11:20:52,896 [3777157] INFO - erver.JavaeeServerInstanceImpl - com.intellij.execution.ExecutionException: <redacted>/build/libs/exploded/myproject.war not found for the web module.
com.intellij.javaee.oss.admin.jmx.JmxAdminException: com.intellij.execution.ExecutionException: <redacted>/build/libs/exploded/myproject.war not found for the web module.
at org.jetbrains.idea.tomcat.admin.TomcatAdminLocalServerImpl$DeployStep.perform(TomcatAdminLocalServerImpl.java:277)
at org.jetbrains.idea.tomcat.admin.TomcatAdminLocalServerImpl.doDeploy(TomcatAdminLocalServerImpl.java:125)
at com.intellij.javaee.oss.admin.jmx.JavaeeJmxAdminServerBase$4.doPerform(JavaeeJmxAdminServerBase.java:120)
at com.intellij.javaee.oss.admin.jmx.JavaeeJmxAdminServerBase$JmxOperation.perform(JavaeeJmxAdminServerBase.java:243)
at com.intellij.javaee.oss.admin.jmx.JavaeeJmxAdminServerBase.doStartDeploy(JavaeeJmxAdminServerBase.java:135)
at com.intellij.javaee.oss.admin.jmx.JavaeeJmxAdminServerBase$2.setDeploymentStatus(JavaeeJmxAdminServerBase.java:90)
at com.intellij.javaee.oss.admin.jmx.JavaeeJmxAdminServerBase$DeploymentModelOperation.doSetDeploymentStatus(JavaeeJmxAdminServerBase.java:270)
at com.intellij.javaee.oss.admin.jmx.JavaeeJmxAdminServerBase$3.doPerform(JavaeeJmxAdminServerBase.java:100)
at com.intellij.javaee.oss.admin.jmx.JavaeeJmxAdminServerBase$JmxOperation.perform(JavaeeJmxAdminServerBase.java:243)
at com.intellij.javaee.oss.admin.jmx.JavaeeJmxAdminServerBase.doStartDeployWithUndeploy(JavaeeJmxAdminServerBase.java:107)
at com.intellij.javaee.oss.admin.jmx.JavaeeJmxAdminServerBase.startDeploy(JavaeeJmxAdminServerBase.java:74)
at org.jetbrains.idea.tomcat.admin.TomcatAdminServerBase.startDeploy(TomcatAdminServerBase.java:115)
at org.jetbrains.idea.tomcat.admin.TomcatAdminLocalServerImpl.startDeploy(TomcatAdminLocalServerImpl.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at com.intellij.remoteServer.agent.impl.ThreadInvocationHandler.a(ThreadInvocationHandler.java:56)
at com.intellij.remoteServer.agent.impl.ThreadInvocationHandler.a(ThreadInvocationHandler.java:100)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at com.intellij.util.concurrency.BoundedTaskExecutor$2.run(BoundedTaskExecutor.java:212)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: com.intellij.execution.ExecutionException: <redacted>/build/libs/exploded/myproject.war not found for the web module.
at org.jetbrains.idea.tomcat.admin.TomcatAdminLocalServerImpl.addApplicationContext(TomcatAdminLocalServerImpl.java:217)
at org.jetbrains.idea.tomcat.admin.TomcatAdminLocalServerImpl$4.doPerform(TomcatAdminLocalServerImpl.java:121)
at org.jetbrains.idea.tomcat.admin.TomcatAdminLocalServerImpl$DeployStep.perform(TomcatAdminLocalServerImpl.java:274)
... 24 more
Before Intellij 2019.2 :
Under Gradle Settings, you have to make sure you don't have checked 'Delegate IDE build/run actions to gradle'
Since Intellij 2019.2 :
Settings is now under 'Build, Execution, Deployment > Build Tools > Gradle'.
You have to select Build and run using : Intellij IDEA
An issue has been opened on jetbrains bug tracker.

zeppelin hive interpreter throws ClassNotFoundException

I have deployed zeppelin 0.6 and configured hive under Jdbc interpreter.
Tried executing
%hive
show databases
Throws:
org.apache.hive.jdbc.HiveDriver class java.lang.ClassNotFoundException
java.net.URLClassLoader.findClass(URLClassLoader.java:381)
java.lang.ClassLoader.loadClass(ClassLoader.java:424)
sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:331)
java.lang.ClassLoader.loadClass(ClassLoader.java:357)
java.lang.Class.forName0(Native Method)
java.lang.Class.forName(Class.java:264)
org.apache.zeppelin.jdbc.JDBCInterpreter.getConnection(JDBCInterpreter.java:220)
org.apache.zeppelin.jdbc.JDBCInterpreter.getStatement(JDBCInterpreter.java:233)
org.apache.zeppelin.jdbc.JDBCInterpreter.executeSql(JDBCInterpreter.java:292)
org.apache.zeppelin.jdbc.JDBCInterpreter.interpret(JDBCInterpreter.java:398)
org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:94)
org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:383)
org.apache.zeppelin.scheduler.Job.run(Job.java:176)
org.apache.zeppelin.scheduler.ParallelScheduler$JobRunner.run(ParallelScheduler.java:162)
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
java.util.concurrent.FutureTask.run(FutureTask.java:266)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
java.lang.Thread.run(Thread.java:745)
I just ran into this issue this morning. I'm not sure if this is the recommended way to fix, but I downloaded the binary packages for Hive 1.2, and Hadoop 2.6.4. I then copied the following jars to ./interpreter/jdbc/ and reloaded zeppelin ./bin/zeppelin-daemon.sh reload
cp ~/Dev/Hadoop/apache-hive-1.2.1-bin/lib/hive-jdbc-1.2.1-standalone.jar ./interpreter/jdbc/
cp ~/Dev/Hadoop/hadoop-2.6.4/share/hadoop/common/hadoop-common-2.6.4.jar ./interpreter/jdbc/
1)
You could download just Hive JDBC driver instead of whole Hive jars set, for example, one from Cloudera:
http://www.cloudera.com/downloads/connectors/hive/jdbc/2-5-17.html
2)
Hive starting with 0.14 will have a standalone jar for JDBC part:
hive-jdbc-standalone.jar
but until https://issues.apache.org/jira/browse/HIVE-9600 is resolved,
you would need two more jars:
hadoop-common.jar
hadoop-auth.jar
to put into classpath along with hive-jdbc-standalone.jar
The top rated answer given here fixes the issue
However I have added the classpath of HADOOP_HOME to interpreter.sh to take the jar files in common
Below is the line which I have added to bin/interpreter.sh inside zeppelin
HADOOP_HOME=/opt/hadoop-2.6.2/
addJarInDirForIntp "${HADOOP_HOME}/share/hadoop/common

zeppelin interpreter error even after giving correct details

getting the below error
and i have given mysql settings in the interpreter:
com.mysql.jdbc.Driver
jdbc:mysql://:3306/
username and password
restarted interpreter and binded it, but still get the error
using commands: use and select commands
enter code herejava.lang.NullPointerException
at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.executeSql(PostgreSqlInterpreter.java:201)
at org.apache.zeppelin.postgresql.PostgreSqlInterpreter.interpret(PostgreSqlInterpreter.java:288)
at org.apache.zeppelin.interpreter.ClassloaderInterpreter.interpret(ClassloaderInterpreter.java:57)
at org.apache.zeppelin.interpreter.LazyOpenInterpreter.interpret(LazyOpenInterpreter.java:93)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterServer$InterpretJob.jobRun(RemoteInterpreterServer.java:300)
at org.apache.zeppelin.scheduler.Job.run(Job.java:169)
at org.apache.zeppelin.scheduler.FIFOScheduler$1.run(FIFOScheduler.java:134)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:334)
at java.util.concurrent.FutureTask.run(FutureTask.java:166)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:178)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:292)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
at java.lang.Thread.run(Thread.java:722)
Adding the jar to $ZEPPELIN_HOME/lib folder like user3921855 didn't work for me.
I got it working adding MYSQL connector to dependency section in the interpreter config (ex: Artifact : mysql:mysql-connector-java:5.1.38)
IMPORTANT :
You need to restart Zeppelin daemon for the interpreter to pick the new jar. Don't know why because it said it had restarted the sub-process. Might be a bug.
Stop / Start Reminder :
$ZEPPELIN_HOME/bin/zeppelin-daemon.sh stop
$ZEPPELIN_HOME/bin/zeppelin-daemon.sh start

Failed to create a child event loop

In IntelliJ IDEA Community Edition 14.0.2, I'm trying to run Java Applet and every time I try to run project, I got this this dummy error.
Error: Failed to create a child event loop
What I've tried
Compiling newly created Hello World program
Update IntelliJ
Uninstall & Reinstall
Allow from Windows Firewall rule
Disabling Antivirus Program
I'm using Windows 8.1 with Anti-virus program Windows Defender.
Error detail from Log file:
java.lang.IllegalStateException: failed to create a child event loop
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:81)
at io.netty.channel.MultithreadEventLoopGroup.<init>(MultithreadEventLoopGroup.java:50)
at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:72)
at io.netty.channel.nio.NioEventLoopGroup.<init>(NioEventLoopGroup.java:58)
at org.jetbrains.io.BuiltInServer.start(BuiltInServer.java:60)
at org.jetbrains.ide.BuiltInServerManagerImpl$1.run(BuiltInServerManagerImpl.java:111)
at com.intellij.openapi.application.impl.ApplicationImpl$8.run(ApplicationImpl.java:405)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
at org.jetbrains.ide.PooledThreadExecutor$1$1.run(PooledThreadExecutor.java:56)
Caused by: io.netty.channel.ChannelException: failed to open a new selector
at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:127)
at io.netty.channel.nio.NioEventLoop.<init>(NioEventLoop.java:119)
at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:97)
at io.netty.channel.nio.NioEventLoopGroup.newChild(NioEventLoopGroup.java:31)
at io.netty.util.concurrent.MultithreadEventExecutorGroup.<init>(MultithreadEventExecutorGroup.java:77)
... 12 more
Caused by: java.io.IOException: Unable to establish loopback connection
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:101)
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:68)
at java.security.AccessController.doPrivileged(Native Method)
at sun.nio.ch.PipeImpl.<init>(PipeImpl.java:170)
at sun.nio.ch.SelectorProviderImpl.openPipe(SelectorProviderImpl.java:50)
at java.nio.channels.Pipe.open(Pipe.java:155)
at sun.nio.ch.WindowsSelectorImpl.<init>(WindowsSelectorImpl.java:127)
at sun.nio.ch.WindowsSelectorProvider.openSelector(WindowsSelectorProvider.java:44)
at io.netty.channel.nio.NioEventLoop.openSelector(NioEventLoop.java:125)
... 16 more
Caused by: java.net.SocketException: Network is unreachable: connect
at sun.nio.ch.Net.connect0(Native Method)
at sun.nio.ch.Net.connect(Net.java:457)
at sun.nio.ch.Net.connect(Net.java:449)
at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:647)
at java.nio.channels.SocketChannel.open(SocketChannel.java:189)
at sun.nio.ch.PipeImpl$Initializer$LoopbackConnector.run(PipeImpl.java:130)
at sun.nio.ch.PipeImpl$Initializer.run(PipeImpl.java:83)
... 24 more
Restarting intelliJ did not help, restarting machine did!
Disable firewall
or add idea.exe full correct path to outgoing connect rules of your firewall.
It helped me with IDEA 14.0.3
I encountered same error when we deploy our application to a Windows Server, however, this server has no antivirus software and the firewall is disabled. In our case, it's due to the Astrill VPN, if we disable the Astrill VPN, the issue would disappear.
My os is Windows, Use Pycharm 2017.3 to run scrapy project encountered a similar problem.
Today, Use Intellij Idea 2016.3.5 to run spring project had the same question.
In my os, Close the Windows Defender can solve this question.
We have to turn off the firewall. It solves the problem. Even in "eclipse" software also turning off the firewall works well.
updating pycharm to a newer version worked for me and then restarting the pycharm ide.