I am trying to start hiveserver2, by going to the bin folder of my Hive install and typing hiveserver2. However, nothing happens - it just hangs there, and when I check if anything is running on Hive ports (the interface on 10002 for example) there is nothing, nor anything in netstat.
Initially I had errors about SLF4J:
SLF4J: Found binding in [jar:file:/usr/lib/hive/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/lib/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
But I solved that by moving the /usr/lib/hive/lib/log4j-slf4j-impl-2.4.1.jar in to the /tmp directory.
Now when I run bin/hiveserver2 it hangs for quite a while, with no output, and then just returns to the command line - and the Hive server isn't started. I'm struggling to find any logs either.
There is a chance that the hiveserver2 running without showing any information. Try this answer with attached argument flag showing running log
Related
In our web application deployed on tomcat, catalina.out file keeps growing (it grows in GB's).
So to disable it I found solution where I set CATALINA_OUT=/dev/null.
This worked as expected but we also use kill -3 <PID> command to capture Thread Dump whenever it requires.
This thread dump gets written into catalina.out file but now I have disabled catalina.out so I cannot see thread dump.
How can I get thread dump?
Since it is a production system therefore we are using JRE and not JDK otherwise we would have used jmap/jcmd/jstack commands to capture thread dump.
So, I would like, tomcat should stop creating catalina.out file or do not write log4j statements in it. And if it is disabled then how to get thread dump?
Finally I found the solution which I was looking for. I added below lines of statement in tomcat/bin/startup.sh file
export CATALINA_OUT=/dev/null
export JAVA_OPTS="$JAVA_OPTS -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:LogFile=thread_dump.log"
CATALINA_OUT set to /dev/null will stop writing log statements in catalina.out log.
JAVA_OPTS -XX:+UnlockDiagnosticVMOptions -XX:+LogVMOutput -XX:LogFile=thread_dump.log configurations is used to write thread dump statements generated using kill –3 <PID> command. It will also print JVM-related configuration being used.
I hope this will help someone.
I'm new flink user and I have the following problem.
I use flink on YARN cluster to transfer related data extracted from RDBMS to HBase.
I write flink batch application on java with multiple ExecutionEnvironments (one per RDB table to transfer table rows in parrallel) to transfer table by table sequentially (because call of env.execute() is blocking).
I start YARN session like this
export YARN_CONF_DIR=/etc/hadoop/conf
export FLINK_HOME=/opt/flink-1.3.1
export FLINK_CONF_DIR=$FLINK_HOME/conf
$FLINK_HOME/bin/yarn-session.sh -n 1 -s 4 -d -jm 2048 -tm 8096
Then I run my application on YARN session started via shell script transfer.sh. Its content is here
#!/bin/bash
export YARN_CONF_DIR=/etc/hadoop/conf
export FLINK_HOME=/opt/flink-1.3.1
export FLINK_CONF_DIR=$FLINK_HOME/conf
$FLINK_HOME/bin/flink run -p 4 transfer.jar
When I start this script from command line manually it works fine - jobs are submitted to YARN session one by one without errors.
Now I should be able to run this script from another java program.
For this aim I use
Runtime.exec("transfer.sh");
(maybe are there better ways to do this? I have seen at REST API but there are some difficulties because job manager is proxied by YARN).
At the beginning is works as usually - first several jobs are submitted to session and finished successfully. But the following jobs are not submitted to YARN session.
In /opt/flink-1.3.1/log/flink-tsvetkoff-client-hadoop-dev1.log I see error (and no another errors found in DEBUG level)
The program execution failed: JobClientActor seems to have died before the JobExecutionResult could be retrieved.
I have tried to analyse this problem by myself and found out that this error has occurred in JobClient class while sending ping request with timeout to JobClientActor (i.e. YARN cluster).
I tried to increase multiple heartbeat and timeout options like akka.*.timeout, akka.watch.heartbeat.* and yarn.heartbeat-delay options but it doesn't solve the problem - new jobs are not submit to YARN session from CliFrontend.
The environment for both case (manual call and call from another program) is the same. When I call
$ ps axu | grep transfer
it will give me output
/usr/lib/jvm/java-8-oracle/bin/java -Dlog.file=/opt/flink-1.3.1/log/flink-tsvetkoff-client-hadoop-dev1.log -Dlog4j.configuration=file:/opt/flink-1.3.1/conf/log4j-cli.properties -Dlogback.configurationFile=file:/opt/flink-1.3.1/conf/logback.xml -classpath /opt/flink-1.3.1/lib/flink-metrics-graphite-1.3.1.jar:/opt/flink-1.3.1/lib/flink-python_2.11-1.3.1.jar:/opt/flink-1.3.1/lib/flink-shaded-hadoop2-uber-1.3.1.jar:/opt/flink-1.3.1/lib/log4j-1.2.17.jar:/opt/flink-1.3.1/lib/slf4j-log4j12-1.7.7.jar:/opt/flink-1.3.1/lib/flink-dist_2.11-1.3.1.jar:::/etc/hadoop/conf org.apache.flink.client.CliFrontend run -p 4 transfer.jar
I also tried to update flink to 1.4.0 release or change parallelism of job (even to -p 1) but error has still occurred.
I have no idea what could be different? Is any workaround by the way?
Thank you for any help.
Finally I find out how to resolve that error
Just replace Runtime.exec(...) with new ProcessBuilder(...).inheritIO().start().
I really don't know why the call of inheritIO helps in that case because as I understand it just redirects IO streams from child process to parent process.
But I have checked that if I comment out this line of code the program begins to fall again.
I'm following these steps outlines on this link, however when I try to start the server nothing happens nor can I connect to anything from the client. Does anyone know how to run this?
when I try from a command prompt instead of double clicking the redis-server.exe I get this message
[11868] 23 Jul 11:58:26.325 # QForkMasterInit: system error caught. error code=0
x000005af, message=VirtualAllocEx failed.: unknown error
http://bartwullems.blogspot.ca/2013/07/unofficial-redis-for-windows.html
The easiest way to install Redis is through NuGet:
Open Visual Studio
Create an empty solution so that NuGet knows where to put the packages
Go the Package Manager Console: Tools –> Library Package Manager –>Package Manager Console
Type Install-Package Redis-64
image
Go to the Packages folder and browse to the Tools folder. Here you’ll find the Redis-server.exe. Double click on it to start it.
Redis is ready to use and start’s listening on a specific port(6379 in
my case)
image
Let’s open up a client and try to put a value into Redis. Start Redis-cli.exe. It already connects to the same port by default.
image
Add a value by executing following command:
image
Read the value again:
image
Try to run with redis-server --maxheap 4000000
Miguel is correct, but it is not that simple. To start redis-server either as a service or from the command prompt, the amount of available RAM and disk space must be sufficient for Redis to run as configured.
Now, if no configuration file is specified when running Redis, it will use the default configuration values. All of this is documented in the redis.windows.conf file as well as in the document "Redis on Windows.docx" (both deployed with the redis installation).
In my experience, errors when starting Redis usually come from lack of available resources (RAM or disk space) or some incorrect configuration of maxhead or maxmemory parameters.
To troubleshoot this kind of behavior, check your system's available resources and try running redis-server from the command line varying the parameters maxmemory, maxheap, and/or heapdir. The loglevel parameter set to verbose might also help diagnosing the issue.
Regards
I am trying to get ActiveMQ server running on a RaspberryPI Debian Squeeze box and all appears to be installed correctly but when I try and start the service I am getting the following...
root#raspberrypi:/var/www/activemq/apache-activemq-5.7.0# bin/activemq
INFO: Loading '/etc/default/activemq'
INFO: Using java '/usr/jre1.7.0_07/bin/java'
/usr/jre1.7.0_07/bin/java: 1: /usr/jre1.7.0_07/bin/java:ELF0
4°: not found
/usr/jre1.7.0_07/bin/java: 2: /usr/jre1.7.0_07/bin/java: Syntax error: "(" unexpected
Tasks provided by the sysv init script:
restart - stop running instance (if there is one), start new instance
console - start broker in foreground, useful for debugging purposes
status - check if activemq process is running
setup - create the specified configuration file for this init script
(see next usage section)
Configuration of this script:
The configuration of this script can be placed on /etc/default/activemq or /root/.activemqrc.
To use additional configurations for running multiple instances on the same operating system
rename or symlink script to a name matching to activemq-instance-<INSTANCENAME>.
This changes the configuration location to /etc/default/activemq-instance-<INSTANCENAME> and
$HOME/.activemqrc-instance-<INSTANCENAME>. Configuration files in /etc have higher precedence.
root#raspberrypi:/var/www/activemq/apache-activemq-5.7.0#
It looks like there is an error somewhere but I am a fairly newbie at this and don't know where to look.
Just adding an answer since,becoz as per the documentation , the command is wrong
to start activemqm, use
Navigate to [installation directory/bin] and run ./activemq start or simple bin/activemq start
if you want to see it live in a window use bin/activemq console
To stop, you have to kill the process
The default ActiveMQ "getting started" link sends u here : http://activemq.apache.org/getting-started.html which is the "Getting Started Guide for ActiveMQ 4.x".
If you scroll down main documentation page, you will find this link with the proper commands :
http://activemq.apache.org/version-5-getting-started.html
I've followed the instructions in this URL: http://pmungai.wordpress.com/sakai-developer-guide/sakai-linux-cheatsheet/ and was able to compile and deploy sakai, however, after restarting tomcat, it will show me this:
root#ip-10-72-129-39:/opt/sakai# sh /opt/tomcat/bin/startup.sh
Using CATALINA_BASE: /opt/tomcat
Using CATALINA_HOME: /opt/tomcat
Using CATALINA_TMPDIR: /opt/tomcat/temp
Using JRE_HOME: /usr/lib/jvm/java-1.6.0-openjdk-amd64
Using CLASSPATH: /opt/tomcat/bin/bootstrap.jar
meaning that tomcat started successfully, but when I try to open the url from a browser, it simply loads forever, waiting for a response. If I try to shutdown tomcat, I get:
root#ip-10-72-129-39:/opt/sakai# sh /opt/tomcat/bin/shutdown.sh
Using CATALINA_BASE: /opt/tomcat
Using CATALINA_HOME: /opt/tomcat
Using CATALINA_TMPDIR: /opt/tomcat/temp
Using JRE_HOME: /usr/lib/jvm/java-1.6.0-openjdk-amd64
Using CLASSPATH: /opt/tomcat/bin/bootstrap.jar
2012-05-24 15:26:34,357 ERROR main org.apache.catalina.startup.Catalina - Catalina.stop:
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
at java.net.Socket.connect(Socket.java:546)
at java.net.Socket.connect(Socket.java:495)
at java.net.Socket.<init>(Socket.java:392)
at java.net.Socket.<init>(Socket.java:206)
at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:395)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:344)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:435)
Which happens when tomcat wasn't running to begin with.
I was able to test that tomcat was running exactly before deploying sakai, and right after, it wasn't able to load a web page.
Does anyone know what is going on?
Before starting a sakai development environment i'd sugges trying the sakai 2.8.x demo, it will provide an internal database and have some most of the standard functionalities.
You should download the demo project from:
https://testdrivesakai.com/portal
After you get used to the flows, configurations etc. you should set up your own instance with your own sakai schema. But when you do so also make sure you have at least some minimum hardware settings available and also the following tomcat startup arguments in place:
-Xms512m
-Xmx1024m
-XX:PermSize=128m
-XX:MaxPermSize=256m
-XX:NewSize=192m
-XX:MaxNewSize=384m
-Djava.awt.headless=true
-Dhttp.agent=Sakai
-Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false
-Dsun.lang.ClassLoader.allowArraySyntax=true
Also make sure you join the sakai-dev mailing list as you will get faster support by posting problems there.
The link to join the sakai dev-list:
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
In my experience this is usually database related. Are you using the out-of-the-box hsqldb or MySQL? After compiling and deploying with Maven into your tomcat directory when you first startup Tomcat it has to auto-create a bunch of tables. In many cases this can fail for a few reasons.
I will assume you are using MySQL as hsqldb has very few issues:
Sometimes the initial tomcat launch can shutdown your mysqlservice and not allow it to start back up again. This is usually due to it being unable to create error log files in the locations specified in your my.cnf file. Try commenting out any instances where these logs are being used and restart the service. Then retry the launch.
On Linux use "# ./catalina.sh run" instead of invoking the startup shell script. This will spawn a 2nd terminal window and show you everything that's happening (including any errors) while Tomcat is attempting to launch.
See if any tables were made in your database. If not then its a database connection issue. If so you should have around 377 tables or so depending on the Sakai version.
If you are getting "cache is not alive" errors in your tomcat logs this is a known race condition. You may need to disable the table auto-creation (assuming the tables have already been made from previous launch trials) and apply the patch outlined here:
KNL-1290
Without having error messages to work with it's difficult to diagnose your problem.