I am running storm and rabbitmq on ubuntu 12.04. rabbitmq is sending data which is being received by the storm spout. I am using the StormSubmitter class to specify a cluster.
I am using the command> bin/storm jar Rabbitmqstorm.jar RabbitmqTopology rabbittest
to run the storm topology.
But I am getting an exception Exception in thread "main" java.lang.NoClassDefFoundError: com/rabbitmq/client/Consumer
This command is working fine when i am using LocalCluster class. However, i want to run this scenario in a cluster environment.
Obviously you don't have a required dependency (which it would be RabittMQ Java driver) on the cluster. Just make sure that you have that dependency on the cluster.
Related
Airflow 1.10.12 Seeing this error in the UI:
Broken DAG: [/home/airflow/dags/something.py] The version of cryptography does not match the loaded shared object. This can happen if you have multiple copies of cryptography installed in your Python path. Please try creating a new virtual environment to resolve this issue. Loaded python version: 2.9.2, shared object version: b'2.9'
The dags compile on the machine with no errors, but these messages appear for almost all the dags.
I have also recreated the virtualenv multiple times, but the error persists.
Anyone seen this before?
Turns out that a celery host had a scheduler running that was inserting the errors in the database. Stopped the extra scheduler and the messages went away
I'm building an application on AWS EMR using YARN (and Dask) version Hadoop 2.7.3-amzn-1. I'm trying to test various failure scenarios and I'm wanting to simulate a container failure. I can't seem to find an easy way to kill a YARN container - only the whole application. Is there a command-line utility for this?
[root#node1 lillcol]# yarn container -help
20/04/24 15:04:14 INFO client.AHSProxy: Connecting to Application History server at node1/127.0.0.1:10200
usage: container
-help Displays help for all commands.
-list <Application Attempt ID> List containers for application
attempt.
-signal <container ID [signal command]> Signal the container. The
available signal commands are
[OUTPUT_THREAD_DUMP,
GRACEFUL_SHUTDOWN,
FORCEFUL_SHUTDOWN] Default
command is OUTPUT_THREAD_DUMP.
-status <Container ID> Prints the status of the
container.
Through the command yarn container -signal [container-ID] GRACEFUL_SHUTDOWN to achieve.
i've tried and int works,I hope that will be helpful.
YARN has no CLI or REST API that kills a container.
The simplest way to create a container failure is to login to a NodeManager host and kill the process (which would be a container) spawned by the NodeManager.
Seems like it's exposed in API starting from version 2.8.0
https://hadoop.apache.org/docs/r2.8.0/api/org/apache/hadoop/yarn/client/api/YarnClient.html#signalToContainer(org.apache.hadoop.yarn.api.records.ContainerId,%20org.apache.hadoop.yarn.api.records.SignalContainerCommand)
I have a Glassfish 3.1.2 cluster.
I have 2 ssh nodes that each have 1 instance.
I added my lib jars in the DAS domains/mydomain/config/mycluster-config/lib/ directory.
When I restart my instances, I see that the jars are copied to each node in the:
nodes/node1/instance1/config/mycluster-config/lib/ directory and
nodes/node2/instance2/config/mycluster-config/lib/ directory.
My application is a JSF 2.2 app with Richfaces 4.3
The problem is that when I deploy my application, the application can't find any of the jars from my libs.
One question would be: How do I set the classpath for the nodes?
I have tried:
export LD_LIBRARY_PATH="/path/to/node1/instance1/config/prodc-config/lib"
and the same on command on the other node.
This did not enable my app to find the libs.
If I deploy my EAR to the standalone domain NOT the cluster then it will deploy without any errors.
When I deploy my app from the web admin console I check availability enabled and make sure the target is pointing to mycluster.
These are some of the errors I am getting:
WELD-000119 Not generating any bean definitions from com.my.domain.Validate because of underlying class loading error
Application was not properly initialized at startup, could not find Factory: javax.faces.context.FacesContextFactory. Attempting to find backup.
The cluster is always able to start.
The instances start and stop just fine.
The full message while deploying my EAR is:
Warning Command succeeded with Warning
"domain/applications/application/my_EAR" created successfully.
WARNING: Command _deploy did not complete successfully on server instance instance1: remote failure:
Failed to load the application on instance instance1.
The application will not run properly. Please fix your application and redeploy.
Exception while shutting down application container : java.lang.NullPointerException. Please see server.log for more details.
WARNING: Command _deploy did not complete successfully on server instance instance2: remote failure:
Failed to load the application on instance instance2. The application will not run properly. Please fix your application and redeploy.
Exception while shutting down application container : java.lang.NullPointerException. Please see server.log for more details.
WARNING: Command _deploy did not complete successfully on server instance instance1: remote failure:
Failed to load the application on instance instance1. The .... msg.seeServerLog
Thank you for any help with my problem.
For GlassFish v3.1.2 I have been using this link: http://docs.oracle.com/cd/E18930_01/html/821-2426/gkrdd.html#gksav
When an app is deployed I have to specify the libraries during deployment. The libraries are relative to the applibs directory. So for a cluster the path would be: ../../config/clustername-config/lib/util.jar
My problem must have been that I did not get the path correct when specifying this directory. That is what I get for not looking close enough the path I was using.
So, short answer: use --libraries when deploying app to a cluster and make sure the path is correct.
I am testing CXF REST services in Karaf using Pax Exam. The tests almost always run without a hitch on my machine. When run in Jenkins (under Maven build) they typically fail. The failures seem random and unpredictable. The error I receive during the failure deals with attempt to run a Karaf command. The commands are executed by the following snippet:
def byteArrayOutputStream = new ByteArrayOutputStream();
def printStream = new PrintStream(byteArrayOutputStream);
CommandProcessor commandProcessor = getOsgiService(CommandProcessor.class);
CommandSession commandSession = commandProcessor.createSession(System.in, printStream, System.err);
commandSession.put("APPLICATION", System.getProperty("karaf.name", "root"));
commandSession.put("USER", "karaf");
commandSession.execute(command)
These are the commands I am trying to execute in the tests setup method:
'features:addurl mvn:org.apache.cxf.karaf/apache-cxf/2.7.2/xml/features', 'features:install http', 'features:install cxf'
This is the exception:
org.apache.felix.gogo.runtime.CommandNotFoundException: Command not found: features:addurl
Apparently occasionally Karaf does not start correctly and cannot process these commands. The error like this one happen randomly in different tests on different Karaf commands. On my machine they are more likely to happen if the machine is under load.
What may cause Karaf to behave in such a manner? How to prevent these errors from happening?
Thank you,
Michael
There is is also pax-exam-karaf, it also has a feature installer which is usable from the configuration. If you want to stick to the "manual" installation you shoul make sure the features service is installed beforehand. For example let the service be injected.
I've followed the instructions in this URL: http://pmungai.wordpress.com/sakai-developer-guide/sakai-linux-cheatsheet/ and was able to compile and deploy sakai, however, after restarting tomcat, it will show me this:
root#ip-10-72-129-39:/opt/sakai# sh /opt/tomcat/bin/startup.sh
Using CATALINA_BASE: /opt/tomcat
Using CATALINA_HOME: /opt/tomcat
Using CATALINA_TMPDIR: /opt/tomcat/temp
Using JRE_HOME: /usr/lib/jvm/java-1.6.0-openjdk-amd64
Using CLASSPATH: /opt/tomcat/bin/bootstrap.jar
meaning that tomcat started successfully, but when I try to open the url from a browser, it simply loads forever, waiting for a response. If I try to shutdown tomcat, I get:
root#ip-10-72-129-39:/opt/sakai# sh /opt/tomcat/bin/shutdown.sh
Using CATALINA_BASE: /opt/tomcat
Using CATALINA_HOME: /opt/tomcat
Using CATALINA_TMPDIR: /opt/tomcat/temp
Using JRE_HOME: /usr/lib/jvm/java-1.6.0-openjdk-amd64
Using CLASSPATH: /opt/tomcat/bin/bootstrap.jar
2012-05-24 15:26:34,357 ERROR main org.apache.catalina.startup.Catalina - Catalina.stop:
java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:327)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:193)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:180)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:384)
at java.net.Socket.connect(Socket.java:546)
at java.net.Socket.connect(Socket.java:495)
at java.net.Socket.<init>(Socket.java:392)
at java.net.Socket.<init>(Socket.java:206)
at org.apache.catalina.startup.Catalina.stopServer(Catalina.java:395)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:616)
at org.apache.catalina.startup.Bootstrap.stopServer(Bootstrap.java:344)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:435)
Which happens when tomcat wasn't running to begin with.
I was able to test that tomcat was running exactly before deploying sakai, and right after, it wasn't able to load a web page.
Does anyone know what is going on?
Before starting a sakai development environment i'd sugges trying the sakai 2.8.x demo, it will provide an internal database and have some most of the standard functionalities.
You should download the demo project from:
https://testdrivesakai.com/portal
After you get used to the flows, configurations etc. you should set up your own instance with your own sakai schema. But when you do so also make sure you have at least some minimum hardware settings available and also the following tomcat startup arguments in place:
-Xms512m
-Xmx1024m
-XX:PermSize=128m
-XX:MaxPermSize=256m
-XX:NewSize=192m
-XX:MaxNewSize=384m
-Djava.awt.headless=true
-Dhttp.agent=Sakai
-Dorg.apache.jasper.compiler.Parser.STRICT_QUOTE_ESCAPING=false
-Dsun.lang.ClassLoader.allowArraySyntax=true
Also make sure you join the sakai-dev mailing list as you will get faster support by posting problems there.
The link to join the sakai dev-list:
http://collab.sakaiproject.org/mailman/listinfo/sakai-dev
In my experience this is usually database related. Are you using the out-of-the-box hsqldb or MySQL? After compiling and deploying with Maven into your tomcat directory when you first startup Tomcat it has to auto-create a bunch of tables. In many cases this can fail for a few reasons.
I will assume you are using MySQL as hsqldb has very few issues:
Sometimes the initial tomcat launch can shutdown your mysqlservice and not allow it to start back up again. This is usually due to it being unable to create error log files in the locations specified in your my.cnf file. Try commenting out any instances where these logs are being used and restart the service. Then retry the launch.
On Linux use "# ./catalina.sh run" instead of invoking the startup shell script. This will spawn a 2nd terminal window and show you everything that's happening (including any errors) while Tomcat is attempting to launch.
See if any tables were made in your database. If not then its a database connection issue. If so you should have around 377 tables or so depending on the Sakai version.
If you are getting "cache is not alive" errors in your tomcat logs this is a known race condition. You may need to disable the table auto-creation (assuming the tables have already been made from previous launch trials) and apply the patch outlined here:
KNL-1290
Without having error messages to work with it's difficult to diagnose your problem.