Activemq Shutdown fails and then kills process - activemq

Question

I am implementing replicated leveldb activemq setup. I have 3 instance of activemq running on same box. I am changing their rmiPort, amqpport and openwire port in config file.
config like lookslie this:
<?xml version="1.0" encoding="UTF-8"?><beans xmlns=" http://www.springframework.org /schema/beans" xmlns:amq=" http://activemq.apache.org/schema/core" xmlns:xsi=" http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation=" http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<!-- Allows us to use system properties as variables in this configuration file -->
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<value>file:${activemq.conf}/credentials.properties</value>
</property>
</bean>
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="activemq_8200" dataDirectory="${activemq.data}">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry producerFlowControl="false" topic=">">
<!-- The constantPendingMessageLimitStrategy is used to prevent
slow topic consumers to block producers and affect other consumers
by limiting the number of messages that are retained
For more information, see:
http://activemq.apache.org/slow-consumer-handling.html
-->
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>
</policyEntry>
<policyEntry producerFlowControl="false" queue=">">
<deadLetterStrategy>
<!--
Use the prifix 'DLQ.' for the destination name, and make the DLQ a queue rather than a topic
-->
<individualDeadLetterStrategy queuePrefix="DLQ." useQueueForQueueMessages="true"/>
</deadLetterStrategy>
<!-- Use VM cursor for better latency
For more information, see:
http://activemq.apache.org/message-cursors.html
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
-->
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<!--
The managementContext is used to configure how ActiveMQ is exposed in
JMX. By default, ActiveMQ uses the MBean server that is started by
the JVM. For more information, see:
http://activemq.apache.org/jmx.html
-->
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<persistenceAdapter>
<replicatedLevelDB bind="tcp://0.0.0.0:0" directory="${activemq.data}/leveldb" replicas="3" zkAddress="gwxdev05.northamerica.cerner.net:2181,gwxdev05.northamerica.cerner.net:2182,gwxdev05.northamerica.cerner.net:2183" zkPassword="password" zkPath="/opt/gwx/activemqdata"/>
</persistenceAdapter>
<systemUsage>
<systemUsage sendFailIfNoSpace="true">
<memoryUsage>
<memoryUsage limit="256 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="1 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="128 mb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
<transportConnector name="openwire" uri="tcp://0.0.0.0:${openwirePort}?maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
<transportConnector name="amqp" uri="amqp://0.0.0.0:${amqpPort}?maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
</transportConnectors>
<!-- destroy the spring context on shutdown to stop jetty -->
<shutdownHooks>
<bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook"/>
</shutdownHooks>
</broker>
<import resource="jetty.xml"/>
my instance file looks like this:
ACTIVEMQ_BASE=`cd "$ACTIVEMQ_BASE" && pwd`
## Add system properties for this instance here (if needed), e.g
#export ACTIVEMQ_OPTS_MEMORY="-Xms256M -Xmx1G"
#export ACTIVEMQ_OPTS="$ACTIVEMQ_OPTS_MEMORY
-Dorg.apache.activemq.UseDedicatedTaskRunner=true
-Djava.util.logging.config.file=logging.properties"
export ACTIVEMQ_SUNJMX_CONTROL="-Dactivemq.jmx.url=service:jmx:rmi:///jndi/rmi://127.0.0.1:8100/jmxrmi"
#
ACTIVEMQ_SUNJMX_START="-Dcom.sun.management.jmxremote.port=8100 "
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.authenticate=false"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.ssl=false"
export ACTIVEMQ_SUNJMX_START=$ACTIVEMQ_SUNJMX_START
export ACTIVEMQ_HOME=/opt/gwx/apache-activemq-5.10-SNAPSHOT
export ACTIVEMQ_BASE=$ACTIVEMQ_BASE
export JAVA_HOME=/opt/gwx/apache-activemq-5.10-SNAPSHOT/jdk1.7.0_25
${ACTIVEMQ_HOME}/bin/activemq "$#"
Here is exception I get:
Connecting to pid: 2410
INFO: failed to resolve jmxUrl for pid:2410, using default JMX url Connecting to JMX URL: service:jmx:rmi:///jndi/rmi://localhost:1099/jmxrmi
ERROR: java.lang.RuntimeException: Failed to execute stop task. Reason: java.io.IOException: Failed to retrieve RMIServer stub:
javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is: java.net.ConnectException: Connection refused] java.lang.RuntimeException: Failed to execute stop task. Reason: java.io.IOException: Failed to retrieve RMIServer stub:
javax.naming.ServiceUnavailableException [Root exception is java.rmi.ConnectException: Connection refused to host: localhost; nested exception is: java.net.ConnectException: Connection refused]
at
I checked firewall. It not issue.
Any idea what might be causing this issue.
Activemq version 5.10 snapshot
Java 1.7
OS linux 6.4

Answer 1

If it still helps someone:
When I had in bin/env this
export ACTIVEMQ_SUNJMX_CONTROL="-Dactivemq.jmx.url=service:jmx:rmi:///jndi/rmi://127.0.0.1:8100/jmxrmi"
#
ACTIVEMQ_SUNJMX_START="-Dcom.sun.management.jmxremote.port=8100 "
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.authenticate=false"
ACTIVEMQ_SUNJMX_START="$ACTIVEMQ_SUNJMX_START -Dcom.sun.management.jmxremote.ssl=false"
, it did not work, process failed even without logging.
I don't know why, but this method works(for AMQ 5.13): u add to xml this:
<managementContext>
<managementContext connectorPort="1099"/>
</managementContext>
and nothing like ACTIVEMQ_SUNJMX_START in env file