Apache ActiveMQ - Transport Connection to: tcp:/ failed: java.io.EOFException /java.net.SocketException: Software caused connection abort: recv failed - activemq

I am using following configuration for ActiveMQ:
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<value>file:${activemq.conf}/credentials.properties</value>
</property>
</bean>
<bean id="logQuery" class="io.fabric8.insight.log.log4j.Log4jLogQuery"
lazy-init="false" scope="singleton"
init-method="start" destroy-method="stop">
</bean>
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" >
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<persistenceAdapter>
<kahaDB directory="${activemq.data}/kahadb"/>
</persistenceAdapter>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage percentOfJvmHeap="70" />
</memoryUsage>
<storeUsage>
<storeUsage limit="100 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="50 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=90000&wireFormat.maxInactivityDuration=10000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=90000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="stomp" uri="stomp://0.0.0.0:61613?maximumConnections=90000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=90000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="ws" uri="ws://0.0.0.0:61614?maximumConnections=90000&wireFormat.maxFrameSize=104857600"/>
</transportConnectors>
<!-- destroy the spring context on shutdown to stop jetty -->
<shutdownHooks>
<bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook" />
</shutdownHooks>
</broker>
<!--
Enable web consoles, REST and Ajax APIs and demos
The web consoles requires by default login, you can disable this in the jetty.xml file
Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
-->
<import resource="jetty.xml"/>
</beans>
<!-- END SNIPPET: example -->
When I try to run on 61616 then I am getting following error constantly and ActiveMQ does not work or does not get started properly,
INFO | jvm 2 | 2021/09/07 17:30:21 | WARN | Transport Connection to: tcp://IP1:61040 failed: java.io.EOFException
INFO | jvm 2 | 2021/09/07 17:30:37 | WARN | Transport Connection to: tcp://IP2:61464 failed: java.net.SocketException: Software caused connection abort: recv failed
INFO | jvm 2 | 2021/09/07 17:30:37 | WARN | Transport Connection to: tcp://IP3:53301 failed: java.net.SocketException: Software caused connection abort: recv failed
INFO | jvm 2 | 2021/09/07 17:30:51 | WARN | Transport Connection to: tcp://IP4:51592 failed: java.io.EOFException
INFO | jvm 2 | 2021/09/07 17:30:51 | WARN | Transport Connection to: tcp://IP5:57330 failed: java.io.EOFException
INFO | jvm 2 | 2021/09/07 17:31:05 | WARN | Transport Connection to: tcp://IP6:51593 failed: java.io.EOFException
But when I try to run it on different port other than 61616 it works fine as there are no incoming requests on that port from any source.
Can anyone help me out here what could be the possible reason for this failure?

Related

Errors on ActiveMQ cluster setup

I am trying to setup a cluster of ActiveMQ, using static mode. I have for the moment only 2 instances. One is TE5021 / 192.165.20.21, with this configuration file (extract):
<beans>
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="te5021" dataDirectory="${activemq.data}" persistent="false" schedulePeriodForDestinationPurge="10000" useJmx="false">
<networkConnectors>
<networkConnector uri="static:(tcp://192.165.20.22:61616)" dynamicOnly="true" networkTTL="3" duplex="true" prefetchSize="1" decreaseNetworkConsumerPriority="false"/>
</networkConnectors>
<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=154857600&timeout=10000&soTimeout=10000" enableStatusMonitor="true" rebalanceClusterClients="true" updateClusterClients="true" updateClusterClientsOnRemove="true"/>
</transportConnectors>
</broker>
</beans>
The second one is TE5022 / 192.165.20.22 with this configuration file (extract):
<beans>
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="te5022" dataDirectory="${activemq.data}" persistent="false" schedulePeriodForDestinationPurge="10000" useJmx="false">
<networkConnectors>
<networkConnector uri="static:(tcp://192.165.20.21:61616)" dynamicOnly="true" networkTTL="3" duplex="true" prefetchSize="1" decreaseNetworkConsumerPriority="false"/>
</networkConnectors>
<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=154857600&timeout=10000&soTimeout=10000" enableStatusMonitor="true" rebalanceClusterClients="true" updateClusterClients="true" updateClusterClientsOnRemove="true"/>
</transportConnectors>
</broker>
</beans>
On the first one, everything seems to works as expected. But on the second one, it show me these errors :
2021-11-02 10:33:23,680 | INFO | Establishing network connection from vm://te5022 to tcp://192.165.20.21:61616 | org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-1
2021-11-02 10:33:23,692 | WARN | Failed to add Connection id=te5022->te5021-38812-1635845361826-27:1, clientId=NC_te5021_inbound_te5022, clientIP=vm://te5022#52 due to Broker: te5022 - Client: NC_te5021_inbound_te5022 already connected from vm://te5022#12 | org.apache.activemq.broker.TransportConnection | triggerStartAsyncNetworkBridgeCreation: remoteBroker=tcp:///192.165.20.21:61616#42870, localBroker= vm://te5022#52
2021-11-02 10:33:23,693 | INFO | Network connection between vm://te5022#52 and tcp:///192.165.20.21:61616#42870 shutdown due to a local error: {} | org.apache.activemq.network.DemandForwardingBridgeSupport | triggerStartAsyncNetworkBridgeCreation: remoteBroker=tcp:///192.165.20.21:61616#42870, localBroker= vm://te5022#52
javax.jms.InvalidClientIDException: Broker: te5022 - Client: NC_te5021_inbound_te5022 already connected from vm://te5022#12
at org.apache.activemq.broker.region.RegionBroker.addConnection(RegionBroker.java:265)
at org.apache.activemq.broker.jmx.ManagedRegionBroker.addConnection(ManagedRegionBroker.java:227)
at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:99)
at org.apache.activemq.advisory.AdvisoryBroker.addConnection(AdvisoryBroker.java:119)
at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:99)
at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:99)
at org.apache.activemq.broker.BrokerFilter.addConnection(BrokerFilter.java:99)
at org.apache.activemq.broker.TransportConnection.processAddConnection(TransportConnection.java:844)
at org.apache.activemq.command.ConnectionInfo.visit(ConnectionInfo.java:139)
at org.apache.activemq.broker.TransportConnection.service(TransportConnection.java:331)
at org.apache.activemq.broker.TransportConnection$1.onCommand(TransportConnection.java:200)
at org.apache.activemq.transport.ResponseCorrelator.onCommand(ResponseCorrelator.java:116)
at org.apache.activemq.transport.MutexTransport.onCommand(MutexTransport.java:50)
at org.apache.activemq.transport.vm.VMTransport.doDispatch(VMTransport.java:165)
at org.apache.activemq.transport.vm.VMTransport.dispatch(VMTransport.java:157)
at org.apache.activemq.transport.vm.VMTransport.oneway(VMTransport.java:134)
at org.apache.activemq.transport.MutexTransport.oneway(MutexTransport.java:68)
at org.apache.activemq.transport.ResponseCorrelator.asyncRequest(ResponseCorrelator.java:81)
at org.apache.activemq.transport.ResponseCorrelator.request(ResponseCorrelator.java:86)
at org.apache.activemq.network.DemandForwardingBridgeSupport.startLocalBridge(DemandForwardingBridgeSupport.java:510)
at org.apache.activemq.network.DemandForwardingBridgeSupport.doStartLocalAndRemoteBridges(DemandForwardingBridgeSupport.java:462)
at org.apache.activemq.network.DemandForwardingBridgeSupport.access$600(DemandForwardingBridgeSupport.java:113)
at org.apache.activemq.network.DemandForwardingBridgeSupport$5.run(DemandForwardingBridgeSupport.java:372)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1130)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:630)
at java.base/java.lang.Thread.run(Thread.java:832)
2021-11-02 10:33:23,719 | INFO | te5022 bridge to te5021 stopped | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ BrokerService[te5022] Task-9
2021-11-02 10:33:25,705 | INFO | Stopping vm://te5022#52 because Failed with SecurityException: Broker: te5022 - Client: NC_te5021_inbound_te5022 already connected from vm://te5022#12 | org.apache.activemq.broker.TransportConnection | ActiveMQ BrokerService[te5022] Task-10
What am I doing wrong ?
Ok, here was the problem : I did setup the network connectors with duplex = true.
In my case, setting duplex = false, and broker attribute networkConnectorStartAsync to true was the solution.

Good use of bridged MQTT brokers

I'm currently working on a project on which an external app sends data coming from many sensors via MQTT protocol.
I want to collect all of this data, and I want to send them to an external server. I want to create 2 MQTT brokers:
one local (on the machine with the app that sends data)
one in the distant server
I will create a network bridge between the two. It's a possibility given by my MQTT server app ActiveMQ (I imagine that's a common feature).
In this way the data producing app will publish on the local broker and, via the bridge, the same data will be published on the remote broker. The point is to let the app working without problems in case of connection loss.
When I lose the network connection between the brokers I don't get the data produced by the app during the time there were no connection. Do you know if it's possible to configure the bridge in order to make it work the way I want?
Will I have to develop a little program which listens on all topics from the local broker, detects connection losses, and re-sends all lost messages to the remote broker?
I add configuration files from my two brokers. My first ActiveMQ server is on the same machine as my app and the second ActiveMQ server is on another machine on the same network. Both computers ping each other perfectly.
Local broker:
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<value>file:${activemq.conf}/credentials.properties</value>
</property>
</bean>
<bean id="logQuery" class="io.fabric8.insight.log.log4j.Log4jLogQuery"
lazy-init="false" scope="singleton"
init-method="start" destroy-method="stop">
</bean>
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" >
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<persistenceAdapter>
<kahaDB directory="${activemq.data}/kahadb"/>
</persistenceAdapter>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage percentOfJvmHeap="70" />
</memoryUsage>
<storeUsage>
<storeUsage limit="100 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="50 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
</transportConnectors>
<networkConnectors>
<networkConnector uri="static:(tcp://192.168.16.100:61616)"/>
</networkConnectors>
<shutdownHooks>
<bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook" />
</shutdownHooks>
</broker>
<import resource="jetty.xml"/>
</beans>
Remote broker:
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<value>file:${activemq.conf}/credentials.properties</value>
</property>
</bean>
<bean id="logQuery" class="io.fabric8.insight.log.log4j.Log4jLogQuery"
lazy-init="false" scope="singleton"
init-method="start" destroy-method="stop">
</bean>
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" >
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<persistenceAdapter>
<kahaDB directory="${activemq.data}/kahadb"/>
</persistenceAdapter>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage percentOfJvmHeap="70" />
</memoryUsage>
<storeUsage>
<storeUsage limit="100 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="50 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
</transportConnectors>
<shutdownHooks>
<bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook" />
</shutdownHooks>
</broker>
<import resource="jetty.xml"/>
</beans>
In order to simulate disconnection between the two brokers I simply disconnects the second computer from the network.
I use MQTTBox on both computer to subscribe to topics I write on. That's how I saw that data sent on a topic in the local broker during the disconnection of the second computer is not published on the same topic of the remote broker when I reconnect it.
EDIT : new infos
I tried again my test today and I notice a checkbox "retain" on my MQTT client MQTTBox.
So :
With the computer A, I publish a message with retain checked on topic /test and computer B was listening on /#
When the 2 computers are connected, it obviously works well, I see the message on computer B.
When I disconnect computer B, publish 2 messages with retain checked then reconnect computer B, I only see the most recent of the 2 messages I published...
It's better, but I'd like to see the other message too... If anyone can help me, i'm lost...
I can also set a QoS for the message I want to publish. I tried with Qos = 0 and QoS = 1 : same thing.
QOS for messages works for bridge connections as well.
So if the bridge is configured for a topic with a QOS greater than 0 then the local broker will queue up the messages while the connection to the remote broker is down and will send them when the connection comes back up.
This way no messages will be lost.
This is perfectly normal deployment pattern for MQTT brokers.

Integration of Apache ActiveMQ to Windows Server 2012 R2 Active Directory

For Apache ActiveMQ
I have one CentOS 7 VM. In this VM I installed Apache ActiveMQ v5.13.3.
My Apache ActiveMQ admin URL is http://192.168.1.186:8161/admin
For Active Directory
I have Windows Server 2012 R2 VM. In this VM I have Active Directory.
IP Address 192.168.1.101/testlabserver.testlab.local
I want to login in ActiveMQ admin URL "http://192.168.1.186:8161/admin" using my Active Directory users.
For this i follow
http://steamingpileofsoftware.blogspot.in/2013/10/secure-your-jetty-activemq-web-console.html
conf/activemq.xml file content
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<value>file:${activemq.conf}/credentials.properties</value>
</property>
</bean>
<bean id="logQuery" class="io.fabric8.insight.log.log4j.Log4jLogQuery"
lazy-init="false" scope="singleton"
init-method="start" destroy-method="stop">
</bean>
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">">
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<managementContext>
<managementContext createConnector="true"/>
</managementContext>
<persistenceAdapter>
<kahaDB directory="${activemq.data}/kahadb"/>
</persistenceAdapter>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage percentOfJvmHeap="70" />
</memoryUsage>
<storeUsage>
<storeUsage limit="100 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="50 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<transportConnectors>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="stomp" uri="stomp://0.0.0.0:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="ws" uri="ws://0.0.0.0:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
</transportConnectors>
<shutdownHooks>
<bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook" />
</shutdownHooks>
</broker>
<import resource="jetty.xml"/>
</beans>
conf/jetty.xml file content
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="securityLoginService" class="org.eclipse.jetty.security.HashLoginService">
<property name="name" value="ActiveMQRealm" />
<property name="config" value="${activemq.conf}/jetty-realm.properties" />
</bean>
<bean id="securityConstraint" class="org.eclipse.jetty.util.security.Constraint">
<property name="name" value="BASIC" />
<property name="roles" value="user,admin" />
<property name="authenticate" value="true" />
</bean>
<bean id="adminSecurityConstraint" class="org.eclipse.jetty.util.security.Constraint">
<property name="name" value="BASIC" />
<property name="roles" value="ActiveMQAdmin" />
<property name="authenticate" value="true" />
</bean>
<bean id="securityConstraintMapping" class="org.eclipse.jetty.security.ConstraintMapping">
<property name="constraint" ref="securityConstraint" />
<property name="pathSpec" value="/api/*,/admin/*,*.jsp" />
</bean>
<bean id="adminSecurityConstraintMapping" class="org.eclipse.jetty.security.ConstraintMapping">
<property name="constraint" ref="adminSecurityConstraint" />
<property name="pathSpec" value="*.action" />
</bean>
<bean id="rewriteHandler" class="org.eclipse.jetty.rewrite.handler.RewriteHandler">
<property name="rules">
<list>
<bean id="header" class="org.eclipse.jetty.rewrite.handler.HeaderPatternRule">
<property name="pattern" value="*"/>
<property name="name" value="X-FRAME-OPTIONS"/>
<property name="value" value="SAMEORIGIN"/>
</bean>
</list>
</property>
</bean>
<bean id="secHandlerCollection" class="org.eclipse.jetty.server.handler.HandlerCollection">
<property name="handlers">
<list>
<ref bean="rewriteHandler"/>
<bean class="org.eclipse.jetty.webapp.WebAppContext">
<property name="contextPath" value="/admin" />
<property name="resourceBase" value="${activemq.home}/webapps/admin" />
<property name="logUrlOnStart" value="true" />
</bean>
<bean class="org.eclipse.jetty.webapp.WebAppContext">
<property name="contextPath" value="/api" />
<property name="resourceBase" value="${activemq.home}/webapps/api" />
<property name="logUrlOnStart" value="true" />
</bean>
<bean class="org.eclipse.jetty.server.handler.ResourceHandler">
<property name="directoriesListed" value="false" />
<property name="welcomeFiles">
<list>
<value>index.html</value>
</list>
</property>
<property name="resourceBase" value="${activemq.home}/webapps/" />
</bean>
<bean id="defaultHandler" class="org.eclipse.jetty.server.handler.DefaultHandler">
<property name="serveIcon" value="false" />
</bean>
</list>
</property>
</bean>
<bean id="securityHandler" class="org.eclipse.jetty.security.ConstraintSecurityHandler">
<property name="loginService" ref="ldapLoginService" />
<property name="strict" value="false" />
<property name="identityService" ref="identityService" />
<property name="authenticator">
<bean class="org.eclipse.jetty.security.authentication.BasicAuthenticator" />
</property>
<property name="constraintMappings">
<list>
<ref bean="adminSecurityConstraintMapping" />
<ref bean="securityConstraintMapping" />
</list>
</property>
<property name="handler" ref="secHandlerCollection" />
</bean>
<bean id="contexts" class="org.eclipse.jetty.server.handler.ContextHandlerCollection">
</bean>
<bean id="jettyPort" class="org.apache.activemq.web.WebConsolePort" init-method="start">
<property name="host" value="0.0.0.0"/>
<property name="port" value="8161"/>
</bean>
<bean id="Server" depends-on="jettyPort" class="org.eclipse.jetty.server.Server"
destroy-method="stop">
<property name="handler">
<bean id="handlers" class="org.eclipse.jetty.server.handler.HandlerCollection">
<property name="handlers">
<list>
<ref bean="contexts" />
<ref bean="securityHandler" />
</list>
</property>
</bean>
</property>
</bean>
<bean id="invokeConnectors" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="targetObject" ref="Server" />
<property name="targetMethod" value="setConnectors" />
<property name="arguments">
<list>
<bean id="Connector" class="org.eclipse.jetty.server.ServerConnector">
<constructor-arg ref="Server" />
<property name="host" value="#{systemProperties['jetty.host']}" />
<property name="port" value="#{systemProperties['jetty.port']}" />
</bean>
</list>
</property>
</bean>
<bean id="configureJetty" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean">
<property name="staticMethod" value="org.apache.activemq.web.config.JspConfigurer.configureJetty" />
<property name="arguments">
<list>
<ref bean="Server" />
<ref bean="secHandlerCollection" />
</list>
</property>
</bean>
<bean id="invokeStart" class="org.springframework.beans.factory.config.MethodInvokingFactoryBean"
depends-on="configureJetty, invokeConnectors">
<property name="targetObject" ref="Server" />
<property name="targetMethod" value="start" />
</bean>
<bean id="ldapLoginService" class="org.eclipse.jetty.plus.jaas.JAASLoginService">
<property name="name" value="LdapRealm" />
<property name="loginModuleName" value="amqLdapLoginModule" />
</bean>
<bean id="identityService" class="org.eclipse.jetty.security.DefaultIdentityService"/>
</beans>
conf/ldap.conf file content
amqLdapLoginModule {
org.eclipse.jetty.plus.jaas.spi.LdapLoginModule required
debug="true"
contextFactory="com.sun.jndi.ldap.LdapCtxFactory"
hostname="192.168.1.101"
port="636"
authenticationMethod="simple"
forceBindingLogin="true"
userBaseDn="OU=TestUser,DC=TESTLAB,DC=local"
userRdnAttribute="sAMAccountName"
userIdAttribute="sAMAccountName"
userPasswordAttribute="unicodePwd"
userObjectClass="user"
roleBaseDn="OU=TestUser,DC=TESTLAB,DC=local"
roleNameAttribute="cn"
roleMemberAttribute="member"
roleObjectClass="group";
};
After this configuration when i start my ActiveMQ it gives me error.
My ActiveMQ logs are below
2016-07-03 16:51:13,546 | INFO | Listening for connections at: tcp://activemq.testlab.local:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2016-07-03 16:51:13,549 | INFO | Connector openwire started | org.apache.activemq.broker.TransportConnector | main
2016-07-03 16:51:13,553 | INFO | Listening for connections at: amqp://activemq.testlab.local:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2016-07-03 16:51:13,556 | INFO | Connector amqp started | org.apache.activemq.broker.TransportConnector | main
2016-07-03 16:51:13,560 | INFO | Listening for connections at: stomp://activemq.testlab.local:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2016-07-03 16:51:13,562 | INFO | Connector stomp started | org.apache.activemq.broker.TransportConnector | main
2016-07-03 16:51:13,565 | INFO | Listening for connections at: mqtt://activemq.testlab.local:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2016-07-03 16:51:13,567 | INFO | Connector mqtt started | org.apache.activemq.broker.TransportConnector | main
2016-07-03 16:51:13,842 | WARN | ServletContext#o.e.j.s.ServletContextHandler#62010f5c{/,null,STARTING} has uncovered http methods for path: / | org.eclipse.jetty.security.SecurityHandler | main
2016-07-03 16:51:13,886 | INFO | Listening for connections at ws://activemq.testlab.local:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.ws.WSTransportServer | main
2016-07-03 16:51:13,889 | INFO | Connector ws started | org.apache.activemq.broker.TransportConnector | main
2016-07-03 16:51:13,892 | INFO | Apache ActiveMQ 5.13.3 (localhost, ID:activemq.testlab.local-33670-1467544873205-0:1) started | org.apache.activemq.broker.BrokerService | main
2016-07-03 16:51:13,894 | INFO | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main
2016-07-03 16:51:13,897 | WARN | Store limit is 102400 mb (current store usage is 20 mb). The data directory: /opt/apache-activemq-5.13.3/data/kahadb only has 33047 mb of usable space. - resetting to maximum available disk space: 33047 mb | org.apache.activemq.broker.BrokerService | main
2016-07-03 16:51:13,899 | WARN | Temporary Store limit is 51200 mb (current store usage is 0 mb). The data directory: /opt/apache-activemq-5.13.3/data only has 33027 mb of usable space. - resetting to maximum available disk space: 33027 mb | org.apache.activemq.broker.BrokerService | main
2016-07-03 16:51:14,002 | WARN | Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'securityHandler' defined in class path resource [jetty.xml]: Cannot resolve reference to bean 'ldapLoginService' while setting bean property 'loginService'; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [org.eclipse.jetty.plus.jaas.JAASLoginService] for bean with name 'ldapLoginService' defined in class path resource [jetty.xml]; nested exception is java.lang.ClassNotFoundException: org.eclipse.jetty.plus.jaas.JAASLoginService | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2016-07-03 16:51:14,007 | INFO | Apache ActiveMQ 5.13.3 (localhost, ID:activemq.testlab.local-33670-1467544873205-0:1) is shutting down | org.apache.activemq.broker.BrokerService | main
2016-07-03 16:51:14,011 | INFO | Connector openwire stopped | org.apache.activemq.broker.TransportConnector | main
2016-07-03 16:51:14,012 | INFO | Connector amqp stopped | org.apache.activemq.broker.TransportConnector | main
2016-07-03 16:51:14,013 | INFO | Connector stomp stopped | org.apache.activemq.broker.TransportConnector | main
2016-07-03 16:51:14,013 | INFO | socketQueue interrupted - stopping | org.apache.activemq.transport.tcp.TcpTransportServer | ActiveMQ Transport Server Thread Handler: amqp://0.0.0.0:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600
2016-07-03 16:51:14,013 | INFO | socketQueue interrupted - stopping | org.apache.activemq.transport.tcp.TcpTransportServer | ActiveMQ Transport Server Thread Handler: stomp://0.0.0.0:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600
2016-07-03 16:51:14,016 | INFO | socketQueue interrupted - stopping | org.apache.activemq.transport.tcp.TcpTransportServer | ActiveMQ Transport Server Thread Handler: mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600
2016-07-03 16:51:14,023 | ERROR | Could not accept connection : java.lang.InterruptedException | org.apache.activemq.broker.TransportConnector | ActiveMQ Transport Server Thread Handler: mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600
2016-07-03 16:51:14,016 | INFO | Connector mqtt stopped | org.apache.activemq.broker.TransportConnector | main
2016-07-03 16:51:14,021 | ERROR | Could not accept connection : java.lang.InterruptedException | org.apache.activemq.broker.TransportConnector | ActiveMQ Transport Server Thread Handler: stomp://0.0.0.0:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600
2016-07-03 16:51:14,020 | ERROR | Could not accept connection : java.lang.InterruptedException | org.apache.activemq.broker.TransportConnector | ActiveMQ Transport Server Thread Handler: amqp://0.0.0.0:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600
2016-07-03 16:51:14,043 | INFO | Connector ws stopped | org.apache.activemq.broker.TransportConnector | main
2016-07-03 16:51:14,051 | INFO | PListStore:[/opt/apache-activemq-5.13.3/data/localhost/tmp_storage] stopped | org.apache.activemq.store.kahadb.plist.PListStoreImpl | main
2016-07-03 16:51:14,052 | INFO | Stopping async queue tasks | org.apache.activemq.store.kahadb.KahaDBStore | main
2016-07-03 16:51:14,053 | INFO | Stopping async topic tasks | org.apache.activemq.store.kahadb.KahaDBStore | main
2016-07-03 16:51:14,055 | INFO | Stopped KahaDB | org.apache.activemq.store.kahadb.KahaDBStore | main
2016-07-03 16:51:14,127 | INFO | Apache ActiveMQ 5.13.3 (localhost, ID:activemq.testlab.local-33670-1467544873205-0:1) uptime 1.387 seconds | org.apache.activemq.broker.BrokerService | main
2016-07-03 16:51:14,128 | INFO | Apache ActiveMQ 5.13.3 (localhost, ID:activemq.testlab.local-33670-1467544873205-0:1) is shutdown | org.apache.activemq.broker.BrokerService | main
2016-07-03 16:51:14,130 | INFO | Closing org.apache.activemq.xbean.XBeanBrokerFactory$1#67b64c45: startup date [Sun Jul 03 16:51:08 IST 2016]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2016-07-03 16:51:14,138 | WARN | Exception thrown from LifecycleProcessor on context close | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
java.lang.IllegalStateException: LifecycleProcessor not initialized - call 'refresh' before invoking lifecycle methods via the context: org.apache.activemq.xbean.XBeanBrokerFactory$1#67b64c45: startup date [Sun Jul 03 16:51:08 IST 2016]; root of context hierarchy
at org.springframework.context.support.AbstractApplicationContext.getLifecycleProcessor(AbstractApplicationContext.java:357)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.doClose(AbstractApplicationContext.java:884)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.close(AbstractApplicationContext.java:843)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.apache.activemq.hooks.SpringContextHook.run(SpringContextHook.java:30)[activemq-spring-5.13.3.jar:5.13.3]
at org.apache.activemq.broker.BrokerService.stop(BrokerService.java:873)[activemq-broker-5.13.3.jar:5.13.3]
at org.apache.activemq.xbean.XBeanBrokerService.stop(XBeanBrokerService.java:122)[activemq-spring-5.13.3.jar:5.13.3]
at org.apache.activemq.xbean.XBeanBrokerService.destroy(XBeanBrokerService.java:115)[activemq-spring-5.13.3.jar:5.13.3]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_91]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_91]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_91]
at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_91]
at org.springframework.beans.factory.support.DisposableBeanAdapter.invokeCustomDestroyMethod(DisposableBeanAdapter.java:351)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.DisposableBeanAdapter.destroy(DisposableBeanAdapter.java:274)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroyBean(DefaultSingletonBeanRegistry.java:578)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingleton(DefaultSingletonBeanRegistry.java:554)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingleton(DefaultListableBeanFactory.java:900)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.destroySingletons(DefaultSingletonBeanRegistry.java:523)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.destroySingletons(DefaultListableBeanFactory.java:907)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.destroyBeans(AbstractApplicationContext.java:915)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:493)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)[xbean-spring-3.18.jar:3.18]
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)[xbean-spring-3.18.jar:3.18]
at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)[activemq-spring-5.13.3.jar:5.13.3]
at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)[activemq-spring-5.13.3.jar:5.13.3]
at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)[activemq-spring-5.13.3.jar:5.13.3]
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)[activemq-broker-5.13.3.jar:5.13.3]
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)[activemq-broker-5.13.3.jar:5.13.3]
at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)[activemq-console-5.13.3.jar:5.13.3]
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)[activemq-console-5.13.3.jar:5.13.3]
at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154)[activemq-console-5.13.3.jar:5.13.3]
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)[activemq-console-5.13.3.jar:5.13.3]
at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)[activemq-console-5.13.3.jar:5.13.3]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_91]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_91]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_91]
at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_91]
at org.apache.activemq.console.Main.runTaskClass(Main.java:262)[activemq.jar:5.13.3]
at org.apache.activemq.console.Main.main(Main.java:115)[activemq.jar:5.13.3]
2016-07-03 16:51:14,146 | ERROR | Failed to load: class path resource [activemq.xml], reason: Error creating bean with name 'securityHandler' defined in class path resource [jetty.xml]: Cannot resolve reference to bean 'ldapLoginService' while setting bean property 'loginService'; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [org.eclipse.jetty.plus.jaas.JAASLoginService] for bean with name 'ldapLoginService' defined in class path resource [jetty.xml]; nested exception is java.lang.ClassNotFoundException: org.eclipse.jetty.plus.jaas.JAASLoginService | org.apache.activemq.xbean.XBeanBrokerFactory | main
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'securityHandler' defined in class path resource [jetty.xml]: Cannot resolve reference to bean 'ldapLoginService' while setting bean property 'loginService'; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [org.eclipse.jetty.plus.jaas.JAASLoginService] for bean with name 'ldapLoginService' defined in class path resource [jetty.xml]; nested exception is java.lang.ClassNotFoundException: org.eclipse.jetty.plus.jaas.JAASLoginService
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveReference(BeanDefinitionValueResolver.java:359)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.BeanDefinitionValueResolver.resolveValueIfNecessary(BeanDefinitionValueResolver.java:108)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.applyPropertyValues(AbstractAutowireCapableBeanFactory.java:1475)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.populateBean(AbstractAutowireCapableBeanFactory.java:1220)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:537)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:755)[spring-beans-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:762)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)[spring-context-4.1.9.RELEASE.jar:4.1.9.RELEASE]
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)[xbean-spring-3.18.jar:3.18]
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)[xbean-spring-3.18.jar:3.18]
at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)[activemq-spring-5.13.3.jar:5.13.3]
at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)[activemq-spring-5.13.3.jar:5.13.3]
at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)[activemq-spring-5.13.3.jar:5.13.3]
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)[activemq-broker-5.13.3.jar:5.13.3]
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)[activemq-broker-5.13.3.jar:5.13.3]
at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)[activemq-console-5.13.3.jar:5.13.3]
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)[activemq-console-5.13.3.jar:5.13.3]
at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154)[activemq-console-5.13.3.jar:5.13.3]
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)[activemq-console-5.13.3.jar:5.13.3]
at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)[activemq-console-5.13.3.jar:5.13.3]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_91]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_91]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_91]
at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_91]
at org.apache.activemq.console.Main.runTaskClass(Main.java:262)[activemq.jar:5.13.3]
at org.apache.activemq.console.Main.main(Main.java:115)[activemq.jar:5.13.3]
Please help me to resolved this issue.
The root of your problem is the following exception, which is causing your Spring context initialization to fail:
java.lang.ClassNotFoundException: org.eclipse.jetty.plus.jaas.JAASLoginService
This means that this class is missing from the classpath. If you're running Jetty 8 still, try adding jetty-plus.jar or jetty-all.jar to your classpath.
If you're running Jetty 9, this is because the package name for the class has changed to org.eclipse.jetty.jaas.JAASLoginService. Try updating your jetty.xml file to use the new class, and make sure to include jetty-jaas.jar in your classpath.

ActiveMQ remote connections refused despite 0.0.0.0 in broker URL

I have an ActiveMQ v5.7.0 broker, running in Karaf v2.3.3, that I want to enable for remote connections. I've set the broker URL to 0.0.0.0:61616, to enable it to listen to network traffic. I've opened the firewall to allow the traffic from the client machines. However, all remote connections are being refused. A quick netstat seems to tell me that the broker isn't listening outside of localhost.
jeremy#server:~$ netstat -pan | grep 61616
tcp6 0 0 127.0.0.1:61616 :::* LISTEN -
Looking at the broker via Hawtio tells me that the URL looks as it should.
Transport connectors Openwire: tcp://0.0.0.0:61616?maximumConnections=1000&wireformat.maxFrameSize=104857600
The firewall is definitely OK, as the connections are being refused rather than just being dropped.
The broker is responding correctly to connections from localhost.
2013-10-14 17:34:29 Connected to localhost:61613
This is the sort of error I get from remote connections:-
Error connecting to xxx.xxx.xxx.xxx:61613: IO::Socket::INET: connect: Connection refused at /usr/local/share/perl/5.14.2/Net/Stomp.pm line 102.
EDIT: telnet output added
Localhost port 61613
jeremy#server:~$ telnet localhost 61613
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
Remote connection port 61613
jeremy#other-server:~$ telnet xxx.xxx.xxx.xxx 61613
Trying xxx.xxx.xxx.xxx...
telnet: Unable to connect to remote host: Connection refused
Localhost connection port 61616 (this one is interesting)
jeremy#server:~$ telnet localhost 61616
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
ðActiveMQ Þ
MaxFrameSizÿÿÿ CacheSize
CacheEnabledSizePrefixDisabled MaxInactivityDurationInitalDelay'TcpNoDelayEnabledMaxInactivityDurationu0TightEncodingEnabledStackTraceEnabledPuTTYConnection closed by foreign host.
Remote connection port 61616
jeremy#other-server:~$ telnet xxx.xxx.xxx.xxx 61616
Trying xxx.xxx.xxx.xxx...
telnet: Unable to connect to remote host: Connection refused
EDIT: remote server karaf log output added
2013-10-15 19:00:46,599 | ERROR | c.event.invited] | faultJmsMessageListenerContainer | .DefaultMessageListenerContainer 909 | 69 - org.springframework.jms - 3.2.4.RELEASE | Could not refresh JMS Connection for destination 'Consumer.notifications.VirtualTopic.event.invited' - retrying in 5000 ms. Cause: Error while attempting to add new Connection to the pool; nested exception is javax.jms.JMSException: Could not connect to broker URL: tcp://xxx.xxx.xxx.xxx:61616. Reason: java.net.ConnectException: Connection refused
Here's the broker.xml.
<?xml version="1.0" encoding="UTF-8"?>
<blueprint xmlns="http://www.osgi.org/xmlns/blueprint/v1.0.0"
xmlns:cm="http://aries.apache.org/blueprint/xmlns/blueprint-cm/v1.0.0"
xmlns:ext="http://aries.apache.org/blueprint/xmlns/blueprint-ext/v1.0.0"
xmlns:amq="http://activemq.apache.org/schema/core">
<ext:property-placeholder />
<broker xmlns="http://activemq.apache.org/schema/core"
brokerName="jellyfish-messaging"
dataDirectory="${karaf.data}/activemq/localhost"
useShutdownHook="false"
persistent="true"
schedulerSupport="true"
startAsync="true">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true" memoryLimit="1mb">
<pendingSubscriberPolicy>
<vmCursor />
</pendingSubscriberPolicy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb">
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<persistenceAdapter>
<kahaDB directory="${karaf.data}/activemq/localhost/kahadb"/>
</persistenceAdapter>
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="64 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="100 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="50 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<!-- The transport connectors ActiveMQ will listen to -->
<transportConnectors>
<!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
<transportConnector name="stomp" uri="stomp://0.0.0.0:61613?maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
</transportConnectors>
</broker>
<bean id="jmsConnectionFactory" class="org.apache.activemq.ActiveMQConnectionFactory">
<property name="brokerURL" value="tcp://0.0.0.0:61616" />
</bean>
<bean id="pooledConnectionFactory" class="org.apache.activemq.pool.PooledConnectionFactory">
<property name="maxConnections" value="8" />
<property name="maximumActive" value="500" />
<property name="connectionFactory" ref="jmsConnectionFactory" />
</bean>
<bean id="resourceManager" class="org.apache.activemq.pool.ActiveMQResourceManager" init-method="recoverResource">
<property name="transactionManager" ref="transactionManager" />
<property name="connectionFactory" ref="jmsConnectionFactory" />
<property name="resourceName" value="activemq.localhost" />
</bean>
<bean id="jmsConfig" class="org.apache.camel.component.jms.JmsConfiguration">
<property name="connectionFactory" ref="pooledConnectionFactory" />
<property name="transacted" value="false" />
<property name="concurrentConsumers" value="10" />
</bean>
<bean id="activemq" class="org.apache.activemq.camel.component.ActiveMQComponent">
<property name="configuration" ref="jmsConfig" />
</bean>
<reference id="transactionManager" interface="javax.transaction.TransactionManager" />
<service ref="pooledConnectionFactory" interface="javax.jms.ConnectionFactory">
<service-properties>
<entry key="name" value="localhost"/>
</service-properties>
</service>
</blueprint>
Can anyone tell me what I'm missing?
Thanks,
J.
I've solved this. It was neither a problem with the firewall, nor with the ActiveMQ configuration.
The Karaf kar file in which the ActiveMQ broker was defined included the activemq-web-console feature. We've not been using this feature, as we're fans of Hawtio, so had never configured it.
As per this blog post, the console was coming up with default settings, including listening on port 61616. This meant that two brokers were in a race condition on start-up and the webconsole-defined one was generally winning. Since by default it isn't configured for remote access, it was locking the port for localhost connections only.
The giveaway was a directory called ${activemq.data} (literally) within the Karaf home directory, containing a second Kahadb repository. All of our broker config was set to use the data directory and we've never specifically set the ActiveMQ environment variables, so this led us to look for where a second broker might have come from.
Might have spotted it more quickly had we done activemq:list inside a Karaf session, as it was listing two brokers.
Simple solution - delete activemq-web-console from the features XML.

How to test this failover activemq queue case

i got my activemq config file and my mule flows with the "jms:activemq-connector" like this:
<jms:activemq-connector name="Active_MQ" specification="1.1" brokerURL="failover://(tcp://localhost:61616,tcp://localhost:61617)?randomize=false" validateConnections="false" maxRedelivery="1" doc:name="Active MQ"/>
This is my activemq.xml
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<!-- Allows us to use system properties as variables in this configuration file -->
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<value>file:${activemq.conf}/credentials.properties</value>
</property>
</bean>
<!--
The <broker> element is used to configure the ActiveMQ broker.
-->
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}" persistent="false">
<!--
For better performances use VM cursor and small memory limit.
For more information, see:
http://activemq.apache.org/message-cursors.html
Also, if your producer is "hanging", it's probably due to producer flow control.
For more information, see:
http://activemq.apache.org/producer-flow-control.html
-->
<destinations>
<queue physicalName="Example.Queue"/>
</destinations>
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" producerFlowControl="true">
<!-- The constantPendingMessageLimitStrategy is used to prevent
slow topic consumers to block producers and affect other consumers
by limiting the number of messages that are retained
For more information, see:
http://activemq.apache.org/slow-consumer-handling.html
-->
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>
</policyEntry>
<policyEntry queue=">" producerFlowControl="true" memoryLimit="1mb" useCache="true">
<!-- Use VM cursor for better latency
For more information, see:
http://activemq.apache.org/message-cursors.html
<pendingQueuePolicy>
<vmQueueCursor/>
</pendingQueuePolicy>
-->
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<!--
The managementContext is used to configure how ActiveMQ is exposed in
JMX. By default, ActiveMQ uses the MBean server that is started by
the JVM. For more information, see:
http://activemq.apache.org/jmx.html
-->
<managementContext>
<managementContext createConnector="false"/>
</managementContext>
<!--
Configure message persistence for the broker. The default persistence
mechanism is the KahaDB store (identified by the kahaDB tag).
For more information, see:
http://activemq.apache.org/persistence.html
-->
<persistenceAdapter>
<kahaDB directory="${activemq.data}/kahadb" cleanupInterval="30000"/>
</persistenceAdapter>
<!--
The systemUsage controls the maximum amount of space the broker will
use before slowing down producers. For more information, see:
http://activemq.apache.org/producer-flow-control.html
If using ActiveMQ embedded - the following limits could safely be used:
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="20 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="1 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="100 mb"/>
</tempUsage>
</systemUsage>
</systemUsage>
-->
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage limit="64 mb"/>
</memoryUsage>
<storeUsage>
<storeUsage limit="100 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="50 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<!--
The transport connectors expose ActiveMQ over a given protocol to
clients and other brokers. For more information, see:
http://activemq.apache.org/configuring-transports.html
-->
<transportConnectors>
<!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
<transportConnector uri="tcp://localhost:61617?maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
<transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=1000&wireformat.maxFrameSize=104857600"/>
</transportConnectors>
<!-- destroy the spring context on shutdown to stop jetty -->
<shutdownHooks>
<bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook" />
</shutdownHooks>
</broker>
<!--
Enable web consoles, REST and Ajax APIs and demos
Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
-->
<import resource="jetty.xml"/>
</beans>
Looks like the failover with those url's "works".
Question, how am i supposed to test the failover??? any suggestions? this is the very first time i implement this.
Thanks.
Per requested:
I actually ran 2 different activemq's in separate cmd prompt, configured the conf files inside each activemq folder. That's all.
It worked by doing "failover://(tcp://localhost:61616,tcp://localhost:61617)" on Mule side.
After shutting down the 61616 one, all the entries would go to the second broker, the 61617 one. so it works great.