Active MQ warnings filling up logs - activemq

I see the below in Active MQ logs all the time and its filling up the log files and making to difficult to read useful log messages. What does this warning mean and how do I fix this? I tried some solutions that I found using google, but none of them helped. Can you help me understand what this log means and how do I fix this? Active MQ version 5.13.0 64 bit Windows 2012
2018-04-26 16:43:22,072 | WARN | Transport Connection to: tcp://127.0.0.1:60736 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:60736#61616
2018-04-26 16:48:18,554 | WARN | Transport Connection to: tcp://127.0.0.1:50693 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:50693#61616
2018-04-26 16:48:18,554 | WARN | Transport Connection to: tcp://127.0.0.1:50690 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:50690#61616
2018-04-26 16:48:18,554 | WARN | Transport Connection to: tcp://127.0.0.1:50689 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:50689#61616
2018-04-26 16:48:18,554 | WARN | Transport Connection to: tcp://127.0.0.1:50553 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:50553#61616
Below ismy activemq.xml config
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<!-- START SNIPPET: example -->
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<!-- Allows us to use system properties as variables in this configuration file -->
<bean class="org.springframework.beans.factory.config.PropertyPlaceholderConfigurer">
<property name="locations">
<value>file:${activemq.conf}/credentials.properties</value>
</property>
</bean>
<!-- Allows accessing the server log -->
<bean id="logQuery" class="io.fabric8.insight.log.log4j.Log4jLogQuery"
lazy-init="false" scope="singleton"
init-method="start" destroy-method="stop">
</bean>
<!--
The <broker> element is used to configure the ActiveMQ broker.
-->
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="EnvironmentName" dataDirectory="${activemq.data}">
<destinationPolicy>
<policyMap>
<policyEntries>
<policyEntry topic=">" >
<!-- The constantPendingMessageLimitStrategy is used to prevent
slow topic consumers to block producers and affect other consumers
by limiting the number of messages that are retained
For more information, see:
http://activemq.apache.org/slow-consumer-handling.html
-->
<pendingMessageLimitStrategy>
<constantPendingMessageLimitStrategy limit="1000"/>
</pendingMessageLimitStrategy>
</policyEntry>
</policyEntries>
</policyMap>
</destinationPolicy>
<!--
The managementContext is used to configure how ActiveMQ is exposed in
JMX. By default, ActiveMQ uses the MBean server that is started by
the JVM. For more information, see:
http://activemq.apache.org/jmx.html
-->
<managementContext>
<!--managementContext createConnector="false"/ -->
<managementContext createConnector="true" connectorPort="1616"/>
</managementContext>
<!--
Configure message persistence for the broker. The default persistence
mechanism is the KahaDB store (identified by the kahaDB tag).
For more information, see:
http://activemq.apache.org/persistence.html
-->
<persistenceAdapter>
<jdbcPersistenceAdapter dataDirectory="${activemq.data}" dataSource="#oracle-ds"/>
</persistenceAdapter>
<!--
The systemUsage controls the maximum amount of space the broker will
use before disabling caching and/or slowing down producers. For more information, see:
http://activemq.apache.org/producer-flow-control.html
-->
<systemUsage>
<systemUsage>
<memoryUsage>
<memoryUsage percentOfJvmHeap="70" />
</memoryUsage>
<storeUsage>
<storeUsage limit="100 gb"/>
</storeUsage>
<tempUsage>
<tempUsage limit="50 gb"/>
</tempUsage>
</systemUsage>
</systemUsage>
<!--
The transport connectors expose ActiveMQ over a given protocol to
clients and other brokers. For more information, see:
http://activemq.apache.org/configuring-transports.html
-->
<transportConnectors>
<!-- DOS protection, limit concurrent connections to 1000 and frame size to 100MB -->
<transportConnector name="openwire" uri="tcp://0.0.0.0:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="amqp" uri="amqp://0.0.0.0:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="stomp" uri="stomp://0.0.0.0:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="mqtt" uri="mqtt://0.0.0.0:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
<transportConnector name="ws" uri="ws://0.0.0.0:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600"/>
</transportConnectors>
<!-- destroy the spring context on shutdown to stop jetty -->
<shutdownHooks>
<bean xmlns="http://www.springframework.org/schema/beans" class="org.apache.activemq.hooks.SpringContextHook" />
</shutdownHooks>
<plugins>
<jaasAuthenticationPlugin configuration="activemq" />
<authorizationPlugin>
<map>
<authorizationMap>
<authorizationEntries>
<authorizationEntry queue="*.>"
read="admins"
write="admins"
admin="admins" />
<authorizationEntry queue="sysQuename"
read="admins,sysusergrp"
write="admins,syssgrp"
admin="admins" />
<authorizationEntry queue="AnotherQueue"
read="admins,anothergroup"
write="admins,anothergroup"
admin="admins" />
<!-- <authorizationEntry topic="*.>"
read="all"
write="all"
admin="all" /> -->
<authorizationEntry topic="ActiveMQ.Advisory.>" read="admins,users,sysusergrp,anothergroup" write="admins,users, sysusergrp, anothergroup" admin="admins,users,sysusergrp,anothergroup"/>
<authorizationEntry topic="Topic1" read="admins" write="admins" admin="admins"/>
<authorizationEntry topic="Topic2" read="admins" write="admins" admin="admins"/>
</authorizationEntries>
<tempDestinationAuthorizationEntry>
<tempDestinationAuthorizationEntry
read="all"
write="all"
admin="all"/>
</tempDestinationAuthorizationEntry>
</authorizationMap>
</map>
</authorizationPlugin>
</plugins>
</broker>
<!-- Oracle DataSource Sample Setup -->
<bean id="oracle-ds" class="oracle.jdbc.pool.OracleDataSource" destroy-method="close">
<!--property name="driverClassName" value="oracle.jdbc.driver.OracleDriver"/>-->
<property name="URL" value="jdbc:oracle:thin:#server:port/schema"/>
<property name="user" value="username"/>
<property name="password" value="password"/>
<!--<property name="maxTotal" value="200"/>-->
<property name="connectionCachingEnabled" value="true"/>
<property name="connectionCacheProperties">
<props>
<prop key="MinLimit">0</prop>
<prop key="MaxLimit">100</prop>
</props>
</property>
<!--<property name="poolPreparedStatements" value="true"/>-->
<!--<property name="poolPreparedStatements" value="true"/>-->
</bean>
<!--
Enable web consoles, REST and Ajax APIs and demos
The web consoles requires by default login, you can disable this in the jetty.xml file
Take a look at ${ACTIVEMQ_HOME}/conf/jetty.xml for more details
-->
<import resource="jetty.xml"/>
</beans>
<!-- END SNIPPET: example -->
Below is the startup log of MQ
2018-04-26 16:58:27,089 | INFO | Database adapter driver override recognized for : [oracle_jdbc_driver] - adapter: class org.apache.activemq.store.jdbc.adapter.OracleJDBCAdapter | org.apache.activemq.store.jdbc.JDBCPersistenceAdapter | WrapperSimpleAppMain
2018-04-26 16:58:27,682 | INFO | Database lock driver override not found for : [oracle_jdbc_driver]. Will use default implementation. | org.apache.activemq.store.jdbc.JDBCPersistenceAdapter | WrapperSimpleAppMain
2018-04-26 16:58:27,682 | INFO | Attempting to acquire the exclusive lock to become the Master broker | org.apache.activemq.store.jdbc.DefaultDatabaseLocker | WrapperSimpleAppMain
2018-04-26 16:58:27,698 | INFO | Becoming the master on dataSource: oracle.jdbc.pool.OracleDataSource#730d22e3 | org.apache.activemq.store.jdbc.DefaultDatabaseLocker | WrapperSimpleAppMain
2018-04-26 16:58:28,073 | INFO | Apache ActiveMQ 5.13.0 (BrokerName, ID:HOSTNAME-49282-1524776307714-0:1) is starting | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2018-04-26 16:58:28,385 | INFO | Listening for connections at: tcp://HOSTNAME:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | WrapperSimpleAppMain
2018-04-26 16:58:28,385 | INFO | Connector openwire started | org.apache.activemq.broker.TransportConnector | WrapperSimpleAppMain
2018-04-26 16:58:28,401 | INFO | Listening for connections at: amqp://HOSTNAME:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | WrapperSimpleAppMain
2018-04-26 16:58:28,401 | INFO | Connector amqp started | org.apache.activemq.broker.TransportConnector | WrapperSimpleAppMain
2018-04-26 16:58:28,401 | INFO | Listening for connections at: stomp://HOSTNAME:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | WrapperSimpleAppMain
2018-04-26 16:58:28,401 | INFO | Connector stomp started | org.apache.activemq.broker.TransportConnector | WrapperSimpleAppMain
2018-04-26 16:58:28,417 | INFO | Listening for connections at: mqtt://HOSTNAME:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | WrapperSimpleAppMain
2018-04-26 16:58:28,448 | INFO | Connector mqtt started | org.apache.activemq.broker.TransportConnector | WrapperSimpleAppMain
2018-04-26 16:58:28,448 | WARN | Transport Connection to: tcp://IPADDRESS:49298 failed: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///IPADDRESS:49298#61616
2018-04-26 16:58:28,448 | WARN | Transport Connection to: tcp://IPADDRESS:49287 failed: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///IPADDRESS:49287#61616
2018-04-26 16:58:28,542 | WARN | Transport Connection to: tcp://IPADDRESS:49311 failed: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///IPADDRESS:49311#61616
2018-04-26 16:58:28,589 | INFO | jetty-9.2.13.v20150730 | org.eclipse.jetty.server.Server | WrapperSimpleAppMain
2018-04-26 16:58:28,651 | INFO | Started o.e.j.s.ServletContextHandler#-7d7eba88{/,null,AVAILABLE} | org.eclipse.jetty.server.handler.ContextHandler | WrapperSimpleAppMain
2018-04-26 16:58:28,698 | INFO | Started ServerConnector#b2498261{HTTP/1.1}{0.0.0.0:61614} | org.eclipse.jetty.server.ServerConnector | WrapperSimpleAppMain
2018-04-26 16:58:28,698 | INFO | Started #5125ms | org.eclipse.jetty.server.Server | WrapperSimpleAppMain
2018-04-26 16:58:28,714 | INFO | Listening for connections at ws://HOSTNAME:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.ws.WSTransportServer | WrapperSimpleAppMain
2018-04-26 16:58:28,714 | INFO | Connector ws started | org.apache.activemq.broker.TransportConnector | WrapperSimpleAppMain
2018-04-26 16:58:28,714 | INFO | Apache ActiveMQ 5.13.0 (BrokerName, ID:HOSTNAME-49282-1524776307714-0:1) started | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2018-04-26 16:58:28,714 | INFO | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2018-04-26 16:58:28,714 | WARN | Store limit is 102400 mb (current store usage is 0 mb). The data directory: D:\Apps\apache-activemq-5.13.0\bin\win64\..\..\data\BrokerName only has 16672 mb of usable space - resetting to maximum available disk space: 16672 mb | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2018-04-26 16:58:28,729 | WARN | Temporary Store limit is 51200 mb (current store usage is 0 mb). The data directory: D:\Apps\apache-activemq-5.13.0\bin\win64\..\..\data\BrokerName\tmp_storage only has 16672 mb of usable space - resetting to maximum available disk space: 16672 mb | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2018-04-26 16:58:28,729 | WARN | Transport Connection to: tcp://IPADDRESS:49283 failed: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///IPADDRESS:49283#61616
2018-04-26 16:58:28,839 | WARN | Transport Connection to: tcp://IPADDRESS:49285 failed: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///IPADDRESS:49285#61616
2018-04-26 16:58:28,854 | WARN | Transport Connection to: tcp://IPADDRESS:49284 failed: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///IPADDRESS:49284#61616
2018-04-26 16:58:28,885 | WARN | Transport Connection to: tcp://IPADDRESS:49286 failed: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///IPADDRESS:49286#61616
2018-04-26 16:58:28,964 | INFO | jetty-9.2.13.v20150730 | org.eclipse.jetty.server.Server | WrapperSimpleAppMain
2018-04-26 16:58:29,182 | INFO | No Spring WebApplicationInitializer types detected on classpath | /admin | WrapperSimpleAppMain
2018-04-26 16:58:29,339 | INFO | Refreshing Root WebApplicationContext: startup date [Thu Apr 26 16:58:29 EDT 2018]; root of context hierarchy | org.springframework.web.context.support.XmlWebApplicationContext | WrapperSimpleAppMain
2018-04-26 16:58:29,339 | INFO | Loading XML bean definitions from ServletContext resource [/WEB-INF/webconsole-embedded.xml] | org.springframework.beans.factory.xml.XmlBeanDefinitionReader | WrapperSimpleAppMain
2018-04-26 16:58:29,354 | INFO | Loading XML bean definitions from ServletContext resource [/WEB-INF/webconsole-query.xml] | org.springframework.beans.factory.xml.XmlBeanDefinitionReader | WrapperSimpleAppMain
2018-04-26 16:58:29,401 | INFO | Loading properties file from URL [file:path/apache-activemq-5.13.0/bin/win64/../../conf/credentials.properties] | org.springframework.beans.factory.config.PropertyPlaceholderConfigurer | WrapperSimpleAppMain
2018-04-26 16:58:29,432 | INFO | ActiveMQ WebConsole available at http://0.0.0.0:8161/ | org.apache.activemq.web.WebConsoleStarter | WrapperSimpleAppMain
2018-04-26 16:58:29,432 | INFO | ActiveMQ Jolokia REST API available at http://0.0.0.0:8161/api/jolokia/ | org.apache.activemq.web.WebConsoleStarter | WrapperSimpleAppMain
2018-04-26 16:58:29,510 | INFO | Initializing Spring FrameworkServlet 'dispatcher' | /admin | WrapperSimpleAppMain
2018-04-26 16:58:29,510 | INFO | FrameworkServlet 'dispatcher': initialization started | org.springframework.web.servlet.DispatcherServlet | WrapperSimpleAppMain
2018-04-26 16:58:29,526 | INFO | Refreshing WebApplicationContext for namespace 'dispatcher-servlet': startup date [Thu Apr 26 16:58:29 EDT 2018]; parent: Root WebApplicationContext | org.springframework.web.context.support.XmlWebApplicationContext | WrapperSimpleAppMain
2018-04-26 16:58:29,526 | INFO | Loading XML bean definitions from ServletContext resource [/WEB-INF/dispatcher-servlet.xml] | org.springframework.beans.factory.xml.XmlBeanDefinitionReader | WrapperSimpleAppMain
2018-04-26 16:58:29,589 | INFO | Mapped URL path [/createDestination.action] onto handler '/createDestination.action' | org.apache.activemq.web.handler.BindingBeanNameUrlHandlerMapping | WrapperSimpleAppMain
2018-04-26 16:58:29,589 | INFO | Mapped URL path [/deleteDestination.action] onto handler '/deleteDestination.action' | org.apache.activemq.web.handler.BindingBeanNameUrlHandlerMapping | WrapperSimpleAppMain
2018-04-26 16:58:29,589 | INFO | Mapped URL path [/createSubscriber.action] onto handler '/createSubscriber.action' | org.apache.activemq.web.handler.BindingBeanNameUrlHandlerMapping | WrapperSimpleAppMain
2018-04-26 16:58:29,589 | INFO | Mapped URL path [/deleteSubscriber.action] onto handler '/deleteSubscriber.action' | org.apache.activemq.web.handler.BindingBeanNameUrlHandlerMapping | WrapperSimpleAppMain
2018-04-26 16:58:29,589 | INFO | Mapped URL path [/sendMessage.action] onto handler '/sendMessage.action' | org.apache.activemq.web.handler.BindingBeanNameUrlHandlerMapping | WrapperSimpleAppMain
2018-04-26 16:58:29,604 | INFO | Mapped URL path [/purgeDestination.action] onto handler '/purgeDestination.action' | org.apache.activemq.web.handler.BindingBeanNameUrlHandlerMapping | WrapperSimpleAppMain
2018-04-26 16:58:29,604 | INFO | Mapped URL path [/deleteMessage.action] onto handler '/deleteMessage.action' | org.apache.activemq.web.handler.BindingBeanNameUrlHandlerMapping | WrapperSimpleAppMain
2018-04-26 16:58:29,604 | INFO | Mapped URL path [/copyMessage.action] onto handler '/copyMessage.action' | org.apache.activemq.web.handler.BindingBeanNameUrlHandlerMapping | WrapperSimpleAppMain
2018-04-26 16:58:29,604 | INFO | Mapped URL path [/moveMessage.action] onto handler '/moveMessage.action' | org.apache.activemq.web.handler.BindingBeanNameUrlHandlerMapping | WrapperSimpleAppMain
2018-04-26 16:58:29,604 | INFO | Mapped URL path [/deleteJob.action] onto handler '/deleteJob.action' | org.apache.activemq.web.handler.BindingBeanNameUrlHandlerMapping | WrapperSimpleAppMain
2018-04-26 16:58:29,870 | INFO | FrameworkServlet 'dispatcher': initialization completed in 344 ms | org.springframework.web.servlet.DispatcherServlet | WrapperSimpleAppMain
2018-04-26 16:58:29,870 | INFO | Started o.e.j.w.WebAppContext#6c5fb9bf{/admin,file:/G:apache-activemq-5.13.0/webapps/admin/,AVAILABLE} | org.eclipse.jetty.server.handler.ContextHandler | WrapperSimpleAppMain
2018-04-26 16:58:29,886 | INFO | ActiveMQ Console at http://ServerConnector#7c9374b6{SSL-HTTP/1.1}{0.0.0.0:8162}/admin | org.eclipse.jetty.webapp.WebAppContext | WrapperSimpleAppMain
2018-04-26 16:58:29,979 | INFO | No Spring WebApplicationInitializer types detected on classpath | /api | WrapperSimpleAppMain
2018-04-26 16:58:30,136 | INFO | jolokia-agent: No access restrictor found at classpath:/jolokia-access.xml, access to all MBeans is allowed | /api | WrapperSimpleAppMain
2018-04-26 16:58:30,636 | INFO | Started o.e.j.w.WebAppContext#3e4a58c7{/api,file:/G:apache-activemq-5.13.0/webapps/api/,AVAILABLE} | org.eclipse.jetty.server.handler.ContextHandler | WrapperSimpleAppMain
2018-04-26 16:58:30,651 | INFO | Apache ActiveMQ REST API at http://ServerConnector#7c9374b6{SSL-HTTP/1.1}{0.0.0.0:8162}/api | org.eclipse.jetty.webapp.WebAppContext | WrapperSimpleAppMain
2018-04-26 16:58:30,698 | INFO | Started ServerConnector#7c9374b6{SSL-HTTP/1.1}{0.0.0.0:8162} | org.eclipse.jetty.server.ServerConnector | WrapperSimpleAppMain
2018-04-26 16:58:30,714 | INFO | Started #7141ms | org.eclipse.jetty.server.Server | WrapperSimpleAppMain
2018-04-26 16:58:31,401 | INFO | Connector vm://BrokerName started | org.apache.activemq.broker.TransportConnector | ActiveMQ Transport: tcp:///AnotherIPADDRESS:59235#61616
2018-04-26 16:58:31,401 | INFO | Started responder end of duplex bridge NC#ID:ANOTHERHOST-59849-1523762308916-0:1 | org.apache.activemq.broker.TransportConnection | ActiveMQ Transport: tcp:///AnotherIPADDRESS:59235#61616
2018-04-26 16:58:31,417 | INFO | Network connection between vm://BrokerName#0 and tcp:///AnotherIPADDRESS:59235#61616 (Broker2) has been established. | org.apache.activemq.network.DemandForwardingBridgeSupport | triggerStartAsyncNetworkBridgeCreation: remoteBroker=tcp:///AnotherIPADDRESS:59235#61616, localBroker= vm://BrokerName#0
2018-04-26 16:58:33,386 | WARN | Transport Connection to: tcp://IPADDRESS:49587 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///IPADDRESS:49587#61616
2018-04-26 16:58:33,401 | WARN | Transport Connection to: tcp://IPADDRESS:49599 failed: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///IPADDRESS:49599#61616
2018-04-26 16:59:43,090 | WARN | Transport Connection to: tcp://IPADDRESS:54942 failed: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///IPADDRESS:54942#61616
2018-04-26 16:59:49,247 | WARN | Transport Connection to: tcp://IPADDRESS:55398 failed: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///IPADDRESS:55398#61616
2018-04-26 17:02:34,183 | WARN | Transport Connection to: tcp://127.0.0.1:50568 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:50568#61616
2018-04-26 17:02:34,183 | WARN | Transport Connection to: tcp://127.0.0.1:50567 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:50567#61616
2018-04-26 17:02:34,183 | WARN | Transport Connection to: tcp://127.0.0.1:50565 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:50565#61616
2018-04-26 17:02:34,183 | WARN | Transport Connection to: tcp://127.0.0.1:50328 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:50328#61616
2018-04-26 17:03:34,626 | WARN | Transport Connection to: tcp://127.0.0.1:55409 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:55409#61616
2018-04-26 17:03:34,626 | WARN | Transport Connection to: tcp://127.0.0.1:55408 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:55408#61616
2018-04-26 17:03:34,626 | WARN | Transport Connection to: tcp://127.0.0.1:55197 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:55197#61616
2018-04-26 17:03:34,626 | WARN | Transport Connection to: tcp://127.0.0.1:55407 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:55407#61616
2018-04-26 17:04:22,626 | WARN | Transport Connection to: tcp://127.0.0.1:59165 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:59165#61616
2018-04-26 17:04:22,626 | WARN | Transport Connection to: tcp://127.0.0.1:59164 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:59164#61616
2018-04-26 17:04:22,626 | WARN | Transport Connection to: tcp://127.0.0.1:59163 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:59163#61616
2018-04-26 17:04:22,642 | WARN | Transport Connection to: tcp://127.0.0.1:58999 failed: java.io.EOFException | org.apache.activemq.broker.TransportConnection.Transport | ActiveMQ Transport: tcp:///127.0.0.1:58999#61616

It gives you information about the connection change(conection timeout, for example).
If it occurs too often, you'd better check your code on the client side. Normally, one client should reuse 1 or several connections with the broker, maybe you create too much connections.

Related

Apache ActiveMQ always failed to restart (Address already in use)

Apache ActiveMQ 5.14.4
I've configured for JDBC persistence.
activemq.xml
<persistenceAdapter>
<jdbcPersistenceAdapter cleanupPeriod="0" dataSource="#oracle-ds"/>
</persistenceAdapter>
<import resource="activemq-jdbc.xml"/>
activemq-jdbc.xml
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<bean class="org.apache.commons.dbcp2.BasicDataSource" destroy-method="close" id="oracle-ds">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver"/>
<property name="url" value="jdbc:oracle:thin:#XX.XX.XX/XXXX"/>
<property name="username" value="XXX"/>
<property name="password" value="XXX"/>
<property name="poolPreparedStatements" value="true"/>
</bean>
</beans>
Step to reproduced
Start ActiveMQ by run activemq.bat and wait until it's initialized completed
Use SQL Developer to kill the session of ActiveMQ.
Wait around 10 seconds and the SQL exception occurs in activemq.log
ActiveMQ try to restart itself but failed to start because port 8161 (ActiveMQ Admin page) is already in use.
ActiveMQ is stopped.
I understand that when the exception occurs on ActiveMQ then it's trying to restart itself. But with this scenario it's always failed to start because "Address already in use".
Then it's required me to manually to stop and start it again to bounce the process.
My questions are:
Is this the expected behavior when the exception is occur and then ActiveMQ restart itself?
Can I increase retry count or retry with delay?
Port 8161 is not successfully closed during restart process?
How to avoid the exception "Address already in use"
Please also find my full stack trace here:
2020-11-05 09:57:47,243 | INFO | Started ServerConnector#475958{HTTP/1.1}{0.0.0.0:8161} | org.eclipse.jetty.server.ServerConnector | WrapperSimpleAppMain
2020-11-05 09:57:47,244 | INFO | Started #62783ms | org.eclipse.jetty.server.Server | WrapperSimpleAppMain
2020-11-05 09:59:51,055 | ERROR | Failed to update database lock: java.sql.SQLRecoverableException: ORA-00028: your session has been killed
| org.apache.activemq.store.jdbc.DefaultDatabaseLocker | ActiveMQ JDBC PA Scheduled Task
java.sql.SQLRecoverableException: ORA-00028: your session has been killed
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:445)
at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:396)
at oracle.jdbc.driver.T4C8Oall.processError(T4C8Oall.java:879)
at oracle.jdbc.driver.T4CTTIfun.receive(T4CTTIfun.java:450)
at oracle.jdbc.driver.T4CTTIfun.doRPC(T4CTTIfun.java:192)
at oracle.jdbc.driver.T4C8Oall.doOALL(T4C8Oall.java:531)
at oracle.jdbc.driver.T4CPreparedStatement.doOall8(T4CPreparedStatement.java:207)
at oracle.jdbc.driver.T4CPreparedStatement.executeForRows(T4CPreparedStatement.java:1044)
at oracle.jdbc.driver.OracleStatement.doExecuteWithTimeout(OracleStatement.java:1329)
at oracle.jdbc.driver.OraclePreparedStatement.executeInternal(OraclePreparedStatement.java:3584)
at oracle.jdbc.driver.OraclePreparedStatement.executeUpdate(OraclePreparedStatement.java:3665)
at oracle.jdbc.driver.OraclePreparedStatementWrapper.executeUpdate(OraclePreparedStatementWrapper.java:1352)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:98)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:98)
at org.apache.commons.dbcp2.DelegatingPreparedStatement.executeUpdate(DelegatingPreparedStatement.java:98)
at org.apache.activemq.store.jdbc.DefaultDatabaseLocker.keepAlive(DefaultDatabaseLocker.java:164)
at org.apache.activemq.broker.LockableServiceSupport.keepLockAlive(LockableServiceSupport.java:127)
at org.apache.activemq.broker.LockableServiceSupport$1.run(LockableServiceSupport.java:98)
at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
at java.base/java.util.concurrent.FutureTask.runAndReset(FutureTask.java:305)
at java.base/java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:305)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)
2020-11-05 09:59:51,059 | ERROR | localhost, no longer able to keep the exclusive lock so giving up being a master | org.apache.activemq.broker.LockableServiceSupport | ActiveMQ JDBC PA Scheduled Task
2020-11-05 09:59:51,061 | INFO | Apache ActiveMQ 5.14.4 (localhost, ID:U6077174-TPL-A-55145-1604545040076-0:1) is shutting down | org.apache.activemq.broker.BrokerService | ActiveMQ JDBC PA Scheduled Task
2020-11-05 09:59:51,064 | INFO | Connector openwire stopped | org.apache.activemq.broker.TransportConnector | ActiveMQ JDBC PA Scheduled Task
2020-11-05 09:59:51,077 | INFO | PListStore:[C:\DF_STP_Environment\apache-activemq-5.14.4\bin\win64\..\..\data\localhost\tmp_storage] stopped | org.apache.activemq.store.kahadb.plist.PListStoreImpl | ActiveMQ JDBC PA Scheduled Task
2020-11-05 09:59:51,089 | INFO | Apache ActiveMQ 5.14.4 (localhost, ID:U6077174-TPL-A-55145-1604545040076-0:1) uptime 2 minutes | org.apache.activemq.broker.BrokerService | ActiveMQ JDBC PA Scheduled Task
2020-11-05 09:59:51,089 | INFO | Apache ActiveMQ 5.14.4 (localhost, ID:U6077174-TPL-A-55145-1604545040076-0:1) is shutdown | org.apache.activemq.broker.BrokerService | ActiveMQ JDBC PA Scheduled Task
2020-11-05 09:59:51,090 | INFO | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1#6128db86: startup date [Thu Nov 05 09:59:51 ICT 2020]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | WrapperSimpleAppMain
2020-11-05 09:59:51,253 | INFO | Using Persistence Adapter: JDBCPersistenceAdapter(org.apache.commons.dbcp2.BasicDataSource#14f00b08) | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2020-11-05 10:00:01,836 | INFO | Database adapter driver override recognized for : [oracle_jdbc_driver] - adapter: class org.apache.activemq.store.jdbc.adapter.OracleJDBCAdapter | org.apache.activemq.store.jdbc.JDBCPersistenceAdapter | WrapperSimpleAppMain
2020-11-05 10:00:18,308 | INFO | Database lock driver override not found for : [oracle_jdbc_driver]. Will use default implementation. | org.apache.activemq.store.jdbc.JDBCPersistenceAdapter | WrapperSimpleAppMain
2020-11-05 10:00:18,308 | INFO | Attempting to acquire the exclusive lock to become the Master broker | org.apache.activemq.store.jdbc.DefaultDatabaseLocker | WrapperSimpleAppMain
2020-11-05 10:00:19,298 | INFO | Becoming the master on dataSource: org.apache.commons.dbcp2.BasicDataSource#14f00b08 | org.apache.activemq.store.jdbc.DefaultDatabaseLocker | WrapperSimpleAppMain
2020-11-05 10:00:19,299 | INFO | PListStore:[C:\DF_STP_Environment\apache-activemq-5.14.4\bin\win64\..\..\data\localhost\tmp_storage] started | org.apache.activemq.store.kahadb.plist.PListStoreImpl | WrapperSimpleAppMain
2020-11-05 10:00:36,271 | INFO | Apache ActiveMQ 5.14.4 (localhost, ID:U6077174-TPL-A-55145-1604545040076-0:2) is starting | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2020-11-05 10:00:44,207 | INFO | Listening for connections at: tcp://U6077174-TPL-A:61616 | org.apache.activemq.transport.TransportServerThreadSupport | WrapperSimpleAppMain
2020-11-05 10:00:44,207 | INFO | Connector openwire started | org.apache.activemq.broker.TransportConnector | WrapperSimpleAppMain
2020-11-05 10:00:44,207 | INFO | Apache ActiveMQ 5.14.4 (localhost, ID:U6077174-TPL-A-55145-1604545040076-0:2) started | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2020-11-05 10:00:44,207 | INFO | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2020-11-05 10:00:44,227 | INFO | jetty-9.2.13.v20150730 | org.eclipse.jetty.server.Server | WrapperSimpleAppMain
2020-11-05 10:00:44,348 | INFO | No Spring WebApplicationInitializer types detected on classpath | /admin | WrapperSimpleAppMain
2020-11-05 10:00:44,406 | INFO | ActiveMQ WebConsole available at http://0.0.0.0:8161/ | org.apache.activemq.web.WebConsoleStarter | WrapperSimpleAppMain
2020-11-05 10:00:44,406 | INFO | ActiveMQ Jolokia REST API available at http://0.0.0.0:8161/api/jolokia/ | org.apache.activemq.web.WebConsoleStarter | WrapperSimpleAppMain
2020-11-05 10:00:44,460 | INFO | Initializing Spring FrameworkServlet 'dispatcher' | /admin | WrapperSimpleAppMain
2020-11-05 10:00:44,533 | INFO | Started o.e.j.w.WebAppContext#66f69a3c{/admin,file:/C:/DF_STP_Environment/apache-activemq-5.14.4/webapps/admin/,AVAILABLE} | org.eclipse.jetty.server.handler.ContextHandler | WrapperSimpleAppMain
2020-11-05 10:00:44,534 | INFO | ActiveMQ Console at http://ServerConnector#54c2c4a4{HTTP/1.1}{0.0.0.0:8161}/admin | org.eclipse.jetty.webapp.WebAppContext | WrapperSimpleAppMain
2020-11-05 10:00:44,619 | INFO | No Spring WebApplicationInitializer types detected on classpath | /api | WrapperSimpleAppMain
2020-11-05 10:00:45,153 | INFO | jolokia-agent: Using policy access restrictor classpath:/jolokia-access.xml | /api | WrapperSimpleAppMain
2020-11-05 10:00:45,159 | INFO | jolokia-agent: jolokia:type=Config is already registered. Adding it with jolokia:type=Config,uuid=ab3a26c8-8be8-4c46-96e3-1ad94283608c, but you should revise your setup in order to either use a qualifier or ensure, that only a single agent gets registered (otherwise history functionality might not work) | /api | WrapperSimpleAppMain
2020-11-05 10:00:45,159 | INFO | jolokia-agent: Cannot register (legacy) MBean handler for config store with name jmx4perl:type=Config since it already exists. This is the case if another agent has been already started within the same JVM. The registration is skipped. | /api | WrapperSimpleAppMain
2020-11-05 10:00:45,159 | INFO | jolokia-agent: Jolokia Discovery MBean registration is skipped because there is already one registered. | /api | WrapperSimpleAppMain
2020-11-05 10:00:45,160 | INFO | Started o.e.j.w.WebAppContext#463e37b0{/api,file:/C:/DF_STP_Environment/apache-activemq-5.14.4/webapps/api/,AVAILABLE} | org.eclipse.jetty.server.handler.ContextHandler | WrapperSimpleAppMain
2020-11-05 10:00:45,160 | INFO | Apache ActiveMQ REST API at http://ServerConnector#54c2c4a4{HTTP/1.1}{0.0.0.0:8161}/api | org.eclipse.jetty.webapp.WebAppContext | WrapperSimpleAppMain
2020-11-05 10:00:45,160 | WARN | FAILED ServerConnector#54c2c4a4{HTTP/1.1}{0.0.0.0:8161}: java.net.BindException: Address already in use: bind | org.eclipse.jetty.util.component.AbstractLifeCycle | WrapperSimpleAppMain
java.net.BindException: Address already in use: bind
at java.base/sun.nio.ch.Net.bind0(Native Method)
at java.base/sun.nio.ch.Net.bind(Net.java:455)
at java.base/sun.nio.ch.Net.bind(Net.java:447)
at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:269)
at org.springframework.beans.factory.config.MethodInvokingBean.invokeWithTargetException(MethodInvokingBean.java:119)
at org.springframework.beans.factory.config.MethodInvokingFactoryBean.afterPropertiesSet(MethodInvokingFactoryBean.java:106)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1631)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1568)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:736)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:762)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)
at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)
at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)
at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154)
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)
at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
at org.apache.activemq.console.Main.main(Main.java:115)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:240)
at java.base/java.lang.Thread.run(Thread.java:834)
2020-11-05 10:00:45,228 | WARN | FAILED org.eclipse.jetty.server.Server#2d86772e: java.net.BindException: Address already in use: bind | org.eclipse.jetty.util.component.AbstractLifeCycle | WrapperSimpleAppMain
java.net.BindException: Address already in use: bind
at java.base/sun.nio.ch.Net.bind0(Native Method)
at java.base/sun.nio.ch.Net.bind(Net.java:455)
at java.base/sun.nio.ch.Net.bind(Net.java:447)
at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
at org.eclipse.jetty.server.ServerConnector.open(ServerConnector.java:321)
at org.eclipse.jetty.server.AbstractNetworkConnector.doStart(AbstractNetworkConnector.java:80)
at org.eclipse.jetty.server.ServerConnector.doStart(ServerConnector.java:236)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at org.eclipse.jetty.server.Server.doStart(Server.java:366)
at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:269)
at org.springframework.beans.factory.config.MethodInvokingBean.invokeWithTargetException(MethodInvokingBean.java:119)
at org.springframework.beans.factory.config.MethodInvokingFactoryBean.afterPropertiesSet(MethodInvokingFactoryBean.java:106)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.invokeInitMethods(AbstractAutowireCapableBeanFactory.java:1631)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1568)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:736)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:762)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)
at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)
at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)
at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154)
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)
at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
at org.apache.activemq.console.Main.main(Main.java:115)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:240)
at java.base/java.lang.Thread.run(Thread.java:834)
2020-11-05 10:00:45,266 | WARN | Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'invokeStart' defined in class path resource [jetty.xml]: Invocation of init method failed; nested exception is java.net.BindException: Address already in use: bind | org.apache.activemq.xbean.XBeanBrokerFactory$1 | WrapperSimpleAppMain
2020-11-05 10:00:45,274 | INFO | Stopped ServerConnector#54c2c4a4{HTTP/1.1}{0.0.0.0:8161} | org.eclipse.jetty.server.ServerConnector | WrapperSimpleAppMain
2020-11-05 10:00:45,280 | INFO | Stopped o.e.j.w.WebAppContext#463e37b0{/api,file:/C:/DF_STP_Environment/apache-activemq-5.14.4/webapps/api/,UNAVAILABLE} | org.eclipse.jetty.server.handler.ContextHandler | WrapperSimpleAppMain
2020-11-05 10:00:45,282 | INFO | Destroying Spring FrameworkServlet 'dispatcher' | /admin | WrapperSimpleAppMain
2020-11-05 10:00:45,284 | INFO | Stopped o.e.j.w.WebAppContext#66f69a3c{/admin,file:/C:/DF_STP_Environment/apache-activemq-5.14.4/webapps/admin/,UNAVAILABLE} | org.eclipse.jetty.server.handler.ContextHandler | WrapperSimpleAppMain
2020-11-05 10:00:45,291 | INFO | Apache ActiveMQ 5.14.4 (localhost, ID:U6077174-TPL-A-55145-1604545040076-0:2) is shutting down | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2020-11-05 10:00:45,292 | INFO | Connector openwire stopped | org.apache.activemq.broker.TransportConnector | WrapperSimpleAppMain
2020-11-05 10:00:45,295 | INFO | PListStore:[C:\DF_STP_Environment\apache-activemq-5.14.4\bin\win64\..\..\data\localhost\tmp_storage] stopped | org.apache.activemq.store.kahadb.plist.PListStoreImpl | WrapperSimpleAppMain
2020-11-05 10:00:46,291 | INFO | Apache ActiveMQ 5.14.4 (localhost, ID:U6077174-TPL-A-55145-1604545040076-0:2) uptime 55.038 seconds | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2020-11-05 10:00:46,292 | INFO | Apache ActiveMQ 5.14.4 (localhost, ID:U6077174-TPL-A-55145-1604545040076-0:2) is shutdown | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2020-11-05 10:00:47,285 | ERROR | Failed to load: class path resource [activemq.xml], reason: Error creating bean with name 'invokeStart' defined in class path resource [jetty.xml]: Invocation of init method failed; nested exception is java.net.BindException: Address already in use: bind | org.apache.activemq.xbean.XBeanBrokerFactory | WrapperSimpleAppMain
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'invokeStart' defined in class path resource [jetty.xml]: Invocation of init method failed; nested exception is java.net.BindException: Address already in use: bind
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1572)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:539)
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:476)
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:303)
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:299)
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:194)
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:736)
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:762)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:480)
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)
at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)
at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)
at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)
at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)
at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154)
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)
at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.apache.activemq.console.Main.runTaskClass(Main.java:262)
at org.apache.activemq.console.Main.main(Main.java:115)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.tanukisoftware.wrapper.WrapperSimpleApp.run(WrapperSimpleApp.java:240)
at java.base/java.lang.Thread.run(Thread.java:834)
Caused by: java.net.BindException: Address already in use: bind
at java.base/sun.nio.ch.Net.bind0(Native Method)
at java.base/sun.nio.ch.Net.bind(Net.java:455)
at java.base/sun.nio.ch.Net.bind(Net.java:447)
at java.base/sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:227)
at java.base/sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:80)
I can solved this by disabled the embedded Web Console and deploy it as a standalone application instead.
https://access.redhat.com/documentation/en-us/red_hat_amq/6.0/html/managing_and_monitoring_a_broker/fmqadminconsoleextrernal

java.io.IOException: Failed to bind error on ActiveMQ service start

I am configuring a new install of ActiveMQ 5.15.10 on a RHEL 7.7 AWS instance. When the process starts I get this error:
[ec2-user#ip-***-***-***-*** activemq]$ more data/activemq.log
2019-10-30 17:36:45,784 | INFO | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1#4667ae56: startup date [Wed Oct 30 17:36:45 UTC 2019]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2019-10-30 17:36:46,568 | INFO | Using Persistence Adapter: KahaDBPersistenceAdapter[/opt/activemq/data/kahadb] | org.apache.activemq.broker.BrokerService | main
2019-10-30 17:36:46,628 | INFO | KahaDB is version 6 | org.apache.activemq.store.kahadb.MessageDatabase | main
2019-10-30 17:36:46,652 | INFO | PListStore:[/opt/activemq/data/***-***-***-***/tmp_storage] started | org.apache.activemq.store.kahadb.plist.PListStoreImpl | main
2019-10-30 17:36:46,821 | INFO | Apache ActiveMQ 5.15.10 (***-***-***-***, ID:ip-***-***-***-***.ec2.internal-36686-1572457006663-0:1) is starting | org.apache.activemq.broker.BrokerService | main
2019-10-30 17:36:46,849 | INFO | Listening for connections at: tcp://ip-***-***-***-***.ec2.internal:61616?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-10-30 17:36:46,851 | INFO | Connector openwire started | org.apache.activemq.broker.TransportConnector | main
2019-10-30 17:36:46,855 | INFO | Listening for connections at: amqp://ip-***-***-***-***.ec2.internal:5672?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-10-30 17:36:46,856 | INFO | Connector amqp started | org.apache.activemq.broker.TransportConnector | main
2019-10-30 17:36:46,859 | INFO | Listening for connections at: stomp://ip-***-***-***-***.ec2.internal:61613?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-10-30 17:36:46,861 | INFO | Connector stomp started | org.apache.activemq.broker.TransportConnector | main
2019-10-30 17:36:46,864 | INFO | Listening for connections at: mqtt://ip-***-***-***-***.ec2.internal:1883?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.TransportServerThreadSupport | main
2019-10-30 17:36:46,865 | INFO | Connector mqtt started | org.apache.activemq.broker.TransportConnector | main
2019-10-30 17:36:46,870 | INFO | Starting Jetty server | org.apache.activemq.transport.WebTransportServerSupport | main
2019-10-30 17:36:46,951 | INFO | Creating Jetty connector | org.apache.activemq.transport.WebTransportServerSupport | main
2019-10-30 17:36:47,043 | WARN | ServletContext#o.e.j.s.ServletContextHandler#7b420819{/,null,STARTING} has uncovered http methods for path: / | org.eclipse.jetty.security.SecurityHandler | main
2019-10-30 17:36:47,091 | INFO | Listening for connections at ws://ip-***-***-***-***.ec2.internal:61614?maximumConnections=1000&wireFormat.maxFrameSize=104857600 | org.apache.activemq.transport.ws.WSTransportServer | main
2019-10-30 17:36:47,092 | INFO | Connector ws started | org.apache.activemq.broker.TransportConnector | main
2019-10-30 17:36:47,093 | INFO | Apache ActiveMQ 5.15.10 (***-***-***-***, ID:ip-***-***-***-***.ec2.internal-36686-1572457006663-0:1) started | org.apache.activemq.broker.BrokerService | main
2019-10-30 17:36:47,094 | INFO | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main
2019-10-30 17:36:47,095 | WARN | Store limit is 102400 mb (current store usage is 0 mb). The data directory: /opt/activemq/data/kahadb only has 47858 mb of usable space. - resetting to maximum available disk space: 47858 mb | org.apache.activemq.broker.BrokerService | main
2019-10-30 17:36:47,096 | WARN | Temporary Store limit is 51200 mb (current store usage is 0 mb). The data directory: /opt/activemq/data only has 47858 mb of usable space. - resetting to maximum available disk space: 47858 mb | org.apache.activemq.broker.BrokerService | main
2019-10-30 17:36:47,726 | INFO | ActiveMQ WebConsole available at http://***-***-***-***:8161/ | org.apache.activemq.web.WebConsoleStarter | main
2019-10-30 17:36:47,726 | INFO | ActiveMQ Jolokia REST API available at http://***-***-***-***:8161/api/jolokia/ | org.apache.activemq.web.WebConsoleStarter | main
2019-10-30 17:36:48,001 | WARN | Exception encountered during context initialization - cancelling refresh attempt: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'invokeStart' defined in class path resource [jetty.xml]: Invocation of init method failed; nested exception is java.io.IOException: Failed to bind to /***-***-***-***:8161 | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2019-10-30 17:36:48,013 | INFO | Apache ActiveMQ 5.15.10 (***-***-***-***, ID:ip-***-***-***-***.ec2.internal-36686-1572457006663-0:1) is shutting down | org.apache.activemq.broker.BrokerService | main
2019-10-30 17:36:48,014 | INFO | Connector openwire stopped | org.apache.activemq.broker.TransportConnector | main
2019-10-30 17:36:48,015 | INFO | Connector amqp stopped | org.apache.activemq.broker.TransportConnector | main
2019-10-30 17:36:48,016 | INFO | Connector stomp stopped | org.apache.activemq.broker.TransportConnector | main
2019-10-30 17:36:48,017 | INFO | Connector mqtt stopped | org.apache.activemq.broker.TransportConnector | main
2019-10-30 17:36:48,021 | INFO | Connector ws stopped | org.apache.activemq.broker.TransportConnector | main
2019-10-30 17:36:48,024 | INFO | PListStore:[/opt/activemq/data/***-***-***-***/tmp_storage] stopped | org.apache.activemq.store.kahadb.plist.PListStoreImpl | main
2019-10-30 17:36:48,025 | INFO | Stopping async queue tasks | org.apache.activemq.store.kahadb.KahaDBStore | main
2019-10-30 17:36:48,025 | INFO | Stopping async topic tasks | org.apache.activemq.store.kahadb.KahaDBStore | main
2019-10-30 17:36:48,026 | INFO | Stopped KahaDB | org.apache.activemq.store.kahadb.KahaDBStore | main
2019-10-30 17:36:48,046 | INFO | Apache ActiveMQ 5.15.10 (***-***-***-***, ID:ip-***-***-***-***.ec2.internal-36686-1572457006663-0:1) uptime 1.495 seconds | org.apache.activemq.broker.BrokerService | main
2019-10-30 17:36:48,047 | INFO | Apache ActiveMQ 5.15.10 (***-***-***-***, ID:ip-***-***-***-***.ec2.internal-36686-1572457006663-0:1) is shutdown | org.apache.activemq.broker.BrokerService | main
2019-10-30 17:36:48,048 | INFO | Closing org.apache.activemq.xbean.XBeanBrokerFactory$1#4667ae56: startup date [Wed Oct 30 17:36:45 UTC 2019]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | main
2019-10-30 17:36:48,049 | ERROR | Failed to load: class path resource [activemq.xml], reason: Error creating bean with name 'invokeStart' defined in class path resource [jetty.xml]: Invocation of init method failed; nested exception is java.io.IOException: Failed to bind to /***-***-***-***:8161 | org.apache.activemq.xbean.XBeanBrokerFactory | main
org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'invokeStart' defined in class path resource [jetty.xml]: Invocation of init method failed; nested exception is java.io.IOException: Failed to bind to /3.231.235.30:8161
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.initializeBean(AbstractAutowireCapableBeanFactory.java:1630)[spring-beans-4.3.24.RELEASE.jar:4.3.24.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:553)[spring-beans-4.3.24.RELEASE.jar:4.3.24.RELEASE]
at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:481)[spring-beans-4.3.24.RELEASE.jar:4.3.24.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory$1.getObject(AbstractBeanFactory.java:312)[spring-beans-4.3.24.RELEASE.jar:4.3.24.RELEASE]
at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:230)[spring-beans-4.3.24.RELEASE.jar:4.3.24.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:308)[spring-beans-4.3.24.RELEASE.jar:4.3.24.RELEASE]
at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:197)[spring-beans-4.3.24.RELEASE.jar:4.3.24.RELEASE]
at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:737)[spring-beans-4.3.24.RELEASE.jar:4.3.24.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:867)[spring-context-4.3.24.RELEASE.jar:4.3.24.RELEASE]
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:542)[spring-context-4.3.24.RELEASE.jar:4.3.24.RELEASE]
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:64)[xbean-spring-4.14.jar:4.14]
at org.apache.xbean.spring.context.ResourceXmlApplicationContext.<init>(ResourceXmlApplicationContext.java:52)[xbean-spring-4.14.jar:4.14]
at org.apache.activemq.xbean.XBeanBrokerFactory$1.<init>(XBeanBrokerFactory.java:104)[activemq-spring-5.15.10.jar:5.15.10]
at org.apache.activemq.xbean.XBeanBrokerFactory.createApplicationContext(XBeanBrokerFactory.java:104)[activemq-spring-5.15.10.jar:5.15.10]
at org.apache.activemq.xbean.XBeanBrokerFactory.createBroker(XBeanBrokerFactory.java:67)[activemq-spring-5.15.10.jar:5.15.10]
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:71)[activemq-broker-5.15.10.jar:5.15.10]
at org.apache.activemq.broker.BrokerFactory.createBroker(BrokerFactory.java:54)[activemq-broker-5.15.10.jar:5.15.10]
at org.apache.activemq.console.command.StartCommand.runTask(StartCommand.java:87)[activemq-console-5.15.10.jar:5.15.10]
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)[activemq-console-5.15.10.jar:5.15.10]
at org.apache.activemq.console.command.ShellCommand.runTask(ShellCommand.java:154)[activemq-console-5.15.10.jar:5.15.10]
at org.apache.activemq.console.command.AbstractCommand.execute(AbstractCommand.java:63)[activemq-console-5.15.10.jar:5.15.10]
at org.apache.activemq.console.command.ShellCommand.main(ShellCommand.java:104)[activemq-console-5.15.10.jar:5.15.10]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)[:1.8.0_191]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)[:1.8.0_191]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)[:1.8.0_191]
at java.lang.reflect.Method.invoke(Method.java:498)[:1.8.0_191]
at org.apache.activemq.console.Main.runTaskClass(Main.java:262)[activemq.jar:5.15.10]
at org.apache.activemq.console.Main.main(Main.java:115)[activemq.jar:5.15.10]
It looks like the web console is logged as available, and then it tries to bind the web console port again? This is on a fresh instance with no other services or java processes running.
Please advise.

ActiveMQ isuue : Puppet Mcollective ping failing

Seeing the following error message wen starting ActiveMQ
2019-05-14 15:25:35,302 | INFO | For help or more information please see: http://activemq.apache.org | org.apache.activemq.broker.BrokerService | main
2019-05-14 15:25:35,815 | INFO | ActiveMQ WebConsole available at http://0.0.0.0:8161/ | org.apache.activemq.web.WebConsoleStarter | main
2019-05-14 15:25:35,867 | INFO | Initializing Spring FrameworkServlet 'dispatcher' | /admin | main
2019-05-14 15:25:36,100 | INFO | jolokia-agent: No access restrictor found at classpath:/jolokia-access.xml, access to all MBeans is allowed | /api | main
2019-05-14 15:25:39,320 | ERROR | Could not accept connection from null: java.io.IOException: javax.net.ssl.SSLException: Received fatal alert: internal_error | org.apa
che.activemq.broker.TransportConnector | ActiveMQ BrokerService[localhost] Task-1
2019-05-14 15:25:51,900 | ERROR | Could not accept connection from null: java.io.IOException: javax.net.ssl.SSLException: Received fatal alert: internal_error | org.apache.activemq.broker.TransportConnector | ActiveMQ BrokerService[localhost] Task-1
2019-05-14 15:25:59,936 | ERROR | Could not accept connection from null: java.io.IOException: javax.net.ssl.SSLException: Received fatal alert: internal_error | org.apache.activemq.broker.TransportConnector | ActiveMQ BrokerService[localhost] Task-1
2019-05-14 15:26:01,366 | ERROR | Could not accept connection from null: java.io.IOException: javax.net.ssl.SSLException: Received fatal alert: internal_error | org.apache.activemq.broker.TransportConnector | ActiveMQ BrokerService[localhost] Task-1
2019-05-14 15:26:03,288 | ERROR | Could not accept connection from null: java.io.IOException: javax.net.ssl.SSLException: Unrecognized SSL message, plaintext connection? | org.apache.activemq.broker.TransportConnector | ActiveMQ BrokerService[localhost] Task-1
mco ping is failing because of this with below error
error 2019/05/14 15:46:13: activemq.rb:149:in `on_ssl_connectfail' SSL session creation with stomp+ssl://mcollective#Node-001.wiley.com:61614 failed: SSL_connect returned=1 errno=0 state=error: certificate verify failed

activemq network of brokers missing username or password

I'm trying to set a network of brokers in activemq, my current setup is having two brokers each running on a different vm. My configuration is:
<networkConnectors>
<networkConnector
name="Q:broker1->broker2"
uri="static:(tcp://10.10.10.12:61612)"
duplex="false"
decreaseNetworkConsumerPriority="true"
networkTTL="2"
dynamicOnly="true"
userName="activemq"
password="password">
<excludedDestinations>
<topic physicalName=">" />
</excludedDestinations>
</networkConnector>
</networkConnectors>
I'm getting this error when I start up activemq on broker1:
2016-11-01 17:49:18,571 | INFO | Establishing network connection from vm://localhost?create=false&async=false to tcp://10.10.10.12:61612 | org.apache.activemq.network.DiscoveryNetworkConnector | ActiveMQ Task-1
2016-11-01 17:49:18,572 | INFO | Connector vm://localhost started | org.apache.activemq.broker.TransportConnector | ActiveMQ Task-1
2016-11-01 17:49:18,578 | WARN | Failed to add Connection localhost->localhost-34161-1478036807089-20:1 due to java.lang.SecurityException: User name [activemq] or password is invalid. | org.apache.activemq.broker.TransportConnection | triggerStartAsyncNetworkBridgeCreation: remoteBroker=tcp:///10.10.10.12:61612#43981, localBroker= vm://localhost#18
2016-11-01 17:49:18,580 | WARN | Security Error occurred on connection to: vm://localhost#18, User name [activemq] or password is invalid. | org.apache.activemq.broker.TransportConnection.Service | triggerStartAsyncNetworkBridgeCreation: remoteBroker=tcp:///10.10.10.12:61612#43981, localBroker= vm://localhost#18
2016-11-01 17:49:18,580 | INFO | Network connection between vm://localhost#18 and tcp:///10.10.10.12:61612#43981 shutdown due to a local error: java.lang.SecurityException: User name [activemq] or password is invalid. | org.apache.activemq.network.DemandForwardingBridgeSupport | triggerStartAsyncNetworkBridgeCreation: remoteBroker=tcp:///10.10.10.12:61612#43981, localBroker= vm://localhost#18
2016-11-01 17:49:18,583 | INFO | Connector vm://localhost stopped | org.apache.activemq.broker.TransportConnector | ActiveMQ BrokerService[localhost] Task-8
2016-11-01 17:49:18,584 | INFO | localhost bridge to localhost stopped | org.apache.activemq.network.DemandForwardingBridgeSupport | ActiveMQ BrokerService[localhost] Task-8
2016-11-01 17:49:20,580 | INFO | Stopping vm://localhost#18 because Failed with SecurityException: User name [activemq] or password is invalid. | org.apache.activemq.broker.TransportConnection | ActiveMQ BrokerService[localhost] Task-5
I set the activemq user in broker2 activemq.xml like the following:
<simpleAuthenticationPlugin>
<users>
<authenticationUser username="activemq" password="${activemq.password}"
groups="admins"/>
and the user activemq is defined in credentials-enc.properties file like:
activemq.password=password
why broker1 is enable to establish a connection with broker2? what I'm missing here?
Add the userName and password attributes like this:
<networkConnector .. userName="admin" password="admin".. >
Network Connector Properties
Part 2:
Do you have a property placeholder bean in your activemq.xml file? You need that for the macro replacement to work. Otherwise, the broker probably thinks the password is literally the text below with the $ sign and { } braces.
${activemq.password}

Why does ActiveMQ restart automatically and how do i prevent it?

We've been using AMQ 5.5.1 in production for several months. Occasionally, we observe that the broker decides to refresh itself with no outside trigger. When this happens, our queue senders fail until the broker is back online some 10 minutes later. I cannot find any information or settings that would cause this behavior .. and let me control it.
Is this normal for the broker to recycle on its own like this? If so, what things would cause it?
2012-12-11 11:02:11,603 | INFO | Refreshing org.apache.activemq.xbean.XBeanBrokerFactory$1#f6ac0b: startup date [Tue Dec 11 11:02:11 EST 2012]; root of context hierarchy | org.apache.activemq.xbean.XBeanBrokerFactory$1 | WrapperSimpleAppMain
2012-12-11 11:02:13,806 | WARN | destroyApplicationContextOnStop parameter is deprecated, please use shutdown hooks instead | org.apache.activemq.xbean.XBeanBrokerService | WrapperSimpleAppMain
2012-12-11 11:02:13,821 | INFO | PListStore:D:\Tools\ActiveMQ\apache-activemq-5.5.1\bin\win32\..\..\data\localhost\tmp_storage started | org.apache.activemq.store.kahadb.plist.PListStore | WrapperSimpleAppMain
2012-12-11 11:02:13,868 | INFO | Using Persistence Adapter: KahaDBPersistenceAdapter[D:\Tools\ActiveMQ\apache-activemq-5.5.1\bin\win32\..\..\data\kahadb] | org.apache.activemq.broker.BrokerService | WrapperSimpleAppMain
2012-12-11 11:02:16,618 | INFO | KahaDB is version 3 | org.apache.activemq.store.kahadb.MessageDatabase | WrapperSimpleAppMain
2012-12-11 11:02:16,697 | INFO | Recovering from the journal ... | org.apache.activemq.store.kahadb.MessageDatabase | WrapperSimpleAppMain
I found that the wrapper exe process was forcing the restart.
I was able to see in the wrapper.log (windows service) that the process was being restarted because the JVM was not responding. So this is not an issue with the broker auto-restarting per se.. it was an issue with the broker JVM somehow becoming hung (separate problem).
Here are the wrapper log entries for those interested:
ERROR | wrapper | 2012/12/11 11:01:58 | JVM appears hung: Timed out waiting for signal from JVM.
ERROR | wrapper | 2012/12/11 11:01:58 | JVM did not exit on request, terminated
STATUS | wrapper | 2012/12/11 11:02:04 | Launching a JVM...