I have a standalone java client program that sends messages to a queue on websphere mq system. I want to avoid the creation of new connection, session and producer for every message so I use the connection pooling facilities that are provided by activemq-jms-pool package (v5.11.1).
The setup of this connection pool is as follows:
this.fConnectionFactory = new PooledConnectionFactory();
((PooledConnectionFactory)this.fConnectionFactory).setConnectionFactory(mqConnectionFactory);
((PooledConnectionFactory)this.fConnectionFactory).setMaxConnections(1);
((PooledConnectionFactory)this.fConnectionFactory).setMaximumActiveSessionPerConnection(1);
((PooledConnectionFactory)this.fConnectionFactory).setExpiryTimeout(10 * 60 * 1000);
((PooledConnectionFactory)this.fConnectionFactory).setIdleTimeout(10 * 60 * 1000);
((PooledConnectionFactory)this.fConnectionFactory).setReconnectOnException(true);
((PooledConnectionFactory)this.fConnectionFactory).setCreateConnectionOnStartup(true);
My MQ client itself is version "7.5.0.5 - p750-005-150424". Websphere MQ itself is version 8.
However from time to time I see the following exception in my logs:
com.ibm.msg.client.wmq.v6.jms.internal.ConfigEnvironment$1: MQJMS2007: failed to send message to MQ queue.
at com.ibm.msg.client.wmq.v6.jms.internal.ConfigEnvironment.newException(ConfigEnvironment.java:379) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.msg.client.wmq.v6.jms.internal.MQMessageProducer.sendInternal(MQMessageProducer.java:1218) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.msg.client.wmq.v6.jms.internal.MQMessageProducer.send(MQMessageProducer.java:2490) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.msg.client.wmq.v6.jms.internal.MQMessageProducer.send(MQMessageProducer.java:2608) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.msg.client.jms.internal.JmsMessageProducerImpl.sendMessage(JmsMessageProducerImpl.java:927) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.msg.client.jms.internal.JmsMessageProducerImpl.send_(JmsMessageProducerImpl.java:783) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.msg.client.jms.internal.JmsMessageProducerImpl.send(JmsMessageProducerImpl.java:363) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.mq.jms.MQMessageProducer.send(MQMessageProducer.java:274) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at org.apache.activemq.jms.pool.PooledProducer.send(PooledProducer.java:95) ~[activemq-jms-pool-5.11.1.jar:5.11.1]
at org.apache.activemq.jms.pool.PooledProducer.send(PooledProducer.java:71) ~[activemq-jms-pool-5.11.1.jar:5.11.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_102]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_102]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_102]
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) [junit-4.12.jar:4.12]
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.12.jar:4.12]
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) [junit-4.12.jar:4.12]
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) [junit-4.12.jar:4.12]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) [junit-4.12.jar:4.12]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.run(ParentRunner.java:363) [junit-4.12.jar:4.12]
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) [.cp/:na]
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) [.cp/:na]
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) [.cp/:na]
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678) [.cp/:na]
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) [.cp/:na]
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) [.cp/:na]
Caused by: com.ibm.mq.MQException: MQJE001: Completion Code '2', Reason '2009'.
at com.ibm.msg.client.wmq.v6.base.internal.MQQueueManager.putMsg2(MQQueueManager.java:617) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.msg.client.wmq.v6.jms.internal.MQMessageProducer.sendInternal(MQMessageProducer.java:1178) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
... 34 common frames omitted
08:37:05.989 [main] WARN o.a.activemq.jms.pool.PooledSession - Caught exception trying close() when putting session back into the pool, will invalidate. com.ibm.msg.client.jms.DetailedIllegalStateException: JMSCC0020: This session is closed.
An application called a method that must not be used after the session is closed.
Ensure that the session is not closed before calling the method.
com.ibm.msg.client.jms.DetailedIllegalStateException: JMSCC0020: This session is closed.
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_102]
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_102]
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_102]
at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_102]
at com.ibm.msg.client.commonservices.j2se.NLSServices.createException(NLSServices.java:319) ~[com.ibm.mq.jmqi.jar:7.5.0.8 - p750-008-170628]
at com.ibm.msg.client.commonservices.nls.NLSServices.createException(NLSServices.java:233) ~[com.ibm.mq.jmqi.jar:7.5.0.8 - p750-008-170628]
at com.ibm.msg.client.jms.internal.JmsErrorUtils.createException(JmsErrorUtils.java:109) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.msg.client.jms.internal.State.checkNotClosed(State.java:150) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.msg.client.jms.internal.JmsSessionImpl.checkNotClosed(JmsSessionImpl.java:2526) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.msg.client.jms.internal.JmsSessionImpl.setMessageListener(JmsSessionImpl.java:1730) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at com.ibm.mq.jms.MQSession.setMessageListener(MQSession.java:1004) ~[com.ibm.mqjms-7.5.0.5.jar:7.5.0.5 - p750-005-150424]
at org.apache.activemq.jms.pool.PooledSession.close(PooledSession.java:103) ~[activemq-jms-pool-5.11.1.jar:5.11.1]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_102]
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_102]
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_102]
at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_102]
at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) [junit-4.12.jar:4.12]
at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) [junit-4.12.jar:4.12]
at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) [junit-4.12.jar:4.12]
at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) [junit-4.12.jar:4.12]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) [junit-4.12.jar:4.12]
at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) [junit-4.12.jar:4.12]
at org.junit.runners.ParentRunner.run(ParentRunner.java:363) [junit-4.12.jar:4.12]
at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) [.cp/:na]
at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) [.cp/:na]
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459) [.cp/:na]
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:678) [.cp/:na]
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382) [.cp/:na]
at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192) [.cp/:na]
What is causing this error (and how can it be avoided) ?
There are no network issues and there is also no firewall in between. The issue seems to occur when there is some time between the sending of different messages (like 5 minutes or so).
Thx for the assistance.
EDIT 06/12/2018: Added other exception
20181204 164448.873 [INFO ] b.g.s.c.j.s.CachingConnectionFactory - Encountered a JMSException - resetting the underlying JMS Connection
com.ibm.msg.client.jms.DetailedJMSException: JMSWMQ1107: A problem with this connection has occurred.
at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:585) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:221) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.wmq.internal.WMQConnection.consumer(WMQConnection.java:785) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.api.RemoteHconn.callEventHandler(RemoteHconn.java:2621) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.api.RemoteHconn.driveEventsEH(RemoteHconn.java:601) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.processHconn(RemoteDispatchThread.java:668) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.impl.RemoteDispatchThread.run(RemoteDispatchThread.java:244) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.runTask(WorkQueueItem.java:214) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.SimpleWorkQueueItem.runItem(SimpleWorkQueueItem.java:105) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.run(WorkQueueItem.java:229) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.WorkQueueManager.runWorkQueueItem(WorkQueueManager.java:303) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.j2se.workqueue.WorkQueueManagerImplementation$ThreadPoolWorker.run(WorkQueueManagerImplementation.java:1241) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
Caused by: com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN').
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:209) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
... 10 common frames omitted
20181204 164448.873 [TRACE] b.g.s.c.j.s.CachingConnectionFactory - Reset connection, first close cached sessions (active: false).
20181204 164448.874 [TRACE] b.g.s.c.j.s.CachingConnectionFactory - Invoking method close via proxied session handler.
20181204 164448.874 [DEBUG] b.g.s.c.j.s.CachingConnectionFactory - Physically closing cached Session: com.ibm.mq.jms.MQQueueSession#4bb27555
20181204 164448.874 [TRACE] b.g.s.c.j.s.CachingConnectionFactory - Closing producer com.ibm.mq.jms.MQQueueSender#1aad157c
20181204 164448.878 [TRACE] b.g.s.c.j.s.CachingConnectionFactory - Could not close cached JMS Session
com.ibm.msg.client.jms.DetailedJMSException: JMSWMQ0019: Failed to disconnect from queue manager 'xx' using connection mode '1' and host name 'xxx(1446)'.
at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:585) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:221) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.wmq.internal.WMQSession.disconnect(WMQSession.java:712) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.wmq.internal.WMQSession.close(WMQSession.java:663) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.jms.internal.JmsSessionImpl.close(JmsSessionImpl.java:513) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.jms.internal.JmsSessionImpl.close(JmsSessionImpl.java:303) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jms.MQSession.close(MQSession.java:298) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at xxx.server.comm.jms.spring.CachingConnectionFactory$CachedSessionInvocationHandler.physicalClose(CachingConnectionFactory.java:567) ~[xxx.server.comm.jms_18.63.7.SNAPSHOT.jar:na]
at xxx.server.comm.jms.spring.CachingConnectionFactory$CachedSessionInvocationHandler.invoke(CachingConnectionFactory.java:356) ~[xxx.server.comm.jms_18.63.7.SNAPSHOT.jar:na]
at com.sun.proxy.$Proxy124.close(Unknown Source) ~[na:na]
at xxx.server.comm.jms.spring.CachingConnectionFactory.resetConnection(CachingConnectionFactory.java:200) ~[xxx.server.comm.jms_18.63.7.SNAPSHOT.jar:na]
at xxx.server.comm.jms.spring.SingleConnectionFactory.onException(SingleConnectionFactory.java:387) [xxx.server.comm.jms_18.63.7.SNAPSHOT.jar:na]
at xxx.server.comm.jms.spring.SingleConnectionFactory$AggregatedExceptionListener.onException(SingleConnectionFactory.java:832) [xxx.server.comm.jms_18.63.7.SNAPSHOT.jar:na]
at com.ibm.msg.client.jms.internal.JmsProviderExceptionListener.run(JmsProviderExceptionListener.java:427) [com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.runTask(WorkQueueItem.java:214) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.SimpleWorkQueueItem.runItem(SimpleWorkQueueItem.java:105) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.run(WorkQueueItem.java:229) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.WorkQueueManager.runWorkQueueItem(WorkQueueManager.java:303) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.j2se.workqueue.WorkQueueManagerImplementation$ThreadPoolWorker.run(WorkQueueManagerImplementation.java:1241) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
Caused by: com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN').
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:209) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
... 17 common frames omitted
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2009
at com.ibm.mq.jmqi.remote.impl.RemoteSession.getConnection(RemoteSession.java:509) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.impl.RemoteSession.allocateMQAPI(RemoteSession.java:1375) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.impl.RemoteSession.allocateMQAPI(RemoteSession.java:1345) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.api.RemoteFAP.MQDISC(RemoteFAP.java:2741) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.ese.jmqi.InterceptedJmqiImpl.MQDISC(InterceptedJmqiImpl.java:335) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.ese.jmqi.ESEJMQI.MQDISC(ESEJMQI.java:237) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.wmq.internal.WMQSession.disconnect(WMQSession.java:693) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
... 16 common frames omitted
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2009;AMQ9208: Error on receive from host 'xxx/xxx (xxx)'. [1=-1,2=ffffffff,3=xxx/ (xxx),4=TCP]
at com.ibm.mq.jmqi.remote.impl.RemoteRcvThread.receiveBuffer(RemoteRcvThread.java:821) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.impl.RemoteRcvThread.receiveOneTSH(RemoteRcvThread.java:757) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.impl.RemoteRcvThread.run(RemoteRcvThread.java:150) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
... 5 common frames omitted
20181204 164448.878 [DEBUG] b.g.s.c.j.s.CachingConnectionFactory - Closing shared JMS Connection: com.ibm.mq.jms.MQQueueConnection#3bbfe008
20181204 164448.879 [DEBUG] b.g.s.c.j.s.CachingConnectionFactory - Could not close shared JMS Connection
com.ibm.msg.client.jms.DetailedJMSException: JMSWMQ0019: Failed to disconnect from queue manager 'TM01' using connection mode '1' and host name 'xxx'.
at com.ibm.msg.client.wmq.common.internal.Reason.reasonToException(Reason.java:585) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:221) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.wmq.internal.WMQConnection.close(WMQConnection.java:709) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.jms.internal.JmsConnectionImpl.close(JmsConnectionImpl.java:347) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jms.MQConnection.close(MQConnection.java:98) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at xxx.server.comm.jms.spring.SingleConnectionFactory.closeConnection(SingleConnectionFactory.java:546) [xxx.server.comm.jms_18.63.7.SNAPSHOT.jar:na]
at xxx.server.comm.jms.spring.SingleConnectionFactory.resetConnection(SingleConnectionFactory.java:413) [xxx.server.comm.jms_18.63.7.SNAPSHOT.jar:na]
at xxx.server.comm.jms.spring.CachingConnectionFactory.resetConnection(CachingConnectionFactory.java:213) [xxx.server.comm.jms_18.63.7.SNAPSHOT.jar:na]
at xxx.server.comm.jms.spring.SingleConnectionFactory.onException(SingleConnectionFactory.java:387) [xxx.server.comm.jms_18.63.7.SNAPSHOT.jar:na]
at xxx.server.comm.jms.spring.SingleConnectionFactory$AggregatedExceptionListener.onException(SingleConnectionFactory.java:832) [xxx.server.comm.jms_18.63.7.SNAPSHOT.jar:na]
at com.ibm.msg.client.jms.internal.JmsProviderExceptionListener.run(JmsProviderExceptionListener.java:427) [com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.runTask(WorkQueueItem.java:214) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.SimpleWorkQueueItem.runItem(SimpleWorkQueueItem.java:105) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.WorkQueueItem.run(WorkQueueItem.java:229) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.workqueue.WorkQueueManager.runWorkQueueItem(WorkQueueManager.java:303) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.commonservices.j2se.workqueue.WorkQueueManagerImplementation$ThreadPoolWorker.run(WorkQueueManagerImplementation.java:1241) [com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
Caused by: com.ibm.mq.MQException: JMSCMQ0001: WebSphere MQ call failed with compcode '2' ('MQCC_FAILED') reason '2009' ('MQRC_CONNECTION_BROKEN').
at com.ibm.msg.client.wmq.common.internal.Reason.createException(Reason.java:209) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
... 14 common frames omitted
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2009
at com.ibm.mq.jmqi.remote.impl.RemoteSession.getConnection(RemoteSession.java:509) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.impl.RemoteSession.allocateMQAPI(RemoteSession.java:1375) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.impl.RemoteSession.allocateMQAPI(RemoteSession.java:1345) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.api.RemoteFAP.MQDISC(RemoteFAP.java:2741) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.ese.jmqi.InterceptedJmqiImpl.MQDISC(InterceptedJmqiImpl.java:335) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.ese.jmqi.ESEJMQI.MQDISC(ESEJMQI.java:237) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.msg.client.wmq.internal.WMQConnection.close(WMQConnection.java:692) ~[com.ibm.mqjms.jar:7.5.0.7 - p750-007-160812]
... 13 common frames omitted
Caused by: com.ibm.mq.jmqi.JmqiException: CC=2;RC=2009;AMQ9208: Error on receive from host 'xxx'. [1=-1,2=ffffffff,3=xxx,4=TCP]
at com.ibm.mq.jmqi.remote.impl.RemoteRcvThread.receiveBuffer(RemoteRcvThread.java:821) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.impl.RemoteRcvThread.receiveOneTSH(RemoteRcvThread.java:757) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
at com.ibm.mq.jmqi.remote.impl.RemoteRcvThread.run(RemoteRcvThread.java:150) ~[com.ibm.mq.jmqi.jar:7.5.0.7 - p750-007-160812]
... 5 common frames omitted
The root cause of the problem is the following linked exception
MQJE001: Completion Code '2', Reason '2009'.
What this means is that the underlying TCP/IP connection has been closed underneath the messsaging client/server. Typically this issue occurs when there is a lack of activity which causes a firewall to close the connection.
To resolve just change the firewall rule that is causing the connection to be closed.
Related
Using Spring Cloud Hoxton.SR4 and Spring Boot 2.2.9.RELEASE libraries, my Spring Boot microservice starts up correctly and runs just fine when running against localstack in my local environment, but when I upgrade to Hoxton.SR7, I get the stack trace below:
com.amazonaws.SdkClientException: Failed to connect to service endpoint:
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100)
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.getToken(InstanceMetadataServiceResourceFetcher.java:91)
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:69)
at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66)
at com.amazonaws.util.EC2MetadataUtils.getItems(EC2MetadataUtils.java:402)
at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:371)
at org.springframework.cloud.aws.context.support.env.AwsCloudEnvironmentCheckUtils.isRunningOnCloudEnvironment(AwsCloudEnvironmentCheckUtils.java:38)
at org.springframework.cloud.aws.context.annotation.OnAwsCloudEnvironmentCondition.matches(OnAwsCloudEnvironmentCondition.java:38)
at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:108)
at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader$TrackedConditionEvaluator.shouldSkip(ConfigurationClassBeanDefinitionReader.java:469)
at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForConfigurationClass(ConfigurationClassBeanDefinitionReader.java:131)
at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitions(ConfigurationClassBeanDefinitionReader.java:120)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:331)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:236)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:280)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:96)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:707)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:533)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
at com.acme.Bootstrapper.main(Bootstrapper.java:33)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
Caused by: java.net.SocketException: Network unreachable (connect failed)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:341)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:362)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1248)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1227)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1015)
at com.amazonaws.internal.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:52)
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:80)
... 32 common frames omitted
2020-08-05 | 17:42:13.698 | main | WARN | com.amazonaws.util.EC2MetadataUtils | Unable to retrieve the requested metadata (/latest/meta-data/instance-id). Failed to connect to service endpoint:
com.amazonaws.SdkClientException: Failed to connect to service endpoint:
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:100)
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.getToken(InstanceMetadataServiceResourceFetcher.java:91)
at com.amazonaws.internal.InstanceMetadataServiceResourceFetcher.readResource(InstanceMetadataServiceResourceFetcher.java:69)
at com.amazonaws.internal.EC2ResourceFetcher.readResource(EC2ResourceFetcher.java:66)
at com.amazonaws.util.EC2MetadataUtils.getItems(EC2MetadataUtils.java:402)
at com.amazonaws.util.EC2MetadataUtils.getData(EC2MetadataUtils.java:371)
at org.springframework.cloud.aws.context.support.env.AwsCloudEnvironmentCheckUtils.isRunningOnCloudEnvironment(AwsCloudEnvironmentCheckUtils.java:38)
at org.springframework.cloud.aws.context.annotation.OnAwsCloudEnvironmentCondition.matches(OnAwsCloudEnvironmentCondition.java:38)
at org.springframework.context.annotation.ConditionEvaluator.shouldSkip(ConditionEvaluator.java:108)
at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader$TrackedConditionEvaluator.shouldSkip(ConfigurationClassBeanDefinitionReader.java:469)
at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitionsForConfigurationClass(ConfigurationClassBeanDefinitionReader.java:131)
at org.springframework.context.annotation.ConfigurationClassBeanDefinitionReader.loadBeanDefinitions(ConfigurationClassBeanDefinitionReader.java:120)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.processConfigBeanDefinitions(ConfigurationClassPostProcessor.java:331)
at org.springframework.context.annotation.ConfigurationClassPostProcessor.postProcessBeanDefinitionRegistry(ConfigurationClassPostProcessor.java:236)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanDefinitionRegistryPostProcessors(PostProcessorRegistrationDelegate.java:280)
at org.springframework.context.support.PostProcessorRegistrationDelegate.invokeBeanFactoryPostProcessors(PostProcessorRegistrationDelegate.java:96)
at org.springframework.context.support.AbstractApplicationContext.invokeBeanFactoryPostProcessors(AbstractApplicationContext.java:707)
at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:533)
at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:141)
at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:747)
at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:397)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:315)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226)
at org.springframework.boot.SpringApplication.run(SpringApplication.java:1215)
at com.acme.Bootstrapper.main(Bootstrapper.java:33)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:566)
at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:48)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:87)
at org.springframework.boot.loader.Launcher.launch(Launcher.java:51)
at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:52)
Caused by: java.net.SocketException: Network unreachable (connect failed)
at java.base/java.net.PlainSocketImpl.socketConnect(Native Method)
at java.base/java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:399)
at java.base/java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:242)
at java.base/java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:224)
at java.base/java.net.Socket.connect(Socket.java:609)
at java.base/sun.net.NetworkClient.doConnect(NetworkClient.java:177)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:474)
at java.base/sun.net.www.http.HttpClient.openServer(HttpClient.java:569)
at java.base/sun.net.www.http.HttpClient.<init>(HttpClient.java:242)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:341)
at java.base/sun.net.www.http.HttpClient.New(HttpClient.java:362)
at java.base/sun.net.www.protocol.http.HttpURLConnection.getNewHttpClient(HttpURLConnection.java:1248)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect0(HttpURLConnection.java:1227)
at java.base/sun.net.www.protocol.http.HttpURLConnection.plainConnect(HttpURLConnection.java:1081)
at java.base/sun.net.www.protocol.http.HttpURLConnection.connect(HttpURLConnection.java:1015)
at com.amazonaws.internal.ConnectionUtils.connectToEndpoint(ConnectionUtils.java:52)
at com.amazonaws.internal.EC2ResourceFetcher.doReadResource(EC2ResourceFetcher.java:80)
... 32 common frames omitted
What new or additional configurations/code do I need to include to turn this EC2 checking off in my local environment when running against localstack?
This is a warning message logged directly from AWS SDK and inside Spring Cloud AWS we have no control over it. Following documentation, just turn this warning off by adding a line to your application.properties config file:
logging.level.com.amazonaws.util.EC2MetadataUtils=error
I am using alfresco-community-5.2.0 version and solr4. Getting the following error while hitting from my API. What can be the solution. Am not getting the exact solution. Can anyone give me a solution for this to try. Following is the error am getting in alfresco log and Solr log
alfresco log
ERROR [org.quartz.core.JobRunShell] [DefaultScheduler_Worker-4] Job DEFAULT.org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean threw an unhandled Exception:
org.springframework.scheduling.quartz.JobMethodInvocationFailedException: Invocation of method 'run' on target class [class org.alfresco.module.org_alfresco_module_wcmquickstart.jobs.DynamicCollectionProcessor] failed; nested exception is org.alfresco.repo.search.impl.lucene.LuceneQueryParserException: 0320217066
at org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean$MethodInvokingJob.executeInternal(MethodInvokingJobDetailFactoryBean.java:321)
at org.springframework.scheduling.quartz.QuartzJobBean.execute(QuartzJobBean.java:114)
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:563)
Caused by: org.alfresco.repo.search.impl.lucene.LuceneQueryParserException: 0320217066
at org.alfresco.repo.search.impl.solr.SolrQueryHTTPClient.executeQuery(SolrQueryHTTPClient.java:540)
at org.alfresco.repo.search.impl.solr.SolrQueryLanguage.executeQuery(SolrQueryLanguage.java:51)
at org.alfresco.repo.search.impl.solr.SolrSearchService.query(SolrSearchService.java:348)
at org.alfresco.repo.search.impl.solr.SolrSearchService.query(SolrSearchService.java:152)
at org.alfresco.repo.search.SearcherComponent.query(SearcherComponent.java:66)
at org.alfresco.repo.search.AbstractSearcherComponent.query(AbstractSearcherComponent.java:53)
at sun.reflect.GeneratedMethodAccessor661.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.alfresco.repo.management.subsystems.SubsystemProxyFactory$1.invoke(SubsystemProxyFactory.java:72)
at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:172)
at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:204)
at com.sun.proxy.$Proxy23.query(Unknown Source)
at org.alfresco.repo.search.impl.SearchServiceSubSystemDelegator.query(SearchServiceSubSystemDelegator.java:91)
at org.alfresco.module.org_alfresco_module_wcmquickstart.jobs.DynamicCollectionProcessor$1$1.execute(DynamicCollectionProcessor.java:245)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:457)
at org.alfresco.repo.transaction.RetryingTransactionHelper.doInTransaction(RetryingTransactionHelper.java:326)
at org.alfresco.module.org_alfresco_module_wcmquickstart.jobs.DynamicCollectionProcessor$1.doWork(DynamicCollectionProcessor.java:234)
at org.alfresco.repo.security.authentication.AuthenticationUtil.runAs(AuthenticationUtil.java:548)
at org.alfresco.module.org_alfresco_module_wcmquickstart.jobs.DynamicCollectionProcessor.runInternal(DynamicCollectionProcessor.java:229)
at org.alfresco.module.org_alfresco_module_wcmquickstart.jobs.DynamicCollectionProcessor.run(DynamicCollectionProcessor.java:192)
at sun.reflect.GeneratedMethodAccessor706.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:483)
at org.springframework.util.MethodInvoker.invoke(MethodInvoker.java:269)
at org.springframework.scheduling.quartz.MethodInvokingJobDetailFactoryBean$MethodInvokingJob.executeInternal(MethodInvokingJobDetailFactoryBean.java:312)
... 3 more
Caused by: java.net.ConnectException: Connection refused
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:345)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:656)
at sun.security.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:460)
at sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:153)
at org.alfresco.encryption.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:168)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at org.alfresco.repo.search.impl.solr.SolrQueryHTTPClient.postQuery(SolrQueryHTTPClient.java:601)
at org.alfresco.repo.search.impl.solr.SolrQueryHTTPClient.postSolrQuery(SolrQueryHTTPClient.java:559)
at org.alfresco.repo.search.impl.solr.SolrQueryHTTPClient.executeQuery(SolrQueryHTTPClient.java:520)
... 28 more
solr log
ERROR [solr.tracker.AbstractTracker] [org.alfresco.solr.AlfrescoCoreAdminHandler#2dc6565_Worker-26] Tracking failed
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at sun.security.ssl.SSLSocketImpl.connect(SSLSocketImpl.java:668)
at sun.security.ssl.SSLSocketImpl.<init>(SSLSocketImpl.java:472)
at sun.security.ssl.SSLSocketFactoryImpl.createSocket(SSLSocketFactoryImpl.java:153)
at org.alfresco.encryption.ssl.AuthSSLProtocolSocketFactory.createSocket(AuthSSLProtocolSocketFactory.java:168)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at org.alfresco.httpclient.AbstractHttpClient.executeMethod(AbstractHttpClient.java:135)
at org.alfresco.httpclient.AbstractHttpClient.sendRemoteRequest(AbstractHttpClient.java:111)
at org.alfresco.httpclient.HttpClientFactory$HttpsClient.sendRequest(HttpClientFactory.java:408)
at org.alfresco.solr.client.SOLRAPIClient.getAclChangeSets(SOLRAPIClient.java:165)
at org.alfresco.solr.tracker.AclTracker.checkRepoAndIndexConsistency(AclTracker.java:354)
at org.alfresco.solr.tracker.AclTracker.trackRepository(AclTracker.java:320)
at org.alfresco.solr.tracker.AclTracker.doTrack(AclTracker.java:111)
at org.alfresco.solr.tracker.AbstractTracker.track(AbstractTracker.java:190)
at org.alfresco.solr.tracker.TrackerJob.execute(TrackerJob.java:54)
at org.quartz.core.JobRunShell.run(JobRunShell.java:216)
at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:563)
I'm regularly getting SocketTimeout Exception on del operation on redis server using Jedis. It's only 1% of all the delete operations done.
Exception:-
Caused by: java.net.SocketTimeoutException: Read timed out
at java.net.SocketInputStream.socketRead0(Native Method) ~[?:1.8.0_77]
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116) ~[?:1.8.0_77]
at java.net.SocketInputStream.read(SocketInputStream.java:170) ~[?:1.8.0_77]
at java.net.SocketInputStream.read(SocketInputStream.java:141) ~[?:1.8.0_77]
at java.net.SocketInputStream.read(SocketInputStream.java:127) ~[?:1.8.0_77]
at redis.clients.util.RedisInputStream.ensureFill(RedisInputStream.java:195) ~[jedis-2.x.jar:?]
at redis.clients.util.RedisInputStream.readByte(RedisInputStream.java:40) ~[jedis-2.x.jar:?]
at redis.clients.jedis.Protocol.process(Protocol.java:141) ~[jedis-2.x.jar:?]
at redis.clients.jedis.Protocol.read(Protocol.java:205) ~[jedis-2.x.jar:?]
at redis.clients.jedis.Connection.readProtocolWithCheckingBroken(Connection.java:297) ~[jedis-2.x.jar:?]
at redis.clients.jedis.Connection.getIntegerReply(Connection.java:222) ~[jedis-2.x.jar:?]
at redis.clients.jedis.Jedis.del(Jedis.java:145) ~[jedis-2.x.jar:?]
at redis.clients.jedis.ShardedJedis.del(ShardedJedis.java:252) ~[jedis-2.x.jar:?]
I checked my shared pool configuration and it is enough to handle the traffic. I did a client list on the server and do not see any client with age over 180 sec. Any broken resource is dropped from the pool.
What could be the issue here and how can I investigate this further?
I am trying to configure hadoop from [https://www.tutorialspoint.com/hadoop/hadoop_mapreduce.htm][1].
I have configured everything.Now at the time of execution command $HADOOP_HOME/bin/hadoop jar /home/abp/unit/unit.jar com.hadoop.ProcessUnits input_dir output_dir
I have getting below exception
abp#abp-Precision-T1700:/usr/local/hadoop/bin$ $HADOOP_HOME/bin/hadoop jar /home/abp/unit/unit.jar com.hadoop.ProcessUnits input_dir output_dir
17/04/26 15:09:09 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
17/04/26 15:09:10 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
17/04/26 15:09:10 INFO client.RMProxy: Connecting to ResourceManager at /0.0.0.0:8032
Exception in thread "main" java.io.IOException: Failed on local exception: org.apache.hadoop.ipc.RpcException: RPC response exceeds maximum data length; Host Details : local host is: "abp-Precision-T1700/127.0.1.1"; destination host is: "localhost":9000;
at org.apache.hadoop.net.NetUtils.wrapException(NetUtils.java:782)
at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1485)
at org.apache.hadoop.ipc.Client.call(Client.java:1427)
at org.apache.hadoop.ipc.Client.call(Client.java:1337)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:227)
at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:116)
at com.sun.proxy.$Proxy10.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getFileInfo(ClientNamenodeProtocolTranslatorPB.java:787)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:398)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeMethod(RetryInvocationHandler.java:163)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invoke(RetryInvocationHandler.java:155)
at org.apache.hadoop.io.retry.RetryInvocationHandler$Call.invokeOnce(RetryInvocationHandler.java:95)
at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:335)
at com.sun.proxy.$Proxy11.getFileInfo(Unknown Source)
at org.apache.hadoop.hdfs.DFSClient.getFileInfo(DFSClient.java:1700)
at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1436)
at org.apache.hadoop.hdfs.DistributedFileSystem$27.doCall(DistributedFileSystem.java:1433)
at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
at org.apache.hadoop.hdfs.DistributedFileSystem.getFileStatus(DistributedFileSystem.java:1433)
at org.apache.hadoop.fs.FileSystem.exists(FileSystem.java:1436)
at org.apache.hadoop.mapred.FileOutputFormat.checkOutputSpecs(FileOutputFormat.java:130)
at org.apache.hadoop.mapreduce.JobSubmitter.checkSpecs(JobSubmitter.java:270)
at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:141)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1341)
at org.apache.hadoop.mapreduce.Job$11.run(Job.java:1338)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
at org.apache.hadoop.mapreduce.Job.submit(Job.java:1338)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:575)
at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:570)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1807)
at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:570)
at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:561)
at org.apache.hadoop.mapred.JobClient.runJob(JobClient.java:870)
at com.hadoop.ProcessUnits.main(ProcessUnits.java:95)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.run(RunJar.java:234)
at org.apache.hadoop.util.RunJar.main(RunJar.java:148)
Caused by: org.apache.hadoop.ipc.RpcException: RPC response exceeds maximum data length
at org.apache.hadoop.ipc.Client$IpcStreams.readResponse(Client.java:1800)
at org.apache.hadoop.ipc.Client$Connection.receiveRpcResponse(Client.java:1155)
at org.apache.hadoop.ipc.Client$Connection.run(Client.java:1052)
Please help me in resolving this.
After restarting my pentaho server , admin console is not up, this throw-ed me following errors ,
This would be the extension of Issue with data sources that are created through the Pentaho Admin Console post .
Do any 1 faced this, do any one have an idea about this
2013-07-16 03:24:52,965 INFO [org.apache.commons.httpclient.HttpMethodDirector] Retrying request
2013-07-16 03:28:01,778 ERROR [com.pentaho.pac.server.PacProServiceImpl]
org.pentaho.pac.server.common.ProxyException: Connection timed out
at org.pentaho.pac.server.common.ThreadSafeHttpClient.executeMethod(ThreadSafeHttpClient.java:183)
at org.pentaho.pac.server.common.ThreadSafeHttpClient.execRemoteMethod(ThreadSafeHttpClient.java:147)
at org.pentaho.pac.server.common.ThreadSafeHttpClient.execRemoteMethod(ThreadSafeHttpClient.java:100)
at org.pentaho.pac.server.common.BiServerTrustedProxy.execRemoteMethod(BiServerTrustedProxy.java:93)
at com.pentaho.pac.server.PacProServiceImpl.k(SourceFile:1175)
at com.pentaho.pac.server.PacProServiceImpl.q(SourceFile:1193)
at com.pentaho.pac.server.PacProServiceImpl.getTestResult(SourceFile:750)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:562)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:188)
at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java:224)
at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java:62)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487)
at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:367)
at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712)
at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139)
at org.mortbay.jetty.Server.handle(Server.java:285)
at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:502)
at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:835)
at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:638)
at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:208)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:378)
at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:226)
at org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:450)
Caused by: java.net.ConnectException: Connection timed out
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.PlainSocketImpl.doConnect(Unknown Source)
at java.net.PlainSocketImpl.connectToAddress(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.connect(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at java.net.Socket.<init>(Unknown Source)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:80)
at org.apache.commons.httpclient.protocol.DefaultProtocolSocketFactory.createSocket(DefaultProtocolSocketFactory.java:122)
at org.apache.commons.httpclient.HttpConnection.open(HttpConnection.java:707)
at org.apache.commons.httpclient.MultiThreadedHttpConnectionManager$HttpConnectionAdapter.open(MultiThreadedHttpConnectionManager.java:1361)
at org.apache.commons.httpclient.HttpMethodDirector.executeWithRetry(HttpMethodDirector.java:387)
at org.apache.commons.httpclient.HttpMethodDirector.executeMethod(HttpMethodDirector.java:171)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:397)
at org.apache.commons.httpclient.HttpClient.executeMethod(HttpClient.java:323)
at org.pentaho.pac.server.common.ThreadSafeHttpClient.executeMethod(ThreadSafeHttpClient.java:163)
... 30 more
2013-07-16 03:31:10,786 INFO [org.apache.commons.httpclient.HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection timed out
2013-07-16 03:31:10,786 INFO [org.apache.commons.httpclient.HttpMethodDirector] Retrying request
2013-07-16 03:31:10,970 INFO [org.apache.commons.httpclient.HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection timed out
2013-07-16 03:31:10,970 INFO [org.apache.commons.httpclient.HttpMethodDirector] Retrying request
2013-07-16 03:34:19,784 INFO [org.apache.commons.httpclient.HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection timed out
2013-07-16 03:34:19,785 INFO [org.apache.commons.httpclient.HttpMethodDirector] Retrying request
2013-07-16 03:34:19,960 INFO [org.apache.commons.httpclient.HttpMethodDirector] I/O exception (java.net.ConnectException) caught when processing request: Connection timed out
2013-07-16 03:34:19,960 INFO [org.apache.commons.httpclient.HttpMethodDirector] Retrying request
Your BI server is not starting up, probably. Can you access the BI server homepage?