I'm using the RabbitMQ 3.6.12 version with java 8 client. I'm getting a frequent exception in the RabbitMQ log and my client is getting exited to consume the message.
logs on the RabbitMQ:
=WARNING REPORT==== 29-DEC-2018::16:02:38 === closing AMQP connection <0.8979.3> (127.0.0.1:57065 -> 127.0.0.1:5672, vhost: '/', user:
'guest'): client unexpectedly closed TCP connection
logs on the Java Client:
[com.rabbitmq.client.impl.ForgivingExceptionHandler] (AMQP Connection 127.0.0.1:5672)
An unexpected connection driver error occured: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:288)
at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:91)
at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:164)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:580)
at java.lang.Thread.run(Thread.java:748)"
Related
I'm able connect IBM Queue directly but when u tried to connect from mule getting the below error and not able to deploy. I'm getting the below error
ERROR 2017-04-25 06:45:13,582
[main]org.mule.retry.notifiers.ConnectNotifier: Failed to connect/reconnect:
WebSphereMQConnector
{
name=WMQ2
lifecycle=initialise
this=5e7abaf7
numberOfConcurrentTransactedReceivers=4
createMultipleTransactedReceivers=true
connected=false
supportedProtocols=[wmq]
serviceOverrides=<none>
}
. Root Exception was: Connection timed out: connect. Type: class java.net.ConnectException
ERROR 2017-04-25 06:50:23,943 [main] org.mule.module.launcher.application.DefaultMuleApplication:
************************************************
Message : JMSWMQ0018: Failed to connect to queue manager 'RQACBRKB' with connection mode 'Client' and host name '172.11.11.11(6912)'.
JMS Code : JMSWMQ0018
Element : /WMQ2 # app:config.xml:14 (WMQ)
--------------------------------------------------------------------------------
Root Exception stack trace:
java.net.ConnectException: Connection timed out: connect
at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
com.ibm.mq.jmqi.JmqiException: CC=2;RC=2538;AMQ9213: A communications error for occurred [1=java.net.ConnectException[Connection timed out: connect],3=rbitbrka.apl.com] at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.connnectUsingLocalAddress(RemoteTCPConnection.java:810) ~[?:?]
PFB connector details:
<wmq:connector name="WMQ5" hostName="${mq.host}" port="${mq.port}" queueManager="${mq.queue.manager}" channel="CLIENTS.SALES.CRM" username="${mq.user}" password="${mq.password}" transportType="CLIENT_MQ_TCPIP" specification="1.1" targetClient="JMS_COMPLIANT" validateConnections="false" doc:name="WMQ" maxRedelivery="-1">
<reconnect frequency="${mq.reconnection.period.ms}" count="${mq.reconnection.attempt}"/>
</wmq:connector>
When i telnet the ip and port getting below error:
C:\Users\111>telnet 172.11.11.11 6912
Connecting To 172.11.11.11...Could not open connection to the host, on port 6912: Connect failed
But when i ping getting responce
C:\Users\111>ping 172.11.11.11
The pertinent pieces of information from your provided error are:-
JMSWMQ0018: Failed to connect to queue manager 'RQACBRKB'
with connection mode 'Client' and host name '172.11.11.11(6912)'.
com.ibm.mq.jmqi.JmqiException: CC=2;RC=2538;
MQRC 2538 is MQRC_HOST_NOT_AVAILABLE which is explained in Knowledge Center. In there it mentions the most common reasons for this error:-
The listener has not been started on the remote system. (please check that your listener is running on port 6912 on the machine at IP address 172.11.11.11)
The connection name in the client channel definition is incorrect. (the connection name your client is using is '172.11.11.11(6912)' - is this correct?)
The network is currently unavailable.
A firewall blocking the port, or protocol-specific traffic.
The security call initializing the IBM MQ client is blocked by a security exit on the SVRCONN channel at the server.
"java.net.ConnectException: Connection timed out: connect" usually occurs when you have a config issue or you are unable to connect to the remote server. As mentioned above do you have an error on the MQ end and if not have you checked the connection properties within config. If these are correct, are you able to reach MQ from another client, such as SOAPUI?
Can you also post the connector and flow details?
Can someone provide your thoughts. what is going wrong on my configuration.
This was one approach:
public CamelContext camelContext() throws Exception {
final SpringCamelContext camelContext = new SpringCamelContext();
camelContext.getProperties().put("http.proxyHost","webproxy.lon.corp.services");
camelContext.getProperties().put("http.proxyPort","80");
camelContext.addRoutes(new CortellisRouteBuilder(env));
return camelContext;
}
Second Approach
https4://xxxxxx/xxxx/ws/rs/xx-v1/match?authUsername=someUsernamexxx&authPassword=xxxxxxxxx&authMethod=Digest&proxyHost=webproxy.services.xxx.com&proxyPort=80&proxyAuthScheme=http4
both does not work. below is the exception that i receive. it does 3 retires and then fails.
Exchange[
Id ID-UC193537-TPL-A-54917-1428598980021-0-2
ExchangePattern InOut
Headers {breadcrumbId=ID-UC193537-TPL-A-54917-1428598980021-0-1,
CamelAcceptContentType=application/x-www-form-urlencoded,
CamelHttpMethod=POST,camelHttpQuery=text=Paracetamol,
camelRedelivered=false, CamelRedeliveryCounter=0}
BodyType null
Body [Body is null]]
====================================================================
I/O exception (java.net.SocketException) caught when processing request:
Software caused connection abort: recv failed
2015-04-09 18:04:36,876 - INFO [http-apr-8080-exec-
4:HttpMethodDirector.executeWithRetry#445] - Retrying request
2015-04-09 18:04:37,015 - INFO [http-apr-8080-exec-
4:HttpMethodDirector.executeWithRetry#439] - I/O exception
(java.net.SocketException) caught when processing request: Software caused
connection abort: recv failed
2015-04-09 18:04:37,015 - INFO [http-apr-8080-exec-
4:HttpMethodDirector.executeWithRetry#445] - Retrying request
2015-04-09 18:04:37,167 - INFO [http-apr-8080-exec-
4:HttpMethodDirector.executeWithRetry#439] - I/O exception
(java.net.SocketException) caught when processing request: Software caused
connection abort: recv failed
2015-04-09 18:04:37,168 - INFO [http-apr-8080-exec-
4:HttpMethodDirector.executeWithRetry#445] - Retrying request
2015-04-09 18:04:37,298 - ERROR [http-apr-8080-exec-
4:MarkerIgnoringBase.error#161] - Failed delivery for (MessageId: ID-
UC193537-TPL-A-54917-1428598980021-0-1 on ExchangeId: ID-UC193537-TPL-A-
54917-1428598980021-0-2). Exhausted after delivery attempt: 1 caught:
java.net.SocketException: Software caused connection abort: recv failed
====================================================================
java.net.SocketException: Software caused connection abort: recv failed
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.read(SocketInputStream.java:173)
at java.net.SocketInputStream.read(SocketInputStream.java:122)
at sun.security.ssl.InputRecord.readFully(InputRecord.java:442)
at sun.security.ssl.InputRecord.read(InputRecord.java:480)
at sun.security.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:934)
at sun.security.ssl.SSLSocketImpl.performInitialHandshake(SSLSocketImpl.java:1332)
at sun.security.ssl.SSLSocketImpl.writeRecord(SSLSocketImpl.java:709)
at sun.security.ssl.AppOutputStream.write(AppOutputStream.java:122)
at java.io.BufferedOutputStream.flushBuffer(BufferedOutputStream.java:82)
at java.io.BufferedOutputStream.flush(BufferedOutputStream.java:140)
at
This error can occur when the local network system aborts a connection, see e.g. java.net.SocketException: Software caused connection abort: recv failed.
The proxy server may abort the connection. Can you try to connect to your service directly (= not through your proxy)?
cortellis.ontologiesNerUrl=https4://xxxx/ls-api-ws/ws/rs/xxxx-v1/ner?proxyPort=80&proxyHost=xxxxxxxx&authMethod=Basic&authUsername=TR_Internal_024&authPassword=ZTYA5S1KLF7WCDMN&**proxyAuthScheme=http4**
with addition of proxyAuthScheme=http4 it worked for rest based URL, But not for SOAP based URL
i have picked the latest camel version 2.15.1
I'm using AMQP connector,and my queues are present in cluster envt. I'm not sure why this message is populated always. But this is not affecting any success running flow. But it is trying to try connections always.
I'm using Amqp transport jar: 3.4.4 ( Even I tried importing with latest jar 3.4.6 jar) same error has happening again.
Amqp client: 3.2.1.
I'm totally confused.
Because I tried the same queues which is not present in cluster it works fine.
Please suggest me where i'm doing wrong.
Using Ack( Manual) changed to 'MULE_AUTO' also. Helpless.
WARN 2015-01-29 14:47:29,639 [amqpReceiver.01] org.mule.transport.amqp.internal.endpoint.receiver.MessageReceiverConsumer: Received shutdown signal for consumer tag: amq.ctag-13inp2uTSjUnIgiNjyFYAA, the message receiver will try to restart.
com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.io.EOFException
at com.rabbitmq.client.impl.AMQConnection.startShutdown(AMQConnection.java:715)
at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:705)
atcom.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:564)
Caused by: java.io.EOFException
at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:290)
at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95)
at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:533)
WARN 2015-01-29 14:47:29,642 [amqpReceiver.03] org.mule.transport.amqp.internal.endpoint.receiver.MessageReceiverConsumer: Received shutdown signal for consumer tag: amq.ctag-G-xCK-Uj-0aF7LcrYIuo3A, the message receiver will try to restart.
com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.io.EOFException
at com.rabbitmq.client.impl.AMQConnection.startShutdown(AMQConnection.java:715)
at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:705)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:564)
Caused by: java.io.EOFException
at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:290)
at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95)
at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:533)
WARN 2015-01-29 14:47:29,642 [amqpReceiver.02] org.mule.transport.amqp.internal.endpoint.receiver.MessageReceiverConsumer: Received shutdown signal for consumer tag: amq.ctag-seeeDsnuB_1sSuo97C8OdQ, the message receiver will try to restart.
com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.io.EOFException
at com.rabbitmq.client.impl.AMQConnection.startShutdown(AMQConnection.java:715)
at com.rabbitmq.client.impl.AMQConnection.shutdown(AMQConnection.java:705)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:564)
Caused by: java.io.EOFException
at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:290)
at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95)
at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:533)
ERROR 2015-01-29 14:47:29,665 [AMQP Connection] org.mule.exception.DefaultSystemExceptionStrategy:
********************************************************************************
Message : Connection shutdown detected for: AMQP_Connector
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. null (java.io.EOFException)
java.io.DataInputStream:290 (null)
2. connection error; reason: java.io.EOFException (com.rabbitmq.client.ShutdownSignalException)
com.rabbitmq.client.impl.AMQConnection:715 (null)
3. Connection shutdown detected for: AMQP_Connector (org.mule.transport.ConnectException)
org.mule.transport.amqp.internal.connector.AmqpConnector$1:296 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/transport/ConnectEx ception.html)
--------------------------------------------------------------------------------
Root Exception stack trace:
java.io.EOFException
at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:290)
at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95)
at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131)
+ 3 more (set debug level logging or '-Dmule.verbose.exceptions=true' for everything)
********************************************************************************
INFO 2015-01-29 14:47:29,696 [AMQP Connection ] org.mule.exception.DefaultSystemExceptionStrategy: Exception caught is a ConnectException, attempting to reconnect...
ERROR 2015-01-29 14:47:29,698 [amqpReceiver.03] org.mule.exception.DefaultSystemExceptionStrategy:
********************************************************************************
Message : Impossible to create new channels on connection: amqp://test#11.11.2000.111:1234/
Code : MULE_ERROR--2
--------------------------------------------------------------------------------
Exception stack is:
1. clean connection shutdown; reason: Attempt to use closed connection (com.rabbitmq.client.AlreadyClosedException)
com.rabbitmq.client.impl.AMQConnection:168 (null)
2. Impossible to create new channels on connection: amqp://test#11.11.2000.111:1234/ (org.mule.transport.ConnectException)
org.mule.transport.amqp.internal.client.ChannelHandler:204 (http://www.mulesoft.org/docs/site/current3/apidocs/org/mule/transport/ConnectEx ception.html)
ERROR 2015-01-29 15:12:51,604 [amqpReceiver.34] org.mule.transport.amqp.internal.endpoint.receiver.MultiChannelMessageSubReceiver: Failed to restart: MultiChannelMessageSubReceiver(.....)
Edit:
org.mule.api.MessagingException: Failed to ack message w/deliveryTag: 1 on channel: AMQChannel(amqp://ezibuy#10.10.19.159:5672/,1) (org.mule.api.DefaultMuleException). Message payload is of type: String
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:32)
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:58)
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
at org.mule.processor.chain.DefaultMessageProcessorChain.doProcess(DefaultMessageProcessorChain.java:94)
at org.mule.processor.chain.AbstractMessageProcessorChain.process(AbstractMessageProcessorChain.java:67)
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.doProcess(InterceptingChainLifecycleWrapper.java:50)
at org.mule.processor.chain.AbstractMessageProcessorChain.process(AbstractMessageProcessorChain.java:67)
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.access$001(InterceptingChainLifecycleWrapper.java:22)
at org.mule.processor.chain.InterceptingChainLifecycleWrapper$1.process(InterceptingChainLifecycleWrapper.java:66)
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
at org.mule.execution.MessageProcessorNotificationExecutionInterceptor.execute(MessageProcessorNotificationExecutionInterceptor.java:58)
at org.mule.execution.MessageProcessorExecutionTemplate.execute(MessageProcessorExecutionTemplate.java:44)
at org.mule.processor.chain.InterceptingChainLifecycleWrapper.process(InterceptingChainLifecycleWrapper.java:61)
at org.mule.exception.TemplateMessagingExceptionStrategy.route(TemplateMessagingExceptionStrategy.java:139)
at org.mule.exception.TemplateMessagingExceptionStrategy.handleException(TemplateMessagingExceptionStrategy.java:45)
at org.mule.execution.HandleExceptionInterceptor.execute(HandleExceptionInterceptor.java:37)
at org.mule.execution.HandleExceptionInterceptor.execute(HandleExceptionInterceptor.java:14)
at org.mule.execution.BeginAndResolveTransactionInterceptor.execute(BeginAndResolveTransactionInterceptor.java:54)
at org.mule.execution.ResolvePreviousTransactionInterceptor.execute(ResolvePreviousTransactionInterceptor.java:44)
at org.mule.execution.SuspendXaTransactionInterceptor.execute(SuspendXaTransactionInterceptor.java:50)
at org.mule.execution.ValidateTransactionalStateInterceptor.execute(ValidateTransactionalStateInterceptor.java:40)
at org.mule.execution.IsolateCurrentTransactionInterceptor.execute(IsolateCurrentTransactionInterceptor.java:41)
at org.mule.execution.ExternalTransactionInterceptor.execute(ExternalTransactionInterceptor.java:48)
at org.mule.execution.RethrowExceptionInterceptor.execute(RethrowExceptionInterceptor.java:28)
at org.mule.execution.RethrowExceptionInterceptor.execute(RethrowExceptionInterceptor.java:13)
at org.mule.execution.TransactionalErrorHandlingExecutionTemplate.execute(TransactionalErrorHandlingExecutionTemplate.java:109)
at org.mule.execution.TransactionalErrorHandlingExecutionTemplate.execute(TransactionalErrorHandlingExecutionTemplate.java:30)
at org.mule.processor.AsyncInterceptingMessageProcessor$AsyncMessageProcessorWorker.doRun(AsyncInterceptingMessageProcessor.java:181)
at org.mule.work.AbstractMuleEventWork.run(AbstractMuleEventWork.java:39)
at org.mule.work.WorkerContext.run(WorkerContext.java:286)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:724)
Caused by: org.mule.api.DefaultMuleException: Failed to ack message w/deliveryTag: 1 on channel: AMQChannel(amqp://ezibuy#10.10.19.159:5672/,1)
at org.mule.transport.amqp.internal.processor.Acknowledger.ack(Acknowledger.java:63)
at org.mule.transport.amqp.internal.processor.Acknowledger.ack(Acknowledger.java:49)
at org.mule.transport.amqp.internal.processor.Acknowledger.process(Acknowledger.java:38)
at org.mule.execution.ExceptionToMessagingExceptionExecutionInterceptor.execute(ExceptionToMessagingExceptionExecutionInterceptor.java:24)
... 32 more
Caused by: com.rabbitmq.client.AlreadyClosedException: channel is already closed due to channel error; protocol method: #method<channel.close>(reply-code=406, reply-text=PRECONDITION_FAILED - unknown delivery tag 1, class-id=60, method-id=80)
at com.rabbitmq.client.impl.AMQChannel.ensureIsOpen(AMQChannel.java:190)
at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:291)
at com.rabbitmq.client.impl.AMQChannel.transmit(AMQChannel.java:285)
at com.rabbitmq.client.impl.ChannelN.basicAck(ChannelN.java:1012)
at org.mule.transport.amqp.internal.processor.Acknowledger.ack(Acknowledger.java:59)
There should be a delivery-tag flow variable that is used to keep track of the connection. This variable is required to be able to acknowledge the message. I suspect the flow variable is disappearing at some point. This can happen if the ACK is done in a separate thread from the one where the flow starts. The reason is that the delivery-tag variable can't be serialized/deserialized as it is moved to another thread.
Is anything being done asynchronously?
You are probably bitten a bug present when using request-response or request-reply.
Please try to use 3.6 where the bug is fixed.
I am using Kafka 0.8.2-beta and have 2 Ubuntu 14 virtual machines:
172.30.141.127 is running Zookeeper
172.30.141.184 is running a Kafka broker
I'm starting the Zookeeper instance and all if fine. Then I try to start the broker and connect it to 172.30.141.127:2181. It seems to be able to connect and establish a session on a specific port, but then it losses connection due to some exception that doesn't seem to be logged.
The broker output:
[2015-01-19 11:03:55,029] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,030] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,031] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,031] INFO Client environment:os.arch=i386 (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,032] INFO Client environment:os.version=3.16.0-23-generic (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,033] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,033] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,037] INFO Client environment:user.dir=/home/osboxes/Desktop/kafka_2.11-0.8.2-beta (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,039] INFO Initiating client connection, connectString=172.30.141.127:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient#1ecf473 (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,129] INFO Opening socket connection to server 172.30.141.127/172.30.141.127:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:55,186] INFO Socket connection established to 172.30.141.127/172.30.141.127:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:55,203] WARN Session 0x0 for server 172.30.141.127/172.30.141.127:2181, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[2015-01-19 11:03:56,552] INFO Opening socket connection to server 172.30.141.127/172.30.141.127:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:56,555] INFO Socket connection established to 172.30.141.127/172.30.141.127:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:56,567] WARN Session 0x0 for server 172.30.141.127/172.30.141.127:2181, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[2015-01-19 11:03:57,131] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2015-01-19 11:03:58,075] INFO Opening socket connection to server 172.30.141.127/172.30.141.127:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:58,077] INFO Socket connection established to 172.30.141.127/172.30.141.127:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:58,195] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:58,196] INFO EventThread shut down (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:58,251] FATAL [Kafka Server 1], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 2000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:157)
at kafka.server.KafkaServer.startup(KafkaServer.scala:83)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:28)
at kafka.Kafka$.main(Kafka.scala:46)
at kafka.Kafka.main(Kafka.scala)
[2015-01-19 11:03:58,279] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)
[2015-01-19 11:03:58,295] INFO [Kafka Server 1], shut down completed (kafka.server.KafkaServer)
[2015-01-19 11:03:58,308] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 2000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:157)
at kafka.server.KafkaServer.startup(KafkaServer.scala:83)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:28)
at kafka.Kafka$.main(Kafka.scala:46)
at kafka.Kafka.main(Kafka.scala)
[2015-01-19 11:03:58,335] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)
Zookeper outputs:
[2015-01-19 11:03:55,245] INFO Accepted socket connection from /172.30.141.184:54089 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2015-01-19 11:03:55,315] WARN Exception causing close of session 0x0 due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:55,329] INFO Closed socket connection for client /172.30.141.184:54089 (no session established for client) (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:56,613] INFO Accepted socket connection from /172.30.141.184:54090 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2015-01-19 11:03:56,615] WARN Exception causing close of session 0x0 due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:56,617] INFO Closed socket connection for client /172.30.141.184:54090 (no session established for client) (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:58,133] INFO Accepted socket connection from /172.30.141.184:54091 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2015-01-19 11:03:58,134] WARN Exception causing close of session 0x0 due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:58,135] INFO Closed socket connection for client /172.30.141.184:54091 (no session established for client) (org.apache.zookeeper.server.NIOServerCnxn)
Ping works between the 2 machines. Telnet to 2181 kind of works, in that it connects, but gets disconnected from time to time. This leads me to think that the problem is with the Zookeeper instance. Both processes are started as root.
Any ideas why this is happening? Thanks
You are probably hitting the max number of connections per host.
This happens if you have [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory#247] - Too many connections from /127.0.0.1 - max is 10 in your zookeeper logs.
Fix it by setting maxClientCnxns=[something more than 10; 0 for unlimited] in your conf/zoo.cfg.
Docs (search for maxClientCnxns)
No idea about the warning, but I had the same Error problem
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 2000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
....
In my case when I was setting up producer and consumer, I provided wrong ip/port. When I changed it to correct one with:
bin/kafka-console-producer.sh --broker-list kafka_ip:kafka_port --topic test
bin/kafka-console-consumer.sh --zookeeper zookeeper_id:zookeper_port --topic test --from-beginning
my problem was solved.
As answered by #RickyA
"You are probably hitting the max number of connections per host."
Try clearing your logs folder which in my case was F:\tmp\kafka-logs.
I'm having difficulty configuring my connection to CloudAMQP in my deployed grails application. I can run the application locally against a locally installed RabbitMQ instance but can't figure out how to correctly define my application to run on CloudBees using the CloudAMQP service.
In my Config.groovy, I'm defining my connection info and a queue:
rabbitmq {
connectionfactory {
username = 'USERNAME'
password = 'PASSWORD'
hostname = 'lemur.cloudamqp.com'
}
queues = {
testQueue autoDelete: false, durable: false, exclusive: false
}
}
When the application starts and tries to connect, I see the following log messages:
2013-08-23 21:29:59,195 [main] DEBUG listener.SimpleMessageListenerContainer - Starting Rabbit listener container.
2013-08-23 21:29:59,205 [SimpleAsyncTaskExecutor-1] DEBUG listener.BlockingQueueConsumer - Starting consumer Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
2013-08-23 21:30:08,405 [SimpleAsyncTaskExecutor-1] WARN listener.SimpleMessageListenerContainer - Consumer raised exception, processing can restart if the connection factory supports it
org.springframework.amqp.AmqpIOException: java.io.IOException
at org.springframework.amqp.rabbit.connection.RabbitUtils.convertRabbitAccessException(RabbitUtils.java:112)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:163)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:228)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils$1.createConnection(ConnectionFactoryUtils.java:119)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.doGetTransactionalResourceHolder(ConnectionFactoryUtils.java:163)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.getTransactionalResourceHolder(ConnectionFactoryUtils.java:109)
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:199)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:524)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:106)
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:102)
at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:124)
at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:381)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:516)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:545)
Caused by: com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.net.SocketException: Connection reset
at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:67)
at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:33)
at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:343)
at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:216)
at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:118)
... 3 more
Caused by: java.net.SocketException: Connection reset
at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95)
at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:508)
2013-08-23 21:30:08,406 [SimpleAsyncTaskExecutor-1] INFO listener.SimpleMessageListenerContainer - Restarting Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
2013-08-23 21:30:08,406 [SimpleAsyncTaskExecutor-1] DEBUG listener.BlockingQueueConsumer - Closing Rabbit Channel: null
2013-08-23 21:30:08,407 [SimpleAsyncTaskExecutor-2] DEBUG listener.BlockingQueueConsumer - Starting consumer Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
Aug 23, 2013 9:30:11 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring FrameworkServlet 'grails'
Aug 23, 2013 9:30:11 PM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-8634
Aug 23, 2013 9:30:11 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8634
According to https://developer.cloudbees.com/bin/view/RUN/CloudAMQP
when you bind your CloudAMQP service to your app - some config params are provided in the pattern of CLOUDAMQP_URL_ - this is the type of thing you would need to put in your config files so they can be wired in when the app launches.
Make sure to specify the virtualHost for CloudAMQP connections. That worked for me.