I am using Kafka 0.8.2-beta and have 2 Ubuntu 14 virtual machines:
172.30.141.127 is running Zookeeper
172.30.141.184 is running a Kafka broker
I'm starting the Zookeeper instance and all if fine. Then I try to start the broker and connect it to 172.30.141.127:2181. It seems to be able to connect and establish a session on a specific port, but then it losses connection due to some exception that doesn't seem to be logged.
The broker output:
[2015-01-19 11:03:55,029] INFO Client environment:java.io.tmpdir=/tmp (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,030] INFO Client environment:java.compiler=<NA> (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,031] INFO Client environment:os.name=Linux (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,031] INFO Client environment:os.arch=i386 (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,032] INFO Client environment:os.version=3.16.0-23-generic (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,033] INFO Client environment:user.name=root (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,033] INFO Client environment:user.home=/root (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,037] INFO Client environment:user.dir=/home/osboxes/Desktop/kafka_2.11-0.8.2-beta (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,039] INFO Initiating client connection, connectString=172.30.141.127:2181 sessionTimeout=6000 watcher=org.I0Itec.zkclient.ZkClient#1ecf473 (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:55,129] INFO Opening socket connection to server 172.30.141.127/172.30.141.127:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:55,186] INFO Socket connection established to 172.30.141.127/172.30.141.127:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:55,203] WARN Session 0x0 for server 172.30.141.127/172.30.141.127:2181, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[2015-01-19 11:03:56,552] INFO Opening socket connection to server 172.30.141.127/172.30.141.127:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:56,555] INFO Socket connection established to 172.30.141.127/172.30.141.127:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:56,567] WARN Session 0x0 for server 172.30.141.127/172.30.141.127:2181, unexpected error, closing socket connection and attempting reconnect (org.apache.zookeeper.ClientCnxn)
java.io.IOException: Connection reset by peer
at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:223)
at sun.nio.ch.IOUtil.read(IOUtil.java:192)
at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:379)
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:68)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:366)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1081)
[2015-01-19 11:03:57,131] INFO Terminate ZkClient event thread. (org.I0Itec.zkclient.ZkEventThread)
[2015-01-19 11:03:58,075] INFO Opening socket connection to server 172.30.141.127/172.30.141.127:2181. Will not attempt to authenticate using SASL (unknown error) (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:58,077] INFO Socket connection established to 172.30.141.127/172.30.141.127:2181, initiating session (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:58,195] INFO Session: 0x0 closed (org.apache.zookeeper.ZooKeeper)
[2015-01-19 11:03:58,196] INFO EventThread shut down (org.apache.zookeeper.ClientCnxn)
[2015-01-19 11:03:58,251] FATAL [Kafka Server 1], Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 2000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:157)
at kafka.server.KafkaServer.startup(KafkaServer.scala:83)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:28)
at kafka.Kafka$.main(Kafka.scala:46)
at kafka.Kafka.main(Kafka.scala)
[2015-01-19 11:03:58,279] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)
[2015-01-19 11:03:58,295] INFO [Kafka Server 1], shut down completed (kafka.server.KafkaServer)
[2015-01-19 11:03:58,308] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 2000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
at kafka.server.KafkaServer.initZk(KafkaServer.scala:157)
at kafka.server.KafkaServer.startup(KafkaServer.scala:83)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:28)
at kafka.Kafka$.main(Kafka.scala:46)
at kafka.Kafka.main(Kafka.scala)
[2015-01-19 11:03:58,335] INFO [Kafka Server 1], shutting down (kafka.server.KafkaServer)
Zookeper outputs:
[2015-01-19 11:03:55,245] INFO Accepted socket connection from /172.30.141.184:54089 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2015-01-19 11:03:55,315] WARN Exception causing close of session 0x0 due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:55,329] INFO Closed socket connection for client /172.30.141.184:54089 (no session established for client) (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:56,613] INFO Accepted socket connection from /172.30.141.184:54090 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2015-01-19 11:03:56,615] WARN Exception causing close of session 0x0 due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:56,617] INFO Closed socket connection for client /172.30.141.184:54090 (no session established for client) (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:58,133] INFO Accepted socket connection from /172.30.141.184:54091 (org.apache.zookeeper.server.NIOServerCnxnFactory)
[2015-01-19 11:03:58,134] WARN Exception causing close of session 0x0 due to java.io.IOException: Connection reset by peer (org.apache.zookeeper.server.NIOServerCnxn)
[2015-01-19 11:03:58,135] INFO Closed socket connection for client /172.30.141.184:54091 (no session established for client) (org.apache.zookeeper.server.NIOServerCnxn)
Ping works between the 2 machines. Telnet to 2181 kind of works, in that it connects, but gets disconnected from time to time. This leads me to think that the problem is with the Zookeeper instance. Both processes are started as root.
Any ideas why this is happening? Thanks
You are probably hitting the max number of connections per host.
This happens if you have [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn$Factory#247] - Too many connections from /127.0.0.1 - max is 10 in your zookeeper logs.
Fix it by setting maxClientCnxns=[something more than 10; 0 for unlimited] in your conf/zoo.cfg.
Docs (search for maxClientCnxns)
No idea about the warning, but I had the same Error problem
org.I0Itec.zkclient.exception.ZkTimeoutException: Unable to connect to zookeeper server within timeout: 2000
at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:880)
....
In my case when I was setting up producer and consumer, I provided wrong ip/port. When I changed it to correct one with:
bin/kafka-console-producer.sh --broker-list kafka_ip:kafka_port --topic test
bin/kafka-console-consumer.sh --zookeeper zookeeper_id:zookeper_port --topic test --from-beginning
my problem was solved.
As answered by #RickyA
"You are probably hitting the max number of connections per host."
Try clearing your logs folder which in my case was F:\tmp\kafka-logs.
Related
I'm using the RabbitMQ 3.6.12 version with java 8 client. I'm getting a frequent exception in the RabbitMQ log and my client is getting exited to consume the message.
logs on the RabbitMQ:
=WARNING REPORT==== 29-DEC-2018::16:02:38 === closing AMQP connection <0.8979.3> (127.0.0.1:57065 -> 127.0.0.1:5672, vhost: '/', user:
'guest'): client unexpectedly closed TCP connection
logs on the Java Client:
[com.rabbitmq.client.impl.ForgivingExceptionHandler] (AMQP Connection 127.0.0.1:5672)
An unexpected connection driver error occured: java.net.SocketException: Unrecognized Windows Sockets error: 0: recv failed
at java.net.SocketInputStream.socketRead0(Native Method)
at java.net.SocketInputStream.socketRead(SocketInputStream.java:116)
at java.net.SocketInputStream.read(SocketInputStream.java:171)
at java.net.SocketInputStream.read(SocketInputStream.java:141)
at java.io.BufferedInputStream.fill(BufferedInputStream.java:246)
at java.io.BufferedInputStream.read(BufferedInputStream.java:265)
at java.io.DataInputStream.readUnsignedByte(DataInputStream.java:288)
at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:91)
at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:164)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:580)
at java.lang.Thread.run(Thread.java:748)"
I'm able connect IBM Queue directly but when u tried to connect from mule getting the below error and not able to deploy. I'm getting the below error
ERROR 2017-04-25 06:45:13,582
[main]org.mule.retry.notifiers.ConnectNotifier: Failed to connect/reconnect:
WebSphereMQConnector
{
name=WMQ2
lifecycle=initialise
this=5e7abaf7
numberOfConcurrentTransactedReceivers=4
createMultipleTransactedReceivers=true
connected=false
supportedProtocols=[wmq]
serviceOverrides=<none>
}
. Root Exception was: Connection timed out: connect. Type: class java.net.ConnectException
ERROR 2017-04-25 06:50:23,943 [main] org.mule.module.launcher.application.DefaultMuleApplication:
************************************************
Message : JMSWMQ0018: Failed to connect to queue manager 'RQACBRKB' with connection mode 'Client' and host name '172.11.11.11(6912)'.
JMS Code : JMSWMQ0018
Element : /WMQ2 # app:config.xml:14 (WMQ)
--------------------------------------------------------------------------------
Root Exception stack trace:
java.net.ConnectException: Connection timed out: connect
at java.net.TwoStacksPlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(Unknown Source)
at java.net.AbstractPlainSocketImpl.connectToAddress(Unknown Source)
at java.net.AbstractPlainSocketImpl.connect(Unknown Source)
at java.net.PlainSocketImpl.connect(Unknown Source)
at java.net.SocksSocketImpl.connect(Unknown Source)
com.ibm.mq.jmqi.JmqiException: CC=2;RC=2538;AMQ9213: A communications error for occurred [1=java.net.ConnectException[Connection timed out: connect],3=rbitbrka.apl.com] at com.ibm.mq.jmqi.remote.impl.RemoteTCPConnection.connnectUsingLocalAddress(RemoteTCPConnection.java:810) ~[?:?]
PFB connector details:
<wmq:connector name="WMQ5" hostName="${mq.host}" port="${mq.port}" queueManager="${mq.queue.manager}" channel="CLIENTS.SALES.CRM" username="${mq.user}" password="${mq.password}" transportType="CLIENT_MQ_TCPIP" specification="1.1" targetClient="JMS_COMPLIANT" validateConnections="false" doc:name="WMQ" maxRedelivery="-1">
<reconnect frequency="${mq.reconnection.period.ms}" count="${mq.reconnection.attempt}"/>
</wmq:connector>
When i telnet the ip and port getting below error:
C:\Users\111>telnet 172.11.11.11 6912
Connecting To 172.11.11.11...Could not open connection to the host, on port 6912: Connect failed
But when i ping getting responce
C:\Users\111>ping 172.11.11.11
The pertinent pieces of information from your provided error are:-
JMSWMQ0018: Failed to connect to queue manager 'RQACBRKB'
with connection mode 'Client' and host name '172.11.11.11(6912)'.
com.ibm.mq.jmqi.JmqiException: CC=2;RC=2538;
MQRC 2538 is MQRC_HOST_NOT_AVAILABLE which is explained in Knowledge Center. In there it mentions the most common reasons for this error:-
The listener has not been started on the remote system. (please check that your listener is running on port 6912 on the machine at IP address 172.11.11.11)
The connection name in the client channel definition is incorrect. (the connection name your client is using is '172.11.11.11(6912)' - is this correct?)
The network is currently unavailable.
A firewall blocking the port, or protocol-specific traffic.
The security call initializing the IBM MQ client is blocked by a security exit on the SVRCONN channel at the server.
"java.net.ConnectException: Connection timed out: connect" usually occurs when you have a config issue or you are unable to connect to the remote server. As mentioned above do you have an error on the MQ end and if not have you checked the connection properties within config. If these are correct, are you able to reach MQ from another client, such as SOAPUI?
Can you also post the connector and flow details?
Hadoop is successfully running in distributed mode.
Getting following error while starting HBase in distributed mode.
Tried everything in hbase-site.xml configuration. No idea how to proceed with the problem?
014-03-10 13:55:42,493 INFO org.apache.zookeeper.ClientCnxn: Opening socket connection to server ip-112-11-1-111.ec2.internal/112.11.1.111:2181.
Will not attempt to authenticate using SASL (Unable to locate a login configuration)
2014-03-10 13:55:42,494 WARN org.apache.zookeeper.ClientCnxn: Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:599)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1068)
2014-03-10 13:55:42,594 WARN org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: Possibly transient ZooKeeper exception: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
2014-03-10 13:55:42,594 ERROR org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper: ZooKeeper exists failed after 3 retries
2014-03-10 13:55:42,595 ERROR org.apache.hadoop.hbase.master.HMasterCommandLine: Failed to start master
java.lang.RuntimeException: Failed construction of Master: class org.apache.hadoop.hbase.master.HMaster
at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2104)
at org.apache.hadoop.hbase.master.HMasterCommandLine.startMaster(HMasterCommandLine.java:152)
at org.apache.hadoop.hbase.master.HMasterCommandLine.run(HMasterCommandLine.java:104)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:76)
at org.apache.hadoop.hbase.master.HMaster.main(HMaster.java:2118)
Caused by: org.apache.zookeeper.KeeperException$ConnectionLossException: KeeperErrorCode = ConnectionLoss for /hbase
at org.apache.zookeeper.KeeperException.create(KeeperException.java:99)
at org.apache.zookeeper.KeeperException.create(KeeperException.java:51)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1041)
at org.apache.zookeeper.ZooKeeper.exists(ZooKeeper.java:1069)
at org.apache.hadoop.hbase.zookeeper.RecoverableZooKeeper.exists(RecoverableZooKeeper.java:199)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:1109)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:1099)
at org.apache.hadoop.hbase.zookeeper.ZKUtil.createAndFailSilent(ZKUtil.java:1083)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.createBaseZNodes(ZooKeeperWatcher.java:162)
at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:155)
at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:345)
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:39)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
at java.lang.reflect.Constructor.newInstance(Constructor.java:513)
at org.apache.hadoop.hbase.master.HMaster.constructMaster(HMaster.java:2099)
Make sure that ZooKeeper is provisioned and running expectedly.
Check zoo.cfg and /etc/hosts to make sure that all zookeeper servers are reachable by the HBase master.
I am not a networking guru so I am probably missing something simple.
I have built both the message echo client and msg echo server as runnable jar files using the netty 4.0.11 http://dl.bintray.com/netty/downloads/netty-4.0.11.Final.tar.bz2
I was able to load all of the correct dependencies using maven and the project builds and runs correctly locally and and on the server. I can run a server on my local host and connect to it from a client on the same (localhost , both on my local machine and on my amazon ec2 instance. Again it works connecting to itself (localhost) on both my machine, and my server computer.
THe problem is that I cannot connect to the server from outside the machine it is running, for example, I want to connect to my echo msg server (running on my ec2 instance) from the echo msg client running on my local computer.
I have setup the amazon security settings to allow UDP from the correct port and from the ip of my local machine. I run the Echo server on the ec2 instance and it correctly starts up:
REGISTERED
ACTIVE
DATAGRAM LISTENING bind=/0.0.0.0:1234 peer=null:0
But I just cannot connect from outside the local host, here is the error message I am getting from the client machine,
I have also turned off all firewalls (temporarily) on my local machine. Still, when i try to connect I get:
CONNECT(/70.36.197.242:1234, null)
10:11:17.000 [connect-0] DEBUG com.barchart.udt.EpollUDT - ep 1 rem [id: 0x3287e50e] DATAGRAM CONNECTING bind=/0.0.0.0:55005 peer=null:0
10:11:17.000 [connect-0] DEBUG com.barchart.udt.EpollUDT - ep 1 add [id: 0x3287e50e] DATAGRAM CONNECTING bind=/0.0.0.0:55005 peer=null:0 ERROR_WRITE
10/24/13 10:11:19 AM ===========================================================
udt.echo.message.MsgEchoClientHandler:
rate:
count = 0
mean rate = 0.00 bytes/s
1-minute rate = 0.00 bytes/s
5-minute rate = 0.00 bytes/s
15-minute rate = 0.00 bytes/s
10:11:20.017 [connect-0] WARN com.barchart.udt.nio.SelectionKeyUDT - logic error :
[id: 0x3287e50e] poll=ERROR_WRITE ready=---- inter=-C-- DATAGRAM CONNECTOR CONNECTING bind=/0.0.0.0:55005 peer=null:0
java.lang.Exception: Unexpected error report.
at com.barchart.udt.nio.SelectionKeyUDT.logError(SelectionKeyUDT.java:436) [barchart-udt-bundle-2.3.0.jar:na]
at com.barchart.udt.nio.SelectionKeyUDT.doRead(SelectionKeyUDT.java:205) [barchart-udt-bundle-2.3.0.jar:na]
at com.barchart.udt.nio.SelectorUDT.doResultsRead(SelectorUDT.java:334) [barchart-udt-bundle-2.3.0.jar:na]
at com.barchart.udt.nio.SelectorUDT.doResults(SelectorUDT.java:309) [barchart-udt-bundle-2.3.0.jar:na]
at com.barchart.udt.nio.SelectorUDT.doEpollExclusive(SelectorUDT.java:234) [barchart-udt-bundle-2.3.0.jar:na]
at com.barchart.udt.nio.SelectorUDT.doEpollEnter(SelectorUDT.java:196) [barchart-udt-bundle-2.3.0.jar:na]
at com.barchart.udt.nio.SelectorUDT.select(SelectorUDT.java:455) [barchart-udt-bundle-2.3.0.jar:na]
at io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:596) [netty-all-4.0.11.Final.jar:na]
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:306) [netty-all-4.0.11.Final.jar:na]
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101) [netty-all-4.0.11.Final.jar:na]
at java.lang.Thread.run(Unknown Source) [na:1.7.0_06]
10:11:20.018 [connect-0] DEBUG com.barchart.udt.EpollUDT - ep 1 rem [id: 0x3287e50e] DATAGRAM CONNECTING bind=/0.0.0.0:55005 peer=null:0
10:11:20.018 [connect-0] ERROR c.barchart.udt.nio.SocketChannelUDT - connect failure : [id: 0x3287e50e] DATAGRAM CONNECTING bind=/0.0.0.0:55005 peer=null:0
10:11:20.018 [connect-0] INFO i.n.handler.logging.LoggingHandler - [id: 0x53f0f817] CLOSE()
Exception in thread "main" java.io.IOException
at com.barchart.udt.nio.SocketChannelUDT.finishConnect(SocketChannelUDT.java:236)
at io.netty.channel.udt.nio.NioUdtMessageConnectorChannel.doFinishConnect(NioUdtMessageConnectorChannel.java:132)
at io.netty.channel.nio.AbstractNioChannel$AbstractNioUnsafe.finishConnect(AbstractNioChannel.java:228)
at io.netty.channel.nio.NioEventLoop.processSelectedKey(NioEventLoop.java:502)
at io.netty.channel.nio.NioEventLoop.processSelectedKeysPlain(NioEventLoop.java:417)
at io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:348)
at io.netty.util.concurrent.SingleThreadEventExecutor$2.run(SingleThreadEventExecutor.java:101)
at java.lang.Thread.run(Unknown Source)
Nevermind, after doing some more trouble shooting I found the solution. I had to in addition to allowing the port in the amazon security settings, also allow the UDP port on the server firewall.
I'm having difficulty configuring my connection to CloudAMQP in my deployed grails application. I can run the application locally against a locally installed RabbitMQ instance but can't figure out how to correctly define my application to run on CloudBees using the CloudAMQP service.
In my Config.groovy, I'm defining my connection info and a queue:
rabbitmq {
connectionfactory {
username = 'USERNAME'
password = 'PASSWORD'
hostname = 'lemur.cloudamqp.com'
}
queues = {
testQueue autoDelete: false, durable: false, exclusive: false
}
}
When the application starts and tries to connect, I see the following log messages:
2013-08-23 21:29:59,195 [main] DEBUG listener.SimpleMessageListenerContainer - Starting Rabbit listener container.
2013-08-23 21:29:59,205 [SimpleAsyncTaskExecutor-1] DEBUG listener.BlockingQueueConsumer - Starting consumer Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
2013-08-23 21:30:08,405 [SimpleAsyncTaskExecutor-1] WARN listener.SimpleMessageListenerContainer - Consumer raised exception, processing can restart if the connection factory supports it
org.springframework.amqp.AmqpIOException: java.io.IOException
at org.springframework.amqp.rabbit.connection.RabbitUtils.convertRabbitAccessException(RabbitUtils.java:112)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:163)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:228)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils$1.createConnection(ConnectionFactoryUtils.java:119)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.doGetTransactionalResourceHolder(ConnectionFactoryUtils.java:163)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.getTransactionalResourceHolder(ConnectionFactoryUtils.java:109)
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:199)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:524)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:106)
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:102)
at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:124)
at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:381)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:516)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:545)
Caused by: com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.net.SocketException: Connection reset
at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:67)
at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:33)
at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:343)
at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:216)
at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:118)
... 3 more
Caused by: java.net.SocketException: Connection reset
at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95)
at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:508)
2013-08-23 21:30:08,406 [SimpleAsyncTaskExecutor-1] INFO listener.SimpleMessageListenerContainer - Restarting Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
2013-08-23 21:30:08,406 [SimpleAsyncTaskExecutor-1] DEBUG listener.BlockingQueueConsumer - Closing Rabbit Channel: null
2013-08-23 21:30:08,407 [SimpleAsyncTaskExecutor-2] DEBUG listener.BlockingQueueConsumer - Starting consumer Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
Aug 23, 2013 9:30:11 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring FrameworkServlet 'grails'
Aug 23, 2013 9:30:11 PM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-8634
Aug 23, 2013 9:30:11 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8634
According to https://developer.cloudbees.com/bin/view/RUN/CloudAMQP
when you bind your CloudAMQP service to your app - some config params are provided in the pattern of CLOUDAMQP_URL_ - this is the type of thing you would need to put in your config files so they can be wired in when the app launches.
Make sure to specify the virtualHost for CloudAMQP connections. That worked for me.