SignalR server in console host not sending keep-alives - asp.net-core

I have an ASP.NET Core application using a SignalR hub. When running via a console application (development mode), no keep-alive requests are sent by the server to the client. Consequently, the connection is re-established every 30 seconds or so.
However, when running the same application via Service Fabric, keep-alive requests are sent and everything works as expected.
Here are the server logs when running under the console app:
dbug: Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionManager[1]
New connection T2NKQg0jyrm7QAPz4p0ZWA created.
dbug: Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionDispatcher[4]
Establishing new connection.
dbug: Microsoft.AspNetCore.SignalR.HubConnectionHandler[5]
OnConnectedAsync started.
dbug: Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport[1]
Socket opened using Sub-Protocol: '(null)'.
trce: Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport[9]
Message received. Type: Text, size: 32, EndOfMessage: True.
dbug: Microsoft.AspNetCore.SignalR.Internal.DefaultHubProtocolResolver[2]
Found protocol implementation for requested protocol: json.
dbug: Microsoft.AspNetCore.SignalR.HubConnectionContext[1]
Completed connection handshake. Using HubProtocol 'json'.
trce: Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport[11]
Sending payload: 3 bytes.
trce: Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport[9]
Message received. Type: Text, size: 11, EndOfMessage: True.
trce: Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport[9]
Message received. Type: Text, size: 11, EndOfMessage: True.
dbug: Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport[4]
Waiting for the application to finish sending data.
dbug: Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport[2]
Socket closed.
trce: Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext[1]
Disposing connection T2NKQg0jyrm7QAPz4p0ZWA.
trce: Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext[2]
Waiting for application to complete.
dbug: Microsoft.AspNetCore.SignalR.HubConnectionHandler[6]
OnConnectedAsync ending.
trce: Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext[3]
Application complete.
dbug: Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionManager[2]
Removing connection T2NKQg0jyrm7QAPz4p0ZWA from the list of connections.
dbug: Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionManager[1]
New connection JW_1AnoGvhvNJ6MdWGb5RA created.
dbug: Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionDispatcher[4]
Establishing new connection.
dbug: Microsoft.AspNetCore.SignalR.HubConnectionHandler[5]
OnConnectedAsync started.
dbug: Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport[1]
Socket opened using Sub-Protocol: '(null)'.
trce: Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport[9]
Message received. Type: Text, size: 32, EndOfMessage: True.
dbug: Microsoft.AspNetCore.SignalR.Internal.DefaultHubProtocolResolver[2]
Found protocol implementation for requested protocol: json.
dbug: Microsoft.AspNetCore.SignalR.HubConnectionContext[1]
Completed connection handshake. Using HubProtocol 'json'.
trce: Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport[11]
Sending payload: 3 bytes.
trce: Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport[9]
Message received. Type: Text, size: 11, EndOfMessage: True.
And the client logs:
Microsoft.AspNetCore.Http.Connections.Client.HttpConnection: Debug: Transport 'WebSockets' started.
Microsoft.AspNetCore.Http.Connections.Client.HttpConnection: Information: HttpConnection Started.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Information: Using HubProtocol 'json v1'.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Debug: Sending Hub Handshake.
Microsoft.AspNetCore.Http.Connections.Client.Internal.WebSocketsTransport: Debug: Received message from application. Payload size: 32.
Microsoft.AspNetCore.Http.Connections.Client.Internal.WebSocketsTransport: Debug: Message received. Type: Text, size: 3, EndOfMessage: True.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Debug: Handshake with server complete.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Debug: Receive loop starting.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Debug: Sending PingMessage message.
Microsoft.AspNetCore.Http.Connections.Client.Internal.WebSocketsTransport: Debug: Received message from application. Payload size: 11.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Debug: Sending PingMessage message completed.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Information: HubConnection started.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Trace: The HubConnection is attempting to transition from the Connecting state to the Connected state.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Trace: Releasing Connection Lock in StartAsyncInner (/_/src/SignalR/clients/csharp/Client.Core/src/HubConnection.cs:280).
The thread 0x8184 has exited with code 0 (0x0).
Microsoft.AspNetCore.SignalR.Client.HubConnection: Trace: Acquired the Connection Lock in order to ping the server.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Debug: Sending PingMessage message.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Debug: Sending PingMessage message completed.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Trace: Releasing Connection Lock in RunTimerActions (/_/src/SignalR/clients/csharp/Client.Core/src/HubConnection.cs:1881).
Microsoft.AspNetCore.Http.Connections.Client.Internal.WebSocketsTransport: Debug: Received message from application. Payload size: 11.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Trace: Waiting on Connection Lock in HandleConnectionClose (/_/src/SignalR/clients/csharp/Client.Core/src/HubConnection.cs:1279).
Microsoft.AspNetCore.Http.Connections.Client.HttpConnection: Debug: Disposing HttpConnection.
Microsoft.AspNetCore.Http.Connections.Client.Internal.WebSocketsTransport: Information: Transport is stopping.
Microsoft.AspNetCore.Http.Connections.Client.Internal.WebSocketsTransport: Debug: Send loop stopped.
Microsoft.AspNetCore.Http.Connections.Client.Internal.WebSocketsTransport: Debug: Transport stopped.
Microsoft.AspNetCore.Http.Connections.Client.HttpConnection: Information: HttpConnection Disposed.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Debug: Canceling all outstanding invocations.
Microsoft.AspNetCore.Http.Connections.Client.Internal.WebSocketsTransport: Debug: Receive loop canceled.
Microsoft.AspNetCore.Http.Connections.Client.Internal.WebSocketsTransport: Debug: Receive loop stopped.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Trace: The HubConnection is attempting to transition from the Connected state to the Reconnecting state.
Microsoft.AspNetCore.SignalR.Client.HubConnection: Error: HubConnection reconnecting due to an error.
I won't include them here, but the logs when running under Service Fabric show that the server is correctly sending keep-alives to the client ("Sent a ping message to the client").
It might seem obvious that there is some difference in configuration between my console and Service Fabric hosts, but I've gone through it carefully and cannot see anything that would explain this. In fact, the SignalR integration differed only in that the development host configured detailed errors to be enabled, but even if I remove that the behavior remains the same.
Short of running my own build of ASP.NET Core (something I'm perhaps lazily attempting to avoid only because it was looking far from trivial to build), is there anything I might be missing that would explain this situation?

Related

I have a RabbitMQ container that has regularly "unexpected_frame" exceptions, what does that mean?

I have an application that is pushing data into RabbitMQ and then some other apps are subscribing to the different exchanges.
But recently, I keep having errors like this after a few hours:
2020-07-09 12:45:12.670 [error] <0.23578.1> Error on AMQP connection <0.23578.1> (172.18.0.5:48230 ->
172.18.0.3:5672, vhost: '/', user: 'guest', state: running), channel 6:
operation basic.publish caused a connection exception unexpected_frame:
"expected content header for class 60, got non content"
2020-07-09 12:45:12.674 [info] <0.23578.1> closing AMQP connection <0.23578.1> (172.18.0.5:48230 ->
172.18.0.3:5672, vhost: '/'
On the client side, I get messages like this:
"Already closed: The AMQP operation was interrupted: AMQP close-reason, initiated by Peer,
code=505, text='UNEXPECTED_FRAME - expected content body, got non content body frame instead',
classId=60, methodId=40"
This is on a docker container.
What could this error be about?
You are sharing a channel for concurrent publishing, use below code
lock (ch) { ch.BasicPublish(); }

Asp Net Core 3 Sample with 503 Service Unavailable

When creating a new ASP.NET Core Web Application using react and .Net Core 3.0 or 3.1 in visual studio and run the application the page shows:
503 Service Unavailable
Failed to connect to server localhost
I am using windows 10 professional and visual studio 2019.
The error happens running in IISExpress and kestrel.
On a different machine this works fine.
Creating a template using "Web Application (Model-View-Controller)" works.
The api call works (https://localhost:5001/WeatherForecast).
I'm troubleshooting a migration in a project from AspNet Core 2.2 to 3.0 with the same error, I checked to see if the template project would work and found out that the error was not specific to the migration. Figuring out why the template is not working will probably solve the error with the migration.
Edit:
Disabling firewall did not solve issue.
Here is the debug log for trying to open the page:
dbug: Microsoft.AspNetCore.Server.Kestrel[39]
Connection id "0HLTKHV201G3Q" accepted.
dbug: Microsoft.AspNetCore.Server.Kestrel[1]
Connection id "0HLTKHV201G3Q" started.
dbug: Microsoft.AspNetCore.Server.Kestrel[39]
Connection id "0HLTKHV201G3R" accepted.
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://localhost:5000/
dbug: Microsoft.AspNetCore.Server.Kestrel[1]
Connection id "0HLTKHV201G3R" started.
trce: Microsoft.AspNetCore.HostFiltering.HostFilteringMiddleware[2]
All hosts are allowed.
dbug: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[4]
The request path / does not match a supported file type
dbug: Microsoft.AspNetCore.Routing.Matching.DfaMatcher[1000]
No candidates found for the request path '/'
dbug: Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware[2]
Request did not match any endpoints
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 27.156ms 503 text/html; charset=UTF-8
dbug: Microsoft.AspNetCore.Server.Kestrel[10]
Connection id "0HLTKHV201G3Q" disconnecting.
dbug: Microsoft.AspNetCore.Server.Kestrel[2]
Connection id "0HLTKHV201G3Q" stopped.
dbug: Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets[6]
Connection id "0HLTKHV201G3Q" received FIN.
dbug: Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets[7]
Connection id "0HLTKHV201G3Q" sending FIN because: "The Socket transport's send loop completed gracefully."
info: Microsoft.AspNetCore.Hosting.Diagnostics[1]
Request starting HTTP/1.1 GET http://localhost:5000/favicon.ico
trce: Microsoft.AspNetCore.HostFiltering.HostFilteringMiddleware[2]
All hosts are allowed.
dbug: Microsoft.AspNetCore.StaticFiles.StaticFileMiddleware[5]
The request path /favicon.ico does not match an existing file
dbug: Microsoft.AspNetCore.Routing.Matching.DfaMatcher[1000]
No candidates found for the request path '/favicon.ico'
dbug: Microsoft.AspNetCore.Routing.EndpointRoutingMiddleware[2]
Request did not match any endpoints
info: Microsoft.AspNetCore.Hosting.Diagnostics[2]
Request finished in 19.8075ms 503 text/html; charset=UTF-8
dbug: Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets[6]
Connection id "0HLTKHV201G3R" received FIN.
dbug: Microsoft.AspNetCore.Server.Kestrel[10]
Connection id "0HLTKHV201G3R" disconnecting.
dbug: Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets[7]
Connection id "0HLTKHV201G3R" sending FIN because: "The client closed the connection."
dbug: Microsoft.AspNetCore.Server.Kestrel[2]
Connection id "0HLTKHV201G3R" stopped.
This was happening because proxy was blocking requests from .Net Host to react and angular.
In AspNetCore 3.0 HttpClient now has a default proxy created from system variables HTTPS_PROXY, HTTP_PROXY and NO_PROXY.
Adding NO_PROXY=localhost to system variables solved my problem.

Kestrel HTTPS redirect results in no response with debug log "the application completed without reading the entire request body."

I'm just running the API as a console app. If I have
app.UseHsts();
app.UseHttpsRedirection();
in my code, when I try to authenticate using Postman I get no response. In Postman I don't have the SSL certificate verification turned on.
The Kestrel output:
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[1]
Request starting HTTP/1.1 POST http://localhost:5000/connect/token applica
tion/x-www-form-urlencoded 74
dbug: Microsoft.AspNetCore.HttpsPolicy.HttpsRedirectionMiddleware[1]
Redirecting to 'https://localhost/connect/token'.
dbug: Microsoft.AspNetCore.Server.Kestrel[9]
Connection id "0HLJ4UL9EORCS" completed keep alive response.
info: Microsoft.AspNetCore.Hosting.Internal.WebHost[2]
Request finished in 0.1439ms 307
dbug: Microsoft.AspNetCore.Server.Kestrel[25]
Connection id "0HLJ4UL9EORCS", Request id "0HLJ4UL9EORCS:00000001": starte
d reading request body.
dbug: Microsoft.AspNetCore.Server.Kestrel[26]
Connection id "0HLJ4UL9EORCS", Request id "0HLJ4UL9EORCS:00000001": done r
eading request body.
info: Microsoft.AspNetCore.Server.Kestrel[32]
Connection id "0HLJ4UL9EORCS", Request id "0HLJ4UL9EORCS:00000001": the ap
plication completed without reading the entire request body.
dbug: Microsoft.AspNetCore.Server.Kestrel[1]
Connection id "0HLJ4UL9EORCT" started.
dbug: Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets[6]
Connection id "0HLJ4UL9EORCT" received FIN.
dbug: Microsoft.AspNetCore.Server.Kestrel[10]
Connection id "0HLJ4UL9EORCT" disconnecting.
dbug: Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets[7]
Connection id "0HLJ4UL9EORCT" sending FIN.
dbug: Microsoft.AspNetCore.Server.Kestrel[2]
Connection id "0HLJ4UL9EORCT" stopped.
What does it mean that it doesn't read the entire request body?
If I run my app without HTTP redirection, it works fine. What is going on, how can I fix this?

Akka.Remote - cannot send messages to remote actor after dissassociation

I am using Akka.Remote to communicate between a server-side service application and multiple desktop client applications. The clients send a request message to the server (using Akka.net) and waits for the server to reply with a response message. The client applications are transient, meaning that they often connect to the server, stay connected for some time, disconnect and then reconnect again.
The problem I encountered is that sometimes when a client disconnects from the server actor (by shutting down its ActorSystem) and then reconnects back to the server, it does not receive any replies from the server for some time. After a few minutes the communication works without any problems. I found out that this issue occurs when the server sends a reply to a client that has disconnected during the request and is no longer reachable. The server cannot deliver the response message and it somehow marks the client endpoint as invalid.
In the log (on the server side) I am getting the following messages when the client is disconnected.
[DEBUG] 2016-01-21 13:04:58.6151 received AutoReceiveMessage <Terminated>: [akka.tcp://qb#client:8090/user/qb] - ExistenceConfirmed=True ServerActor
[DEBUG] 2016-01-21 13:04:58.6550 Stopped Akka.Remote.Transport.ProtocolStateActor
[ INFO] 2016-01-21 13:04:58.6550 Quarantined address [akka.tcp://qb#client:8090] is still unreachable or has not been restarted. Keeping it quarantined. Akka.Event.DummyClassForStringSources
[DEBUG] 2016-01-21 13:04:58.6725 Stopped Akka.Remote.ReliableDeliverySupervisor
[DEBUG] 2016-01-21 13:04:58.6725 no longer watched by [akka://myservice/system/endpointManager/reliableEndpointWriter-akka.tcp%3a%2f%2fqb%40client%3a8090-2] Akka.Remote.EndpointWriter
[DEBUG] 2016-01-21 13:04:58.6725 Disassociated [akka.tcp://myservice#server:8081] <- akka.tcp://qb#client:8090 Akka.Remote.EndpointWriter
[DEBUG] 2016-01-21 13:04:58.6725 Stopped Akka.Remote.EndpointWriter
And then when the client attempts to reconnect, I get:
[DEBUG] 2016-01-21 13:05:15.5883 ConnectResponse [akka.tcp://qb#client:8090/user/qb] ServerActor
[DEBUG] 2016-01-21 13:05:16.0467 Started (Akka.Remote.Transport.ProtocolStateActor) Akka.Remote.Transport.ProtocolStateActor
[DEBUG] 2016-01-21 13:05:16.0467 Stopped Akka.Remote.Transport.ProtocolStateActor
[ WARN] 2016-01-21 13:05:16.0467 AssociationError [akka.tcp://myservice#server:8081] -> akka.tcp://qb#client:8090: Error [Invalid address: akka.tcp://qb#client:8090] [] Akka.Remote.EndpointWriter
[ INFO] 2016-01-21 13:05:16.0467 Quarantined address [akka.tcp://qb#client:8090] is still unreachable or has not been restarted. Keeping it quarantined. Akka.Event.DummyClassForStringSources
[DEBUG] 2016-01-21 13:05:16.0643 Stopped Akka.Remote.ReliableDeliverySupervisor
[DEBUG] 2016-01-21 13:05:16.0711 no longer watched by [akka://myservice/system/endpointManager/reliableEndpointWriter-akka.tcp%3a%2f%2fqb%40client%3a8090-4] Akka.Remote.EndpointWriter
[DEBUG] 2016-01-21 13:05:16.0711 Disassociated [akka.tcp://myservice#server:8081] -> akka.tcp://qb#client:8090 Akka.Remote.EndpointWriter
[DEBUG] 2016-01-21 13:05:16.0711 Stopped Akka.Remote.EndpointWriter
[DEBUG] 2016-01-21 13:05:16.0867 received AutoReceiveMessage <Terminated>: [akka://myservice/system/endpointManager/reliableEndpointWriter-akka.tcp%3a%2f%2fqb%40client%3a8090-4] - ExistenceConfirmed=True Akka.Remote.EndpointManager
[DEBUG] 2016-01-21 13:05:16.0867 Terminated [akka.tcp://qb#client:8090/user/qb] ServerActor
I suspect that this behavior is a feature of Akka.net, however, I need to implement my system so that clients can disconnect and then reconnect back to the server without the need to wait. Is there any way to disable the quarantine mechanism or to gracefully close the client endpoint on the server so that the client endpoint doesn't get quarantined?
[ INFO] 2016-01-21 13:04:58.6550 Quarantined address [akka.tcp://qb#client:8090] is still unreachable or has not been restarted. Keeping it quarantined. - that says it all. The node was quarantined which requires a restart of the actor system.
However, IMHO - just upgrade to Akka.NET 1.0.6, which we released on Monday. We made the remoting policy manager much less brittle than it has been historically.

Configuration of CloudAMQP Connection

I'm having difficulty configuring my connection to CloudAMQP in my deployed grails application. I can run the application locally against a locally installed RabbitMQ instance but can't figure out how to correctly define my application to run on CloudBees using the CloudAMQP service.
In my Config.groovy, I'm defining my connection info and a queue:
rabbitmq {
connectionfactory {
username = 'USERNAME'
password = 'PASSWORD'
hostname = 'lemur.cloudamqp.com'
}
queues = {
testQueue autoDelete: false, durable: false, exclusive: false
}
}
When the application starts and tries to connect, I see the following log messages:
2013-08-23 21:29:59,195 [main] DEBUG listener.SimpleMessageListenerContainer - Starting Rabbit listener container.
2013-08-23 21:29:59,205 [SimpleAsyncTaskExecutor-1] DEBUG listener.BlockingQueueConsumer - Starting consumer Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
2013-08-23 21:30:08,405 [SimpleAsyncTaskExecutor-1] WARN listener.SimpleMessageListenerContainer - Consumer raised exception, processing can restart if the connection factory supports it
org.springframework.amqp.AmqpIOException: java.io.IOException
at org.springframework.amqp.rabbit.connection.RabbitUtils.convertRabbitAccessException(RabbitUtils.java:112)
at org.springframework.amqp.rabbit.connection.AbstractConnectionFactory.createBareConnection(AbstractConnectionFactory.java:163)
at org.springframework.amqp.rabbit.connection.CachingConnectionFactory.createConnection(CachingConnectionFactory.java:228)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils$1.createConnection(ConnectionFactoryUtils.java:119)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.doGetTransactionalResourceHolder(ConnectionFactoryUtils.java:163)
at org.springframework.amqp.rabbit.connection.ConnectionFactoryUtils.getTransactionalResourceHolder(ConnectionFactoryUtils.java:109)
at org.springframework.amqp.rabbit.listener.BlockingQueueConsumer.start(BlockingQueueConsumer.java:199)
at org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer$AsyncMessageProcessingConsumer.run(SimpleMessageListenerContainer.java:524)
at java.lang.Thread.run(Unknown Source)
Caused by: java.io.IOException
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:106)
at com.rabbitmq.client.impl.AMQChannel.wrap(AMQChannel.java:102)
at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:124)
at com.rabbitmq.client.impl.AMQConnection.start(AMQConnection.java:381)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:516)
at com.rabbitmq.client.ConnectionFactory.newConnection(ConnectionFactory.java:545)
Caused by: com.rabbitmq.client.ShutdownSignalException: connection error; reason: java.net.SocketException: Connection reset
at com.rabbitmq.utility.ValueOrException.getValue(ValueOrException.java:67)
at com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(BlockingValueOrException.java:33)
at com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(AMQChannel.java:343)
at com.rabbitmq.client.impl.AMQChannel.privateRpc(AMQChannel.java:216)
at com.rabbitmq.client.impl.AMQChannel.exnWrappingRpc(AMQChannel.java:118)
... 3 more
Caused by: java.net.SocketException: Connection reset
at com.rabbitmq.client.impl.Frame.readFrom(Frame.java:95)
at com.rabbitmq.client.impl.SocketFrameHandler.readFrame(SocketFrameHandler.java:131)
at com.rabbitmq.client.impl.AMQConnection$MainLoop.run(AMQConnection.java:508)
2013-08-23 21:30:08,406 [SimpleAsyncTaskExecutor-1] INFO listener.SimpleMessageListenerContainer - Restarting Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
2013-08-23 21:30:08,406 [SimpleAsyncTaskExecutor-1] DEBUG listener.BlockingQueueConsumer - Closing Rabbit Channel: null
2013-08-23 21:30:08,407 [SimpleAsyncTaskExecutor-2] DEBUG listener.BlockingQueueConsumer - Starting consumer Consumer: tag=[null], channel=null, acknowledgeMode=AUTO local queue size=0
Aug 23, 2013 9:30:11 PM org.apache.catalina.core.ApplicationContext log
INFO: Initializing Spring FrameworkServlet 'grails'
Aug 23, 2013 9:30:11 PM org.apache.coyote.http11.Http11Protocol init
INFO: Initializing Coyote HTTP/1.1 on http-8634
Aug 23, 2013 9:30:11 PM org.apache.coyote.http11.Http11Protocol start
INFO: Starting Coyote HTTP/1.1 on http-8634
According to https://developer.cloudbees.com/bin/view/RUN/CloudAMQP
when you bind your CloudAMQP service to your app - some config params are provided in the pattern of CLOUDAMQP_URL_ - this is the type of thing you would need to put in your config files so they can be wired in when the app launches.
Make sure to specify the virtualHost for CloudAMQP connections. That worked for me.