I've created a simple saga prototype project with RabbitMQ as the transport and RavenDB as the persistence mechanism. The prototype actually runs as expected, but every few seconds i get this error msg:
ERROR NServiceBus.Transports.RabbitMQ.RabbitMqDequeueStrategy Failed to receive messages from [Assembly].Retries
System.AggregateException: One or more errors occurred. --> System.IO.EndOfStreamException: SharedQueue closed
at RabbitMQ.Util.SharedQueue1.EnsureIsOpen()
at RabbitMQ.Util.SharedQueue1.Dequeue(int 32 milliseconds timeout.......
I also get an almost identical message immediately following the above one but it says it Failed to receive messages from RabbitMGPoller.Timeouts
In addition to that there are constant INFO messages that say:
NServiceBus.Transports.RabbitMQ.RabbitMqConnectionManager Disconnected from RabbitMQ broker, reason: AMQP close-reason, initiated Library, code=0 text="End of stream"... cause=System.IOException:Unable to write data to the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host...
I have tried adding a DequeueTimeout=600 value to the transport connection, but the same errors still occur. I've also tried adding the following key in the config file, but it still didn't seem to help.
I eventually figured it out, just my lack of understanding of RabbitMQ and NServiceBus. I changed the RequestedHeartbeat value for the RabbitMQ connection to something larger i.e. RequestedHeartbeat=6000. That solved my issue.
Related
We are getting below error when trying to send message to RabbitMQ. Our application is on .net core 3.0 on linux VM, and the RMQ server is running in a cluster of two on linux VMs. The issue is intermittent and not happening always.
Already closed: The AMQP operation was interrupted: AMQP close-reason, initiated by Library, code=541,
text='Unexpected Exception', classId=0, methodId=0, cause=System.Net.Sockets.SocketException (110):
Connection timed out
at RabbitMQ.Client.Impl.InboundFrame.ReadFrom(NetworkBinaryReader reader)
at RabbitMQ.Client.Framing.Impl.Connection.MainLoopIteration()
at RabbitMQ.Client.Framing.Impl.Connection.MainLoop()
We have enabled automatic recovery by setting AutomaticRecoveryEnabled to true, and other than that every setting is default one.
I am frequently getting the below mentioned errors, the dll version used in the project is - 1.0.488.0
System.TimeoutException: Timeout performing GET
StackExchange.Redis.RedisConnectionException: No connection is available to service this operation: GET
No connection is available to service this operation: EXISTS
Can anyone help me out in figuring what the issue can be?
Have also created an issue on StackExchange's Github repo for the same
Issue created on Github for the same
It looks like your connection broke. And when it did, any commands already sent to Redis would have timed out on the client application, even though they could have executed on the server. If you upgrade to a later version of the StackExchange.Redis client, you will get richer diagnostics information about what the state of the threadpool, CPU etc was on the client application side.
I am running a load test in order to see the performance of the WCF services during the peak time (heavy load). I am using the Step-Load where we push the virtual users Step-by-Step. When I start running the load test for the first few minutes the test runs smoothly and as the load increases by time, after some time all of a sudden the below error is triggering,
"Test method "XYZ" threw exception:
System.ServiceModel.CommunicationException: The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host".
I tried lot of solution that I found online but non of them worked for me. I tried changing the default time-outs, maxconnections, maxconcurrent connections etc., in config files. I would really appreciate any help on this.
I had similar problems while ago when moved to WCF. It have probably something to do with the way WCF handles connections.
Solution for me was to move each test into separate ApplicationDomain.
I have executed service broker within same instances with success. But recently I started exploring Distributed Service broker application between multiple instances without any success.
I have created two instances(both are Developer Editions) in two different systems within my domain, when I send a message from one instance to another, its not sending to receiving/target instance. When I check sys.transmission_queue, I get transmission_status as below:
An error occurred while receiving data:
'10054(An existing connection was forcibly closed by the remote host.)'.
Attach Profiler to the other instance (the one that is closing the connection 'forcibly') and monitor for the Audit Broker Login class. It will trace the reason why it refuses the connection (most likely CONNECT permission is missing on the endpoint).
Also running ssbdiagnose should reveal the issue:
ssbdiagnose runtime connect to -S <machineA> connect to -S <machineB>
I'm making a call to a WCF service but I get a CommunicationException on the client while receiving the response from the service.
System.ServiceModel.CommunicationException: An error occurred while receiving the HTTP response to http://localhost:8080/Service. This could be due to the service endpoint binding not using the HTTP protocol. This could also be due to an HTTP request context being aborted by the server (possibly due to the service shutting down). See server logs for more details. ---> System.Net.WebException: The underlying connection was closed: An unexpected error occurred on a receive. ---> System.IO.IOException: Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host. ---> System.Net.Sockets.SocketException: An existing connection was forcibly closed by the remote host
The client that makes the call is a WCF proxy client.
The service method executes without
any exceptions.
The WCF call works fine in those cases where it does not
take a long time for the serivce
method to finish.
The WCF call fails with the above exception message when the service method is taking long time to finish.
The sendTimeout property of the client's binding has been increased to 30 minutes to accommodate the time it can take for the service method to finish.
Try to set the receiveTimeout equal or greater than the time it takes for the service method to complete. The default value for the receiveTimeout property is 10 minutes. So if the service method takes longer time to complete the connection will be closed (if no other activity takes place before the receiveTimeoutoccurs). The receiveTimeout property is described here.
A very long operation like this should most likely be called asynchronously - in other words, the client asks the server to prepare the data, then gets on with something else while the server does the work. When the server's finished it calls the client back.
Asynchronous WCF operations are discussed here.