Can I get help understanding this error message and where I should look for the root cause
StackExchange.Redis.RedisTimeoutException: Timeout performing GET (5000ms), next: GET ReviewStatSession|cuuchrwt1pattdmytjfveu55, inst: 3, qu: 0, qs: 218, aw: False, rs: TryParseResult, ws: Idle, in: 65536, in-pipe: 90112, out-pipe: 0, serverEndpoint:
redis-17006.predc30.production.int:17006, mc: 1/1/0, mgr: 9 of 10 available, clientName: PWEB184NTV, IOCP: (Busy=1,Free=999,Min=8,Max=1000), WORKER: (Busy=52,Free=32715,Min=8,Max=32767), v: 2.1.30.38891 (Please take a look at this article for some common client-side issues that can cause timeouts: https://stackexchange.github.io/StackExchange.Redis/Timeouts) at StackExchange.Redis.ConnectionMultiplexer.ExecuteSyncImpl[T](Message message, ResultProcessor1 processor, ServerEndPoint server) at StackExchange.Redis.RedisBase.ExecuteSync[T](Message message, ResultProcessor1 processor, ServerEndPoint server) at StackExchange.Redis.RedisDatabase.StringGet(RedisKey key, CommandFlags flags) at ReviewStat.RedisPageStatePersister.Load() at System.Web.UI.Page.LoadPageStateFromPersistenceMedium() at System.Web.UI.Page.LoadAllState() at System.Web.UI.Page.ProcessRequestMain(Boolean includeStagesBeforeAsyncPoint, Boolean includeStagesAfterAsyncPoint)
at at StackExchange.Redis.ConnectionMultiplexer.T ExecuteSyncImpl[T](StackExchange.Redis.Message, StackExchange.Redis.ResultProcessor1[T], StackExchange.Redis.ServerEndPoint)() at StackExchange.Redis.RedisBase.T ExecuteSync[T](StackExchange.Redis.Message, StackExchange.Redis.ResultProcessor1[T], StackExchange.Redis.ServerEndPoint)()
at StackExchange.Redis.RedisDatabase.StackExchange.Redis.RedisValue StringGet(StackExchange.Redis.RedisKey, StackExchange.Redis.CommandFlags)()
at ReviewStat.RedisPageStatePersister.Void Load()()
at System.Web.UI.Page.System.Object LoadPageStateFromPersistenceMedium()()
Related
Our application uses RabbitMQ .NET client to publish and consume messages and during loading testing we are getting the following issue in one of our application where the connection to RabbitMQ gets dropped. The following is the stacktrace of the application,
Message: Already closed: The AMQP operation was interrupted: AMQP
close-reason, initiated by Library, code=541, text='Unexpected
Exception', classId=0, methodId=0, cause=System.IO.IOException: Unable
to read data from the transport connection: An existing connection was
forcibly closed by the remote host.. --->
System.Net.Sockets.SocketException (10054): An existing connection was
forcibly closed by the remote host.
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
--- End of inner exception stack trace ---
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.BufferedStream.ReadByteSlow()
at RabbitMQ.Client.Impl.InboundFrame.ReadFrom(Stream reader, Byte[] frameHeaderBuffer)
at RabbitMQ.Client.Framing.Impl.Connection.MainLoopIteration()
at RabbitMQ.Client.Framing.Impl.Connection.MainLoop() Type: RabbitMQ.Client.Exceptions.AlreadyClosedException ShutdownReason:
AMQP close-reason, initiated by Library, code=541, text='Unexpected
Exception', classId=0, methodId=0, cause=System.IO.IOException: Unable
to read data from the transport connection: An existing connection was
forcibly closed by the remote host.. --->
System.Net.Sockets.SocketException (10054): An existing connection was
forcibly closed by the remote host.
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
--- End of inner exception stack trace ---
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.IO.BufferedStream.ReadByteSlow()
at RabbitMQ.Client.Impl.InboundFrame.ReadFrom(Stream reader, Byte[] frameHeaderBuffer)
at RabbitMQ.Client.Framing.Impl.Connection.MainLoopIteration()
at RabbitMQ.Client.Framing.Impl.Connection.MainLoop() HResult: -2146233088 Source: RabbitMQ.Client Stack Trace: at RabbitMQ.Client.Framing.Impl.Connection.EnsureIsOpen()
at RabbitMQ.Client.Framing.Impl.AutorecoveringConnection.EnsureIsOpen()
at RabbitMQ.Client.Framing.Impl.AutorecoveringConnection.CreateModel()
at Playtech.Neon.MQ.Client.RabbitMQ.RabbitMQClient.<>c__DisplayClass14_0.b__2()
at Polly.Policy.<>c__DisplayClass114_01.<Execute>b__0(Context ctx, CancellationToken ct) at Polly.Retry.RetryEngine.Implementation[TResult](Func3 action, Context context, CancellationToken cancellationToken,
ExceptionPredicates shouldRetryExceptionPredicates, ResultPredicates1 shouldRetryResultPredicates, Action4 onRetry, Int32
permittedRetryCount, IEnumerable1 sleepDurationsEnumerable, Func4
sleepDurationProvider)
Any idea why this happens?
I'm working on an server-side Blazor application which supports the "normal" build-in Identity login as well as AzureAD authentication (where the authenticated user is then mapped to a local one).
The authentication and authorization themselves work fine.
Logged in via build-in Identity I get an authentication cookie.
Logged in via AzureAD I get the Azure cookies and the Identity cookie:
Authentication cookies
But occassionally the applications seems to hang, all cookies are removed and the authentication state is gone. I've got logged out. And I have no idea why this happens.
The DevTools console then shows:
blazor.server.js:1 [2020-05-14T19:46:43.201Z] Information: Normalizing '_blazor' to 'https://wwwxxx/xxx/_blazor'.
blazor.server.js:1 [2020-05-14T19:56:56.047Z] Warning: Timeout from HTTP request.
blazor.server.js:1 GET https://wwwxxx/xxx/_blazor?id=cV_9FcIQ1C1raoggiSYzVg&_=1589486216048 404
blazor.server.js:1 [2020-05-14T19:56:56.559Z] Error: Connection disconnected with error 'Error'.
blazor.server.js:1 [2020-05-14T19:56:59.565Z] Information: Normalizing '_blazor' to 'https://wwwxxx/xxx/_blazor'.
Startup.cs looks as follows:
public void ConfigureServices(IServiceCollection services)
{
services.AddDbContext<ApplicationDbContext>(options =>
options.UseSqlServer(
Configuration.GetConnectionString("DefaultConnection")));
services.AddDefaultIdentity<AppUser>(options =>
{
options.SignIn.RequireConfirmedAccount = true;
options.User.RequireUniqueEmail = true;
})
.AddRoles<IdentityRole>()
.AddEntityFrameworkStores<ApplicationDbContext>()
.AddDefaultTokenProviders();
services.AddHttpContextAccessor();
services.AddRazorPages().AddRazorPagesOptions(options =>
{
options.Conventions.AuthorizeFolder("/");
}).AddMvcOptions(options =>
{
var policy = new AuthorizationPolicyBuilder()
.RequireAuthenticatedUser()
.Build();
options.Filters.Add(new AuthorizeFilter(policy));
}); ;
services.AddServerSideBlazor();
services.AddScoped<AuthenticationStateProvider, RevalidatingIdentityAuthenticationStateProvider<AppUser>>();
services.AddAuthentication()
.AddAzureAD(options => Configuration.Bind("AzureAd", options));
}
public void Configure(IApplicationBuilder app, IWebHostEnvironment env)
{
//Localization
app.UseRequestLocalization(app.ApplicationServices.GetService<IOptions<RequestLocalizationOptions>>().Value);
if (env.IsDevelopment())
{
app.UseDeveloperExceptionPage();
app.UseDatabaseErrorPage();
}
else
{
app.UseExceptionHandler("/Error");
// The default HSTS value is 30 days. You may want to change this for production scenarios, see https://aka.ms/aspnetcore-hsts.
app.UseHsts();
}
app.UseHttpsRedirection();
app.UseStaticFiles();
app.UseRouting();
app.UseAuthentication();
app.UseAuthorization();
app.UseEndpoints(endpoints =>
{
endpoints.MapControllers();
endpoints.MapBlazorHub();
endpoints.MapFallbackToPage("/_Host");
});
}
Does anyone know this problem? Any suggestions?
Thank you in advance!
Update:
It seems that the WebSocket connection breaks... I don't know why:
Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher: Debug: Received hub invocation: InvocationMessage { InvocationId: "", Target: "EndInvokeJSFromDotNet", Arguments: [ 1230, True, [1230,true,null] ], StreamIds: [ ] }.
Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher: Trace: InvocationId (null): Sending result of type 'System.Void'.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 2919 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 2919 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 23565 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 238 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 196 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 178 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 279 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 279 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 279 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 279 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 174 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 543 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 174 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 543 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 174 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 543 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 543 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 104 bytes.
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in Microsoft.AspNetCore.Server.IIS.dll
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in System.Private.CoreLib.dll
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in System.Private.CoreLib.dll
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in System.Private.CoreLib.dll
Exception thrown: 'Microsoft.AspNetCore.Connections.ConnectionResetException' in System.Private.CoreLib.dll
Exception thrown: 'System.Net.WebSockets.WebSocketException' in System.Net.WebSockets.dll
Exception thrown: 'System.Net.WebSockets.WebSocketException' in System.Private.CoreLib.dll
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.1.3\System.Diagnostics.StackTrace.dll'.
'iisexpress.exe' (CoreCLR: clrhost): Loaded 'C:\Program Files\dotnet\shared\Microsoft.NETCore.App\3.1.3\System.Reflection.Metadata.dll'.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Debug: Socket connection closed prematurely.
System.Net.WebSockets.WebSocketException (0x80004005): The remote party closed the WebSocket connection without completing the close handshake.
---> Microsoft.AspNetCore.Connections.ConnectionResetException: The client has disconnected
---> System.Runtime.InteropServices.COMException (0x800704CD): Es wurde versucht, auf eine nicht vorhandene Netzwerkverbindung zuzugreifen. (0x800704CD)
--- End of inner exception stack trace ---
at Microsoft.AspNetCore.Server.IIS.Core.IO.AsyncIOOperation.GetResult(Int16 token)
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadBody()
at System.IO.Pipelines.PipeCompletion.ThrowLatchedException()
at System.IO.Pipelines.Pipe.GetReadResult(ReadResult& result)
at System.IO.Pipelines.Pipe.GetReadAsyncResult()
at Microsoft.AspNetCore.Server.IIS.Core.IISHttpContext.ReadAsync(Memory`1 memory, CancellationToken cancellationToken)
at Microsoft.AspNetCore.Server.IIS.Core.HttpRequestStream.ReadAsyncInternal(Memory`1 buffer, CancellationToken cancellationToken)
at System.Net.WebSockets.ManagedWebSocket.EnsureBufferContainsAsync(Int32 minimumRequiredBytes, CancellationToken cancellationToken, Boolean throwOnPrematureClosure)
at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory`1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter)
at System.Net.WebSockets.ManagedWebSocket.ReceiveAsyncPrivate[TWebSocketReceiveResultGetter,TWebSocketReceiveResult](Memory`1 payloadBuffer, CancellationToken cancellationToken, TWebSocketReceiveResultGetter resultGetter)
at Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsServerTransport.StartReceiving(WebSocket socket)
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Debug: Waiting for the application to finish sending data.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Debug: Socket closed.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext: Trace: Disposing connection 8FE-SmJEB1D0RUWx8dXjLA.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext: Trace: Waiting for application to complete.
Microsoft.AspNetCore.SignalR.HubConnectionHandler: Debug: OnConnectedAsync ending.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionContext: Trace: Application complete.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionManager: Debug: Removing connection qSMJEL5tNU3rTLk5nrX_xw from the list of connections.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionManager: Debug: New connection es3QXXbAgihjUVXi211hVw created.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionDispatcher: Debug: Sending negotiation response.
Microsoft.AspNetCore.Http.Connections.Internal.HttpConnectionDispatcher: Debug: Establishing new connection.
Microsoft.AspNetCore.SignalR.HubConnectionHandler: Debug: OnConnectedAsync started.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Debug: Socket opened using Sub-Protocol: '(null)'.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Message received. Type: Text, size: 38, EndOfMessage: True.
Microsoft.AspNetCore.SignalR.Internal.DefaultHubProtocolResolver: Debug: Found protocol implementation for requested protocol: blazorpack.
Microsoft.AspNetCore.SignalR.HubConnectionContext: Debug: Completed connection handshake. Using HubProtocol 'blazorpack'.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 3 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Message received. Type: Binary, size: 245, EndOfMessage: True.
Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher: Debug: Received hub invocation: InvocationMessage { InvocationId: "0", Target: "ConnectCircuit", Arguments: [ CfDJ8Hy6nGrT4bRAtvHZDDv3QE4idYpgvocWjooUF0ypIU4d2QFQXnLVnBfBIzseKV-LRhC60wxVjdqZkcUdFGKUASZUiIU4a5mWs2TYam17ichgx6gxoM5ueM7_v9JuiXbr91zEbuiw0MHdHeCb6U_QPCA_36siQCR0h3BfgEwTdqEjdB0PwG7CROaiqdapc-obGR9-vIeRtB0GjsLrNSJy2N0 ], StreamIds: [ ] }.
Exception thrown: 'System.Threading.Tasks.TaskCanceledException' in System.Private.CoreLib.dll
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 18869 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 2919 bytes.
Microsoft.AspNetCore.Http.Connections.Internal.Transports.WebSocketsTransport: Trace: Sending payload: 48272 bytes.
Microsoft.AspNetCore.SignalR.Internal.DefaultHubDispatcher: Trace: InvocationId 0: Sending result of type 'System.Boolean'.
I am trying some edge cases in my software. So I have created a very simple test environment:
RabbitMQ server running on CentOS 7
Message consumer written against .NETCore 2.1 in C# running under CentOS 7
Message sender written against .NETCore 2.1 in C# running under CentOS 7
I send a simple text message every 5 seconds. The sender and receiver run on the same UNIX box, while the RabbitMQ server runs on a different machine in the network.
So far so good. Now I stop my RabbitMQ server with systemctl stop rabbitmq-server.
I get errors on the sender and the receiver which was expected.
I restart the RabbitMQ server using systemctl start rabbitmq-server.
And now the fun starts! The sender can recover and continues to send messages but the consumer CANNOT recover and does not receive messages. They are accumulated on the RabbitMQ server!
Here are my log entries from the sender (which works as expected):
2019-01-22 21:18:25.628 +01:00 [ERR] [EasyNetQ.PersistentConnection] [ThreadId 10] Failed to connect to broker infraserver-tbws2, port 5672, vhost testvh
RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable ---> System.AggregateException: One or more errors occurred. (Connection failed) ---> RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: Connection refused 172.16.63.239:5672
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source)
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Net.Sockets.Socket.<>c.<ConnectAsync>b__272_0(IAsyncResult iar)
--- End of stack trace from previous location where exception was thrown ---
at RabbitMQ.Client.TcpClientAdapter.ConnectAsync(String host, Int32 port)
at RabbitMQ.Client.Impl.TaskExtensions.TimeoutAfter(Task task, Int32 millisecondsTimeout)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectOrFail(ITcpClient socket, AmqpTcpEndpoint endpoint, Int32 timeout)
--- End of inner exception stack trace ---
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectUsingAddressFamily(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 timeout, AddressFamily family)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectUsingIPv4(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 timeout)
at RabbitMQ.Client.Impl.SocketFrameHandler..ctor(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 connectionTimeout, Int32 readTimeout, Int32 writeTimeout)
at RabbitMQ.Client.Framing.Impl.IProtocolExtensions.CreateFrameHandler(IProtocol protocol, AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 connectionTimeout, Int32 readTimeout, Int32 writeTimeout)
at RabbitMQ.Client.ConnectionFactory.CreateFrameHandler(AmqpTcpEndpoint endpoint)
at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func`2 selector)
--- End of inner exception stack trace ---
at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func`2 selector)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
--- End of inner exception stack trace ---
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(String clientProvidedName)
at EasyNetQ.ConnectionFactoryWrapper.CreateConnection()
at EasyNetQ.PersistentConnection.TryToConnect()
2019-01-22 21:18:25.632 +01:00 [ERR] [EasyNetQ.PersistentConnection] [ThreadId 10] Failed to connect to any Broker. Retrying in 00:00:05
2019-01-22 21:18:35.444 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 7.' | Num: 7 | Guid: 5345c7e4-61e6-4c79-8179-d4bef7864420'.
2019-01-22 21:18:40.452 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 8.' | Num: 8 | Guid: 3cd8635c-cdfa-45f3-8495-2acb0713d47b'.
2019-01-22 21:18:45.457 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 9.' | Num: 9 | Guid: 099462b8-cd66-40b9-ac10-89c3246819ec'.
2019-01-22 21:18:50.470 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 10.' | Num: 10 | Guid: c25139b2-8e45-4771-9544-830014382e0c'.
2019-01-22 21:18:55.515 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 11.' | Num: 11 | Guid: 90049d91-3805-4aaa-ac18-b61c09164afd'.
2019-01-22 21:19:00.526 +01:00 [INF] [] [ThreadId 1] Sucessfully sent Message 'Message: 'This is test message number 12.' | Num: 12 | Guid: 108ff318-6a34-4e64-94bd-dafa67aa6717'.
This shows the last error message and then one can see that EasyNetQ recovered and can deliver messages again.
The message consumer does NOT work! Here my log entries:
2019-01-22 21:18:25.623 +01:00 [ERR] [EasyNetQ.PersistentConnection] [ThreadId 12] Failed to connect to broker infraserver-tbws2, port 5672, vhost testvh
RabbitMQ.Client.Exceptions.BrokerUnreachableException: None of the specified endpoints were reachable ---> System.AggregateException: One or more errors occurred. (Connection failed) ---> RabbitMQ.Client.Exceptions.ConnectFailureException: Connection failed ---> System.Net.Internals.SocketExceptionFactory+ExtendedSocketException: Connection refused 172.16.63.239:5672
at System.Runtime.ExceptionServices.ExceptionDispatchInfo.Throw(Exception source)
at System.Net.Sockets.Socket.EndConnect(IAsyncResult asyncResult)
at System.Net.Sockets.Socket.<>c.<ConnectAsync>b__272_0(IAsyncResult iar)
--- End of stack trace from previous location where exception was thrown ---
at RabbitMQ.Client.TcpClientAdapter.ConnectAsync(String host, Int32 port)
at RabbitMQ.Client.Impl.TaskExtensions.TimeoutAfter(Task task, Int32 millisecondsTimeout)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectOrFail(ITcpClient socket, AmqpTcpEndpoint endpoint, Int32 timeout)
--- End of inner exception stack trace ---
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectUsingAddressFamily(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 timeout, AddressFamily family)
at RabbitMQ.Client.Impl.SocketFrameHandler.ConnectUsingIPv4(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 timeout)
at RabbitMQ.Client.Impl.SocketFrameHandler..ctor(AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 connectionTimeout, Int32 readTimeout, Int32 writeTimeout)
at RabbitMQ.Client.Framing.Impl.IProtocolExtensions.CreateFrameHandler(IProtocol protocol, AmqpTcpEndpoint endpoint, Func`2 socketFactory, Int32 connectionTimeout, Int32 readTimeout, Int32 writeTimeout)
at RabbitMQ.Client.ConnectionFactory.CreateFrameHandler(AmqpTcpEndpoint endpoint)
at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func`2 selector)
--- End of inner exception stack trace ---
at RabbitMQ.Client.EndpointResolverExtensions.SelectOne[T](IEndpointResolver resolver, Func`2 selector)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
--- End of inner exception stack trace ---
at RabbitMQ.Client.ConnectionFactory.CreateConnection(IEndpointResolver endpointResolver, String clientProvidedName)
at RabbitMQ.Client.ConnectionFactory.CreateConnection(String clientProvidedName)
at EasyNetQ.ConnectionFactoryWrapper.CreateConnection()
at EasyNetQ.PersistentConnection.TryToConnect()
2019-01-22 21:18:25.625 +01:00 [ERR] [EasyNetQ.PersistentConnection] [ThreadId 12] Failed to connect to any Broker. Retrying in 00:00:05
Here it sits and waits forever! Looks like the thing is deadlocked in a way while the messages are accumulating on the RabbitMQ server:
When I stop my consumer application and restart it, the messages are picked up.
I connect in both applications (sender and consumer) with the following code:
private static IBus SetupRabbitMqConnection(string rabbitServer, string rabbitVHost, ushort rabbitPort, string rabbitUser, string rabbitPwd)
{
Log.Logger.Debug($"Creating a connection to RabbitMQ server '{rabbitServer}' on port {rabbitPort.ToString()} " +
$"using the EasyNetQ library....");
try
{
var connStr = $"host={rabbitServer}:{rabbitPort.ToString()};virtualHost={rabbitVHost};username={rabbitUser};" +
$"password={rabbitPwd};publisherConfirms=true;timeout=30;prefetchcount=1;requestedHeartbeat=30";
var msgBus = RabbitHutch.CreateBus(connStr, x => { });
if (!msgBus.IsConnected)
{
var errMsg = $"Currently not connected to RabbitMQ server '{rabbitServer}'.";
Log.Logger.Error(errMsg);
}
Log.Logger.Debug("Successfully connected to RabbitMQ server.");
return msgBus;
}
catch (Exception ex)
{
Log.Logger.Error($"Error to establish a connection to RabbitMQ server '{rabbitServer}'. Error: {ex}");
throw;
}
}
The consumer program registers the listeners as follows:
var msgBus = SetupRabbitMqConnection(rabbitServer, vhost, rabbitPort, rabbitUser, rabbitPwd);
RegisterMsgSubscriptions(msgBus);
private static void RegisterMsgSubscriptions(IBus msgBus)
{
Log.Logger.Debug("Starting to register RabbitMQ message subscriptions...");
try
{
#region Queue declarations
var advancedBus = msgBus.Advanced;
var testQueueOne = new EasyNetQ.Topology.Queue(TestQueueOneName, true);
Log.Logger.Debug("Finished declaring queues.");
#endregion
#region Message Queue Handler registrations
advancedBus.Consume(testQueueOne, registration => registration
.Add<RabbitMessage<TestTextMessageDto>>(MessageProcessor.ProcessRabbitTestMessage));
Log.Logger.Debug("Finished registration of message handlers for several queues.");
#endregion
}
catch (Exception ex)
{
Log.Logger.Error($"Error registering message handler. Error: {ex}");
}
}
Any idea what might go wrong here?? In production we have more than 100 servers consuming messages. These servers are on premise spread all over the country. The RabbitMQ server is in the data center. So the consuming servers MUST recover if the connection gets lost, otherwise this is not usable!
Just add subscription logic on connected event (It will be fired on every broker connected event).
advancedBus.Connected += (sender, args) => {
// subscribe logic
};
Of course You should handle not to call subscribe logice twice on any connect etc.
I'm getting a failure and the following error message when I try to start a Selenium Test.
OpenQA.Selenium.WebDriverException was unhandled by user code
HResult=-2146233088
Message=A exception with a null response was thrown sending an HTTP
request to the remote WebDriver server for URL http://localhost:55754
/session. The status of the exception was ReceiveFailure, and the message
was: The underlying connection was closed: An unexpected error occurred
on a receive.
Source=WebDriver
InnerException:
HResult=-2146233079
Message=The underlying connection was closed: An unexpected error occurred on a receive.
Source=System
StackTrace:
at System.Net.HttpWebRequest.GetResponse()
at OpenQA.Selenium.Remote.HttpCommandExecutor.CreateResponse(WebRequest request)
InnerException:
HResult=-2146232800
Message=Unable to read data from the transport connection: An existing connection was forcibly closed by the remote host.
Source=System
StackTrace:
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.PooledStream.Read(Byte[] buffer, Int32 offset, Int32 size)
at System.Net.Connection.SyncRead(HttpWebRequest request, Boolean userRetrievedStream, Boolean probeRead)
InnerException:
ErrorCode=10054
HResult=-2147467259
Message=An existing connection was forcibly closed by the remote host
NativeErrorCode=10054
Source=System
StackTrace:
at System.Net.Sockets.Socket.Receive(Byte[] buffer, Int32 offset, Int32 size, SocketFlags socketFlags)
at System.Net.Sockets.NetworkStream.Read(Byte[] buffer, Int32 offset, Int32 size)
InnerException:
Here is the failing code, it has been working fine for months.
ChromeOptions options = new ChromeOptions();
//ChromeDriver Driver = new ChromeDriver(#"C:\Program Files (x86)\SeleniumDrivers", options);
////Driver = new ChromeDriver(options);
options.AddArgument("--disable-extensions");
Driver = new ChromeDriver(options);
Uninstall the ChromeDriver and WebDriver nuget packages and then reinstall them.
hi i am trying to connect to microsoft analysis service through olap4j but i got this error every time "org.olap4j.OlapException: This connection encountered an exception while executing a query"
This Full Description of Error :
<code>22:30:45.331 [ERROR] [kimo] 22:30:45.326:XRP4:WARN:RPCManager:org.olap4j.OlapException: This connection encountered an exception while executing a query.undefined - response: {operationId: "listCubes_fetch",
clientContext: Obj,
context: Obj,
transactionNum: 0,
httpResponseCode: 200,
httpResponseText: "//isc_RPCResponseStart-->[{data:"org.ola..."[212],
xmlHttpRequest: [object XMLHttpRequest],
transport: "xmlHttpRequest",
status: -1,
clientOnly: undef,
httpHeaders: Obj,
isStructured: true,
callbackArgs: null,
results: Obj,
data: "org.olap4j.OlapException: This connectio..."[91],
invalidateCache: false,
isDSResponse: true,
queueStatus: -1,
startRow: 0,
endRow: 0,
totalRows: 0}
com.smartgwt.client.core.JsObject$SGWT_WARN: 22:30:45.326:XRP4:WARN:RPCManager:org.olap4j.OlapException: This connection encountered an exception while executing a query.undefined - response: {operationId: "listCubes_fetch",
clientContext: Obj,
context: Obj,
transactionNum: 0,
httpResponseCode: 200,
httpResponseText: "//isc_RPCResponseStart-->[{data:"org.ola..."[212],
xmlHttpRequest: [object XMLHttpRequest],
transport: "xmlHttpRequest",
status: -1,
clientOnly: undef,
httpHeaders: Obj,
isStructured: true,
callbackArgs: null,
results: Obj,
data: "org.olap4j.OlapException: This connectio..."[91],
invalidateCache: false,
isDSResponse: true,
queueStatus: -1,
startRow: 0,
endRow: 0,
totalRows: 0}
at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
at sun.reflect.NativeConstructorAccessorImpl.newInstance(Unknown Source)
at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(Unknown Source)
at java.lang.reflect.Constructor.newInstance(Unknown Source)
at com.google.gwt.dev.shell.MethodAdaptor.invoke(MethodAdaptor.java:105)
at com.google.gwt.dev.shell.MethodDispatch.invoke(MethodDispatch.java:71)
at com.google.gwt.dev.shell.OophmSessionHandler.invoke(OophmSessionHandler.java:172)
at com.google.gwt.dev.shell.BrowserChannelServer.reactToMessages(BrowserChannelServer.java:293)
at com.google.gwt.dev.shell.BrowserChannelServer.processConnection(BrowserChannelServer.java:547)
at com.google.gwt.dev.shell.BrowserChannelServer.run(BrowserChannelServer.java:364)
at java.lang.Thread.run(Unknown Source)
this is the connection function
<code>
public static OlapConnection getConnectionAServices() throws Exception {
Class.forName("org.olap4j.driver.xmla.XmlaOlap4jDriver");
// anonymous connection
Connection connectionXMLA = DriverManager.getConnection(
"jdbc:xmla:Server=http://localhost/olap/msmdpump.dll;" + "Catalog=Analysis_Service;");
OlapWrapper wrapper = (OlapWrapper) connectionXMLA;
OlapConnection olapConnection = wrapper.unwrap(OlapConnection.class);
return olapConnection;
}
Have you set up XML/A HTTP access for SQL Server Analysis Services in IIS?
Configuring HTTP Access to SQL Server 2008 Analysis Services on Microsoft Windows Server 2008
You also need to add a role in the database with appropriate permissions, and to add
the user you are running the application as to that role.