How to track MSDTC error in NServicebus? - nservicebus

We got a weird error that we can't figure the cause of. It happened over 100 times in a matter of seconds and then it stopped.
Stacktrace:
System.Transactions.TransactionAbortedException: The transaction has aborted. ---> System.Transactions.TransactionException: MSDTC was unable to read its configuration information. (Exception from HRESULT: 0x8004D027) ---> System.Runtime.InteropServices.COMException: MSDTC was unable to read its configuration information. (Exception from HRESULT: 0x8004D027)
at System.Transactions.Oletx.IDtcProxyShimFactory.ConnectToProxy(String nodeName, Guid resourceManagerIdentifier, IntPtr managedIdentifier, Boolean& nodeNameMatches, UInt32& whereaboutsSize, CoTaskMemHandle& whereaboutsBuffer, IResourceManagerShim& resourceManagerShim)
at System.Transactions.Oletx.DtcTransactionManager.Initialize()
--- End of inner exception stack trace ---
at System.Transactions.Oletx.OletxTransactionManager.ProxyException(COMException comException)
at System.Transactions.Oletx.DtcTransactionManager.Initialize()
at System.Transactions.Oletx.DtcTransactionManager.get_ProxyShimFactory()
at System.Transactions.Oletx.OletxTransactionManager.CreateTransaction(TransactionOptions properties)
at System.Transactions.TransactionStatePromoted.EnterState(InternalTransaction tx)
--- End of inner exception stack trace ---
at System.Transactions.TransactionStateAborted.CheckForFinishedTransaction(InternalTransaction tx)
at System.Transactions.Transaction.Promote()
at System.Transactions.TransactionInterop.ConvertToOletxTransaction(Transaction transaction)
at System.Transactions.TransactionInterop.GetDtcTransaction(Transaction transaction)
at System.Messaging.MessageQueue.StaleSafeReceiveMessage(UInt32 timeout, Int32 action, MQPROPS properties, NativeOverlapped* overlapped, ReceiveCallback receiveCallback, CursorHandle cursorHandle, IntPtr transaction)
at System.Messaging.MessageQueue.ReceiveCurrent(TimeSpan timeout, Int32 action, CursorHandle cursor, MessagePropertyFilter filter, MessageQueueTransaction internalTransaction, MessageQueueTransactionTyp
Any ideas about how we can track the cause of this error?

Related

.NET Core gives unknown error while processing HTTPS certificate [closed]

Closed. This question is not reproducible or was caused by typos. It is not currently accepting answers.
This question was caused by a typo or a problem that can no longer be reproduced. While similar questions may be on-topic here, this one was resolved in a way less likely to help future readers.
Closed 3 years ago.
Improve this question
I'm trying to build a .NET Core server that uses a HTTPS connection.
I created a self-signed certificate using dotnet dev-certs tool and
set up Kestrel like this:
return WebHost.CreateDefaultBuilder(args)
.UseKestrel(options =>
{
options.Listen(IPAddress.Loopback, 5000, lisOpt => lisOpt.UseHttps("myCert.pfx", "test"));
})
.UseStartup<Startup>()
.Build();
But when I try to connect to it with my client, I keep getting this exception:
fail: Microsoft.AspNetCore.Server.Kestrel[0]
Uncaught exception from the OnConnectionAsync method of an IConnectionAdapter.
System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. ---> System.ComponentModel.Win32Exception: An unknown error occurred while processing the certificate
--- End of inner exception stack trace ---
at System.Net.Security.SslState.StartSendAuthResetSignal(ProtocolToken message, AsyncProtocolRequest asyncRequest, ExceptionDispatchInfo exception)
at System.Net.Security.SslState.CheckCompletionBeforeNextReceive(ProtocolToken message, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartSendBlob(Byte[] incoming, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.ProcessReceivedBlob(Byte[] buffer, Int32 count, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.StartReadFrame(Byte[] buffer, Int32 readBytes, AsyncProtocolRequest asyncRequest)
at System.Net.Security.SslState.PartialFrameCallback(AsyncProtocolRequest asyncRequest)
--- End of stack trace from previous location where exception was thrown ---
at System.Net.Security.SslState.ThrowIfExceptional()
at System.Net.Security.SslState.InternalEndProcessAuthentication(LazyAsyncResult lazyResult)
at System.Net.Security.SslState.EndProcessAuthentication(IAsyncResult result)
at System.Net.Security.SslStream.EndAuthenticateAsServer(IAsyncResult asyncResult)
at System.Net.Security.SslStream.<>c.<AuthenticateAsServerAsync>b__50_2(IAsyncResult iar)
at System.Threading.Tasks.TaskFactory`1.FromAsyncCoreLogic(IAsyncResult iar, Func`2 endFunction, Action`1 endAction, Task`1 promise, Boolean requiresSynchronization)
--- End of stack trace from previous location where exception was thrown ---
at Microsoft.AspNetCore.Server.Kestrel.Https.Internal.HttpsConnectionAdapter.InnerOnConnectionAsync(ConnectionAdapterContext context)
at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.FrameConnection.ApplyConnectionAdaptersAsync()
fail: Microsoft.AspNetCore.Server.Kestrel[0]
Uncaught exception from the OnConnectionAsync method of an IConnectionAdapter.
System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception. ---> System.ComponentModel.Win32Exception: An unknown error occurred while processing the certificate
--- End of inner exception stack trace ---
What am I doing wrong?
This apparently happened because my client sent requests to 127.0.0.1 instead of 'localhost'.

Silverlight + Wcf: CommunicationException : The remote server returned an error: Not Found

I have really strange problem. I load my silverlight application and if I call the service in lets say next 30 seconds (or lets say immediatelly) everything works. However if I call the service like 2 minutes after my application was loaded I will get
{System.ServiceModel.CommunicationException: The remote server returned an error: NotFound. ---> System.Net.WebException: The remote server returned an error: NotFound. ---> System.Net.WebException: The remote server returned an error: NotFound.
at System.Net.Browser.BrowserHttpWebRequest.InternalEndGetResponse(IAsyncResult asyncResult)
at System.Net.Browser.BrowserHttpWebRequest.<>c__DisplayClassa.<EndGetResponse>b__9(Object sendState)
at System.Net.Browser.AsyncHelper.<>c__DisplayClass4.<BeginOnUI>b__0(Object sendState)
--- End of inner exception stack trace ---
at System.Net.Browser.AsyncHelper.BeginOnUI(SendOrPostCallback beginMethod, Object state)
at System.Net.Browser.BrowserHttpWebRequest.EndGetResponse(IAsyncResult asyncResult)
at System.ServiceModel.Channels.HttpChannelFactory.HttpRequestChannel.HttpChannelAsyncRequest.CompleteGetResponse(IAsyncResult result)
--- End of inner exception stack trace ---
at System.ServiceModel.AsyncResult.End[TAsyncResult](IAsyncResult result)
at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.End(SendAsyncResult result)
at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
at System.ServiceModel.ClientBase`1.ChannelBase`1.EndInvoke(String methodName, Object[] args, IAsyncResult result)
at EnrollPortal.UI.SL.EnrollApp.FitC.FitCWatchlistMatcherClient.FitCWatchlistMatcherClientChannel.EndAssessBiometricQuality(IAsyncResult result)
at EnrollPortal.UI.SL.EnrollApp.FitC.FitCWatchlistMatcherClient.EnrollPortal.UI.SL.EnrollApp.FitC.IFitCWatchlistMatcher.EndAssessBiometricQuality(IAsyncResult result)
at EnrollPortal.UI.SL.EnrollApp.FitC.FitCWatchlistMatcherClient.OnEndAssessBiometricQuality(IAsyncResult result)
at System.ServiceModel.ClientBase`1.OnAsyncCallCompleted(IAsyncResult result)}
if i browse the service itself it is working.
Thanks for the hints!

NServiceBus Could not load file or assembly

I am getting the below error when i call "NServiceBus.Host.exe". The issue appears to be itextsharp.dll. I've read that i can use
"Configure.With(AllAssemblies.Except("itextsharp.dll"))... // rest of your config"
But i'm using the following sample and not sure where to add this property.
http://docs.particular.net/samples/web/asp-mvc-application/
#
Unhandled Exception: Magnum.StateMachine.StateMachineException: Exception occurr
ed in Topshelf.Internal.ServiceController1[[NServiceBus.Host.Internal.GenericHo
st, NServiceBus.Host, Version=2.6.0.1504, Culture=neutral, PublicKeyToken=9fc386
479f8a226c]] during state Initial while handling OnStart ---> System.Exception:
Exception when starting endpoint, error has been logged. Reason: Could not load
file or assembly 'file:///D:\dev\MYSITE\MYSITE.EStore\EStore.Server\bin\Debug\itexts
harp.dll' or one of its dependencies. Operation is not supported. (Exception fro
m HRESULT: 0x80131515)
at NServiceBus.Host.Internal.GenericHost.Start() in d:\BuildAgent-02\work\20b
5f701adefe8f8\src\host\NServiceBus.Host\Internal\GenericHost.cs:line 90
at Magnum.StateMachine.EventActionList1.Execute(T stateMachine, Event event,
Object parameter) in :line 0
--- End of inner exception stack trace ---
Server stack trace:
at Magnum.StateMachine.ExceptionActionDictionary1.HandleException(T stateMac
hine, Event event, Object parameter, Exception exception) in :line 0
at Magnum.StateMachine.EventActionList1.Execute(T stateMachine, Event event,
Object parameter) in :line 0
at Magnum.StateMachine.State1.RaiseEvent(T instance, BasicEvent1 eevent, Ob
ject value) in :line 0
at Magnum.StateMachine.StateMachine`1.RaiseEvent(Event raised) in :line 0
at System.Runtime.Remoting.Messaging.StackBuilderSink._PrivateProcessMessage(
IntPtr md, Object[] args, Object server, Int32 methodPtr, Boolean fExecuteInCont
ext, Object[]& outArgs)
at System.Runtime.Remoting.Messaging.StackBuilderSink.SyncProcessMessage(IMes
sage msg, Int32 methodPtr, Boolean fExecuteInContext)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage req
Msg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgDa
ta, Int32 type)
at Topshelf.Internal.ServiceControllerProxy.Start()
at Topshelf.Internal.ServiceCoordinator.Start() in d:\dev\open-source\topshel
f\src\Topshelf\Internal\ServiceCoordinator.cs:line 54
at Topshelf.Internal.Hosts.ConsoleHost.Run() in d:\dev\open-source\topshelf\s
rc\Topshelf\Internal\Hosts\ConsoleHost.cs:line 53
at NServiceBus.Host.Program.Main(String[] args) in d:\BuildAgent-02\work\20b5
f701adefe8f8\src\host\NServiceBus.Host\Program.cs:line 103
When NserviceBus host starts it tries to retrieve all types from assemblies you specify.
In the sample you provide it uses NServiceBus.Configure.WithWeb() method.
This is the same as to call NServiceBus.Configure.With(AppDomain.CurrentDomain.DynamicDirectory)
You get the excpetion because "itextsharp.dll" cannot be loaded to AppDomain (this is done by calling Assembly.LoadFrom(file.FullName) method).
To fix the issue, you may manually specify assemblies you need to scan by using one of NServiceBus.Configure.With() overloads.

The I/O operation has been aborted because of either a thread exit or an application request

I'm running some WCF net.tcp services on a high-traffic website.
In my local environment everything worked as expect.
In the production environment, the website works for few minutes then the service throw an exception.
When I stop and start again the service website it works again for few minutes.
Here's the details:
A TCP error (995: The I/O operation has been aborted because of either a thread exit or an application request) occurred while transmitting data.
System.ServiceModel.Channels.SocketConnection.EndRead()
System.ServiceModel.Channels.TracingConnection.EndRead()
System.ServiceModel.Channels.ConnectionStream.ReadAsyncResult.OnAsyncReadComplete(Object state)
System.ServiceModel.Channels.TracingConnection.TracingConnectionState.ExecuteCallback()
System.ServiceModel.Channels.SocketConnection.AsyncReadCallback(Boolean haveResult, Int32 error, Int32 bytesRead)
System.Runtime.Fx.IOCompletionThunk.UnhandledExceptionFrame(UInt32 error, UInt32 bytesRead, NativeOverlapped* nativeOverlapped)
System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)
The I/O operation has been aborted because of either a thread exit or an application request
I see the same problem on Win8kR2SP1Web:
System.Runtime.InteropServices.COMException (0x800703E3): The I/O operation has been aborted because of either a thread exit or an application request. (Exception from HRESULT: 0x800703E3)
An error occurred while communicating with the remote host. The error code is 0x800703E3.
The I/O operation has been aborted because of either a thread exit or an application request. (Exception from HRESULT: 0x800703E3)
at System.Web.Hosting.IIS7WorkerRequest.RaiseCommunicationError(Int32 result, Boolean throwOnDisconnect)
at System.Web.Hosting.IIS7WorkerRequest.ReadEntityCoreSync(Byte[] buffer, Int32 offset, Int32 size)
at System.Web.HttpRequest.GetEntireRawContent()
at System.Web.HttpRequest.FillInFormCollection()
at System.Web.HttpRequest.get_Form()
at System.Web.Mvc.HttpRequestExtensions.GetHttpMethodOverride(HttpRequestBase request)
at System.Web.Mvc.AcceptVerbsAttribute.IsValidForRequest(ControllerContext controllerContext, MethodInfo methodInfo)
at System.Linq.Enumerable.All[TSource](IEnumerable`1 source, Func`2 predicate)
at System.Web.Mvc.ActionMethodSelector.RunSelectionFilters(ControllerContext controllerContext, List`1 methodInfos)
at System.Web.Mvc.ReflectedControllerDescriptor.FindAction(ControllerContext controllerContext, String actionName)
at System.Web.Mvc.ControllerActionInvoker.FindAction(ControllerContext controllerContext, ControllerDescriptor controllerDescriptor, String actionName)
at System.Web.Mvc.ControllerActionInvoker.InvokeAction(ControllerContext controllerContext, String actionName)
at System.Web.Mvc.Controller.ExecuteCore()
at System.Web.Mvc.ControllerBase.Execute(RequestContext requestContext)
at System.Web.Mvc.MvcHandler.<>c__DisplayClass6.<>c__DisplayClassb.<BeginProcessRequest>b__5()
at System.Web.Mvc.Async.AsyncResultWrapper.<>c__DisplayClass1.<MakeVoidDelegate>b__0()
at System.Web.Mvc.MvcHandler.<>c__DisplayClasse.<EndProcessRequest>b__d()
at System.Web.HttpApplication.CallHandlerExecutionStep.System.Web.HttpApplication.IExecutionSt ep.Execute()
at System.Web.HttpApplication.ExecuteStep(IExecutionStep step, Boolean& completedSynchronously)
First time in 1 year though ...

Asynchronous WCF fails with MessageSecurityException when multiple worker processes (Web Garden)

I have a simple WCF service using ws2007HttpBinding which puts in a configurable delay, then returns. The service is configured to be PerCall. The service is hosted on IIS 7.5 on Windoes 2008 Server R2 SP1
I am call this asynchronously via console app using an imported service reference.
I can easily inject 10000 requests and they are all successful. Each request uses a new WCF proxy. The CPU was running about 70%
I wanted to see if adding another worker process would help throughput and utilise the full 100% cpu. I configured it to have 2 worker processes.
When I ran this it began to fail virtually immediately with a MessageSecurityException. Even if I set the number of requsts to 10, it still fails. I noticed that it fails more if the timer in the WCF service is short ( between 10 and 50 ms). A higher delay causes less failures.
It either fails on the proxy.Open() or proxy.Close()
I investigated this and thought it might be related to secure conversation. So I disabled this by
setting
establishSecurityContext="false"
(Is this correct?) I am confused why this would be the case if I am creating a new proxy each time
This dis not seem to make any difference.
When I return the AppPool to have only 1 worker process it works again
Has anybody had the same issue, and managed to get a solution. If so can you post you solution in explicit detail.
The errors received are as follows
In the WCF trace it reported
The incoming message is not part of an existing security session
The exception shown in trace
System.ServiceModel.EndpointNotFoundException, System.ServiceModel, Version=4.0.0.0, Culture=neutral, PublicKeyToken=b77a5c561934e089
There was no channel that could accept the message with action 'http://tempuri.org/ILoadedService/RunLoad'.
at System.ServiceModel.Dispatcher.ErrorBehavior.ThrowAndCatch(Exception e, Message message)
at System.ServiceModel.Channels.DatagramChannelDemuxer2.ProcessItem(TInnerItem item)
at System.ServiceModel.Channels.DatagramChannelDemuxer2.HandleReceiveResult(IAsyncResult result)
at System.ServiceModel.Channels.DatagramChannelDemuxer2.OnReceiveComplete(IAsyncResult result)
at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
at System.ServiceModel.Channels.ReplyChannel.HelpReceiveRequestAsyncResult.OnReceiveRequest(IAsyncResult result)
at System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame(IAsyncResult result)
at System.Runtime.AsyncResult.Complete(Boolean completedSynchronously)
at System.Runtime.InputQueue1.AsyncQueueReader.Set(Item item)
at System.Runtime.InputQueue1.EnqueueAndDispatch(Item item, Boolean canDispatchOnThisThread)
at System.Runtime.InputQueue1.EnqueueAndDispatch(T item, Action dequeuedCallback, Boolean canDispatchOnThisThread)
at System.ServiceModel.Channels.SingletonChannelAcceptor`3.Enqueue(QueueItemType item, Action dequeuedCallback, Boolean canDispatchOnThisThread)
at System.ServiceModel.Channels.HttpChannelListener.HttpContextReceived(HttpRequestContext context, Action callback)
at System.ServiceModel.Activation.HostedHttpTransportManager.HttpContextReceived(HostedHttpRequestAsyncResult result)
at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.HandleRequest()
at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.BeginRequest()
at System.ServiceModel.Activation.HostedHttpRequestAsyncResult.OnBeginRequest(Object state)
at System.Runtime.IOThreadScheduler.ScheduledOverlapped.IOCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* nativeOverlapped)
at System.Runtime.Fx.IOCompletionThunk.UnhandledExceptionFrame(UInt32 error, UInt32 bytesRead, NativeOverlapped* nativeOverlapped)
at System.Threading._IOCompletionCallback.PerformIOCompletionCallback(UInt32 errorCode, UInt32 numBytes, NativeOverlapped* pOVERLAP)
System.ServiceModel.EndpointNotFoundException: There was no channel that could accept the message with action 'http://tempuri.org/ILoadedService/RunLoad'.
Exception caught on the client
Exception thrown of type [System.ServiceModel.Security.MessageSecurityException] Source [mscorlib]
Exception when receiving response [9] - Message [An unsecured or incorrectly secured fault was received from the other party. See the inner FaultException for the fault code and detail.] StackTrace [
Server stack trace:
at System.Runtime.AsyncResult.End[TAsyncResult](IAsyncResult result)
at System.ServiceModel.Channels.ServiceChannel.SendAsyncResult.End(SendAsyncResult result)
at System.ServiceModel.Channels.ServiceChannel.EndCall(String action, Object[] outs, IAsyncResult result)
at System.ServiceModel.Channels.ServiceChannelProxy.InvokeEndService(IMethodCallMessage methodCall, ProxyOperationRuntime operation)
at System.ServiceModel.Channels.ServiceChannelProxy.Invoke(IMessage message)
Exception rethrown at [0]:
at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at SimpleIISTestClient.SlowClient.ILoadedService.EndRunLoad(IAsyncResult result)
at SimpleIISTestClient.Program.PerfAsynchCallback(IAsyncResult result) in C:\CRF Prototypes\Load\SimpleIISTestClient\Program.cs:line 116]
Inner Exception [9] - Message [The security context token is expired or is not valid. The message was not processed.] StackTrace []