I have a WCF service where client applications can connect via a MSMQ:
[ServiceContract(Namespace="http://example.com", SessionMode = SessionMode.NotAllowed)]
public interface IMyService
{
[OperationContract(IsOneWay = true)]
void Update(DataSet ds);
}
and then:
string queueName = ConfigurationManager.AppSettings["QueueName"];
NetMsmqBinding binding = new NetMsmqBinding("MyBinding");
if (!MessageQueue.Exists(#".\private$\" + queueName))
{
MessageQueue.Create(#".\private$\" + queueName, binding.ExactlyOnce);
}
ServiceHost msmqHost = new ServiceHost(typeof(MyService));
msmqHost.AddServiceEndpoint(typeof(IMyService), binding, "net.msmq://localhost/private/" + queueName);
with the following configuration:
<system.serviceModel>
<bindings>
<netMsmqBinding>
<binding name="MyBinding" durable="false" exactlyOnce="false" maxReceivedMessageSize="20000000">
<security mode="None" />
<readerQuotas maxDepth="32" maxStringContentLength="543192" maxArrayLength="2147483647" maxBytesPerRead="4096" maxNameTableCharCount="8456384" />
</binding>
</netMsmqBinding>
</bindings>
<services>
<service name="MyService" behaviorConfiguration="MsMqBehavior" />
</services>
<behaviors>
<serviceBehaviors>
<behavior name="MsMqBehavior">
<serviceThrottling maxConcurrentCalls="50" maxConcurrentSessions="50" maxConcurrentInstances="50" />
</behavior>
</serviceBehaviors>
</behaviors>
</system.serviceModel>
I have the service with the same configuration already in use without problems on other installations. But now on a new installation I receive only messages from some clients (9 actually - there are 31). The messages I receive are always from the same servers. I can't find an error message anywhere (Windows Event Log (Client/Server), WCF Trace file) and also the MSMQ state says "connected" on the client machines that don't send messages. The dead letter queues are also empty.
The messages must get lost somewhere between the MSMQ Client and Server (I stopped my app and on the server queue I received only messages from the nine Clients - same behaviour if I enable journaling).
Any help would be appreciated
Update
I've used performance counters to monitor the queue. The session counter shows the correct value of 31 sessions. Also the incoming message counter shows correct values. However if I stop the app or enable journaling only a part of the messages are stored in the queue.
The problem comes through cloning the server as described in this blog entry: http://blogs.msdn.com/b/johnbreakwell/archive/2007/02/06/msmq-prefers-to-be-unique.aspx
Basically it says there that you must not clone servers with MSMQ feature turned on. If you do so you either have to re-install the MSMQ feature on your client machines or do a registry change:
1.Stop the MSMQ Service
2.Delete the QMId value completely
3.Add a SysPrep DWORD (Under HKLM\Software\Microsoft\MSMQ\Parameters) and set it to 1
4.Start the MSMQ Service
I'm doing some wcf tracing on a sever with 800 clients (verbose clients) and I am seeing a lot of SocketConnection aborted after Process action 'http://tempuri.org/IConnectionRegister/ValidateUriRoute'. This is during a ** ** activity
I removed (maxed out) all possible throttling (binding, service, port sharing (both on custom binding and SMSvcHost.exe). Still a lot of random timeouts and socket aborted.
Also the 0000 activity is absolutely full (less than every second) of
"The socket connection was aborted. This could be caused by an error processing your message or a receive timeout being exceeded by the remote host, or an underlying network resource issue. Local socket timeout was '10675199.02:48:05.4775807'."
"The I/O operation has been aborted because of either a thread exit or an application request"
Any help would be very much appreciate in identifying who is causing the dropping of connections.
I can also upload a sample svc trace if anyone is interested.
Stack trace of Socket Connection Abort
System.ServiceModel.Channels.SocketConnection.Abort()
System.ServiceModel.Channels.TracingConnection.Abort()
System.ServiceModel.Channels.InitialServerConnectionReader.Abort()
System.ServiceModel.Channels.ServerSessionPreambleConnectionReader.OnValidate()
......
System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame()
System.Runtime.AsyncResult.Complete()
System.ServiceModel.Channels.TransportDuplexSessionChannel.TryReceiveAsyncResult.OnReceive()
System.Runtime.Fx.AsyncThunk.UnhandledExceptionFrame()
System.Runtime.AsyncResult.Complete()
System.ServiceModel.Channels.SynchronizedMessageSource.ReceiveAsyncResult.OnReceiveComplete()
<customBinding>
<binding name="externalRoutingcustomBinding">
<binaryMessageEncoding>
</binaryMessageEncoding>
<security
authenticationMode="UserNameOverTransport"
allowInsecureTransport="true"
enableUnsecuredResponse="true"
>
</security><tcpTransport>
<connectionPoolSettings groupName = "default" maxOutboundConnectionsPerEndpoint = "1000"/>
</tcpTransport>
</binding>
<service name="ZPRoutingWorkflow.ZpRoutingService">
<endpoint address="Transactions" binding="customBinding" bindingConfiguration="externalRoutingcustomBinding"
name="Transactions" contract="TopUpInterface.ITransactionsService">
<identity>
<dns value="appserver.test.com" />
</identity>
</endpoint>
<endpoint address="Retry" binding="customBinding" bindingConfiguration="externalRoutingcustomBinding" name="Retry" contract="IRetryInterface.IRetryService">
<identity>
<dns value="appserver.test.com" />
</identity>
</endpoint>
Config summary:
Multiple net.tcp services over same port hosted in iis. One of the services is a routing service implementing IRequestReplyRouter. Basically all messages go through the router and the router connects internally to the services and passes them their messages
I have a remote private queue that I can write to without issue. I am trying to create a WAS-hosted WCF service to pull the data for processing. I need the service to be on a different machine than the queue; I currently have it in the same site as the writing service - same app pool but a different application.
The queue I am targetting is called PersistenceService/Log.svc and is transactional and authenticated. My service is running at http://appdev.me.com/PersistenceService/Log.svc. The below contains my endpoint and binding:
<bindings>
<msmqIntegrationBinding>
<binding exactlyOnce="true" durable="true" serializationFormat="Xml" maxReceivedMessageSize="2147483647"
closeTimeout="00:00:30" sendTimeout="00:00:30" receiveTimeout="00:00:30" timeToLive="24:00:00"
receiveRetryCount="1" maxRetryCycles="1" retryCycleDelay="00:10:00" receiveErrorHandling="Move">
<security mode="Transport" />
</binding>
</msmqIntegrationBinding>
</bindings>
<services>
<service name="Me.Logging.Persistence.PersistenceService">
<endpoint address="msmq.formatname:DIRECT=OS:DIRECT=OS:meserver\private$\persistenceservice/log.svc"
binding="msmqIntegrationBinding"
bindingNamespace="http://me.logging/services/2012/11"
contract="Me.Logging.Service.Shared.Service.Contracts.ICatchAll" />
</service>
</services>
The error that I get is the below:
An error occurred while opening the queue:The remote computer is not available. (-1072824215, 0xc00e0069). The message cannot be sent or received from the queue. Ensure that MSMQ is installed and running. Also ensure that the queue is available to open with the required access mode and authorization.
I have cleared the way for communications by shutting down McAfee so it's not blocking traffic. I have granted ANONYMOUS LOGON access to the queue temporarily - though it should be coming in as the same user through which I'm writing to the queue, and that user has full rights on the queue.
The WCF trace doesn't reveal anything other than this error. Any ideas as to where to look would be greatly appreciated.
Your endpoint looks to be incorrect. You have "DIRECT=OS:" in there twice. Try:
<endpoint address="msmq.formatname:DIRECT=OS:meserver\private$\persistenceservice/log.svc"
binding="msmqIntegrationBinding"
bindingNamespace="http://me.logging/services/2012/11"
contract="Me.Logging.Service.Shared.Service.Contracts.ICatchAll" />
I've created a WCF service that listens to a private MSMQ for jobs. I've throttled my service so that it will only handle one job at a time (maxConcurrentInstances = 1). The problem I have is that when two messages are submitted and I inspect the queue through my Computer Management console, it's empty. I expect there to be one pending message. When I submit three messages, I'll see one pending message in the MSMQ. From reading MSDN, it looks like the ServiceHost is holding the next job in memory until the current job is done, but I can't find a way to turn it off so that the it doesn't hold the message in memory. Does anyone know of a way to make it so that the ServiceHost won't hold the pending message in memory and leave it in the queue? Thanks!
<configuration>
<system.serviceModel>
<services>
<service
name="MyService"
behaviorConfiguration="DefaultServiceBehavior">
<host>
<baseAddresses>
<add baseAddress="http://localhost:8000/MyService/"/>
</baseAddresses>
</host>
<endpoint
address="net.msmq://localhost/private/MyService"
binding="netMsmqBinding" bindingConfiguration="MsmqBindingNoSecurity"
contract="IMyService" />
<endpoint address="mex"
binding="mexHttpBinding"
contract="IMetadataExchange" />
</service>
</services>
<behaviors>
<serviceBehaviors>
<behavior name="DefaultServiceBehavior">
<serviceMetadata httpGetEnabled="True"/>
<serviceThrottling
maxConcurrentCalls="1"
maxConcurrentSessions="1"
maxConcurrentInstances="1"
/>
</behavior>
</serviceBehaviors>
</behaviors>
<bindings>
<netMsmqBinding>
<binding name="MsmqBindingNoSecurity"
useActiveDirectory="false"
exactlyOnce="false">
<security mode="None">
<transport
msmqAuthenticationMode="None"/>
</security>
</binding>
</netMsmqBinding>
</bindings>
</system.serviceModel>
</configuration>
I have also noticed this behaviour in netMsmqBinding and as far as I know it's not addressable from the service end.
This is only an issue if you're not using transactional queues whereby a failure in your service could result in the in-memory message being dropped permanently.
If you use transactional queues even though the message has been read from the inbound queue it's actually still there on the queue (but it becomes "invisible"). If you suffer a failure on your service at this time the message will become re-queued and then processed when you come back up.
If you cannot use transactional queueing then the only way you can address this is to do so from the client, which means checking to see if a message has been transmitted before making another call. This can be done using System.Messaging or I assume you could bake this into a custom behaviour.
If you can't use a transactional queue (e.g. if you are using netMessagingBinding) you can use the ReceiveContext attribute for more fine-grained control. Its well explained by Juval Lowy here in his book:
http://books.google.co.uk/books?id=PvNrurEhmiEC&pg=PA500&lpg=PA500&dq=ReceiveContext&source=bl&ots=ChDvHuH_Jq&sig=XHhiz2ebmXuu0QNYcBYhtlN99p4&hl=en&sa=X&ei=XgYmUo3kHOOH0AW--oHoCg&ved=0CFsQ6AEwCTgK#v=onepage&q=ReceiveContext&f=false
Also see this MSDN article to see how specifically it can be used in the netMessagingBinding scenario (I know that's not directly relevant to your question but the principal still holds)
http://msdn.microsoft.com/en-us/library/hh532034(VS.103).aspx
I intermittently get the following exception in my .Net WCF Service.
"The HTTP service located at http://MyServer/TestWCF/MyService.svc is too busy."
Am I missing something here?
Am using basic http binding and have enabled WCF throttling.
<basicHttpBinding>
<binding name="BasicHttpBinding_MyService" maxReceivedMessageSize="2147483647"
messageEncoding="Text" textEncoding="utf-16" sendTimeout="00:01:00" >
<readerQuotas maxStringContentLength="2147483647" maxArrayLength="163840000"
maxDepth="2147483647" maxBytesPerRead="2147483647" maxNameTableCharCount="16384" />
</binding>
.
.
.
.
<behavior name="MyWCFServices.MyServiceBehavior">
<serviceMetadata httpGetEnabled="true" />
<serviceDebug includeExceptionDetailInFaults="true" />
<serviceThrottling
maxConcurrentCalls="16"
maxConcurrentInstances="2147483647"
maxConcurrentSessions="10"/>
</behavior>
Will throttling help resolving the issue?
Also,may i know the recommended parameter values for throttling for a high traffic web site?
You could definitely try to increase the maxConcurrentSessions and maxConcurrentCalls in your service throttling behavior to the standard values of 30 or so and see if that makes the error go away. Server too busy would seem to indicate that more requests have come in than area allowed by your service throttling behavior, and they've been discarded since no service instance became available to service them within the given timeout period.
My answer would be, check if the app pool is up and well?
I've seen this error occurring when the app pool has died due to exceptions being thrown that aren't caught.
Consider for example, custom config sections - having an error in there, will cause your app to fail before it's even started. Too many of these in a short space of time will kill the app pool.
If you're service is running under your account (Identity), it's quite possible that you've recently changed your password--you'll need to reset it for its IIS application pool in Advanced Settings | Identity dialog box.
It is not just the maxConcurrentSessions, it is also how long the session lasts.
If the client does not close the connection, it will remain open until it timesout. You could then hit the maxConcurrentSessions limit with very little activity on the server.
Make sure you check the inner exception, too; during our deployments, we disable the application pool of a WCF web service, and clients start getting this error during that time:
System.ServiceModel.ServerTooBusyException: The HTTP service located at https://ourserver.x.com/path/service.svc is too busy. ---> System.Net.WebException: The remote server returned an error: (503) Server Unavailable.
So in this case an HTTP error 503 is being (mis?)interpreted as "server too busy".
The only source of this exception that I am aware of is if you are using sessions, and you manage to hit the MaxPendingChannels throttle,. Its default is something pretty low like 4. You could try setting it higher (128 for example), or if you just want to repro, set it to 1 and you should see it under load testing.
See here for more information about sessions: http://msdn.microsoft.com/en-us/library/ms733795.aspx
I just ran into this error, and it boiled down to a simple configuration problem. I had a service up on the exact same port and same interface (mock service). I ran the service with the appropriate command line switch to run the "original" service I intended. The error went away.
My Solution would be, Check the App.Config file, whether the service tag is there for this particular service.
eg:
<service name="MyServices.ServiceName">
<endpoint address="" binding="wsHttpBinding" bindingConfiguration="TestBinding" contract="MyServices.ServiceName">
<identity>
<dns value="localhost" />
</identity>
</endpoint>
<endpoint address="mex" binding="mexHttpBinding" contract="IMetadataExchange" />
<host>
<baseAddresses>
<add baseAddress="http://localhost:8732/Design_Time_Addresses/MyServices/ServiceName/" />
</baseAddresses>
</host>
</service>