Slow MSMQ within a WCF service - wcf

this is a weird thing.
I created a simple SOAP based web service with WCF. When the 'SubmitTransaction' method is called, the transaction is being passed on to an application service. But if the application service is not available, it is being written to a MSMQ.
Like this:
public void SubmitTransaction(someTransaction)
{
try
{
// pass transaction data to application
}
catch(SomeError)
{
// write to MSMQ
}
}
So when an error occures the transaction is written to the queue. Now, when using the MSMQ API directly in my WCF service, everything is fine. Each call takes a few milliseconds.
E.g.:
...
catch(SomeError)
{
// write to MSMQ
var messageQueue = new MessageQueue(queuePath);
try
{
messageQueue.Send(accountingTransaction, MessageQueueTransactionType.Single);
}
finally
{
messageQueue.Close();
}
}
But since I want to use the message queue functionality at some other points of the system as well, I created a new assembly that takes care of the message queue writing.
Like:
...
catch(SomeError)
{
// write to MSMQ
var messageQueueService = new MessageQueueService();
messageQueueService.WriteToQueue(accountingTransaction);
}
Now when using this setup, the web service is suddenly very slow. From the above-mentioned milliseconds, each call now takes up to 4 seconds. Only because the message queue stuff is encapsulated in a new assembly. The logic is exactly the same. Anyone knows what the problem could be...?
Thanks!

Ok, now I know. It has something to do with my logging setup (log4net). I'll have to check that first. Sorry for stealing your time..

You have two new lines of code here:
var messageQueueService = new MessageQueueService();
messageQueueService.WriteToQueue(accountingTransaction);
Do you know which of the two is causing the problem? Perhaps add some logging, or profiling, or step through in a debugger to see which one seems slow.

Related

ASP.NET Core 2.1 Handling ServiceScope usage with dependencies on WebSocket middleware's every incoming messages

I have some performance issue on using websockets on ASP.NET Core 2.1
At first, I have an implementation of websockets similar to this example:
https://radu-matei.com/blog/aspnet-core-websockets-middleware/
On every incoming websocket message, I have it to parse, call some services, send a message back to websocket.
if (result.MessageType == WebSocketMessageType.Text)
{
using (var scope = service.CreateScope())
{
var communicationService = scope.ServiceProvider.GetSomeService();
await communicationService.HandleConnection(webSocket, result, buffer);
}
}
So as you see on every incoming message I am creating a new Scope, getting Service provider and then calling services on this service's method communicationService.HandleConnection. But if there is a lot of messages, my Azure WebService CPU goes up to 100%.
Can someone tell me if I am using these scope creations correct on every socket message?
It's hard to know for certain with the limited code snippet you provided (i.e. the lifetime and type of communicationService. Having said that, I would look at https://learn.microsoft.com/en-us/azure/app-service/faq-availability-performance-application-issues to capture a snapshot of your app service when the CPU spike to 100%. You may discover the issue might be unrelated to your using (scope).

IErrorHandler interface is confusing me

I have created a class that implements IErrorHandler and IServiceBehavior. I add the behavior to my ServiceHost and run it.
The reason I got around to trying to learn about IErrorHandler is because the frustration I was having in having to wrap all of my service code that interfaces with callback channels in try catch statements, and wondered if there was a global way to catch the exceptions.
Well from what I read here on StackOverflow, I saw that it is indeed the thing I want.
In execution however, it's confusing me greatly. If I have a contract method that does the following...
Dictionary<IChatCallback, string> userChannels = new Dictionary<IChatCallback, string>();
public void SendMessage(string message)
{
IChatCallback callback = OperationContext.Current.GetCallbackChannel<IChatCallback>();
string senderName = userChannels[callback];
SendMessageToAllUsers(message, senderName);
}
public void SendMessageToAllUsers(string message, string sender)
{
foreach (var userChannel in userChannels.Keys)
{
userChannel.ReceiveMessage(new ChatMessage(message, sender));
}
}
If a client is on my server and he drops off the face of the earth and the channel faults, it seems to take a while before my server realizes that the channel is faulted. Even if am subscribed to the events of the ICommunicationObject, it seems to take a while before the server realizes the channel is faulted. For example, if someone is sending messages and someone disconnects, and a message is being propagated at the same time.
Now, I have another method that essentially pings clients at a regular interval to figure out if they're still there or not, so I can remove them from the userChannel dictionary.
How exactly is IErrorHandler supposed to help me here? It seems that it's not exactly helping my service from crashing even if it catches the error. Because my service cannot ping my clients at a super rapid pace, sometimes a message seems to come in immediately(I was testing this) after a previous message that threw an exception. It seemed to have handled the first one, but the second one threw another exception that wasn't caught, and my service crashed, because the server was trying to communicate with a faulted channel. Now, my service DOES clean up those faulted channels, but it does so periodically.
I was hoping that by implementing IErrorHandler I could avoid having to wrap all of my methods in try{}catch{} blocks... but it seems like I still have to, to check if I'm communicating with a faulted channel?
Or maybe I'm just going about using IErrorHandler improperly and thinking it should be doing things it's not supposed to do. Should I maybe have my service implement it, and inject itself into the Servicehost as the IServiceBehavior? And then in my HandleError() method remove the channel from my list of clients? That seems kind of messy, because I'm putting that WCF plumbing stuff in my service class, but that would be the only way I can think of to have the exception code itself remove the channel from the list of callbacks.
I'm trying to do a lot of reading because I'm new to it, but it seems like the world of Error Handling in WCF is daunting.
Even if I try to remove the channel so it's not called on services via subscribing to the Faulted event, it doesn't work fast enough and still throws exception that the server cannot seem to recover from even with IErrorHandler.
For example, when the user first connects, I subscribe to the event.
IChatCallback callback = OperationContext.Current.GetCallbackChannel<IChatCallback>();
ICommunicationObject callbackComm = (ICommunicationObject) callback;
callbackComm.Faulted += (sender, e) =>
{
lock (lockObject)
{
string name = userChannels[callback];
userChannels.Remove(callback);
NotifyOfUserDisconnect(name);
}
};
Do I just need to go into all my methods and check if the channel is closed or faulted, and wrapping everything in a try/catch block to remove the reference and notify clients?

Calling multiple operations on a WCF client

I am currently looking at some code which basically reads data line-by-line from a file and calls an appropriate method on a WCF client for each line. My problem with the code is that a new client is created for each operation ie:
function CallSomeOp(line)
{
using (var client = new Client()
{
client.SomeOp(line);
}
}
This seems to make the client close the connection and reopen a new one for each line in the file which seems like a big overhead. What I would like to do is move the creation of the client outside the function and into initialization, so that multiple operations are called on one client, ie:
using (var client = new Client())
{
for(var line in lines)
{
CallSomeOp(line, client);
}
}
But I'm not sure if this is OK or not. Is there some problem with calling multiple operations on a client, ie will I eventually run out of memory or something?
An operation is typically stateless, so that's expected behaviour. You can change the InstanceContextMode though. This determines if a new instance is created with each call or that there is a single instance. The question is, should you? Like I said, a single call is typically stateless. Having just a single instance doesn't scale very well either. I'd suggest to keep InstanceContextMode to be PerCall, that way, for every call the webserver creates a single instance, I wouldn't worry too much about overhead. If overhead really does become a problem, maybe a service isn't what you need.
Then there's the ConcurrencyMode which allows you to specify multiple threads to be created.
As far as the client is concerned, you don't need to re-create it everytime you want to call an operation. Alternatively, you could also look at the ChannelFactory if both client and server have knowledge of the contract. This answer provides some detail.
It's definitely a good idea to reuse a WCF client across multiple calls, since there is indeed a cost in creating new instances of it, even if that's been improved in later versions.
The most important thing to be aware of when reusing a WCF client, though, is that if a service operation fails, the whole client will be put in a faulted state and can no longer be used.
You will have to take this into consideration in your client code. Depending on your requirements, one way to handle this situation could be to recreate the client proxy in case of failure and continue processing the input, like shown in this example:
var client = new ServiceClient();
foreach (var line in lines)
{
try
{
client.DoSomething(line);
}
catch
{
client.Abort();
client = new ServiceClient()
}
}
Note that you may also want to keep track of how many times the service call fails and with what type of exception, since in some situations it would make more sense to completely abort the operation instead of keep trying, e.g. when the remote service is not reachable.

Calling Abort on faulted WCF channel causes server side error to be logged

When a FaultException is returned from my WCF service, I need to Abort the channel instead of Closing it. Both my client and service work fine with this approach but after implementing IErrorHandler on the service and logging any exception, I can see that calling Abort on the client causes the service to log:
System.ServiceModel.CommunicationException: The socket connection was aborted...
I do not want to pollute my service logs with this information and only want to log service related errors. I know I can obviously stop logging any CommunicationExceptions but my service is also a WCF client for other services and CommunicationExceptions raised by these services should be logged.
How can I stop it doing this?
As nobody else has answered the question (Tomas's answer was not relevant), I asked a few experts in the field. Unfortunately, there is no nice way of stopping this and the best that they could come up with was to add logic in IErrorHandler to not log CommunicationExcepions with a message starting with 'The socket connection was aborted'. Not very elegant, but it does work.
The problem is that you get an exception that covers your underlying exception if you get an exception when calling dispose wich is possible. I wrote a wrapper to deal with scenarios like this, you can read about it on my blog: http://blog.tomasjansson.com/2010/12/disposible-wcf-client-wrapper/
The idea is that you have a wrapper around your channel that deals with the scenario if the dispose method throws an exception.
A small example of how you should use my wrapper:
public class ClientWrapperUsage : IYourInternalInterface
{
public IList<SomeEntity> GetEntitiesForUser(int userId)
{
using(var clientWrapper = new ServiceClientWrapper<ServiceType>())
{
var response = clientWrapper.Channel.GetEntitiesForUser();
var entities = response.YourListOfEntities.TranslateToInternal();
return entities;
}
}
}
Here I have assumed that it existing an extension method for a list that contains the entity that is returned by the service, then you use that method to translate it to internal entities. This is 100 % testable, at least I think :). Just moch the interface IYourInternalInterface everywhere you wan't to fake the service.

Using WCF and NetNamedPipeBinding for IPC

I'm trying to learn WCF to use it as an IPC mechanism for a host/plugin system. The host needs to be able to call the plugin to Start/Stop it, and the plugin needs to call the server back to perform logging.
I made a simple test case where the host creates an endpoint on net.pipe://localhost/SampleServer with the following ServiceContract:
[ServiceContract]
public interface IWcfServer
{
[OperationContract]
void Log(string message);
}
And the plugin creates an endpoint on net.pipe://localhost/SampleClient with the following ServiceContract:
[ServiceContract]
public interface IWcfClient
{
[OperationContract]
string Init();
}
Here's a sample of how I'm setting up each endpoint:
this.server = new ServiceHost(this);
this.server.AddServiceEndpoint(typeof(IWcfServer),
new NetNamedPipeBinding(),
"net.pipe://localhost/SampleServer");
this.server.Open();
And here's a sample of how I'm making the calls:
ChannelFactory<IWcfClient> factory = new ChannelFactory<IWcfClient>(
new NetNamedPipeBinding(),
new EndpointAddress("net.pipe://localhost/SampleClient"));
IWcfClient client = factory.CreateChannel();
using ((IClientChannel)client)
{
client.Init());
}
I already confirmed that the host can call plugin.Init(), and the plugin can call host.Log(message) without issues. However, if this following scenario happens:
Host calls plugin.Init()
During the execution of plugin.Init(), the plugin attempts to call host.Log(message)
The applications freezes, and I get a TimeoutException after 1min. Anyone has any ideas on what I'm doing wrong?
what is the InstanceContextMode of the service host ? If it is a singleton, it will block until Init() returns - resulting in a circular dependency.
1 min is the standard wcf timeout.
Do you have a circular reference?
Also, why do you have 2 contracts, when you make a call to client.init who is listening?
Turn on E2E tracing for WCF to check what exactly is timing out. - http://msdn.microsoft.com/en-us/library/ms733025.aspx. Besides your methods might be causing a deadlock since init might require log and log might require init to happen first or something like that.
"net.pipe://localhost/SampleServer"
"net.pipe://localhost/SampleClient"
You have two different URL for the Server and for the Client. It is a problem!