I have some queries about WCF and multithreading.
My plan is to place items onto the Thread Pool and for it to process messages from the MSMQ queue.
I also will be hosting WCF in WAS.
I am wondering how the threading will work at this point. For example messages will be picked up by the WCF binding to the MSMQ queue and I know that WAS will spin up the service as and when it requires to. But lets say if we have 100 messages to process (100 messages per second for example) - would these be delivered in a threaded way or in a single thread?
If in a threaded manner then how best to commit or abort transactions? Any special considerations?
Sorry for the questions - just need to clarify this.
Its not clear what "placing items onto the Thread Pool" does but on the WCF side, a service using the netMsmqBinding handles "calls" in a similar way as other WCF bindings. The difference is that a "call" is actually an MSMQ message in a queue.
This article on netMsmqBinding gives a very clear explanation of how the binding works. If you configure the WCF service with its default InstanceContext setting (per call or per session depending on the .NET version), the service instances will pick up messages off the queue as-if they were a standard call each. There are setting in MSMQ and WCF that can affect this behavior to make the messages be processed sequentially but that's not the default.
Let WCF handle multi-threading for you by leaving the service set to per call (or per session) and for transactions, look at the code in this sample in MSDN to see how to work with them.
Related
I'm aware of that there are a couple of posts here on SO that discusses this issue, but I can't figure out a straight answer to my scenario.
Lets say i have:
One queue on Server-A with alot of messages coming in
One WCF service with NetMsmq binding on Server-B reading from Server-A's queue
The very same WCF Service deployed to another Server-C with the same binding reading from Server-A's queue
Am I guaranteed that the services on Server-B and Server-C will never process the same message?
Are there any other problems with this setup that needs to be taken care of?
The purpose of this setup with multiple wcf services reading from the same queue is to increase processing speed, and I do not want to use a hardware loadbalancer.
Thanks for your time!
You will need to make your queue transactional and then set the exactlyOnce property on the binding to true.
Reference:
http://msdn.microsoft.com/en-us/library/system.servicemodel.msmqbindingbase.exactlyonce(v=vs.110).aspx
This should take care of it. You can have multiple readers (WCF Services) reading from the same queue. That's basically a load-levelling concept and is the intent of queue-based messaging. Here is some background documentation on WCF and MSMQ.
http://msdn.microsoft.com/en-us/library/ms789048(v=vs.110).aspx
I want to add server-side retry behavior if something specific happens during operation.
Custom IOperationInvoker looks like a good candidate for this functionality, but...
Unfortunately instance on which operation should be performed is already created/resolved, so to correctly implement retry logic everyone should write stateless service implementations which is quite limiting, sometimes it's nice to have InstanceContextMode.PerCall mode and state which lives only during request lifetime.
Is there any place or possibility to force WCF to re-create/resolve service and invoke operation again?
Not so easy to imlement retry logic on your own. You will have to deal with a number of issues like those:
What if all attempts failed?
Should there be an interval between attempts?
What if you server is down between the first and second attempt?
etc.
Fortunately there is an out of the box solution which already does everything for you. Take a look at the MSMQ WCF binding.
It will put you contract object to the dead letter queue if you wish, so you can easyly keep track of such messages and even force to process them again.
You can set an interval between the attempts. So if you server is down for 10 minutes not all attempts are shot.
You can configure it to be durable. It will store the messages on disk, So when your server is back it will try to process the message again.
Etc. Take a look at the MSMQ binding you will find a lot of useful features.
Here's a good article about the queuing in WCF
So in the long run your design will be something like that:
Client Call -> MSMQ -> Service
Please note you don't have to make any changes to the code you have now at all, you just change the binding configuration and set up MSMQ which is fairly easy.
Just in case you can't use msmq binding directly with you client, you can always go with an special server side service which just puts the messages to the queue. So the design will be:
Client service -> HTTP -> QueueService (one method which puts messages to a queue) -> MSMQ -> ProcessService
Hope it helps!
I am calling a WCF service which contains the business logic to process the message objects.
I need to pass the id of the message to WCF service. We are using MSMQ for queuing up the requests.
There could be multiple messages that WCF service need to process which can be handled as follows
Send the message id one by one
Send array of message ids and then WCF service will iterate through each id and process the message object.
Performance point of view I believe second option is better as multiple requests to WCF are not there.
Is my assumption correct?
-
Ram
Number 2 is more efficient in terms of latency but does not give you the chance to spread the processing load by having multiple queue readers
Also be aware that if you use a transactional queue and sessions then WCF may put more than one SOAP message in each MSMQ message
I'm thinking of adding a queue function in a product based on a bunch of WCF services. I've read some about MSMQ, first I thought that was what I needed but I'm not sure and are considering to just put the queue in a database table. I wonder if somone here got some feedback on which way to go.
Basicly I'm planning to have a facade WCF service called over http. The facade service should only write all incoming messages to a queue to give a fast response to the calling system. The messages in the queue should then be processed by another component, either a WCF service or a Windows service depending om my choice of queue.
The product is running in a load balanced enviroment with 2 to n web servers.
The options I'm considering and the questions I got are:
To let the facade WCF write to a MSMQ and then have anothther WCF service reading from this queue to do the processing of the messages. What I don't feel confident about for this alternative from what I've read is how this will work in a load balanced enviroment.
1A. Where should the MSMQ(s) be placed? One on each web server? One on a separate server? Mulitple on a separate server? (not considering need of redundance and that data in rare cases could be lost and re-sent)
1B. How it the design affected if I want the system redundant? I'd like to be alble to lose a server (it never comes up online again) holding the MSMQ without losing the data in that queue. From what I've read about MSMQ that leaves me to the only option of placing the MSMQ on a windows cluster. Is that correct? (I'd like to avoid using a windows cluster fo this).
The second design alternative is to let the facade WCF service write the queue to a database. Then have two or more Windows services to do the processing of the queue. I don't have any questions on this alternative. If you wonder why I don't pick this one as it seems simpler to me then it is because I'd like to build this not introducing any windows services to the solution, that I beleive the MSMQ got functionality I don't want to code myself and I'm also curious about using MSMQ as I've never used it before.
Best Regards
Håkan
OK, so you're not using WCF with MSMQ integration, you're using WCF to create MSMQ messages as an end-product. That simplifies things to "how do I load balance MSMQ?"
The arrangement you use is based on what works best for you.
You could have multiple webservers sending messages to a remote queue on a central machine.
Instead you could have a webservers putting messages in local queues with a central machine polling the queues for new arrivals.
You don't need to cluster MSMQ to make it resilient. You can instead make your code resilient so that it copes with lost messages using dead letter queues, transactional queues, journaling, and so on. Hardware clustering is the easy option :-)
Load-balancing MSMQ - a brief
discussion
Oil and water - MSMQ transactional
messages and load balancing
After reading some more on the subjet I haver decided to not use MSMQ. It seems like I really got no reason to go down this road. I need this to be non-transactional and as I understand it none of the journaling or dead letter techniques will help me with my redundancy requirement.
All my components will be online most of the time (maybe a couple of hours per year when they got access problems).
The MSQM will only add complexity to the exciting solution, another technique and maybe another server to keep track of.
To get full redundance to prevent data loss in MSMQ I will need a windows cluster or implement send/recieve to multiple identical queues. I don't want to do either of those.
All this lead me to front my recieving application with a WCF facade accepting http calls writing to a database queue. This database is already protected from data loss. The queue will be polled by muliple active instances of a Windows Servce containing all the heavy business logic. With low process priority these services could be hosted on the already existing nodes used by the load balaced web application. If I got time to use MSMQ or if I needed it for another reason in my application I might change my decision.
I have a system that sends a object to another service via WCF using MSMQ. The service picks the message up fine and does what it have to with it. But the problem i have now is that i need to send a response to the calling system.
Example:
Create a Customer object
Populate the information
Send the message to the service using WCF over MSMQ
Pick the message up from the queue using a windows service
Call Customer.Insert() method on the windows service
I now need to send the new customer id back to the calling application here.
Any ideas?
As Emmanuel points out - MSMQ messages are by design one-way and have no response, really.
Your best solution would be to have a response queue where the "other service" can drop his response messages into. Your client would then have to monitor that queue, e.g. check it once in a while (every minute, every 30 minutes - whatever makes sense for you) for new messages, and handle those.
There's no duplex (two-way) MSMQ channels - but you can easily create a pair of separate queues for both directions.
Marc
you can use duplex communication with msmq but not natively, take a look to my article
MSMQ Operation needs to one way, the only way I can think of receiving back a message is for your calling application to also Host a service for responses since there's no duplex MSMQ binding.