I am shortly starting a project, which requires messages to be held in a queue for a period of 24 hours, this is because the database can't have any updates at certain times of the month. The service also has to be hosted on windows server 2003, which means it will have to be a windows service.
It is also required that the service use WCF so that in 12 months time when we move over to windows server 2008, the service can hosted in iis 7. At present I am wondering if MSMQ is the best way to handle this.
I've been looking into topics like poison message handling & dead letter queues, but nothing that really covers what I am intending to actually do. Could anyone recommend a sample or a tutorial for this ?
Thanks in advance
Yes, it sounds like this is a perfect scenario for WCF and MSMQ. It should be much easier to use MSMQ than to create your own queuing mechanism with the same robustness. You will want to look into the Message.TimeToBeReceived Property for a message expiring timeout.
If the interval specified by the TimeToBeReceived property expires
before the message is removed from the
queue, Message Queuing discards the
message in one of two ways. If the
message's UseDeadLetterQueue property
is true, the message is sent to the
dead-letter queue. If
UseDeadLetterQueue is false, the
message is ignored.
Here are some good starter tutorials on WCF with MSMQ: link1 and link2
Related
I'm aware of that there are a couple of posts here on SO that discusses this issue, but I can't figure out a straight answer to my scenario.
Lets say i have:
One queue on Server-A with alot of messages coming in
One WCF service with NetMsmq binding on Server-B reading from Server-A's queue
The very same WCF Service deployed to another Server-C with the same binding reading from Server-A's queue
Am I guaranteed that the services on Server-B and Server-C will never process the same message?
Are there any other problems with this setup that needs to be taken care of?
The purpose of this setup with multiple wcf services reading from the same queue is to increase processing speed, and I do not want to use a hardware loadbalancer.
Thanks for your time!
You will need to make your queue transactional and then set the exactlyOnce property on the binding to true.
Reference:
http://msdn.microsoft.com/en-us/library/system.servicemodel.msmqbindingbase.exactlyonce(v=vs.110).aspx
This should take care of it. You can have multiple readers (WCF Services) reading from the same queue. That's basically a load-levelling concept and is the intent of queue-based messaging. Here is some background documentation on WCF and MSMQ.
http://msdn.microsoft.com/en-us/library/ms789048(v=vs.110).aspx
Yes, I have googled and there are loads of samples out there but each and every one is configured differently or is not quite what I am looking for.
I am trying to find a sample where I have MSMQ set up. I want a WCF Service which gets notified as and when a message arrives in the MSMQ queue.
I also want then a client application which binds to that contract and sends a message, placing it in the MSMQ queue.
Can anyone provide examples or links? I'd greatly appreciate it. I'm just getting confused between different types of messaging, different bindings.... list goes on. I am NOT looking for a System.Messaging implementation.
Thank you.
You can find plethora of examples on MSDN Windows Communication Foundation (WCF) and Windows Workflow Foundation (WF) Samples for .NET Framework 4. When you expand the package look under \WCFWFCardSpace\WCF\Basic\Binding\MsmqIntegration for some basic samples
EDIT:
For transactions it's as simple as:
[OperationBehavior(TransactionScopeRequired = true, TransactionAutoComplete = true)]
public void SubmitPurchaseOrder(MsmqMessage<PurchaseOrder> msg)
{}
You can read more about Poison Message Handling on MSDN More about the MSMQ on MSDN - How to: Exchange Messages with WCF Endpoints and Message Queuing Applications
"When the service reads messages from the target queue under a transaction, the service may fail to process the message for various reasons. The message is then put back into the queue to be read again. To deal with messages that fail repeatedly, a set of poison-message handling properties can be configured in the binding. There are four properties: ReceiveRetryCount, MaxRetryCycles, RetryCycleDelay, and ReceiveErrorHandling. "
EDIT: You would use System.Messaging on the service side.
I have some queries about WCF and multithreading.
My plan is to place items onto the Thread Pool and for it to process messages from the MSMQ queue.
I also will be hosting WCF in WAS.
I am wondering how the threading will work at this point. For example messages will be picked up by the WCF binding to the MSMQ queue and I know that WAS will spin up the service as and when it requires to. But lets say if we have 100 messages to process (100 messages per second for example) - would these be delivered in a threaded way or in a single thread?
If in a threaded manner then how best to commit or abort transactions? Any special considerations?
Sorry for the questions - just need to clarify this.
Its not clear what "placing items onto the Thread Pool" does but on the WCF side, a service using the netMsmqBinding handles "calls" in a similar way as other WCF bindings. The difference is that a "call" is actually an MSMQ message in a queue.
This article on netMsmqBinding gives a very clear explanation of how the binding works. If you configure the WCF service with its default InstanceContext setting (per call or per session depending on the .NET version), the service instances will pick up messages off the queue as-if they were a standard call each. There are setting in MSMQ and WCF that can affect this behavior to make the messages be processed sequentially but that's not the default.
Let WCF handle multi-threading for you by leaving the service set to per call (or per session) and for transactions, look at the code in this sample in MSDN to see how to work with them.
I'm thinking of adding a queue function in a product based on a bunch of WCF services. I've read some about MSMQ, first I thought that was what I needed but I'm not sure and are considering to just put the queue in a database table. I wonder if somone here got some feedback on which way to go.
Basicly I'm planning to have a facade WCF service called over http. The facade service should only write all incoming messages to a queue to give a fast response to the calling system. The messages in the queue should then be processed by another component, either a WCF service or a Windows service depending om my choice of queue.
The product is running in a load balanced enviroment with 2 to n web servers.
The options I'm considering and the questions I got are:
To let the facade WCF write to a MSMQ and then have anothther WCF service reading from this queue to do the processing of the messages. What I don't feel confident about for this alternative from what I've read is how this will work in a load balanced enviroment.
1A. Where should the MSMQ(s) be placed? One on each web server? One on a separate server? Mulitple on a separate server? (not considering need of redundance and that data in rare cases could be lost and re-sent)
1B. How it the design affected if I want the system redundant? I'd like to be alble to lose a server (it never comes up online again) holding the MSMQ without losing the data in that queue. From what I've read about MSMQ that leaves me to the only option of placing the MSMQ on a windows cluster. Is that correct? (I'd like to avoid using a windows cluster fo this).
The second design alternative is to let the facade WCF service write the queue to a database. Then have two or more Windows services to do the processing of the queue. I don't have any questions on this alternative. If you wonder why I don't pick this one as it seems simpler to me then it is because I'd like to build this not introducing any windows services to the solution, that I beleive the MSMQ got functionality I don't want to code myself and I'm also curious about using MSMQ as I've never used it before.
Best Regards
Håkan
OK, so you're not using WCF with MSMQ integration, you're using WCF to create MSMQ messages as an end-product. That simplifies things to "how do I load balance MSMQ?"
The arrangement you use is based on what works best for you.
You could have multiple webservers sending messages to a remote queue on a central machine.
Instead you could have a webservers putting messages in local queues with a central machine polling the queues for new arrivals.
You don't need to cluster MSMQ to make it resilient. You can instead make your code resilient so that it copes with lost messages using dead letter queues, transactional queues, journaling, and so on. Hardware clustering is the easy option :-)
Load-balancing MSMQ - a brief
discussion
Oil and water - MSMQ transactional
messages and load balancing
After reading some more on the subjet I haver decided to not use MSMQ. It seems like I really got no reason to go down this road. I need this to be non-transactional and as I understand it none of the journaling or dead letter techniques will help me with my redundancy requirement.
All my components will be online most of the time (maybe a couple of hours per year when they got access problems).
The MSQM will only add complexity to the exciting solution, another technique and maybe another server to keep track of.
To get full redundance to prevent data loss in MSMQ I will need a windows cluster or implement send/recieve to multiple identical queues. I don't want to do either of those.
All this lead me to front my recieving application with a WCF facade accepting http calls writing to a database queue. This database is already protected from data loss. The queue will be polled by muliple active instances of a Windows Servce containing all the heavy business logic. With low process priority these services could be hosted on the already existing nodes used by the load balaced web application. If I got time to use MSMQ or if I needed it for another reason in my application I might change my decision.
I have a system that sends a object to another service via WCF using MSMQ. The service picks the message up fine and does what it have to with it. But the problem i have now is that i need to send a response to the calling system.
Example:
Create a Customer object
Populate the information
Send the message to the service using WCF over MSMQ
Pick the message up from the queue using a windows service
Call Customer.Insert() method on the windows service
I now need to send the new customer id back to the calling application here.
Any ideas?
As Emmanuel points out - MSMQ messages are by design one-way and have no response, really.
Your best solution would be to have a response queue where the "other service" can drop his response messages into. Your client would then have to monitor that queue, e.g. check it once in a while (every minute, every 30 minutes - whatever makes sense for you) for new messages, and handle those.
There's no duplex (two-way) MSMQ channels - but you can easily create a pair of separate queues for both directions.
Marc
you can use duplex communication with msmq but not natively, take a look to my article
MSMQ Operation needs to one way, the only way I can think of receiving back a message is for your calling application to also Host a service for responses since there's no duplex MSMQ binding.