I have a MSMQ queue and I need to implement a listener that is executed periodically or at specified time (i.e. nightly) to process messages in the queue.
WCF provides netMsmqBinding that allows sending messages to other service via MSMQ. I wonder is it possible to implement the WCF service to consume messages at specified time or periodically in equal intervals? Or WCF always consumes message as soon as it arrives?
For example I need to check queue every hour, and if there are any messages - process them.
One more question is about concurrency. Can I configure WCF service to use limited number of threads (e.g. 2) for queue message handling?
Thanks
Your best bet is to host the MSMQ consumer in a windows service and then configure a windows scheduled task to start it up and shut it down (eg with a powershell script) as per your service window requirements.
EDIT: I believe NServiceBus sagas can also support this requirement but it does not use WCF.
Related
We are working towards an architecture like one below but we will have micro services on cloud and some on premises which will talk to each other using queue(s) and bus(es),
Now I am confused with where we should host MassTransit and RabbitMq, also should it be a ASP.NET Core project on its own ? if yes what I will be doing in it ? starting a bus ? creating queues ? I am not able to move forward with this
The simple MassTransit examples are just that, the absolute simplest examples of interacting with queues.
RabbitMQ is your message broker. It is hosted separately.
MassTransit is a development framework that makes it much easier to interact with RabbitMQ (or Azure Service Bus) by abstracting away the implementation-specific "plumbing."
You write any number of .NET services that either publish messages to a queue, or subscribe to queues.
We recently worked on something similar, the way we did it is:
RabbitMQ was hosted separately, and buses/queues creation and management were done from the services that use messaging.
For each service that receives messages you use Maastransit to create a queue because service will be receiving messages using this queue.
You will be using publish/subscribe way of messaging so as mentioned above, inside each service, create a queue with logical name and connect to RabbitMQ server address.
Services that represent senders will publish messages of a custom type you create, and services that represent receivers will subscribe to this type of messages by having a consumer for this type registered inside the bus created.
Hope it helps.
Tech Stack:
.Net 4.6.1
Erlang 18.3
Language: C# 6
NServiceBus 6
RabbitMQ 3.6.3
Windows 7
Context:
We are trying to implement a dumb event publisher with smart subscription in NServiceBus on the RabbitMQ transport.
All interested receivers subscribe to an event.
A Publisher publishes the event to an event channel.
All subscribers receive a copy of the event.
Note that I did not say each TYPE of subscriber receives a copy of the message. If there are multiple instances of a service running, and they all have active subscriptions to an event, each INSTANCE of the subscribing service should get a copy of the message.
However, NServiceBus' notion of Publish-Subscribe delivers a published event to one and only one receiver on a given channel. In our case, one instance of a given subscribed service.
I hesitate to list messaging "patterns" because they don't seem to be named or described particularly consistently. However, I believe we are esentially trying to implement the "Multicast" version of the Publish-Subscribe Channel messaging pattern from the Enterprise Integration Patterns (Hohpe and Woolf) Book.
Problem Statement:
Our business case is this:
We have a configuration service that provides application configuration to all other services in our broader application.
Each service requests its configuration from the configuration service on startup by sending a ConfigurationRequest message.
The configuration services replies to the specific instance that made the ConfigurationRequest with a ConfigurationResponse. This is done using NServiceBus' Full-Duplex (commonly called Request-Response) feature.
There is a website that can modify configuration globally. When it does so, it notifies the configuration service with an UpdateConfiguration command.
The configuration service publishes a ConfigurationUpdated event that all other services are subscribed to.
Each service can have multiple instances running on more than one server. ALL instances of a service need to update their configuration, not just one instance.
Each service instance is calling NServiceBus' EndpointConfiguration.MakeInstanceUniquelyIdentifiable with a distinct discriminator.
Currently when we publish the ConfigurationUpdated event, only one instance of each service type gets the message. The events are distributed round-robin, instead of each instance getting a copy of the message.
We have worked around this problem by keeping a record of running services instances and sending the ConfigurationUpdated event (as a Command in NServiceBus) to each of them individually, but Pub-Sub implies that we should have dumb publishers and smart subscribers, and our workaround is the opposite... Our publisher looks up a listof each subscriber and sends to it implicitly. Is there some configuration I am missing on the subscriber side that would allow each instance of each service to get a copy of the published ConfigurationUpdated event? If not, where should I look in NServiceBus to start implementing such a feature? Routing topology, perhaps?
i'm not familiar enough with nservicebus to know how to do it with that toolset, but the RabbitMQ implementation would be a "fanout" exchange with a queue per consumer.
Every queue bound to a fanout exchange will get a copy of the message. If there is a single consumer for each of those queues, then you will be sending a copy of the message to each of the consumers.
It sounds like you have multiple consumers connected to the same queue. Maybe there's a way to tell nservicebus to create a queue for each consumer instance?
p.s. you're right about pub-sub pattern being multicast. i talk about this in my RMQ Patterns ebook (https://leanpub.com/rmq-patterns) and that's what the EIP book would say, as well.
I know that once a message has been delivered to the MSMQ by a WCF client, the netmsmqbinding provides retries out of the box in case the service faults.
But if my client fails to put the message in the MSMQ in the first place, is there an out of the box client retry available in WCF or do I have to implement a client queue and retry logic in my client code?
Thanks
It's a highly unlikely scenario that your messages sent to the service will not even be placed in the client queue in the first place, if you have MSMQ server running on the client station and the MSMQ listener service is up and running you should have nothing to worry about. I don't think MSMQ offers anything to check this for you, you should code some method on your client to periodically Peek() the local queue and send an acknowledgment receipt for every message that has reached the queue, this is feasible since you can easily access your local queues in code and also every message sent via MSMQ from a client to a service will always go trhough the local queue. You can also tell that the message reaches the queue if your Send() method desn't return an error. But I don't think you really need to worry about message son the client not reaching the local queue first.
We are using MSMQ right now with WCF activation feature, it enables us not to pull queue to read messages. It like push message to application.
As we are looking at porting from MSMQ to RabbitMQ going through what we need from message queue.
I can't anything regarding RabbitMQ .net client support for receiving message notification from subscribed queue?
Is there anything in RabbitMQ with .net which can do push notification to subscriber like MSMQ?
Or we need service running which constantly checks for message?
In AMQP (and RabbitMQ), there are two ways to retrieve messages: basic.get and basic.consume.
Basic.get is used to poll the server for a message. If one exists, it is returned to the client. If not, a get-empty is returned (the .NET method returns null).
Basic.consume sets the consumer for the queue. The broker pushes messages to the consumer as they arrive. You can either derive DefaultBasicConsumer, which gives you your own custom consumer, or you can use the Subscription Message Pattern, which gives you a blocking nextDelivery().
For more information, check out the API guide linked above and the .NET Client Userguide. Also, a great place to ask RabbitMQ-related questions is the rabbitmq-discuss mailing list.
I think you are after something like the EventingBasicConsumer. See also this question/answer
That is a feature provided by WAS (Windows Activation Service). Right now WAS has listener adapters for net.pipe, net.msmq and net.tcp (and its port sharing service). I guess you would need a specific AMQP listener adapter.
This may help http://msdn.microsoft.com/en-us/library/ms789006.aspx
I need to create a service which can process queued requests on a configured time interval. For example go to the web and get financial data from a site the requires we limit requests to once per second. I am new to WCF and I am not sure if (1) WCF with MSMQ a proper choice for implementing this? and (2) if so what is the best mechanism for enforcing the interval? a thread wait? a timer (not sure how that would work).
There's nothing built into WCF that would allow you to handle this explicitly, so you'd still need to do all the work yourself.
While your service could certainly process requests from MSMQ, the MSMQ listeners in WCF will pick and process messages as soon as possible; you can't configure them to process messages every X seconds only (you could fake it given the right tools, but seems to me it wouldn't be all that great).
One option if your delay between processing requests isn't very short, would be to use an intermediate queue to hold pending requests. That is, whatever sends the real requests writes them to a queue nobody is directly listening to (queue A), while your WCF service listens on a differet queue (queue B). Then, have something else (could be as simple as a script run from task scheduler) that runs once every X seconds/minutes/whatever and moves just 1 message from queue A to queue B, thus triggering the actual WCF service to run.
WCF and MSMQ are a great team! Definitely worth checking out.
The part that WCF doesn't provide out of the box is the "check every x seconds". The best approach here would be to host your WCF service inside a Windows NT Service, and have a timer inside the NT Service that goes to check the MSMQ queue only once every x seconds. Shouldn't be too hard to implement, really. The beauty is: you can very easily self-host a WCF Service inside a NT Service - just a few lines of code, and you get complete control over what's happening, and when. See the MSDN docs on How to Host a WCF service in a managed application for details.
Resources:
Tom Hollander's blog post series on MSMQ, WCF, IIS: Getting them to play nice
Motley Queue: MSMQ and WCF Getting Started
SOAizing MSMQ with WCF (and why it's worth it)
Or you could just use a window service to consume the messages instead. If you are not using the WCF functionality of consuming a message as soon as it is posted, then you probably have no reason to use wcf in the first place