I'm hosting NServiceBus in my own application to act as a subscriber.
I have 4 projects in the solution:
1. Contracts - declare the event interfaces
2. Host - class library with API to start the bus.
3. Handlers - here the event handlers are implemented.
4. Console application to run it all.
I see that the endpoint name is set correctly according to the console application name which is what I want and the queues are created accordingly.
I successfully subscribe to the publisher events.
The problem:
When the publisher tries to send a message to the subscriber - it tries to send to a queue that is named according to the event handlers namespace and not the endpoint name.
The exception that I get is that the publisher could not find the subscribers input queue.
Just for a sanity check, I manually created the input queue that is named according to the handlers namespace and indeed I started to receive the events.
So, is this a bug in NServiceBus or have I missed something very crucial?
Thanks....
I found the problem and it was mine...
The publisher still had old subscribers in its Raven DB so it tried to publish the events also to these queues which weren't there anymore...
To make my life easier, I configured the subscriptions to be stored using MSMQ.
I had a similar issue. I renamed my endpoints but when calling Publish() it was still trying to send to the old queues. I went to localhost:8080 (RavenDB) and deleted all documents and databases but still had the same issues. Restarting the RavenDB service solved the issue, so it must cache them in memory or something.
Related
Tech Stack:
.Net 4.6.1
Erlang 18.3
Language: C# 6
NServiceBus 6
RabbitMQ 3.6.3
Windows 7
Context:
We are trying to implement a dumb event publisher with smart subscription in NServiceBus on the RabbitMQ transport.
All interested receivers subscribe to an event.
A Publisher publishes the event to an event channel.
All subscribers receive a copy of the event.
Note that I did not say each TYPE of subscriber receives a copy of the message. If there are multiple instances of a service running, and they all have active subscriptions to an event, each INSTANCE of the subscribing service should get a copy of the message.
However, NServiceBus' notion of Publish-Subscribe delivers a published event to one and only one receiver on a given channel. In our case, one instance of a given subscribed service.
I hesitate to list messaging "patterns" because they don't seem to be named or described particularly consistently. However, I believe we are esentially trying to implement the "Multicast" version of the Publish-Subscribe Channel messaging pattern from the Enterprise Integration Patterns (Hohpe and Woolf) Book.
Problem Statement:
Our business case is this:
We have a configuration service that provides application configuration to all other services in our broader application.
Each service requests its configuration from the configuration service on startup by sending a ConfigurationRequest message.
The configuration services replies to the specific instance that made the ConfigurationRequest with a ConfigurationResponse. This is done using NServiceBus' Full-Duplex (commonly called Request-Response) feature.
There is a website that can modify configuration globally. When it does so, it notifies the configuration service with an UpdateConfiguration command.
The configuration service publishes a ConfigurationUpdated event that all other services are subscribed to.
Each service can have multiple instances running on more than one server. ALL instances of a service need to update their configuration, not just one instance.
Each service instance is calling NServiceBus' EndpointConfiguration.MakeInstanceUniquelyIdentifiable with a distinct discriminator.
Currently when we publish the ConfigurationUpdated event, only one instance of each service type gets the message. The events are distributed round-robin, instead of each instance getting a copy of the message.
We have worked around this problem by keeping a record of running services instances and sending the ConfigurationUpdated event (as a Command in NServiceBus) to each of them individually, but Pub-Sub implies that we should have dumb publishers and smart subscribers, and our workaround is the opposite... Our publisher looks up a listof each subscriber and sends to it implicitly. Is there some configuration I am missing on the subscriber side that would allow each instance of each service to get a copy of the published ConfigurationUpdated event? If not, where should I look in NServiceBus to start implementing such a feature? Routing topology, perhaps?
i'm not familiar enough with nservicebus to know how to do it with that toolset, but the RabbitMQ implementation would be a "fanout" exchange with a queue per consumer.
Every queue bound to a fanout exchange will get a copy of the message. If there is a single consumer for each of those queues, then you will be sending a copy of the message to each of the consumers.
It sounds like you have multiple consumers connected to the same queue. Maybe there's a way to tell nservicebus to create a queue for each consumer instance?
p.s. you're right about pub-sub pattern being multicast. i talk about this in my RMQ Patterns ebook (https://leanpub.com/rmq-patterns) and that's what the EIP book would say, as well.
Does NServiceBus, at any point, for any reason, have to post empty messages to MSMQ, and if so, why and when does it happen? Longer explanation below.
A project I work on makes use of NServiceBus version 4. That version does not allow "multi-hosting" of event handlers for different queues in a single process, which may be inconvenient if your project contains 40 or so different queues.
To overcome this problem in development, I made a small "router" app, which listens to all the necessary MSMQ queues and simply forwards all messages from them into a single "unified" queue. That "unified" queue is specified as the queue name for the "unified endpoint" process, which references all the handlers for all the messages that would normally be handled from those various queues.
The setup kind-of works, it seems (with most handlers, at least), but there is one mysterious behaviour (which, I presume, may have something to do with the set-up not working with some other handlers). Namely, as soon as the project starts up, my "router" immediately discovers a number of empty MSMQ messages posted to the queues it has to listen to. Apparently, NSB is publishing those messages during start-up for some reason (and most probably the router is snitching them up before NSB has the chance to look at them again).
I am sure this is not an artefact of my implementation as this does not happen unless NSB is also started. I am curious about the reasons.
NServiceBus, by default, autosubscribes to all handled events if it knows the endpoint which publishes them. These empty messages you see might be the subscribe messages because they are being sent during the endpoint start-up phase.
The mechanics behind the subscribe messages are documented here. TL;DR for transports that do not provide publishing natively (e.g. MSMQ, Azure Storage Queues) NServiceBus emulates it using subscribe messages and internal subscription lists (storages).
You can verify this by checking the message intent header. If they are not subscribe messages, please share the complete list of headers of such a message for further investigation.
Right now we have a queuing system(activemq) which is storing the messages. And we have written a separate java application that will read the queue and then trigger a exe to do some processing. But we want to do away with this extra application that is linking our activemq and exe. So i want to know whether any queuing system houses a code which will help me run the exe without any extra code written by me.
Any inputs regarding which queuing system can get me this done will be greatly appreciated.
This isn't really how Message Brokers work. You could however embed a broker inside your own application or create a broker plugin to do something. In the end though the best way to do this is to create your own client that can implement your business logic and let the Message Broker do what it was designed to do, route message traffic.
If you want more of a 'push' solution rather than a producer-consumer solution (which sounds to me like you are) you could look into the use of WebSockets. That would be another way of dealing with messages.
As others have said it doesn't look like using a message broker is the solution you want if you don't want to have some additional middleware to provide asynchronous communication.
So you just need something to launch an EXE on message arrival?
Message Queuing Triggers
Just some additional information for you... Triggered applications are natively supported in IBM Websphere MQ via a Trigger Monitor application that runs as a service (in windows implementations) or a daemon (in UNixish implementations).
When a message arrives in a queue, the MQ software will generate another message ("Trigger" message and send it to the "Trigger" queue, which is being monitored by the Trigger Manager app. The app then starts the required application.
So your implementation of an "app to start an app" is not odd ball or strange at all.. IBM do it in their implementation. I see nothing wrong with your implementation and if you can integrate it tighter to activeMQ then you are on a winner.
What about IBM MQ's triggering feature ?
WebSphere MQ provides a feature that enables an application or channel to be started automatically when there are messages available to retrieve from a queue. A message is put to a queue defined as triggered. If a series of conditions are met, the queue manager sends a trigger message to an initiation queue.
https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.dev.doc/q026940_.htm
The MSDN documentation for the BrokeredMessage.Complete method (http://msdn.microsoft.com/en-us/library/microsoft.servicebus.messaging.brokeredmessage.complete.aspx) describes the method as this: "Completes the receive operation of a message and indicates that the message should be marked as processed and deleted or archived."
In my use of this method I've only seen the message deleted once it is processed. This is the one and only instance I've seen in the MSDN documentation, blogs, or anywhere else about Service Bus being capable of archiving old messages.
I could archive the message myself as part of my code that reads and processes a message and then marks it complete. But is it possible to make Windows Server Service Bus archive completed messages for me? If so, how do you turn on and configure this feature?
In case the difference matters, I am using the locally hosted Windows Server Service bus, not the Azure version.
No, Service Bus doesn't archive your messages. I'm going to follow up w/ the documentation folks on what that was supposed to express.
I have spent days reading MSDN, forums and article about this, and cannot find a solution to my problem.
As a PoC, I need to consume a queue from more than one machine since I need fault tolerance on the consumers side. Performance is not an issue since less than 100 messages a day should by exchanged.
I have coded two trivial console application , one as client, the other one as server. Using Framework 4.0 (tested also on 3.5). Messages are using transactions.
Everything runs fines on a single machine (Windows 7), even when running multiple consumers application instance.
Now I have a 2012 and a 2008 R2 virtual test servers running in the same domain (but don't want to use AD integration anyway). I am using IP address or "." in endpoint address attribute to prevent from DNS / AD resolution side effects.
Everything works fine IF the the queue is hosted by the consumer and the producer is submitting messages on the remote private queue. This is also true if I exchange the consumer / producer role of the 2012 and 2008 server.
But I have NEVER been able to make this run, using WCF, when the consumer is reading from remote queue and the producer is submitting messages localy. Submition never fails, my problem is on the consumer side.
My wish is to make this run using netMsmqBinding, but I also tried using msmqIntegrationBinding. For each test, I adapted code and configuration, then confirmed this was running ok when the consumer was consuming from the local queue.
The last test I have done is using WCF (msmqIntegrationBinding) only on the producer (local queue) and System.Messaging.MessageQueue on the consumer (remote queue) : It works fine ! => My goal is to make the same using WCF and netMsmqBinding on both sides.
In my point of view, I have proved this problem is a WCF issue, not an MSMQ one. This has nothing to do with security, authentication, firewall, transport, protocol, MSMQ version etc.
Errors info using MS Service Trace Viewer :
Using msmqIntegrationBinding when receiving the message (openning queue was ok) : An error occurred while receiving a message from the queue: The transaction specified cannot be imported. (-1072824242, 0xc00e004e). Ensure that MSMQ is installed and running. Make sure the queue is available to receive from.
Using netMsmqBinding, on opening the queue : An error occurred when converting the '172.22.1.9\private$\Test' queue path name to the format name: The queue path name specified is invalid. (-1072824300, 0xc00e0014). All operations on the queued channel failed. Ensure that the queue address is valid. MSMQ must be installed with Active Directory integration enabled and access to it is available.
If someone can help to find why my configuration cannot be handled by WCF, a much elegant and configurable way than Messaging, I would greatly appreciate !
Thank you.
You may need to post you consumer code and config to give more of an idea but it could be the construction of the queue name - e.g.
FormatName:DIRECT=TCP:192.168.0.2\SomeQueue
There are several different ways to connect to a queue and it changes when you are remote or local as well.
I have found this article in the past to help:
http://blogs.msdn.com/b/johnbreakwell/archive/2009/02/26/difference-between-path-name-and-format-name-when-accessing-msmq-queues.aspx
Also, MessageQueue Constructor on MSDN...
http://msdn.microsoft.com/en-us/library/ch1d814t.aspx