Cannot find MSMQ message at remote computer - wcf

I have a WCF service that is set up to use MSMQ to transmit to a service on another machine. We are trying to move the client onto a different machine, but it's not working. Enabling the MSMQ.End2End event log gives us
Message with ID {6940f8fa-3d31-4db0-ae2b-59bc98c99f2c}\25321 was sent to queue DIRECT=OS:iisapp1-vvpm\private$\TransactionalEmailService/TransactionalEmail.Service.TransactionalEmailService.svc
which makes me think that it is working correctly from our machine, but we can't find any trace of it on the target machine. The service is not being invoked, and we can't find the message in the dead-letter queue (or anywhere else we can think of to look).
Also, running the code directly from Visual Studio on my machine causes it to work.
Changing the receiving queue to the DEV machine also causes the code to work, which makes me further think it's a problem with the receiving machine. (I just have no idea what)
UPDATE 1:
I came back to it and noticed all the messages I tried to send in the transactional dead-letter queue. The error message is "the time-to-reach-queue has elapsed". Looking at the connection state, it's inactive, and sending another message won't cause it to become connected. I restart the machine, and it is "Connected" again. I try to send the message again, and look at the queue state. There are 12 messages, all of which are unacknowledged (0 are unprocessed).

So it started happening again once the the endpoint machine got restarted. I came across this article, which was the real solution:
http://blogs.msdn.com/b/johnbreakwell/archive/2007/02/06/msmq-prefers-to-be-unique.aspx
( In case the link goes away: )
It looks like there is an ID in MSMQ that is sent as part of the message. The remote host uses that ID as a key into a cache to determine who to send the ack back to. If you clone a machine, it gets the same value in the registry for that ID, which makes the other machine not know who to send it back to. So it will send it to the wrong machine, which will discard it, and the other machine will be stuck with a bunch of messages it keeps trying to send. This also explains why it just started working one day... the cache expired and the "correct" machine got put in as the endpoint.
Reinstalling MSMQ on the cloned machine fixes the issue.

I'm really not sure this may be the case here (I don't have any experience with WCF in the context of MSMQ), but one of the more common reasons of this kind of behaviour you're talking of is missing the obligatory camel casing on FormatName in your queue name when using a MessageQueue constructor (like 'FormatName:DIRECT=...'), or getting the name somehow wrong. The queue name in the message looks a bit odd with the svc ending, but that could just be a WCF thing? Hope this helps at least getting you on the right direction.

Not sure what to say here, but it works now. Reading some stuff helped point me to the status of the queue (click on Outgoing Queues under Features/Message Queueing). From there I found this KB article with a hotfix: http://support.microsoft.com/kb/976438 It didn't seem like it was applicable, but the symptoms people were having were all the same. Our guys tried to install it, but it failed and they didn't restart... but for some reason the message queues started working.
If someone comes along with some insight, I'll gladly upvote them or give them the bounty (if it's soon enough). But I'll just accept this as the answer for now.

This is usually caused by permissions on the remote queue, usual scenario is if you are using a private queue and this is accessed remotely by your wcf service.
Try using a public queue.

Related

How to use NServiceBus with MSMQ

I am experimenting the new version of NServicsBus. I find following step by step sample on particular site.
https://docs.particular.net/samples/step-by-step/
Can any one tell me how to configure MSMQ for Transport. Here is my scenario.
Client create message
Client message should be stored in MSMQ
Server Application running on same machine which subscribe the message.
Server handler get message from MSMQ and process it further. i.e Store in DB or send to other web service.
Retry to process message if it does not worked first time
after 3 retries send message to error queue
How do i configure this sample to use MSMQ for my scenario.
Helpful information to include
Product name:NServiceBus.Core
Version: 6.3.4
Stacktrace:
Description:
Did you know that we have released a LearningTransport and LearningPersistence just for purposes like these? Have a look at it here.
Having said that, the transport swapping should be rather seamless so even if you have setup a small PoC using this transport/persistence, you can change it to MSMQ or other production-ready transports/persistence when you go live.
Again, as stated in the documentation page and as the name suggests, this is not for use in production.
I would recommend you walk through this.
https://docs.particular.net/tutorials/intro-to-nservicebus/
Will answer your questions, and future ones you have.

WCF Service Not Reading Messages in Queue

I have a WCF service that reads from a message queue. I noticed with our latest deployment that there is a message sitting in the queue that has not been read. The WCF service is up and running and it is correctly establishing a connection to the queue (or I would receive and error).
We made some changes to our code recently and we are wondering if perhaps that is related. It seems odd to us that the service would be running and yet the message not read. At a minimum, I would expect WCF to throw some sort of error if there was a malformed message in the queue.
I looked at the properties of the message and it says there have been zero moves, so I don't think it is being sent to a retry sub-queue. It is just sitting there and the service won't read it.
Is there a circumstance where WCF would ignore a message in the queue? How does WCF handle malformed messages?
Sorry if this isn't a lot of information to go on. At this point I am just trying to understand what is preventing the message from being processed.
This situation seemed just too odd. It didn't make sense that WCF would do nothing at all.
I eventually decided to simply try restarting the MSMQ service on the server. Once I did that, my service immediately picked up the messages and everything started working again.
I have no idea if this had something to do with a Windows Update or some other server change. I am glad it was this easy to fix - it will be the first thing I try in the future.

Masstransit: Can it use central msmq server? (Or should I start w/RabbitMQ from the start?)

I set up the Masstransit sample apps, and all was great. Local operation, msmq, looks good.
Now I am starting to put masstransit in my real app. In my real app, I have jobs coming from four servers, and processing happening on two worker systems.
It seems that masstransit always wants to push to:
msmq://localhost/...
But I thought I would set up a single, central, msmq server: msmq:///...
It appears (I may be missing something! Please correct me if I am off!) that when using msmq, that I need to set up msmq on multiple machines, and configure msmq to route from machine to machine.
Am I missing something?
Should I skip msmq and jump to rabbitmq right off, (which appears to solve for this)?
Is there some fundamental msmq knowledge (that is perhaps not in the masstransit docs) ?
thank you!
First off I always suggest people use RabbitMQ over MSMQ unless you MUST use DTC for some reason. And even then, I'd suggest you rethink using DTC.
But given you have some constraint you can't fight. You're welcomed to use a central MSMQ server but it doesn't provide a ton of value. Each server sending messages must have MSMQ installed locally because of how it works. Messages actually end up in an outgoing queue before they are sent over the other machine in question. If you have multi-machine MSMQ setups, in the past for me it's been like:
Core Machine runs MassTransit.RuntimeServices at /mt_subscriptions, and maybe one service at /service_1
Other processing machine runs a specific heavy load service at /service_2 and it's configuration references msmq://coremachine/mt_subscriptions for the subscription service.
Yet another processing machine with similar setup
So with those 3 machines, the only thing you don't have msmq://localhost/ is the reference to the subscription service in configuration.

Can't read from remote transactional private queue using WCF in workgroup mode (can do using System.Messaging !)

I have spent days reading MSDN, forums and article about this, and cannot find a solution to my problem.
As a PoC, I need to consume a queue from more than one machine since I need fault tolerance on the consumers side. Performance is not an issue since less than 100 messages a day should by exchanged.
I have coded two trivial console application , one as client, the other one as server. Using Framework 4.0 (tested also on 3.5). Messages are using transactions.
Everything runs fines on a single machine (Windows 7), even when running multiple consumers application instance.
Now I have a 2012 and a 2008 R2 virtual test servers running in the same domain (but don't want to use AD integration anyway). I am using IP address or "." in endpoint address attribute to prevent from DNS / AD resolution side effects.
Everything works fine IF the the queue is hosted by the consumer and the producer is submitting messages on the remote private queue. This is also true if I exchange the consumer / producer role of the 2012 and 2008 server.
But I have NEVER been able to make this run, using WCF, when the consumer is reading from remote queue and the producer is submitting messages localy. Submition never fails, my problem is on the consumer side.
My wish is to make this run using netMsmqBinding, but I also tried using msmqIntegrationBinding. For each test, I adapted code and configuration, then confirmed this was running ok when the consumer was consuming from the local queue.
The last test I have done is using WCF (msmqIntegrationBinding) only on the producer (local queue) and System.Messaging.MessageQueue on the consumer (remote queue) : It works fine ! => My goal is to make the same using WCF and netMsmqBinding on both sides.
In my point of view, I have proved this problem is a WCF issue, not an MSMQ one. This has nothing to do with security, authentication, firewall, transport, protocol, MSMQ version etc.
Errors info using MS Service Trace Viewer :
Using msmqIntegrationBinding when receiving the message (openning queue was ok) : An error occurred while receiving a message from the queue: The transaction specified cannot be imported. (-1072824242, 0xc00e004e). Ensure that MSMQ is installed and running. Make sure the queue is available to receive from.
Using netMsmqBinding, on opening the queue : An error occurred when converting the '172.22.1.9\private$\Test' queue path name to the format name: The queue path name specified is invalid. (-1072824300, 0xc00e0014). All operations on the queued channel failed. Ensure that the queue address is valid. MSMQ must be installed with Active Directory integration enabled and access to it is available.
If someone can help to find why my configuration cannot be handled by WCF, a much elegant and configurable way than Messaging, I would greatly appreciate !
Thank you.
You may need to post you consumer code and config to give more of an idea but it could be the construction of the queue name - e.g.
FormatName:DIRECT=TCP:192.168.0.2\SomeQueue
There are several different ways to connect to a queue and it changes when you are remote or local as well.
I have found this article in the past to help:
http://blogs.msdn.com/b/johnbreakwell/archive/2009/02/26/difference-between-path-name-and-format-name-when-accessing-msmq-queues.aspx
Also, MessageQueue Constructor on MSDN...
http://msdn.microsoft.com/en-us/library/ch1d814t.aspx

How to debug ActiveMQ client?

I'm a fairly new user of ActiveMQ and I'm looking for a way to get detailed debug information on the client side of a queue connection. My problem is this: I have a server that is sending a message through a queue to a client. Using the admin web page associated with the broker, I can verify the following: the queue was created, there is a consumer associated with the queue, the message has been enqueued, the message has been dispatched, the dispatched queue size is 1, the message has not been dequeued. This setup was working yesterday but mysteriously stopped working today even though I did a restart of the activemq service. The log file at /var/log/activemq.log does not contain any useful information.
At this point I'm stumped; I'm assuming that there is some sort of problem with the configuration, but it hasn't changed since yesterday. Does anybody have a suggestion about what my next step should be?
Turn on debug (or even trace) logging in the broker first of all in conf/log4j.properties.
log4j.logger.org.apache.activemq=DEBUG
restart the broker and re-run your scenario. The logging will hopefully provide you with some information.
Jconsole is also a useful tool to monitor the running broker.
Does your client use any message filters?
You can also enable remote debugging and then connect with an IDE.
To start remote debugging execute
$ ACTIVEMQ_DEBUG=true bin/activemq
and then start a remote debugger to connect to port 5005