I have a bunch of message on an outgoing MSMQ that have been addressed incorrectly due to a typo. They are stuck on the outgoing queue since the box they have been directed at does not exist.
The name of the queue is of the form
DIRECT=OS:foo\private$\MyQueue and is in state Waiting to connect
and I want to send to
DIRECT=OS:bar\private$\MyQueue
Will an entry in the hosts file suffice, or if not is there any other way that I can go this? I don't want to have to create a server called and a process that recieves the messages and forwards them on to the correct server if at all possible
I tried the idea I suggested above of adding the incorrect entry into the Hosts file with the correct IP address. This caused the messages to get dispatched to the remote computer, but I think WCF dropped them as they didn't have the correct endpoint adress.
Not recommended if you don't want to lose the messages! Fortunately for me these were not critical, so it was worth a try.
Oh well
Modan,
I experienced something similar years ago and the only way I could find was to setup the box and Q (in a VM) and forward them to correct queue. Maybe you can delete and resend the messages with the fix? Is that possible? Unfortunately don't think there's a good way around this.
-Bryan
Related
I'm getting an strange behavior in my DEV environment. When I send a message to my queue it's dispatched correctly, but the next message (can be same content or different) always fails and is directly sent to my deadletter queue. And then this pattern is repeated, one OK, one sent to deadletter.
In my local setup, everything is working OK, but not in my DEV env so this makes a little difficult to debug/troubleshoot. Not sure what could be wrong or different. I'm new on RabbitMq so maybe I need to include more information (if so please let me know).
Does anyone have an idea of what could be causing it? Or does anyone have experienced something like this before?
RabbitMq Version is: 3.8.2
My rabbitmq.config file is:
[{rabbitmq_management,[{tcp_config,[{port,15672}]}]}, {rabbit,[{total_memory_available_override_value,3999997952}, {tcp_listeners,[5672]}, {loopback_users,[]}]}].
My two queues are configured this way:
**my-queue.dev**
Type: Classic
Features: D, DLX
**my-queue.dev.deadletter**
Type: Classic
Features: D
Kind regards!
We have an environment for three instances of BizTalk 2016 that are used randomly by the messages.
I would need to know which one has been used when I send a message, but only if it has had any error and it is shown in the Windows logs-->Application is said which computer was used.
But I'd need to know which computer was used when the message was fine, but in the 'Tracked Message Events' this information is not shown and the flow of the message is displayed in the three instances.
Any idea of how I could get this information?
The fact of the matter is that it could be anyone of the three servers within your BizTalk Group. What you could do is track the properties and in the send pipeline promote the machine name. Although I'm not sure why you would need to do this.
Please remember that a send port can be triggered by any server with a particular host instance configured and running and that retry attempts may even be done on other server(s) as long as the triggered host instance is running there.
My question is two-fold:
First, in Redis, is it possible to have multiple publishers to publish messages to the same channel?
And second, if the answer to the first part is yes (which I think it is), is it possible to tell (on the subscriber end) which publisher has sent any given message?
My scenario is that I've got a server which sends events to Redis right now. And I would like to have multiple instances of it and collect all their events. I was wondering if it is possible to centralize their events in one Redis while being able to tell apart the message sources without changing the publisher code. I mean one solution is to have each server include some ID in the message but that requires changing the code which I prefer not to do.
First, in Redis, is it possible to have multiple publishers to publish messages to the same channel?
Yes. You can even easily test it!
And second, if the answer to the first part is yes ...
The message doesn't include the publisher, but the publisher can add its name to the message explicitly. For that you'd have to change the actual code that calls PUBLISH from your application - no two ways about it.
I'm new to NServiceBus, but currently using it with SQL Server Transport to send messages between three machines: one belongs to an endpoint called Server, and two belong to an endpoint called Agent. This is working as expected, with messages sent to the Agent endpoint distributed to one of the two machines via the default round-robin.
I now want to add a new endpoint called PriorityAgent with a different queue and two additional machines. While all endpoints use the same message type, I know where each message should be handled prior to sending it, so normally I can just choose the correct destination endpoint and the message will be processed accordingly.
However, I need to build in a special case: if all machines on the PriorityAgent endpoint are currently down, messages that ordinarily should be sent there should be sent to the Agent endpoint instead, so they can be processed without delay. On the other hand, if all machines on the Agent endpoint are currently down, any Agent messages should not be sent to PriorityAgent, they can simply wait for an Agent machine to return.
I've been researching the proper way to implement this, and haven't seen many results. I imagine this isn't an unheard-of scenario, so my assumption is that I'm searching for the wrong things or thinking about this problem in the wrong way. Still, I came up with a couple potential solutions:
Separately track heartbeats of PriorityAgent machines, and add a mutator or behavior to change the destination of outgoing PriorityAgent messages to the Agent endpoint if those heartbeats stop.
Give PriorityAgent messages a short expiration, and somehow handle the expiration to redirect messages to the Agent endpoint. I'm not sure if this is actually possible.
Is one of these solutions on the right track, or am I off-base entirely?
You have not seen many do this because it's considered an antipattern. Or rather one of two antipatterns.
1) Either you are sending a command, in which case the RECEIVER of the command defines the contract. Why are you sending a command defined by PriorityAgent to Agent? There should be no coupling there. A command belongs to ONE logical endpoint/queue.
2) Or you are publishing an event defined by whoever publishes, with both PriorityAgent and Agent as subscribers. The two subscribers should be 100% autonomous and share nothing. Checking heartbeats/sharing info between these two logical separate entities is a bad thing. Why have them separately in the first place then? If they know about each other "dirty secrets," they should be the same thing.
If your primary concern is that the PriorityAgent messages will not be handled if the machines hosting it are down, and want to use the machines hosting Agent as a backup, simply deploy PriorityAgent there as well. One machine can run more than one endpoint just fine.
That way you can leverage the additional machines, but don't have to get dirty with sending the same command to a different logical endpoint or coupling two different logical endpoints together through some back channel.
I'm Dennis van der Stelt and I work for Particular Software, makers of NServiceBus.
From what I understand, both PriorityAgent and Agent are already scaled out over multiple machines? Then they both work according to competing consumers pattern. In other words, both machines try to pick up messages from the same queue, where only one will win and starts processing the message.
You're also talking about high availability. So when PriorityAgent goes down, another machine will pick it up. That's what I don't understand. Why fail over to Agent, which seems to me to be a logically different endpoint? If it is logically different, how can it handle PriorityAgent messages? If it can handle the same message, it seems logically the same endpoint. Then why make the difference between PriorityAgent and Agent?
Besides that, SQL Server has all kinds of features (like Always-On) to make sure it does not (completely) go down. Why try to solve difficult scenarios with custom build solutions, when SQL Server can already solve this for you?
Another scenario could be that PriorityAgent should handle priority cases. Something like preferred customers, or high-value customers. That is sometimes used when (for example) a lot of orders (read: messages) come in, but we want to deal with high-value customers sooner than regular customers. But due to the amount of messages coming in, high-value customers would also end up in the back of the queue, together with regular customers. A solution could be to publish these messages and have two different endpoints (with different queues) subscribed both to this message. Both receive each unique message, but check whether it's a message they should handle. The Agent will ignore high-value customers, the PriorityAgent will ignore regular customer.
These are some of the solutions available as standard messaging patterns, or infrastructural solutions to solving your issue. Again, it's not completely clear to me what it is you're looking for. If you'd like to continue the discussion; perhaps you want to email support#particular.net and we can continue the discussion there.
TL;DR: I need to "replay" dead letter messages back into their original queues once I've fixed the consumer code that was originally causing the messages to be rejected.
I have configured the Dead Letter Exchange (DLX) for RabbitMQ and am successfully routing rejected messages to a dead letter queue. But now I want to look at the messages in the dead letter queue and try to decide what to do with each of them. Some (many?) of these messages should be replayed (requeued) to their original queues (available in the "x-death" headers) once the offending consumer code has been fixed. But how do I actually go about doing this? Should I write a one-off program that reads messages from the dead letter queue and allows me to specify a target queue to send them to? And what about searching the dead letter queue? What if I know that a message (let's say which is encoded in JSON) has a certain attribute that I want to search for and replay? For example, I fix a defect which I know will allow message with PacketId: 1234 to successfully process now. I could also write a one-off program for this I suppose.
I certainly can't be the first one to encounter these problems and I'm wondering if anyone else has already solved them. It seems like there should be some sort of Swiss Army Knife for this sort of thing. I did a pretty extensive search on Google and Stack Overflow but didn't really come up with much. The closest thing I could find were shovels but that doesn't really seem like the right tool for the job.
Should I write a one-off program that reads messages from the dead letter queue and allows me to specify a target queue to send them to?
generally speaking, yes.
you could set up a delayed re-try to resend the message back to the original queue, using a combination of the delay message exchange plugin.
but this would only automate the retries on an interval, and you may not have fixed the problem before the retries happen.
in some circumstances this is ok - like when the error is caused by an external resource being temporarily unavailable.
in your case, though, i believe your thoughts on creating an app to handle the dead letters is the best way to go, for several reasons:
you need to search through the messages, which isn't possible RMQ
this means you'll need a database to store the messages from the DLX/queue
because you're pulling the messages out of the DLX/queue, you'll need to ensure you get all the header info from the message so that you can re-publish to the correct queue when the time comes.
I certainly can't be the first one to encounter these problems and I'm wondering if anyone else has already solved them.
and you're not!
there are many solutions to this problem that all come down to the solution you've suggested.
some larger "service bus" implementations have this type of feature built in to them. i believe NServiceBus (or the SaaS version of it) has this built in, for example - though I'm not 100% sure of it.
if you want to look into this further, do some search for the term "poison message" - this is generally the term used for this situation. I've found a few things on google with a quick search, that may help you down the path:
http://lists.rabbitmq.com/pipermail/rabbitmq-discuss/2013-January/025019.html
https://web.archive.org/web/20170809194056/http://tafakari.co.ke/2014/07/rabbitmq-poison-messages/
https://web.archive.org/web/20170809170555/http://kjnilsson.github.io/blog/2014/01/30/spread-the-poison/
hope that helps!