Hope someone can help here; I can't be the only one who has had this issue with Glassfish Open Message Queue :)
In short, when a message ends up in the mq.sys.dmq (The default dead message queue for Open MQ), whether the message is then deleted, resolved or simply left in the dead message queue, on restarting the server - domain1 in my case - the message ends back up in the original destination queue. This only happens once. The next time round it disappears. At times I basically have 2 messages, one in the dead message queue - the original - and a new clone back in the original destination queue.
Could someone please tell me if I'm missing something blindingly obvious, like a something I should be setting on the message on being sent from my client?
Thanks in advance.
Related
I'm trying to use RabbitMQ in a more unconventional way (though at this point i can pick any other message queue implementation if needed)
I have one queue (I can have more if needed) that where customers are fetching N messages asynchronous. After they do their work I send the results from the client to the db.
I have two problems: first I don't want that they will work on the same message, second I want to grantee that I wont lose messages in case that my customer will close the browser or just stop working.
I looked at the documentation and saw the TTL which was perfect for me if I could alter that message that got timeout isn't going to be deleted but to move to another queue. can't find a way to alter this.
Moreover I looked at the confirmation option which in the first glance looked what I wanted,that mechanism is working like this: when the consumer gets a message he send confirmation to queue, I thought I can delay this confirm and send it when the work is done on the client side.
my problem was that I can't program the queue that if any message didn't get confirm then return it to the queue (or to another).
I also find how to do a scheduled message but it didn't help either because I don't want that the message will be inserted to the queue in five min,I want that when a customer will receive a message it will be locked in the queue for 5 min until confirm to delete is set otherwise return it to the queue.
Can I do temporary queue that enables my mechanism?
If someone can help with one of the problems or suggest another architecture or option to do it in another MQ it would be great.
Resources:
confirmation:
http://www.rabbitmq.com/blog/2011/02/10/introducing-publisher-confirms/
post about locks but his problem was a batcher component:
Locks and batch fetch messages with RabbitMq
TTL:
https://www.rabbitmq.com/ttl.html
Schedule a message:
https://www.rabbitmq.com/blog/2015/04/16/scheduling-messages-with-rabbitmq/
my problem was that I can't program the queue that if any message
didnt get confirm then return it to the queue (or to another).
RabbitMQ does this anyhow, so all you have to do is switch off the auto-ack flag, you figured this out
I thought I can delay this confirm and send it when the work is done
on the client side.
so just send the ACK once you've finished with processing the message.
All the unacknowledged messages remain in the queue and are re-delivered to next consumer (or the same one when it's up again, depending on your setup)
I have this issue, I want to know my rabbit is working great.
I am not gonna send the message, so, Im not 100% sure is being sent correctly. But the problem is this.
After all is configured and all....
I see at the RabbitMQ web manager
And when I supposedly send a message the I see activity on the "message rates" chart but nothing at the "queued messages" .
I frankly dont know whats going on, is it too fast that doesnt need to queue the messages? Or something is misconfigured?
Any idea of the difference?
Thanks.
In case RabbitMQ receive non-routable message it drop it. So while message was received, it was not queued.
You may configure Alternate Exchanges to catch such messages.
In my case,
Situation1:
when my Exchange in rabbitTemplate.convertAndSend was not set properly -- the message was not sent to the correct queue -- the Queued messages was empty all time.
however, Message rates is not zero, it does show there are message get sent.
Which correspond to what the other answer is saying:
In case RabbitMQ receive non-routable message it drop it.
Situation2:
when my Exchange in rabbitTemplate.convertAndSend was indeed set properly -- the message was sent to the correct queue -- the Queued messages was queuing up the message.
Everything seems fine.
Situation3:
(continue from Situation2)
And now, I turn on the receiver service which has the #RabbitListener.
The Queued messages immediately drops down to 0, and never goes up again.
But the transporting of messages is still working fine.
Situation4:
(continue from Situation2)
And now, I change the receiver service to use the rabbitTemplate.receiveAndConvert.
Which I manually receive the message from the queue every 2s by using a loop.
(message is also sent from sender service every 2s by using a loop, same as the situations before.)
Now, the Queued messages stays at constant -- a straight line
(depends on how many message you have queued up, in my case 1, before the receiver service is up, then it stays at 1).
Conclusion:
I suspect that, when the message is consumed too fast, the Queued messages will just show 0.
Which correspond to what the OP is saying:
is it too fast that doesnt need to queue the messages?
(or, I could screw up some setting in RabbitMQ and led to wrong conclusion. I dont think so, but idk, I am not familiar with RabbitMQ.)
This seems like a pretty basic question, but I seem to be losing messages when the consumer falls over before acknowledging them. I have set up the broker with an exchange audit:exchange and a queue bound to it audti:queue. Both are durable, and as expected if I send messages when no consumer is active they sit on the queue and get processed by the consumer when it starts up. However if I put a break point in the consumer and kill the process half way through, the message is not requeued - it just seems to get lost. The consumer is set up using the annotation
#RabbitListener(queues="audit:queue")
public void process(Message message) {
routeMessage(message) //stop here and kill process - message removed from q
}
I can't reproduce your issue.
With the breakpoint triggered, I see the message still in the queue (unacked=1) on the rabbit console.
When the process is killed; the message goes back to ready.
Have you configured the listener container factory to use Acknowledgemode.NONE?
That will exhibit the behavior you describe.
The default is AUTO which means the message will only be acknowledged when the listener returns successfully.
If you still think there's an issue; please supply the complete test case.
Sorry this was my bad (I just wasted a few hours .. sigh). I was killing the app from within my ide. Which probably detaches and then kills the process - allowing time for it to proceed just enough that it actually does send the ack. When I just killed the process from a terminal it worked exactly as expected. Particualr apologies to you Gary for wasting your time as well.
I am new in activeMQ, i want to be sure that the consumed message is processed normally and nothing is wrong, and if something gets wrong i don't want to lose the message, in this case what should i do? should i save the messages to a database? is there any way in activeMQ to keep the messages in the queue if something is wrong? i tried to save the messages to mysql database and then called the consumer manually to read from the database, it works but i am afraid in future to have a big record in the database and reading to be slow.
You are looking for a 'Dead Letter Queue' or DLQ to be precise.
Whenever during message processing ,if something goes wrong that message is automatically transferred to Default DLQ provided by ActiveMQ, the queue is named activemq.DLQ.
The messages you send should be persistent messages, so that they are stored in DLQ.
for more reference you can have a look here
http://activemq.apache.org/message-redelivery-and-dlq-handling.html
hope this helps!
Good luck!
I am using RabbitMQ version 3.0.2 & I see close to 1000 message in Error queue. I want to know
At what point messages are moved to the error queues?
Is there a way to know why a certain message is being moved to an error queue?
Is there any way to move message from error queue to normal queue?
Thank you
a) they fail to deserialize or b) the consumer throws an exception processing that message five times
Not really... If you peek at the message in the queue, the payload headers might contain a note but I don't think we did that. If you turn logging on (NLog, log4net, etc) you should be able to see the exceptions in your log. You'll have to correlate message ids at that point to figure out exactly why.
There is no built in way via MassTransit. Mostly because there doesn't seem to be a great, generic way to handle this. Everyone wants some process around this. Dru did create a BusDriver app (in the main MT source repo) that could be used to move messages back to the exchange in question. This default behaviour is there so you at least know things have been failing if you don't put in the infrastructure to handle it.
To add to Travis' answer, During my development I found some other reasons for messages going onto the error queue:
The published message type has no consumer
A SAGA and a consumer are expecting the same concrete message type. Even if you try and differentiate using "Accepts" and ".Selected", both a SAGA and a Consumer should not be programmed to receive the same message type.