zeromq sub-pub mode, block and lost - block

I use zeromq sub-pub mode in my app, but I got a problem:
I started sub app and subscribed the message, and then started pub app later, and the pub app will not publish any message in half hour or longer, after this, the pub app will pub messages to the sub-app, but the sub-app was blocked there without any message or some of the messages are lost.
If someone knows what's the problem? Thank you in advance.

Ideally, you should start the publisher first, then the subscribers.
Also, read this section in the guide regarding scenarios where messages between PUB and SUB can get lost: http://zguide.zeromq.org/page:all#Getting-the-Message-Out
Without any additional information, or code snippets of what you're doing, it's hard to troubleshoot this issue.

Related

One message processed OK, Next message rejected and goes to deadletter

I'm getting an strange behavior in my DEV environment. When I send a message to my queue it's dispatched correctly, but the next message (can be same content or different) always fails and is directly sent to my deadletter queue. And then this pattern is repeated, one OK, one sent to deadletter.
In my local setup, everything is working OK, but not in my DEV env so this makes a little difficult to debug/troubleshoot. Not sure what could be wrong or different. I'm new on RabbitMq so maybe I need to include more information (if so please let me know).
Does anyone have an idea of what could be causing it? Or does anyone have experienced something like this before?
RabbitMq Version is: 3.8.2
My rabbitmq.config file is:
[{rabbitmq_management,[{tcp_config,[{port,15672}]}]}, {rabbit,[{total_memory_available_override_value,3999997952}, {tcp_listeners,[5672]}, {loopback_users,[]}]}].
My two queues are configured this way:
**my-queue.dev**
Type: Classic
Features: D, DLX
**my-queue.dev.deadletter**
Type: Classic
Features: D
Kind regards!

MassTransit with RabbitMQ: When is a message moved to the error queue

I am using RabbitMQ version 3.0.2 & I see close to 1000 message in Error queue. I want to know
At what point messages are moved to the error queues?
Is there a way to know why a certain message is being moved to an error queue?
Is there any way to move message from error queue to normal queue?
Thank you
a) they fail to deserialize or b) the consumer throws an exception processing that message five times
Not really... If you peek at the message in the queue, the payload headers might contain a note but I don't think we did that. If you turn logging on (NLog, log4net, etc) you should be able to see the exceptions in your log. You'll have to correlate message ids at that point to figure out exactly why.
There is no built in way via MassTransit. Mostly because there doesn't seem to be a great, generic way to handle this. Everyone wants some process around this. Dru did create a BusDriver app (in the main MT source repo) that could be used to move messages back to the exchange in question. This default behaviour is there so you at least know things have been failing if you don't put in the infrastructure to handle it.
To add to Travis' answer, During my development I found some other reasons for messages going onto the error queue:
The published message type has no consumer
A SAGA and a consumer are expecting the same concrete message type. Even if you try and differentiate using "Accepts" and ".Selected", both a SAGA and a Consumer should not be programmed to receive the same message type.

How to detect alarm-based blocking RabbitMQ producer?

I have a producer sending durable messages to a RabbitMQ exchange. If the RabbitMQ memory or disk exceeds the watermark threshold, RabbitMQ will block my producer. The documentation says that it stops reading from the socket, and also pauses heartbeats.
What I would like is a way to know in my producer code that I have been blocked. Currently, even with a heartbeat enabled, everything just pauses forever. I'd like to receive some sort of exception so that I know I've been blocked and I can warn the user and/or take some other action, but I can't find any way to do this. I am using both the Java and C# clients and would need this functionality in both. Any advice? Thanks.
Sorry to tell you but with RabbitMQ (at least with 2.8.6) this isn't possible :-(
had a similar problem, which centred around trying to establish a channel when the connection was blocked. The result was the same as what you're experiencing.
I did some investigation into the actual core of the RabbitMQ C# .Net Library and discovered the root cause of the problem is that it goes into an infinite blocking state.
You can see more details on the RabbitMQ mailing list here:
http://rabbitmq.1065348.n5.nabble.com/Net-Client-locks-trying-to-create-a-channel-on-a-blocked-connection-td21588.html
One suggestion (which we didn't implement) was to do the work inside of a thread and have some other component manage the timeout and kill the thread if it is exceeded. We just accepted the risk :-(
The Rabbitmq uses a blocking rpc call that listens for a reply indefinitely.
If you look the Java client api, what it does is:
AMQChannel.BlockingRpcContinuation k = new AMQChannel.SimpleBlockingRpcContinuation();
k.getReply(-1);
Now -1 passed in the argument blocks until a reply is received.
The good thing is you could pass in your timeout in order to make it return.
The bad thing is you will have to update the client jars.
If you are OK with doing that, you could pass in a timeout wherever a blocking call like above is made.
The code would look something like:
try {
return k.getReply(200);
} catch (TimeoutException e) {
throw new MyCustomRuntimeorTimeoutException("RabbitTimeout ex",e);
}
And in your code you could handle this exception and perform your logic in this event.
Some related classes that might require this fix would be:
com.rabbitmq.client.impl.AMQChannel
com.rabbitmq.client.impl.ChannelN
com.rabbitmq.client.impl.AMQConnection
FYI: I have tried this and it works.

NServiceBus pub/sub - where have my messages gone?

Well I've been doing this NServiceBus project for a while and once I got it working for PubSub I then spent the rest of the time on the actual workflow logic. However, I can see a serious issue which I want to get around (or rather learn how to handle correctly).
A publisher publishes a message to the storage queues of any subscribers as far as I understand. Great. But what happens when the subscriber isn't running (I've read other posts about this and they don't seem to be asking the same question).
Scenario - I get the publisher to Publish a message when no subscribers are running (attached/requested messages to be relayed to them).. I then find that.. the message is "gone" just simply isn't there! where did it go? Did the publisher say "hey, no one's subscribing to this, so I wont bother publishing it?", shouldn't it NOT do that and require at least one subscriber?
Can anyone shed any light on this? (nservicenewbie)
You should publish an event that has happened - a statement of fact, that other handler may or may not be interested in. It's perfectly valid to have zero subscribers! If this is not the case then maybe you should be Send()ing a command instead of Publish()ing an event.
If you are using a persistent subscription storage, start the subscriber up once and it will always be subscribed. If the subscriber is offline, messages for it will pile up in its Input Queue, ready to be processed when the subscriber comes back online.
If you're just testing with NServiceBus, the NServiceBus.Host.exe is running in the Lite profile, which uses in-memory (non-persistant) subscription storage, which would result in what you are seeing.
Ah ha! Well though it's not always an error to have no subscriber for a message type, there is a way to handle it.
In your publisher simply modify the:
IBus Bus
To use (you will need NServiceBus.Core.dll and the NS NServiceBus.Unicast):
IUnicastBus Bus
Then you can attach an handler to:
Bus.NoSubscribersForMessage += .......
This can then put the message in an error queue.. or perhaps retry forever.. or publish something else etc.. etc.. what ever you want. Thus ensuring there is nothing lost where your particular system (from a business perspective) requires an outcome

ActiveMQ + NMS can't receive messages

I'm trying to consume messages on an ActiveMQ topic from a C# application. I'm using the 1.3 .net release, and I don't receive any messages.
I have existing code that uses older libraries (and libraries built on top of libraries that I don't want to use or upgrade) that work fine, so I know messages are travelling along the topic, but my simple program with newer libraries just doesn't see them.
I'm using code copied and pasted from http://remark.wordpress.com/articles/publish-subscribe-with-activemq-and-nms/ as my test. The SimpleTopicSubscriber (with a simple change to make it a non-durable consumer) just doesn't receive anything. The SimpleTopicPublisher works just fine - I can send a simple message and it gets through, no problem.
Looking at the JMX console, I can see my subscriber connect, see that it's on the right topic, but it just doesn't get any messages.
Any ideas? I've even tried using the 1.2 libraries, but that didn't make any difference.
Thanks
Matt
Turns out there were 2 problems. I edited the example code incorrectly and passed in a durable consumer id instead of a selector. Not terribly clever.
Once I'd fixed this, I could receive messages, but only with the 1.1 release of the NMS/ActiveMq dlls. I couldn't receive messages with version 1.2 or 1.3.
But if I pass wireformat.version=2 as a parameter when connecting, everything works ok. The broker is 5.0.0 and I strongly suspect this would work ok with a later version of the broker.
Thanks
Matt
There are a couple reasons why your client might not be receiving messages, one could be that you didn't call Connection.Start(), without that the message pump won't start dispatching messages to your client. The other reason might be that your now non-durable Topic subscriber was started after the publisher in which case there'd be no messages delivered since you are using a Topic and Topics are like queue's in that once a message is sent its forgotten about so a subscriber that joins up later doesn't receive any messages that were sent before it subscribed.
Regards
Tim.
Open Source Integration: http://fusesource.com