I have experienced the following situation with an ActiveMQ Pub/Sub implementation. If the connection to the message broker is lost the publisher could re-try to establish a connection since the publish method would throw an exception.
However if the connection to the message broker is lost at the subscriber end, the subscriber would not know it. This would be the same if the session expires.
Proposed solution:
One of the solutions I thought was to implement a heartbeat at the subscriber end to periodically publish a ping message to a separate topic so that the subscriber could know if the connection is dropped. This works fine, but the down side is that the amount of ping messages generated by the subscribers available in the system. The second option I thought was to implement the heartbeat to try and create a connection in an interval. WDYT?
Do you see a better way of implementing this? Appreciate your thoughts.
Use the ActiveMQ Failover transport and don't disable the inactivity monitor and the client will check the connection and automatically reconnect as needed. Without more information on you set-up that's about the best answer.
Related
I am new to the world of Message Queues and I am currently evaluating RabbitMQ, ActiveMQ and Kafka. I see that in RabbitMQ, the Producer will create a Connection to the RabbitMQ server and the thread holding the Connection will remain active until the connection is closed. This leads me to believe that there MUST be a separate thread which delivers information to the RMQ Producer thread which will simply publish the message to the queue and keep looping until connection to the RMQ Server is closed? Is this assumption correct? Any thoughts/inputs would be appreciated.
Thanks!
P.S: This isn't the behaviour with Kafka. [ Apache Kafka: Java Producer reusability ]
in general, you should have a single RMQ connection per application instance. that connection can be opened as soon as your application starts.
having a connection does not yet give you the ability to publish or consume messages, though.
to do that, you need to create a channel.
the general best practice is one channel per thread in your application. need to publish a messages from this thread? create a channel for the thread. done with publishing it and not doing any other RMQ work on this channel? close the channel.
unlike connections, channels are cheap and easy to create. they work over the existing RMQ connection, and they take very little resources to create.
you can create thousands of channels in a single connection (though you might want to limit that number for performance reasons)
I wish to run an experiment in which the publisher loses connection with the broker and then enqueues messages in its own queue and then when it regains connectivity it sends all its queued messages to the broker. How can I I do this since if I call close connection, I can no longer send(raises an exception). A trick that I can think of is to use a network of two brokers and simulate the above by breaking the connection between the two brokers. Is there an API call that I can use to do the above?
This is very much like facebook messenger or whatsapp acting as a publisher and enqueuing our to-send messages if we are offline and sending them once we are connected.
There is plenty of solutions you could use to break the connection in order to test, here is a non-comprehensive list :
Make a script that can set/unset a firewall rule on your environement blocking the connection port
If you are working with VMs, you can suspend/resume the one running Activemq, you can even automate it with tools like vagrant (vagrant suspend, then vagrant up)
Tweak the connection manualy accessing the activemq jmx
Develop an activemq plugin able to trash connections on demand (or maybe there is one ?)
Now in order to have the behavior you wish to obtain there is two options :
1) Make sure your connection is failover so it can be reestablished, and store your message on disk before sending them with your producer.
2)Produce to a local broker embbeded in your app, and connect this one to the remote broker.
I am using BeginPeek() /no params/ to subscribe to messages coming in to my private queue. This is being done in a service hosted in NServiceBus host. When NServiceBus encounters transport connection timeout exception (i'm seeing circuit breaker armed logs and timeout exception logs), the peek event subscription seems get lost. When database connectivity becomes stable and new messages come in to my queue, the service is no longer notified.
Any ideas or suggestions on how to address this?
I've got an iOS application which uses a STOMP Client to talk to RabbitMQ. The application loads a lot of state during startup, and then keeps that state in sync by receiving updates published on STOMP. Of course, if it loses its connection, it can no longer be sure it's in sync, and therefore has to re-load that large initial blob. Any kind of network interruption triggers this behavior and makes my customers sad.
There are a lot of big-picture ways to fix this (and I'm working on them) but in the meantime, I'm trying to use persistent queues to solve this problem. The idea is that the server will create a queue, bind it to the appropriate topics, and then start building the large startup bundle. When finished, it will hand everything off to the client. The client will set itself up with the startup bundle, open a subscription to the queue, and then process any updates which happened while the server was getting things ready. Similarly, if the client should become disconnected, it can simply reconnect and resume reading the messages it finds in the queue.
My problem is that while the client successfully receives messages sent after it connects, if there were any messages in the queue before it connected, they are not read. Likewise, if the client becomes disconnected, when it reconnects, it won't see any messages which arrived while it was away.
Can anyone suggest how I might get the client to be able to read those missing messages?
It turns out what was happening was that the STOMP adapter was consuming the messages but failing to deliver them. Thus, when the client reconnected, it wouldn't have any messages waiting for it.
To fix the problem, I changed the "ack" setting on the subscribe request to "client", meaning that STOMP shouldn't consider the message delivered until the client sends back an ACK frame. By changing my client appropriately, messages now get delivered even after the client has been away.
I have a lot of client programs and one service.
This Client programs communicate with the server with http channel with WCF.
The clients have dynamic IP.
They are online 24h/day.
I need the following:
The server should notify all the clients in 3 min interval. If the client is new (started in the moment), is should notify it immediately.
But because the clients have dynamic IP and they are working 24h/day and sometimes the connection is unstable, is it good idea to use wcf duplex?
What happens when the connection goes down? Will it automatically recover?
Is is good idea to use remote MSMQ for this type of notification ?
Regards,
WCF duplex is very resource hungry and per rule of thumb you should not use more than 10. There is a lot of overhead involved with duplex channels. Also there is not auto-recover.
If you know the interval of 3 minutes and you want the client to get information when it starts why not let the client poll the information from the server?
When the connection goes down the callback will throw an exception and the channel will close.
I am not sure MSMQ will work for you unless each client will create an MSMQ queue for you and you push messages to each one of them. Again with an unreliable connection it will not help. I don't think you can "push" the data if you loose the connection to a client, client goes off-line or changes an IP without notifying your system.