I am using JMX MBeans in spring 4 framework. I handle notifications from JMX mBean and process them as events.
For testing alternate approach, I just commented out notification handling part. This implies that my JMX Mbean continues to publish notifications and I am just ignoring them.
P.S:
I handle the JMX notifications via logstash. For testing I have commented out Logstash configuration
What happens to the Notification? Where it will be stored?
Will it impact memory /pile up in application?
JMX notifications are an implementation of the observer pattern. With this pattern, events are not stored, so there is no memory issue.
NotificationBroadcaster JavaDoc:
When an MBean emits a notification, it considers each listener that has been added with addNotificationListener and not subsequently removed with removeNotificationListener. If a filter was provided with that listener, and if the filter's isNotificationEnabled method returns false, the listener is ignored. Otherwise, the listener's handleNotification method is called with the notification...
See also
https://en.m.wikipedia.org/wiki/Observer_pattern
https://docs.oracle.com/javase/7/docs/api/javax/management/NotificationBroadcaster.html
Java Management Extensions (JMX) - Best Practices
Lapsed Listener Problem
The observer pattern link refers to this problem. To be clear, this potential memory link is different than what you worried about originally (storing notifications). This memory leak is caused by listeners not being deregistered, for example the logstash listener. If a listener is not deregistered, then it may not be garbage collected.
If you are concerned about this, you need to confirm that commenting out the logstash configuration prevents the listener from ever registering in the first place (it probably does). Regardless, this is probably not an issue for you, because it is just one listener object. Your concern about notifications was more serious because notifications are continuously created.
After some debugging of code flow of sendNotification Method in Java:
My findings below:
Flow of sendNotification Method:
org.springframework.jmx.export.notification.ModelMBeanNotificationPublisher
javax.management.modelmbean.RequiredModelMBean (Sendnotification)
javax.management.NotificationBroadcasterSupport
Method in NotificationBroadcasterSupport sends notifications to listeners which are registered only (based on listenersList within the class file). If no listeners are registered, then it skips sending notifications.
Hence from this I assume/conclude that Notifications are not stored anywhere.
As #DavidS suggested, if we fail to removeNotificationListener after registering and do not want to receive notifications, we would end up listeners processing notifications. Possible cause of memory leak.
Related
Setting up a CMS consumer with a listener involves two separate calls: first, acquiring a consumer:
cms::MessageConsumer* cms::Session::createConsumer( const cms::Destination* );
and then, setting a listener on the consumer:
void cms::MessageConsumer::setMessageListener( cms::MessageListener* );
Could messages be lost if the implementation subscribes to the destination (and receives messages from the broker/router) before the listener is activated? Or are such messages queued internally and delivered to the listener upon activation?
Why isn't there an API call to create the consumer with a listener as a construction argument? (Is it because the JMS spec doesn't have it?)
(Addendum: this is probably a flaw in the API itself. A more logical order would be to instantiate a consumer from a session, and have a cms::Consumer::subscribe( cms::Destination*, cms::MessageListener* ) method in the API.)
I don't think the API is flawed necessarily. Obviously it could have been designed a different way, but I believe the solution to your alleged problem comes from the start method on the Connection object (inherited via Startable). The documentation for Connection states:
A CMS client typically creates a connection, one or more sessions, and a number of message producers and consumers. When a connection is created, it is in stopped mode. That means that no messages are being delivered.
It is typical to leave the connection in stopped mode until setup is complete (that is, until all message consumers have been created). At that point, the client calls the connection's start method, and messages begin arriving at the connection's consumers. This setup convention minimizes any client confusion that may result from asynchronous message delivery while the client is still in the process of setting itself up.
A connection can be started immediately, and the setup can be done afterwards. Clients that do this must be prepared to handle asynchronous message delivery while they are still in the process of setting up.
This is the same pattern that JMS follows.
In any case I don't think there's any risk of message loss regardless of when you invoke start(). If the consumer is using an auto-acknowledge mode then messages should only be automatically acknowledged once they are delivered synchronously via one of the receive methods or asynchronously through the listener's onMessage. To do otherwise would be a bug in my estimation. I've worked with JMS for the last 10 years on various implementations and I've never seen any kind of condition where messages were lost related to this.
If you want to add consumers after you've already invoked start() you could certainly call stop() first, but I don't see any problem with simply adding them on the fly.
I am currently learning RabbitMQ and AMQP in general. I started working with some tutorials I found online and all of them show more or less the same example - a Spring Boot web app that, upon a REST call, produces a message and puts in onto a RabbitMQ queue and then, another class from the same app, which is configured as the Consumer of that message consumes it and processes the handler method.
I can't wrap my head around why this is beneficial in any way. The upside I understand is that the handler is executed in a separate thread, while the controller method can return right after sending the message to the queue. However, why would this be in any way better than just using Spring's #Async annotation on that handler method and calling it explicitly? In that case I suppose we would achieve the same thing, while not having to host and manage a seperate instance of a message broker like RabbitMQ.
Can someone please explain? Thanks.
Very simply:
with RabbitMq you can have persistent messages and a much safer and consistent exception management. In case the machine crashes, already pushed messages are not lost.
A message can be pushed to an exchange and consumed by more parallel consumers, that helps scaling the application in case the consumer code is too slow.
and a lot of other reasons...
After calling service.subscribeToStreamingNotificationsOnAllFolders(..) and registering a disconnect handler, the handler gets called when the stream disconnects periodically. We are advised to reconnect the stream in the disconnect handler. Question: Is there the possibility to lose events, during the period when the stream is disconnected and reconnected? If so, how does the Microsoft EWS design propose to prevent this from happening?
Yes you can (and will) have missed events especially against cloud mailboxes , Notifications are there to provide a signal that you need to Synchronize a folder not as a synchronization method themselves. I'd suggest reading https://learn.microsoft.com/en-us/exchange/client-developer/exchange-web-services/mailbox-synchronization-and-ews-in-exchange which describes the recommended sync patterns to use.
I'm attempting to create a high level test in my solution, and I want to 'catch' messages sent to the bus.
Here's what I do:
nUnit [SetUp] spins up the WebAPI project in IISExpress
SetUp also creates the bus
Send a HTTP request to the API
Verify whatever I want to verify
The WebAPI part of the whole test works fine. The creation of the bus and kicking it off seems great too. It even finds my fake message handler. The problem is the handler never receives the command from the queue, they just stay in the RabbitMQ queue forever.
Here's how the bus is being configured:
var bus = Configure.With()
.DefineEndpointName("Local")
.Log4Net()
.UseTransport<global::NServiceBus.RabbitMQ>()
.UseInMemoryTimeoutPersister()
.RijndaelEncryptionService()
.UnicastBus();
.CreateBus();
In the log from NServiceBus starting up, I see that my fake handler is being associated with the command:
2014-09-24 15:29:59,007 [Runner thread] DEBUG NServiceBus.Unicast.MessageHandlerRegistry
[(null)] <(null)> - Associated 'Bloo.MyCommand' message with 'Blah.FakeMyCommandHandler' handler
So seeing as the message lands in the correct RabbitMQ queue, I'm assuming everything up until the handler point is working fine.
I've tried putting waits in my [TearDown] so that the bus lives a little longer - hoping to give the handler time to receive the message. I've also tried spinning off the in-memory bus for the consumer part of the interactoin into a new thread with no luck.
Has anyone else tried this?
This is only the first step, what I would love to do is create a fake bus that records messages being sent to it. The need for RabbitMQ is just to get myself going (the bounds of my solution are WebAPI on the front and the bus at the back).
Cheers
You forgot to call .Start() on the bus, that's why it didn't listen for messages.
See here for more info: http://docs.particular.net/nservicebus/hosting-nservicebus-in-your-own-process-v4.x
Also, consider using NServiceBus.Testing for unit testing your handlers and sagas:
https://www.nuget.org/packages/NServiceBus.Testing
I'm guessing your messages are just sitting in your queue forever because your end point is listening on "Local.MachineName" queue instead of "Local"
If you set the ScaleOut to be SingleBrokerQueue this should sort the issue.
Configure.ScaleOut(s => s.UseSingleBrokerQueue());
var bus = Configure.With()
.DefineEndpointName("Local")
...
If you are attempting to do full integration tests, using actual queues, then this answer won't help you.
If you are doing more focused tests, i.e. testing individual components that rely on the bus, I would recommend that you use a mocking framework (I like Moq) and mock out IBus. You can then verify that messages you expected to be sent to the bus were indeed sent.
I was looking for an ActiveMQ broker admin command, to tell it to pause a queue - that is:
continue accepting messages from producing clients
cease delivering to consuming clients, allowing the queue backlog to grow until the queue is resumed, whereupon the backlog is sent to clients.
I was unable to find such a command. The commonest answer was that it should be managed at the client end -- that is, locate every consumer and stop it. Other answers were workarounds, like manipulating network routes or firewalls so that the clients and broker could no longer communicate.
A cursory survey of other message queues indicates that ActiveMQ is not unusual in this regard.
It seems to me there are two reasons this functionality might not be implemented:
It is difficult to implement -- but I can't think of any reason why.
It is counter to the design philosophy of message queues
Which is it, and why?
Being able to pause a queue is supported in the newly released ActiveMQ 5.12.0:
When the queue is "paused":
NO messages sent to the associate consumers
messages still to be enqueued on the queue
ability to be able to browse the queue
all the JMX counters for the queue to be available and correct.
...
implemented pause/resume/isPaused queue view mbean ops and attribute
when paused, there is no dispatch to regular queue consumers, send
and browse work as normal. Any inflight messages will continue inflight
till ackes as normal.
See https://issues.apache.org/jira/browse/AMQ-5229
If you have Jolokia enabled (I think it is enabled by default nowadays), you can use something like the following curl request to pause the queue:
curl --user admin:admin http://127.0.0.1:8161/api/jolokia/exec/org.apache.activemq:brokerName=localhost,destinationName=myQueue,destinationType=Queue,type=Broker/pause
(Using the default username, password and broker name and a queue called myQueue)
Replace "pause" with "resume" in order to resume the queue.
Probably not too complicated to implement - as you say.
I don't know if it's an active design decision of if there has been no demand. Other similar products such as IBM WebSphere MQ implements "get/put inhibited" on queues, so it's obviously is not totally against the philosofy of messaging - rather a tool to operate and trouble shoot live systems.
I'm a bit biased, but I actually like to decouple the sender from the receive (if the are two different systems, that might eventually get switched/upgraded/changed..).
An easy way to decouple the systems, and be able to do what you want is to make the sender send to one queue "DATA.OUT" and the receiver listen to another "DATA.IN". Then you can use Apache Camel (which is typically bundled with ActiveMQ to achieve Enterprise Integration Patterns), to route from DATA.OUT to DATA.IN.
A Camel Route is possible to start/stop via JMX, which will achieve something similar to what you described.
I guess ActiveMQ design in the matter rather have you do these kind of things in a middleware layer, such as Apache Camel, rather than direct on the queues.