Hy,
Is there a way to see the messages that arr waiting in the Celery/RabbitMQ queues?
I would like to see what parameters has each waiting task.
Tx!
You can do it with Management Plugin
UPD:
Message props and headers are plain strings or numbers so you can view them without any extra job.
If you need to see message body and Management Plugin doesn't fits you, another approach is to consume messages from queue, output their content and push them back in queue. There is some chance to loose message during such action if your script fail before message will be requeued back.
Again, if you need to see only parameters or headers then Management Plugin is your best friend.
If you send JSON or XML encoded message body i think it is trivial to write js script and use it with Userscripts to show you message original content.
Related
In our team, we exchange messages via RabbitMQ between two systems. The messages are encoded in protobuf (v3). We use NServiceBus on the sending and receiving side. We use the RabbitMQ management UI for monitoring the error queue. In production, we noticed that it is not easy to make sense of the payload of the messages in the error queue, which are base64-encoded.
What is the easiest way to get human-readable access to the messages in the error queue? We have full control over the decisions in both systems, and also discussed a switch to JSON-encoded messages (instead of protobuf). But we are otherwise happy with our protobuf-based implementation. Which is already implemented after all.
Protobuf v3 supports formatting as json, once you have the data parsed as IMessage (the base type for in-memory protobuf objects).
So you can convert a single message to be human readable as follows:
Use the webUI GetMessage function to get the message as base64 then requeue it
Convert the message back to protobuf binary via Convert.FromBase64String
Parse it back to an IMessage via ProtoMessageTypeGoesHere.Parser.ParseFrom(binaryData)
You can then convert the parsed message to Json via ToString() or Google.Protobuf.JsonFormatter.
As long as your error queue isn't going to be disrupted by the re-queuing (e.g. resetting of timestamps or reprocessing), you should be able to do this for all messages in the queue.
I wouldn't recommend using the management UI for this. A simple script or html page with a stomp client would be a lot easier to use and more error-proof, in my opinion.
However, to answer your question: to simply decode the message and replace the text, a simple javascript solution will work fine.
$(".msg-payload").text(atob($(".msg-payload").text()))
This will simply select the message field on the queue page on the RabbitMQ management UI and replace it with the decoded value (that's the function atob).
To use this, you can either run it from the console or add it as a bookmark in your browser. Simply use the code prefixed with javascript:, like so:
javascript:$(".msg-payload").text(atob($(".msg-payload").text()))
What is the recommended way to deal with message versioning? The main schools of thought appear to be:
Always create a new message class as the message structure changes
Never use (pure) serialized objects as a message. Always use some kind of version header field and a byte stream body field. In this way, the receiver can always accept the message and check the version number before attempting to read the message body.
Never use binary serialized objects as a message. Instead, use a textual form such as JSON. In this way, the receiver can always accept the message, check the version number, and then (when possible) understand the message body.
As I want to keep my messages compact I am considering using Google Protocol Buffers which would allow me to satisfy both 2 & 3.
However I am interested in real world experiences and advice on how to handle versioning of messages as their structure changes?
In this case "version" will be basically some metadata about the message, And these metadata are some instruction/hints to the processing algorithm. So I willsuggest to add such metadata in the header (outside of the payload), so that consumer can read the metadata first before trying to read/understand and process the message payload. For example, if you keep the version info in the payload and due to some reason your (message payload is corrupted) then algorithm will fail parse the message, then it can not event reach the metadata you have put there.
You may consider to have both version and type info of the payload in one header.
we are trying to audit all incoming/outgoing messages, header information in our mule flow.
For same we have tried to use 'wire-tap' which we dint found so useful also its working on mule 3.6.1 but giving error in 3.7.
Any idea/suggestion for auditing?
Ok let me add some more details:
What we are trying to do is- Whatever message comes or flows via flow components we want to copy it in some sub flow (say in queue) without interrupting the main flow so that we can check the message.
you can make it work in several ways
1) Wire tap is one of the choice. You can route your messages asynchronously to sub flow and sub flow will do the auditing work. But I don't know why you didn't found wiretap useful. Can you explain in more.
2) All the messages your receive from the main flow, those you can post to JMS queue. So another flow will read from there and do the auditing work. Use of this multiple projects can use the same piece of code and post JMS queue for auditing.
This can be done in many different ways and you kind of mentioned them in your question such as logger component and interceptor.
All the headers are available as message properties so if you log the entire message they are shown. Simply put an logger component after the inbound endpoint and one before the outbound endpoint and this is easily done.
If you need some log entries transformation you could always put that in a wire tap so you don't interfere with the functionality of your flow.
I have two scripts which generate messages in one queue, one of the scripts would like the queue to generate an additional message once the message has been successfully ack'd - note this is not RPC I want to do additional processing optionally once the first message has completed successfully, but until the first message has been processed successfully I cannot do the second round of processing.
Does anyone have any experience doing this? My initial thought is to send additional parameters to the initial message identifying the "next steps" but this seems a little hackish so I was hoping for a better solution.
check my answer to this question:
RabbitMQ get message send confirm
This functionality should allow you to do what you want.
I'm using the web console against my AMQ 5.2 instance successfully, except for I cannot see the content of all of my messages.
If I send a test message using the web console, I can see the sample text content, but I believe the vendor app I am working with has binary or byte array message content.
Is there something I need to do to be able to view this raw data?
Thanks,
To my knowledge, it is not possible to inspect messages in the Admin Console. You can get some statistics (like how many messages have been sent etc.).
ActiveMQ does not unmarshal messages when receiving them (for performance reasons, unmarshalling is rather expensive).
Thus, if you want to have some way to inspect messages for their content, you can basically do 2 things:
Write a consumer which registers for all topics/queues, through which you can see messages' content. Drawback: if you're using queue-based interaction, your "real" consumers will not get all messages
Write an activeMQ plugin which looks at the messages. Have a look at ActiveMQ's Logger Plugin. Then write your own (you'll need the sources to compile it) and load it with ActiveMQ (see the documentation on how to configure ActiveMQ to load plugins). You want to override the send() method which is called whenever someone sends a message to the broker. There you get a reference to the message and can access its content.
Neither of the two messages provides a convenient viewing-mechanism though. You'll have to resort to standard out, or write your own web-based access.
hawtio now shows first 256 chars of messages. Don't know if that is enough for you. Use browse() method.