Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
I got the contents from path: /api/queues/vhost/name and how are these parameters below?
message_stats:
deliver_get
incoming
Here you can find the stats documentation:
http://hg.rabbitmq.com/rabbitmq-management/raw-file/31c1d2668d39/priv/www/doc/stats.html
EDIT**
message_stats objects
publish Count of messages published.
publish_in Count of messages published "in" to an exchange, i.e. not taking account of routing.
publish_out Count of messages published "out" of an exchange, i.e. taking account of routing.
confirm Count of messages confirmed.
deliver Count of messages delivered in acknowledgement mode to consumers.
deliver_noack Count of messages delivered in no-acknowledgement mode to consumers.
get Count of messages delivered in acknowledgement mode in response to basic.get.
get_noack Count of messages delivered in no-acknowledgement mode in response to basic.get.
deliver_get Sum of all four of the above.
redeliver Count of subset of messages in deliver_get which had the redelivered flag set.
return Count of messages returned to publisher as unroutable.
/api/queues/(vhost)/(name)
incoming Detailed message stats (see section above) for publishes from exchanges >into this queue.
deliveries Detailed message stats for deliveries from this queue into channels.
consumer_details List of consumers on this channel, with some details on each.
Related
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 6 months ago.
Improve this question
When building webhooks, it's a best practice for the consumer of the webhook (e.g. the receiver of the webhook) to immediately drop any messages received into a queue to prevent it from "backing up" the delivery of subsequent messages. This seems to be the "best practice" for years regarding webhook architectures. Nowadays, with the advent of internet-accessible queues (e.g. Amazon SQS), why are we not flipping the script on webhook architecture such that the consumer becomes responsible to "pull" messages off a queue, rather than receive message via an http post?
Yes, this essentially is no longer a "webhook", but the concept is the same. The problem being solved here is that a consumer wants to be made "aware" of events happening in another system. Why not have the publisher of these events store all relevant events in a queue dedicated to a single consumer such that the consumer can pull those messages off the queue at their own leisure, pace, etc. I see many benefits to this, mainly the transfer of responsibility to the consumer to "dequeue" messages according to their own abilities. The publisher drops messages as quickly as they can into the queue, and the consumer pulls them off as quickly as they can. If the consumer goes down for any reason, the messages will still remain in the queue for as long as they need. Once the consumer is back up, they can continue pulling message off. No messages ever lost in this scenario. Right?
The way I see it is mostly an opinion, not necessarily the ultimate answer.
While theoretically there's a good point in advocating for pushing messages straight to the queue by the producers, there's a real world constraint that will be imposed on those producers. Every messaging system has some nuances. This means producers have to be aware of those nuances in order to be able to publish to various messaging services. Authentication is another nuance. All this turns into a nightmare for any producer that issues notifications to various consumers. This is what webhooks have solved. Ubiquitous, established protocol, authentication, etc.
Apologies if this question has already been answered before.
After searching a lot, I understood that in MassTransit(RabbitMQ) if a message need to deliver to multiple consumers then we need to use different endpoint names (queues) for each of the services. Lets say, if a news is to broadcast in 1000 subscribers then we need to register (ReceiveEndPoint) all 1000 consumers with the publisher and each subscriber has its own queue.
I am not satisfied with this approach and I believe I am mistaken. If the same message need to be delivered then why not all the subscribers get a copy of the message from the same queue?
Is it the limitation of the RabbitMQ or there is a way to do it?
I am trying to use Spring JMS and ActiveMQ to process a large number of messages. The context of the problem is the following:
Each customer produces a set of messages that are added to the queue. The messages are added to the queue with the customer id as parameter.
In one case, customer A can add 10k messages to the queue, while customer B only adds 100 messages to that same queue. My issue is customer B needs to wait until all the 10k messages have finished processing before its 100 messages are processed.
Is there a way to process some of messages of customer A and some of the messages of customer B at the same time? I know there is the option to set a higher priority on the messages from customer B, but that does not solve the issue when there is more multiple customers. The customer with the more messages will fill-up the queue while the others will have to wait.
I would appreciate if you could provide some help or advice.
The basic semantic of a queue is first-in-first-out (i.e. FIFO). There's no real way to escape that. I recommend you redesign your application to use multiple queues - one for each "type" of message or independent application you have.
I would say you can fine tune your activemq the number of messages it is processing in batches. Also, there are way you can fine tune a given Broker and queue. For more details refer this link:
http://activemq.apache.org/performance-tuning
https://access.redhat.com/documentation/en-US/Fuse_ESB/4.4.1/html-single/ActiveMQ_Tuning_Guide/index.html
I think I have find the solution to the issue. It involves using message groups. For each message I set the property JMSXGroupID with an identifier for the customer.
Since they are multiple message groups the queue takes care of assigning messages from different groups to different consumers. In that way, documents from customer B can be processed while the ones from customer A are still being processed.
After reading documentation about what is Rabbit and what does, I have some common questions:
Case is: Producer sends one message to some consumers (subscribers).
I need a short explanation for all points of list below, what to use, and what to dig further.
How to clear queue and stop sending message to consumers after
specific time/date?
Can I include to confirmed message user's data like JSON?
Where is stored this data? In the same queue?
How to filter confirmed messages and then clear queue?
How to clear queue after specific time/date?
What happens if not one consumer no confirms message, how long they are stored?
Does consumer subscribe on queue or can subscribe on exchange too?
Using model one to many for sending message, how to set who have to
get message first/last or at the same time, here described that, but not clear is it on client or server side?
If no consumers, how to re-push message to another queue and close
current?
Each consumer has own queue?
Thank you in advance and any comment to this question!
If you can elaborate some of your questions and include what is your use case, I can edit the answer.
1 - As long as consumer is alive rabbitmq sends incoming messages to consumer. You can give TTL to messages/queues if you want them to expire after some time.
https://www.rabbitmq.com/ttl.html
2 - What you mean?
3 - Rabbitmq stores the data in mnesia database.
https://www.rabbitmq.com/persistence-conf.html
https://www.rabbitmq.com/relocate.html
4 - What you mean by filterig messages and clear queue? Successfully consumed messages removed from the queue immediatly.
5 - You can give ttl to queue or declare queue as auto delete
https://www.rabbitmq.com/ttl.html
https://www.rabbitmq.com/queues.html
6 - If consumers don't send ack to rabbit, messages stays unack as long as memory becomes full or rabbit becomes unavailable
7 - Both. A consumer can create its own queue and bind it to an exchange or it can consume from existing queue. It depends on the use case.
8 - It is hard to answer this without knowing details of what you mean by one-to-many. Direct exchange or fanout or whatelse, how many queues etc.
But, in rabbitmq, messages ordered by publish order by default.
According to link you shared, rabbitmq sends messages first to higher priority consumers until consumer prefetch count (unack messages on consumer) becomes its limits.
9 - You need to handle this case in the code. Or you can use management ui with Shovel plugin.
https://www.rabbitmq.com/management.html
https://www.rabbitmq.com/shovel.html
10 - Again, it depends on the design and use case.
I would like to give feedback to users (message issuers) when their messages will be processed (not when they are done).
Theoretically I could receive all the messages, count and reschedule them. This would be stupid due to new arriving messages would be processed immediately due to all the message before the new messages would be rescheduled after the new (initial last) messages.
Is there a way to determine the number of messages before a specific message with RabbitMQ?
I am afraid there is currently no way on the server side to find the number of messages published before a particular message though rabbitmq provides an HTTP API which returns the number of messages currently in the queue.
You will have to maintain this statistic on the client side(like a counter or something) preferably at the publisher end. But make sure to publish the message only once in order to avoid duplicate counts or implement your counter in a such a way(keep a unique identifier per message) so as to avoid counting the same message more than once (if republishing is allowed).
A similar question has been asked before :
RabbitMQ - Get total count of messages enqueued