When you create a queue, you can specify time-to-leave of the messages, deletion of the queue if it has not been used for some time etc.
Those parameters are passed via a dictionary ; is there a place where you can find the proper key-values list accepted ?
“Arguments” are amqp BasicProperties
http://www.rabbitmq.com/releases/rabbitmq-java-client/v3.2.4/rabbitmq-java-client-javadoc-3.2.4/com/rabbitmq/client/AMQP.BasicProperties.html
Some AMQP broker use the BasicProperties to implement their extensions, for example Time-To-Live extension is an RabbitMQ extension.
If you change the broker you lose this functionality.
Read http://www.rabbitmq.com/extensions.html to see the RabbitMQ extensions.
Anyway you can use the properties as you prefer, for example you can add an your custom key value.
The class http://www.rabbitmq.com/releases//rabbitmq-java-client/current-javadoc/com/rabbitmq/client/MessageProperties.html contains some pre-built BasicProperties.
Related
I am integrating a NestJS application with RabbitMQ and i followed this tutorial
From the tutorial i can see that message consumer can connect to the rabbitMQ broker using a queue name, but actually i want to create a temporary queue that connects to an exchange. I cannot find an option to specify the exchange name in the queue options specified in the tutorial. can someone please point to the right configuration to use to specify an exchange instead of queue?
is it possible to specify an exchange name inside the 'queueOptions' structure?
From what I understand from your tutorial (I don't know NestJS), createMicroservice only connects to a queue (or create it if needed). In other words, it performs assertQueue operation:
var queue = 'task_queue';
channel.assertQueue(queue, {
durable: false
});
If you want to bind a queue to an existing exchange, you need to perform bindQueue operation:
channel.bindQueue(queue, exchange, '');
Thus you can't bind to an exchange using queueOptions. Take a look at the official Rabbitmq documentation, especially the "Temporary queues" and following "Bindings" sections.
I want to use RabbitMQ with StormCrawler. I already saw that there is a repository for using RabbitMQ with Storm:
https://github.com/ppat/storm-rabbitmq
How would you use this for the StormCrawler? I would like to use the Producer as well as the consumer.
For the consumer there seems to be some documentation. What about the Producer? Can you just put the config entries in the storm crawler config or would I need to change the source code of the RabbitMQProducer?
You'd want the bolt which sends URLs to RabbitMQ to extend AbstractStatusUpdaterBolt as the super class does a lot of useful things under the bonnet, which means that you would not use the Producer out of the box but will need to write some custom code.
Unless you are certain that there will be no duplicates URLs, you'll need to deduplicate the URLs before sending them to the queues anyway, which could be done e.g. with Redis within your custom status updater.
Rebus has flexible system that allows me to specify different endpoints for different message types, either in web.config or by implementing a custom IDetermineMessageOwnership.
As far as I can tell, message ownership is represented simply by a string. Using the MSMQ transport, this string points to a queue to which the message is delivered. With RabbitMQ, the string is used as a topic for the message, which is then delivered to a generic exchange named "Rebus". Rebus is a nice fellow, so he also sets up a queue in the RabbitMQ server, using the same name, and makes a binding from topic to queue within the Rebus exchange.
My question is this: Is it possible to have Rebus not create queues and bindings, but still deliver the messages to an exchange with a relevant topic set for each message?
Declaring the queues and bindings manually will allow me to set up an awesome topic exchange, using bindings with wildcards and what not. Here is a nice illustration of a topic exchange with funky bindings, just to make my question look more sleek and sexy:
Sounds to me like you want to do something like this:
Configure.With(yourFavoriteContainer)
.Transport(t => t.UseRabbitMq(...)
.ManageSubscriptions()) //< BAM!!1
.(...)
which lets Rebus take advantage of the fact that Rebus' RabbitMqMessageQueue implements IMulticastTransport, which in turn turns handling of all things multicast over to Rabbit.
It's just important that all of your Rabbit-enabled Rebus endpoints agree on letting Rabbit ManageSubscriptions - otherwise, weird stuff might happen ;)
It means that
when you bus.Subscribe<SomeEvent>, you bind a topic with the type name to the subscriber's input queue - e.g. "SomeEvent.SomeNamespace" -> myInputQueue
publishers publish events on a topic that is the type name - e.g. "SomeEvent.SomeNamespace"
message ownership is disregarded when subscribing
Rabbit will do the heavy lifting when doing multicast (which is what Rabbit users are mostly doing)
If you require even more flexibility, you can even take responsibility of deciding the topic to publish to for each .NET type, like so:
Configure.With(yourFavoriteContainer)
.Transport(t => t.UseRabbitMq(...)
.ManageSubscriptions()
.AddEventNameResolver(type => DecideTopic(type))
.(...)
You can add multiple event name resolvers if you want - they will be run in sequence until one of them returns something that is not null.
Does it make sense?
I have exactly 2 types of messages that I want to be sent via RabbitMQ. So I have 2 options how I can do this:
sent a message to default empty-named exchange with routing_key corresponding to the queue name
use direct exchange's routing_key parameter corresponding to consumer's routing_key parameter in queue binding
So which option is preferable and why?
A default exchange is a direct exchange. RabbitMQ creates the default exchange by default, but it uses an empty string for the name. If you look at the RabbitMQ AMQP concepts page, under Default Exchange:
The default exchange is a direct exchange with no name (empty string)
pre-declared by the broker.
You can see this by running rabbitmqctl list_exchanges as well:
direct
Foo direct < Same thing as the above
amq.direct direct
amq.fanout fanout
...and so on
As far as I'm aware, there isn't any benefits of using one over the other. I would stick with the default exchange if you only need to route based on routing keys.
Let's say you direct-bind to an exchange broadcasting logs to routing keys of "info", "warn", and "error". Using the default exchange, you would need to create three different queues with those names to receive all logs; and adjustments to which log levels you receive would require changing your queue declarations. By using a named exchange, you can simply change your queue's bindings and continue processing things as normal.
In short, it provides one extra level of abstraction.
As I see it, the default direct exchange give the possibility for the consumers and the producers to not know about each other, by binding a queue (used by a consumer) to an exchange (used by a producer) implicitly using the queue's name.
I use the default direct-exchange for a specific case: the consumer and producers don't know about each other. In my case, each consumer have its proper queue. From the producer, I cannot know by advance which queues are going to be declared and used, as it depends on the consumers. So it is impossible to define the bindings between a custom direct-exchange and the queues on the producer side.
One way to solve it with a custom (user-defined) direct-exchange would be to define the binding-key on consumer side. But it would means to know about the producer from the consumer side as I need to know the exchange name used by the producer.
Therefore, automatically binding a queue by its name on the default direct-exchange makes it possible, in my case, to only declare a queue on consumer side and send message to it from producer by only knowing the queue's name.
Of course it implies to know the queue's name at runtime, when invoking the producer, as it would be required to know the binding-key of a custom direct-exchange (in my case, the queue's name is given by the application using the producer). But when configuring the broker, producers and consumers don't have to know about each other.
some description in rabbitmq web manager.
//default exchange 's Bindings
Default exchange
The default exchange is implicitly bound to every queue,
with a routing key equal to the queue name.
It is not possible to
explicitly bind to, or
unbind from the default exchange.
It also cannot be deleted.
I have to implement this scenario:
An external application publish message to rabbitmq.
This message has a client_id property. We can place this id to routing key or message header or some other property.
I have to implement sharding in a exchange routng logic - the message should be delivered to specific queue based on the client_id range.
Is it possible to implement in a standard exchanges?
If not what exchange should I take as the base?
How to dynamicly change client_id ranges?
Take a look at the rabbitmq plugin. It's included in the RabbitMQ distribution from v3.6.0 onwards.
Just have your producer put enough info into the routing key that causes the message to go into the right queue on the other side of the Exchange.
So for example, create two queues called 1 and 2 and bind them with routing keys matching the names. Then have your producer decide which routing key to use when producing the event message. Customers with names starting with letters a-m go to 1, n-z go to 2, you get the idea. It pushes the sharding to the producer but that might be OK for your application.
AMQP doesn't have any explicit implementation of sharding, but its architecture should help you to do that.
Spreading messages to several queues is just a rabbitmq challenge (and part of amqp specification), and with routing, way you can attach hetereogeneous consumers to handle specific messages routed via the same exchange. Therefore, producer should push a specific key to be consumed by specific queue/consumer...
You can decide to make a static sharding, perhaps you have 10 queues with one consumer per queue. You could implement a consistent hashing function such that key is CLIENT_ID % 10.
Another ways and none static solutions could be propoused, and you can try to over this architecture.