I use rabbitmq as the middleware in cloud stream ,when I configure the application.yml
spring:
cloud:
stream:
bindings:
usertest: #channelName
content-type: application/json
group: testGroup
destination: topic888 #destination，
binder: rabbit1
I can't understand the key meaning of destination.
official definition is :
destination
The target destination of a channel on the bound middleware (e.g., the RabbitMQ exchange or Kafka topic). If the channel is bound as a consumer, it could be bound to multiple destinations and the destination names can be specified as comma separated String values. If not set, the channel name is used instead.
Also,I found when consumer and producer have different destinations , It did't stop consumer from consuming.
The destination depends on the binder type. For kafka, it's the topic; for RabbitMQ, it's an exchange.
The producer publishes to the exchange.
The consumer binds a queue to the exchange. Anonymous consumers bind a temporary, auto-delete queue, consumers with a group bind a queue called topic888.testGroup in your case (destination.group).
Also,I found when consumer and producer have different destinations , It did't stop consumer from consuming.
I don't see how that's possible, with the binder doing the provisioning, for the reasons discussed above.
It's possible if you manually bind a queue with the correct name to a different exchange that the producer is publishing to.
Related
In Cloud Stream for RabbitMQ I want to set a consumer that will consume from a Queue only the messages that have been forwarded with a specific routing key. Is this possible?
Here are my application.properties.
Producer:
spring.cloud.stream.rabbit.bindings..consumer.bindQueue=false
spring.cloud.stream.rabbit.bindings..consumer.declareExchange=false
spring.cloud.stream.bindings.producer1-out-0.destination=exchange1
spring.cloud.stream.rabbit.bindings.producer1-out-0.producer.routingKeyExpression='routing.key.1'
Consumer:
spring.cloud.stream.rabbit.bindings..consumer.bindQueue=false
spring.cloud.stream.rabbit.bindings..consumer.declareExchange=false
spring.cloud.stream.bindings.consumer1-in-0.destination=exchange1
spring.cloud.stream.bindings.consumer1-in-0.group=reports
spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.queueNameGroupOnly=false
spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.bindingRoutingKeyDelimiter=,
spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.bindingRoutingKey='routing.key.1'
Exchanges and bindings are defined directly in RabbitMQ.
My expectation was if I change spring.cloud.stream.rabbit.bindings.consumer1-in-0.consumer.bindingRoutingKey to something else the consumer will stop consuming but it's not the case.
No; unlike JMS, RabbitMQ has no concept of a message selector; you will get all messages.
The canonical way to solve this issue is to use a different queue for each RK.
bindingRoutingKey is meaningless when bindQueue is false and, in any case, all it does is specify which routing key to use when binding the queue to the exchange.
I have multiple producers that publish to their specific (durable and limited) queues using the amq.direct exchange and particular routing key
Queues:
producer.06
producer.07
...
Routing keys:
"producer.06" -> producer.06
"producer.07" -> producer.07
...
I also have multiple consumers. When they connect, they create their own (exclusive) queue and routing keys to receive all the messages from the queues that are of interest to them. This way multiple consumers can get the same messages.
Queues:
consumer.a
consumer.b
...
Routing keys:
"producer.06" -> consumer.a
"producer.06" -> consumer.b
"producer.07" -> consumer.b
...
I would like to populate the consumer's queue with a snapshot of messages of the relevant producer's queues, prior to binding the routing keys. Loosing a few messages in the interval between the message copy and routing key binding is acceptable, and a better alternative than out-of-order messages for my application. The consumer should not remove messages from producer's queues (as they would be needed by other consumers).
Is there a way to achieve this? -copying a snapshot of a queue into another one- or does anyone has a suggestion on how to achieve this?
I am running RabbitMQ 3.8.4 on Erlang 23.0.2, and using Rabbit .Net client 6.0.0.0 for the consumers.
How do you consume using Spring Cloud Stream (that is using AMQP) messages sent with MQTT to RabbitMQ?
With MQTT on Rabbit, all messages land on an exchange called "amq.topic".
On the consumer side, with Spring Cloud Stream, a queue is created for each destination and an exchange of type "topic" with the name of the destination; the created queue is bound to the exchange.
Now,
I cannot manually bind my queue to the "amq.topic", because it is exclusive:
cannot obtain exclusive access to locked queue '...' in vhost '/'. It could be originally declared on another connection or the exclusive property value does not match that of the original declaration.
I cannot listen directly from the exchange "amq.topic", because, well, you have to listen to a queue...
I created a "tmp" queue bound to "amq.topic" where messages are delivered, but I cannot use it as destination, because RMQ will create a new queue called "tmp.SOME_CLIENT_ID", bound to an exchange called "tmp", that has nothing to do with my "tmp" queue.
Any idea would be welcome!
See Using Existing Queues/Exchanges.
By default, the binder will automatically provision a topic exchange with the name being derived from the value of the destination binding property <prefix><destination>. The destination defaults to the binding name, if not provided. When binding a consumer, a queue will automatically be provisioned with the name <prefix><destination>.<group> (if a group binding property is specified), or an anonymous, auto-delete queue when there is no group. The queue will be bound to the exchange with the "match-all" wildcard routing key (#) for a non-partitioned binding or <destination>-<instanceIndex> for a partitioned binding. The prefix is an empty String by default. If an output binding is specified with requiredGroups, a queue/binding will be provisioned for each group.
There are a number of rabbit-specific binding properties that allow you to modify this default behavior.
If you have an existing exchange/queue that you wish to use, you can completely disable automatic provisioning as follows, assuming the exchange is named myExchange and the queue is named myQueue:
spring.cloud.stream.bindings.<binding name>.destination=myExhange
spring.cloud.stream.bindings.<binding name>.group=myQueue
spring.cloud.stream.rabbit.bindings.<binding name>.consumer.bindQueue=false
spring.cloud.stream.rabbit.bindings.<binding name>.consumer.declareExchange=false
spring.cloud.stream.rabbit.bindings.<binding name>.consumer.queueNameGroupOnly=true
If you want the binder to provision the queue/exchange, but you want to do it using something other than the defaults discussed here, use the following properties. Refer to the property documentation above for more information.
spring.cloud.stream.rabbit.bindings.<binding name>.consumer.bindingRoutingKey=myRoutingKey
spring.cloud.stream.rabbit.bindings.<binding name>.consumer.exchangeType=<type>
spring.cloud.stream.rabbit.bindings.<binding name>.producer.routingKeyExpression='myRoutingKey'
Why do we need routing key to route messages from exchange to queue? Can't we simply use the queue name to route the message? Also, in case of publishing to multiple queues, we can use multiple queue names. Can anyone point out the scenario where we actually need routing key and queue name won't be suffice?
There are several types of exchanges. The fanout exchange ignores the routing key and sends messages to all queues. But pretty much all other exchange types use the routing key to determine which queue, if any, will receive a message.
The tutorials on the RabbitMQ website describes several usecases where different exchange types are useful and where the routing key is relevant.
For instance, tutorial 5 demonstrates how to use a topic exchange to route log messages to different queues depending on the log level of each message.
If you want to target multiple queues, you need to bind them to a fanout exchange and use that exchange in your publisher.
You can't specify multiple queue names in your publisher. In AMQP, you do not publish a message to queues, you publish a message to an exchange. It's the exchange responsability to determine the relevant queues. It's possible that a message is routed to no queue at all and just dropped.
Decoupling queue names from applications is useful for flexibility.
You could establish multiple queues to consume the same message, but queues can't have the same name.
In some cases, message's originator doesn't know the names of queues. (like when you have randomly generated queue names when horizontally scaling a server)
An exchange may be routing messages for more than just one type of consumer. Then you would need some wildcards in your routing keys to route messages to concerned consumers.
Is it possible to bind a single queue to many topics using RabbitMQ STOMP client?
Each time a client sending SUBSCRIBE frame server creates a new queue for it, it makes usage of "prefetch-count" useless for me, because it applies to each subscription individually.
I am just looking for any way to get messages with many topics in the single queue via RabbitMQ Web-STOMP. Any ideas?
See Documentation:User generated queue names for Topic and Exchange destinations
The header x-queue-name specified queue name should binding to same queue if there exist, but will exist multiple subscription on client.
The different between AMQP and STOMP concept not compatible in some ways.