I'm new in to RabbitMQ and I've faced a problem. I'm trying to get messages from queue by API method. I've made that by now I want to get messages from queue by header or property if it is possible. I read the documentation about HTTP API. I have not found such an API for filtering messages by some headers or properties.
I use that kind of API to get messages from queue:
/api/queues/vhost/name/get
and in the body:
{"count":20,"ackmode":"ack_requeue_true","encoding":"auto"}
I was thinking, maybe it is possible to somehow pass some filter in the body so it could filter and return the message what I want.
This is how my message looks like :
I have tried to pass in the body type = "myType" or header = "myHeader"
I've made that by now I want to get messages from queue by header or
property if it is possible.
RabbitMQ only delivers messages in order from a queue. There is no way to filter once a message is in a queue.
You can filter messages as they are published to an exchange, however. Use a headers exchange and bind queues based on header values. Then, each queue will contain the messages you expect and you can then consume from them.
The RabbitMQ tutorials have a section that use a "headers exchange". Use that as a guide.
Finally, only use the HTTP API for testing. It is a very inefficient way to retrieve messages.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
A bit late to the party, but I think you can achieve the same like this
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(hostname);
Connection conn = factory.newConnection();
Channel channel = connection.createChannel();
channel.queueBind(queueName, exchangeName, "");
DeliverCallback deliverCallback = (consumerTag, delivery) -> {
Map<String, Object> headers = delivery.getProperties().getHeaders();
String message = new String(delivery.getBody(), "UTF-8");
System.out.println(" [x] Received '" + message + "', with header : " + headers.get("TestHeader") );
};
channel.basicConsume(queue, true, deliverCallback, consumerTag -> { });
Using the REST API I'm trying to update the priority of an existing queue like so:
PUT /api/queues/%2F/TestEvent_ProcessingService HTTP/1.1
Host: localhost:15672
Authorization: Basic <--snip-->
cache-control: no-cache
{
"durable":true,
"arguments":{
"x-max-priority":2
},
}
The response is:
{
"error": "bad_request",
"reason": "inequivalent arg 'x-max-priority' for queue 'TestEvent_ProcessingService' in vhost '/': received the value '2' of type 'long' but current is none"
}
Is there any way to set this value on an existing queue, or do you need to delete the queue and recreate it with a new "x-max-priority" value?
The answer is in RabbitMQ's documentation:
https://www.rabbitmq.com/priority.html
do you need to delete the queue and recreate it with a new
"x-max-priority" value?
Yes.
(cross-posted question here).
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
I'm trying to write something for my iOS app, using RMQClient that scans for existing exchanges on a rabbitmq server. I came up with this so far.
class AMQPExchangeScanner {
static func scan() {
let connection:RMQConnection = RMQConnection(uri: "amqp://user:user#abc.def.com:5672", delegate: RMQConnectionDelegateLogger())
connection.start()
for exchangeName in Foo.pastExchanges() {
let channel = connection.createChannel()
let exchange = channel.fanout(exchangeName, options: .passive)
"scan \(exchangeName) \(exchange)".print()
channel.close()
}
}
}
I'm not sure how to determine if the exchange actually exists though. The print() statement prints exchange objects. I get a whole bunch of output in the console. I had hoped that I would get back an optional so I could do something like
if let exchange... {
}
But that doesn't appear to be the case. How do I programmatically check if the exchange is real or not? Or get at those errors? Do I need my own connection delegate and have to parse a bunch of text?
The best way to scan for existing exchanges is using rabbitmq management HTTP API - it would be under /api/exchanges.
For a single exchange you could call the declare method, with passive parameter set accordingly (quoting from here):
for the declare method
This method creates an exchange if it does not already exist, and if
the exchange exists, verifies that it is of the correct and expected
class.
for the passive bit
If set, the server will reply with Declare-Ok if the exchange already
exists with the same name, and raise an error if not.
...
If not set and the exchange exists, the server MUST check that the existing exchange has the same values for type, durable, and arguments fields. The server MUST respond with Declare-Ok if the requested exchange matches these fields, and MUST raise a channel exception if not.
Have to add OperationContext.Current.OutgoingMessageProperties to outgoing BizTalk message
How to implement below code in BizTalk?
ConcurrentPrograms_ARClient client1 = new ConcurrentPrograms_ARClient(binding, address);
using (new OperationContextScope(client1.InnerChannel))
{
OperationContext.Current.OutgoingMessageProperties.Add("Property Name", "Property Value");
client1.OPERATION(params...);
}
You might want to take a look at WCF message inspectors.
You can implement one in a separate project and configure it as a behavior in your send port where you would then have full control over both the request and the reply message.
I'm trying to setup my first RabbitMQ dead letter exchange, here are the steps I'm using through the web admin interface:
Create new DIRECT exchange with the name "dead.letter.test"
Create new queue "dead.letter.queue"
Bind "dead.letter.queue" to "dead.letter.test"
Create new queue "test1" with the dead letter exchange set to "dead.letter.test"
Send a message into "test1"
Nack (with requeue = false) the message in "test1"
I am expecting that these steps should put a record into the "dead.letter.queue" through the "dead.letter.test" exchange. This is not happening.
I can manually put a message into the "dead.letter.test" exchange and it shows up in "dead.letter.queue" so I know that is fine.
When I look at the admin UI it shows that the DLX parameter is setup on the queue "test1".
Where am I going wrong?
Gentilissimo Signore was kind enough to answer my question on Twitter. The problem is that if your dead letter exchange is setup as DIRECT you must specify a dead letter routing key. If you just want all your NACKed message to go into a dead letter bucket for later investigation (as I do) then your dead letter exchange should be setup as a FANOUT.
Here are the updated steps that work:
Create new FANOUT exchange with the name "dead.letter.test"
Create new queue "dead.letter.queue"
Bind "dead.letter.queue" to "dead.letter.test"
Create new queue "test1" with the dead letter exchange set to "dead.letter.test"
Send a message into "test1"
Nack (with requeue = false) the message in "test1"
Dead Letter Exchange without routing key and with direct exchange
Follow the steps these will work for sure:-
1. Create a new queue named 'dead_queue'.
2. Create an exchange named 'dead_exchange' and type of exchange should be 'direct'.
3. Bind 'dead_queue' and 'dead_exchange' without routing key.
4. Create a new queue named 'test_queue' and set its 'x-dead-letter-exchange' name as 'dead_exchange'
5. Create an exchange named 'test_exchange' and type of exchange should be 'direct'
6. Bind 'test_exchange' and 'test_queue' without routing key.
And at last we will check it. For this publish something on 'test_exchange' with argument 'expiration' set to 10000. After this when a message is publish on 'test_exchange' it will go to 'test_queue' and when a message is expired with in a queue it will look for DLX Parameter(Dead Letter Exchange name) there that message find the name 'dead_exchange' then that message will reach 'dead_exchange' deliver it to 'dead queue' ..
If still you have any problem regarding this and if i miss understood your problem... write your problem i will surely look over it... Thanks..
Note: Must publish the message on 'test_exchange' because that test_queue and test_exchange binding is without routing key and it will work fine but If you publish message on 'test_queue' default exchange and routing key will be used.Then after expiration of message queue tries to deliver that dead message to dead_exchange with some default routing key and message will not go to that queue.
If you want to use custom routing key on dead letter exchange you have to set x-dead-letter-routing-key when declaring working queue (in your case it is test1), otherwise default routing key will be used. In your case RabbitMQ broker detects cycling and simply drop rejected messages.
What you need is to have x-dead-letter-exchange=dead.letter.test and x-dead-letter-routing-key=dead.letter.queue arguments set on test1 queue.
If you want all your queues to have same dead letter exchange it is easier to set a general policy:
sudo rabbitmqctl -p /my/vhost/path set_policy DLX ".*" '{"dead-letter-exchange":"MyExchange.DEAD"}' --apply-to queues
Don't need to create FANOUT exchange if it is not compulsory.
You can create DIRECT exchange using the same routing key which you have used already for other exchange. And also don't need to create a new queue for the new exchange. You can use existing queues with new exchange. You just need to bind that new exchange with the queue.
Here is my receive.js file:
var amqp = require("amqplib/callback_api");
var crontab = require('node-crontab');
amqp.connect("amqp://localhost", function (err, conn) {
conn.createChannel(function (err, ch) {
var ex = 'direct_logs';
var ex2 = 'dead-letter-test';
var severity = 'enterprise-1-key';
//assert "direct" exchange
ch.assertExchange(ex, 'direct', { durable: true });
//assert "dead-letter-test" exchange
ch.assertExchange(ex2, 'direct', { durable: true });
//if acknowledgement is nack() then message will be stored in second exchange i.e. ex2="dead-letter-test"
ch.assertQueue('enterprise-11', { exclusive: false, deadLetterExchange: ex2 }, function (err, q) {
var n = 0;
console.log(' [*] Waiting for logs. To exit press CTRL+C');
console.log(q);
//Binding queue with "direct_logs" exchange
ch.bindQueue(q.queue, ex, severity);
//Binding the same queue with "dead-letter-test"
ch.bindQueue(q.queue, ex2, severity);
ch.consume(q.queue, function (msg) {
// consume messages via "dead-letter-exchange" exchange at every second.
if (msg.fields.exchange === ex2) {
crontab.scheduleJob("* * * * * *", function () {
console.log("Received by latest exchange %s", msg.fields.routingKey, msg.content.toString());
});
} else {
console.log("Received %s", msg.fields.routingKey, msg.content.toString());
}
if (n < 1) {
// this will executes first time only. Here I'm sending nack() so message will be stored in "deadLetterExchange"
ch.nack(msg, false, false);
n += 1;
} else {
ch.ack(msg)
n = 0
}
}, { noAck: false });
});
});
});
Create new DIRECT exchange with the name "dead.letter.test"
Correct
Create new queue "dead.letter.queue"
Correct
Bind "dead.letter.queue" to "dead.letter.test"
Correct
Create new queue "test1" with the dead letter exchange set to "dead.letter.test"
I am assuming you are creating test1 queue and binding it to dead.letter.test exchange
Send a message into "test1"
If you want your message to be received by dead.letter.queue you will have to provide routing key while sending message and clients consuming dead.letter.queue should also use same routing key
If you are publishing without routing key then only clients who are subscribed to test1 will receive the message.
If you publish message to direct.letter.test exchange then all the queue will receive the message. It will work like a fanout exchange
So, if you want dead.letter.queue to receive message you will have to publish message in that queue or you will have to use same routing key while publishing and subscribing and publish message to exchange
In my case the problem was because the queue had
ackMode="MANUAL"
But I never set it (because a runtime exception), use Defaul ACK instead.
For those who use Spring-AMQP
In my case the problem was different. I wanted a dead-letter-exchange to be of type direct. And i set both x-dead-letter-exchange and x-dead-letter-routing-key for the queue. Plus i had spring.rabbitmq.listener.simple.default-requeue-rejected=false in the application.properties.
Seems everything fine, but while debugging i noticed that my SimpleRabbitListenerContainerFactory has defaultRequeueRejected as null. So the reason was that when you declare SimpleRabbitListenerContainerFactory in your #Configuration, you create a new "non-default" bean. The default one is created for you behind the scene out of your properties. But your SimpleRabbitListenerContainerFactory in the #Config, these properties are not read, you must read it yourself and set in java code.
It happened to me, because i just copy-pasted the config from the Spring-AMQP docs when wanted to configure the concurrency. But you should do everything in one place, either in properties, like
spring.rabbitmq.listener.simple.default-requeue-rejected=false
spring.rabbitmq.listener.simple.concurrency=5
spring.rabbitmq.listener.simple.max-concurrency=10
or completely in java, like
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setConcurrentConsumers(5);
factory.setMaxConcurrentConsumers(10);
factory.setDefaultRequeueRejected(false);
return factory;
}
These 2 above are the same.
I would expect that when i use the second (java) option is still picks up properties from the application.properties and then i customize then in java, but it doesn't work like this.
And yes, "copy-paste" is evil :)