I have some doubts about Spring AMQP and the correlationId of an AMQP message.
I have a Project with two queues (“queue.A” and “queue.B”) and one MessageListener on each:
public class ServerHandlerQueueA implements MessageListener {
#Override
public void onMessage(Message msg)
public class ServerHandlerQueueB implements MessageListener {
#Override
public void onMessage(Message msg)
In some cases, when I receive a message in the “queue.A”, I have to redirect it to “queue.B”:
rabbitTemplate.convertAndSend(routingkey, msg, new MessagePostProcessor()
{ …});
In all cases I send the response to the client using the following:
String routingkey = msg.getMessageProperties().getReplyTo();
rabbitTemplate.convertAndSend(routingkey, respuesta, new MessagePostProcessor() {
#Override
public Message postProcessMessage(Message msg) throws AmqpException
{….}
});
This is working correctly if I use Spring AMQP on the client side:
Object _response = getRabbitOperations().convertSendAndReceive(requestExchange, routingKeyManagement, msg,
new MessagePostProcessor()
{
public Message postProcessMessage(Message message) throws AmqpException
{….}
});
But If I use the java client (on then client side):
RpcClient _rpcClient = new RpcClient(channel, exchangeName, routingKey);
Response _response = _rpcClient.doCall(new AMQP.BasicProperties.Builder()
.contentType("application/json")
.deliveryMode(2)
.priority(1)
.userId("myUser")
.appId("MyApp")
.replyTo(replyQueueName)
.correlationId(corrId)
.type("NewOrder")
.build(),
messageBodyBytes);
I always get a NullPointerException in:
com.rabbitmq.client.RpcClient$1.handleDelivery(RpcClient.java:195)
I think it's because of the correlationId treatment. When I send a message with Spring AMQP I can see the “spring_listener_return_correlation” and “spring_request_return_correlation” headers in the consumer, but the “correlationId” property is always null.
How can I make it compatible with the pure java client and with Spring AMQP? I am doing something wrong?
Thanks!
------ EDIT ----------
I’ve upgraded to Spring AMQP 1.7.4 version.
I send a message like this:
Object respuesta = getRabbitOperations().convertSendAndReceive(requestExchange, routingKey, _object,
new MessagePostProcessor()
{
public Message postProcessMessage(Message message) throws AmqpException
{
message.getMessageProperties().setUserId(“myUser”);
message.getMessageProperties().setType(“myType”);
message.getMessageProperties().setAppId("myApp");
message.getMessageProperties().setMessageId(counter.incrementAndGet() + "-myType");
message.getMessageProperties().setDeliveryMode(MessageDeliveryMode.NON_PERSISTENT);
message.getMessageProperties().setRedelivered(false);
return message;
}
});
On the server I have:
#Override
public void onMessage(Message msg)
{
MessageProperties mp = msg.getMessageProperties();
Gson __gson = new Gson();
String _stringMP = __gson.toJson(mp);
System.out.println("MessageProperties:\n" + _stringMP);
}
And I think the problem is that I always get the correlationId null:
{"headers":{"spring_listener_return_correlation":"49bd0a84-9abb-4719-b8a7-8668a4a77f32","spring_request_return_correlation":"32","__TypeId__":"MyType"},"messageId":"32-MyType","appId":"myApp","type":"MyType","replyTo":"amq.rabbitmq.reply-to.g2dkABByYWJiaXRATkRFUy1QQzAyAAAsMwAAAAgD.ia4+GgHgoeBnajbHxOgW+w\u003d\u003d","contentType":"application/json","contentEncoding":"UTF-8","contentLength":0,"contentLengthSet":false,"priority":0,"redelivered":false,"receivedExchange":"requestExchange","receivedRoutingKey":"inquiry","receivedUserId":"myUser",
"deliveryTag":5,"deliveryTagSet":true,"messageCount":0,"consumerTag":"amq.ctag-4H_P9CbWYZMML-QsmyaQYQ","consumerQueue":"inquiryQueue","receivedDeliveryMode":"NON_PERSISTENT"}
If I use the Java Client I can see the correlationId:
{"headers":{},"appId":"XBID","type":"MyOrders","correlationId":[49], ….
------------ EDIT 2 --------------------------------
I have tried with:
getRabbitOperations().convertAndSend(requestExchange, routingKeyInquiry,
_object,
new MessagePostProcessor()
{
public Message postProcessMessage(Message message) throws AmqpException
{
message.getMessageProperties().setUserId(“myUser”);
message.getMessageProperties().setType(“myType”);
message.getMessageProperties().setAppId("myApp");
message.getMessageProperties().setMessageId(counter.incrementAndGet() + "-myType");
message.getMessageProperties().setDeliveryMode(MessageDeliveryMode.NON_PERSISTENT);
message.getMessageProperties().setRedelivered(false);
message.getMessageProperties().setCorrelationIdString(UUID.randomUUID().toString());
return message;
}
});
But the "correlationId" is always null at the server side.
What version are you using?
The return correlation headers have nothing to do with correlationId; they are used to correlate returned (mandatory) requests and replies.
As long as you copy the correlationId and replyTo from the queue.A message to the queue.B message, it should all work ok.
If you can't figure it out, post debug logs from all 3 servers someplace.
EDIT
This works fine for me...
#SpringBootApplication
public class So46316261Application implements CommandLineRunner {
public static void main(String[] args) {
SpringApplication.run(So46316261Application.class, args).close();
}
#Autowired
private RabbitTemplate template;
#Override
public void run(String... arg0) throws Exception {
Object reply = this.template.convertSendAndReceive("queue.A", "foo");
System.out.println(reply);
Connection conn = this.template.getConnectionFactory().createConnection();
Channel channel = conn.createChannel(false);
RpcClient client = new RpcClient(channel, "", "queue.A");
Response response = client.doCall(new AMQP.BasicProperties.Builder()
.contentType("text/plain")
.deliveryMode(2)
.priority(1)
.userId("guest")
.appId("MyApp")
.replyTo("amq.rabbitmq.reply-to")
.correlationId("bar")
.type("NewOrder")
.build(),
"foo".getBytes());
System.out.println(new String(response.getBody()));
channel.close();
conn.close();
}
#Bean
public Queue queueA() {
return new Queue("queue.A");
}
#Bean
public Queue queueB() {
return new Queue("queue.B");
}
#RabbitListener(queues = "queue.A")
public void listen(Message in) {
System.out.println(in);
this.template.send("queue.B", in);
}
#RabbitListener(queues = "queue.B")
public String listenB(Message in) {
System.out.println(in);
return "FOO";
}
}
(Body:'foo' MessageProperties [headers={}, correlationId=1, replyTo=amq.rabbitmq.reply-to.g2dkABByYWJiaXRAbG9jYWxob3N0AAACyAAAAAAB.hp0xZxgVpXcuj9+5QkcOOw==, contentType=text/plain, contentEncoding=UTF-8, contentLength=0, priority=0, redelivered=false, receivedExchange=, receivedRoutingKey=queue.B, deliveryTag=1, consumerTag=amq.ctag-oanHvT3YyUb_Lajl0gpZSQ, consumerQueue=queue.B])
FOO
(Body:'foo' MessageProperties [headers={}, appId=MyApp, type=NewOrder, correlationId=1, replyTo=amq.rabbitmq.reply-to.g2dkABByYWJiaXRAbG9jYWxob3N0AAACzAAAAAAB.okm02YXf0s0HdqZynVIn2w==, contentType=text/plain, contentLength=0, priority=1, redelivered=false, receivedExchange=, receivedRoutingKey=queue.B, deliveryTag=2, consumerTag=amq.ctag-oanHvT3YyUb_Lajl0gpZSQ, consumerQueue=queue.B])
FOO
Related
I am using Spring Rabbit 2.3.6 for my spring application. I have list of Comment entity and send to message queue via this function
public void deletePost(long postId){
Post post = postRepository.findById(postId)
.orElseThrow(() -> new PostNotFoundException(String.valueOf(postId)));
List<Comment> comments = postBase.getComments(post);
postRepository.delete(post);
//post event
Map<String, Object> inputs= new HashMap<>();
inputs.put(InputParam.OBJECT_ID, postId);
inputs.put(InputParam.COMMENTS, comments);
messagingUtil.postEvent(SnwObjectType.POST, SnwActionType.DELETE, inputs);
}
Listening message
#RabbitListener(queues = MessagingConfig.QUEUE)
public void consumeMessageFromQueue(Map<String, Object> inputs) {
}
And this is data I received after listening message. Data type of each element in comment list have been changed, and I can't use it as Comment.
Have you any solution to data type from not being changed?
Updated: Messaging config
#Configuration
public class MessagingConfig {
public static final String QUEUE = "javatechie_queue";
public static final String EXCHANGE = "javatechie_exchange";
public static final String ROUTING_KEY = "javatechie_routingKey";
#Bean
public Queue queue() {
return new Queue(QUEUE);
}
#Bean
public TopicExchange exchange() {
return new TopicExchange(EXCHANGE);
}
#Bean
public Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(ROUTING_KEY);
}
#Bean
public MessageConverter converter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public AmqpTemplate template(ConnectionFactory connectionFactory) {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(converter());
return rabbitTemplate;
}
}
I want to be sure the message is reaching a queue.
Otherwise, I want an exception.
I have tried publisher returns, but it is not what I need, because it is on a different thread and I think it would be tricky to somehow wait for it on the thread sent the message.
Without the transacted channel, the convertAndSend method returned successfully when the exchange did not be there, with the transacted channel now it throws an exception.
What I need is the same when there is no route based on the routing key.
#SpringBootApplication
public class DemoApplication {
private static final Logger log = Logger.getGlobal();
public static void main(String[] args) {
SpringApplication.run(DemoApplication.class, args);
}
#Bean
RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> log.info(replyCode + "," + replyText));
rabbitTemplate.setChannelTransacted(true);
rabbitTemplate.setMandatory(true);
return rabbitTemplate;
}
#Bean
CommandLineRunner commandLineRunner(RabbitTemplate rabbitTemplate) {
return args -> {
rabbitTemplate.convertAndSend("exchangeName", "routingKey", "message");
log.info("Send is done.");
};
}
}
only property: spring.rabbitmq.publisher-returns=true
Spring boot version: 2.1.7.RELEASE
Actual:
no exchange -> convertAndSend throws exception
no route at exchange -> method returns
Expected:
no exchange -> convertAndSend throws exception
no route at exchange -> convertAndSend throws exception
You need to use publisher confirms and correlation data:
spring.rabbitmq.publisher-returns=true
spring.rabbitmq.publisher-confirms=true
spring.rabbitmq.template.mandatory=true
#SpringBootApplication
public class So57464212Application {
public static void main(String[] args) {
SpringApplication.run(So57464212Application.class, args);
}
#Bean
public ApplicationRunner runner(RabbitTemplate template) {
template.setReturnCallback((message, replyCode, replyText, exchange, routingKey) -> {
System.err.println("Returned: " + replyText);
});
template.setConfirmCallback((correlationData, ack, cause) -> {
System.err.println("ack:" + ack);
});
return args -> {
CorrelationData correlationData = new CorrelationData("foo");
template.convertAndSend("", "NOQUEUE", "bar", correlationData);
correlationData.getFuture().get(10, TimeUnit.SECONDS);
if (correlationData.getReturnedMessage() != null) {
throw new RuntimeException("Message was returned");
}
};
}
}
I am new to spring-amqp. I am trying to manually acknowledge the messages instead of using auto-ack.
I am seeing that the last message is being unacked in the management console.
image for unacked message in managemnet console.
but the queue is empty.
As soon as I stop the server the last message gets acknowledged. How do I handle this and how can I print in logs ,the message id/information which has been unacknowledged..
Here is the code which I have implemented.
RabbitConfig.java:
public class RabbitMQConfig {
final static String queueName = "spring-boot";
#Bean
Queue queue() {
return new Queue(queueName, true,false,false,null);
}
#Bean
TopicExchange exchange() {
return new TopicExchange("spring-boot-exchange");
}
#Bean
Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(queueName);
}
#Bean
SimpleMessageListenerContainer container(ConnectionFactory connectionFactory,
MessageListenerAdapter listenerAdapter) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(queueName);
container.setMessageListener(listenerAdapter);
container.setAcknowledgeMode(AcknowledgeMode.MANUAL);
return container;
}
#Bean
Consumer receiver() {
return new Consumer();
}
#Bean
MessageListenerAdapter listenerAdapter(Consumer receiver) {
return new MessageListenerAdapter(receiver, "receiveMessage");
}
Consumer.java
public class Consumer implements ChannelAwareMessageListener{
#RabbitListener(queues = "spring-boot")
public void receiveMessage(String message, Channel channel, #Header(AmqpHeaders.DELIVERY_TAG) long tag)
throws IOException, InterruptedException {
Thread.sleep(500);
channel.basicAck(tag, true);
System.out.println(tag + "received");
}
#Override
public void onMessage(Message arg0, Channel arg1) throws Exception {
// TODO Auto-generated method stub
}
Producer endpoints:
#RestController
public class HelloController {
private final RabbitTemplate rabbitTemplate;
public HelloController(RabbitTemplate rabbitTemplate) {
this.rabbitTemplate = rabbitTemplate;
}
// Call this end point from the postman or the browser then check in the
// rabbitmq server
#GetMapping(path = "/hello")
public String sayHello() throws InterruptedException {
// Producer operation
for (int i = 0; i < 100; i++) {
Thread.sleep(500);
rabbitTemplate.convertAndSend(RabbitMQConfig.queueName, "Hello World");
}
return "hello";
}
#GetMapping(path = "/hellotwo")
public String sayHellotwo() throws InterruptedException {
// Producer operation
for (int i = 0; i < 50; i++) {
Thread.sleep(500);
rabbitTemplate.convertAndSend(RabbitMQConfig.queueName, "SEcond message");
}
return "hellotwo";
}
You have two listener containers; the container bean and one created by the framework for the #RabbitListener.
I am not entirely sure what's happening without running a test myself, but I suspect the problem is your attempt to call receiveMessage from the simple MessageListenerAdapter.
That adapter is only designed to call a method with one argument (converted from the Message). Also, that adapter doesn't know how to map #Header parameters. I suspect that delivery fails and since you are using MANUAL acks, no more deliveries are attempted to that container because of the unack'd delivery and the default qos (1).
You don't need your container bean; instead configure the message listener container factory to set the ack mode. See the documentation.
If you are new to spring-amqp; why do you think you need manual acks? The default mode (auto) means the container will ack/nack for you (NONE is traditional rabbit auto-ack). It is not common to use manual acks with Spring.
I use RabbitMQ for connection between parts my program. Version of RMQ(3.3.5). It used with java client from repo.
// Connection part
#Inject
public AMQService(RabbitMQConfig mqConfig) throws IOException {
this.mqConfig = mqConfig;
connectionFactory.setHost(mqConfig.getRABBIT_HOST());
connectionFactory.setUsername(mqConfig.getRABBIT_USERNAME());
connectionFactory.setPassword(mqConfig.getRABBIT_PASSWORD());
connectionFactory.setAutomaticRecoveryEnabled(true);
connectionFactory.setPort(mqConfig.getRABBIT_PORT());
connectionFactory.setVirtualHost(mqConfig.getRABBIT_VHOST());
Connection connection = connectionFactory.newConnection();
channel = connection.createChannel();
channel.basicQos(1);
}
//Consume part
private static void consumeResultQueue() {
final QueueingConsumer consumer = new QueueingConsumer(channel);
Future resultQueue = EXECUTOR_SERVICE.submit((Callable<Object>) () -> {
channel.basicConsume("resultQueue", true, consumer);
while (true) {
try {
QueueingConsumer.Delivery delivery = consumer.nextDelivery();
String message = new String(delivery.getBody(), "UTF-8");
resultListener.onMessage(message);
} catch (IOException | InterruptedException e) {
e.printStackTrace();
}
}
});
}
I want leave use inifinty loop. Can RMQ notify client while message can read from queue? Without check?
You can create a class which extends DefaultConsumer and override handleDelivery.
public class MyConsumer extends DefaultConsumer {
public MyConsumer(Channel channel) {
super(channel);
}
#Override
public void handleDelivery(String consumerTag, Envelope envelope,
AMQP.BasicProperties properties, byte[] body) throws IOException {
// do your computation
}
}
And register this consumer with channel.basicConsume(queueName, myConsumerInstance);
Note that by doing this, handleDelivery will run inside rabbitmq client thread pool so you should avoid any long computation inside this function.
some configurations here:
non-durable consumer,non-persistent message,disabled flow control,default prefetch size,optimizeAcknowledge = true,asynsend = true, use jms to connect ActiveMQ
for example,
one producer、one consumer，
Producer————Topic————consumer
the producer send rate can reach 6k/s
but,in this case:
one producer three consumer,
/——consumer
Producer——-Topic——-consumer
\——consumer
the producer send rate drop down to 4k/s
Here is my some of the key code:
sender class:
public class sender {
public Boolean durable=false;
public String clientID=null;
public Boolean transacted=false;
public int ackMode=Session.AUTO_ACKNOWLEDGE;
public int timeToLive=0;
public String queuename = "";
public int persistent = DeliveryMode.NON_PERSISTENT;
public Connection createConnection(String user,String pwd,String url) throws JMSException, Exception {
ActiveMQConnectionFactory connectionFactory = new ActiveMQConnectionFactory(user, pwd, url);
connectionFactory.setDispatchAsync(true);
//connectionFactory.setAlwaysSessionAsync(false);
Connection connection = connectionFactory.createConnection();
if (durable && clientID!=null) {
connection.setClientID(clientID);
}
connection.start();
return connection;
}
public Session createSession(Connection connection) throws Exception {
Session session = connection.createSession(transacted, ackMode);
return session;
}
public MessageProducer createProducer(Session session) throws JMSException {
Queue destination = session.createQueue(queuename);
MessageProducer producer = session.createProducer(destination);
producer.setDeliveryMode(persistent);
if( timeToLive!=0 )
producer.setTimeToLive(timeToLive);
return producer;
}
public void onMessage(Message message) {
//process message
}
}
sendmain method：
public static void main(String[] args) throws JMSException, Exception {
// TODO Auto-generated method stub
sender s = new sender();
s.persistent = DeliveryMode.NON_PERSISTENT;
Connection c = s.createConnection("","","tcp://localhost:61616?jms.useAsyncSend=true");
Session sess = s.createSession(c);
Topic topic = sess.createTopic("topic.test");
MessageProducer mp = sess.createProducer(topic);
StringBuffer tmpsb=new StringBuffer();
for (int j=0;j<1024;j++)
{
tmpsb.append("0");
}
Message m = sess.createTextMessage(tmpsb.toString());
long pre=System.currentTimeMillis();
for (int i=0;i<10000;i++)
{
mp.send(m);
}
long post=System.currentTimeMillis();
mp.close();
System.out.println("sendtime:"+(post-pre));
System.out.println("sendrate:"+10000000/(float)(post-pre));
System.out.println("timenow:"+(post));
}
receiver class code:
public class receiver implements MessageListener
{
public int receivetimes=0;
public long pretime;
public void onMessage(Message msg)
{
//TextMessage tm = (TextMessage) msg;
try {
if (receivetimes==0)
{
pretime=System.currentTimeMillis();
}
receivetimes+=1;
if (receivetimes==10000)
{
long now=System.currentTimeMillis();
System.out.println("time:"+(now-pretime)+"\nrecive rate:"+9999999/(float)(now-pretime));
System.out.println("timenow:"+(now));
receivetimes=0;
}
} catch(Throwable t) {
t.printStackTrace();
}
}
}
receiver class code here has hide some methods,such as createConnection,createSession or something just like sender class does.
receiver main method:
public static void main(String[] args) throws JMSException, Exception {
// TODO Auto-generated method stub
receiver s = new receiver();
Connection c = s.createConnection("","","tcp://localhost:6151?jms.optimizeAcknowledge=true");
Session sess = s.createSession(c);
Topic destination = sess.createTopic("topic.test");
MessageConsumer consumer = sess.createConsumer(destination);
consumer.setMessageListener(new receiver());
}
Every consumer is in a standalone process. I ran three consumer and one producer then I got a result of bad performance. Does any one knows why I get this?
Just as #TimBish said.
The issue is 'producer,consumer,activemq-server on same machine'.When separated,the issue never shows up.
It is so important to test something in a strict way.......