My requirement is to listen to two queues and process the messages
I have created two containers
#Bean("listenerForIncomingMessages")
public SimpleRabbitListenerContainerFactory listenerForIncomingMessages() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(2);
factory.setMaxConcurrentConsumers(2);
factory.setAcknowledgeMode(AcknowledgeMode.MANUAL);
factory.setPrefetchCount(1);
return factory;
}
#Bean("listenerForMissedDeliveries")
public SimpleRabbitListenerContainerFactory listenerForMissedDeliveries() {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory());
factory.setConcurrentConsumers(2);
factory.setMaxConcurrentConsumers(2);
factory.setAcknowledgeMode(AcknowledgeMode.MANUAL);
factory.setPrefetchCount(1);
return factory;
}
I have two #Configuration classes to process the messages
#Configuration
public class ClassOne {
#RabbitListener(containerFactory = "listenerForIncomingMessages", queues = "IncomingMessage")
public void getAmqpMessage(Message inMessage, Channel channel) throws Exception {
}
}
#Configuration
public class ClassTwo {
#RabbitListener(containerFactory = "listenerForMissedDeliveries", queues = "MissedDelivery")
public void getAmqpMessage(Message inMessage, Channel channel) throws Exception {
}
}
The ClassTwo's RabbitListener works only once during server start. After that it is not working -- The messages are arriving at the queue but they are
not consumed.
What am i doing wrong?
Any pointers to get this working.
Thanks,
Related
I am trying to run an integration test for my RabbitListener in Spring AMQP while the broker is not running so I am using TestRabbitTemplate. I am using a Jacksonized object and the Jackson2JsonMessageConverter was set for the TestRabbitTemplate. However upon sending a message to the exchange, I am getting the following exception.
org.springframework.amqp.rabbit.support.ListenerExecutionFailedException: Listener method could not be invoked with the incoming message
Caused by: org.springframework.messaging.converter.MessageConversionException: Cannot convert from [[B] to [com.murex.em.demo.springamqpdemo.message.CustomMessage] for GenericMessage [payload=byte[27], headers={amqp_contentEncoding=UTF-8, amqp_contentLength=27, amqp_replyTo=testRabbitTemplateReplyTo, id=fb0352d2-2abb-ae06-9b2e-03da1fc19e43, amqp_lastInBatch=false, contentType=application/json, __TypeId__=com.murex.em.demo.springamqpdemo.message.CustomMessage, timestamp=1676625212607}], failedMessage=GenericMessage [payload=byte[27], headers={amqp_contentEncoding=UTF-8, amqp_contentLength=27, amqp_replyTo=testRabbitTemplateReplyTo, id=fb0352d2-2abb-ae06-9b2e-03da1fc19e43, amqp_lastInBatch=false, contentType=application/json, __TypeId__=com.murex.em.demo.springamqpdemo.message.CustomMessage, timestamp=1676625212607}]
at org.springframework.messaging.handler.annotation.support.PayloadMethodArgumentResolver.resolveArgument(PayloadMethodArgumentResolver.java:145)
at org.springframework.amqp.rabbit.annotation.RabbitListenerAnnotationBeanPostProcessor$OptionalEmptyAwarePayloadArgumentResolver.resolveArgument(RabbitListenerAnnotationBeanPostProcessor.java:1053)
at org.springframework.messaging.handler.invocation.HandlerMethodArgumentResolverComposite.resolveArgument(HandlerMethodArgumentResolverComposite.java:118)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.getMethodArgumentValues(InvocableHandlerMethod.java:147)
at org.springframework.messaging.handler.invocation.InvocableHandlerMethod.invoke(InvocableHandlerMethod.java:115)
at org.springframework.amqp.rabbit.listener.adapter.HandlerAdapter.invoke(HandlerAdapter.java:77)
at org.springframework.amqp.rabbit.listener.adapter.MessagingMessageListenerAdapter.invokeHandler(MessagingMessageListenerAdapter.java:263)
... 78 more
Here are the relevant chunks of my code:
#Jacksonized
#Builder
#Value
public class CustomMessage {
String id;
String name;
}
#Component
public class InwardMessageConsumer {
#RabbitListener(bindings = #QueueBinding(
value = #Queue(name = "inward.messageQueue", durable = "true"),
exchange = #Exchange(value = "inward", type = "topic"),
key = "inwardRoutingKey")
)
public String processMessage(CustomMessage customMessage) {
return customMessage.getName();
}
}
#SpringJUnitConfig
#SpringBootTest
class BrokerNotRunningIT {
#Autowired
private TestRabbitTemplate template;
#Test
public void testSendAndReceive() {
CustomMessage customMessage = CustomMessage.builder()
.id("123")
.name("name1")
.build();
assertThat(template.convertSendAndReceive("inward", "inward.messageQueue", customMessage)).isEqualTo("name1");
}
#TestConfiguration
public static class RabbitTestConfiguration {
#Bean
public TestRabbitTemplate testRabbitTemplate(
ConnectionFactory mockConnectionFactory,
Jackson2JsonMessageConverter jackson2JsonMessageConverter
) {
TestRabbitTemplate testRabbitTemplate = new TestRabbitTemplate(mockConnectionFactory);
testRabbitTemplate.setMessageConverter(jackson2JsonMessageConverter);
return testRabbitTemplate;
}
#Bean
public ConnectionFactory mockConnectionFactory() throws IOException {
ConnectionFactory factory = mock(ConnectionFactory.class);
Connection connection = mock(Connection.class);
Channel channel = mock(Channel.class);
AMQP.Queue.DeclareOk declareOk = mock(AMQP.Queue.DeclareOk.class);
willReturn(connection).given(factory).createConnection();
willReturn(channel).given(connection).createChannel(anyBoolean());
given(channel.isOpen()).willReturn(true);
given(channel.queueDeclare(anyString(), anyBoolean(), anyBoolean(), anyBoolean(), anyMap()))
.willReturn(declareOk);
return factory;
}
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
return factory;
}
#Bean
public SimpleMessageListenerContainer simpleMessageListenerContainer(ConnectionFactory connectionFactory) {
return new SimpleMessageListenerContainer(connectionFactory);
}
}
}
Apparently I needed to set the message converter in the RabbitListenerContainerFactory as well like this:
#Bean
public SimpleRabbitListenerContainerFactory rabbitListenerContainerFactory(ConnectionFactory connectionFactory, Jackson2JsonMessageConverter jackson2JsonMessageConverter) {
SimpleRabbitListenerContainerFactory factory = new SimpleRabbitListenerContainerFactory();
factory.setConnectionFactory(connectionFactory);
factory.setMessageConverter(jackson2JsonMessageConverter);
return factory;
}
I am using Spring Rabbit 2.3.6 for my spring application. I have list of Comment entity and send to message queue via this function
public void deletePost(long postId){
Post post = postRepository.findById(postId)
.orElseThrow(() -> new PostNotFoundException(String.valueOf(postId)));
List<Comment> comments = postBase.getComments(post);
postRepository.delete(post);
//post event
Map<String, Object> inputs= new HashMap<>();
inputs.put(InputParam.OBJECT_ID, postId);
inputs.put(InputParam.COMMENTS, comments);
messagingUtil.postEvent(SnwObjectType.POST, SnwActionType.DELETE, inputs);
}
Listening message
#RabbitListener(queues = MessagingConfig.QUEUE)
public void consumeMessageFromQueue(Map<String, Object> inputs) {
}
And this is data I received after listening message. Data type of each element in comment list have been changed, and I can't use it as Comment.
Have you any solution to data type from not being changed?
Updated: Messaging config
#Configuration
public class MessagingConfig {
public static final String QUEUE = "javatechie_queue";
public static final String EXCHANGE = "javatechie_exchange";
public static final String ROUTING_KEY = "javatechie_routingKey";
#Bean
public Queue queue() {
return new Queue(QUEUE);
}
#Bean
public TopicExchange exchange() {
return new TopicExchange(EXCHANGE);
}
#Bean
public Binding binding(Queue queue, TopicExchange exchange) {
return BindingBuilder.bind(queue).to(exchange).with(ROUTING_KEY);
}
#Bean
public MessageConverter converter() {
return new Jackson2JsonMessageConverter();
}
#Bean
public AmqpTemplate template(ConnectionFactory connectionFactory) {
final RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setMessageConverter(converter());
return rabbitTemplate;
}
}
I am trying to implement a HTTP request/reply using separate RabbitMQ queues in Spring Integration DSL. It's similar to Spring IntegrationFlow http request to amqp queue. The difference is I want the response back to the original http caller. I could see the test http post message successfully passed to the request queue and transformed (into upper case) into the response queue. The message was consumed from the response queue as well but never returned back to the caller(http://localhost:8080/Tunner). Eventually the call timed out with 500 error. I am new to this so there could be something I totally missed. Could someone provide suggestion? The code is as follows:
public class TunnelApplication
{
public static void main(String[] args)
{
SpringApplication.run(TunnelApplication.class, args);
}
#Value("${outboundQueue}")
private String outboundQueue;
#Value("${inboundQueue}")
private String inboundQueue;
private ConnectionFactory rabbitConnectionFactory;
#Autowired
public TunnelApplication(ConnectionFactory factory) {
rabbitConnectionFactory = factory;
}
#Bean
public Queue targetQueue()
{
return new Queue(outboundQueue, true, false, true);
}
#Bean
public Queue requestQueue()
{
return new Queue(inboundQueue, true, false, true);
}
#Bean
public Jackson2JsonMessageConverter jsonMessageConverter()
{
return new Jackson2JsonMessageConverter();
}
#Bean
public AmqpTemplate amqpTemplate()
{
RabbitTemplate result = new RabbitTemplate(rabbitConnectionFactory);
result.setMessageConverter(jsonMessageConverter());
result.setDefaultReceiveQueue(outboundQueue);
//result.setReplyAddress(outboundQueue);
result.setReplyTimeout(60000);
return result;
}
#Bean
public IntegrationFlow sendReceiveFlow(RabbitTemplate amqpTemplate) {
return IntegrationFlows
.from(Http.inboundGateway("/tunnel"))
.handle(Amqp.outboundGateway(amqpTemplate)
.routingKey(inboundQueue)
.returnChannel(amqpOutboundChannel()))
.log()
.bridge(null)
.get();
}
#Bean
public IntegrationFlow rabbitToWeb(RabbitTemplate amqpTemplate, ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Amqp.inboundGateway(connectionFactory, requestQueue()))
.transform(String.class, String::toUpperCase)
.log()
.handle(Amqp.outboundGateway(amqpTemplate).routingKey(outboundQueue))
.log()
.bridge(null)
.get();
}
#Bean
public IntegrationFlow replyBackToHttp(RabbitTemplate amqpTemplate, ConnectionFactory connectionFactory) {
return IntegrationFlows.from(Amqp.inboundGateway(connectionFactory, targetQueue()))
.handle(Http.outboundGateway("http://localhost:8080/tunnel")
.expectedResponseType(String.class))
.log()
.bridge(null)
.channel(amqpOutboundChannel())
.get();
}
#Bean
public MessageChannel amqpOutboundChannel() {
return new DirectChannel();
}
We have also tried the following code (by my coworker) and we didn't get the response either:
#Configuration
#EnableIntegration
public class FlowConfig {
#Value("${routingKey}")
private String routingKey;
#Value("${rabbitSinkChannel}")
private String rabbitSinkChannel;
#Bean
public MessageChannel rabbitSinkChannel(ConnectionFactory connectionFactory) {
return
Amqp
.channel(rabbitSinkChannel, connectionFactory)
.get();
}
#Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
return new RabbitTemplate(connectionFactory);
}
#Bean
public IntegrationFlow httpFlow(RabbitTemplate rabbitTemplate, ConnectionFactory connectionFactory) {
MessageChannel rabbitSinkChannel = rabbitSinkChannel(connectionFactory);
return IntegrationFlows
.from(
Http.inboundGateway("/sendreceive")
)
.handle(
Amqp.outboundGateway(rabbitTemplate)
.routingKey(routingKey)
.returnChannel(rabbitSinkChannel)
)
.channel(rabbitSinkChannel) // or .handle? if so, what?
.get();
}
}
Here's the almost complete solution to anyone that's frustrated and just wants to move on.
package com.scdf.poc.config;
import org.springframework.amqp.rabbit.connection.ConnectionFactory;
import org.springframework.amqp.rabbit.core.RabbitTemplate;
import org.springframework.amqp.rabbit.listener.SimpleMessageListenerContainer;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.amqp.dsl.Amqp;
import org.springframework.integration.config.EnableIntegration;
import org.springframework.integration.dsl.IntegrationFlow;
import org.springframework.integration.dsl.IntegrationFlows;
import org.springframework.integration.http.dsl.Http;
#Configuration
#EnableIntegration
public class FlowConfig {
#Value("${rabbitSource}")
private String rabbitSource;
#Value("${rabbitSink}")
private String rabbitSink; // Note: Has to be manually created in rabbit mq, the SCDF flows won't auto create this
#Bean
public RabbitTemplate rabbitTemplate(ConnectionFactory connectionFactory) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(connectionFactory);
rabbitTemplate.setReplyAddress(rabbitSink);
return rabbitTemplate;
}
#Bean
public SimpleMessageListenerContainer simpleMessageListenerContainer(RabbitTemplate rabbitTemplate, ConnectionFactory connectionFactory) {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(connectionFactory);
container.setQueueNames(rabbitSink);
container.setMessageListener(rabbitTemplate);
return container;
}
#Bean
public IntegrationFlow httpFlow(RabbitTemplate rabbitTemplate) {
return IntegrationFlows
.from(
Http.inboundGateway("/sendreceive")
.requestPayloadType(String.class)
)
.handle(
Amqp.outboundGateway(rabbitTemplate)
.routingKey(rabbitSource)
)
.get();
}
}
application.properties - note that SCDF uses the stream name as both prefix and suffix for queue names
rabbitSource=pocStream.rabbitSource.pocStream
rabbitSink=pocStream.rabbitSink.pocStream
SCDF stream definition for pocStream - this just echos the request back
rabbitSource: rabbit --queues=rabbitSource | bridge | rabbitSink: rabbit --routing-key=pocStream.rabbitSink.pocStream
You probably misunderstood what is returnChannel on the Amqp.outboundGateway and try to rely your logic on it. Please, make yourself familiar with that Publisher Confirms and Returns feature: https://docs.spring.io/spring-amqp/docs/current/reference/html/#cf-pub-conf-ret.
It is also not clear what is a replyBackToHttp flow purpose, but it confuses at the moment with mixed references to other beans.
You probably need to investigate what is a request-reply configuration from Spring AMQP respective and you would probably don't try to use another queue for replies. Although it is still possible: see replyAddress property or RabbitTemplate: https://docs.spring.io/spring-amqp/docs/current/reference/html/#request-reply
The following update works (I also removed the replyBackToHttp() method):
#Bean
public AmqpTemplate amqpTemplate()
{
RabbitTemplate rabbitTemplate = new RabbitTemplate(rabbitConnectionFactory);
rabbitTemplate.setMessageConverter(jsonMessageConverter());
//result.setDefaultReceiveQueue(outboundQueue);
rabbitTemplate.setReplyAddress(outboundQueue);
rabbitTemplate.setReplyTimeout(60000);
rabbitTemplate.setUseDirectReplyToContainer(false);
return rabbitTemplate;
}
#Bean
public SimpleMessageListenerContainer replyListenerContainer() {
SimpleMessageListenerContainer container = new SimpleMessageListenerContainer();
container.setConnectionFactory(rabbitConnectionFactory);
container.setQueues(replyQueue());
container.setMessageListener((MessageListener) amqpTemplate());
return container;
}
I was wondering if it is possible in Spring AMQP to receive messages from the same queue in multiple classes depending on the payload type.
I am aware of using the #RabbitListener annotation in class and then putting #RabbitHandler on methods, but I would like to split complexity of message handling in multiple classes while keeping a single queue.
Version currently in use: Spring AMQP v2.0.3 along with RabbitMQ.
Well, it isn't possible. The way you would like it won't be a queue then.
That is really an architecture decision to design a single listener and distribute to its methods according the payload type.
As a workaround I can suggest you to delegate the logic from a single #RabbitListener class to those business services:
#RabbitListener(queues = "foo")
public class MyListener {
private final ServiceA serviceA;
private final ServiceB serviceB;
public MyListener(ServiceA serviceA, ServiceB serviceB) {
this.serviceA = serviceA;
this.serviceB = serviceB;
}
#RabbitHandler
public void handleA(A a) {
this.serviceA.handle(a);
}
#RabbitHandler
public void handleB(B b) {
this.serviceB.handle(b);
}
}
Yes you can, but it takes a little different approach:
You need lo listen to the generic Message Type, do some switching, and your own deserialisation. You can of course completly hide that code somewhere (baseclass, annotations...)
Example below can be extended to listen to whatever types extra.
The example above would be to filter on A and B DTO types.
void receive(ADTO dto)
{
System.out.print(dto.url);
}
void receive(BDTO dto)
{
System.out.print(dto.url);
}
#RabbitListener(queues = "your.queue.name")
public void listenMesssage(Message message)
{
try
{
String typeId = message.getMessageProperties().getHeaders().get("__TypeId__").toString();
String contentType = message.getMessageProperties().getContentType();
if (contentType != "application/json" || typeId == null || !typeId.contains(ADTO.class.toString()))
{
//TODO log warning
System.out.print("type not supported by this service");
return;
}
Object receivedObject = new Jackson2JsonMessageConverter().fromMessage(message);
if (receivedObject instanceof ADTO)
{
receive((ADTO)receivedObject);
System.out.print("ADTO");
}
//else
Alternatively, you can also do the serialisation like this:
....
String typeId = message.getMessageProperties().getHeaders().get("__TypeId__").toString();
byte[] binMsg = message.getBody();
String strMsg = new String(binMsg, StandardCharsets.UTF_8);
ObjectMapper mapper = new ObjectMapper();
if (typeId.contains("ADTO"))
{
receive(mapper.readValue(strMsg, ADTO.class ));
}
else
...
Alternatively, I found out, if you have a high enough version of AQMP and Rabbit
(mine = spring-rabit 2.1.3, spring-starter-aqmp 2.1.2) you can do this polymorph:
#RabbitListener(id="multi", queues = "somequeuename")
public class SomeService
{
#RabbitHandler
public void handleADTO(#Payload ADTO adto) {
System.out.print(adto.url);
}
#RabbitHandler
public void handleADTO2(#Payload ADTO2 adto) {
System.out.print(adto.url);
}
}
However, out of the box, this does not work. You need to configure another "bag of Beans":
#Bean
public SimpleRabbitListenerContainerFactory myFactory(
SimpleRabbitListenerContainerFactoryConfigurer configurer, ConnectionFactory connectionFactory) {
SimpleRabbitListenerContainerFactory factory =
new SimpleRabbitListenerContainerFactory();
configurer.configure(factory, connectionFactory);
return factory;
}
#Bean
Jackson2JsonMessageConverter jsonMessageConverter() {
return new Jackson2JsonMessageConverter();
}
#Bean
RabbitTemplate rabbitTemplate(Jackson2JsonMessageConverter converter, ConnectionFactory connectionFactory) {
RabbitTemplate template = new RabbitTemplate(connectionFactory);
template.setMessageConverter(new Jackson2JsonMessageConverter());
return template;
}
Just put it in your app, or some other class.
See also
https://docs.spring.io/spring-amqp/docs/current/reference/html/
I been working on implementing a PUB/SUB service using spring-data-Redis.
I have been researching and following the web and got something to work fine.
my problem is that I need absolute reliability when a message is not processed ( either an Exception is thrown or a logic error occurs ).
In which case I need the message to return to the topic for a retry ( by another subscriber or even the same ).
I have looked at several questions, particularly the following:
Redis Pub/Sub with Reliability
and
How to implement Redis Multi-Exec by using Spring-data-Redis
I have understood that I should use multi, exec for managing a transaction, but I couldn't get it to work.
Here is a simplified version of my code
#Configuration
#PropertySource(value = { "classpath:application.properties" })
public class RedisConfig {
#Autowired
Environment env;
#Bean
public MessageListenerAdapter messageListener() {
MyMessageListenerAdapter messageListeneradapter = new MyMessageListenerAdapter(new RedisMessageSubscriber());
messageListeneradapter.afterPropertiesSet();
return messageListeneradapter;
}
#Bean(name="RedisMessagePublisherBean")
public RedisMessagePublisher messagePublisher() {
return new RedisMessagePublisher();
}
#Bean
public RedisTemplate<String, Object> redisTemplate() {
RedisTemplate<String , Object> template = new RedisTemplate<>();
template.setValueSerializer(new GenericToStringSerializer<Object>(Object.class));
template.setEnableTransactionSupport(true);
template.setConnectionFactory(lettuceConnectionFactory());
return template;
}
#Bean
public RedisMessageListenerContainer redisContainer() {
RedisMessageListenerContainer container
= new RedisMessageListenerContainer();
container.setConnectionFactory(lettuceConnectionFactory());
container.addMessageListener(messageListener(), topic());
return container;
}
#Bean
public LettuceConnectionFactory lettuceConnectionFactory() {
LettuceConnectionFactory factory = new LettuceConnectionFactory();
factory.setValidateConnection(true);
factory.setDatabase(1);
factory.afterPropertiesSet();
return factory;
}
#Bean
public ChannelTopic topic() {
return new ChannelTopic("MQ_TOPIC");
}
public class MyMessageListenerAdapter extends MessageListenerAdapter{
public MyMessageListenerAdapter(RedisMessageSubscriber redisMessageSubscriber) {
super(redisMessageSubscriber);
}
#Override
public void onMessage(Message message, byte[] pattern) {
RedisTemplate<?, ?> template = redisTemplate();
template.execute(new SessionCallback<String>() {
#Override
public <K, V> String execute(RedisOperations<K, V> operations) throws DataAccessException {
operations.multi();
System.out.println("got message");
String result = doSomeLogic(message);
if (result == null)
operations.discard();
else
operations.exec();
return null;
}
}) ;
}
}
}
My requirements are that if a message failed to process ( I can leave without runtime exceptions etc.. strictly logical error would suffice for now ), It will return to the topic.
Any help is appreciated, Thanks!