I'm trying to add custom headers on my message, so whenever an exception occurs and it ends up in the dead-letter-queue, I can see what the exception was. However all my attempts at this have failed.
using .setHeader()
setting header on the outMessage
setting property of the exchange
Setting the exception as a property in the payload is not allowed.
#Component
public class ProcessRoute extends RouteBuilder {
...
#Override
public void configure() throws Exception {
onException(Exception.class)
.log("Error for ${body}! Requeue")
.redeliveryDelay(2000)
.maximumRedeliveries(3)
.handled(true)
.setHeader("TEST", constant("TEST"))
.process(e -> {
e.getOut().setHeader("TEST", "TEST");
e.setProperty("TEST","TEST");
});
from(SOME_ROUTE)
.doSomeStuff()
.to(RABBITMQ);
}
...
}
RABBITMQ-string:
rabbitmq://foo
?exchangeType=topic
&addresses=localhost:1234
&routingKey=#
&autoDelete=false
&queue=bar
&autoAck=false
&deadLetterExchange=DLX
&deadLetterQueue=bar.dlq
&deadLetterExchangeType=direct
&deadLetterRoutingKey=#
&username=foo
&password=bar
Resulting message on the dead-letter-queue:
If you use a header key following the pattern that the Camel RabbitMQ component has established, then your custom header will get picked up when the message is published to RabbitMQ.
Taking from your code above, instead of:
.setHeader("TEST", constant("TEST"))
Do this:
.setHeader("rabbitmq.TEST", constant("TEST"))
The Camel RabbitMQ component seems to ignore all the other non- "rabbitmq.*" headers that might be on the Camel exchange, and probably for good reason. There could be quite a few and most of them wouldn't make sense in the context of a message published to RabbitMQ.
Related
I'd like to know what is the canonical way to handle errors in the following situation (code is a minimal working example):
Messages are sent through a messaging gateway which defines its defaultRequestChannel and a #Gateway method:
#MessagingGateway(name = MY_GATEWAY, defaultRequestChannel = INPUT_CHANNEL)
public interface MyGateway
{
#Gateway
public void sendMessage(String message);
Messages are read from the channel and sent through an AMQP outbound adapter:
#Bean
public IntegrationFlow apiMutuaInputFlow()
{
return IntegrationFlows
.from(INPUT_CHANNEL)
.handle(Amqp.outboundAdapter(rabbitConfig.myTemplate()))
.get();
}
The RabbitMQ configuration is skeletal:
#Configuration
public class RabbitMqConfiguration
{
#Autowired
private ConnectionFactory rabbitConnectionFactory;
#Bean
public RabbitTemplate myTemplate()
{
RabbitTemplate r = new RabbitTemplate(rabbitConnectionFactory);
r.setExchange(INPUT_QUEUE_NAME);
r.setConnectionFactory(rabbitConnectionFactory);
return r;
}
}
I generally include a bean to define the RabbitMQ configuration I'm relying upon (exchange, queues and bindings), and it actually works fine. But while testing for failure scenarios, I found a situation I don't know how to properly handle using Spring Integration. The steps are:
Remove the beans that configure RabbitMQ
Run the flow against an unconfigured, vanilla RabbitMQ instance.
What I would expect is:
The message cannot be delivered because the exchange cannot be found.
Either I find some way to get an exception from the messaging gateway on the caller thread.
Either I find some way to otherwise intercept this error.
What I find:
The message cannot be delivered because the exchange cannot be found, and indeed this error message is logged every time the #Gateway method is called.
2020-02-11 08:18:40.746 ERROR 42778 --- [ 127.0.0.1:5672] o.s.a.r.c.CachingConnectionFactory : Channel shutdown: channel error; protocol method: #method<channel.close>(reply-code=404, reply-text=NOT_FOUND - no exchange 'my.exchange' in vhost '/', class-id=60, method-id=40)
The gateway is not failing, nor have I find a way to configure it to do so (e.g.: adding throws clauses to the interface methods, configuring a transactional channel, setting wait-for-confirm and a confirm-timeout).
I haven't found a way to otherwise catch that CachingConectionFactory error (e.g.: configuring a transactional channel).
I haven't found a way to catch an error message on another channel (specified on the gateway's errorChannel), or in Spring Integration's default errorChannel.
I understand such a failure may not be propagated upstream by the messaging gateway, whose job is isolating callers from the messaging API, but I definitely expect such an error to be interceptable.
Could you point me in the right direction?
Thank you.
RabbitMQ is inherently async, which is one reason that it performs so well.
You can, however, block the caller by enabling confirms and returns and setting this option:
/**
* Set to true if you want to block the calling thread until a publisher confirm has
* been received. Requires a template configured for returns. If a confirm is not
* received within the confirm timeout or a negative acknowledgment or returned
* message is received, an exception will be thrown. Does not apply to the gateway
* since it blocks awaiting the reply.
* #param waitForConfirm true to block until the confirmation or timeout is received.
* #since 5.2
* #see #setConfirmTimeout(long)
* #see #setMultiSend(boolean)
*/
public void setWaitForConfirm(boolean waitForConfirm) {
this.waitForConfirm = waitForConfirm;
}
(With the DSL .waitForConfirm(true)).
This also requires a confirm correlation expression. Here's an example from one of the test cases
#Bean
public IntegrationFlow flow(RabbitTemplate template) {
return f -> f.handle(Amqp.outboundAdapter(template)
.exchangeName("")
.routingKeyFunction(msg -> msg.getHeaders().get("rk", String.class))
.confirmCorrelationFunction(msg -> msg)
.waitForConfirm(true));
}
#Bean
public CachingConnectionFactory cf() {
CachingConnectionFactory ccf = new CachingConnectionFactory(
RabbitAvailableCondition.getBrokerRunning().getConnectionFactory());
ccf.setPublisherConfirmType(CachingConnectionFactory.ConfirmType.CORRELATED);
ccf.setPublisherReturns(true);
return ccf;
}
#Bean
public RabbitTemplate template(ConnectionFactory cf) {
RabbitTemplate rabbitTemplate = new RabbitTemplate(cf);
rabbitTemplate.setMandatory(true); // for returns
rabbitTemplate.setReceiveTimeout(10_000);
return rabbitTemplate;
}
Bear in mind this will slow down things considerably (similar to using transactions) so you may want to reconsider whether you want to do this on every send (unless performance is not an issue).
I have an issue in getting the message to spring-cloud-stream spring-boot app.
I am using rabbitMq as message engine.
Message producer is a non spring-boot app, which sends a message using Spring RestTemplate.
Queue Name: "audit.logging.rest"
The consumer application is setup to listen that queue. This app is spring-boot app(spring-cloud-stream).
Below is the consumer code
application.yml
cloud:
stream:
bindings:
restChannel:
binder: rabbit
destination: audit.logging
group: rest
AuditServiceApplication.java
#SpringBootApplication
public class AuditServiceApplication {
#Bean
public ByteArrayMessageConverter byteArrayMessageConverter() {
return new ByteArrayMessageConverter();
}
#Input
#StreamListener(AuditChannelProperties.REST_CHANNEL)
public void receive(AuditTestLogger logger) {
...
}
AuditTestLogger.java
public class AuditTestLogger {
private String applicationName;
public String getApplicationName() {
return applicationName;
}
public void setApplicationName(String applicationName) {
this.applicationName = applicationName;
}
}
Below is the request being sent from the producer App in JSON format.
{"applicationName" : "AppOne" }
Found couple of issues:
Issue1:
What I noticed is the below method is getting triggered only when the method Parameter is mentioned as Object, as spring-cloud-stream is not able to parse the message into Java POJO object.
#Input
#StreamListener(AuditChannelProperties.REST_CHANNEL)
public void receive(AuditTestLogger logger) {
Issue2:
When I changed the method to receive object. I see the object is of type RMQTextMessage which cannot be parsed. However I see actual posted message within it against text property.
I had written a ByteArrayMessageConverter which even didn't help.
Is there any way to tell spring cloud stream to extract the message from RMQTextMessage using MessageConverter and get the actual message out of it.
Thanks in Advance..
RMQTextMessage? Looks like it is a part of rabbitmq-jms-client.
In case of RabbitMQ Binder you should rely only on the Spring AMQP.
Now let's figure out what your producer application is doing.
Since you get RMQTextMessage as value for the #StreamListener method that says me that the sender really uses rabbitmq-jms-client for producing, and therefore the real AMQP message in queue has that RMQTextMessage as a wrapper for real payload.
Why don't use Spring AMQP there as well?
It's a late reply but I have the exact problem and solved it by sending and receiving the messages in application/json format. use this in the spring cloud stream config.
content-type: application/json
I am trying to create a camel route to transfer a file from an FTP server to an AWS S3 storage.
I have written the following route
private static class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception
{
from("sftp://<<ftp_server_name>>&noop=true&include=<<file_name>>...")
.process(new Processor(){
#Override
public void process(Exchange ex)
{
System.out.println("Hello");
}
})
.to("aws-s3://my-dev-bucket ?
accessKey=ABC***********&secretKey=12abc********+**********");
}
The issue is, this gives me the following exception:
Exception in thread "main" org.apache.camel.FailedToCreateRouteException: Failed to create route route1 at: >>> To[aws-s3://my-dev-bucket?accessKey=ABC*******************&secretKey=123abc******************** <<< in route: Route(route1)[[From[sftp://<<ftp-server>>... because of Failed to resolve endpoint: aws-s3://my-dev-bucket?accessKey=ABC***************&secretKey=123abc************** due to: The request signature we calculated does not match the signature you provided. Check your key and signing method.
I then tried to do this the other way. i.e.writing a method like this:
public void boot() throws Exception {
// create a Main instance
main = new Main();
// enable hangup support so you can press ctrl + c to terminate the JVM
main.enableHangupSupport();
// bind MyBean into the registery
main.bind("foo", new MyBean());
// add routes
AWSCredentials awsCredentials = new BasicAWSCredentials("ABC*****************", "123abc*************************");
AmazonS3 client = new AmazonS3Client(awsCredentials);
//main.bind("client", client);
main.addRouteBuilder(new MyRouteBuilder());
main.run();
}
and invoking using the bound variable #client. This approach does not give any exceptions, but the file transfer does not work.
To make sure that there's nothing wrong with my approach, I tried aws-sqs instead of aws-s3 and that works fine (file succesfully transfers to the SQS queue)
Any idea why this is happening? Is there some basic issue with "aws-s3" connector for camel?
Have you tried of using RAW() function to wrap as like RAW(secretkey or accesskey).
It will help you to pass your keys as it is without encoding.
Any plus signs in you secret key need to be url encoded as %2B, in your case **********+*********** becomes **********%2B***********
When you configure Camel endpoints using URIs then the parameter values gets url encoded by default.
This can be a problem when you want to configure passwords as is.
To do that you can tell Camel to use the raw value, by enclosing the value with RAW(value). See more details at How do I configure endpoints which has an example also.
See Camel Documentation
Your url should looks like:
aws-s3:bucketName?accessKey=RAW(XXXX)&secretKey=RAW(XXXX)
I've a simple route that listens to a Redis channel. For some reason it's not working.
Here is my route. I verified that data is being published into the Redis channel and I can read it back using a normal Jedis subscriber. I'm running Camel inside Jetty and it is deployed as a war.
public class RedisSubscriberRoute extends RouteBuilder{
#Override
public void configure() throws Exception {
from("spring-redis://localhost:6379?command=SUBSCRIBE&channels=mychannel")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String res = exchange.getIn().getBody().toString();
System.out.println("************ " + res);
exchange.getOut().setBody(res);
}
})
.to("log:foo");
}
}
UPDATE (10-May-2013 9:56 AM EST): Adding version information
<properties>
<spring.version>3.2.2.RELEASE</spring.version>
<camel.version>2.11.0</camel.version>
<jetty.version>7.6.8.v20121106</jetty.version>
</properties>
Redis server version is 2.6.11
The sample git project is here.
https://github.com/soumyasd/camelredisdemo
UPDATE 10-May-2013 (10:18 PM EST):
As suggested in the comments below I changed the version of the spring-data to 1.0.0.RELEASE. Looks like the message is getting to the subscriber but I'm still getting an exception.
java.lang.RuntimeException: org.springframework.data.redis.serializer.SerializationException: Cannot deserialize; nested exception is org.springframework.core.serializer.support.SerializationFailedException: Failed to deserialize payload. Is the byte array a result of corresponding serialization for DefaultDeserializer?; nested exception is java.io.StreamCorruptedException: invalid stream header: 77686174
at org.apache.camel.component.redis.RedisConsumer.onMessage(RedisConsumer.java:73)[camel-spring-redis-2.11.0.jar:2.11.0]
at org.springframework.data.redis.listener.RedisMessageListenerContainer.executeListener(RedisMessageListenerContainer.java:242)[spring-data-redis-1.0.0.RELEASE.jar:]
at org.springframework.data.redis.listener.RedisMessageListenerContainer.processMessage(RedisMessageListenerContainer.java:231)[spring-data-redis-1.0.0.RELEASE.jar:]
at org.springframework.data.redis.listener.RedisMessageListenerContainer$DispatchMessageListener$1.run(RedisMessageListenerContainer.java:726)[spring-data-redis-1.0.0.RELEASE.jar:]
at java.lang.Thread.run(Thread.java:680)[:1.6.0_45]
There is something broken in the consumer with v 1.0.3.RELEASE, use 1.0.0.RELEASE instead.
The exception you are getting is something different: Camel producer uses Spring RedisTemplate, which in turn uses JdkSerializationRedisSerializer. To make it symetric, the consumer by default also uses JdkSerializationRedisSerializer to deserialize data. So if you are using Camel producer to publish data, it should work fine w/o hustle. But if you are publishing data to redis using other redis clients (or as in your case some other libraries) you have to use another serializer for the consumer. Long explanation, but to make it work is actually two lines:
from("spring-redis://localhost:6379?command=SUBSCRIBE&channels=mychannel&serializer=#serializer")
Here is a summary of what I had to change to make this work.
As pointed out by #Bilgin Ibryam - you have to use the version 1.0.0.RELEASE of spring-data-redis (as on 11-May-2013)
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-redis</artifactId>
<!-- IMPORTANT - as of 10-May-2013 the Redis Camel
component only works with version 1.0.0.RELASE -->
<version>1.0.0.RELEASE</version>
</dependency>
Other versions that I used in my pom.xml are
3.2.2.RELEASE
2.11.0
7.6.8.v20121106
If you are publishing and consuming using the Camel Redis component you don't have to declare a different serializer. In my case I was publishing from python as well as plain old Java using Jedis. I had to change as my route to include the serializer and define the serializer in my spring/camel config.
#Override
public void configure() throws Exception {
from("spring-redis://localhost:6379?command=SUBSCRIBE&channels=mychannel&serializer=#redisserializer")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String res = exchange.getIn().getBody().toString();
System.out.println("************ " + res);
exchange.getOut().setBody(res);
}
})
.to("log:foo");
}
I'm getting started with NServiceBus and have a question about the Pubsub sample.
My intention was to have multiple instances of Publisher1 running and receiving the message sent from the publisher. I also hacked the Publisher to only send messages of the eventMessage type.
But if I start the publisher and three instances of Subscriber1, only one of them gets the message at a time.
why is that? Is it a config setting or something else?
This is what I've tried which returns an exception "Exception when starting endpoint, error has been logged. Reason: Cannot configure property before the component has been configured. Please call 'Configure' first.":
using NServiceBus;
namespace Subscriber1
{
public class EndpointConfig : IConfigureThisEndpoint, AsA_Server
{
}
public class OverrideInputQueue : IWantCustomInitialization
{
public void Init()
{
Configure
.Instance
.Configurer
.ConfigureProperty<NServiceBus.Config.MsmqTransportConfig>(t => t.InputQueue, "testQueue");
}
}
}
/J
NServiceBus assumes that you have one input queue per process. Make sure that each of your subscribers are configured with a unique input queue. If not all three will be polling the same queue producing the behavior you're describing.
To do this you would probably have to copy paste sub1 to 3 different folders, modfying the app.config and start them up.
Hope this helps!
You should use this-
Configure.Instance.Configurer.ConfigureProperty<NServiceBus.Unicast.Transport.Msmq.MsmqTransport>(msmq => msmq.InputQueue, "SomeQueueHere");
Make sure you use MsmqTransport and not MsmqTransportConfig as you mentioned.