Camel Redis Component subscribe to a channel not working - redis

I've a simple route that listens to a Redis channel. For some reason it's not working.
Here is my route. I verified that data is being published into the Redis channel and I can read it back using a normal Jedis subscriber. I'm running Camel inside Jetty and it is deployed as a war.
public class RedisSubscriberRoute extends RouteBuilder{
#Override
public void configure() throws Exception {
from("spring-redis://localhost:6379?command=SUBSCRIBE&channels=mychannel")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String res = exchange.getIn().getBody().toString();
System.out.println("************ " + res);
exchange.getOut().setBody(res);
}
})
.to("log:foo");
}
}
UPDATE (10-May-2013 9:56 AM EST): Adding version information
<properties>
<spring.version>3.2.2.RELEASE</spring.version>
<camel.version>2.11.0</camel.version>
<jetty.version>7.6.8.v20121106</jetty.version>
</properties>
Redis server version is 2.6.11
The sample git project is here.
https://github.com/soumyasd/camelredisdemo
UPDATE 10-May-2013 (10:18 PM EST):
As suggested in the comments below I changed the version of the spring-data to 1.0.0.RELEASE. Looks like the message is getting to the subscriber but I'm still getting an exception.
java.lang.RuntimeException: org.springframework.data.redis.serializer.SerializationException: Cannot deserialize; nested exception is org.springframework.core.serializer.support.SerializationFailedException: Failed to deserialize payload. Is the byte array a result of corresponding serialization for DefaultDeserializer?; nested exception is java.io.StreamCorruptedException: invalid stream header: 77686174
at org.apache.camel.component.redis.RedisConsumer.onMessage(RedisConsumer.java:73)[camel-spring-redis-2.11.0.jar:2.11.0]
at org.springframework.data.redis.listener.RedisMessageListenerContainer.executeListener(RedisMessageListenerContainer.java:242)[spring-data-redis-1.0.0.RELEASE.jar:]
at org.springframework.data.redis.listener.RedisMessageListenerContainer.processMessage(RedisMessageListenerContainer.java:231)[spring-data-redis-1.0.0.RELEASE.jar:]
at org.springframework.data.redis.listener.RedisMessageListenerContainer$DispatchMessageListener$1.run(RedisMessageListenerContainer.java:726)[spring-data-redis-1.0.0.RELEASE.jar:]
at java.lang.Thread.run(Thread.java:680)[:1.6.0_45]

There is something broken in the consumer with v 1.0.3.RELEASE, use 1.0.0.RELEASE instead.
The exception you are getting is something different: Camel producer uses Spring RedisTemplate, which in turn uses JdkSerializationRedisSerializer. To make it symetric, the consumer by default also uses JdkSerializationRedisSerializer to deserialize data. So if you are using Camel producer to publish data, it should work fine w/o hustle. But if you are publishing data to redis using other redis clients (or as in your case some other libraries) you have to use another serializer for the consumer. Long explanation, but to make it work is actually two lines:
from("spring-redis://localhost:6379?command=SUBSCRIBE&channels=mychannel&serializer=#serializer")

Here is a summary of what I had to change to make this work.
As pointed out by #Bilgin Ibryam - you have to use the version 1.0.0.RELEASE of spring-data-redis (as on 11-May-2013)
<dependency>
<groupId>org.springframework.data</groupId>
<artifactId>spring-data-redis</artifactId>
<!-- IMPORTANT - as of 10-May-2013 the Redis Camel
component only works with version 1.0.0.RELASE -->
<version>1.0.0.RELEASE</version>
</dependency>
Other versions that I used in my pom.xml are
3.2.2.RELEASE
2.11.0
7.6.8.v20121106
If you are publishing and consuming using the Camel Redis component you don't have to declare a different serializer. In my case I was publishing from python as well as plain old Java using Jedis. I had to change as my route to include the serializer and define the serializer in my spring/camel config.
#Override
public void configure() throws Exception {
from("spring-redis://localhost:6379?command=SUBSCRIBE&channels=mychannel&serializer=#redisserializer")
.process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
String res = exchange.getIn().getBody().toString();
System.out.println("************ " + res);
exchange.getOut().setBody(res);
}
})
.to("log:foo");
}

Related

How to set custom headers on RabbitMQ message using Apache Camel?

I'm trying to add custom headers on my message, so whenever an exception occurs and it ends up in the dead-letter-queue, I can see what the exception was. However all my attempts at this have failed.
using .setHeader()
setting header on the outMessage
setting property of the exchange
Setting the exception as a property in the payload is not allowed.
#Component
public class ProcessRoute extends RouteBuilder {
...
#Override
public void configure() throws Exception {
onException(Exception.class)
.log("Error for ${body}! Requeue")
.redeliveryDelay(2000)
.maximumRedeliveries(3)
.handled(true)
.setHeader("TEST", constant("TEST"))
.process(e -> {
e.getOut().setHeader("TEST", "TEST");
e.setProperty("TEST","TEST");
});
from(SOME_ROUTE)
.doSomeStuff()
.to(RABBITMQ);
}
...
}
RABBITMQ-string:
rabbitmq://foo
?exchangeType=topic
&addresses=localhost:1234
&routingKey=#
&autoDelete=false
&queue=bar
&autoAck=false
&deadLetterExchange=DLX
&deadLetterQueue=bar.dlq
&deadLetterExchangeType=direct
&deadLetterRoutingKey=#
&username=foo
&password=bar
Resulting message on the dead-letter-queue:
If you use a header key following the pattern that the Camel RabbitMQ component has established, then your custom header will get picked up when the message is published to RabbitMQ.
Taking from your code above, instead of:
.setHeader("TEST", constant("TEST"))
Do this:
.setHeader("rabbitmq.TEST", constant("TEST"))
The Camel RabbitMQ component seems to ignore all the other non- "rabbitmq.*" headers that might be on the Camel exchange, and probably for good reason. There could be quite a few and most of them wouldn't make sense in the context of a message published to RabbitMQ.

Endpoint message notification logger in mule does not get invoked

I want to log the payloads (centralized logging) being sent out from my HTTP endpoints (requester) for which I figured it is best to use the EndpointMessageNotificationListener
The server starts up correctly but my notification logger does not get called. I expect the onNotification method to be invoked each time a request is sent out from the HTTP Requester.
Now I have configured the spring bean.
<spring:bean name="endpointNotificationLogger" class="EndpointAuditor" />
My Java class is as below
public class EndpointAuditor implements EndpointMessageNotificationListener<EndpointMessageNotification> {
public EndpointAuditor(){
System.out.println("The EndpointAuditor has been instantiated");
}
#Override
public void onNotification(EndpointMessageNotification notification) {
try {
System.out.println("This comes from the endpoint " + notification.getSource().getPayloadAsString());
} catch (Exception e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
}
}
and my notification configuration as below
<notifications>
<notification event="ENDPOINT-MESSAGE" />
<notification-listener ref="endpointNotificationLogger" />
</notifications>
I have picked up all this from here.
Any ideas why Mule is not happy ?
This one turned out to be a tricky one. I was using Mule 3.6.1 EE with the new HTTP Connector and this feature is not implemented with the HTTP Connector in this particular Mule version.
So basically I was doing all the right things and Mule just did'nt have the feature.
But they have implemented it with Mule 3.6.3 EE and Mule 3.7.X. This commit shows how the code written for it.

Camel aws-s3 not working

I am trying to create a camel route to transfer a file from an FTP server to an AWS S3 storage.
I have written the following route
private static class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception
{
from("sftp://<<ftp_server_name>>&noop=true&include=<<file_name>>...")
.process(new Processor(){
#Override
public void process(Exchange ex)
{
System.out.println("Hello");
}
})
.to("aws-s3://my-dev-bucket ?
accessKey=ABC***********&secretKey=12abc********+**********");
}
The issue is, this gives me the following exception:
Exception in thread "main" org.apache.camel.FailedToCreateRouteException: Failed to create route route1 at: >>> To[aws-s3://my-dev-bucket?accessKey=ABC*******************&secretKey=123abc******************** <<< in route: Route(route1)[[From[sftp://<<ftp-server>>... because of Failed to resolve endpoint: aws-s3://my-dev-bucket?accessKey=ABC***************&secretKey=123abc************** due to: The request signature we calculated does not match the signature you provided. Check your key and signing method.
I then tried to do this the other way. i.e.writing a method like this:
public void boot() throws Exception {
// create a Main instance
main = new Main();
// enable hangup support so you can press ctrl + c to terminate the JVM
main.enableHangupSupport();
// bind MyBean into the registery
main.bind("foo", new MyBean());
// add routes
AWSCredentials awsCredentials = new BasicAWSCredentials("ABC*****************", "123abc*************************");
AmazonS3 client = new AmazonS3Client(awsCredentials);
//main.bind("client", client);
main.addRouteBuilder(new MyRouteBuilder());
main.run();
}
and invoking using the bound variable #client. This approach does not give any exceptions, but the file transfer does not work.
To make sure that there's nothing wrong with my approach, I tried aws-sqs instead of aws-s3 and that works fine (file succesfully transfers to the SQS queue)
Any idea why this is happening? Is there some basic issue with "aws-s3" connector for camel?
Have you tried of using RAW() function to wrap as like RAW(secretkey or accesskey).
It will help you to pass your keys as it is without encoding.
Any plus signs in you secret key need to be url encoded as %2B, in your case **********+*********** becomes **********%2B***********
When you configure Camel endpoints using URIs then the parameter values gets url encoded by default.
This can be a problem when you want to configure passwords as is.
To do that you can tell Camel to use the raw value, by enclosing the value with RAW(value). See more details at How do I configure endpoints which has an example also.
See Camel Documentation
Your url should looks like:
aws-s3:bucketName?accessKey=RAW(XXXX)&secretKey=RAW(XXXX)

How do I force GlassFish 2 to load EJBs on startup?

We're using EJB3 on GlassFish v2.
My application includes a GenericServlet called StartupServlet, which has an init method. java.util.TimerTask pollers started from this method cannot lookup facades from the InitialContext.
However if I make an HTTP request and do a lookup, it succeeds. Therefore I have a workaround now where my poller startup code makes an HTTP connection to a page which looks up the interfaces they need.
How can I rearrange my application so I don't need to use such a hack? If possible the solution needs to work on GFv3 as well.
Thanks in advance for your help!
On GF 2, I have a servlet that on start ensures that my timer is created. This looks up a remote session bean and calls it successfully from the init() (not actual code, distilled down to the important parts):
#EJB(name="TimerSessionRef", beanInterface=TimerSessionRemote.class)
public class StartTimers extends HttpServlet {
#Override
public void init() throws ServletException {
super.init();
try {
Context ctx = new InitialContext();
TimerSessionRemote timerSession = (TimerSessionRemote) ctx.lookup("java:comp/env/TimerSessionRef");
timerSession.createTimer();
} catch (NamingException ex) {
logger.blah();
}

How to start Camel routes on slave ActiveMQ only when slave becomes active in failover?

I have a durable consumer to a remote JMS queue in embedded Camel routing. Is it possible to have this kind of routing with master-slave configuration? Now it seems that the Camel routes are started and activated already when slave ActiveMQ is started and not when the actual failover happens.
Now it causes the slave instance to receive the same messages that are also sent to master and this causes duplicate messages to arrive to the queue on failover.
I'm using ActiveMQ 5.3 along with Apache Camel 2.1.
Unfortunately, when the slave broker starts so does the CamelContext along with the routes. However you can accomplish this by doing the following:
On the camelContext deployed with slave broker add the following autoStartup attribute to prevent the routes from starting:
<camelContext id="camel" xmlns="http://camel.apache.org/schema/spring" autoStartup="false">
...
</camelContext>
Next you need to create a class that implements the ActiveMQ Service Interface. A sample of this would be as follows:
package com.fusesource.example;
import org.apache.activemq.Service;
import org.apache.camel.spring.SpringCamelContext;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
* Example used to start and stop the camel context using the ActiveMQ Service interface
*
*/
public class CamelContextService implements Service
{
private final Logger LOG = LoggerFactory.getLogger(CamelContextService.class);
SpringCamelContext camel;
#Override
public void start() throws Exception {
try {
camel.start();
} catch (Exception e) {
LOG.error("Unable to start camel context: " + camel);
e.printStackTrace();
}
}
#Override
public void stop() throws Exception {
try {
camel.stop();
} catch (Exception e) {
LOG.error("Unable to stop camel context: " + camel);
e.printStackTrace();
}
}
public SpringCamelContext getCamel() {
return camel;
}
public void setCamel(SpringCamelContext camel) {
this.camel = camel;
}
}
Then in broker's configuration file, activemq.xml, add the following to register the service:
<services>
<bean xmlns="http://www.springframework.org/schema/beans" class="com.fusesource.example.CamelContextService">
<property name="camel" ref="camel"/>
</bean>
</services>
Now, once the slave broker takes over as the master, the start method will be invoked on the service class and the routes will be started.
I have also posted a blog about this here: http://jason-sherman.blogspot.com/2012/04/activemq-how-to-startstop-camel-routes.html
this shouldn't be an issue because the Camel context/routes on the slave will not start until it becomes the master (when the message store file lock is released by the master)
With camel routepolicies you can decide to suspend/resume certain routes based on your own conditions.
http://camel.apache.org/routepolicy.html
There is an existing ZookeeperRoutePolicy that can be used to do leader election.
http://camel.apache.org/zookeeper.html (see bottom of the page)