Camel aws-s3 not working - amazon-s3

I am trying to create a camel route to transfer a file from an FTP server to an AWS S3 storage.
I have written the following route
private static class MyRouteBuilder extends RouteBuilder {
#Override
public void configure() throws Exception
{
from("sftp://<<ftp_server_name>>&noop=true&include=<<file_name>>...")
.process(new Processor(){
#Override
public void process(Exchange ex)
{
System.out.println("Hello");
}
})
.to("aws-s3://my-dev-bucket ?
accessKey=ABC***********&secretKey=12abc********+**********");
}
The issue is, this gives me the following exception:
Exception in thread "main" org.apache.camel.FailedToCreateRouteException: Failed to create route route1 at: >>> To[aws-s3://my-dev-bucket?accessKey=ABC*******************&secretKey=123abc******************** <<< in route: Route(route1)[[From[sftp://<<ftp-server>>... because of Failed to resolve endpoint: aws-s3://my-dev-bucket?accessKey=ABC***************&secretKey=123abc************** due to: The request signature we calculated does not match the signature you provided. Check your key and signing method.
I then tried to do this the other way. i.e.writing a method like this:
public void boot() throws Exception {
// create a Main instance
main = new Main();
// enable hangup support so you can press ctrl + c to terminate the JVM
main.enableHangupSupport();
// bind MyBean into the registery
main.bind("foo", new MyBean());
// add routes
AWSCredentials awsCredentials = new BasicAWSCredentials("ABC*****************", "123abc*************************");
AmazonS3 client = new AmazonS3Client(awsCredentials);
//main.bind("client", client);
main.addRouteBuilder(new MyRouteBuilder());
main.run();
}
and invoking using the bound variable #client. This approach does not give any exceptions, but the file transfer does not work.
To make sure that there's nothing wrong with my approach, I tried aws-sqs instead of aws-s3 and that works fine (file succesfully transfers to the SQS queue)
Any idea why this is happening? Is there some basic issue with "aws-s3" connector for camel?

Have you tried of using RAW() function to wrap as like RAW(secretkey or accesskey).
It will help you to pass your keys as it is without encoding.

Any plus signs in you secret key need to be url encoded as %2B, in your case **********+*********** becomes **********%2B***********

When you configure Camel endpoints using URIs then the parameter values gets url encoded by default.
This can be a problem when you want to configure passwords as is.
To do that you can tell Camel to use the raw value, by enclosing the value with RAW(value). See more details at How do I configure endpoints which has an example also.
See Camel Documentation

Your url should looks like:
aws-s3:bucketName?accessKey=RAW(XXXX)&secretKey=RAW(XXXX)

Related

How to set custom headers on RabbitMQ message using Apache Camel?

I'm trying to add custom headers on my message, so whenever an exception occurs and it ends up in the dead-letter-queue, I can see what the exception was. However all my attempts at this have failed.
using .setHeader()
setting header on the outMessage
setting property of the exchange
Setting the exception as a property in the payload is not allowed.
#Component
public class ProcessRoute extends RouteBuilder {
...
#Override
public void configure() throws Exception {
onException(Exception.class)
.log("Error for ${body}! Requeue")
.redeliveryDelay(2000)
.maximumRedeliveries(3)
.handled(true)
.setHeader("TEST", constant("TEST"))
.process(e -> {
e.getOut().setHeader("TEST", "TEST");
e.setProperty("TEST","TEST");
});
from(SOME_ROUTE)
.doSomeStuff()
.to(RABBITMQ);
}
...
}
RABBITMQ-string:
rabbitmq://foo
?exchangeType=topic
&addresses=localhost:1234
&routingKey=#
&autoDelete=false
&queue=bar
&autoAck=false
&deadLetterExchange=DLX
&deadLetterQueue=bar.dlq
&deadLetterExchangeType=direct
&deadLetterRoutingKey=#
&username=foo
&password=bar
Resulting message on the dead-letter-queue:
If you use a header key following the pattern that the Camel RabbitMQ component has established, then your custom header will get picked up when the message is published to RabbitMQ.
Taking from your code above, instead of:
.setHeader("TEST", constant("TEST"))
Do this:
.setHeader("rabbitmq.TEST", constant("TEST"))
The Camel RabbitMQ component seems to ignore all the other non- "rabbitmq.*" headers that might be on the Camel exchange, and probably for good reason. There could be quite a few and most of them wouldn't make sense in the context of a message published to RabbitMQ.

Dropwizard Authentication with POST calls failing

I was trying out Dropwizard authentication in my code but facing some issue in POST call at runtime, although its working fine with GET. this is how I am using this in the GET call:
#Override
#GET
#Path("/auth")
public Response doAuth(#Auth User user) {
//do something
}
And then in Post call which is not working:
#Override
#POST
#Path("/")
public Response createLegalEntity(#Auth User user, LegalEntity createLegalEntity) {
// do something
}
While running it is throwing following error:
SEVERE: Missing dependency for method public javax.ws.rs.core.Response org.flipkart.api.LegalEntityResource.createLegalEntity(com.yammer.dropwizard.authenticator.User,org.flipkart.model.LegalEntity) at parameter at index 0
I am new to Dropwizard and not able to figure out the cause of the problem.
UPDATE
Here is how I have registered my ldap authentication configs:
final LdapConfiguration ldapConfiguration = configuration.getLdapConfiguration();
Authenticator<BasicCredentials, User> ldapAuthenticator = new CachingAuthenticator<>(
environment.metrics(),
new ResourceAuthenticator(new LdapAuthenticator(ldapConfiguration)),
ldapConfiguration.getCachePolicy());
environment.jersey().register(new AuthDynamicFeature(
new BasicCredentialAuthFilter.Builder<User>()
.setAuthenticator(ldapAuthenticator)
.setRealm("LDAP")
.buildAuthFilter()));
environment.jersey().register(new AuthValueFactoryProvider.Binder<>(User.class));
The most likely reason is that you have not configured that auth feature correctly. The one thing that most people forget about is the AuthValueFactoryProvider.Binder. An instance of this class also needs to be registed. This would definitely cause the error you are seeing, if unregistered.
// If you want to use #Auth to inject a custom Principal type into your resource
environment.jersey().register(new AuthValueFactoryProvider.Binder<>(User.class));
From Basic Authentication docs
See also:
My comment for Dropwizard issue regarding the same problem. You will get a good explanation of the what causes the problem.

spring-boot property placeholder

I'm not able to figure out why I am not able to inject values into my application.properties file in spring-boot. external property into the logging.file variable. I have an application.properties file which looks like this
logging.file=${mylogfile}
server.port=${myport}
with corresponding Spring-boot Application class
#PropertySources({
#PropertySource("file:///c:/myfolder/externalprops.properties"),
})
#Configuration
#EnableAutoConfiguration
#ComponentScan
public class Application {
public static void main(String[] args) throws Exception {
SpringApplication.run(Application.class, args);
}
}
and external property file
mylogfile=myoutput.log
myport=8060
When I run my spring-boot 1.0.2.REL application I get the following exception every time I try to inject mylogfile into the logging.file property in application.properties
Exception in thread "main" java.lang.IllegalArgumentException: Could not resolve placeholder 'mylogfile' in string value "${mylogfile}"
at org.springframework.util.PropertyPlaceholderHelper.parseStringValue(PropertyPlaceholderHelper.java:174)
Note that I am not having any problems injecting and starting up the application if I inject the server port number on its own.
I am going around in circles on this issue and cannot figure out what I am doing wrong.
I don't think you can use #PropertySource to inject values into "application.properties" - the latter has to be parsed and ready to use before any #Configuration is read, or even known about. Your external properties could go in "${user.dir}/application.properties" and I think that would achieve what you are trying to do.

Apache Http Client Put Request Error

I'm trying to upload a file using the Apache Http Client's PUT method. The code is as below;
def putFile(resource: String, file: File): (Int, String) = {
val httpClient = new DefaultHttpClient(connManager)
httpClient.getCredentialsProvider.setCredentials(AuthScope.ANY, new UsernamePasswordCredentials(un, pw))
val url = address + "/" + resource
val put = new HttpPut(url)
put.setEntity(new FileEntity(file, "application/xml"))
executeHttp(httpClient, put) match {
case Success(answer) => (answer.getStatusLine.getStatusCode, "Successfully uploaded file")
case Failure(e) => {
e.printStackTrace()
(-1, e.getMessage)
}
}
}
When I tried running the method, I get to see the following error:
org.apache.http.NoHttpResponseException: The target server failed to respond
at org.apache.http.impl.conn.DefaultResponseParser.parseHead(DefaultResponseParser.java:101)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:252)
at org.apache.http.impl.AbstractHttpClientConnection.receiveResponseHeader(AbstractHttpClientConnection.java:281)
at org.apache.http.impl.conn.DefaultClientConnection.receiveResponseHeader(DefaultClientConnection.java:247)
at org.apache.http.impl.conn.AbstractClientConnAdapter.receiveResponseHeader(AbstractClientConnAdapter.java:219)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:298)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.client.DefaultRequestDirector.tryExecute(DefaultRequestDirector.java:633)
at org.apache.http.impl.client.DefaultRequestDirector.execute(DefaultRequestDirector.java:454)
at org.apache.http.impl.client.AbstractHttpClient.execute(AbstractHttpClient.java:820)
I do not know what has gone wrong? I'm able to do GET requests, but PUT seems not to work! Any clues as to where I should look for?
Look on the server. If GET Works, but PUT does not, then you have to figure out the receiving end.
Also, you may want to write a simple HTML File that has a form with PUT Method in it to rule out your Java Part.
As a sidenode: Its technically possible that something in between stops the request from going through or the response reaching you. Best setup a dummy HTTP Server to do the testing against.
Maybe its also a timeout issue, so the server takes to long to process your PUT.
The connection you are trying to use is a stale connection and therefore the request is failing.
But why are you only seeing an error for the PUT request and you are not seeing it for the GET request?
If you check the DefaultHttpRequestRetryHandler class you will see that by default HttpClient attempts to automatically recover from I/O exceptions. The default auto-recovery mechanism is limited to just a few exceptions that are known to be safe.
HttpClient will make no attempt to recover from any logical or HTTP protocol errors (those derived from HttpException class).
HttpClient will automatically retry those methods that are assumed to be idempotent. Your GET request, but not your PUT request!!
HttpClient will automatically retry those methods that fail with a transport exception while the HTTP request is still being transmitted to the target server (i.e. the request has not been fully transmitted to the server).
This is why you don't notice any error with your GET request, because the retry mechanism handles it.
You should define a CustomHttpRequestRetryHandler extending the DefaultHttpRequestRetryHandler. Something like this:
public class CustomHttpRequestRetryHandler extends DefaultHttpRequestRetryHandler {
#Override
public boolean retryRequest(IOException exception, int executionCount, HttpContext context) {
if(exception instanceof NoHttpResponseException) {
return true;
}
return super.retryRequest(exception, executionCount, context);
}
}
Then just assign your CustomHttpRequestRetryHandler
final HttpClientBuilder httpClientBuilder = HttpClients.custom();
httpClientBuilder.setRetryHandler(new CustomHttpRequestRetryHandler());
And that's it, now your PUT request is handled by your new RetryHandler (like the GET was by the default one)

How do I force GlassFish 2 to load EJBs on startup?

We're using EJB3 on GlassFish v2.
My application includes a GenericServlet called StartupServlet, which has an init method. java.util.TimerTask pollers started from this method cannot lookup facades from the InitialContext.
However if I make an HTTP request and do a lookup, it succeeds. Therefore I have a workaround now where my poller startup code makes an HTTP connection to a page which looks up the interfaces they need.
How can I rearrange my application so I don't need to use such a hack? If possible the solution needs to work on GFv3 as well.
Thanks in advance for your help!
On GF 2, I have a servlet that on start ensures that my timer is created. This looks up a remote session bean and calls it successfully from the init() (not actual code, distilled down to the important parts):
#EJB(name="TimerSessionRef", beanInterface=TimerSessionRemote.class)
public class StartTimers extends HttpServlet {
#Override
public void init() throws ServletException {
super.init();
try {
Context ctx = new InitialContext();
TimerSessionRemote timerSession = (TimerSessionRemote) ctx.lookup("java:comp/env/TimerSessionRef");
timerSession.createTimer();
} catch (NamingException ex) {
logger.blah();
}