Apache Camel - Build both from and to endpoints dynamically - apache

I have a camel route which processes a message from a process queue and sends it to upload queue.
from("activemq:queue:process" ).routeId("activemq_processqueue")
.process(exchange -> {
SomeImpl impl = new SomeImpl();
impl.process(exchange);
})
.to(ExchangePattern.InOnly, "activemq:queue:upload");
In impl.process I am populating an Id and destination server path. Now I need to define a new route which consumes messages from upload queue ,and copy a local folder (based on Id generated in previous route) and upload it to destination folder which is an ftp server (this is also populated in previous route)
So how to design a new route where both from and to endpoints are dynamic which would look something like below ?
from("activemq:queue:upload" )
.from("file:basePath/"+{idFromExchangeObject})
.to("ftp:"+{serverIpFromExchangeObject}+"/"+{pathFromExchangeObject});

I think there is a better alternative for your case, taking as granted that you are using a Camel version newer than 2.16.(alternatives for a previous version exist but the are more complicated and don't look elegant - ( e.g consumerTemplate & recipientList).
You can replace the first "dynamic from" with pollEnrich which enriches the message using a polling consumer and simple expression to build the dynamic file endpoint. For the second part, as already mentioned, a dynamic uri .toD will do the job. So your route would look like this:
from("activemq:queue:upload" )
.pollEnrich().simple("file:basePath/${header.idFromExchangeObject})
.aggregationStrategy(new ExampleAggregationStrategy()) // * see explanation
.timeout(2000) // the timeout is optional but recommended
.toD("ftp:${header.serverIpFromExchangeObject}/${header.pathFromExchangeObject}")
See content enricher section "Using dynamic uris"
http://camel.apache.org/content-enricher.html .
You will need an aggregation strategy, to combine the original exchange with the resource exchange in order to make sure that the headers serverIpFromExchangeObject, pathFromExchangeObject will be included in the aggregated exchange after the enrichment. If you don't include the custom strategy then Camel will by default use the body obtained from the resource. Have a look at the ExampleAggregationStrategy example in content-enricher.html to see how this works.
For the .toD() have a look at http://camel.apache.org/how-to-use-a-dynamic-uri-in-to.html

Adding a dynamic to endpoint in Camel (as noted in the comment) can be done with the .toD() which is described on this page on the Camel site.
I don't know of any fromD() equivalent. However, you could add a dynamic route by calling the addRoutes method on the CamelContext. This is described on this page on the Camel site.
Expanding slightly on the example from the Camel site here is something that should get you heading in the right direction.
public void process(Exchange exchange) throws Exception {
String idFromExchangeObject = ...
String serverIpFromExchangeObject = ...
String pathFromExchangeObject = ...
exchange.getContext().addRoutes(new RouteBuilder() {
public void configure() {
from("file:basePath/"+ idFromExchangeObject)
.to("ftp:"+ serverIpFromExchangeObject +"/"+pathFromExchangeObject);
}
});
}
There may be other options in Camel as well since this framework has an amazing number of EIP and capabilities.

Related

FeignClient error: url values must be not be absolute

I have an error when starting SpringBootApplication:
Unexpected exception during bean creation; nested exception is java.lang.IllegalArgumentException: url values must be not be absolute.
I'm a beginner is SpringCloud, but I worked with Openshift (On first look it's basically the same things).
I have a cluster with GatewayApplication and some business microservices in it, wrote on Kotlin. Inside cluster microservices communicate by FeignClient without authentification. In consumer-service it looks like it:
#FeignClient(name = "producer-service")
#Headers("Content-Type: application/json")
interface MarketServiceFeign {
#GetMapping("https://somehost.ru/{id}/orders")
fun getUserDevices(
#PathVariable id: String,
): ResponseEntity<List<UserOrder>>
}
I tried find same case, but couldn't.
I tried to:
use #RequestLine from feign-core, but it doesn't work with #FeignClient
use feign.#Param for argument instead of org.springframework.web.bind.annotation.#PathVariable
use url with http instead of https
The thing I didn't take into account is that FeignClient goes to services into cluster by name, not by host. So I fixed it like that:
#FeignClient(name = "producer-service")
#Headers("Content-Type: application/json")
interface MarketServiceFeign {
#GetMapping("/{id}/orders")
fun getUserDevices(
#PathVariable id: String
): ResponseEntity<List<UserDevice>>
}
How I understand, instead of "https://somehost.ru" FeignClient uses service name "producer-service". Result url for FeignClient is "producer-service/{id}/orders".
I hope this helps someone.
If you want to use an absolute URL instead of using load-balancing, you need to pass it via the url attribute in the #FeignClient annotation. It's going to be a URL per Feign client, so you cannot pass it per-request via #RequestMapping annotations. You can only use them to provide the path segments that follow the host url. If you do not pass the url in #FeignClient, the name will be used as serviceId to fetch all the instances of that service (for example, from a service registry) and load-balancing will be performed under the hood to select an instance to send the request to.

Use Accept header to remap request URL (api versioning using accept header)

Anyone can suggest a correct place to inject a code in ocelot (a handler or similar) to get a request, look at it and if there is a header "Accept" with version specs add it to the path.
For a header value application/vnd.myapp.v2+json it would signal that we want to call API v2 and adjust request accordingly.
this logic needs to be executed before route rules are applied because downstream routes will have version in the path:
a call GET /teams/ (with accept header application/vnd.myapp.v2+json) becomes
a call GET /v2/teams/ which, using redirect rule will be send to a service that handles teams calls V2.
example logic (need adjustment to add version in the beginnig)
private static Microsoft.AspNetCore.Http.PathString AppendVersionInPath(DownstreamContext ctx)
{
if (ctx.HttpContext.Request.Headers.TryGetValue("accept", out var acceptHeaderValue))
{
var resultString = Regex.Match(acceptHeaderValue, #"\d+").Value;
if (resultString.Length > 0)
{
var versionPath = $"/v{resultString}";
ctx.HttpContext.Request.Path = ctx.HttpContext.Request.Path.Add(new Microsoft.AspNetCore.Http.PathString(versionPath));
}
}
return ctx.HttpContext.Request.Path;
}
After some considerations we ended up using .net rewrite functionality by implementing pipeline handler that takes request and rewrites URL path based on the header value (if present)
I do not expect this to be the best answer but almost certainly ocelot does not allow such transformation.

WCF Data Service authorization policies

I'm using the WCF Data Service and i need to implement authorization policies.
The polices are dynamic and are stored into a table that contains the target table,
the field and the allowed value.
In order to achieve this, I override the OnStartProcessingRequest method
of the DataService but I try to change the RequestUri I run into "Unauthorization" problem.
There is a way to change the RequestUri parameter in OnStartProcessingRequest method?
this code generate the exeption
protected override void OnStartProcessingRequest(ProcessRequestArgs args) {
Uri uri = new Uri(args.RequestUri + "?$filter=Id eq 3");
args.RequestUri = uri;
}
I can't use the Interceptor because the system is dynamic and entites are unknown.
Currently the adopted solution is to apply filters in client application (html5/js)
and verify the filtering parameters on server (into the OnStartProcessingRequest).
I wonder if there is a way for me to add filter parameters in OnStartProcessingRequest
or any way that can fix this problem.

Options for creating dynamic filters (xpath) in a Camel route

I've the following static route that is loaded at my server startup. It listens for UDP messages on a port and pushes these messages to the seda queue defined in the route below.
from("mina:udp://hostipaddress:9998?sync=false").wireTap(
"seda:sometag?size=100&blockWhenFull=true&multipleConsumers=true");
Now I can have multiple clients that want to receive/subscribe to these messages. They also want to dynamically select which feeds they need.
Each client send a subscription request (REST) to the server (implemented using Spring-MVC, Jetty, Camel).
As soon as the server receives a request I create a new Camel route that looks like:
from("seda:sometag?multipleConsumers=true")
.routeId(RouteIdCreator.createRouteId(toIP, toPort, "sometag"))
.filter()
.xpath(this.xpathFilter).unmarshal().jaxb("sometag").marshal()
.json().wireTap("mina:udp://client_ip_address:20001?sync=false");
Once this route is deployed it will start to send UDP messages to the client_ip_address: 20001 (as specified in the dynamic route above.)
The client can send different filters to the server.
In case this server receives the new filter it does the following
1. checks if there is a route running (based on client ip and port)
2. If there is route running it stops that route and deletes this route with the older filter
3. It then recreates a new route which differs from the last route only in the xpathfilter.
My issue is that step 2 takes a lot of time (to stop and restart)
Is there is a way to resolve this issue?
Basically I want to change the XPath expression in the route without stops/migrating the route.
PS: I've also posted this on the official Camel mailing list.
You can try to store the xpath filter in a database (basically a simple table with the ip and the filter associated) when you receive a new subscription. Then you can read this filter from the database in the route, and use it as a filter.
from("seda:sometag?multipleConsumers=true")
.routeId(RouteIdCreator.createRouteId(toIP, toPort, "sometag"))
.setHeader("ip").constant(client_ip_adresse)
.filter().xpath(simple("${bean:xpathFilterComponent?methode=find}"))
.unmarshal().jaxb("sometag").marshal()
.json().wireTap("mina:udp://client_ip_address:20001?sync=false");
And your bean should look like
public class XpathFilterCompnent {
public void save(String ip, String filter){
//store a filter for an ip in database, when a subscription is received
}
public void find(#Header("ip") String ip){
String filter = ... //retreive filter from database
return filter;
}
}

WCF routing -- how to correctly add filter table programmatically

I am using the WCF 4 routing service, and need to configure the service programmatically (as opposed to via config). The examples I have seen of doing so, which are rare, create a MessageFilterTable as follows:
var filterTable=new MessageFilterTable<IEnumerable<ServiceEndpoint>>();
But, the generic parameter to that method is supposed to be TFilterData (the type of data you are filtering on)? I have my own custom filter that accepts a string -- can I still create the filter table this way?
If this will work...will the routing infrastructure create client endpoints out of the list I pass in?
I have created a WCF 4 routing service and configured it programmatically. My code is a bit more spaced out than it needs to be (maintainability for others being a concern, hence the comments), but it definitely works. This has two filters: one filters some specific Actions to a given endpoint, and the second sends the remaining actions to a generic endpoint.
// Create the message filter table used for routing messages
MessageFilterTable<IEnumerable<ServiceEndpoint>> filterTable = new MessageFilterTable<IEnumerable<ServiceEndpoint>>();
// If we're processing a subscribe or unsubscribe, send to the subscription endpoint
filterTable.Add(
new ActionMessageFilter(
"http://etcetcetc/ISubscription/Subscribe",
"http://etcetcetc/ISubscription/KeepAlive",
"http://etcetcetc/ISubscription/Unsubscribe"),
new List<ServiceEndpoint>()
{
new ServiceEndpoint(
new ContractDescription("ISubscription", "http://etcetcetc/"),
binding,
new EndpointAddress(String.Format("{0}{1}{2}", TCPPrefix, HostName, SubscriptionSuffix)))
},
HighRoutingPriority);
// Otherwise, send all other packets to the routing endpoint
MatchAllMessageFilter filter = new MatchAllMessageFilter();
filterTable.Add(
filter,
new List<ServiceEndpoint>()
{
new ServiceEndpoint(
new ContractDescription("IRouter", "http://etcetcetc/"),
binding,
new EndpointAddress(String.Format("{0}{1}{2}", TCPPrefix, HostName, RouterSuffix)))
},
LowRoutingPriority);
// Then attach the filter table as part of a RoutingBehaviour to the host
_routingHost.Description.Behaviors.Add(
new RoutingBehavior(new RoutingConfiguration(filterTable, false)));
You can find a good example on MSDN here: How To: Dynamic Update Routing Table
Note how they dont directly create an instance of the MessageFilterTable, but instead use the 'FilterTable' property provided by a new RoutingConfiguration instance.
If you have written a custom filter, then you will add it like this:
rc.FilterTable.Add(new CustomMessageFilter("customStringParameter"), new List<ServiceEndpoint> { physicalServiceEndpoint });
The CustomMessageFilter will be your filter, and the "customStringParameter" is the string that (I believe) you are talking about.
When the Router receives a connection request, it will attempt to map it via this table entry, if this is successful, then you are right, the router will create a client endpoint to talk to the ServiceEndpoint that you provided.