Apache Camel validate before insert data with setBody - apache

I have RouteBuilder as below.
from("seda:requestQueue").routeId("service_request").log(LoggingLevel.INFO, "Processing STARTED, {body = ${body}")
.setBody(body())
.to("select * from SERVICE_REQUEST WHERE requestType=:#TYPE AND requestDate=:#date AND owner=:#OWNER)
.split(body()).streaming().process(new Processor() {
#Override
public void process(Exchange exchange) throws Exception {
Map<String, Object> row = exchange.getIn().getBody(Map.class);
if (row == null) {
LOGGER.info("Request is new. No records found.");
return;
}
//Duplicate request. Q(Not sure how to terminate the process with exception)
}
})
.log(LoggingLevel.INFO, "Processing CONTINUE, {body = ${body}")
.setBody(body())
.to("insert into SERVICE_REQUEST (ID,....) ").log(LoggingLevel.INFO, "Processing COMPLETED").end();
I would like to achieve
Whenever request is submitted (for now via SEDA), first check whether the same request has been available in database or not.
If it is not available then only insert into database (new row)
Question:
1. How can I set original request body to insertQuery? As per the above code, body received at seda:requestQueue is not available to to("insert into SERVICE..).

Your splitter will send out messages with a new body (based on the SQL). So, the body itself cannot be used.
Instead, before calling the splitter, set the body to a property of the Exchange. Then, when you need to use it, read the value of the property.
.setProperty("mySampleProperty", simple("${body}")
If you need it back as the body, then -- at that point -- set the body to the value that you previously stored in the Exchange property.
Here's a similar question: Apache Camel: how store variable for later use

Related

Getting a client readable message from an Npgsql.PostgresException

I'm writing a web api using PostgreSQL and am checking database constraints as part of the validation process, but I also have a global exception filter as a fallback in case something gets by when saving. My problem is that the exception doesn't seem to have any message that I can present to the client without some processing. The added image is of the PostgresException data from a breakpoint. For example, in this case I would want something along the lines of "Asset Number x already exists" or just "Asset Number must be unique". Is this something that can be configured somewhere? The place that makes the most sense is at the constraint creation code, but I couldn't find an option to do so.
modelBuilder.Entity<AssetItem>().HasIndex(item => new { item.AssetNumber }).IsUnique();
public class DbExceptionFilter : IExceptionFilter
{
private const string UNIQUE_EXCEPTION = "23505";
public async void OnException(ExceptionContext context)
{
var exceptionType = context.Exception.InnerException.GetType().FullName;
if (exceptionType == "Npgsql.PostgresException")
{
var pgException = (PostgresException) context.Exception.InnerException;
switch(pgException.SqlState)
{
case UNIQUE_EXCEPTION:
var error = new {error = "Unique Error Here"};
await WriteJsonErrorResponse(context.HttpContext.Response, HttpStatusCode.BadRequest, error);
return;
}
}
else
{
var error = new { error = "Unexpected Server Error"};
await WriteJsonErrorResponse(context.HttpContext.Response, HttpStatusCode.InternalServerError, error);
return;
}
}
private async Task WriteJsonErrorResponse(HttpResponse response, HttpStatusCode statusCode, dynamic error)
{
response.ContentType = "application/json";
response.StatusCode = (int) statusCode;
await response.Body.WriteAsync(Encoding.ASCII.GetBytes(JsonConvert.SerializeObject(error)));
}
}
The closest thing to a user-readable message that PostgreSQL provides is the message text exposed on PostgresException.
However, as a general rule it is not a good idea to expose database errors directly to users (including web API users): these are intended to the application directly interacting with the database (i.e. your application). These messages generally don't mean much to the users of your API, and more importantly they leak potentially sensitive information about your database schema and are therefore not secure. It's especially problematic to dump/serialize the entire exception to the user as you seem to be doing (with JsonConvert.SerializeObject).
The best practice here would be to identify legitimate database exceptions that the user may trigger, intercept these and return and appropriately-worded message of your own (e.g. "A user with that name already exists").
As a side note, to identify PostgresException, rather than getting the name of the exception and comparing to that, you can simply use C# pattern matching:
if (context.Exception.InnerException is PostgresException postgresException)
{
// ...
}

Header values getting lost spring integration

Im using header enricher which uses existing headers to set a value of new header. However existing header information is lost and only 3 header remain ie request-id,timestamp and raw-body.
public String vipul(Message<String> message) {
MessageHeaders messageHeaders =message.getHeaders();
if (messageHeaders.containsKey("x-death")) {
List<HashMap<String, Object>> deathList = (List<HashMap<String, Object>>) messageHeaders
.get("x-death");
//logger.debug(message.get("messageId")+" "+deathList);
if (deathList.size() > 0) {
HashMap<String, Object> death = deathList.get(0);
if (death.containsKey("original-expiration")) {
return (String) death.get("original-expiration");
//logger.info(messageHeaders.get("messageId")+" original-expiration = "+death.get("original-expiration"));
}
}
} else {
return null;
}
return "";
}
In this messageHeaders map has only has 3 keys and not all the header keys which are normally there. I need to make a retry system using original expiration .
MY spring integration xml has following snippet :
<int:header-enricher input-channel="fromPushAppointmentErrorHandler1"
output-channel="fromPushAppointmentErrorHandler">
<int:header name="original_expiration" method="vipul" ref="errorhelper"/>
</int:header-enricher>
First of all it looks like you also need an overwrite="true" for that <int:header name="original_expiration"> since the logic in your vipul() is about to produce a new value for existing header and that is not going to happen since the value is already there in headers.
The fact that you are missing some headers in this your logic might be dictated by some upstream <transformer> which returns the whole Message without copying request headers.

Java - Insert a single row at a time into google Big Query ?

I am creating an application where every time a user clicks on an article, I need to capture the article data and the user data to calculate the reach of every article and be able to run analytics on the reached data.
My application is on App Engine.
When I check documentation for inserts into BQ, most of them point towards bulk inserts in the form of Jobs or Streams.
Question:
Is it even a good practice to insert into big Query one row at a time every time a user action is initiated ? If so, could you point me to some Java code to effectively do this ?
There are limits on the number of load jobs and DML queries (1,000 per day), so you'll need to use streaming inserts for this kind of application. Note that streaming inserts are different from loading data from a Java stream.
TableId tableId = TableId.of(datasetName, tableName);
// Values of the row to insert
Map<String, Object> rowContent = new HashMap<>();
rowContent.put("booleanField", true);
// Bytes are passed in base64
rowContent.put("bytesField", "Cg0NDg0="); // 0xA, 0xD, 0xD, 0xE, 0xD in base64
// Records are passed as a map
Map<String, Object> recordsContent = new HashMap<>();
recordsContent.put("stringField", "Hello, World!");
rowContent.put("recordField", recordsContent);
InsertAllResponse response =
bigquery.insertAll(
InsertAllRequest.newBuilder(tableId)
.addRow("rowId", rowContent)
// More rows can be added in the same RPC by invoking .addRow() on the builder
.build());
if (response.hasErrors()) {
// If any of the insertions failed, this lets you inspect the errors
for (Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
// inspect row error
}
}
(From the example at https://cloud.google.com/bigquery/streaming-data-into-bigquery#bigquery-stream-data-java)
Note especially that a failed insert does not always throw an exception. You must also check the response object for errors.
Is it even a good practice to insert into big Query one row at a time every time a user action is initiated ?
Yes, it's pretty typical to stream event streams to BigQuery for analytics. You'll could get better performance if you buffer multiple events into the same streaming insert request to BigQuery, but one row at a time is definitely supported.
A simplified version of Google's example.
Map<String, Object> row1Data = new HashMap<>();
row1Data.put("booleanField", true);
row1Data.put("stringField", "myString");
Map<String, Object> row2Data = new HashMap<>();
row2Data.put("booleanField", false);
row2Data.put("stringField", "myOtherString");
TableId tableId = TableId.of("myDatasetName", "myTableName");
InsertAllResponse response =
bigQuery.insertAll(
InsertAllRequest.newBuilder(tableId)
.addRow("row1Id", row1Data)
.addRow("row2Id", row2Data)
.build());
if (response.hasErrors()) {
// If any of the insertions failed, this lets you inspect the errors
for (Map.Entry<Long, List<BigQueryError>> entry : response.getInsertErrors().entrySet()) {
// inspect row error
}
}
You can use Cloud Logging API to write one row at a time.
https://cloud.google.com/logging/docs/reference/libraries
Sample code from document
public class QuickstartSample {
/** Expects a new or existing Cloud log name as the first argument. */
public static void main(String... args) throws Exception {
// Instantiates a client
Logging logging = LoggingOptions.getDefaultInstance().getService();
// The name of the log to write to
String logName = args[0]; // "my-log";
// The data to write to the log
String text = "Hello, world!";
LogEntry entry =
LogEntry.newBuilder(StringPayload.of(text))
.setSeverity(Severity.ERROR)
.setLogName(logName)
.setResource(MonitoredResource.newBuilder("global").build())
.build();
// Writes the log entry asynchronously
logging.write(Collections.singleton(entry));
System.out.printf("Logged: %s%n", text);
}
}
In this case you need to create sink from dataflow logs. Then message will be redirect to the big Query table.
https://cloud.google.com/logging/docs/export/configure_export_v2

Extract org.restlet.http.headers value from Camel headers inside a .choice()

I'm trying to extract a value from org.restlet.http.headers header collection in a Camel route.
My incoming POST has a http header property called IncomingRequestType: ABCD.
Camel moves this inside the exchange headers collection, but it is buried inside org.restlet.http.headers which is in-itself a collection of headers.
I can extract the value in a process using the code below:
.process(new Processor() {
public void process(Exchange exchange) throws Exception {
org.restlet.util.Series<Header> httpHeaders = null;
httpHeaders = (Series<Header>) exchange.getIn().getHeader("org.restlet.http.headers");
String reqType = httpHeaders.getValues("IncomingRequestType").toString();
}})
Outside of a process I need to access the IncomingRequestType inside a .choice().when()
e.g. i want to be able to do:
.choice()
.when(header("org.restlet.http.headers")["IncomingRequestType"]).isEqualTo("ABCD"))
Any suggestions on how this can be done. I've tried creating predicates but cannot get a suitable solution.
This can be done in the simple language:
.choice()
.when(simple("${in.header.org.restlet.http.headers[IncomingRequestType]} == 'ABCD'"))

Camel check header value after split

I have a problem in my Camel Route when I try to check the value of a header.
So what is happening is that I go to a processor, do my stuff, then I create 2 different message that I put inside the body.
After that I go back on my route, I split my body so I can route the 2 differents messages, and there I use a .choice().when() on the header CamelFileName to check if it contains some value.
It doesn't find the value and then doesn't go inside the .when()
Here is the source code to make it more clear :
// My route
from("myQuartz")
.routeId("myId")
.bean(myProcessor.class)
.split(body())
.to("log:test?showAll=true&multiline=true")
.log('[${header.CamelFileName}]')
.choice()
.when(header('CamelFileName').contains('myString1'))
// do my stuff
.endChoice()
.when(header('CamelFileName').contains('myString2'))
// do my other stuff
.endChoice()
.otherwise()
.log("It did not go inside the when")
.to("log:test?showAll=true&multiline=true")
.endChoice()
.end()
So here I am simply trying to check if the CamelFileName header contains a string (it's not a variable), but it keep going inside the otherwise.
The log just after the split show me that the CamelFileName header is correct and do contains the string I am looking for.
I tried different ways of checking the value inside the when() such as using a simple(), but it doesn't work.
My file is a groovy file.
Thanks for your help.
EDIT :
So to explain what is inside my body I will show you the processor source code.
I create two DefaultMessage,
I set them a body and a CamelFileName header,
I put them into a list and then I put that list into my exchange body.
After that I go back to the route and I split the body so it will separate the two messages and route them.
Here is what's happening in my processor :
// Message 1
DefaultMessage message1 = new DefaultMessage()
message1.setBody(bodyContent)
def fileName1 = "myString1_blablabla.txt"
message1.setHeader("CamelFileName",fileName1)
listMessages.add(message1)
// Message 2
DefaultMessage message2 = new DefaultMessage()
message2.setBody(bodyContent)
def fileName2 = "myString2_blablabla.txt"
message2.setHeader("CamelFileName",fileName2)
listMessages.add(message2)
exchange.in.setBody(listMessages)
I've setup a simpler test for your route. It routes data to the proper when clause. When you split(), the headers get copied for each exchange, so I'm not sure why you would expect (given your route) why the elements of the list would have different header values.
public class SampleTest extends CamelTestSupport{
#Override
protected RouteBuilder createRouteBuilder() throws Exception {
return new RouteBuilder() {
#Override
public void configure() throws Exception {
from("direct:start")
.setHeader("CamelFileName", simple("myString1"))
.split(body())
.choice()
.when(header("CamelFileName").contains("myString1"))
.to("mock:myString1")
.endChoice()
.when(header("CamelFileName").contains("myString2"))
.to("mock:myString2")
.endChoice()
.otherwise()
.to("mock:otherwise")
.endChoice()
.end();
}
};
}
#Test
public void test() throws Exception {
//Setup mock body
java.util.List<String> myList = new java.util.ArrayList<String>();
myList.add("1");
myList.add("2");
MockEndpoint mock1 = getMockEndpoint("mock:myString1");
MockEndpoint mock2 = getMockEndpoint("mock:myString2");
MockEndpoint mockOtherwise = getMockEndpoint("mock:otherwise");
mock1.expectedMessageCount(myList.size());
mock2.expectedMessageCount(0);
mockOtherwise.expectedMessageCount(0);
template.sendBody("direct:start", myList);
assertMockEndpointsSatisfied();
}
}