Apache Camel 3.4 - Various Unmarshalling Errors Using Jackson - jackson

Question

I have succesfully written a simple use-case to unmarshal a JSON response from the Quickbooks oAuth API to refresh tokens.
With that working, I need to go and fetch the actual data:
https://quickbooks.api.intuit.com/v3/company/XXXXXXXXXXXXXX/query?query=SELECT * FROM Invoice WHERE Metadata.LastUpdatedTime%3E='2020-07-01T01:00:00' ORDERBY Metadata.LastUpdatedTime, Id STARTPOSITION 1 MAXRESULTS 1000 &minorversion=47
The HTTP call works OK:
// make the HTTP REST call, without C10y* & Camel* headers:
.toD("https://${header." + Headers.IEP_API_HOST + "}?headerFilterStrategy=C10yHeaderFilterStrategy")
I can check that the JSON returned is OK:
.log(LoggingLevel.DEBUG, API_LOG, "JSON returned: ${body}")
But from here it goes pear-shaped:
.unmarshal().json(JsonLibrary.Jackson, Payload.class)
This is what happens:
With the above log statement a MismatchedInputException is raised with the message "No content to map due to end-of-input".
Without the above log statement a ClassNotFoundException is raised with the message "Payload.class".
Re. 1.
My understanding is that responses are cached by default so it should be possible to read the inputstream multiple times. By default the HTTP endpoint option disableStreamCache is set to false, so it's enabled.
The stack trace is:
com.fasterxml.jackson.databind.exc.MismatchedInputException: No content to map due to end-of-input
at [Source: (org.apache.camel.converter.stream.CachedOutputStream$WrappedInputStream); line: 1, column: 0]
at com.fasterxml.jackson.databind.exc.MismatchedInputException.from(MismatchedInputException.java:59) ~[jackson-databind-2.11.0.jar:2.11.0]
at com.fasterxml.jackson.databind.ObjectMapper._initForReading(ObjectMapper.java:4624) ~[jackson-databind-2.11.0.jar:2.11.0]
at com.fasterxml.jackson.databind.ObjectMapper._readMapAndClose(ObjectMapper.java:4469) ~[jackson-databind-2.11.0.jar:2.11.0]
at com.fasterxml.jackson.databind.ObjectMapper.readValue(ObjectMapper.java:3471) ~[jackson-databind-2.11.0.jar:2.11.0]
at org.apache.camel.component.jackson.JacksonDataFormat.unmarshal(JacksonDataFormat.java:188) ~[camel-jackson-3.4.0.jar:3.4.0]
at org.apache.camel.support.processor.UnmarshalProcessor.process(UnmarshalProcessor.java:64) ~[camel-support-3.4.0.jar:3.4.0]
at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.doRun(RedeliveryErrorHandler.java:702) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.run(RedeliveryErrorHandler.java:616) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:147) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:286) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:203) ~[camel-timer-3.4.0.jar:3.4.0]
at org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76) ~[camel-timer-3.4.0.jar:3.4.0]
at java.base/java.util.TimerThread.mainLoop(Timer.java:556) ~[na:na]
at java.base/java.util.TimerThread.run(Timer.java:506) ~[na:na]
Re. 2.
The stack trace for this is:
org.apache.camel.CamelExecutionException: Exception occurred during execution on the exchange: Exchange[ID-WIN-10-DM-1594724136485-0-1]
at org.apache.camel.CamelExecutionException.wrapCamelExecutionException(CamelExecutionException.java:47) ~[camel-api-3.4.0.jar:3.4.0]
at org.apache.camel.language.simple.SimpleExpressionBuilder$26.evaluate(SimpleExpressionBuilder.java:590) ~[camel-core-languages-3.4.0.jar:3.4.0]
at org.apache.camel.support.ExpressionAdapter.evaluate(ExpressionAdapter.java:36) ~[camel-support-3.4.0.jar:3.4.0]
at org.apache.camel.reifier.language.SimpleExpressionReifier$1.evaluate(SimpleExpressionReifier.java:42) ~[camel-core-engine-3.4.0.jar:3.4.0]
at org.apache.camel.processor.SetHeaderProcessor.process(SetHeaderProcessor.java:48) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.doRun(RedeliveryErrorHandler.java:702) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.processor.errorhandler.RedeliveryErrorHandler$RedeliveryTask.run(RedeliveryErrorHandler.java:616) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.impl.engine.DefaultReactiveExecutor$Worker.schedule(DefaultReactiveExecutor.java:148) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.impl.engine.DefaultReactiveExecutor.scheduleMain(DefaultReactiveExecutor.java:60) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.processor.Pipeline.process(Pipeline.java:147) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:286) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.component.timer.TimerConsumer.sendTimerExchange(TimerConsumer.java:203) ~[camel-timer-3.4.0.jar:3.4.0]
at org.apache.camel.component.timer.TimerConsumer$1.run(TimerConsumer.java:76) ~[camel-timer-3.4.0.jar:3.4.0]
at java.base/java.util.TimerThread.mainLoop(Timer.java:556) ~[na:na]
at java.base/java.util.TimerThread.run(Timer.java:506) ~[na:na]
Caused by: java.lang.ClassNotFoundException: Payload.class
at org.apache.camel.impl.engine.DefaultClassResolver.resolveMandatoryClass(DefaultClassResolver.java:87) ~[camel-base-3.4.0.jar:3.4.0]
at org.apache.camel.language.simple.SimpleExpressionBuilder$26.evaluate(SimpleExpressionBuilder.java:588) ~[camel-core-languages-3.4.0.jar:3.4.0]
... 13 common frames omitted
All of this is running within an Eclipse/Maven project, so I have cleaned, refreshed, compiled, rebuilt, etc, etc, to no avail.
I have written a simple jUnit and it works absolutely fine using the JSON from the above log, saved to a file:
/**
* POJO Jackson unmarshalling
*/
#Test
public void pojoUnmarshallTest() {
ObjectMapper om = new ObjectMapper();
try {
Payload payload = om.readValue(getFile("qb.Payload.Invoice.json"), Payload.class);
assertTrue(payload.toString().startsWith("c10y.model.qb.Payload"));
} catch (Exception e) {
e.printStackTrace();
fail(e.getMessage());
}
}
Lastly, I'm using the latest versions of Camel & Spring Boot:
<properties>
<!-- latest versions # Jul 2020 -->
<java.version>11</java.version>
<camel.version>3.4.0</camel.version> <!-- latest long term support version -->
<maven-compiler-plugin.version>3.8.1</maven-compiler-plugin.version>
<maven-surefire-plugin.version>3.0.0-M5</maven-surefire-plugin.version>
<spring-boot.version>2.3.0.RELEASE</spring-boot.version>
<run.profiles>dev</run.profiles>
</properties>
As ever, thanks in advance for your help!
PS:
The POJO classes that receive the unmarshalled JSON were generated at http://www.jsonschema2pojo.org/. The POJO for the working route was hand-cranked. I mention it just in case it could make a difference (it shouldn't IMHO).
The entire route can be obtained here: https://drive.google.com/file/d/1Qu0vwJaSlggH6BrIgUFMU_BZNpmMQYuh/view?usp=sharing.
I have tried the Gson library and get the same ClassNotFoundException: Payload.class error. This is looking like less of an unmarshalling problem and more of a classpath issue.
Having two .log() statements gives the following output:
2020-07-15 10:06:27.108 DEBUG 6720 --- [mer://startHere] c.c.r.qb.fetch.delta : JSON returned: {"QueryResponse":{"Invoice":[{"AllowIPNPayment":false, ...removed... ,"Balance":0}],"startPosition":1,"maxResults":1,"totalCount":1},"time":"2020-07-15T02:06:21.869-07:00"}
2020-07-15 10:06:27.108 DEBUG 6720 --- [mer://startHere] c.c.r.qb.fetch.delta : JSON returned:
Contrary to the documentation, it looks like the input stream is not, in fact, cached.
PS2:
Adding .streamCaching() to the route and &disableStreamCache=false to the endpoint URI didn't make any difference to the second .log(); it remained empty.
I also tried the following Java Config approach:
#Configuration
public class Config {
#Bean
CamelContextConfiguration contextConfiguration() {
return new CamelContextConfiguration() {
#Override
public void beforeApplicationStart(CamelContext context) {
System.out.println("****** beforeApplicationStart ******");
}
#Override
public void afterApplicationStart(CamelContext context) {
System.out.println("****** afterApplicationStart ******");
context.setStreamCaching(true);
}
};
}
}
I can see the sysout in the console but this didn't work either.
I downloaded jsonschema2pojo-1.0.2 and ran it against a much bigger JSON sample with the following arguments:
--annotation-style JACKSON2
--big-decimals
--date-class java.util.Date
--format-dates
--format-date-times
--package c10y.model.qb.jxon
--remove-old-output
--source C:\Users\...\src\test\resources\qb.Payload.Invoice.json
--source-type JSON
--target c:\temp
--target-language JAVA
--target-version 11
This created the root/base POJO called QbPayloadInvoice, which looks like it's taken from the input file name. I updated my route:
.unmarshal().json(JsonLibrary.Jackson, QbPayloadInvoice.class)
It still raises the java.lang.ClassNotFoundException: Payload.class.
There's nothing in the JSON response, or in any of the other generated POJOs, called Payload.
At the same time, my updated jUnit works fine:
QbPayloadInvoice payload = om.readValue(getFile("qb.Payload.Invoice.json"), QbPayloadInvoice.class);
expected = "c10y.model.qb.jxon.QbPayloadInvoice";
assertEquals(expected, payload.toString().substring(0, expected.length()));
Go figure!

Answer 1

The clue exists above where I wrote "There's nothing in the JSON response, or in any of the other generated POJOs, called Payload." This statement turned out to be absolutely correct.
The unmarshalling had been working all along:
.unmarshal().json(JsonLibrary.Jackson, QbPayloadInvoice.class)
It was the following line that was failing:
.setHeader(QB_START_POS, simple("${bodyAs(Payload.class).QueryResponse.startPosition}"))
This was made more obvious given that I'd changed the unmarshalling class from Payload to QbPayloadInvoice.