#JsonFormat deserialization fails - jackson

I have a class with one field:
#JsonFormat(shape = JsonFormat.Shape.STRING, pattern = "yyyy-MM-dd'T'HH:mm:ss.SSSZ", timezone = "PST")
#JsonProperty("myDate")
private Date myDate;
When I try to deserialize a json string into an object:
ObjectMapper objectMapper = new ObjectMapper();
objectMapper.setTimeZone(TimeZone.getTimeZone("PST"));
ObjectNode node = objectMapper.createObjectNode();
node.put("myDate", "2016-11-06");
Object pojo = objectMapper.treeToValue(node, SomeClass.class);
It fails with this exception:
com.fasterxml.jackson.databind.JsonMappingException: Failed to parse Date value '2016-11-06' (format: "yyyy-MM-dd'T'HH:mm:ss.SSSZ"): Unparseable date: "2016-11-06"
.
.
.
Caused by: java.lang.IllegalArgumentException: Failed to parse Date value '2016-11-06' (format: "yyyy-MM-dd'T'HH:mm:ss.SSSZ"): Unparseable date: "2016-11-06"
at com.fasterxml.jackson.databind.deser.std.DateDeserializers$DateBasedDeserializer._parseDate(DateDeserializers.java:158)
at com.fasterxml.jackson.databind.deser.std.DateDeserializers$DateDeserializer.deserialize(DateDeserializers.java:261)
at com.fasterxml.jackson.databind.deser.std.DateDeserializers$DateDeserializer.deserialize(DateDeserializers.java:245)
at com.fasterxml.jackson.databind.deser.SettableBeanProperty.deserialize(SettableBeanProperty.java:490)
at com.fasterxml.jackson.databind.deser.impl.MethodProperty.deserializeAndSet(MethodProperty.java:95)
at com.fasterxml.jackson.databind.deser.BeanDeserializer.vanillaDeserialize(BeanDeserializer.java:260)
Shouldn't the annotation, #JsonFormat be applicable for serialization only? At least thats what I understood by reading this FAQ: http://wiki.fasterxml.com/JacksonFAQDateHandling. Here I am unable to deserialize a json string into an object.
Even the JsonFormat java docs doesnt talk about it being used during deserialization: http://fasterxml.github.io/jackson-annotations/javadoc/2.0.0/com/fasterxml/jackson/annotation/JsonFormat.html
Did I understand this feature wrong or am I doing something wrong in my code?

No, the #JsonFormat annotation is applicable for both serailization and deserailization. So in de-serailization it is used to parse the datetime string to datetime object using DateFormat.parse method and in serailization the datetime object is formatted to date time string using DateFormat.format method.
For more info, take a look at the DateSerializer & DateDeserializers classes.
You should only use 'Z' if the value is a UTC time. So in your case change your date time format to "yyyy-MM-dd'T'HH:mm:ss.SSS" and pass the date time value as "2016-11-06T05:00:35.657". This will parse successfully.

Related

Type name 'serializeObject' does not exist in the type 'JsonConvert'

I am trying to convert some data to Json to send to the CloudFlare API. I am attempting to use Newtonsoft.Json.JsonConvert.SeriablizeObject() to accomplish this but I am getting the Intellisense error that the type name 'serializeObject' does not exist in the type 'JsonConvert'. I have the NuGet package Newtonsoft.Json installed but it does not recognize the SerializeObject() method. I am not sure what I am missing.
Its because you're calling the method wrong, remove the new operator from the line
var json = new JsonConvert.SerializeObject(updateDnsRecord);
to
var json = JsonConvert.SerializeObject(updateDnsRecord);

Processing Event Hub Capture AVRO files with Azure Data Lake Analytics

I'm attempting to extract data from AVRO files produced by Event Hub Capture. In most cases this works flawlessly. But certain files are causing me problems. When I run the following U-SQL job, I get the error:
USE DATABASE Metrics;
USE SCHEMA dbo;
REFERENCE ASSEMBLY [Newtonsoft.Json];
REFERENCE ASSEMBLY [Microsoft.Analytics.Samples.Formats];
REFERENCE ASSEMBLY [Avro];
REFERENCE ASSEMBLY [log4net];
USING Microsoft.Analytics.Samples.Formats.ApacheAvro;
USING Microsoft.Analytics.Samples.Formats.Json;
USING System.Text;
//DECLARE #input string = "adl://mydatalakestore.azuredatalakestore.net/event-hub-capture/v3/{date:yyyy}/{date:MM}/{date:dd}/{date:HH}/{filename}";
DECLARE #input string = "adl://mydatalakestore.azuredatalakestore.net/event-hub-capture/v3/2018/01/16/19/rcpt-metrics-us-es-eh-metrics-v3-us-0-35-36.avro";
#eventHubArchiveRecords =
EXTRACT Body byte[],
date DateTime,
filename System.String
FROM #input
USING new AvroExtractor(#"
{
""type"":""record"",
""name"":""EventData"",
""namespace"":""Microsoft.ServiceBus.Messaging"",
""fields"":[
{""name"":""SequenceNumber"",""type"":""long""},
{""name"":""Offset"",""type"":""string""},
{""name"":""EnqueuedTimeUtc"",""type"":""string""},
{""name"":""SystemProperties"",""type"":{""type"":""map"",""values"":[""long"",""double"",""string"",""bytes""]}},
{""name"":""Properties"",""type"":{""type"":""map"",""values"":[""long"",""double"",""string"",""bytes""]}},
{""name"":""Body"",""type"":[""null"",""bytes""]}
]
}
");
#json =
SELECT Encoding.UTF8.GetString(Body) AS json
FROM #eventHubArchiveRecords;
OUTPUT #json
TO "/outputs/Avro/testjson.csv"
USING Outputters.Csv(outputHeader : true, quoting : true);
I get the following error:
Unhandled exception from user code: "The given key was not present in the dictionary."
An unhandled exception from user code has been reported when invoking the method 'Extract' on the user type 'Microsoft.Analytics.Samples.Formats.ApacheAvro.AvroExtractor'
Am I correct in assuming the problem is within the AVRO file produced by Event Hub Capture, or is there something wrong with my code?
The Key Not Present error is referring to the fields in your extract statement. It's not finding the data and filename fields. I removed those fields and your script runs correctly in my ADLA instance.
The current implementation only supports primitive types, not complex types of the Avro specification at the moment.
You have to build and use an extractor based on apache avro and not use the sample extractor provided by MS.
We went the same path

How do you serialize a fragment of an Xtext parse tree/AST?

I have some xtend code that gets a parse tree/AST from a file in my DSL.
I want a method that will serialize any node in the tree. This means I want text in the language of my DSL for an EObject somewhere in the parse tree.
So, if a program in my language is
class C {
int foo = 3;
}
the parse tree might look like
MyProgram
|_ MyClass
|_MyFieldDecl
|_Type
| |_IntType
|_Identifier
|_Expression
|_LiteralInteger
If I assigned a variable x the MyFieldDecl object in the tree, then I might want to call
var s = serialize(x)
to get the String, "int foo = 3;"
So, how do I implement that serialize() function?
The code I find when searching around will serialize the entire tree but not a node/fragment.
I found things in the xtext core code that appear to be doing this with either some instance of ISerializer or GrammarAccessExtensions. I tried to inject
#Inject #Extension GrammarAccessExtensions _grammarAccessExtensions
and get the serialization using
val s = _grammarAccessExtensions.grammarFragmentToString(e, prefix);
where e is an EObject, some node in the parse tree.
That failed on the statement
val main = injector.getInstance(MyClass)
with this error:
Exception in thread "main" com.google.inject.ConfigurationException: Guice configuration errors:
1) No implementation for org.eclipse.xtext.xtext.generator.model.project.IXtextProjectConfig was bound.
while locating org.eclipse.xtext.xtext.generator.model.project.IXtextProjectConfig
for field at org.eclipse.xtext.xtext.generator.XtextGeneratorNaming.projectConfig(Unknown Source)
while locating org.eclipse.xtext.xtext.generator.XtextGeneratorNaming
for field at org.eclipse.xtext.xtext.generator.grammarAccess.GrammarAccessExtensions._xtextGeneratorNaming(Unknown Source)
while locating org.eclipse.xtext.xtext.generator.grammarAccess.GrammarAccessExtensions
for field at myorg.MyClass._grammarAccessExtensions(Unknown Source)
while locating myorg.MyClass
1 error
at com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:1004)
at com.google.inject.internal.InjectorImpl.getProvider(InjectorImpl.java:961)
at com.google.inject.internal.InjectorImpl.getInstance(InjectorImpl.java:1013)
at myorg.MyClass.main(MyClass.java:435)
I think this means I need to bind or inject things, but I just started working with this code base a couple of days ago, and all of this is new to me.
I also tried to copy code out of the xtext core to get an ISerializer and call
s = GrammarAccessExtensions.grammarFragmentToString(serializer, e, prefix)
That failed with this:
java.lang.NullPointerException: Invalid context: Grammar returns ScenarioModel

Geode native client deserialise PdxInstanceImpl

I have a REST client that populates a Geode region with Json data which the Geode REST API automatically converts to a PdxInstance type.
The region triggers a C# native client listener AfterCreate(EntryEvent<TKey, TVal> ev) in which the TVal type ev.NewValue is seen as type PdxInstanceImpl and looks like:
PDX[7534066,__GEMFIRE_JSON]{#type=MyClass, Field1=Value1, Field2=Value2}
I've seen from here that the following code can get at the individual Pdx fields
IPdxInstance pdx = (IPdxInstance)ev.NewValue;
pdx.GetField("Field1");
and that works on a Field level, but I want to convert the PdxInstanceImpl that is received to PdxInstance so it can be put into another region directly, or I want to convert all the fields back to Json (as a string) in 1 go and put a Json string into another region, or use it as I like.
So there is apparently a way to autoserialize a PdxInstance to MyClass but if I try
MyClass c = (MyClass)pdx;
then I get System.InvalidCastException: Unable to cast object of type 'Apache.Geode.Client.Internal.PdxInstanceImpl' to type 'MyClass'
I've seen from some Java client examples you can use type PdxInstanceImpl to get at the data but in the C# native client that gives an error: PdxInstanceImpl is inaccessible due to its protection level.
I've added the autoserializer and the results are the same.
Any idea what I am missing here? Thanks
In the end I've used a field by field approach:
IPdxInstance pdx = (IPdxInstance)ev.NewValue;
pdx.GetField("Field1");
pdx.GetField("Field2");
pdx.GetField("Field3");
etc...
Outside of the event handlers, to create a PDX instance I used:
IPdxInstanceFactory writer = Setup.g.GetCache().CreatePdxInstanceFactory("myType");
writer.WriteString("String", "s");
writer.WriteChar("Char", 'c');
writer.WriteDouble("Double", Convert.ToDouble(1000));
IPdxInstance pdx = writer.Create();
To read a PDX instance its:
IPdxInstance pdx = Setup.gpg.GeodeGetPdx("myType", key);
MyType t = new MyType();
t.String1 = (string)pdx.GetField("String1");
t.Int1 = (int)pdx.GetField("Int1");
t.Date1 = (DateTime)pdx.GetField("Date1");
etc...

jackson-dataformat-csv: cannot serialize LocalDate

When I try to serialize object containing Local date, I get following error:
csv generator does not support object values for properties
I have JSR-310 module enabled, with WRITE_DATES_AS_TIMESTAMPS and I can convert the same object to JSON without problem.
For now I resorted to mapping the object to another, string only object, but it's decadent and wasteful.
Is there a way for Jackson csv mapper to acknowledge localDates? Should I somehow enable JSR-310 specifically for csv mapper?
I had the same problem because of configuring mapper after schema. Make sure you are using the latest verson of jackson and its modules. This code works for me:
final CsvMapper mapper = new CsvMapper();
mapper.findAndRegisterModules();
mapper.configure(SerializationFeature.WRITE_DATES_AS_TIMESTAMPS, false); //Optional
final CsvSchema schema = mapper.schemaFor(PojoWithLocalDate.class);
// Use this mapper and schema as you need to: get readers, writers etc.
No additional annotations needed in Pojo class.