jsog-jackson: Serializing object graphs - jackson

I am trying to serialize an object graph with parent/child references, essentially I have an entity that looks like this:
#Entity (name = "Container")
#JsonIdentityInfo(generator=JSOGGenerator.class)
public class Container {
public String type = "parent";
#JsonManagedReference ("child")
#OneToMany (mappedBy = "parent", cascade = CascadeType.PERSIST)
public List<Child> children;
}
#Entity (name = "Child")
#JsonIdentityInfo(generator=JSOGGenerator.class)
public class Child {
public String type = "child";
#JsonBackReference ("child")
#ManyToOne
public Parent parent;
}
when I try to serialize this to the client, then this is what I get:
{
"type": "parent",
#id: 1
"children": [
{
"type": "child",
#id: 2
},
{ ... }
]
}
I see #id properties on all objects but there is no sight of any #ref properties. If I have understood jsog and jsog-jackson right, then this is what should actually be serialized:
{
"type": "parent",
#id: 1
"children": [
{
"type": "child",
#id: 2
#ref: 1
},
{ ... }
]
}
What I would really like to have is a way of restoring the original back reference to the parent after restoring the serialized JSOG in the browser, so that instead of #ref I get the parent property in each child object back.

Your using two conflicting approaches to managing circular relationships. You can either use the JSOGGenerator OR the #JsonManagedReference and #JsonBackReference annotations.
The JSOGGenerator will include the #id and #ref properties in the JSON serialized format which is useful for deserializing the object in another language such as JavaScript.
The #JsonManagedReference and #JsonBackReference use the Java class information to identify the circular reference and subsequently that information is excluded from the JSON serialized format so another language like JavaScript can't deserialize the object because the required information is missing.
Another benefit to the JSOGGenerator is that it can handle deeply nested circular relationships as opposed to a limited parent-child relationship.

Related

How to combine polymorphic and transforming serializers for nested JSON, using Kotlin serialization?

This is a simplified example of the JSON I want to work with:
{
"useless_info": "useless info",
"data": {
"useless_info2": "useless info 2",
"children": [
{
"kind": "Car",
"data": {
"id": 1,
"transmission": "manual"
}
},
{
"kind": "Boat",
"data": {
"id": 2,
"isDocked": true
}
}
]
}
}
children is an array of vehicle objects. vehicle can be boat or car.
My Problem
The information that I want is nested quite deep (the real JSON is much deeply nested). A hack solution is to model the JSON exactly by writing dozens of nested data classes that references each other. I do not want to do this.
My problem is that while I know how to use JsonTransformingSerializer to unwrap arrays of a single type, and JsonContentPolymorphicSerializer to work with objects of various types, in this situation I believe I require both, but I can't get it to work.
What I Did
Assuming a single type
I tried understanding how it would work if it was a single type.
If the objects I wanted where all of the same type, it would be trivial to implement a JsonTransformingSerializer to cut right into the data I want. In this example, I will assume I only care about the ID, so I can just create a generic Vehicle model.
#Serializable
data class VehicleResponse(
#Serializable(with = VehicleResponseSerializer::class)
#SerialName("data")
val vehicles: List<Vehicle>
)
#Serializable
data class Vehicle(val id: Int)
object VehicleResponseSerializer : JsonTransformingSerializer<List<Vehicle>>(ListSerializer(Vehicle.serializer())) {
override fun transformDeserialize(element: JsonElement): JsonElement {
val vehicles = mutableListOf<JsonElement>()
// equals: [{"kind":"Car","data":{"id":1,"transmission":"manual"}},{"kind":"Boat","data":{"id":2,"isDocked":true}}]
val vehicleArray = element.jsonObject["children"]!!.jsonArray
vehicleArray.forEach { vehicle ->
// equals: {"id":1,"transmission":"manual"}}
val vehicleData = vehicle.jsonObject["data"]!!
vehicles.add(vehicleData)
}
return JsonArray(vehicles.toList())
}
}
The code works perfectly. Calling it out from main, printing the result gives me:
VehicleResponse(vehicles=[Vehicle(id=1), Vehicle(id=2)])
But they are actually Polymorphic!
Assuming one type does not work. I need to work with Car and Boat, and call their respective functions and properties.
I tried to model the structure like this:
#Serializable
data class VehicleResponse(
#Serializable(with = VehicleResponseSerializer::class)
#SerialName("data")
val vehicles: List<Vehicle>
)
#Serializable
abstract class Vehicle {
abstract val id: Int
}
#Serializable
data class Car(
override val id: Int,
val transmission: String,
) : Vehicle()
#Serializable
data class Boat(
override val id: Int,
val isDocked: Boolean,
) : Vehicle()
What I Want
I want to receive a JSON from a server, and instantly be able to deserialize it into a list of Vehicle objects, like the one VehicleResponse has.
I want to navigate through a deeply nested JSON, and unwrap an array that contains various Vehicle objects. For this, I assume I need to use JsonTransformingSerializer.
I want to use polymorphic deserialization to convert each of Vehicle into its corresponding subtype.
The actual ACTUAL problem
The thing that is truly throwing me in a loop is that a polymorphic serializer just does not seem to fit. It's called first before I get to parse the JSON. How am I supposed to decide which serializer to use?
Here's a test implementation:
object VehiclePolymorphicSerializer: JsonContentPolymorphicSerializer<VehicleResponse>(VehicleResponse::class) {
override fun selectDeserializer(element: JsonElement): DeserializationStrategy<out VehicleResponse> {
println("\nselectDeserializer()\n" +
"base element:\n" +
"$element\n")
// this return is a temporary hack, I just want to see the base element by printing it to the console
return VehicleResponse.serializer()
}
}
It prints:
selectDeserializer()
base element:
{"useless_info":"useless info","data":{"useless_info2":"useless info 2","children":[{"kind":"Car","data":{"id":1,"transmission":"manual"}},{"kind":"Boat","data":{"id":2,"isDocked":true}}]}}
That's the whole initial JSON! How am I supposed to decide which deserialization strategy to use, if both Car and Boat are in there? The JsonTransformingSerializer is called after the JsonContentPolymorphicSerializer.
Really not sure how am I supposed to proceed here. Would really appreciate even a slight hint.
kotlinx.serialization can handle polymorphic deserialization in this case without custom JsonContentPolymorphicSerializer. You just need to preserve class descriminator in the JSON returned from your JsonTransformingSerializer:
object VehicleResponseSerializer : JsonTransformingSerializer<List<Vehicle>>(ListSerializer(Vehicle.serializer())) {
override fun transformDeserialize(element: JsonElement): JsonElement {
// equals: [{"kind":"Car","data":{"id":1,"totalWheels":"4"}},{"kind":"Boat","data":{"id":2,"isDocked":true}}]
val vehicleArray = element.jsonObject["children"]!!.jsonArray
// equals: [{"type":"Car","id":1,"totalWheels":"4"}, {"type":"Boat","id":2,"isDocked":true}]
return JsonArray(vehicleArray.map {
val data = it.jsonObject["data"]!!.jsonObject
val type = it.jsonObject["kind"]!!
JsonObject(
data.toMutableMap().apply { this["type"] = type }
/*
//Kotlin 1.4 offers a nicer way to do this:
buildMap {
putAll(data)
put("type", type)
}
*/
)
})
}
}
If you declare Vehicle class as sealed (not just abstract), you're already good to go. If you want to keep it abstract, then you need to register all its subclasses in serializersModule
val module = SerializersModule {
polymorphic(Vehicle::class) {
subclass(Car::class)
subclass(Boat::class)
}
}
and pass it to JSON configuration
val kotlinx = Json {
ignoreUnknownKeys = true
serializersModule = module
}
UPDATE
(Alternative approach with combination of JsonTransformingSerializer & JsonContentPolymorphicSerializer)
Actually, it's possible to combine these two serializers.
For the sake of justification for this approach, let's imagine that original JSON doesn't have that nice kind field, and we have to figure out the actual subtype of Vehicle by the shape of JSON. In this case it could be the following heuristic: "if there is a isDocked field, then it's a Boat, othrewise - a Car".
Yes, we may include this logic into JsonTransformingSerializer to create class descriminator on the fly:
val type = when {
"isDocked" in data -> JsonPrimitive("Boat")
else -> JsonPrimitive("Car")
}
But it's more common (and type-safe) to use JsonContentPolymorphicSerializer for this. So we may simplify JsonTransformingSerializer:
object VehicleResponseSerializer : JsonTransformingSerializer<List<Vehicle>>(ListSerializer(Vehicle.serializer())) {
override fun transformDeserialize(element: JsonElement): JsonElement {
// equals: [{"kind":"Car","data":{"id":1,"totalWheels":"4"}},{"kind":"Boat","data":{"id":2,"isDocked":true}}]
val vehicleArray = element.jsonObject["children"]!!.jsonArray
// equals: [{"id":1,"totalWheels":"4"}, {"id":2,"isDocked":true}] // Note that class discriminator is absent here!
return JsonArray(vehicleArray.map { it.jsonObject["data"]!! })
}
}
and define JsonContentPolymorphicSerializer (for Vehicle, not for VehicleResponse!) to handle actual serializer selection:
object VehiclePolymorphicSerializer : JsonContentPolymorphicSerializer<Vehicle>(Vehicle::class) {
override fun selectDeserializer(element: JsonElement): DeserializationStrategy<out Vehicle> = when {
"isDocked" in element.jsonObject -> Boat.serializer()
else -> Car.serializer()
}
}
Since JsonTransformingSerializer (aka VehicleResponseSerializer) is registered for vehicles field serialization it's called before JsonContentPolymorphicSerializer (aka VehiclePolymorphicSerializer). Actually, the latter one is not yet called at all. We need to explicitly register it in serializersModule and pass it to JSON configuration (regardless of whether Vehicle class is declared as abstract or sealed):
val module = SerializersModule {
polymorphicDefault(Vehicle::class) { VehiclePolymorphicSerializer }
}
val kotlinx = Json {
ignoreUnknownKeys = true
serializersModule = module
}

Pick any element from JSON schema while streaming data to parse

I'm writing a de-serializer which reads a huge json file and puts records matching a filter (logic in my application) into database. The json file has a fixed schema as follows:
{
"cityDetails": {
"name": "String",
"pinCodes": "Array of integers ",
"people": [{
"name": "String",
"age": "Integer"
}]
}
}
I am only interested in streaming list of "people" from the file. I am aware that GSON/Jackson provide streaming APIs which I can use but I want to avoid looping through the tokens as I stream them and match their name to see if I am interested in them. I believe that there should be a solution which can do the streaming in background and point/seek the stream to the token I am interested in. I don't see any reason why this should not be possible if I provide my JSON schema. Is there are solution available for this?
Here's a sample instance of my JSON:
{
"cityDetails": {
"name": "mumbai",
"pinCodes": ["400001", "400002"],
"people": [{
"name": "Foo",
"age": 1
}, {
"name": "Bar",
"age": 2
}]
}
}
With GSON I would just create corresponding DTOs for the data to be parsed.
So you have some wrapper that is the root object:
#Getter
public class Wrapper {
private CityDetails cityDetails;
}
and city details:
#Getter
public class CityDetails {
private List<Person> people;
}
and possibly many Persons in the list people:
#Getter
#ToString
public class Person {
private String name;
private Integer age;
}
Then you can simply use for example Reader like below:
#Test
public void test() {
Gson gson = new Gson();
// assuming your json is named "test.json" in the same directory as test
Reader r = new InputStreamReader(getClass().getResourceAsStream("test.json"));
Wrapper wrapper = gson.fromJson(r, Wrapper.class);
wrapper.getCityDetails().getPeople().forEach(p -> log.info("{}", p.toString()));
}
Gson will search and instantiate only what is specified in DTO-classes the rest is ignored when parsing.
A nice way of doing this would be to use JsonPath.
A json path of:
$.cityDetails.people
will return just the contents of the people array:
[
[
{
"name": "Foo",
"age": 1
},
{
"name": "Bar",
"age": 2
}
]
]
Here is a Java implementation...

How to deserialize JSON having references to abstract types in Jackson

I have an issue with object references to abstract classes and JSON serialization and deserialization. The abstracted issue looks like this:
I have a graph consisting of nodes and edges. Each edge connects two nodes. Nodes can be of flavor red and green. Therefore, there is an abstract class Node and two derived classes RedNode and GreenNode. A Node takes an id (#JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class, property = "id")):
#JsonSubTypes({
#JsonSubTypes.Type(value = GreenNode.class, name = "GreenNode"),
#JsonSubTypes.Type(value = RedNode.class, name = "RedNode")
})
#JsonIdentityInfo(generator = ObjectIdGenerators.PropertyGenerator.class, property = "id")
public abstract class Node {
public String id;
}
public class RedNode extends Node {
// ...
}
public class GreenNode extends Node {
// ...
}
An Edge has a source and a target of type Node, which are serialized as references (#JsonIdentityReference(alwaysAsId = true)):
public class Edge {
#JsonIdentityReference(alwaysAsId = true)
public Node source;
#JsonIdentityReference(alwaysAsId = true)
public Node target;
}
The graph is defined as follows:
public class Graph {
public List<GreenNode> greenNodes = new ArrayList();
public List<RedNode> redNodes = new ArrayList();
public List<Edge> edges = new ArrayList();
}
An example JSON looks as follows:
{
"greenNodes" : [ {
"id" : "g",
"content" : "green g",
"greenProperty" : "green"
} ],
"redNodes" : [ {
"id" : "r",
"content" : "red r",
"redProperty" : "red"
} ],
"edges" : [ {
"source" : "g",
"target" : "r"
} ]
}
Using an ObjectMapper cannot read this:
Can not construct instance of com.github.koppor.jsonidentityissue.model.Node: abstract types either need to be mapped to concrete types, have custom deserializer, or contain additional type information
The error location is "line: 13, column: 16". Thus, it is hit at the id of the edge. The nodes themselves are properly serialized.
A workaround is to add type information in the json:
#JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include = JsonTypeInfo.As.PROPERTY,
property = "type")
public abstract class Node {
Then, everything works:
{
"greenNodes" : [ {
"id" : "g",
"type" : "GreenNode",
"content" : "green g",
"greenProperty" : "green"
} ],
"redNodes" : [ {
"id" : "r",
"type" : "RedNode",
"content" : "red r",
"redProperty" : "red"
} ],
"edges" : [ {
"source" : "g",
"target" : "r"
} ]
}
Then, everything works.
Is it really necessary to include type information in the referenced objects to have the reference working? Without the type information, a graph with red and green nodes (and no edges) can be loaded. After an edge comes in, it can't. However, the JSON of an edge contains an id only. The referenced objects are already parsed.
I really like to get rid off the #JsonTypeInfo annotation. Is there a way to have a clean JSON?
The full example is made available at https://github.com/koppor/jackson-jsonidentityreference-issue/tree/issue.
The current solution is to include fake type information. Full code at https://github.com/koppor/jackson-jsonidentityreference-issue.
The Node gets an existing property type, which is not written:
#JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include = JsonTypeInfo.As.EXISTING_PROPERTY,
property = "type")
public abstract class Node {
#JsonIgnore
public abstract String getType();
}
Each subclass specifies itself as defaultImpl and provides an implementation of getType:
#JsonTypeInfo(
use = JsonTypeInfo.Id.NAME,
include = JsonTypeInfo.As.EXISTING_PROPERTY,
property = "type",
defaultImpl=GreenNode.class)
public class GreenNode extends Node {
#Override
public String getType() {
return "GreeNode";
}
}
This way, the JSON remains clean, but Jackson can resolve the id reference without any issues.

Jackson HttpMessageNotReadableException : Already had POJO for id

I dont understand what is wrong in birectional relationship deserializing :
Serialized Registration is :
{
uuid: "d3372f25-ac70-4735-b756-7894977142b2",
payment: [{
"uuid":"15363b3f-7057-4658-3f63-3f3f695d7019",
"type":0,
"amount":10
"registration":"d3372f25-ac70-4735-b756-7894977142b2"
},
{
"uuid":"ee9fb135-2b21-48f4-a1ee-2f71f09f44ab",
"type":0,
"amount":5
"registration":"d3372f25-ac70-4735-b756-7894977142b2"
}]
}
Java is :
#JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator.class,
property = "uuid", scope = Registration.class)
public class Registration implements Serializable {}
#JsonIdentityInfo(
generator = ObjectIdGenerators.PropertyGenerator.class,
property = "uuid", scope = Payment.class)
public class Payment implements Serializable {}
Could not read document: Already had POJO for id (java.util.UUID) [[ObjectId: key=d3372f25-ac70-4735-b756-7894977142b2, type=com.fasterxml.jackson.databind.deser.impl.PropertyBasedObjectIdGenerator, scope=com.mezoo.tdc.model.Registration]] (through reference chain: com.mezoo.tdc.model.Registration["payment"]->java.util.ArrayList[0]->com.mezoo.tdc.model.Payment["registration"]->com.mezoo.tdc.model.Registration["uuid"]); nested exception is com.fasterxml.jackson.databind.JsonMappingException: Already had POJO for id (java.util.UUID) [[ObjectId: key=d3372f25-ac70-4735-b756-7894977142b2, type=com.fasterxml.jackson.databind.deser.impl.PropertyBasedObjectIdGenerator, scope=com.mezoo.tdc.model.Registration]] (through reference chain: com.mezoo.tdc.model.Registration["payment"]->java.util.ArrayList[0]->com.mezoo.tdc.model.Payment["registration"]->com.mezoo.tdc.model.Registration["uuid"])"
Somebody can explain to me why deserializing doesnt works please ?

Can't POST a collection

I have a simple Entity with a single collection mapped.
#Entity
public class Appointment Identifiable<Integer> {
#Id
#GeneratedValue(strategy = GenerationType.AUTO)
#JsonIgnore
private Integer id;
#Column(name="TRAK_NBR")
private String trackNumber;
#OneToMany(fetch =FetchType.EAGER, cascade= CascadeType.ALL)
#JoinColumn(name="CNSM_APT_VER_WRK_I", nullable = false)
private Set<Product> products = new HashSet<Product>();
}
#Entity
public class Product implements Identifiable<Integer> {
#Id
#Column(name = "CNSM_PRD_VER_WRK_I")
#GeneratedValue(strategy = GenerationType.AUTO)
#JsonIgnore
private Integer id;
#Column(name = "PRD_MDL_NBR")
private String model;
#Column(name = "PRD_SPEC_DSC")
private String description;
}
In my application when I only include a PagingAndSortingRepository for Appointment. I can call the POST command with the following payload.
{
"trackNumber" : "XYZ123",
"products": [
{"model" : "MODEL",
"description" : "NAME"
}]
}
When I add a PagingAndSortingRepository for Product and try the same POST I get the following error message.
{
"cause" : {
"cause" : {
"cause" : null,
"message" : null
},
"message" : "(was java.lang.NullPointerException) (through reference chain: com..model.Appointment[\"products\"])"
},
"message" : "Could not read JSON: (was java.lang.NullPointerException) (through reference chain: com.model.Appointment[\"products\"]); nested exception is com.fasterxml.jackson.databind.JsonMappingException: (was java.lang.NullPointerException) (through reference chain: com.model.AppointmentVerification[\"products\"])"
}
My GET payload with both Repositories returns this. This is my desired format. The link to products should be included
{
"trackNumber" : "XYZ123",
"_links" : {
"self" : {
"href" : "http://localhost:8080/consumerappointment/appointments/70"
},
"products" : {
"href" : "http://localhost:8080/consumerappointment/appointments/70/products"
}
}
With only the Appointment repository I get the following payload and can post the list of products.
{
"trackNumber" : "XYZ123",
"products" : [ {
"model" : "MODEL",
"description" : "NAME",
} ],
"_links" : {
"self" : {
"href" : "http://localhost:8080/consumerappointment/appointments/1"
}
}
}
Let's take a step back and make sure you understand what's happening here: if a repository is detected, Spring Data REST exposes a dedicated set of resources for it to manage the aggregates handled by the repository via HTTP. Thus, if you have repositories for multiple entities related to each other, the relationship is represented as a link. This is why you see the products inlined with only the AppointmentRepository in place and the products link in place once you create a ProductRepository.
If you want to expose both repositories as resources, you need to hand the URIs of the Product instances in the payload for the POST to create an Appointment. That means, instead of posting this:
{ "trackNumber" : "XYZ123",
"products": [
{ "model" : "MODEL",
"description" : "NAME"
}
]
}
you'd create a Product first:
POST /products
{ "model" : "MODEL",
"description" : "NAME" }
201 Created
Location: …/products/4711
And then hand the ID of the product to the Appointment payload:
{ "trackNumber" : "XYZ123",
"products": [ "…/products/4711" ]}
In case you don't want any of this (no resources exposed for Product in the first place, use #RepositoryRestResource(exported = false) on PersonRepository. That would still leave you with the bean instance created for the repo, but no resources exported and the resource exposed for Appointments back to inlining related Products.