Jackson Deserialization with no Type - kotlin

I have a question related to Jackson and polymorphism: is there a way to deserialize a JSON string without specifying a type?
Assuming I don't own this message (e.g., external API) and I have two separate messages that come in at separate times:
{
"responseCode": 200
"responseMessage": "You did something successfully"
}
{
"errorCode": 401
"errorDescription": "Permission denied"
}
And I want to deserialize this message with some data classes that I created based on these messages through polymorphism (see abstract class in next code block):
data class MyDataClass(
val responseCode: Int,
val responseMessage: String
): MyAbstractClass()
data class MyOtherDataClass(
val errorCode: Int,
val errorDescription: String
): MyAbstractClass()
And I am resolving these messages through a function that will use the Jackson Object Mapper to deserialize the stringified JSON payload:
#JsonSubTypes(
JsonSubTypes.Type(value = MyDataClass::class),
JsonSubTypes.Type(value = MyOtherDataClass::class)
)
#JsonIgnoreProperties(ignoreProperties = true)
abstract class MyAbstractClass
fun receiveMessage(message: String) {
val convertedMessage = jacksonObjectMapper().readValue<MyAbstractClass>(message)
log.info(convertedMessage)
/* prints either:
MyDataClass(responseCode=200, responseMessage=You did something successfully)
OR
MyOtherDataClass(errorCode=401, errorDescription=Permission denied)
*/
}
But since I haven't described how to identify the data class (using #JsonTypeInfo), it fails.
To repeat, I am curious if there is a way that I can deserialize the incoming message to one of my polymorphic types without having to specify the #JsonTypeInfo. Or if I must describe the #JsonTypeInfo, how would I do this with no similarities between the two child classes of MyAbstractClass?

I would write a custom deserializer which takes it as a generic JSONObject or the like. Then I'd check if a differentiating key exists. For example:
// pseudocode
when (json: JSONObject) {
hasKey("responseCode") -> // deserialize as MyDataClass
hasKey("errorCode") -> // deserialize as MyOtherDataClass
}

Related

Polymorphic serialization of sealed hierarchies with generic type parameters

Using Kotlin serialization, I would like to serialize and deserialize (to JSON) a generic data class with type parameter from a sealed hierarchy. However, I get a runtime exception.
To reproduce the issue:
import kotlinx.serialization.*
import kotlin.test.Test
import kotlin.test.assertEquals
/// The sealed hierarchy used a generic type parameters:
#Serializable
sealed interface Coded {
val description: String
}
#Serializable
#SerialName("CodeOA")
object CodeOA: Coded {
override val description: String = "Code Object OA"
}
#Serializable
#SerialName("CodeOB")
object CodeOB: Coded {
override val description: String = "Code Object OB"
}
/// Simplified class hierarchy
#Serializable
sealed interface NumberedData {
val number: Int
}
#Serializable
#SerialName("CodedData")
data class CodedData<out C : Coded> (
override val number: Int,
val info: String,
val code: C
): NumberedData
internal class GenericSerializerTest {
#Test
fun `polymorphically serialize and deserialize a CodedData instance`() {
val codedData: NumberedData = CodedData(
number = 42,
info = "Some test",
code = CodeOB
)
val codedDataJson = Json.encodeToString(codedData)
val codedDataDeserialized = Json.decodeFromString<NumberedData>(codedDataJson)
assertEquals(codedData, codedDataDeserialized)
}
}
Running the test results in the following runtime exception:
kotlinx.serialization.SerializationException: Class 'CodeOB' is not registered for polymorphic serialization in the scope of 'Coded'.
Mark the base class as 'sealed' or register the serializer explicitly.
This error message does not make sense to me, as both hierarchies are sealed and marked as #Serializable.
I don't understand the root cause of the problem - do I need to explicitly register one of the plugin-generated serializers? Or do I need to roll my own serializer? Why would that be the case?
I am using Kotlin 1.7.20 with kotlinx.serialization 1.4.1
Disclaimer: I do not consider my solution to be very statisfying, but I cannot find a better way for now.
KotlinX serialization documentation about sealed classes states (emphasis mine):
you must ensure that the compile-time type of the serialized object is a polymorphic one, not a concrete one.
In the following example of the doc, we see that serializing child class instead of parent class prevent it to be deserialized using parent (polymorphic) type.
In your case, you have nested polymorphic types, so this is even more complicated I think. To make serialization and deserialization work, then, I've tried multiple things, and finally, the only way I've found to make it work is to:
Remove generic on CodedData (to be sure that code attribute is interpreted in a polymorphic way:
#Serializable
#SerialName("CodedData")
data class CodedData (
override val number: Int,
val info: String,
val code: Coded
): NumberedData
Cast coded data object to NumberedData when encoding, to ensure polymorphism is triggered:
Json.encodeToString<NumberedData>(codedData)
Tested using a little main program based on your own unit test:
fun main() {
val codedData = CodedData(
number = 42,
info = "Some test",
code = CodeOB
)
val json = Json.encodeToString<NumberedData>(codedData)
println(
"""
ENCODED:
--------
$json
""".trimIndent()
)
val decoded = Json.decodeFromString<NumberedData>(json)
println(
"""
DECODED:
--------
$decoded
""".trimIndent()
)
}
It prints:
ENCODED:
--------
{"type":"CodedData","number":42,"info":"Some test","code":{"type":"CodeOB"}}
DECODED:
--------
CodedData(number=42, info=Some test, code=CodeOB(description = Code Object OB))

Deserializing a generic type using gson in kotlin

I have a generic type Animal implemented as a sealed class that can be a Dog or a Cat.
sealed class Animal(val typeOfAnimal: String) {
data class Dog(val barkVolume: Int): Animal("dog")
data class Cat(val hasStripes: Boolean): Animal("cat")
}
According to http://jansipke.nl/serialize-and-deserialize-a-list-of-polymorphic-objects-with-gson/ you can deserialize an Animal by registering a RuntimeTypeAdapterFactory
val animalAdapterFactory = RuntimeTypeAdapterFactory.of(Animal::class.java, "typeOfAnimal").registerSubtype(Dog::class.java, Dog::class.java.qualifiedName).registerSubtype(Cat::class.java, Cat::class.java.qualifiedName)
gson = GsonBuilder().registerTypeAdapterFactory(animalAdapterFactory)
But if I try to deserialize an animal that looks like
jsonStr = "{barkVolume: 100}"
gson.fromJson(json, Animal::class.java)
RuntimeTypeAdapterFactory complains that it can't deserialize Animal as it does not dfine a field named "typeOfAnimal"
To my understanding typeOfAnimal is a field you add to differentiate the subtypes and not something you need in the json you are deserializing. Because my json is really coming from an api I cannot add the field.
typeOfAnimal is required because gson must know which class should choose to deserialize your json. There is no way to guess the type, but you can implement your own deserializator. In a custom deserializator you can implement logic such as:
if (jsonObject.get("barkVolume") != null) {
// retun Dog object
}

Kotlin Flow out generic

I am using flow{} builder to call the api and then emit() the response to ViewModel. I add return type of flow as Flow<Resource<List<RemoteData>>>. However, at some places in emit(), the Android Studio throws
error : Not enough information to infer type variable T
Because the emit(Resource.Error(message = "Couldn't reach server, check your internet connection.")) is expecting values of type List<RemoteData> Please see my Resource class below
sealed class Resource<T>(val data: T? = null, val message: String? = null) {
class Loading<T>(data: T? = null): Resource<T>(data)
class Success<T>(data: T?): Resource<T>(data)
class Error<T>(message: String, data: T? = null): Resource<T>(data, message)
}
My question, Is it safe to change emit to
emit(Resource.Error(
message = "Couldn't reach server, check your internet connection.",
data = null
))
And flow's return type as Flow<Resource<out List<RemoteData>>> ?
Kotlin has declaration site variance. I would put out at the Resource class declaration. Then when you declare your type Flow<Resource<List<RemoteData>>>, it will already be implicitly out List<RemoteData>.
Also, your Resource classes look convoluted to me. If data is the loaded resource, it should not be part of the Loading or Error classes. Why force every instance of Loading and Error to carry a meaningless null data value? Likewise, the message should not be part of the Loading and Success cases.
I would rewrite your sealed class as a sealed interface (since it has no shared state between types) like this, and take advantage of data class features as well. Loading can be an object because it doesn't need to hold state. Loading and Error can both be Resource<Nothing> since the type T is irrelevant to any specific instance of them. That way you won't have to needlessly specify types when using them, like having to put <RemoteData> after is Resource or is Error in a when statement.
sealed interface Resource<out T> {
object Loading: Resource<Nothing>
data class Success<out T>(val data: T): Resource<T>
data class Error(val message: String): Resource<Nothing>
}
This version of the sealed classes will be much easier to use. The compiler will be more lenient with how and where you need to specify generic types.

How to serialize "Any" type in Kotlinx Serialization?

I have a class that gets serialized for network traffic.
#Serializable
data class Packet(val dataType: String, val payload: Any)
I've used Java serialization to send it over the wire. The receiver can't know the type of the payload but Java deserializes it just fine, and then I use when(dataType) as a lookup to correctly cast the Any object to its correct type. Easy breazy.
But Kotlinx Serialization (with ProtoBuf) is a stickler about this Any type for reasons that aren't obvious to me. I can't register a serializer for Any. In the docs they recommend a polymorphic approach, which sorta works but you have to make the packet typed:
data class Packet<out T : Any>(val dataType: String, val payload: T) : SomeBaseClass<T>
but this kinda sucks because it weighs down a lot of code paths with inline reified typing, plus this doesn't solve that the receiving end won't know what type to try to deserialize the payload as without being to look at the dataType field first.
This is the worst catch-22. The framework won't ignore the payload: Any field (gives a compile error) and I can't even write a custom serializer because defining an element of type Any in a customer serializer (for the descriptor) gives the same run-time error of "no serializer registered for Any."
I've used Java serialization to send it over the wire. The receiver can't know the type of the payload but Java deserializes it just fine, and then I use when(dataType) as a lookup to correctly cast the Any object to its correct type. Easy breazy.
This is because java serialization is rather primitive - there is only one way to serialize (and hence to deserialize) an object. In kotlinx.serialization each class can have its own serialization strategy (or even several ones). And this flexibility comes with a price.
Serialization of Any could be handled (for declared list of its subclasses), but dynamic determintaion of deserialization strategy based on dataType field of partly deseriazed object is impossible in general case, because there is no guarantee that dataType field will be deserialized first. Some serialization formats (like JSON or Protobuf) have unordered schema. It could happen that payload is about to be deserialized before dataType, and Decoder interface doesn't allow to go back/make several passes.
If you're sure about the order of properties in your serialization format/message (or just feel lucky) you may go with the following custom serializer:
import kotlinx.serialization.*
import kotlinx.serialization.descriptors.*
import kotlinx.serialization.encoding.*
#Serializable(with = PacketSerializer::class)
data class Packet(val dataType: String, val payload: Any)
object PacketSerializer : KSerializer<Packet> {
override val descriptor: SerialDescriptor = buildClassSerialDescriptor("Packet") {
element("dataType", serialDescriptor<String>())
element("payload", buildClassSerialDescriptor("Any"))
}
#Suppress("UNCHECKED_CAST")
private val dataTypeSerializers: Map<String, KSerializer<Any>> =
mapOf(
"String" to serializer<String>(),
"Int" to serializer<Int>(),
//list them all
).mapValues { (_, v) -> v as KSerializer<Any> }
private fun getPayloadSerializer(dataType: String): KSerializer<Any> = dataTypeSerializers[dataType]
?: throw SerializationException("Serializer for class $dataType is not registered in PacketSerializer")
override fun serialize(encoder: Encoder, value: Packet) {
encoder.encodeStructure(descriptor) {
encodeStringElement(descriptor, 0, value.dataType)
encodeSerializableElement(descriptor, 1, getPayloadSerializer(value.dataType), value.payload)
}
}
#ExperimentalSerializationApi
override fun deserialize(decoder: Decoder): Packet = decoder.decodeStructure(descriptor) {
if (decodeSequentially()) {
val dataType = decodeStringElement(descriptor, 0)
val payload = decodeSerializableElement(descriptor, 1, getPayloadSerializer(dataType))
Packet(dataType, payload)
} else {
require(decodeElementIndex(descriptor) == 0) { "dataType field should precede payload field" }
val dataType = decodeStringElement(descriptor, 0)
val payload = when (val index = decodeElementIndex(descriptor)) {
1 -> decodeSerializableElement(descriptor, 1, getPayloadSerializer(dataType))
CompositeDecoder.DECODE_DONE -> throw SerializationException("payload field is missing")
else -> error("Unexpected index: $index")
}
Packet(dataType, payload)
}
}
}

GSON Deserialization of subtypes in Kotlin

I'm not sure if this is a limitation, a bug or just bad use of GSON. I need to have a hierarchy of Kotlin objects (parent with various subtypes) and I need to deserialize them with GSON. The deserialized object has correct subtype but its field enumField is actually null.
First I thought this is because the field is passed to the "super" constructor but then I found out that "super" works well for string, just enum is broken.
See this example:
import com.google.gson.Gson
import com.google.gson.GsonBuilder
import com.google.gson.typeadapters.RuntimeTypeAdapterFactory
open class Parent(val stringField: String,
val enumField: EnumField) {
enum class EnumField {
SUBTYPE1,
SUBTYPE2,
SUBTYPE3
}
}
class Subtype1() : Parent("s1", EnumField.SUBTYPE1)
class Subtype2(stringField: String) : Parent(stringField, EnumField.SUBTYPE2)
class Subtype3(stringField: String, type: EnumField) : Parent(stringField, type)
val subtypeRAF = RuntimeTypeAdapterFactory.of(Parent::class.java, "enumField")
.registerSubtype(Subtype1::class.java, Parent.EnumField.SUBTYPE1.name)
.registerSubtype(Subtype2::class.java, Parent.EnumField.SUBTYPE2.name)
.registerSubtype(Subtype3::class.java, Parent.EnumField.SUBTYPE3.name)
fun main() {
val gson = GsonBuilder()
.registerTypeAdapterFactory(subtypeRAF)
.create()
serializeAndDeserialize(gson, Subtype1()) // this works (but not suitable)
serializeAndDeserialize(gson, Subtype2("s2")) // broken
serializeAndDeserialize(gson, Subtype3("s3", Parent.EnumField.SUBTYPE3)) // broken
}
private fun serializeAndDeserialize(gson: Gson, obj: Parent) {
println("-----------------------------------------")
val json = gson.toJson(obj)
println(json)
val obj = gson.fromJson(json, Parent::class.java)
println("stringField=${obj.stringField}, enumField=${obj.enumField}")
}
Any ideas how to achieve to deserialization of enumField?
(deps: com.google.code.gson:gson:2.8.5, org.danilopianini:gson-extras:0.2.1)
P.S.: Note that I have to use RuntimeAdapterFactory because I have subtypes with different set of fields (I did not do it in the example so it is easier to understand).
Gson requires constructors without arguments to work properly (see deep-dive into Gson code below). Gson constructs raw objects and then use reflection to populate fields with values.
So if you just add some argument-less dummy constructors to your classes that miss them, like this:
class Subtype1() : Parent("s1", EnumField.SUBTYPE1)
class Subtype2(stringField: String) : Parent(stringField, EnumField.SUBTYPE2) {
constructor() : this("")
}
class Subtype3(stringField: String, type: EnumField) : Parent(stringField, type) {
constructor() : this("", EnumField.SUBTYPE3)
}
you will get the expected output:
-----------------------------------------
{"stringField":"s1","enumField":"SUBTYPE1"}
stringField=s1, enumField=SUBTYPE1
-----------------------------------------
{"stringField":"s2","enumField":"SUBTYPE2"}
stringField=s2, enumField=SUBTYPE2
-----------------------------------------
{"stringField":"s3","enumField":"SUBTYPE3"}
stringField=s3, enumField=SUBTYPE3
Gson deep-dive
If you want to investigate the internals of Gson, a tip is to add an init { } block to Subtype1 since it works and then set a breakpoint there. After it is hit you can move up the call stack, step through code, set more breakpoints etc, to reveal the details of how Gson constructs objects.
By using this method, you can find the Gson internal class com.google.gson.internal.ConstructorConstructor and its method newDefaultConstructor(Class<? super T>) that has code like this (I have simplified for brevity):
final Constructor<? super T> constructor = rawType.getDeclaredConstructor(); // rawType is e.g. 'class Subtype3'
Object[] args = null;
return (T) constructor.newInstance(args);
i.e. it tries to construct an object via a constructor without arguments. In your case for Subtype2 and Subtype3, the code will result in a caught exception:
} catch (NoSuchMethodException e) { // java.lang.NoSuchMethodException: Subtype3.<init>()
return null; // set breakpoint here to see
}
i.e. your original code fails since Gson can't find constructors without arguments for Subtype2 and Subtype3.
In simple cases, the problem with missing argument-less constructors is worked around with the newUnsafeAllocator(Type, final Class<? super T>)-method in ConstructorConstructor, but with RuntimeTypeAdapterFactory that does not work correctly.
I may be missing something in what you're trying to achieve, but is it necessary to use the RuntimeTypeAdapterFactory? If we take out the line where we register that in the Gson builder, so that it reads
val gson = GsonBuilder()
.create()
Then the output returns the enum we would expect, which looks to be serialising / deserialising correctly. I.e. the output is:
-----------------------------------------
{"stringField":"s1","enumField":"SUBTYPE1"}
stringField=s1, enumField=SUBTYPE1
-----------------------------------------
{"stringField":"s2","enumField":"SUBTYPE2"}
stringField=s2, enumField=SUBTYPE2
-----------------------------------------
{"stringField":"s3","enumField":"SUBTYPE3"}
stringField=s3, enumField=SUBTYPE3
It also may be an idea to implement Serializable in Parent. i.e.
open class Parent(val stringField: String, val enumField: EnumField) : Serializable {
enum class EnumField {
SUBTYPE1,
SUBTYPE2,
SUBTYPE3
}
}
Try adding #SerializedName annotation to each enum.
enum class EnumField {
#SerializedName("subtype1")
SUBTYPE1,
#SerializedName("subtype2")
SUBTYPE2,
#SerializedName("subtype3")
SUBTYPE3
}