In Kotlin how is possible to convert a data class from its string representation back to the actual data class without having to manually parse the string?
for example, I have the next data class where both Interest and EmploymentType are enums, being the second element a List.
data class DataFilter(val mainInterest: Interest, val employments: List<EmploymentType>)
with toString I can get the contents its string representation, but if I want to get it back to a data class, how is it done?
For this purpose, you need to use a serialization library. I recommend you use Kotlinx.serialization library. This will help you convert the data object into a JSON String and convert it back to the data object easily.
You can follow the guideline to setup Kotlinx.serialization library.
After the setup is done, look at the following example:
I will define both enum classes Interest and EmploymentType, and the DataFilter class:
import kotlinx.serialization.Serializable
import kotlinx.serialization.decodeFromString
import kotlinx.serialization.encodeToString
import kotlinx.serialization.json.Json
enum class Interest {
SPORTS,
BOOKS,
TRAVEL,
FOOD,
TECHNOLOGY,
ART,
OTHER
}
enum class EmploymentType {
FULL_TIME,
PART_TIME,
CONTRACT,
FREELANCE,
OTHER
}
#Serializable //You have to annotate your class as Serializable to make it works.
data class DataFilter(
val mainInterest: Interest,
val employment: List<EmploymentType>
)
Notice that Interest and EmploymentType are not annotated as #Serializable because enums are Serializable by default, but if you defined any other normal classes to use with DataFilter, then you have to annotate them as well.
Next, I will show you how conversions work:
fun main() {
val filter = DataFilter(
mainInterest = Interest.BOOKS,
employment = listOf(EmploymentType.FULL_TIME, EmploymentType.FREELANCE)
)
// Converting filter object into JSON String:
val filterAsJsonString = Json.encodeToString(filter)
println(filterAsJsonString) //prints {"mainInterest":"BOOKS","employment":["FULL_TIME","FREELANCE"]}
//Convert JSON String back to data object:
val filterAsDataObject: DataFilter = Json.decodeFromString(filterAsJsonString)
println(filterAsDataObject) //prints DataFilter(mainInterest=BOOKS, employment=[FULL_TIME, FREELANCE])
}
Related
Using Kotlin serialization, I would like to serialize and deserialize (to JSON) a generic data class with type parameter from a sealed hierarchy. However, I get a runtime exception.
To reproduce the issue:
import kotlinx.serialization.*
import kotlin.test.Test
import kotlin.test.assertEquals
/// The sealed hierarchy used a generic type parameters:
#Serializable
sealed interface Coded {
val description: String
}
#Serializable
#SerialName("CodeOA")
object CodeOA: Coded {
override val description: String = "Code Object OA"
}
#Serializable
#SerialName("CodeOB")
object CodeOB: Coded {
override val description: String = "Code Object OB"
}
/// Simplified class hierarchy
#Serializable
sealed interface NumberedData {
val number: Int
}
#Serializable
#SerialName("CodedData")
data class CodedData<out C : Coded> (
override val number: Int,
val info: String,
val code: C
): NumberedData
internal class GenericSerializerTest {
#Test
fun `polymorphically serialize and deserialize a CodedData instance`() {
val codedData: NumberedData = CodedData(
number = 42,
info = "Some test",
code = CodeOB
)
val codedDataJson = Json.encodeToString(codedData)
val codedDataDeserialized = Json.decodeFromString<NumberedData>(codedDataJson)
assertEquals(codedData, codedDataDeserialized)
}
}
Running the test results in the following runtime exception:
kotlinx.serialization.SerializationException: Class 'CodeOB' is not registered for polymorphic serialization in the scope of 'Coded'.
Mark the base class as 'sealed' or register the serializer explicitly.
This error message does not make sense to me, as both hierarchies are sealed and marked as #Serializable.
I don't understand the root cause of the problem - do I need to explicitly register one of the plugin-generated serializers? Or do I need to roll my own serializer? Why would that be the case?
I am using Kotlin 1.7.20 with kotlinx.serialization 1.4.1
Disclaimer: I do not consider my solution to be very statisfying, but I cannot find a better way for now.
KotlinX serialization documentation about sealed classes states (emphasis mine):
you must ensure that the compile-time type of the serialized object is a polymorphic one, not a concrete one.
In the following example of the doc, we see that serializing child class instead of parent class prevent it to be deserialized using parent (polymorphic) type.
In your case, you have nested polymorphic types, so this is even more complicated I think. To make serialization and deserialization work, then, I've tried multiple things, and finally, the only way I've found to make it work is to:
Remove generic on CodedData (to be sure that code attribute is interpreted in a polymorphic way:
#Serializable
#SerialName("CodedData")
data class CodedData (
override val number: Int,
val info: String,
val code: Coded
): NumberedData
Cast coded data object to NumberedData when encoding, to ensure polymorphism is triggered:
Json.encodeToString<NumberedData>(codedData)
Tested using a little main program based on your own unit test:
fun main() {
val codedData = CodedData(
number = 42,
info = "Some test",
code = CodeOB
)
val json = Json.encodeToString<NumberedData>(codedData)
println(
"""
ENCODED:
--------
$json
""".trimIndent()
)
val decoded = Json.decodeFromString<NumberedData>(json)
println(
"""
DECODED:
--------
$decoded
""".trimIndent()
)
}
It prints:
ENCODED:
--------
{"type":"CodedData","number":42,"info":"Some test","code":{"type":"CodeOB"}}
DECODED:
--------
CodedData(number=42, info=Some test, code=CodeOB(description = Code Object OB))
I have a generic type Animal implemented as a sealed class that can be a Dog or a Cat.
sealed class Animal(val typeOfAnimal: String) {
data class Dog(val barkVolume: Int): Animal("dog")
data class Cat(val hasStripes: Boolean): Animal("cat")
}
According to http://jansipke.nl/serialize-and-deserialize-a-list-of-polymorphic-objects-with-gson/ you can deserialize an Animal by registering a RuntimeTypeAdapterFactory
val animalAdapterFactory = RuntimeTypeAdapterFactory.of(Animal::class.java, "typeOfAnimal").registerSubtype(Dog::class.java, Dog::class.java.qualifiedName).registerSubtype(Cat::class.java, Cat::class.java.qualifiedName)
gson = GsonBuilder().registerTypeAdapterFactory(animalAdapterFactory)
But if I try to deserialize an animal that looks like
jsonStr = "{barkVolume: 100}"
gson.fromJson(json, Animal::class.java)
RuntimeTypeAdapterFactory complains that it can't deserialize Animal as it does not dfine a field named "typeOfAnimal"
To my understanding typeOfAnimal is a field you add to differentiate the subtypes and not something you need in the json you are deserializing. Because my json is really coming from an api I cannot add the field.
typeOfAnimal is required because gson must know which class should choose to deserialize your json. There is no way to guess the type, but you can implement your own deserializator. In a custom deserializator you can implement logic such as:
if (jsonObject.get("barkVolume") != null) {
// retun Dog object
}
I need to set up a serialization/deserialization mechanism for a polymorphic class hierarchy that also includes primitives and nulls. There are container classes containing collections with polymorphic objects, primitives, and nulls. And, the subclasses for these objects are spread across modules (therefore sealed is not an option).
I have been reading through the kotlinx.serialization polymorphism docs trying to come up with a solution. I've been able to make some incremental progress by working through that tutorial but I seem to still be hitting a wall when I try to put everything together.
The code I am posting here is a minimal example that brings together everything I need. If I can get this example to work, that should cover everything I need for my real project. This example does run without error but introduces some unnecessary readability and efficiency issues.
All classes in my custom class hierarchy are serializable data classes. The outermost container object that needs to be serialized/deserialized is a map wrapper. This map has keys which are each an instance of one of these data classes. And the values of this map can be primitives, nulls, or instances of one of my data classes. I think my main challenge here is to include those primitives and nulls in my polymorphic serialization in a clean way.
The goal of my code below is to represent this problem in the simplest way possible and to serialize and deserialize one container object successfully.
There are two main issues in the code:
I've had to replace null with FakeNull. Without this, I get null cannot be cast to non-null type kotlin.Any. This will reduce the readability and simplicity of my code and I suspect it could decrease efficiency as well.
I've had to add StringClassSerializer and DoubleClassSerializer and wrapper classes. I would also need to add serializers like these for every primitive class. If I don't register these primitives as subclasses of Any, I get Class 'String' is not registered for polymorphic serialization in the scope of 'Any'.. And if I try to register them with their default serializers (like subclass(String::class, String.serializer())) I get Serializer for String of kind STRING cannot be serialized polymorphically with class discriminator.. The problem with using serializers like StringClassSerializer and wrappers like StringWrapper is that it removes the efficiency and readability benefits of using primitives.
The json comes out looking like:
{"type":"MapContainer","map":[{"type":"SubA","data":1.0},{"type":"StringWrapper","s":"valueA"},{"type":"SubB","data":2.0},{"type":"DoubleWrapper","d":2.0},{"type":"SubB","data":3.0},{"type":"SubA","data":1.0},{"type":"SubB","data":4.0},{"type":"matt.play.FakeNull"}]}
I don't like the way this looks. I want the nulls to simply be null and the primitives to simply be primitives.
import kotlinx.serialization.KSerializer
import kotlinx.serialization.PolymorphicSerializer
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.descriptors.buildClassSerialDescriptor
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.json.Json
import kotlinx.serialization.modules.SerializersModule
import kotlinx.serialization.modules.polymorphic
import kotlinx.serialization.modules.subclass
import kotlin.collections.set
#Serializable
abstract class SuperClass
#Serializable
#SerialName("SubA")
data class SubA(val data: Double): SuperClass()
#Serializable
#SerialName("SubB")
data class SubB(val data: Double): SuperClass()
#Serializable
#SerialName("MapContainer")
data class MapContainer<K: SuperClass, V>(val map: Map<K, V>): Map<K, V> by map
#Serializable
#SerialName("StringWrapper")
data class StringWrapper(val s: String)
#Serializable
#SerialName("DoubleWrapper")
data class DoubleWrapper(val d: Double)
object StringClassSerializer: KSerializer<String> {
override val descriptor = buildClassSerialDescriptor("string")
override fun deserialize(decoder: Decoder) = decoder.decodeSerializableValue(StringWrapper.serializer()).s
override fun serialize(encoder: Encoder, value: String) =
encoder.encodeSerializableValue(StringWrapper.serializer(), StringWrapper(value))
}
object DoubleClassSerializer: KSerializer<Double> {
override val descriptor = buildClassSerialDescriptor("double")
override fun deserialize(decoder: Decoder) = decoder.decodeSerializableValue(DoubleWrapper.serializer()).d
override fun serialize(encoder: Encoder, value: Double) =
encoder.encodeSerializableValue(DoubleWrapper.serializer(), DoubleWrapper(value))
}
#Serializable
object FakeNull
fun main() {
val theMap = mutableMapOf<SuperClass, Any?>()
theMap[SubA(1.0)] = "valueA"
theMap[SubB(2.0)] = 2.0
theMap[SubB(3.0)] = SubA(1.0)
theMap[SubB(4.0)] = FakeNull /*wish I could make this just `null`*/
val theMapContainer = MapContainer(theMap)
val format = Json {
allowStructuredMapKeys = true
ignoreUnknownKeys = true
serializersModule = SerializersModule {
polymorphic(SuperClass::class) {
subclass(SubA::class)
subclass(SubB::class)
}
polymorphic(Any::class) {
/*I wish I could remove all of this primitive wrapper stuff*/
default {
when (it) {
StringWrapper::class.simpleName -> StringClassSerializer
DoubleWrapper::class.simpleName -> DoubleClassSerializer
else -> throw RuntimeException("unknown type: ${it}?")
}
}
subclass(String::class, StringClassSerializer)
subclass(Double::class, DoubleClassSerializer)
subclass(SubA::class)
subclass(SubB::class)
subclass(FakeNull::class)
}
polymorphic(
MapContainer::class, MapContainer::class, actualSerializer = MapContainer.serializer(
PolymorphicSerializer(SuperClass::class),
PolymorphicSerializer(Any::class)
) as KSerializer<MapContainer<*, *>>
)
}
}
val encoded = format.encodeToString(PolymorphicSerializer(MapContainer::class), theMapContainer)
println("\n\n${encoded}\n\n")
val decoded = format.decodeFromString(PolymorphicSerializer(MapContainer::class), encoded)
if (theMapContainer != decoded) {
throw RuntimeException("the decoded object is not the same as the original")
} else {
println("success")
}
}
Primitives (such as strings, numbers, and enums) by default are serialized as JSON primitives (e.g., "answer" or 42), not JSON objects ({ ... }). This is why they don't support polymorphic serialization; there is no "space" to place the type information in (the class discriminator).
There is no JSON object to place the class discriminator in, e.g., {"type": "fully.qualified.Name"} by default.
But, kotlinx serialization does allow you to write custom serializers, which allows you to work around this. I wrote a custom serializer for enums since I wanted to register enums as concrete types in polymophic serialization. It sounds like you should be able to do something similar. (Disclosure: I only read your problem description in detail; not your ongoing attempts/solution.)
A serializer which supports registering [Enum]s as subclasses in polymorphic serialization when class discriminators are used.
When class discriminators are used, an enum is not encoded as a structure which the class discriminator can be added to.
An exception is thrown when initializing [Json]: " "Serializer for of kind ENUM cannot be serialized polymorphically with class discriminator."
This serializer encodes the enum as a structure with a single value holding the enum value.
Use this serializer to register the enum in the serializers module, e.g.:
subclass( <enum>::class, PolymorphicEnumSerializer( <enum>.serializer() )
This custom serializer can possibly be generalized to any primitive type and thus support your use case.
I have a simple data class being returned from a REST endpoint.
data class SummarizedReturn(
val NET_CASH_FLOW: BigDecimal,
val ROI_PERCENTAGE: BigDecimal
)
When it is returned, the object looks like this:
{
summarizedReturn: {
net_CASH_FLOW: -194703.12028723184,
roi_PERCENTAGE: -35,
}
}
This is not what I need. I need all letters to be capitalized. So I added the JsonProperty annotation
data class SummarizedReturn(
#JsonProperty("NET_CASH_FLOW")
val NET_CASH_FLOW: BigDecimal,
#JsonProperty("ROI_PERCENTAGE")
val ROI_PERCENTAGE: BigDecimal,
)
This did not change anything. I still get the result the same as above.
I then changed the property names and kept the annotation
data class SummarizedReturn(
#JsonProperty("NET_CASH_FLOW")
val netCashFlow: BigDecimal,
#JsonProperty("ROI_PERCENTAGE")
val roiPercentage: BigDecimal,
)
and that returned what I wanted.
{
summarizedReturn: {
NET_CASH_FLOW: -194703.12028723184,
ROI_PERCENTAGE: -35,
}
}
Why did the annotation not work on the initial version of the class? How can I keep my property names all capitalized and have the Jackson value to be the same?
There is an issue with interoperability of Java annotations in Kotlin code. You can register Jackson's Kotlin module to get rid of this problems:
import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper
fun main() {
val mapper = jacksonObjectMapper() // <= shortcut to ObjectMapper().registerKotlinModule()
println(mapper.writeValueAsString(SummarizedReturn(
BigDecimal("-194703.12028723184"),
BigDecimal("-35"))))
}
Output:
{"NET_CASH_FLOW":-194703.12028723184,"ROI_PERCENTAGE":-35}
This will also require you to add com.fasterxml.jackson.module:jackson-module-kotlin to your dependencies.
PS: Alternatively you can solve it by using slightly different target:
#JsonPropery => #get:JsonPropery (or field:#JsonPropery in case this data class will also be used for deserialization)
I have a Firestore collection that holds different data objects with no common key or values.
In Kotlin, this is represented by something like
sealed class Task()
data class WorkTask(val id: String): Task()
data class ReductionTask(val time: Date): Task()
I would like to deserialize the data from the Firestore collection in a way like:
val tasks = result.toObjects(Task::class.java)
val workTasks = result.filterInstance(WorkTask::class.java)
val reductionTasks= result.filterInstance(ReductionTask::class.java)
In summary, I would like to retrieve a union from Firestore WorkTask | ReductionTask | OtherTask that I would be able to hold in one list and later either filter or patternmatch by instance.
EDIT:
Currently, my workaround is to have 1 common key (type) that holds the type of the object:
inline fun <reifed T: Any> QuerySnapshot.deserializeByType(
crossinline selector: (type:String) -> Class<out T>
): List<T> {
return this.documents.map({ document ->
val type = firestoreDoc.getString("type")
document.toObject(selector(type))
})
}
querySnapshot.deserializeByType<Task> { type ->
when (type) {
"WORK" -> WorkTask::class.java
"REDUCE" -> ReductionTask::class.java
...
}
}
And in theory, I could just provide a list of classes and let it try/catch to deserialize. But that seems to be hacky as hell.
Since the Firestore SDK is implemented in Java, it doesn't know anything about Kotlin sealed classes. It is working purely off what understands about JavaBean style POJO objects that use conventions for names of getter and setter methods on classes. toObject() is simply mapping the names of Firestore document fields to the getters and setters (obtained by reflection) on the provided class instance. That's all.
Your workaround (or some variation of it) is currently the only viable option, since your code needs to make a judgement about which actual class is actually being represented by the document in question. There's really no way for the Firestore SDK to know which class the document should populate - you have to tell it that.