Kotlinx serialization documentation
According to Kotlinx.serialization user-defined annotations doc:
"Inside a process of serialization/deserialization, your own annotation class are available in SerialDescriptor object" :
override fun encodeElement(desc: SerialDescriptor, index: Int): Boolean {
val annotations = desc.getElementAnnotations(index)
...
}
What I want to do
I need a #Transient equivalent, but conditional:
classic way where : Json.stringify(serializer, myClass) works as usual.
custom way where : Json.stringify(customSerializer, myClass) would return usual json but exculding all #MyAnnotation-tagged values.
Here is my code
#SerialInfo
#Target(AnnotationTarget.PROPERTY)
annotation class CustomAnnotation
#Serializable
data class MyClass(val a: String, #CustomAnnotation val b: Int = -1)
And I would like to build a custom Serializer and achieve something like
override fun encodeElement(desc: SerialDescriptor, index: Int): Boolean {
val isTaggedAsCustomAnnotation = desc.getElementAnnotations(index).any{ it is CustomAnnotation }
val myCondition = mySerializer.getMyConditionBlablabla
if(myCondition && isTaggedAsCustomAnnotation) {
encode()
}
...
}
What I found
abstract class ElementValueEncoder : Encoder, CompositeEncoder {
...
open fun encodeElement(desc: SerialDescriptor, index: Int): Boolean = true
}
But I don't know how I can build a custom Serializer so that I can override that function Encoder.encodeElement. Where can I access to ElementValueEncoder in a custom Serializer ?
I also found this sample demo in kotlinx.serialization github repo. It's using TaggedEncoder & TaggedDecoder where I'm able to override encodeTaggedValue. But here again I don't know how I can use those encoder/decoder in a process of serialization/deserialization.
Finally
Where can I override fun encodeElement(desc: SerialDescriptor, index: Int): Boolean, and how I can handle my own-defined serialization annotation ?
Thanks !!
First of all, you need to grasp the difference between Serializer and Encoder. Serializer (represented by KSerializer) defines how your class looks like, and Encoder (represented by e.g. JsonOutput) defines how data will be recorded. You can find more info on that topic here: https://github.com/Kotlin/KEEP/blob/master/proposals/extensions/serialization.md#core-api-overview-and-mental-model .
So, custom annotations feature is mainly used for providing format-specific information to Encoder. Typical usage of such an annotation is ProtoId – property id, specific to protobuf format, that should be recognized by ProtobufEncoder. Such annotations are usually defined by format authors alongside their encoders.
What you want to do here, as I can see, is to use already existing encoder (JSON format), so overriding encodeElement is impossible since Json encoders can not be subclassed. I'd advise you to use custom json transofrming serializer to achieve your goal. Unfortunately, currently kotlinx.serialization does not have mechanism to generalize such a transformation, so you need to write such serializer for each class.
Related
I need to set up a serialization/deserialization mechanism for a polymorphic class hierarchy that also includes primitives and nulls. There are container classes containing collections with polymorphic objects, primitives, and nulls. And, the subclasses for these objects are spread across modules (therefore sealed is not an option).
I have been reading through the kotlinx.serialization polymorphism docs trying to come up with a solution. I've been able to make some incremental progress by working through that tutorial but I seem to still be hitting a wall when I try to put everything together.
The code I am posting here is a minimal example that brings together everything I need. If I can get this example to work, that should cover everything I need for my real project. This example does run without error but introduces some unnecessary readability and efficiency issues.
All classes in my custom class hierarchy are serializable data classes. The outermost container object that needs to be serialized/deserialized is a map wrapper. This map has keys which are each an instance of one of these data classes. And the values of this map can be primitives, nulls, or instances of one of my data classes. I think my main challenge here is to include those primitives and nulls in my polymorphic serialization in a clean way.
The goal of my code below is to represent this problem in the simplest way possible and to serialize and deserialize one container object successfully.
There are two main issues in the code:
I've had to replace null with FakeNull. Without this, I get null cannot be cast to non-null type kotlin.Any. This will reduce the readability and simplicity of my code and I suspect it could decrease efficiency as well.
I've had to add StringClassSerializer and DoubleClassSerializer and wrapper classes. I would also need to add serializers like these for every primitive class. If I don't register these primitives as subclasses of Any, I get Class 'String' is not registered for polymorphic serialization in the scope of 'Any'.. And if I try to register them with their default serializers (like subclass(String::class, String.serializer())) I get Serializer for String of kind STRING cannot be serialized polymorphically with class discriminator.. The problem with using serializers like StringClassSerializer and wrappers like StringWrapper is that it removes the efficiency and readability benefits of using primitives.
The json comes out looking like:
{"type":"MapContainer","map":[{"type":"SubA","data":1.0},{"type":"StringWrapper","s":"valueA"},{"type":"SubB","data":2.0},{"type":"DoubleWrapper","d":2.0},{"type":"SubB","data":3.0},{"type":"SubA","data":1.0},{"type":"SubB","data":4.0},{"type":"matt.play.FakeNull"}]}
I don't like the way this looks. I want the nulls to simply be null and the primitives to simply be primitives.
import kotlinx.serialization.KSerializer
import kotlinx.serialization.PolymorphicSerializer
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.descriptors.buildClassSerialDescriptor
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.json.Json
import kotlinx.serialization.modules.SerializersModule
import kotlinx.serialization.modules.polymorphic
import kotlinx.serialization.modules.subclass
import kotlin.collections.set
#Serializable
abstract class SuperClass
#Serializable
#SerialName("SubA")
data class SubA(val data: Double): SuperClass()
#Serializable
#SerialName("SubB")
data class SubB(val data: Double): SuperClass()
#Serializable
#SerialName("MapContainer")
data class MapContainer<K: SuperClass, V>(val map: Map<K, V>): Map<K, V> by map
#Serializable
#SerialName("StringWrapper")
data class StringWrapper(val s: String)
#Serializable
#SerialName("DoubleWrapper")
data class DoubleWrapper(val d: Double)
object StringClassSerializer: KSerializer<String> {
override val descriptor = buildClassSerialDescriptor("string")
override fun deserialize(decoder: Decoder) = decoder.decodeSerializableValue(StringWrapper.serializer()).s
override fun serialize(encoder: Encoder, value: String) =
encoder.encodeSerializableValue(StringWrapper.serializer(), StringWrapper(value))
}
object DoubleClassSerializer: KSerializer<Double> {
override val descriptor = buildClassSerialDescriptor("double")
override fun deserialize(decoder: Decoder) = decoder.decodeSerializableValue(DoubleWrapper.serializer()).d
override fun serialize(encoder: Encoder, value: Double) =
encoder.encodeSerializableValue(DoubleWrapper.serializer(), DoubleWrapper(value))
}
#Serializable
object FakeNull
fun main() {
val theMap = mutableMapOf<SuperClass, Any?>()
theMap[SubA(1.0)] = "valueA"
theMap[SubB(2.0)] = 2.0
theMap[SubB(3.0)] = SubA(1.0)
theMap[SubB(4.0)] = FakeNull /*wish I could make this just `null`*/
val theMapContainer = MapContainer(theMap)
val format = Json {
allowStructuredMapKeys = true
ignoreUnknownKeys = true
serializersModule = SerializersModule {
polymorphic(SuperClass::class) {
subclass(SubA::class)
subclass(SubB::class)
}
polymorphic(Any::class) {
/*I wish I could remove all of this primitive wrapper stuff*/
default {
when (it) {
StringWrapper::class.simpleName -> StringClassSerializer
DoubleWrapper::class.simpleName -> DoubleClassSerializer
else -> throw RuntimeException("unknown type: ${it}?")
}
}
subclass(String::class, StringClassSerializer)
subclass(Double::class, DoubleClassSerializer)
subclass(SubA::class)
subclass(SubB::class)
subclass(FakeNull::class)
}
polymorphic(
MapContainer::class, MapContainer::class, actualSerializer = MapContainer.serializer(
PolymorphicSerializer(SuperClass::class),
PolymorphicSerializer(Any::class)
) as KSerializer<MapContainer<*, *>>
)
}
}
val encoded = format.encodeToString(PolymorphicSerializer(MapContainer::class), theMapContainer)
println("\n\n${encoded}\n\n")
val decoded = format.decodeFromString(PolymorphicSerializer(MapContainer::class), encoded)
if (theMapContainer != decoded) {
throw RuntimeException("the decoded object is not the same as the original")
} else {
println("success")
}
}
Primitives (such as strings, numbers, and enums) by default are serialized as JSON primitives (e.g., "answer" or 42), not JSON objects ({ ... }). This is why they don't support polymorphic serialization; there is no "space" to place the type information in (the class discriminator).
There is no JSON object to place the class discriminator in, e.g., {"type": "fully.qualified.Name"} by default.
But, kotlinx serialization does allow you to write custom serializers, which allows you to work around this. I wrote a custom serializer for enums since I wanted to register enums as concrete types in polymophic serialization. It sounds like you should be able to do something similar. (Disclosure: I only read your problem description in detail; not your ongoing attempts/solution.)
A serializer which supports registering [Enum]s as subclasses in polymorphic serialization when class discriminators are used.
When class discriminators are used, an enum is not encoded as a structure which the class discriminator can be added to.
An exception is thrown when initializing [Json]: " "Serializer for of kind ENUM cannot be serialized polymorphically with class discriminator."
This serializer encodes the enum as a structure with a single value holding the enum value.
Use this serializer to register the enum in the serializers module, e.g.:
subclass( <enum>::class, PolymorphicEnumSerializer( <enum>.serializer() )
This custom serializer can possibly be generalized to any primitive type and thus support your use case.
I'm trying to create a serializer using kotlinx.serialization for Compose Desktop classes, I have this :
#Serializer(forClass = MutableState::class)
class MutableStateSerializer<T>(private val dataSerializer: KSerializer<T>) : KSerializer<MutableState<T>> {
override fun deserialize(decoder: Decoder) = mutableStateOf(decoder.decodeSerializableValue(dataSerializer))
override val descriptor: SerialDescriptor = dataSerializer.descriptor
override fun serialize(encoder: Encoder, value: MutableState<T>) = encoder.encodeSerializableValue(dataSerializer, value.value)
}
That should be used for instances of MutableState class (as the #Serializer annotation says), but I have to put an explicit serializer for each properties otherwise I get this error :
xception in thread "main" kotlinx.serialization.SerializationException: Class 'SnapshotMutableStateImpl' is not registered for polymorphic serialization in the scope of 'MutableState'.
Mark the base class as 'sealed' or register the serializer explicitly
Code used :
#Serializable
class Test {
var number = mutableStateOf(0)
}
fun main() {
val json = Json { prettyPrint = true }
val serialized = json.encodeToString(Test())
println(serialized)
}
I have to put this annotation on my property :
#Serializable(with = MutableStateSerializer::class)
Isn't there a way to automatically link my serializer to the MutableState interface ? As the SnapshotMutableStateImpl is internal I can't set it to this class.
What you want is currently not possible. Other people seem to have requested a feature similar to what you need on GitHub: Global Custom Serializers.
Currently, for 3rd party classes, you need to specify the serializer in one of three ways:
Pass the custom serializer to the encode/decode method in case you are serializing it as the root object.
Specify the serializer on the property using #Serializable, as you do now.
Specify the serializer to be used by a full file using #file:UseSerializers.
Note that due to type inference, number will be attempted to be serialized as the return type of mutableStateOf. If you specify the type as an interface instead (does it have a supertype?), using polymorphic serialization, you could try to register the concrete type and pass your custom serializer there for the concrete type. Not really what this feature is designed for, but I believe it may work if you don't want to specify your serializer in multiple places. However, the serialized form will then include a type discriminator everywhere.
I have the following code in Kotlin which I aim to use it to convert any instance to a base64 encoded string. The same is not working and it throws the following error :
Serializer for class 'Any' is not found.\nMark the class as #Serializable or provide the serializer explicitly
How can I fix this?
class SerializerAdapter: SerializerPort {
private val logger: Logger = LoggerFactory.getLogger(javaClass.simpleName)
override fun toBase64(input: Any): String {
try {
val jsonString = Json.encodeToString(input)
return Base64.getEncoder().encodeToString(jsonString.toByteArray())
}catch (ex: Exception) {
logger.error("[BASE64 Error] error converting json object to base64 encoded string: ${ex.stackTraceToString()}")
}finally {
return ""
}
}
}
Serializing just Any is not as simple as it sounds. Serialization framework has to know the type of the data to serialize. It can use either compile type (Any in your case) or runtime type (actual type provided to toBase64()). Both options have their drawbacks. Runtime type is incomplete due to type erasure, so e.g. List<Int> and List<String> are the same. On the other hand, compile-time type may be totally lost, e.g. in generics or in cases like yours.
Kotlin serialization generally prefers compile types, especially because reified parameters make them much more usable. Unfortunately, we can't use reified here, because toBase64() is a virtual function, so it can't be inlined.
My suggestion is to change the signature of this function to additionally receive KType of provided data and then create inline function to make it convenient to use:
override fun toBase64(input: Any, type: KType): String {
try {
val serializer = Json.serializersModule.serializer(type)
val jsonString = Json.encodeToString(serializer, input)
...
}
}
#OptIn(ExperimentalStdlibApi::class)
inline fun <reified T> SerializerPort.toBase64(input: Any) = toBase64(input, typeOf<T>())
Alternatively, we can serialize using the runtime type, but note the problems I mentioned earlier - it may not work well with generics.
val serializer = Json.serializersModule.serializer(input::class.starProjectedType)
val jsonString = Json.encodeToString(serializer, input)
kotlinx.serialization is a compile-time library, so you must know up front every object you want to serialise.
Depending on your use case;
You would either need to use a runtime library that uses reflection (such as one of the many JSON, or XML serialisers) to be able to use classes that are not your own.
Or if your requirement is more that you want to be able to serialise other peoples classes then you can use your own interface that others must implement along with #Serializable (see https://github.com/Kotlin/kotlinx.serialization/issues/1005).
I need to inject a local value to a class constructor during deserialization. For example, look at the following class.
#Serializable
class SomeClass(val local: Context, val serialized: String)
I want the field local to be skipped during serialization and substituted with some predefined local value during deserialization.
The reason behind is that I'm going to transfer models through network, but operations on these models rely on a local context which I want to inject.
Because I haven't find any standard ways to achieve it, I've decided to make use of contextual serialization. So I have written the serializer:
class ContextualInjectorSerializer<T>(private val localValue: T) : KSerializer<T> {
override val descriptor = SerialDescriptor("ValueInjection", StructureKind.OBJECT)
override fun deserialize(decoder: Decoder): T {
decoder.beginStructure(descriptor).endStructure(descriptor)
return localValue
}
override fun serialize(encoder: Encoder, value: T) {
encoder.beginStructure(descriptor).endStructure(descriptor)
}
}
And used it this way:
// Context is marked with #Serializable(with = ContextSerializer::class)
val json = Json(JsonConfiguration.Stable, SerializersModule {
contextual(Context::class, ContextualInjectorSerializer(context))
})
// serialize/deserialize
Surprisingly, it works pretty fine on JVM. However, when I compiled it to JS and tested, I got TypeError: Cannot read property 'siteId' of undefined. Here siteId is a field of Context which I try to access.
Is there a standard way to inject local parameters? What's wrong with my trick?
Is there a way to use Parceler with Kotlin data classes and constructor for serialization without using #ParcelProperty annotation for each field?
If I try and use library like this:
#Parcel
data class Valve #ParcelConstructor constructor(val size: Int)
I get Error:Parceler: No corresponding property found for constructor parameter arg0. But if I add #ParcelProperty("size") it works just fine.
Why is that?
Update:
There are other another way to use this library.
I could just remove #ParcelConstructor annotation, but then I will get error
Error:Parceler: No #ParcelConstructor annotated constructor and no default empty bean constructor found.
I think (haven't tested it) I also could make all constructor parameters optional and add #JvmOverloads but that has a side effect that I have to check all properties of the class if they are null or not.
Update 2:
This is what worked for me:
#Parcel
data class Valve(val size: Int? = null)
In short generated Java class must have default empty constructor. One way to achieve that is to do as above - all variables should have default values.
According to the docs, Parceler by default works with public fields. But a usual Kotlin data class (as in your example) is rather a "traditional getter/setter bean", since every Kotlin property is represented by a private field and a getter/[setter].
TL; DR: I think this will work:
#Parcel(Serialization.BEAN)
data class Valve(val size: Int = 10)
Note the default value, it allows Kotlin to automatically generate an additional empty constructor, which is required by the Java Been specification.
Another way would be to mark the constructor that we already have:
#Parcel(Serialization.BEAN)
data class Driver #ParcelConstructor constructor(val name: String)
The specific document: https://github.com/johncarl81/parceler#gettersetter-serialization
I know this question already has an answer, but for future viewers who are also struggling to get Parceler to work with kotlin data objects, I wrote a new annotation processor to generate the Parcelable boilerplate for Kotlin data classes. It's designed to massively reduce the boilerplate code in making your data classes Parcelable:
https://github.com/grandstaish/paperparcel
Usage:
Annotate your data class with #PaperParcel, implement PaperParcelable, and add a JVM static instance of the generated CREATOR e.g.:
#PaperParcel
data class Example(
val test: Int,
...
) : PaperParcelable {
companion object {
#JvmField val CREATOR = PaperParcelExample.CREATOR
}
}
Now your data class is Parcelable and can be passed directly to a Bundle or Intent
Edit: Update with latest API
Just add the default constructor:
#Parcel
data class Valve(val size: Int) {
constructor() : this(0)
}
if you use Kotlin 1.1.4 or above it's easier to use #Parcelize annotation
For doing this first add this to build.gradle
android {
//other codes
//for using latest experimental build of Android Extensions
androidExtensions {
experimental = true
}
}
Then change your class like this
#Parcelize
data class Valve(val size: Int? = null) : Parcelable