With kotlinx.serialization polymorphism, I want to get
{"type":"veh_t","owner":"Ivan","bodyType":"cistern","carryingCapacityInTons":5,"detachable":false}
but I get
{"type":"kotlin.collections.LinkedHashMap","owner":"Ivan","bodyType":"cistern","carryingCapacityInTons":5,"detachable":false}
I use the following models
interface Vehicle {
val owner: String
}
#Serializable
#SerialName("veh_p")
data class PassengerCar(
override val owner: String,
val numberOfSeats: Int
) : Vehicle
#Serializable
#SerialName("veh_t")
data class Truck(
override val owner: String,
val body: Body
) : Vehicle {
#Serializable
data class Body(
val bodyType: String,
val carryingCapacityInTons: Int,
val detachable: Boolean
//a lot of other fields
)
}
I apply the following Json
inline val VehicleJson: Json get() = Json(context = SerializersModule {
polymorphic(Vehicle::class) {
PassengerCar::class with PassengerCar.serializer()
Truck::class with TruckKSerializer
}
})
I use serializer TruckKSerializer because the server adopts a flat structure. At the same time, in the application I want to use an object Truck.Body. For flatten I override fun serialize(encoder: Encoder, obj : T) and fun deserialize(decoder: Decoder): T in Serializator using JsonOutput and JsonInput according to the documentation in these classes.
object TruckKSerializer : KSerializer<Truck> {
override val descriptor: SerialDescriptor = SerialClassDescImpl("Truck")
override fun serialize(encoder: Encoder, obj: Truck) {
val output = encoder as? JsonOutput ?: throw SerializationException("This class can be saved only by Json")
output.encodeJson(json {
obj::owner.name to obj.owner
encoder.json.toJson(Truck.Body.serializer(), obj.body)
.jsonObject.content
.forEach { (name, value) ->
name to value
}
})
}
#ImplicitReflectionSerializer
override fun deserialize(decoder: Decoder): Truck {
val input = decoder as? JsonInput
?: throw SerializationException("This class can be loaded only by Json")
val tree = input.decodeJson() as? JsonObject
?: throw SerializationException("Expected JsonObject")
return Truck(
tree.getPrimitive("owner").content,
VehicleJson.fromJson<Truck.Body>(tree)
)
}
}
And finally, I use stringify(serializer: SerializationStrategy<T>, obj: T)
VehicleJson.stringify(
PolymorphicSerializer(Vehicle::class),
Truck(
owner = "Ivan",
body = Truck.Body(
bodyType = "cistern",
carryingCapacityInTons = 5,
detachable = false
)
)
)
I end up with {"type":"kotlin.collections.LinkedHashMap", ...}, but I need {"type":"veh_t", ...}
How do I get the right type? I want using polymorphism for Vehicle and encode Body object with Truck.Body.serializer() to flatten.
With this serialization, the PassengerCar class runs fine.
VehicleJson.stringify(
PolymorphicSerializer(Vehicle::class),
PassengerCar(
owner = "Oleg",
numberOfSeats = 4
)
)
Result is correct:
{"type":"veh_p","owner":"Oleg","numberOfSeats":4}
I think the problem is the custom serializer TruckKSerializer.
And I noticed if I use in my overridden fun serialize(encoder: Encoder, obj : T) next code
encoder
.beginStructure(descriptor)
.apply {
//...
}
.endStructure(descriptor)
I get the correct type but cannot flatten the object Truck.Body using its serializer.
the correct way to open and close a composite {}
is this code
val composite = encoder.beginStructure(descriptor)
// use composite instead of encoder here
composite.endStructure(descriptor)
and you should be able to serialize Body using .encodeSerializable(Body.serializer(), body)
and always pass the descriptor along otherwise it will fall back to stuff like that LinkedhashMap for the json dictionary
Related
This question already has an answer here:
kotlinx deserialization: different types && scalar && arrays
(1 answer)
Closed 7 months ago.
With a structure similar to the following:
#Serializable
sealed class Parameters
#Serializable
data class StringContainer(val value: String): Parameters()
#Serializable
data class IntContainer(val value: Int): Parameters()
#Serializable
data class MapContainer(val value: Map<String, Parameters>): Parameters()
// more such as list, bool and other fairly (in the context) straight forward types
And the following container class:
#Serializable
data class PluginConfiguration(
// other value
val parameters: Parameters.MapContainer,
)
I want to reach a (de)serialization where the paramters are configured as a flexible json (or other) map, as one would usually expect:
{
"parameters": {
"key1": "String value",
"key2": 12,
"key3": {}
}
}
And so on. Effectively creating a flexible structure that is still structured enough to not be completely uncontrolled as Any would be. There's a fairly clearly defined (de)serialization, but I cannot figure how to do this.
I've tried reading the following
https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serialization-guide.md
And I do have a hunch that a polymorphic serializer is needed, but so far I'm bumping in to either generic structures, which I believe is way overkill for my purpose or that it for some reason cannot find the serializer for my subclasses, when writing a custom serializer for Parameters.
Update
So using custom serializers combined with surrogate classes, most things are working. The current problem is when values are put into the map, I get a kotlin.IllegalStateException: Primitives cannot be serialized polymorphically with 'type' parameter. You can use 'JsonBuilder.useArrayPolymorphism' instead. Even when I enable array polymorphism this error arises
The answer with kotlinx deserialization: different types && scalar && arrays is basically the answer, and the one I will accept. However, for future use, the complete code to my solution is as follows:
Class hierarchy
#kotlinx.serialization.Serializable(with = ParametersSerializer::class)
sealed interface Parameters
#kotlinx.serialization.Serializable(with = IntContainerSerializer::class)
data class IntContainer(
val value: Int
) : Parameters
#kotlinx.serialization.Serializable(with = StringContainerSerializer::class)
data class StringContainer(
val value: String
) : Parameters
#kotlinx.serialization.Serializable(with = MapContainerSerializer::class)
data class MapContainer(
val value: Map<String, Parameters>
) : Parameters
#kotlinx.serialization.Serializable
data class PluginConfiguration(
val plugin: String,
val parameters: MenuRunnerTest.MapContainer
)
Serializers:
abstract class BaseParametersSerializer<T : Parameters> : KSerializer<T> {
override val descriptor: SerialDescriptor = JsonElement.serializer().descriptor
override fun serialize(encoder: Encoder, value: T) {
fun toJsonElement(value: Parameters): JsonElement = when (value) {
is IntContainer -> JsonPrimitive(value.value)
is MapContainer -> JsonObject(
value.value.mapValues { toJsonElement(it.value) }
)
is StringContainer -> JsonPrimitive(value.value)
}
val sur = toJsonElement(value)
encoder.encodeSerializableValue(JsonElement.serializer(), sur)
}
override fun deserialize(decoder: Decoder): T {
with(decoder as JsonDecoder) {
val jsonElement = decodeJsonElement()
return deserializeJson(jsonElement)
}
}
abstract fun deserializeJson(jsonElement: JsonElement): T
}
object ParametersSerializer : BaseParametersSerializer<Parameters>() {
override fun deserializeJson(jsonElement: JsonElement): Parameters {
return when(jsonElement) {
is JsonPrimitive -> when {
jsonElement.isString -> StringContainerSerializer.deserializeJson(jsonElement)
else -> IntContainerSerializer.deserializeJson(jsonElement)
}
is JsonObject -> MapContainerSerializer.deserializeJson(jsonElement)
else -> throw IllegalArgumentException("Only ints, strings and strings are allowed here")
}
}
}
object StringContainerSerializer : BaseParametersSerializer<StringContainer>() {
override fun deserializeJson(jsonElement: JsonElement): StringContainer {
return when(jsonElement) {
is JsonPrimitive -> StringContainer(jsonElement.content)
else -> throw IllegalArgumentException("Only strings are allowed here")
}
}
}
object IntContainerSerializer : BaseParametersSerializer<IntContainer>() {
override fun deserializeJson(jsonElement: JsonElement): IntContainer {
return when (jsonElement) {
is JsonPrimitive -> IntContainer(jsonElement.int)
else -> throw IllegalArgumentException("Only ints are allowed here")
}
}
}
object MapContainerSerializer : BaseParametersSerializer<MapContainer>() {
override fun deserializeJson(jsonElement: JsonElement): MapContainer {
return when (jsonElement) {
is JsonObject -> MapContainer(jsonElement.mapValues { ParametersSerializer.deserializeJson(it.value) })
else -> throw IllegalArgumentException("Only maps are allowed here")
}
}
}
This structure should be expandable for lists, doubles and other structures, not included in the example :)
The Problem
Due to project architecture, backward compatibility and so on, I need to change class discriminator on one abstract class and all classes that inherit from it. Ideally, I want it to be an enum.
I tried to use #JsonClassDiscriminator but Kotlinx still uses type member as discriminator which have name clash with member in class. I changed member name to test what will happen and Kotlinx just used type as discriminator.
Also, outside of annotations, I want to avoid changing these classes. It's shared code, so any non backward compatible changes will be problematic.
Code
I prepared some code, detached from project, that I use for testing behavior.
fun main() {
val derived = Derived("type", "name") as Base
val json = Json {
prettyPrint = true
encodeDefaults = true
serializersModule = serializers
}.encodeToString(derived)
print(json)
}
#Serializable
#JsonClassDiscriminator("source")
data class Derived(
val type: String?,
val name: String?,
) : Base() {
override val source = FooEnum.A
}
#Serializable
#JsonClassDiscriminator("source")
abstract class Base {
abstract val source: FooEnum
}
enum class FooEnum { A, B }
internal val serializers = SerializersModule {
polymorphic(Base::class) {
subclass(Derived::class)
}
}
If I don't change type member name, I got this error:
Exception in thread "main" java.lang.IllegalArgumentException:
Polymorphic serializer for class my.pack.Derived has property 'type'
that conflicts with JSON class discriminator. You can either change
class discriminator in JsonConfiguration, rename property with
#SerialName annotation or fall back to array polymorphism
If I do change the name, I got this JSON which clearly shows, that json type discriminator wasn't changed.
{
"type": "my.pack.Derived",
"typeChanged": "type",
"name": "name",
"source": "A"
}
Kotlinx Serialization doesn't allow for significant customisation of the default type discriminator - you can only change the name of the field.
Encoding default fields
Before I jump into the solutions, I want to point out that in these examples using #EncodeDefault or Json { encodeDefaults = true } is required, otherwise Kotlinx Serialization won't encode your val source.
#Serializable
data class Derived(
val type: String?,
val name: String?,
) : Base() {
#EncodeDefault
override val source = FooEnum.A
}
Changing the discriminator field
You can use #JsonClassDiscriminator to define the name of the discriminator
(Note that you only need #JsonClassDiscriminator on the parent Base class, not both)
However, #JsonClassDiscriminator is more like an 'alternate name', not an override. To override it, you can set classDiscriminator in the Json { } builder
val mapper = Json {
prettyPrint = true
encodeDefaults = true
serializersModule = serializers
classDiscriminator = "source"
}
Discriminator value
You can change the value of type for subclasses though - use #SerialName("...") on your subclasses.
#Serializable
#SerialName("A")
data class Derived(
val type: String?,
val name: String?,
) : Base()
Including the discriminator in a class
You also can't include the discriminator in your class - https://github.com/Kotlin/kotlinx.serialization/issues/1664
So there are 3 options.
Closed polymorphism
Change your code to use closed polymorphism
Since Base is a sealed class, instead of an enum, you can use type-checks on any Base instance
fun main() {
val derived = Derived("type", "name")
val mapper = Json {
prettyPrint = true
encodeDefaults = true
classDiscriminator = "source"
}
val json = mapper.encodeToString(Base.serializer(), derived)
println(json)
val entity = mapper.decodeFromString(Base.serializer(), json)
when (entity) {
is Derived -> println(entity)
}
}
#Serializable
#SerialName("A")
data class Derived(
val type: String?,
val name: String?,
) : Base()
#Serializable
sealed class Base
Since Base is now sealed, it's basically the same as an enum, so there's no need for your FooEnum.
val entity = mapper.decodeFromString(Base.serializer(), json)
when (entity) {
is Derived -> println(entity)
// no need for an 'else'
}
However, you still need Json { classDiscriminator= "source" }...
Content-based deserializer
Use a content-based deserializer.
This would mean you wouldn't need to make Base a sealed class, and you could manually define a default if the discriminator is unknown.
object BaseSerializer : JsonContentPolymorphicSerializer<Base>(Base::class) {
override fun selectDeserializer(element: JsonElement) = when {
"source" in element.jsonObject -> {
val sourceContent = element.jsonObject["source"]?.jsonPrimitive?.contentOrNull
when (
val sourceEnum = FooEnum.values().firstOrNull { it.name == sourceContent }
) {
FooEnum.A -> Derived.serializer()
FooEnum.B -> error("no serializer for $sourceEnum")
else -> error("'source' is null")
}
}
else -> error("no 'source' in JSON")
}
}
This is a good fit in some situations, especially when you don't have a lot of control over the source code. However, I think this is pretty hacky, and it would be easy to make a mistake in selecting the serializer.
Custom serializer
Alternatively you can write a custom serializer.
The end result isn't that different to the content-based deserializer. It's still complicated, and is still easy to make mistakes with. For these reasons, I won't give a complete example.
This is beneficial because it provides more flexibility if you need to encode/decode with non-JSON formats.
#Serializable(with = BaseSerializer::class)
#JsonClassDiscriminator("source")
sealed class Base {
abstract val source: FooEnum
}
object BaseSerializer : KSerializer<Base> {
override val descriptor: SerialDescriptor = buildClassSerialDescriptor("Base") {
// We have to write our own custom descriptor, because setting a custom serializer
// stops the plugin from generating one
}
override fun deserialize(decoder: Decoder): Base {
require(decoder is JsonDecoder) {"Base can only be deserialized as JSON"}
val sourceValue = decoder.decodeJsonElement().jsonObject["source"]?.jsonPrimitive?.contentOrNull
// same logic as the JsonContentPolymorphicSerializer...
}
override fun serialize(encoder: Encoder, value: Base) {
require(encoder is JsonEncoder) {"Base can only be serialized into JSON"}
when (value) {
is Derived -> encoder.encodeSerializableValue(Derived.serializer(), value)
}
}
}
Not sure if it is possible yet but for the life of me I cannot figure out how to serialize this.
sealed class ServiceResult<out T : Any> {
data class Success<out T : Any>(val data: T) : ServiceResult<T>()
data class Error(val exception: Exception) : ServiceResult<Nothing>()
}
Everything that is stuff into T is using #Serializable ex:
#Serializable
data class GalleryDTO(
override val id: Int,
override val dateCreated: Long,
override val dateUpdated: Long,
val name:String,
val description:String,
val photos:List<DTOMin>
) : DTO
As Animesh Sahu already mentioned there is an issue for this topic that is still open, but the solution using a surrogate suggested by Михаил Нафталь for serialization of Error can actually be used also to serialize the polymorphic ServiceResult, by creating a surrogate that mixes the fields of Success and Error. For the sake of simplicity in the example I only represent the exception message.
#Serializable(with = ServiceResultSerializer::class)
sealed class ServiceResult<out T : Any> {
data class Success<out T : Any>(val data: T) : ServiceResult<T>()
data class Error(val exceptionMessage: String?) : ServiceResult<Nothing>()
}
class ServiceResultSerializer<T : Any>(
tSerializer: KSerializer<T>
) : KSerializer<ServiceResult<T>> {
#Serializable
#SerialName("ServiceResult")
data class ServiceResultSurrogate<T : Any>(
val type: Type,
// The annotation is not necessary, but it avoids serializing "data = null"
// for "Error" results.
#EncodeDefault(EncodeDefault.Mode.NEVER)
val data: T? = null,
#EncodeDefault(EncodeDefault.Mode.NEVER)
val exceptionMessage: String? = null
) {
enum class Type { SUCCESS, ERROR }
}
private val surrogateSerializer = ServiceResultSurrogate.serializer(tSerializer)
override val descriptor: SerialDescriptor = surrogateSerializer.descriptor
override fun deserialize(decoder: Decoder): ServiceResult<T> {
val surrogate = surrogateSerializer.deserialize(decoder)
return when (surrogate.type) {
ServiceResultSurrogate.Type.SUCCESS ->
if (surrogate.data != null)
ServiceResult.Success(surrogate.data)
else
throw SerializationException("Missing data for successful result")
ServiceResultSurrogate.Type.ERROR ->
ServiceResult.Error(surrogate.exceptionMessage)
}
}
override fun serialize(encoder: Encoder, value: ServiceResult<T>) {
val surrogate = when (value) {
is ServiceResult.Error -> ServiceResultSurrogate(
ServiceResultSurrogate.Type.ERROR,
exceptionMessage = value.exceptionMessage
)
is ServiceResult.Success -> ServiceResultSurrogate(
ServiceResultSurrogate.Type.SUCCESS,
data = value.data
)
}
surrogateSerializer.serialize(encoder, surrogate)
}
}
This solution can also be easily extended to support nullable Ts. In this case when deserializing you will also have to check if null is a valid value for T (it can be done by checking descriptor.isNullable on tSerializer) and you will also have to cast data as T.
Polymorphic serialization will be a mess in this case (you will have to manually register all possible types passed as a generic parameter to ServiceResult<T>), and will have several limitations (it would be impossible to register primitive types (including Nothing and String) as generic parameters, for instance).
If you only need serialization (aka encoding), I'd recommend to serialize both subtypes independently (for convenience, wrap subtype determination into auxilary function):
inline fun <reified T : Any> serializeServiceResult(x: ServiceResult<T>) = when (x) {
is ServiceResult.Success -> Json.encodeToString(x)
is ServiceResult.Error -> Json.encodeToString(x)
}
To serialize ServiceResult.Success you need just to mark it with #Serializable annotation. The tricky part here is serialization of ServiceResult.Error, or more precisely, serialization of its exception: Exception field. I'd suggest to serialize only its message (via surrogate):
sealed class ServiceResult<out T : Any> {
#Serializable
data class Success<out T : Any>(val data: T) : ServiceResult<T>()
#Serializable(with = ErrorSerializer::class)
data class Error(val exception: Exception) : ServiceResult<Nothing>()
}
#Serializable
private data class ErrorSurrogate(val error: String)
class ErrorSerializer : KSerializer<ServiceResult.Error> {
override val descriptor: SerialDescriptor = ErrorSurrogate.serializer().descriptor
override fun deserialize(decoder: Decoder): ServiceResult.Error {
val surrogate = decoder.decodeSerializableValue(ErrorSurrogate.serializer())
return ServiceResult.Error(Exception(surrogate.error))
}
override fun serialize(encoder: Encoder, value: ServiceResult.Error) {
val surrogate = ErrorSurrogate(value.exception.toString())
encoder.encodeSerializableValue(ErrorSurrogate.serializer(), surrogate)
}
}
I'm trying to pass data class to the service-proxy of Vert.x like this:
data class Entity(val field: String)
#ProxyGen
#VertxGen
public interface DatabaseService {
DatabaseService createEntity(Entity entity, Handler<AsyncResult<Void>> resultHandler);
}
However, the service-proxy requires a DataObject as the parameter type.
Below are what I've tried so far.
First, I rewrite the data class as:
#DataObject
data class Entity(val field: String) {
constructor(json: JsonObject) : this(
json.getString("field")
)
fun toJson(): JsonObject = JsonObject.mapFrom(this)
}
Although this works, the code is redundant, so I tried the kapt with the following generator:
override fun process(annotations: Set<TypeElement>, roundEnv: RoundEnvironment): Boolean {
roundEnv.getElementsAnnotatedWith(ProxyDataObject::class.java).forEach { el ->
val className = el.simpleName.toString()
val pack = processingEnv.elementUtils.getPackageOf(el).toString()
val filename = "Proxy$className"
val classBuilder = TypeSpec.classBuilder(filename)
val primaryConstructorBuilder = FunSpec.constructorBuilder()
val secondaryConstructorBuilder = FunSpec.constructorBuilder().addParameter("json", JsonObject::class)
val secondaryConstructorCodeBlocks = mutableListOf<CodeBlock>()
el.enclosedElements.forEach {
if (it.kind == ElementKind.FIELD) {
val name = it.simpleName.toString()
val kClass = getClass(it) // get the corresponding Kotlin class
val jsonTypeName = getJsonTypeName(it) // get the corresponding type name in methods of JsonObject
classBuilder.addProperty(PropertySpec.builder(name, kClass).initializer(name).build())
primaryConstructorBuilder.addParameter(name, kClass)
secondaryConstructorCodeBlocks.add(CodeBlock.of("json.get$jsonTypeName(\"$name\")"))
}
}
secondaryConstructorBuilder.callThisConstructor(secondaryConstructorCodeBlocks)
classBuilder
.addAnnotation(DataObject::class)
.addModifiers(KModifier.DATA)
.primaryConstructor(primaryConstructorBuilder.build())
.addFunction(secondaryConstructorBuilder.build())
.addFunction(
FunSpec.builder("toJson").returns(JsonObject::class).addStatement("return JsonObject.mapFrom(this)").build()
)
val generatedFile = FileSpec.builder(pack, filename).addType(classBuilder.build()).build()
generatedFile.writeTo(processingEnv.filer)
}
return true
}
Then I can get the correct generated file by simply writing the original data class, but when I execute the building after cleaning, I still get the following error:
Could not generate model for DatabaseService#createEntity(ProxyEntity,io.vertx.core.Handler<io.vertx.core.AsyncResult<java.lang.Void>>): type ProxyEntity is not legal for use for a parameter in proxy
It seems that the generated annotation #DataObject is not processed.
So what should I do? Is there a better solution?
how can I set properties of a dataclass by its name. For example, I have a raw HTTP GET response
propA=valueA
propB=valueB
and a data class in Kotlin
data class Test(var propA: String = "", var propB: String = ""){}
in my code i have an function that splits the response to a key value array
val test: Test = Test()
rawResp?.split('\n')?.forEach { item: String ->
run {
val keyValue = item.split('=')
TODO
}
}
In JavaScript I can do the following
response.split('\n').forEach(item => {
let keyValue = item.split('=');
this.test[keyValue[0]] = keyValue[1];
});
Is there a similar way in Kotlin?
You cannot readily do this in Kotlin the same way you would in JavaScript (unless you are prepared to handle reflection yourself), but there is a possibility of using a Kotlin feature called Delegated Properties (particularly, a use case Storing Properties in a Map of that feature).
Here is an example specific to code in your original question:
class Test(private val map: Map<String, String>) {
val propA: String by map
val propB: String by map
override fun toString() = "${javaClass.simpleName}(propA=$propA,propB=$propB)"
}
fun main() {
val rawResp: String? = """
propA=valueA
propB=valueB
""".trimIndent()
val props = rawResp?.split('\n')?.map { item ->
val (key, value) = item.split('=')
key to value
}?.toMap() ?: emptyMap()
val test = Test(props)
println("Property 'propA' of test is: ${test.propA}")
println("Or using toString: $test")
}
This outputs:
Property 'propA' of test is: valueA
Or using toString: Test(propA=valueA,propB=valueB)
Unfortunately, you cannot use data classes with property delegation the way you would expect, so you have to 'pay the price' and define the overridden methods (toString, equals, hashCode) on your own if you need them.
By the question, it was not clear for me if each line represents a Test instance or not. So
If not.
fun parse(rawResp: String): Test = rawResp.split("\n").flatMap { it.split("=") }.let { Test(it[0], it[1]) }
If yes.
fun parse(rawResp: String): List<Test> = rawResp.split("\n").map { it.split("=") }.map { Test(it[0], it[1]) }
For null safe alternative you can use nullableString.orEmpty()...