This question already has an answer here:
kotlinx deserialization: different types && scalar && arrays
(1 answer)
Closed 7 months ago.
With a structure similar to the following:
#Serializable
sealed class Parameters
#Serializable
data class StringContainer(val value: String): Parameters()
#Serializable
data class IntContainer(val value: Int): Parameters()
#Serializable
data class MapContainer(val value: Map<String, Parameters>): Parameters()
// more such as list, bool and other fairly (in the context) straight forward types
And the following container class:
#Serializable
data class PluginConfiguration(
// other value
val parameters: Parameters.MapContainer,
)
I want to reach a (de)serialization where the paramters are configured as a flexible json (or other) map, as one would usually expect:
{
"parameters": {
"key1": "String value",
"key2": 12,
"key3": {}
}
}
And so on. Effectively creating a flexible structure that is still structured enough to not be completely uncontrolled as Any would be. There's a fairly clearly defined (de)serialization, but I cannot figure how to do this.
I've tried reading the following
https://github.com/Kotlin/kotlinx.serialization/blob/master/docs/serialization-guide.md
And I do have a hunch that a polymorphic serializer is needed, but so far I'm bumping in to either generic structures, which I believe is way overkill for my purpose or that it for some reason cannot find the serializer for my subclasses, when writing a custom serializer for Parameters.
Update
So using custom serializers combined with surrogate classes, most things are working. The current problem is when values are put into the map, I get a kotlin.IllegalStateException: Primitives cannot be serialized polymorphically with 'type' parameter. You can use 'JsonBuilder.useArrayPolymorphism' instead. Even when I enable array polymorphism this error arises
The answer with kotlinx deserialization: different types && scalar && arrays is basically the answer, and the one I will accept. However, for future use, the complete code to my solution is as follows:
Class hierarchy
#kotlinx.serialization.Serializable(with = ParametersSerializer::class)
sealed interface Parameters
#kotlinx.serialization.Serializable(with = IntContainerSerializer::class)
data class IntContainer(
val value: Int
) : Parameters
#kotlinx.serialization.Serializable(with = StringContainerSerializer::class)
data class StringContainer(
val value: String
) : Parameters
#kotlinx.serialization.Serializable(with = MapContainerSerializer::class)
data class MapContainer(
val value: Map<String, Parameters>
) : Parameters
#kotlinx.serialization.Serializable
data class PluginConfiguration(
val plugin: String,
val parameters: MenuRunnerTest.MapContainer
)
Serializers:
abstract class BaseParametersSerializer<T : Parameters> : KSerializer<T> {
override val descriptor: SerialDescriptor = JsonElement.serializer().descriptor
override fun serialize(encoder: Encoder, value: T) {
fun toJsonElement(value: Parameters): JsonElement = when (value) {
is IntContainer -> JsonPrimitive(value.value)
is MapContainer -> JsonObject(
value.value.mapValues { toJsonElement(it.value) }
)
is StringContainer -> JsonPrimitive(value.value)
}
val sur = toJsonElement(value)
encoder.encodeSerializableValue(JsonElement.serializer(), sur)
}
override fun deserialize(decoder: Decoder): T {
with(decoder as JsonDecoder) {
val jsonElement = decodeJsonElement()
return deserializeJson(jsonElement)
}
}
abstract fun deserializeJson(jsonElement: JsonElement): T
}
object ParametersSerializer : BaseParametersSerializer<Parameters>() {
override fun deserializeJson(jsonElement: JsonElement): Parameters {
return when(jsonElement) {
is JsonPrimitive -> when {
jsonElement.isString -> StringContainerSerializer.deserializeJson(jsonElement)
else -> IntContainerSerializer.deserializeJson(jsonElement)
}
is JsonObject -> MapContainerSerializer.deserializeJson(jsonElement)
else -> throw IllegalArgumentException("Only ints, strings and strings are allowed here")
}
}
}
object StringContainerSerializer : BaseParametersSerializer<StringContainer>() {
override fun deserializeJson(jsonElement: JsonElement): StringContainer {
return when(jsonElement) {
is JsonPrimitive -> StringContainer(jsonElement.content)
else -> throw IllegalArgumentException("Only strings are allowed here")
}
}
}
object IntContainerSerializer : BaseParametersSerializer<IntContainer>() {
override fun deserializeJson(jsonElement: JsonElement): IntContainer {
return when (jsonElement) {
is JsonPrimitive -> IntContainer(jsonElement.int)
else -> throw IllegalArgumentException("Only ints are allowed here")
}
}
}
object MapContainerSerializer : BaseParametersSerializer<MapContainer>() {
override fun deserializeJson(jsonElement: JsonElement): MapContainer {
return when (jsonElement) {
is JsonObject -> MapContainer(jsonElement.mapValues { ParametersSerializer.deserializeJson(it.value) })
else -> throw IllegalArgumentException("Only maps are allowed here")
}
}
}
This structure should be expandable for lists, doubles and other structures, not included in the example :)
Related
I'm trying to build a class where certain values are Observable but also Serializable.
This obviously works and the serialization works, but it's very boilerplate-heavy having to add a setter for every single field and manually having to call change(...) inside each setter:
interface Observable {
fun change(message: String) {
println("changing $message")
}
}
#Serializable
class BlahVO : Observable {
var value2: String = ""
set(value) {
field = value
change("value2")
}
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
println(BlahVO().apply { value2 = "test2" })
correctly outputs
changing value2
{"value2":"test2"}
I've tried introducing Delegates:
interface Observable {
fun change(message: String) {
println("changing $message")
}
#Suppress("ClassName")
class default<T>(defaultValue: T) {
private var value: T = defaultValue
operator fun getValue(observable: Observable, property: KProperty<*>): T {
return value
}
operator fun setValue(observable: Observable, property: KProperty<*>, value: T) {
this.value = value
observable.change(property.name)
}
}
}
#Serializable
class BlahVO : Observable {
var value1: String by Observable.default("value1")
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
println(BlahVO().apply { value1 = "test1" }) correctly triggers change detection, but it doesn't serialize:
changing value1
{}
If I go from Observable to ReadWriteProperty,
interface Observable {
fun change(message: String) {
println("changing $message")
}
fun <T> look(defaultValue: T): ReadWriteProperty<Observable, T> {
return OP(defaultValue, this)
}
class OP<T>(defaultValue: T, val observable: Observable) : ObservableProperty<T>(defaultValue) {
override fun setValue(thisRef: Any?, property: KProperty<*>, value: T) {
super.setValue(thisRef, property, value)
observable.change("blah!")
}
}
}
#Serializable
class BlahVO : Observable {
var value3: String by this.look("value3")
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
the result is the same:
changing blah!
{}
Similarly for Delegates.vetoable
var value4: String by Delegates.vetoable("value4", {
property: KProperty<*>, oldstring: String, newString: String ->
this.change(property.name)
true
})
outputs:
changing value4
{}
Delegates just doesn't seem to work with Kotlin Serialization
What other options are there to observe a property's changes without breaking its serialization that will also work on other platforms (KotlinJS, KotlinJVM, Android, ...)?
Serialization and Deserialization of Kotlin Delegates is not supported by kotlinx.serialization as of now.
There is an open issue #1578 on GitHub regarding this feature.
According to the issue you can create an intermediate data-transfer object, which gets serialized instead of the original object. Also you could write a custom serializer to support the serialization of Kotlin Delegates, which seems to be even more boilerplate, then writing custom getters and setters, as proposed in the question.
Data Transfer Object
By mapping your original object to a simple data transfer object without delegates, you can utilize the default serialization mechanisms.
This also has the nice side effect to cleanse your data model classes from framework specific annotations, such as #Serializable.
class DataModel {
var observedProperty: String by Delegates.observable("initial") { property, before, after ->
println("""Hey, I changed "${property.name}" from "$before" to "$after"!""")
}
fun toJson(): String {
return Json.encodeToString(serializer(), this.toDto())
}
}
fun DataModel.toDto() = DataTransferObject(observedProperty)
#Serializable
class DataTransferObject(val observedProperty: String)
fun main() {
val data = DataModel()
println(data.toJson())
data.observedProperty = "changed"
println(data.toJson())
}
This yields the following result:
{"observedProperty":"initial"}
Hey, I changed "observedProperty" from "initial" to "changed"!
{"observedProperty":"changed"}
Custom data type
If changing the data type is an option, you could write a wrapping class which gets (de)serialized transparently. Something along the lines of the following might work.
#Serializable
class ClassWithMonitoredString(val monitoredProperty: MonitoredString) {
fun toJson(): String {
return Json.encodeToString(serializer(), this)
}
}
fun main() {
val monitoredString = obs("obsDefault") { before, after ->
println("""I changed from "$before" to "$after"!""")
}
val data = ClassWithMonitoredString(monitoredString)
println(data.toJson())
data.monitoredProperty.value = "obsChanged"
println(data.toJson())
}
Which yields the following result:
{"monitoredProperty":"obsDefault"}
I changed from "obsDefault" to "obsChanged"!
{"monitoredProperty":"obsChanged"}
You however lose information about which property changed, as you don't have easy access to the field name. Also you have to change your data structures, as mentioned above and might not be desirable or even possible. In addition, this work only for Strings for now, even though one might make it more generic though.
Also, this requires a lot of boilerplate to start with. On the call site however, you just have to wrap the actual value in an call to obs.
I used the following boilerplate to get it to work.
typealias OnChange = (before: String, after: String) -> Unit
#Serializable(with = MonitoredStringSerializer::class)
class MonitoredString(initialValue: String, var onChange: OnChange?) {
var value: String = initialValue
set(value) {
onChange?.invoke(field, value)
field = value
}
}
fun obs(value: String, onChange: OnChange? = null) = MonitoredString(value, onChange)
object MonitoredStringSerializer : KSerializer<MonitoredString> {
override val descriptor: SerialDescriptor = PrimitiveSerialDescriptor("MonitoredString", PrimitiveKind.STRING)
override fun serialize(encoder: Encoder, value: MonitoredString) {
encoder.encodeString(value.value)
}
override fun deserialize(decoder: Decoder): MonitoredString {
return MonitoredString(decoder.decodeString(), null)
}
}
Trying to implement a custom JSONB binding that maps to an object containing a map. Generated code throws a None of the following functions can be called with the arguments supplied error caused by the following line:
val SOME_FIELD: TableField<SomeRecord, Jsonb?> = createField(DSL.name("meta"), SQLDataType.JSONB.nullable(false).defaultValue(DSL.field("'{}'::jsonb", SQLDataType.JSONB)), this, "", JsonbBinding())
Here's my configuration:
class JsonbBinding : Binding<Any, Jsonb> {
private val mapper = ObjectMapper()
override fun converter(): Converter<Any, Jsonb> {
return object : Converter<Any, Jsonb> {
override fun from(dbObject: Any?): Jsonb {
if (dbObject == null) return Jsonb()
val props = mapper.readValue<MutableMap<String, Any>>(dbObject.toString())
return Jsonb(props)
}
override fun to(userObject: Jsonb?): Any? {
return mapper.writeValueAsString(userObject)
}
override fun fromType(): Class<Any> {
return Any::class.java
}
override fun toType(): Class<Jsonb> {
return Jsonb::class.java
}
}
}
override fun sql(ctx: BindingSQLContext<Jsonb>) {
ctx.render()?.let {
if (it.paramType() == ParamType.INLINED) {
it.visit(
DSL.inline(ctx.convert(converter()).value())
).sql("::jsonb")
} else {
it.sql("?::jsonb")
}
}
}
override fun register(ctx: BindingRegisterContext<Jsonb>) {
ctx.statement().registerOutParameter(ctx.index(), Types.VARCHAR)
}
override fun set(ctx: BindingSetStatementContext<Jsonb>) {
ctx.statement().setString(
ctx.index(),
ctx.convert(converter()).value()?.toString()
)
}
override fun set(ctx: BindingSetSQLOutputContext<Jsonb>) {
throw SQLFeatureNotSupportedException()
}
override fun get(ctx: BindingGetResultSetContext<Jsonb>) {
ctx.convert(converter()).value(ctx.resultSet().getString(ctx.index()))
}
override fun get(ctx: BindingGetStatementContext<Jsonb>) {
ctx.convert(converter()).value(ctx.statement().getString(ctx.index()))
}
override fun get(ctx: BindingGetSQLInputContext<Jsonb>) {
throw SQLFeatureNotSupportedException()
}
}
<forcedType>
<userType>org.example.Jsonb</userType>
<binding>org.example.JsonbBinding</binding>
<includeExpression>.*</includeExpression>
<includeTypes>jsonb</includeTypes>
</forcedType>
Also, it seems like the line causing problems is mapping database data to JOOQ's default JSONB object. Is that what's causing the issue? Is there anything I may want to do about it? Is there some other way of doing mapping database JSONB data to a map by JOOQ?
I think you're confusing the type variables on Binding<T, U> here:
T is the database / JDBC type (in this case org.jooq.JSONB)
U is the user type (in this case Any)
You have to implement the binding the other way round: Binding<JSONB?, Any?>. Since jOOQ already takes care of properly binding the JSONB type to JDBC, you can probably do with your Converter<JSONB?, Any?> implementation alone, and attach that to your generated code instead:
class JsonbConverter : Converter<JSONB?, Any?> { ... }
Also, you don't have to use your own Jsonb type to wrap JSON data here.
Not sure if it is possible yet but for the life of me I cannot figure out how to serialize this.
sealed class ServiceResult<out T : Any> {
data class Success<out T : Any>(val data: T) : ServiceResult<T>()
data class Error(val exception: Exception) : ServiceResult<Nothing>()
}
Everything that is stuff into T is using #Serializable ex:
#Serializable
data class GalleryDTO(
override val id: Int,
override val dateCreated: Long,
override val dateUpdated: Long,
val name:String,
val description:String,
val photos:List<DTOMin>
) : DTO
As Animesh Sahu already mentioned there is an issue for this topic that is still open, but the solution using a surrogate suggested by Михаил Нафталь for serialization of Error can actually be used also to serialize the polymorphic ServiceResult, by creating a surrogate that mixes the fields of Success and Error. For the sake of simplicity in the example I only represent the exception message.
#Serializable(with = ServiceResultSerializer::class)
sealed class ServiceResult<out T : Any> {
data class Success<out T : Any>(val data: T) : ServiceResult<T>()
data class Error(val exceptionMessage: String?) : ServiceResult<Nothing>()
}
class ServiceResultSerializer<T : Any>(
tSerializer: KSerializer<T>
) : KSerializer<ServiceResult<T>> {
#Serializable
#SerialName("ServiceResult")
data class ServiceResultSurrogate<T : Any>(
val type: Type,
// The annotation is not necessary, but it avoids serializing "data = null"
// for "Error" results.
#EncodeDefault(EncodeDefault.Mode.NEVER)
val data: T? = null,
#EncodeDefault(EncodeDefault.Mode.NEVER)
val exceptionMessage: String? = null
) {
enum class Type { SUCCESS, ERROR }
}
private val surrogateSerializer = ServiceResultSurrogate.serializer(tSerializer)
override val descriptor: SerialDescriptor = surrogateSerializer.descriptor
override fun deserialize(decoder: Decoder): ServiceResult<T> {
val surrogate = surrogateSerializer.deserialize(decoder)
return when (surrogate.type) {
ServiceResultSurrogate.Type.SUCCESS ->
if (surrogate.data != null)
ServiceResult.Success(surrogate.data)
else
throw SerializationException("Missing data for successful result")
ServiceResultSurrogate.Type.ERROR ->
ServiceResult.Error(surrogate.exceptionMessage)
}
}
override fun serialize(encoder: Encoder, value: ServiceResult<T>) {
val surrogate = when (value) {
is ServiceResult.Error -> ServiceResultSurrogate(
ServiceResultSurrogate.Type.ERROR,
exceptionMessage = value.exceptionMessage
)
is ServiceResult.Success -> ServiceResultSurrogate(
ServiceResultSurrogate.Type.SUCCESS,
data = value.data
)
}
surrogateSerializer.serialize(encoder, surrogate)
}
}
This solution can also be easily extended to support nullable Ts. In this case when deserializing you will also have to check if null is a valid value for T (it can be done by checking descriptor.isNullable on tSerializer) and you will also have to cast data as T.
Polymorphic serialization will be a mess in this case (you will have to manually register all possible types passed as a generic parameter to ServiceResult<T>), and will have several limitations (it would be impossible to register primitive types (including Nothing and String) as generic parameters, for instance).
If you only need serialization (aka encoding), I'd recommend to serialize both subtypes independently (for convenience, wrap subtype determination into auxilary function):
inline fun <reified T : Any> serializeServiceResult(x: ServiceResult<T>) = when (x) {
is ServiceResult.Success -> Json.encodeToString(x)
is ServiceResult.Error -> Json.encodeToString(x)
}
To serialize ServiceResult.Success you need just to mark it with #Serializable annotation. The tricky part here is serialization of ServiceResult.Error, or more precisely, serialization of its exception: Exception field. I'd suggest to serialize only its message (via surrogate):
sealed class ServiceResult<out T : Any> {
#Serializable
data class Success<out T : Any>(val data: T) : ServiceResult<T>()
#Serializable(with = ErrorSerializer::class)
data class Error(val exception: Exception) : ServiceResult<Nothing>()
}
#Serializable
private data class ErrorSurrogate(val error: String)
class ErrorSerializer : KSerializer<ServiceResult.Error> {
override val descriptor: SerialDescriptor = ErrorSurrogate.serializer().descriptor
override fun deserialize(decoder: Decoder): ServiceResult.Error {
val surrogate = decoder.decodeSerializableValue(ErrorSurrogate.serializer())
return ServiceResult.Error(Exception(surrogate.error))
}
override fun serialize(encoder: Encoder, value: ServiceResult.Error) {
val surrogate = ErrorSurrogate(value.exception.toString())
encoder.encodeSerializableValue(ErrorSurrogate.serializer(), surrogate)
}
}
I was coding on Java for quite a long time and trying to migrate to Kotlin. I'm confused with Generics in Kotlin a bit...
I have a DelegateManager class. It should consume only subtypes of IViewData
class DelegateManager<T : IViewData> {
private val delegates: MutableList<AdapterDelegate<T>> = mutableListOf()
fun addDelegate(adapterDelegate: AdapterDelegate<T>) {
delegates.add(adapterDelegate)
}
...
}
Inside TrackListAdapter I want to add a delegate. As you might have seen it's AdapterDelegate<TrackViewData> and TrackViewData is a subtype of IViewData So it should work but it shows error inside init block of TrackListAdapter
class TrackListAdapter : BaseListAdapter<IViewData>() {
init {
delegateManager.addDelegate(TrackViewDelegate()) // error: Type mismatch -> Required: AdapterDelegate<IViewData>, Found: TrackViewDelegate
}
}
class TrackViewDelegate : AdapterDelegate<TrackViewData>() {
override fun onCreateViewHolder(parent: ViewGroup): ListViewHolder<TrackViewData> {
val itemView = LayoutInflater.from(parent.context).inflate(R.layout.track_item, parent, false)
return TrackViewHolder(itemView)
}
override fun isDelegateForDataType(data: IViewData) = data is TrackViewData
}
How to deal with it? How to extend the generic parameter correctly?
I have a Kotlin object that I am trying serialize with Gson. A member that is setup as a delegate does not get serialized. The delegation works if I call it directly, as does the onChange callback, but Gson just ignores it.
Is there any way to get Gson to serialize this without writing a custom serializer?
Here is a simplified example of what I'm trying to do:
class MyDelegate() {
fun getProperty(): String {
return "myDelegate Property"
}
fun observableDelegate(onChange: () -> Unit): ReadWriteProperty<Any?, String> {
return object: ReadWriteProperty<Any?, String> {
override fun getValue(thisRef: Any?, property: KProperty<*>): String {
return getProperty()
}
override fun setValue(thisRef: Any?, property: KProperty<*>, value: String) {
TODO("not implemented")
}
}
}
}
class MyTest(delegate: MyDelegate, val property0: String = "property0" ) {
val property1 = "property1"
var property2 = "property2"
var property3: String by delegate.observableDelegate {
// onChange called
}
}
Testing it with:
#Test
fun testDelegate() {
val t1 = MyTest(MyDelegate())
val s1 = Gson().toJson(t1)
Assert.fail(s1)
}
Output:
{"property1":"property1","property2":"property2","property0":"property0"}
The property3 variable is not field backed. Thus Gson doesn't consider it as field in the Json serialization.
The GsonDesignDocument states for properties as such
Some Json libraries use the getters of a type to deduce the Json elements. We chose to use all fields (up the inheritance hierarchy) that are not transient, static, or synthetic. We did this because not all classes are written with suitably named getters. Moreover, getXXX or isXXX might be semantic rather than indicating properties.
So you might have to implement a custom (de)serializer for your needs.