Disclaimer: I'm a beginner with rust!
Lets assume I have the following to structs and I want the default json serialization from the derived Serialize traits
#[derive(Serialize)]
struct Address {
zip:usize,
street: String,
}
#[derive(Serialize)]
struct Person{
name: String,
age: usize,
address: Address
}
In addition, I have implemented a custom serde::Serializer for a legacy format which takes any Serialize and internally forwards to the serialize methods when it encounters nested structs, enums, etc.
struct MySerializer {...}
impl Serializer for MySerializer {...}
Further, I defined a custom trait MyFormat that extends the Serialize trait and is supposed to work as a tag or something, i.e. if MySerializer encounters something that implements MyFormat, it should not use Serialize, but MyFormat instead.
trait MyFormat: Serialize {
/// by default just use the normal Serialize trait method.
fn serialize<S>(&self, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
Serialize::serialize(&self, serializer)
}
}
impl MyFormat for Address {
fn serialize<S>(&self, serializer: S) -> Result<<S as Serializer>::Ok, <S as Serializer>::Error>
where
S: Serializer,
{
// this can actually look much more complicated, e.g. some additional mapping will be done here.
serializer.serialize_some(&self.zip)?;
serializer.serialize_some(&"some extra field only present in this format")?;
serializer.serialize_some(&self.street)?
}
}
impl MyFormat for Person {}
Now the problem is, when I use the custom serializer, I can, on the most outer level control whether to use the derived Serialize, however, the moment I descent into the structure, the Serializer trait expects Serialize types in the trait method specification which I cannot narrow down to MyFormat, as that would violate the trait.
In the above example name and age are ok, as the format for primitive types is identical, however, the nested Address might require to "inject" some additional fields if serialized to MyFormat.
I assume I just miss some paradigm that lets me do that with traits in rust. In C++ I could use template specialization or SFINAE. Is there anything in rust that can solve this kind of problem?
Kind regards,
Marti
Related
In my Jetpack compose project, I have a field that can contain different types of serializable data.
val httpBody : Serializable?
get() = when(this) {
is InitiatePayment -> this.pendingTransaction //data class annotated with #Serializable
is VerifyPaymentConfirmation -> this.transactionId //Normal String
is ValidateCart -> this.cartItems //Another data class with #Serializable
else -> null
}
Any? Type works. But when using Serializable?,
Gives the error,
Type mismatch.
Required:
Serializable?
Found:
PendingTransactionDTO
In Swift, I could just do
var httpBody : Codable? {
switch self {
case .initiatePayment(let pendingTransaction):
return pendingTransaction //Codable struct
case .verifyStoreCode(let storeCode):
return storeCode //String
case .verifyPaymentConfirmation(let transactionId):
return transactionId //String
default:
return nil
}
}
In this case, httpBody property accepts everything that implements the Codable interface/protocol.
In short, how to get the property type to accept any class/data class/object that is annotated with #Serializable, plus primitives like String and Int, which I suppose are by default, serializable in Kotlin.
Any tips? Or is it even possible to do this in Kotlin?
Are you using the #Serializable annotation from the kotlinx.serialization package? That's just to mark the class for processing by that library - it's unrelated to the Serializable interface in Java which is the type you're using here. That annotation isn't part of the type system, so you can't use it for this kind of thing.
Depending on how you're actually doing the serialisation, it might be enough for you to add the Serializable interface type to your class definitions - like the docs for the interface say, it doesn't actually require any methods to be overridden. But if you want to, say, throw it into a Bundle with putSerializable, you'll probably have more work to do.
Serializable is an annotation, not a type that a class inherits (unless you're talking about Java's unrelated Serializable interface), so this is not possible. Notably, the various Kotlinx Serialization libraries' encodeTo... functions have no restriction on the type of the argument. There is no compile-time check to make sure it is Serializable. You might as well make your function return Any?. The Any? return type is an acceptable argument for encodeToString, et. al.
I need to set up a serialization/deserialization mechanism for a polymorphic class hierarchy that also includes primitives and nulls. There are container classes containing collections with polymorphic objects, primitives, and nulls. And, the subclasses for these objects are spread across modules (therefore sealed is not an option).
I have been reading through the kotlinx.serialization polymorphism docs trying to come up with a solution. I've been able to make some incremental progress by working through that tutorial but I seem to still be hitting a wall when I try to put everything together.
The code I am posting here is a minimal example that brings together everything I need. If I can get this example to work, that should cover everything I need for my real project. This example does run without error but introduces some unnecessary readability and efficiency issues.
All classes in my custom class hierarchy are serializable data classes. The outermost container object that needs to be serialized/deserialized is a map wrapper. This map has keys which are each an instance of one of these data classes. And the values of this map can be primitives, nulls, or instances of one of my data classes. I think my main challenge here is to include those primitives and nulls in my polymorphic serialization in a clean way.
The goal of my code below is to represent this problem in the simplest way possible and to serialize and deserialize one container object successfully.
There are two main issues in the code:
I've had to replace null with FakeNull. Without this, I get null cannot be cast to non-null type kotlin.Any. This will reduce the readability and simplicity of my code and I suspect it could decrease efficiency as well.
I've had to add StringClassSerializer and DoubleClassSerializer and wrapper classes. I would also need to add serializers like these for every primitive class. If I don't register these primitives as subclasses of Any, I get Class 'String' is not registered for polymorphic serialization in the scope of 'Any'.. And if I try to register them with their default serializers (like subclass(String::class, String.serializer())) I get Serializer for String of kind STRING cannot be serialized polymorphically with class discriminator.. The problem with using serializers like StringClassSerializer and wrappers like StringWrapper is that it removes the efficiency and readability benefits of using primitives.
The json comes out looking like:
{"type":"MapContainer","map":[{"type":"SubA","data":1.0},{"type":"StringWrapper","s":"valueA"},{"type":"SubB","data":2.0},{"type":"DoubleWrapper","d":2.0},{"type":"SubB","data":3.0},{"type":"SubA","data":1.0},{"type":"SubB","data":4.0},{"type":"matt.play.FakeNull"}]}
I don't like the way this looks. I want the nulls to simply be null and the primitives to simply be primitives.
import kotlinx.serialization.KSerializer
import kotlinx.serialization.PolymorphicSerializer
import kotlinx.serialization.SerialName
import kotlinx.serialization.Serializable
import kotlinx.serialization.descriptors.buildClassSerialDescriptor
import kotlinx.serialization.encoding.Decoder
import kotlinx.serialization.encoding.Encoder
import kotlinx.serialization.json.Json
import kotlinx.serialization.modules.SerializersModule
import kotlinx.serialization.modules.polymorphic
import kotlinx.serialization.modules.subclass
import kotlin.collections.set
#Serializable
abstract class SuperClass
#Serializable
#SerialName("SubA")
data class SubA(val data: Double): SuperClass()
#Serializable
#SerialName("SubB")
data class SubB(val data: Double): SuperClass()
#Serializable
#SerialName("MapContainer")
data class MapContainer<K: SuperClass, V>(val map: Map<K, V>): Map<K, V> by map
#Serializable
#SerialName("StringWrapper")
data class StringWrapper(val s: String)
#Serializable
#SerialName("DoubleWrapper")
data class DoubleWrapper(val d: Double)
object StringClassSerializer: KSerializer<String> {
override val descriptor = buildClassSerialDescriptor("string")
override fun deserialize(decoder: Decoder) = decoder.decodeSerializableValue(StringWrapper.serializer()).s
override fun serialize(encoder: Encoder, value: String) =
encoder.encodeSerializableValue(StringWrapper.serializer(), StringWrapper(value))
}
object DoubleClassSerializer: KSerializer<Double> {
override val descriptor = buildClassSerialDescriptor("double")
override fun deserialize(decoder: Decoder) = decoder.decodeSerializableValue(DoubleWrapper.serializer()).d
override fun serialize(encoder: Encoder, value: Double) =
encoder.encodeSerializableValue(DoubleWrapper.serializer(), DoubleWrapper(value))
}
#Serializable
object FakeNull
fun main() {
val theMap = mutableMapOf<SuperClass, Any?>()
theMap[SubA(1.0)] = "valueA"
theMap[SubB(2.0)] = 2.0
theMap[SubB(3.0)] = SubA(1.0)
theMap[SubB(4.0)] = FakeNull /*wish I could make this just `null`*/
val theMapContainer = MapContainer(theMap)
val format = Json {
allowStructuredMapKeys = true
ignoreUnknownKeys = true
serializersModule = SerializersModule {
polymorphic(SuperClass::class) {
subclass(SubA::class)
subclass(SubB::class)
}
polymorphic(Any::class) {
/*I wish I could remove all of this primitive wrapper stuff*/
default {
when (it) {
StringWrapper::class.simpleName -> StringClassSerializer
DoubleWrapper::class.simpleName -> DoubleClassSerializer
else -> throw RuntimeException("unknown type: ${it}?")
}
}
subclass(String::class, StringClassSerializer)
subclass(Double::class, DoubleClassSerializer)
subclass(SubA::class)
subclass(SubB::class)
subclass(FakeNull::class)
}
polymorphic(
MapContainer::class, MapContainer::class, actualSerializer = MapContainer.serializer(
PolymorphicSerializer(SuperClass::class),
PolymorphicSerializer(Any::class)
) as KSerializer<MapContainer<*, *>>
)
}
}
val encoded = format.encodeToString(PolymorphicSerializer(MapContainer::class), theMapContainer)
println("\n\n${encoded}\n\n")
val decoded = format.decodeFromString(PolymorphicSerializer(MapContainer::class), encoded)
if (theMapContainer != decoded) {
throw RuntimeException("the decoded object is not the same as the original")
} else {
println("success")
}
}
Primitives (such as strings, numbers, and enums) by default are serialized as JSON primitives (e.g., "answer" or 42), not JSON objects ({ ... }). This is why they don't support polymorphic serialization; there is no "space" to place the type information in (the class discriminator).
There is no JSON object to place the class discriminator in, e.g., {"type": "fully.qualified.Name"} by default.
But, kotlinx serialization does allow you to write custom serializers, which allows you to work around this. I wrote a custom serializer for enums since I wanted to register enums as concrete types in polymophic serialization. It sounds like you should be able to do something similar. (Disclosure: I only read your problem description in detail; not your ongoing attempts/solution.)
A serializer which supports registering [Enum]s as subclasses in polymorphic serialization when class discriminators are used.
When class discriminators are used, an enum is not encoded as a structure which the class discriminator can be added to.
An exception is thrown when initializing [Json]: " "Serializer for of kind ENUM cannot be serialized polymorphically with class discriminator."
This serializer encodes the enum as a structure with a single value holding the enum value.
Use this serializer to register the enum in the serializers module, e.g.:
subclass( <enum>::class, PolymorphicEnumSerializer( <enum>.serializer() )
This custom serializer can possibly be generalized to any primitive type and thus support your use case.
I already made calculator that can compute with integers and real numbers(i made it with go).
Then I want to make it possible to calculate complex and rational numbers by adding those modules.
(it can also calculate when types are mixed)
It can be easy if I check types of operands every time(runtime) and take care of each case, but I want to solve it with dynamic binding. Guys can you tell me the idea of how to solve this problem
I think by dynamic typing, you're probably referring to how in eg C++ and Java, dynamic binding is essentially having reference to a base class that can point to a derived class (because the derived class "is a" base class).
One might say that the base class defines the interface behind which derived classes morph. If the derived classes replace methods of the base class, a reference to the base class can still access those methods in the derived class.
Base class can define some methods to provide some "base" functionality to its derived classes. If those classes don't redefine the method, the base class's method can be called.
In go, both these concepts exist, but they're quite different.
Go has an explicit interface keyword that defines method signatures but no methods. Any value implicitly satisfies that interface if it has methods of the same name with the same signature.
type LivingBeing interface {
TakeInEnergy()
ExpelWaste()
}
The interface type becomes a valid type in the code. We can pass an interface to a function, and without knowing the type satisfying that interface, can call its methods:
func DoLife(being LivingBeing) {
being.TakeInEnergy()
being.ExpelWaste()
}
This is valid code, but not complete code. Unlike with base classes in other languages, interfaces cannot define functions, only their signatures. It is purely and only an interface definition. We have to define the types that satisfy the interface separately from the interface itself.
type Organism struct{}
func (o *Organism) TakeInEnergy() {}
func (o *Organism) ExpelWaste() {}
We now have a type organism that satisfies LivingBeing. It is something like a base class, but if we wanted to build on it, we can't use subclassing because Go doesn't implement it. But Go does provide something similar called embedding types.
In this example I'll define a new organism, Animal, that draws ExpelWaste() from LivingBeing, but defines its own TakeInEnergy():
type Animal struct {
Organism // the syntax for an embedded type: type but no field name
}
func (a *Animal) TakeInEnergy() {
fmt.Printf("I am an animal")
}
What isn't obvious from that code is that because Animal's Organism is not a named field, its fields and methods are accessible directly from Animal. It's almost as if Animal "is an" organism.
However it is *not * an organism. It is a different type, and it would be more accurate to think of object embedding as syntactic sugar to automatically promote Organism's fields and methods to Animal.
Since go is statically and explicitly typed, DoLife cannot take an Organism and then be passed an Animal: it doesn't have the same type:
/* This does not work. Animal embeds organism, but *is not* an organism */
func DoLife(being *Organism) {
being.TakeInEnergy()
being.ExpelWaste()
}
func main() {
var a = &Animal{Organism{}}
DoLife(&Animal{})
}
cannot use &Animal{} (type *Animal) as type *Organism in argument to DoLife
That's why interface exists - so that both Organism and Animal (and indeed, even Plant or ChemotrophBacteria) can be passed as a LivingBeing.
Putting it all together, here's the code I've been working from:
package main
import "fmt"
type LivingBeing interface {
TakeInEnergy()
ExpelWaste()
}
type Organism struct{}
func (o *Organism) TakeInEnergy() {
}
func (o *Organism) ExpelWaste() {}
type Animal struct {
Organism
}
func DoLife(being LivingBeing) {
being.TakeInEnergy()
being.ExpelWaste()
}
func (a *Animal) TakeInEnergy() {
fmt.Printf("I am an animal")
}
func main() {
var a = &Animal{Organism{}}
DoLife(a)
}
There are a few caveats:
Syntactically, If you want to declare an embedded literal you must explicitly provide its type. In my example Organism has no fields so there's nothing to declare, but I still left the explicit type in there to point you in the right direction.
DoLife can call the right TakeInEnergy on a LivingBeing, but Organism's methods cannot. They see only the embedded Organism.
func (o *Organism) ExpelWaste() {
o.getWaste() //this will always be Organism's getWaste, never Animal's
}
func (o *Organism)getWaste() {}
func (a *Animal)getWaste() {
fmt.Println("Animal waste")
}
Simlarly if you pass only the embedded part, then it's going to call its own TakeInEnergy(), not that of Animal; there's no Animal left!
func main() {
var a = &Animal{Organism{}}
DoLife(&a.Organism)
}
In summary,
Define explict interface and use that type wherever you want "polymorphic" behavior
Define base types and embed them in other types to share base functionality.
Don't expect the "base" type to ever "bind" to functions from the "derived" type.
The pattern of having an object-safe trait Foo and a (potentially unsafe) extension trait FooExt implemented for all instances of Foo seems to become standard now.
https://github.com/rust-lang/rfcs/pull/445
This is a problem for me in the case of Iterator<A>, as I have a library that overrides the default method IteratorExt#last() of the old iterator trait (the underlying library has an efficient implementation of last()). This in now impossible, because for any A, there will always be a conflicting trait implementation of IteratorExt, the one that libcore already provides for all Iterator<A>.
iterator.rs:301:1: 306:2 error: conflicting implementations for trait `core::iter::IteratorExt` [E0119]
iterator.rs:301 impl<'a, K: Key> iter::IteratorExt<Vec<u8>> for ValueIterator<'a,K,Vec<u8>> {
iterator.rs:302 fn last(&mut self) -> Option<Vec<u8>> {
iterator.rs:303 self.seek_last();
iterator.rs:304 Some(self.value())
iterator.rs:305 }
iterator.rs:306 }
...
Now, as far as I see, I have two options:
have my own trait and my own last() implementation. That would mean it conflicts if IteratorExt is imported unless carefully used. This also has the danger accidentally using an inefficient version of last() if the version from IteratorExt is used. I'd loose convenient access to IteratorExt.
have my own trait and name the method differently (seek_last()). Disadvantage: I ask the user to learn vocabulary and to always favor my method over that provided by IteratorExt. Same problem: I'd like to avoid accidental usage of last().
Is there any other, better, solution I am missing?
As of rustc 0.13.0-nightly (8bca470c5 2014-12-08 00:12:30 +0000) defining last() as an inherent method on your type should work.
#[deriving(Copy)]
struct Foo<T> {t: T}
impl<T> Iterator<T> for Foo<T> {
fn next(&mut self) -> Option<T> { None }
}
// this does not work
// error: conflicting implementations for trait `core::iter::IteratorExt` [E0119]
// impl<T> IteratorExt<T> for Foo<T> {
// fn last(mut self) -> Option<T> { None }
//}
// but this currently does
impl<T> Foo<T> {
fn last(mut self) -> Option<T> { Some(self.t) }
}
fn main() {
let mut t = Foo{ t: 3u };
println!("{}", t.next())
println!("{}", t.last()) // last has been "shadowed" by our impl
println!("{}", t.nth(3)) // other IteratorExt methods are still available
}
Since you're not supposed to use Extension traits as generic bounds (but just to provide additional methods), this should theoretically work for your scenario, as you can have your own type and its impl in the same crate.
Users of your type will use the inherent last method instead of the one on IteratorExt but still be able to use the other methods on IteratorExt.
last should be moved to Iterator, rather than IteratorExt.
IteratorExt is needed when using Box<Iterator> objects, to allow calling generic methods on them (e.g. map), because you can't put a generic method in a vtable. However, last isn't generic, so it can be put in Iterator.
I want to deserialize a configuration structure like this:
#[deriving(Clone, PartialEq, Decodable, Show)]
pub struct Config {
network1: Network,
network2: Network
}
#[deriving(Clone, PartialEq, Decodable, Show)]
pub struct Network {
interface: Option<String>,
prefer: Option<AddressPreference>,
address: Option<String>,
port: u16
}
#[deriving(Clone, PartialEq, Show)]
pub enum AddressPreference {
IPv4, IPv6, Any
}
It forms a tree of structs and enums. But the decoder I want to use does not support enums, so I thought that I will be able to represent the enum as a string by implementing Decodable for the enum which matches on strings. But I don't know how to do it properly.
The main problem is that I don't know how to write Decodable implementation which handles errors correctly. If I write
impl<E, D: Decoder<E>> Decodable<D, E> for AddressPreference { ... }
the only way I can indicate parsing error is fail!() - I don't know actual error type, so I can't create its instance. This is unacceptable.
Buf if I write
impl Decodable<toml::Decoder, toml::Error> for AddressPreference { ... }
that is, binding Decodable instance to concrete decoder type (which is fine for me), I lose automatic deriving of outer structures, because they are implemented for generic decoder and error:
src/sfc/config.rs:11:30: 11:39 error: expected serialize::serialize::Decodable<__D,__E>, but found serialize::serialize::Decodable<rust-toml::Decoder,rust-toml::Error> (expected type parameter but found struct rust-toml::Decoder)
src/sfc/config.rs:11 #[deriving(Clone, PartialEq, Decodable, Show)]
What should I do in this case? I guess, the proper answer in this particular case is submitting a pull request to the library enabling support for enums serialization, but I think that the question is more broad. What it the library supported enums, but I wanted to customize decoding nonetheless?