This question already has answers here:
How to fix lifetime error when function returns a serde Deserialize type?
(2 answers)
Closed 5 months ago.
struct S1<T>
{
pub data: T
}
impl<'a, T> S1<T> where T: Deserialize<'a> {
fn update() {
let str_json = String::new(); // let's assume read from file
self.data = serve_json::from_str(&str_json).unwrap();
}
}
this function will produce lifetime related issue due to incorrect specifier. What is the best way to overcome the problem?
Deserialize does not necessarily create a copy of the data in str_json. There might be large text sections in str_json that get referenced in the final T instead of being copied into it. That means, the lifetime of the final self.data is identical to the one from str_json, as it might potentially borrow data from str_json. That is also the reason why you have to specify the lifetime 'a in the first place.
Now in your case, the problem is that self.data gets stored in self, while str_json gets dropped at the end of update(). But self.data might still reference data from str_json, so the compiler rightfully complains.
I think what you actually wanted to achieve is that the data from str_json gets copied into self.data instead of just being referenced. For that behaviour, you have the wrong trait, and what you actually want is DeserializeOwned:
use serde::de::DeserializeOwned;
struct S1<T> {
pub data: T,
}
impl<T> S1<T>
where
T: DeserializeOwned,
{
fn update(&mut self) {
let str_json = String::new(); // let's assume read from file
self.data = serde_json::from_str(&str_json).unwrap();
}
}
Related
I have a data format using custom enum of values. From the database I receive a Vec<MyVal>. I want to convert this to a struct (and fail if it doesn't work). I want to use serde because after processing I want to return the API response as a json, and serde makes this super easy.
Playground link for the example
enum MyVal {
Bool(bool),
Text(String)
}
#[derive(Serialize, Deserialize)]
struct User {
name: String,
registered: bool
}
The challenge is with converting the data format into the serde data model. For this I can implement a Deserializer and implement the visit_seq method i.e. visit the Vec<MyVal> as if it were a sequence and return the values one by one. The visitor for User can consume the visited values to build the struct User.
However I'm not able to figure out how to convert the Vec into something visitor_seq can consume. Here's some sample code.
struct MyWrapper(Vec<MyVal>);
impl<'de> Deserializer<'de> for MyWrapper {
type Error = de::value::Error;
// skip unncessary
fn deserialize_seq<V>(self, visitor: V) -> Result<V::Value, Self::Error>
where
V: serde::de::Visitor<'de>,
{
let convert_myval = move |myval: &MyVal| match myval {
MyVal::Bool(b) => visitor.visit_bool(*b),
MyVal::Text(s) => visitor.visit_string(s.clone())
};
// now I have a vec of serde values
let rec_vec: Vec<Result<V::Value, Self::Error>> =
self.0.iter().map(|myval| convert_myval(myval)).collect();
// giving error!!
visitor.visit_seq(rec_vec.into_iter())
}
}
The error is
92 | visitor.visit_seq(rec_vec.into_iter())
| --------- ^^^^^^^^^^^^^^^^^^^ the trait `SeqAccess<'de>` is not implemented for `std::vec::IntoIter<Result<<V as Visitor<'de>>::Value, _::_serde::de::value::Error>>`
| |
| required by a bound introduced by this call
So I looked into SeqAccess and it has an implementor that requires that whatever is passed to it implement the Iterator trait. But I thought I had that covered, because vec.into_iter returns an IntoIter, a consuming iterator which does implement the Iterator trait.
So I'm completely lost as to what's going wrong here. Surely there must be a way to visit a Vec<Result<Value, Error>> as a seq?
Preface: The question wants to treat a Rust data structure Vec<MyData> like a serialized piece of data (e.g.: like a JSON string) and allow deserializing that into any other Rust data structure that implements Deserialize. This is a quite unusual, but not without precedent. And since the MyVals are actually pieces of data with various types which get returned from a database access crate, this approach does make sense.
The main problem with the code in the question is that it tries to deserialize two different data structures (MyWrapper<Vec<MyVal>> and MyVal) with a single Deserializer. The obvious way out is to define a second struct MyValWrapper(MyVal) and implement Deserializer for it:
struct MyValWrapper(MyVal);
impl<'de> Deserializer<'de> for MyValWrapper {
type Error = de::value::Error;
fn deserialize_any<V>(self, visitor: V) -> Result<V::Value, Self::Error>
where
V: de::Visitor<'de>,
{
match self.0 {
MyVal::Bool(b) => visitor.visit_bool(b),
MyVal::Text(s) => visitor.visit_string(s.clone()),
}
}
// Maybe you should call deserialize_any from all other deserialize_* functions to get a passable error message, e.g.:
forward_to_deserialize_any! {
bool i8 i16 i32 i64 i128 u8 u16 u32 u64 u128 f32 f64 char str string
bytes byte_buf option unit unit_struct newtype_struct seq tuple
tuple_struct map struct enum identifier ignored_any
}
// Though maybe you want to handle deserialize_enum differently...
}
MyValWrapper now can deserialize a MyVal. To use MyValWrapper to deserialize a Vec of MyVals, the serde::value::SeqDeserializer adapter is convenient, it can turn an iterator of (Into)Deserializer into a sequence Deserializer:
let data: Vec<MyData> = …;
// Vec deserialization snippet
let mut sd = SeqDeserializer::new(data.into_iter().map(|myval| MyValWrapper(myval)));
let res = visitor.visit_seq(&mut sd)?;
sd.end()?;
Ok(res)
For some reason, SeqDeserializer requires the iterator items to be IntoDeserializer, but no impl<T: IntoDeserializer> Deserializer for T exists, so we need to make sure that MyValWrapper is not only a Deserializer but trivially also an IntoDeserializer:
impl<'de> IntoDeserializer<'de> for MyValWrapper {
type Deserializer = MyValWrapper;
fn into_deserializer(self) -> Self::Deserializer {
self
}
}
Finally, you need to impl Deserializer for MyWrapper (you can use the "Vec deserialization snippet" for that) — if you do actually need Deserializer to be implemented, which I suspect you don't: the SeqDeserializer already implements Deserializer, and it is a wrapper struct (just as MyWrapper is a wrapper struct). Especially, if your final goal is having a function like
fn turn_bad_myvals_into_good_T<T: DeserializeOwned>(v: Vec<MyVal>) -> T {
T::deserialize(SeqDeserializer::new(
v.into_iter().map(|myval| MyValWrapper(myval)),
))
}
then you entirely don't need MyWrapper.
This question already has answers here:
How to get a reference to a concrete type from a trait object?
(2 answers)
Closed 4 years ago.
I have a collection of interfaces that are loaded dynamically from shared libraries. I want to be able to convert those downcasted interfaces to their original type (trait).
struct A {}
fn abstract_a<'l>() -> &'l Any { return &A{} }
trait TargetTrait { fn some_method(); }
impl TargetTrait for A { fn some_method() { println!("HELLO"); } }
fn main() {
let x: &Any = abstract_a();
let y: &TargetTrait = magic_conversion<&TargetTrait> (x);
}
// question: does 'magic_conversion'(or 'dynamic_cast') exist? what is it?
While loading these is not a problem, I have no idea how to get target functionality with such interface. In other words:
/* simplified for readability */
// this part is known
let some_lib = loadlib("path/to/lib.so")
let some_interface: &Any = some_lib.loadfunc<&Any>("constructor_func")()
/* loader does not know what target type constructor has, so it presumes 'Any' */
// the problem:
let dependent_class = Some(class)
dependent_class.graphics = dynamic_cast<IGraphics>(some_interface)
In this example, dependent_class uses an extern interface and does not care about handling libloading and all of that complicated stuff.
If there is another way to achieve my goal, I would also be very happy to see it, but the only solution I came up with is 'dynamic_cast'
I think what you're looking for is downcast_ref::<A>:
let y: &TargetTrait = Any::downcast_ref::<A>(x).expect("Expected an A");
You have to specify the concrete type A. Any trait objects don't hold any information about what traits the underlying type implements, so you can't "cross-cast" from &Any to &TargetTrait directly; you have to know the underlying type.
The expect will panic if downcast_ref returns None; if that's not what you want, you have to decide what you want to happen when x is not an A and match against the result of downcast_ref instead.
I'm trying to implement an iterator on my own struct. My general approach is by generating and storing an iterator the first time next is invoked, and then calling this iterator each time I need a value.
My minimal failing example looks like this, and the heart of it is:
if !self.vals.is_some() {
self.vals = Some(Box::new({
self.display.chars().filter(|&i| i == self.look_for)
}) as Box<std::iter::Iterator<Item = _>>);
}
My code fails to compile, producing the following message:
help: consider using an explicit lifetime parameter as shown: fn next(self: &'a mut Self) -> Option<<Self>::Item>
Following the advice doesn't help (just leads to more compile errors saying that my implementation is incompatible with the Iterator trait definition.
I'd appreciate help understanding what's going wrong and how I can fix it.
The problem is that the closure you pass to filter needs to borrow self, but you can't store a reference to self in the struct itself.
In this case, we can work around it by storing a copy of the value in the closure instead. This is done in two steps:
Assign self.look_for to a local variable, and use the local variable in the closure instead. This way, the closure is not tied to self.
Add move to the closure. The closure will thus capture the local variable by value.
Here's the final code:
impl<'a> Iterator for StatefulCounter<'a> {
type Item = bool;
fn next(&mut self) -> Option<Self::Item> {
if !self.vals.is_some() {
let look_for = self.look_for;
self.vals = Some(Box::new({
self.display.chars().filter(move |&i| i == look_for)
}));
}
if let &Some(v) = &self.vals.as_mut().unwrap().next() {
Some(expensive(v))
} else {
None
}
}
}
The explicit cast on the Box is not necessary, so I removed it.
I have a function that takes an iterator over references to structs. Sometimes I'm iterating over a vector, which works fine, but sometimes I create an iterator that produces new structs, and I'm having trouble figuring that one out. I get that when I create a value in a closure, it goes away when the closure does. Rust is always trying to move values out of things when I don't want it to; why doesn't it here?
struct Thing {
value: u32,
}
fn consume<'a, I: IntoIterator<Item = &'a Thing>>(things: I) {
for thing in things {
println!("{}", thing.value);
}
}
fn main() {
let solid = vec![Thing { value: 0 }];
let ephemeral = (1..5).map(|i| &Thing { value: i }); // Boxing Thing does not work either
consume(solid.iter());
consume(ephemeral);
}
But
error[E0515]: cannot return reference to temporary value
--> src/main.rs:13:36
|
13 | let ephemeral = (1..5).map(|i| &Thing { value: i }); // Boxing Thing does not work either
| ^------------------
| ||
| |temporary value created here
| returns a reference to data owned by the current function
I have the sense I either need to move the struct out of the closure and iterator, or store it somewhere. But Boxing the struct doesn't work and returning a struct rather than a pointer doesn't type check (and I can't find the opposite of .cloned()). What's the approach here?
Short answer: you can't.
Longer explanation:
Here is "an iterator that produces new structs":
let iterator_of_structs = (1..5).map(|value| Thing { value });
The main trick to figuring this out is to always ask "who owns the data?".
Each time we call next, the closure takes ownership of an integer (via value) and constructs a new Thing. The closure returns the Thing, transferring ownership to the code that called next.
While you are borrowing a value (a.k.a. taking a reference), the ownership of the value cannot change hands and the value must last longer than the borrow lasts.
Let's turn to the concept of an iterator of references and ask our question: "who owns the data?".
map(|value| &Thing { value })
Here, we create a Thing and take a reference to it. No variable owns the Thing, so the scope owns it and the value will be destroyed when the scope ends. The closure tries to return the reference, but that violates the axiom that borrowed items must outlive their borrows.
So, how do you fix it? The easiest thing is to change your function to be more accepting:
use std::borrow::Borrow;
struct Thing {
value: u32,
}
fn consume(things: impl IntoIterator<Item = impl Borrow<Thing>>) {
for thing in things {
let thing = thing.borrow();
println!("{}", thing.value);
}
}
fn main() {
let iterator_of_structs = (1..5).map(|value| Thing { value });
consume(iterator_of_structs);
let vector_of_structs: Vec<_> = (1..5).map(|value| Thing { value }).collect();
let iterator_of_references_to_structs = vector_of_structs.iter();
consume(iterator_of_references_to_structs);
}
Here, we accept any type which can be converted into an iterator of items that allow us to borrow a reference to a Thing. This is implemented for any item and any reference to an item.
An iterator of references allows the consumer to keep all the references that the iterator yielded, for as long as they want (at least while the iterator itself remains alive). Obviously to support that, all objects to which the iterator creates references need to be in memory at the same time. There is no way around this with the iterator protocol as-is. So your best course of action is to collect() the iterator into a vector and create an reference iterator from that (as you do with solid). Unfortunately this means losing the laziness.
There is an alternative iterator abstraction, called streaming iterator, which would support this. With streaming iterators, the consumer may only hold onto the reference until it gets the next one. I am not aware of any crates implementing this though, and it would be a completely different trait which no function using std::iter::Iterator supports. In many cases it may even be impossible to use streaming iterators, because the algorithm needs the freedom to reference several values at once.
I have a struct and I would like it to be initialised with some sensible default values.
Typically, the thing to do here is to use a constructor but since go isn't really OOP in the traditional sense these aren't true objects and it has no constructors.
I have noticed the init method but that is at the package level. Is there something else similar that can be used at the struct level?
If not what is the accepted best practice for this type of thing in Go?
There are some equivalents of constructors for when the zero values can't make sensible default values or for when some parameter is necessary for the struct initialization.
Supposing you have a struct like this :
type Thing struct {
Name string
Num int
}
then, if the zero values aren't fitting, you would typically construct an instance with a NewThing function returning a pointer :
func NewThing(someParameter string) *Thing {
p := new(Thing)
p.Name = someParameter
p.Num = 33 // <- a very sensible default value
return p
}
When your struct is simple enough, you can use this condensed construct :
func NewThing(someParameter string) *Thing {
return &Thing{someParameter, 33}
}
If you don't want to return a pointer, then a practice is to call the function makeThing instead of NewThing :
func makeThing(name string) Thing {
return Thing{name, 33}
}
Reference : Allocation with new in Effective Go.
There are actually two accepted best practices:
Make the zero value of your struct a sensible default. (While this looks strange to most people coming from "traditional" oop it often works and is really convenient).
Provide a function func New() YourTyp or if you have several such types in your package functions func NewYourType1() YourType1 and so on.
Document if a zero value of your type is usable or not (in which case it has to be set up by one of the New... functions. (For the "traditionalist" oops: Someone who does not read the documentation won't be able to use your types properly, even if he cannot create objects in undefined states.)
Go has objects. Objects can have constructors (although not automatic constructors). And finally, Go is an OOP language (data types have methods attached, but admittedly there are endless definitions of what OOP is.)
Nevertheless, the accepted best practice is to write zero or more constructors for your types.
As #dystroy posted his answer before I finished this answer, let me just add an alternative version of his example constructor, which I would probably write instead as:
func NewThing(someParameter string) *Thing {
return &Thing{someParameter, 33} // <- 33: a very sensible default value
}
The reason I want to show you this version is that pretty often "inline" literals can be used instead of a "constructor" call.
a := NewThing("foo")
b := &Thing{"foo", 33}
Now *a == *b.
There are no default constructors in Go, but you can declare methods for any type. You could make it a habit to declare a method called "Init". Not sure if how this relates to best practices, but it helps keep names short without loosing clarity.
package main
import "fmt"
type Thing struct {
Name string
Num int
}
func (t *Thing) Init(name string, num int) {
t.Name = name
t.Num = num
}
func main() {
t := new(Thing)
t.Init("Hello", 5)
fmt.Printf("%s: %d\n", t.Name, t.Num)
}
The result is:
Hello: 5
I like the explanation from this blog post:
The function New is a Go convention for packages that create a core type or different types for use by the application developer. Look at how New is defined and implemented in log.go, bufio.go and cypto.go:
log.go
// New creates a new Logger. The out variable sets the
// destination to which log data will be written.
// The prefix appears at the beginning of each generated log line.
// The flag argument defines the logging properties.
func New(out io.Writer, prefix string, flag int) * Logger {
return &Logger{out: out, prefix: prefix, flag: flag}
}
bufio.go
// NewReader returns a new Reader whose buffer has the default size.
func NewReader(rd io.Reader) * Reader {
return NewReaderSize(rd, defaultBufSize)
}
crypto.go
// New returns a new hash.Hash calculating the given hash function. New panics
// if the hash function is not linked into the binary.
func (h Hash) New() hash.Hash {
if h > 0 && h < maxHash {
f := hashes[h]
if f != nil {
return f()
}
}
panic("crypto: requested hash function is unavailable")
}
Since each package acts as a namespace, every package can have their own version of New. In bufio.go multiple types can be created, so there is no standalone New function. Here you will find functions like NewReader and NewWriter.
In Go, a constructor can be implemented using a function that returns a pointer to a modified structure.
type Colors struct {
R byte
G byte
B byte
}
// Constructor
func NewColors (r, g, b byte) *Colors {
return &Color{R:r, G:g, B:b}
}
For weak dependencies and better abstraction, the constructor does not return a pointer to a structure, but an interface that this structure implements.
type Painter interface {
paintMethod1() byte
paintMethod2(byte) byte
}
type Colors struct {
R byte
G byte
B byte
}
// Constructor return intreface
func NewColors(r, g, b byte) Painter {
return &Color{R: r, G: g, B: b}
}
func (c *Colors) paintMethod1() byte {
return c.R
}
func (c *Colors) paintMethod2(b byte) byte {
return c.B = b
}
another way is;
package person
type Person struct {
Name string
Old int
}
func New(name string, old int) *Person {
// set only specific field value with field key
return &Person{
Name: name,
}
}
If you want to force the factory function usage, name your struct (your class) with the first character in lowercase. Then, it won't be possible to instantiate directly the struct, the factory method will be required.
This visibility based on first character lower/upper case work also for struct field and for the function/method. If you don't want to allow external access, use lower case.
Golang is not OOP language in its official documents.
All fields of Golang struct has a determined value(not like c/c++), so constructor function is not so necessary as cpp.
If you need assign some fields some special values, use factory functions.
Golang's community suggest New.. pattern names.
I am new to go. I have a pattern taken from other languages, that have constructors. And will work in go.
Create an init method.
Make the init method an (object) once routine. It only runs the first time it is called (per object).
func (d *my_struct) Init (){
//once
if !d.is_inited {
d.is_inited = true
d.value1 = 7
d.value2 = 6
}
}
Call init at the top of every method of this class.
This pattern is also useful, when you need late initialisation (constructor is too early).
Advantages: it hides all the complexity in the class, clients don't need to do anything.
Disadvantages: you must remember to call Init at the top of every method of the class.