Imagine a tiny map that stores 3 values, the first two for known keys. I'd like to implement an iterator for this map, but I'm running into lifetime issues. What's the appropriate way to return a reference to the value from a generic associated function (K::zero() in the example below)?
FYI, I own the trait, so I tried changing it to the new RFC195 associated const, which didn't help.
I've boiled down my problem to the following code:
extern crate num;
use num::*;
pub struct TinyMap<K: Num, V> {
v0: Option<V>, // value for K::zero()
v1: Option<V>, // value for K::one()
k2: K, // arbitrary K
v2: Option<V>, // value for k2
}
pub struct Iter<'a, K: 'a + Num, V: 'a> {
k0: K,
v0: &'a Option<V>,
v1: &'a Option<V>,
k2: &'a K,
v2: &'a Option<V>,
}
impl<K: Num, V> TinyMap<K, V> {
pub fn iter(&self) -> Iter<K, V> {
Iter {
k0: K::zero(),
v0: &self.v0,
v1: &self.v1,
k2: &self.k2,
v2: &self.v2,
}
}
}
impl<'a, K: 'a + Num, V: 'a> Iterator for Iter<'a, K, V> {
type Item = (&'a K, &'a V);
fn next(&mut self) -> Option<(&'a K, &'a V)> {
if (*self.v0).is_some() {
// code removed that remembers we did this once.
return Some((&self.k0, ((*self.v0).as_ref()).unwrap()));
}
// if (*self.v1).is_some() {
// code removed that remembers we did this once.
// return Some((&K::one(), &((*self.v1).unwrap())));
// }
None
}
}
error[E0495]: cannot infer an appropriate lifetime for borrow expression due to conflicting requirements
--> src/lib.rs:38:26
|
38 | return Some((&self.k0, ((*self.v0).as_ref()).unwrap()));
| ^^^^^^^^
|
note: first, the lifetime cannot outlive the anonymous lifetime #1 defined on the method body at 35:5...
--> src/lib.rs:35:5
|
35 | / fn next(&mut self) -> Option<(&'a K, &'a V)> {
36 | | if (*self.v0).is_some() {
37 | | // code removed that remembers we did this once.
38 | | return Some((&self.k0, ((*self.v0).as_ref()).unwrap()));
... |
44 | | None
45 | | }
| |_____^
note: ...so that reference does not outlive borrowed content
--> src/lib.rs:38:26
|
38 | return Some((&self.k0, ((*self.v0).as_ref()).unwrap()));
| ^^^^^^^^
note: but, the lifetime must be valid for the lifetime 'a as defined on the impl at 32:6...
--> src/lib.rs:32:6
|
32 | impl<'a, K: 'a + Num, V: 'a> Iterator for Iter<'a, K, V> {
| ^^
= note: ...so that the expression is assignable:
expected std::option::Option<(&'a K, &'a V)>
found std::option::Option<(&K, &V)>
It's not possible to do that with the Iterator trait, because of the lifetime of the self reference (which is elided away in your code, but can be explicitly written like this):
type Item = (&'a K, &'a V);
fn next<'s>(&'s mut self) -> Self::Item;
Since 's doesn't appear in the function's return value (and can't appear in there, because Self::Item can't use type parameters of the function), the output is not allowed to hold a reference to any of the iterator's member variables.
That's the mechanics of the mistake, now here's the why part:
Consider a function that does include a reference to a member of self, with all the lifetimes set up correctly:
struct SomeMember;
struct SomeObject {
some_member: SomeMember,
}
impl SomeObject {
fn some_function<'s>(&'s mut self) -> &'s SomeMember {
&self.some_member
}
}
The same way you're trying to return &self.k, but without any other things going on, and with the lifetimes fixed so that it's allowed. However, if I then try to do this:
fn main() {
let mut some_object = SomeObject{some_member: SomeMember};
let _item_1 = some_object.some_function();
let _item_2 = some_object.some_function();
}
error[E0499]: cannot borrow `some_object` as mutable more than once at a time
--> src/main.rs:15:23
|
14 | let _item_1 = some_object.some_function();
| ----------- first mutable borrow occurs here
15 | let _item_2 = some_object.some_function();
| ^^^^^^^^^^^ second mutable borrow occurs here
16 | }
| - first borrow ends here
The second call wasn't allowed, because it borrows some_object twice, mutably, a classic Rust no-no! But if I had tried to implement an iterator with an Item type that borrowed the iterator itself, then Iterator::collect() would be impossible, because it tries to pull more than one item out at once!
So, no, an iterator can't return an item that borrows its contents. That's an explicit, and intentional, part of the trait contract for iterators.
The consensus appears to be that as of this time (Rust 1.29), the only sensible way is to put K::zero() inside TinyMap. Thanks to #SvenMarnach for confirming my suspicions.
Related
I have the following:
impl<'a, K: Hash + Eq, V> Index<K> for &'a LFUCache<K, V> {
type Output = V;
fn index(&self, index: K) -> &Self::Output {
self.get(index).unwrap()
}
}
This compiles fine.
Now when I do:
let mut lfu = LFUCache::new(2);
lfu.set(1, 1);
lfu[1] == 1;
I get an error:
cannot index into a value of type `LFUCache<{integer}, {integer}>`
--> src/lib.rs:154:9
|
154 | lfu[1] == 1;
| ^^^^^^
How do I fix this?
A number without any suffix in rust has no specific int type, (it could be any of i8,i16,i32,u8, etc) so the rust compiler can't infer which one you want your cache to hold. There are three ways to fix this:
Explicitly specify when constructing it: LFUCache::<i32,i32>::new(2)
Explicitly specifying the type of the binding. let mut lfu: LFUCache<i32,i32> =
Explicitly specify the type of int you are inserting with a suffix:lfu[1i32] = 1i32;
I believe option 2 is the most idiomatic in your example.
Fixed it by making it: impl<'a, K:Hash+Eq, V> Index<K> for LFUCache<K, V> {...}
I have a trait
trait B {
type Index: Sized + Copy;
fn bounds(&self) -> (Self::Index, Self::Index);
}
I want to get all the Indexes within bounds:
fn iterate<T: B>(it: &T) {
let (low, high) = it.bounds();
for i in low..high {}
}
This won't work since there's no constraint that the type T can be "ranged" over, and the compiler says as much:
error[E0277]: the trait bound `<T as B>::Index: std::iter::Step` is not satisfied
--> src/main.rs:8:5
|
8 | for i in low..high {}
| ^^^^^^^^^^^^^^^^^^^^^ the trait `std::iter::Step` is not implemented for `<T as B>::Index`
|
= help: consider adding a `where <T as B>::Index: std::iter::Step` bound
= note: required because of the requirements on the impl of `std::iter::Iterator` for `std::ops::Range<<T as B>::Index>`
I tried adding the Step bound to Index
use std::iter::Step;
trait B {
type Index: Sized + Copy + Step;
fn bounds(&self) -> (Self::Index, Self::Index);
}
but apparently it isn't stable:
error: use of unstable library feature 'step_trait': likely to be replaced by finer-grained traits (see issue #42168)
--> src/main.rs:1:5
|
1 | use std::iter::Step;
| ^^^^^^^^^^^^^^^
error: use of unstable library feature 'step_trait': likely to be replaced by finer-grained traits (see issue #42168)
--> src/main.rs:4:32
|
4 | type Index: Sized + Copy + Step;
| ^^^^
Am I missing something or is it just not possible to do so right now?
If you want to require that a Range<T> can be iterated over, just use that as your trait bound:
trait Bounded {
type Index: Sized + Copy;
fn bounds(&self) -> (Self::Index, Self::Index);
}
fn iterate<T>(it: &T)
where
T: Bounded,
std::ops::Range<T::Index>: IntoIterator,
{
let (low, high) = it.bounds();
for i in low..high {}
}
fn main() {}
To do this kind of thing generically the num crate is helpful.
extern crate num;
use num::{Num, One};
use std::fmt::Debug;
fn iterate<T>(low: T, high: T)
where
T: Num + One + PartialOrd + Copy + Clone + Debug,
{
let one = T::one();
let mut i = low;
loop {
if i > high {
break;
}
println!("{:?}", i);
i = i + one;
}
}
fn main() {
iterate(0i32, 10i32);
iterate(5u8, 7u8);
iterate(0f64, 10f64);
}
This question already has answers here:
What is the correct way to return an Iterator (or any other trait)?
(2 answers)
Closed 5 years ago.
I'm trying to write a function that will encapsulate a series of chained iterator method calls (.lines().map(...).filter(...)) which I currently have duplicated. I can't figure out the type signatures to get this to compile. If this is impossible or highly unidiomatic for Rust, I'm open to suggestions for an idiomatic approach.
use std::fs;
use std::io;
use std::io::prelude::*;
use std::iter;
const WORDS_PATH: &str = "/usr/share/dict/words";
fn is_short(word: &String) -> bool {
word.len() < 7
}
fn unwrap(result: Result<String, io::Error>) -> String {
result.unwrap()
}
fn main_works_but_code_dupe() {
let file = fs::File::open(WORDS_PATH).unwrap();
let reader = io::BufReader::new(&file);
let count = reader.lines().map(unwrap).filter(is_short).count();
println!("{:?}", count);
let mut reader = io::BufReader::new(&file);
reader.seek(io::SeekFrom::Start(0));
let sample_size = (0.05 * count as f32) as usize; // 5% sample
// This chain of iterator logic is duplicated
for line in reader.lines().map(unwrap).filter(is_short).take(sample_size) {
println!("{}", line);
}
}
fn short_lines<'a, T>
(reader: &'a T)
-> iter::Filter<std::iter::Map<std::io::Lines<T>, &FnMut(&str, bool)>, &FnMut(&str, bool)>
where T: io::BufRead
{
reader.lines().map(unwrap).filter(is_short)
}
fn main_dry() {
let file = fs::File::open(WORDS_PATH).unwrap();
let reader = io::BufReader::new(&file);
let count = short_lines(reader).count();
println!("{:?}", count);
// Would like to do this instead:
let mut reader = io::BufReader::new(&file);
reader.seek(io::SeekFrom::Start(0));
let sample_size = (0.05 * count as f32) as usize; // 5% sample
for line in short_lines(reader).take(sample_size) {
println!("{}", line);
}
}
fn main() {
main_works_but_code_dupe();
}
I can't figure out the type signatures to get this to compile.
The compiler told you what it was.
error[E0308]: mismatched types
--> src/main.rs:35:5
|
35 | reader.lines().map(unwrap).filter(is_short)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected reference, found fn item
|
= note: expected type `std::iter::Filter<std::iter::Map<_, &'a for<'r> std::ops::FnMut(&'r str, bool) + 'a>, &'a for<'r> std::ops::FnMut(&'r str, bool) + 'a>`
found type `std::iter::Filter<std::iter::Map<_, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String {unwrap}>, for<'r> fn(&'r std::string::String) -> bool {is_short}>`
Now, granted, you can't just copy+paste this directly. You have to replace the _ type with the actual one you already had (it left it out because it was already correct). Secondly, you need to delete the {unwrap} and {is_short} bits; those are because function items have unique types, and that's how the compiler annotates them. Sadly, you can't actually write these types out.
Recompile and...
error[E0308]: mismatched types
--> src/main.rs:35:5
|
32 | -> std::iter::Filter<std::iter::Map<std::io::Lines<T>, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String>, for<'r> fn(&'r std::string::String) -> bool>
| -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- expected `std::iter::Filter<std::iter::Map<std::io::Lines<T>, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String>, for<'r> fn(&'r std::string::String) -> bool>` because of return type
...
35 | reader.lines().map(unwrap).filter(is_short)
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ expected fn pointer, found fn item
|
= note: expected type `std::iter::Filter<std::iter::Map<_, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String>, for<'r> fn(&'r std::string::String) -> bool>`
found type `std::iter::Filter<std::iter::Map<_, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String {unwrap}>, for<'r> fn(&'r std::string::String) -> bool {is_short}>`
Remember what I said about function items have unique types? Yeah, that. To fix that, we cast from a function item to a function pointer. We don't even need to specify what we're casting to, we just have to let the compiler know we want it to do a cast.
fn short_lines<'a, T>
(reader: &'a T)
-> std::iter::Filter<std::iter::Map<std::io::Lines<T>, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String>, for<'r> fn(&'r std::string::String) -> bool>
where T: io::BufRead
{
reader.lines().map(unwrap as _).filter(is_short as _)
}
error[E0308]: mismatched types
--> src/main.rs:41:29
|
41 | let count = short_lines(reader).count();
| ^^^^^^ expected reference, found struct `std::io::BufReader`
|
= note: expected type `&_`
found type `std::io::BufReader<&std::fs::File>`
= help: try with `&reader`
Again, the compiler tells you exactly what to do. Make the change and...
error[E0507]: cannot move out of borrowed content
--> src/main.rs:35:5
|
35 | reader.lines().map(unwrap as _).filter(is_short as _)
| ^^^^^^ cannot move out of borrowed content
Right, that's because you got the input of short_lines wrong. One more change:
fn short_lines<T>
(reader: T)
-> std::iter::Filter<std::iter::Map<std::io::Lines<T>, fn(std::result::Result<std::string::String, std::io::Error>) -> std::string::String>, for<'r> fn(&'r std::string::String) -> bool>
where T: io::BufRead
{
reader.lines().map(unwrap as _).filter(is_short as _)
}
And now all you have to deal with are warnings.
In short: read the compiler messages. They're useful.
I'd suggest doing it the simple way - collecting the iterator into a vector:
use std::fs;
use std::io;
use std::io::prelude::*;
const WORDS_PATH: &str = "/usr/share/dict/words";
fn main() {
let file = fs::File::open(WORDS_PATH).unwrap();
let reader = io::BufReader::new(&file);
let short_lines = reader.lines()
.map(|l| l.unwrap())
.filter(|l| l.len() < 7)
.collect::<Vec<_>>(); // the element type can just be inferred
let count = short_lines.len();
println!("{:?}", count);
let sample_size = (0.05 * count as f32) as usize; // 5% sample
for line in &short_lines[0..sample_size] {
println!("{}", line);
}
}
I am trying to create a function that takes two iterators as as parameters and iterates over the items by reference. Each Iterator item should implement PartialEq.
My first attempt was:
fn compute<T: Iterator>(first: T, second: T, len: usize) -> usize
where
T::Item: std::cmp::PartialEq,
{
// ...
}
This compiles but iterates (as far as I understand) not by reference but by value and the compiler complains about a move when iterating.
My second attempt was something like:
fn compute<'a, T>(first: T, second: T, len: usize) -> usize
where
T: Iterator<Item = &'a std::cmp::PartialEq>,
{
//...
}
resulting in a compiler error:
error[E0393]: the type parameter `Rhs` must be explicitly specified
--> src/main.rs:3:28
|
3 | T: Iterator<Item = &'a std::cmp::PartialEq>,
| ^^^^^^^^^^^^^^^^^^^ missing reference to `Rhs`
|
= note: because of the default `Self` reference, type parameters must be specified on object types
What does the Rhs (Right hand side?) the compiler refers to here mean? Why do I need a reference to it? How do I pass a bounded reference-based Iterator into a function?
PartialEq is a trait that allows you to compare two values. Those two values do not have to be of the same type! The generic type Rhs is used to specify what type we are comparing with. By default, the value of Rhs is the same as the type that is being compared to:
pub trait PartialEq<Rhs = Self>
where
Rhs: ?Sized,
In this case, you are actually requesting that the iterator value be the trait object &PartialEq. As the error message states:
because of the default Self reference, type parameters must be specified on object types
We could specify it:
fn compute<'a, T>(first: T, second: T, len: usize) -> usize
where
T: Iterator<Item = &'a std::cmp::PartialEq<i32>>,
or
fn compute<'a, T: 'a>(first: T, second: T, len: usize) -> usize
where
T: Iterator<Item = &'a std::cmp::PartialEq<&'a T>>,
but iterates (as far as I understand) not by reference but by value
It's quite possible for it to iterate by reference. Remember that T is any type and that i32, &i32, and &mut i32 are all types. Your first example is the formulation of the signature I would use:
fn compute<T: Iterator>(first: T, second: T, len: usize) -> usize
where
T::Item: std::cmp::PartialEq,
{
42
}
fn main() {
let a = [1, 2, 3];
let b = [4, 5, 6];
compute(a.iter(), b.iter(), 1);
compute(a.iter(), b.iter(), 2);
compute(a.iter(), b.iter(), 3);
}
I'd like to use Peekable as the basis for a new cautious_take_while operation that acts like take_while from IteratorExt but without consuming the first failed item. (There's a side question of whether this is a good idea, and whether there are better ways to accomplish this goal in Rust -- I'd be happy for hints in that direction, but mostly I'm trying to understand where my code is breaking).
The API I'm trying to enable is basically:
let mut chars = "abcdefg.".chars().peekable();
let abc : String = chars.by_ref().cautious_take_while(|&x| x != 'd');
let defg : String = chars.by_ref().cautious_take_while(|&x| x != '.');
// yielding (abc = "abc", defg = "defg")
I've taken a crack at creating a MCVE here, but I'm getting:
:10:5: 10:19 error: cannot move out of borrowed content
:10 chars.by_ref().cautious_take_while(|&x| x != '.');
As far as I can tell, I'm following the same pattern as Rust's own TakeWhile in terms of my function signatures, but I'm seeing different different behavior from the borrow checker. Can someone point out what I'm doing wrong?
The funny thing with by_ref() is that it returns a mutable reference to itself:
pub trait IteratorExt: Iterator + Sized {
fn by_ref(&mut self) -> &mut Self { self }
}
It works because the Iterator trait is implemented for the mutable pointer to Iterator type. Smart!
impl<'a, I> Iterator for &'a mut I where I: Iterator, I: ?Sized { ... }
The standard take_while function works because it uses the trait Iterator, that is automatically resolved to &mut Peekable<T>.
But your code does not work because Peekable is a struct, not a trait, so your CautiousTakeWhileable must specify the type, and you are trying to take ownership of it, but you cannot, because you have a mutable pointer.
Solution, do not take a Peekable<T> but &mut Peekable<T>. You will need to specify the lifetime too:
impl <'a, T: Iterator, P> Iterator for CautiousTakeWhile<&'a mut Peekable<T>, P>
where P: FnMut(&T::Item) -> bool {
//...
}
impl <'a, T: Iterator> CautiousTakeWhileable for &'a mut Peekable<T> {
fn cautious_take_while<P>(self, f: P) -> CautiousTakeWhile<&'a mut Peekable<T>, P>
where P: FnMut(&T::Item) -> bool {
CautiousTakeWhile{inner: self, condition: f,}
}
}
A curious side effect of this solution is that now by_ref is not needed, because cautious_take_while() takes a mutable reference, so it does not steal ownership. The by_ref() call is needed for take_while() because it can take either Peekable<T> or &mut Peekable<T>, and it defaults to the first one. With the by_ref() call it will resolve to the second one.
And now that I finally understand it, I think it might be a good idea to change the definition of struct CautiousTakeWhile to include the peekable bit into the struct itself. The difficulty is that the lifetime has to be specified manually, if I'm right. Something like:
struct CautiousTakeWhile<'a, T: Iterator + 'a, P>
where T::Item : 'a {
inner: &'a mut Peekable<T>,
condition: P,
}
trait CautiousTakeWhileable<'a, T>: Iterator {
fn cautious_take_while<P>(self, P) -> CautiousTakeWhile<'a, T, P> where
P: FnMut(&Self::Item) -> bool;
}
and the rest is more or less straightforward.
This was a tricky one! I'll lead with the meat of the code, then attempt to explain it (if I understand it...). It's also the ugly, unsugared version, as I wanted to reduce incidental complexity.
use std::iter::Peekable;
fn main() {
let mut chars = "abcdefg.".chars().peekable();
let abc: String = CautiousTakeWhile{inner: chars.by_ref(), condition: |&x| x != 'd'}.collect();
let defg: String = CautiousTakeWhile{inner: chars.by_ref(), condition: |&x| x != '.'}.collect();
println!("{}, {}", abc, defg);
}
struct CautiousTakeWhile<'a, I, P> //'
where I::Item: 'a, //'
I: Iterator + 'a, //'
P: FnMut(&I::Item) -> bool,
{
inner: &'a mut Peekable<I>, //'
condition: P,
}
impl<'a, I, P> Iterator for CautiousTakeWhile<'a, I, P>
where I::Item: 'a, //'
I: Iterator + 'a, //'
P: FnMut(&I::Item) -> bool
{
type Item = I::Item;
fn next(&mut self) -> Option<I::Item> {
let return_next =
match self.inner.peek() {
Some(ref v) => (self.condition)(v),
_ => false,
};
if return_next { self.inner.next() } else { None }
}
}
Actually, Rodrigo seems to have a good explanation, so I'll defer to that, unless you'd like me to explain something specific.