Pass an iterator of structs to a function accepting references in Rust? - iterator

Question

I have a function that takes an iterator over references to structs. Sometimes I'm iterating over a vector, which works fine, but sometimes I create an iterator that produces new structs, and I'm having trouble figuring that one out. I get that when I create a value in a closure, it goes away when the closure does. Rust is always trying to move values out of things when I don't want it to; why doesn't it here?
struct Thing {
value: u32,
}
fn consume<'a, I: IntoIterator<Item = &'a Thing>>(things: I) {
for thing in things {
println!("{}", thing.value);
}
}
fn main() {
let solid = vec![Thing { value: 0 }];
let ephemeral = (1..5).map(|i| &Thing { value: i }); // Boxing Thing does not work either
consume(solid.iter());
consume(ephemeral);
}
But
error[E0515]: cannot return reference to temporary value
--> src/main.rs:13:36
|
13 | let ephemeral = (1..5).map(|i| &Thing { value: i }); // Boxing Thing does not work either
| ^------------------
| ||
| |temporary value created here
| returns a reference to data owned by the current function
I have the sense I either need to move the struct out of the closure and iterator, or store it somewhere. But Boxing the struct doesn't work and returning a struct rather than a pointer doesn't type check (and I can't find the opposite of .cloned()). What's the approach here?

Answer 1

Short answer: you can't.
Longer explanation:
Here is "an iterator that produces new structs":
let iterator_of_structs = (1..5).map(|value| Thing { value });
The main trick to figuring this out is to always ask "who owns the data?".
Each time we call next, the closure takes ownership of an integer (via value) and constructs a new Thing. The closure returns the Thing, transferring ownership to the code that called next.
While you are borrowing a value (a.k.a. taking a reference), the ownership of the value cannot change hands and the value must last longer than the borrow lasts.
Let's turn to the concept of an iterator of references and ask our question: "who owns the data?".
map(|value| &Thing { value })
Here, we create a Thing and take a reference to it. No variable owns the Thing, so the scope owns it and the value will be destroyed when the scope ends. The closure tries to return the reference, but that violates the axiom that borrowed items must outlive their borrows.
So, how do you fix it? The easiest thing is to change your function to be more accepting:
use std::borrow::Borrow;
struct Thing {
value: u32,
}
fn consume(things: impl IntoIterator<Item = impl Borrow<Thing>>) {
for thing in things {
let thing = thing.borrow();
println!("{}", thing.value);
}
}
fn main() {
let iterator_of_structs = (1..5).map(|value| Thing { value });
consume(iterator_of_structs);
let vector_of_structs: Vec<_> = (1..5).map(|value| Thing { value }).collect();
let iterator_of_references_to_structs = vector_of_structs.iter();
consume(iterator_of_references_to_structs);
}
Here, we accept any type which can be converted into an iterator of items that allow us to borrow a reference to a Thing. This is implemented for any item and any reference to an item.

Answer 2

An iterator of references allows the consumer to keep all the references that the iterator yielded, for as long as they want (at least while the iterator itself remains alive). Obviously to support that, all objects to which the iterator creates references need to be in memory at the same time. There is no way around this with the iterator protocol as-is. So your best course of action is to collect() the iterator into a vector and create an reference iterator from that (as you do with solid). Unfortunately this means losing the laziness.
There is an alternative iterator abstraction, called streaming iterator, which would support this. With streaming iterators, the consumer may only hold onto the reference until it gets the next one. I am not aware of any crates implementing this though, and it would be a completely different trait which no function using std::iter::Iterator supports. In many cases it may even be impossible to use streaming iterators, because the algorithm needs the freedom to reference several values at once.