Idiomatic way to collect all errors from an iterator - error-handling

Let's say I have a attrs: Vec<Attribute> of some function attributes and a function fn map_attribute(attr: &Attribute) -> Result<TokenStream, Error> that maps the attributes to some code.
I know that I could write something like this:
attrs.into_iter()
.map(map_attribute)
.collect::<Result<Vec<_>, _>()?
However, this is not what I want. What I want is spit out all errors at once, not stop with the first Error. Currently I do something like this:
let mut codes : Vec<TokenStream> = Vec::new();
let mut errors: Vec<Error> = Vec::new();
for attr in attrs {
match map_attribute(attr) {
Ok(code) => codes.push(code),
Err(err) => errors.push(err)
}
}
let mut error_iter = errors.into_iter();
if let Some(first) = error_iter.nth(0) {
return Err(iter.fold(first, |mut e0, e1| { e0.combine(e1); e0 }));
}
This second version does what I want, but is considerably more verbose than the first version. Is there a better / more idiomatic way to acchieve this, if possible without creating my own iterator?

The standard library does not have a convenient one-liner for this as far as I know, however the excellent itertools library does:
use itertools::Itertools; // 0.9.0
fn main() {
let foo = vec![Ok(42), Err(":("), Ok(321), Err("oh noes")];
let (codes, errors): (Vec<_>, Vec<_>)
= foo.into_iter().partition_map(From::from);
println!("codes={:?}", codes);
println!("errors={:?}", errors);
}
(Permalink to the playground)

I ended up writing my own extension for Iterator, which allows me to stop collecting codes when I encounter my first error. This is in my use case probably a bit more efficient than the answer by mcarton, since I only need the first partition bucket if the second one is empty. Also, I need to fold the errors anyways if I want to combine them into a single error.
pub trait CollectToResult
{
type Item;
fn collect_to_result(self) -> Result<Vec<Self::Item>, Error>;
}
impl<Item, I> CollectToResult for I
where
I : Iterator<Item = Result<Item, Error>>
{
type Item = Item;
fn collect_to_result(self) -> Result<Vec<Item>, Error>
{
self.fold(<Result<Vec<Item>, Error>>::Ok(Vec::new()), |res, code| {
match (code, res) {
(Ok(code), Ok(mut codes)) => { codes.push(code); Ok(codes) },
(Ok(_), Err(errors)) => Err(errors),
(Err(err), Ok(_)) => Err(err),
(Err(err), Err(mut errors)) => { errors.combine(err); Err(errors) }
}
})
}
}

Related

Failed to fill whole buffer with rust read_exact

I have a little code snippet where I'm trying to write struct to a file and then read it. I have seen other similar posts where the asker has forgotten to zero initialise the buffer they are trying to read into. I have made sure not to do this, but I still am getting the error that 'failed to fill whole buffer' error when using read_exact, even though my buffer size and the size of the file I'm trying to read are the same.
Here is the code:
use std::fs::{File, OpenOptions};
use std::io::prelude::*;
use bincode::*;
use serde::{Serialize, Deserialize};
#[derive(Serialize, Deserialize, PartialEq, Debug)]
pub struct Node {
pub name: String,
end_ptr: u32 // number of bytes away the next node is
}
impl Node {
pub fn to_string(&self) -> String {
return self.name.clone();
}
}
fn main() {
let node = Node { name: String::from("node_1"), end_ptr: 0 };
let node_as_buf = bincode::serialize(&node).unwrap();
let len_of_bytes_serialised: usize = node_as_buf.len();
let mut file = OpenOptions::new()
.read(true)
.write(true)
.create(true)
.truncate(true)
.open("test.txt")
.unwrap();
match file.write_all(&node_as_buf) {
Ok(result) => {println!("Write success")},
Err(err) => {println!("{}", &err)}
}
println!("{}", file.metadata().unwrap().len()); // this and
println!("{}", len_of_bytes_serialised); // this are the same size
let mut buffer = vec![0; len_of_bytes_serialised];
match file.read_exact(&mut buffer[..]) {
Ok(result) => println!("Read success"),
Err(err) => println!("{}", &err) // prints 'failed to fill whole buffer'
}
let read_node: Node = bincode::deserialize(&buffer[..]).unwrap();
}
Because you use the same stream for reading and writing, the cursor is moved to the end of the file and any reading will attempt to read there, which will of course fail. You can observe that if you'll print file.stream_position().
You need to rewind() before reading.

How to get error using if else method in rust?

I know the idiomatic way to handle errors in rust is using match statement but I am looking for other ways to do the same.
use std::fs;
fn main() {
if let Ok(f) = fs::read_dir("/dummy"){
println!("First: {:?}", f);
}else{
println!("Error");
}
}
This works but I need the original Result error from fs::read_dir("/dummy") to be printed in the else statement. How can I do it?
Generally, I'd consider such an approach a bad idea, but you do have a few options, the most obvious two being "multiple if lets" and a "functional style" approach. I've included the match version for comparison. The code is available on the playground.
fn multiple_if_lets() {
let f = std::fs::read_dir("/dummy");
if let Ok(f) = &f {
println!("First: {:?}", f);
}
if let Err(f) = &f {
println!("Error: {:?}", f);
}
}
fn functional_style() {
std::fs::read_dir("/dummy")
.map(|f| println!("First: {:?}", f))
.unwrap_or_else(|f| println!("Error: {:?}", f));
}
fn match_style() {
match std::fs::read_dir("/dummy") {
Ok(f) => println!("First: {:?}", f),
Err(f) => println!("Error: {:?}", f),
}
}
I'm not quite sure why you don't want to use match because it is like a better if let! But you might find it useful to use an external crate like anyhow. You can very easily propagate all errors as one type and also add context where it makes sense.
In your Cargo.toml
[dependencies]
anyhow = "1"
In your code
use std::fs;
use anyhow::Context;
fn main() -> anyhow::Result<()> {
let dir = fs::read_dir("/dummy").context("failed to read dir")?;
// another fallible function which can return a `Result` with a
// different error type.
do_something(dir).context("failed to do something")?;
Ok(())
}
If read_dir had to fail here your program would exit and it would output the following
Error: failed to read dir
Caused by:
No such file or directory (os error 2)
If you wanted to throw away the error but still print it out you could still use match.
let dir = match fs::read_dir("/dummy").context("failed to read dir") {
Ok(dir) => Some(dir),
Err(err) => {
eprintln!("Error: {:?}", err);
None
}
};
This would output something like the following but still continue:
Error: failed to read dir
Caused by:
No such file or directory (os error 2)
Since Rust 1.65.0 (Nov. 2022), as mentioned by Mara Bos, you can do:
let Ok(f) = fs::read_dir("/dummy") else{ println!("Error"); return };
println!("First: {:?}", f);
let-else statements.
You can now write things like:
let Ok(a) = i32::from_str("123") else { return };
without needing an if or match statement.
This can be useful to avoid deeply nested if statements.

Is there a more ergonomic syntax for Either when using futures?

Here's an example of using Tokio to run a function that returns a future:
use futures::sync::oneshot;
use futures::Future;
use std::thread;
use std::time::Duration;
use tokio;
#[derive(Debug)]
struct MyError {
error_code: i32,
}
impl From<oneshot::Canceled> for MyError {
fn from(_: oneshot::Canceled) -> MyError {
MyError { error_code: 1 }
}
}
fn deferred_task() -> impl Future<Item = i32, Error = MyError> {
let (sx, rx) = oneshot::channel();
thread::spawn(move || {
thread::sleep(Duration::from_millis(100));
sx.send(100).unwrap();
});
return rx.map_err(|e| MyError::from(e));
}
fn main() {
tokio::run(deferred_task().then(|r| {
println!("{:?}", r);
Ok(())
}));
}
However, when the function in question (i.e. deferred_task) is non-trivial, the code becomes much more complex when I write it, because the ? operation doesn't seem to easily mix with returning a future:
fn send_promise_to_worker(sx: oneshot::Sender<i32>) -> Result<(), ()> {
// Send the oneshot somewhere in a way that might fail, eg. over a channel
thread::spawn(move || {
thread::sleep(Duration::from_millis(100));
sx.send(100).unwrap();
});
Ok(())
}
fn deferred_task() -> impl Future<Item = i32, Error = MyError> {
let (sx, rx) = oneshot::channel();
send_promise_to_worker(sx)?; // <-------- Can't do this, because the return is not a result
return rx.map_err(|e| MyError::from(e));
}
A Future is a Result, it's meaningless to wrap it in result, and it breaks the impl Future return type.
Instead you get a deeply nested chain of:
fn deferred_task() -> impl Future<Item = i32, Error = MyError> {
let (sx, rx) = oneshot::channel();
match query_data() {
Ok(_i) => match send_promise_to_worker(sx) {
Ok(_) => Either::A(rx.map_err(|e| MyError::from(e))),
Err(_e) => Either::B(futures::failed(MyError { error_code: 2 })),
},
Err(_) => Either::B(futures::failed(MyError { error_code: 2 })),
}
}
full code
The more results you have, the deeper the nesting; exactly what the ? operator solves normally.
Am I missing something? Is there some syntax sugar to make this easier?
I do not see how async / await syntax will categorically help you with Either. Ultimately, you still need to return a single concrete type, and that's what Either provides. async / await will reduce the need for combinators like Future::map or Future::and_then however.
See also:
Why can impl trait not be used to return multiple / conditional types?
That being said, you don't need to use Either here.
You have consecutive Result-returning functions, so you can borrow a trick from JavaScript and use an IIFE to use use the ? operator. Then, we can "lift up" the combined Result into a future and chain it with the future from the receiver:
fn deferred_task() -> impl Future<Item = i32, Error = MyError> {
let (tx, rx) = oneshot::channel();
let x = (|| {
let _i = query_data().map_err(|_| MyError { error_code: 1 })?;
send_promise_to_worker(tx).map_err(|_| MyError { error_code: 2 })?;
Ok(())
})();
future::result(x).and_then(|()| rx.map_err(MyError::from))
}
In the future, that IIFE could be replaced with a try block, as I understand it.
You could also go the other way and convert everything to a future:
fn deferred_task() -> impl Future<Item = i32, Error = MyError> {
let (tx, rx) = oneshot::channel();
query_data()
.map_err(|_| MyError { error_code: 1 })
.into_future()
.and_then(|_i| {
send_promise_to_worker(tx)
.map_err(|_| MyError { error_code: 2 })
.into_future()
})
.and_then(|_| rx.map_err(MyError::from))
}
This would be helped with async / await syntax:
async fn deferred_task() -> Result<i32, MyError> {
let (tx, rx) = oneshot::channel();
query_data().map_err(|_| MyError { error_code: 1 })?;
send_promise_to_worker(tx).map_err(|_| MyError { error_code: 2 })?;
let v = await! { rx }?;
Ok(v)
}
I have also seen improved syntax for constructing the Either by adding left and right methods to the Future trait:
foo.left();
// vs
Either::left(foo);
However, this doesn't appear in any of the current implementations.
A Future is a Result
No, it is not.
There are two relevant Futures to talk about:
From the futures 0.1 crate
From the (nightly) standard library
Notably, Future::poll returns a type that can be in two states:
Complete
Not complete
In the futures crate, "success" and "failure" are tied to "complete", whereas in the standard library they are not. In the crate, Result implements IntoFuture, and in the standard library you can use future::ready. Both of these allow converting a Result into a future, but that doesn't mean that Result is a future, no more than saying that a Vec<u8> is an iterator, even though it can be converted into one.
It's possible that the ? operator (powered by the Try trait), will be enhanced to automatically convert from a Result to a specific type of Future, or that Result will even implement Future directly, but I have not heard of any such plans.
Is there some syntax sugar to make this easier?
Yes, it's called async/await, but it's not quite ready for wide consumption. It is only supported on nightly, it uses a slightly different version of futures that Tokio only supports via an interop library that causes additional syntactic overhead, and documentation for the whole thing is still spotty.
Here are some relevant links:
What is the purpose of async/await in Rust?
https://jsdw.me/posts/rust-asyncawait-preview/
https://areweasyncyet.rs/

How to include the file path in an IO error in Rust?

In this minimalist program, I'd like the file_size function to include the path /not/there in the Err so it can be displayed in the main function:
use std::fs::metadata;
use std::io;
use std::path::Path;
use std::path::PathBuf;
fn file_size(path: &Path) -> io::Result<u64> {
Ok(metadata(path)?.len())
}
fn main() {
if let Err(err) = file_size(&PathBuf::from("/not/there")) {
eprintln!("{}", err);
}
}
You must define your own error type in order to wrap this additional data.
Personally, I like to use the custom_error crate for that, as it's especially convenient for dealing with several types. In your case it might look like this:
use custom_error::custom_error;
use std::fs::metadata;
use std::io;
use std::path::{Path, PathBuf};
use std::result::Result;
custom_error! {ProgramError
Io {
source: io::Error,
path: PathBuf
} = #{format!("{path}: {source}", source=source, path=path.display())},
}
fn file_size(path: &Path) -> Result<u64, ProgramError> {
metadata(path)
.map(|md| md.len())
.map_err(|e| ProgramError::Io {
source: e,
path: path.to_path_buf(),
})
}
fn main() {
if let Err(err) = file_size(&PathBuf::from("/not/there")) {
eprintln!("{}", err);
}
}
Output:
/not/there: No such file or directory (os error 2)
While Denys Séguret's answer is correct, I like using my crate SNAFU because it provides the concept of a context. This makes the act of attaching the path (or anything else!) very easy to do:
use snafu::{ResultExt, Snafu}; // 0.2.3
use std::{
fs, io,
path::{Path, PathBuf},
};
#[derive(Debug, Snafu)]
enum ProgramError {
#[snafu(display("Could not get metadata for {}: {}", path.display(), source))]
Metadata { source: io::Error, path: PathBuf },
}
fn file_size(path: impl AsRef<Path>) -> Result<u64, ProgramError> {
let path = path.as_ref();
let md = fs::metadata(&path).context(Metadata { path })?;
Ok(md.len())
}
fn main() {
if let Err(err) = file_size("/not/there") {
eprintln!("{}", err);
}
}

How to map on a vec and use a closure with pattern matching in Rust

I'd like to use map to iterate over an array and do stuff per item and get rid of the for loop. An error which I do not understand blocks my attempt. What I want to achieve is to iterate through a vector of i32 and match on them to concat a string with string literals and then return it at the end.
Function:
pub fn convert_to_rainspeak(prime_factors: Vec<i32>) -> String {
let mut speak = String::new();
prime_factors.iter().map(|&factor| {
match factor {
3 => { speak.push_str("Pling"); },
5 => { speak.push_str("Plang"); },
7 => { speak.push_str("Plong"); },
_ => {}
}
}).collect();
speak
}
fn main() {}
Output:
error[E0282]: type annotations needed
--> src/main.rs:10:8
|
10 | }).collect();
| ^^^^^^^ cannot infer type for `B`
Iterator::collect is defined as:
fn collect<B>(self) -> B
where
B: FromIterator<Self::Item>
That is, it returns a type that is up to the caller. However, you have completely disregarded the output, so there's no way for it to infer a type. The code misuses collect when it basically wants to use for.
In your "fixed" version (which has since been edited, making this paragraph make no sense), you are being very inefficient by allocating a string in every iteration. Plus you don't need to specify any explicit types other than those on the function, and you should accept a &[i32] instead:
fn convert_to_rainspeak(prime_factors: &[i32]) -> String {
prime_factors.iter()
.map(|&factor| {
match factor {
3 => "Pling",
5 => "Plang",
7 => "Plong",
_ => "",
}
})
.collect()
}
fn main() {
println!("{}", convert_to_rainspeak(&[1, 2, 3, 4, 5]));
}
The error in this case is that the compiler can't figure out what type you are collecting into. If you add a type annotation to collect, then it will work:
pub fn convert_to_rainspeak(prime_factors:Vec<i32>) -> String{
let mut speak = String::new();
prime_factors.iter().map(| &factor| {
match factor {
3 => {speak.push_str("Pling");},
5 => {speak.push_str("Plang");},
7 => {speak.push_str("Plong");},
_ => {}
}
}).collect::<Vec<_>>();
speak
}
However, this is really not the idiomatic way to do this. You should use for instead:
pub fn convert_to_rainspeak(prime_factors:Vec<i32>) -> String {
let mut speak = String::new();
for factor in prime_factors.iter() {
match *factor {
3 => speak.push_str("Pling"),
5 => speak.push_str("Plang"),
7 => speak.push_str("Plong"),
_ => {}
}
}
speak
}
You could also use flat_map() instead of map().
This way you can map to Option and return None instead of empty string if there is no corresponding value.
fn convert_to_rainspeak(prime_factors: &[i32]) -> String {
prime_factors
.iter()
.flat_map(|&factor| match factor {
3 => Some("Pling"),
5 => Some("Plang"),
7 => Some("Plong"),
_ => None,
})
.collect()
}
fn main() {
println!("{}", convert_to_rainspeak(&[1, 2, 3, 7]));
}
few minutes later I solved it myself (any upgrade appreciated):
pub fn convert_to_rainspeak(prime_factors:Vec<i32>) -> String{
let mut speak:String = prime_factors.iter().map(|&factor| {
match factor {
3 => {"Pling"},
5 => {"Plang"},
7 => {"Plong"},
_ => {""}
}
}).collect();
speak
}
The issue was that I was not aware that .collect() is awaiting the result of map. I then assigned prime_factors.iter()... to a string and rearranged var bindings so it now all works.
EDIT: refactored redundant assignments to the speak vector