get structure header names with serde Serialize preserving order - serialization

Rust data structure.
This is my implementation to get headers preserving order:
use serde::{Deserialize, Serialize};
use csv::{Reader, Writer, StringRecord};
#[derive(Default, Debug, Deserialize, Serialize)]
pub struct Colunas {
#[serde(rename = "Arquivo Teste")]
arquivo: String,
#[serde(rename = "CNPJ dos Estabelecimentos")]
cnpj: Option<String>,
#[serde(rename = "Ano do Período de Apuração")]
ano: Option<usize>,
#[serde(rename = "Mês")]
m: u32,
}
impl Colunas {
fn get_headers(self) -> StringRecord {
// https://docs.rs/csv/1.0.0/csv/struct.Writer.html
let mut wtr = Writer::from_writer(vec![]);
wtr.serialize( Colunas::default() ).ok();
let colunas = String::from_utf8(wtr.into_inner().ok().unwrap()).ok().unwrap();
// split-a-string-in-rust
let mut lines = colunas.lines();
let first_line = lines.next().unwrap();
//println!("first_line: {:#?}", &first_line);
let vec: Vec<&str> = first_line.split(',').collect();
//println!("vec: {:#?}", vec);
//https://docs.rs/csv/1.0.0-beta.2/csv/struct.StringRecord.html
// https://docs.rs/csv/1.0.0/csv/struct.Reader.html
let mut rdr = Reader::from_reader("".as_bytes());
rdr.set_headers(StringRecord::from(vec));
let headers = rdr.headers().ok().unwrap();
//println!("headers: {:#?}", headers);
headers.clone()
}
}
fn main() {
let headers = Colunas::default().get_headers();
println!("headers: {:#?}", headers);
}
Is there any other more elegant and concise way?
Rust playground:
https://play.rust-lang.org/?version=stable&mode=debug&edition=2021&gist=24cd9eeed8f5fbfa2abf0eff467a03a2
The output is:
headers: StringRecord(["Arquivo Teste", "CNPJ dos Estabelecimentos", "Ano do Período de Apuração", "Mês"])

I don't know of any concise and elegant way. But I can give you concise via a trip through serde_json::Value
serde_json::to_value(&Colunas::default())
.unwrap()
.as_object()
.unwrap()
.keys()
.collect()
(Enabling the preserve_order feature on serde_json should preserve the order of the fields in the struct.)
As for elegant, one part I consider inelegant is that you have to make Colunas impl Default for this. This can be avoided with a custom deserializer, which has access to the structure without ever needing a value.
struct FieldTracingDeserializer<'a> {
fields: &'a mut Vec<&'static str>,
}
impl<'de, 'a> serde::Deserializer<'de> for FieldTracingDeserializer<'a> {
type Error = serde::de::value::Error;
fn deserialize_any<V>(self, _visitor: V) -> Result<V::Value, Self::Error>
where
V: serde::de::Visitor<'de>,
{
panic!("Only works for structs");
}
serde::forward_to_deserialize_any! {
bool i8 i16 i32 i64 i128 u8 u16 u32 u64 u128 f32 f64 char str string
bytes byte_buf option unit unit_struct newtype_struct seq tuple
tuple_struct map enum identifier ignored_any
}
fn deserialize_struct<V>(
self,
_name: &'static str,
fields: &'static [&'static str],
_visitor: V,
) -> Result<V::Value, Self::Error>
where
V: serde::de::Visitor<'de>,
{
// Would be cleaner to return fields through a custom error struct.
// But also more work.
self.fields.extend_from_slice(fields);
Err(serde::de::Error::custom("success"))
}
}
let mut headers = vec![];
Colunas::deserialize(FieldTracingDeserializer {
fields: &mut headers,
})
.ok();
headers.into_iter().collect()
I don't really think this can be called elegant… There are however crates that implement this exact functionality (e.g. serde-aux):
serde_aux::serde_introspection::serde_introspect::<Colunas>()

Related

How to properly handle empty, null and valid JSON?

I need to deserialize a JSON file into either None or Some(T) in Rust. The source we are using will provide null or empty, '{}', JSON fields when no values are present. I want to handle both as the None case and only deserialize when the JSON field is not null or empty.
input: {"test": null} -> output: {"test": None}
input: {"test": {}} -> output: {"test": None}
input: {"test": {"valid_json": 42}} -> output: {"test": {"valid_json": 42}}
All of the answers I could find address one case or another but not both.
use serde::{Deserialize, Deserializer};
#[derive(Deserialize, Debug, PartialEq)]
struct Foo {
#[serde(deserialize_with = "object_empty_as_none")]
bar: Option<Bar>,
}
#[derive(Deserialize, Debug, PartialEq)]
struct Bar {
inner: u32,
}
pub fn object_empty_as_none<'de, D, T>(deserializer: D) -> Result<Option<T>, D::Error>
where
D: Deserializer<'de>,
for<'a> T: Deserialize<'a>,
{
#[derive(Deserialize, Debug)]
#[serde(deny_unknown_fields)]
struct Empty {}
#[derive(Deserialize, Debug)]
#[serde(untagged)]
enum Aux<T> {
T(T),
Empty(Empty),
Null,
}
match Deserialize::deserialize(deserializer)? {
Aux::T(t) => Ok(Some(t)),
Aux::Empty(_) | Aux::Null => Ok(None),
}
}
fn main() {
let data = r#"{"bar": null}"#;
let v: Foo = serde_json::from_str(data).unwrap();
assert_eq!(v, Foo { bar: None });
let data = r#"{"bar": {}}"#;
let v: Foo = serde_json::from_str(data).unwrap();
assert_eq!(v, Foo { bar: None });
let data = r#"{"bar": {"inner": 42}}"#;
let v: Foo = serde_json::from_str(data).unwrap();
assert_eq!(
v,
Foo {
bar: Some(Bar { inner: 42 })
}
);
let data = r#"{"bar": {"not_inner": 42}}"#;
let v: Result<Foo, _> = serde_json::from_str(data);
assert!(v.is_err());
}
Should be enough for most case. Remove #[serde(deny_unknown_fields)] on Empty if you want to.
This page tells you how to implement a custom map deserializer, which requires customizing how visit_map produces key-value pairs from the input data. I've basically copied that page and produced a minimal example that implements what you're looking for. Link to playground.
use std::fmt;
use std::marker::PhantomData;
use serde::de::{Deserialize, Deserializer, MapAccess, Visitor};
use serde_json::Value as JsonValue;
use std::collections::HashMap;
#[derive(Debug)]
struct MyMap(HashMap<String, JsonValue>);
impl MyMap {
fn with_capacity(capacity: usize) -> Self {
Self(HashMap::with_capacity(capacity))
}
}
struct MyMapVisitor {
marker: PhantomData<fn() -> MyMap>,
}
impl MyMapVisitor {
fn new() -> Self {
MyMapVisitor {
marker: PhantomData,
}
}
}
impl<'de> Visitor<'de> for MyMapVisitor {
// The type that our Visitor is going to produce.
type Value = MyMap;
// Format a message stating what data this Visitor expects to receive.
fn expecting(&self, formatter: &mut fmt::Formatter) -> fmt::Result {
formatter.write_str("a very special map")
}
// Deserialize MyMap from an abstract "map" provided by the
// Deserializer. The MapAccess input is a callback provided by
// the Deserializer to let us see each entry in the map.
fn visit_map<M>(self, mut access: M) -> Result<Self::Value, M::Error>
where
M: MapAccess<'de>,
{
let mut map = MyMap::with_capacity(access.size_hint().unwrap_or(0));
// While there are entries remaining in the input, add them
// into our map. Empty Objects get turned into Null.
while let Some((key, value)) = access.next_entry()? {
let value = match value {
JsonValue::Object(o) if o.is_empty() => JsonValue::Null,
_ => value,
};
map.0.insert(key, value);
}
Ok(map)
}
}
// This is the trait that informs Serde how to deserialize MyMap.
impl<'de> Deserialize<'de> for MyMap {
fn deserialize<D>(deserializer: D) -> Result<Self, D::Error>
where
D: Deserializer<'de>,
{
// Instantiate our Visitor and ask the Deserializer to drive
// it over the input data, resulting in an instance of MyMap.
deserializer.deserialize_map(MyMapVisitor::new())
}
}
fn main() -> serde_json::Result<()> {
let json_str = r#"{"a": null, "b": {}, "c": {"valid_json": 42}}"#;
let v: MyMap = serde_json::from_str(json_str)?;
println!("{:?}", v);
Ok(())
}
This prints MyMap({"b": Null, "c": Object({"valid_json": Number(42)}), "a": Null}) which I believe is what you're after.

How can I return an iterator over a locked struct member in Rust?

Here is as far as I could get, using rental, partly based on How can I store a Chars iterator in the same struct as the String it is iterating on?. The difference here is that the get_iter method of the locked member has to take a mutable self reference.
I'm not tied to using rental: I'd be just as happy with a solution using reffers or owning_ref.
The PhantomData is present here just so that MyIter bears the normal lifetime relationship to MyIterable, the thing being iterated over.
I also tried changing #[rental] to #[rental(deref_mut_suffix)] and changing the return type of MyIterable.get_iter to Box<Iterator<Item=i32> + 'a> but that gave me other lifetime errors originating in the macro that I was unable to decipher.
#[macro_use]
extern crate rental;
use std::marker::PhantomData;
pub struct MyIterable {}
impl MyIterable {
// In the real use-case I can't remove the 'mut'.
pub fn get_iter<'a>(&'a mut self) -> MyIter<'a> {
MyIter {
marker: PhantomData,
}
}
}
pub struct MyIter<'a> {
marker: PhantomData<&'a MyIterable>,
}
impl<'a> Iterator for MyIter<'a> {
type Item = i32;
fn next(&mut self) -> Option<i32> {
Some(42)
}
}
use std::sync::Mutex;
rental! {
mod locking_iter {
pub use super::{MyIterable, MyIter};
use std::sync::MutexGuard;
#[rental]
pub struct LockingIter<'a> {
guard: MutexGuard<'a, MyIterable>,
iter: MyIter<'guard>,
}
}
}
use locking_iter::LockingIter;
impl<'a> Iterator for LockingIter<'a> {
type Item = i32;
#[inline]
fn next(&mut self) -> Option<Self::Item> {
self.rent_mut(|iter| iter.next())
}
}
struct Access {
shared: Mutex<MyIterable>,
}
impl Access {
pub fn get_iter<'a>(&'a self) -> Box<Iterator<Item = i32> + 'a> {
Box::new(LockingIter::new(self.shared.lock().unwrap(), |mi| {
mi.get_iter()
}))
}
}
fn main() {
let access = Access {
shared: Mutex::new(MyIterable {}),
};
let iter = access.get_iter();
let contents: Vec<i32> = iter.take(2).collect();
println!("contents: {:?}", contents);
}
As user rodrigo has pointed out in a comment, the solution is simply to change #[rental] to #[rental_mut].

How do I perform iterator computations over iterators of Results without collecting to a temporary vector?

I'm looking for a way to eliminate the temporary vector allocation in this example:
fn doit<T: Iterator<Item = Result<i32, &'static str>>>(name: &str, iter: T) {
println!(
"{}: {:?}",
name,
iter.collect::<Result<Vec<_>, _>>()
.map(|v| v.into_iter().min())
);
}
fn main() {
let without_errors = vec![Ok(1), Ok(2), Ok(3)];
let with_errors = vec![Ok(1), Err("error"), Ok(2)];
doit("without errors", without_errors.into_iter());
doit("with errors", with_errors.into_iter());
}
This is a variation of the iterator with error handling theme, except that I don't want to create a collection (so collect() doesn't quite do the job), but I want to perform further operations on the elements being iterated over.
Note that this gives the wrong result because Ok is less than Err:
fn doit<T: Iterator<Item = Result<i32, &'static str>>>(name: &str, iter: T) {
println!("{}: {:?}", name, iter.min());
}
It would give the right result for max() by accident, but it would not stop iterating on the first error.
Iterator::try_fold provides the framework for what you need, and it's available since Rust 1.27 (Playground):
fn fold_ok<I, T, E, F>(mut iter: I, f: F) -> Result<Option<T>, E>
where
I: Iterator<Item = Result<T, E>>,
T: Ord,
F: Fn(T, T) -> T,
{
iter.try_fold(None, |r, i| {
let i = i?;
Ok(Some(if let Some(r) = r { f(r, i) } else { i }))
})
}
fn main() {
let without_errors = vec![Ok(1), Ok(2), Ok(3)];
let with_errors = vec![Ok(1), Err("error"), Ok(2)];
fn doit<'r, T>(name: &str, iter: T)
where
T: Iterator<Item = &'r Result<i32, &'static str>> + Clone,
{
println!("{}: {:?}", name, fold_ok(iter.cloned(), ::std::cmp::min));
}
doit("without errors", without_errors.iter());
doit("with errors", with_errors.iter());
}
Before that, I think your only option is manually iterating (Playground)
fn fold_ok<I, T, E, F>(mut iter: I, f: F) -> Result<Option<T>, E>
where
I: Iterator<Item = Result<T, E>>,
T: Ord,
F: Fn(T, T) -> T,
{
let mut result = match iter.next() {
None => return Ok(None),
Some(r) => r?,
};
for item in iter {
result = f(result, item?);
}
Ok(Some(result))
}
fn main() {
let without_errors = vec![Ok(1), Ok(2), Ok(3)];
let with_errors = vec![Ok(1), Err("error"), Ok(2)];
fn doit<'r, T>(name: &str, iter: T)
where
T: Iterator<Item = &'r Result<i32, &'static str>> + Clone,
{
println!(
"{}: {:?}",
name,
fold_ok(iter.clone().cloned(), ::std::cmp::min)
);
}
doit("without errors", without_errors.iter());
doit("with errors", with_errors.iter());
}
"Lifting" a function to handle an iterator of results is a fairly common pattern and, as usual, itertools has a solution — process_results:
use itertools; // 0.8.0
fn doit(name: &str, iter: impl Iterator<Item = Result<i32, &'static str>>) {
let min = itertools::process_results(iter, |i| i.min());
println!("{}: {:?}", name, min);
}
This code began life as ResultShunt in the standard library before being extracted to itertools. It's what underlies the implementation of sum and product for iterators of Result.
It's possible to abuse collect() for this:
pub struct Min<T> {
value: Option<T>,
}
impl<T> Min<T> {
pub fn value(self) -> Option<T> {
self.value
}
}
impl<T> std::iter::FromIterator<T> for Min<T>
where
T: Ord,
{
fn from_iter<I: IntoIterator<Item = T>>(iter: I) -> Self {
let mut iter = iter.into_iter();
match iter.next() {
None => Min { value: None },
Some(mut value) => {
for i in iter {
value = std::cmp::min(value, i);
}
Min { value: Some(value) }
}
}
}
}
This can be used via iter.collect::<Min<_>>().value(). This is a lot of machinery, and I don't see a way to abstract over it (so that you only need to supply std::cmp::min or some other semigroup operation).
I didn't look in the direction of Iterator::try_fold, which provides most of the machinery.

How can I automatically implement FromIterator?

I have written a trait that specifies some methods similar to those of Vec:
pub trait Buffer {
type Item;
fn with_capacity(c: usize) -> Self;
fn push(&mut self, item: Self::Item);
}
I would like to implement FromIterator for all types that implement Buffer, as follows:
impl<T> iter::FromIterator<T::Item> for T
where T: Buffer
{
fn from_iter<I>(iter: I) -> Self
where I: IntoIterator<Item = T>
{
let mut iter = iter.into_iter();
let (lower, _) = iter.size_hint();
let ans = Self::with_capacity(lower);
while let Some(x) = iter.next() {
ans.push(x);
}
ans
}
}
The compiler won't let me:
error[E0210]: type parameter `T` must be used as the type parameter
for some local type (e.g. `MyStruct<T>`); only traits defined in the
current crate can be implemented for a type parameter
I think I understand the error message; it is preventing me from writing code that is incompatible with possible future changes to the standard library.
The only way around this error appears to be to implement FromIterator separately for every type for which I implement Buffer. This will involve copying out exactly the same code many times. Is there a a way to share the same implementation between all Buffer types?
You can't implement a trait from another crate for an arbitrary type, only for a type from your crate. However, you can move the implementation to a function and reduce amount of duplicated code:
fn buffer_from_iter<I, B>(iter: I) -> B
where I: IntoIterator<Item = B::Item>,
B: Buffer
{
let mut iter = iter.into_iter();
let (lower, _) = iter.size_hint();
let mut ans = B::with_capacity(lower);
while let Some(x) = iter.next() {
ans.push(x);
}
ans
}
struct S1;
impl Buffer for S1 {
type Item = i32;
fn with_capacity(c: usize) -> Self { unimplemented!() }
fn push(&mut self, item: Self::Item) { unimplemented!() }
}
impl std::iter::FromIterator<<S1 as Buffer>::Item> for S1 {
fn from_iter<I>(iter: I) -> Self
where I: IntoIterator<Item = <S1 as Buffer>::Item>
{
buffer_from_iter(iter)
}
}
This implementation of FromIterator can be wrapped into a macro to further reduce code duplication.

How to read an integer input from the user in Rust 1.0?

Existing answers I've found are all based on from_str (such as Reading in user input from console once efficiently), but apparently from_str(x) has changed into x.parse() in Rust 1.0. As a newbie, it's not obvious how the original solution should be adapted taking this change into account.
As of Rust 1.0, what is the easiest way to get an integer input from the user?
Here is a version with all optional type annotations and error handling which may be useful for beginners like me:
use std::io;
fn main() {
let mut input_text = String::new();
io::stdin()
.read_line(&mut input_text)
.expect("failed to read from stdin");
let trimmed = input_text.trim();
match trimmed.parse::<u32>() {
Ok(i) => println!("your integer input: {}", i),
Err(..) => println!("this was not an integer: {}", trimmed),
};
}
If you are looking for a way to read input for the purpose of competitive programming on websites like codeforces where you do not have access to text_io, this solution is for you.
I use the following macro to read different values from stdin:
#[allow(unused_macros)]
macro_rules! read {
($out:ident as $type:ty) => {
let mut inner = String::new();
std::io::stdin().read_line(&mut inner).expect("A String");
let $out = inner.trim().parse::<$type>().expect("Parsable");
};
}
#[allow(unused_macros)]
macro_rules! read_str {
($out:ident) => {
let mut inner = String::new();
std::io::stdin().read_line(&mut inner).expect("A String");
let $out = inner.trim();
};
}
#[allow(unused_macros)]
macro_rules! read_vec {
($out:ident as $type:ty) => {
let mut inner = String::new();
std::io::stdin().read_line(&mut inner).unwrap();
let $out = inner
.trim()
.split_whitespace()
.map(|s| s.parse::<$type>().unwrap())
.collect::<Vec<$type>>();
};
}
Use it as follows:
fn main(){
read!(x as u32);
read!(y as f64);
read!(z as char);
println!("{} {} {}", x, y, z);
read_vec!(v as u32); // Reads space separated integers and stops when newline is encountered.
println!("{:?}", v);
}
Here are a few possibilities (Rust 1.7):
use std::io;
fn main() {
let mut n = String::new();
io::stdin()
.read_line(&mut n)
.expect("failed to read input.");
let n: i32 = n.trim().parse().expect("invalid input");
println!("{:?}", n);
let mut n = String::new();
io::stdin()
.read_line(&mut n)
.expect("failed to read input.");
let n = n.trim().parse::<i32>().expect("invalid input");
println!("{:?}", n);
let mut n = String::new();
io::stdin()
.read_line(&mut n)
.expect("failed to read input.");
if let Ok(n) = n.trim().parse::<i32>() {
println!("{:?}", n);
}
}
These spare you the ceremony of pattern matching without depending on extra libraries.
Probably the easiest part would be to use the text_io crate and write:
#[macro_use]
extern crate text_io;
fn main() {
// read until a whitespace and try to convert what was read into an i32
let i: i32 = read!();
println!("Read in: {}", i);
}
If you need to read more than one value simultaneously, you might need to use Rust nightly.
See also:
Is there a way to check that the user entered an integer with text_io's read!() macro?
parse is more or less the same; it’s read_line that’s unpleasant now.
use std::io;
fn main() {
let mut s = String::new();
io::stdin().read_line(&mut s).unwrap();
match s.trim_right().parse::<i32>() {
Ok(i) => println!("{} + 5 = {}", i, i + 5),
Err(_) => println!("Invalid number."),
}
}
You can create an extension method if you want a simple syntax:
use std::error::Error;
use std::io;
use std::str::FromStr;
trait Input {
fn my_read<T>(&mut self) -> io::Result<T>
where
T: FromStr,
T::Err: Error + Send + Sync + 'static;
}
impl<R> Input for R where R: io::Read {
fn my_read<T>(&mut self) -> io::Result<T>
where
T: FromStr,
T::Err: Error + Send + Sync + 'static,
{
let mut buff = String::new();
self.read_to_string(&mut buff)?;
buff.trim()
.parse()
.map_err(|e| io::Error::new(io::ErrorKind::InvalidInput, e))
}
}
// Usage:
fn main() -> io::Result<()> {
let input: i32 = io::stdin().my_read()?;
println!("{}", input);
Ok(())
}
I would definitely use the file system Rust-Lang provides std::fs (See more here: https://doc.rust-lang.org/stable/std/fs/) But more particularly https://doc.rust-lang.org/stable/std/fs/fn.read_to_string.html
Let's say you just want to read input of a text file, try this :
use std::fs
or
use std::fs::read_to_string
fn main() {
println!("{}", fs::read_to_string("input.txt"));
}
you can try this piece of code
fn main() {
let mut line = String::new();
// read input line string and store it into line
std::io::stdin().read_line(&mut line).unwrap();
// convert line to integer
let number : i32 = line.trim().parse().unwrap();
println!("Your number {}",number);
}
now you can write a function for taking user input and use it everytime like below
fn main() {
let first_number = get_input();
let second_number = get_input();
println!("Summation : {}",first_number+second_number);
}
fn get_input() -> i32{
let mut line = String::new();
std::io::stdin().read_line(&mut line).unwrap();
let number : i32 = line.trim().parse().unwrap();
return number ;
}