I am developing an API using the tower_web framework, and for some API calls, it would be best to just return bytes with a content type like "application/octet-stream". This data is not directly read from a file, but generated dynamically, so I can't refer to the example code for serving a static file.
For implementing the desired functionality, I tried using a http::Response<Vec<u8>> with a body using a u8 array. Here are the relevant snippets of my code, for testing purposes I used a u8 array filled with zeroes, that would later be the data I want to send to the client:
extern crate tower_web;
use tower_web::ServiceBuilder;
use http::Response as HttpResponse;
const SERVER_ADDRESS : &str = "127.0.0.1:8080";
[...]
#[derive(Clone, Debug)]
pub struct HttpApi;
impl_web! {
impl HttpApi {
[...]
#[get("/test")]
#[content_type("application/octet-stream")]
fn test(&self) -> http::Response<Vec<u8>> {
let response = HttpResponse::builder().status(200).body([0u8;16]);
println!("{:?}", response);
response
}
}
}
pub fn main() {
let addr = SERVER_ADDRESS.parse().expect("Invalid address");
println!("Listening on http://{}", addr);
ServiceBuilder::new()
.resource(HttpApi)
.run(&addr)
.unwrap();
}
However, when I try to build it using cargo, I get this error:
the trait `tower_web::codegen::futures::Future` is not implemented for `http::Response<Vec<u8>>`
I also tried setting the type returned by the function to a Result wrapping the http::Response<Vec<u8>> and (), as it would work that way when I try returning strings or json. This gave me the error
the trait `BufStream` is not implemented for `Vec<u8>`
Is there any simple way to solve this, or any other approach for returning bytes as the body of a HTTP Response in Tower Web?
Related
I'm starting to learn Rust and the rocket framework https://crates.io/crates/rocket.
I have a dumb question that I can't figure out.
How do I return my_universe that I created on the first line of main() when calling GET /universe/ports/21?
fn main() {
let my_universe = universe::model::Universe::new();
rocket::ignite().mount("/universe", routes![ports]).launch();
}
#[get("/ports/<id>")]
fn ports(id: u16) -> universe::model::Universe {
// need to return my_universe here
}
The issue I'm having is that if I define my_universe within the route controller ports(), it'll recreate the my_universe object on each request. Instead, I need the route to return the same my_universe object on each request
Sharing non-mutable state in rocket is fairly easy. You can add the state with manage during the build of rocket.
rocket::build()
.manage(my_universe) // put the shared state here
.mount("/universe", routes![ports])
If you want to return this state in a route you will have to add both serde as a dependency and the json feature of rocket.
[dependencies]
rocket = { version = "0.5.0-rc.2", features = ["json"]}
serde = "1.0.147"
You can now annotate your struct with Serialize so we can send it as a JSON response later.
#[derive(Serialize)]
struct Universe {
/* ... */
}
And access this state in your route with a &State parameter.
#[get("/ports/<id>")]
fn ports(id: u16, universe: &State<Universe>) -> Json<&Universe> {
Json(universe.inner())
}
Here we can access the inner value of the state and return it as Json.
So far, the state is immutable and can not be changed in the route which might not be what you want. Consider wrapping your state into a Mutex to allow for interior mutability.
I'm trying to use an actix-web server as a gateway to a small stack to guarantee a strict data format inside of the stack while allowing some freedoms for the user.
To do that, I want to deserialize a JSON string to the struct, then validate it, serialize it again and publish it on a message broker. The main part of the data is an array of arrays that contain integers, floats and datetimes. I'm using serde for deserialization and chrono to deal with datetimes.
I tried using a struct combined with an enum to allow the different types:
#[derive(Serialize, Deserialize)]
pub struct Data {
pub column_names: Option<Vec<String>>,
pub values: Vec<Vec<ValueType>>,
}
#[derive(Serialize, Deserialize)]
#[serde(untagged)]
pub enum ValueType {
I32(i32),
F64(f64),
#[serde(with = "datetime_handler")]
Dt(DateTime<Utc>),
}
Since chrono::DateTime<T> does not implement Serialize, I added a custom module for that similar to how it is described in the serde docs.
mod datetime_handler {
use chrono::{DateTime, TimeZone, Utc};
use serde::{self, Deserialize, Deserializer, Serializer};
pub fn serialize<S>(dt: &DateTime<Utc>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let s = dt.to_rfc3339();
serializer.serialize_str(&s)
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<DateTime<Utc>, D::Error>
where
D: Deserializer<'de>,
{
println!("Checkpoint 1");
let s = String::deserialize(deserializer)?;
println!("{}", s);
println!("Checkpoint 2");
let err1 = match DateTime::parse_from_rfc3339(&s) {
Ok(dt) => return Ok(dt.with_timezone(&Utc)),
Err(e) => Err(e),
};
println!("Checkpoint 3");
const FORMAT1: &'static str = "%Y-%m-%d %H:%M:%S";
match Utc.datetime_from_str(&s, FORMAT1) {
Ok(dt) => return Ok(dt.with_timezone(&Utc)),
Err(e) => println!("{}", e), // return first error not second if both fail
};
println!("Checkpoint 4");
return err1.map_err(serde::de::Error::custom);
}
}
This tries 2 different time formats one after the other and works for DateTime strings.
The Problem
It seems like the combination of `#[derive(Serialize, Deserialize)]`, `#[serde(untagged)]` and `#[serde(with)]` does something unexpected. `serde:from_str(...)` tries to deserialize every entry in the array with my custom `deserialize` function.
I would expect it to either try to deserialize into `ValueType::I32` first, succeed and continue with the next entry, as [the docs](https://serde.rs/enum-representations.html) say:
Serde will try to match the data against each variant in order and the first one that deserializes successfully is the one returned.
What happens is that the custom deserializeis applied to e.g. "0" fails and the deserialization stops.
What's going on? How do I solve it?
My ideas are that I either fail to deserialize in the wrong way or that I somehow "overwrite" the derived deserialize with my own.
#jonasbb helped me realize the code works when using [0,16.9,"2020-12-23 00:23:14"] but it does not when trying to deserialize ["0","16.9","2020-12-23 00:23:14"]. Serde does not serialize numbers from strings by default, the attempts for I32 and F64 just fail silently. This is discussed in this serde-issue and can be solved using the inofficial serde-aux crate.
Many crates will implement serde and other common utility crates, but will leave them as optional features. This can help save time when compiling. You can check a crate by viewing the Cargo.toml file to see if there is a feature for it or the dependency is included but marked as optional.
In your case, I can go to chrono on crates.io and select the Repository link to view the source code for the crate. In the Cargo.toml file, I can see that serde is used, but is not enabled by default.
[features]
default = ["clock", "std", "oldtime"]
alloc = []
std = []
clock = ["libc", "std", "winapi"]
oldtime = ["time"]
wasmbind = ["wasm-bindgen", "js-sys"]
unstable-locales = ["pure-rust-locales", "alloc"]
__internal_bench = []
__doctest = []
[depenencies]
...
serde = { version = "1.0.99", default-features = false, optional = true }
To enable it you can go into the Cargo.toml for your project and add it as a feature to chrono.
[depenencies]
chrono = { version: "0.4.19", features = ["serde"] }
Alternatively, chrono lists some (but not all?) of their optional features in their documentation. However, not all crates do this and docs can sometimes be out of date so I usually prefer the manual method.
As for the issue between the interaction of deserialize_with and untagged on enums, I don't see any issue with your code. It may be a bug in serde so I suggest you create an issue on the serde Repository so they can further look into why this error occurs.
I am trying to port Ethereum DeFi contracts into Solana's Rust programs...
I have learned about saving a struct or an array in programs' account data, but still do not know how to save a HashMap<address in string, amount in u64> into a program's account data...
Then how to read this HashMap's values like checking each address' staked amount.
Please help. Thank you!
My Solana Rust program:
pub fn get_init_hashmap() -> HashMap<&'static str, u64> {
let mut staked_amount: HashMap<&str, u64> = HashMap::new();
staked_amount.insert("9Ao3CgcFg3RB2...", 0);
staked_amount.insert("8Uyuz5PUS47GB...", 0);
staked_amount.insert("CRURHng6s7DGR...", 0);
staked_amount
}
pub fn process_instruction(...) -> ProgramResult {
msg!("about to decode account data");
let acct_data_decoded = match HashMap::try_from_slice(&account.data.borrow_mut()) {
Ok(data) => data,//to be of type `HashMap`
Err(err) => {
if err.kind() == InvalidData {
msg!("InvalidData so initializing account data");
get_init_hashmap()
} else {
panic!("Unknown error decoding account data {:?}", err)
}
}
};
msg!("acct_data_decoded: {:?}", acct_data_decoded);
Solana doesn't expose a HashMap like that. In Solidity, it is common to have a top-level HashMap that tracks addresses to user values.
On Solana a common pattern to replace it would be to use PDAs (Program derived addresses). You can Hash user SOL wallet to ensure unique PDAs and then iterate over them using an off-chain crank.
Answered by Solana dev support on Discord:
HashMap doesnt work on-chain at the moment, so you'll have to use BTreeMap.
As for actually saving it, you can iterate through the key-value pairs and serializing each of those.
In general though, we suggest using mutiple accounts, and program-derived addresses for each account: https://docs.solana.com/developing/programming-model/calling-between-programs#program-derived-addresses
I just started with Rust and I have some trouble with deserialization.
I'm actually trying to use the function ProjectDatabaseDocumentCreateDocumentCall from the following crate google_firestore1. I want to populate the field fields of the struct Document. The documentation of the struct is clear, it's expecting a HashMap<String, google_firestore1::Value> as a value.
The question is, how can I deserialize a JSON string to a HashMap<String, google_firestore1::Value> ?
Here is the code I wrote for the moment:
extern crate google_firestore1 as firestore1;
use google_firestore1::Document;
use std::collections::HashMap;
use serde_json;
pub fn go() {
let _my_doc = Document::default();
let test = "{\"test\":\"test\", \"myarray\": [1]}";
// Working perfectly fine
let _working: HashMap<String, serde_json::Value> = serde_json::from_str(test).unwrap();
// Not working
let _not_working: HashMap<String, firestore1::Value> = serde_json::from_str(test).unwrap();
// Later I want to do the following
// _my_doc.fields = _not_working
}
Obvsiouly this is not working, and it crashes with the following error.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("invalid type: string \"test\", expected struct Value", line: 1, column: 14)', src/firestore.rs:17:85
stack backtrace:
Of course, I noticed that serde_json::Value and firestore1::Value are not the same Struct.
But I gave a look at the source code and it seems that firestore1::Value is implementing the Deserialize trait.
So why is it not working ? In this case, do I need to iterate over the first HashMap and deserialize serde_json::Value to firestore1::Value again ? Is there a cleaner way to do what I want ?
Thanks for your answer !
The definition of the firestore1::Value is:
/// A message that can hold any of the supported value types.
///
/// This type is not used in any activity, and only used as *part* of another schema.
///
#[derive(Default, Clone, Debug, Serialize, Deserialize)]
pub struct Value {
/// A bytes value.
///
/// Must not exceed 1 MiB - 89 bytes.
/// Only the first 1,500 bytes are considered by queries.
#[serde(rename="bytesValue")]
pub bytes_value: Option<String>,
/// A timestamp value.
///
/// Precise only to microseconds. When stored, any additional precision is
/// rounded down.
#[serde(rename="timestampValue")]
pub timestamp_value: Option<String>,
...
}
This means each entry for a firestore1::Value must be an object.
I suspect that only one of the fields would actually be set, corresponding
to the actual type of the value (as they're all optional).
So your json would need to be something like:
let test = r#"{
"test":{"stringValue":"test"},
"myarray": {
"arrayValue":{"values":[{"integerValue":1}]}
}
}"#;
This is pretty ugly, so if you're doing a lot of your own JSON to firestore conversations, I'd probably write some helpers to convert from the serde_json::Value to firestore1::Value.
It would probably look something like this:
fn my_firestore_from_json(v:serde_json::Value) -> firestore1::Value {
match v {
serde_json::Value::Null => firestore::Value {
// I don't know why this is a Option<String>
null_value: Some("".to_string),
..Default::default(),
},
serde_json::Value::Bool(b) => firestore::Value {
bool_value: Some(b),
..Default::default(),
},
// Implement this
serde_json::Value::Number(n) => my_firestore_number(n),
serde_json::Value::String(s) => firestore::Value {
string_value: Some(s),
..Default::default(),
},
serde_json::Value::Array(v) => firestore::Value {
array_value:
Some(firestore1::ArrayValue{
values:v.into_iter().map(my_firestore_from_json)
}),
..Default::default(),
},
// Implement this
serde_json::Value::Object(d) => my_firststore_object(/* something */)
}
}
This would be a bit neater if there were various implementations of From<T> for the firestore1::Value, but using the implementation of
Default makes this not too ugly.
It is also worth noting that not all firebase types are created here,
since the types expressed in serde_json are different from those supported by firebase.
Anyway this allows you to use your JSON as written by doing something like:
let test = "{\"test\":\"test\", \"myarray\": [1]}";
let working: HashMap<String, serde_json::Value> = serde_json::from_str(test).unwrap();
let value_map: HashMap<String, firestore1::Value> = working.iter().map(|(k,v)| (k, my_firestore_from_json(v)).collect();
I'm learning Rust and trying to solve some basic algorithm problems with it. In many cases, I want to read lines from stdin, perform some transformation on each line and return a vector of resulting items. One way I did this was like this:
// Fully working Rust code
let my_values: Vec<u32> = stdin
.lock()
.lines()
.filter_map(Result::ok)
.map(|line| line.parse::<u32>())
.filter_map(Result::ok)
.map(|x|x*2) // For example
.collect();
This works but of course silently ignores any errors that may occur. Now what I woud like to do is something along the lines of:
// Pseudo-ish code
let my_values: Result<Vec<u32>, X> = stdin
.lock()
.lines() // Can cause std::io::Error
.map(|line| line.parse::<u32>()) // Can cause std::num::ParseIntError
.map(|x| x*2)
.collect();
Where X is some kind of error type that I can match on afterwards. Preferably I want to perform the whole operation on one line at a time and immediately discard the string data after it has been parsed to an int.
I think I need to create some kind of Enum type to hold the various possible errors, possibly like this:
#[derive(Debug)]
enum InputError {
Io(std::io::Error),
Parse(std::num::ParseIntError),
}
However, I don't quite understand how to put everything together to make it clean and avoid having to explicitly match and cast everywhere. Also, is there some way to automatically create these enum error types or do I have to explicilty enumerate them every time I do this?
You're on the right track.
The way I'd approach this is by using the enum you've defined,
then add implementations of From for the error types you're interested in.
That will allow you to use the ? operator on your maps to get the kind of behaviour you want.
#[derive(Debug)]
enum MyError {
IOError(std::io::Error),
ParseIntError(std::num::ParseIntError),
}
impl From<std::io::Error> for MyError {
fn from(e:std::io::Error) -> MyError {
return MyError::IOError(e)
}
}
impl From<std::num::ParseIntError> for MyError {
fn from(e:std::num::ParseIntError) -> MyError {
return MyError::ParseIntError(e)
}
}
Then you can implement the actual transform as either
let my_values: Vec<_> = stdin
.lock()
.lines()
.map(|line| -> Result<u32,MyError> { Ok(line?.parse::<u32>()?*2) } )
.collect();
which will give you one entry for each input, like: {Ok(x), Err(MyError(x)), Ok(x)}.
or you can do:
let my_values: Result<Vec<_>,MyError> = stdin
.lock()
.lines()
.map(|line| -> Result<u32,MyError> { Ok(line?.parse::<u32>()?*2) } )
.collect();
Which will give you either Err(MyError(...)) or Ok([1,2,3])
Note that you can further reduce some of the error boilerplate by using an error handling crate like snafu, but in this case it's not too much.