I'm starting to learn Rust and the rocket framework https://crates.io/crates/rocket.
I have a dumb question that I can't figure out.
How do I return my_universe that I created on the first line of main() when calling GET /universe/ports/21?
fn main() {
let my_universe = universe::model::Universe::new();
rocket::ignite().mount("/universe", routes![ports]).launch();
}
#[get("/ports/<id>")]
fn ports(id: u16) -> universe::model::Universe {
// need to return my_universe here
}
The issue I'm having is that if I define my_universe within the route controller ports(), it'll recreate the my_universe object on each request. Instead, I need the route to return the same my_universe object on each request
Sharing non-mutable state in rocket is fairly easy. You can add the state with manage during the build of rocket.
rocket::build()
.manage(my_universe) // put the shared state here
.mount("/universe", routes![ports])
If you want to return this state in a route you will have to add both serde as a dependency and the json feature of rocket.
[dependencies]
rocket = { version = "0.5.0-rc.2", features = ["json"]}
serde = "1.0.147"
You can now annotate your struct with Serialize so we can send it as a JSON response later.
#[derive(Serialize)]
struct Universe {
/* ... */
}
And access this state in your route with a &State parameter.
#[get("/ports/<id>")]
fn ports(id: u16, universe: &State<Universe>) -> Json<&Universe> {
Json(universe.inner())
}
Here we can access the inner value of the state and return it as Json.
So far, the state is immutable and can not be changed in the route which might not be what you want. Consider wrapping your state into a Mutex to allow for interior mutability.
Related
I am developing an API using the tower_web framework, and for some API calls, it would be best to just return bytes with a content type like "application/octet-stream". This data is not directly read from a file, but generated dynamically, so I can't refer to the example code for serving a static file.
For implementing the desired functionality, I tried using a http::Response<Vec<u8>> with a body using a u8 array. Here are the relevant snippets of my code, for testing purposes I used a u8 array filled with zeroes, that would later be the data I want to send to the client:
extern crate tower_web;
use tower_web::ServiceBuilder;
use http::Response as HttpResponse;
const SERVER_ADDRESS : &str = "127.0.0.1:8080";
[...]
#[derive(Clone, Debug)]
pub struct HttpApi;
impl_web! {
impl HttpApi {
[...]
#[get("/test")]
#[content_type("application/octet-stream")]
fn test(&self) -> http::Response<Vec<u8>> {
let response = HttpResponse::builder().status(200).body([0u8;16]);
println!("{:?}", response);
response
}
}
}
pub fn main() {
let addr = SERVER_ADDRESS.parse().expect("Invalid address");
println!("Listening on http://{}", addr);
ServiceBuilder::new()
.resource(HttpApi)
.run(&addr)
.unwrap();
}
However, when I try to build it using cargo, I get this error:
the trait `tower_web::codegen::futures::Future` is not implemented for `http::Response<Vec<u8>>`
I also tried setting the type returned by the function to a Result wrapping the http::Response<Vec<u8>> and (), as it would work that way when I try returning strings or json. This gave me the error
the trait `BufStream` is not implemented for `Vec<u8>`
Is there any simple way to solve this, or any other approach for returning bytes as the body of a HTTP Response in Tower Web?
I'm trying to use an actix-web server as a gateway to a small stack to guarantee a strict data format inside of the stack while allowing some freedoms for the user.
To do that, I want to deserialize a JSON string to the struct, then validate it, serialize it again and publish it on a message broker. The main part of the data is an array of arrays that contain integers, floats and datetimes. I'm using serde for deserialization and chrono to deal with datetimes.
I tried using a struct combined with an enum to allow the different types:
#[derive(Serialize, Deserialize)]
pub struct Data {
pub column_names: Option<Vec<String>>,
pub values: Vec<Vec<ValueType>>,
}
#[derive(Serialize, Deserialize)]
#[serde(untagged)]
pub enum ValueType {
I32(i32),
F64(f64),
#[serde(with = "datetime_handler")]
Dt(DateTime<Utc>),
}
Since chrono::DateTime<T> does not implement Serialize, I added a custom module for that similar to how it is described in the serde docs.
mod datetime_handler {
use chrono::{DateTime, TimeZone, Utc};
use serde::{self, Deserialize, Deserializer, Serializer};
pub fn serialize<S>(dt: &DateTime<Utc>, serializer: S) -> Result<S::Ok, S::Error>
where
S: Serializer,
{
let s = dt.to_rfc3339();
serializer.serialize_str(&s)
}
pub fn deserialize<'de, D>(deserializer: D) -> Result<DateTime<Utc>, D::Error>
where
D: Deserializer<'de>,
{
println!("Checkpoint 1");
let s = String::deserialize(deserializer)?;
println!("{}", s);
println!("Checkpoint 2");
let err1 = match DateTime::parse_from_rfc3339(&s) {
Ok(dt) => return Ok(dt.with_timezone(&Utc)),
Err(e) => Err(e),
};
println!("Checkpoint 3");
const FORMAT1: &'static str = "%Y-%m-%d %H:%M:%S";
match Utc.datetime_from_str(&s, FORMAT1) {
Ok(dt) => return Ok(dt.with_timezone(&Utc)),
Err(e) => println!("{}", e), // return first error not second if both fail
};
println!("Checkpoint 4");
return err1.map_err(serde::de::Error::custom);
}
}
This tries 2 different time formats one after the other and works for DateTime strings.
The Problem
It seems like the combination of `#[derive(Serialize, Deserialize)]`, `#[serde(untagged)]` and `#[serde(with)]` does something unexpected. `serde:from_str(...)` tries to deserialize every entry in the array with my custom `deserialize` function.
I would expect it to either try to deserialize into `ValueType::I32` first, succeed and continue with the next entry, as [the docs](https://serde.rs/enum-representations.html) say:
Serde will try to match the data against each variant in order and the first one that deserializes successfully is the one returned.
What happens is that the custom deserializeis applied to e.g. "0" fails and the deserialization stops.
What's going on? How do I solve it?
My ideas are that I either fail to deserialize in the wrong way or that I somehow "overwrite" the derived deserialize with my own.
#jonasbb helped me realize the code works when using [0,16.9,"2020-12-23 00:23:14"] but it does not when trying to deserialize ["0","16.9","2020-12-23 00:23:14"]. Serde does not serialize numbers from strings by default, the attempts for I32 and F64 just fail silently. This is discussed in this serde-issue and can be solved using the inofficial serde-aux crate.
Many crates will implement serde and other common utility crates, but will leave them as optional features. This can help save time when compiling. You can check a crate by viewing the Cargo.toml file to see if there is a feature for it or the dependency is included but marked as optional.
In your case, I can go to chrono on crates.io and select the Repository link to view the source code for the crate. In the Cargo.toml file, I can see that serde is used, but is not enabled by default.
[features]
default = ["clock", "std", "oldtime"]
alloc = []
std = []
clock = ["libc", "std", "winapi"]
oldtime = ["time"]
wasmbind = ["wasm-bindgen", "js-sys"]
unstable-locales = ["pure-rust-locales", "alloc"]
__internal_bench = []
__doctest = []
[depenencies]
...
serde = { version = "1.0.99", default-features = false, optional = true }
To enable it you can go into the Cargo.toml for your project and add it as a feature to chrono.
[depenencies]
chrono = { version: "0.4.19", features = ["serde"] }
Alternatively, chrono lists some (but not all?) of their optional features in their documentation. However, not all crates do this and docs can sometimes be out of date so I usually prefer the manual method.
As for the issue between the interaction of deserialize_with and untagged on enums, I don't see any issue with your code. It may be a bug in serde so I suggest you create an issue on the serde Repository so they can further look into why this error occurs.
I just started with Rust and I have some trouble with deserialization.
I'm actually trying to use the function ProjectDatabaseDocumentCreateDocumentCall from the following crate google_firestore1. I want to populate the field fields of the struct Document. The documentation of the struct is clear, it's expecting a HashMap<String, google_firestore1::Value> as a value.
The question is, how can I deserialize a JSON string to a HashMap<String, google_firestore1::Value> ?
Here is the code I wrote for the moment:
extern crate google_firestore1 as firestore1;
use google_firestore1::Document;
use std::collections::HashMap;
use serde_json;
pub fn go() {
let _my_doc = Document::default();
let test = "{\"test\":\"test\", \"myarray\": [1]}";
// Working perfectly fine
let _working: HashMap<String, serde_json::Value> = serde_json::from_str(test).unwrap();
// Not working
let _not_working: HashMap<String, firestore1::Value> = serde_json::from_str(test).unwrap();
// Later I want to do the following
// _my_doc.fields = _not_working
}
Obvsiouly this is not working, and it crashes with the following error.
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Error("invalid type: string \"test\", expected struct Value", line: 1, column: 14)', src/firestore.rs:17:85
stack backtrace:
Of course, I noticed that serde_json::Value and firestore1::Value are not the same Struct.
But I gave a look at the source code and it seems that firestore1::Value is implementing the Deserialize trait.
So why is it not working ? In this case, do I need to iterate over the first HashMap and deserialize serde_json::Value to firestore1::Value again ? Is there a cleaner way to do what I want ?
Thanks for your answer !
The definition of the firestore1::Value is:
/// A message that can hold any of the supported value types.
///
/// This type is not used in any activity, and only used as *part* of another schema.
///
#[derive(Default, Clone, Debug, Serialize, Deserialize)]
pub struct Value {
/// A bytes value.
///
/// Must not exceed 1 MiB - 89 bytes.
/// Only the first 1,500 bytes are considered by queries.
#[serde(rename="bytesValue")]
pub bytes_value: Option<String>,
/// A timestamp value.
///
/// Precise only to microseconds. When stored, any additional precision is
/// rounded down.
#[serde(rename="timestampValue")]
pub timestamp_value: Option<String>,
...
}
This means each entry for a firestore1::Value must be an object.
I suspect that only one of the fields would actually be set, corresponding
to the actual type of the value (as they're all optional).
So your json would need to be something like:
let test = r#"{
"test":{"stringValue":"test"},
"myarray": {
"arrayValue":{"values":[{"integerValue":1}]}
}
}"#;
This is pretty ugly, so if you're doing a lot of your own JSON to firestore conversations, I'd probably write some helpers to convert from the serde_json::Value to firestore1::Value.
It would probably look something like this:
fn my_firestore_from_json(v:serde_json::Value) -> firestore1::Value {
match v {
serde_json::Value::Null => firestore::Value {
// I don't know why this is a Option<String>
null_value: Some("".to_string),
..Default::default(),
},
serde_json::Value::Bool(b) => firestore::Value {
bool_value: Some(b),
..Default::default(),
},
// Implement this
serde_json::Value::Number(n) => my_firestore_number(n),
serde_json::Value::String(s) => firestore::Value {
string_value: Some(s),
..Default::default(),
},
serde_json::Value::Array(v) => firestore::Value {
array_value:
Some(firestore1::ArrayValue{
values:v.into_iter().map(my_firestore_from_json)
}),
..Default::default(),
},
// Implement this
serde_json::Value::Object(d) => my_firststore_object(/* something */)
}
}
This would be a bit neater if there were various implementations of From<T> for the firestore1::Value, but using the implementation of
Default makes this not too ugly.
It is also worth noting that not all firebase types are created here,
since the types expressed in serde_json are different from those supported by firebase.
Anyway this allows you to use your JSON as written by doing something like:
let test = "{\"test\":\"test\", \"myarray\": [1]}";
let working: HashMap<String, serde_json::Value> = serde_json::from_str(test).unwrap();
let value_map: HashMap<String, firestore1::Value> = working.iter().map(|(k,v)| (k, my_firestore_from_json(v)).collect();
I understand that in Kotlin there is no such thing as "Non-local variables" or "Global Variables" I am looking for a way to modify variables in another "Scope" in Kotlin by using the function below:
class Listres(){
var listsize = 0
fun gatherlistresult(){
var listallinfo = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
listallinfo.addOnSuccessListener {
listResult -> listsize += listResult.items.size
}
}
}
the value of listsize is always 0 (logging the result from inside of the .addOnSuccessListener scope returns 8) so clearly the listsize variable isn't being modified. I have seen many different posts about this topic on other sites , but none fit my usecase.
I simply want to modify listsize inside of the .addOnSuccessListener callback
This method will always be returned 0 as the addOnSuccessListener() listener will be invoked after the method execution completed. The addOnSuccessListener() is a callback method for asynchronous operation and you will get the value if it gives success only.
You can get the value by changing the code as below:
class Demo {
fun registerListResult() {
var listallinfo = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
listallinfo.addOnSuccessListener {
listResult -> listsize += listResult.items.size
processResult(listsize)
}
listallinfo.addOnFailureListener {
// Uh-oh, an error occurred!
}
}
fun processResult(listsize: Int) {
print(listResult+"") // you will get the 8 here as you said
}
}
What you're looking for is a way to bridge some asynchronous processing into a synchronous context. If possible it's usually better (in my opinion) to stick to one model (sync or async) throughout your code base.
That being said, sometimes these circumstances are out of our control. One approach I've used in similar situations involves introducing a BlockingQueue as a data pipe to transfer data from the async context to the sync context. In your case, that might look something like this:
class Demo {
var listSize = 0
fun registerListResult() {
val listAll = FirebaseStorage.getInstance()
.getReference()
.child("MainTimeline/")
.listAll()
val dataQueue = ArrayBlockingQueue<Int>(1)
listAll.addOnSuccessListener { dataQueue.put(it.items.size) }
listSize = dataQueue.take()
}
}
The key points are:
there is a blocking variant of the Queue interface that will be used to pipe data from the async context (listener) into the sync context (calling code)
data is put() on the queue within the OnSuccessListener
the calling code invokes the queue's take() method, which will cause that thread to block until a value is available
If that doesn't work for you, hopefully it will at least inspire some new thoughts!
Consider the pattern where there are several states registered with a dispatcher and each state knows what state to transition to when it receives an appropriate event. This is a simple state transition pattern.
struct Dispatcher {
states: HashMap<Uid, Rc<RefCell<State>>>,
}
impl Dispatcher {
pub fn insert_state(&mut self, state_id: Uid, state: Rc<RefCell<State>>) -> Option<Rc<RefCell<State>>> {
self.states.insert(state_id, state)
}
fn dispatch(&mut self, state_id: Uid, event: Event) {
if let Some(mut state) = states.get_mut(&state_id).cloned() {
state.handle_event(self, event);
}
}
}
trait State {
fn handle_event(&mut self, &mut Dispatcher, Event);
}
struct S0 {
state_id: Uid,
move_only_field: Option<MOF>,
// This is pattern that concerns me.
}
impl State for S0 {
fn handle_event(&mut self, dispatcher: &mut Dispatcher, event: Event) {
if event == Event::SomeEvent {
// Do some work
if let Some(mof) = self.mof.take() {
let next_state = Rc::new(RefCell::new(S0 {
state_id: self.state_id,
move_only_field: mof,
}));
let _ = dispatcher.insert(self.state_id, next_state);
} else {
// log an error: BUGGY Logic somewhere
let _ = dispatcher.remove_state(&self.state_id);
}
} else {
// Do some other work, maybe transition to State S2 etc.
}
}
}
struct S1 {
state_id: Uid,
move_only_field: MOF,
}
impl State for S1 {
fn handle_event(&mut self, dispatcher: &mut Dispatcher, event: Event) {
// Do some work, maybe transition to State S2/S3/S4 etc.
}
}
With reference to the inline comment above saying:
// This is pattern that concerns me.
S0::move_only_field needs to be an Option in this pattern because self is borrowed in handle_event, but I am not sure that this is best way to approach it.
Here are the ways I can think of with demerits of each one:
Put it into an Option as I have done: this feels hacky and every time I need
to check the invariant that the Option is always Some otherwise
panic! or make it a NOP with if let Some() = and ignore
the else clause, but this causes code-bloat. Doing an unwrap
or bloating the code with if let Some() feels a bit off.
Get it into a shared ownership Rc<RefCell<>>: Need to heap allocate
all such variables or construct another struct called Inner or
something that has all these non-clonable types and put that into an
Rc<RefCell<>>.
Pass stuff back to Dispatcher indicating it to basically remove us
from the map and then move things out of us to the next State which
will also be indicated via our return value: Too much coupling,
breaks OOP, does not scale as Dispatcher needs to know about all the
States and needs frequent updating. I don't think this is a good
paradigm, but could be wrong.
Implement Default for MOF above: Now we can mem::replace it with
the default while moving out the old value. The burden of panicking OR
returning an error OR doing a NOP is now hidden in implementation of
MOF. The problem here is we don't always have the access to MOF
type and for those that we do, it again takes the point of bloat
from user code to the code of MOF.
Let the function handle_event take self by move as fn handle_event(mut self, ...) -> Option<Self>: Now instead of Rc<RefCell<>> you will need to have Box<State> and move it out each time in the dispatcher and if the return is Some you put it back. This almost feels like a sledgehammer and makes many other idioms impossible, for instance if I wanted to share self further in some registered closure/callback I would normally put a Weak<RefCell<>> previously but now sharing self in callbacks etc is impossible.
Are there any other options? Is there any that is considered the "most idiomatic" way of doing this in Rust?
Let the function handle_event take self by move as fn handle_event(mut self, ...) -> Option<Self>: Now instead of Rc<RefCell<>> you will need to have Box<State> and move it out each time in the dispatcher and if the return is Some you put it back.
This is what I would do. However, you don't need to switch from Rc to Box if there is only one strong reference: Rc::try_unwrap can move out of an Rc.
Here's part of how you might rewrite Dispatcher:
struct Dispatcher {
states: HashMap<Uid, Rc<State>>,
}
impl Dispatcher {
fn dispatch(&mut self, state_id: Uid, event: Event) {
if let Some(state_ref) = self.states.remove(&state_id) {
let state = state_ref.try_unwrap()
.expect("Unique strong reference required");
if let Some(next_state) = state.handle_event(event) {
self.states.insert(state_id, next_state);
}
} else {
// handle state_id not found
}
}
}
(Note: dispatch takes state_id by value. In the original version, this wasn't necessary -- it could have been changed to pass by reference. In this version, it is necessary, since state_id gets passed to HashMap::insert. It looks like Uid is Copy though, so it makes little difference.)
It's not clear whether state_id actually needs to be a member of the struct that implements State anymore, since you don't need it inside handle_event -- all the insertion and removal happens inside impl Dispatcher, which makes sense and reduces coupling between State and Dispatcher.
impl State for S0 {
fn handle_event(self, event: Event) -> Option<Rc<State>> {
if event == Event::SomeEvent {
// Do some work
let next_state = Rc::new(S0 {
state_id: self.state_id,
move_only_field: self.mof,
});
Some(next_state)
} else {
// Do some other work
}
}
}
Now you don't have to handle a weird, should-be-impossible corner case where the Option is None.
This almost feels like a sledgehammer and makes many other idioms impossible, for instance if I wanted to share self further in some registered closure/callback I would normally put a Weak<RefCell<>> previously but now sharing self in callbacks etc is impossible.
Because you can move out of an Rc if you have the only strong reference, you don't have to sacrifice this technique.
"Feels like a sledgehammer" might be subjective, but to me, what a signature like fn handle_event(mut self, ...) -> Option<Self> does is encode an invariant. With the original version, each impl State for ... had to know when to insert and remove itself from the dispatcher, and whether it did or not was uncheckable. For example, if somewhere deep in the logic you forgot to call dispatcher.insert(state_id, next_state), the state machine wouldn't transition, and might get stuck or worse. When handle_event takes self by-value, that's not possible anymore -- you have to return the next state, or the code simply won't compile.
(Aside: both the original version and mine do at least two hashtable lookups each time dispatch is called: once to get the current state, and again to insert the new state. If you wanted to get rid of the second lookup, you could combine approaches: store Option<Rc<State>> in the HashMap, and take from the Option instead of removing it from the map entirely.)