Noticed that related API deprecated from v0.1.12
https://docs.rs/tokio/0.1.12/tokio/net/struct.TcpStream.html
pub fn set_keepalive(&self, keepalive: Option<Duration>) -> Result<(), Error>
In newest version, https://docs.rs/tokio/1.21.2/tokio/net/struct.TcpStream.html#method.set_keepalive
How can I set it?
use socket2::{SockRef, TcpKeepalive};
...
info!("connecting to {addr}");
let tcp = TcpStream::connect(&addr).await?;
info!("tcp connected to {addr}");
let ka = TcpKeepalive::new().with_time(std::time::Duration::from_secs(30));
let sf = SockRef::from(&tcp);
sf.set_tcp_keepalive(&ka)?;
...
Related
I'm experimenting with Rocket, Rust and SQLx and I'd like to test what happens when two parallel transactions try to insert a duplicated record on my table.
My insert fn contains nothing special and it works fine:
async fn insert_credentials<'ex, EX>(&self, executor: EX, credentials: &Credentials) -> Result<u64, Errors>
where
EX: 'ex + Executor<'ex, Database = Postgres>,
{
sqlx::query!(
r#"INSERT INTO credentials (username, password)
VALUES ($1, crypt($2, gen_salt('bf')))"#,
credentials.username,
credentials.password,
)
.execute(executor)
.await
.map(|result| result.rows_affected())
.map_err(|err| err.into())
}
My test, though, hangs indefinitely since it waits for a commit that never happens:
#[async_std::test]
async fn it_should_reject_duplicated_username_in_parallel() {
let repo = new_repo();
let db: Pool<Postgres> = connect().await;
let credentials = new_random_credentials();
println!("TX1 begins");
let mut tx1 = db.begin().await.unwrap();
let rows_affected = repo.insert_credentials(&mut tx1, &credentials).await.unwrap();
assert_eq!(rows_affected, 1);
println!("TX2 begins");
let mut tx2 = db.begin().await.unwrap();
println!("It hangs on the next line");
let rows_affected = repo.insert_credentials(&mut tx2, &credentials).await.unwrap();
assert_eq!(rows_affected, 1);
println!("It never reaches this line");
tx1.commit().await.unwrap();
tx2.commit().await.unwrap();
}
How do I create and execute those TXs in parallel, such that the assertions pass but the test fails when trying to commit the second TX?
For reference, this is my Cargo.toml
[package]
name = "auth"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
async-trait = "0.1.52"
serde = "1.0.136"
thiserror = "1.0.30"
# TODO https://github.com/SergioBenitez/Rocket/issues/1893#issuecomment-1002393878
rocket = { git = "https://github.com/SergioBenitez/Rocket", features = ["json"] }
[dependencies.redis]
version = "0.21.5"
features = ["tokio-comp"]
[dependencies.sqlx]
version = "0.5.11"
features = ["macros", "runtime-tokio-rustls", "postgres"]
[dependencies.uuid]
version = "1.0.0-alpha.1"
features = ["v4", "fast-rng", "macro-diagnostics"]
## DEV ##
[dev-dependencies]
mockall = "0.11.0"
[dev-dependencies.async-std]
version = "1.11.0"
features = ["attributes", "tokio1"]
You can use a async_std::future::timeout or tokio::time::timeout. Example using async_std:
use async_std::future;
use std::time::Duration;
let max_duration = Duration::from_millis(100);
assert!(timeout(max_duration, tx2.commit()).await.is_err());
If you want to continue to tx2 before completing tx1, you can async_std::task::spawn or tokio::spawn the tx1 first:
async_std::task::spawn(async move {
assert!(tx1.commit().await.is_ok());
});
#Mika pointed me the right direction, I could spawn both transactions and add a bit of timeout to give the concurrent TXs some time to execute.
let handle1 = tokio::spawn(async move {
let repo = new_repo();
let mut tx = db1.begin().await.unwrap();
let rows_affected = repo.insert_credentials(&mut tx, &credentials1).await.unwrap();
assert_eq!(rows_affected, 1);
tokio::time::sleep(Duration::from_millis(100)).await;
tx.commit().await.unwrap()
});
let handle2 = tokio::spawn(async move {
let repo = new_repo();
let mut tx = db2.begin().await.unwrap();
let rows_affected = repo.insert_credentials(&mut tx, &credentials2).await.unwrap();
assert_eq!(rows_affected, 1);
tokio::time::sleep(Duration::from_millis(100)).await;
tx.commit().await.unwrap()
});
let (_first, _second) = rocket::tokio::try_join!(handle1, handle2).unwrap();
I thought this way both TXs would execute in parallel until the sleep line, then one would commit and the other one would fail on the commit line. But no, actually both TXs execute in parallel, TX1 runs until the sleep and TX2 blocks on the insert line until TX1 commits, then TX2 fails on the insert line.
I guess that's just how DB works on this case and maybe I could change that by messing with TX isolation, but that's not my intent here. I'm just playing to learn more, and that's enough learning for today :)
Akka - Discriminated Unions as messages in F#
I am unable to use discriminated unions as messages to akka actors. If anyone can point me at an example that does this, it would be much appreciated.
My own attempt at this is at git#github.com:Tweega/AkkaMessageIssue.git. (snippets below). It is a cutdown version of a sample found at https://github.com/rikace/AkkaActorModel.git (Chat project)
Problem
The DU message never finds its target on the server actor, but is sent to the deadletter box. If I send Objects, instead, they do arrive.
If I send a DU, but set my server actor to listen for generic Objects, the message does arrive, but its type is
seq [seq [seq []]
and I can't get at underlying DU.
The DU I am trying to send as message
type PrinterJob =
| PrintThis of string
| Teardown
The client code
let system = System.create "MyClient" config
let chatClientActor =
spawn system "ChatClient" <| fun mailbox ->
let server = mailbox.Context.ActorSelection("akka.tcp://MyServer#localhost:8081/user/ChatServer")
let rec loop nick = actor {
let! (msg:PrinterJob) = mailbox.Receive()
server.Tell(msg)
return! loop nick
}
loop ""
while true do
let input = Console.ReadLine()
chatClientActor.Tell(PrintThis(input))
Messages are forwarded to the client from console input
while true do
let input = Console.ReadLine()
chatClientActor.Tell(PrintThis(input))
The server code
let system = System.create "MyServer" config
let chatServerActor =
spawn system "ChatServer" <| fun (mailbox:Actor<_>) ->
let rec loop (clients:Akka.Actor.IActorRef list) = actor {
let! (msg:PrinterJob) = mailbox.Receive()
printfn "Received %A" msg //Received seq [seq [seq []]; seq [seq [seq []]]] ???
match msg with
| PrintThis str ->
Console.WriteLine("Printing: {0} Do we get this?", str)
return! loop clients
| Teardown ->
Console.WriteLine("Tearing down now")
return! loop clients
}
loop []
Dependencies
(I am not using paket here) - PM commands below:
Install-Package Akka -Version 1.4.23
Install-Package Akka.Remote -Version 1.4.23
Install-Package Akka.FSharp -Version 1.4.23
I am hosting the application in net5.0
Constructor argument names - oddity?
When passing in class instances as objects, akka seems to be sensitive to the name of constructor parameters. The message gets handled, but the data is not copied across from client to server. If you have a property called Username, the constructor parameter cannot be, for example, uName, otherwise its value is null when it reaches the server. Code for this is in branch params.
type DoesWork(montelimar: string) =
member x.Montelimar = montelimar
type DoesNotWork(montelimaro: string) =
member x.Montelimar = montelimaro
I opened an issue in the Akka.NET repository: https://github.com/akkadotnet/akka.net/issues/5194
And added a detailed reproduction for this: https://github.com/akkadotnet/akka.net/pull/5196
But it looks like Newtonsoft.Json really can't perform this deserialization without being given a type hint, which Akka.NET's network serialization does not do by default for JSON:
type TestUnion =
| A of string
| B of int * string
type TestUnion2 =
| C of string * TestUnion
| D of int
[<Fact(Skip="JSON.NET really does not support even basic DU serialization")>]
member _.``JSON.NET must serialize DUs`` () =
let du = C("a-11", B(11, "a-12"))
let settings = new JsonSerializerSettings()
settings.Converters.Add(new DiscriminatedUnionConverter())
let serialized = JsonConvert.SerializeObject(du, settings)
let deserialized = JsonConvert.DeserializeObject(serialized, settings)
Assert.Equal(du :> obj, deserialized)
That test will not pass and it doesn't use any of Akka.NET's infrastructure at all - so the default JSON serializer simply won't work for real-world F# use cases.
We can try changing the defaults of our serialization system to include a type hint, but that will take a lot of validation testing (for old Akka.Persistence data serialized without one).
A better solution, which my pull request validates, is to use Hyperion for polymorphic serialization instead - it will be similarly transparent to you but it has much more robust handling for complex types than Newtonsoft.Json and is actually faster: https://getakka.net/articles/networking/serialization.html#how-to-setup-hyperion-as-default-serializer
How do I pass an authentication callback to git2::Repository::clone()? (set_remote_callbacks sets up the callbacks).
I have some code like the following:
let mut cb = git2::RemoteCallbacks::new();
Self::set_remote_callbacks(&mut cb);
let rr = Repository::clone(url, path.to_str().ok_or("bad string".to_string())?);
What I want is like, as an example, when I do I fetch, I do this, which passes my callbacks to the fetch:
let mut fetchOptions = FetchOptions::new();
let mut cb = git2::RemoteCallbacks::new();
Self::set_remote_callbacks(&mut cb);
fetchOptions.remote_callbacks(cb);
let mut remote = self.repo.find_remote(remote)?;
remote.fetch(&[branch], Some(&mut fetchOptions), None)?;
Use git2::build::RepoBuilder.
Credit goes to issue 329 on the git2 issue tracker.
We are on: akka-stream-experimental_2.11 1.0.
Inspired by the example
We wrote a TCP receiver as follows:
def bind(address: String, port: Int, target: ActorRef)
(implicit system: ActorSystem, actorMaterializer: ActorMaterializer): Future[ServerBinding] = {
val sink = Sink.foreach[Tcp.IncomingConnection] { conn =>
val serverFlow = Flow[ByteString]
.via(Framing.delimiter(ByteString("\n"), maximumFrameLength = 256, allowTruncation = true))
.map(message => {
target ? new Message(message); ByteString.empty
})
conn handleWith serverFlow
}
val connections = Tcp().bind(address, port)
connections.to(sink).run()
}
However, our intention was to have the receiver not respond at all and only sink the message. (The TCP message publisher does not care about response ).
Is it even possible? to not respond at all since akka.stream.scaladsl.Tcp.IncomingConnection takes a flow of type: Flow[ByteString, ByteString, Unit]
If yes, some guidance will be much appreciated. Thanks in advance.
One attempt as follows passes my unit tests but not sure if its the best idea:
def bind(address: String, port: Int, target: ActorRef)
(implicit system: ActorSystem, actorMaterializer: ActorMaterializer): Future[ServerBinding] = {
val sink = Sink.foreach[Tcp.IncomingConnection] { conn =>
val targetSubscriber = ActorSubscriber[Message](system.actorOf(Props(new TargetSubscriber(target))))
val targetSink = Flow[ByteString]
.via(Framing.delimiter(ByteString("\n"), maximumFrameLength = 256, allowTruncation = true))
.map(Message(_))
.to(Sink(targetSubscriber))
conn.flow.to(targetSink).runWith(Source(Promise().future))
}
val connections = Tcp().bind(address, port)
connections.to(sink).run()
}
You are on the right track. To keep the possibility to close the connection at some point you may want to keep the promise and complete it later on. Once completed with an element this element published by the source. However, as you don't want any element to be published on the connection, you can use drop(1) to make sure the source will never emit any element.
Here's an updated version of your example (untested):
val promise = Promise[ByteString]()
// this source will complete when the promise is fulfilled
// or it will complete with an error if the promise is completed with an error
val completionSource = Source(promise.future).drop(1)
completionSource // only used to complete later
.via(conn.flow) // I reordered the flow for better readability (arguably)
.runWith(targetSink)
// to close the connection later complete the promise:
def closeConnection() = promise.success(ByteString.empty) // dummy element, will be dropped
// alternatively to fail the connection later, complete with an error
def failConnection() = promise.failure(new RuntimeException)
How can I get the page to https? Tried that.
open Sys
open Array
open Str
open List
open Printf
open Unix
open Http_client.Convenience
open Nethttp
open Http_client.Convenience
open Netsys_tls;;
let provider = Netsys_tls.current_tls();;
let tls_config =
Netsys_tls.create_x509_config
~system_trust:true
~peer_auth:`Required
provider;;
let getPage _ =
let pipeline = new Http_client.pipeline in
let call = new Http_client.get ("https://vk.com/") in
call#set_redirect_mode Http_client.Do_not_redirect;
pipeline#add call;
pipeline#run ();
let cookies = Header.get_set_cookie call#response_header in
List.iter (fun c -> print_string(c.cookie_name^"="^c.cookie_value^"\n")) cookies;;
getPage();;
But when compiling I've got this error:
Error: Unbound module Netsys_tls
c:\wodi32\opt\wodi32\bin\ocamlfind ocamlopt unix.cmxa bigarray.cmxa str.cmxa -I +labltk labltk.cmxa -I c:\wodi32\opt\wodi32\lib\ocaml\pkg-lib\netsys\ netsys_oothr.cmxa netsys.cmxa -I c:\wodi32\opt\wodi32\lib\ocaml\pkg-lib\netstring\ netstring.cmxa -I c:\wodi32\opt\wodi32\lib\ocaml\pkg-lib\equeue\ equeue.cmxa -I c:\wodi32\opt\wodi32\lib\ocaml\pkg-lib\netclient\ netclient.cmxa vk.ml -I c:\wodi32\opt\wodi32\lib\ocaml\pkg-lib\equeue-ssl\ equeue_ssl.cmxa -I c:\wodi32\opt\wodi32\lib\ocaml\pkg-lib\cryptgps\ cryptgps.cmxa -o vk.exe
where can I get the module Netsys_tls
If you just want to get the url, and have no other reasons to stick with OCamlnet, then you can just use curl library, that will take care of all caveats:
let fetch fname url =
let fd = open_in fname in
let write s = Out_channel.output_string fd s; String.length s in
let conn = Curl.init () in
Curl.set_writefunction conn (write);
Curl.set_failonerror conn true;
Curl.set_followlocation conn true;
Curl.set_url conn url;
Curl.perform conn;
Curl.cleanup conn;
close_out fd
It will fetch any url supported by curl, including https