How to test two parallel transactions in Rust SQLx? - testing

I'm experimenting with Rocket, Rust and SQLx and I'd like to test what happens when two parallel transactions try to insert a duplicated record on my table.
My insert fn contains nothing special and it works fine:
async fn insert_credentials<'ex, EX>(&self, executor: EX, credentials: &Credentials) -> Result<u64, Errors>
where
EX: 'ex + Executor<'ex, Database = Postgres>,
{
sqlx::query!(
r#"INSERT INTO credentials (username, password)
VALUES ($1, crypt($2, gen_salt('bf')))"#,
credentials.username,
credentials.password,
)
.execute(executor)
.await
.map(|result| result.rows_affected())
.map_err(|err| err.into())
}
My test, though, hangs indefinitely since it waits for a commit that never happens:
#[async_std::test]
async fn it_should_reject_duplicated_username_in_parallel() {
let repo = new_repo();
let db: Pool<Postgres> = connect().await;
let credentials = new_random_credentials();
println!("TX1 begins");
let mut tx1 = db.begin().await.unwrap();
let rows_affected = repo.insert_credentials(&mut tx1, &credentials).await.unwrap();
assert_eq!(rows_affected, 1);
println!("TX2 begins");
let mut tx2 = db.begin().await.unwrap();
println!("It hangs on the next line");
let rows_affected = repo.insert_credentials(&mut tx2, &credentials).await.unwrap();
assert_eq!(rows_affected, 1);
println!("It never reaches this line");
tx1.commit().await.unwrap();
tx2.commit().await.unwrap();
}
How do I create and execute those TXs in parallel, such that the assertions pass but the test fails when trying to commit the second TX?
For reference, this is my Cargo.toml
[package]
name = "auth"
version = "0.1.0"
edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
[dependencies]
async-trait = "0.1.52"
serde = "1.0.136"
thiserror = "1.0.30"
# TODO https://github.com/SergioBenitez/Rocket/issues/1893#issuecomment-1002393878
rocket = { git = "https://github.com/SergioBenitez/Rocket", features = ["json"] }
[dependencies.redis]
version = "0.21.5"
features = ["tokio-comp"]
[dependencies.sqlx]
version = "0.5.11"
features = ["macros", "runtime-tokio-rustls", "postgres"]
[dependencies.uuid]
version = "1.0.0-alpha.1"
features = ["v4", "fast-rng", "macro-diagnostics"]
## DEV ##
[dev-dependencies]
mockall = "0.11.0"
[dev-dependencies.async-std]
version = "1.11.0"
features = ["attributes", "tokio1"]

You can use a async_std::future::timeout or tokio::time::timeout. Example using async_std:
use async_std::future;
use std::time::Duration;
let max_duration = Duration::from_millis(100);
assert!(timeout(max_duration, tx2.commit()).await.is_err());
If you want to continue to tx2 before completing tx1, you can async_std::task::spawn or tokio::spawn the tx1 first:
async_std::task::spawn(async move {
assert!(tx1.commit().await.is_ok());
});

#Mika pointed me the right direction, I could spawn both transactions and add a bit of timeout to give the concurrent TXs some time to execute.
let handle1 = tokio::spawn(async move {
let repo = new_repo();
let mut tx = db1.begin().await.unwrap();
let rows_affected = repo.insert_credentials(&mut tx, &credentials1).await.unwrap();
assert_eq!(rows_affected, 1);
tokio::time::sleep(Duration::from_millis(100)).await;
tx.commit().await.unwrap()
});
let handle2 = tokio::spawn(async move {
let repo = new_repo();
let mut tx = db2.begin().await.unwrap();
let rows_affected = repo.insert_credentials(&mut tx, &credentials2).await.unwrap();
assert_eq!(rows_affected, 1);
tokio::time::sleep(Duration::from_millis(100)).await;
tx.commit().await.unwrap()
});
let (_first, _second) = rocket::tokio::try_join!(handle1, handle2).unwrap();
I thought this way both TXs would execute in parallel until the sleep line, then one would commit and the other one would fail on the commit line. But no, actually both TXs execute in parallel, TX1 runs until the sleep and TX2 blocks on the insert line until TX1 commits, then TX2 fails on the insert line.
I guess that's just how DB works on this case and maybe I could change that by messing with TX isolation, but that's not my intent here. I'm just playing to learn more, and that's enough learning for today :)

Related

Why such a simple BufWriter operation didn't work

The following code is very simple. Open a file as a write, create a BufWriter using the file, and write a line of string.
The program reports no errors and returns an Ok(10) value, but the file just has no content and is empty.
#[tokio::test]
async fn save_file_async() {
let path = "./hello.txt";
let inner = tokio::fs::OpenOptions::new()
.create(true)
.write(true)
//.truncate(true)
.open(path)
.await
.unwrap();
let mut writer = tokio::io::BufWriter::new(inner);
println!(
"{} bytes wrote",
writer.write("1234567890".as_bytes()).await.unwrap()
);
}
Need an explicit flush:
writer.flush().await.unwrap();

Rust macro to generate multiple individual tests

Is it possible to have a macro that generates standalone tests? I have two text files, one with an input and another with an output. Each new line in the text file represents a new test.
Currently, this is how I run my tests:
#[test]
fn it_works() {
let input = read_file("input.txt").expect("failed to read input");
let input = input.split("\n").collect::<Vec<_>>();
let output = read_file("output.txt").expect("failed to read output");
let output = output.split("\n").collect::<Vec<_>>();
input.iter().zip(output).for_each(|(a, b)| {
println!("a: {}, b: {}", a, b);
assert_eq!(b, get_result(a));
})
But, as you can see, if one test fail, all of them fail, since there's a loop inside a single test. And I need each iteration to be a single and isolated test, without having to repeat myself.
So I was wondering if it's possible to achieve that by using macros?
The macro ideally would output something like:
#[test]
fn it_works_1() {
let input = read_file("input.txt").expect("failed to read input");
let input = input.split("\n").collect::<Vec<_>>();
let output = read_file("output.txt").expect("failed to read output");
let output = output.split("\n").collect::<Vec<_>>();
assert_eq!(output[0], get_result(input[0])); // first test
}
#[test]
fn it_works_2() {
let input = read_file("input.txt").expect("failed to read input");
let input = input.split("\n").collect::<Vec<_>>();
let output = read_file("output.txt").expect("failed to read output");
let output = output.split("\n").collect::<Vec<_>>();
assert_eq!(output[1], get_result(input[1])); // second test
}
// ... the N remaining tests: it_works_n()
You can't do this with a declarative macro because a declarative macro cannot generate an identifier to name the test functions. However you can use a crate such as test-case, which can run the same test with different inputs:
use test_case::test_case;
#[test_case(0)]
#[test_case(1)]
#[test_case(2)]
#[test]
fn it_works(index: usize) {
let input = read_file("input.txt").expect("failed to read input");
let input = input.split("\n").collect::<Vec<_>>();
let output = read_file("output.txt").expect("failed to read output");
let output = output.split("\n").collect::<Vec<_>>();
assert_eq!(output[index], get_result(input[index])); // first test
}
If you have a lot of different inputs to test, you could use a declarative macro to generate the code above, which would add all of the #[test_case] annotations.
After Peter Hall answer, I was able to achieve what I wanted. I added the seq_macro crate to generate the repeated #[test_case]'s. Maybe there's a way to loop through all test cases instead of manually defining the amount of tests (like I did), but this is good for now:
macro_rules! test {
( $from:expr, $to:expr ) => {
#[cfg(test)]
mod tests {
use crate::{get_result, read_file};
use seq_macro::seq;
use test_case::test_case;
seq!(N in $from..$to {
#(#[test_case(N)])*
fn it_works(index: usize) {
let input = read_file("input.txt").expect("failed to read input");
let input = input.split("\n").collect::<Vec<_>>();
let output = read_file("output.txt").expect("failed to read output");
let output = output.split("\n").collect::<Vec<_>>();
let res = get_result(input[index]);
assert_eq!(
output[index], res,
"Test '{}': Want '{}' got '{}'",
input[index], output[index], res
);
}
});
}
};
}
test!(0, 82);

In git2-rs, how do I authenticate when cloning?

How do I pass an authentication callback to git2::Repository::clone()? (set_remote_callbacks sets up the callbacks).
I have some code like the following:
let mut cb = git2::RemoteCallbacks::new();
Self::set_remote_callbacks(&mut cb);
let rr = Repository::clone(url, path.to_str().ok_or("bad string".to_string())?);
What I want is like, as an example, when I do I fetch, I do this, which passes my callbacks to the fetch:
let mut fetchOptions = FetchOptions::new();
let mut cb = git2::RemoteCallbacks::new();
Self::set_remote_callbacks(&mut cb);
fetchOptions.remote_callbacks(cb);
let mut remote = self.repo.find_remote(remote)?;
remote.fetch(&[branch], Some(&mut fetchOptions), None)?;
Use git2::build::RepoBuilder.
Credit goes to issue 329 on the git2 issue tracker.

How do i create a TCP receiver that only consumes messages using akka streams?

We are on: akka-stream-experimental_2.11 1.0.
Inspired by the example
We wrote a TCP receiver as follows:
def bind(address: String, port: Int, target: ActorRef)
(implicit system: ActorSystem, actorMaterializer: ActorMaterializer): Future[ServerBinding] = {
val sink = Sink.foreach[Tcp.IncomingConnection] { conn =>
val serverFlow = Flow[ByteString]
.via(Framing.delimiter(ByteString("\n"), maximumFrameLength = 256, allowTruncation = true))
.map(message => {
target ? new Message(message); ByteString.empty
})
conn handleWith serverFlow
}
val connections = Tcp().bind(address, port)
connections.to(sink).run()
}
However, our intention was to have the receiver not respond at all and only sink the message. (The TCP message publisher does not care about response ).
Is it even possible? to not respond at all since akka.stream.scaladsl.Tcp.IncomingConnection takes a flow of type: Flow[ByteString, ByteString, Unit]
If yes, some guidance will be much appreciated. Thanks in advance.
One attempt as follows passes my unit tests but not sure if its the best idea:
def bind(address: String, port: Int, target: ActorRef)
(implicit system: ActorSystem, actorMaterializer: ActorMaterializer): Future[ServerBinding] = {
val sink = Sink.foreach[Tcp.IncomingConnection] { conn =>
val targetSubscriber = ActorSubscriber[Message](system.actorOf(Props(new TargetSubscriber(target))))
val targetSink = Flow[ByteString]
.via(Framing.delimiter(ByteString("\n"), maximumFrameLength = 256, allowTruncation = true))
.map(Message(_))
.to(Sink(targetSubscriber))
conn.flow.to(targetSink).runWith(Source(Promise().future))
}
val connections = Tcp().bind(address, port)
connections.to(sink).run()
}
You are on the right track. To keep the possibility to close the connection at some point you may want to keep the promise and complete it later on. Once completed with an element this element published by the source. However, as you don't want any element to be published on the connection, you can use drop(1) to make sure the source will never emit any element.
Here's an updated version of your example (untested):
val promise = Promise[ByteString]()
// this source will complete when the promise is fulfilled
// or it will complete with an error if the promise is completed with an error
val completionSource = Source(promise.future).drop(1)
completionSource // only used to complete later
.via(conn.flow) // I reordered the flow for better readability (arguably)
.runWith(targetSink)
// to close the connection later complete the promise:
def closeConnection() = promise.success(ByteString.empty) // dummy element, will be dropped
// alternatively to fail the connection later, complete with an error
def failConnection() = promise.failure(new RuntimeException)

Get test outcome/result using TFS API

Using the TFS API, how can I get the outcome/result of a specific test case in a given test suite and plan?
With outcome/result I mean the value that tests are grouped by in MTM:
Passed, failed, active, in progress or blocked
This is how I do it.
To get passed and totalTests I use:
ITestRun run*
run.PassedTests and run.TotalTests
To see run state I use:
TestRunSTate.Aborted and TestRunState.InProgress
To see if the failed or is inconclusive I use:
TestOutcome.Failed or TestOutcome.Inconclusive
First I only used ITestRun to easy se results, but I see they lack any kind of "failed" there which I find very disturbing.
So to send the right numbers to my test report that is mailed to the product owner I do the following when talking to the tfs api:
var tfs = Connect(optionsModel.CollectionUri);
var tcm = GetService<ITestManagementService>(tfs);
var wis = GetService<WorkItemStore>(tfs);
_testProject = tcm.GetTeamProject(optionsModel.TeamProjectName);
var plan = _testProject.TestPlans.Find(optionsModel.PlanId);
if (plan == null)
throw new Exception("Could not find plan with that id.");
var run = plan.CreateTestRun(true);
var testSuite = _testProject.TestSuites.Find(optionsModel.SuiteId);
if (testSuite == null)
throw new Exception("Could not find suite with that id.");
AddTestCasesBySuite(testSuite, optionsModel.ConfigId, plan, run);
run.Title = optionsModel.Title;
run.Save();
var failedTests = run.QueryResultsByOutcome(TestOutcome.Failed).Count;
var inconclusiveTests = run.QueryResultsByOutcome(TestOutcome.Inconclusive).Count;
Hope this helps
optionsmodel is the information I take in from the user running the tsts
I was trying to do the same thing, but using the REST API.
Just in case it helps someone, I managed to do that obtaining the testpoints from the suite:
https://dev.azure.com/{organization}/{project}/_apis/testplan/Plans/{planId}/Suites/{suiteId}/TestPoint?api-version=5.1-preview.2
More info: https://learn.microsoft.com/en-us/rest/api/azure/devops/testplan/test%20point/get%20points%20list?view=azure-devops-rest-5.1
You can use ITestManagementService and TestPlan query to get the result of specific Test plan
var server = new Uri("http://servername:8080/tfs/collectionname");
var tfs = TfsTeamProjectCollectionFactory.GetTeamProjectCollection(server);
var service = tfs.GetService<ITestManagementService>();
var testProject = service.GetTeamProject(teamProject);
var plans = testProject.TestPlans.Query("SELECT * FROM TestPlan").Where(tp => tp.Name == YOURTESTPLANNAME).FirstOrDefault();
ITestPlanCollection plans = tfsConnectedTeamProject.TestPlans.Query("Select * From TestPlan");
foreach (ITestPlan plan in plans)
{
if (plan.RootSuite != null && plan.RootSuite.Entries.Count > 0)
{
foreach (ITestSuiteEntry suiteEntry in plan.RootSuite.Entries)
{
var suite = suiteEntry.TestSuite as IStaticTestSuite;
if (suite != null)
{
ITestSuiteEntryCollection suiteentrys = suite.TestCases;
foreach (ITestSuiteEntry testcase in suiteentrys)
{
// Write code to get the test case
}
}
}
}
}
I hope this may help you.