I have a function which creates a process on Windows.
pub fn create_process(url: String) {
thread::spawn(move || {
let _child = process::Command::new("cmd.exe")
.arg("/C")
.arg("ping")
.arg(&url)
.arg("-t")
.spawn()
.expect("Couldn't run 'ping'");
});
}
I have a function which I want to terminate (kill) the process created by 'create_process()':
pub fn stop() {
// ?????
}
How can I access the _child created in create_process function to terminate its process? Are there any other ways to kill that process?
TL;DR: Use the kill method. For example:
use std::{process, thread, time::Duration};
fn main() {
let mut child = process::Command::new("ping")
.arg("8.8.8.8")
.arg("-t")
.spawn()
.expect("Couldn't run 'ping'");
thread::sleep(Duration::from_secs(5));
child.kill().expect("!kill");
}
Notice how you don't need a separate thread, since the spawned process is already parallel to the parent process.
In your question, there is a code example which uses "cmd /C" to run the "ping". This spawns not one but two processes: the "cmd" process and the "ping" process. Killing the child will kill the "cmd" process but might leave the "ping" process running.
Using "cmd /C" is also dangerous, allowing for command injection.
How can I access the _child created in the create_process function to terminate its process?
The _child is Send, which means that you can send it from thread to thread. The particulars of sending data across threads is likely to have been covered already in a number of corresponding Stack Overflow questions.
Are there any other ways to kill that process?
You can use the native platform APIs. For example:
[dependencies]
gstuff = "0.5.2"
winapi = {version = "0.3.6", features = ["psapi", "shellapi"]}
#[macro_use]
extern crate gstuff;
use std::process;
use std::ptr::null_mut;
use std::thread;
use std::time::Duration;
use winapi::shared::minwindef::DWORD;
use winapi::shared::ntdef::HANDLE;
use winapi::um::processthreadsapi::{OpenProcess, TerminateProcess};
use winapi::um::winnt::{PROCESS_QUERY_INFORMATION, PROCESS_TERMINATE};
struct Process(HANDLE);
impl Process {
fn open(pid: DWORD) -> Result<Process, String> {
// https://msdn.microsoft.com/en-us/library/windows/desktop/ms684320%28v=vs.85%29.aspx
let pc = unsafe { OpenProcess(PROCESS_QUERY_INFORMATION | PROCESS_TERMINATE, 0, pid) };
if pc == null_mut() {
return ERR!("!OpenProcess");
}
Ok(Process(pc))
}
fn kill(self) -> Result<(), String> {
unsafe { TerminateProcess(self.0, 1) };
Ok(())
}
}
impl Drop for Process {
fn drop(&mut self) {
unsafe { winapi::um::handleapi::CloseHandle(self.0) };
}
}
fn main() {
let child = process::Command::new("ping")
.arg("8.8.8.8")
.arg("-t")
.spawn()
.expect("Couldn't run 'ping'");
let pid = child.id();
let pc = Process::open(pid as DWORD).expect("!open");
println!("Process {} opened.", pid);
thread::sleep(Duration::from_secs(5));
pc.kill().expect("!kill");
println!("Process {} killed.", pid);
}
Related
What is the best way to create and use a struct with only one instantiation in the system? Yes, this is necessary, it is the OpenGL subsystem, and making multiple copies of this and passing it around everywhere would add confusion, rather than relieve it.
The singleton needs to be as efficient as possible. It doesn't seem possible to store an arbitrary object on the static area, as it contains a Vec with a destructor. The second option is to store an (unsafe) pointer on the static area, pointing to a heap allocated singleton. What is the most convenient and safest way to do this, while keeping syntax terse?
Non-answer answer
Avoid global state in general. Instead, construct the object somewhere early (perhaps in main), then pass mutable references to that object into the places that need it. This will usually make your code easier to reason about and doesn't require as much bending over backwards.
Look hard at yourself in the mirror before deciding that you want global mutable variables. There are rare cases where it's useful, so that's why it's worth knowing how to do.
Still want to make one...?
Tips
In the 3 following solutions:
If you remove the Mutex then you have a global singleton without any mutability.
You can also use a RwLock instead of a Mutex to allow multiple concurrent readers.
Using lazy-static
The lazy-static crate can take away some of the drudgery of manually creating a singleton. Here is a global mutable vector:
use lazy_static::lazy_static; // 1.4.0
use std::sync::Mutex;
lazy_static! {
static ref ARRAY: Mutex<Vec<u8>> = Mutex::new(vec![]);
}
fn do_a_call() {
ARRAY.lock().unwrap().push(1);
}
fn main() {
do_a_call();
do_a_call();
do_a_call();
println!("called {}", ARRAY.lock().unwrap().len());
}
Using once_cell
The once_cell crate can take away some of the drudgery of manually creating a singleton. Here is a global mutable vector:
use once_cell::sync::Lazy; // 1.3.1
use std::sync::Mutex;
static ARRAY: Lazy<Mutex<Vec<u8>>> = Lazy::new(|| Mutex::new(vec![]));
fn do_a_call() {
ARRAY.lock().unwrap().push(1);
}
fn main() {
do_a_call();
do_a_call();
do_a_call();
println!("called {}", ARRAY.lock().unwrap().len());
}
Using std::sync::LazyLock
The standard library is in the process of adding once_cell's functionality, currently called LazyLock:
#![feature(once_cell)] // 1.67.0-nightly
use std::sync::{LazyLock, Mutex};
static ARRAY: LazyLock<Mutex<Vec<u8>>> = LazyLock::new(|| Mutex::new(vec![]));
fn do_a_call() {
ARRAY.lock().unwrap().push(1);
}
fn main() {
do_a_call();
do_a_call();
do_a_call();
println!("called {}", ARRAY.lock().unwrap().len());
}
A special case: atomics
If you only need to track an integer value, you can directly use an atomic:
use std::sync::atomic::{AtomicUsize, Ordering};
static CALL_COUNT: AtomicUsize = AtomicUsize::new(0);
fn do_a_call() {
CALL_COUNT.fetch_add(1, Ordering::SeqCst);
}
fn main() {
do_a_call();
do_a_call();
do_a_call();
println!("called {}", CALL_COUNT.load(Ordering::SeqCst));
}
Manual, dependency-free implementation
There are several existing implementation of statics, such as the Rust 1.0 implementation of stdin. This is the same idea adapted to modern Rust, such as the use of MaybeUninit to avoid allocations and unnecessary indirection. You should also look at the modern implementation of io::Lazy. I've commented inline with what each line does.
use std::sync::{Mutex, Once};
use std::time::Duration;
use std::{mem::MaybeUninit, thread};
struct SingletonReader {
// Since we will be used in many threads, we need to protect
// concurrent access
inner: Mutex<u8>,
}
fn singleton() -> &'static SingletonReader {
// Create an uninitialized static
static mut SINGLETON: MaybeUninit<SingletonReader> = MaybeUninit::uninit();
static ONCE: Once = Once::new();
unsafe {
ONCE.call_once(|| {
// Make it
let singleton = SingletonReader {
inner: Mutex::new(0),
};
// Store it to the static var, i.e. initialize it
SINGLETON.write(singleton);
});
// Now we give out a shared reference to the data, which is safe to use
// concurrently.
SINGLETON.assume_init_ref()
}
}
fn main() {
// Let's use the singleton in a few threads
let threads: Vec<_> = (0..10)
.map(|i| {
thread::spawn(move || {
thread::sleep(Duration::from_millis(i * 10));
let s = singleton();
let mut data = s.inner.lock().unwrap();
*data = i as u8;
})
})
.collect();
// And let's check the singleton every so often
for _ in 0u8..20 {
thread::sleep(Duration::from_millis(5));
let s = singleton();
let data = s.inner.lock().unwrap();
println!("It is: {}", *data);
}
for thread in threads.into_iter() {
thread.join().unwrap();
}
}
This prints out:
It is: 0
It is: 1
It is: 1
It is: 2
It is: 2
It is: 3
It is: 3
It is: 4
It is: 4
It is: 5
It is: 5
It is: 6
It is: 6
It is: 7
It is: 7
It is: 8
It is: 8
It is: 9
It is: 9
It is: 9
This code compiles with Rust 1.55.0.
All of this work is what lazy-static or once_cell do for you.
The meaning of "global"
Please note that you can still use normal Rust scoping and module-level privacy to control access to a static or lazy_static variable. This means that you can declare it in a module or even inside of a function and it won't be accessible outside of that module / function. This is good for controlling access:
use lazy_static::lazy_static; // 1.2.0
fn only_here() {
lazy_static! {
static ref NAME: String = String::from("hello, world!");
}
println!("{}", &*NAME);
}
fn not_here() {
println!("{}", &*NAME);
}
error[E0425]: cannot find value `NAME` in this scope
--> src/lib.rs:12:22
|
12 | println!("{}", &*NAME);
| ^^^^ not found in this scope
However, the variable is still global in that there's one instance of it that exists across the entire program.
Starting with Rust 1.63, it can be easier to work with global mutable singletons, although it's still preferable to avoid global variables in most cases.
Now that Mutex::new is const, you can use global static Mutex locks without needing lazy initialization:
use std::sync::Mutex;
static GLOBAL_DATA: Mutex<Vec<i32>> = Mutex::new(Vec::new());
fn main() {
GLOBAL_DATA.lock().unwrap().push(42);
println!("{:?}", GLOBAL_DATA.lock().unwrap());
}
Note that this also depends on the fact that Vec::new is const. If you need to use non-const functions to set up your singleton, you could wrap your data in an Option, and initially set it to None. This lets you use data structures like Hashset which currently cannot be used in a const context:
use std::sync::Mutex;
use std::collections::HashSet;
static GLOBAL_DATA: Mutex<Option<HashSet<i32>>> = Mutex::new(None);
fn main() {
*GLOBAL_DATA.lock().unwrap() = Some(HashSet::from([42]));
println!("V2: {:?}", GLOBAL_DATA.lock().unwrap());
}
Alternatively, you could use an RwLock, instead of a Mutex, since RwLock::new is also const as of Rust 1.63. This would make it possible to read the data from multiple threads simultaneously.
If you need to initialize with non-const functions and you'd prefer not to use an Option, you could use a crate like once_cell or lazy-static for lazy initialization as explained in Shepmaster's answer.
From What Not To Do In Rust
To recap: instead of using interior mutability where an object changes
its internal state, consider using a pattern where you promote new
state to be current and current consumers of the old state will
continue to hold on to it by putting an Arc into an RwLock.
use std::sync::{Arc, RwLock};
#[derive(Default)]
struct Config {
pub debug_mode: bool,
}
impl Config {
pub fn current() -> Arc<Config> {
CURRENT_CONFIG.with(|c| c.read().unwrap().clone())
}
pub fn make_current(self) {
CURRENT_CONFIG.with(|c| *c.write().unwrap() = Arc::new(self))
}
}
thread_local! {
static CURRENT_CONFIG: RwLock<Arc<Config>> = RwLock::new(Default::default());
}
fn main() {
Config { debug_mode: true }.make_current();
if Config::current().debug_mode {
// do something
}
}
Use SpinLock for global access.
#[derive(Default)]
struct ThreadRegistry {
pub enabled_for_new_threads: bool,
threads: Option<HashMap<u32, *const Tls>>,
}
impl ThreadRegistry {
fn threads(&mut self) -> &mut HashMap<u32, *const Tls> {
self.threads.get_or_insert_with(HashMap::new)
}
}
static THREAD_REGISTRY: SpinLock<ThreadRegistry> = SpinLock::new(Default::default());
fn func_1() {
let thread_registry = THREAD_REGISTRY.lock(); // Immutable access
if thread_registry.enabled_for_new_threads {
}
}
fn func_2() {
let mut thread_registry = THREAD_REGISTRY.lock(); // Mutable access
thread_registry.threads().insert(
// ...
);
}
If you want mutable state(NOT Singleton), see What Not to Do in Rust for more descriptions.
Hope it's helpful.
If you are on nightly, you can use LazyLock.
It more or less does what the crates once_cell and lazy_sync do. Those two crates are very common, so there's a good chance they might already by in your Cargo.lock dependency tree. But if you prefer to be a bit more "adventurous" and go with LazyLock, be prepered that it (as everything in nightly) might be a subject to change before it gets to stable.
(Note: Up until recently std::sync::LazyLock used to be named std::lazy::SyncLazy but was recently renamed.)
A bit late to the party, but here's how I worked around this issue (rust 1.66-nightly):
#![feature(const_size_of_val)]
#![feature(const_ptr_write)]
static mut GLOBAL_LAZY_MUT: StructThatIsNotSyncNorSend = unsafe {
// Copied from MaybeUninit::zeroed() with minor modifications, see below
let mut u = MaybeUninit::uninit();
let bytes = mem::size_of_val(&u);
write_bytes(u.as_ptr() as *const u8 as *mut u8, 0xA5, bytes); //Trick the compiler check that verifies pointers and references are not null.
u.assume_init()
};
(...)
fn main() {
unsafe {
let mut v = StructThatIsNotSyncNorSend::new();
mem::swap(&mut GLOBAL_LAZY_MUT, &mut v);
mem::forget(v);
}
}
Beware that this code is unbelievably unsafe, and can easily end up being UB if not handled correctly.
You now have a !Send !Sync value as a global static, without the protection of a Mutex. If you access it from multiple threads, even if just for reading, it's UB. If you don't initialize it the way shown, it's UB, because it calls Drop on an actually unitialized value.
You just convinced the rust compiler that something that is UB is not UB. You just convinced that putting a !Sync and !Send in a global static is fine.
If unsure, don't use this snippet.
My limited solution is to define a struct instead of a global mutable one. To use that struct, external code needs to call init() but we disallow calling init() more than once by using an AtomicBoolean (for multithreading usage).
static INITIATED: AtomicBool = AtomicBool::new(false);
struct Singleton {
...
}
impl Singleton {
pub fn init() -> Self {
if INITIATED.load(Ordering::Relaxed) {
panic!("Cannot initiate more than once")
} else {
INITIATED.store(true, Ordering::Relaxed);
Singleton {
...
}
}
}
}
fn main() {
let singleton = Singleton::init();
// panic here
// let another_one = Singleton::init();
...
}
My problem is that server process doesn't get shut down after the last integration test.
In integration.rs, I have:
lazy_static! {
static ref SERVER: Arc<Mutex<duct::ReaderHandle>> = {
println!("Starting server");
Arc::new(Mutex::new(
cmd!("cargo", "run", "--", "13000")
.reader()
.expect("Valid server"),
))
};
}
async fn wait_for_server() {
lazy_static::initialize(&SERVER);
// Code to wait
}
#[tokio::test]
async fn integration_test_query_amount() -> Result<()> {
wait_for_server().await;
let client = reqwest::Client::new();
// Etc...
}
The tests work, but the server stays running after the cargo test invocation finishes. Is there a nice recipe for starting up and shutting down a server like this?
You can make a Drop wrapper for a process which will kill it when it goes out of scope. Something along the lines of:
struct KillOnDrop(std::process::Child);
impl Drop for KillOnDrop {
fn drop(&mut self) {
self.0.kill()
}
}
Alternatively, as it looks like you're using tokio already, tokio::process supports this out of the box.
I have a program that takes an argument called start-master. When it is passed, I need to start a daemon listening on a port.
I have a function:
fn startMaster() {
// Sockets, connections, logic, etc
}
Can I make a new process that executes startMaster in the background so that a user can call ./mainprogram start-master without using bash's &?
The process has to be independent from the main process as it must not stop when the caller process finishes. I need a non blocking solution.
My main program would be:
fn main() {
let args: Vec<String> = env::args().collect();
let arg1 = &args[1];
// ...
match arg1.as_ref() {
"start-master" => {
// Starts process in background which executes startMaster function
}
_ => println!("Not valid argument")
}
// Ends program but now there is an active daemon
}
I'm making a small ncurses application in Rust that needs to communicate with a child process. I already have a prototype written in Common Lisp. I'm trying to rewrite it because CL uses a huge amount of memory for such a small tool.
I'm having some trouble figuring out how to interact with the sub-process.
What I'm currently doing is roughly this:
Create the process:
let mut program = match Command::new(command)
.args(arguments)
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
{
Ok(child) => child,
Err(_) => {
println!("Cannot run program '{}'.", command);
return;
}
};
Pass it to an infinite (until user exits) loop, which reads and handles input and listens for output like this (and writes it to the screen):
fn listen_for_output(program: &mut Child, output_viewer: &TextViewer) {
match program.stdout {
Some(ref mut out) => {
let mut buf_string = String::new();
match out.read_to_string(&mut buf_string) {
Ok(_) => output_viewer.append_string(buf_string),
Err(_) => return,
};
}
None => return,
};
}
The call to read_to_string however blocks the program until the process exits. From what I can see read_to_end and read also seem to block. If I try running something like ls which exits right away, it works, but with something that doesn't exit like python or sbcl it only continues once I kill the subprocess manually.
Based on this answer, I changed the code to use BufReader:
fn listen_for_output(program: &mut Child, output_viewer: &TextViewer) {
match program.stdout.as_mut() {
Some(out) => {
let buf_reader = BufReader::new(out);
for line in buf_reader.lines() {
match line {
Ok(l) => {
output_viewer.append_string(l);
}
Err(_) => return,
};
}
}
None => return,
}
}
However, the problem still remains the same. It will read all lines that are available, and then block. Since the tool is supposed to work with any program, there is no way to guess out when the output will end, before trying to read. There doesn't appear to be a way to set a timeout for BufReader either.
Streams are blocking by default. TCP/IP streams, filesystem streams, pipe streams, they are all blocking. When you tell a stream to give you a chunk of bytes it will stop and wait till it has the given amout of bytes or till something else happens (an interrupt, an end of stream, an error).
The operating systems are eager to return the data to the reading process, so if all you want is to wait for the next line and handle it as soon as it comes in then the method suggested by Shepmaster in Unable to pipe to or from spawned child process more than once (and also in his answer here) works.
Though in theory it doesn't have to work, because an operating system is allowed to make the BufReader wait for more data in read, but in practice the operating systems prefer the early "short reads" to waiting.
This simple BufReader-based approach becomes even more dangerous when you need to handle multiple streams (like the stdout and stderr of a child process) or multiple processes. For example, BufReader-based approach might deadlock when a child process waits for you to drain its stderr pipe while your process is blocked waiting on it's empty stdout.
Similarly, you can't use BufReader when you don't want your program to wait on the child process indefinitely. Maybe you want to display a progress bar or a timer while the child is still working and gives you no output.
You can't use BufReader-based approach if your operating system happens not to be eager in returning the data to the process (prefers "full reads" to "short reads") because in that case a few last lines printed by the child process might end up in a gray zone: the operating system got them, but they're not large enough to fill the BufReader's buffer.
BufReader is limited to what the Read interface allows it to do with the stream, it's no less blocking than the underlying stream is. In order to be efficient it will read the input in chunks, telling the operating system to fill as much of its buffer as it has available.
You might be wondering why reading data in chunks is so important here, why can't the BufReader just read the data byte by byte. The problem is that to read the data from a stream we need the operating system's help. On the other hand, we are not the operating system, we work isolated from it, so as not to mess with it if something goes wrong with our process. So in order to call to the operating system there needs to be a transition to "kernel mode" which might also incur a "context switch". That is why calling the operating system to read every single byte is expensive. We want as few OS calls as possible and so we get the stream data in batches.
To wait on a stream without blocking you'd need a non-blocking stream. MIO promises to have the required non-blocking stream support for pipes, most probably with PipeReader, but I haven't checked it out so far.
The non-blocking nature of a stream should make it possible to read data in chunks regardless of whether the operating system prefers the "short reads" or not. Because non-blocking stream never blocks. If there is no data in the stream it simply tells you so.
In the absense of a non-blocking stream you'll have to resort to spawning threads so that the blocking reads would be performed in a separate thread and thus won't block your primary thread. You might also want to read the stream byte by byte in order to react to the line separator immediately in case the operating system does not prefer the "short reads". Here's a working example: https://gist.github.com/ArtemGr/db40ae04b431a95f2b78.
P.S. Here's an example of a function that allows to monitor the standard output of a program via a shared vector of bytes:
use std::io::Read;
use std::process::{Command, Stdio};
use std::sync::{Arc, Mutex};
use std::thread;
/// Pipe streams are blocking, we need separate threads to monitor them without blocking the primary thread.
fn child_stream_to_vec<R>(mut stream: R) -> Arc<Mutex<Vec<u8>>>
where
R: Read + Send + 'static,
{
let out = Arc::new(Mutex::new(Vec::new()));
let vec = out.clone();
thread::Builder::new()
.name("child_stream_to_vec".into())
.spawn(move || loop {
let mut buf = [0];
match stream.read(&mut buf) {
Err(err) => {
println!("{}] Error reading from stream: {}", line!(), err);
break;
}
Ok(got) => {
if got == 0 {
break;
} else if got == 1 {
vec.lock().expect("!lock").push(buf[0])
} else {
println!("{}] Unexpected number of bytes: {}", line!(), got);
break;
}
}
}
})
.expect("!thread");
out
}
fn main() {
let mut cat = Command::new("cat")
.stdin(Stdio::piped())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.expect("!cat");
let out = child_stream_to_vec(cat.stdout.take().expect("!stdout"));
let err = child_stream_to_vec(cat.stderr.take().expect("!stderr"));
let mut stdin = match cat.stdin.take() {
Some(stdin) => stdin,
None => panic!("!stdin"),
};
}
With a couple of helpers I'm using it to control an SSH session:
try_s! (stdin.write_all (b"echo hello world\n"));
try_s! (wait_forĖ¢ (&out, 0.1, 9., |s| s == "hello world\n"));
P.S. Note that await on a read call in async-std is blocking as well. It's just instead of blocking a system thread it only blocks a chain of futures (a stack-less green thread essentially). The poll_read is the non-blocking interface. In async-std#499 I've asked the developers whether there's a short read guarantee from these APIs.
P.S. There might be a similar concern in Nom: "we would want to tell the IO side to refill according to the parser's result (Incomplete or not)"
P.S. Might be interesting to see how stream reading is implemented in crossterm. For Windows, in poll.rs, they are using the native WaitForMultipleObjects. In unix.rs they are using mio poll.
Tokio's Command
Here is an example of using tokio 0.2:
use std::process::Stdio;
use futures::StreamExt; // 0.3.1
use tokio::{io::BufReader, prelude::*, process::Command}; // 0.2.4, features = ["full"]
#[tokio::main]
async fn main() {
let mut cmd = Command::new("/tmp/slow.bash")
.stdout(Stdio::piped()) // Can do the same for stderr
.spawn()
.expect("cannot spawn");
let stdout = cmd.stdout().take().expect("no stdout");
// Can do the same for stderr
// To print out each line
// BufReader::new(stdout)
// .lines()
// .for_each(|s| async move { println!("> {:?}", s) })
// .await;
// To print out each line *and* collect it all into a Vec
let result: Vec<_> = BufReader::new(stdout)
.lines()
.inspect(|s| println!("> {:?}", s))
.collect()
.await;
println!("All the lines: {:?}", result);
}
Tokio-Threadpool
Here is an example of using tokio 0.1 and tokio-threadpool. We start the process in a thread using the blocking function. We convert that to a stream with stream::poll_fn
use std::process::{Command, Stdio};
use tokio::{prelude::*, runtime::Runtime}; // 0.1.18
use tokio_threadpool; // 0.1.13
fn stream_command_output(
mut command: Command,
) -> impl Stream<Item = Vec<u8>, Error = tokio_threadpool::BlockingError> {
// Ensure that the output is available to read from and start the process
let mut child = command
.stdout(Stdio::piped())
.spawn()
.expect("cannot spawn");
let mut stdout = child.stdout.take().expect("no stdout");
// Create a stream of data
stream::poll_fn(move || {
// Perform blocking IO
tokio_threadpool::blocking(|| {
// Allocate some space to store anything read
let mut data = vec![0; 128];
// Read 1-128 bytes of data
let n_bytes_read = stdout.read(&mut data).expect("cannot read");
if n_bytes_read == 0 {
// Stdout is done
None
} else {
// Only return as many bytes as we read
data.truncate(n_bytes_read);
Some(data)
}
})
})
}
fn main() {
let output_stream = stream_command_output(Command::new("/tmp/slow.bash"));
let mut runtime = Runtime::new().expect("Unable to start the runtime");
let result = runtime.block_on({
output_stream
.map(|d| String::from_utf8(d).expect("Not UTF-8"))
.fold(Vec::new(), |mut v, s| {
print!("> {}", s);
v.push(s);
Ok(v)
})
});
println!("All the lines: {:?}", result);
}
There's numerous possible tradeoffs that can be made here. For example, always allocating 128 bytes isn't ideal, but it's simple to implement.
Support
For reference, here's slow.bash:
#!/usr/bin/env bash
set -eu
val=0
while [[ $val -lt 10 ]]; do
echo $val
val=$(($val + 1))
sleep 1
done
See also:
How do I synchronously return a value calculated in an asynchronous Future in stable Rust?
If Unix support is sufficient, you can also make the two output streams as non-blocking and poll over them as you would do it on TcpStream with the set_nonblocking function.
The ChildStdout and ChildStderr returned by the Command spawn are Stdio (and contain a file descriptor), you can modify directly the read behavior of these handle to make it non-blocking.
Based on the work of jcreekmore/timeout-readwrite-rs and anowell/nonblock-rs, I use this wrapper to modify the stream handles:
extern crate libc;
use std::io::Read;
use std::os::unix::io::AsRawFd;
use libc::{F_GETFL, F_SETFL, fcntl, O_NONBLOCK};
fn set_nonblocking<H>(handle: &H, nonblocking: bool) -> std::io::Result<()>
where
H: Read + AsRawFd,
{
let fd = handle.as_raw_fd();
let flags = unsafe { fcntl(fd, F_GETFL, 0) };
if flags < 0 {
return Err(std::io::Error::last_os_error());
}
let flags = if nonblocking{
flags | O_NONBLOCK
} else {
flags & !O_NONBLOCK
};
let res = unsafe { fcntl(fd, F_SETFL, flags) };
if res != 0 {
return Err(std::io::Error::last_os_error());
}
Ok(())
}
You can manage the two streams as any other non-blocking stream. The following example is based on the polling crate which makes really easy to handle read event and BufReader for line reading:
use std::process::{Command, Stdio};
use std::path::PathBuf;
use std::io::{BufReader, BufRead};
use std::thread;
extern crate polling;
use polling::{Event, Poller};
fn main() -> Result<(), std::io::Error> {
let path = PathBuf::from("./worker.sh").canonicalize()?;
let mut child = Command::new(path)
.stdin(Stdio::null())
.stdout(Stdio::piped())
.stderr(Stdio::piped())
.spawn()
.expect("Failed to start worker");
let handle = thread::spawn({
let stdout = child.stdout.take().unwrap();
set_nonblocking(&stdout, true)?;
let mut reader_out = BufReader::new(stdout);
let stderr = child.stderr.take().unwrap();
set_nonblocking(&stderr, true)?;
let mut reader_err = BufReader::new(stderr);
move || {
let key_out = 1;
let key_err = 2;
let mut out_closed = false;
let mut err_closed = false;
let poller = Poller::new().unwrap();
poller.add(reader_out.get_ref(), Event::readable(key_out)).unwrap();
poller.add(reader_err.get_ref(), Event::readable(key_err)).unwrap();
let mut line = String::new();
let mut events = Vec::new();
loop {
// Wait for at least one I/O event.
events.clear();
poller.wait(&mut events, None).unwrap();
for ev in &events {
// stdout is ready for reading
if ev.key == key_out {
let len = match reader_out.read_line(&mut line) {
Ok(len) => len,
Err(e) => {
println!("stdout read returned error: {}", e);
0
}
};
if len == 0 {
println!("stdout closed (len is null)");
out_closed = true;
poller.delete(reader_out.get_ref()).unwrap();
} else {
print!("[STDOUT] {}", line);
line.clear();
// reload the poller
poller.modify(reader_out.get_ref(), Event::readable(key_out)).unwrap();
}
}
// stderr is ready for reading
if ev.key == key_err {
let len = match reader_err.read_line(&mut line) {
Ok(len) => len,
Err(e) => {
println!("stderr read returned error: {}", e);
0
}
};
if len == 0 {
println!("stderr closed (len is null)");
err_closed = true;
poller.delete(reader_err.get_ref()).unwrap();
} else {
print!("[STDERR] {}", line);
line.clear();
// reload the poller
poller.modify(reader_err.get_ref(), Event::readable(key_err)).unwrap();
}
}
}
if out_closed && err_closed {
println!("Stream closed, exiting process thread");
break;
}
}
}
});
handle.join().unwrap();
Ok(())
}
Additionally, used with a wrapper over an EventFd, it becomes possible to easily stop the process from another thread without blocking nor active polling and uses and only a single thread.
EDIT: It seems the polling crate sets automatically the polled handles in non-blocking mode following my tests. The set_nonblocking function is still useful in case you want to directly use the nix::poll object.
I have encountered enough use-cases where it was useful to interact with a subprocess over line-delimited text that I wrote a crate for it, interactive_process.
I expect the original problem has long since been solved, but I thought it might be helpful to others.
I'd like to store a std::io::process::Process inside a struct in rust. However, the program goes defunct as soon as I pass the Process instance to the struct. Why is that? Code:
use std::io::process::{Command, Process};
use std::rc::Rc;
use std::cell::RefCell;
struct DzenInst {
process: Rc<RefCell<Process>>
//stdin: std::io::pipe::PipeStream
}
impl DzenInst {
// Write a string to the stdin of dzen
fn write(&mut self, s : String) {
let mut stdin = self.process.borrow_mut().stdin.take().unwrap();
println!("Writing to dzen inst");
match stdin.write_str((s + "\n").as_slice()) {
Err(why) => panic!("couldn't write to dzen stdin: {}", why.desc),
Ok(_) => println!("Wrote string to dzen"),
};
}
}
fn CreateDzen() -> DzenInst {
DzenInst {process: Rc::new(RefCell::new(
match Command::new("dzen2").spawn() {
Err(why) => panic!("couldn't spawn dzen: {}", why.desc),
Ok(process) => process,
}))}
}
fn main() {
let mut d1 = CreateDzen();
d1.write("Test".to_string());
std::io::timer::sleep(std::time::duration::Duration::seconds(1));
}
If I write to the process stdin immediately inside CreateDzen, then it works just fine (i.e. the program does not go defunct). I'm assuming that copying the Process instance is causing its destructor to be invoked, which closes the process. Any ideas how to store the Process instance without causing the process to go defunct?
Your Rust code is fine. The problem here is dzen2. You need to add the -p flag to make dzen2 persist EOF (which is in the dzen2 README).
Instead of
match Command::new("dzen2").spawn() {
Use
match Command::new("dzen2").arg("-p").spawn() {