Cannot get apache mod_perl with mpirun to work - apache

Question

I am trying to build a simple web service that runs on an ubuntu machine with apache and mod_perl2. The service runs mpirun and returns the output of the call.
I invoke a call of the apache response handler via the web browser. The problem is that the mpirun command seems to hang.
Important:
This problem occurs on a server running Ubuntu (12.04.4) with apache, mod_perl and openmpi. When running it on my mac (Macos 10.9.3), it works fine and mpirun returns. On both machines, openmpi is installed in the same version (1.6.5)
Here my mod_perl handler:
package MyHandler;
use Apache2::Const '-compile' => 'OK';
sub handler {
my $command = "mpirun -np 4 echo test";
my $out = qx($command);
print $out;
return Apache2::Const::OK;
}
1;
The mpirun job does not seem to finish. A ps aux | grep mpirun gives me this:
www-data 24023 0.0 0.1 23600 2424 ? S 13:02 0:00 mpirun -np 4 echo test
When I do a kilall -9 mpirun, the service comes back with the result.
No errors are written to the apache error log.
Here is what I tried/tested:
made sure that the command mpirun -np 4 echo test generates the correct output when run as user www-data
tried to invoke mpirun in different ways: Using IPC::Run and IPC::Run3, as suggested by Sergei, I also tried using pipes, but everytime mpirun dies not finish.
tried to call the handler directly via a perl script and not via the browser: mpirun finishes and the handler prints the desired output.
compared the outputs of ompi_info --param mpi all on both machines, mac and ubuntu, but found no differences
Any idea why mpirun would hang in my situation or any idea how I could debug this?
Edit
I tried to use Apache2::SubProcess as suggested by hrunting. Here my code following the simple example from the link:
package MyHandler;
use Apache2::SubProcess ();
use Apache2::Const '-compile' => 'OK';
use Apache2::Request;
use Config;
use constant PERLIO_IS_ENABLED => $Config{useperlio};
sub handler {
my $r = shift;
my $command = "mpirun -np 4 echo test";
my ($in_fh, $out_fh, $err_fh) = $r->spawn_proc_prog($command);
$r->content_type('text/plain');
my $output = read_data($out_fh);
my $error = read_data($err_fh);
print "output : $output \n";
print "error : $error \n";
return Apache2::Const::OK;
}
# helper function to work w/ and w/o perlio-enabled Perl
sub read_data {
my ($fh) = #_;
my $data;
if (PERLIO_IS_ENABLED || IO::Select->new($fh)->can_read(10)) {
$data = <$fh>;
}
return defined $data ? $data : '';
}
1;
This does not work for me. When calling the handler from the browser, I get the output:
output :
error :
and ps aux tells me that mpirun is not running.
Any further ideas of how I could debug this and get mpirun to work with my configuration?

Answer 1

Look at Apache2::SubProcess. When you're running external processes within a mod_perl handler, Apache memory, I/O and process management come into play. Remember, your code is running within Apache itself and is subject to the Apache environment. The Apache2::SubProcess module is designed to make exec()- and system()-style calls work properly under within Apache.
Note that the module documentation outlines caveats for dealing with different Perl configurations.

Answer 2

Try IPC::Run or IPC::Run3 to run your command.

Answer 3

Capture::Tiny works for me. I'm not sure it'll work well under mod_perl (it may interact badly with the file handles for the request and response) but it works fine as a regular script:
use Capture::Tiny 'capture';
my ( $stdout, $stderr, $exit ) = capture {
system( qw(mpirun -np 4 echo test) );
};
print "stdout: $stdout\n";
print "stderr: $stderr\n";
print "exit: $exit\n";
Prints:
stdout: test
test
test
test
stderr:
exit: 0