How do I store the value returned by either run or shell? - raku

Let's say I have this script:
# prog.p6
my $info = run "uname";
When I run prog.p6, I get:
$ perl6 prog.p6
Linux
Is there a way to store a stringified version of the returned value and prevent it from being output to the terminal?
There's already a similar question but it doesn't provide a specific answer.

You need to enable the stdout pipe, which otherwise defaults to $*OUT, by setting :out. So:
my $proc = run("uname", :out);
my $stdout = $proc.out;
say $stdout.slurp;
$stdout.close;
which can be shortened to:
my $proc = run("uname", :out);
say $proc.out.slurp(:close);
If you want to capture output on stderr separately than stdout you can do:
my $proc = run("uname", :out, :err);
say "[stdout] " ~ $proc.out.slurp(:close);
say "[stderr] " ~ $proc.err.slurp(:close);
or if you want to capture stdout and stderr to one pipe, then:
my $proc = run("uname", :merge);
say "[stdout and stderr] " ~ $proc.out.slurp(:close);
Finally, if you don't want to capture the output and don't want it output to the terminal:
my $proc = run("uname", :!out, :!err);
exit( $proc.exitcode );

The solution covered in this answer is concise.
This sometimes outweighs its disadvantages:
Doesn't store the result code. If you need that, use ugexe's solution instead.
Doesn't store output to stderr. If you need that, use ugexe's solution instead.
Potential vulnerability. This is explained below. Consider ugexe's solution instead.
Documentation of the features explained below starts with the quote adverb :exec.
Safest unsafe variant: q
The safest variant uses a single q:
say qx[ echo 42 ] # 42
If there's an error then the construct returns an empty string and any error message will appear on stderr.
This safest variant is analogous to a single quoted string like 'foo' passed to the shell. Single quoted strings don't interpolate so there's no vulnerability to a code injection attack.
That said, you're passing a single string to the shell which may not be the shell you're expecting so it may not parse the string as you're expecting.
Least safe unsafe variant: qq
The following line produces the same result as the q line but uses the least safe variant:
say qqx[ echo 42 ]
This double q variant is analogous to a double quoted string ("foo"). This form of string quoting does interpolate which means it is subject to a code injection attack if you include a variable in the string passed to the shell.

By default run just passes the STDOUT and STDERR to the parent process's STDOUT and STDERR.
You have to tell it to do something else.
The simplest is to just give it :out to tell it to keep STDOUT. (Short for :out(True))
my $proc = run 'uname', :out;
my $result = $proc.out.slurp(:close);
my $proc = run 'uname', :out;
for $proc.out.lines(:close) {
.say;
}
You can also effectively tell it to just send STDOUT to /dev/null with :!out. (Short for :out(False))
There are more things you can do with :out
{
my $file will leave {.close} = open :w, 'test.out';
run 'uname', :out($file); # write directly to a file
}
print slurp 'test.out'; # Linux
my $proc = run 'uname', :out;
react {
whenever $proc.out.Supply {
.print
LAST {
$proc.out.close;
done; # in case there are other whenevers
}
}
}
If you are going to do that last one, it is probably better to use Proc::Async.

Related

Running command in perl6, commands that work in shell produce failure when run inside perl6

I'm trying to run a series of shell commands with Perl6 to the variable $cmd, which look like
databricks jobs run-now --job-id 35 --notebook-params '{"directory": "s3://bucket", "output": "s3://bucket/extension", "sampleID_to_canonical_id_map": "s3://somefile.csv"}'
Splitting the command by everything after notebook-params
my $cmd0 = 'databricks jobs run-now --job-id 35 --notebook-params ';
my $args = "'{\"directory\": \"$in-dir\", \"output\": \"$out-dir\",
\"sampleID_to_canonical_id_map\": \"$map\"}'"; my $run = run $cmd0,
$args, :err, :out;
Fails. No answer given either by Databricks or the shell. Stdout and stderr are empty.
Splitting the entire command by white space
my #cmd = $cmd.split(/\s+/);
my $run = run $cmd, :err, :out
Error: Got unexpected extra arguments ("s3://bucket", "output":
"s3://bucket/extension",
"sampleID_to_canonical_id_map":
"s3://somefile.csv"}'
Submitting the command as a string
my $cmd = "$cmd0\"$in-dir\", \"output\": \"$out-dir\", \"sampleID_to_canonical_id_map\": \"$map\"}'";
again, stdout and stderr are empty. Exit code 1.
this is something about how run can only accept arrays, and not strings (I'm curious why)
If I copy and paste the command that was given to Perl6's run, it works when given from the shell. It doesn't work when given through perl6. This isn't good, because I have to execute this command hundreds of times.
Perhaps Perl6's shell https://docs.perl6.org/routine/shell would be better? I didn't use that, because the manual suggests that run is safer. I want to capture both stdout and stderr inside a Proc class
EDIT: I've gotten this running with shell but have encountered other problems not related to what I originally posted. I'm not sure if this qualifies as being answered then. I just decided to use backticks with perl5. Yes, backticks are deprecated, but they get the job done.
I'm trying to run a series of shell commands
To run shell commands, call the shell routine. It passes the positional argument you provide it, coerced to a single string, to the shell of the system you're running the P6 program on.
For running commands without involving a shell, call the run routine. The first positional argument is coerced to a string and passed to the operating system as the filename of the program you want run. The remaining arguments are concatenated together with a space in between each argument to form a single string that is passed as a command line to the program being run.
my $cmd0 = 'databricks jobs run-now --job-id 35 --notebook-params ';
That's wrong for both shell and run:
shell only accepts one argument and $cmd0 is incomplete.
The first argument for run is a string interpreted by the OS as the filename of a program to be run and $cmd0 isn't a filename.
So in both cases you'll get either no result or nonsense results.
Your other two experiments are also invalid in their own ways as you discovered.
this is something about how run can only accept arrays, and not strings (I'm curious why)
run can accept a single argument. It would be passed to the OS as the name of the program to be run.
It can accept two arguments. The first would be the program name, the second the command line passed to the program.
It can accept three or more arguments. The first would be the program name, the rest would be concatenated to form the command line passed to the program. (There are cases where this is more convenient coding wise than the two argument form.)
run can also accept a single array. The first element would the program name and the rest the command line passed to it. (There are cases where this is more convenient.)
I just decided to use backticks with perl5. Yes, backticks are deprecated, but they get the job done.
Backticks are subject to code injection and shell interpolation attacks and errors. But yes, if they work, they work.
P6 has direct equivalents of most P5 features. This includes backticks. P6 has two variants:
The safer P6 alternative to backticks is qx. The qx quoting construct calls the shell but does not interpolate P6 variables so it has the same sort of level of danger as using shell with a single quoted string.
The qqx variant is the direct equivalent of P5 backticks or using shell with a double quoted string so it suffers from the same security dangers.
Two mistakes:
the simplistic split cuts up the last, single parameter into multiple arguments
you are passing $cmd to run, not #cmd
use strict;
my #cmd = ('/tmp/dummy.sh', '--param1', 'param2 with spaces');
my $run = run #cmd, :err, :out;
print(#cmd ~ "\n");
print("EXIT_CODE:\t" ~ $run.exitcode ~ "\n");
print("STDOUT:\t" ~ $run.out.slurp ~ "\n");
print("STDERR:\t" ~ $run.err.slurp ~ "\n");
output:
$ cat /tmp/dummy.sh
#!/bin/bash
echo "prog: '$0'"
echo "arg1: '$1'"
echo "arg2: '$2'"
exit 0
$ perl6 dummy.pl
/tmp/dummy.sh --param1 param2 with spaces
EXIT_CODE: 0
STDOUT: prog: '/tmp/dummy.sh'
arg1: '--param1'
arg2: 'param2 with spaces'
STDERR:
If you can avoid generating $cmd as single string, I would generate it into #cmd directly. Otherwise you'll have to implement complex split operation that handles quoting.

Perl 6 $*ARGFILES.handles in binary mode?

I'm trying out $*ARGFILES.handles and it seems that it opens the files in binary mode.
I'm writing a zip-merge program, that prints one line from each file until there are no more lines to read.
#! /usr/bin/env perl6
my #handles = $*ARGFILES.handles;
# say $_.encoding for #handles;
while #handles
{
my $handle = #handles.shift;
say $handle.get;
#handles.push($handle) unless $handle.eof;
}
I invoke it like this: zip-merge person-say3 repeat repeat2
It fails with: Cannot do 'get' on a handle in binary mode in block at ./zip-merge line 7
The specified files are text files (encoded in utf8), and I get the error message for non-executable files as well as executable ones (with perl6 code).
The commented out line says utf8 for every file I give it, so they should not be binary,
perl6 -v: This is Rakudo version 2018.10 built on MoarVM version 2018.10
Have I done something wrong, or have I uncovered an error?
The IO::Handle objects that .handles returns are closed.
my #*ARGS = 'test.p6';
my #handles = $*ARGFILES.handles;
for #handles { say $_ }
# IO::Handle<"test.p6".IO>(closed)
If you just want get your code to work, add the following line after assigning to #handles.
.open for #handles;
The reason for this is the iterator for .handles is written in terms of IO::CatHandle.next-handle which opens the current handle, and closes the previous handle.
The problem is that all of them get a chance to be both the current handle, and the previous handle before you get a chance to do any work on them.
(Perhaps .next-handle and/or .handles needs a :!close parameter.)
Assuming you want it to work like roundrobin I would actually write it more like this:
# /usr/bin/env perl6
use v6.d;
my #handles = $*ARGFILES.handles;
# a sequence of line sequences
my $line-seqs = #handles.map(*.open.lines);
# Seq.new(
# Seq.new( '# /usr/bin/env perl6', 'use v6.d' ), # first file
# Seq.new( 'foo', 'bar', 'baz' ), # second file
# )
for flat roundrobin $line-seqs {
.say
}
# `roundrobin` without `flat` would give the following result
# ('# /usr/bin/env perl6', 'foo'),
# ('use v6.d', 'bar'),
# ('baz')
If you used an array for $line-seqs, you will need to de-itemize (.<>) the values before passing them to roundrobin.
for flat roundrobin #line-seqs.map(*.<>) {
.say
}
Actually I personally would be more likely to write something similar to this (long) one-liner.
$*ARGFILES.handles.eager».open».lines.&roundrobin.flat.map: *.put
:bin is always set in this type of objects. Since you are working on the handles, you should either read line by line as instructed on the example, or reset the handle so that it's not in binary mode.

TCL, How to name a variable that includes another variable

In TCL, I'm writing a procedure that returns the depth of a clock.
But since I have several clocks I want to name the var: depth_$clk
proc find_depth {} {
foreach clk $clocks {
…
set depth_$clk $max_depth
echo $depth_$clk
}
}
But I get:
Error: can't read "depth_": no such variable
Use error_info for more info. (CMD-013)
Your problem is this line:
echo $depth_$clk
The issue is that the syntax for $ only parses a limited set of characters afterwards for being part of the variable name; the $ is not part of that. Instead, you can use the set command with one argument; $ is effectively syntactic sugar for that, but the command lets you use complex substitutions.
echo [set depth_$clk]
HOWEVER!
The real correct thing to do here is to switch to using an associative array. It's a bit larger change to your code, but lets you do more as you've got proper access to substitutions in array element names:
proc find_depth {} {
foreach clk $clocks {
…
set depth($clk) $max_depth
echo $depth($clk)
}
}
echo ${depth}_$cell
This can help too.
Thanks !

perl6 Is there a way to do editable prompt input?

In bash shell, if you hit up or down arrows, the shell will show you your previous or next command that you entered, and you can edit those commands to be new shell commands.
In perl6, if you do
my $name = prompt("Enter name: ");
it will print "Enter name: " and then ask for input; is there a way to have perl6 give you a default value and then you just edit the default to be the new value. E.g.:
my $name = prompt("Your name:", "John Doe");
and it prints
Your name: John Doe
where the John Doe part is editable, and when you hit enter, the edited string is the value of $name.
https://docs.raku.org/routine/prompt does not show how to do it.
This is useful if you have to enter many long strings each of which is just a few chars different from others.
Thanks.
To get the editing part going, you could use the Linenoise module:
zef install Linenoise
(https://github.com/hoelzro/p6-linenoise)
Then, in your code, do:
use Linenoise;
sub prompt($p) {
my $l = linenoise $p;
linenoiseHistoryAdd($l);
$l
}
Then you can do your loop with prompt. Remember, basically all Perl 6 builtin functions can be overridden lexically. Now, how to fill in the original string, that I haven't figure out just yet. Perhaps the libreadline docs can help you with that.
Well by default, programs are completely unaware of their terminals.
You need your program to communicate with the terminal to do things like pre-fill an input line, and it's unreasonable to expect Perl 6 to handle something like this as part of the core language.
That said, your exact case is handled by the Readline library as long as you have a compatible terminal.
It doesn't look like the perl 6 Readline has pre-input hooks setup so you need to handle the callback and read loop yourself, unfortunately. Here's my rough attempt that does exactly what you want:
use v6;
use Readline;
sub prompt-prefill($question, $suggestion) {
my $rl = Readline.new;
my $answer;
my sub line-handler( Str $line ) {
rl_callback_handler_remove();
$answer = $line;
}
rl_callback_handler_install( "$question ", &line-handler );
$rl.insert-text($suggestion);
$rl.redisplay;
while (!$answer) {
$rl.callback-read-char();
}
return $answer;
}
my $name = prompt-prefill("What's your name?", "Bob");
say "Hi $name. Go away.";
If you are still set on using Linenoise, you might find the 'hints' feature good enough for your needs (it's used extensively by the redis-cli application if you want a demo). See the hint callback used with linenoiseSetHintsCallback in the linenoise example.c file. If that's not good enough you'll have to start digging into the guts of linenoise.
Another solution :
Use io-prompt
With that you can set a default value and even a default type:
my $a = ask( "Life, the universe and everything?", 42, type => Num );
Life, the universe and everything? [42]
Int $a = 42
You can install it with:
zef install IO::Prompt
However, if just a default value is not enough. Then it is better you use the approach Liz has suggested.

Powershell piping to variable and write-host at the same time

Hello all and thanks for your time in advance;
I'm running into a slight issue.
I'm running a command and piping it into a variable so i can manipulate the output.
$variable = some command
this normally works fine but doesn't output what's happening to the screen, which is fine most of the time. However occasionally this command requires some user input (like yes or no or skip for example), and since it's not actually piping anything to the command window, it just sits there hanging instead of prompting the user. If you know what to expect you can hit y or n or s and it'll proceed normally.
Is there anyway to run the command so that the output is piped to a variable AND appears on screen? I've already tried:
($variable = some command)
I've also tried:
write-host ($variable = some command)
But neither work. Note that the command running isn't a native windows or shell command and I cannot just run it twice in a row.
To clarify (because i probably wasn't clear)
I've also tried :
$variable = some command : Out-host
and
$variable = some command | out-default
with all their parameters, But the "prompt" from the command (to write y, n, s) doesn't show up.
Being able to pass S automatically would also be acceptable.
Sounds like you need Tee-Object. Example:
some command | Tee-Object -Variable MyVariable
This should pass everything from the command down the pipe, as well as store all output from the command in the $MyVariable variable for you.
You need to give some specific example that doesn't work. I tried this and it works:
function test { $c = read-host "Say something"; $c }
$x = test
I still see "Say something". read-host does not output to standard output so your problem is surprising. Even this works:
read-host "Say something" *> out
=== EDIT ===
Since this is interaction with cmd.exe you have two options AFAIK. First, test command:
test.cmd
#echo off
set /p something="Say something: "
echo %something%
This doesn't work as you said: $x= ./test.cmd
To make it work:
a) Replace above command with: "Say something:"; $x= ./test.cmd. This is obviously not ideal in general scenario as you might not know in advance what the *.cmd will ask. But when you do know its very easy.
b) Try this:
Start-transcript test_out;
./test.cmd;
Stop-transcript;
gc .\test_out | sls 'test.cmd' -Context 0,1 | select -Last 1 | set y
rm test_out
$z = ($y -split "`n").Trim()
After this $z variable contains: Say something: something. This could be good general solution that you could convert to function:
$z = Get-CmdOutput test.cmd
Details of text parsing might be slightly different in general case - I assumed here that only 1 question is asked and answer is on the same line but in any case with some work you will be able to get everything cmd.exe script outputed in general case:
=== EDIT 2 ===
This might be a better general extraction:
$a = gi test_out; rm test_out
$z = $a | select -Index 14,($a.count-5)
$z
$variable = ""
some command | % {$variable += $_;"$_"}
This executes the command, and each line of output is both added to $variable and printed to the console.