How to iterate through string one word at a time in zsh - while-loop

How do I modify the following code so that when run in zsh it expands $things and iterates through them one at a time?
things="one two"
for one_thing in $things; do
echo $one_thing
done
I want the output to be:
one
two
But as written above, it outputs:
one two
(I'm looking for the behavior that you get when running the above code in bash)

In order to see the behavior compatible with Bourne shell, you'd need to set the option SH_WORD_SPLIT:
setopt shwordsplit # this can be unset by saying: unsetopt shwordsplit
things="one two"
for one_thing in $things; do
echo $one_thing
done
would produce:
one
two
However, it's recommended to use an array for producing word splitting, e.g.,
things=(one two)
for one_thing in $things; do
echo $one_thing
done
You may also want to refer to:
3.1: Why does $var where var="foo bar" not do what I expect?

Another way, which is also portable between Bourne shells (sh, bash, zsh, etc.):
things="one two"
for one_thing in $(echo $things); do
echo $one_thing
done
Or, if you don't need $things defined as a variable:
for one_thing in one two; do
echo $one_thing
done
Using for x in y z will instruct the shell to loop through a list of words, y, z.
The first example uses command substitution to transform the string "one two" into a list of words, one two (no quotes).
The second example is the same thing without echo.
Here's an example that doesn't work, to understand it better:
for one_thing in "one two"; do
echo $one_thing
done
Notice the quotes. This will simply print
one two
because the quotes mean the list has a single item, one two.

You can use the z variable expansion flag to do word splitting on a variable
things="one two"
for one_thing in ${(z)things}; do
echo $one_thing
done
Read more about this and other variable flags in man zshexpn, under "Parameter Expansion Flags."

You can assume the Internal Field Separator (IFS) on bash to be \x20 (space). This makes the following work:
#IFS=$'\x20'
#things=(one two) #array
things="one two" #string version
for thing in ${things[#]}
do
echo $thing
done
With this in mind you can implement this in many different ways just manipulating the IFS; even on multi-line strings.

Related

Running command in perl6, commands that work in shell produce failure when run inside perl6

I'm trying to run a series of shell commands with Perl6 to the variable $cmd, which look like
databricks jobs run-now --job-id 35 --notebook-params '{"directory": "s3://bucket", "output": "s3://bucket/extension", "sampleID_to_canonical_id_map": "s3://somefile.csv"}'
Splitting the command by everything after notebook-params
my $cmd0 = 'databricks jobs run-now --job-id 35 --notebook-params ';
my $args = "'{\"directory\": \"$in-dir\", \"output\": \"$out-dir\",
\"sampleID_to_canonical_id_map\": \"$map\"}'"; my $run = run $cmd0,
$args, :err, :out;
Fails. No answer given either by Databricks or the shell. Stdout and stderr are empty.
Splitting the entire command by white space
my #cmd = $cmd.split(/\s+/);
my $run = run $cmd, :err, :out
Error: Got unexpected extra arguments ("s3://bucket", "output":
"s3://bucket/extension",
"sampleID_to_canonical_id_map":
"s3://somefile.csv"}'
Submitting the command as a string
my $cmd = "$cmd0\"$in-dir\", \"output\": \"$out-dir\", \"sampleID_to_canonical_id_map\": \"$map\"}'";
again, stdout and stderr are empty. Exit code 1.
this is something about how run can only accept arrays, and not strings (I'm curious why)
If I copy and paste the command that was given to Perl6's run, it works when given from the shell. It doesn't work when given through perl6. This isn't good, because I have to execute this command hundreds of times.
Perhaps Perl6's shell https://docs.perl6.org/routine/shell would be better? I didn't use that, because the manual suggests that run is safer. I want to capture both stdout and stderr inside a Proc class
EDIT: I've gotten this running with shell but have encountered other problems not related to what I originally posted. I'm not sure if this qualifies as being answered then. I just decided to use backticks with perl5. Yes, backticks are deprecated, but they get the job done.
I'm trying to run a series of shell commands
To run shell commands, call the shell routine. It passes the positional argument you provide it, coerced to a single string, to the shell of the system you're running the P6 program on.
For running commands without involving a shell, call the run routine. The first positional argument is coerced to a string and passed to the operating system as the filename of the program you want run. The remaining arguments are concatenated together with a space in between each argument to form a single string that is passed as a command line to the program being run.
my $cmd0 = 'databricks jobs run-now --job-id 35 --notebook-params ';
That's wrong for both shell and run:
shell only accepts one argument and $cmd0 is incomplete.
The first argument for run is a string interpreted by the OS as the filename of a program to be run and $cmd0 isn't a filename.
So in both cases you'll get either no result or nonsense results.
Your other two experiments are also invalid in their own ways as you discovered.
this is something about how run can only accept arrays, and not strings (I'm curious why)
run can accept a single argument. It would be passed to the OS as the name of the program to be run.
It can accept two arguments. The first would be the program name, the second the command line passed to the program.
It can accept three or more arguments. The first would be the program name, the rest would be concatenated to form the command line passed to the program. (There are cases where this is more convenient coding wise than the two argument form.)
run can also accept a single array. The first element would the program name and the rest the command line passed to it. (There are cases where this is more convenient.)
I just decided to use backticks with perl5. Yes, backticks are deprecated, but they get the job done.
Backticks are subject to code injection and shell interpolation attacks and errors. But yes, if they work, they work.
P6 has direct equivalents of most P5 features. This includes backticks. P6 has two variants:
The safer P6 alternative to backticks is qx. The qx quoting construct calls the shell but does not interpolate P6 variables so it has the same sort of level of danger as using shell with a single quoted string.
The qqx variant is the direct equivalent of P5 backticks or using shell with a double quoted string so it suffers from the same security dangers.
Two mistakes:
the simplistic split cuts up the last, single parameter into multiple arguments
you are passing $cmd to run, not #cmd
use strict;
my #cmd = ('/tmp/dummy.sh', '--param1', 'param2 with spaces');
my $run = run #cmd, :err, :out;
print(#cmd ~ "\n");
print("EXIT_CODE:\t" ~ $run.exitcode ~ "\n");
print("STDOUT:\t" ~ $run.out.slurp ~ "\n");
print("STDERR:\t" ~ $run.err.slurp ~ "\n");
output:
$ cat /tmp/dummy.sh
#!/bin/bash
echo "prog: '$0'"
echo "arg1: '$1'"
echo "arg2: '$2'"
exit 0
$ perl6 dummy.pl
/tmp/dummy.sh --param1 param2 with spaces
EXIT_CODE: 0
STDOUT: prog: '/tmp/dummy.sh'
arg1: '--param1'
arg2: 'param2 with spaces'
STDERR:
If you can avoid generating $cmd as single string, I would generate it into #cmd directly. Otherwise you'll have to implement complex split operation that handles quoting.

awk/sed - generate an error if 2nd address of range is missing

We are currently using sed to filter output of regression runs. Sometimes we have a filter that looks like this:
/copyright/,/end copyright/d
If that end copyright is ever missing, the rest of the file is deleted. I'm wondering if there's some way to generate an error for this? awk would also be okay to use. I don't really want to add code that reads the file line by line and issues an error if it hits EOF.
here's a string
copyright
2016 jan 15
end copyright
date 2016 jan 5 time 15:36
last one
I'd like to get an error if end copyright is missing. The real filter also would replace the date line with DATE, so it's more that just ripping out the copyright.
You can persuade sed to generate an error if you reach end of input (i.e. see address $) between your start and end, but it won't be a very helpful message:
/copyright/,/end copyright/{
$s//\1/ # here
d
}
This will error if end copyright is missing or on the last line, with an exit status of 1 and the helpful message:
sed: -e expression #1, char 0: invalid reference \1 on `s' command's RHS
If you're using this in a makefile, you might want to echo a helpful message first, or (better) to wrap this in something that catches the error and produces a more useful one.
I tested this with GNU sed; though if you are using GNU sed, you could more easily use its useful extension:
q [EXIT-CODE]
This command only accepts a single address.
Exit 'sed' without processing any more commands or input. Note
that the current pattern space is printed if auto-print is not
disabled with the -n options. The ability to return an exit code
from the 'sed' script is a GNU 'sed' extension.
Q [EXIT-CODE]
This command only accepts a single address.
This command is the same as 'q', but will not print the contents of
pattern space. Like 'q', it provides the ability to return an exit
code to the caller.
So you could simply write
/copyright/,/end copyright/{
$Q 42
d
}
Never use range expressions /start/,/end/ as they make trivial code very slightly briefer but require a complete rewrite or duplicate conditions when you have the tiniest requirements change. Always use a flag instead. Note that since sed doesn't support variables, it doesn't support flag variables, and so you shouldn't be using sed you should be using awk instead.
In this case your original code would be:
awk '/copyright/{f=1} !f; /end copyright/{f=0}' file
And your modified code would be:
awk '/copyright/{f=1} !f; /end copyright/{f=0} END{if (f) print "Missing end copyright"}' file
The above is obviously untested since you didn't provide any sample input/output we could test a potential solution against.
With sed you can build a loop:
sed -e '/copyright/{:a;/end copyright/d;N;ba;};' file
:a defines the label "a"
/copyright end/d deletes the pattern space, only when "end copyright" matches
N appends the next line to the pattern space
ba jumps to the label "a"
Note that d ends the loop.
In this way you can avoid to delete the text until the end.
If you don't want the text to be displayed at all and prefer an error message when a "copyright" block stays unclosed, you obviously need to wait the end of the file. You can do it with sed too storing all the lines in the buffer space until the end:
sed -n -e '/copyright/{:a;/end copyright/d;${c\ERROR MESSAGE
;};N;ba;};H;${g;p};' file
H appends the current line to the buffer space
g put the content of the buffer space to the pattern space
The file content is only displayed once the last line reached with ${g;p} otherwise when the closing "end copyright" is missing, the current line is changed in the error message with ${c\ERROR MESSAGE\n;} inside the loop.
This way you can test what returns sed before redirecting it to whatever you want.

Until a specific substring

I need a batch command to return everything until after a certain substring.
What I mean, is when I have a string like this: "Hi! How are you doing? I don't care!!!!!" I can execute a command that gives me everything until after "?".
I looked around the web and didn't find anything that I wanted. I found one method that took everything until after a substring and changed it:
set name=123456789
set blablabla=%name:*5=5%
This returns "56789" to the variable blablabla. The strings in my program are not going to be specific, so this won't work.
Thank you for any help!
Use sed and regular expressions:
$ string1='Hello my name is Foobar? What do I care!'
$ string2=$(echo $string1 | sed 's/^.*\? //')
$ echo $string2
What do I care!
I have not understood your requirements fully. But I think PowerShell is a better option for this. However, you can use following script to get all characters upto given delimiter.
#echo off
set delim=%1
set input=%2
for /f "delims=%upto%" %%i in ("%input%") do (
echo %i
goto :eof
)
Sample run command:
stringupto.bat 5 123456789
Output:
1234

csh alias with variable number of arguments

I would like to create a csh alias that performs one operation if invoked without arguments and a second operation if invoked with a single argument. Does anyone know how to do this? (Attempting to refer to an argument that wasn't passed triggers an error).
I know this is a bit late but I just ran into needing something similar and hope it might still be relevant to somebody.
You can set the arguments as an array and query based on the size of the array:
alias testing 'set args_=(\!*); if ($#args_ > 0) echo "this command has $#args_ arguments" endif'
Aliases in tcsh are limited; for more advanced things, I've found that the best way is to source a (t)csh script, like so:
alias my-cmd 'source ~/.tcsh/my-cmd.tcsh'
And ~/.tcsh/my-cmd.tcsh would contain something like:
if ( $1 != '' ) then
echo "we have an argument: $1"
else
echo "we don't have an argument"
endif
Example output:
% my-cmd
we don't have an argument
% my-cmd hello
we have an argument: hello
Now, it may also be possible to do this with just an alias, but this will be much more maintainable & cleaner in the long run, IMHO.
(I've assumed tcsh here since almost all, or perhaps even all, c shells are tcsh these days).
Easy to do - sorry I'm late to the party.
alias iftest 'if (\\!:0 != \\!:$) echo "Last arg="\\!:$;if (\\!:0 == \\!:$) echo "No args given."'
This merely checks whether the 0th argument (=the 'iftest' itself) and the last arguments are the same, and if they are, assumes there is no argument. This is, of course, not necessarily true, but hopefully works in praxis.

Using echo and read $variable not working in UNIX! (UNIX beginner!)

So I just started learning UNIX yesterday, and I'm trying to create a basic script that asks for your contact details (name, address, phone number), and then stores that into a file called details.out.
This is driving me NUTS! Its such an easy/basic thing, yet I cant do it, and I've been stuck on it for a solid hour now...
after much googling and searching, I still can't find the answer. So this is what I've done so far, and was wondering where I am going wrong!
echo Please type your first and last name
read $firstname $lastname
echo Please type in your address
read $address
echo Please type in your phone number
read $phone
echo Thank you very much!
echo The details have been stored in '"details.out"'
cat >> details.out <<EOF
Name: echo $firstname echo $lastname
Address: echo $address
Phone Number: echo $phone
EOF
When I read "details.out" it it displays as follows:
Name: echo
Address: echo
Phone Number: echo
ANY help would be appreciated! (and if you get try and point me in the right directions as opposed to straight up giving me the answer, I would appreciate that!)
P.S I'm using Putty if that helps!
when you use read (or declaring variables), don't put $ sigil on the variable names
when you display a variable, always put double quotes around : ex. echo "$var"
when you use here-doc, no need to put echo command
when you use echo, use quotes :
"Double quote" every expansion, and anything that could contain a special character, eg. "$var", "$#", "${array[#]}", "$(command)". Use 'single quotes' to make something literal, eg. 'Costs $5 USD'. See http://mywiki.wooledge.org/Quotes http://mywiki.wooledge.org/Arguments and http://wiki.bash-hackers.org/syntax/words
Whenever you put a $ before a variable name, you're retrieving the current value of that variable. You don't want to do that in your read command. The variables are empty when the script starts, the empty values are put in place of the $firstname and $lastname and read is called with no arguments, causing it to read a line and discard it.
Setting a variable with assignment:
var=value
Setinng a variable with read:
read var
Neither of them use $var because they don't want to look at the current value, they want to replace it.
There's no need for those echos in the heredoc either. They aren't in command position, so they'll just get copied as part of the input to cat.