I'm trying to make an awk file executable. I've written the script, and did chmod +x filename. Here is the code:
#!/bin/awk -v
'TOPNUM = $1
## pick1 - pick one random number out of y
## main routine
BEGIN {
## set seed
srand ()
## get a random number
select = 1 +int(rand() * TOPNUM)
# print pick
print select
}'
When I try and run the program and put in a variable for the TOPNUM:
pick1 50
I get the response:
-bash: /home/petersone/bin/pick1: /bin/awk: bad interpreter: No such file or directory
I'm sure that there's something simple that I'm messing up, but I simply cannot figure out what it is. How can I fix this?
From a command line, run this command:
which awk
This will print the path of AWK, which is likely /usr/bin/awk. Correct the first line and your script should work.
Also, your script shouldn't have the single-quote characters at the beginning and end. You can run AWK from the command line and pass in a script as a quoted string, or you can write a script in a file and use the #!/usr/bin/awk first line, with the commands just in the file.
Also, the first line of your script isn't going to work right. In AWK, setup code needs to be inside the BEGIN block, and $1 is a reference to the first word in the input line. You need to use ARGV[1] to refer to the first argument.
http://www.gnu.org/software/gawk/manual/html_node/ARGC-and-ARGV.html
As #TrueY pointed out, there should be a -f on the first line:
#!/usr/bin/awk -f
This is discussed here: Invoking a script, which has an awk shebang, with parameters (vars)
Working, tested version of the program:
#!/usr/bin/awk -f
## pick1 - pick one random number out of y
## main routine
BEGIN {
TOPNUM = ARGV[1]
## set seed
srand ()
## get a random number
select = 1 +int(rand() * TOPNUM)
# print pick
print select
}
Actually this form is more preferrable:
#! /bin/awk -E
Man told:
-E Similar to -f, however, this is option is the last one processed and should be used with #! scripts, particularly for CGI applications, to avoid passing in options or source code (!) on the command line from a URL. This option disables command-line variable assignments
Related
I have 20 lines in file i want to remove all lines after nth line
nth line number is set using varible
eg var1=5 then remove line 5 to 20 using sed
Tried
sed ""$var1",$d" file -i
But it produces following error
sed: -e expression #1, char 5: unexpected `,'```
We can try
var="3"
awk -v line="$var" '1; FNR==line{exit}' file > temp && mv temp file
Details:
Here we are creating a shell variable var with value 3, then we are passing that variable to awk program, now the awk program is printing every line till the line number is 3 then the program is exited.
Use this Perl one-liner:
export var1=5
perl -i.bak -pe 'last if $. == $ENV{var1}' in_file
The Perl one-liner uses these command line flags:
-e : Tells Perl to look for code in-line, instead of in a file.
-p : Loop over the input one line at a time, assigning it to $_ by default. Add print $_ after each loop iteration.
-i.bak : Edit input files in-place (overwrite the input file). Before overwriting, save a backup copy of the original file by appending to its name the extension .bak.
$. : Current input line number.
SEE ALSO:
perldoc perlrun: how to execute the Perl interpreter: command line switches
I was beginner to linux thats why i asked such question.
Since no one answered.
I kept trying.
Now i learned how to debug linux shells and hence while using set -x i encountered mistake i was doing while writing up this command.
Now it is solved.
var=2
sed ""$(echo "$var")",\$d" file
I'm writing a bash scrip that would translate one file to another, and am encountering an issue.
Whenever the program sees something like this(......not included):
......Mul(-a1+b2-c3...+f+e)......
change it to:
......M(-a1)*M(b2)*M(-c3)*...*M(f)*M(e)......
the number of the variables in Mul is unknown and there could be multiple occurrence of Mul in the file. There are also other places in the file where + or - appears. And Variables could be one or more characters.
I tried grouping in sed, with a group followed by a "*", but it doesn't seem to be working due to the need of replacing unknown amount of variables.
Here is a sed script that will do it:
:a
s/\(Mul(.[^)]*\)\([+-].\)/\1)*Mul(\2/
ta
s/Mul(+\{0,1\}/M(/g
The trick is to use the test to jump back to the beginning after making a substitution (e.g. "Mul(a+b+c)"=>"Mul(a)*Mul(+b+c)").
$ cat tst.awk
match($0,/Mul\([^()]+\)/) {
tgt = substr($0,RSTART+4,RLENGTH-5)
gsub(/[-+][[:alnum:]]+/,"*M(&)",tgt)
gsub(/\+/,"",tgt)
sub(/^\*/,"",tgt)
print substr($0,1,RSTART-1) tgt substr($0,RSTART+RLENGTH)
}
$ awk -f tst.awk file
......M(-a1)*M(b2)*M(-c3)*M(f)*M(e)......
The above was run on this input file:
$ cat file
......Mul(-a1+b2-c3+f+e)......
Still having trouble figuring out how to preserve the contents of a given file using the following code that is attempting to rename the file based on a specific regex match within said file (i.e. within a given file there will always be one SMILE followed by 12 digits, e.g., SMILE000123456789).
for f in FILENAMEX_*; do awk '/SMILE[0-9]/ {OUT=$f ".txt"}; OUT {print >OUT}' ${f%.*}; done
This code is naming the file correctly but is simply printing out everything after the match instead of the entire contents of the file.
The list of files to be processed don't currently have an extension (and they need one for the next step) because I was using csplit to parse out the content from a larger file.
There are two problems: the first is using a shell variable in your awk program, and the second is the logic of the awk program itself.
To use a shell variable in awk, you can use
awk -v var="$var" '<program>'
and then use just var inside of awk.
For the second problem: if a line doesn't match your pattern and OUT is not set, you don't print the line. After the first line matching the pattern, OUT is set and you print. Since the match might be anywhere in the file, you have to store the lines at least up to the first match.
Here is a version that should work and is pretty close to your approach:
for f in FILENAMEX_*; do
awk -v f="${f%.*}" '
/SMILE[0-9]/ {
out=f".txt"
for (i=1;i<NR;++i) # Print file so far
print lines[i] > out
}
out { print > out } # Match has been seen: print
! out { lines[NR] = $0 } # No match yet: store
' "$f"
done
You could do some trickery and work with FILENAME or similar to do everything in a single invocation of awk, but since the main purpose is to find the presence of a pattern in the file, you're much better off using grep -q, which returns an exit status of 0 if the pattern has been found:
for f in FILENAMEX_*; do grep -q 'SMILE[0-9]' "$f" && cp "$f" "${f%.*}".txt; done
perhaps a different approach and just do each step separately..
ie pseudocode
for all files with some given text
extract text
rename file
I have a file with a content like this:
bird://localhost:${xfire.port}/${plfservice.url}
${configtool.store.folder}/config/hpctemplates.htb
I'd like to run a script which resolves all variables to get their values. Let's call the script resolve.sh. I'd like to loop through all matching pattern, but want to get an output including the original file, where the variables are substituted by the resolved values. Example for output:
bird://localhost:8043/net-littlecat-rbp.jms
/cat/dog/installation/configtool/config/hpctemplates.htb
So, resolve.sh runs where it finds a string between ${ } and the rest of the file remains the original. How can I get this?
Read the file and replace it globally with ENV vars then output it to the file which includes the origin filename, right?
Here is resolve.sh:
#!/bin/bash
PORT=1234 # xfire.port
URL=abc # plfservice.url
DIR=xyz # configtool.store.folder
# subsitude with ENV vars
C=$(sed -e "s/\${xfire.port}/${PORT}/g" \
-e "s_\${plfservice.url}_${URL}_g" \
-e "s_\${configtool.store.folder}_${DIR}_g"< $1)
# original filename
F="original filename: ${1%}"
# output to file which specified by $2
cat >$2 <<END
${C}
${F}
END
Call resolve.sh: ./resolve.sh <your-input-file> <your-output-file>
I haven't been able to figure how to do it awk nor sed. But in Perl it works like a charm:
perl -ape 's/\${([^}]*)}/$ENV{$1}/g'
first post here, but been a lurker for ages. i have googled for ages, but cant find what i want (many abigious topic subjects which dont request what the topic suggests it does ...). not new to awk or scripting, just a little rusty :)
i'm trying to write an awk script which will set shell env values as it runs - for another bash script to pick up and use later on. i cannot simply use stdout from awk to report this value i want setting (i.e. "export whatever=awk cmd here"), as thats already directed to a 'results file' which the awkscript is creating (plus i have more than one variable to export in the final code anyway).
As an example test script, to demo my issue:
echo $MYSCRIPT_RESULT # returns nothing, not currently set
echo | awk -f scriptfile.awk # do whatever, setting MYSCRIPT_RESULT as we go
echo $MYSCRIPT_RESULT # desired: returns the env value set in scriptfile.awk
within scriptfile.awk, i have tried (without sucess)
1/) building and executing an adhoc string directly:
{
cmdline="export MYSCRIPT_RESULT=1"
cmdline
}
2/) using the system function:
{
cmdline="export MYSCRIPT_RESULT=1"
system(cmdline)
}
... but these do not work. I suspect that these 2 commands are creating a subshell within the shell awk is executing from, and doing what i ask (proven by touching files as a test), but once the "cmd"/system calls have completed, the subshell dies, unfortunatley taking whatever i have set with it - so my env setting changes dont stick from "the caller of awk"'s perspective.
so my question is, how do you actually set env variables within awk directly, so that a calling process can access these env values after awk execution has completed? is it actually possible?
other than the adhoc/system ways above, which i have proven fail for me, i cannot see how this could be done (other than writing these values to a 'random' file somewhere to be picked up and read by the calling script, which imo is a little dirty anyway), hence, help!
all ideas/suggestions/comments welcomed!
You cannot change the environment of your parent process. If
MYSCRIPT_RESULT=$(awk stuff)
is unacceptable, what you are asking cannot be done.
You can also use something like is described at
Set variable in current shell from awk
unset var
var=99
declare $( echo "foobar" | awk '/foo/ {tmp="17"} END {print "var="tmp}' )
echo "var=$var"
var=
The awk END clause is essential otherwise if there are no matches to the pattern declare dumps the current environment to stdout and doesn't change the content of your variable.
Multiple values can be set by separating them with spaces.
declare a=1 b=2
echo -e "a=$a\nb=$b"
NOTE: declare is bash only, for other shells, use eval with the same syntax.
You can do this, but it's a bit of a kludge. Since awk does not allow redirection to a file descriptor, you can use a fifo or a regular file:
$ mkfifo fifo
$ echo MYSCRIPT_RESULT=1 | awk '{ print > "fifo" }' &
$ IFS== read var value < fifo
$ eval export $var=$value
It's not really necessary to split the var and value; you could just as easily have awk print the "export" and just eval the output directly.
I found a good answer. Encapsulate averything in a subshell!
The comand declare works as below:
#Creates 3 variables
declare var1=1 var2=2 var3=3
ex1:
#Exactly the same as above
$(awk 'BEGIN{var="declare "}{var=var"var1=1 var2=2 var3=3"}END{print var}')
I found some really interesting uses for this technique. In the next exemple I have several partitions with labels. I create variables using the labels as variable names and the device name as variable values.
ex2:
#Partition data
lsblk -o NAME,LABEL
NAME LABEL
sda
├─sda1
├─sda2
├─sda5 System
├─sda6 Data
└─sda7 Arch
#Creates a subshell to execute the text
$(\
#Pipe lsblk to awk
lsblk -o NAME,LABEL | awk \
#Initiate the variable with the text for the declare command
'BEGIN{txt="declare "}'\
#Filters devices with labels Arch or Data
'/Data|Arch/'\
#Concatenate txt with itself plus text for the variables(name and value)
#substr eliminates the special caracters before the device name
'{txt=txt$2"="substr($1,3)" "}'\
#AWK prints the text and the subshell execute as a command
'END{print txt}'\
)
The end result of this is 2 variables: Data with value sda6 and Arch with value sda7.
The same exemple in a single line.
$(lsblk -o NAME,LABEL | awk 'BEGIN{txt="declare "}/Data|Arch/{txt=txt$2"="substr($1,3)" "}END{print txt}')