I have this daunting chunk of code:
exec zip -r $FULLPATH error.log [ append datetime $RECENT ".bwdb" ] [ append txt $testName ".txt" ] [ append lancap $testName "-lan.cap" ] \
[ append lanmcap $testName "-lan-m.cap" ] [ append wancap $testName "-wan.cap" ] [ append wanmcap $testName "-wan-m.cap" ] [ append conf $confFile ".conf" ] \
start.txt start-lan.cap start-lan-m.cap start-wan.cap start-wan-m.cap [ append comments "comments-" $RECENT ".bwc" ]
Which is actually just zipping a bunch of files together. I'm changing the way I generate the zip files (more accurately I'm changing when I generate them). The way I've decided to do it is to save every command into a text file and then just go through every command in the file when I need to create them. The issue is that I have a bunch of variables that I need to convert into literal form.
Does anyone know how I could convert this command into it's literal values and store it into a text file?
Edit: I'd also be willing to hear other suggestions that solve the same issue, and/or any pros/cons in doing it the way I'm suggesting.
Edit2: Thanks for all the help, I opted to do this:
set data123 "exec zip -r $FULLPATH error.log [ append datetime $RECENT \".bwdb\" ] [ append txt $testName \".txt\" ] [ append lancap $testName \"-lan.cap\" ] \
[ append lanmcap $testName \"-lan-m.cap\" ] [ append wancap $testName \"-wan.cap\" ] [ append wanmcap $testName \"-wan-m.cap\" ] [ append conf $confFile \".conf\" ] \
start.txt start-lan.cap start-lan-m.cap start-wan.cap start-wan-m.cap [ append comments \"comments-\" $RECENT \".bwc\" ]"
set datafile123 "datafile123.txt"
set fileId123 [ open $datafile123 "w" ]
puts $fileId123 $data123
close $fileId123
However, when I look at the resulting file I see:
exec zip -r /home/IOL/TR069_Certification/Results/TEST_Round99_GetRPCMethods_CDR1_20130410172812 error.log 20130410172812.bwdb20130410172812".bwdb" GetRPCMethods.txtGetRPCMethods".txt" GetRPCMethods-lan.capGetRPCMethods"-lan.cap" GetRPCMethods-lan-m.capGetRPCMethods"-lan-m.cap" GetRPCMethods-wan.capGetRPCMethods"-wan.cap" GetRPCMethods-wan-m.capGetRPCMethods"-wan-m.cap" IOL.confIOL".conf" start.txt start-lan.cap start-lan-m.cap start-wan.cap start-wan-m.cap comments-20130410172812.bwc"comments-"20130410172812".bwc"
Which appears to be making a second copy of the concatenated string at the end of each append and I can't figure out why.
Why not as simple string? You know that a " string can span multiple lines?
set f [open /p/t/f w]
puts $f "error.log
$datetume$RECENT.bwdb
$txt$testName.txt
$lancap$testName-lan.cap
$lanmcap$testName-lan-m.cap
$wancap$testName-wan.cap
$wanmcap$testName-$wan.cap
$conf$confFile.conf
start.txt
start-lan.cap
start-lan-m.cap
start-wan.cap
start-wan-m.cap
${comments}comments-$RECENT.bwc"
close $f
Edit: You should not indent the follow up lines, otherwise this indentation will show up in the resulting file.
I personally usually use a template file, and use subst to replace the variables.
I would basically just do what you proposed: wrote the literal values to a text file.
One way to do that would be as simple as this:
set s ""
append s \
error.log \n \
$datetime $RECENT .bwdb \n \
$txt $testName .txt \n \
$lancap $testName -lan.cap \n \
$lanmcap $testName -lan-m.cap \n \
$wancap $testName -wan.cap \n \
$wanmcap $testName -wan-m.cap \n \
$conf $confFile .conf \n \
start.txt \n start-lan.cap \n \
start-lan-m.cap \n start-wan.cap \n start-wan-m.cap \n \
$comments comments- $RECENT .bwc
set f [open /path/to/file w]
puts $f $s
close $f
This would create a text files with one filename per line.
Another — more direct — rework could look like this:
set files [list]
lappend files \
error.log \
[ append datetime $RECENT ".bwdb" ] \
[ append txt $testName ".txt" ] \
[ append lancap $testName "-lan.cap" ] \
[ append lanmcap $testName "-lan-m.cap" ] \
[ append wancap $testName "-wan.cap" ] \
[ append wanmcap $testName "-wan-m.cap" ] \
[ append conf $confFile ".conf" ] \
start.txt start-lan.cap start-lan-m.cap start-wan.cap start-wan-m.cap \
[ append comments "comments-" $RECENT ".bwc" ]
set f [open /path/to/file w]
puts $f [join $files \n]
close $f
I'm suspicious about this usage of append though as here this command seems to be used solely for its side-effect. append takes the strings of its second to last arguments and appends them to a string located in a variable whose name is passed as its first argument, returning the resulting string. And I dislike the fact that in your original example (and in my second one) append seems to be used just for that return value.
So probably a more idiomatic way would be to use the Tcl's way of simple string concatenation, like this:
set files [list]
lappend files \
error.log \
${datetime}${RECENT}.bwdb \
${txt}${testName}.txt \
${lancap}${testName}-lan.cap \
${lanmcap}${testName}-lan-m.cap \
${wancap}${testName}-wan.cap \
${wanmcap}${testName}-wan-m.cap \
${conf}${confFile}.conf \
start.txt start-lan.cap start-lan-m.cap start-wan.cap start-wan-m.cap \
${comments}comments-${RECENT}.bwc
set f [open /path/to/file w]
puts $f [join $files \n]
close $f
Related
I tried to run my nextflow script and the first two precess worded fine, but the third process Conbinevcf reported an error, showing that the variable prefix was not found.
process Annovar_genebased {
publishDir "${params.output}/annovar", mode: 'copy'
input:
path 'snp_anatation' from anatation1.flatMap()
val humandb
val refgene
output:
path "*.exonic_variant_function" into end
"""
prefix=\$(basename \$(readlink snp_anatation) .avinput)
perl $refgene -geneanno -dbtype refGene -out \${prefix}.anatation -buildver hg19 $snp_anatation $humandb -hgvs
rm *.log
rm *.variant_function
"""
}
process Annovar {
publishDir "${params.output}/annovar", mode: 'copy'
input:
path 'snp_anatation' from anatation2.flatMap()
val annovar_table
val humandb
output:
path "*.csv" into end1
"""
prefix=\$(basename \$(readlink snp_anatation) .avinput)
perl $annovar_table $snp_anatation $humandb -buildver hg19 -out \${prefix}.anatation -remove -protocol refGene,cytoBand,exac03,clinvar_20200316,gnomad211_exome -operation g,r,f,f,f -nastring . -csvout -polish
"""
}
I got stuck on this process
process Combinevcf {
publishDir "${params.output}/combinevcf", mode: 'copy'
input:
path 'genebased' from end.flatMap()
path 'allbased' from end1.flatMap()
output:
path "*_3.csv" into end3
"""
prefix=\$(basename \$(readlink genebased) .exonic_variant_function)
prefix1=\$(basename \$(readlink allbased) .csv)
cat ${prefix}.exonic_variant_function | tr -s ‘[:blank:]’ ‘,’ | awk 'BEGIN{FS=",";OFS="," }{ print \$3,\$13,\$22}' | awk ' BEGIN { OFS=", "; print "refGene", "refGene", "refGene", "refGene", "refGene", "Zogysity","chr", "filter" } { print \$0, "" } ' > ${prefix}_1.csv
awk 'BEGIN{FS=",";OFS="," }{ print \$1,\$2,\$3,\$4,\$5,\$6,\$7,\$8,\$9,\$10,\$15,\$21,\$24,\$25}' ${prefix1}.csv > ${prefix1}_2.csv
paste ${prefix}_1.csv ${prefix1}_2.csv > ${prefix}_3.csv
"""
}
I am not sure what went wrong, any help would be appreciated.
You need to escape your ${prefix} with backslashes to tell nextflow that the variable prefix is in the script block scope, and not in the nextflow scope.
See https://www.nextflow.io/docs/latest/process.html#script for more info:
Since Nextflow uses the same Bash syntax for variable substitutions in strings, you must manage them carefully depending on whether you want to evaluate a Nextflow variable or a Bash variable
I'm trying to change the value of a Boolean variable using expression in bash script (Debian Jessie) but it's not working as intended!
Here is my code
vld=true
while ! [ $vld ]
do
echo "Enter a number: [1-30]:"
read myinput
$vld=[ $myinput -ge 1 ] && [ $myinput -le 30 ]
done
echo "Your number is $myinput
When I this script it says:
8: ./test.sh true=[: not found
Then it keeps on the loop as the variable is not receiving the result of the expression but is used itself!
#!/bin/bash
echo "Please type the file name"
read filename
echo "Please type the word or phrase you wish to look for"
read string
grep '$string' /home/pi/$filename
I was wondering how I could make grep a variable so I could use a code like this:
if [ $var=~$string ];
then
echo "the string is there
else
echo "sorry string doesn't exist"
To assign the output of grep to a variable:
var = $(grep '$string' /home/pi/$filename)
EDIT
As #staticx pointed out, in your case, where you grep for $string and see if the result again matches $string, it would be easier just to see if grep finds the element by piping it wc -l.
count = $(grep '$string' /home/pi/$filename | wc -l)
if [ count -gt 0 ]; then
# do stuff
fi
I'm having trouble making a script to replace a string that is inside double quotes. The files sections looks like this:
"regA~1" : "FF_NODE~94"
"regA~2" : "FF_NODE~105"
"regA~3" : "FF_NODE~116"
"regA~4" : "FF_NODE~127"
"regA~5" : "FF_NODE~138"
"regA~6" : "FF_NODE~149"
"regA~7" : "FF_NODE~154"
"regA~8" : "FF_NODE~155"
"regA~9" : "FF_NODE~156"
"regA~1" : "FF_NODE~95"
"regA~11" : "FF_NODE~96"
It works if I do
awk '/"regA~1"/{c++;if(c==2){sub("regA~1","regA~10");}}1' file > file_out
but when trying to make this a script where I pass a variable regA~1 and the value for c it doesn't work.
s="regA~1";
r="regA~10";
n=2;
awk -v search="$s" -v replace="$r" -v count=$n '/search/{c++;if(c==count){sub(search,replace);}}1' file > file_out
I also tried
awk -v search=$s -v replace=$r -v count=$n '/search/{c++;if(c==count){sub(search,replace);}}1' file > file_out
The syntax you need to match an RE that's stored as a string in a variable is
$0 ~ var
not
/var/
Thanks to Ed Morton for the tip. Here's the bash script in case anyone needs something like this. Not very sophisticated but it works for me.
#!/bin/bash
# Replaces a specific occurrence of a search string with a replace string
if [ $# -lt 4 ] ; then
echo -e "Wrong number of parameters."
echo -e "Usage:"
echo -e "repnthstr file search replace n"
echo -e "repnthstr fileext search replace n"
exit 1
fi
for file in $1
do
if [ -f $file -a -r $file ]; then
awk -v search=$2 -v replace=$3 -v cnt=$4 '$0 ~ search{c++;if(c==cnt){sub(search,replace);}}1' "$file" > temp && mv temp "$file"
else
echo "Error: Cannot read $file"
fi
done
This is similar to another issue, but I only want make to prompt for a value if I'm running a specific target and a mandatory variable has not been specified.
The current code:
install-crontab: PASSWORD ?= "$(shell read -p "Password: "; echo "$$REPLY")"
install-crontab: $(SCRIPT_PATH)
#echo "#midnight \"$(SCRIPT_PATH)\" [...] \"$(PASSWORD)\""
This just results in the following output and no prompt:
Password: read: 1: arg count
#midnight [...] ""
The important point here is that I have to ask only when running this target, and only if the variable has not been defined. I can't use a configure script, because obviously I shouldn't store passwords in a config script, and because this target is not part of the standard installation procedure.
Turns out the problem was that Makefiles don't use Dash / Bash-style quotation, and that Dash's read built-in needs a variable name, unlike Bash. Resulting code:
install-crontab-delicious: $(DELICIOUS_TARGET_PATH)
#while [ -z "$$DELICIOUS_USER" ]; do \
read -r -p "Delicious user name: " DELICIOUS_USER;\
done && \
while [ -z "$$DELICIOUS_PASSWORD" ]; do \
read -r -p "Delicious password: " DELICIOUS_PASSWORD; \
done && \
while [ -z "$$DELICIOUS_PATH" ]; do \
read -r -p "Delicious backup path: " DELICIOUS_PATH; \
done && \
( \
CRONTAB_NOHEADER=Y crontab -l || true; \
printf '%s' \
'#midnight ' \
'"$(DELICIOUS_TARGET_PATH)" ' \
"\"$$DELICIOUS_USER\" " \
"\"$$DELICIOUS_PASSWORD\" " \
"\"$$DELICIOUS_PATH\""; \
printf '\n') | crontab -
Result:
$ crontab -r; make install-crontab-delicious && crontab -l
Delicious user name: a\b c\d
Delicious password: e f g
Delicious backup path: h\ i
no crontab for <user>
#midnight "/usr/local/bin/export_Delicious" "a\b c\d" "e f g" "h\ i"
$ DELICIOUS_PASSWORD=foo make install-crontab-delicious && crontab -l
Delicious user name: bar
Delicious backup path: baz
#midnight "/usr/local/bin/export_Delicious" "a\b c\d" "e f g" "h\ i"
#midnight "/usr/local/bin/export_Delicious" "bar" "foo" "baz"
This code:
treats all input characters as literals, so it works with spaces and backslashes,
avoids problems if the user presses Enter without writing anything,
uses environment variables if they exist, and
works whether crontab is empty or not.
l0b0's answer helped me with a similar problem where I wanted to exit if the user doesn't input 'y'. I ended up doing this:
#while [ -z "$$CONTINUE" ]; do \
read -r -p "Type anything but Y or y to exit. [y/N] " CONTINUE; \
done ; \
if [ ! $$CONTINUE == "y" ]; then \
if [ ! $$CONTINUE == "Y" ]; then \
echo "Exiting." ; exit 1 ; \
fi \
fi
I hope that helps someone. It's hard to find more info about using user input for an if/else in a makefile.