I have mutilple list of line, for example:
root : root
user : user adm cdrom sudo dip
I want it to look like this.
root : root
user : user
user : adm
user : cdrom
user : sudo
user : dip
Thankyou In Advance
Can be achieved f.ex. with a simple Perl one-liner:
$ cat dummy.txt
root : root
user : user adm cdrom sudo dip
$ perl <dummy.txt -lane 'print "$F[0] : $_" for (#F[2..$#F])'
root : root
user : user
user : adm
user : cdrom
user : sudo
user : dip
BONUS CODE: dissection of the Perl one-liner:
$ perl -MO=Deparse -lane 'print "$F[0] : $_" for (#F[2..$#F])'
BEGIN { $/ = "\n"; $\ = "\n"; }
LINE: while (defined($_ = readline ARGV)) {
chomp $_;
our #F = split(' ', $_, 0);
print "$F[0] : $_" foreach (#F[2 .. $#F]);
}
Related
I have a configuration file structured as follows:
[section1]
path = <path1>
read only = yes
guest ok = yes
[section2]
path = <path2>
read only = no
guest ok = yes
I would need to be able to replace a whole section of the configuration file with a new section using the sed command.
Example of what I would like to achieve:
sudo sed -E -i ':a;N;$!ba;\[section1\]<regex_match_until_end_of_section1>/<new_section_1>/' <config_path>
Expected result:
<new_section_1>
[section2]
path = <path2>
read only = no
guest ok = yes
sudo sed -E -i ':a;N;$!ba;\[section2\]<regex_match_until_end_of_section2>/<new_section_2>/' <config_path>
Expected result:
[section1]
path = <path1>
read only = yes
guest ok = yes
<new_section_2>
While sed may be able to do this but using awk is much more intuitive and clean:
awk -v s='[section1]' -v RS= '$1 == s {$0 = "<new_section_1>"}
{ORS=RT} 1' file
<new_section_1>
[section2]
path = <path2>
read only = no
guest ok = yes
Or:
awk -v s='[section2]' -v RS= '$1 == s {$0 = "<new_section_2>"}
{ORS=RT} 1' file
[section1]
path = <path1>
read only = yes
guest ok = yes
<new_section_2>
With your shown samples only(considering that in between a complete section you don't have new lines but your sections are separated by new lines) please try following awk code, written and tested with GNU awk should work in any awk. Where s1 is awk variable which has value to be compared(for section) and s2 is containing new value in it.
awk -v s1='[section1]' -v s2='<new_section_1>' '
$0~/^\[/{
val=found=""
if($0==s1){ found=1 }
}
!NF{
if(!found){ print val }
else { print s2 }
}
{
val=(val?val ORS:"")$0
}
!NF
END{
if(!found){ print val }
}
' Input_file
I have a perl program which I am downloading and using in a docker file.
This program is calling tbl2asn but I do not need the output and this part runs very long. Therefore, I want to comment a few lines out of this perl program.
runcmd(
"tbl2asn -V b -a r10k -l paired-ends -M n -N 1 -y 'Annotated using $EXE $VERSION from $URL' -Z".
" \Q$outdir/$prefix.err\E -i \Q$outdir/$prefix.fsa\E 2> /dev/null"
);
delfile("$outdir/errorsummary.val");
delfile( map { "$outdir/$prefix.$_" } qw(dr fixedproducts ecn val) );
msg("Repairing broken .GBK output that tbl2asn produces...");
runcmd("sed 's/COORDINATES: profile/COORDINATES:profile/' < \Q$outdir/$prefix.gbf\E > \Q$outdir/$prefix.gbk\E");
delfile("$outdir/$prefix.gbf");
These are the lines I want to comment and at least this line "tbl2asn -V b -a r10k -l paired-ends -M n -N 1 -y 'Annotated using $EXE $VERSION from $URL' -Z". only occurs once in the perl program.
How can I use awk or sed to comment this line via a regex, one before and eight lines after?
Thank you.
Awk solution:
Sample file.pl:
some text
runcmd(
"tbl2asn -V b -a r10k -l paired-ends -M n -N 1 -y 'Annotated using $EXE $VERSION from $URL' -Z".
" \Q$outdir/$prefix.err\E -i \Q$outdir/$prefix.fsa\E 2> /dev/null"
);
delfile("$outdir/errorsummary.val");
delfile( map { "$outdir/$prefix.$_" } qw(dr fixedproducts ecn val) );
msg("Repairing broken .GBK output that tbl2asn produces...");
runcmd("sed 's/COORDINATES: profile/COORDINATES:profile/' < \Q$outdir/$prefix.gbf\E > \Q$outdir/$prefix.gbk\E");
delfile("$outdir/$prefix.gbf");
some text
awk '/"tbl2asn -V b .+ -Z"\.$/{ n = NR + 9 }NR <= n{ rec = "#"rec }
rec{ print rec }{ rec = $0 }
END{ print rec }' file.pl
The output:
some text
#runcmd(
# "tbl2asn -V b -a r10k -l paired-ends -M n -N 1 -y 'Annotated using $EXE $VERSION from $URL' -Z".
# " \Q$outdir/$prefix.err\E -i \Q$outdir/$prefix.fsa\E 2> /dev/null"
#);
#delfile("$outdir/errorsummary.val");
#delfile( map { "$outdir/$prefix.$_" } qw(dr fixedproducts ecn val) );
#
#msg("Repairing broken .GBK output that tbl2asn produces...");
#runcmd("sed 's/COORDINATES: profile/COORDINATES:profile/' < \Q$outdir/$prefix.gbf\E > \Q$outdir/$prefix.gbk\E");
#delfile("$outdir/$prefix.gbf");
some text
I have an smb.conf ini file which is overwritten whenever edited with a certain GUI tool, wiping out a custom setting. This means I need a cron job to ensure that one particular section in the file contains a certain option=value pair, and insert it at the end of the section if it doesn't exist.
Example
Ensure that hosts deny=192.168.23. exists within the [myshare] section:
[global]
printcap name = cups
winbind enum groups = yes
security = user
[myshare]
path=/mnt/myshare
browseable=yes
enable recycle bin=no
writeable=yes
hosts deny=192.168.23.
[Another Share]
invalid users=nobody,nobody
valid users=nobody,nobody
path=/mnt/share2
browseable=no
Long-winded solution using awk
After a long time struggling with sed, I concluded that it might not be the right tool for the job. So I moved over to awk and came up with this:
#!/bin/sh
file="smb.conf"
tmp="smb.conf.tmp"
section="myshare"
opt="hosts deny=192.168.23."
awk '
BEGIN {
this_section=0;
opt_found=0;
}
# Match the line where our section begins
/^[ \t]*\['"$section"'\][ \t]*$/ {
this_section=1;
print $0;
next;
}
# Match lines containing our option
this_section == 1 && /^[ \t]*'"$opt"'[ \t]*$/ {
opt_found=1;
}
# Match the following section heading
this_section == 1 && /^[ \t]*\[.*$/ {
this_section=0;
if (opt_found != 1) {
print "\t'"$opt"'";
}
}
# Print every line
{ print $0; }
END {
# In case our section is the very last in the file
if (this_section == 1 && opt_found != 1) {
print "\t'"$opt"'";
}
}
' $file > $tmp
# Overwrite $file only if $tmp is different
diff -q $file $tmp > /dev/null 2>&1
if [ $? -ne 0 ]; then
mv $tmp $file
# reload smb.conf here
else
rm $tmp
fi
I can't help feeling that this is a long script to achieve a simple task. Is there a more efficient/elegant way to insert a property in an ini file using basic shell tools like sed and awk?
Consider using Python 3's configparser:
#!/usr/bin/python3
import sys
from configparser import SafeConfigParser
cfg = SafeConfigParser()
cfg.read(sys.argv[1])
cfg['myshare']['hosts deny'] = '192.168.23.';
with open(sys.argv[1], 'w') as f:
cfg.write(f)
To be called as ./filename.py smb.conf (i.e., the first parameter is the file to change).
Note that comments are not preserved by this. However, since a GUI overwrites the config and doesn't preserve custom options, I suspect that comments are already nuked and that this is not a worry in your case.
Untested, should work though
awk -vT="hosts deny=192.168.23" 'x&&$0~T{x=0}x&&/^ *\[[^]]+\]/{print "\t\t"T;x=0}
/^ *\[myshare\]/{x++}1' file
This solution is a bit awkward. It uses the INI section header as the record separator. This means that there is an empty record before the first header, so when we match the header we're interested in, we have to read the next record to handle that INI section. Also, there are some printf commands because the records still contain leading and trailing newlines.
awk -v RS='[[][^]]+[]]' -v str="hosts deny=192.168.23." '
{printf "%s", $0; printf "%s", RT}
RT == "[myshare]" {
getline
printf "%s", $0
if (index($0, str) == 0) print str
printf "%s", RT
}
' smb.conf
RS is the awk variable that contains the regex to split the text into records.
RT is the awk variable that contains the actual text of the current record separator.
With GNU awk for a couple of extensions:
$ cat tst.awk
index($0,str) { found = 1 }
match($0,/^\s*\[([^]]+).*/,a) {
if ( (name == tgt) && !found ) { print indent str }
name = a[1]
found = 0
}
{ print; indent=gensub(/\S.*/,"","") }
.
$ awk -v tgt="myshare" -v str="hosts deny=192.168.23." -f tst.awk file
[global]
printcap name = cups
winbind enum groups = yes
security = user
[myshare]
path=/mnt/myshare
browseable=yes
enable recycle bin=no
writeable=yes
hosts deny=192.168.23.
[Another Share]
invalid users=nobody,nobody
valid users=nobody,nobody
path=/mnt/share2
browseable=no
.
$ awk -v tgt="myshare" -v str="fluffy bunny" -f tst.awk file
[global]
printcap name = cups
winbind enum groups = yes
security = user
[myshare]
path=/mnt/myshare
browseable=yes
enable recycle bin=no
writeable=yes
hosts deny=192.168.23.
fluffy bunny
[Another Share]
invalid users=nobody,nobody
valid users=nobody,nobody
path=/mnt/share2
browseable=no
I'm having trouble making a script to replace a string that is inside double quotes. The files sections looks like this:
"regA~1" : "FF_NODE~94"
"regA~2" : "FF_NODE~105"
"regA~3" : "FF_NODE~116"
"regA~4" : "FF_NODE~127"
"regA~5" : "FF_NODE~138"
"regA~6" : "FF_NODE~149"
"regA~7" : "FF_NODE~154"
"regA~8" : "FF_NODE~155"
"regA~9" : "FF_NODE~156"
"regA~1" : "FF_NODE~95"
"regA~11" : "FF_NODE~96"
It works if I do
awk '/"regA~1"/{c++;if(c==2){sub("regA~1","regA~10");}}1' file > file_out
but when trying to make this a script where I pass a variable regA~1 and the value for c it doesn't work.
s="regA~1";
r="regA~10";
n=2;
awk -v search="$s" -v replace="$r" -v count=$n '/search/{c++;if(c==count){sub(search,replace);}}1' file > file_out
I also tried
awk -v search=$s -v replace=$r -v count=$n '/search/{c++;if(c==count){sub(search,replace);}}1' file > file_out
The syntax you need to match an RE that's stored as a string in a variable is
$0 ~ var
not
/var/
Thanks to Ed Morton for the tip. Here's the bash script in case anyone needs something like this. Not very sophisticated but it works for me.
#!/bin/bash
# Replaces a specific occurrence of a search string with a replace string
if [ $# -lt 4 ] ; then
echo -e "Wrong number of parameters."
echo -e "Usage:"
echo -e "repnthstr file search replace n"
echo -e "repnthstr fileext search replace n"
exit 1
fi
for file in $1
do
if [ -f $file -a -r $file ]; then
awk -v search=$2 -v replace=$3 -v cnt=$4 '$0 ~ search{c++;if(c==cnt){sub(search,replace);}}1' "$file" > temp && mv temp "$file"
else
echo "Error: Cannot read $file"
fi
done
This is similar to another issue, but I only want make to prompt for a value if I'm running a specific target and a mandatory variable has not been specified.
The current code:
install-crontab: PASSWORD ?= "$(shell read -p "Password: "; echo "$$REPLY")"
install-crontab: $(SCRIPT_PATH)
#echo "#midnight \"$(SCRIPT_PATH)\" [...] \"$(PASSWORD)\""
This just results in the following output and no prompt:
Password: read: 1: arg count
#midnight [...] ""
The important point here is that I have to ask only when running this target, and only if the variable has not been defined. I can't use a configure script, because obviously I shouldn't store passwords in a config script, and because this target is not part of the standard installation procedure.
Turns out the problem was that Makefiles don't use Dash / Bash-style quotation, and that Dash's read built-in needs a variable name, unlike Bash. Resulting code:
install-crontab-delicious: $(DELICIOUS_TARGET_PATH)
#while [ -z "$$DELICIOUS_USER" ]; do \
read -r -p "Delicious user name: " DELICIOUS_USER;\
done && \
while [ -z "$$DELICIOUS_PASSWORD" ]; do \
read -r -p "Delicious password: " DELICIOUS_PASSWORD; \
done && \
while [ -z "$$DELICIOUS_PATH" ]; do \
read -r -p "Delicious backup path: " DELICIOUS_PATH; \
done && \
( \
CRONTAB_NOHEADER=Y crontab -l || true; \
printf '%s' \
'#midnight ' \
'"$(DELICIOUS_TARGET_PATH)" ' \
"\"$$DELICIOUS_USER\" " \
"\"$$DELICIOUS_PASSWORD\" " \
"\"$$DELICIOUS_PATH\""; \
printf '\n') | crontab -
Result:
$ crontab -r; make install-crontab-delicious && crontab -l
Delicious user name: a\b c\d
Delicious password: e f g
Delicious backup path: h\ i
no crontab for <user>
#midnight "/usr/local/bin/export_Delicious" "a\b c\d" "e f g" "h\ i"
$ DELICIOUS_PASSWORD=foo make install-crontab-delicious && crontab -l
Delicious user name: bar
Delicious backup path: baz
#midnight "/usr/local/bin/export_Delicious" "a\b c\d" "e f g" "h\ i"
#midnight "/usr/local/bin/export_Delicious" "bar" "foo" "baz"
This code:
treats all input characters as literals, so it works with spaces and backslashes,
avoids problems if the user presses Enter without writing anything,
uses environment variables if they exist, and
works whether crontab is empty or not.
l0b0's answer helped me with a similar problem where I wanted to exit if the user doesn't input 'y'. I ended up doing this:
#while [ -z "$$CONTINUE" ]; do \
read -r -p "Type anything but Y or y to exit. [y/N] " CONTINUE; \
done ; \
if [ ! $$CONTINUE == "y" ]; then \
if [ ! $$CONTINUE == "Y" ]; then \
echo "Exiting." ; exit 1 ; \
fi \
fi
I hope that helps someone. It's hard to find more info about using user input for an if/else in a makefile.