Assign variable to cut -f field - awk

Using cut, I want to know how to use it as:
awk -v id=3 -v RS= -F '::' '($1==id) {print $3}' jenny | a=1 ;cut -d$'\n' -f$a
I want to use it in a loop where i is replaced with, e.g., -f 1...3
Input
0::chkconfig --list autofs::
autofs 0:off 1:off 2:on 3:on 4:on 5:on 6:off
1::grep "^PROMPT=" /etc/sysconfig/init::
PROMPT=yes
2::rpm -q prelink::
prelink-0.4.0-2.el5
3::if [ -z "$(grep -l "hard core" /etc/security/limits.conf /etc/security/limits.d/*)" ]; then echo "empty"; else echo -e "$(grep -l "hard core" /etc/security/limits.conf /etc/security/limits.d/*)"; fi::
/etc/security/limits.conf
/etc/security/limits.d/test
4::sysctl fs.suid_dumpable::
fs.suid_dumpable = 0
5::stat /etc/motd::
File: `/etc/motd'
Size: 17 Blocks: 16 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 10125343 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2019-04-09 07:56:19.000000000 +0500
Modify: 2019-03-30 19:22:13.000000000 +0500
Change: 2019-03-30 19:22:13.000000000 +0500
Expected Output
/etc/security/limits.conf
/etc/security/limits.d/test
As field 1 and field currently it's all coming in $3. I tried separating with newline in awk; it doesn't seem to catch.

To get your desired output from the given input, try:
$ awk '/^$/{f=0} f{print} /3::/{f=1}' file
/etc/security/limits.conf
/etc/security/limits.d/test
To get only one output line as selected with a variable i:
$ awk -v i=1 '/3::/{n=NR+i} n==NR' file
/etc/security/limits.conf
$ awk -v i=2 '/3::/{n=NR+i} n==NR' file
/etc/security/limits.d/test
The awk variable i can, of course, be set to the value of a shell variable i:
$ i=2
$ awk -v i="$i" '/3::/{n=NR+i} n==NR' file
/etc/security/limits.d/test
The stanza can also be selected from a variable:
$ i=2
$ k=3
$ awk -v i="$i" -v k="$k" -F:: '$1==k{n=NR+i} n==NR' file
/etc/security/limits.d/test
How it works:
-v i="$i" -v k="$k"
These options set awk variable i and k to the values of the shell variables $i and $k, respectively.
-F::
This sets the field separator to ::.
$1==k {n=NR+i}
If the first field of the current line equals the variable k, then set variable n to the current line number, NR, plus i.
n==NR
If the current line number, NR, is n, then print this line.

With sed:
$ id=3; sed -En "/^$id::/,/^$/{/^[[:blank:]]*\//p}" jenny
/etc/security/limits.conf
/etc/security/limits.d/test
Explanations:
Your shell will interpret the command and replace id by its value.
/^$id::/,/^$/{} the scope {} will be executed only between the lines that starts with the value of id followed by :: (/^$id::/) until an empty line (/^$/)
/^[[:blank:]]*\//p for the lines that start with some POSIX blank character class (e.g. space/tab) followed by / print the line. This will print your two paths.
To specify a line:
$ id=3; line=1; sed -En "/^$id::/,/^$/{/^[[:blank:]]*\//p}" jenny | cut -d$'\n' -f"$line"
/etc/security/limits.conf
$ id=3; line=2; sed -En "/^$id::/,/^$/{/^[[:blank:]]*\//p}" jenny | cut -d$'\n' -f"$line"
/etc/security/limits.d/test
$ id=3; line=1; sed -En "/^$id::/,/^$/{/^[[:blank:]]*\//p}" jenny | sed -n "${line}p"
/etc/security/limits.conf
$ id=3; line=2; sed -En "/^$id::/,/^$/{/^[[:blank:]]*\//p}" jenny | sed -n "${line}p"
/etc/security/limits.d/test

Assuming you want to build onto your previous question rather than coming up with a completely different approach
$ awk -v id=3 -v lineNr=1 -v RS= -F '::' '$1==id{ split($3,lines,/\n/); print lines[lineNr+1] }' file
/etc/security/limits.conf
$ awk -v id=3 -v lineNr=2 -v RS= -F '::' '$1==id{ split($3,lines,/\n/); print lines[lineNr+1] }' file
/etc/security/limits.d/test

Related

AWKing or GREPing brackets [ ]?

I've searched all over and couldn't find a solution.
How would I awk or grep the following:
$ mbimcli -d /dev/cdc-wdm0 -p --query-ip-configuration
[/dev/cdc-wdm0] IPv4 configuration available: 'address, gateway, dns'
IP [0]: '11.22.333.44/55'
Gateway: '14.13.198.4'
DNS [0]: '172.17.1.101'
DNS [1]: '172.17.1.102'
DNS [2]: '172.17.1.101'
DNS [3]: '172.17.1.102'
So that I end up with:
11.22.33.44/55
I've tried a bunch of different combinations with both grep and awk and couldn't find a solution.
Using cat file as I don't have mbimcli -d /dev/cdc-wdm0 -p --query-ip-configuration:
$ cat file | awk -F"'" '/IP \[/{print $2}'
11.22.333.44/55
$ cat file | awk -F"'" '/Gateway/{print $2}'
14.13.198.4
or maybe this is all you need if the output of that command always looks like the example you posted:
$ cat file | awk -v RS= -F"'" '{print $5}'
11.22.333.44/55
$ cat file | awk -v RS= -F"'" '{print $8}'
14.13.198.4
You can do this in a single awk:
mbimcli -d /dev/cdc-wdm0 -p --query-ip-configuration |
awk '$1 == "IP" {gsub(/\047/, "", $NF); print $NF}'
11.22.333.44/55
something like this:
grep '[0-9]' file_with_text | awk '{print $NF}
grep [0-9] only lines with numbers and pipe the output into awk.
The $NF will return the last element.
If you want only the line that has the /, just add it to grep [0-9]/.
Also, for a complete answer, you can pipe the output of the command into grep:
mbimcli -d /dev/cdc-wdm0 -p --query-ip-configuration | grep '[0-9]/' | awk '{print $NF}
I would harness tr for preprocessing and GNU AWK for processing, let file.txt content be
IP [0]: '11.22.333.44/55'
Gateway: '14.13.198.4'
DNS [0]: '172.17.1.101'
DNS [1]: '172.17.1.102'
DNS [2]: '172.17.1.101'
DNS [3]: '172.17.1.102'
then
cat file.txt | tr -d "'" | awk '/IP/{print $NF}'
output
11.22.333.44/55
Explanation: use tr to delete ' then awk to print last column ($NF) if row contain IP.
(tested in tr (GNU coreutils) 8.30 and GNU Awk 5.0.1)

Print paragraph if it contains a string stored in a variable (blank lines separate paragraphs)

I am trying to isolate the header of a mail in the /var/spool/mail/mysuser file.
Print a paragraph if it contains AAA (blank lines separate paragraphs)
sed is working when searching with the string "AAA"
$ sed -e '/./{H;$!d;}' -e 'x;/AAA/!d;' /var/spool/mail/mysuser
When using a variable is does not work :
$ MyVar="AAA"
$ sed -e '/./{H;$!d;}' -e 'x;/$MyVar/!d;' /var/spool/mail/mysuser
=> No output as the single quotes prevent the expantion of the variable
Trying with singles quotes
$ sed -e "/./{H;$!d;}" -e "x;/$MyVar/!d; /var/spool/mail/mysuser
sed: -e expression #2, char 27: extra characters after command
Actually, the first search is also not working with doubles quotes
$ sed -e "/./{H;$!d;}" -e 'x;/AAA/!d;" /var/spool/mail/mysuser
sed -e "/./{H;$!d;}" -e "x;/AAA/date;" /var/spool/mail/mysuser
sed: -e expression #2, char 9: extra characters after command
I am also considering awk without success so far
Any advices ?
should be trivial with awk
$ awk -v RS= '/AAA/' file
with a variable, little more needed
$ awk -v RS= -v var='AAA' '$0~var'
or if it's defined elsewhere
$ awk -v RS= -v var="$variable_holding_value" '$0~var'
That is happening because of the single quotes. You need to go out of the single quotes to enable interpolation:
sed -e '/./{H;$!d;}' -e 'x;/'$MyVar'/!d;' /var/spool/mail/mysuser
or, better put the variable in double quotes:
sed -e '/./{H;$!d;}' -e 'x;/'"$MyVar"'/!d;' /var/spool/mail/mysuser
Thanks to karakfa
It works with :
MyVar="AAA"
awk -v RS= -v X=$MyVar '$0~X' file

How to paste values of specific columns of a file into another command?

I want to use the fastacmd to extract specific regions of fasta sequences.
To do that I need to put the name of the fasta file -d, the name of the sequence -s and the position of the sequence to extract -L. For example:
fastacmd -d OAP11402.1.fa -s OAP11402.1 -L 50,100
But the problem is that I have hundreds of files (each file has one sequence with the same name of the file) and the info of position of each sequence to extract is in a protein database (info_sequences.txt). So, I want to make a loop to paste the name of the file, sequence and the positions to extract from the protein database info_sequences.txt in the fastacmd.
The look of info_sequences.txt is like this:
File seq_id position_start position_end
OAP11402.1.fa OAP11402.1 50 100
OAP15774.1.fa OAP15774.1 75 200
OAP10214.1.fa OAP10214.1 33 310
I think that awk could help but i'm struggling with the way to paste the info in the fastcmd
source <(
awk 'NR > 1 {
printf "echo fastacmd -d %s -s %s -L %d,%d\n", $1, $2, $3, $4
}' info_sequences.txt
)
The awk command spits out all the commands.
Then the source <( ... ) evaluates the commands in your current shell.
Same advice as Cyrus, if it looks OK remove the echo
Or, do it all in awk:
awk 'NR > 1 {
cmd = "echo fastacmd -d " $1 " -s " $2 " -L " $3 "," $4
system(cmd)
}' info_sequences.txt
awk 'NR>1 {print "-d",$1,"-s",$2,"-L",$3","$4}' info_sequences.txt | xargs -I {} echo fastacmd {}
Output:
fastacmd -d OAP11402.1.fa -s OAP11402.1 -L 50,100
fastacmd -d OAP15774.1.fa -s OAP15774.1 -L 75,200
fastacmd -d OAP10214.1.fa -s OAP10214.1 -L 33,310
If everything looks okay, remove echo.

Linux Grep or Awk to find strings and store into array

I would like to print the string in the following pattern. And I would like to store it in a array. Please help me, I need O/p as follows
test11
orcl
My commands/Tries
egrep -i ":Y|:N" /etc/oratab | cut -d":" -f1 | grep -v "\#" | grep -v "\*" | tr -d '\n' | sed 's/ /\n/g' | awk '{print $1}'
Above commands O/p:
test11orcl
Contents of Oratab will be as follows,
[oracle#rhel6112 scripts]$ cat/etc/oratab
#
# This file is used by ORACLE utilities. It is created by root.sh
# and updated by the Database Configuration Assistant when creating
# a database.
# A colon, ':', is used as the field terminator. A new line terminates
# the entry. Lines beginning with a pound sign, '#', are comments.
#
# Entries are of the form:
# $ORACLE_SID:$ORACLE_HOME:<N|Y>:
#
# Multiple entries with the same $ORACLE_SID are not allowed.
#
#
test11:/u01/app/oracle/product/11.2.0/dbhome_1:N
orcl:/u01/app/oracle/product/10.2.0/db_1:N
End of Cat Output
From the above file am trying to extract the STRING before the :/
As a start, try this:
$ cat input.txt
test11:/u01/app/oracle/product/11.2.0/dbhome_1:N
orcl:/u01/app/oracle/product/10.2.0/db_1:N
$ awk -F: '{print $1}' input.txt
test11
orcl
update
Using bash:
#!/bin/bash
ARRAY=()
while read -r line
do
[[ "$line" = \#* ]] && continue
data=$(awk -F: '{print $1}' <<< $line)
ARRAY+=($data)
done < input.txt
for i in "${ARRAY[#]}"
do
echo "$i"
done
In action:
$ ./db.sh
test11
orcl
You could use sed also,
sed -r 's/^([^:]*):.*$/\1/g' file
Example:
$ cat cc
test11:/u01/app/oracle/product/11.2.0/dbhome_1:N
orcl:/u01/app/oracle/product/10.2.0/db_1:N
$ sed -r 's/^([^:]*):.*$/\1/g' cc
test11
orcl
OR
$ sed -nr 's/^(.*):\/.*$/\1/p' file
test11
orcl

Awk or grep question

I have this datafile
[abc]
def
ghi
[jkl]
[mno]
From this file; i can run grep and easily get all lines that have "[" in them. How can I get the contents of text inside "[]".
For example:
abc
jkl
mno
Thanks
Give this a try:
sed -n 's/\[\([^]]*\)\]/\1/p'
or
awk -F "[][]" '$2 != "" {print $2}'
or
grep -Po '(?<=\[)[^]]*(?=])'
sed -n 's/\[\(.*\)\]/\1/p' file
Explanation: -n suppresses the printing of each line to STDOUT, but the /p at the end of the regex re-enables this behavior causing all matching lines to be printed. The regex itself matches everything between brackets and replaces the entire line with it.
grep "\[" | sed -e 's/\[//' -e 's/\]//'
here's how you can do it with awk
$ cat file
[abc]
def [ xxx]
ghi
[jkl]
[mno]
[zz
zzzz]
$ awk 'BEGIN{RS="]";FS="["}/\[/{print $NF }' file
abc
xxx
jkl
mno
zz
zzzz
Ruby(1.9+)
ruby -0777 -ne 'puts $_.scan(/\[(.*?)\]/m)' file
Or you can do it with just the shell
$ var=$(<file)
$ IFS="]"
$ set -- $var
$ for i in $#; do echo ${i##*[}; done