Trying to turn some butchered data into bar delimited, unbutchered data...
here's some sample data
asd1276vdjs12897364vsk Tue Apr 2 08:19:12 2013 [pid 3] [words] FAIL UPLOAD: Client "00.005.006.006", "/0801NSJH.bbf", 0.00Kbyte/sec
into
asd1276vdjs12897364vsk|Tue Apr 2 08:19:12 2013|[pid 3]|[words]|FAIL UPLOAD: Client "00.005.006.006"|"/0801NSJH.bbf"|0.00Kbyte/sec
The regex's are simple enough, but I don't know how to say first field = regex, second field = regex etc.
This sed is functional but kind of hacky, I'd like to make it work in gawk.
sed 's/ Sun/|Sun/'
sed 's/ Mon/|Mon/'
sed 's/ Tue/|Tue/'
sed 's/ Wed/|Wed/'
sed 's/ Thu/|Thu/'
sed 's/ Fri/|Fri/'
sed 's/ Sat/|Sat/'
sed 's/ Sun/|Sun/'
sed -e 's% \[%|\[%g' -e 's%\] %\]|%g' -e 's%, %|%g'
$ cat tst.awk
{ print gensub(/\
([^[:space:]]+)[[:space:]]+\
([^[]+)[[:space:]]+\
([[][^]]+[]])[[:space:]]+\
([[][^]]+[]])[[:space:]]+\
([^,]+),[[:space:]]+\
([^,]+),[[:space:]]+\
/,
"\\1|\\2|\\3|\\4|\\5|\\6|","")
}
$ awk -f tst.awk file
asd1276vdjs12897364vsk|Tue Apr 2 08:19:12 2013|[pid 3]|[words]|FAIL UPLOAD: Client "00.005.006.006"|"/0801NSJH.bbf"|0.00Kbyte/sec
Related
I am trying to isolate the header of a mail in the /var/spool/mail/mysuser file.
Print a paragraph if it contains AAA (blank lines separate paragraphs)
sed is working when searching with the string "AAA"
$ sed -e '/./{H;$!d;}' -e 'x;/AAA/!d;' /var/spool/mail/mysuser
When using a variable is does not work :
$ MyVar="AAA"
$ sed -e '/./{H;$!d;}' -e 'x;/$MyVar/!d;' /var/spool/mail/mysuser
=> No output as the single quotes prevent the expantion of the variable
Trying with singles quotes
$ sed -e "/./{H;$!d;}" -e "x;/$MyVar/!d; /var/spool/mail/mysuser
sed: -e expression #2, char 27: extra characters after command
Actually, the first search is also not working with doubles quotes
$ sed -e "/./{H;$!d;}" -e 'x;/AAA/!d;" /var/spool/mail/mysuser
sed -e "/./{H;$!d;}" -e "x;/AAA/date;" /var/spool/mail/mysuser
sed: -e expression #2, char 9: extra characters after command
I am also considering awk without success so far
Any advices ?
should be trivial with awk
$ awk -v RS= '/AAA/' file
with a variable, little more needed
$ awk -v RS= -v var='AAA' '$0~var'
or if it's defined elsewhere
$ awk -v RS= -v var="$variable_holding_value" '$0~var'
That is happening because of the single quotes. You need to go out of the single quotes to enable interpolation:
sed -e '/./{H;$!d;}' -e 'x;/'$MyVar'/!d;' /var/spool/mail/mysuser
or, better put the variable in double quotes:
sed -e '/./{H;$!d;}' -e 'x;/'"$MyVar"'/!d;' /var/spool/mail/mysuser
Thanks to karakfa
It works with :
MyVar="AAA"
awk -v RS= -v X=$MyVar '$0~X' file
Using cut, I want to know how to use it as:
awk -v id=3 -v RS= -F '::' '($1==id) {print $3}' jenny | a=1 ;cut -d$'\n' -f$a
I want to use it in a loop where i is replaced with, e.g., -f 1...3
Input
0::chkconfig --list autofs::
autofs 0:off 1:off 2:on 3:on 4:on 5:on 6:off
1::grep "^PROMPT=" /etc/sysconfig/init::
PROMPT=yes
2::rpm -q prelink::
prelink-0.4.0-2.el5
3::if [ -z "$(grep -l "hard core" /etc/security/limits.conf /etc/security/limits.d/*)" ]; then echo "empty"; else echo -e "$(grep -l "hard core" /etc/security/limits.conf /etc/security/limits.d/*)"; fi::
/etc/security/limits.conf
/etc/security/limits.d/test
4::sysctl fs.suid_dumpable::
fs.suid_dumpable = 0
5::stat /etc/motd::
File: `/etc/motd'
Size: 17 Blocks: 16 IO Block: 4096 regular file
Device: fd00h/64768d Inode: 10125343 Links: 1
Access: (0644/-rw-r--r--) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2019-04-09 07:56:19.000000000 +0500
Modify: 2019-03-30 19:22:13.000000000 +0500
Change: 2019-03-30 19:22:13.000000000 +0500
Expected Output
/etc/security/limits.conf
/etc/security/limits.d/test
As field 1 and field currently it's all coming in $3. I tried separating with newline in awk; it doesn't seem to catch.
To get your desired output from the given input, try:
$ awk '/^$/{f=0} f{print} /3::/{f=1}' file
/etc/security/limits.conf
/etc/security/limits.d/test
To get only one output line as selected with a variable i:
$ awk -v i=1 '/3::/{n=NR+i} n==NR' file
/etc/security/limits.conf
$ awk -v i=2 '/3::/{n=NR+i} n==NR' file
/etc/security/limits.d/test
The awk variable i can, of course, be set to the value of a shell variable i:
$ i=2
$ awk -v i="$i" '/3::/{n=NR+i} n==NR' file
/etc/security/limits.d/test
The stanza can also be selected from a variable:
$ i=2
$ k=3
$ awk -v i="$i" -v k="$k" -F:: '$1==k{n=NR+i} n==NR' file
/etc/security/limits.d/test
How it works:
-v i="$i" -v k="$k"
These options set awk variable i and k to the values of the shell variables $i and $k, respectively.
-F::
This sets the field separator to ::.
$1==k {n=NR+i}
If the first field of the current line equals the variable k, then set variable n to the current line number, NR, plus i.
n==NR
If the current line number, NR, is n, then print this line.
With sed:
$ id=3; sed -En "/^$id::/,/^$/{/^[[:blank:]]*\//p}" jenny
/etc/security/limits.conf
/etc/security/limits.d/test
Explanations:
Your shell will interpret the command and replace id by its value.
/^$id::/,/^$/{} the scope {} will be executed only between the lines that starts with the value of id followed by :: (/^$id::/) until an empty line (/^$/)
/^[[:blank:]]*\//p for the lines that start with some POSIX blank character class (e.g. space/tab) followed by / print the line. This will print your two paths.
To specify a line:
$ id=3; line=1; sed -En "/^$id::/,/^$/{/^[[:blank:]]*\//p}" jenny | cut -d$'\n' -f"$line"
/etc/security/limits.conf
$ id=3; line=2; sed -En "/^$id::/,/^$/{/^[[:blank:]]*\//p}" jenny | cut -d$'\n' -f"$line"
/etc/security/limits.d/test
$ id=3; line=1; sed -En "/^$id::/,/^$/{/^[[:blank:]]*\//p}" jenny | sed -n "${line}p"
/etc/security/limits.conf
$ id=3; line=2; sed -En "/^$id::/,/^$/{/^[[:blank:]]*\//p}" jenny | sed -n "${line}p"
/etc/security/limits.d/test
Assuming you want to build onto your previous question rather than coming up with a completely different approach
$ awk -v id=3 -v lineNr=1 -v RS= -F '::' '$1==id{ split($3,lines,/\n/); print lines[lineNr+1] }' file
/etc/security/limits.conf
$ awk -v id=3 -v lineNr=2 -v RS= -F '::' '$1==id{ split($3,lines,/\n/); print lines[lineNr+1] }' file
/etc/security/limits.d/test
cat sample.log.txt|
grep INext-DROP-DEFLT|
sed -e 's/^.*SRC=//' -e 's/ .*DPT=/:/' -e 's/ .*$//' |
sort|
uniq|
ip.awk
I'm trying to send this command to an awk script but I don't know how to pipe it. The ip.awk is the script I want to send it to
I would like to print the string in the following pattern. And I would like to store it in a array. Please help me, I need O/p as follows
test11
orcl
My commands/Tries
egrep -i ":Y|:N" /etc/oratab | cut -d":" -f1 | grep -v "\#" | grep -v "\*" | tr -d '\n' | sed 's/ /\n/g' | awk '{print $1}'
Above commands O/p:
test11orcl
Contents of Oratab will be as follows,
[oracle#rhel6112 scripts]$ cat/etc/oratab
#
# This file is used by ORACLE utilities. It is created by root.sh
# and updated by the Database Configuration Assistant when creating
# a database.
# A colon, ':', is used as the field terminator. A new line terminates
# the entry. Lines beginning with a pound sign, '#', are comments.
#
# Entries are of the form:
# $ORACLE_SID:$ORACLE_HOME:<N|Y>:
#
# Multiple entries with the same $ORACLE_SID are not allowed.
#
#
test11:/u01/app/oracle/product/11.2.0/dbhome_1:N
orcl:/u01/app/oracle/product/10.2.0/db_1:N
End of Cat Output
From the above file am trying to extract the STRING before the :/
As a start, try this:
$ cat input.txt
test11:/u01/app/oracle/product/11.2.0/dbhome_1:N
orcl:/u01/app/oracle/product/10.2.0/db_1:N
$ awk -F: '{print $1}' input.txt
test11
orcl
update
Using bash:
#!/bin/bash
ARRAY=()
while read -r line
do
[[ "$line" = \#* ]] && continue
data=$(awk -F: '{print $1}' <<< $line)
ARRAY+=($data)
done < input.txt
for i in "${ARRAY[#]}"
do
echo "$i"
done
In action:
$ ./db.sh
test11
orcl
You could use sed also,
sed -r 's/^([^:]*):.*$/\1/g' file
Example:
$ cat cc
test11:/u01/app/oracle/product/11.2.0/dbhome_1:N
orcl:/u01/app/oracle/product/10.2.0/db_1:N
$ sed -r 's/^([^:]*):.*$/\1/g' cc
test11
orcl
OR
$ sed -nr 's/^(.*):\/.*$/\1/p' file
test11
orcl
I have this datafile
[abc]
def
ghi
[jkl]
[mno]
From this file; i can run grep and easily get all lines that have "[" in them. How can I get the contents of text inside "[]".
For example:
abc
jkl
mno
Thanks
Give this a try:
sed -n 's/\[\([^]]*\)\]/\1/p'
or
awk -F "[][]" '$2 != "" {print $2}'
or
grep -Po '(?<=\[)[^]]*(?=])'
sed -n 's/\[\(.*\)\]/\1/p' file
Explanation: -n suppresses the printing of each line to STDOUT, but the /p at the end of the regex re-enables this behavior causing all matching lines to be printed. The regex itself matches everything between brackets and replaces the entire line with it.
grep "\[" | sed -e 's/\[//' -e 's/\]//'
here's how you can do it with awk
$ cat file
[abc]
def [ xxx]
ghi
[jkl]
[mno]
[zz
zzzz]
$ awk 'BEGIN{RS="]";FS="["}/\[/{print $NF }' file
abc
xxx
jkl
mno
zz
zzzz
Ruby(1.9+)
ruby -0777 -ne 'puts $_.scan(/\[(.*?)\]/m)' file
Or you can do it with just the shell
$ var=$(<file)
$ IFS="]"
$ set -- $var
$ for i in $#; do echo ${i##*[}; done