How to process fields between matching patterns? - awk

Question

Related post: How to select lines between two patterns?
#fedorqui, thanks for providing all these different options for awk. I have been using this to parse through var log messages when troubleshooting ooms and it works great. I want to extend this further but I have not been able to figure out how to proceed. What I am trying to do:
Print the lines between rss and out of memory. I have done that with the example
Order the sections between each match by rss field. I have not been able to figure this out
Add an extra column with its own header and perform some mathematical operation. I have been able to do this somewhat but I am running into some formatting issues. I am not sure how to skip the first and last line when adding the column so I lose those lines. I am also not able to keep the spacing from the original if I do any operations other than print.
Here's the command I am using right now:
less /var/log/messages'|awk '/swapents/{x=1; print "=================="};/Out of memory/{x=0} x'|sed 's/[]\[]//g'
Here's the source Data:
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.617265 pid uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.622250 1828 0 1828 4331 116 14 3 0 -1000 udevd
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.627310 2664 0 2664 28002 53 23 3 0 -1000 auditd
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.633181 2680 0 2680 62032 1181 24 4 0 0 rsyslogd
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.638888 2694 0 2694 3444 61 11 3 0 0 irqbalance
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.644912 2710 81 2710 5430 56 14 3 0 0 dbus-daemon
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.651108 2779 0 2779 19958 203 42 3 0 -1000 sshd
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.656670 2789 0 2789 5622 56 17 3 0 0 xinetd
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.653452 Out of memory: Kill process 43390 (mysql) score 1000 or sacrifice child
blah
blah
blah
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.617265 pid uid tgid total_vm rss nr_ptes nr_pmds swapents oom_score_adj name
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.622250 1828 0 1828 4331 116 14 3 0 -1000 udevd
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.627310 2664 0 2664 28002 53 23 3 0 -1000 auditd
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.633181 2680 0 2680 62032 1181 24 4 0 0 rsyslogd
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.638888 2694 0 2694 3444 61 11 3 0 0 irqbalance
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.644912 2710 81 2710 5430 56 14 3 0 0 dbus-daemon
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.651108 2779 0 2779 19958 203 42 3 0 -1000 sshd
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.656670 2789 0 2789 5622 56 17 3 0 0 xinetd
Sep 8 11:35:15 ip-10-23-15-70 kernel: 11810061.653452 Out of memory: Kill process 43390 (mysql) score 1000 or sacrifice child
Here's what my output looks like:
================== 0MB
Sep 8 11:35:15 pid 0MB name <---- should be header (Pid virt rss etc)
Sep 8 11:35:15 1828 0MB udevd
Sep 8 11:35:15 2664 0MB auditd
Sep 8 11:35:15 2680 4MB rsyslogd
Sep 8 11:35:15 2694 0MB irqbalance
Sep 8 11:35:15 2710 0MB dbus-daemon
Sep 8 11:35:15 2779 0MB sshd
Sep 8 11:35:15 2789 0MB xinetd
Sep 8 11:35:15 2822 0MB crond
Sep 8 11:35:15 Out 0MB or <---- should be footer (out of memory etc)
================== 0MB
Sep 8 11:35:15 pid 0MB name <---- should be header (Pid virt rss etc)
Sep 8 11:35:15 1828 0MB udevd
Sep 8 11:35:15 2664 0MB auditd
Sep 8 11:35:15 2680 4MB rsyslogd
Sep 8 11:35:15 2694 0MB irqbalance
Sep 8 11:35:15 2710 0MB dbus-daemon
Sep 8 11:35:15 2779 0MB sshd
Sep 8 11:35:15 2789 0MB xinetd
Sep 8 11:35:15 2822 0MB crond
Sep 8 11:35:15 Out 0MB or <---- should be footer (out of memory etc)
================== 0MB
You can see the from the output that the separator I added for each oom field, awk tries to calculate values for it, I would love to avoid this if possible. Also the header and footer are getting chopped off and it would be nice to avoid that too.
Here's what I would like:
========================
Sep 8 11:35:15 pid rss memused_MB oom_score_adj name
Sep 8 11:35:15 2664 53 {rss*4/1024} -1000 auditd
Sep 8 11:35:15 2789 56 {rss*4/1024} 0 xinetd
Sep 8 11:35:15 2710 56 {rss*4/1024} 0 dbus-dae
Sep 8 11:35:15 2694 61 {rss*4/1024} 0 irqbalan
Sep 8 11:35:15 1828 116 {rss*4/1024} -1000 udevd
Sep 8 11:35:15 2680 181 {rss*4/1024} 0 rsyslogd
Sep 8 11:35:15 2779 203 {rss*4/1024} -1000 sshd
Sep 8 11:35:15 Out of memory: Kill process 43390 (mysql) score 1000 or sacrifice child
========================
Sep 8 11:35:15 pid rss memused_MB oom_score_adj name
Sep 8 11:35:15 2664 53 {rss*4/1024} -1000 auditd
Sep 8 11:35:15 2789 56 {rss*4/1024} 0 xinetd
Sep 8 11:35:15 2710 56 {rss*4/1024} 0 dbus-dae
Sep 8 11:35:15 2694 61 {rss*4/1024} 0 irqbalan
Sep 8 11:35:15 1828 116 {rss*4/1024} -1000 udevd
Sep 8 11:35:15 2680 181 {rss*4/1024} 0 rsyslogd
Sep 8 11:35:15 2779 203 {rss*4/1024} -1000 sshd
Sep 8 11:35:15 Out of memory: Kill process 43390 (mysql) score 1000 or sacrifice child
========================

Answer 1

awk solution:
$ cat tst.awk
/swapents/ {
x=1;
print "=================="
printf( "%s %s %s pid\t%4s\tmemused_MB\toom_score_adj\tname\n", $1, $2, $3, "rss");
next
}
/Out of memory/ {
printf( "%s %s %s %s\n", $1, $2, $3, substr($0,index($0,$7)));
x=0
}
x {
printf( "%s %s %s %s\t%4d\t%10.5f\t%13d\t%s\n", $1, $2, $3, $7, $11, ($11*4)/1024, $15, $16 )
}
You can play around with the formatting, like the precision in column 6, using the specifiers in the printf function. Call this with:
$ awk -f tst.awk /var/log/messages
EDIT: with sorting
OP asked to sort the output by rss columns. Using a standard sort wouldn't work here because you want to sort in between starting and ending matches. You can solve this by saving the intermediate result in an array and sort that with a self-defined function. Like this:
$ cat tst2.awk
/swapents/ {
x=1;
print "=================="
printf( "%s %s %s pid\t%4s\tmemused_MB\toom_score_adj\tname\n", $1, $2, $3, "rss");
next
}
/Out of memory/ {
n=asort(a, sorted, "cmp_rss")
for (i=1; i<=n; i++) {
print sorted[i]
}
delete a;
printf( "%s %s %s %s\n", $1, $2, $3, substr($0,index($0,$7)));
x=0
}
x {
a[i++] = sprintf( "%s %s %s %s\t%4d\t%10.5f\t%13d\t%s", $1, $2, $3, $7, $11, ($11*4)/1024, $15, $16 );
}
function cmp_rss(i1, v1, i2, v2)
{
split(v1, a1, " ")
split(v2, a2, " ")
rss1=a1[5];
rss2=a2[5];
return (rss1 - rss2)
}
which leads to:
$ awk -f tst2.awk input.txt
==================
Sep 8 11:35:15 pid rss memused_MB oom_score_adj name
Sep 8 11:35:15 2664 53 0.20703 -1000 auditd
Sep 8 11:35:15 2710 56 0.21875 0 dbus-daemon
Sep 8 11:35:15 2789 56 0.21875 0 xinetd
Sep 8 11:35:15 2694 61 0.23828 0 irqbalance
Sep 8 11:35:15 1828 116 0.45312 -1000 udevd
Sep 8 11:35:15 2779 203 0.79297 -1000 sshd
Sep 8 11:35:15 2680 1181 4.61328 0 rsyslogd
Sep 8 11:35:15 Out of memory: Kill process 43390 (mysql) score 1000 or sacrifice child
==================
Sep 8 11:35:15 pid rss memused_MB oom_score_adj name
Sep 8 11:35:15 2664 53 0.20703 -1000 auditd
Sep 8 11:35:15 2710 56 0.21875 0 dbus-daemon
Sep 8 11:35:15 2789 56 0.21875 0 xinetd
Sep 8 11:35:15 2694 61 0.23828 0 irqbalance
Sep 8 11:35:15 1828 116 0.45312 -1000 udevd
Sep 8 11:35:15 2779 203 0.79297 -1000 sshd
Sep 8 11:35:15 2680 1181 4.61328 0 rsyslogd
Sep 8 11:35:15 Out of memory: Kill process 43390 (mysql) score 1000 or sacrifice child

Answer 2

Based on Marc Lambrichs response I was able to create this one liner that does the job. Thanks so much. The only thing missing now is sorting by the rss column, I haven't been able to get the rss field sorted though
less /var/log/messages|awk '/swapents/ {x=1; print "==================";gsub(/\[|\]/, "") ;printf "%s %s %s %s %10s %10s %10s %15s memory_used %-s\n", $1,$2,$3,$4,$7,$10,$11,$15,$16 ;next } {gsub(/\[|\]/, "")} /Out of memory/ {print $0 ;x=0 } x {printf "%s %s %s %s %10s %10s %10s %15s %9.2fMB %-s\n", $1,$2,$3,$4,$7,$10,$11,$15,$11*4/1024,$16}'
For readability, the awk code formatted:
/swapents/ {
x=1;
print "==================";
gsub(/\[|\]/, "");
printf "%s %s %s %s %10s %10s %10s %15s memory_used %-s\n", $1,$2,$3,$4,$7,$10,$11,$15,$16 ;
next
}
{
gsub(/\[|\]/, "")
}
/Out of memory/ {
print $0;
x=0
}
x {
printf "%s %s %s %s %10s %10s %10s %15s %9.2fMB %-s\n", $1,$2,$3,$4,$7,$10,$11,$15,$11*4/1024,$16
}