I have the next script
cat foo.txt | awk '/ERROR/,/INFO/'
With the input of:
FooFoo
ERROR
Foo1
INFO
FooFoo
Now the result is:
ERROR
Foo1
INFO
I am looking for the next result:
Foo1
INFO
How I can make it work?
Thanks for your help
Give this a try:
awk '/ERROR/,/INFO/' foo.txt | tail -n +2
If your input is from a file, you don't need the cat. just awk '...' file
Could you please try following, written and tested with shown samples in GNU awk.
awk '
/ERROR/{
found=1
next
}
found{
val=(val?val ORS:"")$0
}
/INFO/{
print val
val=count=found=""
}
' Input_file
Explanation: Adding detailed explanation for above.
awk ' ##Starting awk program from here.
/ERROR/{ ##Checking if line contains ERROR then do following.
found=1 ##Setting found variable here.
next ##next will skip all further statements from here.
}
found{ ##Checking here if found is SET then do following.
val=(val?val ORS:"")$0 ##Creating variable val and keep adding value to it in form of current line.
}
/INFO/{ ##Checking condition if INFO is found in current line then do following.
print val ##Printing val here.
val=count="" ##Nullifying val and count here.
}
' Input_file ##Mentioning Input_file name here.
Like this:
awk '
seen # a true (1) condition makes awk to print current line
/ERROR/{seen=1} # if we grep ERROR, assign 1 to seen flag
/INFO/{seen=0} # if we grep INFO, assign 0 to seen flag
' file
Output
Foo1
INFO
Related
Example file:
Pattern 1
AAAAAAAAAA
BBBBBBBBBB
Pattern 2
I want to print the lines between two patterns in a file in one line.
From a previous question How to print lines between two patterns, inclusive or exclusive (in sed, AWK or Perl)? I found the very nice
awk '/Pattern 1/{flag=1; next} /Pattern 2/{flag=0} flag' file
With output:
AAAAAAAAAA
BBBBBBBBBB
My desired output:
AAAAAAAAAABBBBBBBBBB
Have your awk program in this way, written and tested in GNU awk.
awk '
/Pattern 2/{
if(found){
print val
}
found=""
next
}
/Pattern 1/{
found=1
next
}
found{
val=val $0
}
' Input_file
Explanation: Adding detailed explanation for above.
awk ' ##Starting awk program from here.
/Pattern 2/{ ##Checking if Pattern 2 is found here then do following.
if(found){ ##Checking if found is set then do following.
print val ##Printing val here.
}
found="" ##Nullifying found here.
next ##next will skip all statements from here.
}
/Pattern 1/{ ##Checking if Pattern 1 is found in current line.
found=1 ##Setting found to 1 here.
next ##next will skip all statements from here.
}
found{ ##Checking condition if found is SET then do following.
val=val $0 ##Creating val variable here which is keep adding current line values in it.
}
' Input_file ##Mentioning Input_file name here.
You may use this awk:
awk '/Pattern 2/ {if (s!="") print s; s=f=""} f {s = s $0} /Pattern 1/ {f=1}' file
AAAAAAAAAABBBBBBBBBB
And also with awk:
awk -v RS= '!/Pattern/ {sub(/\n/,"");print}' file
AAAAAAAAAABBBBBBBBBB
With GNU awk for multi-char RS and assuming your "Pattern"s really to take up whole lines and can't occur elsewhere in your input (easy fix if that's wrong):
$ awk -v RS='Pattern 2' 'sub(/.*Pattern 1/,""){gsub(/\n/,""); print}' file
AAAAAAAAAABBBBBBBBBB
or with any awk:
awk 'f{ if (/Pattern 2/){print buf; f=0} else buf=buf $0 } /Pattern 1/{f=1; buf=""}' file
AAAAAAAAAABBBBBBBBBB
You can set the output record separator to an empty string by using -v ORS=:
awk -v ORS= '/Pattern 1/{flag=1; next} /Pattern 2/{flag=0} flag' file
See an online demo.
To print a newline at the end, add END{print "\n"}:
awk -v ORS= '/Pattern 1/{flag=1; next} /Pattern 2/{flag=0} flag; END{print "\n"}' file > newfile
See the Ubuntu 18 screenshot:
I have multiple instances of the following pattern in my document:
Dipole Moment: [D]
X: 1.5279 Y: 0.1415 Z: 0.1694 Total: 1.5438
I want to extract the total dipole moment, so 1.5438. How can I pull this off?
When I throw in grep "Dipole Moment: [D]" filename, I don't get the line after. I am new to these command line interfaces. Any help you can provide would be greatly appreciated.
Could you please try following. Written and tested with shown samples in GNU awk.
awk '/Dipole Moment: \[D\]/{found=1;next} found{print $NF;found=""}' Input_file
Explanation: Adding detailed explanation for above.
awk ' ##Starting awk program from here.
/Dipole Moment: \[D\]/{ ##Checking if line contains Dipole Moment: \[D\] escaped [ and ] here.
found=1 ##Setting found to 1 here.
next ##next will skip all further statements from here.
}
found{ ##Checking condition if found is NOT NULL then do following.
print $NF ##Printing last field of current line here.
found="" ##Nullifying found here.
}
' Input_file ##Mentioning Input_file name here.
Sed alternative:
sed -rn '/^Dipole/{n;s/(^[[:space:]]{5}.*[[:space:]]{5})(.*)(([[:space:]]{5}.*+[:][[:space:]]{5}.*){3})/\2/p}' file
Search for the line beginning with "Dipole" then read the next line. Split this line into three sections based on regular expressions and substitute the line for the second section only, printing the result.
I have the following code that only displays the lines that are not in the first file:
'NR==FNR{a[$0];next} !($0 in a)' compareAgainst myFile
How can I include the original line number next to the output?
Could you please try following.
awk 'NR==FNR{a[$0];next} !($0 in a){print FNR,$0}' compareAgainst myFile
Explanation: Adding detailed explanation for above.
awk ' ##Starting awk program from here.
NR==FNR{ ##Checking condition FNR==NR which will be TRUE when first Input_file compareAgainst is being read.
a[$0] ##Creating array a with index $0 here.
next ##next will skip all further statements from here.
}
!($0 in a){ ##Checking condition if current Line is not present in array a then do following.
print FNR,$0 ##Printing line number and current line here.
}
' compareAgainst myFile ##Mentioning Input_file names here.
Trying to find a way to grep all names on one line for 100 files. grepping all names available in each file must appear on the same line.
FILE1
"company":"COMPANY1","companyDisplayName":"CM1","company":"COMPANY2","companyDisplayName":"CM2","company":"COMPANY3","companyDisplayName":"CM3",
FILE2
"company":"COMPANY99","companyDisplayName":"CM99"
The output i actually want is, ( include file name as prefix.)
FILE1:COMPANY1,COMPANY2,COMPANY3
FILE2:COMPANY99
i tried grep -oP '(?<="company":")[^"]*' * but i get results like this :
FILE1:COMPANY1
FILE1:COMPANY2
FILE1:COMPANY3
FILE2:COMPANY99
Could you please try following.
awk -F'[,:]' '
BEGIN{
OFS=","
}
{
for(i=1;i<=NF;i++){
if($i=="\"company\""){
val=(val?val OFS:"")$(i+1)
}
}
gsub(/\"/,"",val)
print FILENAME":"val
val=""
}
' Input_file1 Input_file2
Explanation: Adding explanation for above code.
awk -F'[,:]' ' ##Starting awk program here and setting field separator as colon OR comma here for all lines of Input_file(s).
BEGIN{ ##Starting BEGIN section of awk here.
OFS="," ##Setting OFS as comma here.
} ##Closing BEGIN BLOCK here.
{ ##Starting main BLOCK here.
for(i=1;i<=NF;i++){ ##Starting a for loop which starts from i=1 to till value of NF.
if($i=="\"company\""){ ##Checking condition if field value is equal to "company" then do following.
val=(val?val OFS:"")$(i+1) ##Creating a variable named val and concatenating its own value to it each time cursor comes here.
} ##Closing BLOCK for if condition here.
} ##Closing BLOCK for, for loop here.
gsub(/\"/,"",val) ##Using gsub to gklobally substitute all " in variable val here.
print FILENAME":"val ##Printing filename colon and variable val here.
val="" ##Nullifying variable val here.
} ##Closing main BLOCK here.
' Input_file1 Input_file2 ##Mentioning Input_file names here.
Output will be as follows.
Input_file1:COMPANY1,COMPANY2,COMPANY3
Input_file2:COMPANY99
EDIT: Adding solution in case OP needs to use grep and want to get final output from its output(though I will recommend to use awk solution itself since we are NOT using multiple commands or sub-shells).
grep -oP '(?<="company":")[^"]*' * | awk 'BEGIN{FS=":";OFS=","} prev!=$1 && val{print prev":"val;val=""} {val=(val?val OFS:"")$2;prev=$1} END{if(val){print prev":"val}}'
There are two tools that can take the output of your grep command and reformat it the way you want. First tool is GNU datamash. Second is tsv-summarize from eBay's tsv-utils package (disclaimer: I'm the author). Both tools solve this in similar ways:
$ # The grep output
$ echo $'FILE1:COMPANY1\nFILE1:COMPANY2\nFILE1:COMPANY3\nFILE2:COMPANY99' > grep-output.txt
$ cat grep-output.txt
FILE1:COMPANY1
FILE1:COMPANY2
FILE1:COMPANY3
FILE2:COMPANY99
$ # Using GNU datamash
$ cat grep-output.txt | datamash -field-separator : --group 1 unique 2
FILE1:COMPANY1,COMPANY2,COMPANY3
FILE2:COMPANY99
$ # Using tsv-summarize
$ cat grep-output.txt | tsv-summarize --delimiter : --group-by 1 --unique-values 2 --values-delimiter ,
FILE1:COMPANY1,COMPANY2,COMPANY3
FILE2:COMPANY99
I'd like to remove all the lines of a file based on matching a string from another file. This is what I have used but it only deletes some:
grep -vFf to_delete.csv inputfile.csv > output.csv
Here are sample lines from my input file (inputfile.csv):
Ata,Aqu,Ama3,Abe,0.053475,0.025,0.1,0.11275,0.1,0.15,0.83377
Ata135,Aru2,Aba301,A29,0.055525,0.025,0.1,0.082825,0.075,0.125
Ata135,Atb,Aca,Am54,0.14695,0.1,0.2,0.05255,0.025,0.075,0.8005,
Adc,Aru7,Ama301,Agr84,0.002075,0,0.025,0.240075,0.2,0.
My file "to_delete.csv" looks like this for example:
Aqu
Aca
So any line with those strings should get deleted, in this case, lines 1 and 3 should get deleted. Sample desired output:
Ata135,Aru2,Aba301,A29,0.055525,0.025,0.1,0.082825,0.075,0.125
Adc,Aru7,Ama301,Agr84,0.002075,0,0.025,0.240075,0.2,0.
EDIT: Since OP had carriage characters in his files so adding solution for that too now.
cat -v Input_file ##To check if carriage returns are there or not.
tr -d '\r' < Input_file > temp_file && mv temp_file Input_file
Since your samples of Input_file and expected output is not clear so couldn't fully test it, could you please try following.(if you are ok with awk), append > temp_file && mv temp_file Input_file in code to save output into Input_file itself.
awk -F, 'FNR==NR{a[$0];next} {for(i=1;i<=NF;i++){if($i in a){next}}} 1' to_delete.csv Input_file > temp_file && mv temp_file Input_file
Explanation: Adding explanation for above code too now.
awk -F, ' ##Setting field separator as comma here.
FNR==NR{ ##checking condition FNR==NR which will be TRUE when first Input_file is being read.
a[$0] ##Creating an array named a whose index is $0.
next ##next will skip all further statements from here.
}
{
for(i=1;i<=NF;i++){ ##Starting a for loop from value i=1 to till value of NF.
if($i in a){ ##checking if $i is present in array a if yes then go into this condition block.
next ##next will skip all further statements(since we DO NOt want to print any matching contents)
} ##Closing if block now.
} ##Closing for block here.
} ##Closing block which should be executed for 2nd Input_file here.
1 ##awk works on pattern and action method so making condition TRUE here and not mentioning any action so by default print of current line will happen.
' to_delete.csv Input_file ##Mentioning Input_file names here now.