how to split data in 5 letters around one specific letter for each section - awk

Question

I have a data like this
>sp|Q96A73|P33MX_HUMAN Putative monooxygenase p33MONOX OS=Homo sapiens OX=9606 GN=KIAA1191 PE=1 SV=1
RNDDDDTSVCLGTRQCSWFAGCTNRTWNSSAVPLIGLPNTQDYKWVDRNSGLTWSGNDTCLYSCQNQTKGLLYQLFRNLFCSYGLTEAHGKWRCADASITNDKGHDGHRTPTWWLTGSNLTLSVNNSGLFFLCGNGVYKGFPPKWSGRCGLGYLVPSLTRYLTLNASQITNLRSFIHKVTPHR
>sp|P13674|P4HA1_HUMAN Prolyl 4-hydroxylase subunit alpha-1 OS=Homo sapiens OX=9606 GN=P4HA1 PE=1 SV=2
VECCPNCRGTGMQIRIHQIGPGMVQQIQSVCMECQGHGERISPKDRCKSCNGRKIVREKKILEVHIDKGMKDGQKITFHGEGDQEPGLEPGDIIIVLDQKDHAVFTRRGEDLFMCMDIQLVEALCGFQKPISTLDNRTIVITSHPGQIVKHGDIKCVLNEGMPIYRRPYEKGRLIIEFKVNFPENGFLSPDKLSLLEKLLPERKEVEE
>sp|Q7Z4N8|P4HA3_HUMAN Prolyl 4-hydroxylase subunit alpha-3 OS=Homo sapiens OX=9606 GN=P4HA3 PE=1 SV=1
MTEQMTLRGTLKGHNGWVTQIATTPQFPDMILSASRDKTIIMWKLTRDETNYGIPQRALRGHSHFVSDVVISSDGQFALSGSWDGTLRLWDLTTGTTTRRFVGHTKDVLSVAFSSDNRQIVSGSRDKTIKLWNTLGVCKYTVQDESHSEWVSCVRFSPNSSNPIIVSCGWDKLVKVWNLANCKLK
>sp|P04637|P53_HUMAN Cellular tumor antigen p53 OS=Homo sapiens OX=9606 GN=TP53 PE=1 SV=4
IQVVSRCRLRHTEVLPAEEENDSLGADGTHGAGAMESAAGVLIKLFCVHTKALQDVQIRFQPQL
>sp|P10144|GRAB_HUMAN Granzyme B OS=Homo sapiens OX=9606 GN=GZMB PE=1 SV=2
MQPILLLLAFLLLPRADAGEIIGGHEAKPHSRPYMAYLMIWDQKSLKRCGGFLIRDDFVLTAAHCWGSSINVTLGAHNIKEQEPTQQFIPVKRPIPHPAYNPKNFSNDIMLLQLERKAKRTRAVQPLRLPSNKAQVKPGQTCSVAGWGQTAPLGKHSHTLQEVKMTVQEDRKCES
>sp|Q9UHX1|PUF60_HUMAN Poly(U)-binding-splicing factor PUF60 OS=Homo sapiens OX=9606 GN=PUF60 PE=1 SV=1
MGKDYYQTLGLARGASDEEIKRAYRRQALRYHPDKNKEPGAEEKFKEIAEAYDVLSDPRKREIFDRYGEEGLKGSGPSGGSGGGANGTSFSYTFHGDPHAMFAEFFGGRNPFDTFFGQRNGEEGMDIDDPFSGFPMGMGGFTNVNFGRSRSAQEPARKKQDPPVTHDLRVSLEEIYSGCTKKMKISHK
>sp|Q06416|P5F1B_HUMAN Putative POU domain, class 5, transcription factor 1B OS=Homo sapiens OX=9606 GN=POU5F1B PE=5 SV=2
IVVKGHSTCLSEGALSPDGTVLATASHDGYVKFWQIYIEGQDEPRCLHEWKPHDGRPLSCLLFCDNHKKQDPDVPFWRFLITGADQNRELKMWCTVSWTCLQTIRFSPDIFSSVSVPPSLKVCLDLSAEYLILSDVQRKVLYVMELLQNQEEGHACFSSISEFLLTHPVLSFGIQVVSRCRLRHTEVLPAEEENDSLGADGTHGAGAMESAAGVLIKLFCVHTKALQDVQIRFQPQLNPDVVAPLPTHTAHEDFTFGESRPELGSEGLGSAAHGSQPDLRRIVELPAPADFLSLSSETKPKLMTPDAFMTPSASLQQITASPSSSSSGSSSSSSSSSSSLTAVSAMSSTSAVDPSLTRPPEELTLSPKLQLDGSLTMSSSGSLQASPRGLLPGLLPAPADKLTPKGPGQVPTATSALSLELQEVEP
>sp|O14683|P5I11_HUMAN Tumor protein p53-inducible protein 11 OS=Homo sapiens OX=9606 GN=TP53I11 PE=1 SV=2
MIHNYMEHLERTKLHQLSGSDQLESTAHSRIRKERPISLGIFPLPAGDGLLTPDAQKGGETPGSEQWKFQELSQPRSHTSLKVSNSPEPQKAVEQEDELSDVSQGGSKATTPASTANSDVATIPTDTPLKEENEGFVKVTDAPNKSEISKHIEVQVAQETRNVSTGSAENEEKSEVQAIIESTPELDMDKDLSGYKGSSTPTKGIENKAFDRNTESLFEELSSAGSGLIGDVDEGADLLGMGREVENLILENTQLLETKNALNIVKNDLIAKVDELTCEKDVLQGELEAVKQAKLKLEEKNRELEEELRKARAEAEDARQKAKDDDDSDIPTAQRKRFTRVEMARVLMERNQYKERLMELQEAVRWTEMIRASRENPAMQEKKRSSIWQFFSRLFSSSSNTTKKPEPPVNLKYNAPTSHVTPSVK
I am trying to find 5 letter left and 5 letters right to each F for each section and then calculate the number of E or D in each of them
a representative output looks like below
>sp|Q96A73|P33MX_HUMAN Putative monooxygenase p33MONOX OS=Homo sapiens OX=9606 GN=KIAA1191 PE=1 SV=1
RQCSWFAGCTN 0 0
LLYQLFRNLFC 0 0
LFRNLFCSYGL 0 0
NNSGLFFLCGN 0 0
NSGLFFLCGNG 0 0
GVYKGFPPKWS 0 0
TNLRSFIHKVT 0 0
>sp|P13674|P4HA1_HUMAN Prolyl 4-hydroxylase subunit alpha-1 OS=Homo sapiens OX=9606 GN=P4HA1 PE=1 SV=2
GQKITFHGEGD 1 1
KDHAVFTRRGE 1 1
RGEDLFMCMDI 1 2
EALCGFQKPIS 1 0
RLIIEFKVNFP 1 0
EFKVNFPENGF 2 0
FPENGFLSPDK 1 0
>sp|Q7Z4N8|P4HA3_HUMAN Prolyl 4-hydroxylase subunit alpha-3 OS=Homo sapiens OX=9606 GN=P4HA3 PE=1 SV=1
ATTPQFPDMIL 0 1
RGHSHFVSDVV 0 1
SSDGQFALSGS 0 1
TTTRRFVGHTK 0 0
VLSVAFSSDNR 0 1
VSCVRFSPNSS 0 0
>sp|P04637|P53_HUMAN Cellular tumor antigen p53 OS=Homo sapiens OX=9606 GN=TP53 PE=1 SV=4
VLIKLFCVHTK 0 0
DVQIRFQPQL 0 1
>sp|P10144|GRAB_HUMAN Granzyme B OS=Homo sapiens OX=9606 GN=GZMB PE=1 SV=2
LLLLAFLLLPR 0 0
KRCGGFLIRDD 0 2
LIRDDFVLTAA 0 2
EPTQQFIPVKR 1 0
YNPKNFSNDIM 0 1
>sp|Q9UHX1|PUF60_HUMAN Poly(U)-binding-splicing factor PUF60 OS=Homo sapiens OX=9606 GN=PUF60 PE=1 SV=1
GAEEKFKEIAE 4 0
RKREIFDRYGE 2 1
ANGTSFSYTFH 0 0
SFSYTFHGDPH 0 1
DPHAMFAEFFG 0 1
AMFAEFFGGRN 1 0
MFAEFFGGRNP 1 0
GGRNPFDTFFG 0 1
NPFDTFFGQRN 0 1
PFDTFFGQRNG 0 1
DIDDPFSGFPM 0 3
DPFSGFPMGMG 0 1
MGMGGFTNVNF 0 0
FTNVNFGRSRS 0 0
>sp|Q06416|P5F1B_HUMAN Putative POU domain, class 5, transcription factor 1B OS=Homo sapiens OX=9606 GN=POU5F1B PE=5 SV=2
DGYVKFWQIYI 0 1
LSCLLFCDNHK 0 1
DPDVPFWRFLI 0 2
VPFWRFLITGA 0 0
LQTIRFSPDIF 0 1
FSPDIFSSVSV 0 1
EGHACFSSISE 0 0
SSISEFLLTHP 1 0
HPVLSFGIQVV 0 0
VLIKLFCVHTK 0 0
DVQIRFQPQLN 0 1
TAHEDFTFGES 2 1
HEDFTFGESRP 2 1
PAPADFLSLSS 0 1
MTPDAFMTPSA 0 1
>sp|O14683|P5I11_HUMAN Tumor protein p53-inducible protein 11 OS=Homo sapiens OX=9606 GN=TP53I11 PE=1 SV=2
ISLGIFPLPAG 0 0
SEQWKFQELSQ 2 0
EENEGFVKVTD 3 1
IENKAFDRNTE 2 1
NTESLFEELSS 3 0
AQRKRFTRVEM 1 0
SSIWQFFSRLF 0 0
SIWQFFSRLFS 0 0
FFSRLFSSSSN 0 0
at the begging I thought of finding the 5 letters left and right to F.
but I could not figure it out how to do that

Answer 1

In awk:
$ awk '
NR%2 {print; next } # print every odd record
{ # the even records are processed
while(match($0,/.{5}F.{0,5}/)) { # get 5 before and upto 5 after F
# 5 before F ^^^ ^^^ 0-5 chars after F
# change to /.{0,5}F.{0,5}/ if needed
print s=substr($0,RSTART,RLENGTH), # print match
gsub(/E/,"E",s), # count of Es
gsub(/D/,"D",s) # count of Ds
$0=substr($0,RSTART+1) # shorten the search string
}
}' file
Some output:
>sp|Q96A73|P33MX_HUMAN Putative monooxygenase p33MONOX OS=Homo sapiens OX=9606 GN=KIAA1191 PE=1 SV=1
RQCSWFAGCTN 0 0
LLYQLFRNLFC 0 0 # notice another F in the 5+F+5 window
LFRNLFCSYGL 0 0 # .. getting handled
NNSGLFFLCGN 0 0
NSGLFFLCGNG 0 0
GVYKGFPPKWS 0 0
TNLRSFIHKVT 0 0
...
>sp|P04637|P53_HUMAN Cellular tumor antigen p53 OS=Homo sapiens OX=9606 GN=TP53 PE=1 SV=4
VLIKLFCVHTK 0 0
DVQIRFQPQL 0 1 # ...F{0,5}

Answer 2

Check out this Perl solution
perl -lne ' if (/^[^>]/) { while(/(?<=(\w{5}))F(?=(\w{5}))/g) { $x="$1F$2";$e=()=$x=~/E/g; $d=()=$x=~/D/g; print "$x $e $d" } } else { print }' 5letter.txt
EDIT1:
DVQIRFQPQL 0 1 # Edge case
For accommodating the edge case as mentioned by OP in the comments - the string to the right of F can be less than 5 letters if it is at the end of the line
perl -lne ' if (/^[^>]/) { while(/(?<=(.{5}))F(?=(.{0,5}))/g) { $x="$1F$2";$e=()=$x=~/E/g; $d=()=$x=~/D/g; print "$x $e $d" } } else { print }' 5letter.txt
For each line that doesn't start with >, use positive lookbehind to match 5 letter \w to the left of F and positive lookahead to match 5 letter \w to the right of F. Using while loop and /g on the match operator scan the line and for each match copy the $1F$2 into variable $x. Use list context and count the occurrences of E and D. Print the result finally.
with given inputs
$ perl -lne ' if (/^[^>]/) { while(/(?<=(.{5}))F(?=(.{0,5}))/g) { $x="$1F$2";$e=()=$x=~/E/g; $d=()=$x=~/D/g; print "$x $e $d" } } else { print }' 5letter.txt
>sp|Q96A73|P33MX_HUMAN Putative monooxygenase p33MONOX OS=Homo sapiens OX=9606 GN=KIAA1191 PE=1 SV=1
RQCSWFAGCTN 0 0
LLYQLFRNLFC 0 0
LFRNLFCSYGL 0 0
NNSGLFFLCGN 0 0
NSGLFFLCGNG 0 0
GVYKGFPPKWS 0 0
TNLRSFIHKVT 0 0
>sp|P13674|P4HA1_HUMAN Prolyl 4-hydroxylase subunit alpha-1 OS=Homo sapiens OX=9606 GN=P4HA1 PE=1 SV=2
GQKITFHGEGD 1 1
KDHAVFTRRGE 1 1
RGEDLFMCMDI 1 2
EALCGFQKPIS 1 0
RLIIEFKVNFP 1 0
EFKVNFPENGF 2 0
FPENGFLSPDK 1 1
>sp|Q7Z4N8|P4HA3_HUMAN Prolyl 4-hydroxylase subunit alpha-3 OS=Homo sapiens OX=9606 GN=P4HA3 PE=1 SV=1
ATTPQFPDMIL 0 1
RGHSHFVSDVV 0 1
SSDGQFALSGS 0 1
TTTRRFVGHTK 0 0
VLSVAFSSDNR 0 1
VSCVRFSPNSS 0 0
>sp|P04637|P53_HUMAN Cellular tumor antigen p53 OS=Homo sapiens OX=9606 GN=TP53 PE=1 SV=4
VLIKLFCVHTK 0 0
DVQIRFQPQL 0 1
>sp|P10144|GRAB_HUMAN Granzyme B OS=Homo sapiens OX=9606 GN=GZMB PE=1 SV=2
LLLLAFLLLPR 0 0
KRCGGFLIRDD 0 2
LIRDDFVLTAA 0 2
EPTQQFIPVKR 1 0
YNPKNFSNDIM 0 1
>sp|Q9UHX1|PUF60_HUMAN Poly(U)-binding-splicing factor PUF60 OS=Homo sapiens OX=9606 GN=PUF60 PE=1 SV=1
GAEEKFKEIAE 4 0
RKREIFDRYGE 2 1
ANGTSFSYTFH 0 0
SFSYTFHGDPH 0 1
DPHAMFAEFFG 1 1
AMFAEFFGGRN 1 0
MFAEFFGGRNP 1 0
GGRNPFDTFFG 0 1
NPFDTFFGQRN 0 1
PFDTFFGQRNG 0 1
DIDDPFSGFPM 0 3
DPFSGFPMGMG 0 1
MGMGGFTNVNF 0 0
FTNVNFGRSRS 0 0
>sp|Q06416|P5F1B_HUMAN Putative POU domain, class 5, transcription factor 1B OS=Homo sapiens OX=9606 GN=POU5F1B PE=5 SV=2
DGYVKFWQIYI 0 1
LSCLLFCDNHK 0 1
DPDVPFWRFLI 0 2
VPFWRFLITGA 0 0
LQTIRFSPDIF 0 1
FSPDIFSSVSV 0 1
EGHACFSSISE 2 0
SSISEFLLTHP 1 0
HPVLSFGIQVV 0 0
VLIKLFCVHTK 0 0
DVQIRFQPQLN 0 1
TAHEDFTFGES 2 1
HEDFTFGESRP 2 1
PAPADFLSLSS 0 1
MTPDAFMTPSA 0 1
>sp|O14683|P5I11_HUMAN Tumor protein p53-inducible protein 11 OS=Homo sapiens OX=9606 GN=TP53I11 PE=1 SV=2
ISLGIFPLPAG 0 0
SEQWKFQELSQ 2 0
EENEGFVKVTD 3 1
IENKAFDRNTE 2 1
NTESLFEELSS 3 0
AQRKRFTRVEM 1 0
SSIWQFFSRLF 0 0
SIWQFFSRLFS 0 0
FFSRLFSSSSN 0 0
$
PS:
$ echo "RNDDDDTSVCLGTRQCSWFAGCTNRTWNSSAVPLIGLPNTQDYKWVDRNSGLTWSGNDTCLYSCQNQTKGLLYQLFRNLFCSYGLTEAHGKWRCADASITNDKGHDGHRTPTWWLTGSNLTLSVNNSGLFFLCGNGVYKGFPPKWSGRCGLGYLVPSLT
RYLTLNASQITNLRSFIHKVTPHR" | perl -ne ' if (/^[^>]/) { $y=$_;while(/(.{10})/g) { $x=$1; $c++ for($x=~/F/g) ; print "$x $c\n"; $c=0 } } '
RNDDDDTSVC
LGTRQCSWFA 1
GCTNRTWNSS 0
AVPLIGLPNT 0
QDYKWVDRNS 0
GLTWSGNDTC 0
LYSCQNQTKG 0
LLYQLFRNLF 2
CSYGLTEAHG 0
KWRCADASIT 0
NDKGHDGHRT 0
PTWWLTGSNL 0
TLSVNNSGLF 1
FLCGNGVYKG 1
FPPKWSGRCG 1
LGYLVPSLT 0
RYLTLNASQI 0
TNLRSFIHKV 1
$

Answer 3

With any awk in any shell on any UNIX box:
$ cat tst.awk
/^>/ { print; next }
{
fpos = 0
while ( match(substr($0,fpos+1),/F/) ) {
fpos += RSTART
str = substr($0,fpos-5,11)
print str, gsub(/E/,"&",str), gsub(/D/,"&",str)
}
}
.
$ awk -f tst.awk file
>sp|Q96A73|P33MX_HUMAN Putative monooxygenase p33MONOX OS=Homo sapiens OX=9606 GN=KIAA1191 PE=1 SV=1
RQCSWFAGCTN 0 0
LLYQLFRNLFC 0 0
LFRNLFCSYGL 0 0
NNSGLFFLCGN 0 0
NSGLFFLCGNG 0 0
GVYKGFPPKWS 0 0
TNLRSFIHKVT 0 0
>sp|P13674|P4HA1_HUMAN Prolyl 4-hydroxylase subunit alpha-1 OS=Homo sapiens OX=9606 GN=P4HA1 PE=1 SV=2
GQKITFHGEGD 1 1
KDHAVFTRRGE 1 1
RGEDLFMCMDI 1 2
EALCGFQKPIS 1 0
RLIIEFKVNFP 1 0
EFKVNFPENGF 2 0
FPENGFLSPDK 1 1
>sp|Q7Z4N8|P4HA3_HUMAN Prolyl 4-hydroxylase subunit alpha-3 OS=Homo sapiens OX=9606 GN=P4HA3 PE=1 SV=1
ATTPQFPDMIL 0 1
RGHSHFVSDVV 0 1
SSDGQFALSGS 0 1
TTTRRFVGHTK 0 0
VLSVAFSSDNR 0 1
VSCVRFSPNSS 0 0
>sp|P04637|P53_HUMAN Cellular tumor antigen p53 OS=Homo sapiens OX=9606 GN=TP53 PE=1 SV=4
VLIKLFCVHTK 0 0
DVQIRFQPQL 0 1
>sp|P10144|GRAB_HUMAN Granzyme B OS=Homo sapiens OX=9606 GN=GZMB PE=1 SV=2
LLLLAFLLLPR 0 0
KRCGGFLIRDD 0 2
LIRDDFVLTAA 0 2
EPTQQFIPVKR 1 0
YNPKNFSNDIM 0 1
>sp|Q9UHX1|PUF60_HUMAN Poly(U)-binding-splicing factor PUF60 OS=Homo sapiens OX=9606 GN=PUF60 PE=1 SV=1
GAEEKFKEIAE 4 0
RKREIFDRYGE 2 1
ANGTSFSYTFH 0 0
SFSYTFHGDPH 0 1
DPHAMFAEFFG 1 1
AMFAEFFGGRN 1 0
MFAEFFGGRNP 1 0
GGRNPFDTFFG 0 1
NPFDTFFGQRN 0 1
PFDTFFGQRNG 0 1
DIDDPFSGFPM 0 3
DPFSGFPMGMG 0 1
MGMGGFTNVNF 0 0
FTNVNFGRSRS 0 0
>sp|Q06416|P5F1B_HUMAN Putative POU domain, class 5, transcription factor 1B OS=Homo sapiens OX=9606 GN=POU5F1B PE=5 SV=2
DGYVKFWQIYI 0 1
LSCLLFCDNHK 0 1
DPDVPFWRFLI 0 2
VPFWRFLITGA 0 0
LQTIRFSPDIF 0 1
FSPDIFSSVSV 0 1
EGHACFSSISE 2 0
SSISEFLLTHP 1 0
HPVLSFGIQVV 0 0
VLIKLFCVHTK 0 0
DVQIRFQPQLN 0 1
TAHEDFTFGES 2 1
HEDFTFGESRP 2 1
PAPADFLSLSS 0 1
MTPDAFMTPSA 0 1
>sp|O14683|P5I11_HUMAN Tumor protein p53-inducible protein 11 OS=Homo sapiens OX=9606 GN=TP53I11 PE=1 SV=2
ISLGIFPLPAG 0 0
SEQWKFQELSQ 2 0
EENEGFVKVTD 3 1
IENKAFDRNTE 2 1
NTESLFEELSS 3 0
AQRKRFTRVEM 1 0
SSIWQFFSRLF 0 0
SIWQFFSRLFS 0 0
FFSRLFSSSSN 0 0

Answer 4

This is an adaptation of the excellent solution of James Brown. The adaptation makes it POSIX compliant, and also corrects for the single case where two F values are less than 5 characters apart. Example:
...RQCSWFAGFCTNRQS...
^ ^
The first window should detect RQCSWFAGFCT while the second window should detect SWFAGFCTNRQ. In the proposed solution, it will only detect AGFCTNRQ, or it might not detect the second F at all. (depending on the usage of .{0,5}F.{0,5} or .{5}F.{0,5} as a regex.
awk '
NR%2 {print; next } # print every odd record
{ # the even records are processed
seq=$0; l=lseq=length($0)
while(match(seq,/F/)) { # find `F`
n = l - lseq + RSTART # get position in $0
print s=substr($0,n-5,(n<6?n+5:11)), # print match and
# correct if F is in the first 5
gsub(/E/,"E",s), # count of Es
gsub(/D/,"D",s) # count of Ds
seq=substr(seq,RSTART+1) # shorten the search string
lseq=lseq-RSTART
}
}' file.fasta
You might be also interested in BioAwk, it is an adapted version of awk which is tuned to process FASTA files.
This gives the result:
bioawk -c fastx '{ print ">" $name }
{ tmp=$seq; l=ltmp=length($seq)
while(match(tmp,/F/)) {
n=l-ltmp+RSTART
print s=substr($seq,n,(n<6?n+5:11)),
gsub(/E/,"E",s),
gsub(/D/,"D",s)
tmp=substr(tmp,RSTART+1)
ltmp=ltmp-RSTART
}}' file.fasta
Here $name is the sequence name (everything after >), and $seq is the complete sequence, even if you have a sequence spanning multiple lines.
Note: BioAwk is based on Brian Kernighan's awk which is documented in "The AWK Programming Language",
by Al Aho, Brian Kernighan, and Peter Weinberger
(Addison-Wesley, 1988, ISBN 0-201-07981-X)
. I'm not sure if this version is compatible with POSIX.