I am trying to filter a file_to_filter by using another filter_file, which is just a list of strings in $1. I think I am close but can not seem to include the header row in the output. The file_to_filter is tab delimited as well. Thank you :).
file_to_filter
Chr Start End Ref Alt Func.refGene Gene.refGene
chr1 160098543 160098543 G A exonic ATP1A2
chr1 172410967 172410967 G A exonic PIGC
filter_file
PIGC
desired output (header included)
Chr Start End Ref Alt Func.refGene Gene.refGene
chr1 172410967 172410967 G A exonic PIGC
awk with current output (header not included)
awk -F'\t' 'NR==1{A[$1];next}$7 in A' file test
chr1 172410967 172410967 G A exonic PIGC
Assuming your fields really are tab-separated:
awk -F'\t' 'NR==FNR{tgts[$1]; next} (FNR==1) || ($7 in tgts)' filter_file file_to_filter
To start learning awk, read the book Effective Awk Programing, 4th Edition, by Arnold Robbins.
Related
In the awk below I am trying to skip the # in f1 and match $4 in f1 with $6 of f2 and if there is a match the contents in $5 in f1 are appended to $8 in f2.
I added comments as well.
f1 tab-delimited
#
#
chr1 2019345 2030758 GABRD {There is a lot of text here}
f2 tab-delimited
chr1 2028270 2028270 G A GABRD This has text in it.
chr1 2028297 2028302 CAT C GABRD This has text in it.
chr1 2229406 2229406 A G SKI This has text in it.
chr1 2304553 2304553 G A SKI This has text in it.
chr1 2306636 2306636 C T SKI This has text in it.
desired tab-delimited
chr1 2028270 2028270 G A GABRD This has text in it. {There is a lot of text here}
chr1 2028297 2028302 CAT C GABRD This has text in it. {There is a lot of text here}
chr1 2229406 2229406 A G SKI This has text in it. unknown
chr1 2304553 2304553 G A SKI This has text in it. unknown
chr1 2306636 2306636 C T SKI This has text in it. unknown
awk
awk '/^[^#]/ # skipping lines starting with # in f1
FNR==NR{ # checking condition which will be TRUE when f2 is being read.
a[$4]=$6 # creating array a with index of $4 and value of $6 here.
next # next will skip all further statements from here.
}
{
print $0,($4 in a?a[$4]:"unknown") # Printing current line and checking if 1st field is there in and print a[$4] else print unknown.
}' f1 f2 # close and inputs
Assumptions:
skip all lines (all files) that start with a #
f1/$5 does not contain any embedded tabs
if f1/$4 == f2/$6 then replace f2/$8 with f1/$5 else set f2/$8 = "unknown"
One awk idea:
awk '
BEGIN { FS=OFS="\t" }
/^#/ { next }
FNR==NR { a[$4]=$5; next }
{ $8= ($6 in a) ? a[$6] : "unknown" }
1
' f1 f2
This generates:
chr1 2028270 2028270 G A GABRD This has text in it. {There is a lot of text here}
chr1 2028297 2028302 CAT C GABRD This has text in it. {There is a lot of text here}
chr1 2229406 2229406 A G SKI This has text in it. unknown
chr1 2304553 2304553 G A SKI This has text in it. unknown
chr1 2306636 2306636 C T SKI This has text in it. unknown
In the awk I am splitting on the space or : after the chrxx (it is not consistent so I added both as FS, then splitting on the -. I can not seem to duplicate $2 if there is no - after it. Lines 2,3 are examples. If there is a - after the number then the value to the right of it is $3 in the ouput. The awk seems close but isn't duplicating the value. Thank you :).
in
chr17 7124137-7124146 ACADVL
chr1 229568460 ACTA1
chr10 90708637 ACTA2
awk
awk -F"[ :-]" '$3=$3?$3:$2' OFS='\t' in
current
chr17 7124137 7124146 ACADVL
chr1 229568460 ACTA1
chr10 90708637 ACTA2
desired output
chr17 7124137 7124146 ACADVL
chr1 229568460 229568460 ACTA1
chr10 90708637 90708637 ACTA2
If number of fields is three, copy 3rd field to 4th, and 2nd to 3rd. Force recomputing of whole record to make output tab separated regardless of what's done before.
awk -F'[ :-]' 'NF==3{$4=$3;$3=$2} {$1=$1} 1' OFS='\t' in
$ perl -lane 'if($F[1]=~/\-/){$F[1]=~s/-/ /}else{splice #F, 1, 0, $F[1];}print "#F" ' temp
chr17 7124137 7124146 ACADVL
chr1 229568460 229568460 ACTA1
chr10 90708637 90708637 ACTA2
[netcrk#o2uk1061 infinys_root]$
In the below awk I am trying to match $2 in file1 up until the ., with $4 in file2 up to the first undescore _. If a match is found then that portion of file2 is up dated with the matching $1 value in file1. I think it is close but not sure how to account for the . in file1. In my real data there are thousands of lines, but they are all in the below format and a match may not always be found. The awk as is does execute but file2 is not updated, I think because the . is not matching. Thank you :).
file 1 space delimited
TGFBR1 NM_004612.3
TGFBR2 NM_003242.5
TGFBR3 NM_003243.4
file 2 tab-delimited
chr1 92149295 92149414 NM_003243_cds_0_0_chr1_92149296_r
chr1 92161228 92161336 NM_003243_cds_1_0_chr1_92161229_r
chr1 92163645 92163687 NM_003243_cds_2_0_chr1_92163646_r
chr3 30648375 30648469 NM_003242_cds_0_0_chr3_30648376_f
chr3 30686238 30686407 NM_003242_cds_1_0_chr3_30686239_f
chr9 101867487 101867584 NM_004612_cds_0_0_chr9_101867488_f
chr9 101904817 101904985 NM_001130916_cds_3_0_chr9_101904818_f
desired output tab-delimited
chr1 92149295 92149414 TGFBR3_cds_0_0_chr1_92149296_r
chr1 92161228 92161336 TGFBR3_cds_1_0_chr1_92161229_r
chr1 92163645 92163687 TGFBR3_cds_2_0_chr1_92163646_r
chr3 30648375 30648469 TGFBR2_cds_0_0_chr3_30648376_f
chr3 30686238 30686407 TGFBR2_cds_1_0_chr3_30686239_f
chr9 101867487 101867584 TGFBR1_cds_0_0_chr9_101867488_f
awk
awk 'FNR==NR {A[$1]=$1; next} $4 in A {sub ($4, $4 "_" A[$4]) }1' OFS='\t' file1 FS='\t' file2
Following awk may help you on same. Also you could change you FS field separator as per your Input_file(s) too, eg--> Input_file1 is space delimited then use FS=" " before it and Input_file2 is TAB delimited then use FS="\t" before its name.
awk '
FNR==NR{
val=$2;
sub(/\..*/,"",val);
a[val]=$1;
next
}
{
split($4,array,"_")
}
((array[1]"_"array[2]) in a){
sub(/.*_cds/,a[array[1]"_"array[2]]"_cds",$4);
print
}
' Input_file1 Input_file2
Output will be as follows:
chr1 92149295 92149414 TGFBR3_cds_0_0_chr1_92149296_r
chr1 92161228 92161336 TGFBR3_cds_1_0_chr1_92161229_r
chr1 92163645 92163687 TGFBR3_cds_2_0_chr1_92163646_r
chr3 30648375 30648469 TGFBR2_cds_0_0_chr3_30648376_f
chr3 30686238 30686407 TGFBR2_cds_1_0_chr3_30686239_f
chr9 101867487 101867584 TGFBR1_cds_0_0_chr9_101867488_f
I have an awk that seemed straight-forward, but I seem to be having a problem. In the file below if $5 starts with a ( then to that string a ) is added at the end. However if$5does not start with a(then nothing is done. The out is separated by a tab. Theawkis almost right but I am not sure how to add the condition to only add a)if the field starts with a(`. Thank you :).
file
chr7 100490775 100491863 chr7:100490775-100491863 ACHE
chr7 100488568 100488719 chr7:100488568-100488719 ACHE;DJ051769
chr1 159174749 159174770 chr1:159174749-159174770 (ACKR1
chr1 159175223 159176240 chr1:159175223-159176240 (ACKR1
awk tried
awk -v OFS='\t' '{print $1,$2,$3,$4,""$5")"}' file
current output
chr7 100490775 100491863 chr7:100490775-100491863 ACHE)
chr7 100488568 100488719 chr7:100488568-100488719 ACHE;DJ051769)
chr1 159174749 159174770 chr1:159174749-159174770 (ACKR1)
chr1 159175223 159176240 chr1:159175223-159176240 (ACKR1)
desired output (line 1 and 2 nothing is done but line 3 and 4 have a ) added to the end)
chr7 100490775 100491863 chr7:100490775-100491863 ACHE
chr7 100488568 100488719 chr7:100488568-100488719 ACHE;DJ051769
chr1 159174749 159174770 chr1:159174749-159174770 (ACKR1)
chr1 159175223 159176240 chr1:159175223-159176240 (ACKR1)
$ awk -v OFS='\t' '{p = substr($5,1,1)=="(" ? ")" : ""; $5=$5 p}1' mp.txt
chr7 100490775 100491863 chr7:100490775-100491863 ACHE
chr7 100488568 100488719 chr7:100488568-100488719 ACHE;DJ051769
chr1 159174749 159174770 chr1:159174749-159174770 (ACKR1)
chr1 159175223 159176240 chr1:159175223-159176240 (ACKR1)
Check the first character of the 5th field. If it is ( append a ) to the end, otherwise append the empty string.
By appending something (where one of the somethings is "nothing" :) in all cases, we force awk to reconstitute the record with the defined (tab) output separator, which saves us from having to print the individual fields. The trailing 1 acts as an always-true pattern whose default action is simply to print the reconstituted line.
I am trying to clean-up some tab-delineated files and thought that the awk below would remove field 18 Otherinfo from the file. I also tried cut and can not seem to get the desired output. Thank you :).
file
Chr Start End Ref Alt Func.refGene Gene.refGene GeneDetail.refGene ExonicFunc.refGene AAChange.refGene PopFreqMax CLINSIG CLNDBN CLNACC CLNDSDB CLNDSDBID common Otherinfo
chr1 949654 949654 A G exonic ISG15 . synonymous SNV ISG15:NM_005101:exon2:c.294A>G:p.V98V 0.96 . . . . . . 1 3825.28 624 chr1 949654 . A G 3825.28 PASS AF=1;AO=621;DP=624;FAO=399;FDP=399;FR=.;FRO=0;FSAF=225;FSAR=174;FSRF=0;FSRR=0;FWDB=0.00425236;FXX=0.00249994;HRUN=1;LEN=1;MLLD=97.922;OALT=G;OID=.;OMAPALT=G;OPOS=949654;OREF=A;PB=0.5;PBP=1;QD=38.3487;RBI=0.0367904;REFB=0.0353003;REVB=-0.0365438;RO=2;SAF=335;SAR=286;SRF=0;SRR=2;SSEN=0;SSEP=0;SSSB=0.00332809;STB=0.5;STBP=1;TYPE=snp;VARB=-3.42335e-05;ANN=ISG15 GT:GQ:DP:FDP:RO:FRO:AO:FAO:AF:SAR:SAF:SRF:SRR:FSAR:FSAF:FSRF:FSRR 1/1:171:624:399:2:0:621:399:1:286:335:0:2:174:225:0:0 GOOD 399 reads
desired output
Chr Start End Ref Alt Func.refGene Gene.refGene GeneDetail.refGene ExonicFunc.refGene AAChange.refGene PopFreqMax CLINSIG CLNDBN CLNACC CLNDSDB CLNDSDBID common
chr1 949654 949654 A G exonic ISG15 0 synonymous SNV ISG15:NM_005101:exon2:c.294A>G:p.V98V 0.96 . . . . . .
awk (runs but doesn't remove field 18)
awk '{ $18=""; print }' file1
cut (removes all field except 18)
cut -f18 file1
By default, awk uses blanks as delimiters. Therefore, you have to specify to use tabs as delimiters in your output (OFS):
awk 'BEGIN{FS=OFS="\t"}{$18=""; gsub(/\t\t/,"\t")}1' file1