This question already has an answer here:
awk: function to escape regex operators from a string
(1 answer)
Closed 6 months ago.
Trying to replace | by , using awk
$ awk '{gsub("|",","); print}' sample.txt | tee sample.txt
sample file contains ||| characters and target is to replace with ,,, when fired above command the output is ,|,|,| where it should be ,,,
Try awk '{gsub(/\|/,","); print}' sample.txt | tee output.txt. Note that "|" need to be escaped with "\", and the result is then "tee"ed to another file. Writing back to the same file may not be safe.
Related
I have the following data in multiple lines:
1
2
3
4
5
6
7
8
9
10
I want to convert them to lines separated by "|" and "()":
(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|10
I made a mistake. I'm sorry,I want to convert them to lines separated by "|" and "()":
(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|(10)
What I have tried is:
seq 10 | sed -r 's/(.*)/(\1)/'|paste -sd"|"
What's the best unix one-liner to do that?
This might work for you (GNU sed):
sed 's/.*/(&)/;H;1h;$!d;x;s/\n/|/g' file
Surround each line by parens.
Append all lines to the hold space except for the first line which replaces the hold space.
Delete all lines except the last.
On the last line, swap to the hold space and replace all newlines by |'s.
N.B. When a line is deleted no further commands are invoked and the command cycle begins again. That is why the last two commands are only executed on the last line of the file.
Alternative:
sed -z 's/\n$//;s/.*/(&)/mg;y/\n/|/' file
With your shown samples please try following awk code. This should work in any version of awk.
awk -v OFS="|" '{val=(val?val OFS:"") "("$0")"} END{print val}' Input_file
Using GNU sed
$ sed -Ez ':a;s/([0-9]+)\n/(\1)|/;ta;s/\|$/\n/' input_file
(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|(10)
Here is another simple awk command:
awk 'NR>1 {printf "%s|", p} {p="(" $0 ")"} END {print p}' file
(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|(10)
Here it is:
sed -z 's/^/(/;s/\n/)|(/g;s/|($//' your_input
where -z allows you to treat the whole file as a single string with embedded \ns.
In detail, the sed script above consists of 3 commands separated by ;s:
s/^/(/ inserts a ( at the beginning of the whole file,
s/\n/)|(/g changes every \n to )|(;
s/|($// removes the trailing |( resulting from the \n at EOF, that is likely in your file since you are on linux.
With perl:
$ seq 10 | perl -pe 's/.*/($&)/; s/\n/|/ if !eof'
(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|(10)
s/.*/($&)/ to surround input lines with ()
s/\n/|/ if !eof will change newline to | except for the last input line.
Here's a solution with paste (just for fun):
$ seq 10 | paste -d'()' /dev/null - /dev/null | paste -sd'|'
(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|(10)
Using any awk:
$ seq 10 | awk '{printf "%s(%s)", sep, $0; sep="|"} END{print ""}'
(1)|(2)|(3)|(4)|(5)|(6)|(7)|(8)|(9)|(10)
This question already has answers here:
How do I use shell variables in an awk script?
(7 answers)
Closed 1 year ago.
I have a csv file that I need to add a number of columns at the end. The new columns are variables taken from other files.
STNO=3389
STNNAME=SWORDS
awk -F "," '{ stnname='"$STNNAME"';stno='"$STNO"';print $0","stnname","stno }' infile
example of the output.
992501062,-6.278983000,202105210736,,3389
The stno is written fine but the stnname is blank. It seems like I can use numeric variables but not text.
any help appreciated.
thanks.
You are interpolating the literal symbol SWORDS where apparently you were hoping to interpolate a quoted string. Awk has no variable named SWORDS so it resolves to an empty string.
Better hygiene and stability altogether is to avoid interpolating junk right into your Awk script. Use -v on the Awk command line to pass in values.
awk -v stnname="$STNNAME" -v stno="$STNO" 'BEGIN {FS=OFS=","}
{ print $0, stnname , stno }' infile
Tangentially, avoid upper case for your private shell variables.
It is very easy to get lost in a sea of quotes. Maybe catch the env variables using -v like this:
awk -v stnname="$STNNAME" -v stno="$STNO" -F "," '{ print $0,stnname,stno }' infile
then you can use them in the command directly without trying to piece together a string
This question already has answers here:
Why does my tool output overwrite itself and how do I fix it?
(3 answers)
Closed 2 years ago.
I am trying to put a char at the end of each line in a file:
a
aa
aaa
aah
aahed
aahing
aahs
aal
aalii
aaliis
using sed/awk:
awk '{print $0 "A"}' words_alpha.txt > words_alpha_hyp.txt
sed -i 's/$/A/' words_alpha.txt > words_alpha_hyp.txt
I got always instead the char 'a' a prefix:
A
A
Aa
Ah
Ahed
Ahing
Ahs
Al
Alii
Aliis
Als
Am
Ani
Ardvark
Ardvarks
Interestingly if I try to '\n' before the char 'a':
awk '{print $0 "\n a"}' words_alpha.txt > words_alpha_hyp.txt
a
a
aa
a
aaa
a
aah
a
aahed
a
aahing
a
aahs
a
aal
a
I does what it supposed to do... I don't understand it...
Your input file most likely has DOS CRLF line terminators i.e. instead of \n being a \r\n. Modify your record separator to handle those lines, needs GNU awk for having to use a multi character record separator
awk -v RS="\r\n" '{ $0 = $0 "a" }1' file
Or make the obvious choice of converting the file to *nix line endings \n by doing dos2unix file and run your original command again.
This question already has answers here:
How do I use shell variables in an awk script?
(7 answers)
Closed 2 years ago.
I want to write a script which replaces 'gene' feature from the 3rd column of the $1 file into 'quant'.
#!/bin/bash
awk -F "\t" '{gsub("gene","quant",$3);print}' $1
The code works well, however I would like to read "gene" as an argument, so how can I specify argument $2 instead of 'gene' in the above code?
Thanks!
Use -v awkvar="$value" to create an awk variable with a given value. Thus:
#!/bin/bash
awk -v orig="$2" -F '\t' '{gsub(orig,"quant",$3);print}' "$1"
This question already has answers here:
How can I prepend a string to the beginning of each line in a file?
(9 answers)
Closed 3 years ago.
I am working with ffmpeg and want to automate making one big video from many smaller. I can use a list file, but each line Must Be file 'name.ext'. I can't figure out how to get sed or awk to NOT see the ' as a control character. Any way to do that?
I have tried using a variable as instead of the string file ', tried a two statement script where i set file # then use another cmd to change the # to ', but it fails every time
awk '{print "line #" $0}' uselessInfo.txt >goofy2.txt
sed '/#/\'/g' goofy2.txt >goofy3.txt
tried the sed line with " around the ' also
Neither sed nor awk are seeing ' as a control character. In fact they aren't seeing the ' at all in the code you posted - the shell doesn't allow you to use single quotes inside a single quote delimited script.
Your question isn't clear. Is this what you're trying to do?
$ echo 'foo' | awk '{print "file \047" $0 "\047"}'
file 'foo'
$ echo 'foo' | sed 's/.*/file '\''&'\''/'
file 'foo'
$ echo 'foo' | sed "s/.*/file '&'/"
file 'foo'
If not then edit your question to clarify and provide a concrete example we can test against.
If you just want to add a single quote at the front of each line:
> cat test.txt
a
b
c
> sed "s/^/'/" test.txt
'a
'b
'c
You can then output this to whatever file you wish as in your example.
The solution to your problem, I believe, lies in the fact that characters within single quotes on the command line are not interpreted. This means that when you try to escape a quote inside a single quoted string, this will not work and you just get a backslash in your string. Comparing this to a string bound by double quotes, we see that the backslash is interpreted before being passed to the command echo as an argument.
> echo '\'
\
> echo "\""
"
another useful trick is defining single quote as a variable
$ echo "part 1" | awk -v q="'" '{print "line " q $0 q}'
line 'part 1'