Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 6 months ago.
Improve this question
I have a list of product codes grouped in 2 or 3 lines. I need to return the group where codes are not same (or consecutive)
9003103
9003103
9003978
9003979
9003763
9003728
9003543
9003543
9003543
In this case, only the third group should be returned:
9003763
9003728
I would harness GNU AWK for this task following way, let file.txt content be
9003103
9003103
9003978
9003979
9003763
9003728
9003543
9003543
9003543
then
awk 'BEGIN{RS=""}{diff=$NF-$1;diff=diff>0?diff:-diff}diff>NF' file.txt
gives output
9003763
9003728
Explanation: I set RS to empty string to provoke paragraph mode, thus every block is treated as single line, then for each block I compute absolute of difference between first and last field, if difference is bigger than number of field block is printed.
(tested in GNU Awk 5.0.1)
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 5 months ago.
Improve this question
Let's say i have the following file:
balloons:
- 2
- 3
Each number above should represents how many times i want to print the string. So for example I would like to process this to output as following:
balloons:
- red
- red
- blue
- blue
- blue
I only have red and blue balloons. The digits will vary from one file to another, so my search string would be a simple regex search sed -e "/[[:digit:]]\+/ perform_my_action"
Try:
awk 'BEGIN{idx[2]="red"; idx[3]="blue"}
/^-[ \t]+[0-9]+/{for(i=1;i<=$2;i++) print idx[$2]; next}
1
' file
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 1 year ago.
Improve this question
I have a long text file with the following format:
>foo_bar
TATGTTCTGCAACTGTATAATGGTATAAAAACATTGCAAAATGTAATGAAACTTGTTATTTTGTGAAATACATTCTATAAATATCACTATTTCATGAAAA
ATATTGAAAATCATTTATTTTCGACAAGTAGAACCATAGGTTCTGTAATTGTAAATAGTTCTGCAAACTTAACCTGTTTTGCAGAAGAATATGTTTTCAC
TAGTTAACTTGTAGAATGTTTAGGATTGTTAAAATTTTTAACAAAATAAGATTTTATAGAACATGATTTGCAAAATAACACATTTTGCAATATTTTTATA
CCATATATAGTTGCAGAACATATGGGGACTACGGGCAGCCGGTAAATATGTGGACTACATGGAACTTGTTCAGATACATCTGGAGCAAAGAGCCACCGCT
CTAAATTATCTCTTCTCATTTCCAGTATTATATCTCTCATGCTAAATTATCTCTACAAATCATGACCTCTCTTAGCAATCTCCCTGAGCATCTCCGTAGG
GAGCAGATATTCACCCGTCTTCCGATGAAAGACCTAATGGTCCTCGCATCTGCAAGTCATGTCTTGCGTTAATCTTTCTCTCTCTTTTTGTGGAATCCCA
TCTCTCCTCTTATCAACTAAACCAGATACAGTTTGCACCAACTTTCTTCACTCCCCTGTTACATGAGAAGGCCAGACTTAGGTAGCTTCTGAATCAGAAC
CCGGTCATTCCAAGCATGGGATTTCTTGTTGATCTCTTGTTTTTATGTAATAGTGATCATTTGATATCTGGTGTTGATGGGAATTCAGATGTATGGGACT
TTGTTTATTGTTGATGTGGAATTCTTATATTTTACTGTGTACTATAAAATTTTAGTGATACCTACTATCTATTGTATAAATTGATTAATTGATGTTCTTA
>bar_foo
TATGTTCTGCAACTGTATAATGGTATAAAAACATTGCAAAATGTAATGAAACTTGTTATTTTGTGAAATACATTCTATAAATATCACTATTTCATGAAAA
ATATTGAAAATCATTTATTTTCGACAAGTAGAACCATAGGTTCTGTAATTGTAAATAGTTCTGCAAACTTAACCTGTTTTGCAGAAGAATATGTTTTCAC
TAGTTAACTTGTAGAATGTTTAGGATTGTTAAAATTTTTAACAAAATAAGATTTTATAGAACATGATTTGCAAAATAACACATTTTGCAATATTTTTATA
CCATATATAGTTGCAGAACATATGGGGACTACGGTACTACGGTAAATATGTGGACTACATGGAACTTGTTCAGATACATCTGGAGCAAAGAGCCACCGCT
CTAAATTATCTCTTCTCATTTCCAGCTGCATATCTCTCATGCTAAATTATCTCTACAAATCATGACCTCTCTTAGCAATCTCCCTGAGCATCTCCGTAGG
GAGCAGATATTCACCCGTCTTCCGATGAAAGACCTAATGGTCCTCGCATCTGCAAGTCATGTCTTGCGTTAATCTTTCTCTCTCTTTTTGTGGAATCCCA
TCTCTCCTCTTATCAACTAAACCAGATACAGTTTGCACCAACTTTCTTCACTCCCCTGTTACATGAGAAGGCCAGACTTAGGTAGCTTCTGAATCAGAAC
CCGGTCATTCCAAGCATGGGATTTCTTGTTGATCTCTTGTTTTTATGTAATAGTGATCATTTGATATCTGGTGTTGATGGGAATTCAGATGTATGGGACT
TTGTTTATTGTTGATGTGGAATTCTTATATTTTACTGTGTACTATAAAATTTTAGTGATACCTACTATCTATTGTATAAATTGATTAATTGATGTTCTTA
I.e., there is a header line which begins with a ">", and then an arbitrary number of lines with no more than 100 letters in them. I would like to find the positions within the non-header lines that match either "GCAGC" or "GCTGC". Overlapping match sites would both get recorded individually.
An example output would be a three column text file where the first column contained the header line for that block of text minus the ">", the second column contained the start position of a pattern match (i.e., the number of characters into the text block, excluding line-break characters), and the third column recorded which of the two patterns were matched. E.g.:
foo_bar 109 GCAGC
bar_foo 58289 GCTGC
Not sure how complex this task is, and in particular whether there is a memory-efficient way to perform this operation in a streaming fashion. awk or sed seem like two utilities which might work, but the required command is beyond my limited understanding of the programs.
A tiny tweak on yesterdays answer:
sub(/^>/,"") {
hdr = $0
next
}
{
while ( match($0,/GC[AT]GC/) ) {
print hdr, RSTART, substr($0,RSTART,RLENGTH)
$0 = substr($0,1,RSTART-1) " " substr($0,RSTART+1)
}
}
Please get the book Effective AWK Programming, 5th Edition, by Arnold Robbins to learn the basics of awk.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 2 years ago.
Improve this question
I am trying to do the following in my Bigquery sql:
input: myword1 myword my 3433123 other stuff
output: myword1 myword my
input: myword 23498780000123 more stuff
output: myword
I want the output shown above based on the above input.
I need everything before the numbers.
I tried using ^([\s\w\s]+)(?=[^\d\r\n]+\d+[^\d\r\n]+$) but re2 doesn't like it.
Re2 doesn't like ?= . Hope that helps
It seems like you want everything before the first digit. If so, you can use regexp_replace():
regexp_replace(mycol, '\s*\d.*$', '')
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I have a very big text file (1 GB) and I see that there are few places where the http url field has a space there.
For example in the lines below we have space between "brad pitt" and "[30 wet=]". They should be changed to "bradpitt" and "[30wet=]" but they can occur in any url or trim_url. I am currently finding these places using my program and then manually fixing it vim. Is there a way using awk/sed we can do it?
0.0 q:hello url:http://sapient.com/bapper/30/brad pitt/C345/surf trim_url:http://sapient.com/bapper/30/brad pitt/C345 rating:good
0.0 q:hello url:http://sick.com/bright/[30 wet=]/sound trim_url:http://sick.com/bright/[30 wet=]rating:good
What I tried to do was sed:
sed -i -e 's/*http*[:space:]*/*http*/g' test.txt
Using perl and a proper module to URI encode the URL:
perl -MURI::Escape -pe 's!(https?://)(.*)!$1 . uri_escape($2)!e' file
You even can replace the file in place with -i switch (just like sed) perl -MURI::Escape -i -pe [...]
Output
0.0 q:hello url:http://sapient.com%2Fbapper%2F30%2Fbrad%20pitt%2FC345%2Fsurf%20trim_url%3Ahttp%3A%2F%2Fsapient.com%2Fbapper%2F30%2Fbrad%20pitt%2FC345%20rating%3Agood
0.0 q:hello url:http://sick.com%2Fbright%2F%5B30%20wet%3D%5D%2Fsound%20trim_url%3Ahttp%3A%2F%2Fsick.com%2Fbright%2F%5B30%20wet%3D%5Drating%3Agood
URI::Escape - Percent-encode and percent-decode unsafe characters
Note
As msanford said in comments, spaces in a URL are meaningful. You can't decide to cut them without breaking the link in something that just become not reachable
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I need to pull the first word from a variable in my batch script to another variable.
ex
if %hello% had "apples are awesome" in it and was pulled and put into %hi%
%hi% would say "apples"
thanks in Advance
This can be done using a for loop:
for /f %%h in ("%hello%") do [command that uses %%h]
The behaviour of "for" in this circumstance is to split its input up into lines (there is only one, assuming there are no newline characters in your input variable), then split each line into tokens on spaces (you can change the delimiter using the "delim=[chars]" option) and pass the first token of each line to the specified command (you can use "tokens=n,n,..." to get at tokens other than the first on the line).
Note that AIUI you can only use a single letter variable name for the variable to receive the word, so you can't use %%hi as you requested.
(This is all untested, as I'm not at a machine running Windows at the moment, but ought to work if I'm reading the documentation correctly.)
set "hi="
for %%h in (%hello%) do if not defined hi set "hi=%%h"
echo %hi%
should work, as would
set "hi="
for %%h in (%hello%) do set "hi=%%h"&goto done
:done
echo %hi%
Note that the set "var=string" syntax ensures that trailing spaces on the line, as left by some editors, are not included in the value assigned.
You don't say clearly whether the value of hello is apples are awesomeor "apples are awesome" - the first is a string of three words with space-separators, the second a single string containing one "word" which contains spaces. I've assumed the former.