I used command before and it worked fine, but now there is some error:
grep -oha "domain\.com/[^\"]*" "/Users/test5/folder5” -r | grep -v word1 | grep -v word2 | awk '!a[$0]++' > results
In the folder5 there are many files in which I need to find "domain.com" string and then filter duplicates and then write results to file.
But I have error:
zsh: event not found: a[
What is wrong with my command?
Thank you!
The answer given by #boppy in a comment is correct:
After the /Users path there is a "beautified" quote ” - that might be the problem...
Related
I am trying to use awk to print the unique lines returned by a command. For simplicity, assume the command is ls -alh.
If I run the following command in my Z shell, awk shows all lines printed by ls -alh
ls -alh | awk '!seen[$0]++'
However, if I run the same command with $SHELL -c while escaping the ! with backslash, I only see the first line of the output printed.
$SHELL -c "ls -alh | awk '\!seen[$0]++'"
How can I ensure the latter command prints the exact same outputs as the former?
EDIT 1:
I initially thought the ! could be the issue. But changing the expression '!seen[$0]++' to 'seen[$0]++==0' has the same problem.
EDIT 2:
It looks like I should have escaped $ too. Since I do not know the reason behind it, I will not post an answer.
In the second form, $0 is being treated as a shell variable in the double-quoted string. The substitution creates an interestingly mangled awk command:
> print $SHELL -c "ls -alh | awk '\!seen[$0]++'"
/bin/zsh -c ls -alh | awk '!seen[-zsh]++'
The variable is not substituted in the first form since it is inside single quotes.
This answer discusses how single- and double-quoted strings are treated in bash and zsh:
Difference between single and double quotes in Bash
Escaping the $ so that $0 is passed to awk should work, but note that quoting in commands that are parsed multiple times can get really tricky.
> print $SHELL -c "ls -alh | awk '\!seen[\$0]++'"
/bin/zsh -c ls -alh | awk '!seen[$0]++'
Problem Set (Raspberry Pi OS):
I have a file example.conf that contains a line IPv4addr=XXXXX. I am attempting to change this to the IP that is generated the in the command
ipTest=$(ip --brief a show | grep eth0 | awk '{ print $3 }')
I want to automate this file change during a script install.sh, the line I am attempting is:
IPtest=$(ip --brief a show | grep eth0 | awk '{ print $3 }')
sudo sed -e "/IPv4addr/s/[^=]*$/$IPtest/" example.conf
Returns error:
sed: -e expression #1, char 32: unknown option to `s'
A simple line in that code works, such as SimpleTest='Works'
Any thoughts? I am open to other solutions as well, however I am not an experienced linux user so I am using the tools I know to work with other problem sets.
$IPtest contains the / character; try something like this:
IPtest=$(ip --brief a show | grep eth0 | awk '{ print $3 }')
sudo sed -e '/IPv4addr/s#[^=]*$#'"$IPtest"'#' example.conf
You can shorten your variable and allow awk to do the job of grep at the same time
IPtest=$(ip --brief a s | awk '/eth0/{print $3}')
Using sed grouping and back referencing
sed -i.bak "s|\([^=]*.\).*|\1$IPtest|" example.conf
I am trying to read some files located on a server and write only a certain number of columns from those files onto my local machine. I am tried to do this in a for loop to avoid inputting my password for each file. Below is what I was able to cobble till now.
The following code works but writes all the output to a single file which is not manageable due to its large size.
ssh user#xx.xxx.xxx.xx 'for loc in /hel/insur/*/201701*; do zcat $loc | grep -v NUMBER | awk -F',' -v OFS="," '\''{print $1,$2,$3,$4,$5}'\'' | gzip; done' > /cygdrive/c/Users/user1/Desktop/test/singlefile.csv.gz
So, I tried to write each file individually as shown below but it gives me an error saying that it cannot find the location(possibly because I am sshed ito the remote server).
ssh user#xx.xxx.xxx.xx 'for loc in /hel/insur/*/201701*; do zcat $loc | grep -v NUMBER | awk -F',' -v OFS="," '\''{print $1,$2,$3,$4,$5}'\'' | gzip > /cygdrive/c/Users/user1/Desktop/test/`echo $loc | cut -c84-112` ; done'
Any ideas on how to solve this?
i am writting a script and i need to get a number out of the shell command output. The command & its return is
$ git branch -a -v --no-abbrev --contains $(git rev-parse HEAD)
* (HEAD detached at c5246b6) c5246b6907e46795741853852462914e7a5f60de Merge pull request 1166 from testPR into dev
remotes/origin-pull/1166/merge c5246b6907e46795741853852462914e7a5f60de Merge pull request 1166 from testPR into dev
i am trying to extract the 1166 out of the result by using sed over the piped result. Something like
$ git branch -a -v --no-abbrev --contains $(git rev-parse HEAD) | sed <pattern>
to get the 1166
My patterns so far doesn't seem to get the number i am expecting.
I seems that you're trying to extract the part of your remote branch name between last 2 slashes. And you may use grep with perl interpreted pattern to achieve that, here you are,
$ git branch ... | grep -oP '[^\/]+(?=\/[^\/]+$)'
1166
Brief explanation,
-o: Print only the matched (non-empty) parts
[^\/]+ : grep command would print this part, non-slash pattern
(?=\/[^\/]+$) : matches words ahead of the las slash of the line [^\/]+$
Not the answer to my exact question, but i am able to get what i want by modifying my bash command.
git branch -r -v --no-abbrev --contains $(git rev-parse HEAD) | awk '{print $1}'
This returns: origin-pull/1166/merge , which is what i want
notice the -r in the command, -a will return both local and remote git branch info. This way, i can cheat on the sed pattern again.
I have this find awk line to get python code analyse::
$ find ./ -name '*.py' -exec wc -l {} \; | sort -n| awk '{print $0}{s+=$0}END{print s}'
12 ./gb/__init__.py
23 ./gb/value_type.py
40 ./setup.py
120 ./gb/libcsv.py
200
$
I try to put it in a Makefile::
$ cat Makefile
python_count_lines: clean
#find ./ -name '*.py' -exec wc -l {} \; | sort -n| awk '{print \$0}{s+=\$0}END{print s}'
But this did not work::
$ make python_count_lines
awk: line 1: syntax error at or near }
Makefile:12: recipe for target 'python_count_lines' failed
make: *** [python_count_lines] Error 2
$
Bertrand Martel is correct that you need to escape dollar signs from make by doubling them, not prefixing them with backslashes (see info here).
However, the rest of that suggestion is not right and won't work; first, you should almost never use the shell function in a recipe. Second, using the info function here cannot work because in the first line you've set a shell variable RES equal to some value, then you try to print the make variable RES in the second line; not only that but each line is run in a separate shell, and also all make variable and function references are expanded up-front, before any part of the recipe is passed to the shell.
You just need to do this:
python_count_lines: clean
#find ./ -name '*.py' -exec wc -l {} \; | sort -n| awk '{print $$0}{s+=$$0}END{print s}'