I have a big csv file delimited with "|". How do I encrypt the 2nd column value into SHA256 using bash for full file.
Thanks in advance
for data in cat test.txt; do d1=$(echo $data | cut -d "|" -f1); d2=$(echo $data | cut -d "|" -f2 | sha256sum); rest=$(echo $data | cut -d "|" -f3-); echo "$d1|$d2|$rest";done >> new_file.txt
Related
When I try this command:
/usr/bin/curl -s sketch*.zip "https://www.sketch.com/downloads/mac/" |\
grep 'download.sketchapp.com/sketch-' | awk 'NR==1{print $3}'
The output is:
content="0;URL='https://download.sketchapp.com/sketch-68.2-102594.zip
what I am looking to get is:
68.2
Any help would be appreciated.
It seems you want to extract the number after your pattern, only for the first matcing row. You can use one grep command:
... | grep -oPm1 '(?<=download.sketchapp.com/sketch-)[^-]+' file
or as this is the 3rd field of your 1st curl output row you want, you can use one awk command (split field using hyphen as separator to array and print the element in the middle):
awk '/download.sketchapp.com/sketch-/ && NR==1 {split($3,a,"-"); print a[2]; exit}'
Using sed:
/usr/bin/curl -s sketch*.zip "https://www.sketch.com/downloads/mac/" | \
sed -n 's!.*download.sketchapp.com/sketch-\([^-]*\).*!\1!p;' | \
head -1
head is to get rid of multiple matches. sed command extracts non-hyphen characters after download.sketchapp.com/sketch-.
If I use cp inside a bash script the copied file will have weird charachters around the destination filename.
The destination name comes from the results of an operation, it's put inside a variable, and echoing the variable shows normal output.
The objective is to name a file after a string.
#!/bin/bash
newname=`cat outputfile | grep 'hostname ' | sed 's/hostname //g'
newecho=`echo $newname`
echo $newecho
cp outputfile "$newecho"
If I launch the script the echo looks ok
$ ./rename.sh
mo-swc-56001
However the file is named differently
~$ ls
'mo-swc-56001'$'\r'
As you can see the file contains extra charachters which the echo does not show.
Edit: the newline of the file is like this
# file outputfile
outputfile: ASCII text, with CRLF, CR line terminators
I tried in every possible way to get rid of the ^M charachter but this is an example of the hundreds of attempts
# cat outputfile | grep 'hostname ' | sed 's/hostname //g' | cat -v
mo-swc-56001^M
# cat outputfile | grep 'hostname ' | sed 's/hostname //g' | cat -v | sed 's/\r//g' | cat -v
mo-swc-56001^M
This newline will stay there. Any ideas?
Edit: crazy, the only way is to perform a dos2unix on the output...
Looks like your outputfile has \r characters in it, so you could add logic there to remove them and give it a try then.
#!/bin/bash
##remove control M first from outputfile by tr command.
tr -d '\r' < outputfile > temp && mv temp outputfile
newname=$(sed 's/hostname //g' outputfile)
newecho=`echo $newname`
echo $newecho
cp outputfile "$newecho"
The only way was to use dos2unix
I have a variable and that variable only needs a '\' in front of it.
I would say that the sed command is the ideal tool for it?
I tried using single quotes, double quotes, multiple variables, combination of variables, ...
I don't get an error returned but the end result is not showing what I need it do be
FOLDER=$(echo `cat file.XML | grep "Value" | cut -d \" -f2`)
echo $FOLDER
sed -i "s#"$FOLDER"#"\\$FOLDER"#g" ./file.XML
echo $FOLDER
After execution, I get
$ ./script.sh
b4c17422-1365-4fbe-bccd-04e0d7dbb295
b4c17422-1365-4fbe-bccd-04e0d7dbb295
Eventually I need to have a result like
$ ./script.sh
b4c17422-1365-4fbe-bccd-04e0d7dbb295
\b4c17422-1365-4fbe-bccd-04e0d7dbb295
Fixed thanks to the input of Cyrus and Ed Morton.
FOLDER=$(echo `cat file.XML | grep "Value" | cut -d \" -f2`)
NEW_FOLDER="\\$FOLDER"
sed -i "s#$FOLDER#\\$NEW_FOLDER#g" ./file.XML
I am trying to set the variable COGLINE to be the output of my grep line (which is searching my config.json file for the regExthe "cogs"). When I execute the grep line it correctly outputs the proper line number, but when I echo the variable it comes up blank.
COGLINE = $(grep -n \"cogs\" ~/Desktop/Repos/pronghorn/config.json | cut -f1 -d:)
all:
grep -n \"cogs\" ~/Desktop/Repos/pronghorn/config.json | cut -f1 -d:
echo $(COGLINE)
Here is the output:
GlennMBP:test glenn$ make all
grep -n \"cogs\" ~/Desktop/Repos/pronghorn/config.json | cut -f1 -d:
2
echo
You can see that the line number is properly found as "2", but the variable comes up blank as if it were not set. What am I doing wrong?
grep is not a make function. That COGLINE = line is a make assignment.
You either need to use
COGLINE := $(shell grep -n \"cogs\" ~/Desktop/Repos/pronghorn/config.json | cut -f1 -d:)
if you want that run at make parse time and want it in a make variable.
Or
all:
COGLINE=$$(grep -n \"cogs\" ~/Desktop/Repos/pronghorn/config.json | cut -f1 -d:); \
echo "$${COGLINE}"
to run it at all recipe execution time and have it in a shell variable.
There are middle grounds as well but those are the two basic ideas.
I have two .txt files, one with TAB as field delimiter and another with | as field delimiter.
I want to change the delimiter from TAB to CTRL-A and save as .txt file for the first file and for the second file change the delimiter from | to CTRL-A and save as .txt file.
These two files are separate files.
How can we do it using awk or sed?
For file one, try:
cat file1 | sed -e 's/\t/\x01/g' >file1.txt
For file two, try
cat file2 | sed -e 's/\|/\x01/g' >file2.txt
This is a great use for tr:
tr '\t' '\001' <file1 >file1-new
That will perform the transformation from horizontal tabs on file1 and output the results to file1-new. You can do the same thing with pipes.
An alternative using perl:
Replacing pipes:
echo "a|b|c" | perl -pe '$c=chr(1); s/\|/$c/g' | cat -A
a^Ab^Ac$
Replacing tabs:
echo -e "a\tb\tc" | perl -pe '$c=chr(1); s/\t/$c/g' | cat -A
a^Ab^Ac$