Using SED to match emails in a sql dump and replace them - sql
I am trying to scrub the emails from a SQL dump file, and I could use some advice. I am doing this because I want to send some developers "mostly correct" information, without sharing actual user information. I have a BASH script that loops through line-by-line, so I am trying to do a SED replace on the INSERT statements. I need to iterate through the dumb because I have some other scrubbing stuff, which is working. I have some regex that works (I think), but I cannot seem to get it into SED. The regex of:
'(.*#.*?)'
Will match 'emailname#emaildomain.com', but I'm having trouble getting it into SED, and I'm sure that there is a better REGEX. Here's my example line.
'firstname','emailname#emaildomain.com','lastname'
I hope to be able to replace whenever I have an # between quotes with 'empty#invalid'. Any advice would be greatly appreciated.
Try:
sed "s/'[^#']*#[^#']*'/'empty#invalid'/g"
I've replaced your .* with the more specific [*#']*, which only matches strings which don't have any single-quotes ' or #, which is needed because sed is greedy.
Related
Multi-line text in a .env file
In vue, is there a way to have a value span multiple lines in an .env file. Ex:
Instead of:
someValue=[{"someValue":"Here is a really really long piece which should be split into multiple lines"}]
I want to do something like:
someValue=`[{"someValue":"Here is a really
really long piece which
should be split into multiple lines"}]`
Doing the latter gives me a JSON parsing error if I try to do JSON.parse(someValue) in my code
I don't know if this will work, but I can't format a comment appropriately enough to get the point across so see if this will work:
someValue=[{"someValue":"Here is a really\
really long piece which\
should be split into multiple lines"}]
Where "\" should escape the newline similar to how you can write long bash commands while escaping the newline. I'm not certain the .env interpreter will support it though.
EDIT
Looks like this won't work. This syntax was actually proposed, but I don't think it was incorporated. See motdotla/dotenv#333 (which is what Vue uses to parse .env).
Like #zero298 said, this isn't possible. Likely you could delimit the entry with a character that wouldn't show up normally in the text (^ is a good candidate), then parse it within the application using string.replace('^', '\n');
Sed script needed to insert LF before each time match in a large single string
I have lengthy string that I need to put a line feed before each instance of a time stamp. 03:38:11,03/07/2017,node,cpu,user,sys,idle,intr/s,ctxt/s,0,0,0,9,91,0,1,0,24,75,0,total,0,17,83,2370,3574,1,0,3,4, 93,1,1,10,4,86,1,total,7,4,89,2922,4653,03:39:11,03/07/2017,node,cpu,user,sys,idle,intr/s,ctxt/s,0,0,4,25,71,0,1,5 ,16,79,0,total,4,21,75,2487,3876,1,0,0,3,97,1,1,1,1,98,1,total,1,2,98,2880,4728,03:40:11,03/07/2017,node,cpu,user, sys,idle,intr/s,ctxt/s,0,0,1,30,69,0,1,1,30,69,0,total,1,30,69,3237,4344,1,0,3,49,47,1,1,10,47,43,1,total,6,48,45, 3920,5702, I need to see about formatting it as such: 03:38:11,03/07/2017,node,cpu,user,sys,idle,intr/s,ctxt/s,0,0,0,9,91,0,1,0,24,75,0,total,0,17,83,2370,3574,1,0,3,4,93,1,1,10,4,86,1,total,7,4,89,2922,4653, 03:39:11,03/07/2017,node,cpu,user,sys,idle,intr/s,ctxt/s,0,0,4,25,71,0,1,5,16,79,0,total,4,21,75,2487,3876,1,0,0,3,97,1,1,1,1,98,1,total,1,2,98,2880,4728, 03:40:11,03/07/2017,node,cpu,user,sys,idle,intr/s,ctxt/s,0,0,1,30,69,0,1,1,30,69,0,total,1,30,69,3237,4344,1,0,3,49,47,1,1,10,47,43,1,total,6,48,45,3920,5702, I am currently trying to use the following: sed -e 's/^[[:digit:]][[:digit:]]\:[[:digit:]][[:digit:]]/\n&/g' cpu.log
The ^ line anchor forces sed to only match the first date stamp. Remove it and you should be fine. To avoid roplacing the first, maybe massage the script to require something before the match (hard-coding a comma would seem to work, based on your sample data); or just post-process the output to remove the first newline.
sed 's/[0-9][0-9]:[0-9][0-9]:[0-9][0-9]/\n&/g'
Escape all commas in line except first and last
I have a CSV file which I'm trying to import to a SQL Server table. The file contains lines of 3 columns each, separated by a comma. The only problem is that some of the data in the second column contains an arbitrary number of commas. For example: 1281,I enjoy hunting, fishing, and boating,smith317 I would like to escape all occurrences of commas in each line except the first and the last, such that the result of this line would be: 1281,I enjoy hunting\, fishing\, and boating,smith317 I know I will need some type of regular expression to accomplish this task, but my knowledge of regular expressions is very limited. Currently, I'm trying to use Notepad++ find/replace with regex, but I am open to other ideas. Any help would be greatly appreciated :-)
Okay, could be a manual stuff. Do this: Normal find all the , and replace it with \,. Escape everything. Regex find ^(.*)(\\,) and replace it with $1,. Regex find (\\,)(.*)$ and replace it with ,$2. Worked for me in Sublime Text 2.
sed replacing without untouching a string
Im trying to replace all lines within files that contains: /var/www/webxyz/html to /home/webxyz/public_html the string: webxyz is variable: like web1, web232 So only the string before and after webxyz should be replaced. Tried this without solution: sed -i 's/"var/www/web*/html"/"home/web*/public_html"/g' Also i want this should check and replace files (inclusive subdirectory and files), the * operator don't work.
Within a regular expression, you’ll need to escape the delimiting character that surround them, in your case the /. But you can also use a different delimiter like ,: sed -i 's,"var/www/web*/html","home/web*/public_html",g' But to get it working as intended, you’ll also need to remove the " and replace the b* (sed doesn’t understand globbing wildcards) to something like this: sed -i 's,var/www/web\([^/]*\)/html,home/web\1/public_html,g' Here \([^/]*\) is used to match anything after web except a /. The matching string is then referenced by \1 in the replacement part.
Here is what your replacement operation should look like (see sed for more info): sed -i 's/var\/www\(\/.*\/\)html/home\1public_html/g' Note that \(...\) is a grouping, and specifies a "match variable" which shows up in the replacement side as \1. Note also the .* which says "match any single character (dot) zero or more times (star)". Note further that your / characters must be escaped so that they are not treated as part of the sed control structure.
Is there a tool to clean the output of the script(1) tool?
script(1) is a tool for keeping a record of an interactive terminal session; by default it writes to the file transcript. My problem is that I use ksh93, which has readline features, and so the transcript is mucked up with all sorts of terminal escape sequences and it can be very difficult to reconstruct the command that was actually executed. Not to mention the stray ^M's and the like. I'm looking for a tool that will read a transcript file written by script, remove all the junk, and reconstruct what the shell thought it was executing, so I have something that shows $PS1 and the commands actually executed. Failing that, I'm looking for suggestions on how to write such a tool, ideally using knowledge from the terminfo database, or failing that, just using ANSI escape sequences. A cheat that looks in shell history, as long as it really really works, would also be acceptable.
Doesn't cat/more work by default for browsing the transcript? Do you intend to create a script out of the commands actually executed (which in my experience can be dangerous)?
Anyway, 3 years without an answer, so I will give it a shot with an incomplete solution. If your are only interested in the commands actually typed, remove the non-printable characters, then replace PS1' with something readable and unique, and grep for that unique string. Like this:
$ sed -i 's/[^[:print:]]//g' transcript
$ sed 's/]0;cartman#southpark: ~cartman#southpark:~/CARTMAN/g' transcript | grep CARTMAN
Explanation: After first sed, PS1' can be taken from one of the first few lines of the transcript file, as is -- PS1' is different from PS1 -- and can be modified with a unique readable string ("CARTMAN" here). Note that the dollar sign at the end of the prompt was left out intentionally.
In the few examples that I tried, this didn't solve everything but took care of most issues.
This is essentially the same question asked recently in Can I programmatically “burn in” ANSI control codes to a file using unix utils? -- removing all nonprinting characters will not fix embedded escape sequences backspace/overstriking for underlining use of carriage-returns for overstriking