I'm looking to search some files via SSH with the grep command but I have some special chars.
The string I'm looking for is:
"$GLOBALS['....'];"
I tried this one
grep -r -H "\$GLOBALS\\['*'\\]\;" /var/www/
but nothing happens. Any help will be welcome.
Your RE actually matches "$GLOBALS['''''''];" with one or more ' there.
try this one:
grep -rHP "[$]GLOBALS\['.*?']\;" file
I use [$] instead of \$, is because ESCAPE IS SOMEHOW TRICKY, some environment you need use \\\$.
Update, less than 10 chars inside the []:
grep -rHP "[$]GLOBALS\['.{0,10}']\;" file
Related
I need to loop through a directory the first time I log into a new VM and set permissions on WSL shared folders. The SSH agent will not add private keys unless they have 0700 permissions set.
How do I use grep to get a list of file names and then loop through them?
I tried:
for file $(grep -lr "-----BEGIN OPENSSH PRIVATE KEY-----")
do
echo "Working on File: $file"
chmod 700 $file
done
The results were a mess. The grep command is being resolved into a single string of file names. Any mapped directories that have spaces in them (from the Win11 side of the house) messed things up further.
Suggesting to accomplish the task in one line:
chmod -v 700 $(grep -lr "-----BEGIN OPENSSH PRIVATE KEY-----" $PWD)
Explanation:
-v : Print verbose report on each changed file.
grep -lr "-----BEGIN OPENSSH PRIVATE KEY-----" $PWD : List all matched files to be changed, from all files under current directory.
Note: also try to replace $PWD with $HOME.
Here is my solution:
grep will output filenames with a null character after each item with the "-Z" flag.
Zsh can take that null delimiter and use it in an expanded string with the "0" flag.
The final result:
for file in ${(0o)"$(grep -lrZ "-----BEGIN OPENSSH PRIVATE KEY-----" *)"}
do
echo "Working on $file"
chmod 700 $file
done
A detailed explanation for posterity:
Zsh will execute anything inside () in a subshell. so the results of grep are in a nameless variable $().
Zsh does variable/parameter expansion with ${}. Anything inside gets expanded like hitting "Tab" at the command prompt.
We know that grep is going to return file paths with spaces in them. Therefore we can't use whitespace to expand our file list. We can use flags to make it happen with grep and Zsh using null characters as delimiters instead of spaces.
Zsh flags are specified immediately after the curly braces are opened up.
${(0o)$foo} will result in $foo being expanded into an array of items, splitting $foo on nulls because of the "0" and sorting them in ascending order because of the "o" flag.
We need the extra step of enclosing our grep results in double-quotes. ${(0o)"$foo"} We are after a string that contains both spaces and nulls. We are going to split the string on the nulls and preserve the spaces in the path names.
You could sort them in descending order with a capitalized "O" if you wanted to do it that way instead.
grep -lrZ will look for the string "-----BEGIN OPENSSH PRIVATE KEY-----" in all files found with the * wildcard.
The "r" is recursively look in sub-directories
The -l outputs just the filename
And -Z outputs a null separator at the end of each result.
So you end up with grep finding the files, Zsh expands the list and then loops through each of the items in the returned array.
As a side note in WSL, you have to turn on metadata in order to mount a windows folder and set Linux permissions. It is a nice solution to keep your ssh keys in a secure, backed-up location and use them on Linux images.
Iterate over all the files, but only operate on the ones that grep succeeds on. (You probably don't have that many files; don't worry so much over how many times grep has to run.)
for f in *; do
if grep -q "-----BEGIN OPENSSH PRIVATE KEY-----" "$f"; then
echo "Working on File: $f"
chmod 700 $f
fi
done
And because this is zsh, you can embed the call to grep in the glob:
is_key () {
grep -q "-----BEGIN OPENSSH PRIVATE KEY-----" "$REPLY"
}
for f in *(+is_key); do
echo "Working on File: $f"
chmod 700 "$f"
done
or even dispense with the explicit loop
chmod 700 *(+is_key)
(The shell function isn't actually necessary, but it does help with the readability of the glob qualifier.)
If I want to search an LDAP entry from a LDAP server using ForgeRock Directory Server(OpenDJ), the document says in Linux it works like the following.
$ ldapsearch \
--hostname localhost \
--port 1389 \
--baseDn dc=example,dc=com \
"(cn=Abagail Abadines)" \
cn mail street l
If I want to run this command in Windows, it doesn't seem to work.
What do I need to change in this command to run it on Windows's cmd.
It works the same, you just need to use ldapsearch.bat, and use the Windows continuation character (^) instead of \, or put all arguments on a single line.
Also, when reading the documentation, the upper right corner icon allows you to copy the example command to your clipboard.
I figured it out after two hours of trying! I hope this might help someone. In Linux, \ means new line (of course, it also means escape sequence). It created new line. So in Window's cmd mode, just type the whole command without \
Also, even in Linux, typing the whole command without \ seem to work fine without any error. (By putting \ every line somehow didn't work in the middle. )
In Unix/Linux, Windows, and Mac, '/' (slash) and '\' ( backslash ) are confusing a little. So when executing commands or setting paths, gotta watch out.
I'm trying to use the --immediate-submit on a PBSPro cluster. I tried using an in-place modification of the dependencies string to adapt it to PBSPro, similar to what is done here.
snakemake --cluster "qsub -l wd -l mem={cluster.mem}GB -l ncpus={threads} -e {cluster.stderr} -q {cluster.queue} -l walltime={cluster.walltime} -o {cluster.stdout} -S /bin/bash -W $(echo '{dependencies}' | sed 's/^/depend=afterok:/g' | sed 's/ /:/g')"
This last part gets converted into, for example:
-W depend=afterok: /g/data1a/va1/dk0741/analysis/2018-03-25_marmo_test/.snakemake/tmp.cyrhf51c/snakejob.trimmomatic_pe.7.sh
There are two problems here:
How can I get the dependencies string to output job ID instead of the script path? The qsub command normally outputs the job ID to stdout, so I'm not sure why it's not doing so here.
How do I get rid of the space after afterok:? I've tried everything!
As an aside, it would be helpful if there were some option to debug the submission or not to delete the tmp.cyrhf51c directory in .snakemake -- is there some way to do this?
Thanks,
David
I suggest to use a profile for this, instead of trying to find an ad-hoc solution. This will also help with debugging. E.g., there is already a pbs-torque profile available (https://github.com/Snakemake-Profiles/pbs-torque), probably there is not much to change towards pbspro?
i have a 6gb httpd log file and i want to remove lines beginging in
66.249 (ip block of googlebot) i did have a
SetEnvIf Remote_Addr "66\.249\.\." dontlog
entry in my httpd.conf file but it didnt seem to work
so is there a linux command like
grep -removelines-starting "66.49" acessslog
Using sed: Use -i flag if you make changes in file directly.
sed '/^66\.49/d' logfile
Using grep: This will print lines apart from lines starting with 66.49
grep -v '^66\.49' logfile
Using awk:This will print lines apart from lines starting with 66.49
awk '!/^66\.49/' logfile
I can imagine sed is a better fit for this task.
sed -i '/66\.249/d' ./acessslog
"d" is for deleting matched pattern, while -i is for overwriting input file.
I am trying to create a patch that users can use to remotely edit a file in a pre-defined way using sed, and I could do this manually on each computer, but it would take a long time.
The line I am struggling with is as follows:
host=[hostname]
port=[portnum]
ssh -t $host -p $port "cp ~/file1 ~/file1.bak ; sed -i \"s/fcn1('param1', $2)\n/fcn2('param2'):$zoom\n/g\" ~/file1"
This makes a backup of file1 and then edits a line in the file. I actually want to edit more than one line, but this line demonstrates the problems:
The command works, provided no $ signs are used within the sed command.
I have tried a number of ways of escaping these $ signs but cannot seem to find one that works.
I can use a . wildcard in the find, but obviously not in the replace string.
I would use single quotes for the sed command, in order to avoid expanding the $2, but single quotes are already used inside the command.
Does anyone have any ideas of how to overcome this problem? Thanks in advance for any suggestions!
This should work as well:
ssh -t $host -p $port "cp ~/file1 ~/file1.bak && sed -i \"s/fcn1('param1', \\\$2)/fcn2('param2'):\\\$zoom/g\" file1"
You need 3 back slashes as you have to escape the $ sign in the string passed in the remote bash to sed. And you have to escape that back slash and the $ sign when sending it over via ssh.