I use procmail to forward certain 'From' to a Gmail account
/home/user/.procmailrc
:0c
* !^FROM_MAILER
* ^From: .*aaa | bbb | ccc.*
! ^X-Loop: user#gmail\.com
| formail -k -X "From:" -X "Subject:" \
-I "To: user#gmail.com" \
-I "X-Loop: user#gmail.com"
:0
* ^From: .*aaa | bbb | ccc.*
$DEFAULT
This works fine but on my server inbox I also get an 'undelivered' mail
The mail system <"^X-Loop:"#my-name-server.com> (expanded from
<"^X-Loop:">): unknown user:
"^x-loop:"
How can I avoid this?
I've tried to delete these mails.
This is not the best way.
Anyway It does not work.
:0B * <"\^X-Loop:"#my-name-server.com>
/dev/null
The recipe contains multiple syntax errors, but the bounce message comes because you lack an asterisk on one of the condition lines, which makes it an action line instead.
The general syntax of a Procmail recipe is
:0flags # "prelude", with optional flags
* condition # optional, can have zero conditions
* condition # ...
action
The action can be a mailbox name, or ! followed by a destination mailbox to forward the message to, or | followed by a shell pipeline.
So your first recipe is "If not from mailer and matching From: ..., forward to ^X-Loop:.
The | formail ... line after that is then simply a syntax error and ignored, because it needs to come after a prelude line :0 and (optionally) some condition lines.
Additionally, the ^From: regex is clearly wrong. It will match From: .*aaa or bbb (with spaces on both sides, in any header, not just the From: header) or ccc.
Finally, the intent is apparently to actually forward the resulting message somewhere.
:0c
* ! ^FROM_MAILER
* ^From:(.*\<)?(aaa|bbb|ccc)
* ! ^X-Loop: user#gmail\.com
| formail -I "X-Loop: user#gmail.com" | $SENDMAIL $SENDMAILFLAGS user#gmail.com
If you simply want to forward the incoming message, the other -X and -I and certainly -k options are superfluous or wrong. If they do accomplish something which is irrelevant for this question, maybe you need to add some or all of them back (and also remember to extract with -X any new headers you add with -I, as otherwise they will be suppressed; this sucks).
Your second recipe is also superfluous, unless you have more Procmail recipes later in the file which should specifically be bypassed for these messages. (If so, you will need to fix the From: regex there as well.)
Related
I'm using screen to monitor several parallel jobs to test small variations of my program. I gave each screen session a different logfile. I do not remember which logfile I set for which session, and now wish I did!
Is there a way to query which session name (usually of the form #####.ttys000N.hostname) goes with which logfile, or vice-versa?
(To whom it concerns: the gnu-screen tag suggests determining which SX site the question is most relevant to. Based on the help pages of SuperUser and StackOverflow, this question appears roughly equally applicable to either community. Feel free to migrate it if you think it belongs elsewhere.)
I didn't find my suggested comment of using screen -ls to list the process ids, and then doing an lsof -p on these to find the filenames very satisfactory, so here is another not entirely satisfactory alternative:
There is an option -X to send commands to a remote screen, but unfortunately any output is shown on the remote. There is an option -Q to send a command and print the result locally, but it only accepts a very limited set of commands. However, one of these is lastmsg, which repeats the last message displayed.
So you can use -X logfile to display the name of the logfile remotely, then immediately use -Q lastmsg to duplicate that display locally! There is, of course, the possibility of some event occurring in the middle of this non-atomic action. The two commands cannot be combined. Here's an example:
#!/bin/bash
screen -ls |
while read session rest
do if [[ "$session" =~ [0-9]+\..+ ]]
then screen -S "$session" -X logfile # shows in status
msg=$(screen -S "$session" -Q lastmsg)
# logfile is '/tmp/xxxxx'
echo "$session $msg"
fi
done
and some typical output:
21017.test2 logfile is '/tmp/xxxxx'
20166.test logfile is '/tmp/mylog.%n'
In modsecurity default-script:
base_rules/modsecurity_crs_20_protocol_violations.conf
there is a rule, 960011:
SecRule REQUEST_METHOD "^(?:GET|HEAD)$" \
"msg:'GET or HEAD Request with Body Content.',\
severity:'2',\
id:'960011',\
ver:'OWASP_CRS/2.2.9',\
rev:'1',\
maturity:'9',\
accuracy:'9',\
phase:1,\
block,\
logdata:'%{matched_var}',\
t:none,\
tag:'OWASP_CRS/PROTOCOL_VIOLATION/INVALID_HREQ',\
tag:'CAPEC-272',\
chain"
SecRule REQUEST_HEADERS:Content-Length "!^0?$"\
"t:none,\
setvar:'tx.msg=%{rule.msg}',\
setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},\
setvar:'tx.%{rule.id}-OWASP_CRS/PROTOCOL_VIOLATION/INVALID_HREQ-%{matched_var_name}=%{matched_var}'"
I only want to disable logging for this rule (it gives too many false positives),
and therefore add my own script
base_rules/z99_logging_suppress.conf
to remove the default-rule and create a new identical rule -- only without logging:
SecRuleRemoveById 960011
SecRule REQUEST_METHOD "^(?:GET|HEAD)$" \
"msg:'GET or HEAD Request with Body Content.',\
severity:'2',\
id:'9960011',\
ver:'OWASP_CRS/2.2.9',\
rev:'1',\
maturity:'9',\
accuracy:'9',\
phase:1,\
block,nolog,\
logdata:'%{matched_var}',\
t:none,\
tag:'OWASP_CRS/PROTOCOL_VIOLATION/INVALID_HREQ',\
tag:'CAPEC-272',\
chain"
SecRule REQUEST_HEADERS:Content-Length "!^0?$"\
"t:none,\
setvar:'tx.msg=%{rule.msg}',\
setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},\
setvar:'tx.%{rule.id}-OWASP_CRS/PROTOCOL_VIOLATION/INVALID_HREQ-%{matched_var_name}=%{matched_var}'"
The only differences to the original rule are the new id 9960011, and the nolog additions:
...
id:'9960011',\
...
block,nolog,\
...
But when I restart httpd with this additional rule, I get error:
AH00526: Syntax error on line 18 of /path/base_rules/z99_logging_suppress.conf:
ModSecurity: Execution phases can only be specified by chain starter rules.
The same strategy --- SecRuleRemoveById + then re-create it with new id --- works for all other default-rules I tried, but not for this one.
Anyone can tell me why that is?
It basically says that the phase command can only be in the first rule in a chain and not in a subsequent rule which forms part of the chain.
There is nothing wrong with the rule as you have written it, phase is only specified in the first SecRule. In fact I've tried it on my instance and it works. So either one of two things has gone wrong:
You have copied and pasted it incorrectly into this question.
The rule above where you have defined this, has chain in it and so has left an open chain, that your rule 9960011 is then effectively trying to continue on from.
Or something else weird is happening! But I'm going with 1 or 2 for now :-)
In the repository home page , i can see comments posted in recent activity at the bottom, bit it only shows 10 commnets.
i want to all the comments posted since beginning.
Is there any way
Comments of pull requests, issues and commits can be retrieved using bitbucket’s REST API.
However it seems that there is no way to list all of them at one place, so the only way to get them would be to query the API for each PR, issue or commit of the repository.
Note that this takes a long time, since bitbucket has seemingly set a limit to the number of accesses via API to repository data: I got Rate limit for this resource has been exceeded errors after retrieving around a thousand results, then I could retrieve about only one entry per second elapsed from the time of the last rate limit error.
Finding the API URL to the repository
The first step is to find the URL to the repo. For private repositories, it is necessary to get authenticated by providing username and password (using curl’s -u switch). The URL is of the form:
https://api.bitbucket.org/2.0/repositories/{repoOwnerName}/{repoName}
Running git remote -v from the local git repository should provide the missing values. Check the forged URL (below referred to as $url) by verifying that repository information is correctly retrieved as JSON data from it: curl -u username $url.
Fetching comments of commits
Comments of a commit can be accessed at $url/commit/{commitHash}/comments.
The resulting JSON data can be processed by a script. Beware that the results are paginated.
Below I simply extract the number of comments per commit. It is indicated by the value of the member size of the retrieved JSON object; I also request a partial response by adding the GET parameter fields=size.
My script getNComments.sh:
#!/bin/sh
pw=$1
id=$2
json=$(curl -s -u username:"$pw" \
https://api.bitbucket.org/2.0/repositories/{repoOwnerName}/{repoName}/commit/$id/comments'?fields=size')
printf '%s' "$json" | grep -q '"type": "error"' \
&& printf "ERROR $id\n" && exit 0
nComments=$(printf '%s' "$json" | grep -o '"size": [0-9]*' | cut -d' ' -f2)
: ${nComments:=EMPTY}
checkNumeric=$(printf '%s' "$nComments" | tr -dc 0-9)
[ "$nComments" != "$checkNumeric" ] \
&& printf >&2 "!ERROR! $id:\n%s\n" "$json" && exit 1
printf "$nComments $id\n"
To use it, taking into account the possibility for the error mentioned above:
A) Prepare input data. From the local repository, generate the list of commits as wanted (run git fetch -a prior to update the local git repo if needed); check out git help rev-list for how it can be customised.
git rev-list --all | sort > sorted-all.id
cp sorted-all.id remaining.id
B) Run the script. Note that the password is passed here as a parameter – so first assign it to a variable safely using stty -echo; IFS= read -r passwd; stty echo, in one line; also see security considerations below. The processing is parallelised onto 15 processes here, using the option -P.
< remaining.id xargs -P 15 -L 1 ./getNComments.sh "$passwd" > commits.temp
C) When the rate limit is reached, that is when getNComments.sh prints !ERROR!, then kill the above command (Ctrl-C), and execute these below to update the input and output files. Wait a while for the request limit to increase, then re-execute the above one command and repeat until all the data is processed (that is when wc -l remaining.id returns 0).
cat commits.temp >> commits.result
cut -d' ' -f2 commits.result | sort | comm -13 - sorted-all.id > remaining.id
D) Finally, you can get the commits which received comments with:
grep '^[1-9]' commits.result
Fetching comments of pull requests and issues
The procedure is the same as for fetching commits’ comments, but for the following two adjustments:
Edit the script to replace in the URL commit by pullrequests or by issues, as appropriate;
Let $n be the number of issues/PRs to search. The git rev-list command above becomes: seq 1 $n > sorted-all.id
The total number of PRs in the repository can be obtained with:
curl -su username $url/pullrequests'?state=&fields=size'
and, if the issue tracker is set up, the number of issues with:
curl -su username $url/issues'?fields=size'
Hopefully, the repository has few enough PRs and issues so that all data can be fetched in one go.
Viewing comments
They can be viewed normally via the web interface on their commit/PR/issue page at:
https://bitbucket.org/{repoOwnerName}/{repoName}/commits/{commitHash}
https://bitbucket.org/{repoOwnerName}/{repoName}/pull-requests/{prId}
https://bitbucket.org/{repoOwnerName}/{repoName}/issues/{issueId}
For example, to open all PRs with comments in firefox:
awk '/^[1-9]/{print "https://bitbucket.org/{repoOwnerName}/{repoName}/pull-requests/"$2}' PRs.result | xargs firefox
Security considerations
Arguments passed on the command line are visible to all users of the system, via ps ax (or /proc/$PID/cmdline). Therefore the bitbucket password will be exposed, which could be a concern if the system is shared by multiple users.
There are three commands getting the password from the command line: xargs, the script, and curl.
It appears that curl tries to hide the password by overwriting its memory, but it is not guaranteed to work, and even if it does, it leaves it visible for a (very short) time after the process starts. On my system, the parameters to curl are not hidden.
A better option could be to pass the sensitive information through environment variables. They should be visible only to the current user and root via ps axe (or /proc/$PID/environ); although it seems that there are systems that let all users access this information (do a ls -l /proc/*/environ to check the environment files’ permissions).
In the script simply replace the lines pw=$1 id=$2 with id=$1, then pass pw="$passwd" before xargs in the command line invocation. It will make the environment variable pw visible to xargs and all of its descendent processes, that is the script and its children (curl, grep, cut, etc), which may or may not read the variable. curl does not read the password from the environment, but if its password hiding trick mentioned above works then it might be good enough.
There are ways to avoid passing the password to curl via the command line, notably via standard input using the option -K -. In the script, replace curl -s -u username:"$pw" with printf -- '-s\n-u "%s"\n' "$authinfo" | curl -K - and define the variable authinfo to contain the data in the format username:password. Note that this method needs printf to be a shell built-in to be safe (check with type printf), otherwise the password will show up in its process arguments. If it is not a built-in, try with print or echo instead.
A simple alternative to an environment variable that will not appear in ps output in any case is via a file. Create a file with read/write permissions restricted to the current user (chmod 600), and edit it so that it contains username:password as its first line. In the script, replace pw=$1 with IFS= read -r authinfo < "$1", and edit it to use curl’s -K option as in the paragraph above. In the command line invocation replace $passwd with the filename.
The file approach has the drawback that the password will be written to disk (note that files in /proc are not on the disk). If this too is undesirable, it is possible to pass a named pipe instead of a regular file:
mkfifo pipe
chmod 600 pipe
# make sure printf is a builtin, or use an equivalent instead
(while :; do printf -- '%s\n' "username:$passwd"; done) > pipe&
pid=$!
exec 3<pipe
Then invoke the script passing pipe instead of the file. Finally, to clean up do:
kill $pid
exec 3<&-
This will ensure the authentication info is passed directly from the shell to the script (through the kernel), is not written to disk and is not exposed to other users via ps.
You can go to Commits and see the top line for each commit, you will need to click on each one to see further information.
If I find a way to see all without drilling into each commit, I will update this answer.
I am currently working with LDAP which is on a UNIX server. The main purpose for me right now is to modify two entries in the directory by Ldapmodify through command line. The command is as follows:
ldapmodify -a -D 'cn=ldap,dc=cs,dc=ttu,dc=edu' \
-w *password* -H *server address* -f Documents/user.ldif
user.ldif file is as follows:
dn: cn=*username*,cn=Users,dc=ttu,dc=edu
changetype: modify
replace: displayName
displayName: abc
-
replace: loginShell
loginShell: /bin/tc
-
I get the following error when i try to run it:
ldapmodify: invalid format (line 5) entry: "cn=*username*,cn=Users,dc=ttu,dc=edu"
I checked the "dn" and it is correct. Why am I getting this error then?
I have a similar problem. In my case, my dn was correct, but it says "invalid format (line 16) ..." .
This is how I solved this:
The error msg can be misleading, because the dn is indeed correct. But line number "(line 5)" is an accurate pointer to the problem. In your case, I think the dash '-' is causing the problem. Try to remove the dashes and it may work!
is the cn RDN of the entry?
One can have an entry in directory sn=john, ou=Users .....
sn: john
cn: doe
displayName: johnny
the RDN is an attribute used as reference, in this case
dn: cn=doe,....
would be wrong, no matter that the entry has an attribute in specified value.
In the example you provided, DN is first line, and error says about 5th line.
Assuming you put it in some batch, maybe SOME entry has some special character in it's CN ?
And "Users" is usually an "ou" or "dc" or something, even if the container has "cn" attribute with "Users" as one of cn values. Are entries separated by empty newlines? Didn't you put minus sign after last modification of previous entry above, instead of one ( or more ) newline(s) ?
Oh, and when putting ldif with many entries ( changes ), it's wise to try it with "-n". And (actually for some reason I don't remember) I personally prefer to pipe ldif instead of "-f"
cat user.ldif | ldapmodify ...opts...
?
PS. haha :) try to remove "-a", it turns ldapmodify to ldapadd actually :) And add "-x" if you want to use "simple" auth ( with -w password ).
It must be the problem of character encoding of dash(-)
I met the same problem before because I edited file on windows and used it on centOS in Virtualbox
Then I made a new file in centos using touch and edited it using vi and it works.
I've ran into the same issue. In my case, I had an extra space after the dash (-) character separating the entries.
For example, note the space after the dash on line 5:
$cat -A sample.ldif
dn: uid=noelp,ou=People,dc=example,dc=co,dc=us$
changetype: modify$
replace: employeeType$
employeeType: fulltime$
- $
replace: title$
title: Sr IT Analyst$
-$
replace: employeeNumber$
employeeNumber: 1299$
Same issue can happen with a blank line with a space afer it.
I'm trying to write some code to work with an htdigest password file. The documentation I can find seems to claim that the format of that file is:
user:realm:MD5(user:realm:pass)
If that is the case, then why doesn't this work for me? I created a file with the command line htdigest thus:
htdigest -c test b a
When prompted for a password I entered 'c'. This creates a file with the contents:
a:b:02cc8f08398a4f3113b554e8105ebe4c
However if I try to derive this hash I can't,
echo a:b:c | md5
gives me "49d6ea7ca1facf323ca1928995420354". Is there something obvious that I'm missing here?
Thanks
echo by default adds a trailing new line:
echo -n a:b:c | md5
Should work as you expect.
Hm, I seem to have answered my own question. My test case was flawed, 'echo' is adding extra characters (not sure which). For instance
echo a:b:c | wc
gives 6 characters instead of 5. Calculating the hash at http://md5-hash-online.waraxe.us/ gives the correct value. Sorry everyone!
Here is how you set the password for a given user.
sudo htdigest /etc/apache2/.htdigest yourrealm.com yourusername