I'm trying to configure modsecurity for Apache to limit the number of times a given resource can be accessed.
I wrote this code, and it works (I'm getting a 429 rejection as I wanted), but I can't reinitiate ip.counter at a certain point of time (last line).
SecAction initcol:ip=%{REMOTE_ADDRESS},pass,nolog,id:132
SecAction "phase:2,setvar:ip.counter=+1,pass,nolog,id:332"
SecRule IP:COUNTER "#ge 1" "phase:3,id:'9000080007',pause:10,deny,status:429,setenv:RATELIMITED,skip:1,nolog,id:232"
SecRule TIME "^10:37:00$" "phase:2,id:'9000080008',setvar:!ip.counter"
However, if I switch the last line to use TIME_HOUR instead, the SecRule does apply correctly:
SecRule TIME_HOUR "#eq 10" "phase:2,id:'9000080008',setvar:!ip.counter"
Any help please for using TIME variable in SecRule to match the exact time?
Congratulations on getting a very advanced recipe to work properly. This is really cool.
Now your rule does not work, because the online reference is wrong about the format of the TIME variable (The Handbook is correct though).
Here is how to debug this on ModSec debug log level 9:
SecRule TIME "#unconditionalMatch" "id:1000,phase:2,pass,log,msg:'Key : Value : |%{MATCHED_VAR_NAME}| : |%{MATCHED_VAR}|'"
Leads to:
...4c20][/][5] Rule 562b28db5420: SecRule "TIME" "#unconditionalMatch " "phase:2,auditlog,id:1007,pass,log,msg:'Key : Value : |%{MATCHED_VAR_NAME}| : |%{MATCHED_VAR}|'"
...4c20][/][4] Transformation completed in 0 usec.
...4c20][/][4] Executing operator "unconditionalMatch" with param "" against TIME.
...4c20][/][9] Target value: "20220829070111"
...4c20][/][4] Operator completed in 0 usec.
...4c20][/][9] Resolved macro %{MATCHED_VAR_NAME} to: TIME
...4c20][/][9] Resolved macro %{MATCHED_VAR} to: 20220829070111
...4c20][/][2] Warning. Unconditional match in SecAction. [file "/apache/conf/httpd.conf_pod_2022-08-29_06:58"] [line "209"] [id "1007"] [msg "Key : Value : |TIME| : |20220829070111|"]
...4c20][/][4] Rule returned 1.
...4c20][/][9] Match -> mode NEXT_RULE.
Related
This question already has answers here:
How can I troubleshoot my Perl CGI script?
(8 answers)
Closed 3 months ago.
I am still fairly new to this and I am trying to run this CGI Script on my apache server. When I go to the webpage all I get is a blank page. What am I doing wrong?
#!/usr/bin/perl
use CGI qw(:standard);
print"Content-type: text/html \n\n";
$cost=param('cost');
$num=param('number');
$rev=param('revenue');
$avg = $cost/$num;
$avg=sprintf("%.2f",$avg);
$gp = $rev - $cost;
print "Project Cost Last Year was \$ $cost .<p>";
print "We completed $num projects during the year.";
print " That works out to an average of \$ $avg cost per project.";
print "<p>Our annual project revenue was \$ $rev <br>";
print "We made a gross profit of \$ $gp \n";
That is my current code for the script. I have made sure that my the file is executable as well.
the url i used was 192.168.1.49/cgi-bin/cgi.cgi
I believe you mean http://192.168.1.49/cgi-bin/cgi.cgi.
Given that request URL, what exactly do you expect to happen for the following?
$cost=param('cost');
$num=param('number');
$rev=param('revenue');
$avg = $cost/$num;
Since you didn't provide a value for the number parameter, $num is undef and treated as zero in $avg = $cost/$num;.
Division by zero makes the CPU sad.
You should have figured this out yourself.
An exception like this would have caused the server to return an HTTP status of 500. This indicates you should read your error logs, where you would have found the following message:
Illegal division by zero at [filename] line 7.
If you had used use warnings; as normal, you would also have received these errors:
Use of uninitialized value $num in division (/) at [filename] line 7.
Use of uninitialized value $cost in division (/) at [filename] line 7.
Always use use strict; use warnings;.
Your code suffers from MAJOR security bugs.
Specifically, your code suffers from code injection bugs which can easily be exploited for cross-site scripting attacks and url redirection attacks.
Text included in HTML needs to be converted to HTML.
There is some problem in my samtools_dup rule.
It says "SyntaxError in line 201 of /data/mypipeline.smk:
No rule keywords allowed after run/shell/script/wrapper/cwl in rule samtools_dup. (mypipeline.smk, line 201)".
If I google the error, I found that a person says that, in their code, might be that he placed the "log:" after the "shell:" (and that the shell should be the last thing in each rule), but in my code that is not the case. In many other forums I saw people posting it but no answer was recorded. I am not sure where else this mistake can be... any thoughts? Thank you !
Here I post the code for you to take a look.
dup_fun="rmdup"
# Mark or remove duplicates with Samtools
if ( mrDup == "mark" or mrDup == "rm" ):
rule samtools_dup:
input: f'{bamDir}' + '/{sample}_sort.bam')
params: fun = dup_fun
output: protected(f'{dupDir}' + "/" + f'{mrDup}dup.bam')
shell: "samtools {params.fun} -s {input} {output}"
I see a syntax error in your snippet: there is a closing bracket without an opening bracket in the input section:
input: f'{bamDir}' + '/{sample}_sort.bam')
^ where is the opening bracket?
There may be other syntax errors in your file, but you definitely provide just a small portion of it.
I have both Apache and Modsecurity working together. I'm trying to limit hit rate by request's header (like "facebookexternalhit"). And then return a friendly "429 Too Many Requests" and "Retry-After: 3".
I know I can read a file of headers like:
SecRule REQUEST_HEADERS:User-Agent "#pmFromFile ratelimit-bots.txt"
But I'm getting trouble building the rule.
Any help would be really appreciated. Thank you.
After 2 days of researching and understanding how Modsecurity works, I finally did it. FYI I'm using Apache 2.4.37 and Modsecurity 2.9.2 This is what I did:
In my custom file rules: /etc/modsecurity/modsecurity_custom.conf I've added the following rule:
# Limit client hits by user agent
SecRule REQUEST_HEADERS:User-Agent "#pm facebookexternalhit" \
"id:400009,phase:2,nolog,pass,setvar:global.ratelimit_facebookexternalhit=+1,expirevar:global.ratelimit_facebookexternalhit=3"
SecRule GLOBAL:RATELIMIT_FACEBOOKEXTERNALHIT "#gt 1" \
"chain,id:4000010,phase:2,pause:300,deny,status:429,setenv:RATELIMITED,log,msg:'RATELIMITED BOT'"
SecRule REQUEST_HEADERS:User-Agent "#pm facebookexternalhit"
Header always set Retry-After "3" env=RATELIMITED
ErrorDocument 429 "Too Many Requests"
Explanation:
Note: I want to limit to 1 request every 3 seconds.
The first rule matches the request header user agent against "facebookexternalhit". If the match was succesful, it creates the ratelimit_facebookexternalhit property in the global collection with the initial value of 1 (it will increment this value with every hit matching the user agent). Then, it sets the expiration time of this var in 3 seconds. If we receive a new hit matching "facebookexternalhit" it will sum 1 to ratelimit_facebookexternalhit. If we don't receive hits matching "facebookexternalhit" after 3 seconds, ratelimit_facebookexternalhit will be gone and this process will be restarted.
If global.ratelimit_clients > 1 (we received 2 or more hits within 3 seconds) AND user agent matches "facebookexternalhit" (this AND condition is important because otherwise all requests will be denied if a match is produced), we set RATELIMITED=1, stop the action with a 429 http error, and log a custom message in Apache error log: "RATELIMITED BOT".
RATELIMITED=1 is set just to add the custom header "Retry-After: 3". In this case, this var is interpreted by Facebook's crawler (facebookexternalhit) and will retry operation in the specified time.
We map a custom return message (in case we want) for the 429 error.
You could improve this rule by adding #pmf and a .data file, then initializing global collection like initcol:global=%{MATCHED_VAR}, so you are not limited just to a single match by rule. I didn't test this last step (this is what I needed right now). I'll update my answer in case I do.
UPDATE:
I've adapted the rule to be able to have a file with all user agents I want to rate limit, so a single rule can be used across multiple bots/crawlers:
# Limit client hits by user agent
SecRule REQUEST_HEADERS:User-Agent "#pmf data/ratelimit-clients.data" \
"id:100008,phase:2,nolog,pass,setuid:%{tx.ua_hash},setvar:user.ratelimit_client=+1,expirevar:user.ratelimit_client=3"
SecRule USER:RATELIMIT_CLIENT "#gt 1" \
"chain,id:1000009,phase:2,deny,status:429,setenv:RATELIMITED,log,msg:'RATELIMITED BOT'"
SecRule REQUEST_HEADERS:User-Agent "#pmf data/ratelimit-clients.data"
Header always set Retry-After "3" env=RATELIMITED
ErrorDocument 429 "Too Many Requests"
So, the file with user agents (one per line) is located inside a subdirectory under the same directory of this rule: /etc/modsecurity/data/ratelimit-clients.data. Then we use #pmf to read and parse the file (https://github.com/SpiderLabs/ModSecurity/wiki/Reference-Manual-(v2.x)#pmfromfile). We initialize the USER collection with the user agent: setuid:%{tx.ua_hash} (tx.ua_hash is in the global scope in /usr/share/modsecurity-crs/modsecurity_crs_10_setup.conf). And we simply use user as collection instead of global. That's all!
Might be better to use "deprecatevar",
And you can allow a bit bigger burst leneanancy
# Limit client hits by user agent
SecRule REQUEST_HEADERS:User-Agent "#pmf data/ratelimit-clients.data" \
"id:100008,phase:2,nolog,pass,setuid:%{tx.ua_hash},setvar:user.ratelimit_client=+1,deprecatevar:user.ratelimit_client=3/1"
SecRule USER:RATELIMIT_CLIENT "#gt 1" \
"chain,id:100009,phase:2,deny,status:429,setenv:RATELIMITED,log,msg:'RATELIMITED BOT'"
SecRule REQUEST_HEADERS:User-Agent "#pmf data/ratelimit-clients.data"
Header always set Retry-After "6" env=RATELIMITED
ErrorDocument 429 "Too Many Requests"
In modsecurity default-script:
base_rules/modsecurity_crs_20_protocol_violations.conf
there is a rule, 960011:
SecRule REQUEST_METHOD "^(?:GET|HEAD)$" \
"msg:'GET or HEAD Request with Body Content.',\
severity:'2',\
id:'960011',\
ver:'OWASP_CRS/2.2.9',\
rev:'1',\
maturity:'9',\
accuracy:'9',\
phase:1,\
block,\
logdata:'%{matched_var}',\
t:none,\
tag:'OWASP_CRS/PROTOCOL_VIOLATION/INVALID_HREQ',\
tag:'CAPEC-272',\
chain"
SecRule REQUEST_HEADERS:Content-Length "!^0?$"\
"t:none,\
setvar:'tx.msg=%{rule.msg}',\
setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},\
setvar:'tx.%{rule.id}-OWASP_CRS/PROTOCOL_VIOLATION/INVALID_HREQ-%{matched_var_name}=%{matched_var}'"
I only want to disable logging for this rule (it gives too many false positives),
and therefore add my own script
base_rules/z99_logging_suppress.conf
to remove the default-rule and create a new identical rule -- only without logging:
SecRuleRemoveById 960011
SecRule REQUEST_METHOD "^(?:GET|HEAD)$" \
"msg:'GET or HEAD Request with Body Content.',\
severity:'2',\
id:'9960011',\
ver:'OWASP_CRS/2.2.9',\
rev:'1',\
maturity:'9',\
accuracy:'9',\
phase:1,\
block,nolog,\
logdata:'%{matched_var}',\
t:none,\
tag:'OWASP_CRS/PROTOCOL_VIOLATION/INVALID_HREQ',\
tag:'CAPEC-272',\
chain"
SecRule REQUEST_HEADERS:Content-Length "!^0?$"\
"t:none,\
setvar:'tx.msg=%{rule.msg}',\
setvar:tx.anomaly_score=+%{tx.critical_anomaly_score},\
setvar:'tx.%{rule.id}-OWASP_CRS/PROTOCOL_VIOLATION/INVALID_HREQ-%{matched_var_name}=%{matched_var}'"
The only differences to the original rule are the new id 9960011, and the nolog additions:
...
id:'9960011',\
...
block,nolog,\
...
But when I restart httpd with this additional rule, I get error:
AH00526: Syntax error on line 18 of /path/base_rules/z99_logging_suppress.conf:
ModSecurity: Execution phases can only be specified by chain starter rules.
The same strategy --- SecRuleRemoveById + then re-create it with new id --- works for all other default-rules I tried, but not for this one.
Anyone can tell me why that is?
It basically says that the phase command can only be in the first rule in a chain and not in a subsequent rule which forms part of the chain.
There is nothing wrong with the rule as you have written it, phase is only specified in the first SecRule. In fact I've tried it on my instance and it works. So either one of two things has gone wrong:
You have copied and pasted it incorrectly into this question.
The rule above where you have defined this, has chain in it and so has left an open chain, that your rule 9960011 is then effectively trying to continue on from.
Or something else weird is happening! But I'm going with 1 or 2 for now :-)
I'm trying to set up a ModSecurity whitelist for arguments with an unknown name, but matching a value. For example, I want to whitelist any parameter that is a timestamp (e.g. timestamp=2016-01-01 00:00:00). Currently, this triggers rule 981173 (Restricted SQL Character Anomaly Detection Alert - Total # of special characters exceeded)
The following will work, but will skip checks on all parameters if at least one matches, so it doesn't catch the badvalue parameter in https://www.example.com/?timestamp=2016-01-01+00:00:00&badvalue=2016-01-01+00:00:00:00.
SecRule ARGS "#rx ^2[0-9]{3}-[0-1][0-9]-[0-3][0-9] [0-2][0-9]:[0-5][0-9]:[0-5][0-9]$" \
"id:'99001', phase:1, nolog, pass, t:none, \
ctl:ruleRemoveTargetByTag=OWASP_CRS/WEB_ATTACK/SQL_INJECTION;ARGS"
The following works if I hardcode the parameter name.
SecRule ARGS:timestamp "#rx ^2[0-9]{3}-[0-1][0-9]-[0-3][0-9] [0-2][0-9]:[0-5][0-9]:[0-5][0-9]$" \
"id:'99001', phase:1, nolog, pass, t:none, \
ctl:ruleRemoveTargetByTag=OWASP_CRS/WEB_ATTACK/SQL_INJECTION;ARGS:timestamp"
I've tried the following, but they haven't worked.
SecRule ARGS "#rx ^2[0-9]{3}-[0-1][0-9]-[0-3][0-9] [0-2][0-9]:[0-5][0-9]:[0-5][0-9]$" \
"id:'99001', phase:1, nolog, pass, t:none, \
ctl:ruleRemoveTargetByTag=OWASP_CRS/WEB_ATTACK/SQL_INJECTION;/%{MATCHED_VAR_NAME}/"
SecRule ARGS "#rx ^2[0-9]{3}-[0-1][0-9]-[0-3][0-9] [0-2][0-9]:[0-5][0-9]:[0-5][0-9]$" \
"id:'99001', phase:1, nolog, pass, t:none, \
ctl:ruleRemoveTargetByTag=OWASP_CRS/WEB_ATTACK/SQL_INJECTION;MATCHED_VAR_NAME"
Is this possible with ModSecurity? Is there a way to use MATCHED_VAR_NAME for this use case? I would rather not have to add a rule for every argument name that might contain a timestamp.
Unfortunately, it is currently not possible to use Macro Expansion within the ctl action argument.
As evidence consider the following examples:
SecRule ARGS "#contains bob" "id:1,t:none,pass,ctl:ruleRemoveTargetById=2;ARGS:x"
SecRule ARGS "#contains hello" "id:2,deny,status:403"
When providing the following request: 'http://localhost/?x=bobhello' we will see the following in the debug log when evaluating the second rule
[04/Aug/2016:00:44:07 --0400] [localhost/sid#55e47aa583e0][rid#55e47ad7cb10][/][4] Recipe: Invoking rule 55e47ab14638; [file "/etc/httpd/modsecurity.d/includeOWASP.conf"] [line "12"] [id "2"].
[04/Aug/2016:00:44:07 --0400] [localhost/sid#55e47aa583e0][rid#55e47ad7cb10][/][5] Rule 55e47ab14638: SecRule "ARGS" "#contains hello" "phase:2,log,auditlog,id:2,deny,status:403"
[04/Aug/2016:00:44:07 --0400] [localhost/sid#55e47aa583e0][rid#55e47ad7cb10][/][4] Transformation completed in 0 usec.
[04/Aug/2016:00:44:07 --0400] [localhost/sid#55e47aa583e0][rid#55e47ad7cb10][/][9] fetch_target_exception: Found exception target list [ARGS:x] for rule id 2
[04/Aug/2016:00:44:07 --0400] [localhost/sid#55e47aa583e0][rid#55e47ad7cb10][/][9] fetch_target_exception: Target ARGS:x will not be processed.
[04/Aug/2016:00:44:07 --0400] [localhost/sid#55e47aa583e0][rid#55e47ad7cb10][/][4] Executing operator "contains" with param "hello" against ARGS:x skipped.
[04/Aug/2016:00:44:07 --0400] [localhost/sid#55e47aa583e0][rid#55e47ad7cb10][/][4] Rule returned 0.
However, When we provide the same request ('http://localhost/?x=bobhello') While have Macro Expansion within our ctl action (as follows):
SecRule ARGS "#contains bob" "id:1,t:none,pass,ctl:ruleRemoveTargetById=2;%{MATCHED_VAR_NAME}"
SecRule ARGS "#contains hello" "id:2,deny,status:403"
Our Debug log will appear as follows:
[04/Aug/2016:00:44:41 --0400] [localhost/sid#559f82a0b3e0][rid#559f82d2fb50][/][5] Rule 559f82ac76e8: SecRule "ARGS" "#contains hello" "phase:2,log,auditlog,id:2,deny,status:403"
[04/Aug/2016:00:44:41 --0400] [localhost/sid#559f82a0b3e0][rid#559f82d2fb50][/][4] Transformation completed in 0 usec.
[04/Aug/2016:00:44:41 --0400] [localhost/sid#559f82a0b3e0][rid#559f82d2fb50][/][9] fetch_target_exception: Found exception target list [%{MATCHED_VAR_NAME}] for rule id 2
[04/Aug/2016:00:44:41 --0400] [localhost/sid#559f82a0b3e0][rid#559f82d2fb50][/][4] Executing operator "contains" with param "hello" against ARGS:x.
[04/Aug/2016:00:44:41 --0400] [localhost/sid#559f82a0b3e0][rid#559f82d2fb50][/][9] Target value: "bobhello"
[04/Aug/2016:00:44:41 --0400] [localhost/sid#559f82a0b3e0][rid#559f82d2fb50][/][4] Operator completed in 2 usec.
[04/Aug/2016:00:44:41 --0400] [localhost/sid#559f82a0b3e0][rid#559f82d2fb50][/][4] Rule returned 1.
I cannot think of a method of accomplishing this goal without excessive overhead. At this point the best solution would likely be to manually whitelist each offending argument.