my SFTP-Server with fail2ban and UFW is running well. But now I recognized that it seems that IP-adresses are not banned when there connection is dropped by "past MaxStartups".
My auth.log looks like
Feb 26 23:19:42 SFTPSERVER sshd[2719]: drop connection #10 from [1xx.1xx.xx.xx]:57970 on [xx.xx.xx.xx]:22 past MaxStartups
In the fail2ban.log there is no entry for that ip. So it seems that this logentry is not filtered by fail2ban. How can I add a filter for this log entry? Do I have to add some regex to the sshd.conf in filter.d of my fail2ban folder?
MaxStartups can hit anyone, i.e. it's the maximum amount of unauthenticated connections (from ALL sources, combined) before the SSH daemon will kill any new ones, until some of those unauthenticated connections have dropped.
So, Fail2Ban not banning on those, is correct... (else, it might ban you, before you even had a chance to authenticate...)
Enable more fail2ban rules:
Banning IP addresses bases on being "past MaxStartups" would be quite random and it is not implemented for that reason.
Rather, consider banning IP's that contribute to MaxStartups counters while not being covered by the default fail2ban rules. The rules are typically located in:
/etc/fail2ban/filter.d/sshd.conf
Specifically, look for rules that match the term 'preauth' - those are not included in defaults since they are not conclusively attacks (it could be you trying to connect but the network failed just at that time). They do however significantly contribute to your MaxStartups counter as they commonly dangle for a long time before timing out or similar. If your ban is not too aggressive, you might want to include those by uncommenting them.
Other strategies:
Increase MaxStartups and just allow that background noise to consume a bit of your resources
Change your SSH port to drastically reduce background noise (troublesome for many apps, though)
Set IP filters in your firewall to limit sources allowed to connect to SSH
Use port knocking (this practically reduces SSH use to your own though)
Related
I have received some results from a security scan that say that something is executing DNS A record look-ups on the URL in the Host header.
Having looked at the application code I can't see any such requests so I'm looking further up the stack.
I don't think Apache should be doing this but it's using mod_headers and mod_rewrite and maybe there is a configuration item in there that I have overlooked.
A long time ago, I came across an Apache httpd that was configured to do a reverse-lookup for IP-addresses before logging. While this was long denied, some requests were served quickly, while others took a long time (depending on the time required for the reverse lookup). And it became obvious once we looked at the logs (DNS names mixed with IP-addresses)
I don't see any reason why Headers and Rewrite would ever need to resolve any of the domains - they're purely working on strings/regexp.
Recommendation to figure out what's going on: Capture the traffic and figure out what domains/addresses are looked up when. With DNS still being largely unencrypted, this might be fairly easy, and point you to the smoking gun.
What techniques and/or modules are available to implement robust rate limiting (requests|bytes/ip/unit time) in apache?
The best
mod_evasive (Focused more on reducing DoS exposure)
mod_cband (Best featured for 'normal' bandwidth control)
and the rest
mod_limitipconn
mod_bw
mod_bwshare
As stated in this blog post it seems possible to use mod_security to implement a rate limit per second.
The configuration is something like this:
SecRuleEngine On
<LocationMatch "^/somepath">
SecAction initcol:ip=%{REMOTE_ADDR},pass,nolog
SecAction "phase:5,deprecatevar:ip.somepathcounter=1/1,pass,nolog"
SecRule IP:SOMEPATHCOUNTER "#gt 60" "phase:2,pause:300,deny,status:509,setenv:RATELIMITED,skip:1,nolog"
SecAction "phase:2,pass,setvar:ip.somepathcounter=+1,nolog"
Header always set Retry-After "10" env=RATELIMITED
</LocationMatch>
ErrorDocument 509 "Rate Limit Exceeded"
There are numerous way including web application firewalls but the easiest thing to implement if using an Apache mod.
One such mod I like to recommend is mod_qos. It's a free module that is veryf effective against certin DOS, Bruteforce and Slowloris type attacks. This will ease up your server load quite a bit.
It is very powerful.
The current release of the mod_qos module implements control mechanisms to manage:
The maximum number of concurrent requests to a location/resource
(URL) or virtual host.
Limitation of the bandwidth such as the
maximum allowed number of requests per second to an URL or the maximum/minimum of downloaded kbytes per second.
Limits the number of request events per second (special request
conditions).
Limits the number of request events within a defined period of time.
It can also detect very important persons (VIP) which may access the
web server without or with fewer restrictions.
Generic request line and header filter to deny unauthorized
operations.
Request body data limitation and filtering (requires mod_parp).
Limits the number of request events for individual clients (IP).
Limitations on the TCP connection level, e.g., the maximum number of
allowed connections from a single IP source address or dynamic
keep-alive control.
Prefers known IP addresses when server runs out of free TCP
connections.
This is a sample config of what you can use it for. There are hundreds of possible configurations to suit your needs. Visit the site for more info on controls.
Sample configuration:
# minimum request rate (bytes/sec at request reading):
QS_SrvRequestRate 120
# limits the connections for this virtual host:
QS_SrvMaxConn 800
# allows keep-alive support till the server reaches 600 connections:
QS_SrvMaxConnClose 600
# allows max 50 connections from a single ip address:
QS_SrvMaxConnPerIP 50
# disables connection restrictions for certain clients:
QS_SrvMaxConnExcludeIP 172.18.3.32
QS_SrvMaxConnExcludeIP 192.168.10.
http://opensource.adnovum.ch/mod_qos/
In Apache 2.4, there's a new stock module called mod_ratelimit. For emulating modem speeds, you can use mod_dialup. Though I don't see why you just couldn't use mod_ratelimit for everything.
Sadly, mod_evasive won't work as expected when used in non-prefork configurations (recent apache setups are mainly MPM)
Depends on why you want to rate limit.
If it's to protect against overloading the server, it actually makes sense to put NGINX in front of it, and configure rate limiting there. It makes sense because NGINX uses much less resources, something like a few MB per ten thousand connections. So, if the server is flooded, NGINX will do the rate limiting(using an insignificant amount of resources) and only pass the allowed traffic to Apache.
If all you're after is simplicity, then use something like mod_evasive.
As usual, if it's to protect against DDoS or DoS attacks, use a service like Cloudflare which also has rate limiting.
One more option - mod_qos
Not simple to configure - but powerful.
http://opensource.adnovum.ch/mod_qos/
I've just enabled mod_security on my CentOS server via WHM with OWASP ModSecurity Core Rule Set.
I've seen some very useful stuff so far but also some very annoying, not very useful things such as:
016-06-23 10:00:01 127.0.0.1 WARNING 200 960008: Request Missing a Host Header
2016-06-23 10:00:01 127.0.0.1 NOTICE 200 960009: Request Missing a User Agent Header More
2016-06-23 10:00:01 127.0.0.1 200 981204: Inbound Anomaly Score Exceeded (Total Inbound Score: 5): Request Missing a User Agent Header
Basically 95% of the hits (which are a lot) are like this, form localhost.
What is this exactly? And are there security concerns should I simply whitelist localhost?
Thanks
Only you can answer this to be honest.
It sounds like you have something running locally which connects to you webserver. Maybe a script to check if the site is up? Maybe something else that gets something webserver.
Whatever it is not setting:
The Host Header: which site you want to visit - so guessing script is using IP address to connect or just not passing the hostname.
The user agent: all browsers pass a user agent so when this is not present its highly likely it's a script. That's not to say whether that script is malicious or useful.
Neither of those rules are really problems or security concerns in themselves. They are just a sign that it's likely not a browser so likely is a script so maybe want to block.
To be honest I'd find out what is doing this. As I say I would guess it might be a heartbeat check script to check site it still up. If so, then no problem whitelisting localhost for these two rules. If not then you can decide whether you want to whitelist or not.
I personally prefer to only whitelist what's necessary rather than blanket whitelist all of localhost, so to whitelist these rules you can add config like this BEFORE the rules (as the ctl command expects the rules it's amending to come after it):
SecRule REMOTE_ADDR "#ipMatch 127.0.0.1" "id:1000,phase:2,nolog,ctl:ruleRemoveById=960008;ctl:ruleRemoveById=960009"
I have a web-service where users download files tunneled via apache2 reverse proxies.
I am using mod rewrite and the P flag in conjunction with a rewrite map.
Basically it looks like this:
<Location /my-identifier>
RewriteEngine on
RewriteRule /my-identifier/(.*) ${my-rewrite-map:$1} [P]
</Location>
I know I can limit the bandwidth of one connection or even one ip-address per server using mod_bandwidth or something similar.
However I want the limit to take effect only for certain users (namely those who make a lot of traffic and exceeded the fair use volumes).
I also want it to span across multiple servers.
It is possible for me to set a custom environment variable, if that helps (I have full control over the URL where I can encode it into and can set it using the rewrite rule)!
Basically what I want is for example for a user who reached their limit to get only 5 mbps of speed, no matter how many connections they use or how many servers they connect to.
Is it somehow possible? Is there a module?
My thought would be a centralized data-store where the servers report their traffic stats per ip to. Probably some sort of RRD data structure. Then they can select the traffic for the ip over a specified time interval (for example the last 60 seconds) and apply a throttle factor according to it.
But I really don't want to do this all by myself, I could but it would take me months... I am also not bound to apache. I am also using Nginx servers for the same thing, if there is something for Nginx I can switch to it!
I don't know about Apache, but since you listed Nginx as a tag, how about something like the approach below?
Set up Nginx as a reverse proxy to your Apache servers or web-services with more or less the following configuration:
upstream serverlist {
server www1.example.com;
server www2.example.com;
server www3.example.com;
}
location / {
proxy_pass http://serverlist;
}
The "overall connections" requirement you have is not directly mappable, but you can probably get reasonably close to what you want with a combination of the following directives added to the location block:
limit_rate this is per connection
limit_con this allows you to limit the number of connections
limit_req this allows you to limit the number of requests/sec and allowable bursts
limit_zone sets up the zone for your limits
UPDATE:
There's a 3th party Nginx module limiting the overall rate per IP to be found here.
What techniques and/or modules are available to implement robust rate limiting (requests|bytes/ip/unit time) in apache?
The best
mod_evasive (Focused more on reducing DoS exposure)
mod_cband (Best featured for 'normal' bandwidth control)
and the rest
mod_limitipconn
mod_bw
mod_bwshare
As stated in this blog post it seems possible to use mod_security to implement a rate limit per second.
The configuration is something like this:
SecRuleEngine On
<LocationMatch "^/somepath">
SecAction initcol:ip=%{REMOTE_ADDR},pass,nolog
SecAction "phase:5,deprecatevar:ip.somepathcounter=1/1,pass,nolog"
SecRule IP:SOMEPATHCOUNTER "#gt 60" "phase:2,pause:300,deny,status:509,setenv:RATELIMITED,skip:1,nolog"
SecAction "phase:2,pass,setvar:ip.somepathcounter=+1,nolog"
Header always set Retry-After "10" env=RATELIMITED
</LocationMatch>
ErrorDocument 509 "Rate Limit Exceeded"
There are numerous way including web application firewalls but the easiest thing to implement if using an Apache mod.
One such mod I like to recommend is mod_qos. It's a free module that is veryf effective against certin DOS, Bruteforce and Slowloris type attacks. This will ease up your server load quite a bit.
It is very powerful.
The current release of the mod_qos module implements control mechanisms to manage:
The maximum number of concurrent requests to a location/resource
(URL) or virtual host.
Limitation of the bandwidth such as the
maximum allowed number of requests per second to an URL or the maximum/minimum of downloaded kbytes per second.
Limits the number of request events per second (special request
conditions).
Limits the number of request events within a defined period of time.
It can also detect very important persons (VIP) which may access the
web server without or with fewer restrictions.
Generic request line and header filter to deny unauthorized
operations.
Request body data limitation and filtering (requires mod_parp).
Limits the number of request events for individual clients (IP).
Limitations on the TCP connection level, e.g., the maximum number of
allowed connections from a single IP source address or dynamic
keep-alive control.
Prefers known IP addresses when server runs out of free TCP
connections.
This is a sample config of what you can use it for. There are hundreds of possible configurations to suit your needs. Visit the site for more info on controls.
Sample configuration:
# minimum request rate (bytes/sec at request reading):
QS_SrvRequestRate 120
# limits the connections for this virtual host:
QS_SrvMaxConn 800
# allows keep-alive support till the server reaches 600 connections:
QS_SrvMaxConnClose 600
# allows max 50 connections from a single ip address:
QS_SrvMaxConnPerIP 50
# disables connection restrictions for certain clients:
QS_SrvMaxConnExcludeIP 172.18.3.32
QS_SrvMaxConnExcludeIP 192.168.10.
http://opensource.adnovum.ch/mod_qos/
In Apache 2.4, there's a new stock module called mod_ratelimit. For emulating modem speeds, you can use mod_dialup. Though I don't see why you just couldn't use mod_ratelimit for everything.
Sadly, mod_evasive won't work as expected when used in non-prefork configurations (recent apache setups are mainly MPM)
Depends on why you want to rate limit.
If it's to protect against overloading the server, it actually makes sense to put NGINX in front of it, and configure rate limiting there. It makes sense because NGINX uses much less resources, something like a few MB per ten thousand connections. So, if the server is flooded, NGINX will do the rate limiting(using an insignificant amount of resources) and only pass the allowed traffic to Apache.
If all you're after is simplicity, then use something like mod_evasive.
As usual, if it's to protect against DDoS or DoS attacks, use a service like Cloudflare which also has rate limiting.
One more option - mod_qos
Not simple to configure - but powerful.
http://opensource.adnovum.ch/mod_qos/