Let me start about Why...
Why
Since most people agree that the most effective way of blocking Ip's is by using a solution like iptables. So with that in mind why would i wanna block such a huge list directly in Apache. As a maintainer of servers and after investigating "unwanted traffic" trying to do all kinds of things that the online service isnt made for i started looking for a solution to stop that once and for all....
So in the past 2 years i identified all/most possible networks that produce "unwanted traffic".
These networks have no use on the online service at all and are bassicly bad for the environment when u look at all the energy and resources that they consume!
So in my attempt to remove all the power from these services i made a list of all worldwide hosting addresses and bussines connections that are outside of the targetted area from the online service (which means all business connections unless they are inside 2 targetted countries and all hosting addresses).
This resulted in almost 300k subnets which after combining the subnets resulted in almost 290k subnets left that i wanna block.
When using this list in a solution like iptables it would also block updating services so to prevent this it's better to use apache since it will then only affect apache and nothing else on this server, protecting an entire server is not inside the scope of this question.
Possible solutions
Adding the subnets to .htaccess (Don't rlly wanna go there since this will make apache reload the list on every request)
Using
RewriteEngine on
RewriteMap hosts-deny "txt:/path/to/hosts.deny"
RewriteCond "${hosts-deny:%{REMOTE_ADDR}|NOT-FOUND}" "!=NOT-FOUND" [OR]
RewriteCond "${hosts-deny:%{REMOTE_HOST}|NOT-FOUND}" "!=NOT-FOUND"
RewriteRule "^" "-" [F]
Works very well when using single addresses but converting 300k subnets to individual addresses doens't look very appealing to me, so unless this can be transformed to work with subnets this isn't going to work either.
Using expressions
RewriteCond expr "-R '12.345.678.90/28'"
This works untill i try to add more then 1 subnet, i think it goes wrong with RequireAll, RequireAny, RequireNone or the location it is placed.
Hopefully someone can shed a light on this and guide me towards the right way of implementing my solution.
Related
After Apache upgrade on my shared server, I have been having nightmare issues with the form input on all of my reseller hosting accounts. The clients get 403 (or in case of Wordpress install, 404, which has really confused me) after the simplest, most innocent looking form input. For example "he is having a lot of trouble" in a text field results in 403!
It took almost two weeks to figure out what's going on, as the error seemed random and hard to replicate, but after I asked for exact text from the clients that they were not able to enter, we got to the modsec issue. The answer from tech help was "While checking the issue in detail, we found that a mod_security rule was getting triggered on the server while trying to submit the content as "he is having a lot of trouble". We have whitelisted the rule for the website which resolved the issues.".
My question is - how can I deal with this proactively? Is there a list of rules for mod_security that I can check, test some input, ask for additional whitelisting etc? With about a 100 accounts all having problems, it's enough to want to get out of the hosting business all together.
I don't understand your scenario or your question. Are you managing the host or not?
It sounds like you are hosting sites on a shared server so do not have access to the full server but are setting hosts up for clients - is that right?
Running a WAF like ModSecurity requires monitoring log files to identify false positives like this. If you do not have access to the log files then you need to ask your hosting provider what there options are for managing this sort of thing? Or will they do nothing until you raise it?
You can also ask to turn off ModSecurity completely. Most sites get on fine without a WAF - though personally I think they do add value and security.
Finally as to what rules are running on your instance only your tech help can answer that. ModSecurity itself is only an engine and comes with no rules. People can write their own, but some, or use free sets of rules like the OWASP Core Rule Set. So depending what you have would depend how you can test this. Most rules are fairly generic in nature so do result in false positives unless tweaked.
I'm struggling with a mod_rewrite problem. Basically I need to do a secret redirect on the domain name, going from
http://domainname.com.someotherstuff.com
to
http://domainname.com
This rule should affect all subdirectories as well.
I've understood there are three steps:
tell the system if the path matches what we're looking for
define the RewriteRule
pass the new path to the old one so that the system knows (even if it doesn't show) that the two match
I've looked up several posts and resources (the closest ones being this and this) but none of them can solve both my problems – rewriting and secrecy – at once.
Can anybody point me in the right direction?
Moreover, can someone explain the tradeoff between a hidden redirect and a 301? Hidden redirect is not search engine friendly, correct?
Thanks a lot!
referring to an older post for clarification on rewrite vs redirect
If you want the customer's browser to say http://domainname.com, but fetch the content from http://domainname.com.someotherstuff.com, then what you want is a rewrite. You will point your customer at http://domainname.com and that answering frontend (server/LB/etc...) will then rewrite "domainname.com" to "domainname.com.someotherstuff.com" and send the request on to a backend service that will answer that request. I prefer to SNAT in this case, so the backend responds directly to the frontend, which then returns the content to the customer none the wiser.
You have several moving parts here:
DNS entries for domainname.com and domainname.com.someotherstuff.com
frontend - F5s are my favorites, but you can achieve similar results with any linux server; needs to be able to resolve domainname.com.someotherstuff.com and has network connectivity to the backend; servicing requests for http://domainname.com
backend - web server; servicing requests from frontend for http://domainname.com.someotherstuff.com
Background: we're running Drupal 6 on an Apache server. I've scoured the internet but can't seem to find anything on exactly what I'm looking for, maybe someone here has an idea.
As a website with a decent amount of traffic, we tend to get a lot of low-level attack attempts. Any time I look through logs, there's at least a handful of "page not found" errors from script kiddies and bots trying to access pages like wp-login.php or admin.php. Obviously these attacks never get very far, but it's still a sometimes significant load on our server to serve up all the 404s.
These attempts are often quite amateurish. Generally, they all come from one IP address over a period of a few minutes. So I'm wondering if there's some way to implement temporary blocks by IP address for anyone who tries something that's obviously an attack. For example, maybe there's a way to configure .htaccess to say:
If (bot_IP tries to access wp_login.php, admin.php, administer/index.php, phpmyadmin.php and so on)
Deny from bot_IP for next four hours
Has anyone ever tried anything like this? It would be wonderful if we could reduce the amount of time we spend sending 404s to attackers, and it seems to me like a lot of people could find it useful.
Thanks!
If you do this in a Drupal module, you'll spend more resources checking logs and filtering requests than you do sending 404's. If you have root access to the server, http://www.fail2ban.org/wiki/index.php/Main_Page will work very well. It scans the apache error logs, uses regex based rules to match log entries and updates the OS firewall rules to handle the blocking.
I am looking for a solution which would redirect the externally facing http://mycompany.com/external/* to be redirected/proxied to http://internal-host:1234/internal/*
(the asterisk is used as a wildcard)
OK, I guess the sentence above is not enough, so here are the details:
In my intranet I have several servers, (names, addresses, ports, and context paths are obviously made-up for the sake of simplicity):
HRServer running at address 10.10.10.10:1010/hr
MailServer running at address 20.20.20.20:2020/mail
My system is accessible from internet only from ip 78.78.78.78, and the constraint here is that I can use only one port (e.g. 8080). In other words - whatever the solution of my problem is - the external address should start with 78.78.78.78:8080
What I need to do is to expose both HR and Mail services though this port.
The first thing which came to my mind was to write two simple portlets (or an HTML with two frames) and to embed them in a simple web page at 78.78.78.78:8080/
But obviously this will not work, as the portlets will redirect the browser to e.g 10.10.10.10:1010/hr which is not visible from the internet.
So my next thought was - OK, lets find a reverse proxy which has dispatching capabilities. Then I can make
78.78.78.78:8080/hr to "redirect" to the internal 10.10.10.10:1010/hr
78.78.78.78:8080/mail to "redirect" to the internal 20.20.20.20:2020/mail
I'd also expect that if let's say the mail server unread messages are seen on 20.20.20.20:2020/mail/unread the unread messages to be also accessible from internet.
Roughly speaking - I'd expect
78.78.78.78:8080/mail/* to redirect to the internal 20.20.20.20:2020/mail/* (the asterisk is used as a wildcard)
I really feel I am missing the obvious here, but honestly - I've spent quite a while in researching several proxies and I did not find the answer. I might be looking for the wrong words or something, but I could not find reverse proxy which can be configured to dispatch external path to different internal paths.
So please - if the answer is e.g. the Apache mod_proxy - please give me a hint about the parameter names that I should be looking for.
Lastly - I am going to run thin in a FreeBSD OS, but this is not a strong requirement (other *nix OSes are also fine)
Thanks!
It took quite a while, but here is the answer:
A good solution is nginx (pronounced "Engine X").
To reroute all traffic which comes to
https://mycompany.com/external/* to
http://internal-host:1234/internal/* (the asterisk is used as a wildcard) you need to have the following configuration:
location ~ ^/internal/ {
rewrite ^/internal/(.*)$ /$1 break;
proxy_pass http://internal-host:1234;
}
And this approach can be used for all the other addresses - e.g. HR portal, mail, etc.
Finally, to give you a heads up - the following configuration does not work:
location ~ ^/internal/(.*)$ {
proxy_pass http://internal-host:1234/internal/$1;
}
It turns out nginx will always proxypass the whole URI when regex is used, so the rule has to be the one above (which does url-rewrite).
Do any of you folks do redirection by IP (within Apache) for people coming
to your website from different countries?
What do ye use? mod_geoip and/or a whole pile of mod_rewrite rules?
Is mod_geoip reliable/well-performing/accurate?
I used to use mod_geoip but recently dumped Apache for Cherokee (which has its own GEOIP logic).
I can't say I've had any performance problems with either but Cherokee is definitely easier to get set up.
Accuracy is pretty spot on but (obviously) neither cannot tell if somebody is browsing through a transparent proxy in another country.