On page: http://code.google.com/p/modwsgi/wiki/FileWrapperExtension , Graham Dumpleton recommends the following:
"Do note however that for the best performance, static files should
always be served by a web server. In the case of mod_wsgi this means
by Apache itself rather than mod_wsgi or the WSGI application."
I'd like to pre-build a large number of static pages, then have a python program (running under apache/mod_wsgi 3.3/python3.1, daemon mode, no django involved) decide which of them to serve to each user. I'd like the python program to decide, for example, that this guy needs "12345.html" and have it tell Apache, "please serve static file '12345.html' to this guy", rather than having to use python to open the file, read the contents, turn it into a python string, and return it to mod_wsgi as "[output]".
Is this possible? If so, how?
If not, what's the best way to do this?
There are numerous ways one could do it.
X-Sendfile implemented by mod_xsendfile and Apache.
Location/mod_rewrite tricks using mod_wsgi daemon mode.
X-Accel-Redirect if also using nginx as front end to Apache.
Read up on (1) and (3) as more widely used options.
Update with instructions for (2).
Have the WSGI application return a 200 response with empty body and 'Location' response header with URL path to local resource hosted on same Apache server and mod_wsgi when daemon mode is being used will trigger an internal redirect to that URL.
Thus if your Apache has:
Alias /generated-files/ /some/path/
<Directory /some/path>
Order allow, deny
Allow from all
</Directory>
then generate your file as /some/path/foo.txt in file system and then have the 'Location' response header have value '/generated-files/foo.txt' and it will be served up.
Note that anything under '/generated-files' is publicly accessible. If you didn't want this and wanted it to be private and so only returnable via the specific request which generated the 'Location' response header, you need to add mod_rewrite magic that blocks access to that URL except for an internally generated sub request. That from memory needs to be something like:
RewriteCond %{IS_SUBREQ} false
RewriteRule ^/generated-files/ - [F]
Related
We have a situation where ideally we would like a user to access a page on our site at a URL such as https://example.com/path/to/page. However, the HTML to render that page is sitting on an entirely different server (S3 to be exact) that we have control over, and we would like to render that page for that URL without redirecting (i.e. changing the URL itself).
I took a brief look at the Apache mod_proxy module, but it doesn't seem to do the job as we just get 500 or 404 errors. Here is an example entry from our .htaccess:
<IfModule mod_proxy.c>
RewriteRule "/path/to/page/(.*)$" "https://bucketname.s3-website-eu-west-1.amazonaws.com/path/to/page/$1" [P]
</IfModule>
Any help or a pointer in the right direction would be appreciated.
Most likely you stumble over the fact that you are using an absolute path inside a dynamic cohnfiguration files RewriteRule. Have a try with that instead:
RewriteEngine on
RewriteRule "/?path/to/page/(.*)$" "https://bucketname.s3-website-eu-west-1.amazonaws.com/path/to/page/$1" [P]
That slightly modified will work in dynamic configuration files and in the real http servers host configuration.
But as mentioned in the comment I wonder why you should not be able to use the proxy module directly to simplify things. You'd have to do that in in http servers host configuration though, this is not possible in dynamic configuration files:
ProxyRequests off
ProxyPass "/path/to/page/" "https://bucketname.s3-website-eu-west-1.amazonaws.com/path/to/page/"
ProxyPassReverse "/path/to/page/" "https://bucketname.s3-website-eu-west-1.amazonaws.com/path/to/page/"
And a general hint: you should always prefer to place such rules inside the http servers host configuration instead of using dynamic configuration files (".htaccess"). Those files are notoriously error prone, hard to debug and they really slow down the server. They are only provided as a last option for situations where you do not have control over the host configuration (read: really cheap hosting service providers) or if you have an application that relies on writing its own rewrite rules (which is an obvious security nightmare).
I'm looking for an easy way to blacklist IP addresses in Apache 2.4.x. My web site logs ip addresses that tried illegal operations into a text file. I would like to use this text file within Apache to deny all access to all vhosts to this ip list. What would be the best way (easiest and least resource consuming way) ?
Found this but this is only for 2.2.. Not sure how this applies to 2.4..
Cheers.
edit: this is a windows x64 box running apache x64
#vastlysuperiorman called it right, csf/lfd is the best at this. Unfortunately, they only run on linux.
This free utility promises to provide the same functionality: dynamically monitor access attempts and auto-block IP addresses. You can unblock with a command, in case of false positives. Certainly worth a short.
An alternative could be to create a VM (if your platform supports virtualization) deploy a very small spec linux box, and use that as a proxy. This should be easy to implement. BTW, why not just use linux? .. :-)
(this should have been a comment on #vastlysuperiorman's post, but I don't have enough SO reps to comment on the post of others)
Edited to suggest a possible apache 2.4 based solution:
To translate ACL directives between the 2.2 and 2.4 in apache
2.2 Syntax
order Deny,Allow
include conf/IPList.conf
Allow from all
2.4 Syntax
DocumentRoot /some/local/dir
<Directory /some/local/dir/>
<RequireAll>
Require all granted
Include conf/IPList.conf
</RequireAll>
</Directory>
#this will also work
<Location />
<RequireAll>
Require all granted
Include conf/IPList.conf
</RequireAll>
</Directory>
# conf/IPLIst.com is actually in /etc/apache2/conf/IPList.conf
# (ie, paths are relative to where apache is installed.
# I guess you can also use the full path to the list.
And inside conf/IPList.conf, you will have individual lines with entries like the following
Require not ip 10.10.1.23
Require not ip 192.168.22.199
Require not ip 10.20.70.100
Using mod-rewrite and a list of IPs for banning
For a redirect-to-another-page to work, you need to keep the RewriteRule outside the base URL you are guarding.
For instance, the redirect would not work under a Directory directive on DocumentRoot or a Location directive on '/', because the ban affects the status page we want to display.
So, best to keep this outside a Directory or Location directive, or link to a status page on another unprotected web server.
#Required set of rewrite rules
RewriteEngine on
RewriteMap hosts-deny txt:/etc/apache/banned-hosts
RewriteCond ${hosts-deny:%{REMOTE_ADDR}|NOT-FOUND} !=NOT-FOUND [OR]
RewriteCond ${hosts-deny:%{REMOTE_HOST}|NOT-FOUND} !=NOT-FOUND
RewriteRule ^ /why-am-i-banned.html
## inside our banned hosts file, we have:
## /etc/apache2/banned-hosts (maintain the format .. its not just a plain text file)
##
193.102.180.41 -
192.168.111.45 -
www.example.com -
www.sumwia.net -
# inside our status page, could be html as below or a plain text file with '.txt' extension
#/var/www/html/why-am-i-banned.html
#
<html xmlns="http://www.w3.org/1999/xhtml">
<head>
<title>Why is my IP banned?</title>
</head>
<body>
<h2>Why is my IP address banned?</h2>
<p>
To manage spammers and for other security needs, our server automatically blocks
suspicious IP address. If however you reckon your IP address has been blocked
wrongfully, please contact us.
</p>
</body>
</html>
And of course, you can parse your log files and populate conf/IPList.conf or /etc/apache2/banned-hosts as appropriate ..
As a short term solution
An alternative that will allow you to use the 2.2 syntax, is to install mod_access_compat module and continue using your deprecated 2.2 style 'Deny,Allow' directives. But this is only advisable as a short-term solution since that module is just there to aid transition, and would probably go away in future versions of apache 2.4
I too have not seen a good alternative for blocking access dynamically from within Apache itself. There are "hacky" ways: you could set an environment variable to contain a list of IPs and then use the module with ${REMOTE_ADDR} and the env function, but that's a stretch. Details on the Expression Parser
However, I have used several light weight modules that are helpful in protecting your Apache server.
ConfigServer Firewall (CSF/LFD) is a great solution for linux systems. It provides a simple method for managing iptables, and can be set up to do brute force detection and blocking. Info here
EDIT:
Add the following line to /etc/csf/csf.deny to include your custom IP block list:
Include /var/www/example.deny
Alternately, update your script to append IP addresses to csf.deny either directly:
echo $badIP >> /etc/csf/csf.deny
or using the CSF command line option (preferred):
csf -d 10.20.30.40
CSF readme here
mod_security is one of my favorite Apache/nginx modules. It detects dangerous GET and POST requests and blocks access accordingly. When set up properly, it will trigger CSF to block the IP addresses that frequently violate rules. Details here
Is it possible to browse a directory on an Apache server with a running website?
Example: I have myserver/mydirectory with an index.html and 'test.txt`. Can I list somehow those files assuming browsing is enabled?
there are a couple of things you can try:
in httpd.conf find the line that begins with "DirectoryIndex" and replace it with "DirectoryIndex disabled" this way apache will not server default files like index.html and just list files. however you can explicitly request it if you want.
if default document setting is important to you, you can also configure apache to listen to another port and setup a virtual host on that port and do the same thing with "DirectoryIndex" for virtual host, this way you have two ports , one that serves default documents and one that only list files.
if you want to use only one port for this , you can try no. 2 option and then set a proxy that sends all requests that begin for example with /list/ to the other virtual host, this way you work on one port and if you want list of files instead of writing "/myserver/mydirectory" you request "/list/myserver/mydirectory"
hope it helps.
The DirectoryIndex directive in the Apache configuration tells Apache which index files to look for. Default settings includes index.html, so if you have such a file in your directory, this is the file that Apache will serve if you enter the site without specifying a specific file (this you properly already know, but included for completeness).
To enable directory listing in Apache, have a look at the Options Indexes option. For example in your case (assuming your website is located in /var/www/website:
<Directory /var/www/website/mydirectory>
Options Indexes FollowSymLinks
</Directory>
This will, however, only enable listing of files if Apache do not find an index file. A solution is therefore either to delete (or rename index.html), or to use a website scripting language like PHP to enable directory listing (For this, Google is your friend :-)
Given
ProxyPassMatch ^/(.*\.php(/.*)?)$ fcgi://127.0.0.1:9000/var/www/$1
how can one prevent malicious code execution when a fake image is uploaded in a folder which is then called via
http://www.foo.bar/uploads/malicious.jpg/fake.php
If I understand correctly, the request above will let Apache pass it to PHP-FPM which will execute /uploads/malicious.jpg.
I know I could add an .htaccess file in the uploads folder that removes the ProxyPassMatch, but this is something my customers don't know and they could end up being compromised.
There's a new setting in php-fpm since php 5.3.9, 'security.limit_extensions', that limits which files php-fpm will execute. The default is '.php', so the 'malicious.jpg' would not be executed.
I'm starting up a new web-site, and I'm having difficulties enforcing my desired file/folder organization:
For argument's sake, let's say that my website will be hosted at:
http://mywebsite.com/
I'd like (have set up) Apache's Virtual Host to map http://mywebsite.com/ to the /fileserver/mywebsite_com/www folder.
The problem arises when I've decided that I'd like to put a few files (favicon.ico and robots.txt) into a folder that is ABOVE the /www that Apache is mounting the http://mywebsite.com/ into
robots.txt+favicon.ico go into => /fileserver/files/mywebsite_com/stuff
So, when people go to http://mywebsite.com/robots.txt, Apache would be serving them the file from /fileserver/mywebsite_com/stuff/robots.txt
I've tried to setup a redirection via mod_rewrite, but alas:
RewriteRule ^(robots\.txt|favicon\.ico)$ ../stuff/$1 [L]
did me no good, because basically I was telling apache to serve something that is above it's mounted root.
Is it somehow possible to achieve the desired functionality by setting up Apache's (2.2.9) Virtual Hosts differently, or defining a RewriteMap of some kind that would rewrite the URLs in question not into other URLs, but into system file paths instead?
If not, what would be the preffered course of action for the desired organization (if any)?
I know that I can access the before mentioned files via PHP and then stream them - say with readfile(..), but I'd like to have Apache do as much work as necessary - it's bound to be faster than doing I/O through PHP.
Thanks a lot, this has deprived me of hours of constructive work already. Not to mention poor Apache getting restarted every few minutes. Think of the poor Apache :)
It seems you are set to using a RewriteRule. However, I suggest you use an Alias:
Alias /robots.txt /fileserver/files/mywebsite_com/stuff/robots.txt
Additionally, you will have to tell Apache about the restrictions on that file. If you have more than one file treated this way, do it for the complete directory:
<Directory /fileserver/files/mywebsite_com/stuff>
Order allow,deny
Allow from all
</Directory>
Can you use symlinks?
ln -s /fileserver/files/mywebsite_com/stuff/robots.txt /fileserver/files/mywebsite_com/stuff/favicon.ico /fileserver/mywebsite_com/www/
(ln is like cp, but creates symlinks instead of copies with -s.)