all!
On our server we have got a directory 'files' with materials for students and teachers:
http://sample.ru/files/4students/
http://sample.ru/files/4teacherszzz/
http://sample.ru/files/markszzz/
...
Each directory, when you type it into your web-browser shows you 'Index of /files/4students/' page with list of files/subdirs.
But, we've got problem: when the student types 'http://sample.ru/files/' he can see all other folders! But students must not see teachers files (in folders 4teacherzzz and markszzz).
So, we've put blank index.html into /files/ . And by typing this url, student can't see other folders.
Now, this is question: can anyone student (he only knows 'http://sample.ru /files/4students/' url) get list of urls in 'http://sample.ru /files/' (may be by special typing '/files/4students/../' or so on)? and how? and how to deny this?
The problem is that we must leave that folders system --- it is more convenient for teachers (they only copy files to a folder on server and files automatically appears in "Index of ..." page)
Thanks for your replies!
Depending on your brand of web server - no. Most brands of normal origin servers (like Apache, IIS) will be particularly careful about collapsing any attempts at ../.. and similar naughtiness. (Note that some servers optimized for proxy/caching and similar non-origin use may well NOT do that).
But regardless you want to read the documentation very carefully. And note that most servers, like Apache and IIS, have specific directives for just this particular problem.
Have a look at "Indexes" - .i.e.
<Directory /something/>
option -Indexes
...
note the minus - will very explicitly disable indexes (Be careful with symbolic links though - or also use -FollowSymLinks ). http://httpd.apache.org/docs/2.0/mod/core.html has the full story.
Check specifically the section 'Directory Directive' and note the text:
Be careful with the directory-path arguments: They have to literally
match the filesystem path which Apache uses to access the files.
Directives applied to a particular will not apply to files
accessed from that same directory via a different path, such as via
different symbolic links.
and the example just below it. So you typically will want to lock down (-Indexes) on the lower level and then do a directory(match) one down for the students where you do allow Indexes.
Your trick of adding an index.html usually works - but note that this is just because mod_autoindex snarfs it in a timely fashion. This is not as guaranteed as above - in particular when you have MultiViews or other localisation/internationalization and similar (language/charset) variants in your server. See http://httpd.apache.org/docs/2.2/mod/mod_autoindex.html.
Hope this helps,
Dw.
Related
Currently I have a website setup with apache on ubuntu.
I'd like to customize the "Index of" file (the one that comes up when there is no index.html/php/etc as defined under DirectoryIndex) to remove the line of text that says "Apache/x.x.x (Ubuntu) Server at domain Port #".
I'm not sure where this file is located (in order to locate it and edit it). Additionally, I'd like to know how to have Apache point towards a different file (e.g. comment out the indexof.file and have it point towards a customized indexof2.file).
Image of the "Index of" file that I'm talking about:
Summary of my questions:
1) Where is the "Index of" file located
2) How can I direct Apache to use a different "Index of" file (when there are no index.html/php/etc in that directory).
The automatic index Apache generates is handled by this module (available by default): https://httpd.apache.org/docs/2.4/mod/mod_autoindex.html
The customisation options would be way to long to explain here but they definitely exist, and are detailed in the link above.
More specifically, you want to have a look at the styling options from this chapter: https://httpd.apache.org/docs/2.4/mod/mod_autoindex.html#indexoptions
You can also find "premade" option packages like this one: http://adamwhitcroft.com/apaxy/
It's probably worth looking at this to understand how it works and get a good example of what's possible.
In this specific example, there's a lot of IndexOptions involved, as you can see in https://github.com/AdamWhitcroft/apaxy/blob/master/apaxy/htaccess.txt
Definitely not something easy to implement as this not just a file to edit, like you would edit a template, but there's room for customisation.
Thanks to Capsule's response, I was able to research in a more meaningful manner and I stumbled upon a nice guide. For anybody who may visit this stackoverflow question of mine in future and is looking for a guide on editting the "Index of" directory page, the guide may help you too: https://perishablepress.com/better-default-directory-views-with-htaccess/
Hope someone will give me a hand with this problem I have. So here it goes.
There is a website with integrated vBulletin forum inside. The forum is accessible through
https://site.de/forum domain. The main site itself has many other domains based on locale. That is to say, there is a https://site.ch, https://site.it, https://site.at, etc (each one is in corresponding language).
Now there is a need to have this forum under at least 2 of this additional domains. I mean, there should be https://site.ch/forum domain, wich will contain the same forum, but with some differences in style and, of course, will have working inside-forum links with it's own domain (site.ch). The whole system is to be SEO-ed also.
So now my question is how to achieve this? I know there are some sort of plugins to manage multi-domain access, but they are not supported and are still in beta version.
At first, how to setup the forum to work under multiple domains?
And then, maybe I need to manually change some code to set the $vbulletin->options['bburl'] that is used to generate the links inside forum?
And the last one, how do I make all this search engine optimized??
You're asking numerous questions, you might get better results if you created a separate question for each of:
1) How to use one forum directory for multiple domains? (with the vbulletin tag and the tag for the web server you are using)
2) How to set the language based on the current domain in vbulletin? (with the vbulletin tag and one or more of these tags: localized, locale, multi-language, multilanguage)
3) Best practices for duplicate content presented in multiple languages on multiple domains (with the seo and vbulletin tags)
Some Answers:
1) If you're using the apache web server, you could add something like this to your httpd.conf file:
Alias /forums /var/www/...xxx.../forum_directory // use the path to your forum directory, no trailing slash
<Directory /var/www/...xxx.../forum_directory>
Order allow,deny
Allow from all
</Directory>
Then in the vbulletin ACP, change the setting for your basepath URL to "No":
Admin Control Panel -> Site Name / URL / Contact Details -> Always use Forum URL as Base Path
2) There are a few plugins that detect the language used by the browser and set vBulletin to use that language:
Language Detection
Set forum-language automatic to browser-language for first-time-visitors
3) SEO covers many things, but to deal with having duplicate content on multiple domains you can look at the Google Webmaster Central Blog.
This posting is helpful:
Working with multi-regional websites
A section from the post: Dealing with duplicate content on global websites
Websites that provide content for different regions and in different languages sometimes create content that is the same or similar but available on different URLs. This is generally not a problem as long as the content is for different users in different countries. While we strongly recommend that you provide unique content for each different group of users, we understand that this may not always be possible for all pages and variations from the start. There is generally no need to "hide" the duplicates by disallowing crawling in a robots.txt file or by using a "noindex" robots meta tag. However, if you're providing the same content to the same users on different URLs (for instance, if both "example.de/" and "example.com/de/" show German language content for users in Germany), it would make sense to choose a preferred version and to redirect (or use the "rel=canonical" link element) appropriately.
I don't have anything on the other search engines.
Is it possible to replace a url name like http://mysite.com/sub/ with http://sub.mysite.com using htaccess?
I don't want to make a redirect rather than just to map a sub-directory address to a sub-domain address. So when a person types an address like http//sub.mysite.com/image.jpg this address remains in the browser but it reads the content from http//mysite.com/sub/image.jpg
Yes it is possible but you should have root access to the server to start with, you will need to also make some DNS record changes so ensure you have this access also before starting.
I have used both methods previously and they both work, however I found using folders was the winner at the end of the day for our usage, this simplified things significantly for us and we didn't have to worry about changing linkages in scripts, e.g. from http://www.my-site.com/images to http://images.my-site.com depending on the code structure being used.
Instead of typing these long instructions out I am going to give you 2 references that have slightly different approaches depending on if you have a physical folder to use or if it is a variable in the URL. They say it probably as well as I can anyway ;-)
Physical folder method :: URL variable method
I hope this helps you
I work for an ISP and I have a server with thousands of users 10MB of free storage. They get this free storage with every e-mail account they have with us. An example of a users storage address: http://users.example.com/~username/
One problem I can see is scanning the server for user names to see what accounts are available, basically getting a list of all our customers valid e-mail addresses. This would be very, very bad.
So I'm wanting to redirect to our homepage if someone comes across a users account that is empty (I'd say 90% of them are completely empty). I also do not want to simply -Indexes them and use a custom 403 because the few customers that do use them, want +Indexes.
I know I can always just tell the customers to put a htaccess file in their directory with Options +indexes if they want directory listing, but that's a last resort.
How can I make it pretty much impossible to tell what accounts are on the server but not in use at all?
I can't see a way to do this with Apache rules alone - and even if, it would be pretty expensive, scanning for files on every incoming request.
I would build a script that puts the appropriate .htaccess file, redirecting to your home page, into every completely empty account.
Maybe run it hourly, and make users aware that if they populate a directory for the first time, it may take up to an hour until their changes take place? I think that would be a reasonable time frame.
I have a rather complicated situation where I run a personal blog where every Friday and Sunday, I will post up music on the blog by uploading the mp3s into a folder, where a Flash mp3 player accesses it and plays it for the world.
Recently, some website called Dizzler, which is like a spider for mp3 files (Like the ones I host on my server!) and lets people play them via their own proprietary player. Now, I normally wouldn't be against other people using my server for their own gain but this recently got out of hand. In the last week of December, they managed to rack up 100k hits on one song and used up 6GB of bandwidth.
In that last week of December, I edited my .htaccess file to remove access to mp3s on my server without taking away access to my mp3s (So "deny all" isn't an option!) and I used this code:
RewriteEngine on
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^(www\.)?mydomain.com [NC]
RewriteRule \.(mp3)$ - [NC,F]
Options -Indexes
It worked pretty well with one exception - it broke every Wordpress installation on my server. What I mean is that outside of the index page, if you clicked on an entry in Wordpress, it wouldn't be able to find it. My host's solution was to add "RewriteEngine on" to every .htaccess file for every installation and in the root of the web server root.
That was a great fix and all the pages work again - but it is no longer blocking my mp3 files in that folder.
What can I do?
PS. For clarification, the code above is in an .htaccess file in the folder containing the mp3s. Hope that helps!
Huge thanks to Vinko Vrsalovic for all the help, definitely helped point me in the right direction, currently using the following code:
SetEnvIfNoCase Referer www\.dizzler\.com bad_referer
SetEnvIfNoCase Referer ".*(dizzler|beemp3|skreemr).*" BlockedReferer
SetEnvIfNoCase REMOTE_ADDR ".*(220.181.38.82|202.108.23.172|66.232.150.219).*" BlockedAddress
# deny any matches from above and send a 403 denied
<FilesMatch "\.mp3$">
order deny,allow
deny from env=bad_referer
deny from env=BlockedReferer
deny from env=BlockedAddress
</FilesMatch>
Testing it out tonight, will report back tomorrow if it works!
I'm posting this as another answer instead of adding this to my other post because it approaches the problem from a different angle. Here I am assuming that all your mp3s are in the same folder.
The problem you are facing is due to sloppy coding on the part of whoever made the media-player thing that wordpress uses. What happens is that the player runs on the visiting user's machine, and actually downloads the mp3 and plays it locally. The problem arises because the player does not provide any useful headers at all: the useragent is that of your browser, the referrer is blank, etc. As such, it is completely impossible to tell if the request is coming from the player, or from a browser that clicked your link in an audio search engine. Really, the only way to protect your mp3s from being indexed is to change the link as often as possible.
Which is precisely the plan. In a nutshell, here is what we are going to do:
change the path to your mp3s. This stays SECRET.
create a script to proxy for the mp3s, which requires a valid key which changes every hour
change all your uses of the mp3 player to use the mp3 proxy script but with a placeholder key
create a script to proxy for your webserver, which replaces the key placeholder with the actual key
use .htaccess to rewrite all requests to your server to use the webserver proxy script.
The upshot of all of this is that your user experience will not change, but if a crawler crawls your links, they will only be valid until midnight of that day, at which point requests to that url will result in a snippy message (or even an mp3 of you asking them to please not download your stuff).
Ready? OK, lets go!
Step 1:
First things first, make sure you renamed your mp3s folder! This will break all existing links (and failing to do this will mean all the links already crawled will remain valid). Secondly, create a robots.txt file to stop google and other search engines from indexing your mp3s folder.
Now, create a file in your root directory called mp3serve.php with the following contents:
<?php
/* This script checks 'key', and if it's valid, serves the mp3
* A valid key is defined as the md5 of the current date in
* yyyy-mm-dd-hh format concatenated with the string
* "Hello there :)"
*
* The key can be anything so long as we are consistent in this
* and the viewer proxy thing we're going to make.
*/
// edit this variable to reflect your server
$music_folder = "/new/path/to/mp3s/";
// get inputs of 'file' and 'key'
// 'file' should be the filename of the mp3 WITHOUT the extension
$file = $_GET['file'];
$key = $_GET['key'];
// get todays date
$date = date("Y-m-d-H");
// calculate the valid key
$valid = md5($date+"Hello there :)");
if ($key == $valid)
{
// if the key is valid, get the song in the path:
print(file_get_contents("$music_folder/$file.mp3"));
}
else
{
// if the key is invalid, print an admonishing message:
print("Please don't try to download my songs, poopface.");
}
?>
What this does is it takes the filename of an MP3 and a key of some kind, and serves the file contents if the key is valid. Note that this script:
makes no checks at all that $file points to what you expect it to, other than the fact that it tries to make sure it will only ever return mp3 files.
does not return valid headers for mp3 files - they'll render as text in a browser. This is easy to fix but the correct header eludes me for the moment... and anyway the wordpress mp3 player doesn't care, so it's all good :)
Step 2:
Now for the slightly tricky part: we have to rewrite the links dynamically. The easiest way to do this is to write a "local-proxy" thing, which really is a lot easier than it sounds. What we will do is write a script that gets what your page would have outputted and corrects the mp3 links. In my example we will edit all of your articles with mp3s in them, but if you want to get fancy this is not completely necessary.
First, edit all of your articles with mp3-players in them. You could automate this, but unless WP has a "find/replace in all articles" function I would advise against it for the sole reason that you might screw up and destroy your articles. In any case, edit them and replace the mp3 links in the players from
/path/to/mp3s/<filename>.mp3
to
/mp3serve.php?file=<filename>&key=[{mp3_file_key}]
Now, create another php script in your root directory called proxyviewer.php with the following contents:
<?php
/*
* The purpose of this file is to act as a proxy in which we can dynamically
* rewrite the page contents. Specifically, we want to get the page that the
* user WOULD have seen, and replace all instances of our key placeholder
* with the actual correct key
*/
// get the requested path
$request = $_GET['req'];
// get what the source output WOULD have been
// NOTE: depending on your server's config, you -might- have to
// replace 'localhost' with your actual site-name. This will
// however increase page-load times. If localhost doesn't work
// ask your host how to access your site locally. To clarify,
// maybe show him this file.
$source = file_get_contents("http://localhost/$request");
// The reason we need to pass the request through apache (i.e. use the whole
// "http://localhost/" thing is because we need the PHP to be rendered, and
// I can't think of another way to do that using the original request uri
// calculate the correct key
$key = md5(date("Y-m-d-H")+"Hello there :)");
// replace all instances of "[{mp3_file_key}]" with the key
$output = str_replace("[{mp3_file_key}]",$key,$source);
//output the source
print($output);
?>
Step 3:
Now for the last part: set up your .htaccess file to redirect all requests from
http://yoursite/some/request/here
to
http://yoursite/proxyviewer.php?req=some/request/here
Unfortunately I'm really not good with .htaccess files so I won't be able to give you the exact code, but I imagine it shouldn't be too hard to do.
Congrats, you're done!
Disclaimer:
Please note that the code in here is not production-level code. First of all, I haven't tested it at all - although unless there's a typo somewhere they should all work, I would advise you to look through them carefully before going live with them. I have been fairly careful not to allow any Bad Things to happen, but it doesn't do any serious checking, and it's the wee hours of the morning here so I may have overlooked something.
FilesMatch is the directive you need:
<FilesMatch "\.mp3$">
Order Allow, Deny
Allow from localhost #Or the address of your player
Deny From All
</FilesMatch>
I think my other answer is much better, but this is still worth considering
Reading through some of the answers, I am struck by another idea: Have your page log the IP addresses of all visitors to your site within the last two (or however many) hours. Then, create a job that gets run ever 2 seconds or so which rewrites your .htaccess file to only allow access to mp3 files to those IP addresses in the log.
That way, only those users who have been served a page from your website in the last two hours will have access to your music. This, for the vast majority of people finding your mp3s in audio search-engines, will prove to be false.