Good evening everyone! Thank you for opening this post.
I currently bought myself the ProCDN from MediaTemple (basically EdgeCast) and have setup a CDN where now I go to cdn-small.DOMAIN.com (or cdn-large.DOMAIN.com) it loads the normal website just fine...
However, I'm not sure which one to use.. Would I use this for the whole complete site to optimize, or use the links to add one by one for each script/stylesheet based on file size? (e.g. All JS/CSS will have the cdn-small while anything larger such as 300kb will have the cdn-large link)
And to say, if the correct way is to load the whole site as one link (e.g. everything is linked normally like js/jquery.js instead of a full link like https://cdn-small.domain.com/js/jquery.js).. Would I set a redirect from DOMAIN.com to cdn-small.DOMAIN.com for the best loading and that they only need to type in the domain not the full sub-CDN-domain?
Apologize if this isn't making sense or anything, but trying to do my best. To put it much more simple terms again is that I'm trying to find the best way to use my cdn-small/cdn-large for my website by having the user enter in the domain (https:// or http://) normally to serve my content as fast as possible near the user.
Kindly appreciate your time for reading this and wish you all a positive weekend.
Here is my live site if it even matters or want to experiement; http://bit.ly/1eGCShX
I'm working on a site which shows different products for different countries. The current url scheme I'm using is "index.php?country=US" for the main page, and "product.php?country=US&id=1234" to show a product from an specific country.
I'm planning now to implement url rewrite to use cleaner urls. The idea would be using each country as subdomain, and product id as a page. Something like this:
us.example.com/1234 -> product.php?country=US&id=1234
I have full control of my dns records and web server, and currently have set a * A record to point to my IP in order to receive *.example.com requests. This seems to work ok.
Now my question is what other things I'd need to take care of. Is it right to assume that just adding a .htaccess would be enough to handle all requests? Do I need to add VirtualHost to each subdomain I use as well? Would anything else be needed or avoided as well?
I'm basically trying to figure out what the simplest and correct way of designing this would be best.
The data you need to process the country is already in the request URL (from the hostname). Moving this to a GET variable introduces additional complications (how do you deal with POSTs).
You don't need seperate vhosts unless the domains have different SSL certs.
I've been asked to create a solution where people log in and are able to upload and download off of our work server. So John uploads a photo, and Jen can download it, for example. They also have to authenticate themselves.
Can someone give me a rough overview of how to implement this? I'm familiar enough with MySQL, C#, and JavaScript.
The rough overview
This should just be a matter of planning out the pieces.
at the very top of the page, put some code that checks if a user is logged in. If not, show a login form (or redirect to...). If they are logged in, show the rest of the page. If not, you'll need some logic to show a form, and then check it once it's submitted for authentication, and set a SESSION cookie or something similar.
Once the user is logged in, on the homepage, you might have an file-upload form and a listing of existing files. How you would style would depend on how many files you might expect to have. To keep things extremely simple, you could simple iterate through whatever files are in the upload directory. If you expect many more files than that, you may consider using a db.
Handle a file upload by sanitizing filenames (checking for filetype/filesize if you want to limit those) and putting the file into the directory.
Force the users to download the files (instead of having the browser decide what to do with them) for security purposes. Implementing this on certain filetypes may also be acceptable.
Other thoughts
You probably would not want the users to be able to excecute any files, so keeping the file directory hidden would be a good idea.
Keeping track of who uploaded and downloaded what is also doable, but would add another layer of complication to the script.
I have a rather complicated situation where I run a personal blog where every Friday and Sunday, I will post up music on the blog by uploading the mp3s into a folder, where a Flash mp3 player accesses it and plays it for the world.
Recently, some website called Dizzler, which is like a spider for mp3 files (Like the ones I host on my server!) and lets people play them via their own proprietary player. Now, I normally wouldn't be against other people using my server for their own gain but this recently got out of hand. In the last week of December, they managed to rack up 100k hits on one song and used up 6GB of bandwidth.
In that last week of December, I edited my .htaccess file to remove access to mp3s on my server without taking away access to my mp3s (So "deny all" isn't an option!) and I used this code:
RewriteEngine on
RewriteCond %{HTTP_REFERER} .
RewriteCond %{HTTP_REFERER} !^(www\.)?mydomain.com [NC]
RewriteRule \.(mp3)$ - [NC,F]
Options -Indexes
It worked pretty well with one exception - it broke every Wordpress installation on my server. What I mean is that outside of the index page, if you clicked on an entry in Wordpress, it wouldn't be able to find it. My host's solution was to add "RewriteEngine on" to every .htaccess file for every installation and in the root of the web server root.
That was a great fix and all the pages work again - but it is no longer blocking my mp3 files in that folder.
What can I do?
PS. For clarification, the code above is in an .htaccess file in the folder containing the mp3s. Hope that helps!
Huge thanks to Vinko Vrsalovic for all the help, definitely helped point me in the right direction, currently using the following code:
SetEnvIfNoCase Referer www\.dizzler\.com bad_referer
SetEnvIfNoCase Referer ".*(dizzler|beemp3|skreemr).*" BlockedReferer
SetEnvIfNoCase REMOTE_ADDR ".*(220.181.38.82|202.108.23.172|66.232.150.219).*" BlockedAddress
# deny any matches from above and send a 403 denied
<FilesMatch "\.mp3$">
order deny,allow
deny from env=bad_referer
deny from env=BlockedReferer
deny from env=BlockedAddress
</FilesMatch>
Testing it out tonight, will report back tomorrow if it works!
I'm posting this as another answer instead of adding this to my other post because it approaches the problem from a different angle. Here I am assuming that all your mp3s are in the same folder.
The problem you are facing is due to sloppy coding on the part of whoever made the media-player thing that wordpress uses. What happens is that the player runs on the visiting user's machine, and actually downloads the mp3 and plays it locally. The problem arises because the player does not provide any useful headers at all: the useragent is that of your browser, the referrer is blank, etc. As such, it is completely impossible to tell if the request is coming from the player, or from a browser that clicked your link in an audio search engine. Really, the only way to protect your mp3s from being indexed is to change the link as often as possible.
Which is precisely the plan. In a nutshell, here is what we are going to do:
change the path to your mp3s. This stays SECRET.
create a script to proxy for the mp3s, which requires a valid key which changes every hour
change all your uses of the mp3 player to use the mp3 proxy script but with a placeholder key
create a script to proxy for your webserver, which replaces the key placeholder with the actual key
use .htaccess to rewrite all requests to your server to use the webserver proxy script.
The upshot of all of this is that your user experience will not change, but if a crawler crawls your links, they will only be valid until midnight of that day, at which point requests to that url will result in a snippy message (or even an mp3 of you asking them to please not download your stuff).
Ready? OK, lets go!
Step 1:
First things first, make sure you renamed your mp3s folder! This will break all existing links (and failing to do this will mean all the links already crawled will remain valid). Secondly, create a robots.txt file to stop google and other search engines from indexing your mp3s folder.
Now, create a file in your root directory called mp3serve.php with the following contents:
<?php
/* This script checks 'key', and if it's valid, serves the mp3
* A valid key is defined as the md5 of the current date in
* yyyy-mm-dd-hh format concatenated with the string
* "Hello there :)"
*
* The key can be anything so long as we are consistent in this
* and the viewer proxy thing we're going to make.
*/
// edit this variable to reflect your server
$music_folder = "/new/path/to/mp3s/";
// get inputs of 'file' and 'key'
// 'file' should be the filename of the mp3 WITHOUT the extension
$file = $_GET['file'];
$key = $_GET['key'];
// get todays date
$date = date("Y-m-d-H");
// calculate the valid key
$valid = md5($date+"Hello there :)");
if ($key == $valid)
{
// if the key is valid, get the song in the path:
print(file_get_contents("$music_folder/$file.mp3"));
}
else
{
// if the key is invalid, print an admonishing message:
print("Please don't try to download my songs, poopface.");
}
?>
What this does is it takes the filename of an MP3 and a key of some kind, and serves the file contents if the key is valid. Note that this script:
makes no checks at all that $file points to what you expect it to, other than the fact that it tries to make sure it will only ever return mp3 files.
does not return valid headers for mp3 files - they'll render as text in a browser. This is easy to fix but the correct header eludes me for the moment... and anyway the wordpress mp3 player doesn't care, so it's all good :)
Step 2:
Now for the slightly tricky part: we have to rewrite the links dynamically. The easiest way to do this is to write a "local-proxy" thing, which really is a lot easier than it sounds. What we will do is write a script that gets what your page would have outputted and corrects the mp3 links. In my example we will edit all of your articles with mp3s in them, but if you want to get fancy this is not completely necessary.
First, edit all of your articles with mp3-players in them. You could automate this, but unless WP has a "find/replace in all articles" function I would advise against it for the sole reason that you might screw up and destroy your articles. In any case, edit them and replace the mp3 links in the players from
/path/to/mp3s/<filename>.mp3
to
/mp3serve.php?file=<filename>&key=[{mp3_file_key}]
Now, create another php script in your root directory called proxyviewer.php with the following contents:
<?php
/*
* The purpose of this file is to act as a proxy in which we can dynamically
* rewrite the page contents. Specifically, we want to get the page that the
* user WOULD have seen, and replace all instances of our key placeholder
* with the actual correct key
*/
// get the requested path
$request = $_GET['req'];
// get what the source output WOULD have been
// NOTE: depending on your server's config, you -might- have to
// replace 'localhost' with your actual site-name. This will
// however increase page-load times. If localhost doesn't work
// ask your host how to access your site locally. To clarify,
// maybe show him this file.
$source = file_get_contents("http://localhost/$request");
// The reason we need to pass the request through apache (i.e. use the whole
// "http://localhost/" thing is because we need the PHP to be rendered, and
// I can't think of another way to do that using the original request uri
// calculate the correct key
$key = md5(date("Y-m-d-H")+"Hello there :)");
// replace all instances of "[{mp3_file_key}]" with the key
$output = str_replace("[{mp3_file_key}]",$key,$source);
//output the source
print($output);
?>
Step 3:
Now for the last part: set up your .htaccess file to redirect all requests from
http://yoursite/some/request/here
to
http://yoursite/proxyviewer.php?req=some/request/here
Unfortunately I'm really not good with .htaccess files so I won't be able to give you the exact code, but I imagine it shouldn't be too hard to do.
Congrats, you're done!
Disclaimer:
Please note that the code in here is not production-level code. First of all, I haven't tested it at all - although unless there's a typo somewhere they should all work, I would advise you to look through them carefully before going live with them. I have been fairly careful not to allow any Bad Things to happen, but it doesn't do any serious checking, and it's the wee hours of the morning here so I may have overlooked something.
FilesMatch is the directive you need:
<FilesMatch "\.mp3$">
Order Allow, Deny
Allow from localhost #Or the address of your player
Deny From All
</FilesMatch>
I think my other answer is much better, but this is still worth considering
Reading through some of the answers, I am struck by another idea: Have your page log the IP addresses of all visitors to your site within the last two (or however many) hours. Then, create a job that gets run ever 2 seconds or so which rewrites your .htaccess file to only allow access to mp3 files to those IP addresses in the log.
That way, only those users who have been served a page from your website in the last two hours will have access to your music. This, for the vast majority of people finding your mp3s in audio search-engines, will prove to be false.
I'm making something that requires me to pass information from one domain to a subdomain. The subdomain would be in an iframe on the domain. I know I can use cookies, sessions, or a database. But I'm trying to save processing time so I thought about using the referrer. I know that some people turn the referrer off for some reason, but exactly just how many. If they do, this won't work for them.
Oh and I can't use the URL to pass information.
I'd say < 0.001 % of all Internet users have ever heard about referrers. Even a smaller portion of them will be willing to switch them off. Even a smaller number of them will be able to.