Using Apache and mod_rewrite I can rewrite a complex request to a simple filename, eg:
RewriteRule ^shortcut/(.*)$ /long/way/around/$1
Can this work in reverse? I want a simple request to be rewritten to an unknown file, but I can identify which file should be served by a unique ID number prefixed to it's filename. I want Apache to "guess" using a regular expression which file to serve based on the ID.
For example:
GET /img/29281.jpg
Directory of /img/:
...
29280-filename-here.jpg
29281-other-filename-here.jpg <-- Apache should serve this one
29282-more-files-here.jpg
...
So, a regular expression rewrite could perhaps be:
^(\d+)\.jpg$ --> ^$1\-[a-zA-Z0-9-_]+.jpg$
How to integrate this into Apache (if it's possible)?
Many thanks for any suggestions.
P.S. Renaming all the filenames to the simple ID number isn't an option in this instance.
I think one of the apache-way is to use mod-rewrite with the RewriteMap directive, and using the :prg mapping of the rewritemap.
This will load at apache start a simple program (may be a perl one as suggested in documentation, a PHP program may get more memleaks & performance problems, IMHO). This program will have to do the last part of the filename guess.
You can find some examples of such programs here (PHP), there (C), and here a Perl basic one.
I'm not enough perl-enabled in my brain to get the perl way of extracting real filename taking the extension and the int identifier but that should be easy and short (which is important for a PRG rewriteMap).
You could use a PHP script as an ErrorDocument for the 404 page.
This script would find the desired file, and send a redirection to the wanted file.
Related
I want to archive an old website which was built with PHP. Its URLs are full of .phps and query strings.
I don't want anything to actually change from the perspective of the visitor -- the URLs should remain the same. The only actual difference is that it will no longer be interactive or dynamic.
I ran wget --recursive to spider the site and grab all the static content. So now I have thousands of files such as page.php?param1=a¶m2=b. I want to serve them up as they were before, so that means they'll mostly have Content-Type: text/html, and the webserver needs to treat ? and & in the URL as literal ? and & in the files it looks up on disk -- in other words it needs to not support query strings.
And ideally I'd like to host it for free.
My first thought was Netlify, but deployment on Netlify fails if any files have ? in their filename. I'm also concerned that I may not be able to tell it that most of these files are to be served as text/html (and one as application/rss+xml) even though there's no clue about that in their filenames.
I then considered https://surge.sh/, but hit exactly the same problems.
I then tried AWS S3. It's not free but it's pretty close. I got further here: I was able to attach metadata to the files I was uploading so each would have the correct content type, and it doesn't mind the files having ? and & in their filenames. However, its webserver interprets ?... as a query string, and it looks up and serves the file without that suffix. I can't find any way to disable query strings.
Did I miss anything -- is there a way to make any of the above hosts act the way I want them to?
Is there another host which will fit the bill?
If all else fails, I'll find a way to transform all the filenames and all the links between the files. I found how to get wget to transform ? to #, which may be good enough. It would be a shame to go this route, however, since then the URLs are all changing.
I found a solution with Netlify.
I added the wget options --adjust-extension and --restrict-file-names=windows.
The --adjust-extension part adds .html at the end of filenames which were served as HTML but didn't already have that extension, so now we have for example index.php.html. This was the simplest way to get Netlify to serve these files as HTML. It may be possible to skip this and manually specify the content types of these files.
The --restrict-file-names=windows alters filenames in a few ways, the most important of which is that it replaces ? with #. This is needed since Netlify doesn't let us deploy files with ? in the name. It's a bit of a hack; this is not really what this option is meant for.
This gives static files with names like myfile.php#param1=value1¶m2=value2.html and myfile.php.html.
I did some cleanup. For example, I needed to adjust a few link and resource paths to be absolute rather than relative due to how Netlify manages presence or lack of trailing slashes.
I wrote a _redirects file to define URL rewriting rules. As the Netlify redirect options documentation shows, we can test for specific query parameters and capture their values. We can use those values in the destinations, and we can specify a 200 code, which makes Netlify handle it as a rewrite rather than a redirection (i.e. the visitor still sees the original URL). An exclamation mark is needed after the 200 code if a "query-string-less" version (such as mypage.php.html) exists, to tell Netlify we are intentionally shadowing.
/mypage.php param1=:param1 param2=:param2 /mypage.php#param1=:param1¶m2=:param2.html 200!
/mypage.php param1=:param1 /mypage.php#param1=:param1.html 200!
/mypage.php param2=:param2 /mypage.php#param2=:param2.html 200!
If not all query parameter combinations are actually used in the dumped files, not all of the redirect lines need to be included of course.
There's no need for a final /mypage.php /mypage.php.html 200 line, since Netlify automatically looks for a file with a .html extension added to the requested URL and serves it if found.
I wrote a _headers file to set the content type of my RSS file:
/rss.php
Content-Type: application/rss+xml
I hope this helps somebody.
I'm using a script on several webhosting providers and want just to transfer the whole script to the server if a new version is released.
But every server has its own absolute path to the AuthFile (for directory protection). .htaccess needs an absolute path to the AuthFile and this absolute path is different on every server.
My first approach was to use one .htaccess-file with several <Directory>-directives. Each with an absolute path which sets the AuthFile for the specific server.
But I got a 500 internal server error: .htaccess: <Directory not allowed here
The second idea was to use SetEnv and <IfDefine>. But IfDefine is not able to read environment-variables as shown in this blog entry.
The specific paths and servers are known.
Is there a way to find out on which server the .htaccess is called and to set the specific path for the AuthFile?
Now I solved this problem to create a script which parses the .htaccess files and checks whether the paths are set correctly for the specific server.
Not the solution I hoped for, but better than no solution :-/
I have the following URL format:
www.example.com/members/admin/projects/?projectid=41
And I would like to rewrite them to the following format:
www.example.com/avits/projectname/
Project names do not have to be unique when a user creates them therefore I will be checking for an existing name and appending an integer to the end of the project name if a project of the same name already exists. e.g. example.project, example.project1, example.project2 etc.
I am happy setting up the GET request to query the database by project name however I am having huge problems setting up these pretty url's.
I am using Apache with Nginx Admin installed which mens that all static content is served via Nginx without the overhead of apache.
I am totally confused as to whether I should be employing an nginx rewrite rule in my nginx.conf file or standard rewrites in my .htaccess file.
To confuse matters further although this is a rather large custom appliction it is build on top of a wordpress backbone for easy blogging functionality meaning that I also have the built in wordpress rewrite module at my disposal.
I have tried all three methods with absolutely no success. I have read a lot on the matter but simply cannot seem to get anything to work. I am certain this is purely down to a complete lack of understanding on with regards to URL rewriting. Combined with the fact that I don't know which type of rewriting should be applicable in my case means that I am doing nothing more than going round in circles.
Can anyone clear up this matter for me and explain how to rewrite my URLs in the manner described above?
Many thanks.
If you are proxying all the non static file requests to Apache, do the rewrites there - you don't need to do anything on nginx as it will just pass the requests to the back end.
The problem with what you are proposing is that it's not actually a rewrite, a rewrite is taking the first URL and just changing it around or moving the user to another location.
What you need actually takes logic to extrapolate the project name from the project ID.
For example you can rewrite:
www.example.com/members/admin/projects/?projectid=41
To:
www.example.com/avits/41/
Fairly easily, but can you map that /41/ in your app code to change it to /projectname/ - because a URL rewrite can't do that.
I've taken over development of a fairly heavy-duty LAMP application. The original dev used an .htaccess file with RewriteMap and a PHP script to handle certain conditions of the app.
Specifically, when certain subdomain patterns are requested by the client, the RewriteMap catches them and sends them to the appropriate application module.
I'm quite comfortable with typical mod_rewrite redirects, and I think I've got the basic RewriteMap concept figured out; but I'm struggling to find decent documentation on how RewriteLock works. According to the Apache docs:
This directive sets the filename for a synchronization lockfile which mod_rewrite needs to communicate with RewriteMap programs. Set this lockfile to a local path (not on a NFS-mounted device) when you want to use a rewriting map-program. It is not required for other types of rewriting maps.
But this is still a little vague for me. Whats the exact purpose and function of RewriteLock and how does it work?
RewriteLock is used with the prg: keyword. RewriteMap can be used with several keywords, to use text files (txt:), hashfiles (dbm:), randomized text (rnd:) or external mapping scripts ( this one is the prg: keyword ). In this mode the external script is launched when apache start. Then for every incoming request, when mod-rewrite is calling the prg: mapping, apache sends input to that script and reads the output stream to get the value.
RewriteLock must be used in that case to prevent parallel requests (so parallel inputs to that external process) to mix answers on this process standard output. It's a locking mechanism (a file, the given path, which is a classical token, only one user) to enforce serialization of the calls to this external mapping script. IMHO it should be transparently applied by mod-rewrite when using prg: as I never found a prg case where this locking thing is not mandatory.
Edit:
Well in fact you could use an external prg: without the rewriteLock if randomization of the output is not a problem, i.e. for a given entry you can get a response which was given for another entry, like in a script doing some advanced rnd:, your own round-robin service. But if the output must reflect the entry, then you need that semaphore, which of course can slow down the rewritemap process.
So if you're only using the hashmap or textmap you do not need to set the RewriteLock.
Edit:
You may find useful details on this thread, like the fact the lock file exists only for a few milliseconds, when apache calls the prg and waits for an answer.
Edit:
On the question one strange fact is:
The original dev used an .htaccess file with RewriteMap
This is strange because RewriteMap cannot work on .htaccess files, .htaccess are configuration entries read dynamically and RewriteMap as stated here in the Context line can only be set in the main configuration or in a VirtualHost configuration. It cannot be in a Location, a Directory or a .htaccess. So chances are this will never work in a .htaccess.
Now #puk asked for an example of RewriteMap usage. Well, searching for "RewriteMap" in Stack overflow will show you several real examples:
here in a question
here a list of example in my answer
another here
Apache hangs if you define more than one RewriteLock directives or if you use it in a VHOST config.
The RewriteLock should be specified at server config level and ONLY ONCE. This lock file will be used by all prg type maps. So if you want to use multiple prg maps, I suggest using an internal locking mechanism, for example in PHP there is the flock function, and simply ignore the warning apache writes in the error log.
See here for more info:
http://books.google.com/books?id=HUpTYMf8-aEC&lpg=PP1&pg=PA298#v=onepage&q&f=false
One of my clients (before I came along) decided to use htaccess redirects as their form of URL shortening/search engine friendly URLs. They have literally thousands of them.
The new version of the site now has friendly urls but they aren't equivalent to their redirects so they still need them.
My question to you all is: Is there another way than to populate this file with thousands of lines of "Redirects /folder1 /folder2"?
Thanks
If you cannot make simple rules to catch all of them as in the #chris henry solution you can use the RewriteMap utility of mod_rewrite. You'll be able to write these thousand rules in a text file, then make this text file an hash file, and mode_rewrite will try to match url in this file (if it's an hash file it's quite fast). After that mode_rewwrite can generate a redirect 301 with the [L,R=301] tag.
Yep, look at using the Apache config (httpd.conf or httpd-vhosts.conf) to set up site wide folder aliasing. Eg:
Alias /folder1 c:/www/folder2
Look at http://httpd.apache.org/docs/2.0/mod/core.html#directory for more info.
Depending on how different the URLs being redirected are, one solution might be to come up with an rewrite rule that covers all of them, and maintain the short / long URLs in your application, or even a database.