I am working on a site overhaul. As a result I am moving several pages over to a new format. They aren't keeping the same file name as before so the migration is a little tricky.
Example:
news.alpinezone.com/93467/ is becoming
http://alpinezone.com/still-more-skiing-and-riding-at-whiteface/
The news subdomain has accumulated in several years over 3000 articles. Is it OK to put 3000 + 301 redirects into an .htaccess file?
On a side note, for proper SEO, should I also make sure I use http:// instead of http:// www and also make sure they are fully lower case and also close with a / at the end of the URL. I am redesigning into wordpress and any combination pretty much works but I understand that for Google they can be considered unique but similar URL's so I want to stick with one as much as possible.
Thanks!
Apache does have some stuff for this, like RewriteMap or RewriteProg. I think htaccess files are read on every request, so I wouldn't want to make the size of it explode with 3000 lines of text - although I gut tells me it would handle it just fine. I think RewriteMap is only loaded once per server start or somethign like that, so thats a benefit.
But personally, I think I would just do an internal rewrite of any request to the news subdomain to a serverside script like php, and then inspect the uri, query the database to get the most current/up to date url slug for the id, and then do an external 301 redirect to the new url.
Have you considered mod_authnz_ldap to offload the authentication and authorization lookups to another server? I use this particular module on several enterprise servers with no problems whatsoever. It easily allows you to set up access to pages by group etc.
Related
I am currently migrating my website from Apache to nginx, but my .htaccess file is not working. My website is inside the /usr/share/nginx/html/mywebsite folder. How can I use .htaccess in my nginx server?
This is my .htaccess file:
RewriteEngine on
RewriteRule video/watch/([a-zA-Z0-9_#$*-]+)/?$ "videos-single.php?id=$1" [NC]
Nginx doesn't support .htaccess (see here: "You can’t do this. You shouldn’t. If you need .htaccess, you’re probably doing it wrong.").
You've two choices (as I know):
import your .htaccess to nginx.conf (maybe the htaccess to nginx converter helps you)
use authd-htpasswd (I didn't try it)
Disclosure: I am the author of htaccess for nginx, which is now open source software.
Over the past years, I created a plugin which implements htaccess behaviour into nginx, especially things like RewriteRule, Allow and Deny, which can be crucial for web security. The plugin is used in my own productive environments without a problem.
I totally share the point of efficiency and speed in nginx, and why they didn't implement htaccess.
However, think about it. You cannot make it worse if you're using nginx plus htaccess. You still keep the great performance of nginx, plus you can drive your legacy appliances effortlessly on one webserver.
This is not supported officially in nginx. If you need this kind of functionality you will need to use Apache or some other http server which supports it.
That said, the official nginx reasoning is flawed because it conflates what users want to do with the way it is done. For example, nginx could easily check the directories only every 10 seconds / minute or so, or it could use inotify and similar mechanisms. This would avoid the need to check it on every request... But knowing that doesn't help you. :)
You could get around this limitation by writing a script that would wait for nginx config files to appear and then copy them to /etc/nginx/conf.d/. However there might be some security implications - as there is no native support for .htaccess in nginx, there is also no support for limiting allowed configuration directives in config files. YMMV.
Using the config file is one option, but the cool thing about the .htaccess file is that it provided a way for a web developer to have some control over server settings without having root access to the server. There doesn't seem to be anything like this on nginx which is a real bummer.
I understand how the way it's setup on apache slows down response times, but hoped there could be an nginx way to do the same thing without the performance hit... At least a way to do rewrites with regex on urls if nothing else.
"Is there no nginx way to do bulk redirects using regular expressions that doesn't slow down response times."
Just edit your database with myphpmyadmin.
Open myphpmyadmin select your database then find your "yourprefix_Posts" table.
Open it then click the "Search" tab, then "Find and Replace".
Select "post_content" in the dropdown
In the "Find" field, type URL you want to change: "website.com/oldURL".
In the "Replace" field, type the new URL: "website.com/newURL".
(To use regular expression, tick the "Regular Expression" box.)
NOTE: You can test this out by simply leaving the "Replace" field blank.
ALWAYS BACKUP database before making changes. This might sound scary but its really not. Its super simple and can be used to quickly replace just about anbything.
my site e.g. carparts.co.uk has 355000 unique urls. (it is a car parts catalogue site) (on webmaster tools it shows that 174000 of these are indexed)
We want to move our site to a new shopping cart platform (prestashop), and have completely changed the structure of the catalogue, which means we now have a new set of urls. (although the main domain is unchanged and is still carparts.co.uk)
i now have a excel sheet where I have a column of the 355000 'old' urls matched against the closest equivalent url on the new catalogue.
e.g.
old url: "carparts.co.uk/ford-ranger-alternator belts.htm"
goes to: "carparts.co.uk/belt-drive"
(and there are 355,000 of similar redirects)
my question is how should i do this?
i've that you can use htaccess to do this, but i'm worried because i've read that htaccess slows down sites if it is very large (is this slowness only encounted when trying to access one of the old urls?, or will it impact the speed of all my urls?
so what is the best thing for me to do with such a large number of urls?
Your best bet is probably setting up a RewriteMap. This requires server vhost config access as you can't configure the map from an htaccess file (though you can use one). The mapping is cached by apache so you don't need to worry about constant file access.
Something simple like:
RewriteMap redirects txt:/full/path/to/redirect-map.txt
Then in the file redirect-map.txt would simply have a "from" and "to":
"ford-ranger-alternator belts.htm" belt-drive
old-url.htm new-url
etc...
Then in either your htaccess file or in vhost config, just do:
RewriteCond $(redirects:$1|0) !=0
RewriteRule ^(.*)$ $(redirects:$1) [L,R]
Use of htaccess slows down the website because it needs to check several files for each request, and these are checked dynamically for every request.
It's more a problem for deep routed sites. For example, a request to:
www.example.com/folder1/folder2/folder3/folder4/index.htm
would need to check
The main config file.
Then add any overrides in the document root
htaccess file.
Then add any overrides in the folder1 htaccess file.
Then add any overrides in the folder2 htaccess file.
...etc.
However if you don't have deep nesting then it's not so bad. Still slower than not using them, but may not be noticeable on most sites.
The benefit of htaccess for you here, would mean that you wouldn't need to put all the redirects in one place, and could split them up amongst the htaccess file. I'm not sure of the impact of adding 355,000 redirects to the main Apache config, but it is a fair number, so imagine it could have a performance impact. The htaccess files, on the other hand, are read dynamically as the request is made, so all the redirects would not need to be loaded into Apache.
So, this might be one of the few use cases where htaccess might be a better solution, even if you do have access to the main config files.
I have the following URL format:
www.example.com/members/admin/projects/?projectid=41
And I would like to rewrite them to the following format:
www.example.com/avits/projectname/
Project names do not have to be unique when a user creates them therefore I will be checking for an existing name and appending an integer to the end of the project name if a project of the same name already exists. e.g. example.project, example.project1, example.project2 etc.
I am happy setting up the GET request to query the database by project name however I am having huge problems setting up these pretty url's.
I am using Apache with Nginx Admin installed which mens that all static content is served via Nginx without the overhead of apache.
I am totally confused as to whether I should be employing an nginx rewrite rule in my nginx.conf file or standard rewrites in my .htaccess file.
To confuse matters further although this is a rather large custom appliction it is build on top of a wordpress backbone for easy blogging functionality meaning that I also have the built in wordpress rewrite module at my disposal.
I have tried all three methods with absolutely no success. I have read a lot on the matter but simply cannot seem to get anything to work. I am certain this is purely down to a complete lack of understanding on with regards to URL rewriting. Combined with the fact that I don't know which type of rewriting should be applicable in my case means that I am doing nothing more than going round in circles.
Can anyone clear up this matter for me and explain how to rewrite my URLs in the manner described above?
Many thanks.
If you are proxying all the non static file requests to Apache, do the rewrites there - you don't need to do anything on nginx as it will just pass the requests to the back end.
The problem with what you are proposing is that it's not actually a rewrite, a rewrite is taking the first URL and just changing it around or moving the user to another location.
What you need actually takes logic to extrapolate the project name from the project ID.
For example you can rewrite:
www.example.com/members/admin/projects/?projectid=41
To:
www.example.com/avits/41/
Fairly easily, but can you map that /41/ in your app code to change it to /projectname/ - because a URL rewrite can't do that.
One of my clients (before I came along) decided to use htaccess redirects as their form of URL shortening/search engine friendly URLs. They have literally thousands of them.
The new version of the site now has friendly urls but they aren't equivalent to their redirects so they still need them.
My question to you all is: Is there another way than to populate this file with thousands of lines of "Redirects /folder1 /folder2"?
Thanks
If you cannot make simple rules to catch all of them as in the #chris henry solution you can use the RewriteMap utility of mod_rewrite. You'll be able to write these thousand rules in a text file, then make this text file an hash file, and mode_rewrite will try to match url in this file (if it's an hash file it's quite fast). After that mode_rewwrite can generate a redirect 301 with the [L,R=301] tag.
Yep, look at using the Apache config (httpd.conf or httpd-vhosts.conf) to set up site wide folder aliasing. Eg:
Alias /folder1 c:/www/folder2
Look at http://httpd.apache.org/docs/2.0/mod/core.html#directory for more info.
Depending on how different the URLs being redirected are, one solution might be to come up with an rewrite rule that covers all of them, and maintain the short / long URLs in your application, or even a database.
We are putting up a company blog at companyname.com/blog but for now the blog is a Wordpress installation that lives on a different server (blog.companyname.com).
The intention is to have the blog and web site both on the same server in a month or two, but that leaves a problem in the interim.
At the moment I am using mod_rewrite to do the following:
http://companyname.com/blog/article-name redirects to http://blog.companyname.com/article-name
Can I somehow keep the address bar displaying companyname.com/blog even though the content is coming from the latter blog.companyname.com?
I can see how to do this if it is on the same server and vhost, but not across a different server?
Thanks
Rather than using mod_rewrite, you could use mod_proxy to set up a reverse proxy on companyname.com, so that requests to http://companyname.com/blog/article-name are proxied (rather than redirected) to http://blog.companyname.com/article-name.
Here are more instructions and examples.
There is functionality with ZoneEdit called webforwards which could probably do this and hide what you are actually doing (unless someone looked into it).
The only thing that mod_rewrite can do is send HTTP header redirects, and those redirects (across servers) always result in the browser address bar reflecting the reality.
You should instead consider writing a 404 script that 'reflects' the blog. This would essentially be a transparent proxy, and many are already written.
The script would find if the requested page (that was 404'd) started with http://mycompany.com/blog/ . If it did, it would download and then send onto the client the blog page and associated files (probably caching them as well).
So requesting http://mycompany.com/blog/article_xyz would cause the 404 script to download and send http://blog.companyname.com/article_xyz.
It's probably more work than it's worth, but you might be able to design a simple enough 404 script that it's worthwhile.
-Adam