I have a subdomain which has been indexed in Google. The pages (a WordPress development project) are no longer there, so I want Google to realise that. I figured that a 410 is the way to go, but rather than putting them on individual posts that no longer exist, I was thinking maybe it could be a catch-all for the entire folder.
Would that be possible and would it be a good idea?
In the subdirectory that this subdomain resolves to you could use the following mod_rewrite directive that returns a 410 Gone for all requests:
RewriteEngine On
RewriteRule ^ - [G]
The ^ matches all requests. - indicates no substitution. The G flag results in a 410 Gone response being sent to the client (shorthand for R=410).
The default Apache 410 response will be sent to the client - unless you have defined a custom error document.
Yes, this is a good idea in order to get Google to drop the URLs from its search results in the shortest time possible. You could also consider using Google's URL removal tool as well.
Related
I have implemented SEO URLs using Apache 301 redirects to a 'redirect.cfm' in the root of the website which handles all URL building and content delivering.
Post data is lost during a 301 redirect.
Unable to find a solution so far, have tried excluding post method from rewrites - worst case scenario we could use the old type URLs for post methods.
Is there something that can be done?
Thanks
Using a 307 should be exactly what you want
307 Temporary Redirect (since HTTP/1.1)
In this case, the request should be repeated with another URI; however, future requests
should still use the original URI.[2] In contrast to how 302 was historically implemented,
the request method is not allowed to be changed when reissuing the original request. For
instance, a POST request should be repeated using another POST request
- Wikipedia
Update circa 2021
The original answer here was written before 307 status code redirect worked consistently across browsers. As per Hashbrown's answer below, the 307 status code should be used.
Old Answer
POST data is discarded on redirect as a client will perform a GET request to the URL specified by the 301. Period.
The only option is to convert the POST parameters to GET parameters and tack them onto the end of the URL you're redirecting to. This cannot be done in a .htaccess file rewrite.
One option is to catch POST requests to the url to be redirected and pass it off to a page to handle the redirect. You'd need to do the transposition of the parameters in code then issue the redirect header with the parameter appended new url that way.
Update: As pointed out in the comments to this answer, if you do redirect to another URL specifying POST parameters and that URL is also accessed without paramters (or the params are variable), you should specify a link to the canonical URL for the page.
Say the POST form redirects transposed to the following GET resource:
http://www.example.com/finalpage.php?form_data_1=123&form_data_2=666
You would add this link record to the head section of the page:
<link rel="canonical" href="http://www.example.com/finalpage.php" />
This would ensure all SEO value would be given to http://www.example.com/finalpage.php and avoid possible issues with duplicate content.
Using 301 redirects for general URL rewriting is not the way to go.
This is a performance issue (especially for mobile, but also in general), since it doubles the number of requests for your page.
Think about using a URL rewriting tool like Tuckey's URLrewriteFilter or apache mod_rewrite.
What Ray said is all true, this is just an additional comment on your general approach.
ok I previously had some issues setting up a wild card redirect to strip parameters from an old url format that were non-seo friendly to our root. With the help of stackoverflow we've got it corrected and working but now I am experiencing issues with index.php?
Im seeing tons of duplicate urls in the engines using this url format index.php?cPath=# and Google is stressing me because they think its duplicate index pages.
I tried setting up this redirect in our htaccess but no go. It won't strip out the parameters or redirect it just duplicates the page with a valid 200 response.
RewriteRule ^index.php/.*$ /? [R=301,NE,NC,L]
I want to redirect anything with index.php?cPath=# to our root domain. Any ideas on how i can tackle this 301 redirect using a mod_rewrite on apache?
I've also just noticed that our site has another issue with this url format:
domain.com/?cPath=#...
so now I also need to write a rewrite for domain.com/?cPath=# I don't know where this one popped up from but I can see its going to cause issues hence the valid 200 response code it gives. The engines are really going to love me for this one.
I want to ask how do I redirect Search Engines to take a second look on my new, fresh, rewritten URL-s?
So, my former URL-s were structured like this :
http://www.sample.com/tutorials.php?name=something
and now they look much more cleaner and better :
http://www.sample.com/tutorials/programming/something.php
So, as I said, I want Google (and other engines) to take a look at my new links, which are much more SEO friendly and for that I will be indexed better.
I was told the 301 redirect method was the best, but I don't have a clue what is it, how it works and where to learn how to use it. So, I am asking you.
Side note : Would updating my sitemap.xml file and re-submitting it to Google Webmaster Tools help in this process?
Thanks in advance!
There are 2 kinds (in this context) redirects. When a client, be it a browser, search engine indexing bot, or whatever, requests a URI, the server can tell the client "What you are looking for exists, but it's somewhere else". In the case of a 302 or temporary redirect, it's essentially telling the client "What you are looking for exists, but it's temporarily over here at this URL". In the case of a 301 or permanent redirect, it's essentially telling the client "What you are looking for exists, but it has permanently moved over to this URL".
In the case of the later, browsers, proxy servers, and search engine indexes know that the old URL is no longer valid and to stop using it, and from now on to use the new URL that was returned by the server via a 301 redirect. In the case of a search engine like Google, it has an index of the old URL and all the data that its accumulated over the lifetime of that URL assoicated with it. When one of its bots sees a 301, it knows that the old URL, and its content, isn't gone, but it just permanently moved to another URL. All of the associated data Google has collected for the old URL gets trasnfered to the new URL. Google can probably figure most of this stuff out without a 301 redirect, but it's a sure way to make sure Google has got a right.
You can do such a redirect via mod_rewrite:
RewriteEngine On
RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /tutorials\.php\?name=([^&\ ]+)
RewriteRule ^ /tutorials/programming/%1.php [L,R=301]
You should put this near the top of the htaccess file in your document root. The condition checks that an actual request has been made for /tutorials.php with a query string name="something". The "something" part gets grouped by the match and is accessed via the %1 backreference.
The 301 redirect is a response that the server can make which signals to the user (or search engine) that the page they are looking for has been permanently moved to a specified other page. It is possible to configure apache to give a 301 for certain urls, but it is probably easier to have the whatever server-side language you are using take the request, and then issue a 301.
The chances are that Google will work out what is going on fairly quickly without 301s or anything else, but submitting a sitemap to them or using the URL Parameters functionality in Google's Webmaster Tools might help.
I am using Magento 1.6.2.0 on a shared host running Litespeed web server, and I have begun investigating ways to speed up page loads. Currently I am using Pingdom to look at requests and it appears that I am losing an entire second from the get-go when I type my URL without the www. The browser redirects to the www page, it's just that it takes so long. Is this something I can fix? I presume that I can change Magento's base-url to not include the www, but then I presume I'll have the same delay when going to the www url instead.
I took a look at the link you gave, and I indeed see an about 1 second delay before I receive a 302 redirect to the URL with www. prepended. Not entirely coincidentally, the actual page HTML also takes quite long (about 1.7 seconds) to load.
This is a fairly common issue with large web applications: to return even a simple response like a redirect, the entire application must load and run its startup code. Couple this with a not so fast shared webserver that isn't optimized for that one application, and you can get quite slow page load times. It's nothing unique to Magento; I've seen the same effect with MediaWiki myself, and I expect that it happens with other applications too.
The obvious solution is just to avoid redirects: as long as you make sure all your URLs have the right hostname, the extra delay due to wrong hostnames will not appear. Magento itself will presumably take care of this for its own URLs, but if you have any other code (or static pages) that link to your Magento URLs, make sure they use the right hostname.
You can also sign up for Google Webmaster Tools (and similar tools for other search engines) and configure your preferred domain there (it's under Site configuration → Settings) so that Google will automatically prepend www. to any links to your site it indexes.
You can (and should) also try to reduce Magento's startup time in general. This will speed up not only redirects, but all other page loads as well. I'm not familiar enough with Magento to be able to give much detailed advice on this, but the obvious first step for any massive PHP application is to make sure you're using a PHP accelerator such as APC.
Finally, the fastest way to redirect visitors to the correct hostname is to make your webserver send the redirect directly without ever invoking Magento at all. The details on how to do this depend on the server software you're using, but apparently LiteSpeed supports the same RewriteRule syntax as Apache's mod_rewrite, so you should be able to do this just by adding the following lines to your main .htaccess file:
Options +FollowSymLinks
RewriteEngine On
RewriteBase /
RewriteCond %{HTTP_HOST} !^www\.mmmspeciosa\.com$ [NC]
RewriteRule ^(.*)$ http://www.mmmspeciosa.com/$1 [R=301,L]
(By the way, I'm using HTTP 301 permanent redirects here instead of the HTTP 302 temporary redirects Magento seems to be using. This is not only more appropriate according to the HTTP standard, but also works better with search engines, which treat a 301 redirect as an indication to index the target URL instead of the source of the redirect. If this redirect type is not configurable in Magento, I would consider it a bug. If it is configurable, you should set it to 301.)
We need to make changes to an app that will cause all its URLS to change, we don't want to lose value, and have too many urls redirect to 301. I am looking to change a mod rewritten URL to a non-written one.
My thoughts would be to
Leave the mod rewritten URLS active (Temporarily)
Place a canonical tag with the NEW correct URL
Make sure no links are currently linking to old URLS - all internal links updated etc
Make sure our robots.txt and sitemap submissions are updated to date.
Would a massive change in URLs - even if backed up by canonical URLs and updated sitemap.xml - have a negative affect on listings in google?
What are peoples thoughts / experience in this?
Thinking about it, if you're using mod_rewrite and are wanting to switch to a non mod rewritten URL then the chances are you can make the changes purely by adding the 301 response to the end of your rewrite rule to make something like this:
RewriteRule ^whatever/(.*)$ http://www.domain.com/$1/ [R=301,L]
Actually, a 301 redirect should not impact your search ranking - it's exactly how you're supposed to do that kind of thing and it's search engine independant. The "canonical" header is an invention of Google and has the disadvantage that people still using the old URLs from outside links will not be redirected and thus keep using the old URLs in links and bookmarks.
Using a permanent HTTP redirect is the best solution for both, your users and the search engines.
I'd also be interested to know then on this topic how these 301's should be handled, in .htaccess or at a code level? Surely 100's of 301's in a .htaccess is too many?