We have got a deprecated domain www.deprecateddomain.com. Specific fact is that we have got reverse proxy working and redirecting all requests from this domain to the new one www.newdomain.com.
The problem is when you type "deprecateddomain.com" in google search, there is a link to www.deprecateddomain.com in search results besides results with "newdomain.com". It means that there is such entries in google index. Our customer don't want to see links to old site.
We were suggested to create fake robots.txt with Disallow: / directive for www.deprecateddomain.com and reverse proxy rules to get this file from some directory. But after investigation the subject I started hesitating that it will help. Will it remove entries with old domain from index?
Why not to just create the request in search console to remove www.deprecateddomain.com from index? In my opinion it might help.
Anyway, I'm novice in this question. Could you give me advice what to do?
Google takes time to remove old/obsolete entries from its ranking, especially on low visited or low value pages. You have no control on it. Google needs to revisit each page to see the redirection you have implemented.
So DO NOT implement a disallow on the old website, because it will make the problem worse. Bots won't be able to crawls those pages and see the redirection you have implemented. So they will stay longer in the rankings.
You must also make sure you implement a proper 301 redirection (i.e. a permanent one, not a temporary) for all pages of the old website. Else, some pages may stay in the ranking for quite some time.
If some pages are obsolete and should be deleted rather than redirected, return a 404 for them. Google will remove them quickly from its index.
Related
History/Context: I have a site, 'www.oldexample.com' built in 1998, HTML 4.01 transitional on Apache, cpanel server. Until last fall our main keywords got us to top 10. After mobile changes and Panda etc, Dropped to page 2 or 3 for all but one very specific keyword. The old site, 'www.oldexample.com' has many good back links and history in google and all main directories. I am rebuilding a test site now which is on 'mycompany.myshopify.com' as it addresses all my google errors issues on oldsite. I have set up my 'www.newexample.com' to redirect to the shopify site which is called up under 'www.newexample.com'.The myshopify.com URL does not show up at all.
Question: If I were to do cpanel 301 redirect of whole 'oldexample.com' to 'newexample.com' would I still benefit from the many links and history of oldsite?
When you say that the shopify URL doesn't show at all, do you mean it's not showing when you search for those keywords, or it's not indexed at all? If it's the latter, prompt Google to index it using Google Search Console. If it's the former, there are a number of things that could have affected this:
the authority of the new site - if you've just launched it, it naturally won't have the authority of the previous site and therefore is less likely to get visibility
you are correct that the backlinks would have played a major part in this. What you need to do is to redirect the old domain to the new one you want to appear in Google. For example, if you want to actually take people to newsite.shopify.com, you should redirect the old domain directly to that one. If you redirect the old one to newdomain.com, which you then redirect to newsite.shopify.com the result won't be the same. Link value is lost via redirects. Ideally, you should get in touch with as many 3rd party websites linking to your old domain and ask them to update the links to point to newsite.shopify.com
Even if you do that you might still not see those rankings because of various other factors. If you fancy posting the actual URLs and keywords in question, I can spare a few minutes to have a look.
Couple of months ago, we revamped our web site. We adopted totally new site structure, specifically merged several pages into one. Everything looks charming.
However, there are lots of dead links which produce a large number of 404 errors.
So how can I do with it? If I leave it alone, could it bite back someday, say eating up my pr?
One basic option is using 301 redirect, however it is almost impossible considering the number of it.
So is there any workaround? Thanks for your considering!
301 is an excellent idea.
Consider you can take advantage of global configurations to map a group of pages. You don't necessary need to write one redirect for every 404.
For example, if you removed the http://example/foo folder, using Apache you can write the following configuration
RedirectMatch 301 ^/foo/(.*)$ http://example.org/
to catch all 404 generated from the removed folder.
Also, consider to redirect selectively. You can use Google Webmaster Tools to check which 404 URI are receiving the highest number inbound links and create a redirect configuration only for those.
Chances are the number of redirection rules you need to create will decrease drastically.
301 is definitely the correct route to go down to preserve your page rank.
Alternatively, you could catch 404 errors and redirect either to a "This content has moved" type page, or your home page. If you do this I would still recommend cherry picking busy pages and important content and setting up 301s for these - then you can preserve PR on your most important content, and deal gracefully with the rest of the dead links...
I agree with the other posts - using mod_rewrite you can remap URLs and return 301s. Note - it's possible to call an external program or database with mod_rewrite - so there's a lot you can do there.
If your new and old site don't follow any remapable pattern, then I suggest you make your 404 page as useful as possible. Google has a widget which will suggest the page the user is probably looking for. This works well once Google has spidered your new site.
Along with the other 301 suggestions, you could also split the requested url string into a search string routing to your default search page (if you have one) passing those parameters automatically to the search.
For example, if someone tries to visit http://example.com/2009/01/new-years-was-a-blast, this would route to your search page and automatically search for "new years was a blast" returning the best result for those key words and hopefully your most relevant article.
This will require a little setup. Trust me that this is for a good cause.
The Background
A friend of mine has run a non-profit public interest website for two years. The site is designed to counteract misinformation about a certain public person. Of course, over the last two years those of us who support what he is doing have relentlessly linked to the site in order to boost it in Google so that it appears very highly when you search for this public person's name. (In fact it is the #2 result, right below the public person's own site). He does not have the support of this public person, but what he is doing is in the public interest and good.
The friend had a stroke recently. Coincidentally, the domain name came up for renewal right when he was in the hospital and his wife missed the email about it. A domain squatter snapped up the domain, and put up content diametrically opposed to his intent. This squatter is now benefitting from his Google placement and page rank.
Fortunately there were other domains he owned which were aliased to point to this domain, i.e. they used a DNS mapping or HTTP 301 redirect (I'm not sure which) to send people to the right site. We reconfigured one of the alias domains to point directly to the original content.
We have publicized this new name for the site and the community has now created thousands of links to the new domain, and is fixing all the old links. We can see from the cache that Google has in fact crawled the original site at the new address, and has re-crawled the imposter site.
The Problem
Even though Google has crawled both sites, you can't get the site to appear in relevant searches under the new URL!
It appears to me that Google remembers the old redirect between the two names (probably because someone linked to the new domain back when it was an alias). It is treating the two sites as if they are the same site in all results. The results for the site name, and using the "link:" operator to find sites that link to this site, are entirely consistent with Google being convinced they are the same site.
Keep in mind that we do not have control of the content of the old domain, and we do not have the cooperation of the person that these sites relate to.
How can we convince the Googlebot that domain "a" and domain "b" are now two different sites and should be treated as such in results?
EDIT: Forward was probably DNS, not HTTP based.
Google will detect the decrease in links to the old domain and that will hurt it.
Include some new interesting content on the new domain. This will encourage Google to crawl this domain.
The 301 redirects will be forgotten, in time. Perhaps several months. Note that they redirected one set of URLs to another set, not from one domain to another. Get some links to some new pages within the site, not just the homepage, as these URLs will not be in the old redirected set.
Set up Google Webmaster Tools and submit an XML sitemap. Thoroughly check everything in Webmaster Tools about once per week.
Good luck.
Time heals all wounds...
Losing control of the domain is a big blow, and it will take time to recover. It sounds like you're following all the correct procedures (getting people to change links, using 301s, etc.)
Has the content of the original site changed since being put up again? If not, you should probably make some changes. If Google re-crawls the page and finds it substantially identical to the one previously indexed, it might consider it a copy and that's why it's using the original URL.
Also, I believe that Google has a resolution process for just such situations. I'm not sure what the form to fill out is or who to contact, but surely some other SO citizens could help.
Good luck!
With ASP.NET MVC (or using HttpHandlers) you can dynamically generate URLs, like the one in this question, which includes the title.
What happens if the title changes (for example, editing it) and there's a link pointing to the page from another site, or Google's Pagerank was calculated for that URL?
I guess it's all lost right? (The link points to nowhere and the pagerank calculated is lost)
If so, is there a way to avoid it?
I use the same system as is in place here, everything after the number in the URL is not used in the db query, then I 301 redirect anything else to be the title.
In other words, if the title changed, then it would redirect to the correct place. I do it in PHP rather than htaccess as it's easier to manage more complex ideas.
I think you're generally best off having the server send a permanent redirect to the new location, if possible.
That way any rank which is gained from third party links should, in theory, be transferred to the new location. I'm not convinced whether this happens in practice, but it should.
The way Stackoverflow seems to be implemented everything after the question number is superfluous as far as linking to the question goes. For instance:
SEO and hard links with dynamic URLs
links to this question, despite the fact that I just made up the 'question title' part out of thin air. So the link will not point to nowhere and the PageRank is not lost (though it may be split between the two URLs, depending on whether or not Google can canonicalize them into a single URL).
Have your app redirect the old URL via a 301 Redirect. This will tell Google to transfer the pagerank to the new URL.
If a document is moved to a different URL, the server should be configured to return a HTTP status code of 301 (Moved Permanently) for the old URL to tell the client where the document has been moved to. With Apache, this is done using mod_rewrite and RewriteRule.
The best thing to help Google in this instance is to return a permanent redirect on the old URL to the new one.
I'm not an ASP.NET hacker - so I can't recommend the best way to implement this - but Googling the topic looks fairly productive :-)
Yes, all SEO is lost upon a url change -- it forks to an entirely new record. The way to handle that is to leave a 301 redirect at the old title to the new one, and some search engines (read: Google) is smart enough to pick that up.
EDIT: Fixed to 301 redirect!
i've been tasked with re-organizing a pure HTML site into a CMS. if all goes well, the new site will eventually become the main URL, and the old domain will be phased out. the old domain has a decent enough page rank, and the company wishes to mitigate any loss of page rank for that. in looking over the options available, i've discovered a few things:
it's better to use a 301 redirect when you're ready to make the switch (source).
the current site does not have a sitemap, so adding one and submitting it may help their future page rank.
i'll need to suggest to them that they contact people currently linking to them to update their links.
the process for regaining an old page rank takes awhile, so plan on rebuilding links while we see if the new site is flexible enough to warrant switching over completely.
my question is: as a result of a move to a CMS driven site, the links to various pages will change to accommodate the new structure. will this be an issue for trying to maintain (or improve) the current page rank? what sort of methods are available to mitigate the issue of changing individual page URL's? is there a preferable method beyond mapping individual pages to their new locations with 301 redirects? (the site has literally hundreds of pages, ugh...)
ex.
http://domain.com/Messy_HTML_page_with_little_categorization.html ->
http://newdomain.com/nice/structured/pages.php
i realize this isn't strictly a programming question, however i felt the information could be useful to developers who are tasked with handling this sort of thing in addition to development of the site.
edit: additions in italics
If you really truly want to ensure that page rank is not lost, you will want to replace the old content with something that performs a proper 301 redirect to the new location. With a 301 redirect the search spiders will know that the content is moved and the page rank typically carries over. It also helps external links.
However, the down side is that after a certain period of time you just have to get rid of the old domains.
You can make a handler for HTML files and map the old pages to the new structure with a 301 redirect.