Sometimes, before launching new web projects, i put the site / app under a subdomain like new.domain.com or beta.domain.com.
These URLs are only meant for my clients. So they don't get submitted at search engines and there aren't any public links to them.
However, I noticed in a few occasions, these subdomains get indexed by Bing anyway. How is this possible ?
Does Bing crawl generic subdomain names like new, old, archive, beta, ... ?
Or do URLs sent in mails, get scraped in Office 365 (which my clients use) and get indexed ?
This is possible the user has installed a toolbar from that search engine.
The best way to prevent from this, is adding no-index tag for all pages in sub domain and even you can block using robot.txt.
Related
We have a website that is used to showcase our various products. The website uses MVC4 and subdomains areas.
product1.website.com
product2.website.com
We use the subdomain to determine which area to route the request.
Lately we have been getting http HEAD requests to our site using the IP only. Without the subdomain we can't know which area to send the request.
What should we do?
Send back a 404
Redirect to our most important area/product
Redirect to our company website
why not redirect users to an overview page where they see a short list of the products. In this way you can redirect them behind the screens to whereever you want without hem knowing and this also has the ability to be used when user make typo in the url so that they are 'guided' to the right product and even find other ones.
-a 404 usually makes people seek elsewhere since tey think they have the wrong IP
- redirect to most important product may result in confusion when you change your major product (users tend to bookmark a lot of useless urls)
- redirect to the company website is to my opinion the lesser of all evils, but users tend to get lost when redirected to a 'general' website.
example: you're looking for Windows 8 download and have the IP bookmarked
- 404 error: oh the page no longer exists
- main product: windows 9 is out but for some reason you still need windows 8: you spend more time looking for what you really need and probably find it elsewhere
- overview page: you see what you need in a list and if the list is short you quickly find it, otherwise a simple search reveals the item also.
so redirect to overview page is still a winning shot in my opinion
I am doing some research on Canonical pages in our site.
Does Google create two indexes in this case:
http://www.foo.com/folder/index.html
http://www.foo.com/folder/
Or does it only index one of the above?
I am curious if I need to add a rel="canonical" or if I am just overthinking this simple idea.
After research it depends on the web server.
In our case it was a Sun One web server that you could hit both foo.com/ and foo.com/index.jsp
Even though these pulled up the same content, they are two different URLs and Google saw them as two sperate pages with duplicate content. This was bumping down our SEO.
The fix was to modify the web server to auotmatically redirect /index.jsp pages to the /.
So yes, google will index any page that you can browse to in your browser, unless its on you robots.txt or you are manually telling google not to index in some fashion.
I am renewing my web page and changing the site structure. It was in Asp and now it will be in Asp.Net
So page URLs will be modified. And some pages will be removed, some will be added. But mostly, the content and page names are same, only URLs will change.
The site has SEO work in it and we want to loose it minimum.Site is registered in Analytics and Webmaster Tools.
Google searches will end up blank pages and I don't want to loose my rank.
So I'm looking for a way to inform Google about new page URLs. Domain is same, only URLs. For example: the home page was /default.asp and now /home.aspx
Is there a way to tell Google that a particular URL address or page name has changed?
If all that is changing are the page URLs, Google Analytics cannot "know" that a page is the same, just with diferent URL.
But, you could apply a customized pageview using the _trackPageView() method, giving it the original url as parameter.
If you choose to do this, you will have to exclude the line that uses the method in the original GA code and apply it elsewhere, or pass the parameter to it directly with the orignial URL. All this is done in each page.
You can also read more about the method here.
For IIS (Asp.Net) you want to look into the following to find out how to do 301 redirects:
Response.RedirectPermanent(...) for redirecting from a page
or
"IIS 7 Routing Module and web.config" to set up bulk redirecting
I'd also suggest you consider supporting Search Engine Friendly (SEF) URLs while your making the move. The Routing Module can help you there as well.
You need to implement some form of 301 (301 is key) redirects. This way when google or any other search hits the old page, the index is refreshed with the new page. Asp.net allows you to do these redirects even at the IIS level, and where I'd suggest that they live. You'll also want to submit an up to date site map on webmaster tools.
Edit: Here's a good link on the redirects, http://www.iis.net/ConfigReference/system.webServer/httpRedirect
If i generate a site map at http://www.xml-sitemaps.com/, for aromapersona.com, I get 9 pages, however there are a bunch more pages that should show up. For example, aromapersona.com/candle_holder is in the same "front" directory as the other 9 pages, but doesnt generate in sitemap. Is this because no other pages on my site link to it? Im trying to get these other URLs indexed, and I even edited the site map to include this URL as well as others and submitted to google via webmaster tools, and still nothing. Advice?
I'm not familiar with aromapersona.com but it will only be able to list pages that are linked to from the initial page you give it (or ones they link to) unless you provide the site with FTP access (which I presume you dont).
If you include the URL's in your sitemap for goggle it should eventually list them, but linking to them from other parts of your site is probably the most effective.
I have not checked the website, but do also take the cause is not because of noindex, nofollow, robots.txt, javascript links, mixing http/https etc.
In clear wording: There is no link pointing to the subpage "candle_holder", hence the XML site generator (which works by following links on your site) cannot detect it.
You can add it manually to the XML, but then again, it should be accessible from the site directly.
When I searching our web site on Google I found three sites with the same content show up. I always thought we were using only one site www.foo.com, but it turn out we have www.foo.net and www.foo.info with the same content as www.foo.com.
I know it is extremely bad to have the same content under different URL. And it seems we have being using three domains for years and I have not seen punitive blunt so far. What is going on? Is Google using new policy like this blog advocate?http://www.seodenver.com/duplicate-content-over-multiple-domains-seo-issues/ Or is it OK using DNS redirect? What should I do? Thanks
If you are managing the websites via Google Webmaster Tools, it is possible to specify the "primary domain".
However, the world of search engines doesn't stop with Google, so your best bet is to send a 301 redirect to your primary domain. For example.
www.foo.net should 301 redirect to www.foo.com
www.foo.net/bar should 301 redirect to www.foo.com/bar
and so on.
This will ensure that www.foo.com gets the entire score, rather than (potentially) a third of the score that you might get for link-backs (internal and external).
Look into canonical links, as documented by Google.
If your site has identical or vastly
similar content that's accessible
through multiple URLs, this format
provides you with more control over
the URL returned in search results. It
also helps to make sure that
properties such as link popularity are
consolidated to your preferred
version.
They explicitly state it will work cross-domain.