I am trying to submit my site http://jobmigo.com to Google but after 3 weeks it has not been indexed. Possible problems are one I only have two urls to submit as this is a password protected site, but these urls have a lot of content on them. Two this is an angular based website, so I use prerender to cache pages with rendered content for Google. Third the index.jsp redirects to the home page, this might also bea problem. Which of these are most likely causing the problem?
<meta name="robots" content="noindex,follow">
on http://jobmigo.com/jobs-admin/index?origin=landing basically tells google not to index your page.. So that might be the main issue here.
All the pages I looked at had a noindex tag on.
Related
My cousin has given me her old website to "redo" which is on a Google Sites template. I have separately created a static website with Ruby on Rails setup and Bootstrap but am now scratching my head on how to transition her old Google Site to this new layout (I want to keep the original domain name).
Also, the good thing about Google Sites is that when I search for her business, it pops up on the right side of the page and sometimes pops up at the top for google local search, which is obviously great for marketing/SEO.
I did read something that Google doesn't support other platforms (I think) which is why I'm wondering if I can redirect and still retain the SEO benefits. Any resources or suggestions would be great, thanks!
Setup your new site with new ROR template.
Register and verify your new domain and old one on Google Webmaster
Create a custom 404 page for old domain which suggests visiting new domain.
Redirect your old urls to new one by a 1:1 redirection with 301, "Moved Permanently", like www.oldone.com/games to www.newone.com/games
Submit your sitemap of old one to Google and BIng, they will see 301 and rewrite their crawl cache.
Fill out change of address in Google webmaster and add new site's sitemap.
Above method is the best SEO friendly site moving method.
ps: Not exactly ROR question .
Edit: I don't really know about google pages but one of doing this would be, deploy ROR site somewhere and use iframe in google page to project it to google page.
In your google page,
.
use options like height="100%" to cover the entire page.
I created a CMS for a website and integrated Google Analytics. The site changes it's content every week (adding, editing, removing pages and URLs) and I rewrite the sitemap every time when one of this actions occurred.
The problem is that the web crawlers from Google detect a lot of 404 error pages.
What I am doing wrong?
Getting reports about 404s is perfectly normal and generally no need to worry about them.
Check where does Google find those 404 URLs, you can see that in Search Console (formerly Webmaster Tools), and see if you can fix them. If you cannot, if you have great content, sooner or later you'll get better links.
What you could do additionally, is to create custom 404 pages, where you link to content on your site that's similar to the missing page (if it's possible to determine that), or that's popular on the site.
Also if you feel that the page is for content that won't be coming back on the site. you can remove the URL for their index by using the remove URL option.
i want to remove the pages that i have removed from the server from google
or redirect them
the pages that i have removed from the server are
www.mysite.com/id?=9898
and
www.mysite.com/pagename.html
the new pages are
www.mysite.com/pagename
so i removed the sitemap from google and created a new one and uploaded it
my problem now is google give me crawl error because of the removed pages like www.sitename.com/contact.html and the
indexed page now are only 2 pages
why he can't see that i have removed this pages and when i search on google the removed pages still appears
Better then uploading a new sitemap, is to redirect from your old pages to your new ones. Here you can geht informations about a 301 redirect.
Your problem is, that Google in fact can not know that the pages www.blabalbal.com/id?234234 and www.blabalbal.com/speaking_url are the same page. So you have to tell Google. A good method is the redirect from above.
You should fix this, because Google maybe crawls both webpages as unique ones and comes to the conclusion that this two pages are duplicate content, which is a bad thing for your rankings.
You have to use Disavow Tool - Google to get this done completely.On google search for Disavow Tool - Google and you will get the first link ,login to this by google ID and follow the instructions.USe the same google ID through which sitemap was upload in google webmaster
I have a website that has two different pages structure - one for mobile visitors, and one for desktop. That's why I have two sitemap files - one for the mobile and one for desktop.
I want to create a robots.txt file that will "tell" search engines bots to scan the mobile sitemap for mobile sites, and the desktop sitemap for desktop sites.
How can I do that?
I thought of creating a sitemap index file which will point to both of those site maps, and to add the following directive to the robots.txt file:
sitemap: [sitemap-index-location]
It this the right way?
I can not give you a certainty, but I believe the best practice is to inform the two sitemaps in robots.txt. In mobile sitemap you already have the markings <mobile:mobile/> is reporting that a mobile version.
Another interesting question is perhaps also create a sitemap index:
<sitemapindex xmlns="http://www.sitemaps.org/schemas/sitemap/0.9">
<sitemap>
<loc>http://example.com/sitemap.desktop.xml</loc>
</sitemap>
<sitemap>
<loc>http://m.example.com/sitemap.mobile.xml</loc>
</sitemap>
</sitemapindex>
And your robots.txt will look like:
# Sitemap index
Sitemap: http://example.com/sitemap.xml
# Other sitemaps. I know it is already declared in the sitemap index, but I believe it will do no harm also set here
Sitemap: http://example.com/sitemap.desktop.xml
Sitemap: http://example.com/sitemap.mobile.xml
robots.txt does not tell the search engine which mobile end which is the end of your PC, and he can only declare a sitemap.This is sufficient Well, I think you can add a judge in the html page header, the pc end pc side web mobile end mobile page, this is not good?Site Map there is on the page there is a link to, then more to promote inclusion.
I think I will recommend you for the responsive website design.
With the help of a responsive web design technique, you can build the alter web pages using CSS3 media queries. Here, there is one HTML code for the page regardless of the device accessing it. But, its presentation changes through CSS media queries to specify as to which CSS rules apply to the browser for displaying the page.
With responsive website design, you can keep both the desktop and mobile content on a single URL. It is easier for your users to interact with, share, and link to and for Google’s algorithms to assign the indexing properties to your web content.
Besides, Google will crawl your content effectively and there won’t be any need to crawl a web page using a different Googlebot user agent.
You can simply define a single sitemap and put in robots.txt file. It will crawl both for your desktop and mobile content.
In addition to stating the files in robots.txt, you should log into Google Webmaster Tools and submit the sitemaps there. That will tell you
If the sitemap url you submitted is correct
That the sitemap file has the correct syntax
How many of the files in the sitemap have been crawled
How many of the files in the sitemap have been indexed
I have an app running in Heroku.
I am using sitemap_generator to generate sitemap and save it into s3.
I have added the robots.txt to contain my sitemap location.
My question are.
How can I know my sitemap are successfully find by search engine like google?
How can I monitor my sitemap?
If my sitemap is located in my app server I can add the sitemap manually into google webmaster tools for monitoring. Because when I click on "Test/Add sitemap" in Google webmaster tools, it default to the same server.
Thanks for your help.
I got it to work.
Google has something called cross submission: http://googlewebmastercentral.blogspot.com/2007/10/dealing-with-sitemap-cross-submissions.html
You might want to visit this blog as well:
http://stanicblog.blogspot.sg/2012/02/how-to-add-your-sitemap-file-located-in.html
Thanks for your help, yacc.
Let me answer your two first questions, one at a time (I'm not sure what you mean by 'how can I monitor my sitemap' so I'll skip it):
Manually submit a sitemap to Google
If you can't use Google webmaster form to submit your sitemap, use an HTTP get request to notify Google of your new site map.
If your sitemap is located at https://s3.amazonaws.com/sitemapbucket/sitemap.gz , first URL encode your sitemap URL (you can use this online URL encoder/decoder for that) then using curl or wget to submit your encoded URL to Google:
curl www.google.com/webmasters/tools/ping?sitemap=https%3A%2F%2Fs3.amazonaws.com%2Fsitemapbucket%2Fsitemap.gz
If your request is successful you'll get a 200 answer with a message like this:
... cut ...
<body><h2>Sitemap Notification Received</h2>
<br>
Your Sitemap has been successfully added to our list of Sitemaps to crawl.
... cut ...
Checking that Google knows about your new sitemap
Open Webmaster Tools, navigate to Site sonfiguration->Sitemaps, there you should see the sitemaps that you've submited. It might take sometime for a new sitemap to show up there, so check frequently.