How to stop stop google crawl for this page https://www.example.com/index.php/en/component/jomres/?Itemid=0&task=search&page=1&towns%5B0%5D=calangute in jomres, is it possible to do in robot.txt file or any other solution?
Yes you can through robots.txt for this page by adding the below line
Disallow : /index.php/en/component/jomres/?Itemid=0&task=search&page=1&towns%5B0%5D=calangute
The above line will block only this page, if you want to block all the pages with the above parameters, use the this one
Disallow : /index.php/en/component/jomres/?Itemid=*&task=*&page=*&towns%5B0%5D=*
Here we changed the dynamic values to stars *, you can customize it as you want and for any specific case.
Related
I have an e-commerce site. For the T-shirts page I want the only URL indexed by search engines to be
mysite/items/t-shirts/
But there are filters on the page so a user can filter by size, colour, material etc.
I thought about keeping the URL to always display as
mysite/items/t-shirts/
regardless of which filters are chosen so that search engines don't index pages like:
mysite/items/t-shirts/?colour=red
mysite/items/t-shirts/?colour=red&size=xl
mysite/items/t-shirts/?colour=red&size=xl&material=cotton
but this breaks the back button.
So I'm thinking of keeping the URL parameters so the back button can be used and to prevent search engines from indexing urls with parameters I'll put the following in the robots.txt file:
User-agent: *
# Block crawlers from storing pages with parameters
Disallow: /*/*?
Is this the best approach or are there better alternatives?
We have pages which have been split into multiple pages as they are too in depth. The structure currently...
Page (www.domain.com/page)
We have split this up like so...
Page + Subtitle (www.new-domain.com/page-subtitle-1)
Page + Subtitle (www.new-domain.com/page-subtitle-2)
Page + Subtitle (www.new-domain.com/page-subtitle-3)
I need to know the correct way of adding in multiple canonical tags on the original page. Is it search engine friendly to add say 3/4 canonical tags linking to 3/4 separate pages?
Well, this is what you should do -
Keep the complete page even if you are dividing into component pages.
Use rel="next" and rel="prev" links to indicate the relationship between component URLs. This markup provides a strong hint to Google that you would like it to treat these pages as a logical sequence, thus consolidating their linking properties and usually sending searchers to the first page.
In each of the component pages, add a rel="canonical" link to the original (all content) page. This will tell google about the all content page.
This strategy is recommended by google - read here.
Canonical tags are basically to consolidate link signals for duplicate or similar content. With that said, you are not supposed to have multiple canonical tags in a page. You have two options.
If your old page is going to go away, then you should pick one primary page(in the split pages) and do a 301 redirect, so the SEO value are carried over to that new primary URL.
If its going to stay, you can create internal links to the new pages. But make sure the content is different, so that it does not count as duplicate pages.
Hope this helps.
If I have a site called http://example.com, and under it I have articles, such as:
http://example.com/articles/norwegian-statoil-ceo-resigns
Basically, I don't want the text from the frontpage to show on Google results, so that when you search for "statoil ceo", you ONLY get the article itself, but not the frontpage which contains this text but is not of the article itself.
If you did that, then Google could still display your home page with a note under the link saying they couldnt crawl the page. This is because robots.txt doesnt stop a page being indexed. You could noindex the home page, though personally I wouldnt recommend it.
I tried searching for similar posts but none of them satisfied my need. Hence I have to ask a new question
I have some domain space on my university website like this <university>.edu/~<myname> which is actually <university>.edu/~<myname>/index.html
I want all the traffic from here to go to <myname>.wix.com/resume. But I don't want other pages like my <university>.edu/~<myname>/somethingelse to redirect anywhere.
Also I want the address bar to still show <university>.edu/~<myname> even when it has readirected to my site on wix.com.
Kindly let me know if this is possible.
Thanks in advance.
No browser that I know of will allow itself to be redirected to a different site, without displaying the URL of the new site in the address bar. However, there are ways that you can display content from <myname>.wix.com/resume on your page at <university>.edu/~<myname>/index.html. One way is to use a frame (or an iframe) on <myname>.wix.com/resume which would load content from <university>.edu/~<myname>/index.html. Another (perhaps more seamless) solution would be to use a server-side scripting language (such as PHP) to create a script at <university>.edu/~<myname>/index.php which would download the content 'on the fly' from <myname>.wix.com/resume and display it at <university>.edu/~<myname>/index.php.
I have a TYPO3 site with Indexed Search Engine extension...
The problem is that not all the content is indexed (with debug option activated in conf content not all the page is present but the page size is corect), only the firts part of the page (witch is the head/title and the begining of the menu...).
So for every page the words are only from the begining of the page (title, menus).
I have tried using the Indexed Search Engine begin and end tags but no effect...
What did I do wrong?
I am stupid :-( I have figured out the problem...
The begin/end tags where not corect I used <!--TYPO3SEARCH_begin>
<!--TYPO3SEARCH_end> insted
of <!--TYPO3SEARCH_begin--> <!--TYPO3SEARCH_end-->