Google Custom Search - Add/remove sites to search dynamically - google-custom-search

Google Custom Search has a feature to specify the sites you want the search engine to search - "Sites to search" feature.
I have a requirement to add/remove these sites on the fly. Is there any api or any other way provided by Google with which I can achieve this?

Here you can find the relevant information:
https://developers.google.com/custom-search/docs/tutorial/creatingcse
To create a custom search engine:
Sign into Control Panel using your Google Account (get an account if you don't have one).
In the Sites to search section, add the pages you want to include in your search engine. You can include any sites you want, not just
the sites you own. You can include whole site URLs or individual pages
URLs. You can also use URL patterns.
https://support.google.com/customsearch/answer/71826?hl=en
URL patterns
URL patterns are used to specify what pages you want included in your
custom search engine. When you use the control panel or the Google
Marker to add sites, you're generating URL patterns. Most URL patterns
are very simple and simply specify a whole site. However, by using
more advanced patterns, you can more precisely pick out portions of
sites.
For example, the pattern 'www.foo.com/bar' will only match the single
page 'www.foo.com/bar'. To cover all the pages where the URL starts
with ' www.foo.com/bar', you must explicitly add a '' at the end. In
the form-based interfaces for adding sites, 'foo.com' defaults to
'.foo.com/*'. If this is not what you want, you can change it back in
the control panel. No such defaulting occurs for patterns that you
upload. Also note that URLs are case sensitive - if your site URLs
include capital letters, you'll need to make sure your patterns do as
well.
In addition, the use of wildcards in URL patterns allows you to
include or exclude multiple pages or portions of a site all at once.
So, basically you've to navigate to the "Sites to search section" and enter the needed sites there. If you want to change these site on the fly, you've to manipulate your URL pattern.
There's also an option to use the XML configuration files. You just have to add (or remove) your sites there:
https://developers.google.com/custom-search/docs/annotations
Annotations: The annotations XML file lists the webpages or websites
you want your search engine to cover, and indicates any preferences
you have about how these sites should be ranked in your search
results. Each site and its associated information is called an
annotation. More information about the annotations XML file.
An example for an annotation:
<Annotation about="http://www.solarenergy.org/*">
<Label name="_cse_abcdefghijk"/>
</Annotation>

Using api we can add filter "siteSearch"=>"somedmain.com somdomain2.com","siteSearchFilter"=>"e" but there will be spacing between seperate domains.

Related

Search Database via Google Custom Search? Attached Google CSE to (SQL/NoSQL) Database for website?

TOPIC - Google Search Engine / Custom Search - with Database
References
Search for "Google Search Engine" and "Google Custom Search"
(New to StackOverflow; just joined the other day.I'm limited to 2 links I can post right now).
NOTE:
I have not YET decided/committed to any specific coding language, framework, etc. Not until I figure out how to accomplish my question (below).
BACKGROUND INFO
What I'm trying to do (for now) is add a "search-box/ search engine" to a simple website I'm building out. Before I get too far into it (planning ahead) I would like to use Google CSE if all possible (which can do A LOT of things and works well). However, I will have a database (not sure on type YET. Will depend on what my options and I can do with CSE) of "items" that I want to be able to quickly search (in the search-box) i.e. like Amazon.com.
QUESTION:
Is there any way at all, to use Google Custom Search and or Custom Search API to search/attach a database (SQL, NoSQL, or others)? I would HIGHLY prefer being able to do all of this in Google Cloud Platform, and use one of their storage/database products.
If I get what you try to do, Google CSE is enough.
From the google doc you linked :
#Defining a Custom Search Engine in Control Panel
In the Sites to search section, add the pages you want to include in
your search engine. You can include any sites you want, not just the
sites you own. You can include whole site URLs or individual pages
URLs. You can also use URL patterns.
#Enabling Autocomplete
[...]you can enable or disable autocomplete feature using
enableAutoComplete attribute.
For the Is there any way at all [..] to search a database, I'll said not directly, but it's not a big problem.
Google CSE work on "indexable web pages", so it'll not work again a raw DB, restricted internet, or custom network not under http(s)://.
But in your case, if you make a DB, I suppose you'll have to make web page to display the data you store inside to your users ? (like products pages on Amazon)
If yes, then you'll run Google CSE again these pages by adding your http://[server ip] or http://[domain name] in the white list.
As far as I know, custom search won't guarantee all your content will be indexed.
You probably want to try exporting a full sitemap.xml, a RSS feed and if the custom search results from either of these won't satisfy you, you will probably want to look at the google search appliance product.
There's also http://sphinxsearch.com/ by the way.

Duplicate content and international sites clarification

Something is not clear, here is my case:
i want to have have the same content for us and uk people,
could i safely avoid duplicate content with thoses url:
www.example.us/info.html (hosted on us server)
www.example.co.uk/info.html (hosted on uk server)
from google :
Websites that provide content for different regions and in different languages sometimes create content that is the same or similar but available on different URLs. This is generally not a problem as long as the content is for different users in different countries. While we strongly recommend that you provide unique content for each different group of users, we understand that this might not always be possible. There is generally no need to "hide" the duplicates by disallowing crawling in a robots.txt file or by using a "noindex" robots meta tag. However, if you're providing the same content to the same users on different URLs (for instance, if both example.de/ and example.com/de/ show German language content for users in Germany), you should pick a preferred version and redirect (or use the rel=canonical link element) appropriately. In addition, you should follow the guidelines on rel-alternate-hreflang to make sure that the correct language or regional URL is served to searchers.
Seems not clear for me, what do you think about my case ?!
flau
Go for hreflang. When implemented properly, you will avoid all duplicate content issues.
if you're providing the same content to the same users on different URLs (for instance, if both example.de/ and example.com/de/ show German language content for users in Germany), you should pick a preferred version and redirect (or use the rel=canonical link element) appropriately. In addition, you should follow the guidelines on rel-alternate-hreflang to make sure that the correct language or regional URL is served to searchers
That covers your scenario:
Choose one as your preferred URL for the US and make it redirect (or use canonical), and
Follow hreflang guidelines: https://support.google.com/webmasters/answer/189077?hl=en

custom google search autosuggestion

I have created a custom google search application for my website.
Below is the url
https://www.google.com/cse/
to create application.
Under the Auto complete section i have enabled autocomplete,but still it dont suggest me the options when i start typing on search box.
It suggest me only the keywords that We define under custom autocompletions.
SO my question is : Do we need to provide all the custom keywords that we want to autocomplete in search box or google just creates its own autosuggestion from the website ?
It creates autosuggestions from the website, but it may take few days to collect all data. It also takes into account user's queries on your website.
As previous answer suggests, autocomplete can take a few days to begin working. In addition you must specify one or more specific sites / pages to which to restrict the custom search engine. On occasion, you may also have to specify that the CSE only search those site(s) / page(s) instead of simply emphasizing them in the results.

How to restrict Google Custom Search to specific url prefixes?

If I go to the Google CSE control panel (https://www.google.com/cse/all) I see a list of my custom search engines.
When I click on one I can see in the list the option "sites to search". There I can list
example.com/cool-path
example.com/awesome-path
etc
How do I use the API to do the same? To add multiple domains, sites, or paths to the search? I can't find any documentation specifying this behavior.
The API will connect to a search engine that is specified on your custom search panel.
So you create the search engines you need (each with it's own list of sites to search) and then you connect to whichever one you need using the API using the engine's unique code

vBulletin forum under multiple domains

Hope someone will give me a hand with this problem I have. So here it goes.
There is a website with integrated vBulletin forum inside. The forum is accessible through
https://site.de/forum domain. The main site itself has many other domains based on locale. That is to say, there is a https://site.ch, https://site.it, https://site.at, etc (each one is in corresponding language).
Now there is a need to have this forum under at least 2 of this additional domains. I mean, there should be https://site.ch/forum domain, wich will contain the same forum, but with some differences in style and, of course, will have working inside-forum links with it's own domain (site.ch). The whole system is to be SEO-ed also.
So now my question is how to achieve this? I know there are some sort of plugins to manage multi-domain access, but they are not supported and are still in beta version.
At first, how to setup the forum to work under multiple domains?
And then, maybe I need to manually change some code to set the $vbulletin->options['bburl'] that is used to generate the links inside forum?
And the last one, how do I make all this search engine optimized??
You're asking numerous questions, you might get better results if you created a separate question for each of:
1) How to use one forum directory for multiple domains? (with the vbulletin tag and the tag for the web server you are using)
2) How to set the language based on the current domain in vbulletin? (with the vbulletin tag and one or more of these tags: localized, locale, multi-language, multilanguage)
3) Best practices for duplicate content presented in multiple languages on multiple domains (with the seo and vbulletin tags)
Some Answers:
1) If you're using the apache web server, you could add something like this to your httpd.conf file:
Alias /forums /var/www/...xxx.../forum_directory // use the path to your forum directory, no trailing slash
<Directory /var/www/...xxx.../forum_directory>
Order allow,deny
Allow from all
</Directory>
Then in the vbulletin ACP, change the setting for your basepath URL to "No":
Admin Control Panel -> Site Name / URL / Contact Details -> Always use Forum URL as Base Path
2) There are a few plugins that detect the language used by the browser and set vBulletin to use that language:
Language Detection
Set forum-language automatic to browser-language for first-time-visitors
3) SEO covers many things, but to deal with having duplicate content on multiple domains you can look at the Google Webmaster Central Blog.
This posting is helpful:
Working with multi-regional websites
A section from the post: Dealing with duplicate content on global websites
Websites that provide content for different regions and in different languages sometimes create content that is the same or similar but available on different URLs. This is generally not a problem as long as the content is for different users in different countries. While we strongly recommend that you provide unique content for each different group of users, we understand that this may not always be possible for all pages and variations from the start. There is generally no need to "hide" the duplicates by disallowing crawling in a robots.txt file or by using a "noindex" robots meta tag. However, if you're providing the same content to the same users on different URLs (for instance, if both "example.de/" and "example.com/de/" show German language content for users in Germany), it would make sense to choose a preferred version and to redirect (or use the "rel=canonical" link element) appropriately.
I don't have anything on the other search engines.