Getting The Results of a Google Search Programmatically via Custom Search - google-custom-search

I am trying to send a request to Google.com via Custom Search API and get the response with a proper format (XML or HTML preferably). From the Custom Search API website, I have seen that this is actually possible via "Retrieving the Code for the Search Results" (it is here). The thing is that I cannot get it working. Every time the broken robot from Google shows up. I was wondering if anyone had any experience with it and they could help me. I am trying to use the search results for a small project.
Here are the things I have done:
I can use the Google search API in general (I have used it with the search text box)
I have set up my Custom Search API to search the entire web.
Here are a bunch of things that I am not going to do:
I'm not trying to have a Google search box in my site.
I'm not trying to grab what Google says by parsing the Google.com page.
Here is what I need to do:
I need the content of what Google returns as search results via whatever API Google has to offer.
I will be using PHP for writing this program. If anyone has a better way to get these search results in a proper format, it is very appreciated.

Related

Knowledge graph API returning different results than site

When I Google for something and click on a heading from the Knowledge Graph, Google displays a list of the results just below the search bar:
Screenshot here
I typed in the same thing in my Knowledge Graph API:
https://kgsearch.googleapis.com/v1/entities:search?query=apple%20products&key=[MY_KEY]&indent=True&limit=100
and the results are not the same. Short of scraping, is there a way of obtaining the exact list shown within the results via a Google API? It doesn't seem to be Knowledge Graph, or at least the queries are not the same.
Short of scraping, is there a way of obtaining the exact list shown
within the results via a Google API? It doesn't seem to be Knowledge
Graph
Apparently not, you are using the correct API but unfortunately that is how the API works—it usually gives you a different result than if you do a simple Google search.
Why this is the case—I don't know. I decided scrape google instead.
The Knowledge Graph API doesn't have access to the same data sources as a simple Google search. You can read about here if interested (https://web.archive.org/web/20130329151128/http://zecblog.com/2012/09/16/the-short-life-of-the-open-knowledge-graph/).
But yes in short, you'll have to do your own scraping if you want the same information, although it's difficult because Google purposely obfuscates the HTML tags.

Google Custom search API for BOTS

I am creating a kind of BOT. I would like to add web search functionality to my bot. i.e. I would like my BOT to search the web for user if your says so.
To implement that I am thinking of using GOOGLE search engine. But on overviewPage of its Custom Search API, it says
Create custom search engines that search across a specified collection
of sites or pages.
Is there any way that this specified collection defaults to whole web. or is there another way round this.

Using google custom search API to find images

I want to use Google's Custom Search API to find images using GET request. Like described here. But I don't want to look on images on one specific site, but want to get results like this. How I can do this with the Custom Search API?
Sending this request returns nothing:
https://www.googleapis.com/customsearch/v1?key=[MY_KEY]&cx=017576662512468239146:omuauf_lfve&q=some+request&&safe=off&searchType=image&fileType=png. This cx is from google's manual
I must set cx param, but this value links my request to site, specified in cse control panel. However I don't want to limit my results to one site.
According to the documentation here:
Google Custom Search > Overview
Google Custom Search enables you to create a search engine for your website, your blog, or a collection of websites.
It doesn't sound like that is what you are trying to do (you don't seem to want to search a predefined subset of resources).
I'm aware the question is over 6 years old, but better late than never. It's possible to create a Google programmable search engine that can search the entire web for images. When you create a programmable search engine, there is an option to search the entire web instead of just web sites that you specify. See https://programmablesearchengine.google.com/about/.
As an alternative, you could try Bing Image Search API. There is a limit to the number of image searches you can submit per day for free, but I have rarely hit the limit.

Can I “reuse” Google search results in my application?

I would like to understand if and how it is possible to achieve the following:
Create an app that sends search requests to Google (search requests
inputted by a user!)
Fetch the search results present in a different
format/graphics/layout the search results to the user.
Is that possible or Google would prevent me from doing such a thing? (via a
CAPTCHA for example)
If it is possible what kind of instrument would I need to use to capture (parse?) the search results???
My application would be developed for Android.
Screen-scraping the search results from the Google search site is a violation of their terms of service. I don't think they would use CAPTCHA to prevent you from doing so, but you might get a letter from their lawyers.
However, you could use the Google Custom Search API which allows you to search Google and customize layout of the results. However, you need to acquire a developer key and also follow their terms of service.
Also, Android has a search component built into it that you can probably use. See http://developer.android.com/guide/topics/search/index.html.

How can I retrieve a search snippets from google?

I'm working with a simple app, that allows user to enter some text, then delegates it to the google.com, searching for this text. Something like a Google search bar in Firefox.
In Google toolbar, there is a feature: when you start typing the text, some snippets appear (sorted, as far as I understand, by the number of search results), that allow you to enter your text quickly by choosing one of the snippets.
My question is how can I retrieve these snippets from Google? Does Google provide any API for doing it? I've googled this question, but Google seems to provide a search API, and allows to retrieve search results, not snippets.
p.s. I would appreciate any links, related to the question, or tutorials, or code snippets or something. I am writing a simple WinAPI application in C/C++. Thanks in advance!
Looks like I've got it!
Google provide a simple, but handy REST API. You form a special URL, then send an HTTP request and receive a formatted list of necessary search suggestions. Looks like Google supports many formatting options, for example JSON and XML.
see more info on:
http://blogoscoped.com/archive/2006-08-17-n22.html