Custom Search API not returning all results - google-custom-search

I am a long time customer of using the Custom Search API.
The problem - as described in the CSE documentation - is that the API is intended to search your own site and not the web in general. It misses results, for example from books.google.com, and results from other languages etc.
Is there another (paid) API that returns all results?
Sample search string: "الاستخدامات التالية من التطبيق"
(The above search gets 1 result in Google Search but 0 results in the Custom Search I am paying for.)
Thanks.

I didn't want to switch to Bing, but I was getting better results in the end.
For anyone else having this issue:
https://learn.microsoft.com/en-us/rest/api/cognitiveservices/bing-web-api-v7-reference

Related

Google Search API and Get result from specific Website

Is it possible for me to get a search results based on a query from specific website using google search api. For example query = "car fix" and only show results from "www.wikihow.com".
I prefer the return in json/html format.
Thanks all.

bing search api azure location based results

I am using the bing search api https://api.datamarket.azure.com/Bing/Search to do searches and get the results back in json. This works however ....
When I do a search using the bing.com web page and search for say camping the results are biased towards my location (aberdeen)so I get campsites in aberdeen in the results. Is there any way I can add my location to the search query in the api to get the same location based results
Unfortunately this is not possible. According to their documentation you can pass in Lat / Long but it still does not bring back location based results.
I did some tests just to make sure by using their dataset explorer and it confirmed that location based results, even with a supplied lat/ long, does not work.
Example Query:
https://api.datamarket.azure.com/Bing/Search/v1/Web?Query=%27english%20pub%27&Latitude=26.3686&Longitude=80.1000

Google custom search engine result counts changing as results are paged

I have implemented a Google custom search engine in a website. When searching a word, it brings out results like totalResults - 168 and I am retrieving it by 10 results per page.
Up to 60 results it works fine, but in the 7th page, the total results from Google api response changes to a total of 67.
I am using a free version of the Google custom search API. I don't know whether it is working correctly or not. Please provide me a solution if it is wrong or correct me if I am wrong.
It is because of google search behaviour well explained in following links.
https://productforums.google.com/forum/m/#!topic/customsearch/D4--2TfYk9A
https://productforums.google.com/forum/m/#!topic/customsearch/SjlrUMa-X-k

Google Custom Search API: Using it as a scraper?

Is this API simply for searching your website only, or can any standard google search (even advanced search features) be submitted to it? I understand there is a limit of 100 per day, I am just curious if it can be invoked from say your own machine as the code samples and introduction indicate its intended use is for displaying results on your website. I want to search outside of a given domain and scrape standard google results for any given search. This will not be an ajax call.
My current understanding:
You're limited to 100/day only if you don't pay.
You do have to specify domains, but some tlds are fine (eg: .uk)
There's a limit to 100 search results for any given search query (ten pages of up to ten responses)
It can be invoked from your own machine.

Filter google query results

I'm writing a search engine for wikipedia articles using lucene on the wiki xml dump and I want to calculate the accuracy of the engine when compared to google wiki result on a particular query, when I give "site:en.wikipedia.org" along with the query. I want to do it for multiple queries so I'm getting the google search result URLs manually. I got Google APIs to use a bot to search Google but the problem is I want to get rid off certain type of results like
"/Category:"
"/icon:"
"/file:"
"/photo:"
and user pages.
But I haven't found a convenient way to do this except for using an iterative method of issuing a query, get n number of results, then filter out by using regular expressions, then retrieve the remaining (n-x) results and so on. Google keeps blocking me when I do that.
Is there an intelligent way to get Google results the way I want using Java?
Thanks in advance guys.
You could just try excluding those pages from the Google results, like this:
living people site:en.wikipedia.org -inurl:category -inurl:category_talk -inurl:file -inurl:file_talk -inurl:user -inurl:user_talk