I would like to get up-to-date information on Google's index of a website, and it seems
that results vary depending on which datacentre happens to process your search query.
You can use one of the many tools designed specifically for this. One of them is Google Datacenter Search at iWebTool.
(It's really all about getting a list of data center IPs and sending the same GET variable to them as to google.com.)
Related
i am using the Google CSE Json API in order to receive results on generic queries from Google.
I have configured the Google Engine to return results from all web sites.
i though that this setting would make me use Google as if i was using it from the Google web site via the regular search engine.
BUT - i don't get same results that i would have expected to get from the site. i have major differences and i was wondering why.
From the past reading that i have made, i know that the API uses a certain server(s) around UK, making the results not the same due to the locale settings.
i have read the documentation on the CSE site and saw that there are 2 parameters that i thought would have improved the state:
googlehost - specifying the domain to use. this parameter is deprecated according to the documentation, hence i used the second parameter
gl - specify a country code for the search
both parameters didn't affect my results at all.
i am struggling with this for quite a long time and would appreciate a proper solution for it.
all i want is to have a CSE that acts the same as the google website. Running a search here and there should not differ in the returned results.
your help is highly appreciated.
Regards,
attiass
When I Google for something and click on a heading from the Knowledge Graph, Google displays a list of the results just below the search bar:
Screenshot here
I typed in the same thing in my Knowledge Graph API:
https://kgsearch.googleapis.com/v1/entities:search?query=apple%20products&key=[MY_KEY]&indent=True&limit=100
and the results are not the same. Short of scraping, is there a way of obtaining the exact list shown within the results via a Google API? It doesn't seem to be Knowledge Graph, or at least the queries are not the same.
Short of scraping, is there a way of obtaining the exact list shown
within the results via a Google API? It doesn't seem to be Knowledge
Graph
Apparently not, you are using the correct API but unfortunately that is how the API works—it usually gives you a different result than if you do a simple Google search.
Why this is the case—I don't know. I decided scrape google instead.
The Knowledge Graph API doesn't have access to the same data sources as a simple Google search. You can read about here if interested (https://web.archive.org/web/20130329151128/http://zecblog.com/2012/09/16/the-short-life-of-the-open-knowledge-graph/).
But yes in short, you'll have to do your own scraping if you want the same information, although it's difficult because Google purposely obfuscates the HTML tags.
Is it possible to fetch ads is search results using the google custom search API ?
Need this to do a simple experiment. Search something and to see if any ad appears in the results.
Is it possible to do this ?
You probably want to screen scrap the search result window. Use some tool like phantomjs and inspect the results.
You could open one Google AdWords account and there you can make sample queries and experiments and whatever you want.
Google are not really a big fan of the experiments on their front-end, on Google AdSense, on Google CSE, CSA, etc. You could try, but you might get a wrong picture - there are some protection algorithms on the front-end.
Im looking to use Google Adwords Keyword tool data on a website. Ive been looking around in the API and I cant find much to match what I need. I noticed a lot of keyword research tool websites use google as their main source for their information. How would I go about doing this and extracting the data and have it run on a website automatically so it wouldnt need to be updated manually each month?
you can use the Traffic Estimator service in the AdWords API:
https://developers.google.com/adwords/api/docs/reference/v201409/TrafficEstimatorService
Be warned that this is notoriously inaccurate (which is odd given that you would think Google had its own data to call upon!)
I use the TargetingIdea service in the AdWords API to generate lists of keywords to use for building AdWords campaigns. (https://developers.google.com/adwords/api/docs/reference/v201409/TargetingIdeaService.TargetingIdea)
First off you need an API key - they're not that easy to get and your app needs to offer a whole lot of features to meet the required minimum functionality - take a look here https://developers.google.com/adwords/api/docs/requirements
Once you've jumped over that hurdle you get the data from Google by sending a request to the service. That request includes some targeting criteria like location and language and also a "seed" keyword. You can also specify if you want closely related results or broadly related results.
For example if you sold tractors you'd put 'tractors' in as a seed keyword and then the API would return either closely related terms like 'tractors for sale', 'used tractor spares' etc or more broadly related terms like 'agricultural machinery'.
I would like to understand if and how it is possible to achieve the following:
Create an app that sends search requests to Google (search requests
inputted by a user!)
Fetch the search results present in a different
format/graphics/layout the search results to the user.
Is that possible or Google would prevent me from doing such a thing? (via a
CAPTCHA for example)
If it is possible what kind of instrument would I need to use to capture (parse?) the search results???
My application would be developed for Android.
Screen-scraping the search results from the Google search site is a violation of their terms of service. I don't think they would use CAPTCHA to prevent you from doing so, but you might get a letter from their lawyers.
However, you could use the Google Custom Search API which allows you to search Google and customize layout of the results. However, you need to acquire a developer key and also follow their terms of service.
Also, Android has a search component built into it that you can probably use. See http://developer.android.com/guide/topics/search/index.html.