I am building a software that processes Google search results for a linguistic purpose. It makes multiple searches at a fast rate and opens links (10 to 100 links per search) at a fast rate to access the websites and extract texts.
I will be accessing Google through Custom Search API. The Custom Search terms of service appear to disallow this type of access because they say that the results cannot be modified or crawled; however, I have found an interpretation that allows some wiggle room for my software. Now my question: If I go ahead and build the software, will the Custom Search block the access?
Related
I need to find the maximum elevation along a given path (defined by start and end latitude/longitude), but in searching the internet the only option I seem to find is the Google elevation api. Unfortunately, that's not an option because the google terms of service explicitly state that:
The Google Maps Elevation API may only be used in conjunction with displaying results on a Google map. It is prohibited to use Google Maps Elevation API data without displaying a Google map.
Since my usage does not involve a google map (I'm using Open streetmap data combined with FAA sectional charts), using the Google Maps Elevation API would directly violate this. Additionally, I want to say that Google prohibits commercial usage, which this is (although we may be able to pay an exorbitant fee to get around that restriction).
In any case, given that the Google Maps Elevation API isn't an option, what other options do I have, either in a web-based API similar to Google, or in some sort of a local database download that I can query?
The USGS offers downloads of elevation data in various formats at https://nationalmap.gov/small_scale/atlasftp.html I was able to download a GeoTIFF of the area of interest, load it into a PostgreSQL database, and use the PostGIS extension to perform queries against the data to retrieve the desired information. No licensing issues or number of data point restrictions, plus it's all local.
I want to use Google's Custom Search API to find images using GET request. Like described here. But I don't want to look on images on one specific site, but want to get results like this. How I can do this with the Custom Search API?
Sending this request returns nothing:
https://www.googleapis.com/customsearch/v1?key=[MY_KEY]&cx=017576662512468239146:omuauf_lfve&q=some+request&&safe=off&searchType=image&fileType=png. This cx is from google's manual
I must set cx param, but this value links my request to site, specified in cse control panel. However I don't want to limit my results to one site.
According to the documentation here:
Google Custom Search > Overview
Google Custom Search enables you to create a search engine for your website, your blog, or a collection of websites.
It doesn't sound like that is what you are trying to do (you don't seem to want to search a predefined subset of resources).
I'm aware the question is over 6 years old, but better late than never. It's possible to create a Google programmable search engine that can search the entire web for images. When you create a programmable search engine, there is an option to search the entire web instead of just web sites that you specify. See https://programmablesearchengine.google.com/about/.
As an alternative, you could try Bing Image Search API. There is a limit to the number of image searches you can submit per day for free, but I have rarely hit the limit.
I have started a small IT service company recently in India. I was approached by an Big Enterprise asking to design a custom search engine (for internal/web usage) using the Google's custom search engine api.
it has to fetch info from a specific set of websites (internal websites / common web search)
if a particular website is selected before search, it should display the info from that website only.
Enterprise has a webpages that can be accessed with authentication. when user logged-in, the search has to fetch info from those authenticated webpages as well.
the contents should be segregated into images, books, movies, music, etc.
auto complete and search while typing
once we finish this design, may have to run it as a android application using phone gap(cordova lib).
My Questions:
Is Google allow me to do implementation for the Enterprise (they ready to use and pay Usage Charge for using Google search api), and Can I charge the enterprise for implementation and support (agreement will be made between my company and enterprise)?
Is Google provide ad free search results for custom search? If not can we do restriction on the result?
Is Google search engine can be extended to android app?
If your answer is "No" for above questions, if I develop search engine solution on my own, how can I do that?
Google Custom Search is searching only publicly available documents. For internal websites, you can check out Google Search Appliance: http://www.google.com/enterprise/search/products/gsa.html
I would like to understand if and how it is possible to achieve the following:
Create an app that sends search requests to Google (search requests
inputted by a user!)
Fetch the search results present in a different
format/graphics/layout the search results to the user.
Is that possible or Google would prevent me from doing such a thing? (via a
CAPTCHA for example)
If it is possible what kind of instrument would I need to use to capture (parse?) the search results???
My application would be developed for Android.
Screen-scraping the search results from the Google search site is a violation of their terms of service. I don't think they would use CAPTCHA to prevent you from doing so, but you might get a letter from their lawyers.
However, you could use the Google Custom Search API which allows you to search Google and customize layout of the results. However, you need to acquire a developer key and also follow their terms of service.
Also, Android has a search component built into it that you can probably use. See http://developer.android.com/guide/topics/search/index.html.
I remember years ago there used to be a widget you could use from the Google Custom Search engine that allowed you to embed details of popular searches on your public web pages. I have been looking for the documentation again for that recently so I can implemement it on a site that has a Custom Search Engine but now cannot find it - even with Google! Is this functionality still available or is it something that should be done with Google Analytics.
For an answer I am looking for links to official documentation.
There's a Custom Search help page that discusses this. Custom Search Help: Stats
The Stats tab of the Statistics and Logs page shows you popular
queries for your Custom Search Engine. You can view this info by day,
week, month, or year—even the entire lifetime of your search engine.
The Stats page provides accurate data about the search usage and the
total number of searches completed.
To display popular queries on your site (for example, to help
searchers find popular pages), copy the code in the Popular web
queries page and paste it into your pages.