Google Search Image "no longer available: 403" [duplicate] - google-search-api

I am using google image search API. Till yesterday it was working, but today morning it says "This API is no longer available"
Is it officially closed, Or any error at my side
Request
https://ajax.googleapis.com/ajax/services/search/images?v=1.0&rsz=8&q=cute+kittens
Response
{"responseData": null, "responseDetails": "This API is no longer available.", "responseStatus": 403}

The answer I found was using Google's Custom Search Engine (CSE) API. Note that this is limited to 100 free requests per day.
Creating cx and modifying it to search for images
Create custom search engine at https://cse.google.com/cse/create/new based on your search criteria.
Choose sites to search (leave this blank if you want to search the entire web, otherwise you can enter a site to search in one particular site)
Enter a name and a language for your search engine.
Click "create." You can now find cx in your browser URL.
Under "Modify your search engine," click the "Control Panel" button. In the "edit" section you will find an "Image Search" label with an ON/OFF button, change it to ON. Click "update" to save your changes.
Conducting a search with the API
The API endpoint url is https://www.googleapis.com/customsearch/v1
The following JSON parameters are used for this API:
q: specifies search text
num: specifies number of results. Requires an integer value between 1 and 10 (inclusive)
start: the "offset" for the results, which result the search should start at. Requires an integer value between 1 and 101.
imgSize: the size of the image. I used "medium"
searchType: must be set to "image"
filetype: specifies the file type for the image. I used `"jpg", but you can leave this out if file extension doesn't matter to you.
key: an API key, obtained from https://console.developers.google.com/
cx: the custom search engine ID from the previous section
Simply make a GET request by passing above parameters as JSON to the API endpoint (also listed above).
Note: If you set a list of referrers in the search engine settings, visiting the URL via your browser will likely not work. You will need to make an AJAX call (or the equivalent from another language) from a server specified in this list. It will work for only the referrers which were specified in the configuration settings.
Reference:
https://developers.google.com/custom-search/json-api/v1/reference/cse/list

Now You can search images with Custom image search API.
You can do this with two steps:
Get CUSTOM_SEARCH_ID
Go to - https://cse.google.ru/cse/all
Here you must create new Search Engine. Do this and enable Image Search at there.
Screen(i am Russian... sorry)
then get this search engine ID. To do this press at Get Code button:
And there find line with cx = "here will be your CUSTOM_SEARCH_ID":
Ok. It's done, now second step:
Get SERVER_KEY
Go to google Console - https://console.developers.google.com/project
Press to Create project button, enter the name and other required information.
Pick this project and go to Enable Apis
Now find Custom Search Engine.
And Enable it.
Now we must go to Credentials and create new Server Key:
Ok. Now we can use Image Search.
Query:
https://www.googleapis.com/customsearch/v1?key=SERVER_KEY&cx=CUSTOM_SEARCH_ID&q=flower&searchType=image&fileType=jpg&imgSize=xlarge&alt=json
Replace the SERVER_KEY and CUSTOM_SEARCH_ID and call this request.
Limit: for free you can search only 100 images per day.

If this is just for your own purposes (not for production) and you're not planning to abuse Google Image Search, you can simply extract first image URL from Google search results using JSOUP.
For example:
Code to retrieve image URL of the first thumbnail:
public static String FindImage(String question, String ua) {
String finRes = "";
try {
String googleUrl = "https://www.google.com/search?tbm=isch&q=" + question.replace(",", "");
Document doc1 = Jsoup.connect(googleUrl).userAgent(ua).timeout(10 * 1000).get();
Element media = doc1.select("[data-src]").first();
String finUrl = media.attr("abs:data-src");
finRes= "<img src=\"" + finUrl.replace("&quot", "") + "\" border=1/>";
} catch (Exception e) {
System.out.println(e);
}
return finRes;
}
Guide:
question - image search term
ua - user agent of the browser

After I read several responses I compiled a response with images:
Access the website: https://developers.google.com/custom-search/v1/introduction, on the page you will find this part, so click in the button Get a Key:
Create or select a project, and then NEXT:
Copy the API KEY:
Access the website to create your CX: https://cse.google.com/cse/create/new, write some random domain like “www.anypage.com”, (after we will delete), select a language, and define some name for your search engine. Click on the Button CREATE.
Will see this page, then click in Control Panel:
Copy the Search engine ID for later (this is your CX). After you can set to search in all websites (active Search the entire web, select on the random website www.anypage.com then click on the button Delete) and you can active Image search. So will see like this:
And Using REST you can get the results, using this example code (searching for flower):
<html lang="pt">
<head>
<title>JSON Custom Search API Example</title>
</head>
<body>
<div id="content"></div>
<script>
function hndlr(response) {
console.log(response);
for (var i = 0; i < response.items.length; i++) {
var item = response.items[i];
// in production code, item.htmlTitle should have the HTML entities escaped.
document.getElementById("content").innerHTML += "<br>" + item.htmlTitle;
}
}
</script>
<script src="https://www.googleapis.com/customsearch/v1?key=API_KEY&cx=SEARCH_ENGINE_KEY&q=flower&searchType=image&callback=hndlr"></script>
</body>
</html>
The base code is found here: https://developers.google.com/custom-search/v1/using_rest
After setting your API_KEY (key) and your SEARCH ENGINE KEY (cx), the result will see like this:
Thanks to #Vijay Shegokar, #aftamat4ik and #Alladinian

This is the full URL template to be used
We can eliminate unnecessary parameters.
https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&lr={language?}&safe={safe?}&cx={cx?}&cref={cref?}&sort={sort?}&filter={filter?}&gl={gl?}&cr={cr?}&googlehost={googleHost?}&c2coff={disableCnTwTranslation?}&hq={hq?}&hl={hl?}&siteSearch={siteSearch?}&siteSearchFilter={siteSearchFilter?}&exactTerms={exactTerms?}&excludeTerms={excludeTerms?}&linkSite={linkSite?}&orTerms={orTerms?}&relatedSite={relatedSite?}&dateRestrict={dateRestrict?}&lowRange={lowRange?}&highRange={highRange?}&searchType={searchType}&fileType={fileType?}&rights={rights?}&imgSize={imgSize?}&imgType={imgType?}&imgColorType={imgColorType?}&imgDominantColor={imgDominantColor?}&alt=json
I am using
https://www.googleapis.com/customsearch/v1?key=ap_key&cx=cx&q=hello&searchType=image&imgSize=xlarge&alt=json&num=10&start=1

Change the API url to
Google Custom Image search
Provide the same parameters along with with API KEY and CX.
More Info and Explorer

The Yahoo Boss API is a reasonable substitute, although it's not free and the results are not quite as good.
UPDATE: YAHOO BOSS JSON Search API will discontinue on March 31, 2016

SerpAPI enables to search through Google Images and returns a clean json. it integrates with most of the programming languages: python, php, java, golang, nodejs...
https://serpapi.com/images-results
Google limit the number of search per day.
but this service provides unlimited searches...

looks like we need to implement google custom search API
https://developers.google.com/custom-search/
says so on top of the page you provided yourself

Related

Google Custom Search Engine - Edit the URLs in SERP

Good evening, I'm working with Google Custom Search Engine: https://cse.google.it/cse/ and I need to add, only for some URLs (in total 10 URLs) a parameter at the end of each search results URL. For example: if the domain is www.dominio-test.com then the URL to show is www.dominio-test.com/123456 the CSE searches on the web without restriction.
The code I use is the following:
<script async src = "https://cse.google.com/cse.js cx=014431187084467459449:v2cmjgvgjr0"> </script>
<div class = "gcse-search"> </div>
I thought of proceeding with reading the DOM with getElementsByClassName (), extracting the content and adding it to a variable, at this point add the if / else / replace controls and output again.
But I don't know how to proceed, can you please support me?
Thanks and good job
Search Element callbacks might work for you: https://developers.google.com/custom-search/docs/element#callbacks

Extract portion of HTML from website?

I'm trying to use VBA in Excel, to navigate a site with Internet explorer, to download an Excel file for each day.
After looking through the HTML code of the site, it looks like each day's page has a similar structure, but there's a portion of the website link that seems completely random. But this completely random part stays constant and does not change each time you want to load the page.
The following portion of the HTML code contains the unique string:
<a href="#" onClick="showZoomIn('222698519','b1a9134c02c5db3c79e649b7adf8982d', event);return false;
The part starting with "b1a" is what is used in the website link. Is there any way to extract this part of the page and assign it as a variable that I then can use to build my website link?
Since you don't show your code, I will talk too in general terms:
1) You get all the elements of type link (<a>) with a Set allLinks = ie.document.getElementsByTagName("a"). It will be a vector of length n containing all the links you scraped from the document.
2) You detect the precise link containing the information you want. Let's imagine it's the 4th one (you can parse the properties to check which one it is, in case it's dynamic):
Set myLink = allLinks(3) '<- 4th : index = 3 (starts from zero)
3) You get your token with a simple split function:
myToken = Split(myLink.onClick, "'")(3)
Of course you can be more synthetic if the position of your link containing the token is always the same, like always the 4th link:
myToken = Split(ie.document.getElementsByTagName("a")(3).onClick,"'")(3)

Implementing Google Custom Search with filtering

I have to implement a page with multiple google-powered search forms. We have a license from Google for CSE, and this is the situation:
I have a search form that's present at the top of every page that performs a simple search and displays the results in a separate page. This works.
I have a particular page that, in addition, shows another two search forms: one should filter articles by category, another should filter articles by category and restrict the result to a certain month. I have added a meta key with the publication date to each article for this.
I have gotten a bit lost in the documentation, though: if I add
<gcse:searchbox-only resultsUrl="/[site]/stat/search/google_search_results.html"></gcse:searchbox-only></div>
to the page, I can't filter the results. If I start to meddle with a CustomSearchObject, I don't see an option to show results on a different page.
For category-based filtering, I've tried appending
more:pagemap:metatags-taxonomies:news
to the query argument in the results page URL, and it does work, but I don't understand how to inject this to the form.
For restricting based on dates, I tried adding
&sort=more:pagemap:metatags-pubdate:r:YYYYMMDD:YYYYMMDD
but haven't been able to make it work. Getting the XML does work:
http://www.google.com/search?q=intitle:[mysite]%20more:pagemap:metatags-taxonomies:News&sort=metatags-pubdate:r:20120401:20120830&cx=[mykey]client=google-csbe&output=xml
returns correct results.
Is there documentation that doesn't assume so much? All I find are code snippets without context. I've checked Filtering and sorting, Custom Search Element Control API, and of course this site, but I can't put all the pieces together.
I managed to implement what I wanted. In the search page, I built simple forms pointing to my results pages (this might not be doable if you must implement google branding), and in the results page I put the following:
(in the <head>)
<script src="http://www.google.com/jsapi"></script>
<script>
// This function extracts the query from the URL (if GET) or builds a search query.
// Code removed to simplify the example.
function buildQuery () {
return '<?php echo $_POST['q'];?> more:pagemap:metatags-taxonomias:News'); // injecting the taxonomy metatag filter
}
google.load('search', '1', {language : 'es'});
google.setOnLoadCallback(function() {
var customSearchOptions = {};
customSearchOptions[google.search.Search.RESTRICT_EXTENDED_ARGS] = {'sort':'metatags-pubdate:d,metatags-pubdate:r:<?php echo $_POST['startdate'];?>:<?php echo $_POST['enddate'];?>'}; // these come from the POST request, are processed earlier in the script.
var customSearchControl = new google.search.CustomSearchControl('XXXXXXXXXXX', customSearchOptions); // Put your own App key here.
customSearchControl.setResultSetSize(google.search.Search.FILTERED_CSE_RESULTSET);
var drawOptions = new google.search.DrawOptions();
drawOptions.enableSearchResultsOnly(); // I don't want the search box here
customSearchControl.draw('cse-results-press', drawOptions);
var query = parseQuery();
if (query) {
customSearchControl.execute(query);
}
}, true);
</script>
In the <body>:
<div id="cse-results-press">Loading...</div>

How to get the most popular page under a section using Site Catalyst (Omniture)?

I want to use the site catalyst api to get the most popular page (page that has maximum pageviews) under a given site-section? Let me explain better with an example.
My website has multiple channels (News/Journals/Books/Events etc). Each of the channel has many pages under it. I want a api call that will get the most popular news page or Journals page or Books page etc.
I am passing data to sitecatalyst like this..
s.pageName = o_title;
s.channel = o_structure
s.prop1 = o_iden
where o_title has the page title of a newspage or a journalpage etc..
o_structure refers to either "News" or "Journals" or "Books" etc.
Currently I am able to only get the most popular page on the entire site. I would appreciate if someone can help me find the most popular page per section.
Thanks,
Rag
I assume that you already know the basics of using the Omniture API, how to queue up a report and look for status and get it etc... here is a basic REST data string to get site sections (s.channel) broken down by page names (s.pageName) with page views as metric. You will use the Report.QueueRanked API method, and you need to specify the rsid(s) to get the data from and the date ranges.
{
"reportDescription": {
"reportSuiteID":"RSID",
"dateFrom":"YYYY-MM-DD",
"dateTo":"YYYY-MM-DD",
"metrics":[
{
"id":"pageViews"
}
],
"sortBy":"pageViews",
"elements":[
{
"id":"siteSection"
},
{
"id":"page"
}
]
}
}

Handling a return from Google Checkout

I need to capture server side a notification of a users payment. I've gone through a good majority of the documentation with no luck, but will go through it again. Is there a way to enable a "return to site" after they complete the order?
I searched google checkout return url and got http://www.phpexpertsforum.com/return-url-in-google-checkout-after-payment-t561.html
veerlapallavi wrote...
There are 2 methods through which you
can provide the return URL. 1) Go to
your Google checkout account and Open
the settings. In the settings>>Profile
you can find a field with name "Public
business website:" provide your return
URL there.
2) The second method is , You can pass
the value to Google Checkout with the
HTML form as a hidden variable.
<input name="continue_shopping_url"
type="hidden"
value="http://www.yousite.com/payment_success.php">
OR
Set a function as below
SetContinueShoppingUrl("http://www.yousite.com/payment_success.php");
function SetContinueShoppingUrl($url)
{ $this->continue_shopping_url =
$url; }
Hope that helps.
It looks like you can configure gcheckout to send you XML notifications, but I don't see how you can associate it with a particular order ID other than matching fields or something...
http://code.google.com/apis/checkout/developer/Google_Checkout_XML_API_Notification_API.html#new_order_notifications