Good evening, I'm working with Google Custom Search Engine: https://cse.google.it/cse/ and I need to add, only for some URLs (in total 10 URLs) a parameter at the end of each search results URL. For example: if the domain is www.dominio-test.com then the URL to show is www.dominio-test.com/123456 the CSE searches on the web without restriction.
The code I use is the following:
<script async src = "https://cse.google.com/cse.js cx=014431187084467459449:v2cmjgvgjr0"> </script>
<div class = "gcse-search"> </div>
I thought of proceeding with reading the DOM with getElementsByClassName (), extracting the content and adding it to a variable, at this point add the if / else / replace controls and output again.
But I don't know how to proceed, can you please support me?
Thanks and good job
Search Element callbacks might work for you: https://developers.google.com/custom-search/docs/element#callbacks
Related
So I have this bit as a part of the a code that came with the html template that I purchased. I was told that in order for this to work, I need to use the absolute URL of 'api/tweet.php'.
This is all I one line:
(function($){$.fn.twittie=function(options){var settings=$.extend({'count':10,'hideReplies':false,'dateFormat':'%b/%d/%Y','template':'{{date}} - {{tweet}}'},options);var linking=function(tweet){var parts=tweet.split(' ');var twit='';for(var i=0,len=parts.length;i<len;i++){var text=parts[i];var link="https://twitter.com/#!/";if(text.indexOf('#')!==-1){text=''+text+''}if(text.indexOf('#')!==-1){text=''+text+''}if(text.indexOf('http://')!==-1){text=''+text+''}twit+=text+' '}return twit};var dating=function(twt_date){var time=twt_date.split(' ');twt_date=new Date(Date.parse(time[1]+' '+time[2]+', '+time[5]+' '+time[3]+' UTC'));var months=['January','February','March','April','May','June','July','August','September','October','November','December'];var _date={'%d':twt_date.getDate(),'%m':twt_date.getMonth()+1,'%b':months[twt_date.getMonth()].substr(0,3),'%B':months[twt_date.getMonth()],'%y':String(twt_date.getFullYear()).slice(-2),'%Y':twt_date.getFullYear()};var date=settings.dateFormat;var format=settings.dateFormat.match(/%[dmbByY]/g);for(var i=0,len=format.length;i<len;i++){date=date.replace(format[i],_date[format[i]])}return date};var templating=function(data){var temp=settings.template;var temp_variables=['date','tweet','avatar'];for(var i=0,len=temp_variables.length;i<len;i++){temp=temp.replace(new RegExp('{{'+temp_variables[i]+'}}','gi'),data[temp_variables[i]])}return temp};this.html('<span>Loading...</span>');var that=this;$.getJSON('api/tweet.php',{count:settings.count,exclude_replies:settings.hideReplies},function(twt){that.find('span').fadeOut('fast',function(){that.html('<ul></ul>');for(var i=0;i<settings.count;i++){if(twt[i]){var temp_data={date:dating(twt[i].created_at),tweet:linking(twt[i].text),avatar:'<img src="'+twt[i].user.profile_image_url+'" />'};that.find('ul').append('<li>'+templating(temp_data)+'</li>')}else{break}}})})}})(jQuery);
Does anyone know how to use or get the absolute URL of tweet.php? I've tried researching or doing a google search on it but none seem to work.
The location of the tweet.php is
http://exampledomain.com/api/tweet.php
EDIT:
This is the thread discussion that I posted on their support website. I didn't share the link since it requires visitors to open an account with them just to view responses
Support Thread Picture
According to the Tweetie jQuery plugin documentation, you have to use the apiPath option:
$('.foo').twittie({
'apiPath': 'http://exampledomain.com/api/tweet.php',
});
But specifying the domain is discouraged and unnecessary. So :
$('.foo').twittie({
'apiPath': '/api/tweet.php',
});
I am templating pdfs in Netsuite using freemarker and I want to display the footer only on the last page. I have been doing some research, but couldn't find a solution (since looks like the environment does not allow me to include or import libs), so I thought that just comparing the number of the page with the total pages in an if tag would be a nice and easy workaround. I already know how to display the numbers by using the <pagenumber/> and <totalpages/> tags, but still cannot get them as values so I can use them like this:
<#if (pagenumber == totalpages) >
... footer html...
</#if>
Any ideas of how or where can I get those values from?
The approach you are trying won't work, because you are mixing BFO and Freemarker syntax. Netsuite uses two different "engines" to process PDF Templates. The first step is Freemarker, which merges the record fields with your template and produces an XML file, which is then converted by BFO into a PDF file. The <totalpages/> element is meaningless to Freemarker, as it is only converted into a number by BFO later.
Unfortunately, the ability to add a footer to only the last page of a document is currently a limitation of BFO, as per the BFO FAQ:
At the moment we do not have a facility for explicitly assigning a
footer or header to the last page in a document when the number of
pages is unknown.
You CAN add it after a page break - and put the page break at the end of the body
<pbr footer="nlfooter" footer-height="25%"></pbr>
</body>
The issue here is - on a one page output - you will get 2 pages minimum... it will always ADD a page for the disclaimer / footer...
I am using google image search API. Till yesterday it was working, but today morning it says "This API is no longer available"
Is it officially closed, Or any error at my side
Request
https://ajax.googleapis.com/ajax/services/search/images?v=1.0&rsz=8&q=cute+kittens
Response
{"responseData": null, "responseDetails": "This API is no longer available.", "responseStatus": 403}
The answer I found was using Google's Custom Search Engine (CSE) API. Note that this is limited to 100 free requests per day.
Creating cx and modifying it to search for images
Create custom search engine at https://cse.google.com/cse/create/new based on your search criteria.
Choose sites to search (leave this blank if you want to search the entire web, otherwise you can enter a site to search in one particular site)
Enter a name and a language for your search engine.
Click "create." You can now find cx in your browser URL.
Under "Modify your search engine," click the "Control Panel" button. In the "edit" section you will find an "Image Search" label with an ON/OFF button, change it to ON. Click "update" to save your changes.
Conducting a search with the API
The API endpoint url is https://www.googleapis.com/customsearch/v1
The following JSON parameters are used for this API:
q: specifies search text
num: specifies number of results. Requires an integer value between 1 and 10 (inclusive)
start: the "offset" for the results, which result the search should start at. Requires an integer value between 1 and 101.
imgSize: the size of the image. I used "medium"
searchType: must be set to "image"
filetype: specifies the file type for the image. I used `"jpg", but you can leave this out if file extension doesn't matter to you.
key: an API key, obtained from https://console.developers.google.com/
cx: the custom search engine ID from the previous section
Simply make a GET request by passing above parameters as JSON to the API endpoint (also listed above).
Note: If you set a list of referrers in the search engine settings, visiting the URL via your browser will likely not work. You will need to make an AJAX call (or the equivalent from another language) from a server specified in this list. It will work for only the referrers which were specified in the configuration settings.
Reference:
https://developers.google.com/custom-search/json-api/v1/reference/cse/list
Now You can search images with Custom image search API.
You can do this with two steps:
Get CUSTOM_SEARCH_ID
Go to - https://cse.google.ru/cse/all
Here you must create new Search Engine. Do this and enable Image Search at there.
Screen(i am Russian... sorry)
then get this search engine ID. To do this press at Get Code button:
And there find line with cx = "here will be your CUSTOM_SEARCH_ID":
Ok. It's done, now second step:
Get SERVER_KEY
Go to google Console - https://console.developers.google.com/project
Press to Create project button, enter the name and other required information.
Pick this project and go to Enable Apis
Now find Custom Search Engine.
And Enable it.
Now we must go to Credentials and create new Server Key:
Ok. Now we can use Image Search.
Query:
https://www.googleapis.com/customsearch/v1?key=SERVER_KEY&cx=CUSTOM_SEARCH_ID&q=flower&searchType=image&fileType=jpg&imgSize=xlarge&alt=json
Replace the SERVER_KEY and CUSTOM_SEARCH_ID and call this request.
Limit: for free you can search only 100 images per day.
If this is just for your own purposes (not for production) and you're not planning to abuse Google Image Search, you can simply extract first image URL from Google search results using JSOUP.
For example:
Code to retrieve image URL of the first thumbnail:
public static String FindImage(String question, String ua) {
String finRes = "";
try {
String googleUrl = "https://www.google.com/search?tbm=isch&q=" + question.replace(",", "");
Document doc1 = Jsoup.connect(googleUrl).userAgent(ua).timeout(10 * 1000).get();
Element media = doc1.select("[data-src]").first();
String finUrl = media.attr("abs:data-src");
finRes= "<img src=\"" + finUrl.replace(""", "") + "\" border=1/>";
} catch (Exception e) {
System.out.println(e);
}
return finRes;
}
Guide:
question - image search term
ua - user agent of the browser
After I read several responses I compiled a response with images:
Access the website: https://developers.google.com/custom-search/v1/introduction, on the page you will find this part, so click in the button Get a Key:
Create or select a project, and then NEXT:
Copy the API KEY:
Access the website to create your CX: https://cse.google.com/cse/create/new, write some random domain like “www.anypage.com”, (after we will delete), select a language, and define some name for your search engine. Click on the Button CREATE.
Will see this page, then click in Control Panel:
Copy the Search engine ID for later (this is your CX). After you can set to search in all websites (active Search the entire web, select on the random website www.anypage.com then click on the button Delete) and you can active Image search. So will see like this:
And Using REST you can get the results, using this example code (searching for flower):
<html lang="pt">
<head>
<title>JSON Custom Search API Example</title>
</head>
<body>
<div id="content"></div>
<script>
function hndlr(response) {
console.log(response);
for (var i = 0; i < response.items.length; i++) {
var item = response.items[i];
// in production code, item.htmlTitle should have the HTML entities escaped.
document.getElementById("content").innerHTML += "<br>" + item.htmlTitle;
}
}
</script>
<script src="https://www.googleapis.com/customsearch/v1?key=API_KEY&cx=SEARCH_ENGINE_KEY&q=flower&searchType=image&callback=hndlr"></script>
</body>
</html>
The base code is found here: https://developers.google.com/custom-search/v1/using_rest
After setting your API_KEY (key) and your SEARCH ENGINE KEY (cx), the result will see like this:
Thanks to #Vijay Shegokar, #aftamat4ik and #Alladinian
This is the full URL template to be used
We can eliminate unnecessary parameters.
https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&lr={language?}&safe={safe?}&cx={cx?}&cref={cref?}&sort={sort?}&filter={filter?}&gl={gl?}&cr={cr?}&googlehost={googleHost?}&c2coff={disableCnTwTranslation?}&hq={hq?}&hl={hl?}&siteSearch={siteSearch?}&siteSearchFilter={siteSearchFilter?}&exactTerms={exactTerms?}&excludeTerms={excludeTerms?}&linkSite={linkSite?}&orTerms={orTerms?}&relatedSite={relatedSite?}&dateRestrict={dateRestrict?}&lowRange={lowRange?}&highRange={highRange?}&searchType={searchType}&fileType={fileType?}&rights={rights?}&imgSize={imgSize?}&imgType={imgType?}&imgColorType={imgColorType?}&imgDominantColor={imgDominantColor?}&alt=json
I am using
https://www.googleapis.com/customsearch/v1?key=ap_key&cx=cx&q=hello&searchType=image&imgSize=xlarge&alt=json&num=10&start=1
Change the API url to
Google Custom Image search
Provide the same parameters along with with API KEY and CX.
More Info and Explorer
The Yahoo Boss API is a reasonable substitute, although it's not free and the results are not quite as good.
UPDATE: YAHOO BOSS JSON Search API will discontinue on March 31, 2016
SerpAPI enables to search through Google Images and returns a clean json. it integrates with most of the programming languages: python, php, java, golang, nodejs...
https://serpapi.com/images-results
Google limit the number of search per day.
but this service provides unlimited searches...
looks like we need to implement google custom search API
https://developers.google.com/custom-search/
says so on top of the page you provided yourself
my problem is the following. I'm currently making a blog-page with get-page, get-resources, form-it, and wayfinder. This question requires a decent amount of knowledge about Modx and snippits. I've got the page numberin and all working and i've got a template page with all my calls in it (called weblogTemplate). This template has the following wayfinder call in it :
[[!getResources? &parents=`5` &limit=`5` &tpl=`blogPost`]]
[[!getPage?
&elementClass=`modSnippet`
&element=`getResources`
&parents=`4`
&depth=`2`
&limit=`5`
&pageNavOuterTpl=`[[+first]][[+prev]][[+pages]][[+next]][[+last]]`
&pageVarKey=`page`
&pageFirstTpl=`<li class="controlFirst"><a[[+classes]][[+title]] href="[[+href]]">Eerste pagina</a></li>`
&pageLastTpl=`<li class="controlLast"><a[[+classes]][[+title]] href="[[+href]]">Laatste pagina</a></li>`
&pagePrevTpl=`<li class="controlPrev"><a[[+classes]][[+title]] href="[[+href]]"><<</a></li>`
&pageNextTpl=`<li class="controlNext"><a[[+classes]][[+title]] href="[[+href]]">>></a></li>`
&includeTVs=`1`
&includeContent=`1`
&tpl=`blogListPost`
]]
as you can see the parent is set here to id number 5. This is fine for the homepage but any child page connected in the blog page also uses the same template and so would also have the same menu as the parent. You could use a fix to simply create 1 template for a page and keep using a different getResource call but keep in mind that it is a blog im making, new pages keep getting added. The user can't (, and wouldn't even understand to) make a template and edit any code. A solution i thought of would be to make the parent id dynamic, so it adjust to whatever page it is currently on. So for example if it was on the page with id number 12 it would make the parent call set to 12 and so show all the content under id number 12. If anyone has any ideas / thoughts / solutions i would be very grateful to hear them.
(a link about wayfinder that i used.)
The best solution would be to use two templates - one for main and one for the blog pages and use in blog templates:
&parents=`[[*id]]`
The problem with the user solveds by setting default_template in the system settings.
This worked for me:
[[!getPage?
&elementClass=`modSnippet`
&element=`getResources`
&parents=`[[*id]]`
&depth=`0`
&limit=`10`
&pageNavOuterTpl=`[[+first]][[+prev]][[+pages]][[+next]][[+last]]`
&pageVarKey=`page`
&pageFirstTpl=`<li class="controlFirst"><a[[+classes]][[+title]] href="[[+href]]">Eerste pagina</a></li>`
&pageLastTpl=`<li class="controlLast"><a[[+classes]][[+title]] href="[[+href]]">Laatste pagina</a></li>`
&pagePrevTpl=`<li class="controlPrev"><a[[+classes]][[+title]] href="[[+href]]"><<</a></li>`
&pageNextTpl=`<li class="controlNext"><a[[+classes]][[+title]] href="[[+href]]">>></a></li>`
&includeTVs=`1`
&includeContent=`1`
&tpl=`blogListPost`
]]
Thanks to Vasis for the provided help.
Does anyone know of a tool or script that will crawl my website and count the number of headings on every page within my website? I would like to know how many pages in my website have more than 4 headings (h1). I have Screaming Frog, but it only counts the first two H1 elements. Any help is appreciated.
My Xidel can do that, e.g.:
xidel http://stackoverflow.com/questions/14608312/seo-web-crawling-tool-to-count-number-of-headings-h1-h2-h3 -e 'concat($url, ": ", count(//h1))' -f '//a[matches(#href, "http://[^/]*stackoverflow.com/")]'
The xpath expression in the -e argument tells it to count the h1-tags and the -f option on which pages
This is such a specific task that I would just recommend you write it yourself. The simplest thing you need is an XPATH selector to give you the h1/h2/h3 tags.
Counting the headings:
Pick any one of your favorite programming languages.
Issue a web request for a page on your website (Ruby, Perl, PHP).
Parse the HTML.
Invoke the XPATH heading selector and count the number of elements that it returns.
Crawling your site:
Do step 2 through 4 for all of your pages (you'll probably have to have a queue of pages that you want to crawl). If you want to crawl all of the pages, then it will be just a little more complicated:
Crawl your home page.
Select all anchor tags.
Extract the URL from each href and discard any URLs that don't point to your website.
Perform a URL-seen test: if you have seen it before, then discard, otherwise queue for crawling.
URL-Seen test:
The URL-seen test is pretty simple: just add all the URLs you've seen so far to a hash map. If you run into a URL that is in your hash map, then you can ignore it. If it's not in the hash map, then add it to the crawl queue. The key for the hash map should be the URL and the value should be some kind of a structure that allows you to keep statistics for the headings:
Key = URL
Value = struct{ h1Count, h2Count, h3Count...}
That should be about it. I know it seems like a lot, but it shouldn't be more than a few hundred lines of code!
I found a tool in Code Canyon: Scrap(e) Website Analyser: http://codecanyon.net/item/scrap-website-analyzer/3789481.
As you will see from some of my comments, there was a small amount of configuration, but it is working well so far.
Thanks BeniBela, I will also look at your solution and report back.
You might use xPather chrome extension or similar, and the xPath query:
count(//*[self::h1 or self::h2 or self::h3])
Thanks to:
SEO/Web Crawling Tool to Count Number of Headings (H1, H2, H3...)
https://devhints.io/xpath