How to adjust Kibana Dashboard link in ElastAlert - elastalert

I have written the following rule
type: frequency
filter:
- query:
query_string:
query: "category:foo.bar AND msg._:*Failure*"
alert_text: "Total number of errors cross threshold..... <a href='{0}'>Kibana link</a>"
alert_text_args:
- kibana_link
alert_text_type: alert_text_only
my config.yaml is
# Kibana Dashboard
use_kibana4_dashboard: http://mykibana.com/
When an alert is raised and I click on the hyperlink which I am putting in the message. It takes me to my dashboard.
But what I want is that instead of a dashboard it takes to the data discovery screen and there it issues the very same query that it had issued when the alert was raised.
This way I want to see exactly the query results which the elastalert had seen when the alert was raised.

Actually, I was able to solve this myself. I am writing my solution here.
So basically, I did the exact same search as my criteria above in kibana and saved that search. Next I created a dashboard and pulled the saved query into the dashboard.
Next I pointed the rule to the new dashboard which contains the saved query.
When the link is generated, the elastalert will create the link in a way the time period is injected into the hyperlink. When you click on the link you will see exactly what the alert saw.

Related

yadcf externally triggered filters 'shut off' the actual filtering

I am trying to set my yadcf filters up so they can be triggered from a call (link) from another page. I have an angular single page application that has three tabs on it. If a user clicks a link on lets say the first tab, they will go to another tab (separate table) that contains detailed information relevant to the link they click. (e.g. They are on a row in a table that deals with Apple Mac Pro computers. They see that there are 20 skus currently in the system. They click the number 20 and they go to a lower tab (different table) that contains all the information for those skus). There is no server call in the middle. All the data is loaded in all the tables when the application loads up. So, they are simply clicking a link that applies a filter to the detail table.
yadcf can do this through externally_triggered filters. However, when I set 'externally_triggered': true, it stops the actual filters from working on the details table. (In other words, I can no longer go to that table and manually adjust the filters.)
Does anyone know a way around this issue?
It appears the externally_triggered: true switch does not need to be turned on to use yadcf.exFilterColumn() method. I do not understand when it does need to be turned on, but I am able to call the exFilterColumn method and pass it the options needed to 'prefilter' the table while still retaining the ability to filter the table manually.
externally_triggered and yadcf.exFilterColumn are not related in any way, indeed when yadcf.exFilterColumn is used filters behave a bit differently - they are not filtering on change/keyup/etc , but rather only when the uadcf.exFilterExternallyTriggered function is called (its on purpose and all is explained in the docs)
Here is the relevant text from the docs of the externally_triggered, here it is:
* externally_triggered
Required: false
Type: boolean
Default value: false
Description: Filters will filter only when yadcf.exFilterExternallyTriggered(table_arg) is called
Special notes: Useful when you want to build some form with filters and you want to trigger the filter when that form
"submit" button is clicked (instead of filtering per filter input change)
Here is the showcase page

Google Search Image "no longer available: 403" [duplicate]

I am using google image search API. Till yesterday it was working, but today morning it says "This API is no longer available"
Is it officially closed, Or any error at my side
Request
https://ajax.googleapis.com/ajax/services/search/images?v=1.0&rsz=8&q=cute+kittens
Response
{"responseData": null, "responseDetails": "This API is no longer available.", "responseStatus": 403}
The answer I found was using Google's Custom Search Engine (CSE) API. Note that this is limited to 100 free requests per day.
Creating cx and modifying it to search for images
Create custom search engine at https://cse.google.com/cse/create/new based on your search criteria.
Choose sites to search (leave this blank if you want to search the entire web, otherwise you can enter a site to search in one particular site)
Enter a name and a language for your search engine.
Click "create." You can now find cx in your browser URL.
Under "Modify your search engine," click the "Control Panel" button. In the "edit" section you will find an "Image Search" label with an ON/OFF button, change it to ON. Click "update" to save your changes.
Conducting a search with the API
The API endpoint url is https://www.googleapis.com/customsearch/v1
The following JSON parameters are used for this API:
q: specifies search text
num: specifies number of results. Requires an integer value between 1 and 10 (inclusive)
start: the "offset" for the results, which result the search should start at. Requires an integer value between 1 and 101.
imgSize: the size of the image. I used "medium"
searchType: must be set to "image"
filetype: specifies the file type for the image. I used `"jpg", but you can leave this out if file extension doesn't matter to you.
key: an API key, obtained from https://console.developers.google.com/
cx: the custom search engine ID from the previous section
Simply make a GET request by passing above parameters as JSON to the API endpoint (also listed above).
Note: If you set a list of referrers in the search engine settings, visiting the URL via your browser will likely not work. You will need to make an AJAX call (or the equivalent from another language) from a server specified in this list. It will work for only the referrers which were specified in the configuration settings.
Reference:
https://developers.google.com/custom-search/json-api/v1/reference/cse/list
Now You can search images with Custom image search API.
You can do this with two steps:
Get CUSTOM_SEARCH_ID
Go to - https://cse.google.ru/cse/all
Here you must create new Search Engine. Do this and enable Image Search at there.
Screen(i am Russian... sorry)
then get this search engine ID. To do this press at Get Code button:
And there find line with cx = "here will be your CUSTOM_SEARCH_ID":
Ok. It's done, now second step:
Get SERVER_KEY
Go to google Console - https://console.developers.google.com/project
Press to Create project button, enter the name and other required information.
Pick this project and go to Enable Apis
Now find Custom Search Engine.
And Enable it.
Now we must go to Credentials and create new Server Key:
Ok. Now we can use Image Search.
Query:
https://www.googleapis.com/customsearch/v1?key=SERVER_KEY&cx=CUSTOM_SEARCH_ID&q=flower&searchType=image&fileType=jpg&imgSize=xlarge&alt=json
Replace the SERVER_KEY and CUSTOM_SEARCH_ID and call this request.
Limit: for free you can search only 100 images per day.
If this is just for your own purposes (not for production) and you're not planning to abuse Google Image Search, you can simply extract first image URL from Google search results using JSOUP.
For example:
Code to retrieve image URL of the first thumbnail:
public static String FindImage(String question, String ua) {
String finRes = "";
try {
String googleUrl = "https://www.google.com/search?tbm=isch&q=" + question.replace(",", "");
Document doc1 = Jsoup.connect(googleUrl).userAgent(ua).timeout(10 * 1000).get();
Element media = doc1.select("[data-src]").first();
String finUrl = media.attr("abs:data-src");
finRes= "<img src=\"" + finUrl.replace("&quot", "") + "\" border=1/>";
} catch (Exception e) {
System.out.println(e);
}
return finRes;
}
Guide:
question - image search term
ua - user agent of the browser
After I read several responses I compiled a response with images:
Access the website: https://developers.google.com/custom-search/v1/introduction, on the page you will find this part, so click in the button Get a Key:
Create or select a project, and then NEXT:
Copy the API KEY:
Access the website to create your CX: https://cse.google.com/cse/create/new, write some random domain like “www.anypage.com”, (after we will delete), select a language, and define some name for your search engine. Click on the Button CREATE.
Will see this page, then click in Control Panel:
Copy the Search engine ID for later (this is your CX). After you can set to search in all websites (active Search the entire web, select on the random website www.anypage.com then click on the button Delete) and you can active Image search. So will see like this:
And Using REST you can get the results, using this example code (searching for flower):
<html lang="pt">
<head>
<title>JSON Custom Search API Example</title>
</head>
<body>
<div id="content"></div>
<script>
function hndlr(response) {
console.log(response);
for (var i = 0; i < response.items.length; i++) {
var item = response.items[i];
// in production code, item.htmlTitle should have the HTML entities escaped.
document.getElementById("content").innerHTML += "<br>" + item.htmlTitle;
}
}
</script>
<script src="https://www.googleapis.com/customsearch/v1?key=API_KEY&cx=SEARCH_ENGINE_KEY&q=flower&searchType=image&callback=hndlr"></script>
</body>
</html>
The base code is found here: https://developers.google.com/custom-search/v1/using_rest
After setting your API_KEY (key) and your SEARCH ENGINE KEY (cx), the result will see like this:
Thanks to #Vijay Shegokar, #aftamat4ik and #Alladinian
This is the full URL template to be used
We can eliminate unnecessary parameters.
https://www.googleapis.com/customsearch/v1?q={searchTerms}&num={count?}&start={startIndex?}&lr={language?}&safe={safe?}&cx={cx?}&cref={cref?}&sort={sort?}&filter={filter?}&gl={gl?}&cr={cr?}&googlehost={googleHost?}&c2coff={disableCnTwTranslation?}&hq={hq?}&hl={hl?}&siteSearch={siteSearch?}&siteSearchFilter={siteSearchFilter?}&exactTerms={exactTerms?}&excludeTerms={excludeTerms?}&linkSite={linkSite?}&orTerms={orTerms?}&relatedSite={relatedSite?}&dateRestrict={dateRestrict?}&lowRange={lowRange?}&highRange={highRange?}&searchType={searchType}&fileType={fileType?}&rights={rights?}&imgSize={imgSize?}&imgType={imgType?}&imgColorType={imgColorType?}&imgDominantColor={imgDominantColor?}&alt=json
I am using
https://www.googleapis.com/customsearch/v1?key=ap_key&cx=cx&q=hello&searchType=image&imgSize=xlarge&alt=json&num=10&start=1
Change the API url to
Google Custom Image search
Provide the same parameters along with with API KEY and CX.
More Info and Explorer
The Yahoo Boss API is a reasonable substitute, although it's not free and the results are not quite as good.
UPDATE: YAHOO BOSS JSON Search API will discontinue on March 31, 2016
SerpAPI enables to search through Google Images and returns a clean json. it integrates with most of the programming languages: python, php, java, golang, nodejs...
https://serpapi.com/images-results
Google limit the number of search per day.
but this service provides unlimited searches...
looks like we need to implement google custom search API
https://developers.google.com/custom-search/
says so on top of the page you provided yourself

Is "logstash-" a mandatory prefix of indices in kibana?

If I put messages on index "[logstash-example-]YYYY.MM.DD" then kibana can show the log message in charts but if it's on "[example-]YYYY.MM.DD" then it won't find it.
(curl query gives back the correct result in latter case)
According to documentation it should work:
"For example [web-]YYYY.MM.DD,[mail-]YYYY.MM.DD Please also note that indices should rollover at midnight UTC."
(Elasticsearch 1.3.4, Kibana 3.1.0)
You have to modify your kibana dashboard setting
Click Configure dashboarad in Kibana on Right Top.
Select Index tab.
Modify Index pattern to your new index pattern. For example: [example-]YYYY.MM.DD
Hope this can help you.

Apex - passing a variable from one page to another

I have a form on one page which prompts a user to enter their email address. When they click "next" I want Apex to redirect the user to a different page which shows them a report which selects the records from the users table where the email address matches the one that they entered.
E.g.
SELECT *
FROM USERS
WHERE EMAIL_ADDRESS = (the email address that they entered on the previous page);
Can someone please explain the easiest way to do this?
I have only in the last 5 seconds heard about 'Apex' Lol ... But I think I found what you need to do.
You need to define a 'Branch' which will allow you to POST to whichever page you need to.
Here's some documentation on 'Branches': http://docs.oracle.com/cd/B32472_01/doc/appdev.300/b32469/pdf_report.htm#BABICIJG
Also this snipet may help:
It's [The 'branch'] in the middle (page processing section) of the
development page at the bottom. The branch will be executed whenever a
post submission occurs but you can put conditions on the branch.
Which I found at: http://dbaforums.org/oracle/index.php?showtopic=8139
The "Next" button should not redirect but should submit. The value of the page item has to be pushed to the session state for it to be available in the (apex) session. On submit you can then define a branch which will take you to the page with your report. Your report can then reference the page item by using bind variable notation in the region source:
SELECT * FROM USERS WHERE EMAIL_ADDRESS = :Px_EMAIL_ADDRESS

sharepoint crawl rule to exclude AllItems.aspx , but get an item/document in search resu lts if queried in the search box

I followed this blog Tips 1and created a crawl rule http://.*forms/allitems.aspx and ran full crawl. I no longer get the results with AllItems.aspx. However, if there is any document with name Something.doc in a Document Library , it no longer gets pulled in the search results.
I think what I desire is a basic functionality, like the user should not get to see Allitems.aspx in the search results but should get the item/document with names entered in the search box.
Please let me know if I am missing anything. I have already put in 24 hours...googled the max I could.
It seems that an Index Reset is required. Here's the steps I did:
1. Add the following crawl rule to exclude: *://*allitems.aspx.
2. Index Reset.
3. Full Crawl.
I could not find a good way to do this using crawl rules. Instead, I opted to set up a restriction on the search results web part.
In the search results web part properties, select "Change Query"
Add a property filter to exclude anything with "AllItems" (and any other exclusions you want in place.
Used Steve Mann's blog as a reference and for the images: http://stevemannspath.blogspot.com/2013/04/sharepoint-2013-search-removing-junk.html