I don't know elastalert much.
I just wanted to know whether it is possible to get notification when cluster status is RED using elastalert.
Thank you
its possible. but you need to access it via the cluster health query.
sample:
input{
exec {
curl -u <username>:<password> <elasticsearch-ip:port>/_cluster/health
codec => rubydebug
type => cluster-health
}
}
output {
if "health" in [type] {
elasticsearch{
index => "cluster-health-%{+YYYY.MM.dd}"
hosts => ["elasticsearch-host:port"]
}
}
}
if you have filters that are not parsing it add a filter with if [type] == "cluster-health" section to filter and parse it as json.
this gives you basic details. though you may need to update the field mapping of status.. at least i am facing issue and you have readily availability. Now you can wuery as you normally would under elastalert.
Looks like they don't have this functionality out of the box yet. But there are some hacky ways to achieve it https://github.com/Yelp/elastalert/issues/903
They recommend to use Marvel instead.
No as far as I know.
Current Elastalert documentation state that the index field is mandatory in the rule .yaml file.
The current way elasticsearch cluster provide health status is via the healthapi so no index is available for querying.
Related
Is there a more efficient way to get the list of all databases in notion? I have tried using the https://api.notion.com/v1/databases endpoint but it's deprecated now. Another option is the /search endpoint but it is returning all the records within the database as well.
Can someone provide a better way to list all databases which are accessed by an integration?
you need to filter for databases in the /search endpoint to only get databases back. In Python your payload would look like this:
payload = {
'filter': {
'value': 'database',
'property': 'object'
}
}
In Kibana I'm able to enter queries with AND / OR / NOT / "..." etc, but all examples I can find for the API (Python or NodeJS or .NET) use the Elasticsearch JSON query format to build queries in code. I would like users to be able to enter 'hot AND soup' or '"hot soup" AND cabbage"' etc which are possible in Lucene and Kibana but I cannot find how to use those via the API. I read all SO entries and Elasticsearch docs I could find about the subject but still missed it.
Is it at all possible and if it is, where can I find examples of that? As I cannot find it, it might not be possible at all; I just want to make sure.
Whatever the language you're programming in, you simply need to use the query_string query and pass the use input in there.
GET /_search
{
"query": {
"query_string": {
"query": "\"hot soup\" AND cabbage",
"default_field": "content"
}
}
}
Beware, though, that the query_string query is very sensitive to the syntax, so your users might not enter correct queries. To alleviate that, a more permissive query would be the simple_query_string query
Before you start reading: I have looked at the GraphQL documentation, but my usecase is so specific and I only need the data once, and therefore I allow myself to ask the community for help on this one to save some time and frustration (not planning to learn GraphQL in the future)
Intro
I am a CS student developing an app for Flutter on the side, where I need information about the name and location of every bus stop in a specific county in Norway. Luckily, there's an open GraphQL API for this (API URL: https://api.entur.io/stop-places/v1/graphql). The thing is, I don't know how to query a GraphQL API, and I do not want to spend time learning it as I am only going to fetch the data once and be done with it.
Here's the IDE for the API: https://api.entur.io/stop-places/v1/ide
And this is the exact query I want to perform as I want to fetch bus stops located in the county of Trondheim:
{
stopPlace(stopPlaceType: onstreetBus, countyReference: "Trondheim") {
name {
value
}
... on StopPlace {
quays {
geometry {
coordinates
}
}
}
}
}
The problem with this query though, is that I don't get any data when passing "Trondheim" to the countyReference (without countyReference I get the data, but not for Trondheim). I've tried using the official municipal number for the county as well without any luck, and the documentation of the API is rather poor... Maybe this is something I'll have to contact the people responsible for the API to figure out, which shouldn't be a problem.
But now back to the real problem - how can I make this query using the GraphQL package for Dart? Here's the package I'm planning to use: (https://pub.dev/packages/graphql)
I want to create a bus stop object for each bus stop, and I want to put them all in a list. Here is my bus stop model:
class BusStop with ChangeNotifier {
final String id;
final String name;
final LatLng location;
BusStop({
this.id,
this.name,
this.location
});
}
When it comes to authentication, here's what the documentation says:
This API is open under NLOD licence, however, it is required that all consumers identify themselves by using the header ET-Client-Name. Entur will deploy strict rate-limiting policies on API-consumers who do not identify with a header and reserves the right to block unidentified consumers. The structure of ET-Client-Name should be: "company - application"
Header examples: "brakar - journeyplanner" "fosen_utvikling - departureboard" "norway_bussekspress - nwy-app"
Link to API documentation: https://developer.entur.org/pages-nsr-nsr
Would be great to know how I should go about this as well! I'm grateful for every answers to this, I know I am being lazy here as of learning GraphQL, but for my usecase I thought it would take less time and frustration by asking here!
Getting the query right
First of all you seem to have GraphQL quite figured out. There isn't really much more to it than what you are doing. What queries an API supports depends on the API. The problem you seem to have is more related to the specific API that you are using. I might have figured the right query out for you and if not I will quickly explain what I did and maybe you can improve the query yourself:
{
stopPlace(stopPlaceType: onstreetBus, municipalityReference: "KVE:TopographicPlace:5001") {
name {
value
}
... on StopPlace {
quays {
geometry {
coordinates
}
}
}
}
}
So to get to this I started finding out more about "Trondheim" bei using the topographicPlace query.
{
topographicPlace(query: "Trondheim") {
id
name {
value
}
topographicPlaceType
parentTopographicPlace {
id
name {
value
}
}
}
}
If you do that you will see that "Trondheim" is not a county according to the API: "topographicPlaceType": "municipality". I have no idea what municipality is but the is a different filter for this type on the query that you provided. Then putting "Trondheim" there didn't yield any results so I tried the ID of Trondheim. This now gives me a bunch of results.
About the GraphQL client that you are using:
This seems to be an "Apollo Client" clone in Dart. Apollo Client is a heavy piece of software that comes with a lot of awesome features when used in a frontend application. You probably just want to make a single GraphQL request from a backend. I would recommend using a simple HTTP client to send a POST request to the GraphQL API and a JSON body (don't forget content type header) with the following properties: query containing the query string from above and variables a JSON object mapping variable names to values (only needed if you decide to add variables to your query.
I find ObjectFilter doesn't work in SoftLayer.
I even tried the example provided in the SoftLayer webpage here:
https://sldn.softlayer.com/article/object-filters
REST:
List the ID and hostname of all servers in dal05
https://api.softlayer.com/rest/v3/SoftLayer_Account/getVirtualGuests?objectMask=mask[id,hostname]&objectFilter={"datacenter":{"name":{"operation":"dal05"}}}
When I ran this command, it still returns all the virtual guests, regardless what data center that virtual guest belongs to.
try this request:
GET https://api.softlayer.com/rest/v3/SoftLayer_Account/getVirtualGuests?objectMask=mask[id,hostname,datacenter]&objectFilter={"virtualGuests":{"datacenter":{"name":{"operation":"dal05"}}}}
The issue with your request is that you are missing the "virtualGuests" property, keep in mind that the objectFilter is filtering over the data in the database, so you need to tell it over what table work and over what record of the table work. e.g. using the "SoftLayer_Account" that implies that all the work will be over the "SoftLayer_Account" table now you need to tell id over what property/record of that table work in this case you need to work over the "virtualGuests" and so on. Please keep in mind that and you review the documentation about the valid properties/records e.g. these are the valid properties/record for Softlayer_Account:
http://sldn.softlayer.com/reference/datatypes/SoftLayer_Account
Regards
Maybe you can try adding virtualGuestsin the filter, something like this:
objectFilter={ "virtualGuests": { "datacenter": { "longName": { "operation": "Dallas 6" } } } }
or please see the first examples of https://sldn.softlayer.com/article/object-filters, like this:
object_filter = {
'virtualGuests': {
'datacenter': {
'name': {'operation': 'dal05'}
}
}
}
Ok, this is probably a stupid question but I have been reading and trying different queries and for some reason I cannot get this to work without throwing an error. This is my first time working with MongoDB and it is in an RoR project. We set up charities to have a twitter handle field, but it was not put into the model originally. So we populated the DB with charities, but now none of them have the twitter handle field. I added it to the model so now all others created will have it.
My issue is when I try to update the charities already in my DB I keep getting an error pointing at $set:
namespace :add_tw_handles_fields_2013_6_13 do
desc "add_tw_handle"
task :add_tw_handle => :environment do |t, args|
# db.charity.update( { featured: false }, { $set: { tw_handle : "test"}}, false, true)
# got your 6
Charity.update({ },
{
$set: { "tw_handle": "test"}
},
{ multi: true }
})
end
end
I tried the 2 synax calls above, I was reading in these 2 docs http://docs.mongodb.org/manual/reference/method/db.collection.update/ http://docs.mongodb.org/manual/core/update/#Updating-The%24positionaloperator.
I always get this error tho:
add_tw_handles_fields_2013_6_13.rake:16: syntax error, unexpected ':', expecting tASSOC
$set: {
As far as I can tell that is the correct syntax. I am running this in the script so I don't think I need the db. before my Model name (as shown in the uncommented update) right? I am new to this, but I literally copied and pasted the example and filled out my info, and nothing. I then tried adding a query, but there is never an error until it gets to $set: and I have no idea why. It is exactly as is shown in the Mongo docs linked above.
Any insight into what my issue would be greatly appreciated.
Thanks,
Alan
The error you're getting is from Ruby, not MongoDB, because you're trying to use MongoDB's JSON syntax inside of Ruby, which Ruby does not like. :) Your update query looks fine but you need to translate it to Ruby syntax which is a bit different.
coll.update( { }, { "$set" => { "tw_handle" => "test" } } );
will work assuming coll is your Collection object.
See here for a good tutorial (written by the MongoDB Ruby team) on using the Ruby driver.
Ok so after looking around and talking to the other dev on this project this is what I have found:
Inside a rails script you want to access the objects through rails not the mongoDB directly:
Following example is for running a script from within lib/tasks
Uses Rails activerecord to update the entry through the framework
m = ModelName.find("51b610972f52760fcc003331")
m.update_attributes( :attribute_name => "what you want to assign" )
m.save
Finds the object that has the id given from your model in rails (accesses mongo db directly)
object = ModelName.find({
"$in" => {
"_id" => "51b610972f52760fcc003331"
}
})
object.first.update_attributes(:attribute_name => "what you want to assign")
From within the console
Within the rails console you can use the first segments syntax using the activerecord models to access the objects you are querying for. But if you want to go directly into the mongo console the syntax is slightly different then above.
after going to the root directory of your project launch mongo in the console and find an object from one of your collections based on its id:
>mongo
>show dbs
>use dbsname
>show collections
>db.collection_name.find({ _id: { $in: [ ObjectId("51b610972f52760fcc003331") ] }})
Hopefully this is helpful to others just learning, syntaxs are different in each. I was told that mongo console runs with javascript, where the rails console (and within the project) is using the activerecord to run the call then manipulate the MongoDB. The commenter above was only addressing accessing the mongoDB directly from within a script. So hopefully this is a little more complete in the different ways you could run into this issue.