I'm testing out elastalert, and there was a strange issue on Wednesday, before the holiday weekend, so I just removed all the alert configurations and rebooted elastalert so it had no alerts over the weekend. Now on Monday I turned it back on and its querying all the logs since Wednesday and its taking quite a long time to catch up. I only want to query recent data. Is this a setting? How do I disable it if I want?
Here is an example alert config:
name: alert-name
type: frequency
num_events: 500
timeframe:
minutes: 60
realert:
minutes: 60
index: index.name-*
filter:
- query:
query_string:
query: "message:\"Message\" AND context.debug.type.something"
alert_subject: "This alert happened"
alert:
- pagerduty:
pagerduty_service_key: "nice_try_fbi"
pagerduty_client_name: "company"
Here is what I'm seeing in the logs (today is 11/26)
INFO:elastalert:Queried rule alert-name from 2018-11-22 12:49 UTC to 2018-11-22 13:04 UTC: 83 / 83 hits
INFO:elastalert:Queried rule alert-name from 2018-11-22 13:04 UTC to 2018-11-22 13:19 UTC: 83 / 83 hits
INFO:elastalert:Queried rule alert-name from 2018-11-22 13:19 UTC to 2018-11-22 13:34 UTC: 89 / 89 hits
INFO:elastalert:Queried rule alert-name from 2018-11-22 13:34 UTC to 2018-11-22 13:49 UTC: 91 / 91 hits
INFO:elastalert:Queried rule alert-name from 2018-11-22 13:49 UTC to 2018-11-22 14:04 UTC: 87 / 87 hits
See how its querying stuff from 4 days ago in 15 minute increments? Timeframe is set to 60 mins. I only want to query the most recent 60 mins. Am I missing something here?
I figured out the answer - looks like remembering the "state" of when the last query was run is a feature of elastalert, as specified here under Reliability: https://github.com/Yelp/elastalert/blob/master/docs/source/elastalert.rst#Reliability
I also found that it stores the "state" of alerts in indices of the elasticsearch cluster its running on: https://elastalert.readthedocs.io/en/latest/elastalert_status.html
So I just deleted all of the elastalert* indices in elasticsearch and it seems elastalert treated the alerts as "new" alerts and didn't try to process data since the last successful run. Perhaps there is a better way (like via an alert setting?), but this worked for me.
Related
I'm trying to create a stop report for my plant while learning SQL.
The goal is to get a report containing the amount of minutes stopped planned and unplanned and the amount of stops planned and unplanned. All of this should be reported by day.
Currently I have these values:
Datum: Date
Unplanned: The amount of minutes our plant stopped that day when it was not planned
#unpplanned: the amount of stops we had that day that were unplanned
Daysunplanned: The amount of days the unplanned stop lasted
planned: The amount of minutes our plant stopped that day when it was planned
#planned: The amount of stops we had that day that were planned
dayspl: The amount of days the planned stop lasted
nextday: The amount of minutes that the stop took that should be counted to the next day. A stop can last from 20:00 7/02/2023 till 05:00 8/02/2023).
Can't embed yet :/
My issue is the following:
On 2023-02-06 the plant stopped 1130 minutes unplanned. The stop lasted till 07-02-2023. You can see this because dayunpla = 1.
So the results should be:
2023-02-07 1378
2023-02-06 1130
If dayunpla = 4 we should get:
2023-02-11 1378
2023-02-10 1440
2023-02-09 1440
2023-02-08 1440
2023-02-06 1130
1440 is 24 hours.
I can't think of a way to get from my current table to what I want.
I tried all google searches I could think of but did not find anything helpfull.
Is there a way to fix this using just SQL?
Thanks for your time!
Tried google and a lot of tinkering
I'm working on a web application with a postgresql database. We have a table that contains rows persisted by another app.
These rows arrived from Kafka everyday.
This is the structure of this table:
creation_date status
...
...
20/07/2021 11:47 CREATED
20/07/2021 11:47 CREATED
20/07/2021 11:47 TREATED
20/07/2021 11:45 TREATED
20/07/2021 11:45 TREATED
20/07/2021 11:44 TREATED
20/07/2021 11:44 TREATED
...
...
19/07/2021 16:20 TREATED
19/07/2021 16:16 TREATED
...
...
19/07/2021 08:10 TREATED
19/07/2021 08:08 ERROR
19/07/2021 08:05 TREATED
19/07/2021 08:04 TREATED
19/07/2021 08:02 ERROR
...
...
so everyday we have new rows persisted .
My issue is that I want to use these rows to know what is the interval where we receive rows everyday
(ex: between 7am and 9am or 8am and 10am...).
I want to know this interval in order to execute everyday an SQL query to count the rows in this X interval.
If the count is 0 I will send an alert to say there is a problem and we don't receive events today else everything is okay.
Do you have any idea how I can know this time interval using the previous data?
Regards.
I need my server to reboot after every 5 days of uptime
if it is not rebooted and again reboot after exactly 5 days and continue this process consecutively.
I would like to know how to do this with cronjob or any other script
You can use the dow, or Day of Week, part of the crontab syntax, to execute the command on every occurrence of a particular day, using numbers in the form:
0-6 Sunday to Saturday
1-7 Monday to Sunday
Or even shortened word notation such as WED or THU for Wednesday or Thursday
# m h dom mon dow command
00 07 * * 5 your-command-here
The example above would run the command at 7:00 am every Friday
I need First Login and Last Logout report for CC agents, I have information of every login and logout they perform during their shift (i.e Logout for lunch/smoke/breaktime etc).
We have the Following shifts:
S.No Shift Possible Login Possible Log Out
1. 08 – 04 8: 07 16:05
2. 10 – 06 10:03 18:09
3. 04 – 00 16:08 00:02
4. 06 – 02 18:04 02:01
5. 00 – 08 23:57 08:04
I have this view for collecting relevant information as below:
Problem with this report is that if I am generating a single day report as 20/06/2016 then I am not able to capture information on Shift S.No. (3,4,5) because there is a day change.
For Example:
Agent Login Date/Time is: 20/06/2016 18:10
And
Agent Logout Date/Time is: 21/06/2016 02:05
I need something like to have Interval column where day starts at 20/06/2016 03:00 and day ends at 21/06/2016 03:00 how to achieve this Interval ? Or if you have any other Idea for this report requirement.
If I understood you correctly, based on the view you've shown there, you need to provide data for all the people that were working on, say, 20/06/2016? If that's so, I believe you could select from the view all the people whose
CAST(FirstLogin AS date) = '20160620'
If you needed to also include the people who started working on 19/06/2016 and ended on 20/06/2016 then you would do the same for LastLogOut.
In addition, if you were to write it as a dynamic query, if you needed to provide a report for yesterday for example, you'd substitute the 20/06/2016 from the where clause with
CAST(DATEADD(day, -1, GETDATE()) AS date)
I want to have a list of records under accounts object from a particular time to a particular time
say for example this my api query:
https://ax1.salesforce.com/services/data/v29.0/sobjects/Account/updated/?start=2015-06-30T06%3A49%3A00%2B00%3A00&end=2015-06-30T16%3A30%3A26%2B00%3A00
In my salesforce the time I have chosen is Indian Time which is UTC + 5:30
I created an account at 16:45 pm on 30th June in Indian time(as per salesforce this time is shown in the created by field of the account)
but for the above query in which I have chosen the start time and end time is 06:49 AM and 16:30 PM respectively
I got the record id which I have added at 16 45 pm Indian time but it shouldn't come in the response
The following is the response:
"ids": [
"0019000001QeOINAA3"
],
"latestDateCovered": "2015-06-30T09:00:00.000+0000"
}
Also the latestDateCovered it says 09 AM only
I don't understand this system
Could somebody help me on how this works?
The API REST API will be working with UTC DateTime values.
So you searched for records between:
2015-06-30 06:49:00 UTC
2015-06-30 16:30:26 UTC
Which I make to be:
2015-06-30 12:19:00 IST
2015-06-30 22:00:26 IST
So it would make sense that a record created at 16:45 pm on the 30th of June IST would appear in the results.
Try checking the SystemModStamp and LastModifiedDate fields in the API for the record in question, as the values will also be in UTC there.