splunk and how to do alerting on the first/last business day - splunk

Splunk cron job does not provide a way for running scheduled queries on last and first business days of a month.
In that case I need to provide the data in the query and pass it to alerting condition.
For example:
I need to check each last business day if a file has been delivered. So an ideal alerting condition is:
count < 1 AND is_last_business_day=1
Here I need to check if the query was executed on the last business day.
I wonder if someone can help with a query to check if it is the last business day and pass it as a field. So it can be used in alert conditions.

| eval Date=now(),dayofmonth=strftime(Date,"%d"),weekday=strftime(Date,"%w"),lastDayOfMonth=strftime(relative_time(now(),"#month-1d"),"%d")
| eval is_last_business_day=case(lastDayOfMonth-dayofmonth<7 AND (weekday IN(1,2,3,4,5)),1,True(),0)
Using this statement would work.
it adds the fields Date,dayofmonth,weekday and lastDayOfMonth.
then uses these to calculate if its a last BusinessDay.
Which in this I have defined as the last weekday in the month.
You can stack eval statements by separating them with commas, feel free to put this all in one line or split them up for readability

After trying many things and also using some parts of the query from Daniel here is my solution.
I think this can be improved and may become a shorter query. I tried to write a query that is not the shortest but easy to understand.
The | stats count in the beginning is just to give eval values in the query output even without any results. So you can test the query without any additional conditions or waiting for some dataset results.
I have tested this query with multiple dates and it is working.
| stats count
| eval mydate=strptime("07/28/2022", "%m/%d/%Y")
| eval today_weekday=strftime(mydate,"%w")
| eval today_number=strftime(mydate,"%d")
| eval is_today_business_day=if(today_weekday>=1 and today_weekday<=5,1,0)
``` caluclate last day of month ```
| eval last_day_number=strftime(relative_time(now(),"#month-1d"),"%d")
``` tomorrow for checking if it is a business day ```
| eval tomorrow=relative_time(mydate,"+1d#d")
| eval tomorrow_day=strftime(tomorrow,"%w")
| eval tomorrow_number =strftime(tomorrow,"%d")
| eval is_tomorrow_business_day=if(tomorrow_day>=1 and tomorrow_day<=5,1,0)
| eval is_tomorrow_day_one=if(tomorrow_number=1,1,0)
| eval is_today_in_last_3days=if(today_number>last_day_number-3,1,0)
``` is today a business day AND is today in the last 3 days of month and ( is tomorrow not a businessday OR is tomorrow day one of next month ```
| eval is_today_last_business_day=if(is_today_business_day=1 and is_today_in_last_3days=1 and (is_tomorrow_business_day!=1 OR is_tomorrow_day_one=1),1,0)

| eval Date=now(),dayofmonth=strftime(Date,"%d"),weekday=strftime(Date,"%w"),lastDayOfMonth=strftime(relative_time(now(),"#month-1d"),"%d")
| eval daystillend=lastDayOfMonth-dayofmonth, daystillnextweekday=case(weekday=5,2,True(),0)
| eval is_last_business_day=case(DaysTillEOM<=DaysTillNextWeekDay AND (weekday IN(1,2,3,4,5)),1,True(),0)
New Answer to this question to fix the old one's issues
Approach is get number of days till end of month, and number of days till next weekday
We compare the number of days till end of month and days till next weekday to see if this is the last weekday before end of month (filtering out saturday and sunday too)

Related

Kusto: How to filter Logs in a certian time period?

I want to filter SignIn-Logs with Kusto whose timestamps are only between 6pm and 6am.
Something like that
SignInLogs
| where TimeGenerated between(dateStart .. dateEnd)
All examples I have found are always based on a full timestamp with exact date, like (2014-05-25T08:20:03.123456Z). But I am only interested in the time.
Any idea how to solve this?
Kusto: How to filter Logs in a certian time period?
between operator - Filters a record set for data that falls within an inclusive range of values.
between is used to allow a certain range, but you can also use !between to exclude a time range.
Here Iam excluding from 6 am to 6 pm , so it gives the left over time range i.e.. from 6pm to 6 am
Try the below query
SignInLogs
| where TimeGenerated > ago(1d)
| extend hour = datetime_part("hour", TimeGenerated)
| where hour !between (6 .. 18)

Count overnight hours as one day

I have a dataset where certain operations occur during the overnight hours which I'd like to attribute to the day before.
For example, anything happening between 2/23 8pm and 2/24 6am should be included in 2/23's metrics rather than 2/24. Anything from 6:01 am to 7:59pm should be counted in 2/24's metrics.
I've seen a few posts about decrementing time by 6 hours but that doesn't work in this case.
Is there a way to use an If function to specify that midnight-6am should be counted as date-1 rather than date without affecting the metrics for the 6am - 7:59pm hours?
Thanks in advance! Also, a SQL newbie here so apologies if I have lots of followup questions.
You can use date_add with -6 hours and then optionally cast the timestamp as a date.
create table t (dcol datetime);
insert into t values
('2022-02-25 06:01:00'),
('2022-02-25 06:00:00'),
('2022-02-25 05:59:00');
SELECT CAST(DATE_ADD(dcol, INTERVAL -6 HOUR)AS DATE) FROM t;
| CAST(DATE_ADD(dcol, INTERVAL -6 HOUR)AS DATE) |
| :-------------------------------------------- |
| 2022-02-25 |
| 2022-02-25 |
| 2022-02-24 |
db<>fiddle here
As said in the comments, your requirement is the occurrences in a 6 AM to 6 AM day instead of a 12-12 day. You can achieve this by decreasing the time by 6 hours as shown in #Kendle’s answer. Another way to do it is to use an IF condition as shown below. Here, the date is decremented if the time is before 6 AM on each day and the new date is put in a new column.
Query:
SELECT
IF
(TIME(eventTime) <= "06:00:00",
DATE_ADD(DATE(eventTime), INTERVAL -1 DAY),
DATE(eventTime)) AS newEventTime
FROM
`project.dataset.table`
ORDER BY
eventTime;
Output from sample data:
As seen in the output, timestamps before 6 AM are considered for the previous day while the ones after are considered in the current day.

How to return only the first date of a sequence of dates?

I'm breaking my head trying to create a query for the following situation: I'm using an oracle database, I have a job that always runs at 00 o'clock, so I will fetch the beginning of a user's recess, in the example below we see how are the dates.
DATE_RECESS | USER_ID
---------------------------
22/09/21 | 1
21/09/21 | 1
20/09/21 | 1
19/09/21 | 1
18/09/21 | 1
I will need to notify him missing 10 days to the beginning of the recess, this notification will be sent only 1 time ...
So, looking at these dates with example, I should send 1 single notification on 08/09/21, I should not notify on the other days, only on the first day.
I can't send notification for every day, I should have some return just missing 10 days to start. Summarizing there's my doubt:
How to create a query, (which will be executed by a job that will run every day), and that does not bring result every day? Only when the first date is found?
For me to know what dates are missing 10 days I will take as base my current date, today + 10 days... if I find any date in this period then there I have a recess, but the return of the query should only bring something if it is date is the day 18/09/21, because there is beginning of the recess ... if today + 10 fall on the 19th I will bring nothing, because there will be the second day ...

How can I calculate the number of minutes per day between a daterange

First off I apologize I do not even know where to start and haven't been able to find anything specific to this particular question.
I have a table with datetimes (start and end) and i need to find a way to get minutes/hours between those days. It could either be a sum of the time on weekdays or a some kind of pivot on each day and grouping by the ID number. I had thought to assign a value to the number of days however the times are random and do not start/end at midnight so I am at a loss as how to approach this.
Here are some examples of the date/time format if that helps.
startdate 2018-12-14 10:53:01
enddate 2018-12-27 11:50:00
Any helps or hints would be greatly appreciated!
Edit
forgot to include I am working in SQL Server (SSMS)
Editing For Additional Clarification
Here is a sample date range with an ID number, I wanted to keep it simple.
|ID number| start time |end time
|1 |12/14/2018 10:53|12/17/2018 12:00
here is what I'm trying to achieve (the separation of each date range/ID #)
ID number| start time |end time |mins|
1 | 12/14/2018 10:53|12/14/2018 23:59|786 |
1 | 12/15/2018 0:00 |12/15/2018 23:59|1439|
1 | 12/16/2018 0:00 |12/16/2018 23:59|1439|
1 | 12/17/2018 0:00 |12/17/2018 12:00|960 |
The MINUTE parameter of the DATEDIFF function can be used to determine the difference in minutes between two datetime columns. As below, the second parameter is the start date and the third parameter is the end date, with the result being the amount of time in the specified interval (days, minutes, etc.) from the start to the end date. If you need to find the number of hours between these two columns the HOUR parameter can be used for this. Grouping can be performed as well, as in the second example.
DATEDIFF:
SELECT DATEDIFF(MINUTE, StartDateColumn, EndDateColumn)
DATEDIFF with Grouping and Aggregation:
SELECT ColumnA, SUM(DATEDIFF(MINUTE, StartDateColumn, EndDateColumn)) as DifferenceInMinutes
FROM YourSchema.YourTable
GROUP BY ColumnA

Sum the differences between dates

I have 2 tables, master and detail, that both contain dates related to events. The first contain the master record's begin and end date. The second contains various gaps, that also have a beginning and an end, related to the master record and falling between its begin and end date.
I can successfully calculate the total number of days between the master record's start and end but, yet I'm failing to see how i can aggregate the sum of the "off days" in the details table and group them by month. With this i mean:
Master table
Start date (MM/DD/YYYY): 01/01/2015
End date (MM/DD/YYYY): 01/25/2015
Total number of days: 25
Details table
Start date (MM/DD/YYYY) | End date (MM/DD/YYYY) :
01/02/2015 | 01/05/2015
01/09/2015 | 01/15/2015
01/18/2015 | 01/19/2015
Total number of "off days": 13
The DB environment is Oracle 11g.
Can you help me?
Try this :
select sum(End_date-Start_date+1) from details_table;
The sum function will sum the total of all the dates, which should give you the 13 "off days" you wanted.
If you want to add the start_date/end_date conditions, you can do it like this.
select sum(End_date-Start_date+1) from details_table
where Start_date>=to_date('01/01/2015','mm/dd/yyyy')
and End_date<=to_date('01/25/2015','mm/dd/yyyy');