Splunk: Split a time period into hourly intervals - splunk

index="dummy" url="https://www.dummy.com" status="200 OK"
| stats count by id
| where count > 10
If I apply this above query for 1 day, I would get, for example
id count
ABC 50
XYZ 60
..
This would mean ABC hit https://www.dummy.com 50 times in 1 day, and XYZ called that 60 times.
Now I want to check this for 1 day but with every two hours interval
Suppose, ABC called that request 25 times at 12:00 AM, then 25 times at 3:AM,
and XYZ called all the 60 requests between 12 AM and 2 AM
I want the output to look like this (time format doesn't matter)
id count time
XYZ 60 12:00 AM
ABC 25 12:00 AM
ABC 25 2:00 AM
..

You can use bin to group events into time buckets. You can use any span value, but for the 2 hours you mentioned, the updated query would be:
index="dummy" url="https://www.dummy.com" status="200 OK"
| bin _time span=2h
| stats count by id, _time
| where count > 10

Related

Calculate total manufacturing output over a shift for each location

We currently have a master table stored in our SQL server with the following example information:
Site
Shift Num
Start Time
End Time
Daily Target
A
1
8:00AM
4:00PM
10000
B
1
7:00AM
3:00PM
12000
B
2
4:00PM
2:00AM
7000
C
1
6:00AM
2:00PM
5000
As you can see, there are multiples sites each with their own respective shift start & end times as well as a total daily target for the day.
Another table in the DB is populated by users via the use of a PowerApp. This PowerApp will push output values to the server like so:
Site
Shift Number
Output
Timestamp
A
1
2500
3/15/2022 9:45 AM
A
1
4200
3/15/2022 11:15 AM
A
1
5600
3/15/2022 12:37 PM
A
1
7500
3/15/2022 2:15 PM
This table contains a log of all time-stamped output entries for each site / shift.
What I would like to do is do a daily trend of output vs. target. In order to do so, all output values over a specific shift would have to be aggregated in a SUM function for a given shift grouped by the shift day. The resulting view would need to look like this:
Site
Shift Number
Day
Actual
Target
A
1
3/14
9500
10000
B
1
3/14
13000
12000
A
1
3/15
8000
10000
B
1
3/15
10000
12000
This is easy enough for daytime shifts (group by day and sum the output values). However, if you notice in the master table, Site B / Shift 2 crosses midnight. In this example, I would need to sum values from the previous day 4PM up until 2AM of today. The date grouping would be done by the Shift End Time. Here's an example of the problem area:
Site
Shift Number
Output
Timestamp
B
2
3300
3/15/2022 5:45 PM
B
2
2200
3/15/2022 8:15 PM
B
2
1600
3/16/2022 12:37 AM
B
2
2500
3/16/2022 1:15 AM
I would need these four rows to be aggregated in the view as one row like so:
Site
Shift Number
Day
Actual
Target
B
2
3/16
9600
10000
The values should be listed under March 16th since the end time of the shift occurs then. The values are summated and the target is taken from the daily target master table.
How can I properly calculate these outputs for each shift every day irrespective if it crosses into a new day or not in a view? Or should I go a different route altogether?

Extract 30 minutes from timestamp and group it by 30 mins time interval -PGSQL

In PostgreSQL I am extracting hour from the timestamp using below query.
select count(*) as logged_users, EXTRACT(hour from login_time::timestamp) as Hour
from loginhistory
where login_time::date = '2021-04-21'
group by Hour order by Hour;
And the output is as follows
logged_users | hour
--------------+------
27 | 7
82 | 8
229 | 9
1620 | 10
1264 | 11
1990 | 12
1027 | 13
1273 | 14
1794 | 15
1733 | 16
878 | 17
126 | 18
21 | 19
5 | 20
3 | 21
1 | 22
I want the same output for same SQL for 30 mins. Please suggest
SELECT to_timestamp((extract(epoch FROM login_time::timestamp)::bigint / 1800) * 1800)::timestamp AS interval_30_min
, count(*) AS logged_users
FROM loginhistory
WHERE login_time::date = '2021-04-21' -- inefficient!
GROUP BY 1
ORDER BY 1;
Extracting the epoch gets the number of seconds since the epoch. Integer division truncates. Multiplying back effectively rounds down, achieving the same as date_trunc() for arbitrary time intervals.
1800 because 30 minutes contain 1800 seconds.
Detailed explanation:
Truncate timestamp to arbitrary intervals
The cast to timestamp makes me wonder about the actual data type of login_time? If it's timestamptz, the cast depends on your current time zone setting and sets you up for surprises if that setting changes. See:
How do I match an entire day to a datetime field?
Subtract hours from the now() function
Ignoring time zones altogether in Rails and PostgreSQL
Depending on the actual data type, and exact definition of your date boundaries, there is a more efficient way to phrase your WHERE clause.
You can change the column on which you're aggregating to use the minute too:
select
count(*) as logged_users,
CONCAT(EXTRACT(hour from login_time::timestamp), '-', CASE WHEN EXTRACT(minute from login_time::timestamp) < 30 THEN 0 ELSE 30 END) as HalfHour
from loginhistory
where login_time::date = '2021-04-21'
group by HalfHour
order by HalfHour;

Querying in time intervals throughout history in Splunk

I have a query that returns me a count.
I want to get all the counts of a daily/weekly/monthly granularity, spanning a year back.
Currently I can get the counts manually from the presets (last 30 days, last 15 days, etc), or the date range (e.g. Between 20180101 - 20180201), but what I really want is a query that says
"get me a weekly count that spans a year back from today", and it'll return:
2018-11-15 to 2018-11-22 : count = 10
2018-11-08 to 2018-11-15 : count = 3
2018-11-01 to 2018-11-08 : count = 6
...
2017-11-15 to 2017-11-22 : count = 11
This should get you started.
index=foo earliest=-1y | bucket span=1w _time | stats count by _time

Show data depending on the time of the day

I have a table that shows the amount of units produced by an employee. It also has the transaction time.
Is it possible to display data in such a way that when the current time is between 7 am to 3 pm it should only display the transaction that took place during that period of time and then when the current time is 3pm then it should only display the transactions between 3-11 pm.
sample data
units | name | TIME
-------------------------
10 | aa | 08:33:22
26 | bb | 10:33:22
36 | cc | 16:33:22
11 | dd | 18:33:22
Now if the current time is 13:00:00 i want all the transcations between 7am to 3pm which will be just the first 2. But when the time is 15:00:00 then it should automatically show all the transactions between 3pm - 11pm
You can use where:
where (datepart(hour, getdate()) between 7 and 14 and
datepart(hour, transactiondatetime) between 7 and 14
) or
(datepart(hour, getdate()) not between 7 and 14 and
datepart(hour, transactiondatetime) between 11 and 22
)

Comparing multiple rows of data within a date range to create a new table in MS Access

I'm a novice to SQL & MS-Access, however I have a table of data in MS-Access that looks like:
ID | Start_Time | End_Time
1 | 1:00:00 PM | 1:00:30 PM
2 | 2:15:10 PM | 2:15:50 PM
3 | 2:15:30 PM | 2:18:40 PM
4 | 2:17:00 PM | 2:17:30 PM
5 | 2:45:10 PM | 3:03:10 PM
Each row is sequentially recorded into the database. I want to compare the start and end times of each and combine together rows that overlap. For instance, ID 1's Start_Time and End_Time do not overlap any other times in the table, therefore, it would get posted into the new table. However, ID 2 through 4 have Start_Times and End_Times that overlap with ID 2's Start_Time as Start_Time of the group and ID 3's End_Time as the End_Time of the group ID 2 through 4.
The end result would be a new table that should look like:
ID | Start_Time | End_Time | Duration_seconds
1 | 1:00:00 PM | 1:00:30 PM | 30
2 | 2:15:10 PM | 2:18:40 PM | 210
3 | 2:45:10 PM | 3:03:10 PM | 1080
How can I do this in SQL/MS-Access?
Thank you!!
This might need a few passes through the recordsets.
Define 2 new variables. NewStart and NewEnd. As you grab each ID, assign the existing start/end times.
Using a nested loop compare each record to every other record (new times). If the start time is between the time range then replace that IDs NewStart as the other IDs start time. The NewEnd will be assigned the greater of the current ID or comparitive IDs end time.
As you cycle through, items 3 and 4 will have the same "new" times as ID 2. All will have transformed "new" times.
After this, you query distinct times for each ID