I'm new to kql and defender, looking for help in creating a hunting kql query which checks the avg number of alerts in the last 7 days on defender for endpoint and if at any hour the number of generated alerts spikes and goes above the 1week average number of alerts, it should trigger an alert. For now I have this, but it just checks against a fixed threshold (10), is it possible to change the fixed threshold into a 1week average? And check every hour if in the last hour more alerts were generated Then the week average number of alerts?
let Threshold = 10;
let starttime = 7d;
let endtime = 0d;
let timeframe = 1h;
AlertEvidence
| join (AlertInfo) on AlertId
| project Timestamp, AlertId, Severity, ServiceSource, EntityType, DeviceId, DeviceName, Categories, DetectionSource, Title
| where Timestamp between (startofday(ago(starttime))..startofday(ago(endtime)))
| summarize Total = dcount(AlertId) by Timestamp=bin(Timestamp, 1h), Severity
| extend AboveThreshold = iff(Total > Threshold, "YES", "NO")
| where AboveThreshold=="YES"
Thank you for any help in learning kql
How about this example, which you can adapt?
AppEvents
| where TimeGenerated between ( ago(14d) .. ago(1d) )
// avg for 13 days
| summarize avg_=count()/13 by Name
| join
(
AppEvents
| where TimeGenerated between ( ago(1d) .. now() )
// count for today only
| summarize count() by Name
) on Name
// only show those above the daily average
| where count_ >= avg_
https://portal.azure.com##4bd2cd73-7c32-48aa-8a02-646c8bc0d343/blade/Microsoft_Azure_Monitoring_Logs/DemoLogsBlade/resourceId/%2FDemo/source/LogsBlade.AnalyticsShareLinkToQuery/q/H4sIAAAAAAAAA42QMQ%252BCQAyFd37FG48FQnTFhMG4ObmTQypg5EqOA4Lxx9sDgqudXtPXd18v67rzSMb1wQdTTZZwa1q6kCGrHZUoyE1EBgq6YpUcyxBRtGqRYRDH0GOFB1skB5R69kH90LbaNm%252Fyszy982CcCmMxFDOuuiXxPLkxCFQAqWyH8N1fICuH4Ul5Cr8mJMtDC4tjQQGb17xF%252FpA2mh0lFNuqJMFvoK95gqu59%252Fw8kmiS0xoZ6VFwKsL%252BW0tajlO6nPoFgiy5R04BAAA%253D
Related
I have a table which stores a set of events whose schema can be simplified for this example to
CREATE TABLE events (
id INTEGER PRIMARY KEY,
time INTEGER NOT NULL,
data BLOB
);
CREATE INDEX by_time ON events(time);
Given a time interval min to max, I want to get the number of events in each 1-hour interval between min and max (concrete example below).
The most obvious way to achieve this that I can think of is to compute the required intervals in my code and then for each one run the query
SELECT count(*) FROM events WHERE ? <= time AND time < ?;
Is there a faster way to achieve this by making SQLite deal with splitting the interval into smaller chunks?
If this makes the solution simpler, we can assume min and max are exactly at the start/end of an hour interval.
Example
Suppose events contains events with times
100, 200, 1600, 3000,
3800, 4000,
7400,
15000, 15200, 17000,
20400,
22300, 23000
Then I would want a query with min = 3600, max = 21600 to return something like
start | end | count
-------------------
3600 | 7200 | 2
7200 | 10800 | 1
10800 | 14400 | 0
14400 | 18000 | 3
18000 | 21600 | 1
It doesn't matter exactly what the format of the output is as long as it contains the required counts and a way to identify which interval they refer to.
You can use a recursive CTE to get the time intervals and then LEFT join the table to aggregate:
WITH
cte(min, max) AS (SELECT 3600, 21600),
intervals AS (
SELECT min from_time, min + 3600 to_time, max
FROM cte
WHERE min + 3600 <= max
UNION ALL
SELECT to_time, to_time + 3600, max
FROM intervals
WHERE to_time + 3600 <= max
)
SELECT i.from_time, i.to_time,
COUNT(e.id) count
FROM intervals i LEFT JOIN events e
ON e.time >= i.from_time AND e.time < i.to_time
GROUP BY i.from_time, i.to_time;
See the demo.
I'm new to SQL and I would appreciate any advice!
I have a table that stores the history of an order. It includes the following columns: ORDERID, ORDERMILESTONE, NOTES, TIMESTAMP.
There is one TIMESTAMP for every ORDERMILESTONE in an ORDERID and vice versa.
What I want to do is compare the TIMESTAMPs for certain ORDERMILESTONEs to obtain the amount of time it takes to go from start to finish or from order to shipping, etc.
To get this, I have to gather all of the lines for a specific ORDERID, then somehow iterate through them... while I was trying to do this by declaring a TVP for each ORDERID, but this is just going to take more time because some of my datasets are like 20000 rows long.
What do you recommend? Thanks in advance.
EDIT:
In my problem, I want to find the number of days that the order spends in QA. For example, once an order is placed, we need to make the item requested and then send it to QA. so there's a milestone "Processing" and a milestone "QA". The item could be in "Processing" then "QA" once, and get shipped out, or it could be sent back to QA several times, or back and forth between "Processing" and "Engineering". I want to find the total amount of time that the item spends in QA.
Here's some sample data:
ORDERID | ORDERMILESTONE | NOTES | TIMESTAMP
43 | Placed | newly ordered custom time machine | 07-11-2020 12:00:00
43 | Processing | first time assembling| 07-11-2020 13:00:05
43 | QA | sent to QA | 07-11-2020 13:30:12
43 | Engineering | Engineering is fixing the crank on the time machine that skips even years | 07-12-2020 13:00:02
43 | QA | Sent to QA to test the new crank. Time machine should no longer skip even years. | 07-13-2020 16:00:18
0332AT | Placed | lightsaber custom made with rainbow colors | 07-06-2020 01:00:09
0332AT | Processing| lightsaber being built | 07-06-2020 06:00:09
0332AT | QA | lightsaber being tested | 07-06-2020 06:00:09
I want the total number of days that each order spends with QA.
So I suppose I could create a lookup table that has each QA milestone and its next milestone. Then sum up the difference between each QA milestone and the one that follows. My main issue is that I don't necessarily know how many times the item will need to be sent to QA on each order...
To get the hours to complete a specific mile stone of all orders you can do
select orderid,
DATEDIFF(hh, min(TIMESTAMP), max(TIMESTAMP))
from your_table
where ORDERMILESTONE = 'mile stone name'
group by orderid
Assuming you are using SQL Server and your milestones are not repeated, then you can use:
select om.orderid,
datediff(seconds, min(timestamp), max(timestamp))
from order_milestones om
where milestone in ('milestone1', 'milestone2')
group by om.orderid;
If you want to do this more generally on every row, you can use a cumulative aggregation function:
select om.*,
datediff(seconds,
timestamp,
min(case when milestone = 'order' then timestamp end) over
(partition by orderid
order by timestamp
rows between current row and unbounded following
)
) as time_to_order
from order_milestones om
group by om.orderid;
You can create a lookup table taking a milestone and giving you the previous milestone. Then you can left join to it and left join back to the original table to get the row for the same order at the previous milestone and compare the dates (sqlfiddle):
select om.*, datediff(minute, pm.TIMESTAMP, om.TIMESTAMP) as [Minutes]
from OrderMilestones om
left join MilestoneSequence ms on ms.ORDERMILESTONE = om.ORDERMILESTONE
left join OrderMilestones pm on pm.ORDERID = om.ORDERID
and pm.ORDERMILESTONE = ms.PREVIOUSMILESTONE
order by om.TIMESTAMP
My below query returns a table with the corresponding values
union (traces), (customEvents)
| where timestamp <= now()
| summarize Users=dcount(user_AuthenticatedId) by Country=client_CountryOrRegion
| sort by Users desc
Results:
When pinning the query to the dashboard, I see different results:
The only difference that I can see is the time range set directly on the dashboard. I set this one to custom: 2016-07-06 to now to simulate the same value than in the query. I have checked and I only have logs from 2019 anyway.
Has anyone a clue?
Whenever I have seen this it is due to time slicing. You could add min and max timestamp values to the query in order to understand the exact ranges:
union (traces), (customEvents)
| where timestamp <= now()
| summarize Users=dcount(user_AuthenticatedId), FirstRecord=min(timestamp), LastRecord=max(timestamp) by Country=client_CountryOrRegion
| sort by Users desc
I have a table that houses customer balances and I need to be able to see when accounts figures have dropped by a certain percentage over the previous month's balance per account.
My output consists of an account id, year_month combination code, and the month ending balance. So I want to see if February's balance dropped by X% from January's, and if January's dropped by the same % from December. If it did drop then I would like to be able to see what year_month code it dropped in, and yes I could have 1 account with multiple drops and I hope to see that.
Anyone have an ideas on how to perform this within SQL?
EDIT: Adding some sample data as requested. On the table I am looking at I have year_month as a column, but I do have access to get the last business day date per month as well
account_id | year_month | ending balance
1 | 2016-1 | 50000
1 | 2016-2 | 40000
1 | 2016-3 | 25
Output that I would like to see is the year_month code when the ending balance has at least a 50% decline from the previous month.
First I would recommend making Year_Month a yyyy-mm-dd format date for this calculation. Then take the current table and join it to itself, but the date that you join on will be the prior month. Then perform your calculation in the select. So you could do something like this below.
SELECT x.*,
x.EndingBalance - y.EndingBalance
FROM Balances x
INNER JOIN Balances y ON x.AccountID = y.AccountID
and x.YearMonth = DATEADD(month, DATEDIFF(month, 0, x.YearMonth) - 1, 0)
I have a SSRS report, and I'm trying to sum up rows conditionally. I have:
11/15/2010 12:14:43 AM | Current Rate | Current Speed | Amount used in that minute (Speed*Rate/60)
etc etc etc
I am trying to add all the rows that happened in an hour, so that my report will show:
11/15/2010 | 12 AM - 1 AM | Amount used for that hour (say, 7 gallons)
I cannot find anywhere how to conditionally sum up a row per hour, or how to get my report to say the above.
Thank you in advance!
Using the following table for testing:
CREATE TABLE `log` (
`entry_date` datetime DEFAULT NULL,
`amount_used` int(11) DEFAULT NULL
)
With some test data:
entry_date amount_used
2010-11-01 10:00:00, 3
2010-11-01 10:30:00, 1
2010-11-01 11:00:00, 6
Use this query to get the date, hour range and total amount used:
SELECT DATE(entry_date) AS entry_date,
CONCAT_WS('-',CONVERT(MIN(HOUR(entry_date)), char(2)), CONVERT(MAX(HOUR(entry_date)),CHAR(2))) hours,
SUM(amount_used) amount_used
FROM (
SELECT entry_date, SUM(amount_used) AS amount_used
FROM log
GROUP BY DATE(entry_date), HOUR(entry_date)
) T;
All the CONCAT/CONVERT stuff is just to get the range of hours in that particular day, as a string. This is the result:
entry_date hours amount_used
2010-11-01, 10-11, 10