I'm trying to write a query to report on BlueToothPolicyTriggered events, that will return all the details to show when a device was blocked by policy AND the details of that device.
Our BT policy basically should allow everything but block file transfer over BT. That seems to be working as expected, but before rolling out wider, want a quick way to 'see' if any other devices are being blocked incorrectly or be able to refer to it if a user reports an issue so we can get all the details of the device blocked to add an exception etc.
However (and I'm new to kql) it seems once I filter a table using an 'ActionType' the columns available to report on are restricted, and in this case we lose details of the BT device that has been blocked
This shows all events that have triggered the policy and whether it was 'accepted' or 'blocked' but not the details of the device
search in (DeviceEvents) ActionType == "BluetoothPolicyTriggered"
| extend parsed=parse_json(AdditionalFields)
| extend Result = tostring(parsed.Accepted)
| extend BluetoothMACAddress = tostring(parsed.BluetoothMacAddress)
| extend PolicyName = tostring(parsed.PolicyName)
| extend PolicyPath = tostring(parsed.PolicyPath)
| summarize arg_max(Timestamp, *) by DeviceName, BluetoothMACAddress
| sort by Timestamp desc
| project Timestamp, DeviceName, DeviceId, Result, ActionType, BluetoothMACAddress, PolicyPath, PolicyName, ReportId
Then I have this which will show every BT connection, the device details im looking for, but not whether it was blocked or accepted
DeviceEvents
| extend parsed=parse_json(AdditionalFields)
| extend MediaClass = tostring(parsed.ClassName)
| extend MediaDeviceId = tostring(parsed.DeviceId)
| extend MediaDescription = tostring(parsed.DeviceDescription)
| extend MediaSerialNumber = tostring(parsed.SerialNumber)
| where MediaClass == "Bluetooth"
| project Timestamp, DeviceId, DeviceName, MediaClass, MediaDeviceId, MediaDescription, parsed
| order by Timestamp desc
Ive been trying to somehow join these together (despite being the same DeviceEvents table) with not much success. I don't trust the output as im seeing entries saying a device was blocked when I know it wasnt.
DeviceEvents
| where ActionType == "BluetoothPolicyTriggered"
| extend parsed=parse_json(AdditionalFields)
| extend Result = tostring(parsed.Accepted)
| extend BluetoothMACAddress = tostring(parsed.BluetoothMacAddress)
| extend PolicyName = tostring(parsed.PolicyName)
| extend PolicyPath = tostring(parsed.PolicyPath)
| project Timestamp, DeviceName, DeviceId, Result, ActionType, BluetoothMACAddress, PolicyPath, PolicyName, ReportId
| join kind =inner (DeviceEvents
| extend parsed=parse_json(AdditionalFields)
| extend MediaClass = tostring(parsed.ClassName)
| extend MediaDeviceId = tostring(parsed.DeviceId)
| extend MediaDescription = tostring(parsed.DeviceDescription)
| extend MediaSerialNumber = tostring(parsed.SerialNumber)
) on DeviceName
| where MediaClass == "Bluetooth"
| project Timestamp, DeviceName, Result, ActionType, MediaClass, MediaDeviceId, MediaDescription,BluetoothMACAddress
| sort by Timestamp desc
Am i going about this completely wrong ?
Related
I am working within Microsoft Sentinel Analytics Rules with the Kusto Query Language. (KQL)
I need to work in a Table called CrowdstrikeReplicatorLogs_CL which contains rows that contain a) data rows for which I need to alert on and b) metadata. that contains information about the subject in the alert.
This means I need to self-join the KQL table with itself to get the final result.
The column in question to join the table itself is the aid_g column.
ThreatIntelligenceIndicator
| where foo == bar
| join kind=innerunique (
CrowdstrikeReplicatorLogs_CL
| where TimeGenerated >= ago(dt_lookBack)
| where event_simpleName_s has_any ("NetworkConnectIP4", "NetworkConnectIP6")
| extend json=parse_json(custom_fields_message_s)
| extend ip4 = json["RemoteAddressIP4"], ip6=json["RemoteAddressIP6"]
| extend CS_ipEntity = tostring(iff(isnotempty(ip4), ip4, ip6))
| extend CommonSecurityLog_TimeGenerated = TimeGenerated
) on $left.TI_ipEntity == $right.CS_ipEntity
| join kind=innerunique (
CrowdstrikeReplicatorLogs_CL
| where custom_fields_message_s has "ComputerName"
| extend customFields=parse_json(custom_fields_message_s)
| project Hostname=customFields['ComputerName'], Platform=event_platform_s, aid_g
) on $left.aid_g == $right.aid_g
;
However, this raises a Query contains incompatible 'set' commands. error in Sentinel.
Is there a proper way to self-join tables?
I want to create a pie chart showing the counts of open (not closed) alerts which is working. However, I want it to default to 0 in the chart when there is no alert for a particular severity
alertsmanagementresources
|extend Sev = tostring(parse_json(properties.essentials.severity)),
LastModifiedTime = todatetime(properties.essentials.lastModifiedDateTime)
| where tostring(parse_json(properties.essentials.alertState)) <> 'Closed'
| where resourceGroup =='ai-eazyfuel-eu-prd-rg'
| where Sev =='Sev0'
|where LastModifiedTime >=datetime(2022/07/26)
|summarize count() by Sev
Is this even possible because I understand there are no results to show but you know what end users are like
While it's feasible to write the KQL query:
Azure Resource Graph uses only a limited subset of KQL which makes the query syntax cumbersome.
Azure Resource Graph cannot display 0 size slice.
P.S.
Please note the removal of unnecessary transformations of properties and the use of ISO format for datetime.
resources
| take 1
| mv-expand severity = range(0,4) to typeof(string)
| project severity = strcat("Sev", severity)
| join kind=leftouter
(
alertsmanagementresources
| extend severity = tostring(properties.essentials.severity)
,lastModifiedDateTime = todatetime(properties.essentials.lastModifiedDateTime)
| where properties.essentials.alertState <> "Closed"
and resourceGroup == "ai-eazyfuel-eu-prd-rg"
and severity == "Sev0"
and lastModifiedDateTime >= datetime("2022-07-26")
| summarize count() by severity
) on severity
| project severity, count_ = coalesce(count_, 0)
I want to receive a high severity alert in Sentinel when a user is added to a defined "high severity" group (via watchlist), however, I want to omit any users that are connected to a Zscaler IP address. The query below is working, however, I'm not sure this is the neatest/most optimized logic. Is there a shorter/better way to write this?
I'm only concerned about the lines beginning with asterisks (which are only added for clarity).
watchlist "aadgroups"
Group
Severity
Prod Owners
High
Prod Contributors
High
watchlist "ZSIPs"
zscaler_ip
location
165.225.0.0/23
Chicago
165.225.60.0/22
Chicago
165.225.56.0/22
Chicago
let HighSeverityGroups = (_GetWatchlist('aadgroups') | where severity == "High" | project group_name, severity);
let ZSIPs = (_GetWatchlist('zscaler_ip') | project zscaler_ip);
AuditLogs
| where ActivityDisplayName == "Add member to group"
| where parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[1].newValue)) has_any (HighSeverityGroups)
| extend InitiatedByActor = tostring(parse_json(tostring(InitiatedBy.user)).userPrincipalName)
| extend GroupName = tostring(parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[1].newValue)))
| extend Actor_ipv4 = tostring(parse_json(tostring(InitiatedBy.user)).ipAddress)
| extend TargetUser = tostring(TargetResources[0].userPrincipalName)
| project-reorder TimeGenerated,SourceSystem,InitiatedBy,ActivityDisplayName,TargetUser,GroupName,InitiatedByActor,Actor_ipv4,Result
| where TargetUser <> ""
** | evaluate ipv4_lookup(ZSIPs, Actor_ipv4, zscaler_ip, return_unmatched = true)
** | where isempty(zscaler_ip)
A couple of things you can try to optimize the query:
This filter is quite costly: | where parse_json(tostring(parse_json(tostring(TargetResources[0].modifiedProperties))[1].newValue)) has_any (HighSeverityGroups) - if TargetResources will rarely have strings from HighSeverityGroups, then before this filter, you can add a much more efficient filter, that will filter out most of the records: | where TargetResources has_any (HighSeverityGroups) - this way, the heavy parsing will be done only on a small amount of records
You're parsing some of the data more than once, for example tostring(parse_json(tostring(InitiatedBy.user)) - instead, you need to use the extend operator to parse them only once, and then use later on in the query
I'm running following Log Analytics Kusto query to get data what uses and thus generetes our Log Analytics cost
Usage
| where IsBillable == true
| summarize BillableDataGB = sum(Quantity) by Solution, DataType
| sort by Solution asc, DataType asc
and then the output is following:
What kinda query should I use if I want to deep dive more eg to ContainerInsights/InfrastructureInsights/ServiceMap/VMInsights/LogManagement so to get more detailed data what name or namespaces really cost?
Insightmetrics table have e.g these names and namespaces.
I was able maybe able to get something out using following query but something is still missing. Not totally sure if I'm on right or wrong way
union withsource = tt *
| where _IsBillable == true
| extend Namespace, Name
Here is the code for getting the name and namespace details. using Kusto query
let startTimestamp = ago(1h);
KubePodInventory
| where TimeGenerated > startTimestamp
| project ContainerID, PodName=Name, Namespace
| where PodName contains "name" and Namespace startswith "namespace"
| distinct ContainerID, PodName
| join
(
ContainerLog
| where TimeGenerated > startTimestamp
)
on ContainerID
// at this point before the next pipe, columns from both tables are available to be "projected". Due to both
// tables having a "Name" column, we assign an alias as PodName to one column which we actually want
| project TimeGenerated, PodName, LogEntry, LogEntrySource
| summarize by TimeGenerated, LogEntry
| order by TimeGenerated desc
For more information you can go through the Microsoft document and here is the Kust Query Tutorial.
I am trying to create a column dependent on whether a certain value exists in a list regardless of when it occurs.
In my table currently I have:
Attendee No - unique number for every attendance Tracking Activity -
Description of Activity Tracking Date/Time - Date & Time when
Activity took place Activity Type - <<<< I need to calculate this
column based on specific logic
[Attendee No] can have multiple [Tracking Activity] & associated [Tracking Date/Time]
Table example. Tracker
+-------------+-------------------+--------------------+---------------+
| Attendee_No | Tracking Activity | Tracking Date/Time | Activity Type |
+-------------+-------------------+--------------------+---------------+
| 3623 | Eat | 05/04/2020 16:28 | Physical |
| 3623 | Music | 05/04/2020 07:16 | Physical |
| 3623 | Run | 05/04/2020 03:52 | Physical |
| 3623 | Booked in | 05/04/2020 03:42 | Physical |
| 3624 | Sleep | 05/04/2020 15:47 | Physical |
| 3624 | Walk | 05/04/2020 11:55 | Physical |
| 3624 | TV | 05/04/2020 11:54 | Physical |
| 3624 | Booked in | 05/04/2020 11:52 | Physical |
+-------------+------------------+--------------------+----------------+
Using the example above what im looking to do is:
For every Attendee No if the Tracker Activity = "Run", "Walk", "Jog", "Gym" regardless when it occurred the Activity Type should = "Physical"
im a SQL noob so have no idea what im doing really so your help will be SOOOO GRATEFULLY appreciated!
For every Attendee No if the Tracker Activity = "Run", "Walk", "Jog", "Gym" regardless when it occurred the Activity Type should = "Physical"
SELECT
CASE
WHEN activity IN('Run','Walk','Jog','Gym') THEN 'Physical'
--WHEN .... other boolean test .. THEN ... other output value...
--ELSE .. catch all value...
END as ActivityType
FROM
...
The -- is commented out - i put these things in as comments to show you how to add more cases, and how to add an else. If you want to use them, uncomment them and modify them.
CASE WHEN has to perform X number of tests and return a single value. It cannot return multiple values
CASE WHEN can also be written like:
CASE activity
WHEN 'Walk' THEN 'Physical'
WHEN 'Jog' THEN 'Physical'
...
--ELSE ...
In this format, you can only supply a single value after the WHEN, and it is compared using =. You can't CASE column WHEN > 0 THEN.. or CASE column WHEN value AND othercolumn = blah THEN..
If you want those kinds of complexity you have to use the CASE WHEN column > 0 AND othercolumn = blah THEN.. form
If you get lots of variations, or you expect to be able to add more in the future without modifying the sql, create another table that has two columns:
Activity,ActivityType
------------
Run,Physical
Walk,Physical
Jog,Physical
Gym,Physical
Study,Mental
...
And join it in:
SELECT * FROM
table t
INNER JOIN activityTypes a ON a.Activity = t.Activity
YOu can have a CASE logic to calculate the Activity Type.
As a general best practice, when you are naming the columns, avoid
below things. They will lead to issues one or other way.
Names with spaces
Keywords
Names with -
types names
SELECT [Attendee No], [Tracking Activity],[Tracking Date/Time],
CASE WHEN [Tracking Activity] IN ( 'Run', 'Walk', 'Jog', 'Gym') THEN 'Physical'
ELSE 'NOT Physical' END AS [Activity Type]
FROM Tracker
I started off with the case statement but what im getting blanks when the Tracking Activity is not in the list. I need ALL the activity type to say "Physical" if the attendee has 'Run', 'Walk', 'Jog', 'Gym' at anytime in the Attendee_No
You can use SQL CASE syntax to achieve it.
See SQL CASE on W3SCHOOLS
Modify your SQL to something like the following and it should do the trick.
SQL
SELECT Attendance_No, Tracking_date,
CASE
WHEN Tracking_activity = 'Run' THEN 'Physical'
WHEN Tracking_ctivity = 'Walk' THEN 'Physical'
WHEN Tracking_ctivity = 'Jog' THEN 'Physical'
WHEN Tracking_ctivity = 'Gym' THEN 'Physical'
ELSE `Other`
END AS 'Activity-type'
FROM Tracker