I'm using Azure Log Analytics to review certain events of interest.
I would like to obtain timestamps from data that meets a certain criteria, and then reuse these timestamps in further queries, i.e. to see what else occurred around these times.
The following query returns the desired results, but I'm stuck at how to use the interestingTimes var to then perform further searches and show data within X minutes of each previously returned timestamp.
let interestingTimes =
Event
| where TimeGenerated between (datetime(2021-04-01T11:57:22) .. datetime('2021-04-01T15:00:00'))
| where EventID == 1
| parse EventData with * '<Data Name="Image">' ImageName "<" *
| where ImageName contains "MicrosoftEdge.exe"
| project TimeGenerated
;
Any pointers would be greatly appreciated.
interestingTimes will only be available for use in the query where you declare it. You can't use it in another query, unless you define it there as well.
By the way, you can make your query much more efficient by adding a filter that will utilize the built-in index for the EventData column, so that the parse operator will run on a much smaller amount of records:
let interestingTimes =
Event
| where TimeGenerated between (datetime(2021-04-01T11:57:22) .. datetime('2021-04-01T15:00:00'))
| where EventID == 1
| where EventData has "MicrosoftEdge.exe" // <-- OPTIMIZATION that will filter out most records
| parse EventData with * '<Data Name="Image">' ImageName "<" *
| where ImageName contains "MicrosoftEdge.exe"
| project TimeGenerated
;
Related
I am working within Microsoft Sentinel Analytics Rules with the Kusto Query Language. (KQL)
I need to work in a Table called CrowdstrikeReplicatorLogs_CL which contains rows that contain a) data rows for which I need to alert on and b) metadata. that contains information about the subject in the alert.
This means I need to self-join the KQL table with itself to get the final result.
The column in question to join the table itself is the aid_g column.
ThreatIntelligenceIndicator
| where foo == bar
| join kind=innerunique (
CrowdstrikeReplicatorLogs_CL
| where TimeGenerated >= ago(dt_lookBack)
| where event_simpleName_s has_any ("NetworkConnectIP4", "NetworkConnectIP6")
| extend json=parse_json(custom_fields_message_s)
| extend ip4 = json["RemoteAddressIP4"], ip6=json["RemoteAddressIP6"]
| extend CS_ipEntity = tostring(iff(isnotempty(ip4), ip4, ip6))
| extend CommonSecurityLog_TimeGenerated = TimeGenerated
) on $left.TI_ipEntity == $right.CS_ipEntity
| join kind=innerunique (
CrowdstrikeReplicatorLogs_CL
| where custom_fields_message_s has "ComputerName"
| extend customFields=parse_json(custom_fields_message_s)
| project Hostname=customFields['ComputerName'], Platform=event_platform_s, aid_g
) on $left.aid_g == $right.aid_g
;
However, this raises a Query contains incompatible 'set' commands. error in Sentinel.
Is there a proper way to self-join tables?
The objective of the query im trying to write is to take a count of raw data from the previous month and add that to a count from a lookup table (.csv)
What I have attempted to do is…
index=*** source=***
| stats count(_raw) as monthCount
| join
[ | inputlookup Log_Count_YTD.csv]
| eval countYTD = toNumber(monthCount) + toNumber(TOTAL_COUNT_YTD)
| table countYTD
This query doesn’t return any value on a table. The TOTAL_COUNT_YTD is the only field from the inputlookup file. Let me know if there is any other information you need to help me out with this one. Thanks!
The stats command transforms the data so it has only 1 field: monthCount. The inputlookup returns only the TOTAL_COUNT_YTD field. The join command works by comparing values of common fields between the main search and the subsearch. Since there are no common fields no events are joined.
There is no need for join in this case. The appendcols command will do, assuming the CSV contains a single field in a single row.
index=*** source=***
| stats count() as monthCount
| appendcols
[ | inputlookup Log_Count_YTD.csv]
| eval countYTD = toNumber(monthCount) + toNumber(TOTAL_COUNT_YTD)
| table countYTD
FWIW, the tonumber function is unnecessary, but doesn't hurt.
I'm running following Log Analytics Kusto query to get data what uses and thus generetes our Log Analytics cost
Usage
| where IsBillable == true
| summarize BillableDataGB = sum(Quantity) by Solution, DataType
| sort by Solution asc, DataType asc
and then the output is following:
What kinda query should I use if I want to deep dive more eg to ContainerInsights/InfrastructureInsights/ServiceMap/VMInsights/LogManagement so to get more detailed data what name or namespaces really cost?
Insightmetrics table have e.g these names and namespaces.
I was able maybe able to get something out using following query but something is still missing. Not totally sure if I'm on right or wrong way
union withsource = tt *
| where _IsBillable == true
| extend Namespace, Name
Here is the code for getting the name and namespace details. using Kusto query
let startTimestamp = ago(1h);
KubePodInventory
| where TimeGenerated > startTimestamp
| project ContainerID, PodName=Name, Namespace
| where PodName contains "name" and Namespace startswith "namespace"
| distinct ContainerID, PodName
| join
(
ContainerLog
| where TimeGenerated > startTimestamp
)
on ContainerID
// at this point before the next pipe, columns from both tables are available to be "projected". Due to both
// tables having a "Name" column, we assign an alias as PodName to one column which we actually want
| project TimeGenerated, PodName, LogEntry, LogEntrySource
| summarize by TimeGenerated, LogEntry
| order by TimeGenerated desc
For more information you can go through the Microsoft document and here is the Kust Query Tutorial.
I am crafting a sql query that dynamically builds a where clause. I was able to transform the separate pieces of the where clause as return rows like so:
-------------------------------------------
| ID | Query Part |
-------------------------------------------
| TOKEN 1 | (A = 1 OR B = 2) |
-------------------------------------------
| TOKEN 2 | ([TOKEN 1] or C = 3 |
-------------------------------------------
| TOKEN 3 | ([TOKEN 2] and D = 4) |
-------------------------------------------
My goal is to wrap the current return results above in a stuff and or replace (or something entirely different I hadn't considered) to output the following result:
(((A=1 OR B=2) OR C=3) AND D=4)
Ideally there would be no temp table necessary but I am open to recommendations.
Thank you for any guidance, this has had me pretty stumped at work.
Its unusual. It looks like the query part you want is only Token 3. Then the process should replace any [token] tags in this query part with the corresponding query parts. With the subsequent resulting query part, again the process should replace any [token] tags with the corresponding query parts. This continues until there are no more [token] tags to replace.
I think there should there be a way of indicating the master query (ie token 3) , then use a recursive common table expression to build the expression up until there are no more [token]s.
I need to build a query which will compare off one of two value pairs in my table, my table structure looks something like this:
product_id | psi_a | gpm_a | psi_b | gpm_b |
-------------------------------------------------------------
PRODUCT_123 | 1000 | 400 | 8000 | 300 |
-------------------------------------------------------------
PRODUCT_456 | 2804 | 3006 | 5800 | 579 |
When my psi_a and gpm_a are a value pair as are psi_b and gpm_b, I currently have to run two SQL querys to get the values I require to render my site page correctly, however this results in two sets of results being appended to the page.
Markup
$flowQ = $function->flow_query( $pType, $pVal, $gVal, $class_style, $cVal, $pageCat );
$highQ = $function->high_query( $pType, $pVal, $gVal, $class_style, $cVal, $pageCat );
if(empty($flowQ)===false){
$function->generate_view( $flowQ, $pType, $pVal, $gVal, $class_style, $cVal, $pageCat );
}
The current SQL built by these functions are as follow:
flow_query();
$query = $this->db->prepare( "SELECT * FROM `pumps` WHERE `pump_type` = ? AND `psi_a` >= ? AND gpm_a >= ? AND `pump_category` = ? ORDER BY pump_type DESC" );
$query->bindValue(1, $pType);
$query->bindValue(2, $pVal);
$query->bindValue(3, $gVal);
$query->bindValue(4, $cVal);
The second query is pretty much identical, but it uses psi_a and gpm_a as value parameters. Is there any way to combine these querys to return a single result set that will reference psi_a and gpm_a, and if that returns no results then it references psi_b and gpm_b?
I am relatively novice to SQL so if this is not possible then I shall seek an alternative solution.
May as well call it an answer. You can use and / or clauses in you where statement.
where (psi_a = ? and gpm_a = ? ) or (psi_b = ? and gpm_b = ? )
You can also put a case clause in the select statement that will show you which where clause found the match if it's needed.