Combine KQL Queries into a single query - kql

I have two separate KQL queries I'm using on Azure Log analytics to find blocks on an Azure WAF, the first one finds the transactionid or requests that were blocked.
//Query 1
AzureDiagnostics
| where ResourceType == "APPLICATIONGATEWAYS" and OperationName == "ApplicationGatewayFirewall"
| where hostname_s in ('www.website.com')
| where Message !startswith "Mandatory"
| where action_s in ('Blocked', 'Detected')
| project transactionId_g
To find all the parts of the request that combine to generate the block I then copy/paste the transactionid's to a 2nd query and that all requests that are part of that transaction which gives me the data I need to create exceptions on the WAF if it's a false positive.
//Query 2
AzureDiagnostics
| where ResourceType == "APPLICATIONGATEWAYS" and OperationName == "ApplicationGatewayFirewall"
| where Message !startswith "Mandatory"
| where transactionId_g == "8eb316fd-4a5b-66c7-0136-fc67e21d282b"
| project transactionId_g, TimeGenerated, clientIp_s, hostname_s, requestUri_s, ruleId_s, ruleSetVersion_s, ruleGroup_s, action_s, Message, details_message_s, details_data_s
| sort by transactionId_g
Is there a way to combine these two queries together into a single query that finds the transactionid's of blocked requests, then shows ALL log lines containing those transactionid's?

if there's a single transaction ID returned by the 1st query, you could use the toscalar() function:
let tid = toscalar(
AzureDiagnostics
| where ResourceType == "APPLICATIONGATEWAYS" and OperationName ==
"ApplicationGatewayFirewall"
| where hostname_s in ('www.website.com')
| where action_s in ('Blocked', 'Detected')
| where Message !startswith "Mandatory"
| project transactionId_g
| take 1
);
AzureDiagnostics
| where transactionId_g == tid // <-- this is where you use the result of the 1st subquery
| where ResourceType == "APPLICATIONGATEWAYS" and OperationName == "ApplicationGatewayFirewall"
| where Message !startswith "Mandatory"
| project transactionId_g, TimeGenerated, clientIp_s, hostname_s, requestUri_s, ruleId_s, ruleSetVersion_s, ruleGroup_s, action_s, Message, details_message_s, details_data_s
| sort by transactionId_g
or, if the first query returns a set of transaction IDs and not a single one, you could use the in operator:
let tids =
AzureDiagnostics
| where ResourceType == "APPLICATIONGATEWAYS" and OperationName ==
"ApplicationGatewayFirewall"
| where hostname_s in ('www.website.com')
| where action_s in ('Blocked', 'Detected')
| where Message !startswith "Mandatory"
| project transactionId_g
;
AzureDiagnostics
| where transactionId_g in (tids) // <-- this is where you use the result of the 1st subquery
| where ResourceType == "APPLICATIONGATEWAYS" and OperationName == "ApplicationGatewayFirewall"
| where Message !startswith "Mandatory"
| project transactionId_g, TimeGenerated, clientIp_s, hostname_s, requestUri_s, ruleId_s, ruleSetVersion_s, ruleGroup_s, action_s, Message, details_message_s, details_data_s
| sort by transactionId_g

Related

How better I can optimize this Kusto Query to get my logs

I have below query which I am running and getting logs for Azure K8s, but its takes hour to generate the logs and i am hoping there is a better way to write what i have already written. Can some Kusto experts advice here as how can I better the performance?
AzureDiagnostics
| where Category == 'kube-audit'
| where TimeGenerated between (startofday(datetime("2022-03-26")) .. endofday(datetime("2022-03-27")))
| where (strlen(log_s) >= 32000
and not(log_s has "aksService")
and not(log_s has "system:serviceaccount:crossplane-system:crossplane")
or strlen(log_s) < 32000
| extend op = parse_json(log_s)
| where not(tostring(op.verb) in ("list", "get", "watch"))
| where substring(tostring(op.responseStatus.code), 0, 1) == "2"
| where not(tostring(op.requestURI) in ("/apis/authorization.k8s.io/v1/selfsubjectaccessreviews"))
| extend user = op.user.username
| extend decision = tostring(parse_json(tostring(op.annotations)).["authorization.k8s.io/decision"])
| extend requestURI = tostring(op.requestURI)
| extend name = tostring(parse_json(tostring(op.objectRef)).name)
| extend namespace = tostring(parse_json(tostring(op.objectRef)).namespace)
| extend verb = tostring(op.verb)
| project TimeGenerated, SubscriptionId, ResourceId, namespace, name, requestURI, verb, decision, ['user']
| order by TimeGenerated asc
You could try starting your query as follow.
Please note the additional condition at the end.
AzureDiagnostics
| where TimeGenerated between (startofday(datetime("2022-03-26")) .. endofday(datetime("2022-03-27")))
| where Category == 'kube-audit'
| where log_s hasprefix '"code":2'
I assumed that code is integer, in case it is string, use the following (added qualifier)
| where log_s has prefix '"code":"2'

Get the last row in Slick

I have PostgreSQL table.
Messages:
|id | phone | message | login |
-----------------------------------
* |1 | 85543422 | Hello! | User1 |
|2 | 85543422 | i love | User2 |
-----------------------------------*
I need to filter the table by phone value (85543422) and get the username from the last row.
Now I have a method that allows you to pull the username from the first line.
//return "User1"
def getUserByPhone():String = {
val query = outgoing.filter(_.phone === "85543422").map(_.login).result.headOption
val result = Await.result(db.run(query), Duration.Inf)
if (result.value.isEmpty) return "None"
result.value.get
}
Help with the implementation of the method for removing the user name from the last line.
I need get "User2"
You may sort by id desc to get the username on the last row
eg
val query = outgoing.filter(_.phone === "85543422")
.sortBy(_.id.desc)
.map(_.login).result.headOption

Condition with SQL

I've come to see you for a question. Is there a condition in SQL that allows you to do that:
IF(sup = void) {
}
Database
id | name | lastname | city | mail | number | picture | ...
1 | kiwi | kiwi | USA | kiwi#gmail.com | 0000000000 | default.img | (vide)
SELECT * FROM your_table WHERE sup IS NULL
https://www.w3schools.com/sql/sql_null_values.asp
Update after reading your comment.
$test = $db->query("SELECT * FROM ressource_view WHERE ID = 1")
Will give you the result of your query. Be careful as there could be multiple rows returned.
To fetch the first row
$row = $result->fetch_array()
And then to check if the sup column of your row is null you can use:
if(is_null($row['sup']))
{
}
Or this will have the same effect
if($row['sup'] === NULL)
{
}
But best to tag your question with PHP, MySQL. Your problem seems to be more on the PHP side and someone else could provide a better answer.

Is there a conditional where statement in SQLite?

I have a Todo application with a database for the todos. It contains a column done which will be set to false or true. Now I want to filter the data by all or unfinished. So either done is irrelevant or it has to be false.
I am using SQLite3 in a TypeScript application. Currently I do this by using string templates but I'd prefer an SQL-based solution.
db.prepare(`
select
rowid as id,
title,
description
from todo
${selectAll ? '' : 'where done = 0'}
limit ?
offset ?
`).all(limit, offset);
My idea was to use the CASE clause but it seems not to work around the WHERE clause.
Is there any better solution?
selectAll is a TypeScript variable that is set depending on the query parameters of the app.
Output when selectAll is false
id | title | description | done
1 | Clean the kitchen | as the title says... | 1
2 | Do the shopping | potatoes, tomatoes | 0
3 | Program stuff | Todo app | 1
Output when selectAll is true
id | title | description | done
2 | Do the shopping | potatoes, tomatoes | 0
You can use boolean logic:
where ? = '' or done = 0

tell me the sql

I have a table like this.
Table name : message
mid | mfrom | mto | msg
------------------------
1 | a | b | hi
2 | b | a | hello
3 | a | c | how are you
4 | c | a | fine
i am able to show all rows by
`$sql = mysql_query("SELECT *
FROM message
WHERE mto = '$to'
OR mfrom = '$to';");
while($row = mysql_fetch_array($sql))
{
echo $row['msg'];
}`
but i want to show only one result if mfrom and mto or mto and mfrom is equal. Such as if mfrom =a and mto=b or mfrom=b and mto=a. I want to show only one result from them like this
mid | mfrom | mto | msg
-------------------------
2 | b | a | hello
4 | c | a | fine
please tell me the query.
Your question is not clear because your "query" seems wrong ("mfrom and mto or mto and mfrom is equal" doesn't seem correct).
Nevertheless, as you named the table "message", I'm guessing that you want each message once even though it appears twice in the table.
What you could do is query from the table "twice" and then use the bigger of the ids:
SELECT first.* FROM message AS first LEFT JOIN message AS second
ON first.mfrom = second.mto AND first.mto = second.mfrom
WHERE first.mid > second.mid OR second.mid is NULL;
This would give you the result you requested. It would also give you any message that doesn't have a "partner" row.
If you want only matching messages, you can remove the "OR second.mid is NULL".