How to query nested object in kusto? - kql

Question

I have a kusto table with one of the columns as dynamic type with nested json. The dynamic object is two-dimensional array.
{
"OtherField": "Unknown",
"First": [
{
"Id": "",
"Second": [
{
"ConfidenceLevel": "Low",
"Count": 3
}
]
},
{
"Id": "",
"Second": [
{
"ConfidenceLevel": "High",
"Count": 2
},
{
"ConfidenceLevel": "Low",
"Count": 2
}
]
}
]
}
Previously we use "tostring(ColumnName) has_cs '"Level":"High"'" to select rows if "Level" was matched, but now I want to select "Level == 'High' and Count > 0".
For this two-dimensional array, if one item was matched, then this row should be selected. How to implement nested object query in kusto?
I tried regex,
tostring(ColumnName) matches regex '"Level":"High","Count":[^0]'
but during review, regex was not allowed.
Then I tried "mv-expand" or "mv-apply", but it seems passing column name to toscalar function was not allowed.
How to pass column name to toscalar function?
let T = datatable(ColumnName:dynamic)
[
dynamic({"OtherField": "Unknown","First": [{"Id": "","Second": [{"ConfidenceLevel": "Low","Count": 3}]},{"Id": "","Second":[{"ConfidenceLevel": "High","Count": 0}]}]}),
dynamic({"OtherField": "Unknown","First": [{"Id": "","Second": [{"ConfidenceLevel": "Low","Count": 3}]},{"Id": "","Second":[{"ConfidenceLevel": "High","Count": 2}]}]})
];
let result = T
// The following line works, but regex is not allowed during review.
// | where tostring(ColumnName) matches regex '"ConfidenceLevel":"High","Count":[^0]'
| where isnotnull(toscalar(
// print s = '{"OtherField": "Unknown","First": [{"Id": "","Second": [{"ConfidenceLevel": "Low","Count": 3}]},{"Id": "","Second":[{"ConfidenceLevel": "High","Count": 0}]}]}'
print s = tostring(ColumnName) // Error here: The name 'ColumnName' does not refer to any column, table, varible or function.
| project obj0 = parse_json(s)
| mv-expand obj1 = obj0.First
| mv-expand obj2 = obj1.Second
| where obj2.ConfidenceLevel == "High" and obj2.Count > 0)
)
;
result
I tried to use mv-expand function but got an error "The name 'ColumnName' does not refer to any column, table, varible or function."
Expected result (The second row will be selected):
ColumnName
{"OtherField":"Unknown","First":[{"Id":"","Second":[{"ConfidenceLevel":"Low","Count":3}]},{"Id":"","Second":[{"ConfidenceLevel":"High","Count":2}]}]}

Answer 1

Nested mv-apply, to deal with the nested arrays
let T = datatable(ColumnName:dynamic)
[
dynamic({"OtherField": "Unknown","First": [{"Id": "","Second": [{"ConfidenceLevel": "Low","Count": 3}]},{"Id": "","Second":[{"ConfidenceLevel": "High","Count": 0}]}]}),
dynamic({"OtherField": "Unknown","First": [{"Id": "","Second": [{"ConfidenceLevel": "Low","Count": 3}]},{"Id": "","Second":[{"ConfidenceLevel": "High","Count": 2}]}]})
];
T
| mv-apply ColumnName.First on
(
mv-apply ColumnName_First.Second on
(
where ColumnName_First_Second.ConfidenceLevel == "High"
and ColumnName_First_Second.Count > 0
)
)
| project ColumnName
ColumnName
{"OtherField":"Unknown","First":[{"Id":"","Second":[{"ConfidenceLevel":"Low","Count":3}]},{"Id":"","Second":[{"ConfidenceLevel":"High","Count":2}]}]}
Fiddle