Use a jq range variable to feed into a new jq filter - variables

I have some JSON that looks like this:
{
"vertices": [
{
"id": 71597,
"ns": "ca",
"alias": "polarized"
},
{
"id": 129748,
"ns": "av",
"name": "Polarized"
},
{
"id": 129898,
"ns": "av",
"name": "False"
}
],
"edgeGroups": {
"hasAttributeValue": [
[
0,
1
],
[
0,
2
]
]
}
}
The various entries in .vertices are related by their array index. So in this case, the first vertex (id: 71597 - with an array index of 0) has attribute values 129748 and 129898 (with array indices 1 and two respectively).
I can get the array index for the attribute I'm searching for with the following jq filter:
.vertices | range(0;length) as $i | select(.[$i].alias=="polarized" and .[$i].ns=="ca") | $i
That returns an $i value of 0, since the object with alias "polarized" is the first item in the array. You can try it out here on jq play:
https://jqplay.org/s/DsHYi7ixyn
Now I want to use that $i value in a different filter, so instead of outputting 0 it outputs something like the result of this:
.edgeGroups.hasAttributeValue[] | select(.[0] == 0)
I've tried using the pipe operator like this but it just gives me an error:
.vertices | range(0;length) as $i | select(.[$i].alias=="polarized" and .[$i].ns=="ca") | .edgeGroups.hasAttributeValue[] | select(.[0] == $i)
If I could understand how to use the $i in a chained filter, I think I could solve my main goal, which is to chain several filters together so that I can get all the items associated with the 71597 object - i.e.
{
"id": 129748,
"ns": "av",
"name": "Polarized"
},
{
"id": 129898,
"ns": "av",
"name": "False"
}
Thanks for any help in advance!

Your jq filter could be fixed with a couple of parentheses:
(.vertices
| range(0;length) as $i
| select(.[$i].alias=="polarized" and .[$i].ns=="ca")
| $i) as $i
| .edgeGroups.hasAttributeValue[]
| select(.[0] == $i)

Related

create an array of strings from and existing array keys

I have the following column returned from ARG:
{
"a": {
"key1": [
"text1",
"text2"
]
}
"b": {
"key2": [
"text1",
"text2"
]
}
}
I'm trying to create another column which would contain a list of all the keys.
So in the example above, the new column would contain:
["key1, "key2"]
I also see that I don't have all the functionality to run in ARG so I'm not sure if what I'm trying to do is possilbe.
| mv-expand kind=array doc
| summarize make_list(bag_keys(doc[1]))

How do I KQL to verify presence of property in child object?

I have query which returns object below. How do I return results only if policies array below contains element with policyDefinitionId equal to somevalue without using contains keyword
{
"isComplianceCheck": "False",
"resourceLocation": "southcentralus",
"ancestors": "thc-platform-mg,8f5a5a7f-3cdb-48f1-a894-351a54b84920",
"policies": "[{\"policyDefinitionId\":\"/providers/Microsoft.Authorization/policyDefinitions/34f95f76-5386-4de7-b824-0d8478470c9d/\",\"policySetDefinitionId\":\"/providers/Microsoft.Authorization/policySetDefinitions/1f3afdf9-d0c9-4c3d-847f-89da613e70a8/\",\"policyDefinitionReferenceId\":\"diagnosticsLogsInLogicAppsMonitoring\",\"policySetDefinitionName\":\"1f3afdf9-d0c9-4c3d-847f-89da613e70a8\",\"policyDefinitionName\":\"34f95f76-5386-4de7-b824-0d8478470c9d\",\"policyDefinitionEffect\":\"AuditIfNotExists\",\"policyAssignmentId\":\"/providers/Microsoft.Management/managementGroups/8f5a5a7f-3cdb-48f1-a894-351a54b84920/providers/Microsoft.Authorization/policyAssignments/a45ca010a72c41ceac351431/\",\"policyAssignmentName\":\"a45ca010a72c41ceac351431\",\"policyAssignmentScope\":\"/providers/Microsoft.Management/managementGroups/8f5a5a7f-3cdb-48f1-a894-351a54b84920\",\"policyExemptionIds\":[]}]",
"eventCategory": "Policy",
"entity": "/subscriptions/3adcdebe-b99e-4781-bcdb-65a58a976594/resourceGroups/thc-man-scus-monitoring-rg/providers/Microsoft.Logic/workflows/this-man-scus-reboot-logic",
"message": "Microsoft.Authorization/policies/audit/action",
"hierarchy": "",
"caller": "me#me.com",
"eventDataId": "474c5466-033a-4910-90a1-0ce47d80f1c5",
"eventSubmissionTimestamp": "2021-11-24T15:22:22.7433954Z",
"httpRequest": "{\"clientIpAddress\":\"47.188.89.222\"}",
"resource": "this-man-scus-reboot-logic",
"resourceGroup": "THC-MAN-SCUS-MONITORING-RG",
"resourceProviderValue": "MICROSOFT.LOGIC",
"subscriptionId": "3adcdebe-b99e-4781-bcdb-65a58a976594",
"activityStatusValue": "Success"
}
Here you go:
let MyTable = datatable(d:dynamic) [
dynamic({
"prop1": "value1",
"prop2": "value2",
"policies": "[{\"policyKey1\":\"policyValue1\",\"policyKey2\":\"policyValue2\",\"policyKey3\":\"policyValue3\"},{\"policyKey10\":\"policyValue10\",\"policyKey20\":\"policyValue20\",\"policyKey30\":\"policyValue30\"}]"
}),
dynamic({
"prop1": "value10",
"prop2": "value20",
"policies": "[{\"policyKeyA\":\"policyValueA\",\"policyKeyB\":\"policyValueB\",\"policyKeyC\":\"policyValueC\"},{\"policyKeyAA\":\"policyValueAA\",\"policyKeyBB\":\"policyValueBB\",\"policyKeyCC\":\"policyValueCC\"}]"
}),
dynamic({
"prop1": "value100",
"prop2": "value200",
"policies": "[{\"policyKeyA\":\"policyValueAA\",\"policyKeyB\":\"policyValueB\",\"policyKeyC\":\"policyValueC\"},{\"policyKeyAA\":\"policyValueAA\",\"policyKeyBB\":\"policyValueBB\",\"policyKeyCC\":\"policyValueCC\"}]"
}),
];
MyTable
| mv-apply policy = todynamic(tostring(d.policies)) on
(
mv-expand policy
| where policy['policyKeyA'] == 'policyValueA'
)
| project-away policy
Result:
d
{ "prop1": "value10", "prop2": "value20", "policies": "[{"policyKeyA":"policyValueA","policyKeyB":"policyValueB","policyKeyC":"policyValueC"},{"policyKeyAA":"policyValueAA","policyKeyBB":"policyValueBB","policyKeyCC":"policyValueCC"}]"}
Explanation:
you'll need to use two tricks to solve this one:
You need to use mv-apply in order to iterate over all the items in the policy object, and then filter for the exact policy you're looking for (for example, | where policy['policyKeyA'] == 'policyValueA').
Because the value of policies isn't really a json, but rather a string that represents a json, and because when extracting from a dynamic object you get a dynamic object as well - you can't just iterate over d.policies - you'll need to first convert this from dynamic to string, and then create a dynamic from this string, like this: todynamic(tostring(d.policies))

Azure LogAnalytics Parse JSON Array

I am ingesting some custom logs to Azure LogAnalytics. One of the columns contains nested json objects. I would like to return each nested object to a separate column value.
Was trying the mvexpand statement but have not had any luck.
customLog_CL
| extend test = parsejson(target_s)
| mvexpand test
The column data looks like below.
[ { "id": "00phb49dl40lBsasC0h7", "type": "PolicyEntity", "alternateId": "unknown", "displayName": "Default Policy", "detailEntry": "#{policyType=hello}" }, { "id": "0pri9mxp9vSc4lpiU0h7", "type": "PolicyRule", "alternateId": "00phb49dl40lBsasC0h7", "displayName": "All Users Login", "detailEntry": null } ]
I'm in the exact same situation, so hopefully we can share the knowledge.
I ended up doing something like this, if it's the correct way of doing it, or I have any bugs, I honestly can't tell you right now (still doing my data validation, so I'll update later on), but this should at least get you started.
customLog_CL
| mvexpand parsejson(target_s)
| extend Id=target_s["id"]
| extend type=target_s["type"]
| extend OtherId=target_s["alternateId"]
| project Id, type, OtherId
This should work:
datatable(d:dynamic)
[
dynamic(
[
{ "id": "00phb49dl40lBsasC0h7", "type": "PolicyEntity", "alternateId": "unknown", "displayName": "Default Policy", "detailEntry": "#{policyType=hello}" },
{ "id": "0pri9mxp9vSc4lpiU0h7", "type": "PolicyRule", "alternateId": "00phb49dl40lBsasC0h7", "displayName": "All Users Login", "detailEntry": "" }
]
)
]
| mv-expand(d)
| project key = tostring(d['id']), value = d
| extend p = pack(key, value)
| summarize bag = make_bag(p)
| evaluate bag_unpack(bag)
Output
Please check if this fits your requirement.
let hosts_object = parsejson('{"hosts": [ { "id": "00phb49dl40lBsasC0h7", "type": "PolicyEntity", "alternateId": "unknown", "displayName": "Default Policy", "detailEntry": "#{policyType=hello}" }, { "id": "0pri9mxp9vSc4lpiU0h7", "type": "PolicyRule", "alternateId": "00phb49dl40lBsasC0h7", "displayName": "All Users Login", "detailEntry": null } ]}');
print hosts_object
| extend json1 = hosts_object.hosts[0] , json2 = hosts_object.hosts[1]
Output for this should be as below
Additional Documentation Reference
Hope this helps.

RavenDb facet search in string array field with wildcard

Is it possible to have a RavenDb faceted search, in a string[] field, where I would want to show facets (counts) for only values starting with a particular string, rather a range?
I'll try to explain myself better to with a simple example, imagine having an index with the below entries
ID | Values
-------------------------
1 | CatPersian, CatNormal, DogLabrador
2 | CatPersian, Camel, DogPoodle
3 | CatNormal, CatBengali, DogNormal
4 | DogNormal
I would perform a query on the above documents, and the Facet search would include a range of 'Cat*', on the 'Values' field. Is this possible? Then, I would get a result based on just the different values for cats, like:
CatPersian [2]
CatNormal [2]
CatBengali [1]
Yes, you can do that. Index the array, and then just use facets normally.
Let's see the full example. You have the following documents:
{
"Name": "John",
"FavoriteAnimals": [
"Cats",
"Dogs",
"Snails"
],
"#metadata": {
"#collection": "Kids"
}
}
{
"Name": "Jane",
"FavoriteAnimals": [
"Cats",
"Rabits"
],
"#metadata": {
"#collection": "Kids"
}
}
Now, you create the following index:
from k in docs.Kids
from animal in k.FavoriteAnimals
select new { Animal = animal }
And run this query:
from index 'YourIndex'
where startsWith(Animal , 'ca')
select facet('Animal')
And the result will be:
{
"Name": "Animal",
"Values": [
{
"Count": 2,
"Range": "cats"
}
]
}
Alternatively, you can use this index:
from k in docs.Kids
select new { k.FavoriteAnimals }
And run this query:
from index 'YourIndex'
where startsWith(FavoriteAnimals , 'ca')
select facet('FavoriteAnimals')
The difference here is that you'll get all matches for the documents that have a match.
So in this case
{
"Name": "Animal",
"Values": [
{
"Count": 2,
"Range": "cats"
},
{
"Count": 1,
"Range": "dogs"// also, snails, rabbits
}
]
}

How to inject the array indices of objects in an array into the objects, using jq

Given an array of objects, I would like to inject a property with its position in the array.
For example:
[ { "w" : "Hello" }, { "w" : "World } ]
I would like to produce:
[ { "w" : "Hello", p: 0 }, { "w" : "World, p:1 } ]
where p is the zero-based position in the array.
Is there a way to get the index of the element?
I tried this but it is not working:
keys[] as $i | [ .[] | .p= $i ]
I get:
[ { "w" : "Hello", p: 0 }, { "w" : "World, p:0 } ]
You could do it like this:
[ keys[] as $i | .[$i] | .p=$i ]
Alternatively, you could make it work using to_entries like this:
[ to_entries[] | (.value.p=.key).value ]
Both of which yields:
[
{
"w": "Hello",
"p": 0
},
{
"w": "World",
"p": 1
}
]
Here is a solution which uses reduce
reduce keys[] as $i (.; .[$i].p = $i)