I have Azure AD audit event sent to a log analytics workspace and I'd like to build a query that shows me all Unified Groups created with the IsPublic property set to True.
I have the relevant events in TargetResources[0].modifiedProperties however this is a multi-valued object and depending on how it was provisioned the position of the attribute I look for is different.
for ex.
TargetResources[0].modifiedProperties contains the IsPublic on the 3rd position, but sometimes it's on the second or fourth position.
[
{"displayName":"DisplayName","oldValue":"[]","newValue":"[\"Test Group\"]"},
{"displayName":"GroupType","oldValue":"[]","newValue":"[\"Unified\"]"},
{"displayName":"IsPublic","oldValue":"[]","newValue":"[false]"}
]
I am guessing there is a way to find the exact property and value dynamically?
Sincerely,
Tonino Bruno
You could use the mv-apply operator.
below are a few examples:
datatable(i:int, TargetResources:dynamic)
[
1, dynamic([{"p2":"v2","modifiedProperties":[{"displayName":"DisplayName","oldValue":"[]","newValue":"[\"Test Group\"]"},{"displayName":"GroupType","oldValue":"[]","newValue":"[\"Unified\"]"},{"displayName":"IsPublic","oldValue":"[]","newValue":"[false]"}]}]),
2, dynamic([{"p4":"v4","modifiedProperties":[{"displayName":"DisplayName","oldValue":"[]","newValue":"[\"Test Group\"]"},{"displayName":"GroupType","oldValue":"[]","newValue":"[\"Unified\"]"},{"displayName":"IsSomething","oldValue":"[]","newValue":"[false]"},{"displayName":"IsPublic","oldValue":"[]","newValue":"[true]"}]}]),
3, dynamic([{"p2":"v2","modifiedProperties":[{"displayName":"DisplayName","oldValue":"[]","newValue":"[\"Test Group\"]"},{"displayName":"GroupType","oldValue":"[]","newValue":"[\"Unified\"]"},{"displayName":"IsPublic","oldValue":"[]","newValue":"[true]"}]}]),
]
| project i, mp = TargetResources[0].modifiedProperties
| mv-apply mp on (
where mp.displayName == "IsPublic" and mp.newValue == '[true]'
)
datatable(i:int, TargetResources:dynamic)
[
1, dynamic([{"p2":"v2","modifiedProperties":[{"displayName":"DisplayName","oldValue":"[]","newValue":"[\"Test Group\"]"},{"displayName":"GroupType","oldValue":"[]","newValue":"[\"Unified\"]"},{"displayName":"IsPublic","oldValue":"[]","newValue":"[false]"}]}]),
2, dynamic([{"p4":"v4","modifiedProperties":[{"displayName":"DisplayName","oldValue":"[]","newValue":"[\"Test Group\"]"},{"displayName":"GroupType","oldValue":"[]","newValue":"[\"Unified\"]"},{"displayName":"IsSomething","oldValue":"[]","newValue":"[false]"},{"displayName":"IsPublic","oldValue":"[]","newValue":"[true]"}]}]),
3, dynamic([{"p2":"v2","modifiedProperties":[{"displayName":"DisplayName","oldValue":"[]","newValue":"[\"Test Group\"]"},{"displayName":"GroupType","oldValue":"[]","newValue":"[\"Unified\"]"},{"displayName":"IsPublic","oldValue":"[]","newValue":"[true]"}]}]),
]
| project i, mp = TargetResources[0].modifiedProperties
| mv-apply mp on (
project displayName = tostring(mp.displayName), newValue = tostring(parse_json(tostring(mp.newValue))[0])
| summarize b = make_bag(pack(displayName, newValue))
)
| where b.GroupType == "Unified" and b.IsPublic == "true"
Related
I have a JSON data structure like this stored in a column:
{
"Attributes": "[\"AnAttribute\"]",
"WorkItems": "[\"674746\"]",
"AuthorAliases": "[\"AnAlias\"]"
}
I want to extract the first element from one of the arrays. I try the below, but it doesn't work. What am I misunderstanding here?
Query
let t = datatable (data:string) [
#'{"Attributes": "[\"AnAttribute\"]","WorkItems": "[\"674746\"]","AuthorAliases": "[\"AnAlias\"]"}'
];
t
| extend asDyn = todynamic(data)
| extend WorkItems = asDyn["WorkItems"]
| extend WorkItem = WorkItems[0]
Result
data
asDyn
WorkItems
WorkItem
{"Attributes": "["AnAttribute"]","WorkItems": "["674746"]","AuthorAliases": "["AnAlias"]"}
{"Attributes": "["AnAttribute"]", "WorkItems": "["674746"]", "AuthorAliases": "["AnAlias"]"}
["674746"]
let t = datatable (data:string) [
#'{"Attributes": "[\"AnAttribute\"]","WorkItems": "[\"674746\"]","AuthorAliases": "[\"AnAlias\"]"}'
];
t
| extend asDyn = parse_json(data)
| extend WorkItems = parse_json(tostring(asDyn["WorkItems"]))
| extend WorkItem = WorkItems[0]
data
asDyn
WorkItems
WorkItem
{"Attributes": "["AnAttribute"]","WorkItems": "["674746"]","AuthorAliases": "["AnAlias"]"}
{"Attributes":"["AnAttribute"]","WorkItems":"["674746"]","AuthorAliases":"["AnAlias"]"}
["674746"]
674746
Fiddle
The below code gives the error: A recognition error occurred
let vips = datatable (name: string)
['xxxx',
'yyyy',
'zzzz',
'gggg'];
DeviceLogonEvents
| where AccountName in~ (vips)
| summarize by DeviceName
| summarize vippc = make_list(DeviceName)
DeviceAlertEvents
| where DeviceName in (vippc)
Any suggestions how I can search for the items in the list vippc in the DeviceAlertEvents in the column DeviceName?
you could try this:
let vips = datatable(name: string)
[
'xxxx',
'yyyy',
'zzzz',
'gggg'
]
;
let vippc =
DeviceLogonEvents
| where AccountName in~ (vips)
| distinct DeviceName
;
DeviceAlertEvents
| where DeviceName in (vippc)
I need to extract the email from an intricate 'dict' (I am new to sql)
I have seen several previous posts on the same topic (e.g. this one) however, none seem to work on my data
select au.details
from table_au au
result:
{
"id":3526,
"contacts":[
{
"contactType":"EMAIL",
"value":"name#email.be",
"private":false
},
{
"contactType":"PHONE",
"phoneType":"PHONE",
"value":"025/6251111",
"private":false
}
]
}
I need:
name#email.be
select d.value -> 0 -> 'value' as Email
from json_each('{"id":3526,"contacts":[{"contactType":"EMAIL","value":"name#email.be","private":false},{"contactType":"PHONE","phoneType":"PHONE","value":"025/6251111","private":false}]}') d
where d.key::text = 'contacts'
Output:
| | email |
-------------------
|1 |"name#email.be"|
You can run it here: https://rextester.com/VHWRQ89385
I have one column table in my snowflake database that contain a JSON mapping structure as following
ColumnMappings : {"Field Mapping": "blank=Blank,E=East,N=North,"}
How to write a query that if I feed the Field Mapping a value of E I will get East or if the value if N I will get North so on and so forth without hard coding the value in the query like what CASE statement provides.
You really want your mapping in this JSON form:
{
"blank" : "Blank",
"E" : "East",
"N" : "North"
}
You can achieve that in Snowflake e.g. with a simple JS UDF:
create or replace table x(cm variant) as
select parse_json(*) from values('{"fm": "blank=Blank,E=East,N=North,"}');
create or replace function mysplit(s string)
returns variant
language javascript
as $$
res = S
.split(",")
.reduce(
(acc,val) => {
var vals = val.split("=");
acc[vals[0]] = vals[1];
return acc;
},
{});
return res;
$$;
select cm:fm, mysplit(cm:fm) from x;
-------------------------------+--------------------+
CM:FM | MYSPLIT(CM:FM) |
-------------------------------+--------------------+
"blank=Blank,E=East,N=North," | { |
| "E": "East", |
| "N": "North", |
| "blank": "Blank" |
| } |
-------------------------------+--------------------+
And then you can simply extract values by key with GET, e.g.
select cm:fm, get(mysplit(cm:fm), 'E') from x;
-------------------------------+--------------------------+
CM:FM | GET(MYSPLIT(CM:FM), 'E') |
-------------------------------+--------------------------+
"blank=Blank,E=East,N=North," | "East" |
-------------------------------+--------------------------+
For performance, you might want to make sure you call mysplit only once per value in your mapping table, or even pre-materialize it.
I'm trying to query against app-engine datastore backup data. In python, the entities are described as something like this:
class Bar(ndb.Model):
property1 = ndb.StringProperty()
property2 = ndb.StringProperty()
class Foo(ndb.Model):
bar = ndb.StructuredProperty(Bar, repeated=True)
baz = ndb.StringProperty()
Unfortunately when Foo gets backed up and loaded into bigquery, the table schema gets loaded as:
bar | RECORD | NULLABLE
bar.property1 | STRING | REPEATED
bar.property2 | STRING | REPEATED
baz | STRING | NULLABLE
What I would like to do is to get a table of all bar.property1 and associated bar.property2 where baz = 'baz'.
Is there a simple way to flatten Foo so that the bar records are "zipped" together? If that's not possible, is there another solution?
As hinted in a comment by #Mosha, it seems that big query supports User Defined Functions (UDF). You can input it in the UDF Editor tab on the web UI. In this case, I used something like:
function flattenTogether(row, emit) {
if (row.bar && row.bar.property1) {
for (var i=0; i < row.bar.property1.length; i++) {
emit({property1: row.bar.property1[i],
name: row.bar.property2[i]});
}
}
};
bigquery.defineFunction(
'flattenBar',
['bar.property1', 'bar.property2'],
[{'name': 'property1', 'type': 'string'},
{'name': 'property2', 'type': 'string'}],
flattenTogether);
And then the query looked like:
SELECT
property1,
property2,
FROM
flattenBar(
SELECT
bar.property1,
bar.property2,
FROM
[dataset.foo]
WHERE
baz = 'baz')
Since baz is not repeated, you can simply filter on it in WHERE clause without any flattening:
SELECT bar.property1, bar.property2 FROM t WHERE baz = 'baz'