I am trying to figure out a way to return multiple values in a single column in KQL in Microsoft 365 Defender. For example, if I am trying to output multiple file names from the EmailAttachmentInfo schema, how would I go about doing so?
EmailAttachmentInfo
| where FileName matches regex "Interesting_File_\d+\.zip"
| project FileName
Thank you!
// Data sample generation. Not part of the solution
let EmailAttachmentInfo =
datatable(customer_id:int, FileName:string)
[
1 ,'file1.zip'
,1 ,'file2.zip'
,1 ,'file1.zip'
,1 ,'file3.zip'
,1 ,'file2.zip'
,2 ,'file3.zip'
,2 ,'file3.zip'
,2 ,'file4.zip'
];
// Solution starts here
EmailAttachmentInfo
| summarize make_set(FileName), make_list(FileName) by customer_id
customer_id
set_FileName
list_FileName
1
["file1.zip","file2.zip","file3.zip"]
["file1.zip","file2.zip","file1.zip","file3.zip","file2.zip"]
2
["file3.zip","file4.zip"]
["file3.zip","file3.zip","file4.zip"]
Fiddle
Related
In Vertica DB we have an attribute column that is either comma-separated or enclosed within inverted commas (double and single applicable). When we do an s3 export query on Vertica DB we get the CSV file but when we validate it through an online CSV validator or s3 select query formatted we get an error.
SELECT S3EXPORT(* USING PARAMETERS url='xxxxxxxxxxxxxxxxxxxx.csv', delimiter=',', enclosed_by='\"', prepend_hash=false, header=true, chunksize='10485760'....
and suggestions on how to resolve this issue?
PS: Reading manually every row and checking columns is not the choice
example attributes:-
select uid, cid, att1 from table_name where uid in (16, 17, 15);
uid | cid | att1
-----+-------+---------------------
16 | 78940 | yel,k
17 | 78940 | master#$;#
15 | 78940 | "hello , how are you"
S3EXPORT() is deprecated as from Version 11. We are at Version 12 currently.
Now, you would export like so:
EXPORT TO DELIMITED(
directory='s3://mybucket/mydir'
, filename='indata'
, addHeader='true'
, delimiter=','
, enclosedBy='"'
) OVER(PARTITION BEST) AS
SELECT * FROM indata;
With your three lines, this would generate the below:
dbadmin#gessnerm-HP-ZBook-15-G3:~$ cat /tmp/export/indata.csv
uid,cid,att1
15,78940,"\"hello \, how are you\""
16,78940,"yel\,k"
17,78940,"master#$;#"
Do you need a different format?
Then, try this : ...
EXPORT TO DELIMITED(
directory='/tmp/csv'
, filename='indata'
, addHeader='true'
, delimiter=','
, enclosedBy=''
) OVER(PARTITION BEST) AS
SELECT
uid
, cid
, QUOTE_IDENT(att1) AS att1
FROM indata;
... to get this:
dbadmin#gessnerm-HP-ZBook-15-G3:~$ cat /tmp/csv/indata.csv
uid,cid,att1
15,78940,"""hello \, how are you"""
16,78940,"yel\,k"
17,78940,"master#$;#"
I am trying to build a schedule, I generate an array of objects on the client containing date ranges
[
{start: "2020-07-06 0:0", end: "2020-07-10 23:59"},
{start: "2020-07-13 0:0", end: "2020-07-17 23:59"}
]
I have a column of type daterange[] what is the proper way to format this data to insert it into my table?
This is what I have so far:
INSERT INTO schedules(owner, name, dates) VALUES (
1,
'work',
'{
{[2020-07-06 0:0,2020-07-10 23:59]},
{[2020-07-13 0:0,2020-07-17 23:59]}
}'
)
I think you want:
insert into schedules(owner, name, dates) values (
1,
'work',
array[
'[2020-07-06, 2020-07-11)'::daterange,
'[2020-07-13, 2020-07-18)'::daterange
]
);
Rationale:
you are using dateranges, so you cannot have time portions (for this, you would need tsrange instead); as your code stands, it seems like you want an inclusive lower bound and an exclusive upper bound (hence [ at the left side, and ) at the right side)
explicit casting is needed so Postgres can recognize the that array elements have the proper datatype (otherwise, they look like text)
then, you can surround the list of ranges with the array[] constructor
Demo on DB Fiddle:
owner | name | dates
----: | :--- | :----------------------------------------------------
1 | work | {"[2020-07-06,2020-07-11)","[2020-07-13,2020-07-18)"}
Folks, I'm trying to extract value of 'status' from below string(column name: people) in hive. The problem is, the column is neither a complete JSON nor stored as an Array.
I tried to make it look like a JSON by replacing '=' with ':', which didnt help.
[{name=abc, org=true, self=true, status=accepted, email=abc#gmail.com}, {name=cab abc, org=false, self=false, status=needsAction, email=cab#google.com}]
Below is the query I used:
SELECT
str.name,
str.org,
str.status
FROM table
LATERAL VIEW EXPLODE (TRANSLATE(people,'=',':')) exploded as str;
but I'm getting below error:
FAILED: UDFArgumentException explode() takes an array or a map as a parameter
Need output something like this:
name | org | status
-------- ------- ------------
abc | true | accepted
cab abc | false | needsAction
Note: There is a table already, the datatype is string, and I
can't change the table schema.
Solution for Hive. It possibly can be optimized. Read comments in the code:
with your_table as ( --your data example, you select from your table instead
select "[{name=abc, org=true, self=true, status=accepted, email=abc#gmail.com}, {name=cab abc, org=false, self=false, status=needsAction, email=cab#google.com}]" str
)
select --get map values
m['org'] as org ,
m['name'] as name ,
m['self'] as self ,
m['status'] as status ,
m['email'] as email
from
(--remove spaces after commas, convert to map
select str_to_map(regexp_replace(a.s,', +',','),',','=') m --map
from your_table t --replace w your table
lateral view explode(split(regexp_replace(str,'\\[|\\{|]',''),'}, *')) a as s --remove extra characters: '[' or '{' or ']', split and explode
)s;
Result:
OK
true abc true accepted abc#gmail.com
false cab abc false needsAction cab#google.com
Time taken: 1.001 seconds, Fetched: 2 row(s)
*outdated Oracle version
I have a table for receipt data.
I want to get some data from field EXT_ATTR. such as PAYMENT_RECEIPT_NO
The field "EXT_ATTR" is varchar(4000) stored JSON value
SerialId | EXT_ATTR
1 |
{
"PAYMENT_RECEIPT_NO": "PS00000000000000001",
"IS_CORPOR": "1",
"POSTCODE1": "51000",
"POSTCODE2": "51000",
"BILLADDR1PART1": "BILLADDR1PART1_DATA",
"BILLADDR1PART2": "BILLADDR1PART2_DATA",
"NEED_PRINT_WHT": "1",
"WHT_AMT": "0",
"TRXAMT": "2340600",
"LOCATIONID": "02140",
"PAYMENT_METHOD_NAME": "Cash",
"WITH_TAX": "1"
}
2 |
{
"PAYMENT_RECEIPT_NO": "PS00000000000000055",
"IS_CORPOR": "1",
"POSTCODE1": "51000",
"POSTCODE2": "51000",
"BILLADDR1PART1": "BILLADDR1PART1_DATA",
"BILLADDR1PART2": "BILLADDR1PART2_DATA",
"NEED_PRINT_WHT": "1",
"WHT_AMT": "0",
"TRXAMT": "2340600",
"LOCATIONID": "02140",
"PAYMENT_METHOD_NAME": "Cash",
"WITH_TAX": "1"
}
How can I extract varchar filed to get only value.
SerialId | PAYMENT_RECEIPT_NO
1 | PS00000000000000001
2 | PS00000000000000055
Thank you very much.
to work with json documents you can use PL/JSON
if you want to parse it without json Tools, than you can use substr, instr function in Oracle.
depending on what your string looks like, you have to adjust string positions.
create table tab (json varchar2(1000));
insert into tab values('{"PAYMENT_RECEIPT_NO": "PS00000000000000001","IS_CORPOR": "1","POSTCODE1": "51000","POSTCODE2": "51000","BILLADDR1PART1": "BILLADDR1PART1_DATA","BILLADDR1PART2": "BILLADDR1PART2_DATA","NEED_PRINT_WHT": "1","WHT_AMT": "0","TRXAMT": "2340600","LOCATIONID": "02140","PAYMENT_METHOD_NAME": "Cash","WITH_TAX": "1"}');
insert into tab values('{"PAYMENT_RECEIPT_NO": "PS00000000000000055","IS_CORPOR": "1","POSTCODE1": "51000","POSTCODE2": "51000","BILLADDR1PART1": "BILLADDR1PART1_DATA","BILLADDR1PART2": "BILLADDR1PART2_DATA","NEED_PRINT_WHT": "1","WHT_AMT": "0","TRXAMT": "2340600","LOCATIONID": "02140","PAYMENT_METHOD_NAME": "Cash","WITH_TAX": "1"}');
select substr(json,instr(json,': ',1,1)+3,instr(json,',',1,1)-instr(json,': ',1,1)-4)
from tab;
| SUBSTR(JSON,INSTR(JSON,':',1,1)+3,INSTR(JSON,',',1,1)-INSTR(JSON,':',1,1)-4) |
| :--------------------------------------------------------------------------- |
| PS00000000000000001 |
| PS00000000000000055 |
db<>fiddle here
JSON functions are defined for Database Oracle12c+ version. APEX_JSON package with release 5.0+ should be installed for the previous releases. Whenever installation complete, then the following code might be used as an XML data type manner through APEX_JSON.TO_XMLTYPE() function in order to extract the desired values :
WITH t AS
(
SELECT SerialId, APEX_JSON.TO_XMLTYPE(Payment_Receipt_No) AS xml_data
FROM tab
)
SELECT SerialId, Payment_Receipt_No
FROM t
CROSS JOIN
XMLTABLE('/json'
PASSING xml_data
COLUMNS
Payment_Receipt_No VARCHAR2(100) PATH 'PAYMENT_RECEIPT_NO'
)
Weaving my way through the XML string world - I've come across this issue I'm having.
So I have two XML string that are super similar to each other - only thing is - is that they have different info inside the nodes.
XML string 1:
<DocumentElement>
<Readings>
<ReadingID>1</ReadingID>
<ReadingDate>2013-12-19T00:00:00-05:00</ReadingDate>
<Sys>120</Sys>
<Dia>80</Dia>
<PageNumber>4</PageNumber>
<AddedDate>2015-04-17T19:30:22.2255116-04:00</AddedDate>
<UpdateDate>2015-04-17T19:30:22.2255116-04:00</UpdateDate>
</Readings>
<Readings>
<ReadingID>2</ReadingID>
<ReadingDate>2014-01-10T00:00:00-05:00</ReadingDate>
<Sys>108</Sys>
<Dia>86</Dia>
<PageNumber>8</PageNumber>
<AddedDate>2015-04-17T19:32:08.5121747-04:00</AddedDate>
<UpdateDate>2015-04-17T19:32:08.5121747-04:00</UpdateDate>
</Readings>
</DocumentElement>
XML String 2:
<DocumentElement>
<Readings>
<ReadingID>1</ReadingID>
<ReadingDate>2013-12-20T00:00:00-05:00</ReadingDate>
<Sys>140</Sys>
<Dia>70</Dia>
<PageNumber>10</PageNumber>
<AddedDate>2015-04-17T19:30:22.2255116-04:00</AddedDate>
<UpdateDate>2015-04-17T19:30:22.2255116-04:00</UpdateDate>
</Readings>
</DocumentElement>
Now this is really just an example - I could have an infinite amount of strings just like this that I would want to pull data from. In this case I have two strings and I'm looking to extract all info on <Sys>, <Dia> and <ReadingDate>
I would also like to display this info in a table like this:
Reading Date | Sys | Dia
----------------------------
12/29/2013 | 120 | 80
----------------------------
1/10/2014 | 108 | 86
----------------------------
12/20/2013 | 140 | 70
I am totally unsure how to proceed with this - any and all help is appreciated!
Assuming those XML's are in an XML column named MyXmlColumn, in a table named MyTable*, you can try something like this :
SELECT
R.value('ReadingDate[1]', 'DATETIME') as ReadingDate
, R.value('Sys[1]', 'INT') as Sys
, R.value('Dia[1]', 'INT') as Dia
FROM MyTable t
CROSS APPLY t.MyXmlColumn.nodes('/DocumentElement/Readings') as readings(R)
SQL Fiddle
*: next time you should've provided these info in the first place