The below code gives the error: A recognition error occurred
let vips = datatable (name: string)
['xxxx',
'yyyy',
'zzzz',
'gggg'];
DeviceLogonEvents
| where AccountName in~ (vips)
| summarize by DeviceName
| summarize vippc = make_list(DeviceName)
DeviceAlertEvents
| where DeviceName in (vippc)
Any suggestions how I can search for the items in the list vippc in the DeviceAlertEvents in the column DeviceName?
you could try this:
let vips = datatable(name: string)
[
'xxxx',
'yyyy',
'zzzz',
'gggg'
]
;
let vippc =
DeviceLogonEvents
| where AccountName in~ (vips)
| distinct DeviceName
;
DeviceAlertEvents
| where DeviceName in (vippc)
Related
This is as example taken from another thread, but essentially I would like to achieve this:
Sample data
ID Name Value
1 TV1 {"URL": "www.url.com", "Icon": "some_icon"}
2 TV2 {"URL": "www.url.com", "Icon": "some_icon", "Facebook": "Facebook_URL"}
3 TV3 {"URL": "www.url.com", "Icon": "some_icon", "Twitter": "Twitter_URL"}
..........
Expected output
ID Name URL Icon Facebook Twitter
1 TV1 www.url.com some_icon NULL NULL
2 TV2 www.url.com some_icon Facebook_URL NULL
3 TV3 www.url.com some_icon NULL Twitter_URL
I'm totally new to Snowflake so I'm shaking my head on how to do this easily (and hopefully automatically, in the case where some rows might have more elements in the json than other rows, which would be tedious to assign manually). Some lines might have sub-categories too.
I found the parse_json function for Snowflake, but it's only giving me the same json column in a new column, still in json format.
TIA!
You can create a view over your table with the following SELECT:
SELECT ID,
Name,
Value:URL::varchar as URL,
Value:Icon::varchar as Icon,
Value:Facebook::varchar as Facebook,
Value:Twitter::varchar as Twitter
FROM tablename;
Additional attributes will be ignored unless you add them to the view. There is no way to "automatically" include them into the view, but you could create a stored procedure that dynamically generates the view based on all the attributes that are in the full variant content of a table.
You can create a SP to automatically build the CREATE VIEW for you based on the JSON data in the VARIANT.
I have some simple example below:
-- prepare the table and data
create or replace table test (
col1 int, col2 string,
data1 variant, data2 variant
);
insert into test select 1,2, parse_json(
'{"URL": "test", "Icon": "test1", "Facebook": "http://www.facebook.com"}'
), parse_json(
'{"k1": "test", "k2": "test1", "k3": "http://www.facebook.com"}'
);
insert into test select 3,4,parse_json(
'{"URL": "test", "Icon": "test1", "Twitter": "http://www.twitter.com"}'
), parse_json(
'{"k4": "v4", "k3": "http://www.ericlin.me"}'
);
-- create the SP, we need to know which table and
-- column has the variant data
create or replace procedure create_view(
table_name varchar
)
returns string
language javascript
as
$$
var final_columns = [];
// first, find out the columns
var query = `SHOW COLUMNS IN TABLE ${TABLE_NAME}`;
var stmt = snowflake.createStatement({sqlText: query});
var result = stmt.execute();
var variant_columns = [];
while (result.next()) {
var col_name = result.getColumnValue(3);
var data_type = JSON.parse(result.getColumnValue(4));
// just use it if it is not a VARIANT type
// if it is variant type, we need to remember this column
// and then run query against it later
if (data_type["type"] != "VARIANT") {
final_columns.push(col_name);
} else {
variant_columns.push(col_name);
}
}
var columns = {};
query = `SELECT ` + variant_columns.join(', ') + ` FROM ${TABLE_NAME}`;
stmt = snowflake.createStatement({sqlText: query});
result = stmt.execute();
while (result.next()) {
for(i=1; i<=variant_columns.length; i++) {
var sub_result = result.getColumnValue(i);
if(!sub_result) {
continue;
}
var keys = Object.keys(sub_result);
for(j=0; j<keys.length; j++) {
columns[variant_columns[i-1] + ":" + keys[j]] = keys[j];
}
}
}
for(path in columns) {
final_columns.push(path + "::STRING AS " + columns[path]);
}
var create_view_sql = "CREATE OR REPLACE VIEW " +
TABLE_NAME + "_VIEW\n" +
"AS SELECT " + "\n" +
" " + final_columns.join(",\n ") + "\n" +
"FROM " + TABLE_NAME + ";";
snowflake.execute({sqlText: create_view_sql});
return create_view_sql + "\n\nVIEW created successfully.";
$$;
Execute the SP will return below string:
call create_view('TEST');
+---------------------------------------+
| CREATE_VIEW |
|---------------------------------------|
| CREATE OR REPLACE VIEW TEST_VIEW |
| AS SELECT |
| COL1, |
| COL2, |
| DATA1:Facebook::STRING AS Facebook, |
| DATA1:Icon::STRING AS Icon, |
| DATA1:URL::STRING AS URL, |
| DATA2:k1::STRING AS k1, |
| DATA2:k2::STRING AS k2, |
| DATA2:k3::STRING AS k3, |
| DATA1:Twitter::STRING AS Twitter, |
| DATA2:k4::STRING AS k4 |
| FROM TEST; |
| |
| VIEW created successfully. |
+---------------------------------------+
Then query the VIEW:
SELECT * FROM TEST_VIEW;
+------+------+-------------------------+-------+------+------+-------+-------------------------+------------------------+------+
| COL1 | COL2 | FACEBOOK | ICON | URL | K1 | K2 | K3 | TWITTER | K4 |
|------+------+-------------------------+-------+------+------+-------+-------------------------+------------------------+------|
| 1 | 2 | http://www.facebook.com | test1 | test | test | test1 | http://www.facebook.com | NULL | NULL |
| 3 | 4 | NULL | test1 | test | NULL | NULL | http://www.ericlin.me | http://www.twitter.com | v4 |
+------+------+-------------------------+-------+------+------+-------+-------------------------+------------------------+------+
Query the source table:
SELECT * FROM TEST;
+------+------+------------------------------------------+-----------------------------------+
| COL1 | COL2 | DATA1 | DATA2 |
|------+------+------------------------------------------+-----------------------------------|
| 1 | 2 | { | { |
| | | "Facebook": "http://www.facebook.com", | "k1": "test", |
| | | "Icon": "test1", | "k2": "test1", |
| | | "URL": "test" | "k3": "http://www.facebook.com" |
| | | } | } |
| 3 | 4 | { | { |
| | | "Icon": "test1", | "k3": "http://www.ericlin.me", |
| | | "Twitter": "http://www.twitter.com", | "k4": "v4" |
| | | "URL": "test" | } |
| | | } | |
+------+------+------------------------------------------+-----------------------------------+
You can refine this SP to detect nested data and have them added to the columns list as well.
There are three tables.
Tables :
Trip
id | start_destination_id | end_destination_id | arrive_time |
-------------------------------------------------------------------
1 | S | E | 09:00 |
Destination
id | name
---------
S | Start
E | End
Schedule
id | start_destination_id | end_destination_id | should_arrive |
-------------------------------------------------------------------
1 | S | E | 08:00 |
2 | A | E | 10:00 |
Query
SELECT
Trip.*,
Schedule.should_arrive
FROM
Trip
LEFT JOIN
Schedule
ON
Trip.start_destination_id = Schedule.start_destination_id
AND
Trip.end_destination_id = Schedule.end_destination_id
I am trying to include Schedule in Trip.findAll but receive error
Exception: SequelizeEagerLoadingError: Schedule is not associated to Trip!
Is there a way that I can join them together without using foreign keys and raw queries?
Many thanks.
Finally I found a solution (not sure if it is a hack).
Schedule.ts
// add these lines
...
#ForeignKey(() => Trip)
#Column({ type: DataType.VIRTUAL })
private _dummyForTrip: undefined;
...
Then create an association between Schedule and Trip.
Trip.ts
#HasMany(() => Schedule)
public schedules: Schedule[] | null
Then you can include Schedule inside Trip by using include.on
const trips = await Trip.findAll({
include: [{
model: Schedule,
on: {
'$schedules.start$': { [Op.col]: "Trip.start_destination" },
'$schedules.end$': { [Op.col]: "Trip.end_destination" },
}
}],
where: {
id: { [Op.in]: payload.inputTripIdArr }
}
});
I need to extract the email from an intricate 'dict' (I am new to sql)
I have seen several previous posts on the same topic (e.g. this one) however, none seem to work on my data
select au.details
from table_au au
result:
{
"id":3526,
"contacts":[
{
"contactType":"EMAIL",
"value":"name#email.be",
"private":false
},
{
"contactType":"PHONE",
"phoneType":"PHONE",
"value":"025/6251111",
"private":false
}
]
}
I need:
name#email.be
select d.value -> 0 -> 'value' as Email
from json_each('{"id":3526,"contacts":[{"contactType":"EMAIL","value":"name#email.be","private":false},{"contactType":"PHONE","phoneType":"PHONE","value":"025/6251111","private":false}]}') d
where d.key::text = 'contacts'
Output:
| | email |
-------------------
|1 |"name#email.be"|
You can run it here: https://rextester.com/VHWRQ89385
I have one column table in my snowflake database that contain a JSON mapping structure as following
ColumnMappings : {"Field Mapping": "blank=Blank,E=East,N=North,"}
How to write a query that if I feed the Field Mapping a value of E I will get East or if the value if N I will get North so on and so forth without hard coding the value in the query like what CASE statement provides.
You really want your mapping in this JSON form:
{
"blank" : "Blank",
"E" : "East",
"N" : "North"
}
You can achieve that in Snowflake e.g. with a simple JS UDF:
create or replace table x(cm variant) as
select parse_json(*) from values('{"fm": "blank=Blank,E=East,N=North,"}');
create or replace function mysplit(s string)
returns variant
language javascript
as $$
res = S
.split(",")
.reduce(
(acc,val) => {
var vals = val.split("=");
acc[vals[0]] = vals[1];
return acc;
},
{});
return res;
$$;
select cm:fm, mysplit(cm:fm) from x;
-------------------------------+--------------------+
CM:FM | MYSPLIT(CM:FM) |
-------------------------------+--------------------+
"blank=Blank,E=East,N=North," | { |
| "E": "East", |
| "N": "North", |
| "blank": "Blank" |
| } |
-------------------------------+--------------------+
And then you can simply extract values by key with GET, e.g.
select cm:fm, get(mysplit(cm:fm), 'E') from x;
-------------------------------+--------------------------+
CM:FM | GET(MYSPLIT(CM:FM), 'E') |
-------------------------------+--------------------------+
"blank=Blank,E=East,N=North," | "East" |
-------------------------------+--------------------------+
For performance, you might want to make sure you call mysplit only once per value in your mapping table, or even pre-materialize it.
It seems like sql group by is more of aggregate functions (COUNT, MAX, MIN, SUM, AVG).
select count(Id), Country
from Customer
where Country <> 'CountryX'
group by Country
But do we have a linq-like query where we want to return all results grouped by a certain column, E.g. in linq I would do
id | title | category | email
------------------------------------------
1 | tname-1 | cat1 | test#example.com
2 | tname-2 | cat1 | test1#example.com
3 | tname-3 | cat2 | TEst#example.com
linq group-by:
var groupedBy = list.GroupBy(item => item.Email);
or even throw in some comparison
var groupedBy = list.GroupBy(item => item.Email, StringComparer.OrdinalIgnoreCase);
and a result will be something like:
key | items
----------------------------------------------------------------------------------------------
test#example.com | [{Id :1, Title : "tname-1", category: "cat1", email: "test#example.com" },{Id :3, Title : "tname-3", category: "cat2", email: "TEst#example.com" } ]
test1#example.com| [{Id :2, Title : "tname-2", category: "cat1", email: "test1#example.com" }]
but with sql I would definitely want to return only the subset of the columns, say id, title and email.