Update ARRAY IN ARRAY BigQuery (GA4 Data) - sql

I want to update a value in array with a value in another table.
table1:
event_params.key
event_params.value.string_value
country
US
table2:
country
new_country
US
NL
I try
UPDATE table1
SET event_params = ARRAY(SELECT AS STRUCT * REPLACE ( new_country AS value.string_value ) FROM UNNEST(event_params)
WHERE key = "country")
FROM(SELECT country, new_country FROM table2)
WHERE
(SELECT value.string_value from unnest(event_params) where key = 'country') = t2.country
It doesnt work.. The problem is the value.string_value because is also an array.
The final table should look like
table1:
event_params.key
event_params.value.string_value
country
NL
Now I want to update table1 with table2 to udate the country from US to NL in table1.
How can I take the array in array?

I also saw this code:
UPDATE `project.dataset.your_table` t
SET hits =
ARRAY(
SELECT AS STRUCT * REPLACE(
ARRAY(
SELECT AS STRUCT product.* REPLACE(
CASE WHEN map.raw_name = product.productCategory THEN category
ELSE productCategoryAttribute END AS productCategoryAttribute)
FROM UNNEST(product) product
LEFT JOIN UNNEST(agg_map.map) map
ON map.raw_name = product.productCategory
) AS product)
FROM UNNEST(hits) hit
)
FROM (SELECT ARRAY_AGG(row) map FROM `project.dataset.map` row) agg_map
WHERE TRUE
BigQuery UPDATE nested array field
But I cant implement it.

Related

Use 2 keys in a query

I have in where event_params.key = 'page_referrer'
but, i want to see in select values event_params.key = 'traffic_type'
I need both fields
I wanted to use JOIN tables with itself, but didn't work out
Would you try below ?
SELECT (SELECT value.string_value FROM t.event_params WHERE key = 'traffic_type') traffic_type
FROM `coolclever-1148.analytic_xxxxxx.events_20230130` t
WHERE event_name = 'view_item' AND 'page_referrer' IN UNNEST(event_params.key);
Example Query
To show you working example with public dataset:
SELECT (SELECT value.string_value FROM t.event_params WHERE key = 'page_location') page_location
FROM `bigquery-public-data.ga4_obfuscated_sample_ecommerce.events_*` t
WHERE event_name = 'page_view' AND 'page_title' IN UNNEST(event_params.key);
Query results

how to delete data array on jsonb postgresql

how to update array data in jsonb column on database postgresql?
for example on table table1 i have column attribute that have value like this:
id
attribute
1
[{"task_customs": ["a", "b", "c"]}]
2
[{"task_customs": ["d", "e", "f"]}]
for example if i want to delete b from id 1, so it will be like this on attribute column
id
attribute
1
[{"task_customs": ["a", "c"]}]
2
[{"task_customs": ["d", "e", "f"]}]
already do some research but didn't get what i need..
try this :
(a) Delete 'b' acccording to its position in the array :
UPDATE table1
SET attribute = attribute #- array['0', 'task_customs', '1'] :: text[]
WHERE id = 1
(b) Delete 'b' without knowing its position in the array :
WITH list AS
( SELECT id, to_jsonb(array[jsonb_build_object('task_customs', jsonb_agg(i.item ORDER BY item_id))]) AS new_attribute
FROM table1
CROSS JOIN LATERAL jsonb_array_elements_text(attribute#>'{0,task_customs}') WITH ORDINALITY AS i(item,item_id)
WHERE id = 1
AND i.item <> 'b'
GROUP BY id
)
UPDATE table1 AS t
SET attribute = l.new_attribute
FROM list AS l
WHERE t.id = l.id
see the test result in dbfiddle.
One option is to start splitting the JSONB value by using jsonb_to_recordset such as
UPDATE table1 AS t
SET attribute =
(
SELECT json_build_array(
jsonb_build_object('task_customs',task_customs::JSONB - 'b')
)
FROM table1,
LATERAL jsonb_to_recordset(attribute) AS (task_customs TEXT)
WHERE id = t.id
)
WHERE id = 1
Demo
Edit : If you need more elements as expressed within the comment then you can rather prefer using
UPDATE table1 AS t
SET attribute =
(
SELECT jsonb_agg(
jsonb_build_object(key,je.value::JSONB - 'b')
)
FROM table1,
LATERAL jsonb_array_elements_text(attribute) AS atr,
LATERAL jsonb_each_text(atr::JSONB) AS je
WHERE id = t.id
)
WHERE id = 1
Demo
i solve this issue by combining both answer from Edouard and Barbaros
this is my final query
UPDATE table1 AS t
SET attribute =
jsonb_set(
attribute,
'{0,task_customs}',
(
SELECT task_customs::JSONB - 'b'
FROM table1
CROSS JOIN LATERAL jsonb_to_recordset(attribute) AS (task_customs TEXT)
WHERE id = t.id
)
)
WHERE id = 1
see the test result in dbfiddle

Join three rows if the same value in one column

There is a Postgres database and the table has three columns. The data structure is in external system so I can not modify it.
Every object is represented by three rows (identified by column element_id - rows with the same value in this column represents the same object), for example:
key value element_id
-----------------------------------
status active 1
name exampleNameAAA 1
city exampleCityAAA 1
status inactive 2
name exampleNameBBB 2
city exampleCityBBB 2
status inactive 3
name exampleNameCCC 3
city exampleCityCCC 3
I want to get all values describing every objects (name, status and city).
For this example the output should be like:
exampleNameAAA | active | exampleCityAAA
exampleNameBBB | inactive | exampleCityBBB
exampleNameCCC | inactive | exampleCityCCC
I know how to join two rows:
select a.value as name,
b.value as status
from the_table a
join the_table b
on a.element_id = b.element_id
and b."key" = 'status'
where a."key" = 'name';
How is it possible to join three columns?
You can try below
DEMO
select a.value as name,
b.value as status,c.value as city
from t1 a
join t1 b
on a.element_id = b.element_id and b."keys" = 'status'
join t1 c on a.element_id = c.element_id and c."keys" = 'city'
where a."keys" = 'name';
OUTPUT
name status city
exampleNameAAA active exampleCityAAA
exampleNameBBB inactive exampleCityBBB
exampleNameCCC inactive exampleCityCCC
One option is to simply add another join for each value you need (this is one of the big disadvantages of the EAV (anti) pattern you are using:
select a.value as name,
b.value as status,
c.value as city
from the_table a
join the_table b on a.element_id = b.element_id and b."key" = 'status'
join the_table c on a.element_id = c.element_id and c."key" = 'city'
where a."key" = 'name';
Another option is to aggregate all key/value pairs for an element into a JSON then you can easily access each one without additional joins:
select t.element_id,
t.obj ->> 'city' as city,
t.obj ->> 'status' as status,
t.obj ->> 'name' as name
from (
select e.element_id, jsonb_object_agg("key", value) as obj
from element e
group by e.element_id
) t;
If the table is really big this might be a lot slower than the join version due to the aggregation step. If you limit the query to only some elements (e.g. by adding a where element_id = 1 or where element_id in (1,2,3)) then this should be quite fast.
It has the advantage that you always have all key/value pairs for each element_id available regardless on what you do. The inner select could be put into a view, to make things easier.
Online example: https://rextester.com/MSZOWU37182
Seems like you want to PIVOT
One way to do that is via conditional aggregation.
select
-- t.element_id,
max(case when t.key = 'name' then t.value end) as name,
max(case when t.key = 'status' then t.value end) as status,
max(case when t.key = 'city' then t.value end) as city
from the_table t
group by t.element_id;
db<>fiddle here
Or use crosstab:
select
-- element_id,
name,
status,
city
from crosstab (
'select t.element_id, t.key, t.value
from the_table t'
) as ct (element_id int, name varchar(30), status varchar(30), city varchar(30));
But if you do like those joins, here's a way
select
-- el.element_id,
nm.value as name,
st.value as status,
ci.value as city
from
(
select distinct t.element_id
from the_table t
where t.key in ('name','status','city')
) as el
left join the_table as nm on (nm.element_id = el.element_id and nm.key = 'name')
left join the_table as st on (st.element_id = el.element_id and st.key = 'status')
left join the_table as ci on (ci.element_id = el.element_id and ci.key = 'city');

How to find the key for the minimum value in jsonb column of postgres?

I need to find the key of the minimum value in a jsonb object,I have found out minimum value, need to find the key of the same in the same query.
Query I am using
SELECT id,min((arr ->> 2)::numeric) AS custom_value
FROM (
SELECT id, jdoc
FROM table,
jsonb_each(column1) d (key, jdoc)
) sub,
jsonb_each(jdoc) doc (key, arr)
group by 1
This will do the job.
The left join ... on 1=1 is for keeping IDs with empty json
select t.id
,j.key
,j.value
from mytable t
left join lateral (select j.key,j.value
from jsonb_each(column1) as j
order by j.value
limit 1
) j
on 1=1

SQL Elaborate Joins Query

I'm trying to solve the below problem.
I feel like it is possible, but I can't seem to get it.
Here's the scenario:
Table 1 (Assets)
1 Asset-A
2 Asset-B
3 Asset-C
4 Asset-D
Table 2 (Attributes)
1 Asset-A Red
2 Asset-A Hard
3 Asset-B Red
4 Asset-B Hard
5 Asset-B Heavy
6 Asset-C Blue
7 Asset-C Hard
If I am looking for something having the same attributes as Asset-A, then it should identify Asset-B since Asset-B has all the same attributes as Asset-A (it should discard heavy, since Asset-A didn't specify anything different or the similar). Also, if I wanted the attributes for only Asset-A AND Asset-B that were common, how would I get that?
Seems simple, but I can't nail it...
The actual table I am using, is almost precisely Table2, simply an association of an AssetId, and an AttributeId so:
PK: Id
int: AssetId
int: AttributeId
I only included the idea of the asset table to simplify the question.
SELECT ato.id, ato.value
FROM (
SELECT id
FROM assets a
WHERE NOT EXISTS
(
SELECT NULL
FROM attributes ata
LEFT JOIN
attributes ato
ON ato.id = ata.id
AND ato.value = ata.value
WHERE ata.id = 1
AND ato.id IS NULL
)
) ao
JOIN attributes ato
ON ato.id = ao.id
JOIN attributes ata
ON ata.id = 1
AND ata.value = ato.value
, or in SQL Server 2005 (with sample data to check):
WITH assets AS
(
SELECT 1 AS id, 'A' AS name
UNION ALL
SELECT 2 AS id, 'B' AS name
UNION ALL
SELECT 3 AS id, 'C' AS name
UNION ALL
SELECT 4 AS id, 'D' AS name
),
attributes AS
(
SELECT 1 AS id, 'Red' AS value
UNION ALL
SELECT 1 AS id, 'Hard' AS value
UNION ALL
SELECT 2 AS id, 'Red' AS value
UNION ALL
SELECT 2 AS id, 'Hard' AS value
UNION ALL
SELECT 2 AS id, 'Heavy' AS value
UNION ALL
SELECT 3 AS id, 'Blue' AS value
UNION ALL
SELECT 3 AS id, 'Hard' AS value
)
SELECT ato.id, ato.value
FROM (
SELECT id
FROM assets a
WHERE a.id <> 1
AND NOT EXISTS
(
SELECT ata.value
FROM attributes ata
WHERE ata.id = 1
EXCEPT
SELECT ato.value
FROM attributes ato
WHERE ato.id = a.id
)
) ao
JOIN attributes ato
ON ato.id = ao.id
JOIN attributes ata
ON ata.id = 1
AND ata.value = ato.value
I don't completely understand the first part of your question, identifying assets based on their attributes.
Making some assumptions about column names, the following query would yield the common attributes between Asset-A and Asset-B:
SELECT [Table 2].Name
FROM [Table 2]
JOIN [Table 1] a ON a.ID = [Table 2].AssetID AND a.Name = 'Asset-A'
JOIN [Table 1] b ON b.ID = [Table 2].AssetID AND b.Name = 'Asset-B'
GROUP BY [Table 2].Name
Select * From Assets A
Where Exists
(Select * From Assets
Where AssetId <> A.AssetID
And (Select Count(*)
From Attributes At1 Join Attributes At2
On At1.AssetId <> At2.AssetId
And At1.attribute <> At2.Attribute
Where At1.AssetId = A.AssetId Asset) = 0 )
And AssetId = 'Asset-A'
select at2.asset, count(*)
from attribute at1
inner join attribute at2 on at1.value = at2.value
where at1.asset = "Asset-A"
and at2.asset != "Asset-A"
group by at2.asset
having count(*) = (select count(*) from attribute where asset = "Asset-A");
Find all assets who have every attribute that "A" has (but also may have additional attributes):
SELECT Other.ID
FROM Assets Other
WHERE
Other.AssetID <> 'Asset-A' -- do not return Asset A as a match to itself
AND NOT EXISTS (SELECT NULL FROM Attributes AttA WHERE
AttA.AssetID='Asset-A'
AND NOT EXISTS (SELECT NULL FROM Attributes AttOther WHERE
AttOther.AssetID=Other.ID AND AttOther.AttributeID = AttA.AttributeID
)
)
I.e., "find any asset where there is no attribute of A that is not also an attribute of this asset".
Find all assets who have exactly the same attributes as "A":
SELECT Other.ID
FROM Assets Other
WHERE
Other.AssetID <> 'Asset-A' -- do not return Asset A as a match to itself
AND NOT EXISTS (SELECT NULL FROM Attributes AttA WHERE
AttA.AssetID='Asset-A'
AND NOT EXISTS (SELECT NULL FROM Attributes AttOther WHERE
AttOther.AssetID=Other.ID
AND AttOther.AttributeID = AttA.AttributeID
)
)
AND NOT EXISTS (SELECT NULL FROM Attributes AttaOther WHERE
AttaOther.AssetID=Other.ID
AND NOT EXISTS (SELECT NULL FROM Attributes AttaA WHERE
AttaA.AssetID='Asset-A'
AND AttaA.AttributeID = AttaOther.AttributeID
)
)
I.e., "find any asset where there is no attribute of A that is not also an attribute of this asset, and where there is no attribute of this asset that is not also an attribute of A."
This solution works as prescribed, thanks for the input.
WITH Atts AS
(
SELECT
DISTINCT
at1.[Attribute]
FROM
Attribute at1
WHERE
at1.[Asset] = 'Asset-A'
)
SELECT
DISTINCT
Asset,
(
SELECT
COUNT(ta2.[Attribute])
FROM
Attribute ta2
INNER JOIN
Atts b
ON
b.[Attribute] = ta2.[attribute]
WHERE
ta2.[Asset] = ta.Asset
)
AS [Count]
FROM
Atts a
INNER JOIN
Attribute ta
ON
a.[Attribute] = ta.[Attribute]
Find all assets that have all the same attributes as asset-a:
select att2.Asset from attribute att1
inner join attribute att2 on att2.Attribute = att1.Attribute and att1.Asset <> att2.Asset
where att1.Asset = 'Asset-A'
group by att2.Asset, att1.Asset
having COUNT(*) = (select COUNT(*) from attribute where Asset=att1.Asset)
I thought maybe I can do this with LINQ and then work my way backwards with:
var result = from productsNotA in DevProducts
where productsNotA.Product != "A" &&
(
from productsA in DevProducts
where productsA.Product == "A"
select productsA.Attribute
).Except
(
from productOther in DevProducts
where productOther.Product == productsNotA.Product
select productOther.Attribute
).Single() == null
select new {productsNotA.Product};
result.Distinct()
I thought that translating this back to SQL with LinqPad would result into a pretty SQL query. However it didn't :). DevProducts is my testtable with a column Product and Attribute. I thought I'd post the LINQ query anyways, might be useful to people who are playing around with LINQ.
If you can optimize the LINQ query above, please let me know (it might result in better SQL ;))
I'm using following DDL
CREATE TABLE Attributes (
Asset VARCHAR(100)
, Name VARCHAR(100)
, UNIQUE(Asset, Name)
)
Second question is easy
SELECT Name
FROM Attributes
WHERE Name IN (SELECT Name FROM Attributes WHERE Asset = 'A')
AND Asset = 'B'
First question is not more difficult
SELECT Asset
FROM Attributes
WHERE Name IN (SELECT Name FROM Attributes WHERE Asset = 'A')
GROUP BY Asset
HAVING COUNT(*) = (SELECT COUNT(*) FROM FROM Attributes WHERE Asset = 'A')
Edit:
I left AND Asset != 'A' out of the WHERE clause of the second snippet for brevity