create a new nested JSON column in Snowflake - pandas

Question

I'm trying to convert a few columns in Snowflake table into a nested JSON and have tried OBJECT_CONSTRUCT & ARRAY_CONSTRUCT - but, unable to create a nested JSON
Input:
id
product_1
product_1_purchase_date
product_2
product_2_purchase_date
100
XCTMR
01/02/2003
IOPWER
01/02/2005
200
AQWYU
11/20/2016
XCTMR
09/09/2021
Output:
id
json_combined
100
[ { "product_1": { "name": "XCTMR", "product_1_purchase_date": "01/02/2003" } }, { "product_2": { "name": "IOPWER", "product_2_purchase_date": "01/02/2005" } } ]
200
[ { "product_1": { "name": "AQWYU", "product_1_purchase_date": "11/20/2016" } }, { "product_2": { "name": "XCTMR", "product_2_purchase_date": "09/09/2021" } } ]

Answer 1

Solution, with hard-coding field-names only, just need to put functions ARRAY_CONSTRUCT and OBJECT_CONSTRUCT in proper order.
with cte(id,product_1,product_1_purchase_date,product_2,product_2_purchase_date) as
(select * from values
(100,'XCTMR','01/02/2003','IOPWER','01/02/2005'),
(200,'AQWYU','11/20/2016','XCTMR','09/09/2021')
)select
id,
array_construct(object_construct('product_1',
object_construct('name',product_1,'product_1_purchase_date',
product_1_purchase_date)),
object_construct('product_2',
object_construct('name',product_2,'product_2_purchase_date',
product_2_purchase_date)) ) as JSON_COMBINED
from cte;
+-----+-----------------------------------------------+
| ID | JSON_COMBINED |
|-----+-----------------------------------------------|
| 100 | [ |
| | { |
| | "product_1": { |
| | "name": "XCTMR", |
| | "product_1_purchase_date": "01/02/2003" |
| | } |
| | }, |
| | { |
| | "product_2": { |
| | "name": "IOPWER", |
| | "product_2_purchase_date": "01/02/2005" |
| | } |
| | } |
| | ] |
| 200 | [ |
| | { |
| | "product_1": { |
| | "name": "AQWYU", |
| | "product_1_purchase_date": "11/20/2016" |
| | } |
| | }, |
| | { |
| | "product_2": { |
| | "name": "XCTMR", |
| | "product_2_purchase_date": "09/09/2021" |
| | } |
| | } |
| | ] |
+-----+-----------------------------------------------+
Below is with more dynamism -
CTE is pure data.
CTE_1 is creating two pseudo-columns, one for field name product_x and
another to be used as grouping criteria later in OBJECT_AGG We
cannot use aggregate function inside another aggregate function, so
we need to do aggregation multiple times (twice here). Change divide by 2 in this CTE to the number of product_x columns. We can also add a sub-query here to do this calculation, but for the purpose of this solution I've left it as hard-coded.
CTE_1 is also doing the main job of UNPIVOT product related column into rows for dynamic grouping.
CTE_2 is aggregation number - 1, with output like below (showing truncated output). Final query does the main ARRAY_AGG.
+-----+-----------+----------------------------------------------+
| ID | RN | JSON_COMBINED_1 |
|-----+-----------+----------------------------------------------|
| 100 | product_1 | { |
| | | "product_1": { |
| | | "PRODUCT_1_PURCHASE_DATE": "01/02/2003", |
| | | "name": "XCTMR" |
| | | } |
| | | } |
| 200 | product_2 | { |
| | | "product_2": { |
| | | "PRODUCT_2_PURCHASE_DATE": "09/09/2021", |
| | | "name": "XCTMR" |
| | | } |
| | | } |
+-----+-----------+----------------------------------------------+
Main query -
with cte(id,product_1,product_1_purchase_date,product_2,product_2_purchase_date) as
(select * from values
(100,'XCTMR','01/02/2003','IOPWER','01/02/2005'),
(200,'AQWYU','11/20/2016','XCTMR','09/09/2021')
),cte_1 as (
select id,
case when regexp_like(product_field,'product_[[:digit:]]+','i')
then 'name' else product_field end p_field,
product_val,
concat('product_',
to_char(ceil(row_number() over (partition by id order by null)/2))) rn
from cte
unpivot (product_val for product_field in (product_1,product_1_purchase_date,
product_2,product_2_purchase_date))
), cte_2 as (
select id,rn,
object_construct(rn,object_agg(p_field,to_variant(product_val))) JSON_COMBINED_1
from cte_1
group by id,rn
) select id, array_agg(JSON_COMBINED_1) JSON_COMBINED
from cte_2
group by id
order by id;
Final output from above query (with dynamism) -
+-----+------------------------------------------------+
| ID | JSON_COMBINED |
|-----+------------------------------------------------|
| 100 | [ |
| | { |
| | "product_1": { |
| | "PRODUCT_1_PURCHASE_DATE": "01/02/2003", |
| | "name": "XCTMR" |
| | } |
| | }, |
| | { |
| | "product_2": { |
| | "PRODUCT_2_PURCHASE_DATE": "01/02/2005", |
| | "name": "IOPWER" |
| | } |
| | } |
| | ] |
| 200 | [ |
| | { |
| | "product_2": { |
| | "PRODUCT_2_PURCHASE_DATE": "09/09/2021", |
| | "name": "XCTMR" |
| | } |
| | }, |
| | { |
| | "product_1": { |
| | "PRODUCT_1_PURCHASE_DATE": "11/20/2016", |
| | "name": "AQWYU" |
| | } |
| | } |
| | ] |
+-----+------------------------------------------------+
2 Row(s) produced.