Postgresql update record column based on it's JSONB field values sum - sql

I have a table in Postgresql 9.6.5 with some fields like:
CREATE TABLE test (
id SERIAL,
data JSONB,
amount DOUBLE PRECESION,
PRIMARY KEY(id)
);
In data column there are json objects like this:
{
"Type": 1,
"CheckClose":
{"Payments":
[
{"Type": 4, "Amount": 2068.07},
{"Type": 1, "Amount": 1421.07}
]
}
}
What i need to do is tu put into amount field of each row the SUM of Amount values of Payments field od this data object. For example, for this particular object there should be 2068.07 + 1421.07 = 3489.14.
I've read some stuff about Postgres json and jsonb functions, so here where i am now:
UPDATE test SET amount=sum((jsonb_array_elements(data::jsonb->'CheckClose'->'Payments')->>'Amount')::FLOAT)
That's not working - i get an error about not using agregate functions in UPDATE.
I tried to do this something like this:
UPDATE test SET amount=temp.sum
FROM (
SELECT sum((jsonb_array_elements(data::jsonb->'CheckClose'->'Payments')->>'Amount')::FLOAT) AS "sum"
FROM test WHERE id=test.id
) as "temp"
Now i'm getting an error set-valued function called in context that cannot accept a set
How should i do this? I just need to calculate sum and put it into another row, is that such a hard task?
Please, anyone, help me to figure this out. Thanks.

the set returning fn() aggregation try:
t=# with c(j) as (values('{"Payments":
[
{"Type": 4, "Amount": 2068.07},
{"Type": 1, "Amount": 1421.07}
]
}'::jsonb))
select sum((jsonb_array_elements(j->'Payments')->>'Amount')::float) from c;
error:
ERROR: aggregate function calls cannot contain set-returning function calls
LINE 7: select sum((jsonb_array_elements(j->'Payments')->>'Amount'):...
^
HINT: You might be able to move the set-returning function into a LATERAL FROM item.
can easily be overcame by another cte:
t=# with c(j) as (values('{"Payments":
[
{"Type": 4, "Amount": 2068.07},
{"Type": 1, "Amount": 1421.07}
]
}'::jsonb))
, a as (select (jsonb_array_elements(j->'Payments')->>'Amount')::float am from c)
select sum(am) from a;
sum
---------
3489.14
(1 row)
so now just update from CTE:
with s as (SELECT ((jsonb_array_elements(data::jsonb->'CheckClose'->'Payments')->>'Amount')::FLOAT) AS "sm", id
FROM test
)
, a as (select sum(sm), id from s group by id)
UPDATE test SET amount = sum
FROM a
WHERE id=test.id

Related

In Snowflake, how do I parse an unnamed JSON array and access each key with key value rather than array slicing methods?

I have a JSON array in Snowflake, where I need to parse into a table, i.e., convert all information into a row. Assume my table includes two columns: id and json_tag column.
An example row is as below:
{ [
json_tag
[
{
"key": "app.name",
"value": "myapp1"
},
{
"key": "device.name",
"value": "myiPhone11"
},
{
"key": "iOS.dist",
"value": "latestDist5"
}
]
This is not a standard example, another example can have 5 or even 6 with new key names. An example is below:
{ [
json_tag
[
{
"key": "app.name",
"value": "myapp2"
},
{
"key": "app.cost",
"value": "$2.99"
},
{
"key": "device.name",
"value": "myiPhoneX"
},
{
"key": "device.color",
"value": "gold"
},
{
"key": "iOS.dist",
"value": "latestDist4.9"
}
]
What I want is working on a table (without creating a new one) where the json row is split into columns as below:
id app.name app.cost device.name device.color iOS.dist
1 myapp1 null myiPhone11 null latestDist5
2 myapp2 $2.99 myiPhoneX gold latestDist4.9
I tried the following snippet:
with parsed_tb as (
select id,
to_variant(parse_json(json_tag)) as parsed_json_tag
from mytable
)
select parsed_json_tag[0]:value::varchar as app_name,
parsed_json_tag[1]:value::varchar as app_cost,
parsed_json_tag[2]:value::varchar as device_name
from tb;
As you can imagine, the snippet above does not work when there is no app.cost in key values or every row differs in number of keys and values.
I tried lateral flatten command in Snowflake, but out creates many rows and I cannot figure out how to put them in columns in the same row. I also tried using recursive command, and could not achieve it.
So my question is:
How can I access a key by its name rather than slicing an array as I do above? - this would solve my problem I guess.
If #1 solution I imagine does not fix, how can I attain the table above?
I have found a solution to this problem with the help of following thread:
Mysql, reshape data from long / tall to wide
Solution is first parsing json column with unique identifier, and then lateral flattening and then grouping the results by unique identifier [formatting table from long to wide as given in the link above].
So for the examples I provided above, the solution would be as follows:
with tb as (
select id,
to_variant(parse_json(json_tags)) as parsed_json_tags
from mytable
),
flattened as (
select id,
VALUE:value::string as value,
VALUE:key::string as key
from mytable, lateral flatten(input => (tb.parsed_json_tags))
)
select id,
MAX( IFF( key='app.name', value, NULL ) ) AS app_name,
MAX( IFF( key='app.cost', value, NULL ) ) AS app_cost,
MAX( IFF( key='device.name', value, NULL ) ) AS device_name,
MAX( IFF( key='device.color', value, NULL ) ) AS device_color,
MAX( IFF( key='iOS.dist', value, NULL ) ) AS ios_dist
from flattened
group by 1;

PostgreSQL JSON - String_agg in json data with multiple objects

I have create table with Json datatype field in PostgreSQL.
Also i have inserted data with multiple object in json data
like this
[
{
"Type": "1",
"Amount": "1000",
"Occurrence": 2,
"StartDate": "1990-01-19",
"EndDate": "1999-04-03"
},
{
"Type": "2",
"Amount": "2000",
"Occurrence": 2,
"StartDate": "1984-11-19",
"EndDate": "1997-09-29"
}
]
Now i have to retrieve my data as per below formate in single row like string_agg() function output.
Type Amount
1--2 1000-2000
also i have checked inbuilt function for json in PostgreSQL (https://www.postgresqltutorial.com/postgresql-json/) but not find any solutions for the same.
You will have to unnest the array and aggregate the individual keys:
The following assumes you have some kind of primary key column on the table (in addition to your JSON column):
select t.id,
string_agg(x.element ->> 'Type', '-') as types,
string_agg(x.element ->> 'Amount', '-') as amounts
from the_table t
cross join jsonb_array_elements(t.data) as x(element)
group by t.id;
jsonb_array_elements() extracts each array element and the main query then aggregates that back per ID using string_agg().
The above assumes your column is defined with the type jsonb (which it should be). If it is not, you need to use json_array_elements() instead.
Online example

Query all objects within json array

I'm looking to do a query on a column in my database but the column is of type jsonb. This is an example of the structure:
select json_column->>'left' from schema.table;
[{"id": 123, "name": "Joe"},
{"id": 456, "name": "Jane"},
{"id": 789, "name": "John"},
{"id": 159, "name": "Jess"}]
Essentially I'm just trying to return all the name fields from this but I can't figure it out.
I have tried
select json_column->'left'->>'name' from schema.table
But this returns a blank value just.
I have also tried:
select elem->>'name'
from schema.table m,
jsonb_array_elements(json_column->'left') elem;
But that gives me:
ERROR: cannot extract elements from an object
This seems to work when I have a where clause inserted, for example:
select elem->>'name'
from schema.table m,
jsonb_array_elements(json_column->'left') elem
where m.id = 1;

How to read JSON key values as a data column in Snowflake?

I have the below sample JSON:
{
"Id1": {
"name": "Item1.jpg",
"Status": "Approved"
},
"Id2": {
"name": "Item2.jpg",
"Status": "Approved"
}
}
and I am trying to get the following output:
_key name Status
Id1 Item1.jpg Approved
Id2 Item2.jpg Approved
Is there any way I can achieve this in Snowflake using SQL?
You should use Snowflake's VARIANT data type in any column holding JSON data. Let's break this down step by step:
create temporary table FOO(v variant); -- Temp table to hold the JSON. Often you'll see a variant column simply called "V"
-- Insert into the variant column. Parse the JSON because variants don't hold string types. They hold semi-structured types.
insert into FOO select parse_json('{"Id1": {"name": "Item1.jpg", "Status": "Approved"}, "Id2": {"name": "Item2.jpg", "Status": "Approved"}}');
-- See how it looks in its raw state
select * from FOO;
-- Flatten the top-level JSON. The flatten function breaks down the JSON into several usable columns
select * from foo, lateral flatten(input => (foo.v)) ;
-- Now traverse the JSON using the column name and : to get to the property you want. Cast to string using ::string.
-- If you must have exact case on your column names, you need to double quote them.
select KEY as "_key",
VALUE:name::string as "name",
VALUE:Status::string as "Status"
from FOO, lateral flatten(input => (FOO.V)) ;

BigQuery select expect double nested column

I am trying to remove a column from a BigQuery table and I've followed the instructions as stated here:
https://cloud.google.com/bigquery/docs/manually-changing-schemas#deleting_a_column_from_a_table_schema
This did not work directly as the column I'm trying to remove is nested twice in a struct. The following SO questions are relevant but none of them solve this exact case.
Single nested field:
BigQuery select * except nested column
Double nested field (solution has all fields in the schema enumerated, which is not useful for me as my schema is huge):
BigQuery: select * replace from multiple nested column
I've tried adapting the above solutions and I think I'm close but can't quite get it to work.
This one will remove the field, but returns only the nested field, not the whole table (for the examples I want to remove a.b.field_name. See the example schema at the end):
SELECT AS STRUCT * EXCEPT(a), a.* REPLACE (
(SELECT AS STRUCT a.b.* EXCEPT (field_name)) AS b
)
FROM `table`
This next attempt gives me an error: Scalar subquery produced more than one element:
WITH a_tmp AS (
SELECT AS STRUCT a.* REPLACE (
(SELECT AS STRUCT a.b.* EXCEPT (field_name)) AS b
)
FROM `table`
)
SELECT * REPLACE (
(SELECT AS STRUCT a.* FROM a_tmp) AS a
)
FROM `table`
Is there a generalised way to solve this? Or am I forced to use the enumerated solution in the 2nd link?
Example Schema:
[
{
"name": "a",
"type": "RECORD",
"fields": [
{
"name": "b",
"type": "RECORD"
"fields": [
{
"name": "field_name",
"type": "STRING"
},
{
"name": "other_field_name".
"type": "STRING"
}
]
},
]
}
]
I would like the final schema to be the same but without field_name.
Below is for BigQuery Standard SQL
#standardSQL
SELECT * REPLACE(
(SELECT AS STRUCT(SELECT AS STRUCT a.b.* EXCEPT (field_name)) b)
AS a)
FROM `project.dataset.table`
you can test, play with it using dummy data as below
#standardSQL
WITH `project.dataset.table` AS (
SELECT STRUCT<b STRUCT<field_name STRING, other_field_name STRING>>(STRUCT('1', '2')) a
)
SELECT * REPLACE(
(SELECT AS STRUCT(SELECT AS STRUCT a.b.* EXCEPT (field_name)) b)
AS a)
FROM `project.dataset.table`