I am generating JSON using a query that looks like this
SELECT
json_build_object(
'key1', t1.field1,
'key2', t1.field2,
'key3', t1.field3
)
FROM table1 as t1
WHERE ...
and I get results that look like this
{"key1": 123, "key2": "some string", "key3": 100}
or like this
{"key1": 123, "key2": "some string", "key3": null}
What I need is a way to make the second example drop the whole key instead of including the null value. That is, I want to somehow omit key3 when the value in field3 is null. And end up with this JSON
{"key1": 123, "key2": "some string"}
How do I go about this? I am running PostgreSQL 9.4, so some JSON operations are not available to me.
You can use json_object_agg:
SELECT
(SELECT json_object_agg(name, value)
FROM (VALUES
('key1', to_json(t1.field1)),
('key2', to_json(t1.field2)),
('key3', to_json(t1.field3))
) AS props(name, value)
WHERE value IS NOT NULL
) AS json
FROM table1 as t1
WHERE ...
Related
I want to extract some values for particular keys from a table with json string as below.
raw_data
...
{"label": "XXX", "lines":[{"amount":1000, "category": "A"}, {"amount":100, "category": "B"}, {"amount":10, "category": "C"}]}
...
I am expecting an outcome like
label
amount
category
XXX
[1000, 100, 10]
['A', 'B', 'C']
I am using the following sql query to achieve that
select
JSON_EXTRACT(raw_data, '$.lines[*].amount') AS amount,
JSON_EXTRACT(raw_data, '$.lines[*].category') AS category,
JSON_EXTRACT(raw_data, '$.label') AS label
from table
I can get a specific element of the list with [0] , [1] etc. But the sql code doesn't work with [*]. I am getting the following error
Invalid JSON path: '$.lines[*].amount'
Edit
I am using Presto
Json path support in Presto is very limited, so you need to do some processing manually for example with casts and array functions:
-- sample data
with dataset (raw_data) as (
values '{"label": "XXX", "lines":[{"amount":1000, "category": "A"}, {"amount":100, "category": "B"}, {"amount":10, "category": "C"}]}'
)
-- query
select label,
transform(lines, l -> l['amount']) amount,
transform(lines, l -> l['category']) category
from (
select JSON_EXTRACT(raw_data, '$.label') AS label,
cast(JSON_EXTRACT(raw_data, '$.lines') as array(map(varchar, json))) lines
from dataset
);
Output:
label
amount
category
XXX
[1000, 100, 10]
["A", "B", "C"]
In Trino json path support was vastly improved, so you can do next:
-- query
select JSON_EXTRACT(raw_data, '$.label') label,
JSON_QUERY(raw_data, 'lax $.lines[*].amount' WITH ARRAY WRAPPER) amount,
JSON_QUERY(raw_data, 'lax $.lines[*].category' WITH ARRAY WRAPPER) category
from dataset;
You can use json_table and json_arrayagg:
select json_extract(t.raw_data, '$.label'),
(select json_arrayagg(t1.v) from json_table(t.raw_data, '$.lines[*]' columns (v int path '$.amount')) t1),
(select json_arrayagg(t1.v) from json_table(t.raw_data, '$.lines[*]' columns (v text path '$.category')) t1)
from tbl t
I was able to get the expected output using unnest to flatten and array_agg to aggregate in Presto. Below is the SQL used and output generated:
WITH dataset AS (
SELECT
* from sf_73535794
)
SELECT raw_data.label,array_agg(t.lines.amount) as amount,array_agg(t.lines.category) as category FROM dataset
CROSS JOIN UNNEST(raw_data.lines) as t(lines) group by 1
Output:
I have this Gaint Array of (dicts) loaded from a Json in a date partitioned big query external table with table structure as below as
Field name
Type.
Mode
meta
Record
Nullable
Messages
String
Repeated
date
Integer
Nullable
Every "Messages" Field is in its own row/record in my Bigquery table (New_line_delimited_Json)
I am trying to parse the "messages" field/column to extract some fields Key1 and Key2 which happens to be inside an Array (of dicts). For sake of simplicity ,below is the snippet of json of which "messages" is a field that I am trying to unnest/explode.
Ignore this schema;updated schema below***
[
{
"meta": {
"table": "FEED",
"source": "CP1"
},
"Messages": [
"{
"Key1":"2022-01-10",
"Key2":"H21257061"
}"
],
"date": "20220110"
},
{
"meta": {
"table": "FEED",
"source": "CP1"
},
"Messages": [
"{
"Key1":"2022-01-11",
"Key2":"H21257062"
}"
],
"date": "20220111"
}
]
updated schema on 01/17
{
"meta": {
"table": "FEED",
"source": "CP1"
},
"Messages": [
"{
"Key1":"2022-01-10",
"Key2":"H21257061"
}",
"{
"Key1":"2022-01-10",
"Key2":"H21257062"
}"
],
"date": "20220110"
},
updated schema representation on 01/17:
so far I have tried this but I am getting sql output of key1 and Key2 as Nulls
WITH table AS (SELECT Messages as array_column FROM `project.dataset.table` )
SELECT
json_extract_scalar(flattened_array, '$.Messages.key1') as key1,
json_extract_scalar(flattened_array, '$.Messages.key2') as key2
FROM table t
CROSS JOIN UNNEST(t.array_column) AS flattened_array
Still a little ambiguous so I assume below correctly represents your table (at least it matches the structure/schema in your question)
If my assumption correct - consider below approach
select * except(id) from (
select to_json_string(t) id, kv[offset(0)] as key, kv[safe_offset(1)] as value
from your_table t,
t.messages as message,
unnest([struct( split(translate(message, '"', ''), ':') as kv)])
)
pivot (min(value) for key in ('Key1', 'Key2'))
If / when applied to above sample data - output is
Edit: Trying to help you further - Ok so, looks like your table looks like below
In this case - try below (quite light modification of previous version)
select * except(id) from (
select to_json_string(t) id, kv[offset(0)] as key, kv[safe_offset(1)] as value
from your_table t,
unnest(regexp_extract_all(messages, r'"[^"]+":"[^"]+"')) as message,
unnest([struct( split(translate(message, '"', ''), ':') as kv)])
)
pivot (min(value) for key in ('Key1', 'Key2'))
with output
But obviously, I would use below simplest approach
select
json_extract_scalar(messages, '$.Key1') as Key1,
json_extract_scalar(messages, '$.Key2') as Key2
from your_table
SELECT
JSON_QUERY(message,"$.Key1") as Key1,
JSON_QUERY(message,"$.Key2") as Key2
FROM
`project.dataset.table` as table
CROSS JOIN UNNEST(table.Messages) as message
CROSS JOIN for flattening the array,which will return a row for each message.
After that “JSON_QUERY” to extract the needed values from the JSON string.
I have a jsonb data column made up of various objects. Here is an example.
{"LicensePlates": {"Type": "LicensePlateList", "Value": ["XXXXX"]}, "SubscriptionInfo": {"Type": "SubscriptionInfoList", "Value": [{"id": "1", "lastname": "rossi", "firstname": "paola"}, {"id": "2", "lastname": "Scicolone", "firstname": "Paolo"}]}}
Now I'm searching a specific info in SubscriptionInfo key like this:
SELECT * FROM column WHERE (data -> 'SubscriptionInfo') -> 'Value' #> '[{"firstname": "Paolo"}]';
It works fine, but I would also like to search for "partial" information, eg. searching for the string "pa" (using ILIKE or anything else similar) should return the entire record. it's possible?
You have two options (demo)
convert data to lower case
select *
from
test
where
lower(data -> 'SubscriptionInfo' ->> 'Value')::jsonb #> lower('[{"firstname": "paolo"}]')::jsonb;
Use cross join and extract JSON then use ilike
select distinct on (t.id) t.*
from
test t
cross join jsonb_array_elements(data -> 'SubscriptionInfo' -> 'Value') ej
where
value ->> 'firstname' ilike '%paolo%';
If you are using Postgres 13 or later, you can use a SQL/JSON path expression:
select t.*
from the_table t
where t.data ## '$.SubscriptionInfo.Value[*].firstname like_regex "paolo" flag "i"'
I have field in database with type json:
[{"id": "1"}, {"id": "2"}, {"id": "3"}]
and I need get matches in json with array ["1", "2"] for at least one element.
Something like this:
select t.*
from the_table t
where exists (select *
from jsonb_array_elements(t.the_column) as x(item)
join jsonb_array_elements_text('["1", "2"]') as e(id)
on x.item ->> 'id' = e.id);
Online example
I'm trying to read the column which type is json, values in column look like this
column1
---------------------------------------------
"[{'name': 'Kate', 'position': 'painter'}]"
Im using this query, but all I get is null, what can I do to get the values for each keys?
SELECT
column1 ->> 'name' AS name
FROM
table1;
Then you use jsonb_pretty that Returns from_json as indented JSON text.
select jsonb_pretty('[{"name": "Kate", "position": "painter"}]');
Output display you:
jsonb_pretty
-------------------------------
[ +
{ +
"name": "Kate", +
"position": "painter"+
} +
]
so in your case you use
SELECT jsonb_pretty(column1) AS name FROM table1;
Use json_array_elements function:
SELECT json_array_elements(t) -> 'name'
FROM table1;