I am running an SQL query to extract nested JSON data.
SELECT watsonResponse.responseId,
watsonResponse.chatId,
d.*
FROM watson_response_table watsonResponse
CROSS JOIN LATERAL (
SELECT d2.*
FROM jsonb_array_elements(watsonResponse.watsonresponse_output) AS d(events)
CROSS JOIN LATERAL (
SELECT d2.events ->> 'data' AS watsonResponse_ouput_data
, d2.events ->> 'text' AS watsonResponse_output_text
, d2.events ->> 'uiActionCode' AS watsonResponse_output_uiActionCode
FROM jsonb_array_elements(d.events) AS d2(events)
) d2
WHERE d.events ->> 'uiActionCode' = 'TextWithButton'
) d;
It fails with message SQL Error [22023]: ERROR: cannot extract elements from an object
I am using PostgresSQL 11+. Here is what the JSON looks like,
[
{
"text": [
"Some text!"
],
"uiActionCode": "textOnly"
},
{
"data": {
"type": "options",
"options": [
{ "label": "test", "value": "testvalue" },
{ "label": "test2", "value": "testvalue2" },
{
"label": "test3",
"value": "testQuestion?"
}
]
},
"text": ["testQuestion2?"],
"uiActionCode": "TextWithButton"
}
]
If I am following this correctly, one level of unnesting is sufficient. You can then use the JSON accessors to get the results you want:
SELECT
r.responseId,
r.chatId,
d.events ->> 'uiActionCode' AS output_uiActionCode,
d.events -> 'text' ->> 0 AS output_text,
d.events -> 'data' AS output_data,
FROM watson_response_table watsonResponse r
CROSS JOIN LATERAL jsonb_array_elements(r.watsonresponse_output) AS d(events)
WHERE d.events ->> 'uiActionCode' = 'TextWithButton'
Note that there is an important difference between accessors -> and ->>. The former returns an object, while the latter returns a text value. You need to carefully pick the correct operator according to what needs to be done for each field.
Related
I have a table "blobs" with a column "metadata" in jsonb data-type,
Example:
{
"total_count": 2,
"items": [
{
"name": "somename",
"metadata": {
"metas": [
{
"id": "11258",
"score": 6.1,
"status": "active",
"published_at": "2019-04-20T00:29:00",
"nvd_modified_at": "2022-04-06T18:07:00"
},
{
"id": "9251",
"score": 5.1,
"status": "active",
"published_at": "2018-01-18T23:29:00",
"nvd_modified_at": "2021-01-08T12:15:00"
}
]
}
]
}
I want to identify statuses in the "metas" array that match with certain, given strings. I have tried the following so far but without results:
SELECT * FROM blobs
WHERE metadata is not null AND
(
SELECT count(*) FROM jsonb_array_elements(metadata->'metas') AS cn
WHERE cn->>'status' IN ('active','reported')
) > 0;
It would also be sufficient if I could compare the string with "status" in the first array object.
I am using PostgreSQL 9.6.24
for some clarity I usually break code into series of WITH statements. My idea for your problem would be to use json path (https://www.postgresql.org/docs/12/functions-json.html#FUNCTIONS-SQLJSON-PATH) and function jsonb_path_query.
Below code gives a list of counts, I will leave the rest to you, to get final data.
I've added ID column just to have something to join on. Otherwise join on metadata.
Also, note additional " in where condition. Left join in blob_ext is there just to have null value if metadata is not present or that path does not work.
with blob as (
select row_number() over()"id", * from (VALUES
(
'{
"total_count": 2,
"items": [
{
"name": "somename",
"metadata": {
"metas": [
{
"id": "11258",
"score": 6.1,
"status": "active",
"published_at": "2019-04-20T00:29:00",
"nvd_modified_at": "2022-04-06T18:07:00"
},
{
"id": "9251",
"score": 5.1,
"status": "active",
"published_at": "2018-01-18T23:29:00",
"nvd_modified_at": "2021-01-08T12:15:00"
}
]
}
}
]}'::jsonb),
(null::jsonb)) b(metadata)
)
, blob_ext as (
select bb.*, blob_sts.status
from blob bb
left join (
select
bb2.id,
jsonb_path_query (bb2.metadata::jsonb, '$.items[*].metadata.metas[*].status'::jsonpath)::character varying "status"
FROM blob bb2
) as blob_sts ON
blob_sts.id = bb.id
)
select bbe.id, count(*) cnt, bbe.metadata
from blob_ext bbe
where bbe.status in ('"active"', '"reported"')
group by bbe.id, bbe.metadata;
A way is to peel one layer at a time with jsonb_extract_path() and jsonb_array_elements():
with cte_items as (
select id,
metadata,
jsonb_extract_path(jx.value,'metadata','metas') as metas
from blobs,
lateral jsonb_array_elements(jsonb_extract_path(metadata,'items')) as jx),
cte_metas as (
select id,
metadata,
jsonb_extract_path_text(s.value,'status') as status
from cte_items,
lateral jsonb_array_elements(metas) s)
select distinct
id,
metadata
from cte_metas
where status in ('active','reported');
I am using PostgreSQL 9.4 and I have a requirement to have an 'addresses' array which contains closed JSON objects for different types of address (residential and correspondence). The structure should look like this:
[
{
"addresses": [
{
"addressLine1": "string",
"type": "residential"
},
{
"addressLine1": "string",
"type": "correspondence"
}
],
"lastName": "string"
}
]
...and here's some example data to illustrate the desired result:
[
{
"addresses": [
{
"addressLine1": "54 ASHFIELD PADDOCK",
"type": "residential"
},
{
"addressLine1": "135 MERRION HILL",
"type": "correspondence"
}
],
"lastName": "WRIGHT"
},
{
"addresses": [
{
"addressLine1": "13 BOAKES GROVE",
"type": "residential"
},
{
"addressLine1": "46 BEACONSFIELD GRANGE",
"type": "correspondence"
}
],
"lastName": "DOHERTY"
}
]
This is where I've gotten to with my SQL:
SELECT
json_agg(
json_build_object('addresses',(SELECT json_agg(json_build_object('addressLine1',c2.address_line_1,
'type',c2.address_type
)
)
FROM my_customer_table c2
WHERE c2.person_id=c.person_id
),
'addresses',(SELECT json_agg(json_build_object('addressLine1',c2.corr_address_line_1,
'type',c2.corr_address_type
)
)
FROM my_customer_table c2
WHERE c2.person_id=c.person_id
),
'lastName',c.surname
)
) AS customer_json
FROM
my_customer_table c
WHERE
c.corr_address_type IS NOT NULL /*exclude customers without correspondence addresses*/
...and this runs, however it repeats the 'addresses' object twice and has an array for each address variant, not around the overall array.
What I stupidly thought would work, is the following:
SELECT
json_agg(
json_build_object('addresses',(SELECT json_agg(json_build_object('addressLine1',c2.address_line_1,
'type',c2.address_type
),
json_build_object('addressLine1',c2.corr_address_line_1,
'type',c2.corr_address_type
)
)
FROM my_customer_table c2
WHERE c2.person_id=c.person_id
),
'lastName',c.surname
)
) AS customer_json
FROM
my_customer_table c
WHERE
c.corr_address_type IS NOT NULL /*exclude customers without correspondence addresses*/
...however this throws an error:
"ERROR: function json_agg(json, json) does not exist.
LINE 3: json_build_object('addresses',(SELECT json_agg(json_build_o...
HINT: No function matches the given name and argument types. You might need to add explicit type casts."
I've Googled this, however no posts found seem to relate to the same kind of result I'm trying to get.
Does anyone know if it's possible to have multiple JSON_BUILD_OBJECT entries inside of an array?
A colleague has found a solution to this. It's totally different to the way I was approaching it, but works nicely. Here's the working code:
SELECT
json_agg(subquery.customer_json) AS customer_json
FROM
(
SELECT
row_to_json(t) AS customer_json
FROM (
SELECT
(
SELECT array_to_json(array_agg(addresses_union))
FROM (
SELECT
c.address_line_1 AS "addressLine1",
c.address_type as type
UNION ALL
SELECT
c.corr_address_Line_1 AS "addressLine1",
c.corr_address_type as type
) AS addresses_union
) as addresses,
c.surname AS "lastName"
FROM
my_customer_table c
WHERE
c.corr_address_type IS NOT NULL /*exclude customers without correspondence address*/
) t
) subquery
I have a jsonb column with the following structure:
{
"key1": {
"type": "...",
"label": "...",
"variables": [
{
"label": "Height",
"value": 131315.9289,
"variable": "myVar1"
},
{
"label": "Width",
"value": 61085.7525,
"variable": "myVar2"
}
]
},
}
I want to query for the average height across all rows. The top-level key values are unknown, so I have something like this:
select id,
avg((latVars ->> 'value')::numeric) as avg
from "MyTable",
jsonb_array_elements((my_json_field->jsonb_object_keys(my_json_field)->>'variables')::jsonb) as latVars
where my_json_field is not null
group by id;
It's throwing the following error:
ERROR: set-returning functions must appear at top level of FROM
Moving the jsonb_array_elements function above MyTable in the FROM clause doesn't work.
I'm following the basic advice found in this SO answer to no avail.
Any advice?
jsonb_array_elements is not relevant until my_json_field is a json array at the top level.
You can use instead the jsonb_path_query function based on the jsonpath language if postgres >= 12 :
select id
, avg(v.value :: numeric) as avg
from "MyTable"
, jsonb_path_query(my_json_field, '$.*.variables[*] ? (#.label == "Height").value') AS v(value)
where my_json_field is not null
group by id;
I have the following JSON document stored in a PostgreSQL JSON column:
{
"status": "Success",
"message": "",
"data": {
"serverIp": "XXXX",
"ruleId": 32321,
"results": [
{
"versionId": 555555,
"PriceID": "8abf35ec-3e0e-466b-a4e5-2af568e90eec",
"price": 350,
"Convert": 0.8,
"Cost": 15
"Customer_ID":1
},
{
"versionId": 4444,
"PriceID": "b5a1dbd5-17b4-4847-8b3c-da334f95276a",
"price": 550,
"Convert": 0.7,
"Cost": 10,
"Customer_ID":10
}
]
}
}
I am trying to retrieve the price for specific customer_ID
I am using this query to get the price for Customer_ID=1
select json_array_elements(t.info -> 'data' -> 'results') ->> 'price'
from mytable t
where exists (
select
from json_array_elements(t.info -> 'data' -> 'results') x(elt)
where (x.elt ->> 'Customer_ID')::int = 1
)
The problem is that i am getting the same results for Customer_ID=1 and for Customer_ID=10
I am basically getting both elements of the array instead of just one.
I don't know what i am doing wrong
You can use a lateral join to unnest the array elements, and then filter on the customer with a where clause; this leaves you with just one row, from which you can extract the price:
select x.elt ->> 'price' price
from mytable t
cross join lateral json_array_elements(t.info -> 'data' -> 'results') x(elt)
where x.elt ->> 'Customer_ID' = 1
Note that you don't need to cast the customer id to an int, as it is already stored in the proper format in the json array.
I'm trying to merge some nested JSON arrays without looking at the id. Currently I'm getting this when I make a GET request to /surveyresponses:
{
"surveys": [
{
"id": 1,
"name": "survey 1",
"isGuest": true,
"house_id": 1
},
{
"id": 2,
"name": "survey 2",
"isGuest": false,
"house_id": 1
},
{
"id": 3,
"name": "survey 3",
"isGuest": true,
"house_id": 2
}
],
"responses": [
{
"question": "what is this anyways?",
"answer": "test 1"
},
{
"question": "why?",
"answer": "test 2"
},
{
"question": "testy?",
"answer": "test 3"
}
]
}
But I would like to get it where each survey has its own question and answers so something like this:
{
"surveys": [
{
"id": 1,
"name": "survey 1",
"isGuest": true,
"house_id": 1
"question": "what is this anyways?",
"answer": "test 1"
}
]
}
Because I'm not going to a specific id I'm not sure how to make the relationship work. This is the current query I have that's producing those results.
export function getSurveyResponse(id: number): QueryBuilder {
return db('surveys')
.join('questions', 'questions.survey_id', '=', 'surveys.id')
.join('questionAnswers', 'questionAnswers.question_id', '=', 'questions.id')
.select('surveys.name', 'questions.question', 'questions.question', 'questionAnswers.answer')
.where({ survey_id: id, question_id: id })
}
Assuming jsonb in current Postgres 10 or 11, this query does the job:
SELECT t.data, to_jsonb(s) AS new_data
FROM t
LEFT JOIN LATERAL (
SELECT jsonb_agg(s || r) AS surveys
FROM (
SELECT jsonb_array_elements(t.data->'surveys') s
, jsonb_array_elements(t.data->'responses') r
) sub
) s ON true;
db<>fiddle here
I unnest both nested JSON arrays in parallel to get the desired behavior of "zipping" both directly. The number of elements in both nested JSON arrays has to match or you need to do more (else you lose data).
This builds on implementation details of how Postgres deals with multiple set-returning functions in a SELECT list to make it short and fast. See:
What is the expected behaviour for multiple set-returning functions in select clause?
One could be more explicit with a ROWS FROM expression, which works properly since Postgres 9.4:
SELECT t.data
, to_jsonb(s) AS new_data
FROM tbl t
LEFT JOIN LATERAL (
SELECT jsonb_agg(s || r) AS surveys
FROM ROWS FROM (jsonb_array_elements(t.data->'surveys')
, jsonb_array_elements(t.data->'responses')) sub(s,r)
) s ON true;
The manual about combining multiple table functions.
Or you could use WITH ORDINALITY to get original order of elements and combine as you wish:
PostgreSQL unnest() with element number