I am pretty new to Snowflake and I am now trying to parse a JSON field and pull its attributes to return in the response.
I tried a few variations but every time, the attribute is populating as null.
attributes column in my table has this JSON:
{
"Status": [
"ACTIVE"
],
"Coverence": [
{
"Sub": [
{
"EndDate": [
"2020-06-22"
],
"Source": [
"Test"
],
"Id": [
"CovId1"
],
"Type": [
"CovType1"
],
"StartDate": [
"2019-06-22"
],
"Status": [
"ACTIVE"
]
}
]
}
]
}
What I tried:
SELECT DISTINCT *
from
(
TRIM(mt."attributes":Status, '[""]')::string as STATUS,
TRIM(r.value:"Sub"."Id", '[""]')::string as ID,
TRIM(r.value:"Sub"."Source", '[""]')::string as SOURCE
from "myTable" mt,
lateral flatten ( input => mt."attributes":"Coverence", outer => true) r
)
GROUP BY
STATUS,
ID,
SOURCE;
Later I tried:
SELECT DISTINCT *
from
(
TRIM(mt."attributes":Status, '[""]')::string as STATUS,
TRIM(r.value:"Id", '[""]')::string as ID,
TRIM(r.value:"Source", '[""]')::string as SOURCE
from "myTable" mt,
lateral flatten ( input => mt."attributes":"Coverence":"Sub", outer => true) r
)
GROUP BY
STATUS,
ID,
SOURCE;
But nothing worked. The STATUS is populating as expected. But ID and SOURCE are populating null.
Am I missing something or have I done something dumb? Please shed some light.
Assuming that Coverence could contain multiple Sub, therefore FLATTEN twice. At lowest level only first element is chosen (EndDate[0], Source[0] etc):
SELECT
mt."attributes":Status[0]::TEXT AS Status
,r2.value:EndDate[0]::TEXT AS EndDate
,r2.value:Source[0]::TEXT AS Source
,r2.value:Id[0]::TEXT AS Id
FROM myTable AS mt,
LATERAL FLATTEN(input => mt."attributes",
path => 'Coverence',
outer => true) r1,
LATERAL FLATTEN(input => r1.value,
path => 'Sub',
outer => true) r2;
Output:
All your elements are array type and the overall JSON is not making much sense... so to access all your individual elements, you have to use [] notation and then you can access the element values. You don't need to use flatten also, if you just have to access individual elements via index.
Related
I done
let statuses = await t.any(`SELECT DISTINCT status FROM mails`)
and got
"statuses": [
{
"status": "error"
},
{
"status": "success"
}
]
How can I get array with keys of objects ? ['error', 'success'] ?
Assuming status is a jsonb column (which it should be), you can do:
select distinct st.status
from mails m
cross join jsonb_array_elements(m.status -> 'statuses') as st(status)
If status is a json column you will need to use json_array_elements() instead
I am running an SQL query to extract nested JSON data.
SELECT watsonResponse.responseId,
watsonResponse.chatId,
d.*
FROM watson_response_table watsonResponse
CROSS JOIN LATERAL (
SELECT d2.*
FROM jsonb_array_elements(watsonResponse.watsonresponse_output) AS d(events)
CROSS JOIN LATERAL (
SELECT d2.events ->> 'data' AS watsonResponse_ouput_data
, d2.events ->> 'text' AS watsonResponse_output_text
, d2.events ->> 'uiActionCode' AS watsonResponse_output_uiActionCode
FROM jsonb_array_elements(d.events) AS d2(events)
) d2
WHERE d.events ->> 'uiActionCode' = 'TextWithButton'
) d;
It fails with message SQL Error [22023]: ERROR: cannot extract elements from an object
I am using PostgresSQL 11+. Here is what the JSON looks like,
[
{
"text": [
"Some text!"
],
"uiActionCode": "textOnly"
},
{
"data": {
"type": "options",
"options": [
{ "label": "test", "value": "testvalue" },
{ "label": "test2", "value": "testvalue2" },
{
"label": "test3",
"value": "testQuestion?"
}
]
},
"text": ["testQuestion2?"],
"uiActionCode": "TextWithButton"
}
]
If I am following this correctly, one level of unnesting is sufficient. You can then use the JSON accessors to get the results you want:
SELECT
r.responseId,
r.chatId,
d.events ->> 'uiActionCode' AS output_uiActionCode,
d.events -> 'text' ->> 0 AS output_text,
d.events -> 'data' AS output_data,
FROM watson_response_table watsonResponse r
CROSS JOIN LATERAL jsonb_array_elements(r.watsonresponse_output) AS d(events)
WHERE d.events ->> 'uiActionCode' = 'TextWithButton'
Note that there is an important difference between accessors -> and ->>. The former returns an object, while the latter returns a text value. You need to carefully pick the correct operator according to what needs to be done for each field.
PosgreSQL 9.5
Field type: jsonb
Here json
{
"options": [
{
"name": "method"
},
{
"name": "flavor"
},
{
"name": "weight",
"value": {
"name": "300g"
}
}
]
}
And here query that get value of item (weight) with index = 2 from array:
SELECT
id,
product.data #>'{title,en}' AS title_en,
product.data #>>'{options, 2, value, name }' as options_weight_value
FROM product
Nice. It's work fine.
But the problem that weight can be in any index in array. First or second and so on.
So I need to get value of name (300g) in node "weight" .
I need smt like this:
SELECT
id,
product.data #>'{title,en}' AS title_en,
product.data #>>'{options, *, value, name, weight }' as options_weight_value
FROM product
Is it possible ?
I think I found solution:
SELECT
id,
p.data #>'{title,en}' AS title_en,
p.data #>'{weight,qty}' AS weight_qty,
(select *
from jsonb_array_elements(p.data -> 'options') AS options_array
where
options_array ->> 'name' = 'weight'
) #>'{value,name}' as options_weight
from product p
And now find value of weight(if exist) in any array's item. In this example it = 300g
I'm new with jsonb request and i got a problem. Inside an 'Items' table, I have 'id' and 'data' jsonb. Here is what can look like a data:
[
{
"paramId": 3,
"value": "dog"
},
{
"paramId": 4,
"value": "cat"
},
{
"paramId": 5,
"value": "fish"
},
{
"paramId": 6,
"value": "",
"fields": [
{
"paramId": 3,
"value": "cat"
},
{
"paramId": 4,
"value": "dog"
}
]
},
{
"paramId": 6,
"value": "",
"fields": [
{
"paramId": 5,
"value": "cat"
},
{
"paramId": 3,
"value": "dog"
}
]
}
]
The value in data is always an array with object inside but sometimes the object can have a 'fields' value with objects inside. It is maximum one level deep.
How can I select the id of the items which as for example an object containing "paramId": 3 and "value": "cat" and also have an object with "paramId": 5 and "value" LIKE '%ish%'.
I already have found a way to do that when the object is on level 0
SELECT i.*
FROM items i
JOIN LATERAL jsonb_array_elements(i.data) obj3(val) ON obj.val->>'paramId' = '3'
JOIN LATERAL jsonb_array_elements(i.data) obj5(val) ON obj2.val->>'paramId' = '5'
WHERE obj3.val->>'valeur' = 'cat'
AND obj5.val->>'valeur' LIKE '%ish%';
but I don't know how to search inside the fields array if fields exists.
Thank you in advance for you help.
EDIT:
It looks like my question is not clear. I will try to make it better.
What I want to do is to find all the 'item' having in the 'data' column objects who match my search criteria. This without looking if the objects are at first level or inside a 'fields' key of an object.
Again for example. This record should be selected if I search:
'paramId': 3 AND 'value': 'cat
'paramId': 4 AND 'value': LIKE '%og%'
the matching ones are in the 'fields' key of the object with 'paramId': 6 and I don't know how to do that.
This can be expressed using a JSON/Path expression without the need for unnesting everything
To search for paramId = 3 and value = 'cat'
select *
from items
where data #? '$[*] ? ( (#.paramId == 3 && #.value == "cat") || exists( #.fields[*] ? (#.paramId == 3 && #.value == "cat")) )'
The $[*] part iterates over all elements of the first level array. To check the elements in the fields array, the exists() operator is used to nest the expression. #.fields[*] iterates over all elements in the fields array and applies the same expression again. I don't see a way how repeating the values could be avoided though.
For a "like" condition, you can use like_regex:
select *
from items
where data #? '$[*] ? ( (#.paramId == 4 && #.value like_regex ".*og.*") || exists( #.fields[*] ? (#.paramId == 4 && #.value like_regex ".*og.*")) )'
For now I have found a solution but it is not really clean and I don't know how it will perform in production with 10M records.
SELECT i.id, i.data
FROM ( -- A;
select it.id, it.data, i as value
from items it,
jsonb_array_elements(it.data) i
union
select it.id, it.data, f as value
from items it,
jsonb_array_elements(it.data) i,
jsonb_array_elements(i -> 'fields') f
) as i
WHERE (i.value ->> 'paramId' = '5' -- B1;
AND i.value ->> 'value' LIKE '%ish%')
OR (i.value ->> 'paramId' = '3' -- B2;
AND i.value ->> 'value' = 'cat')
group by i.id, i.data
having COUNT(*) >= 2; -- C;
A: I "flatten" the first and second level (second level is in 'fields' key)
B1, B2: These are my search criteria
C: I make sure the fields have all the criteria matching. If 3 criteria --> COUNT(*) >=3
It really doesn't look clean to me. It is working for dev purpose but I think there is a better way to do it.
If somebody have an idea Big thanks to him/her!
My JSON data looks like this:
[{
"id": 1,
"payload": {
"location": "NY",
"details": [{
"name": "cafe",
"cuisine": "mexican"
},
{
"name": "foody",
"cuisine": "italian"
}
]
}
}, {
"id": 2,
"payload": {
"location": "NY",
"details": [{
"name": "mbar",
"cuisine": "mexican"
},
{
"name": "fdy",
"cuisine": "italian"
}
]
}
}]
given a text "foo" I want to return all the tuples that have this substring. But I cannot figure out how to write the query for the same.
I followed this related answer but cannot figure out how to do LIKE.
This is what I have working right now:
SELECT r.res->>'name' AS feature_name, d.details::text
FROM restaurants r
, LATERAL (SELECT ARRAY (
SELECT * FROM json_populate_recordset(null::foo, r.res#>'{payload,
details}')
)
) AS d(details)
WHERE d.details #> '{cafe}';
Instead of passing the whole text of cafe I want to pass ca and get the results that match that text.
Your solution can be simplified some more:
SELECT r.res->>'name' AS feature_name, d.name AS detail_name
FROM restaurants r
, jsonb_populate_recordset(null::foo, r.res #> '{payload, details}') d
WHERE d.name LIKE '%oh%';
Or simpler, yet, with jsonb_array_elements() since you don't actually need the row type (foo) at all in this example:
SELECT r.res->>'name' AS feature_name, d->>'name' AS detail_name
FROM restaurants r
, jsonb_array_elements(r.res #> '{payload, details}') d
WHERE d->>'name' LIKE '%oh%';
db<>fiddle here
But that's not what you asked exactly:
I want to return all the tuples that have this substring.
You are returning all JSON array elements (0-n per base table row), where one particular key ('{payload,details,*,name}') matches (case-sensitively).
And your original question had a nested JSON array on top of this. You removed the outer array for this solution - I did the same.
Depending on your actual requirements the new text search capability of Postgres 10 might be useful.
I ended up doing this(inspired by this answer - jsonb query with nested objects in an array)
SELECT r.res->>'name' AS feature_name, d.details::text
FROM restaurants r
, LATERAL (
SELECT * FROM json_populate_recordset(null::foo, r.res#>'{payload, details}')
) AS d(details)
WHERE d.details LIKE '%oh%';
Fiddle here - http://sqlfiddle.com/#!15/f2027/5