I have a table say types, which had a JSON column, say location that looks like this:
{ "attribute":[
{
"type": "state",
"value": "CA"
},
{
"type": "distance",
"value": "200.00"
} ...
]
}
Each row in the table has the data, and all have the "type": "state" in it. I want to just extract the value of "type": "state" from every row in the table, and put it in a new column. I checked out several questions on SO, like:
Query for element of array in JSON column
Index for finding an element in a JSON array
Query for array elements inside JSON type
but could not get it working. I do not need to query on this. I need the value of this column. I apologize in advance if I missed something.
create table t(data json);
insert into t values('{"attribute":[{"type": "state","value": "CA"},{"type": "distance","value": "200.00"}]}'::json);
select elem->>'value' as state
from t, json_array_elements(t.data->'attribute') elem
where elem->>'type' = 'state';
| state |
| :---- |
| CA |
dbfiddle here
I mainly use Redshift where there is a built-in function to do this. So on the off-chance you're there, check it out.
redshift docs
It looks like Postgres has a similar function set:
https://www.postgresql.org/docs/current/static/functions-json.html
I think you'll need to chain three functions together to make this work.
SELECT
your_field::json->'attribute'->0->'value'
FROM
your_table
What I'm trying is a json extract by key name, followed by a json array extract by index (always the 1st, if your example is consistent with the full data), followed finally by another extract by key name.
Edit: got it working for your example
SELECT
'{ "attribute":[
{
"type": "state",
"value": "CA"
},
{
"type": "distance",
"value": "200.00"
}
]
}'::json->'attribute'->0->'value'
Returns "CA"
2nd edit: nested querying
#McNets is the right, better answer. But in this dive, I discovered you can nest queries in Postgres! How frickin' cool!
I stored the json as a text field in a dummy table and successfully ran this:
SELECT
(SELECT value FROM json_to_recordset(
my_column::json->'attribute') as x(type text, value text)
WHERE
type = 'state'
)
FROM dummy_table
Related
I have the below sample JSON:
{
"Id1": {
"name": "Item1.jpg",
"Status": "Approved"
},
"Id2": {
"name": "Item2.jpg",
"Status": "Approved"
}
}
and I am trying to get the following output:
_key name Status
Id1 Item1.jpg Approved
Id2 Item2.jpg Approved
Is there any way I can achieve this in Snowflake using SQL?
You should use Snowflake's VARIANT data type in any column holding JSON data. Let's break this down step by step:
create temporary table FOO(v variant); -- Temp table to hold the JSON. Often you'll see a variant column simply called "V"
-- Insert into the variant column. Parse the JSON because variants don't hold string types. They hold semi-structured types.
insert into FOO select parse_json('{"Id1": {"name": "Item1.jpg", "Status": "Approved"}, "Id2": {"name": "Item2.jpg", "Status": "Approved"}}');
-- See how it looks in its raw state
select * from FOO;
-- Flatten the top-level JSON. The flatten function breaks down the JSON into several usable columns
select * from foo, lateral flatten(input => (foo.v)) ;
-- Now traverse the JSON using the column name and : to get to the property you want. Cast to string using ::string.
-- If you must have exact case on your column names, you need to double quote them.
select KEY as "_key",
VALUE:name::string as "name",
VALUE:Status::string as "Status"
from FOO, lateral flatten(input => (FOO.V)) ;
I have a JSON object that's written in a weird way.
> {"custom": [ { "name": "addressIdNum", "valueNum": 12345678}, {
> "name": "cancelledDateAt", "valueAt": "2017-02-30T01:43:04.000Z" }] }
Not sure how to parse something like this. The keys are addressIdNum and cancelledDateAt and the values are 12345678 and 2017-02-30T01:43:04.000Z respectively.
How do I parse this using Snowflake SQL?
Thanks for all your help!
Best,
Preet Rajdeo
If your input is ALWAYS in this form (two elements in an array, with the same fields in the same element), you can combine PARSE_JSON function and the path access.
Just try this:
with input as (
select parse_json(
'{"custom": [ { "name": "addressIdNum", "valueNum": 12345678}, {"name": "cancelledDateAt", "valueAt": "2017-02-30T01:43:04.000Z" }] }')
as json)
select json:custom[0].valueNum::integer, json:custom[1].valueAt::timestamp from input;
----------------------------------+-----------------------------------+
JSON:CUSTOM[0].VALUENUM::INTEGER | JSON:CUSTOM[1].VALUEAT::TIMESTAMP |
----------------------------------+-----------------------------------+
12345678 | 2017-03-01 17:43:04 |
----------------------------------+-----------------------------------+
However, if the structure of your data might be different (e.g. elements in the array might be in a different order), it's probably best to write a JavaScript UDF in Snowflake to convert such messy data into something easier.
I have an issue where I have some JSON stored in my oracle database, and I need to extract values from it.
The problem is, there are some fields that are duplicated.
When I try this, it works as there is only one firstname key in the options array:
SELECT
JSON_VALUE('{"increment_id":"2500000043","item_id":"845768","options":[{"firstname":"Kevin"},{"lastname":"Test"}]}', '$.options.firstname') AS value
FROM DUAL;
Which returns 'Kevin'.
However, when there are two values for the firstname field:
SELECT JSON_VALUE('{"increment_id":"2500000043","item_id":"845768","options":[{"firstname":"Kevin"},{"firstname":"Okay"},{"lastname":"Test"}]}', '$.options.firstname') AS value
FROM DUAL;
It only returns NULL.
Is there any way to select the first occurence of 'firstname' in this context?
JSON_VALUE returns one SQL VALUE from the JSON data (or SQL NULL if the key does not exists).
If you have a collection of values (a JSON array) an you want one specific item of the array you use array subscripts (square brackets) like in JavaScript, for example [2] to select the third item. [0] selects the first item.
To get the first array item in your example you have to change the path expression from '$.options.firstname' to '$.options[0].firstname'
You can follow this query:-
SELECT JSON_VALUE('{
"increment_id": "2500000043",
"item_id": "845768",
"options": [
{
"firstname": "Kevin"
},
{
"firstname": "Okay"
},
{
"lastname": "Test"
}
]
}', '$.options[0].firstname') AS value
FROM DUAL;
I'm new to the postgreSQL(9.5) Json world. Looking for help writing this query. Take this simplified table as an example.
CREATE TABLE activity_log (uri varchar,
data jsonb );
Example of data inside of 'data' column.
"{"ListingInputFilterBean":{"searchItems": [], "listingStatus": "ACTIVE"}"
"{"ListingInputFilterBean":{"searchItems": [{"name": "Dachshund", "type": "BREED"}], "listingStatus": "ACTIVE"}}"
"{"ListingInputFilterBean":{"searchItems": [{"name": "Lab", "type": "BREED"}, {"name": "Black Lab", "type": "CST"}], "listingStatus": "ACTIVE"}}"
The 'data' column is used to log specific sets of data for each URI call. In this case the searchItems array contain the items used in the search. I'm looking to write a query that finds the most searched for 'breed'. I'd like to count the number of times each 'name' is used when type is 'BREED'.
My initial approach was to pull back each of the 'searchItems'. Turn those into a row set using jsonb_to_recordset, but I quickly got in over my head when reading the documentation (sorry, I'm a noob).
Any suggestions on how to write that SQL?
WITH log_activity(data) AS ( VALUES
('{"ListingInputFilterBean":{"searchItems": [], "listingStatus": "ACTIVE"}}'::JSONB),
('{"ListingInputFilterBean":{"searchItems": [{"name": "Dachshund", "type": "BREED"}], "listingStatus": "ACTIVE"}}'::JSONB),
('{"ListingInputFilterBean":{"searchItems": [{"name": "Lab", "type": "BREED"}, {"name": "Black Lab", "type": "CST"}], "listingStatus": "ACTIVE"}}'::JSONB)
)
SELECT search_item->>'name',count(search_item->>'name')
FROM
log_activity la,
jsonb_array_elements(la.data#>'{ListingInputFilterBean,searchItems}') as search_item
WHERE search_item->>'type' = 'BREED'
GROUP BY search_item;
Result:
name | count
-----------+-------
Lab | 1
Dachshund | 1
(2 rows)
Here you just need to iterate over the list of searchItems and group only those entries, which do match your criteria. Steps are the following:
Get jsonb array of searchItems with #> operator, it will get JSON object at specified path;
Iterate over the list of elements retrieved from step 1 with jsonb_array_elements(), function which expands a JSON array to a set of JSON values;
count() names where searchItems' type = BREED, you can get actual text value with ->> operator;
UPDATE
With jsonb_to_recordset() it looks shorter, but you need explicitly define search_item columns' types:
SELECT search_item.name ,count(search_item.name)
FROM
log_activity la,
jsonb_to_recordset(la.data#>'{ListingInputFilterBean,searchItems}') as search_item(name text,type text)
WHERE search_item.type = 'BREED'
GROUP BY search_item.name;
I have a table with JSON array data I'd like to search.
CREATE TABLE data (id SERIAL, json JSON);
INSERT INTO data (id, json)
VALUES (1, '[{"name": "Value A", "value": 10}]');
INSERT INTO data (id, json)
VALUES (2, '[{"name": "Value B1", "value": 5}, {"name": "Value B2", "value": 15}]');
As described in this answer, i created a function, which also allows to create an index on the array data (important).
CREATE OR REPLACE FUNCTION json_val_arr(_j json, _key text)
RETURNS text[] AS
$$
SELECT array_agg(elem->>_key)
FROM json_array_elements(_j) AS x(elem)
$$
LANGUAGE sql IMMUTABLE;
This works nicely if I want to find an entire value (eg. "Value B1"):
SELECT *
FROM data
WHERE '{"Value B1"}'::text[] <# (json_val_arr(json, 'name'));
Now my questions:
Is it possible to find values with a wildcard (eg. "Value*")? Something like the following (naive) approach:
...
WHERE '{"Value%"}'::text[] <# (json_val_arr(json, 'name'));
Is it possible to find numeric values with comparison operators (eg. >= 10)? Again, a naive and obviously wrong approach:
...
WHERE '{10}'::int[] >= (json_val_arr(json, 'value'));
I tried to create a new function returning int[] but that did not work.
I created a SQL Fiddle to illustrate my problem.
Or would it be better to use a different approach like the following working queries:
SELECT *
FROM data,
json_array_elements(json) jsondata
WHERE jsondata ->> 'name' LIKE 'Value%';
and
...
WHERE cast(jsondata ->> 'value' as integer) <= 10;
However, for these queries, I was not able to create any index that was actually picked up by the queries.
Also, I'd like to implement all this in Postgresql 9.4 with JSONB eventually, but I think for the above questions this should not be an issue.
Thank you very much!
I know its been a while but I was just chugging on something similar (using wild cards to query json datatypes) and thought I'd share what I found.
Firstly, this was a huge point in the right direction:
http://schinckel.net/2014/05/25/querying-json-in-postgres/
The take away is that your method of exploding the json element into something else (a record-set) is the way to go. It lets you query the json elements with normal postgres stuff.
In my case:
#Table:test
ID | jsonb_column
1 | {"name": "", "value": "reserved", "expires_in": 13732}
2 | {"name": "poop", "value": "{\"ns\":[\"Whaaat.\"]}", "expires_in": 4554}
3 | {"name": "dog", "value": "{\"ns\":[\"woof.\"]}", "expires_in": 4554}
Example Query
select * from test jsonb_to_recordset(x) where jsonb_column->>'name' like '%o%';
# => Returns
# 2 | {"name": "poop", "value": "{\"ns\":[\"Whaaat.\"]}", "expires_in": 4554}
And to answer your question about jsonb: It looks like jsonb is the better route MOST of the time. It has more methods and faster read (but slower write) times.
Sources:
http://www.postgresql.org/docs/9.4/static/functions-json.html
http://www.postgresql.org/docs/9.4/static/datatype-json.html
Happy hunting!