I have a Nested JSON like
{
"Col1" : Val1,
"Col2" : [
"NestedCol1" : "Nested Value 1",
"NestedCol2" : "Nested Value 2",
"NestedCol3" : "Nested Value 3",
]
"Col3" : [
"NestedCol1" : "Nested Value 1",
"NestedCol2" : "Nested Value 2",
"NestedCol3" : "Nested Value 3",
]
}
Both Nested Columns will have same column names,
i want to select based on the parameter either NestedCol2 values or NestedCol3 Values.
want to create more like genric pl/sql fucntions where user can pass the column names , as in future more nested columns with same structure can exists too.
Related
I have JSONB data in a Postgres column like this:
{
"Id": "5c6d3210-1def-489b-badd-2bcc4a1cda28",
"Name": "Jane Doe",
"Tags": [
{
"Key": "Project",
"Value": "1004345"
}
]
}
How can I query data where Name contains "Jane" or "Tags.Key" contains "4345"?
I tried this but this only matches the exact "Key" value:
select * from documents where data->'Tags' #> '[{ "Value":"1004345"}]';
You can use a JSON path operator using like_regex
select *
from documents
where data ## '$.Tags[*].Value like_regex "4345"'
you can do this way
select *
from documents
where 'Tags' ->> 'Value' = '1004345';
Have a table with json column, this json is quite big, so I want to filter/select only speicific nested keys.
Json Example:
{
"title":{
"nested_1":"This is nested key 1",
"nested_2":"This is nested key 2",
"nested_3":"This is nested key 3",
"nested_4":"This is nested key 4",
"nested_5":"This is nested key 5"
},
"description":{
"nested_1":"This is nested key 1",
"nested_2":"This is nested key 2",
"nested_3":"This is nested key 3",
"nested_4":"This is nested key 4",
"nested_5":"This is nested key 5"
},
"meta":{
"nested_1":"This is nested key 1",
"nested_2":"This is nested key 2",
"nested_3":"This is nested key 3",
"nested_4":"This is nested key 4",
"nested_5":"This is nested key 5"
}
}
In example I want to select only nested_3 and nested_5 (But keep the json structure):
{
"title":{
"nested_3":"This is nested key 3",
"nested_5":"This is nested key 5"
},
"description":{
"nested_3":"This is nested key 3",
"nested_5":"This is nested key 5"
},
"meta":{
"nested_3":"This is nested key 3",
"nested_5":"This is nested key 5"
}
}
What I tried so far:
select
id,
(select json_object_agg(key, val) from (
values
('nested_3', (select json_object_agg(t.token, t.content->'nested_3') from json_each(json_col) as t(token, content))),
('nested_5', (select json_object_agg(t.token, t.content->'nested_5') from json_each(json_col) as t(token, content)))
) as foo(key, val)) as json_col
from my_table
This is working but gives the opposite result (logical, based on above query):
{
"nested_3": {
"title": "This is nested key 3",
"description": "This is nested key 3",
"meta": "This is nested key 3"
},
"nested_5": {
"title": "This is nested key 5",
"description": "This is nested key 5",
"meta": "This is nested key 5"
}
}
I've also tried to nest:
(select json_object_agg(t.token, json_object_agg(t.content->'nested_3', t.languages->'nested_5')) from json_each(json_col) as t(token, content))
But this gives me an error: aggregate function calls cannot be nested
There is a way to select only specific nested keys but preserve the json structure?
Postgres version: 12
With a declared instead of hard-coded list of keys, in ('nested_3', 'nested_5').
select id, (
select jsonb_object_agg(key, (
select jsonb_object_agg(key, value)
from jsonb_each(t.value)
where key in ('nested_3', 'nested_5')
)) json_filtered
from jsonb_each(jsonfield) t
) from the_table;
SQL Fiddle
Found a way to do this: by using json_build_object
So what happens in json_each on each loop we create new object with only needed keys/value, and after push it to our output using json_object_agg if the key already exists will push to previous object otherwise will create new key.
select
id,
(
select
json_object_agg(t.token, json_build_object('nested_3', t.content->'nested_3', 'nested_5', t.content->'nested_5'))
from json_each(json_col) as t(token, content)
) as json_col
from my_table
If someone has better solution or maybe better description explanation, please post your answer.
{
"description": "test",
"id": "1",
"name": "test",
"prod": [
{
"id": "1",
"name": "name",
"re": [
{
"name": "name1",
"value": "1"
},
{
"name": "name2",
"value": "1"
},
{
"name": "name3",
"value": "0"
},
{
"name": "name4",
"value": "0"
}
]
}
]
}
Here is the best I can do with your JSON input and your sample output.
Note that your document has a unique "id" and "name" ("1" and "test" in your example). Then it has an array named "productSpecificationRelationship". Each element of this array is an object with its own "id" - in the query, I show this id with the column name PSR_ID (PSR for Product Specification Relationship). Also, each object in this first-level array contains a sub-array (second level), with objects with "name" ("name" again!) and "value" keys. (This looks very much like an entity-attribute-value model - very poor practice.) In the intermediate step in my query (before pivoting), I call these RC_NAME and RC_VALUE (RC for Relationship Characteristic).
In your sample output you have more than one value in the ID and NAME columns. I don't see how that is possible; perhaps from unpacking more than one document? The JSON document you shared with us has "id" and "name" as top-level attributes.
In the output, I understand (or rather, assume, since I didn't understand too much from your question) that you should also include the PSR_ID - there is only one in your document, with value "10499", but in principle there may be more than one, and the output will have one row per such id.
Also, I assume the "name" values are limited to the four you mentioned (or, if there can be more, you are only interested in those four in the output).
With all that said, here is the query. Note that I called the table ES for simplicity. Also, you will see that I had to go to nested path twice (since your document includes an array of arrays, and I wanted to pick up the PSR_ID from the outer array and the tokens from the nested arrays).
TABLE SETUP
create table es (payloadentityspecification clob
check (payloadentityspecification is json) );
insert into es (payloadentityspecification) values (
'{
"description": "test",
"id": "1",
"name": "test",
"productSpecificationRelationship": [
{
"id": "10499",
"relationshipType": "channelRelation",
"relationshipCharacteristic": [
{
"name": "out_of_home",
"value": "1"
},
{
"name": "out_of_home_ios",
"value": "1"
},
{
"name": "out_of_home_android",
"value": "0"
},
{
"name": "out_of_home_web",
"value": "0"
}
]
}
]
}');
commit;
QUERY
with
prep (id, name, psr_id, rc_name, rc_value) as (
select id, name, psr_id, rc_name, rc_value
from es,
json_table(payloadentityspecification, '$'
columns (
id varchar2(10) path '$.id',
name varchar2(40) path '$.name',
nested path '$.productSpecificationRelationship[*]'
columns (
psr_id varchar2(10) path '$.id',
nested path '$.relationshipCharacteristic[*]'
columns (
rc_name varchar2(50) path '$.name',
rc_value varchar2(50) path '$.value'
)
)
)
)
)
select id, name, psr_id, ooh, ooh_android, ooh_ios, ooh_web
from prep
pivot ( min(case rc_value when '1' then 'TRUE'
when '0' then 'FALSE' else 'UNDEFINED' end)
for rc_name in ( 'out_of_home' as ooh,
'out_of_home_android' as ooh_android,
'out_of_home_ios' as ooh_ios,
'out_of_home_web' as ooh_web
)
)
;
OUTPUT
ID NAME PSR_ID OOH OOH_ANDROID OOH_IOS OOH_WEB
-- ---- ------ ----------- ----------- ----------- -----------
1 test 10499 TRUE FALSE TRUE FALSE
Conditional aggregation might be used in order to pivot the result set after extracting the values by using JSON_TABLE() and JSON_VALUE() functions such as
SELECT JSON_VALUE(payloadentityspecification, '$.name') AS channel_map_name,
MAX(CASE WHEN name = 'out_of_home' THEN
DECODE(value,1,'TRUE',0,'FALSE','UNDEFINED')
END) AS ooh,
MAX(CASE WHEN name = 'out_of_home_android' THEN
DECODE(value,1,'TRUE',0,'FALSE','UNDEFINED')
END) AS ooh_android,
MAX(CASE WHEN name = 'out_of_home_ios' THEN
DECODE(value,1,'TRUE',0,'FALSE','UNDEFINED')
END) AS ooh_ios,
MAX(CASE WHEN name = 'out_of_home_web' THEN
DECODE(value,1,'TRUE',0,'FALSE','UNDEFINED')
END) AS ooh_web
FROM EntitySpecification ES,
JSON_TABLE (payloadentityspecification, '$.productSpecificationRelationship[*]'
COLUMNS ( NESTED PATH '$.relationshipCharacteristic[*]'
COLUMNS (
description VARCHAR2(250) PATH '$.description',
name VARCHAR2(250) PATH '$.name',
value VARCHAR2(250) PATH '$.value'
)
)) jt
WHERE payloadentityspecification IS JSON
GROUP BY JSON_VALUE(payloadentityspecification, '$.name')
Demo
Hi how to create index on array field my sample doc is
{
"name": [ {
"family": "Smith",
"given": [
"Kam"
],
"prefix": [
"Mrs."
],
"use": "official"
},
{
"family": "Johns",
"given": [
"Kam"
],
"use": "maiden"
}
]
}
I want to write a search query (like) on family and given fields ...How to create a index and suggest query ..Im new to couchbase
This query that selects the customers with family name "Smith" and given name "Kam":
select * from customer
where any n in name satisfies n.family = 'Smith' and
any fn in n.given satisfies fn = 'Kam' end end
Note the use of a nested ANY clause because of the use of a nested array in the data.
You can then create an index on the family name like this:
CREATE INDEX customer_name ON customer
( DISTINCT ARRAY n.family FOR n IN name END)
The index gets used without any hints. You can see that it is being used by adding EXPLAIN to the beginning of the query. That will get you a query plan in JSON that includes an index scan operator.
You can learn more about array indexing here:
https://developer.couchbase.com/documentation/server/current/n1ql/n1ql-language-reference/indexing-arrays.html
ElasticSearch Version: 0.90.2
Here's the problem: I want to find documents in the index so that they:
match all query tokens across multiple fields
fields own analyzers are used
So if there are 4 documents:
{ "_id" : 1, "name" : "Joe Doe", "mark" : "1", "message" : "Message First" }
{ "_id" : 2, "name" : "Ann", "mark" : "3", "message" : "Yesterday Joe Doe got 1 for the message First"}
{ "_id" : 3, "name" : "Joe Doe", "mark" : "2", "message" : "Message Second" }
{ "_id" : 4, "name" : "Dan Spencer", "mark" : "2", "message" : "Message Third" }
And the query is "Joe First 1" it should find ids 1 and 2. I.e., it should find documents which contain all the tokens from search query, no matter in which fields they are (maybe all tokens are in one field, or maybe each token is in its own field).
One solution would be to use elasticsearch "_all" field functionality: that way it will merge all the fields I need (name, mark, message) into one and I'll be able to query it with something like
"match": {
"_all": {
"query": "Joe First 1",
"operator": "and"
}
}
But this way I can specify analyzer for the "_all" field only. And I need "name" and "message" fields to have different set of tokenizers/token filters (let's say name will have phonetic analyzer and message will have some stemming token filter).
Is there a way to do this?
Thanks to guys at elasticsearch group, here's the solution... pretty simple need to say :)
All I needed to do is to use query_string query http://www.elasticsearch.org/guide/reference/query-dsl/query-string-query/ with default_operator = AND and it will do the trick:
{
"query": {
"query_string": {
"fields": [
"name",
"mark",
"message"
],
"query": "Joe First 1",
"default_operator": "AND"
}
}
}
I think using a multi match query makes sense here. Something like:
"multi_match": {
"query": "Joe First 1",
"operator": "and"
"fields": [ "name", "message", "mark"]
}
As you say, you can set the analyzer (or search_analyzer/index_analyzer) to be used on the _all field. It seems to me that should indeed be your first step to achieve the query results you're looking for.
From http://jontai.me/blog/2012/10/lucene-scoring-and-elasticsearch-_all-field/, we have this tasty quote:
... the _all field copies the text from the other fields and analyzes
them again; it doesn’t copy the pre-analyzed tokens. You can set a
separate analyzer for the _all field.
Which I interpret to mean that you should set your _all analyzer(s) as well as individual field analyzer(s). The _all field won't re-analyze the individual field data; it will grab the original field contents.