Oracle Select From JSON Array - sql

I have a table for some 'settings' and in that table I have a record with a json array. It is a simple array, like this:
"['scenario1','scenario2','scenario3']"
I want to use a sub-select statement in a view to pull this information out so I can use it like this:
select * from table where field_scenario in (select ????? from settings_table where this=that)
I have been looking through documentation and googling for this but for the life of me I can't figure out how to 'pivot' the returning array into individual elements in order to use it.
Oracle 12c I believe, thanks in advance.

Do NOT use regular expression to parse JSON. Use a proper JSON parser:
select *
from table_name
where field_scenario in (
SELECT j.value
FROM settings_table s
OUTER APPLY (
SELECT value
FROM JSON_TABLE(
s.json,
'$[*]'
COLUMNS(
value VARCHAR2(50) PATH '$'
)
)
) j
)
Which, for the sample data:
CREATE TABLE settings_table ( json CLOB CHECK ( json IS JSON ) );
INSERT INTO settings_table ( json ) VALUES ( '["scenario1","scenario2","scenario3"]');
INSERT INTO settings_table ( json ) VALUES ( '["scenario5"]');
INSERT INTO settings_table ( json ) VALUES ( '["scenario \"quoted\""]');
INSERT INTO settings_table ( json ) VALUES ( '["scenario2,scenario4"]');
CREATE TABLE table_name ( id, field_scenario ) AS
SELECT LEVEL, 'scenario'||LEVEL FROM DUAL CONNECT BY LEVEL <= 6 UNION ALL
SELECT 7, 'scenario "quoted"' FROM DUAL;
Outputs:
ID | FIELD_SCENARIO
-: | :----------------
1 | scenario1
2 | scenario2
3 | scenario3
5 | scenario5
7 | scenario "quoted"
db<>fiddle here

Related

Json - Flatten Key and Values in Hive

In a Hive table having a record JSON column value as: {"XXX": ["123","456"],"YYY": ["246","135"]} and ID as ABC
Need to flatten it as below in Hive query.
Key
Value
ID
XXX
123
ABC
XXX
456
ABC
YYY
246
ABC
YYY
135
ABC
The following uses get_json_object to extract json keys before using regexp_replace and split to convert the remaining values to arrays. With the assistance of explode and lateral views from the resulting subquery, the data has been extracted. The full reproducible example is below:
WITH input_df AS (
SELECT '{"XXX": ["123","456"],"YYY": ["246","135"]}' my_col
)
SELECT
t.key,
kv.kval as value
FROM (
SELECT
explode(map(
'XXX',
split(regexp_replace(get_json_object(my_col,'$.XXX'),'"|\\[|\\]',''),','),
'YYY',
split(regexp_replace(get_json_object(my_col,'$.YYY'),'"|\\[|\\]',''),',')
))
FROM
input_df
) t LATERAL VIEW explode(t.value) kv as kval
You may use the query below if your table/view is named input_df and your json column is my_col
SELECT
t.key,
kv.kval as value
FROM (
SELECT
explode(map(
'XXX',
split(regexp_replace(get_json_object(my_col,'$.XXX'),'"|\\[|\\]',''),','),
'YYY',
split(regexp_replace(get_json_object(my_col,'$.YYY'),'"|\\[|\\]',''),',')
))
FROM
input_df
) t LATERAL VIEW explode(t.value) kv as kval
Response To Updated Question 1:
SELECT
t.key,
kv.kval as value,
'ABC' as ID
FROM (
SELECT
explode(map(
'XXX',
split(regexp_replace(get_json_object(my_col,'$.XXX'),'"|\\[|\\]',''),','),
'YYY',
split(regexp_replace(get_json_object(my_col,'$.YYY'),'"|\\[|\\]',''),',')
))
FROM
input_df
) t LATERAL VIEW explode(t.value) kv as kval
Let me know if this works for you.

Query json dictionary data in SQL

My CLOB field in a table contains JSON and looks as following:
{"a":"value1", "b":"value2", "c":"value3"}
And I'm trying to write an SQL query to return a table with key and value fields like following:
key|value
---|------
a |value1
b |value2
c |value3
Any help would be hugely appreciated!
Use JSON_TABLE and then UNPIVOT if you want the values in rows instead of columns:
SELECT *
FROM (
SELECT p.*
FROM table_name t
CROSS JOIN
JSON_TABLE(
t.value,
'$'
COLUMNS (
a PATH '$.a',
b PATH '$.b',
c PATH '$.c'
)
) p
)
UNPIVOT ( value FOR key IN ( a, b, c ) );
So for some sample data:
CREATE TABLE table_name (
value CLOB CONSTRAINT ensure_json CHECK (value IS JSON)
);
INSERT INTO table_name ( value ) VALUES ( '{"a":"value1", "b":"value2", "c":"value3"}' );
This outputs:
KEY | VALUE
:-- | :-----
A | value1
B | value2
C | value3
db<>fiddle here
If you want to do it dynamically then you can parse the JSON in PL/SQL and use GET_KEYS to get a collection of key names and then access the correct one by its position and correlate that to the value using FOR ORDINALITY:
CREATE FUNCTION get_key(
pos IN PLS_INTEGER,
json IN CLOB
) RETURN VARCHAR2
AS
doc_keys JSON_KEY_LIST;
BEGIN
doc_keys := JSON_OBJECT_T.PARSE ( json ).GET_KEYS;
RETURN doc_keys( pos );
END get_key;
/
Then:
SELECT get_key( j.pos, t.value ) AS key,
j.value
FROM table_name t
CROSS APPLY JSON_TABLE(
t.value,
'$.*'
COLUMNS (
pos FOR ORDINALITY,
value PATH '$'
)
) j;
Outputs:
KEY | VALUE
:-- | :-----
a | value1
b | value2
c | value3
db<>fiddle here

How to use REGEXP_SUBSTR properly?

Currently in my select statement I have id and value. The value is json which looks like this:
{"layerId":"nameOfLayer","layerParams":{some unnecessary data}
I would like to have in my select id and nameOfLayer so the output would be for example:
1, layerName
2, layerName2
etc.
The json looks always the same so the layerID is the first.
Could you tell me how can I use REGEXP_SUBSTR properly in my select query which looks like this now?
select
id,
value
from
...
where
table1.id = table2.bookmark_id
and ...;
In Oracle 11g, you can extract the layerId using the following regular expression, where js is the name of your JSON column:
regexp_replace(js, '^.*"layerId":"([^"]+).*$', '\1')
This basically extracts the string between double quotes after "layerId":.
In more recent versions, you would add a check constraint on the table to ensure that the document is valid JSON, and then use the dot notation to access the object attribute as follows:
create table mytable (
id int primary key,
js varchar2(200),
constraint ensure_js_is_json check (js is json)
);
insert into mytable values (1, '{"layerId":"nameOfLayer","layerParams":{} }');
select id, t.js.layerId from mytable t;
Demo on DB Fiddle:
ID | LAYERID
-: | :----------
1 | nameOfLayer
Don't use regular expressions; use a JSON_TABLE or JSON_VALUE to parse JSON:
Oracle 18c Setup:
CREATE TABLE test_data (
id INTEGER,
value VARCHAR2(4000)
);
INSERT INTO test_data ( id, value )
SELECT 1, '{"layerId":"nameOfLayer","layerParams":{"some":"unnecessary data"}}' FROM DUAL UNION ALL
SELECT 2, '{"layerParams":{"layerId":"NOT THIS ONE!"},"layerId":"nameOfLayer"}' FROM DUAL UNION ALL
SELECT 3, '{"layerId":"Name with \"Quotes\"","layerParams":{"layerId":"NOT THIS ONE!"}}' FROM DUAL;
Query 1:
SELECT t.id,
j.layerId
FROM test_data t
CROSS JOIN
JSON_TABLE(
t.value,
'$'
COLUMNS (
layerId VARCHAR2(50) PATH '$.layerId'
)
) j
Query 2:
If you only want a single value you could, alternatively, use JSON_VALUE:
SELECT id,
JSON_VALUE( value, '$.layerId' ) AS layerId
FROM test_data
Output:
Both output:
ID | LAYERID
-: | :-----------------
1 | nameOfLayer
2 | nameOfLayer
3 | Name with "Quotes"
Query 3:
You can try regular expressions but they do not always work as expected:
SELECT id,
REPLACE(
REGEXP_SUBSTR( value, '[{,]"layerId":"((\\"|[^"])*)"', 1, 1, NULL, 1 ),
'\"',
'"'
) AS layerID
FROM test_data
Output:
ID | LAYERID
-: | :-----------------
1 | nameOfLayer
2 | NOT THIS ONE!
3 | Name with "Quotes"
So if you can guarantee that no-one is going to put data into the database where the JSON is in a different order then this may work; however the JSON specification allows key-value pairs to be in any order so regular expressions are not a general solution that will parse every JSON string. You should be using a proper JSON parser and there are 3rd party solutions available for Oracle 11g or you can upgrade to Oracle 12c where there is a native solution.
db<>fiddle here
I think you can use regexp_substr like this:
regexp_substr(str, '[^"]+',1,2) as layer_id,
regexp_substr(str, '[^"]+',1,4) as layername
Db<>fiddle demo
Cheers!!

Unexpected behavior in Oracle when using Group By with JSON_TABLE

I have a denormalized VIEW we'll call VIEW_X which looks like the following (It's just a regular simple view - not materialized or anything like that):
ID GROUP_ID PART_1_ID PART_2_ID
1 1723189 cd69f0f4-a5ed-4196-916d-401e98ffec75 X1
1 1723189 cd69f0f4-a5ed-4196-916d-401e98ffec75 X2
2 1723185 8d5132cb-1b6e-4e79-9698-fd1962eb808f K1
2 1723188 a191cb01-32ac-4ab4-bd6b-3ef777e395ca K1
It's denormalized in that it actually represents a structure like this:
{
id: 1,
group_id: 1723189,
part_1_id: 'cd69f0f4-a5ed-4196-916d-401e98ffec75'
part_2_ids: ["X1", "X2"]
}
the PART_2_ID in this view is the result of selecting from a JSON_TABLE where the data in the original table is stored in an array like ["X1", "X2"]:
JSON_TABLE(a.PART_2_IDS, '$' COLUMNS (
NESTED PATH '$[*]'
COLUMNS (
PART_2_ID VARCHAR2(4000) PATH '$'
)
)) p2
When I run a query like this on this view I get 0 results although the expected result is a single result with the ID of 2:
SELECT ID
FROM VIEW_X
WHERE PART_2_ID IN ('K1')
GROUP BY ID
HAVING COUNT(DISTINCT(PART_2_ID)) = 1
ID
--
(no results)
figure 1
Curiously enough if I run just the following I get the expected two results as there are two rows with ID 2 where there is a match on PART_2_ID as K1:
SELECT ID
FROM VIEW_X
WHERE PART_2_ID IN ('K1')
ID
--
2
2
If, however, I run either of the following queries I get a match on ID 1:
SELECT ID
FROM VIEW_X
WHERE PART_2_ID IN ('X1')
GROUP BY ID
HAVING COUNT(DISTINCT(PART_2_ID)) = 1
ID
--
1
SELECT ID
FROM VIEW_X
WHERE PART_2_ID IN ('X1', 'X2')
GROUP BY ID
HAVING COUNT(DISTINCT(PART_2_ID)) = 2
ID
--
1
I don't understand why figure 1 is not returning the expected result - is there something I'm overlooking? Is this a quirk with how JSON_TABLE works?
I cannot replicate this in:
Oracle Database 12c Enterprise Edition Release 12.1.0.2.0 - 64bit Production; or
Oracle Database 18c Enterprise Edition Release 18.0.0.0.0 - Production on https://livesql.oracle.com
Oracle Setup:
CREATE TABLE table1 ( document CLOB CONSTRAINT ensure_json CHECK (document IS JSON) );
INSERT INTO table1 ( document ) VALUES ( '{"id":1,"group_id":1723189,"part_1_id":"cd69f0f4-a5ed-4196-916d-401e98ffec75","part_2_ids":["X1","X2"]}' );
INSERT INTO table1 ( document ) VALUES ( '{"id":2,"group_id":1723185,"part_1_id":"8d5132cb-1b6e-4e79-9698-fd1962eb808f","part_2_ids":["K1"]}' );
INSERT INTO table1 ( document ) VALUES ( '{"id":2,"group_id":1723188,"part_1_id":"a191cb01-32ac-4ab4-bd6b-3ef777e395ca","part_2_ids":["K1"]}' );
CREATE VIEW VIEW_X AS
SELECT p.*
FROM table1 t
CROSS JOIN
JSON_TABLE(
t.document,
'$'
COLUMNS (
id PATH '$.id',
group_id PATH '$.group_id',
part_1_id PATH '$.part_1_id',
NESTED PATH '$.part_2_ids[*]'
COLUMNS (
PART_2_ID VARCHAR2(4000) PATH '$'
)
)
) p;
Query 1:
SELECT *
FROM VIEW_X;
Results:
ID GROUP_ID PART_1_ID PART_2_ID
---------- ---------- ------------------------------------ ---------
1 1723189 cd69f0f4-a5ed-4196-916d-401e98ffec75 X1
1 1723189 cd69f0f4-a5ed-4196-916d-401e98ffec75 X2
2 1723185 8d5132cb-1b6e-4e79-9698-fd1962eb808f K1
2 1723188 a191cb01-32ac-4ab4-bd6b-3ef777e395ca K1
Query 2:
SELECT ID
FROM VIEW_X
WHERE PART_2_ID IN ('K1')
GROUP BY ID
HAVING COUNT(DISTINCT(PART_2_ID)) = 1;
Results:
ID
--
2

how to give column values as xml element names in Oracle

I have two tables. I am trying to join them and prepare an XML by using SQL/XML (SQLX) in oracle. Here the problem is XMLELEMENT function is taking hardcoded values for element name, but I want to have those names as a column data. Is it possible?
create table PRODUCTEDIT
(
PRODUCTEDIT_NUM NUMBER(12) primary key,
API_NAME VARCHAR2(255)
);
create table PRODUCTEDITPARAMETER
(
PRODUCTEDIT_NUM NUMBER(12) not null,
PARAMETER_SEQ NUMBER(9) not null,
PARAMETER_VALUE VARCHAR2(4000),
CONSTRAINT fk_producteditparameter
FOREIGN KEY (PRODUCTEDIT_NUM )
REFERENCES PRODUCTEDIT(PRODUCTEDIT_NUM )
);
There are 2 records in first table.
PRODUCTEDIT_NUM Api_Name
1 ModifyProd
2 CreateProd
Records in second table:
PRODUCTEDIT_NUM PARAMETER_SEQ PARAMETER_VALUE
1 1 10
1 2 Data
1 3 1
1 4 Data1
1 5 1
2 1 11
2 2 Voice
2 3 1
Now I want to get XMLOUTPUT like below:
<?xml version='1.0'?>
<ModifyProd>
<1>10</1>
<2>Data</2>
<3>1</3>
<4>Data1</4>
<5>1</5>
</ModifyProd>
<CreateProd>
<1>11</1>
<2>Voice</2>
<3>1</3>
</CreateProd>
In above XML we have XMLELEMENT names (ModifyProd,CreateProd, 1,2 e.t.c) coming from table data.I am not able to achieve that by using SQLXML in oracle.
I tried below, but doesn't seem to be working. XMLELEMENT is not taking the value the way I am passing.
SELECT XMLROOT(
XMLELEMENT(d.api_name,
(SELECT XMLAGG(
XMLELEMENT(e.parameter_seq,e.parameter_value
)
)
FROM producteditparameter e
WHERE e.productedit_num = d.productedit_num
)
),version '1.0', standalone yes
)
FROM productedit d
I assume you are looking for this:
WITH t AS
(SELECT 'foo' AS ELEMENT_NAME, 'bar' AS ELEMENT_CONTENT FROM dual)
SELECT XMLELEMENT(EVALNAME ELEMENT_NAME, ELEMENT_CONTENT)
FROM t;
<foo>bar</foo>
Update based on additional input
Your result is not a well-formed XML. A XML document must have only one single root element. So, either you ask for several XML documents, then you can do this:
SELECT
XMLELEMENT(
EVALNAME Api_Name,
XMLAGG(XMLELEMENT(EVALNAME parameter_seq, e.parameter_value) ORDER BY parameter_seq)
) AS xml_result
FROM PRODUCTEDITPARAMETER e
JOIN PRODUCTEDIT d USING (productedit_num)
GROUP BY productedit_num, Api_Name;
<ModifyProd><1>10</1><2>Data</2><3>1</3><4>Data1</4><5>1</5></ModifyProd>
<CreateProd><1>11</1><2>Voice</2><3>1</3></CreateProd>
or if you need a single XML you have to enclose it by another element, e.g.
SELECT
XMLELEMENT("Products",
XMLAGG(
XMLELEMENT(
EVALNAME Api_Name,
XMLAGG(XMLELEMENT(EVALNAME parameter_seq, e.parameter_value) ORDER BY parameter_seq)
)
)
) AS xml_result
FROM PRODUCTEDITPARAMETER e
JOIN PRODUCTEDIT d USING (productedit_num)
GROUP BY productedit_num, Api_Name;
<Products><ModifyProd><1>10</1><2>Data</2><3>1</3><4>Data1</4><5>1</5></ModifyProd><CreateProd><1>11</1><2>Voice</2><3>1</3></CreateProd></Products>
Another solution if I understood what you want:
WITH t AS
( SELECT LEVEL col1, 'test' || LEVEL col2
FROM DUAL
CONNECT BY LEVEL < 10)
SELECT XMLSERIALIZE (DOCUMENT (xmltype (CURSOR (SELECT col1, col2 FROM t))) INDENT SIZE = 0)
FROM DUAL;
Then, you can use a XSD to transform the output as you want.
Hope it helps