Append Multiple Objects into Existing Array Using JSON_MODIFY - sql

The problem was in appending new JSON array to the existing JSON array:
Suppose I have the following JSON Array
[{"id": 1, "data": "One"}, {"id": 2, "data": "Two"}]
How do I append [{"id": 3, "data": "Three"}, {"id": 4, "data": "Four"}] to it using JSON_MODIFY?
resulting value for updated column:
[{"id": 1, "data": "One"}, {"id": 2, "data": "Two"}, {"id": 3, "data": "Three"}, {"id": 4, "data": "Four"}]

I don't think, that you can merge two JSON arrays with one JSON_MODIFY() call, but the following statement (using JSON_MODIFY()) is a possible solution:
Statement:
DECLARE #json NVARCHAR(500)='[{"id": 1, "data": "One"}, {"id": 2, "data": "Two"}]'
DECLARE #new NVARCHAR(500)='[{"id": 3, "data": "Three"}, {"id": 4, "data": "Four"}]'
SELECT #json = JSON_MODIFY(
#json,
'append $',
JSON_QUERY([value])
)
FROM OPENJSON(#new)
SELECT #json
Result:
[{"id": 1, "data": "One"}, {"id": 2, "data": "Two"},{"id": 3, "data": "Three"},{"id": 4, "data": "Four"}]

You can use "JSON_MODIFY" function and append key to modify JSON object like below:
SQL-FIDDLE
It's for individual JSON array:
DECLARE #json1 NVARCHAR(500)='[{"id": 1, "data": "One"}, {"id": 2, "data": "Two"}]';
DECLARE #json2 NVARCHAR(500)='[{"id": 3, "data": "Three"}, {"id": 4, "data": "Four"}]';
SELECT t.id, t.[data]
FROM
(
SELECT * FROM OPENJSON(#json1) WITH(id int,[data] NVARCHAR(MAX))
UNION ALL
SELECT * FROM OPENJSON(#json2) WITH(id int,[data] NVARCHAR(MAX))
) t
FOR JSON PATH;
It's for individual JSON hash:
DECLARE #info NVARCHAR(500)='[{"id": 1, "data": "One"}, {"id": 2, "data": "Two"}]';
PRINT #info;
SET #info = JSON_MODIFY(#info, 'append $', JSON_QUERY('{"id": 3, "data": "Three"}'))
SET #info = JSON_MODIFY(#info, 'append $', JSON_QUERY('{"id": 4, "data": "Four"}'))
PRINT #info;

Workaround I found for my project
I have two tables t1 and t2 which are identical in structure. Table t1 keeps records of supplier certificates. Table t2 gets with API new certificates obtained by supplier. So Table t1 shall be updated with new certificates from Table t2. Certificates data are placed in JSON array of objects, similar to the example of the topic starter.
Task
JSON array in t1 col JSON_t1 shall be appended with JSON array from t2 col JSON_t2. Here's the structure simplified for the example purposes:
Table "t1"
recordId
JSON_t1
1
[{"id": 1, "data": "One"}, {"id": 2, "data": "Two"}]
Table "t2"
recordId
JSON_t2
1
[{"id": 3, "data": "Three"}, {"id": 4, "data": "Four"}]
RESULT
appended t1.JSON_t1
[{"id": 1, "data": "One"}, {"id": 2, "data": "Two"},{"id": 3, "data": "Three"}, {"id": 4, "data": "Four"}]
SQL method
SELECT
t1.JSON_t1,
t2.JSON_t2,
concat('[', replace(replace(json_modify(t1.JSON_t1, 'append $', json_query(t2.JSON_t2)), '[', ''), ']', ''), ']') as "appended t1.JSON_t1"
FROM t1
INNER JOIN t2 ON t1.recordId = t2.recordId
Method explained
JSON_t2 is converted to JSON format with json_query(t2.JSON_t2) to avoid escaping of characters
JSON_t1 is appended with JSON_t2 with json_modify(t1.JSON_t1, 'append $', json_query(t2.JSON_t2)) resulting to the following output: [{"id": 1, "data": "One"}, {"id": 2, "data": "Two"},[{"id": 3, "data": "Three"}, {"id": 4, "data": "Four"}]]. Note square brackets in bold which shall be fixed as this will not be a correct final array of objects.
Final JSON is stripped from all square brackets with replace method used two times: for "[" and "]" replace(replace(json_modify(t1.JSON_t1, 'append $', json_query(t2.JSON_t2)), '[', ''), ']', '')
Final JSON is added with two square brackets at start and end to make a valid JSON array concat('[', replace(replace(json_modify(t1.JSON_t1, 'append $', json_query(t2.JSON_t2)), '[', ''), ']', ''), ']')
You can test if the final JSON is valid with ISJSON()
Points to note
If you don't use json_query, you get the following result:
[{"id": 1, "data": "One"}, {"id": 2, "data": "Two"},"[{\"id\": 3, \"data\": \"Three\"}, {\"id\": 4, \"data\": \"Four\"}]"]. See more on this here.
I tried to strip from square brackets only JSON_t2 and use json_modify like this json_modify(t1.JSON_t1, 'append $', json_query('{"id": 3, "data": "Three"}, {"id": 4, "data": "Four"}') but this results to appending only the first item from the JSON_t2 like this: [{"id": 1, "data": "One"}, {"id": 2, "data": "Two"},{"id": 3, "data": "Three"}]
NB. if you have nested arrays, this method is not suitable. In my case it works well since I have a simple array of objects (certificates with various key/value pairs like validity, type, issue date, etc.)

Related

How to insert JSON to an existing table without specifying column names?

To a table with two columns, named Id and Name, how can I insert the following json as rows, but without explicitly specifying the column names of the table (i.e. without WITH (Id ..., Name ...)?
[
{
"Id": "f08af9c2-8e67-4a7f-9413-1afffa2de96b",
"SomeOtherKey": " ... ",
"Name": "The One",
...
},
{
"Name": "All is one"
"Id": "9bbb094b-aa64-4c36-90a2-50e10f91c6a3",
"Whatever": 99,
...
},
{
"Id": "af9d22d8-1e46-4d57-8179-75f094d2efa1",
"SomeArrayWhyNot": [0, 1, 1, 2, 3, 5, 8, 13, 21]
"Surprise": "This one does not have a Name value!!! 😱"
...
},
...
]
The question is basically how to make SQL match the key-name to its suitable column name, ignoring json values with keys that do not have suitable column names, resulting with the following table (for the above json example):
Id
Name
f08af9c2-8e67-4a7f-9413-1afffa2de96b
The One
9bbb094b-aa64-4c36-90a2-50e10f91c6a3
All is one
af9d22d8-1e46-4d57-8179-75f094d2efa1
NULL
...
...
Not sure why you want this, but you can also do this:
INSERT INTO YourTable (Id, Name)
SELECT JSON_VALUE(x.value, '$.Id'), JSON_VALUE(x.value, '$.Name')
FROM OPENJSON('[{
"Id": "f08af9c2-8e67-4a7f-9413-1afffa2de96b",
"SomeOtherKey": " ... ",
"Name": "The One"
},
{
"Name": "All is one",
"Id": "9bbb094b-aa64-4c36-90a2-50e10f91c6a3",
"Whatever": 99
},
{
"Id": "af9d22d8-1e46-4d57-8179-75f094d2efa1",
"SomeArrayWhyNot": [0, 1, 1, 2, 3, 5, 8, 13, 21],
"Surprise": "This one doesn''t have a Name value!!! 😱"
}]') x
EDIT dynamic version.
It has many caveats though, your column must match exactly the case in the json, it's very longwinded but if that's what floats your vessel...
declare #json nvarchar(max) = '[{
"Id": "f08af9c2-8e67-4a7f-9413-1afffa2de96b",
"SomeOtherKey": " ... ",
"Name": "The One"
},
{
"Name": "All is one",
"Id": "9bbb094b-aa64-4c36-90a2-50e10f91c6a3",
"Whatever": 99
},
{
"Id": "af9d22d8-1e46-4d57-8179-75f094d2efa1",
"SomeArrayWhyNot": [0, 1, 1, 2, 3, 5, 8, 13, 21],
"Surprise": "This one doesn''t have a Name value!!! 😱"
}]'
create table YourTable (Id nvarchar(100), Name NVARCHAR(MAX))
create table #cols (name sysname, row_id int identity)
insert into #cols (name)
select STRING_ESCAPE(name, 'json')
from sys.columns sc
where sc.object_id = OBJECT_ID('YourTable')
DECLARE #sql nvarchar(max)
set #sql = N'
INSERT INTO YourTable ([COLS])
SELECT [JSONS]
FROM OPENJSON(#json) x'
SET #sql = REPLACE(#sql, '[COLS]',
STUFF((select ',' + QUOTENAME(name) from #cols order by row_id for xml Path(''), type).value('.', 'nvarchar(max)'), 1, 1, ''))
set #sql = replace(#sql, '[JSONS]',
stuff((SELECT ', JSON_VALUE(x.value, ''$."' + REPLACE(name,'''', '''''') + '"'')'
from #cols order by row_id for xml path(''), type).value('.', 'nvarchar(max)'), 1, 1, ''))
exec sp_executesql #sql, N'#json nvarchar(max)', #json = #json
select * FROM YourTable
You can use the OPENJSON function in SQL Server to parse the JSON data and then use INSERT INTO to insert the data into the table.
INSERT INTO YourTable (Id, Name)
SELECT *
FROM OPENJSON('[
{
"Id": "f08af9c2-8e67-4a7f-9413-1afffa2de96b",
"SomeOtherKey": " ... ",
"Name": "The One"
},
{
"Name": "All is one",
"Id": "9bbb094b-aa64-4c36-90a2-50e10f91c6a3",
"Whatever": 99
},
{
"Id": "af9d22d8-1e46-4d57-8179-75f094d2efa1",
"SomeArrayWhyNot": [0, 1, 1, 2, 3, 5, 8, 13, 21],
"Surprise": "This one doesn\'t have a Name value!!! 😱"
}
]') WITH (Id nvarchar(50) '$.Id', Name nvarchar(50) '$.Name')

Is there a way to filter rows in BigQuery by the contents of an array?

I have data in a BigQuery table that looks like this:
[
{ "id": 1, "labels": [{"key": "a", "value": 1}, {"key": "b", "value": 2}] },
{ "id": 2, "labels": [{"key": "a", "value": 1}, {"key": "b", "value": 3}] },
// a lot more rows
]
My question is, how can I find all rows where "key" = "a", "value" = 1, but also "key" = "b" and "value" = 3?
I've tried various forms of using UNNEST but I haven't been able to get it right. The CROSS JOIN leaves me with one row for every object in the labels array, leaving me unable to query by both of them.
Try this:
select *
from mytable
where exists (select 1 from unnest(labels) where key = "a" and value=1)
and exists (select 1 from unnest(labels) where key = "b" and value=3)
Assuming there is no duplicate entries in labels array - you can use below
select *
from `project.dataset.table` t
where 2 = (
select count(1)
from t.labels kv
where kv in (('a', 1), ('b', 3))
)
You can try parse JSON and then you can apply different filter conditions on it as per your requirements, in following query I have tried to identified which all records have Key=a then I tried to identify which record has value=30 then joined them through id :-
WITH data1 AS (
SELECT '{ "id": 1, "labels": [{"key": "a", "value": 11}, {"key": "b", "value": 22}] }' as c1
union all
SELECT '{ "id": 2, "labels": [{"key": "a", "value": 10}, {"key": "b", "value": 30}] }' AS C1
)
select T1.id, T1.C1, T1.key, T2.value from
(SELECT JSON_EXTRACT_SCALAR(c1 , "$.id") AS id,
json_extract_scalar(curSection, '$.value') as value, json_extract_scalar(curSection, '$.key') as key,
c1
FROM data1 tbl LEFT JOIN unnest(json_extract_array(tbl.c1, '$.labels')
) curSection
where json_extract_scalar(curSection, '$.key')='a') T1,
(
SELECT JSON_EXTRACT_SCALAR(c1 , "$.id") AS id,
json_extract_scalar(curSection, '$.value') as value, json_extract_scalar(curSection, '$.key') as key,
c1
FROM data1 tbl LEFT JOIN unnest(json_extract_array(tbl.c1, '$.labels')
) curSection
where json_extract_scalar(curSection, '$.value')='30') T2
where T1.id = T2.id
Hopefully it may work for you.

How to build rows of JSON from a table with one one-to-many relationship

I have a table main like this:
create foreign table main (
"id" character varying not null,
"a" character varying not null,
"b" character varying not null
)
And I have another table, not_main, like this:
create foreign table not_main (
"id" character varying not null,
"fk" character varying not null,
"d" character varying not null,
"e" character varying not null
)
Should I want a query whose return is like:
json
0 {"id": "id_main_0", "a": "a0", "b": "b0", "cs": [{"id": "id_not_main_0", "fk": "id_main_0", "d": "d0", "e": "e0"}, {"id": "id_not_main_1", "fk": "id_main_0", "d": "d1", "e": "e1"}]}
1 {"id": "id_main_1", "a": "a1", "b": "b1", "cs": [{"id": "id_not_main_2", "fk": "id_main_1", "d": "d2", "e": "e3"}, {"id": "id_not_main_3", "fk": "id_main_1", "d": "d3", "e": "e3"}]}
How should I do it?
I tried:
select
json_build_object(
'id', m."id",
'a', m."a",
'b', m."b",
'cs', json_build_array(
json_build_object(
'd', nm."d",
'e', nm."e"
)
)
)
from main m
left join not_main nm on
nm."requisitionId" = m.id;
But it returns only one element in cs:
json
0 {"id": "id_main_0", "a": "a0", "b": "b0", "cs": [{"id": "id_not_main_0", "fk": "id_main_0", "d": "d0", "e": "e0"}]}
1 {"id": "id_main_1", "a": "a1", "b": "b1", "cs": [{"id": "id_not_main_2", "fk": "id_main_1", "d": "d2", "e": "e3"}]}
OBS: consider that the constraints of and between main and not_main are properly modeled, e.g., that I actually have both id columns as PKs and that fk is references the id column of main.
You want json array aggregation. Basically, you just need to change json_build_array() to json_agg(), and to add a group by clause:
select
json_build_object(
'id', m.id,
'a', m.a,
'b', m.b,
'cs', json_agg(
json_build_object(
'd', nm.d,
'e', nm.e
)
)
)
from main m
left join not_main nm on
nm.requisitionId = m.id
group by m.id, m.a, m.b

PLSQL: remove unwanted double quotes in a Json

I have a Json like this (it is contained in a clob variable):
{"id": "33", "type": "abc", "val": "2", "cod": "", "sg1": "1", "sg2": "1"}
{"id": "359", "type": "abcef", "val": "52", "cod": "aa", "sg1": "", "sg2": "0"}
…
I need to remove " from values of: id, val, sg1, sg2
Is it possibile?
For example, I need to obtain this:
{"id": 33, "type": "abc", "val": 2, "cod": "", "sg1": 1, "sg2": 1}
{"id": 359, "type": "abcef", "val": 52, "cod": "aa", "sg1": , "sg2": 0}
…
If you are using Oracle 12 (R2?) or later then you can convert your JSON to the appropriate data types and then convert it back to JSON.
Oracle 18 Setup:
CREATE TABLE test_data ( value CLOB );
INSERT INTO test_data ( value )
VALUES ( '{"id": "33", "type": "abc", "val": "2", "cod": "", "sg1": "1", "sg2": "1"}' );
INSERT INTO test_data ( value )
VALUES ( '{"id": "359", "type": "abcef", "val": "52", "cod": "aa", "sg1": "", "sg2": "0"}' );
Query:
SELECT JSON_OBJECT(
'id' IS j.id,
'type' IS j.typ,
'val' IS j.val,
'cod' IS j.cod,
'sg1' IS j.sg1,
'sg2' IS j.sg2
) AS JSON
FROM test_data t
CROSS JOIN
JSON_TABLE(
t.value,
'$'
COLUMNS
id NUMBER(5,0) PATH '$.id',
typ VARCHAR2(10) PATH '$.type',
val NUMBER(5,0) PATH '$.val',
cod VARCHAR2(10) PATH '$.cod',
sg1 NUMBER(5,0) PATH '$.sg1',
sg2 NUMBER(5,0) PATH '$.sg2'
) j
Output:
| JSON |
| :--------------------------------------------------------------- |
| {"id":33,"type":"abc","val":2,"cod":null,"sg1":1,"sg2":1} |
| {"id":359,"type":"abcef","val":52,"cod":"aa","sg1":null,"sg2":0} |
Or, if you want to use regular expressions (you shouldn't if you have the choice and should use a proper JSON parser instead) then:
Query 2:
SELECT REGEXP_REPLACE(
REGEXP_REPLACE(
value,
'"(id|val|sg1|sg2)": ""',
'"\1": "null"'
),
'"(id|val|sg1|sg2)": "(\d+|null)"',
'"\1": \2'
) AS JSON
FROM test_data
Output:
| JSON |
| :-------------------------------------------------------------------------- |
| {"id": 33, "type": "abc", "val": 2, "cod": "", "sg1": 1, "sg2": 1} |
| {"id": 359, "type": "abcef", "val": 52, "cod": "aa", "sg1": null, "sg2": 0} |
db<>fiddle here

In PostgreSQL, what's the best way to select an object from a JSONB array?

Right now, I have an an array that I'm able to select off a table.
[{"_id": 1, "count: 3},{"_id": 2, "count: 14},{"_id": 3, "count: 5}]
From this, I only need the count for a particular _id. For example, I need the count for
_id: 3
I've read the documentation but I haven't been able to figure out the correct way to get the object.
WITH test_array(data) AS ( VALUES
('[
{"_id": 1, "count": 3},
{"_id": 2, "count": 14},
{"_id": 3, "count": 5}
]'::JSONB)
)
SELECT val->>'count' AS result
FROM
test_array ta,
jsonb_array_elements(ta.data) val
WHERE val #> '{"_id":3}'::JSONB;
Result:
result
--------
5
(1 row)