OpenJson using a wildcard - sql

I have a SQL query using OPENJSON to import JSON data into a table. My problem is that the data I need is nested. How can I use a wildcard in the JSON path to get what I need?
SELECT #Set =
BulkColumn FROM OPENROWSET
(BULK 'Sets.json', DATA_SOURCE = 'MyAzureJson', SINGLE_BLOB) JSON;
INSERT INTO [Sets]
SELECT [name]
FROM OPENJSON(#Set)
WITH(
[name] nvarchar(50) '$.*.name'
)
my json file is set up like this..
{
"testOne" : {
name: "nameOne"
},
"testTwo : {
name: "nameTwo"
}
}
the error I'm getting with everything I try..
JSON path is not properly formatted. Unexpected character '*' is found at position 2.
I've tried . * [] and nothing works

As far as I know there is no support for wildcards in OPENJSON.
Instead you can do a workaround by ignoring the field name in your search. Use JSON_VALUE for this.
INSERT INTO [Sets]
SELECT
JSON_VALUE([value], '$.name')
FROM
OPENJSON(#Set)
Explanation: If you don't define the variables of OPENJSON inside a WITH clause and instead do a simple SELECT * FROM OPENJSON(#Set) query, you will get a result with key, value and type columns (see example output below). Because key contains your problematic field name, you can ignore that part and just look into the value column of the data.
[key] [value] [type]
----- ------- ------
testOne { name: "nameOne" } 5
testTwo { name: "nameTwo" } 5

Related

Reading JSON string and find the max value as integer

I have a JSON string as follows:
DECLARE #json nvarchar(max)
SET #json = '{"value": [
{
"AEDAT": "20211110"
},
{
"AEDAT": "20211110"
},
{
"AEDAT": "20211110"
},
{
"AEDAT": "20211112"
},
{
"AEDAT": "20211112"
},
{
"AEDAT": "20211112"
}
]}';
Now I want to read this JSON in SQL Server using OPENJSON() and find the MAX value for each AEDAT. For this, I am using the following query:
SELECT MAX(value)
FROM OPENJSON(#json, '$.value')
The above query is returning a row with key value pair as below:
{"AEDAT":"20211112"}
My objective is to get only 20211112 as integer.
How to achieve this?
If you want to get the max value as integer, you need to use OPENJSON() with explicit schema (the WITH clause with columns definitions). This schema depends on the structure of the parsed JSON (in your case it's a JSON array):
SELECT MAX(AEDAT) AS MaxAEDAT
FROM OPENJSON(#json, '$.value') WITH (
AEDAT int '$.AEDAT'
)
If the parsed values are dates, you may try a different statement:
SELECT MAX(TRY_CONVERT(date, AEDAT, 112)) AS MaxAEDAT
FROM OPENJSON(#json, '$.value') WITH (
AEDAT varchar(8) '$.AEDAT'
)
OPENJSON without explicit schema, gives you the value column which, in your example, will contain an object such as {"AEDAT": "20211110"} having type = 5. Use JSON_VALUE on that object:
select max(cast(json_value(j.value, '$.AEDAT') as int))
from openjson(#json, '$.value') as j

Edit json with array of objects error, it's replacing the array with one object

I have a JSON column like this
[
{
"JoinedTime": "2021-04-13T20:09:40.654Z",
"LeftTime": "2021-04-13T20:09:53.368Z",
},
{
"JoinedTime": "2021-04-13T20:09:40.654Z",
"LeftTime": null,
},
]
And I have to update all null 'LeftTime' properties to GETUTCDATE(), so change that one 'null' value to the current GETUTCDATE().
I've gotten this far
UPDATE JoinedLeft
SET JsonColumn = JSON_MODIFY(js.[value], '$.LeftTime', FORMAT(GETUTCDATE(), 'yyyy-MM-dd"T"HH:mm:ss"Z"'))
FROM JoinedLeft JL
CROSS APPLY OPENJSON(JL.JsonColumn) AS js
WHERE
JSON_VALUE(js.[value], '$.LeftTime') IS NULL AND
JSON_VALUE(js.[value], '$.JoinedTime') IS NOT NULL
But it just replaces the column with just the object that I wanted to edit, instead of editing the object and saving the array again.
Can someone help me?
When you parse a JSON array with OPENJSON() and default schema (without the WITH clause), the result is a table with rows for each item in the parsed JSON array. This explains the enexpected results from the UPDATE statement. I don't think that JSON_MODIFY() supports wildcards as path parameter, so one possible option is to parse, modify and build the JSON array again:
Table:
SELECT JsonColumn
INTO JoinedLeft
FROM (VALUES
('[
{"JoinedTime": "2021-04-13T20:09:40.654Z","LeftTime": "2021-04-13T20:09:53.368Z"},
{"JoinedTime": "2021-04-14T21:09:40.654Z", "LeftTime": null}
]'),
('[
{"JoinedTime": "2021-05-14T21:09:40.654Z", "LeftTime": null}
]')
) t (JsonColumn)
Statement:
UPDATE JL
SET JL.JsonColumn = V.JsonColumn
FROM JoinedLeft JL
CROSS APPLY (
SELECT
JoinedTime,
COALESCE(LeftTime, FORMAT(GETUTCDATE(), 'yyyy-MM-dd"T"HH:mm:ss"Z"')) AS LeftTime
FROM OPENJSON(JL.JsonColumn, '$') WITH (
JoinedTime varchar(24) '$.JoinedTime',
LeftTime varchar(24) '$.LeftTime'
) AS JsonColumn
FOR JSON PATH
) V (JsonColumn);
Result:
JsonColumn
[{"JoinedTime":"2021-04-13T20:09:40.654Z","LeftTime":"2021-04-13T20:09:53.368Z"},{"JoinedTime":"2021-04-14T21:09:40.654Z","LeftTime":"2021-05-18T12:12:35Z"}]
[{"JoinedTime":"2021-05-14T21:09:40.654Z","LeftTime":"2021-05-18T12:12:35Z"}]

Extracting specific value from large string SQL

I've used a combination of CHARINDEX and SUBSTRING but can't get it working.
I get passed a variable in SQL that contains a lot of text but has an email in it. I need to extract the email value.
I have to use SQL 2008.
I'm trying to extract the value between "EmailAddress":" and ",
An example string is here:
{ "Type":test,
"Admin":test,
"User":{
"UserID":"16959191",
"FirstName":"Test",
"Surname":"Testa",
"EmailAddress":"Test.Test#test.com",
"Address":"Test"
}
}
Assuming you can't upgrade to 2016 or higher, you can use a combination of substring and charindex.
I've used a common table expression to make it less cumbersome, but you don't have to.
DECLARE #json varchar(4000) = '{ "Type":test,
"Admin":test,
"User":{
"UserID":"16959191",
"FirstName":"Test",
"Surname":"Testa",
"EmailAddress":"Test.Test#test.com",
"Address":"Test"
}
}';
WITH CTE AS
(
SELECT #Json as Json,
CHARINDEX('"EmailAddress":', #json) + LEN('"EmailAddress":') As StartIndex
)
SELECT SUBSTRING(Json, StartIndex, CHARINDEX(',', json, StartIndex) - StartIndex)
FROM CTE
Result: "Test.Test#test.com"
The first hint is: Move to v2016 if possible to use JSON support natively. v2008 is absolutely outdated...
The second hint is: Any string action (and all my approaches below will need some string actions too), will suffer from forbidden characters, unexpected blanks or any other surprise you might find within your data.
Try it like this:
First I create a mockup scenario to simulate your issue
DECLARE #tbl TABLE(ID INT IDENTITY,YourJson NVARCHAR(MAX));
INSERT INTO #tbl VALUES
(N'{ "Type":"test1",
"Admin":"test1",
"User":{
"UserID":"16959191",
"FirstName":"Test1",
"Surname":"Test1a",
"EmailAddress":"Test1.Test1#test.com",
"Address":"Test1"
}
}')
,(N'{ "Type":"test2",
"Admin":"test2",
"User":{
"UserID":"16959191",
"FirstName":"Test2",
"Surname":"Test2a",
"EmailAddress":"Test2.Test2#test.com",
"Address":"Test2"
}
}');
--Starting with v2016 there is JSON support
SELECT JSON_VALUE(t.YourJson, '$.User.EmailAddress')
FROM #tbl t
--String-methods
--use CHARINDEX AND SUBSTRING
DECLARE #FirstBorder NVARCHAR(MAX)='"EMailAddress":';
DECLARE #SecondBorder NVARCHAR(MAX)='",';
SELECT t.*
,A.Pos1
,B.Pos2
,SUBSTRING(t.YourJson,A.Pos1,B.Pos2 - A.Pos1) AS ExtractedEMail
FROM #tbl t
OUTER APPLY(SELECT CHARINDEX(#FirstBorder,t.YourJson)+LEN(#FirstBorder)) A(Pos1)
OUTER APPLY(SELECT CHARINDEX(#SecondBorder,t.YourJson,A.Pos1)) B(Pos2);
--use a XML trick
SELECT CAST('<x>' + REPLACE(REPLACE((SELECT t.YourJson AS [*] FOR XML PATH('')),'"EmailAddress":','<mailAddress value='),',',' />') + '</x>' AS XML)
.value('(/x/mailAddress/#value)[1]','nvarchar(max)')
FROM #tbl t
Some explanations:
JSON-support will parse the value directly from a JSON path.
For CHARINDEX AND SUBSTRING I use APPLY. The advantage is, that you can use the computed positions like a variable. No need to repeat the CHARINDEX statements over and over.
The XML approach will transform your JSON to a rather strange and ugly XML. The only sensefull element is <mailAddress> with an attribute value. We can use the native XML method .value() to retrieve the value you are asking for:
An intermediate XML looks like this:
<x>{ "Type":"test1" />
"Admin":"test1" />
"User":{
"UserID":"16959191" />
"FirstName":"Test1" />
"Surname":"Test1a" />
<mailAddress value="Test1.Test1#test.com" />
"Address":"Test1"
}
}</x>

Reading file containing multiple JSON objects using MS SQL Server

I need help reading JSON file in MSSQL
I have a json format like this
{
"_id":{
"$oid":"5c6ceb395916c77f71d9f531"
},
"uuid":"8337df01-7d98-4cdd-b5eb-7fafa88d3740",
"firstName":"TESTFN",
"lastName":"TESTLN",
"middleName":"TESTMN",
"gender":"MALE",
"nationality":"",
"mobileNumber":"",
"card":"TACC-00001",
"claims":{
"$numberLong":"1"
},
"hisId":{
"$numberLong":"0"
},
"hospitals":[
{
"$ref":"hospital",
"$id":{
"$oid":"5bd80e4e36ec8c600f3e750a"
}
}
]
}
I'm trying to read json using below code. It is working fine.
But when I use the exact source file which is repeated of above format and has a lot of fields, I'm only retrieving one row.
SELECT *
FROM OPENROWSET (BULK 'C:\demo_json\claims.json', SINGLE_CLOB) as j
CROSS APPLY OPENJSON(BulkColumn)
I've searched it and found out it is because of multi nested json.
Used below code..
SELECT *
FROM OPENROWSET (BULK 'C:\demo_json\claims.json', SINGLE_CLOB) as j
CROSS APPLY OPENJSON('['+BulkColumn+']')
But I'm getting error: JSON text is not properly formatted. Unexpected character '{' is found at position 2760.
Note: I have no control or cant edit the source file JSON
What can I do to make sure JSON file is in correct format or make it correct for me to read it.
Source file : claims
It seems, that your input file has lines, which are valid JSON objects. One possible approach in this situation is to import this file using BULK INSERT and appropriate ROWTERMINATOR:
Text file (claims.json):
{"_id":{"$oid":"5c6ceb395916c77f71d9f536"},"_class":"stash.ph.data.model.mongo.claims.Claims","uuid":"02735305-220e-4399-9217-0b99939797d2","submittedAt":{"$date":{"$numberLong":"1550641993123"}},"submittedByUuid":"c60d326e-0893-4718-a0e0-f64ac697dd2e","status":"SUBMITTED","billingDesignation":"Hospital","createdAt":{"$date":{"$numberLong":"1550641977093"}},"updatedAt":{"$date":{"$numberLong":"1550641977093"}},"particulars":[{"_id":{"$oid":"5c6ceb395916c77f71d9f530"},"uuid":"b2b3cc59-b9a8-49f1-8f92-30021d2c7abb","particular":"ER_CONSULTATION","procedure":[],"doctors":[{"_id":{"$oid":"5c6ceb395916c77f71d9f52f"},"uuid":"494e1bf9-a3f9-4c3a-bdc0-3ef15b4760a2","type":"PRIMARY","professionalFee":{"$numberDouble":"500.0"},"doctor":{"$ref":"doctor","$id":{"$oid":"5bd8156736ec8c600f3e7604"}}}]}],"patient":{"_id":{"$oid":"5c6ceb395916c77f71d9f531"},"uuid":"8337df01-7d98-4cdd-b5eb-7fafa88d3740","firstName":"TESTFN","email":"","lastName":"TESTLN","middleName":"TESTMN","suffix":"","prefix":"","birthday":"","age":"","gender":"MALE","nationality":"","mobileNumber":"","card":"TACC-00001","cardExpiration":"","company":"","claims":{"$numberLong":"1"},"phicNo":"","hisId":{"$numberLong":"0"},"hospitals":[{"$ref":"hospital","$id":{"$oid":"5bd80e4e36ec8c600f3e750a"}}],"hmos":[]},"files":[{"_id":{"$oid":"5c6ceb395916c77f71d9f533"},"uuid":"8731cb71-e424-4147-8158-38558df56ad5","filename":"8731cb71-e424-4147-8158-38558df56ad5.pdf","description":"files desc"}],"medicalData":{"_id":{"$oid":"5c6ceb395916c77f71d9f534"},"uuid":"617e193b-2280-491b-b34f-adf6eaeb2244","complain":[""],"diagnosis":["A90 | Dengue fever [classical dengue]"],"remarks":"","dateExamined":{"$date":{"$numberLong":"1546272000000"}}},"loa":{"_id":null,"uuid":"b4bc96d1-554b-4cc7-9511-c56d325cf5de","referenceId":""},"payment":{"_id":{"$oid":"5c6ceb395916c77f71d9f535"},"uuid":"9714fdf1-311f-4898-8070-f3fd1ddd0658","phicBill":{"$numberDouble":"0.0"},"hospitalBill":{"$numberDouble":"0.0"},"total":{"$numberDouble":"500.0"},"balance":{"$numberDouble":"500.0"},"createdAt":{"$date":{"$numberLong":"1550641977101"}},"updatedAt":{"$date":{"$numberLong":"1550641977101"}}},"hmoRepresentative":"","remarks":"","registryNumber":{"$numberLong":"0"},"soaNumber":"MBC-MCD-OUT-190000001","soaUuid":"2366abe1-672b-456b-bfd9-5584bf9e7f5a","batchName":"BATCH-190000001","claimLogs":[{"_id":{"$oid":"5c6ceb395916c77f71d9f532"},"uuid":"e2079956-0fbd-4565-9a93-0791a898fbcd","status":"PENDING","date":{"$date":{"$numberLong":"1550641977093"}},"userUuid":"c60d326e-0893-4718-a0e0-f64ac697dd2e","firstName":"Medicard","middleName":"","lastName":"Hospital"}],"messages":[],"hospital":{"$ref":"hospital","$id":{"$oid":"5bd80e4e36ec8c600f3e750a"}},"hmo":{"$ref":"hmo","$id":{"$oid":"598615f970d8a672a291132e"}}}
{"_id":{"$oid":"5c6ceda95916c77f71d9f574"},"_class":"stash.ph.data.model.mongo.claims.Claims","uuid":"73e6627e-187a-4f9f-a141-e4e01666c7c4","submittedAt":{"$date":{"$numberLong":"1550642786142"}},"submittedByUuid":"c60d326e-0893-4718-a0e0-f64ac697dd2e","status":"RETURN","billingDesignation":"Hospital","createdAt":{"$date":{"$numberLong":"1550642601507"}},"updatedAt":{"$date":{"$numberLong":"1550642601507"}},"particulars":[{"_id":{"$oid":"5c6ceda95916c77f71d9f56e"},"uuid":"8c774847-e0ca-4732-9669-6bf6a4579cc6","particular":"ER_CONSULTATION","procedure":[],"doctors":[{"_id":{"$oid":"5c6ceda95916c77f71d9f56d"},"uuid":"0520ffdf-ceb9-4413-8cec-c5eb728b8972","type":"PRIMARY","professionalFee":{"$numberDouble":"500.0"},"doctor":{"$ref":"doctor","$id":{"$oid":"5bd8156736ec8c600f3e7604"}}}]}],"patient":{"_id":{"$oid":"5c6ceda95916c77f71d9f56f"},"uuid":"e71116f6-710e-4e13-b806-c88cf172f5ca","firstName":"FGSDF","email":"","lastName":"FDGDS","middleName":"FGHDSF","suffix":"","prefix":"","birthday":"","age":"","gender":"MALE","nationality":"","mobileNumber":"","card":"TACC-00002","cardExpiration":"","company":"","claims":{"$numberLong":"1"},"phicNo":"","hisId":{"$numberLong":"0"},"hospitals":[{"$ref":"hospital","$id":{"$oid":"5bd80e4e36ec8c600f3e750a"}}],"hmos":[]},"files":[{"_id":{"$oid":"5c6ceda95916c77f71d9f571"},"uuid":"cc53c964-9440-413e-87b6-24fa86ccaa39","filename":"cc53c964-9440-413e-87b6-24fa86ccaa39.pdf","description":"files desc"}],"medicalData":{"_id":{"$oid":"5c6ceda95916c77f71d9f572"},"uuid":"d683a97b-f440-45a3-adef-b3df81be2fdc","complain":[""],"diagnosis":["A90 | Dengue fever [classical dengue]"],"remarks":"","dateExamined":{"$date":{"$numberLong":"1546358400000"}}},"loa":{"_id":null,"uuid":"3e929137-44c3-45f0-8cb5-fef1d83ff866","referenceId":""},"payment":{"_id":{"$oid":"5c6ceda95916c77f71d9f573"},"uuid":"6b0ed0d3-2993-4605-b4cd-421d84656921","phicBill":{"$numberDouble":"0.0"},"hospitalBill":{"$numberDouble":"0.0"},"total":{"$numberDouble":"500.0"},"balance":{"$numberDouble":"500.0"},"createdAt":{"$date":{"$numberLong":"1550642601512"}},"updatedAt":{"$date":{"$numberLong":"1550642601512"}}},"hmoRepresentative":"","remarks":"lack of documents","registryNumber":{"$numberLong":"0"},"approvedByUuid":"6fe1a2df-2a87-4b1e-a9b7-85a8b46eec19","soaNumber":"MBC-ILE-OUT-190000001","soaUuid":"6198678b-4254-4c89-a3d1-54e0366e751c","batchName":"BATCH-190000002","claimLogs":[{"_id":{"$oid":"5c6ceda95916c77f71d9f570"},"uuid":"96077dba-166b-47bd-a8d9-b06bee38e763","status":"PENDING","date":{"$date":{"$numberLong":"1550642601507"}},"userUuid":"c60d326e-0893-4718-a0e0-f64ac697dd2e","firstName":"Medicard","middleName":"","lastName":"Hospital"}],"messages":[],"hospital":{"$ref":"hospital","$id":{"$oid":"5bd80e4e36ec8c600f3e750a"}},"hmo":{"$ref":"hmo","$id":{"$oid":"5985c32870d8a672a291129d"}}}
Statement:
CREATE TABLE #Data (
BulkColumn nvarchar(max)
)
BULK INSERT #Data
FROM 'C:\demo_json\claims.json'
WITH (ROWTERMINATOR = '0x0A')
SELECT *
FROM #Data d
CROSS APPLY OPENJSON(d.BulkColumn) j
Notes:
You may try to use basic string transformations to build a valid JSON array, but reading the file line by line should be your first option:
SELECT *
FROM OPENROWSET (BULK 'C:\demo_json\claims.json', SINGLE_CLOB) d
CROSS APPLY OPENJSON('[' + REPLACE(d.BulkColumn, '}' + CHAR(10) + '{', '},{') + ']') j

SQL Server 2017 Selecting JSON embedded within a JSON field

In SQL Server 2017, I'd like to "SELECT" a JSON object embedded within another as a string so we can store/process them later.
eg JSON:
[
{"key1":"value1",
"level2_Obj":{"key2":"value12"}
},
{"key1":"value2",
"level2_Obj":{"key22":"value22"}
},
]
From above JSON, I'd like to SELECT whole of the level2Obj JSON object, see below for what I'd like to see the "selection" result.
value1 |{"key2" :"value12"}
value2 |{"key22":"value22"}
I tried below with no luck:
SELECT * FROM
OPENJSON(#json,'$."data1"')
WITH(
[key1] nvarchar(50),
[embedded_json] nvarchar(max) '$."level2Obj"'
) AS DAP
Can some one please help how I select the contents of the 2nd level JSON object as a string?
The idea is to Write 1st level JSON properties into individual cells and rest of JSON levels into a single column of type nvarchar(max) (i.e whole of sub-level JSON object into a single column as a string for further processing in later stages).
Good day,
Firstly, Your JSON text is not properly formatted. There is extra comma after the last object in the array. I will remove this extra comma for the sake of the answer, but if this is the format you have then first step will be to clear the text and make sure that is is well formatted.
Please check if this solve your needs:
declare #json nvarchar(MAX) = '
[
{
"key1":"value1",
"level2_Obj":{"key2":"value12"}
}
,
{
"key1":"value2",
"level2_Obj":{"key22":"value22"}
}
]
'
SELECT JSON_VALUE (t1.[value], '$."key1"'), JSON_QUERY (t1.[value], '$."level2_Obj"')
FROM OPENJSON(#json,'$') t1