How can I correctly translate this complicated JSON into a SQL table? - sql

Question

I have the following JSON (some people may recall I've posted here before about a care management system I'm extracting info from in JSON form) to then use in an automated process of KPI gathering. A single resident strand of this particular JSON export is below, though there is one for each resident contained within the extract:
{
"ServiceUserDetails": [
{
"CellDetails": [
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": [
{
"careNoteID": "00000000-0000-0000-0000-000000000000",
"CareAttempted": true,
"hour": null,
"DateDone": "2019-10-20T18:51:36.966Z",
"worker": null,
"sliderOpt": 0,
"DisplayAnswerType": 1
}
]
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": [
{
"careNoteID": "00000000-0000-0000-0000-000000000000",
"CareAttempted": true,
"hour": null,
"DateDone": "2019-10-25T12:55:02.006Z",
"worker": null,
"sliderOpt": 3,
"DisplayAnswerType": 1
}
]
},
{
"CareNotes": []
},
{
"CareNotes": []
},
{
"CareNotes": [
{
"careNoteID": "00000000-0000-0000-0000-000000000000",
"CareAttempted": true,
"hour": null,
"DateDone": "2019-10-28T13:42:42.136Z",
"worker": null,
"sliderOpt": 2,
"DisplayAnswerType": 1
},
{
"careNoteID": "00000000-0000-0000-0000-000000000000",
"CareAttempted": true,
"hour": null,
"DateDone": "2019-10-28T22:29:06.226Z",
"worker": null,
"sliderOpt": 4,
"DisplayAnswerType": 1
}
]
},
{
"CareNotes": [
{
"careNoteID": "00000000-0000-0000-0000-000000000000",
"CareAttempted": true,
"hour": null,
"DateDone": "2019-10-29T13:52:32.961Z",
"worker": null,
"sliderOpt": 7,
"DisplayAnswerType": 1
}
]
},
{
"CareNotes": []
},
{
"CareNotes": [
{
"careNoteID": "00000000-0000-0000-0000-000000000000",
"CareAttempted": true,
"hour": null,
"DateDone": "2019-10-31T16:36:05.347Z",
"worker": null,
"sliderOpt": 4,
"DisplayAnswerType": 1
},
{
"careNoteID": "00000000-0000-0000-0000-000000000000",
"CareAttempted": true,
"hour": null,
"DateDone": "2019-10-31T17:53:42.264Z",
"worker": null,
"sliderOpt": 2,
"DisplayAnswerType": 1
}
]
}
],
"ServiceUserName": "Resident Name",
"ServiceUserID": "45k045839-d88b-436b-92b3-8829dce683be",
"ServiceUserDateOfBirth": "16/01/30"
}
]
}
Basically, this is data regarding a number of days when a resident has potentially been for a number two! In this single example, the person in question is independent and only occasionally requires assistance, and the data export shows each day in the month, with the majority of "CareNotes" being empty to reflect this.
I have created the following query in SQL:
use CMUtility;
DECLARE #JSON VARCHAR(MAX)
SELECT #JSON = BulkColumn
FROM OPENROWSET
(BULK 'C:\Users\user\Documents\Embarcadero\Studio\Projects\CMUServer\Win32\Debug\BowelMovementChart-Home Name.json', SINGLE_CLOB)
AS j
DROP TABLE IF EXISTS [JSONBMs-Home Name]
SELECT * INTO [JSONBMs-Home Name]
FROM
OPENJSON(#JSON, '$.ServiceUserDetails')
WITH
(
ServiceUserName nvarchar(100) '$.ServiceUserName',
ServiceUserID nvarchar(255) '$.ServiceUserID',
ServiceUserDateOfBirth nvarchar(10) '$.ServiceUserDateOfBirth',
CareNotes nvarchar(max) '$.CellDetails' as JSON
) as j1
CROSS APPLY OPENJSON(j1.CareNotes) WITH
(
CareNoteID nvarchar(200) '$.CareNoteID',
CareAttempted nvarchar(10) '$.CareAttempted',
[hour] nvarchar(50) '$.hour',
DateDone nvarchar(10) '$.DateDone',
worker nvarchar(100) '$.worker',
sliderOpt nvarchar(10) '$.sliderOpt',
DisplayAnswerType nvarchar(10) '$.DisplayAnswerType'
) j2
SELECT * FROM [JSONBMs-Home Name]
If I run the code as is, I get the following result (1302 rows):
But if I quit at the end of the j1, before the cross apply and then run a select * from table, I get the following 42 rows (which is the number of residents):
It is obviously in the second half of this code that I am going wrong. In other JSON parsing, I've had to use OUTER APPLY rather than CROSS APPLY, but interchanging them has no impact on the results. The CareNotes JSON fields remain NULL, and I think it is to do with the number of additional elements in here? 🤷‍♂️ I still don't 100% understand the difference between objects and arrays within JSON, and this has puzzled me for the last 10hours or so!
My understanding would be that to get to the additional JSON in CareNotes, I would need to point from ServiceUserDetails > CellDetails > CareNotes, but it is at this point I come unstuck. If I add ", '$.CareNotes'" to the second half's OPENJSON, then I get no results at all. I would appreciate any assistance!!
Thanks
Ant
EDIT: I've added an additional layer of cross apply, to make it now look like this, and I do now get results in the last variables:
select * into [JSONBMs-Home Name]
from
OPENJSON(#JSON, '$.ServiceUserDetails')
with
(
ServiceUserName nvarchar(100) '$.ServiceUserName',
ServiceUserID nvarchar(255) '$.ServiceUserID',
ServiceUserDateOfBirth nvarchar(10) '$.ServiceUserDateOfBirth',
CellDetails nvarchar(max) '$.CellDetails' as JSON
) as j1
cross apply openjson(j1.CellDetails) with
(
CareNotes nvarchar(max) '$.CareNotes' as JSON
) as j2
cross apply openjson(j2.CareNotes) with
(
CareNoteID nvarchar(200) '$.CareNoteID',
CareAttempted nvarchar(10) '$.CareAttempted',
[hour] nvarchar(50) '$.hour',
DateDone nvarchar(10) '$.DateDone',
worker nvarchar(100) '$.worker',
sliderOpt nvarchar(10) '$.sliderOpt',
DisplayAnswerType nvarchar(10) '$.DisplayAnswerType'
) j3
select * from [JSONBMs-Home Name]
My results are now like this:
I think this is now what I was looking for, because if then order the output by ServiceUserName it shows me the individual entries that appear in the report itself. So I think I've answered my own question!

Answer 1

as a rule of thumb, every time there is a json array an apply operator is needed.
since CareNotes is an array, you'll need an APPLY on the CareNotes to get its elements.
SELECT *
FROM
OPENJSON(#JSON, '$.ServiceUserDetails')
WITH
(
ServiceUserName nvarchar(100) '$.ServiceUserName',
ServiceUserID nvarchar(255) '$.ServiceUserID',
ServiceUserDateOfBirth nvarchar(10) '$.ServiceUserDateOfBirth',
CareNotes nvarchar(max) '$.CellDetails' as JSON
) as j1
OUTER APPLY OPENJSON(j1.CareNotes) --, '$')
WITH
(
CareNotesElements nvarchar(max) '$.CareNotes' AS JSON
) as notearray
OUTER APPLY OPENJSON(notearray.CareNotesElements) WITH
(
CareNoteID nvarchar(200) '$.careNoteID', ---typo here, json has careN.. ,query had CareN...
CareAttempted nvarchar(10) '$.CareAttempted',
[hour] nvarchar(50) '$.hour',
DateDone nvarchar(10) '$.DateDone',
worker nvarchar(100) '$.worker',
sliderOpt nvarchar(10) '$.sliderOpt',
DisplayAnswerType nvarchar(10) '$.DisplayAnswerType'
) j2

Answer 2

Because I hadn't 'drilled down' low enough, I needed to add another cross apply to get to the JSON itself.
This is my final code, which does more or less give me the data I need. Apologies if this was obvious to some, but this isn't my day job and I'm learning as I go!
Thanks
use CMUtility;
DECLARE #JSON VARCHAR(MAX)
SELECT #JSON = BulkColumn
FROM OPENROWSET
(BULK 'C:\Users\user\Documents\Embarcadero\Studio\Projects\CMUServer\Win32\Debug\BowelMovementChart-Home Name.json', SINGLE_CLOB)
AS j
drop table if exists [JSONBMs-Home Name]
select * into [JSONBMs-Home Name]
from
OPENJSON(#JSON, '$.ServiceUserDetails')
with
(
ServiceUserName nvarchar(100) '$.ServiceUserName',
ServiceUserID nvarchar(255) '$.ServiceUserID',
ServiceUserDateOfBirth nvarchar(10) '$.ServiceUserDateOfBirth',
CellDetails nvarchar(max) '$.CellDetails' as JSON
) as j1
cross apply openjson(j1.CellDetails) with
(
CareNotes nvarchar(max) '$.CareNotes' as JSON
) as j2
cross apply openjson(j2.CareNotes) with
(
CareNoteID nvarchar(200) '$.CareNoteID',
CareAttempted nvarchar(10) '$.CareAttempted',
[hour] nvarchar(50) '$.hour',
DateDone nvarchar(10) '$.DateDone',
worker nvarchar(100) '$.worker',
sliderOpt nvarchar(10) '$.sliderOpt',
DisplayAnswerType nvarchar(10) '$.DisplayAnswerType'
) j3
select * from [JSONBMs-Home Name]
order by ServiceUserName