Related
Say I have a JSON data stored in a varchar(max) column in a database. Is it possible to use SQL to get all the JSON paths present in that data. For example for the following JSON:
{
"dog":
{
"name":"Rover",
"age": 6,
"likes:["catch", "eating"]
}
}
I would get the output of:
$.
$.dog
$.dog.name
$.dog.age
$.dog.likes[0]
$.dog.likes[1]
I have looked at functions including json_query and json_value but they seem to be more about getting data from the JSON rather than the metadata I require.
I am using SQL Server 2018.
Try a recursive CTE
DECLARE #s varchar(max) = '{
"dog":
{
"name":"Rover",
"age": 6,
"likes":["catch", "eating"]
}
}';
with cte as (
select [type], '$' + case when roottype = 4 then '['+[key]+']' else '.'+[key] end as path
from (
select r.[type] , dummy.[type] roottype, r.[key]
from OPENJSON('{"dummy":' + #s +'}', '$') dummy
cross apply OPENJSON(#s, '$') r
) t
union all
select j.[type], path + case when cte.[type] = 4 then '['+j.[key]+']' else '.'+j.[key] end
from cte
cross apply OPENJSON(#s, path) j
where cte.[type] >= 4
)
select *
from cte;
Returns
type path
5 $.dog
1 $.dog.name
2 $.dog.age
4 $.dog.likes
1 $.dog.likes[0]
1 $.dog.likes[1]
I have a table containing rows and one of the columns is of type text storing Json. I want to count how many Json data items there are (I want to count the instances of a string 'aodata.id" in each) and store that number in a column of type int called JsonRowsInThis.
I tried to select like this
SELECT
value
FROM
STRING_SPLIT((select top 1 MLSImportedGroupsOfItems_ImportedContent from RE_MLSImportedGroupsOfItems), 'odata.id')
but I am using SQL Server 2014 and string_split does not work for it.
Is there a solution to count the number of a character using replace but it did not work for my case?
A sample of the content of a cell below (partial content, it is very long)
{"#odata.context":"https://mywebsite.com/$metadata#Media","value":[{"#odata.id":"https://mywebsite.com/Media('5b9f36ec5928d64cdb53a8f7')","MediaKey":"5b9f36ec5928d64cdb53a8f7","ImageWidth":290,"ImageHeight":218,"ImageSizeDescription":"290x218","MediaURL":"https://s3.Cloudserver24.com/mlsgrid/images/eb726144-0302-4f04-a1b1-554b8a476ba3.jpeg","Modified":"2019-02-03T11:18:18.300Z","ModificationTimestamp":"2019-02-03T11:18:18.300Z","ResourceRecordKey":"CAR28791079","ResourceRecordID":"CARNCM530248","ResourceName":"property","OriginatingSystemName":"carolina","MlgCanView":true,"Order":"0"},{"#odata.id":"https://mywebsite.com/Media('5b9f36ec5928d64cdb53a8fa')","MediaKey":"5b9f36ec5928d64cdb53a8fa","ImageWidth":290,"ImageHeight":218,"ImageSizeDescription":"290x218","MediaURL":"https://s3.Cloudserver24.com/mlsgrid/images/82120215-cf3c-44b2-9786-cadd60d68db0.jpeg","Modified":"2019-02-03T11:18:18.300Z","ModificationTimestamp":"2019-02-03T11:18:18.300Z","ResourceRecordKey":"CAR28791079","ResourceRecordID":"CARNCM530248","ResourceName":"property","OriginatingSystemName":"carolina","MlgCanView":true,"Order":"3"},{"#odata.id":"https://mywebsite.com/Media('5b9f36ec5928d64cdb53a8f8')","MediaKey":"5b9f36ec5928d64cdb53a8f8","ImageWidth":290,"ImageHeight":218,"ImageSizeDescription":"290x218","MediaURL":"https://s3.Cloudserver24.com/mlsgrid/images/641b2e26-f022-44af-bfbf-c482df6a873c.jpeg","Modified":"2019-02-03T11:18:18.300Z","ModificationTimestamp":"2019-02-03T11:18:18.300Z","ResourceRecordKey":"CAR28791079","ResourceRecordID":"CARNCM530248","ResourceName":"property","OriginatingSystemName":"carolina","MlgCanView":true,"Order":"1"},{"#odata.id":"https://mywebsite.com/Media('5b9f36ec5928d64cdb53a903')","MediaKey":"5b9f36ec5928d64cdb53a903","ImageWidth":290,"ImageHeight":218,"ImageSizeDescription":"290x218","MediaURL":"https://s3.Cloudserver24.com/mlsgrid/images/066c658b-ede7-4f90-9cae-ef71774bc152.jpeg","Modified":"2019-02-03T11:18:18.300Z","ModificationTimestamp":"2019-02- etc ......}
You can count the values like this:
select ( len(value) - len(replace(value, 'aodata.id')) ) / len('aodata.id) as num_aodata_id
EDIT:
You should not be using the text data type. You can solve this by converting to varchar(max):
select ( len(v.val) - len(replace(v.val, 'aodata.id')) ) / len('aodata.id) as num_aodata_id
from t cross apply
(values (convert(varchar(max), t.value)) v(val)
You can use recursive cte as substitute for string_split function.
;with cte as (
select 0 as ctr, charindex('odata.id', MLSImportedGroupsOfItems_ImportedContent ) as pos
, MLSImportedGroupsOfItems_ImportedContent as strdata
from RE_MLSImportedGroupsOfItems
union all
select ctr + 1, charindex('odata.id', strdata , pos + 1), strdata
from cte
where pos > 0
)
select count(1) from cte where pos > 0
This answer may help if you can use SQL Server 2016 or higher. The data in your column is a valid JSON, so instead of using string based approach (based on STRING_SPLIT()), you may try to use JSON functions.
JSON:
DECLARE #json nvarchar(max) = N'{
"#odata.context":"https://mywebsite.com/$metadata#Media",
"value":[
{
"#odata.id":"https://mywebsite.com/Media(''5b9f36ec5928d64cdb53a8f7'')",
"MediaKey":"5b9f36ec5928d64cdb53a8f7",
"ImageWidth":290,
"ImageHeight":218,
"ImageSizeDescription":"290x218",
"MediaURL":"https://s3.Cloudserver24.com/mlsgrid/images/eb726144-0302-4f04-a1b1-554b8a476ba3.jpeg",
"Modified":"2019-02-03T11:18:18.300Z",
"ModificationTimestamp":"2019-02-03T11:18:18.300Z",
"ResourceRecordKey":"CAR28791079",
"ResourceRecordID":"CARNCM530248",
"ResourceName":"property",
"OriginatingSystemName":"carolina",
"MlgCanView":true,
"Order":"0"
},
{
"#odata.id":"https://mywebsite.com/Media(''5b9f36ec5928d64cdb53a8fa'')",
"MediaKey":"5b9f36ec5928d64cdb53a8fa",
"ImageWidth":290,
"ImageHeight":218,
"ImageSizeDescription":"290x218",
"MediaURL":"https://s3.Cloudserver24.com/mlsgrid/images/82120215-cf3c-44b2-9786-cadd60d68db0.jpeg",
"Modified":"2019-02-03T11:18:18.300Z",
"ModificationTimestamp":"2019-02-03T11:18:18.300Z",
"ResourceRecordKey":"CAR28791079",
"ResourceRecordID":"CARNCM530248",
"ResourceName":"property",
"OriginatingSystemName":"carolina",
"MlgCanView":true,
"Order":"3"
},
{
"#odata.id":"https://mywebsite.com/Media(''5b9f36ec5928d64cdb53a8f8'')",
"MediaKey":"5b9f36ec5928d64cdb53a8f8",
"ImageWidth":290,
"ImageHeight":218,
"ImageSizeDescription":"290x218",
"MediaURL":"https://s3.Cloudserver24.com/mlsgrid/images/641b2e26-f022-44af-bfbf-c482df6a873c.jpeg",
"Modified":"2019-02-03T11:18:18.300Z",
"ModificationTimestamp":"2019-02-03T11:18:18.300Z",
"ResourceRecordKey":"CAR28791079",
"ResourceRecordID":"CARNCM530248",
"ResourceName":"property",
"OriginatingSystemName":"carolina",
"MlgCanView":true,
"Order":"1"
}
]
}'
Table:
CREATE TABLE RE_MLSImportedGroupsOfItems (
MLSImportedGroupsOfItems_Id [int] IDENTITY(1,1) NOT NULL,
MLSImportedGroupsOfItems_ImportedContent text
)
INSERT INTO RE_MLSImportedGroupsOfItems (MLSImportedGroupsOfItems_ImportedContent)
VALUES (#json), (#json)
Statement (count all items in the JSON column):
SELECT
r.MLSImportedGroupsOfItems_Id,
Cnt = (
SELECT COUNT(*)
FROM OPENJSON(r.MLSImportedGroupsOfItems_ImportedContent, '$.value')
)
FROM RE_MLSImportedGroupsOfItems r
Statement (count all items in the JSON column, which have "#odata.id" key):
SELECT
r.MLSImportedGroupsOfItems_Id,
Cnt = (
SELECT COUNT(*)
FROM OPENJSON(r.MLSImportedGroupsOfItems_ImportedContent, '$.value')
WITH ([#odata.id] nvarchar(100) '$."#odata.id"')
WHERE [#odata.id] IS NOT NULL
)
FROM RE_MLSImportedGroupsOfItems r
We have begun to use FOR JSON PATH with in our SQL code for many different reasons. In this scenario though I'm having a little bit of issues with it. I am trying to get a list of OrderItemIds into a single JSON string. But what it produces is an array of objects with a single Property -> {"OrderItemId": "00000000-0000-0000-0000-000000000000"}. My goal is to produce a json string with just the values of the Guids and not their key.
SELECT
OrderItemId
FROM
OrderItems
FOR JSON PATH
Expected:
"["00000000-0000-0000-0000-000000000000","00000000-0000-0000-0000-000000000000"]"
Actual:
"[{"OrderItemId":"00000000-0000-0000-0000-000000000000"},{"OrderItemId":"00000000-0000-0000-0000-000000000000"}]"
Attempt 1: (Successful, but only in part)
CONCAT(
'[',
SUBSTRING(
(
SELECT ',"'+ST1.[value]+'"' AS [text()]
FROM #table ST1
FOR XML PATH ('')
), 2, 9999),
']')
Results: So this little bit of code works exactly how I would want my result to work. This is using FOR XML PATH instead of JSON PATH, which is fine of course because it produces what I want. But this bit of code is kind of daunting to toss and use all over the place where I want it. So we thought why not stick it into a function and we can pass the values to it.
Attempt 2: (AGAIN successful, but only in part) Make a custom type and function
CREATE TYPE ValueList
AS TABLE ([value] [nvarchar](100) NULL)
CREATE FUNCTION ConvertToValueList(#table ValueList READONLY)
RETURNS NVARCHAR(max)
AS BEGIN
RETURN CONCAT(
'[',
SUBSTRING(
(
SELECT ',"'+ST1.[value]+'"' AS [text()]
FROM #table ST1
FOR XML PATH ('')
), 2, 9999),
']')
Usage example:
DECLARE
#OrderItemIds ValueList;
INSERT INTO #OrderItemIds
(
[value]
)
SELECT
[OrderItemId]
FROM
[dbo].[OrderItems]
SELECT [dbo].[ConvertToValueList](#OrderItemIds)
Results:
This ends up working exactly as planned. The issue I run into now is when I am wanting to use that function in a View. I can't because I need to declare the value list in order to pass into the function.
End Note:
So for now I am just using the Concat statement from attempt 1 until we can come up with a better solution.
with XML PATH:
DECLARE #OrderItems TABLE (OrderItemId UniqueIdentifier PRIMARY KEY)
INSERT INTO #OrderItems
VALUES ( NEWID () )
, ( NEWID () )
, ( NEWID () )
, ( NEWID () )
, ( NEWID () )
SELECT CONCAT (STUFF ( ( SELECT ',' +QUOTENAME (CONVERT (VARCHAR(36), OrderItemId ),'"')
FROM #OrderItems
FOR XML PATH (''), TYPE
). value('text()[1]','varchar(max)')
, 1, 1, '"['
)
,']"'
)
;
Result:
"["135B908B-E5FD-4658-B69C-4E380509581B","4EEDF234-167D-4141-8542-A0173482BFD6","53901E4C-0486-44D5-A0AB-C6BC2FAE39B1","A263D28A-0948-461D-BC46-F9A6D167E37F","5795D046-098E-4AFB-9B2C-FB3DC56F6F31"]"
I have the following JSON data :
set #json = N'{
"Book":{
"IssueDate":"02-15-2019"
, "Detail":{
"Type":"Any Type"
, "Author":{
"Name":"Annie"
, "Sex":"Female"
}
}
, "Chapter":[
{
"Section":"1.1"
, "Title":"Hello world."
}
,
{
"Section":"1.2"
, "Title":"Be happy."
}
]
, "Sponsor":["A","B","C"]
}
}'
The expected result is
topKey Key Value
Book IssueDate 02-15-2019
Book Detail { "Type":"Any Type", "Author":{ "Name":"Annie" , "Sex":"Female"}
Book Chapter [{ "Section":"1.1", "Title":"Hello world." }, { "Section":"1.2", "Title":"Be happy." }]
Book Sponsor ["A","B","C"]
Detail Type Any Type
Detail Author { "Name":"Annie" ,"Sex":"Female"}
Author Name Annie
Author Sex Female
Chapter Section 1.1
Chapter Title Hello world
Chapter Section 1.2
Chapter Title Be happy.
I found that when the field "Value" is JSON, I need to keep parsing it.
So I created a function to do the parsing work but it returns '' which does not meet the requirement.
create function ParseJson(#json nvarchar(max))
returns #tempTable table ([key] nvarchar(max), [value] nvarchar(max))
as
begin
insert #tempTable
select
x.[key]
, x.[value]
from
openjson(#json) x
cross apply ParseJson(x.[value]) y
where ISJSON(x.[value])=1
end
A string may be passed to the function.
select * from ParseJson(#json)
I'm not sure if your expectation of the results is reasonable but clearly the returning table of your function doesn't match what you stated -- it lacks topKey column. For this reason, I'd rather aggregate the path of the hierarchy. Here we go:
create function ParseJson(
#parent nvarchar(max), #json nvarchar(max))
returns #tempTable table (
[key] nvarchar(max), [value] nvarchar(max))
as
begin
; with cte as (
select
iif(#parent is null, [key]
, concat(#parent, '.', [key])) [key]
, [value]
from
openjson(#json)
)
insert
#tempTable
select
x.*
from
cte x
union all
select
x.*
from
cte y
cross apply ParseJson(y.[key], y.[value]) x
where isjson(y.[value])=1
return
end
And the results:
I want to execute a select query on a table and I need this result in a JSON format or in an excel sheet. I want to do this using query only and I'm using SQL Server 2014.
Here is the table schema
CREATE TABLE TestTable
(
Id int primary key identity(1,1),
Name nvarchar(200),
About nvarchar(2000),
Age int,
AddressId int
)
I need to get values from Id, Name, About and Age into a JSON List
JSON AUTO would make quick work of this in but JSON support is available only in SQL Server 2016 and later, including Azure SQL Database. For a T-SQL solution in SQL 2014 and earlier, you'll need to build the JSON string yourself.
Below is an example that uses a FOR XML subquery to concatenate the result into JSON format and adds the outermost [ and ] for the array. Note that this does not handle characters that must be escaped in JSON (\ and ") so you'll need to use REPLACE to escape those if contained in your data.
SELECT '[' + STUFF((
SELECT
',{'
+ '"Id":'+CAST(Id AS varchar(10)) + ','
+ COALESCE('"Name":"' + Name + '",','')
+ COALESCE('"About":"' + About + '",','')
+ COALESCE('"Age":'+CAST(Age AS varchar(10)) + ',','')
+ COALESCE('"AddressId":'+CAST(AddressId AS varchar(10)), '')
+ '}'
FROM TestTable
FOR XML PATH(''), TYPE).value('.', 'varchar(MAX)'),1,1,'')
+ ']';
For SQL server 2017
CREATE TABLE mytable(
ID int PRIMARY KEY,
Name varchar(50),
teamName varchar(255),
Create_Date DATETIME
);
INSERT INTO mytable VALUES (1,NULL,'TEST1','2017-01-02');
INSERT INTO mytable VALUES (2,NULL,'TEST2',NULL);
INSERT INTO mytable VALUES (3,'KK','TEST3','2017-01-02');
INSERT INTO mytable VALUES (4,NULL,NULL,NULL);
Try below way here i provide an example
SELECT
ID,
Name,
teamName,
Create_Date
FROM mytable
FOR JSON AUTO
http://www.sqlfiddle.com/#!18/81350/1
JSON_F52E2B61-18A1-11d1-B105-00805F49916B
[{"ID":1,"teamName":"TEST1","Create_Date":"2017-01-02T00:00:00"},{"ID":2,"teamName":"TEST2"},{"ID":3,"Name":"KK","teamName":"TEST3","Create_Date":"2017-01-02T00:00:00"},{"ID":4}]
For below vesion of sql 2017 server:
1st create a scaler function
create FUNCTION [dbo].[udf-Str-JSON] (#IncludeHead int,#ToLowerCase int,#XML xml)
Returns varchar(max)
AS
Begin
Declare #Head varchar(max) = '',#JSON varchar(max) = ''
; with cteEAV as (Select RowNr=Row_Number() over (Order By (Select NULL))
,Entity = xRow.value('#*[1]','varchar(100)')
,Attribute = xAtt.value('local-name(.)','varchar(100)')
,Value = xAtt.value('.','varchar(max)')
From #XML.nodes('/row') As R(xRow)
Cross Apply R.xRow.nodes('./#*') As A(xAtt) )
,cteSum as (Select Records=count(Distinct Entity)
,Head = IIF(#IncludeHead=0,IIF(count(Distinct Entity)<=1,'[getResults]','[[getResults]]'),Concat('{"status":{"successful":"true","timestamp":"',Format(GetUTCDate(),'yyyy-MM-dd hh:mm:ss '),'GMT','","rows":"',count(Distinct Entity),'"},"results":[[getResults]]}') )
From cteEAV)
,cteBld as (Select *
,NewRow=IIF(Lag(Entity,1) over (Partition By Entity Order By (Select NULL))=Entity,'',',{')
,EndRow=IIF(Lead(Entity,1) over (Partition By Entity Order By (Select NULL))=Entity,',','}')
,JSON=Concat('"',IIF(#ToLowerCase=1,Lower(Attribute),Attribute),'":','"',Value,'"')
From cteEAV )
Select #JSON = #JSON+NewRow+JSON+EndRow,#Head = Head From cteBld, cteSum
Return Replace(#Head,'[getResults]',Stuff(#JSON,1,1,''))
End
-- Parameter 1: #IncludeHead 1/0
-- Parameter 2: #ToLowerCase 1/0 (converts field name to lowercase
-- Parameter 3: (Select * From ... for XML RAW)
Then use this function json conversion below query is an example
Declare #Table table (ID int,Active bit,First_Name varchar(50),Last_Name varchar(50),EMail varchar(50))
Insert into #Table values
(1,1,'John','Smith','john.smith#email.com'),
(2,0,'Jane','Doe' ,'jane.doe#email.com')
Select A.ID
,A.Last_Name
,A.First_Name
,B.JSON
From #Table A
Cross Apply (Select JSON=[dbo].[udf-Str-JSON](0,1,(Select A.* For XML Raw)) ) B
From SQL-Server 2016+, you can use JSON AUTO to return Json:
Select *
From Customers
FOR JSON AUTO;
If you expect to return just a single row, then you can add Without_Array_Wrapper after FOR JSON AUTO with a comma after the AUTO keyword, and you will get an object instead of an array:
Select *
From Customers
FOR JSON AUTO, Without_Array_Wrapper;
Note: If you add the Without_Array_Wrapper and return multiple rows, you will get a newly populated object type but won't get any error - which requires carefulness as you won't know if anything is wrong and might hit you later when exercising the data.
From SQL-Server 2016+, you can use JSON AUTO to return Json:
I know two way to return query in JSON format.
If your query is from one table or only return a single table (as usual) you can use JSON PATH or JSON AUTO
CREATE TABLE TestTable
(
Id int primary key identity(1,1),
Name nvarchar(200),
About nvarchar(2000),
Age int,
AddressId int
)
INSERT INTO dbo.TestTable(Name,About,Age,AddressId)
VALUES (N'John' , NULL ,21, 16),(N'Mehdi' , 'Developer' ,32, 15)
query is
select * from [TestTable] for JSON PATH
output :
[{
"Id": 1,
"Name": "Mehdi",
"About": "Developer",
"Age": 32,
"AddressId": 15
}, {
"Id": 3,
"Name": "John",
"Age": 21,
"AddressId": 16
}
]
Notice that NULL values do not show on result. For showing the null values, [INCLUDE_NULL_VALUES] must be added at the end of the query.
SELECT * FROM [TestTable] WHERE id=2 FOR JSON PATH , INCLUDE_NULL_VALUES
output:
[{
"Id": 3,
"Name": "John",
"About": null,
"Age": 21,
"AddressId": 16
}
]
For the query that only return an object, WITHOUT_ARRAY_WRAPPER must be added at the end of query the brackets are remove from the result.
SELECT * FROM [TestTable] WHERE id=2 FOR JSON PATH , INCLUDE_NULL_VALUES , WITHOUT_ARRAY_WRAPPER
output :
{
"Id": 3,
"Name": "John",
"About": null,
"Age": 21,
"AddressId": 16
}
For adding sub query for example the user's contracts or user's family names.
there two way :
1-
SELECT
* ,
ContractList = (SELECT* FROM contract where UserId = 2 FOR JSON PATH)
FROM TestTable FOR JSON PATH
2-
SELECT
* ,
(JSON_QUERY((SELECT* FROM contract WHERE UserId = 2 FOR JSON PATH))) AS ContractList
FROM TestTable FOR JSON PATH
output:
[{
"Id": 3,
"Name": "John",
"About": null,
"Age": 21,
"AddressId": 16 ,
"ContractList" : [{...}, {...}]
}
]
Thank you so much, your answer came to my rescue after a week-long search.
Had to make a few changes to suit my scenario and add quotes around the code
SELECT '[' + STUFF((
SELECT
',['
+ '"'+ CAST(s.Code AS varchar(10)) + '"' + ','
+ COALESCE(''+CAST(COUNT(s.ControlNumber) AS varchar(10)), '')
+ ']'
FROM school s
INNER JOIN [dbo].[Class] cc ON cc.SchoolID = s.SchoolID
INNER JOIN [dbo].[Subject] s ON s.subjectID = cc.subjectID
INNER JOIN [dbo].Marks ms ON s.MarkID = c.CaseStatusID
WHERE ms.Created BETWEEN '2021-04-01 00:00:00' AND '2021-04-07 23:59:59'
AND s.subjectID IN (1, 2, 3, 4, 5, 6, 7, 8, 9)
Group By s.Code
FOR XML PATH(''), TYPE).value('.', 'varchar(MAX)'),1,1,'')
+ ']';
Output
[["za-ec",14],["za-fs",5],["za-gt",3],["za-mp",14],["za-nc",2],["za-nl",8],["za-np",5],["za-nw",6],["za-wc",15]]