I've used a combination of CHARINDEX and SUBSTRING but can't get it working.
I get passed a variable in SQL that contains a lot of text but has an email in it. I need to extract the email value.
I have to use SQL 2008.
I'm trying to extract the value between "EmailAddress":" and ",
An example string is here:
{ "Type":test,
"Admin":test,
"User":{
"UserID":"16959191",
"FirstName":"Test",
"Surname":"Testa",
"EmailAddress":"Test.Test#test.com",
"Address":"Test"
}
}
Assuming you can't upgrade to 2016 or higher, you can use a combination of substring and charindex.
I've used a common table expression to make it less cumbersome, but you don't have to.
DECLARE #json varchar(4000) = '{ "Type":test,
"Admin":test,
"User":{
"UserID":"16959191",
"FirstName":"Test",
"Surname":"Testa",
"EmailAddress":"Test.Test#test.com",
"Address":"Test"
}
}';
WITH CTE AS
(
SELECT #Json as Json,
CHARINDEX('"EmailAddress":', #json) + LEN('"EmailAddress":') As StartIndex
)
SELECT SUBSTRING(Json, StartIndex, CHARINDEX(',', json, StartIndex) - StartIndex)
FROM CTE
Result: "Test.Test#test.com"
The first hint is: Move to v2016 if possible to use JSON support natively. v2008 is absolutely outdated...
The second hint is: Any string action (and all my approaches below will need some string actions too), will suffer from forbidden characters, unexpected blanks or any other surprise you might find within your data.
Try it like this:
First I create a mockup scenario to simulate your issue
DECLARE #tbl TABLE(ID INT IDENTITY,YourJson NVARCHAR(MAX));
INSERT INTO #tbl VALUES
(N'{ "Type":"test1",
"Admin":"test1",
"User":{
"UserID":"16959191",
"FirstName":"Test1",
"Surname":"Test1a",
"EmailAddress":"Test1.Test1#test.com",
"Address":"Test1"
}
}')
,(N'{ "Type":"test2",
"Admin":"test2",
"User":{
"UserID":"16959191",
"FirstName":"Test2",
"Surname":"Test2a",
"EmailAddress":"Test2.Test2#test.com",
"Address":"Test2"
}
}');
--Starting with v2016 there is JSON support
SELECT JSON_VALUE(t.YourJson, '$.User.EmailAddress')
FROM #tbl t
--String-methods
--use CHARINDEX AND SUBSTRING
DECLARE #FirstBorder NVARCHAR(MAX)='"EMailAddress":';
DECLARE #SecondBorder NVARCHAR(MAX)='",';
SELECT t.*
,A.Pos1
,B.Pos2
,SUBSTRING(t.YourJson,A.Pos1,B.Pos2 - A.Pos1) AS ExtractedEMail
FROM #tbl t
OUTER APPLY(SELECT CHARINDEX(#FirstBorder,t.YourJson)+LEN(#FirstBorder)) A(Pos1)
OUTER APPLY(SELECT CHARINDEX(#SecondBorder,t.YourJson,A.Pos1)) B(Pos2);
--use a XML trick
SELECT CAST('<x>' + REPLACE(REPLACE((SELECT t.YourJson AS [*] FOR XML PATH('')),'"EmailAddress":','<mailAddress value='),',',' />') + '</x>' AS XML)
.value('(/x/mailAddress/#value)[1]','nvarchar(max)')
FROM #tbl t
Some explanations:
JSON-support will parse the value directly from a JSON path.
For CHARINDEX AND SUBSTRING I use APPLY. The advantage is, that you can use the computed positions like a variable. No need to repeat the CHARINDEX statements over and over.
The XML approach will transform your JSON to a rather strange and ugly XML. The only sensefull element is <mailAddress> with an attribute value. We can use the native XML method .value() to retrieve the value you are asking for:
An intermediate XML looks like this:
<x>{ "Type":"test1" />
"Admin":"test1" />
"User":{
"UserID":"16959191" />
"FirstName":"Test1" />
"Surname":"Test1a" />
<mailAddress value="Test1.Test1#test.com" />
"Address":"Test1"
}
}</x>
I have the following JSON (only a couple of nodes for display purposes):
[
{
"CareNotes": [
{
"CareNoteID": "34289e11-6433-4020-9734-224eb8caa11a",
"CareNoteExtendedID": "00000000-0000-0000-0000-000000000000",
"ADLName": "Mobility",
"FlagsText": "",
"Note": "Help with walking, used as four wheel walker, was content.",
"AnswerType": 1,
"Fragment": "Help with walking",
"RemedialText": null,
"Details": null,
"ServiceUserID": "bc300962-3653-491a-9ba9-afab10964af4",
"ServiceUser": "Betty Test",
"ServiceUserLastName": "Test",
"ServiceUserForeNames": "Betty",
"ServiceUserDateofBirth": "19/03/1901",
"ServiceUserLocation": 15,
"WorkerID": "53e6c7b9-2c80-451e-ba8c-abfb309380ac",
"Worker": "Beth Beth",
"VoidedByWorker": null,
"_supersedeStackID": null,
"SupersededByWorker": null,
"WorkerLastName": "Beth",
"DisplayOnShiftHandover": 0,
"WorkerInitials": "B.B.",
"SliderData": "Walk",
"SliderData2": "Not entered",
"SliderIcons": [
{
"IconID": 1093,
"CareNoteText": "was content"
},
{
"IconID": 1156,
"CareNoteText": "used as four wheel walker"
}
],
"DateDone": "2019-09-30T21:24:41.994+00:00",
"DateDoneSU": "2019-09-30T21:24:41.994+00:00",
"Duration": "9 minutes",
"DurationInt": 9,
"ActionIconID": 6001,
"mraCareOrder": 5000,
"wasPlanned": false,
"qrVerified": false,
"qrData": null,
"nfcVerified": null,
"inVerified": null,
"ViaMonitor": null
}
]
}
]
I am not particularly good at SQL and have been frantically relearning what I did at uni and an old job 13yrs ago, in order to complete a project that takes JSON data from APIs of a care management solution, into my Delphi application that then processes the data to work out this and that. The JSON format is different depending on the report, and in the case of the CareNotesReport, the above JSON is what is produced.
My Delphi app pulls this JSON verbatim and dumps it to a .json file, with an ADO query that then executes the following code (along with some other select queries that are irrelevant here):
use CMUtility;
DECLARE #JSON VARCHAR(MAX)
SELECT #JSON = BulkColumn
FROM OPENROWSET
(BULK 'C:\Users\User\Documents\Embarcadero\Studio\Projects\CMU\Win32\Debug\carenotesreport.json', SINGLE_CLOB)
AS j
drop table if exists jsoncarenotes
select * into JSONCareNotes
from OPENJSON(#JSON,'$.CareNotes')
with (
DateDone nvarchar(10) '$.DateDone',
ServiceUser nvarchar(100) '$.ServiceUser',
ServiceUserLastName nvarchar(50) '$.ServiceUserLastName',
SUDOB nvarchar(15) '$.ServiceUserDateofBirth',
Note nvarchar(255) '$.Note',
ADLName nvarchar(200) '$.ADLName',
FlagsText nvarchar(255) '$.FlagsText',
Fragment nvarchar(255) '$.Fragment',
RemedialText nvarchar(255) '$.RemedialText',
Worker nvarchar(30) '$.Worker',
ServiceUserID nvarchar(100) '$.ServiceUserID',
WorkerID nvarchar(100) '$.WorkerID',
CareNoteID nvarchar(255) '$.CareNoteID',
SID1 nvarchar(255) '$.SliderIcons[0].IconID',
SText1 nvarchar(255) '$.SliderIcons[0].CareNoteText',
SID2 nvarchar(255) '$.SliderIcons[1].IconID',
SText2 nvarchar(255) '$.SliderIcons[1].CareNoteText',
SID3 nvarchar(255) '$.SliderIcons[2].IconID',
SText3 nvarchar(255) '$.SliderIcons[2].CareNoteText',
SID4 nvarchar(255) '$.SliderIcons[3].IconID',
SText4 nvarchar(255) '$.SliderIcons[3].CareNoteText',
SID5 nvarchar(255) '$.SliderIcons[4].IconID',
SText5 nvarchar(255) '$.SliderIcons[4].CareNoteText'
)
as CareNotes
I have a couple of issues. With the above code, I've had to strip the first [ and ] from the JSON file to get it to work, but due to the size of some returns I've had to change the use of a memorystream to a filestream within Delphi. This has created the problem in that although I can trim the last ] of the file, I am so far unable to find a reliable (and easy) method for trimming the first [. Therefore, I'm forced to conclude that it is my SQL code that is the weak link, and that it needs to be able to process the JSON with these two characters included.
Please can someone show me where I'm going wrong. I'm aware that the original JSON appears to be array, object, array, object, but I don't know where to go from that. Also, I've fudged the ability to read SliderIcons as I know there are a maximum of 5 objects of that array, but would prefer a more dynamic solution if possible.
Any assistance given in terms of either the way to remove that first [ in Delphi, or better SQL to handle the original JSON would be greatly appreciated.
Regards
Ant
You can parse this JSON input using SQL Server capabilities. If your JSON input has this fixed format (an array with one item and nested JSON arrays), you need an additional APPLY operator with OPENJSON() call to parse the nested JSON array. Note, that when the referenced property contains an inner JSON object or array you need to use the AS JSON option in the column definition.
JSON:
DECLARE #json nvarchar(max)
--SELECT #json = BulkColumn
--FROM OPENROWSET (BULK 'C:\Users\User\Documents\Embarcadero\Studio\Projects\CMU\Win32\Debug\carenotesreport.json', SINGLE_CLOB) AS j
SELECT #json = N'[
{
"CareNotes":[
{
"CareNoteID":"34289e11-6433-4020-9734-224eb8caa11a",
"CareNoteExtendedID":"00000000-0000-0000-0000-000000000000",
"ADLName":"Mobility",
"FlagsText":"",
"Note":"Help with walking, used as four wheel walker, was content.",
"AnswerType":1,
"Fragment":"Help with walking",
"RemedialText":null,
"Details":null,
"ServiceUserID":"bc300962-3653-491a-9ba9-afab10964af4",
"ServiceUser":"Betty Test",
"ServiceUserLastName":"Test",
"ServiceUserForeNames":"Betty",
"ServiceUserDateofBirth":"19/03/1901",
"ServiceUserLocation":15,
"WorkerID":"53e6c7b9-2c80-451e-ba8c-abfb309380ac",
"Worker":"Beth Beth",
"VoidedByWorker":null,
"_supersedeStackID":null,
"SupersededByWorker":null,
"WorkerLastName":"Beth",
"DisplayOnShiftHandover":0,
"WorkerInitials":"B.B.",
"SliderData":"Walk",
"SliderData2":"Not entered",
"SliderIcons":[
{
"IconID":1093,
"CareNoteText":"was content"
},
{
"IconID":1156,
"CareNoteText":"used as four wheel walker"
}
],
"DateDone":"2019-09-30T21:24:41.994+00:00",
"DateDoneSU":"2019-09-30T21:24:41.994+00:00",
"Duration":"9 minutes",
"DurationInt":9,
"ActionIconID":6001,
"mraCareOrder":5000,
"wasPlanned":false,
"qrVerified":false,
"qrData":null,
"nfcVerified":null,
"inVerified":null,
"ViaMonitor":null
}
]
}
]'
Statement:
SELECT
j1.DateDone,
j1.Note,
j2.IconID,
j2.CareNoteText
--INTO JSONCareNotes
FROM OPENJSON(#json, '$[0].CareNotes') WITH (
DateDone nvarchar(10) '$.DateDone',
Note nvarchar(255) '$.Note',
-- add additional columns definitons
SliderIcons nvarchar(max) AS JSON
) j1
CROSS APPLY OPENJSON(j1.SliderIcons) WITH (
IconID int '$.IconID',
CareNoteText nvarchar(100) '$.CareNoteText'
) j2
Result:
DateDone Note IconID CareNoteText
2019-09-30 Help with walking, used as four wheel walker, was content. 1093 was content
2019-09-30 Help with walking, used as four wheel walker, was content. 1156 used as four wheel walker
Notes (JSON basics):
When you want to parse JSON string and get results as table, use OPENJSON table-valued function, with default or explicit schema.
Function JSON_QUERY extracts an object or an array from a JSON string. If the value is not an object or an array, the result is NULL in lax mode and an error in strict mode.
Function JSON_VALUE extracts a scalar value from a JSON string. If the path points to not a scalar value, the result is NULL in lax mode and an error in strict mode
Notes (Delphi and SQL Server):
You can organize your logic as a stored procedure, that has one parameter - the JSON text. In this situation, you'll send the JSON directly to the SQL Server and using OPENROWSET() won't be needed (OPENJSON() needs additional permissions).
Executing a stored procedure with Delphi is an easy task using ADO for example.
i have a column in table which contain data like XML,i would like to get data in rows.
My table data as-
select printDataColumn from Mytable
It returns value-
<PrintData>
<Line1>.MERCHANT ID: *****4005</Line1>
<Line2>.CLERK ID: ADMIN</Line2>
<Line3>.</Line3>
<Line4>. VOID SALE</Line4>
<Line5>.</Line5>
<Line6>.VISA ************0006</Line6>
<Line7>.ENTRY METHOD: SWIPED</Line7>
<Line8>.DATE: 03/05/2019 TIME: 16:57:20</Line8>
<Line9>.</Line9>
<Line10>.INVOICE: 1551785225020</Line10>
<Line11>.REFERENCE: 1008</Line11>
<Line12>.AUTH CODE: 08354A</Line12>
<Line13>.</Line13>
<Line14>.AMOUNT USD$ 1.14</Line14>
<Line15>. ==========</Line15>
<Line16>.TOTAL USD$ 1.14</Line16>
<Line17>.</Line17>
<Line18>. APPROVED - THANK YOU</Line18>
<Line19>.</Line19>
<Line20>.I AGREE TO PAY THE ABOVE TOTAL AMOUNT</Line20>
<Line21>.ACCORDING TO CARD ISSUER AGREEMENT</Line21>
<Line22>.(MERCHANT AGREEMENT IF CREDIT VOUCHER)</Line22>
<Line23>.</Line23>
<Line24>.</Line24>
<Line25>.</Line25>
<Line26>.x_______________________________________</Line26>
<Line27>. Merchant Signature</Line27>
<Line28>.</Line28>
</PrintData>
but i want to use this information in another way like that
MERCHANT ID: *****4005
CLERK ID: ADMIN
SALE
AMEX ***********1006
ENTRY METHOD: CHIP
DATE: 03/07/2019 TIME: 14:37:23
INVOICE: 1551949638173
REFERENCE: 1005
AUTH CODE: 040749. . . . .and so on.
any help is appreciable.
Besides the fact, that it always is a good idea to use the appropriate type to store your data, you can use a cast on the fly to use your xml-like-data with XML methods:
DECLARE #tbl TABLE(ID INT IDENTITY,PrintData VARCHAR(4000));
INSERT INTO #tbl VALUES
('<PrintData>
<Line1>.MERCHANT ID: *****4005</Line1>
<Line2>.CLERK ID: ADMIN</Line2>
<Line3>.</Line3>
<Line4>. VOID SALE</Line4>
<!-- more lines -->
</PrintData>');
SELECT t.ID
,A.Casted.value(N'(/PrintData/Line1/text())[1]','nvarchar(max)') AS Line1
FROM #tbl t
CROSS APPLY(SELECT CAST(t.PrintData AS XML)) A(Casted);
In this case I use CROSS APPLY to add a column A.Casted to the result set, which is a row-wise casted XML.
This will break, in cases of invalid XML (of course). You might try TRY_CAST instead. This would return NULL, but will hide data errors...
Some more background
The cast to XML is a rather expensive operation. Doing this whenever you want to read out of your data is some heavy load for your server. Furthermore, using VARCHAR is prone to two major errors:
If there are foreign characters you might get question marks
If the XML is not valid, you will not see it - until you use it.
If possible, try to change the table's design to use native XML.
And one more hint
It is a bad approach to name-number elements (same for columns). Instead of <Line1><Line2><Line3> better use <Line nr="1"><Line nr="2"><Line nr="3">...
I have a logs message that I have to extract columns from with a sql query,
this how the message looks like:
"device=EOHCS-ZA-JIS-FW severity=high from=EOHCloudFAZ(FL1KVM0000005594) trigger=Syslog Critical System Alerts log="logver=54 itime=1528457940 devid=FG1K5D3I13800425 devname=FWJIS01 vd=95_LHC date=2018-06-08 time=13:34:55 logid=0100044546 type=event subtype=system level=information logdesc="Attribute configured" user="JoshuaK" ui="ha_daemon" action=Edit cfgtid=701760128 cfgpath="system.settings" cfgattr="gui-allow-unnamed-policy[disable->enable]" msg="Edit system.settings """
can someone give me an idea
I have a solution for SQL-Server, you could use PATINDEX and extract the log message.
Below is the code to extract from value
declare #input nVarchar(max),#from nVarchar(MAX)
declare #FromStart int,#FromEnd int
set #input='device=EOHCS-ZA-JIS-FW severity=high from=EOHCloudFAZ(FL1KVM0000005594) trigger=Syslog Critical System Alerts log="logver=54 itime=1528457940 devid=FG1K5D3I13800425 devname=FWJIS01 vd=95_LHC date=2018-06-08 time=13:34:55 logid=0100044546 type=event subtype=system level=information logdesc="Attribute configured" user="JoshuaK" ui="ha_daemon" action=Edit cfgtid=701760128 cfgpath="system.settings" cfgattr="gui-allow-unnamed-policy[disable->enable]" msg="Edit system.settings';
SET #FromStart=PATINDEX('%from=%',#input)+5;
SET #FromEnd=PATINDEX('% trigger=%',#input)-#FromStart;
SELECT #from=SUBSTRING(#input,#FromStart,#FromEnd)
SELECT #from
Note : use equivalent of PATINDEX for your corresponding DB server. Also note that this works only if the input string have parameters in a defined order.
In SQL Server 2017, I'd like to "SELECT" a JSON object embedded within another as a string so we can store/process them later.
eg JSON:
[
{"key1":"value1",
"level2_Obj":{"key2":"value12"}
},
{"key1":"value2",
"level2_Obj":{"key22":"value22"}
},
]
From above JSON, I'd like to SELECT whole of the level2Obj JSON object, see below for what I'd like to see the "selection" result.
value1 |{"key2" :"value12"}
value2 |{"key22":"value22"}
I tried below with no luck:
SELECT * FROM
OPENJSON(#json,'$."data1"')
WITH(
[key1] nvarchar(50),
[embedded_json] nvarchar(max) '$."level2Obj"'
) AS DAP
Can some one please help how I select the contents of the 2nd level JSON object as a string?
The idea is to Write 1st level JSON properties into individual cells and rest of JSON levels into a single column of type nvarchar(max) (i.e whole of sub-level JSON object into a single column as a string for further processing in later stages).
Good day,
Firstly, Your JSON text is not properly formatted. There is extra comma after the last object in the array. I will remove this extra comma for the sake of the answer, but if this is the format you have then first step will be to clear the text and make sure that is is well formatted.
Please check if this solve your needs:
declare #json nvarchar(MAX) = '
[
{
"key1":"value1",
"level2_Obj":{"key2":"value12"}
}
,
{
"key1":"value2",
"level2_Obj":{"key22":"value22"}
}
]
'
SELECT JSON_VALUE (t1.[value], '$."key1"'), JSON_QUERY (t1.[value], '$."level2_Obj"')
FROM OPENJSON(#json,'$') t1