Parse XML data from a column in SQL Server - sql

I'm trying to put together a report for a badge program. Some the data for custom fields we created are stored in a single column in the table as XML. I need to return a couple of the items in there to the report and am having a hard time getting the proper syntax to get it to parse out.
Aside from the XML, the query itself is simple:
SELECT
Person1.firstName
,Person1.lastName
,Person1.idNumber
,Person1.idNumber2
,Person1.idNumber3
,Person1.status
,Person1.customdata
FROM
Person1
the field "customdata" is the XML field that I need to pull Title, and 2 different dates out of. This is what the XML looks like:
<person1_7:CustomData xmlns:person1_7="http://www.badgepass.com/Person1_7">
<Title>IT Director</Title>
<Gaming_x0020_Level>Level 1</Gaming_x0020_Level>
<Gaming_x0020_Issue_x0020_Date>2021-02-18T12:00:00Z</Gaming_x0020_Issue_x0020_Date>
<Gaming_x0020_Expire_x0020_Date>2022-02-18T12:00:00Z</Gaming_x0020_Expire_x0020_Date>
<Betting_x0020_Level>Level 1</Betting_x0020_Level>
<Betting_x0020_Issue_x0020_Date>2021-02-18T12:00:00Z</Betting_x0020_Issue_x0020_Date>
<Betting_x0020_Expire_x0020_Date>2022-02-18T12:00:00Z</Betting_x0020_Expire_x0020_Date>
<BadgeType>Dual Employee</BadgeType>
<Gaming_x0020_Status>TEMP</Gaming_x0020_Status>
<Betting_x0020_Status>TEMP</Betting_x0020_Status>
</person1_7:CustomData>
I have tried a couple of different methods trying to follow the advice from How to query for Xml values and attributes from table in SQL Server? and then tried declaring a XML namespace with the following query:
WITH XMLNAMESPACES ('http://www.badgepass.com/Person1_7' as X)
SELECT
Person1.firstName
,Person1.lastName
,Person1.idNumber
,Person1.idNumber2
,Person1.idNumber3
,Person1.status
,Person1.customdata.value('(/X:person1_7:customdata/X:Title)[1]', 'varchar(100)') AS Title
FROM
Person1
So far all of my results keep returning "XQuery [Person1.customData.value()]: ")" was expected.
" I'm assuming I have a syntax issue that I'm overlooking as I've never had to manipulate XML with SQL before. Thank you in advance for any help.

Please try the following solution.
Notable points:
XQuery .nodes() method establishes a context so you can access any
XML element right away without long XPath expressions.
Use of the text() in the XPath expressions is for performance
reasons.
SQL
-- DDL and sample data population, start
DECLARE #person1 TABLE (firstname varchar(50), customdata xml);
INSERT INTO #person1(firstname, customdata) VALUES
('John', '<person1_7:CustomData xmlns:person1_7="http://www.badgepass.com/Person1_7">
<Title>IT Director</Title>
<Gaming_x0020_Level>Level 1</Gaming_x0020_Level>
<Gaming_x0020_Issue_x0020_Date>2021-02-18T12:00:00Z</Gaming_x0020_Issue_x0020_Date>
<Gaming_x0020_Expire_x0020_Date>2022-02-18T12:00:00Z</Gaming_x0020_Expire_x0020_Date>
<Betting_x0020_Level>Level 1</Betting_x0020_Level>
<Betting_x0020_Issue_x0020_Date>2021-02-18T12:00:00Z</Betting_x0020_Issue_x0020_Date>
<Betting_x0020_Expire_x0020_Date>2022-02-18T12:00:00Z</Betting_x0020_Expire_x0020_Date>
<BadgeType>Dual Employee</BadgeType>
<Gaming_x0020_Status>TEMP</Gaming_x0020_Status>
<Betting_x0020_Status>TEMP</Betting_x0020_Status>
</person1_7:CustomData>');
-- DDL and sample data population, end
WITH XMLNAMESPACES ('http://www.badgepass.com/Person1_7' as person1_7)
SELECT firstName
, c.value('(Title/text())[1]', 'VARCHAR(100)') AS Title
, c.value('(Gaming_x0020_Issue_x0020_Date/text())[1]', 'DATETIME') GamingIssueDate
FROM #person1
CROSS APPLY customdata.nodes('/person1_7:CustomData') AS t(c);
Output
+-----------+-------------+-------------------------+
| firstName | Title | GamingIssueDate |
+-----------+-------------+-------------------------+
| John | IT Director | 2021-02-18 12:00:00.000 |
+-----------+-------------+-------------------------+

Related

Finding every instance of XML Element in SQL output

I'm a beginner when it comes to SQL and have no experience with XML so I'm after a little bit of help.
At the moment I am looking at a single table and just using the query below
select
name,
convert(xml, convert(varbinary(max), orders)) ClientOrders
from client;
In the second columns of SQL output, I have a very lengthy bit of XML similar to the example below. I've used "..." just to skip over some of the output and give a general idea.
Name
ClientOrders
Client1
<report ... ><QueryParameter></QueryParameter Name = "#hello1"><commandtext> ...<value>Example1</value>....<value>Example2</value>...<value>Example3</value>...</commandtext></report>
Client2
<report ... ><QueryParameter></QueryParameter Name = "#hello2"><commandtext> ...<value>Example4</value>....<value>Example5</value>...<value>Example6</value>...</commandtext></report>
I have this for a lot of rows and this output is so long that it exceeds the Excel cell character limit. I'm only looking for the values Example1 through to Example6 in the example given above. Is there an SQL command I can run to get the above string between the open and close value?
I am using SSMS version 18.9.1
Cheers

In SQL, in XML datatype field, how do I query the data like a simple sql record?

I have one XML field in table.
It stores data as
'<NewDataSet>
<ClaimExpense>
<ClaimNo>3003-LOB-0003</ClaimNo>
<Office>3003</Office>
<BranchId>1</BranchId>
<CostCenterId>35</CostCenterId>
<ServiceLineId>14</ServiceLineId>
<ProjectId>62</ProjectId>
<LCAmountCurr>AED</LCAmountCurr>
<LCAmount>367.25</LCAmount>
<FCCurr>USD</FCCurr>
<FCAmount>100</FCAmount>
<ExchangeRate>3.67252</ExchangeRate>
<ExpenseDate>2020-11-03T00:00:00+04:00</ExpenseDate>
<ClaimItemNo>ITM-004</ClaimItemNo>
<GLAccount>10000000</GLAccount>
<VatRate>5%</VatRate>
<VatBaseAmount>349.76</VatBaseAmount>
<VatAmount>17.49</VatAmount>
<ClaimType>LOB</ClaimType>
<ForPayment>357.25</ForPayment>
<ForDeduction>0.00</ForDeduction>
<EmpCode>2019-1194</EmpCode>
</ClaimExpense>
<ClaimExpense>
<ClaimNo>3003-LOB-0003</ClaimNo>
<Office>3003</Office>
<BranchId>1</BranchId>
<CostCenterId>35</CostCenterId>
<ServiceLineId>14</ServiceLineId>
<ProjectId>62</ProjectId>
<LCAmountCurr>AED</LCAmountCurr>
<LCAmount>90.00</LCAmount>
<FCCurr>AED</FCCurr>
<FCAmount>90</FCAmount>
<ExchangeRate>1</ExchangeRate>
<ExpenseDate>2020-11-03T00:00:00+04:00</ExpenseDate>
<ClaimItemNo>ITM-005</ClaimItemNo>
<GLAccount>10000000</GLAccount>
<VatRate />
<ClaimType>LOB</ClaimType>
<ForPayment>357.25</ForPayment>
<ForDeduction>0.00</ForDeduction>
<EmpCode>2019-1194</EmpCode>
</ClaimExpense>
</NewDataSet>'
Now I want to simply query it like an sql record how do I do that?
I tried with conversion and everything but no luck with that. I just want it be simply like any sql record which can be selected, inserted deleted.
Tired but no luck with this:
SELECT
Tbl.Col.value('GLAccount[0]', 'varchar')
FROM #xml.nodes('/NewDataSet/ClaimExpense/GLAccount') Tbl(Col)
Check it out how to do it for your XML.
SQL Fiddle
SQL
CREATE table tbl (ID INT IDENTITY PRIMARY KEY, xmldata XML);
insert into tbl (xmldata) values
(
N'<NewDataSet>
<ClaimExpense>
<ClaimNo>3003-LOB-0003</ClaimNo>
<Office>3003</Office>
<BranchId>1</BranchId>
<CostCenterId>35</CostCenterId>
<ServiceLineId>14</ServiceLineId>
<ProjectId>62</ProjectId>
<LCAmountCurr>AED</LCAmountCurr>
<LCAmount>367.25</LCAmount>
<FCCurr>USD</FCCurr>
<FCAmount>100</FCAmount>
<ExchangeRate>3.67252</ExchangeRate>
<ExpenseDate>2020-11-03T00:00:00+04:00</ExpenseDate>
<ClaimItemNo>ITM-004</ClaimItemNo>
<GLAccount>10000000</GLAccount>
<VatRate>5%</VatRate>
<VatBaseAmount>349.76</VatBaseAmount>
<VatAmount>17.49</VatAmount>
<ClaimType>LOB</ClaimType>
<ForPayment>357.25</ForPayment>
<ForDeduction>0.00</ForDeduction>
<EmpCode>2019-1194</EmpCode>
</ClaimExpense>
<ClaimExpense>
<ClaimNo>3003-LOB-0003</ClaimNo>
<Office>3003</Office>
<BranchId>1</BranchId>
<CostCenterId>35</CostCenterId>
<ServiceLineId>14</ServiceLineId>
<ProjectId>62</ProjectId>
<LCAmountCurr>AED</LCAmountCurr>
<LCAmount>90.00</LCAmount>
<FCCurr>AED</FCCurr>
<FCAmount>90</FCAmount>
<ExchangeRate>1</ExchangeRate>
<ExpenseDate>2020-11-03T00:00:00+04:00</ExpenseDate>
<ClaimItemNo>ITM-005</ClaimItemNo>
<GLAccount>10000000</GLAccount>
<VatRate/>
<ClaimType>LOB</ClaimType>
<ForPayment>357.25</ForPayment>
<ForDeduction>0.00</ForDeduction>
<EmpCode>2019-1194</EmpCode>
</ClaimExpense>
</NewDataSet>');
select c.value('(ClaimNo/text())[1]', 'VARCHAR(20)') as ClaimNo
-- everything in between with proper data types
, c.value('(EmpCode/text())[1]', 'VARCHAR(10)') as EmpCode
from tbl cross apply xmldata.nodes('/NewDataSet/ClaimExpense') t(c);
As I mention in the comments, you need to use the names of your nodes; your XML has no node "row" so of course SQL Server isn't going to find any data. I also recommend use the text() function, as it is far more efficient. For example:
SELECT NDS.CE.value('(ClaimNo/text())[1]','int') AS ClaimNo
FROM #XML.nodes('NewDataSet/ClaimExpense') NDS(CE);

Parse a varchar column containing XML like data into row wise

i have a column in table which contain data like XML,i would like to get data in rows.
My table data as-
select printDataColumn from Mytable
It returns value-
<PrintData>
<Line1>.MERCHANT ID: *****4005</Line1>
<Line2>.CLERK ID: ADMIN</Line2>
<Line3>.</Line3>
<Line4>. VOID SALE</Line4>
<Line5>.</Line5>
<Line6>.VISA ************0006</Line6>
<Line7>.ENTRY METHOD: SWIPED</Line7>
<Line8>.DATE: 03/05/2019 TIME: 16:57:20</Line8>
<Line9>.</Line9>
<Line10>.INVOICE: 1551785225020</Line10>
<Line11>.REFERENCE: 1008</Line11>
<Line12>.AUTH CODE: 08354A</Line12>
<Line13>.</Line13>
<Line14>.AMOUNT USD$ 1.14</Line14>
<Line15>. ==========</Line15>
<Line16>.TOTAL USD$ 1.14</Line16>
<Line17>.</Line17>
<Line18>. APPROVED - THANK YOU</Line18>
<Line19>.</Line19>
<Line20>.I AGREE TO PAY THE ABOVE TOTAL AMOUNT</Line20>
<Line21>.ACCORDING TO CARD ISSUER AGREEMENT</Line21>
<Line22>.(MERCHANT AGREEMENT IF CREDIT VOUCHER)</Line22>
<Line23>.</Line23>
<Line24>.</Line24>
<Line25>.</Line25>
<Line26>.x_______________________________________</Line26>
<Line27>. Merchant Signature</Line27>
<Line28>.</Line28>
</PrintData>
but i want to use this information in another way like that
MERCHANT ID: *****4005
CLERK ID: ADMIN
SALE
AMEX ***********1006
ENTRY METHOD: CHIP
DATE: 03/07/2019 TIME: 14:37:23
INVOICE: 1551949638173
REFERENCE: 1005
AUTH CODE: 040749. . . . .and so on.
any help is appreciable.
Besides the fact, that it always is a good idea to use the appropriate type to store your data, you can use a cast on the fly to use your xml-like-data with XML methods:
DECLARE #tbl TABLE(ID INT IDENTITY,PrintData VARCHAR(4000));
INSERT INTO #tbl VALUES
('<PrintData>
<Line1>.MERCHANT ID: *****4005</Line1>
<Line2>.CLERK ID: ADMIN</Line2>
<Line3>.</Line3>
<Line4>. VOID SALE</Line4>
<!-- more lines -->
</PrintData>');
SELECT t.ID
,A.Casted.value(N'(/PrintData/Line1/text())[1]','nvarchar(max)') AS Line1
FROM #tbl t
CROSS APPLY(SELECT CAST(t.PrintData AS XML)) A(Casted);
In this case I use CROSS APPLY to add a column A.Casted to the result set, which is a row-wise casted XML.
This will break, in cases of invalid XML (of course). You might try TRY_CAST instead. This would return NULL, but will hide data errors...
Some more background
The cast to XML is a rather expensive operation. Doing this whenever you want to read out of your data is some heavy load for your server. Furthermore, using VARCHAR is prone to two major errors:
If there are foreign characters you might get question marks
If the XML is not valid, you will not see it - until you use it.
If possible, try to change the table's design to use native XML.
And one more hint
It is a bad approach to name-number elements (same for columns). Instead of <Line1><Line2><Line3> better use <Line nr="1"><Line nr="2"><Line nr="3">...

Reading dynamic XML nodes in SQL Server

I have the following XML structure:
set #MailXML =
'<MailingCompany>
<Mailman>
<Name>Jamie</Name>
<Age> 24 </Age>
<Letter>
<DestinationAddress> 440 Mountain View Parade </DestinationAddress>
<DestinationCountry> USA </DestinationCountry>
<OriginCountry> Australia </OriginCountry>
<OriginAddress> 120 St Kilda Road </OriginAddress>
</Letter>
</Mailman>
</MailingCompany>'
My SQL currently looks like this:
-- Mail Insertion
INSERT INTO mailDB.dbo.Mailman
SELECT
m.value('Name[1]','varchar(50)') as Name,
m.value('Age[1]','varchar(50)') as Age
FROM
#MailXML.nodes('/MailingCompany/Mailman') as A(m)
SET #MailPersonFK = SCOPE_IDENTITY();
-- Letter Insertion
INSERT INTO mailDB.dbo.Letter
SELECT
l.value('DestinationAddress[1]', 'varchar(50)') as DestinationAddress,
l.value('DestinationCountry[1]', 'varchar(50)') as DestinationCountry,
l.value('OriginCountry[1]', 'varchar(50)') as OriginCountry,
l.value('OriginAddress[1]', 'varchar(50)') as OriginAddress
#MailPersonFK as MailID
FROM
#MailXML.nodes('MailingCompany/Mailman/Letter') as B(l)
I am trying to extract the Mailman and Letter data into their own respective tables. I have got that working however my issue is that the MailCompany node is dynamic. Sometimes it may be MailVehicle, for example, and I still need
to read the corresponding Mailman and Letter node data and insert them into their own respective tables.
So both
FROM #MailXML.nodes('/MailingCompany/Mailman') as A(t)
and
FROM #MailXML.nodes('MailingCompany/Mailman/Letter') as B(l)
Will need to be changed to allow MailingCompany to be dynamic.
I have tried to extract the parent node and concatenate it into a string to put into the .nodes function like the following:
set #DynXML = '/' + #parentNodeVar + '/Mailman'
FROM #MailXML.nodes(#DynXML) as A(t)
However I get the following error:
The argument 1 of the XML data type method "nodes" must be a string literal.
How can I overcome this dynamic XML issue?
Thank you very much in advance
Look at this reduced example:
DECLARE #xml1 XML=
N'<MailingCompany>
<Mailman>
<Name>Jamie</Name>
<Letter>
<DestinationAddress> 440 Mountain View Parade </DestinationAddress>
</Letter>
</Mailman>
</MailingCompany>';
DECLARE #xml2 XML=
N'<OtherName>
<Mailman>
<Name>Jodie</Name>
<Letter>
<DestinationAddress> This is the other address </DestinationAddress>
</Letter>
</Mailman>
</OtherName>';
SELECT #xml1.value(N'(*/Mailman/Name)[1]','nvarchar(max)') AS Mailman_Name
,#xml1.value(N'(*/Mailman/Letter/DestinationAddress)[1]','nvarchar(max)') AS DestinationAddress
SELECT #xml2.value(N'(*/Mailman/Name)[1]','nvarchar(max)') AS Mailman_Name
,#xml2.value(N'(*/Mailman/Letter/DestinationAddress)[1]','nvarchar(max)') AS DestinationAddress
You can replace a node's name with *.
Another trick is the deep search with // (same result as before):
SELECT #xml1.value(N'(//Name)[1]','nvarchar(max)') AS Mailman_Name
,#xml1.value(N'(//DestinationAddress)[1]','nvarchar(max)') AS DestinationAddress
SELECT #xml2.value(N'(//Name)[1]','nvarchar(max)') AS Mailman_Name
,#xml2.value(N'(//DestinationAddress)[1]','nvarchar(max)') AS DestinationAddress
The general rule: Be as specific as possible.

Using Fields[0].Value to get XML from FOR XML RAW, ELEMENTS query is messed up

I have a query that uses FOR XML RAW, ELEMENTS to return a SELECT query as a structured XML document. However, when I get the result using a TSQLDataSet by using Fields[0].Value, the result is different from what I see when I run the query in SQL Server Management Studio.
What I see in the result from the TSQLDataSet:
੄customerIdфname၄governmentNumberไdebtorAddress1ไdebtorAddress2ไdebtorAddress3ไdebtorAddress4ࡄpostCodeୄcontactNameՄphonë́faxൄcustomerSinceՄtermsلactiveไcurrentBalanceلDebtorခŁ䄁ഃӤ
What I see in the result in SSMS:
<Debtor>
<customerId>C0E449E5B2C </customerId>
<name>New Customer 2 </name>
<governmentNumber> </governmentNumber>
<debtorAddress1>Address Line 1 </debtorAddress1>
<debtorAddress4>Address Line 4 </debtorAddress4>
<postCode>1234 </postCode>
<phone>1234567890 </phone>
<fax>1234567890 </fax>
<customerSince>2013-06-10T18:16:06.213</customerSince>
<terms>M </terms>
<active>true</active>
<currentBalance>0.0000</currentBalance>
</Debtor>
Is there a particular way it should be executed to get the right result?
AFAIK this is a DbExpress limitation. I know how overcome this, but using ADO (the returned data must be requested using a special parametrized object and a set of ADO streams). However you can use a workaround converting the XML data to a string in the server side sorrounding the sentence with a select (subquery) or just using a simple CAST statement.
For example if you sentence is like so
SELECT Foo, Bar FROM FooTable FOR XML RAW, ELEMENTS
you can rewrite to
SELECT (SELECT Foo, Bar FROM FooTable FOR XML RAW, ELEMENTS)
or you can rewrite to (use a CAST VARCHAR or NVARCHAR)
SELECT CAST( (SELECT Foo, Bar FROM FooTable FOR XML RAW, ELEMENTS) AS VARCHAR(MAX))
and finally
Retrieve the result like this
SQLDataSet1.Fields[0].AsString