Extracting child node value from XML using T-SQL - sql

I'm stuck on trying to get the 'availability' node's value out of an envelope returned via T-SQL from a Microsoft Lync database. The usual methods of .value('(/MyElement/Something)[1]') doesn't seem to work for me.
<state xsi:type="aggregateState" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://schemas.microsoft.com/2006/09/sip/state">
<availability>3500
</availability>
<delimiter xmlns="http://schemas.microsoft.com/2006/09/sip/commontypes" />
<timeZoneBias>-60
</timeZoneBias>
<timeZoneName>GMT Daylight Time
</timeZoneName>
<timeZoneAbbreviation>GMT Daylight Time
</timeZoneAbbreviation>
<device>computer
</device>
<end xmlns="http://schemas.microsoft.com/2006/09/sip/commontypes" />
</state>
This is the query I've been experimenting with:
SELECT TOP 1
CAST(SUBSTRING(i.Data, 0, 4000) as XML).value('(/state//availability)[1]', 'varchar(256)')
FROM dbo.PublishedCategoryInstanceView AS i
INNER JOIN dbo.CategoryDef AS d
ON (d.CategoryId = i.CategoryId)
WHERE i.PublisherId = (SELECT ResourceId FROM dbo.Resource
WHERE UserAtHost = 'my.email#mydomain.local')
ORDER BY i.LastPubTime DESC
All I get back is 'NULL' unless I do CAST(SUBSTRING(i.Data, 0, 4000) as XML).value('(/)[1]', 'varchar(256)') which returns 3500-60GMT Daylight TimeGMT Daylight Timecomputer
I do know that when I strip out the three attributes on the state element I can perform normal XML queries against the data so I can get around this by manipulating the string with a few replace statements but I'd rather learn exactly what I'm doing wrong here, if anyone can help?

You're just plain ignoring the XML namespace that exists on your XML root node:
<state xsi:type="aggregateState"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns="http://schemas.microsoft.com/2006/09/sip/state">
******************************************************
You need to include that in your T-SQL query!
Try something like this:
;WITH XMLNAMESPACES(DEFAULT 'http://schemas.microsoft.com/2006/09/sip/state')
SELECT TOP 1
CAST(SUBSTRING(i.Data, 0, 4000) as XML).value('(/state//availability)[1]', 'varchar(256)')
FROM
dbo.PublishedCategoryInstanceView AS i

Related

Fetch a value from a column which contains XML

How to fetch a value from a column which contains XML in SQL Server?
below is my sample XML column value and the id's can be swap anytime(101,100) or (201,100,101,321).
<Questions>
<item id="101">Yes</item>
<item id="100">No</item>
</Questions>
I want to fetch a value based on Id. Like fetching Yes from id=101.
Sample code much appreciated.
I tried with below sample, but Unable to retrieve value "Yes"
select Y.value('#item[1]','varchar[3]') as valT from tbl_storeXML s cross apply s.Questions.nodes('Questions/item') as X(Y) where e.empId=256 and Y.value('#id','int')=101
Please help on this.
Ps. It's not a home work, I am learning handling xml in sql server.
Use of the value is not done correct, you do:
Y.value('#id','int')
This should be: Y.value('(#id)[1]','int')
round braces around #id, see: docs: value() Method
and Y.value('item[1]','varchar[3]').
This should be: Y.value('(#item)[1]','varchar(3)').
The # is removed because item is not an attribute
varchar should have round braces, not square braces.
Your try, after changes will become:
select
Y.value('(item)[1]','varchar(3)') as valT
from tbl_storeXML s
cross apply s.Questions.nodes('Questions/item') as X(Y)
where e.empId=256 and Y.value('(#id)','int')=101
This is not tested, because I do not have those tables. (I do think Y.value('(item)[1]','varchar(3)') might need to be written as Y.value('(.)[1]','varchar(3)') )
But the same approach can be seen in this DBFIDDLE
DECLARE #xml XML = '<Questions>
<item id="101">Yes</item>
<item id="100">No</item>
</Questions>';
select
X.y.value('(#id)[1]','VARCHAR(20)') id,
X.y.value('(.)[1]','VARCHAR(20)') value
from #xml.nodes('Questions/item') as X(y);
output:
id
value
101
Yes
100
No

Can't parse XML with outer apply

I have an XML column in a table which i am trying to parse out values from to flat table structure.
I am trying to input the XML here but stackoverflow ses it as code and when i try and format as code it still won't accept it.
I can't even get data from "Header" level.
<RequestMessage xmlns="http://iec.ch/TC57/2011/schema/message" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="Message.xsd">
<Header>
<Verb>created</Verb>
<Noun>MeterReadings</Noun>
<Timestamp>2021-03-08T00:57:18+01:00</Timestamp>
<Source>Ipsum Lorum</Source>
<AsyncReplyFlag>true</AsyncReplyFlag>
<AckRequired>true</AckRequired>
<MessageID>Ipsum Lorum</MessageID>
<CorrelationID />
</Header>
<Payload>
<MeterReadings xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#" xmlns="http://iec.ch/TC57/2011/MeterReadings#">
<MeterReading>
<IntervalBlocks>
<IntervalReadings>
<timeStamp>2021-03-07T01:00:00+01:00</timeStamp>
<value>480.196</value>
<ReadingQualities>
<ReadingQualityType ref="3.0.0" />
</ReadingQualities>
</IntervalReadings>
<IntervalReadings>
<ReadingType ref="11.0.7.3.1.2.12.1.1.0.0.0.0.101.0.3.72.0" />
</IntervalReadings>
</IntervalBlocks>
<Meter>
<mRID>0000000000000</mRID>
<status>
<remark>Ipsum Lorum</remark>
<value>ESP</value>
</status>
</Meter>
<UsagePoint>
<mRID>73599900000000</mRID>
</UsagePoint>
</MeterReading>
</MeterReadings>
</Payload>
</RequestMessage>
I am not able to parse it and i have tried using examples from other threads. I am trying to not use OPENXML solution because requires DECLARE and executing the built in procedure for clearing the XML from memmory periodically. I am trying to use the OUTER APPLY solution.
Like Shugos solution in How to parse XML data in SQL server table or Query XML with nested nodes on Cross Apply.
It doesn't work.
It returns null for the timestamp column.
select
t.file_created_time
,c.value('(Timestamp)[1]','varchar(max)') as timestamp
from load.t t
OUTER APPLY t.xml_data.nodes('RequestMessage/Header') as m(c)
Please try the following solution.
Starting from SQL Server 2005 onwards, it is better to use XQuery language, based on the w3c standards, while dealing with the XML data type.
Microsoft proprietary OPENXML and its companions sp_xml_preparedocument and sp_xml_removedocument are kept just for backward compatibility with the obsolete SQL Server 2000. Their use is diminished just to very few fringe cases.
I had to comment out the following tag <!--<IntervalReadings>--> to make your XML well-formed.
XML Header fragment has a default namespace:
xmlns="http://iec.ch/TC57/2011/schema/message"
XML Payload fragment has its own two additional namespaces:
xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#"
xmlns="http://iec.ch/TC57/2011/MeterReadings#"
Namespaces should be taken into account.
Check it out below.
SQL
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, xml_data XML);
INSERT INTO #tbl (xml_data) VALUES
(N'<RequestMessage xmlns="http://iec.ch/TC57/2011/schema/message"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="Message.xsd">
<Header>
<Verb>created</Verb>
<Noun>MeterReadings</Noun>
<Timestamp>2021-03-08T00:57:18+01:00</Timestamp>
<Source>Ipsum Lorum</Source>
<AsyncReplyFlag>true</AsyncReplyFlag>
<AckRequired>true</AckRequired>
<MessageID>Ipsum Lorum</MessageID>
<CorrelationID/>
</Header>
<Payload>
<MeterReadings xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#"
xmlns="http://iec.ch/TC57/2011/MeterReadings#">
<MeterReading>
<IntervalBlocks>
<IntervalReadings>
<timeStamp>2021-03-07T01:00:00+01:00</timeStamp>
<value>480.196</value>
<ReadingQualities>
<ReadingQualityType ref="3.0.0"/>
</ReadingQualities>
</IntervalReadings>
<!--<IntervalReadings>-->
<ReadingType ref="11.0.7.3.1.2.12.1.1.0.0.0.0.101.0.3.72.0"/>
</IntervalBlocks>
<Meter>
<mRID>0000000000000</mRID>
<status>
<remark>Ipsum Lorum</remark>
<value>ESP</value>
</status>
</Meter>
<UsagePoint>
<mRID>73599900000000</mRID>
</UsagePoint>
</MeterReading>
</MeterReadings>
</Payload>
</RequestMessage>');
-- DDL and sample data population, end
WITH XMLNAMESPACES(DEFAULT 'http://iec.ch/TC57/2011/schema/message')
SELECT id
, c.value('(Noun/text())[1]','VARCHAR(30)') AS Noun
, c.value('(Timestamp/text())[1]','DATETIMEOFFSET(0)') AS [timestamp]
FROM #tbl
CROSS APPLY xml_data.nodes('/RequestMessage/Header') AS t(c);
Output
+----+---------------+----------------------------+
| id | Noun | timestamp |
+----+---------------+----------------------------+
| 1 | MeterReadings | 2021-03-08 00:57:18 +01:00 |
+----+---------------+----------------------------+
You need to respect and include the XML namespace in your XML document in your XQuery!
<RequestMessage xmlns="http://iec.ch/TC57/2011/schema/message"
**********************************************
Try something like this:
WITH XMLNAMESPACES(DEFAULT N'http://iec.ch/TC57/2011/schema/message')
SELECT
t.id,
c.value('(Timestamp)[1]','varchar(max)') as timestamp
FROM
load.t t
CROSS APPLY
t.xml_data.nodes('RequestMessage/Header') AS m(c)
Also when trying to run this on my SQL Server, I get an error that the XML as shown is malformed.....
UPDATE:
If you need to also access bits in the Payload section - you need to also respect that XML namespace there:
<MeterReadings xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#"
xmlns="http://iec.ch/TC57/2011/MeterReadings#">
***********************************************
Try this:
WITH XMLNAMESPACES(N'http://iec.ch/TC57/2011/schema/message' as hdr,
N'http://iec.ch/TC57/2011/MeterReadings#' as mr)
SELECT
t.id,
c.value('(hdr:Timestamp)[1]', 'varchar(50)') AS timestamp,
col.value('(mr:MeterReading/mr:IntervalBlocks/mr:IntervalReadings/mr:timeStamp)[1]', 'varchar(50)') AS MeterReadingsTimestamp
FROM
load.t t
CROSS APPLY
t.xml_data.nodes('/hdr:RequestMessage/hdr:Header') AS m(c)
CROSS APPLY
t.xml_data.nodes('/hdr:RequestMessage/hdr:Payload/mr:MeterReadings') AS mr(col)

How do I get a value from XML column in SQL?

So, I have a table with a large chunk of data stored in XML.
The partial XML schema (down to where I need) looks like this:
<DecisionData>
<Customer>
<SalesAttemptNumber />
<SubLenderID>IN101_CNAC</SubLenderID>
<DecisionType>Decision</DecisionType>
<DealerID />
<CustomerNumber>468195994772076</CustomerNumber>
<CustomerId />
<ApplicationType>Personal</ApplicationType>
<ApplicationDate>9/16/2008 11:32:07 AM</ApplicationDate>
<Applicants>
<Applicant PersonType="Applicant">
<CustNum />
<CustomerSSN>999999999</CustomerSSN>
<CustLastName>BRAND</CustLastName>
<CustFirstName>ELIZABETH</CustFirstName>
<CustMiddleName />
<NumberOfDependants>0</NumberOfDependants>
<MaritalStatus>Single</MaritalStatus>
<DateOfBirth>1/1/1911</DateOfBirth>
<MilitaryRank />
<CurrentAddress>
<ZipCode>46617</ZipCode>
Unfortunately, I am unfamiliar with pulling from XML, and my google-fu has failed me.
select TransformedXML.value('(/DecisionData/Customer/Applicants/Applicant PersonType="Applicant"/CurrentAddress/ZipCode/node())[1]','nvarchar(max)') as zip
from XmlDecisionInputText as t
I believe my problem lies with the portion that goes Applicant PersonType="Applicant", but am unsure how to deal with it.
Thanks for any help.
The xpath in its simplest form would be:
TransformedXML.value('(//ZipCode)[1]', 'nvarchar(100)') AS zip
This will find the first ZipCode node anywhere inside your document. If there are multiple, just be specific (as much as you want but not any more):
TransformedXML.value('(/DecisionData/Customer/Applicants/Applicant[#PersonType="Applicant"]/CurrentAddress/ZipCode)[1]', 'nvarchar(100)') AS zip
DB Fiddle
If there are MULTIPLE applicants, you can use a CROSS APPLY
Example
Select A.ID
,B.*
From XmlDecisionInputText A
Cross Apply (
Select PersonType = x.v.value('#PersonType','VARCHAR(150)')
,CustLastName = x.v.value('CustLastName[1]','VARCHAR(150)')
,CustFirstName = x.v.value('CustFirstName[1]','VARCHAR(150)')
,ZipCode = x.v.value('CurrentAddress[1]/ZipCode[1]','VARCHAR(150)')
From XmlDecisionInputText.nodes('DecisionData/Customer/Applicants/*') x(v)
) B

Flattening xml data in sql

I'm trying to flatten XML data in a SQL query but I always seem to get nulls.
I tried the cross/outer apply method described here.
The column with XML data is called Data.
I'm guessing that the xml data with these links need to be somehow also added?
Could you please help to get a proper SQL query?
Query I tried:
SELECT
v.name
,pref.value('(LocalId/text())[1]', 'nvarchar(10)') as localid
FROM [database].[requests] v
outer apply v.Data.nodes('/DataForm') x(pref)
GO
example of xml data in that column:
<Dataform xmlns="http://somelongasslink.org/hasalsosomestuffhere" xmlns:i="http://somexlmschemalink/">
<DeleteDate xmlns="http://somelongasslink.org/hasalsosomestuffhere" i:nil="true" />
<LocalId xmlns="http://somelongasslink.org/hasalsosomestuffhere">5325325</LocalId>
...
You can use this code to get the result you're looking for:
;WITH XMLNAMESPACES(DEFAULT 'http://somelongasslink.org/hasalsosomestuffhere')
SELECT
rq.Name,
LocalID = TC.value('(LocalId)[1]', 'nvarchar(10)')
FROM
[database].[requests] rq
CROSS APPLY
rq.Data.nodes('/Dataform') AS TX(TC)
GO
There were two problems with your code:
you're not respecting / including the XML namespace that's defined on the XML document
<Dataform xmlns="http://somelongasslink.org/hasalsosomestuffhere"
*******************************************************
you didn't pay attention to the case-sensitivity of XML in your call to .nodes() - you need to use .nodes('/Dataform') (not /DataForm - the F is not capitalized in your XML)

Extracting XML data in SQL - too many cross apply statements

I have an xml document containing details from a Statement:
<Statement>
<Id />
<Invoices>
<Invoice>
<Id />
<Date />
<AmountDue />
etc.
</Invoice>
<Invoice>
<Id />
<Date />
<AmountDue />
etc.
</Invoice>
<Invoice>
<Id />
<Date />
<AmountDue />
etc.
</Invoice>
</Invoices>
</Statement>
This works fine for the Statement specific details:
SET #statementId = #xml.value('(Id)[1]', 'UNIQUEIDENTIFIER');
but it requires a singleton, and only returns the first value. I need ALL of the values for the invoices, not just the first so a singleton won't work.
I am able to get the information out using cross apply statements like this:
SELECT
#statementId AS STATEMENT_ID
Id.value('.', 'uniqueidentifier') AS INVOICE_ID
Date.value('.', 'smalldatetime') AS INVOICE_DATE
Due.value('.', 'decimal') AS INVOICE_AMOUNT_DUE
FROM #xml.nodes('Statement') A(S)
cross apply S.nodes('Invoices/Invoice') B(InvoiceD)
cross apply InvoiceD.nodes('Id') C(Id)
cross apply InvoiceD.nodes('Date') D(Date)
cross apply InvoiceD.nodes('AmountDue') E(Due)
This returns an Id, date, and amount from each Invoice in the Statement - perfect.
My problem comes when I try to extract all of the invoice details. I currently have seven cross apply statements and I got the following message:
"The query processor ran out of internal resources and could not
produce a query plan. This is a rare event and only expected for
extremely complex queries or queries that reference a very large
number of tables or partitions. Please simplify the query. If you
believe you have received this message in error, contact Customer
Support Services for more information."
What I want to do is have one cross apply for the Invoice and narrow down the exact field in the select statement, but unless I use '.' I must make the statement return a singleton and I don't get all of the data that I need.
I have done some research about specifying a namespace within the select statement, but all of the examples set the namespace to be an http address instead of a node in an xml document and I haven't gotten anything to return yet using this approach.
The result I'm looking for is something like this, but with more Invoice Details:
STATEMENT_ID INVOICE_ID INVOICE_DATE INVOICE_AMOUNT_DUE ...
Statement-1-Id Invoice-1-Id Invoice-1-Date Invoice-1-AmountDue ...
Statement-1-Id Invoice-2-Id Invoice-2-Date Invoice-2-AmountDue ...
Statement-1-Id Invoice-3-Id Invoice-3-Date Invoice-3-AmountDue ...
Where should I go from here?
EDIT: I removed some unnecessary information. Getting all of the invoice-specific details is my goal here.
select #XML.value('(Statement/Id/text())[1]', 'uniqueidentifier') as StatementId,
T.N.value('(Id/text())[1]', 'uniqueidentifier') as InvoiceId,
T.N.value('(Date/text())[1]', 'smalldatetime') as InvoiceDate,
T.N.value('(AmountDue/text())[1]', 'decimal') as AmountDue
from #XML.nodes('/Statement/Invoices/Invoice') as T(N)
.nodes will shred your XML to rows so that each row T.N is pointing to an Invoice node of its own. On that node there is only a single Id node so fetching the value specifying a singleton Id[1] works.
You can use Id[1] or (Id/text())[1] but the latter will give you a more efficient execution plan.