TSQL-failing to parse XML(namespaces) - sql

Here is a my SQL:
create table sqm (data xml)
insert into sqm
select '<DataSet xmlns="http://www.bnr.ro/xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.bnr.ro/xsd nbrfxrates.xsd">
<Cube date="2017-06-30">
<Rate currency="AED">1.0867</Rate>
<Rate currency="AUD">3.0665</Rate>
<Rate currency="BGN">2.3284</Rate>
</Cube>
</DataSet>'
select
m.c.value('#date', 'date') as valuta
from sqm as s
outer apply s.data.nodes('/DataSet/Body/Cube') as m(c)
After spending hours trying to find out why my SQL kept returning NULL in the db, I discovered that my problem was due to Hyperlink references at the very beginning of the XML(after DataSet).
I really want to know why is this happening and who can I delete everything between <DataSet end >. Is there any other option?
if you can think of a better title, please edit.

You need to declare the namespace using WITH XMLNAMESPACES
Also your example XML has no Body element so I removed that from the Xpath.
Demo
WITH XMLNAMESPACES (DEFAULT 'http://www.bnr.ro/xsd')
select
m.c.value('#date', 'date') as valuta
from sqm as s
outer apply s.data.nodes('/DataSet/Cube') as m(c)
Or alternatively you can use
select
m.c.value('#date', 'date') as valuta
from sqm as s
outer apply s.data.nodes('/*:DataSet/*:Cube') as m(c)

Related

Can't parse XML with outer apply

I have an XML column in a table which i am trying to parse out values from to flat table structure.
I am trying to input the XML here but stackoverflow ses it as code and when i try and format as code it still won't accept it.
I can't even get data from "Header" level.
<RequestMessage xmlns="http://iec.ch/TC57/2011/schema/message" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="Message.xsd">
<Header>
<Verb>created</Verb>
<Noun>MeterReadings</Noun>
<Timestamp>2021-03-08T00:57:18+01:00</Timestamp>
<Source>Ipsum Lorum</Source>
<AsyncReplyFlag>true</AsyncReplyFlag>
<AckRequired>true</AckRequired>
<MessageID>Ipsum Lorum</MessageID>
<CorrelationID />
</Header>
<Payload>
<MeterReadings xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#" xmlns="http://iec.ch/TC57/2011/MeterReadings#">
<MeterReading>
<IntervalBlocks>
<IntervalReadings>
<timeStamp>2021-03-07T01:00:00+01:00</timeStamp>
<value>480.196</value>
<ReadingQualities>
<ReadingQualityType ref="3.0.0" />
</ReadingQualities>
</IntervalReadings>
<IntervalReadings>
<ReadingType ref="11.0.7.3.1.2.12.1.1.0.0.0.0.101.0.3.72.0" />
</IntervalReadings>
</IntervalBlocks>
<Meter>
<mRID>0000000000000</mRID>
<status>
<remark>Ipsum Lorum</remark>
<value>ESP</value>
</status>
</Meter>
<UsagePoint>
<mRID>73599900000000</mRID>
</UsagePoint>
</MeterReading>
</MeterReadings>
</Payload>
</RequestMessage>
I am not able to parse it and i have tried using examples from other threads. I am trying to not use OPENXML solution because requires DECLARE and executing the built in procedure for clearing the XML from memmory periodically. I am trying to use the OUTER APPLY solution.
Like Shugos solution in How to parse XML data in SQL server table or Query XML with nested nodes on Cross Apply.
It doesn't work.
It returns null for the timestamp column.
select
t.file_created_time
,c.value('(Timestamp)[1]','varchar(max)') as timestamp
from load.t t
OUTER APPLY t.xml_data.nodes('RequestMessage/Header') as m(c)
Please try the following solution.
Starting from SQL Server 2005 onwards, it is better to use XQuery language, based on the w3c standards, while dealing with the XML data type.
Microsoft proprietary OPENXML and its companions sp_xml_preparedocument and sp_xml_removedocument are kept just for backward compatibility with the obsolete SQL Server 2000. Their use is diminished just to very few fringe cases.
I had to comment out the following tag <!--<IntervalReadings>--> to make your XML well-formed.
XML Header fragment has a default namespace:
xmlns="http://iec.ch/TC57/2011/schema/message"
XML Payload fragment has its own two additional namespaces:
xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#"
xmlns="http://iec.ch/TC57/2011/MeterReadings#"
Namespaces should be taken into account.
Check it out below.
SQL
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, xml_data XML);
INSERT INTO #tbl (xml_data) VALUES
(N'<RequestMessage xmlns="http://iec.ch/TC57/2011/schema/message"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="Message.xsd">
<Header>
<Verb>created</Verb>
<Noun>MeterReadings</Noun>
<Timestamp>2021-03-08T00:57:18+01:00</Timestamp>
<Source>Ipsum Lorum</Source>
<AsyncReplyFlag>true</AsyncReplyFlag>
<AckRequired>true</AckRequired>
<MessageID>Ipsum Lorum</MessageID>
<CorrelationID/>
</Header>
<Payload>
<MeterReadings xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#"
xmlns="http://iec.ch/TC57/2011/MeterReadings#">
<MeterReading>
<IntervalBlocks>
<IntervalReadings>
<timeStamp>2021-03-07T01:00:00+01:00</timeStamp>
<value>480.196</value>
<ReadingQualities>
<ReadingQualityType ref="3.0.0"/>
</ReadingQualities>
</IntervalReadings>
<!--<IntervalReadings>-->
<ReadingType ref="11.0.7.3.1.2.12.1.1.0.0.0.0.101.0.3.72.0"/>
</IntervalBlocks>
<Meter>
<mRID>0000000000000</mRID>
<status>
<remark>Ipsum Lorum</remark>
<value>ESP</value>
</status>
</Meter>
<UsagePoint>
<mRID>73599900000000</mRID>
</UsagePoint>
</MeterReading>
</MeterReadings>
</Payload>
</RequestMessage>');
-- DDL and sample data population, end
WITH XMLNAMESPACES(DEFAULT 'http://iec.ch/TC57/2011/schema/message')
SELECT id
, c.value('(Noun/text())[1]','VARCHAR(30)') AS Noun
, c.value('(Timestamp/text())[1]','DATETIMEOFFSET(0)') AS [timestamp]
FROM #tbl
CROSS APPLY xml_data.nodes('/RequestMessage/Header') AS t(c);
Output
+----+---------------+----------------------------+
| id | Noun | timestamp |
+----+---------------+----------------------------+
| 1 | MeterReadings | 2021-03-08 00:57:18 +01:00 |
+----+---------------+----------------------------+
You need to respect and include the XML namespace in your XML document in your XQuery!
<RequestMessage xmlns="http://iec.ch/TC57/2011/schema/message"
**********************************************
Try something like this:
WITH XMLNAMESPACES(DEFAULT N'http://iec.ch/TC57/2011/schema/message')
SELECT
t.id,
c.value('(Timestamp)[1]','varchar(max)') as timestamp
FROM
load.t t
CROSS APPLY
t.xml_data.nodes('RequestMessage/Header') AS m(c)
Also when trying to run this on my SQL Server, I get an error that the XML as shown is malformed.....
UPDATE:
If you need to also access bits in the Payload section - you need to also respect that XML namespace there:
<MeterReadings xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#"
xmlns="http://iec.ch/TC57/2011/MeterReadings#">
***********************************************
Try this:
WITH XMLNAMESPACES(N'http://iec.ch/TC57/2011/schema/message' as hdr,
N'http://iec.ch/TC57/2011/MeterReadings#' as mr)
SELECT
t.id,
c.value('(hdr:Timestamp)[1]', 'varchar(50)') AS timestamp,
col.value('(mr:MeterReading/mr:IntervalBlocks/mr:IntervalReadings/mr:timeStamp)[1]', 'varchar(50)') AS MeterReadingsTimestamp
FROM
load.t t
CROSS APPLY
t.xml_data.nodes('/hdr:RequestMessage/hdr:Header') AS m(c)
CROSS APPLY
t.xml_data.nodes('/hdr:RequestMessage/hdr:Payload/mr:MeterReadings') AS mr(col)

Parsing XML Data Into SQL Server

I am struggling with importing XML Data into SQL Server 2016. I have tried a few things, but keep either getting errors or just no data is returned.
I have this XML Data stored in an XML file (limited the data because it is pretty sensitive:
<?xml version='1.0' encoding='UTF-8'?>
<wd:Report_Data xmlns:wd="urn:com.workday.report/Worker_Details_-_EXPORT_-_Workplace">
<wd:Report_Entry>
<wd:Active_Status>0</wd:Active_Status>
<wd:Legal_Name_-_First_Name>Charlotte</wd:Legal_Name_-_First_Name>
<wd:Position>Executive Housekeeper I</wd:Position>
<wd:Worker_Management_Level>Supervisor</wd:Worker_Management_Level>
<wd:continuous_service_date>1979-04-29-08:00</wd:continuous_service_date>
<wd:Hire_Date>1979-04-29-08:00</wd:Hire_Date>
<wd:termination_date>2019-12-22-08:00</wd:termination_date>
<wd:Anniversary_Month>04</wd:Anniversary_Month>
<wd:Years_of_Service>40</wd:Years_of_Service>
<wd:Employee_Type>Hotel</wd:Employee_Type>
<wd:Time_Type>Full Time</wd:Time_Type>
<wd:Pay_Rate_Type>Salary</wd:Pay_Rate_Type>
<wd:Marital_Status>Single</wd:Marital_Status>
</wd:Report_Entry>
<wd:Report_Entry>
<wd:Active_Status>0</wd:Active_Status>
<wd:Legal_Name_-_First_Name>Robert</wd:Legal_Name_-_First_Name>
<wd:Cost_Center_-_Name>Electronics</wd:Cost_Center_-_Name>
<wd:Work_Address_-_State_Province>Missouri</wd:Work_Address_-_State_Province>
<wd:Position>Manager Of Voice Networks</wd:Position>
<wd:Worker_Management_Level>Manager</wd:Worker_Management_Level>
<wd:continuous_service_date>1980-02-25-08:00</wd:continuous_service_date>
<wd:Hire_Date>1980-02-25-08:00</wd:Hire_Date>
<wd:termination_date>2020-03-22-07:00</wd:termination_date>
<wd:Anniversary_Month>02</wd:Anniversary_Month>
<wd:Years_of_Service>40</wd:Years_of_Service>
<wd:Employee_Type>Corporate</wd:Employee_Type>
<wd:Time_Type>Full Time</wd:Time_Type>
<wd:Pay_Rate_Type>Salary</wd:Pay_Rate_Type>
<wd:Marital_Status>Married</wd:Marital_Status>
</wd:Report_Entry>
</wd:Report_Data>
I have this code that I am trying to use, but keep getting just an empty result:
SELECT
XMLCol.ReportEntry.query('Active_Status').value('.', 'VARCHAR(20)') AS ActiveStatus
FROM
(SELECT
CAST(XMLCol AS XML)
FROM
OPENROWSET(BULK '\\afcn2011\root\DATA\VisualCron\Employee Export\EmployeeExport.xml', SINGLE_BLOB) AS T(XMLCol)
) AS T(XMLCol)
CROSS APPLY
XMLCol.nodes('Report_Data/Report_Entry') AS XMLCol(ReportEntry);
You need to respect and include the XML namespace defined in your document.
Try something like this:
-- define the namespace and give it a prefix - here "wd"
;WITH XMLNAMESPACES ('urn:com.workday.report/Worker_Details_-_EXPORT_-_Workplace' as wd)
SELECT
-- you need to include namespace prefix when referring to the XML element
-- also: is "VARCHAR(20)" really the best datatype?? Looks more like "INT" to me ...
XMLCol.ReportEntry.value('(wd:Active_Status/text())[1]', 'VARCHAR(20)') AS ActiveStatus
FROM
(SELECT
CAST(XMLCol AS XML)
FROM
OPENROWSET(BULK '\\afcn2011\root\DATA\VisualCron\Employee Export\EmployeeExport.xml', SINGLE_BLOB) AS T(XMLCol)
) AS T(XMLCol)
CROSS APPLY
-- you need to include namespace prefix in your XPath expression
XMLCol.nodes('/wd:Report_Data/wd:Report_Entry') AS XMLCol(ReportEntry);

SQL query for XML data

I have a SQL Server database table with a column called XML that contains XML data which is structured like this:
<Item xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://test/data">
<Roots>
<Root>
<Name>Field Name</Name>
<Value>Field Value</Value>
</Root>
<Root>
<Name>Field Name</Name>
<Value>Field Value</Value>
</Root>
</Roots>
I want to use T-SQL to get the Value where Name = Total. I have tried the following but it isn't returning any data:
SELECT [XML]
FROM [BusinessAccount]
WHERE [XML].value('(/Root/Name)[13]', 'VARCHAR(MAX)') LIKE '%Total%'
If anyone could tell me where I've gone wrong?
You are missing the required WITH XMLNAMESPACES for your XML and the path is incorrect.
If you want to bring back rows where the 13th element consists of the text Total you can use the below.
WITH XMLNAMESPACES (DEFAULT 'http://test/data')
SELECT [XML]
FROM [BusinessAccount]
WHERE 1 = [XML].exist('(/Item/Roots/Root/Name)[13][text() = "Total"]')
Otherwise you can add the WITH XMLNAMESPACES to your original query and fix the path there too.
You need to specify namespaces. You can then match <Name> and <Value> pairs and extract the contents of <Value> like so:
SELECT NameNode.value('declare namespace x="http://test/data"; (../x:Value)[1]', 'varchar(100)')
FROM [BusinessAccount]
CROSS APPLY [XML].nodes('declare namespace x="http://test/data"; //x:Root/x:Name') AS n(NameNode)
WHERE NameNode.value('.', 'varchar(100)') = 'Total'
Demo on db<>fiddle

How to get value from a node in XML via SQL Server

I've found several pieces of information online about this but I can't get it working for the life of me.
This is the XML I have:
I need to extract the ID & Name value for each node. There are a lot.
I tried to do this but it returns NULL:
select [xml].value('(/Alter/Object/ObjectDefinition/MeasureGroup/Partitions/Partition/ID)[1]', 'varchar(max)')
from test_xml
I understand the above would return only 1 record. My question is, how do I return all records?
Here's the XML text (stripped down version):
<Alter xmlns="http://schemas.microsoft.com/analysisservices/2003/engine" AllowCreate="true" ObjectExpansion="ExpandFull">
<ObjectDefinition>
<MeasureGroup xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<ID>ts_homevideo_sum_20140430_76091ba1-3a51-45bf-a767-f9f3de7eeabe</ID>
<Name>table_1</Name>
<StorageMode valuens="ddl200_200">InMemory</StorageMode>
<ProcessingMode>Regular</ProcessingMode>
<Partitions>
<Partition>
<ID>123</ID>
<Name>2012</Name>
</Partition>
<Partition>
<ID>456</ID>
<Name>2013</Name>
</Partition>
</Partitions>
</MeasureGroup>
</ObjectDefinition>
</Alter>
You need something like this:
DECLARE #MyTable TABLE (ID INT NOT NULL, XmlData XML)
INSERT INTO #MyTable (ID, XmlData)
VALUES (1, '<Alter xmlns="http://schemas.microsoft.com/analysisservices/2003/engine" AllowCreate="true" ObjectExpansion="ExpandFull">
<ObjectDefinition>
<MeasureGroup xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<ID>ts_homevideo_sum_20140430_76091ba1-3a51-45bf-a767-f9f3de7eeabe</ID>
<Name>table_1</Name>
<StorageMode valuens="ddl200_200">InMemory</StorageMode>
<ProcessingMode>Regular</ProcessingMode>
<Partitions>
<Partition>
<ID>123</ID>
<Name>2012</Name>
</Partition>
<Partition>
<ID>456</ID>
<Name>2013</Name>
</Partition>
</Partitions>
</MeasureGroup>
</ObjectDefinition>
</Alter>')
;WITH XMLNAMESPACES(DEFAULT 'http://schemas.microsoft.com/analysisservices/2003/engine')
SELECT
tbl.ID,
MeasureGroupID = xc.value('(ID)[1]', 'varchar(200)'),
MeasureGroupName = xc.value('(Name)[1]', 'varchar(200)'),
PartitionID = xp.value('(ID)[1]', 'varchar(200)'),
PartitionName = xp.value('(Name)[1]', 'varchar(200)')
FROM
#MyTable tbl
CROSS APPLY
tbl.XmlData.nodes('/Alter/ObjectDefinition/MeasureGroup') AS XT(XC)
CROSS APPLY
XC.nodes('Partitions/Partition') AS XT2(XP)
WHERE
ID = 1
First of all, you must respect and include the default XML namespace defined in the root of your XML document.
Next, you need to do a nested call to .nodes() to get all <MeasureGroup> and all contained <Partition> nodes, so that you can reach into those XML fragments and extract the ID and Name from them.
This should then result in something like this as output:

issue with xml parsing data to sql query

I am working on SQL query where i need to parse xml to sql .Query is working for other xml but not working for below xml. query i am using is as below
DECLARE #xmldata XML
SET #xmldata =
N'<SearchProductsResponse xmlns="http://api.abc.com/">
<productItems>
<productItem id="5d0ee86d84bcc5edef43236d61419a59">
<trackingLinks>
<trackingLink adspaceId="100">
<ppv>
abc.com
</ppv>
<ppc>
abc.com
</ppc>
</trackingLink>
</trackingLinks>
</productItem>
</productItems>
</SearchProductsResponse>';
select
t1.c.value('#id', 'varchar(300)') as itemid,
c.c.value('#id', 'int') as adspaceId
from
#xmldata.nodes('*:SearchProductsResponse/*:productItems/*:productItem') as t1(c)
OUTER APPLY t1.c.nodes('*:trackingLinks/*:trackingLink') as c(c)
Output I am getting is
itemid adspaceId
5d0ee86d84bcc5edef43236d61419a59 NULL
But I should return 100 instead of NULL . This query is working for other XML but don't know whats wrong with this XML. I have double check XML and query nothing different from other XML. Hope I am not missing some silly mistake
Change the attribute selection for your adspaceId column to #adspaceId.
select
t1.c.value('#id', 'varchar(300)') as itemid,
c.c.value('#adspaceId', 'int') as adspaceId
from #xmldata.nodes('*:SearchProductsResponse/*:productItems/*:productItem') as t1(c)
OUTER APPLY t1.c.nodes('*:trackingLinks/*:trackingLink') as c(c)