Parse XML data to SQL with multiple child node data - sql

I have a xml file having following data
<SearchProgramsResponse xmlns="http://api.abc.com/2011-03-01/">
<page>1</page>
<items>50</items>
<total>3129</total>
<programItems>
<programItem id="7779">
<name>Coolblue NL</name>
<adrank>5.4</adrank>
<categories>
<category id="40">Cell, Phone & Fax</category>
<category id="11">Internet services</category>
</categories>
</programItem>
<programItem id="7780">
<name>ABC NL</name>
<adrank>2.4</adrank>
<categories>
<category id="42">Cell</category>
<category id="12">services</category>
</categories>
</programItem>
</programItems>
</SearchProgramsResponse>
I want to Transfer this XML data to SQL server tables. But as there are multiple categories for one ITEM.. So how do i process this XML.
I mean I want to transfer DATA to two tables
In tableA one row per record eg :
TableA(ITEM) : programItem , Name , adrank
TABLEB(Category) : ITEMID, categoryID ,CategoryName
I have gone through XML reference but all are with Single value xml attribute , HOW can I process this type of XML ?
I have tried both queries below but no success
DECLARE #xmldata XML
SET #xmldata =
N'<SearchProgramsResponse xmlns="http://api.abc.com/2011-03-01/">
<page>1</page>
<items>50</items>
<total>3129</total>
<programItems>
<programItem id="7779">
<name>Coolblue NL</name>
<adrank>5.4</adrank>
<categories>
<category id="40">Cell, Phone & Fax</category>
<category id="11">Internet services</category>
</categories>
</programItem>
<programItem id="7780">
<name>ABC NL</name>
<adrank>2.4</adrank>
<categories>
<category id="42">Cell</category>
<category id="12">services</category>
</categories>
</programItem>
</programItems>
</SearchProgramsResponse>';
SELECT
Tbl.Col.value('name[1]', 'varchar(70)')
FROM #xmldata.nodes('/SearchProgramsResponse/programItems/programItem') Tbl(Col)
SELECT
XReservering.value('#programItem', 'int'),
XOpties.value('(.)', 'varchar(50)')
FROM
#xmldata.nodes('/SearchProgramsResponse/programItems/programItem') AS XTbl1(XReservering)
CROSS APPLY
XReservering.nodes('categories/category') AS XTbl2(XOpties)

Well you can apply nodes() function twice to get all the categories with item_id:
;with xmlnamespaces(default 'http://api.abc.com/2011-03-01/')
select
t.c.value('#id', 'int') as id,
t.c.value('(name/text())[1]', 'nvarchar(max)') as name,
t.c.value('(adrank/text())[1]', 'decimal(29,10)') as adrank
from #data.nodes('SearchProgramsResponse/programItems/programItem') as t(c)
;with xmlnamespaces(default 'http://api.abc.com/2011-03-01/')
select
t.c.value('#id', 'int') as item_id,
c.c.value('#id', 'int') as category_id,
c.c.value('text()[1]', 'nvarchar(max)') as category_name
from #data.nodes('SearchProgramsResponse/programItems/programItem') as t(c)
outer apply t.c.nodes('categories/category') as c(c)

Related

Parse out XML data into Rows in SQL Server

I am trying to parse WordPress data in our SQL Server from an Elasticsearch structure.
This is the contents of one field on one record;
<category domain="category" nicename="featured">Featured</category>
<category domain="post_tag" nicename="name1">Name 1</category>
<category domain="post_tag" nicename="name-2">Name 2</category>
<category domain="post_tag" nicename="different-name">Different Name</category>
<category domain="type" nicename="something-else">Something Else</category>
I'd like to parse this out as a table with the headers Domain, NiceName and Contents and a row for each of these nodes in the data. Something along these lines;
Domain
NiceName
Contents
category
featured
Featured
post_tag
name1
Name 1
post_tag
name-2
Name 2
post_tag
different-name
Different Name
type
something-else
Something Else
The number of nodes is different for each row in the data and can appear in any order. Currently the data is stored in a varchar data type but this can be modified if it's best to parse using something like XML.
It's recommended that you use the xml data type for storing XML data. But if you must store it in a varchar column you can use try_cast to cast it to XML (which results in null if it's not actually valid XML) and then work with it using the normal nodes(), query() and value() XML methods such as the following...
create table dbo.Records (
OneField varchar(max)
);
insert dbo.Records (OneField) values
('<category domain="category" nicename="featured">Featured</category>
<category domain="post_tag" nicename="name1">Name 1</category>
<category domain="post_tag" nicename="name-2">Name 2</category>
<category domain="post_tag" nicename="different-name">Different Name</category>
<category domain="type" nicename="something-else">Something Else</category>');
select
Category.value('#domain', 'varchar(50)') as [Domain],
Category.value('#nicename', 'varchar(50)') as [NiceName],
Category.value('(text())[1]', 'varchar(50)') as [Contents]
from dbo.Records R
cross apply (select try_cast(OneField as XML)) X(OneFieldXML)
cross apply OneFieldXML.nodes('/category') N(Category);
Domain
NiceName
Contents
category
featured
Featured
post_tag
name1
Name 1
post_tag
name-2
Name 2
post_tag
different-name
Different Name
type
something-else
Something Else

How to get value from a node in XML via SQL Server

I've found several pieces of information online about this but I can't get it working for the life of me.
This is the XML I have:
I need to extract the ID & Name value for each node. There are a lot.
I tried to do this but it returns NULL:
select [xml].value('(/Alter/Object/ObjectDefinition/MeasureGroup/Partitions/Partition/ID)[1]', 'varchar(max)')
from test_xml
I understand the above would return only 1 record. My question is, how do I return all records?
Here's the XML text (stripped down version):
<Alter xmlns="http://schemas.microsoft.com/analysisservices/2003/engine" AllowCreate="true" ObjectExpansion="ExpandFull">
<ObjectDefinition>
<MeasureGroup xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<ID>ts_homevideo_sum_20140430_76091ba1-3a51-45bf-a767-f9f3de7eeabe</ID>
<Name>table_1</Name>
<StorageMode valuens="ddl200_200">InMemory</StorageMode>
<ProcessingMode>Regular</ProcessingMode>
<Partitions>
<Partition>
<ID>123</ID>
<Name>2012</Name>
</Partition>
<Partition>
<ID>456</ID>
<Name>2013</Name>
</Partition>
</Partitions>
</MeasureGroup>
</ObjectDefinition>
</Alter>
You need something like this:
DECLARE #MyTable TABLE (ID INT NOT NULL, XmlData XML)
INSERT INTO #MyTable (ID, XmlData)
VALUES (1, '<Alter xmlns="http://schemas.microsoft.com/analysisservices/2003/engine" AllowCreate="true" ObjectExpansion="ExpandFull">
<ObjectDefinition>
<MeasureGroup xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<ID>ts_homevideo_sum_20140430_76091ba1-3a51-45bf-a767-f9f3de7eeabe</ID>
<Name>table_1</Name>
<StorageMode valuens="ddl200_200">InMemory</StorageMode>
<ProcessingMode>Regular</ProcessingMode>
<Partitions>
<Partition>
<ID>123</ID>
<Name>2012</Name>
</Partition>
<Partition>
<ID>456</ID>
<Name>2013</Name>
</Partition>
</Partitions>
</MeasureGroup>
</ObjectDefinition>
</Alter>')
;WITH XMLNAMESPACES(DEFAULT 'http://schemas.microsoft.com/analysisservices/2003/engine')
SELECT
tbl.ID,
MeasureGroupID = xc.value('(ID)[1]', 'varchar(200)'),
MeasureGroupName = xc.value('(Name)[1]', 'varchar(200)'),
PartitionID = xp.value('(ID)[1]', 'varchar(200)'),
PartitionName = xp.value('(Name)[1]', 'varchar(200)')
FROM
#MyTable tbl
CROSS APPLY
tbl.XmlData.nodes('/Alter/ObjectDefinition/MeasureGroup') AS XT(XC)
CROSS APPLY
XC.nodes('Partitions/Partition') AS XT2(XP)
WHERE
ID = 1
First of all, you must respect and include the default XML namespace defined in the root of your XML document.
Next, you need to do a nested call to .nodes() to get all <MeasureGroup> and all contained <Partition> nodes, so that you can reach into those XML fragments and extract the ID and Name from them.
This should then result in something like this as output:

Parse an xml data column on a table in SQL Server 2012

How do I parse an xml column on a table of data in SQL Server 2012
Sample data
<GetOfferAvailabilityResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p3="http://somewebsite.com/v2.0" xmlns="http://somewebsite.com/v2.0" p3:TransactionID="281234567">
<p3:RuleResultList xsi:nil="true" />
<p3:ResultList>
<p3:ProviderResult p3:ProviderID="01" p3:ResultID="1234" p3:ResultType="NotAvailable" p3:ResultCode="NotAvailable" p3:BrokerID="55" p3:Structure="None">
<p3:EntityState>None</p3:EntityState>
<p3:ResultText>No Orders returned</p3:ResultText>
<p3:ShortDescription>Not Available</p3:ShortDescription>
<p3:LongDescription>We're sorry, but offers are currently not available for your service address.</p3:LongDescription>
<p3:ResultAction>ErrorMessage</p3:ResultAction>
<p3:SourceResultCode xsi:nil="true" />
</p3:ProviderResult>
</p3:ResultList>
</GetOfferAvailabilityResponse>'
I tried:
DECLARE #x xml
SET #x =
DECLARE #test TABLE (ID INT, XmlRule XML)
Insert into #test VALUES(1,'
<GetOfferAvailabilityResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" ---GetOfferAvailabilityResponse>')
When I use Select #test.query ('\') I get the entire xml but when I try Select #test.query ('\GetOfferAvailabilityResponse') I receive an empty result
You can try something like this:
DECLARE #XmlTbl TABLE (ID INT, XMLDATA XML)
INSERT INTO #XmlTbl
( ID, XMLDATA )
VALUES ( 1, '<GetOfferAvailabilityResponse xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:p3="http://somewebsite.com/v2.0" xmlns="http://somewebsite.com/v2.0" p3:TransactionID="281234567">
<p3:RuleResultList xsi:nil="true" />
<p3:ResultList>
<p3:ProviderResult p3:ProviderID="01" p3:ResultID="1234" p3:ResultType="NotAvailable" p3:ResultCode="NotAvailable" p3:BrokerID="55" p3:Structure="None">
<p3:EntityState>None</p3:EntityState>
<p3:ResultText>No Orders returned</p3:ResultText>
<p3:ShortDescription>Not Available</p3:ShortDescription>
<p3:LongDescription>We''re sorry, but offers are currently not available for your service address.</p3:LongDescription>
<p3:ResultAction>ErrorMessage</p3:ResultAction>
<p3:SourceResultCode xsi:nil="true" />
</p3:ProviderResult>
</p3:ResultList>
</GetOfferAvailabilityResponse>')
;WITH XMLNAMESPACES('http://somewebsite.com/v2.0' AS p3, DEFAULT 'http://somewebsite.com/v2.0')
SELECT
ProviderID = XmlData.value('(/GetOfferAvailabilityResponse/p3:ResultList/p3:ProviderResult/#p3:ProviderID)[1]', 'varchar(50)'),
EntityState = XmlData.value('(/GetOfferAvailabilityResponse/p3:ResultList/p3:ProviderResult/p3:EntityState)[1]', 'varchar(50)'),
ResultText = XmlData.value('(/GetOfferAvailabilityResponse/p3:ResultList/p3:ProviderResult/p3:ResultText)[1]', 'varchar(50)'),
ShortDescription = XmlData.value('(/GetOfferAvailabilityResponse/p3:ResultList/p3:ProviderResult/p3:ShortDescription)[1]', 'varchar(250)'),
LongDescription = XmlData.value('(/GetOfferAvailabilityResponse/p3:ResultList/p3:ProviderResult/p3:LongDescription)[1]', 'varchar(250)'),
ResultAction = XmlData.value('(/GetOfferAvailabilityResponse/p3:ResultList/p3:ProviderResult/p3:ResultAction)[1]', 'varchar(50)')
FROM
#XmlTbl
From a table that contains a column of type XML, select those bits and pieces that you need, taking into account the defined XML namespaces on your XML data
This gives me a result of:

Read XML child node attributes using SQL query

I have one XML column (Criteria) in table (Qualifications) which contains different XML:
<training ID="173"><badge ID="10027" /><badge ID="10028" /></training>
<book Category="Hobbies And Interests" PropertyName="C#" CategoryID="44" />
<sport Category="Hobbies And Interests" PropertyName="Cricket" CategoryID="46" />
<education ID="450" School="Jai ambe vidyalaya"></education>
I want to read the "badge" node "ID" attributes for all nodes under the "training" node.
Can anyone help?
IDs of badge elements inside training only
select t.c.value('.', 'int') ID
from Qualifications q
cross apply q.Criteria.nodes('//training[badge]/badge[#ID]/#ID') t(c)
IDs of badge elements anywhere (not only inside training)
select t.c.value('.', 'int') ID
from Qualifications q
cross apply q.Criteria.nodes('//badge[#ID]/#ID') t(c)
If Criteria column is nvarchar type, you can cast to xml as:
select t.c.value('.', 'int') ID
from Qualifications q
cross apply (select convert(xml, q.Criteria) xmlCriteria) a
cross apply a.xmlCriteria.nodes('//training[badge]/badge[#ID]/#ID') t(c)
Try this sample, it should help (just replace #xml with your table/column name)
DECLARE #xml XML
SET #xml ='
<training ID="173">
<badge ID="10027" />
<badge ID="10028" />
</training>
<book Category="Hobbies And Interests" PropertyName="C#" CategoryID="44" />
<sport Category="Hobbies And Interests" PropertyName="Cricket" CategoryID="46" />
<education ID="450" School="Jai ambe vidyalaya"></education>'
SELECT data.col.value('(#ID)[1]', 'int')
FROM #xml.nodes('(/training/badge)') AS data(col)
Output:
10027
10028

SQL Server XML Column - Result free of Concat

I have an xml column in my SQL Server database that has records in the following format
<items>
<item>
<data alias="Number">123N</data>
<data alias="Description">4 sq.mm Feed Through Terminal block in Grey colour</data>
<data alias="Standard Packing Quantity">100</data>
</item>
<item>
<data alias="Number">234N</data>
<data alias="Description">Toy</data>
<data alias="Standard Packing Quantity">100</data>
</item>
<item>
<data alias="Number">579N</data>
<data alias="Description">Doll</data>
<data alias="Standard Packing Quantity">100</data>
</item>
<item>
<data alias="Catalouge Number">234</data>
<data alias="Description">Vehicle</data>
<data alias="Standard Packing Quantity">324234</data>
</item>
</items>
So to extract the data here I use:
SELECT
CatalogueNo,Description,StdPackingQty
from
(select
CAST(xml as xml).query('//data alias=''Description'']').value('.','nvarchar(225)') [Description],
CAST(xml as xml).query('//data [#alias=''Catalouge Number'']')
.value('.','nvarchar(225)')[CatalogueNo],
CAST(xml as xml).query('//data [#alias=''Standard Packing Quantity'']').value('.','nvarchar(225)')[StdPackingQty]
from [dbo].[cmsContentXml] )as hierarchy
Where CatalogueNo is not null
The problem I face is that the data that is extracted is all concatenated.
I need data in separate rows for each item,so the data needs to be in 3
columns and 4 rows.
Kindly help me resolve the issue asap and help write a query that would fetch data
free of concat
You can use Cross Apply to breakdown the XML into separate data rows, and pull the data from those:
SELECT
CatalogueNo,Description,StdPackingQty
from (
select
i.value('data[#alias="Description"][1]','nvarchar(225)') [Description],
i.value('data[#alias="Catalouge Number"][1]','nvarchar(225)') [CatalogueNo],
i.value('data[#alias="Standard Packing Quantity"][1]','nvarchar(225)') [StdPackingQty]
from [Connectwell].[dbo].[cmsContentXml]
cross apply xml.nodes('/items/item') x(i)
) as hierarchy
--Where CatalogueNo is not null
Note: I've remmed out the where clause at the end as it will reduce you to just one row, whereas the question state you want 4 rows as the result.
If you know that the order of your elements always are the same you can use position() to get the values:
select X.N.value('data[1]', 'nvarchar(255)') as Number,
X.N.value('data[2]', 'nvarchar(255)') as Descritpion,
X.N.value('data[3]', 'nvarchar(255)') as Quanatity
from #T as T
cross apply T.XMLColumn.nodes('/items/item') as X(N)
Otherwise you need to get the value using alias. Your fourth row has an alias "Catalouge Number" that is different from the other rows so you can either have that as a separate column:
select X.N.value('data[#alias="Number"][1]', 'nvarchar(255)') as Number,
X.N.value('data[#alias="Catalouge Number"][1]', 'nvarchar(255)') as CatalougeNumber,
X.N.value('data[#alias="Description"][1]', 'nvarchar(255)') as Descritpion,
X.N.value('data[#alias="Standard Packing Quantity"][1]', 'nvarchar(255)') as Quanatity
from #T as T
cross apply T.XMLColumn.nodes('/items/item') as X(N)
Or you can combine the two in the same column:
select X.N.value('data[#alias=("Number","Catalouge Number")][1]', 'nvarchar(255)') as Number,
X.N.value('data[#alias="Description"][1]', 'nvarchar(255)') as Descritpion,
X.N.value('data[#alias="Standard Packing Quantity"][1]', 'nvarchar(255)') as Quanatity
from #T as T
cross apply T.XMLColumn.nodes('/items/item') as X(N)
Your where clause can be added to the queries like this:
where X.N.exist('data[#alias="Catalouge Number"]') = 1
Or if you want to check against both number aliases:
where X.N.exist('data[#alias=("Number","Catalouge Number")]') = 1
Test data:
declare #T table(XMLColumn xml)
insert into #T values
('<items>
<item>
<data alias="Number">123N</data>
<data alias="Description">4 sq.mm Feed Through Terminal block in Grey colour</data>
<data alias="Standard Packing Quantity">100</data>
</item>
<item>
<data alias="Number">234N</data>
<data alias="Description">Toy</data>
<data alias="Standard Packing Quantity">100</data>
</item>
<item>
<data alias="Number">579N</data>
<data alias="Description">Doll</data>
<data alias="Standard Packing Quantity">100</data>
</item>
<item>
<data alias="Catalouge Number">234</data>
<data alias="Description">Vehicle</data>
<data alias="Standard Packing Quantity">324234</data>
</item>
</items>
')
Edit:
I see that you cast your XML column to XML. That is not necessary if the column already is of data type XML so I guess you have a varchar(max) or something in there. If so you need to do like this to cast the to XML before applying .nodes() function:
select X.N.value('data[#alias=("Number","Catalouge Number")][1]', 'nvarchar(255)') as Number,
X.N.value('data[#alias="Description"][1]', 'nvarchar(255)') as Descritpion,
X.N.value('data[#alias="Standard Packing Quantity"][1]', 'nvarchar(255)') as Quanatity
from #T as T
cross apply (select cast(XMLColumn as xml)) as X1(XMLColumn)
cross apply X1.XMLColumn.nodes('/items/item') as X(N)
where X.N.exist('data[#alias=("Number","Catalouge Number")]') = 1
Edit 2
select xml.value('data[#alias=("Number","Catalouge Number")][1]', 'nvarchar(255)') as Number,
xml.value('data[#alias="Description"][1]', 'nvarchar(255)') as Descritpion,
xml.value('data[#alias="Standard Packing Quantity"][1]', 'nvarchar(255)') as Quanatity
from [dbo].[cmsContentXml] as Hierarchy
cross apply (select cast(xml as xml)) as X1(xml)
cross apply X1.xml.nodes('/items/item') as xml(N)
where xml.N.exist('data[#alias=("Number","Catalouge Number")]')=1