Parsing Dynamic XML to SQL Server tables with Parent and child relation - sql

I have a XML in Source Table. I need to parse this XML to 3 different tables which has Parent Child relationship. I can do this in C# but currently for this i need to implement it at SQL server side.
The sample xml looks like:
<ROWSET>
<ROW>
<HEADER_ID>5001507</HEADER_ID>
<ORDER_NUMBER>42678548</ORDER_NUMBER>
<CUST_PO_NUMBER>LSWQWE1</CUST_PO_NUMBER>
<CUSTOMER_NUMBER>38087</CUSTOMER_NUMBER>
<CUSTOMER_NAME>UNIVERSE SELLER</CUSTOMER_NAME>
<LINE>
<LINE_ROW>
<HEADER_ID>5001507</HEADER_ID>
<LINE_ID>12532839</LINE_ID>
<LINE_NUMBER>1</LINE_NUMBER>
<ITEM_NUMBER>STAGEPAS 600I-CA</ITEM_NUMBER>
<ORDER_QUANTITY>5</ORDER_QUANTITY>
</LINE_ROW>
<LINE_ROW>
<HEADER_ID>5001507</HEADER_ID>
<LINE_ID>12532901</LINE_ID>
<LINE_NUMBER>3</LINE_NUMBER>
<ITEM_NUMBER>CD-C600 RK</ITEM_NUMBER>
<ORDER_QUANTITY>6</ORDER_QUANTITY>
</LINE_ROW>
<LINE_ROW>
<HEADER_ID>5001507</HEADER_ID>
<LINE_ID>12532902</LINE_ID>
<LINE_NUMBER>4</LINE_NUMBER>
<ITEM_NUMBER>CD-S300 RK</ITEM_NUMBER>
<ORDER_QUANTITY>8</ORDER_QUANTITY>
</LINE_ROW>
</LINE>
<PRCADJ>
<PRCADJ_ROW>
<PRICE_ADJUSTMENT_ID>43095064</PRICE_ADJUSTMENT_ID>
<HEADER_ID>5001507</HEADER_ID>
<LINE_ID>12532839</LINE_ID>
<ADJUSTED_AMOUNT>-126</ADJUSTED_AMOUNT>
</PRCADJ_ROW>
<PRCADJ_ROW>
<PRICE_ADJUSTMENT_ID>43095068</PRICE_ADJUSTMENT_ID>
<HEADER_ID>5001507</HEADER_ID>
<LINE_ID>12532840</LINE_ID>
<ADJUSTED_AMOUNT>-96.6</ADJUSTED_AMOUNT>
</PRCADJ_ROW>
</PRCADJ>
</ROW>
</ROWSET>
The issue is the Parent can have multiple child and each child can multiple sub child. How can i write query to transfer this into Sql Server 2005

You need to use three CROSS APPLY operators to break up the "list of XML elements" into separate pseudo tables of XML rows, so you can access their properties - something like this:
SELECT
HeaderID = XCRow.value('(HEADER_ID)[1]', 'int'),
OrderNumber = XCRow.value('(ORDER_NUMBER)[1]', 'int'),
LineHeaderID = XCLine.value('(HEADER_ID)[1]', 'int'),
LineID = XCLine.value('(LINE_ID)[1]', 'int'),
LineNumber = XCLine.value('(LINE_NUMBER)[1]', 'int'),
PriceAdjustmentID = XCPrc.value('(PRICE_ADJUSTMENT_ID)[1]', 'int'),
AdjustedAmount = XCPrc.value('(ADJUSTED_AMOUNT)[1]', 'decimal(20,4)')
FROM
dbo.YourTableNameHere
CROSS APPLY
Data.nodes('/ROWSET/ROW') AS XTRow(XCRow)
CROSS APPLY
XCRow.nodes('LINE/LINE_ROW') AS XTLine(XCLine)
CROSS APPLY
XCRow.nodes('PRCADJ/PRCADJ_ROW') AS XTPrc(XCPrc)
With this, the first CROSS APPLY will handle all the elements that are found directly under <ROWSET> / <ROW> (the header information), the second one will enumerate all instances of <LINE> / <LINE_ROW> below that header element, and the third CROSS APPLY handles the <PRCADJ> / <PRCADJ_ROW> elements, also below the header.
You might need to tweak the outputs a bit - and I only picked two or three of the possible values - extend and adapt to your own needs! But this should show you the basic mechanism - the .nodes() method returns a "pseudo table" of XML fragments, one for each match of the XPath expression you define.

you can do some thing like this. using cross apply you will get node elements and then extract the value using value clause. you need to specify the column type i.e int or varchar etc.
The result can then be inserted using insert into select query.
insert into Table1 values ( header_id, order_number, cust_po_number)
select R.value('(HEADER_ID)[1]', 'int') As header_id,
R.value('(ORDER_NUMBER)[1]', 'int') as order_number,
R.value('(CUST_PO_NUMBER)[1]', 'varchar(256)') as cust_po_number
from table
cross apply XMLdata.nodes('/ROWSET/ROW') AS P(R)
insert into Table2 values ( header_id, line_id, line_number)
select R.value('(HEADER_ID)[1]', 'int') As header_id,
R.value('(LINE_ID)[1]', 'int') as line_id,
R.value('(LINE_NUMBER)[1]', 'int') as line_number
from table
cross apply XMLdata.nodes('/ROWSET/ROW/LINE/LINE_ROW') AS P(R)

Related

Importing XML to SQL Server

I am wondering how I can insert an XML file into a SQL Server DB. Below is the XML I have but I am unsure how to do this in a way that will scale. My thought is a Insert Into Select statement but I do not know if that is going to work as the data increases. Thank you in advance!
<Records>
<Record>
<ID SpecNum="5069580" IssueNum="001" SpecStatus="Pre-Approved">
<NutritionDetails>
<NutrientFacts>
<NutrientNameId>ENERC_KCAL</NutrientNameId>
<NutrientName>ENERC_KCAL</NutrientName>
<NutrientPer100gUnrounded>1.91</NutrientPer100gUnrounded>
<NutrientPer100gRounded>191</NutrientPer100gRounded>
</NutrientFacts>
</NutritionDetails>
</ID>
</Record>
</Records>
Once you've successfully created a proper, valid XML - you should be able to use this T-SQL code to grab the details:
SELECT
-- get the attributes from the <ID> node
IDSpecNum = XC.value('(ID/#SpecNum)[1]', 'int'),
IDIsseNum = XC.value('(ID/#IssueNum)[1]', 'int'),
IDSpecStatus = XC.value('(ID/#SpecStatus)[1]', 'varchar(100)'),
-- get the element values from the children of the <NutrientFacts> node
NutrientNameId = NUT.value('(NutrientNameId)[1]', 'varchar(100)'),
NutrientName = NUT.value('(NutrientName)[1]', 'varchar(100)'),
NutrientPer100gUnrounded = NUT.value('(NutrientPer100gUnrounded)[1]', 'decimal(20,4)'),
NutrientPer100gRounded = NUT.value('(NutrientPer100gRounded)[1]', 'decimal(20,4)')
FROM
dbo.YourTable
CROSS APPLY
-- get one XML fragment per <Record>
XmlData.nodes('/Records/Record') AS XT(XC)
CROSS APPLY
-- get one XML fragment per <NutrientFacts> inside
XC.nodes('ID/NutritionDetails/NutrientFacts') AS XT2(NUT)
The first CROSS APPLY basically get an "inline pseudo table" with one XML fragment for each <Record> node in your XML in the XmlData column of your table (this is just an assumption on my part - adapt to your reality!). These XML fragments are referenced as "pseudo-table" XT with a single column XC.
With that XC column's XML fragment, you can "reach in" and grab the attribute values from the <ID> node in the <Record> - that's the first three values.
Then, based on the XT pseudo table, I apply another CROSS APPLY to get all the <NutrientFacts> nodes inside <ID> / <NutritionDetails> - those are referenced as pseudo-table XT2 with column NUT, which again holds an XML fragment for each <NutrientFacts> node; I reach into that XML node and extract the values from the sub-elements of that node - those are the four additional values that are shown in the select.
Now that you have a SELECT that returns all the values - you can easily get those bits you need and use them in a INSERT INTO dbo.MyTable(list-of-columns) SELECT list-of-columns :...... scenario. Enjoy!
UPDATE: to import an XML file from disk (local disk on your SQL Server machine's file system) into your table - use something like this:
INSERT INTO dbo.YourTable(XmlData)
SELECT
CONVERT(XML, BulkColumn) AS BulkColumn
FROM
OPENROWSET(BULK 'C:\temp\records.xml', SINGLE_BLOB) AS x;
Again: adapt to your needs - I don't know if you want to insert additional information into dbo.YourTable - and I don't even know your table name; you can load one XML at a time from disk

Parse saved xml from table MSSQL server

I have a table with a column named xml. Table is text type but contains xml responses. I need 2 values from this column:
PL81300032102 from <ie801:Traderid>
Some Company sp. z o.o. from <ie801:TraderName>.
It is possible in SQL Server using a query?
<?xml version="1.0" encoding="UTF-8"?><EMCSToTrader xmlns="urn:publicid:-:PL:GOV:MF:EMCS:PHASE3:EMCS-TRADER:REQUEST:V1.00" xmlns:ie801="urn:publicid:-:EC:DGTAXUD:EMCS:PHASE3:IE801:V1.51" xmlns:tms="urn:publicid:-:EC:DGTAXUD:EMCS:PHASE3:TMS:V1.51" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Message><ie801:IE801>
<ie801:Header>
<tms:MessageSender>NDEA.PL</tms:MessageSender>
<tms:MessageRecipient>PL61300032004</tms:MessageRecipient>
<tms:DateOfPreparation>2018-07-17</tms:DateOfPreparation>
<tms:TimeOfPreparation>11:16:44.631</tms:TimeOfPreparation>
<tms:MessageIdentifier>PL#IE801#69474394</tms:MessageIdentifier>
</ie801:Header>
<ie801:Body>
<ie801:EADContainer>
<ie801:ConsigneeTrader language="pl">
<ie801:Traderid>PL81300032102</ie801:Traderid>
<ie801:TraderName>Some Company sp. z o.o.</ie801:TraderName> <...>
Table structure:
I was able to convert text data to xml type using:
SELECT TOP (10) * FROM (
SELECT CAST([xml] AS XML) AS xmlcontent
FROM [emcskomunikaty]
) det
Now trying to get value from xml.
I suppose you can do this:
SELECT
xmldata.value('declare namespace ns1="urn:publicid:-:EC:DGTAXUD:EMCS:PHASE3:IE801:V1.51"; (//ns1:Traderid)[1]', 'VARCHAR(100)') AS Traderid,
xmldata.value('declare namespace ns1="urn:publicid:-:EC:DGTAXUD:EMCS:PHASE3:IE801:V1.51"; (//ns1:TraderName)[1]', 'VARCHAR(100)') AS TraderName
FROM #t
CROSS APPLY (SELECT CAST(xml AS XML)) AS CA(xmldata)
The only tricky part here is handling the namespaces. If you choose ignore namespaces then just use //*:Traderid.

Flattening xml data in sql

I'm trying to flatten XML data in a SQL query but I always seem to get nulls.
I tried the cross/outer apply method described here.
The column with XML data is called Data.
I'm guessing that the xml data with these links need to be somehow also added?
Could you please help to get a proper SQL query?
Query I tried:
SELECT
v.name
,pref.value('(LocalId/text())[1]', 'nvarchar(10)') as localid
FROM [database].[requests] v
outer apply v.Data.nodes('/DataForm') x(pref)
GO
example of xml data in that column:
<Dataform xmlns="http://somelongasslink.org/hasalsosomestuffhere" xmlns:i="http://somexlmschemalink/">
<DeleteDate xmlns="http://somelongasslink.org/hasalsosomestuffhere" i:nil="true" />
<LocalId xmlns="http://somelongasslink.org/hasalsosomestuffhere">5325325</LocalId>
...
You can use this code to get the result you're looking for:
;WITH XMLNAMESPACES(DEFAULT 'http://somelongasslink.org/hasalsosomestuffhere')
SELECT
rq.Name,
LocalID = TC.value('(LocalId)[1]', 'nvarchar(10)')
FROM
[database].[requests] rq
CROSS APPLY
rq.Data.nodes('/Dataform') AS TX(TC)
GO
There were two problems with your code:
you're not respecting / including the XML namespace that's defined on the XML document
<Dataform xmlns="http://somelongasslink.org/hasalsosomestuffhere"
*******************************************************
you didn't pay attention to the case-sensitivity of XML in your call to .nodes() - you need to use .nodes('/Dataform') (not /DataForm - the F is not capitalized in your XML)

SQL Server 2005 with XML Namespaces - Query Multiple Nodes and Properties

I have the following XML and code and can pull back the AccountNumber and Status. I also need to pull back...
The ID property of the MFRRequest
Several elements from the CaseInformation node
I am not sure how to get the ID property and I'm wondering... can I get all of this information (from multiple nodes) back in the same query?
Thank you!
DECLARE #doc xml
SET #doc =
'<?xml version="1.0" encoding="UTF-8"?>
<p:OrderRequest xmlns:p="http://XXX.Schemas.OrderRequest/2" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://XXX.Schemas.OrderRequest/2 OrderRequestV2.xsd ">
<p:Header>
<p:SCRARequestCount>0</p:SCRARequestCount>
<p:MFRRequestCount>1</p:MFRRequestCount>
<p:FileCreatedAt>2001-12-31T12:00:00</p:FileCreatedAt>
<p:RequestFileName>p:RequestFileName</p:RequestFileName>
</p:Header>
<p:RequestItems>
<p:MFRRequest id="1" priority="">
<p:AccountNumber>9999999</p:AccountNumber>
<Status>Initial</Status>
<p:CaseInformation>
<ReferralDate>2011-01-01</ReferralDate>
<LoanType>1A</LoanType>
<ARM>Yes</ARM>
<InvestorNumber>InvestorNumber</InvestorNumber>
<PropertyAddress>PropertyAddress</PropertyAddress>
<PrivateLabel>Yes</PrivateLabel>
<CaseNumber>01-11111/AK/</CaseNumber>
<SuspenseBalance>9999.00</SuspenseBalance>
<TitleOrderedDate>2011-09-01</TitleOrderedDate>
<TotalMonthlyPayment>876.99</TotalMonthlyPayment>
</p:CaseInformation>
</p:MFRRequest>
</p:RequestItems>
</p:OrderRequest>'
;WITH XMLNAMESPACES('http://XXX.Schemas.OrderRequest/2' AS p)
SELECT
Y.i.value('Status[1]', 'varchar(10)') AS Status ,
Y.i.value('p:AccountNumber[1]', 'varchar(10)') AS AccountNumber
FROM #doc.nodes('/p:OrderRequest/p:RequestItems/p:MFRRequest') AS Y(i)
Try something like this:
;WITH XMLNAMESPACES('http://XXX.Schemas.OrderRequest/2' AS p)
SELECT
MFRRequestID = Y.i.value('(#id)[1]', 'int'),
RequestStatus = Y.i.value('Status[1]', 'varchar(10)') ,
AccountNumber = Y.i.value('p:AccountNumber[1]', 'varchar(10)'),
ReferralDate = CIF.value('(ReferralDate)[1]', 'varchar(25)'),
CaseInfoType = CIF.value('(Type)[1]', 'varchar(25)')
FROM
#doc.nodes('/p:OrderRequest/p:RequestItems/p:MFRRequest') AS Y(i)
CROSS APPLY
i.nodes('p:CaseInformation') AS Tbl(CIF)
Basically, once you have the <p:MFRRequest> XML element in your Y(i) pseudo-table, you can then again use a second call to .nodes() to get the (possibly multiple) <p:CaseInfomration> nodes and use .value() calls on this new second pseudo-table to extract single bits of information from it.

Safely retrieve values from element that can contain different values

We have some xml elements in a database that [for older data] can sometimes contain guids and sometimes contain integers.
is there a nice way of pulling out all the integrs only?
This will fail if the value element contains a guid!
select
ra.*,
t.c.value('.', 'int') as organisationId
from
Audit.EmployeeAudit ra
cross apply ra.EmployeeXml.nodes('//*:employee/*:property[*:name="ORG"]/*:value') t(c)
Sample Xml
<employee>
<property>
<name>ORG</name>
<value>39</value> <!-- Sometimes this will be a guid -->
<description>Leeds</description>
</property>
</employee>
You could add a predicate to only match entries of less than or equal to 10 characters.
;with EmployeeAudit as
(
SELECT CAST('<employee><property>
<name>ORG</name>
<value>39</value> <!-- Sometimes this will be a guid -->
<description>Leeds</description>
</property></employee>
' AS XML) AS EmployeeXml
UNION ALL
SELECT CAST('<employee><property>
<name>ORG</name>
<value>2FD29F11-59FC-47FD-BC30-DD330A53284E</value>
<description>Leeds</description>
</property></employee>
' AS XML)
)
select
ra.*,
t.c.value('.', 'int') as organisationId
from
EmployeeAudit ra
cross apply
ra.EmployeeXml.nodes
('//*:employee/*:property[*:name="ORG"]/*:value[string-length() <= 10]') t(c)
Or actually this might be a bit more robust
('//*:employee/*:property[*:name="ORG"]/*:value[ceiling(.) = .]') t(c)