XML To Sql Server - sql

I am trying to import an xml file into the sql server with no success yet.
The xml file is structured like this:
<Users xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<User>
<PartitionKey>be-BY</PartitionKey>
</User>
</Users>
I am using the following code:
SELECT
xmldata.value('(PartitionKey)[1]', 'NCHAR(10)') AS 'partition_key'
FROM
(SELECT CAST(x AS XML)
FROM OPENROWSET(
BULK 'C:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\Backup\SkillageXML\userstest1111.xml',
SINGLE_BLOB
)
AS T(x)
) AS T(x)
CROSS APPLY
x.nodes('/Users/User') AS X(xmldata);
However, I don't see any value after it is done with processing the file. Is there anything missing?

This works like a charm for me:
SELECT
XUsers.value('(PartitionKey)[1]', 'NCHAR(10)') AS 'partition_key'
FROM
(SELECT
BulkXML = CAST(BulkColumn AS XML)
FROM
OPENROWSET(BULK 'C:\Program Files\Microsoft SQL Server\MSSQL10.MSSQLSERVER\MSSQL\Backup\SkillageXML\userstest1111.xml', SINGLE_BLOB) AS BX
) AS T
CROSS APPLY
BulkXml.nodes('/Users/User') AS XTbl(XUsers);and returns:
partition_key
be-BY
I think you're approach of using the AS T(x) alias twice is causing confusion - try to use something more meaningful and not the same alias for both things.

Related

SSIS Extraction/ DTSX

I got this amazing code here on Stack but it was 8 years ago but works perfectly but, when run the t-script the column ObjectName always is NULL, someone can help me to fix it if possible?
The SQL Script
SELECT Pkg.props.value('../../DTS:Property[#DTS:Name="ObjectName"]
[1]','varchar(MAX)') ObjectName,
Pkg.props.value('(#SQLTask:SqlStatementSource)[1]', 'NVARCHAR(MAX)') AS
SqlStatement FROM (select cast(pkgblob.BulkColumn as XML) pkgXML from
openrowset(bulk '\\MYDTS.dtsx',single_blob)
as pkgblob) t CROSS APPLY pkgXML.nodes('//DTS:ObjectData//SQLTask:SqlTaskData') Pkg(props)
UNION
SELECT Pkg.props.value('../../../../DTS:Property[#DTS:Name="ObjectName"]
[1]','varchar(MAX)') ObjectName,
Pkg.props.value('data(./properties/property[#name=''SqlCommand''])[1]',
'varchar(max)') SqlStatement FROM(select cast(pkgblob.BulkColumn as XML)
pkgXML from openrowset(bulk '\\MYDTS.dtsx',single_blob) as
pkgblob) t CROSS APPLY
pkgXML.nodes('//DTS:Executable//pipeline//components//component') Pkg(props)
WHERE Pkg.props.value('data(./properties/property[#name=''SqlCommand''])
[1]', 'varchar(max)') <>''
And the result has been attached below. I would like to show the ObjectName instead NULL and I don't know how I can fix it.
ObjectName is no longer what/where it use to be from those years ago.
You'll need to browse the current dtsx in XML form and find your equivalent ObjectName.
SELECT CONVERT(XML, BulkColumn) AS BulkColumn FROM OPENROWSET(BULK 'C:\PathToPackage\MYDTS.dtsx', SINGLE_BLOB) AS x;
My sample dtsx looks like this in XML form:
<DTS:Executable xmlns:DTS="www.microsoft.com/SqlServer/Dts" DTS:refId="Package" DTS:CreationDate="7/23/2019 7:23:29 AM" DTS:CreationName="Microsoft.Package" DTS:CreatorComputerName="Overflow" DTS:CreatorName="Overflow\Stack" DTS:DTSID="{DEED1FFC-FCE3-4C63-A90C-3382EDA388A1}" DTS:ExecutableType="Microsoft.Package" DTS:LastModifiedProductVersion="14.0.3002.113" DTS:LocaleID="1033" DTS:ObjectName="deduplicate" DTS:PackageType="5" DTS:VersionBuild="20" DTS:VersionGUID="{27B38889-4D80-45F6-9CB0-F6B430B1DBBE}">
<DTS:Property DTS:Name="PackageFormatVersion">8</DTS:Property>
<DTS:Variables />
<DTS:Executables>
<DTS:Executable DTS:refId="Package\Execute SQL Task" DTS:CreationName="Microsoft.ExecuteSQLTask" DTS:Description="Execute SQL Task" DTS:DTSID="{A6053AE9-7445-4973-9AFD-E3AFFE97B68F}" DTS:ExecutableType="Microsoft.ExecuteSQLTask" DTS:LocaleID="-1" DTS:ObjectName="Execute SQL Task" DTS:TaskContact="Execute SQL Task; Microsoft Corporation; SQL Server 2017 RC1; © 2017 Microsoft Corporation; All Rights Reserved;http://www.microsoft.com/sql/support/default.asp;1" DTS:ThreadHint="1">
<DTS:Variables />
<DTS:ObjectData>
<SQLTask:SqlTaskData xmlns:SQLTask="www.microsoft.com/sqlserver/dts/tasks/sqltask" SQLTask:SqlStatementSource="Select GetDate()" />
</DTS:ObjectData>
</DTS:Executable>
...
In this specific XML, the next todos:
Pull out everything that is a SqlTaskData
Retrieve the SqlStatementSource value
Walk up two levels "../../" to DTS:Executable and Retrieve the ObjectName Attribute
WITH XMLNAMESPACES(
'www.microsoft.com/SqlServer/Dts' AS DTS,
'www.microsoft.com/sqlserver/dts/tasks/sqltask' AS SQLTask
),
dtsxAsXML AS (
SELECT CONVERT(XML, BulkColumn) AS BulkColumn
FROM OPENROWSET(BULK 'C:\PathToPackage\MYDTS.dtsx', SINGLE_BLOB) AS x
)
SELECT
X.Y.value('(../../#DTS:ObjectName)[1]', 'NVARCHAR(MAX)') AS ObjectName,
X.Y.value('(#SQLTask:SqlStatementSource)[1]', 'NVARCHAR(MAX)') AS SqlStatementSource
FROM dtsxAsXML d
CROSS APPLY d.BulkColumn.nodes('//DTS:ObjectData//SQLTask:SqlTaskData') X(Y)
ObjectName
SqlStatementSource
Execute SQL Task
Select GetDate()
Repeat this process for the dataflow component, the bottom union in your example.
Since you're already getting your SQL Statements, you don't need to use these example queries, you just need to review the xml from your dtsx, then find and change the relative paths in your existing query to the value you want.

Parsing XML Data Into SQL Server

I am struggling with importing XML Data into SQL Server 2016. I have tried a few things, but keep either getting errors or just no data is returned.
I have this XML Data stored in an XML file (limited the data because it is pretty sensitive:
<?xml version='1.0' encoding='UTF-8'?>
<wd:Report_Data xmlns:wd="urn:com.workday.report/Worker_Details_-_EXPORT_-_Workplace">
<wd:Report_Entry>
<wd:Active_Status>0</wd:Active_Status>
<wd:Legal_Name_-_First_Name>Charlotte</wd:Legal_Name_-_First_Name>
<wd:Position>Executive Housekeeper I</wd:Position>
<wd:Worker_Management_Level>Supervisor</wd:Worker_Management_Level>
<wd:continuous_service_date>1979-04-29-08:00</wd:continuous_service_date>
<wd:Hire_Date>1979-04-29-08:00</wd:Hire_Date>
<wd:termination_date>2019-12-22-08:00</wd:termination_date>
<wd:Anniversary_Month>04</wd:Anniversary_Month>
<wd:Years_of_Service>40</wd:Years_of_Service>
<wd:Employee_Type>Hotel</wd:Employee_Type>
<wd:Time_Type>Full Time</wd:Time_Type>
<wd:Pay_Rate_Type>Salary</wd:Pay_Rate_Type>
<wd:Marital_Status>Single</wd:Marital_Status>
</wd:Report_Entry>
<wd:Report_Entry>
<wd:Active_Status>0</wd:Active_Status>
<wd:Legal_Name_-_First_Name>Robert</wd:Legal_Name_-_First_Name>
<wd:Cost_Center_-_Name>Electronics</wd:Cost_Center_-_Name>
<wd:Work_Address_-_State_Province>Missouri</wd:Work_Address_-_State_Province>
<wd:Position>Manager Of Voice Networks</wd:Position>
<wd:Worker_Management_Level>Manager</wd:Worker_Management_Level>
<wd:continuous_service_date>1980-02-25-08:00</wd:continuous_service_date>
<wd:Hire_Date>1980-02-25-08:00</wd:Hire_Date>
<wd:termination_date>2020-03-22-07:00</wd:termination_date>
<wd:Anniversary_Month>02</wd:Anniversary_Month>
<wd:Years_of_Service>40</wd:Years_of_Service>
<wd:Employee_Type>Corporate</wd:Employee_Type>
<wd:Time_Type>Full Time</wd:Time_Type>
<wd:Pay_Rate_Type>Salary</wd:Pay_Rate_Type>
<wd:Marital_Status>Married</wd:Marital_Status>
</wd:Report_Entry>
</wd:Report_Data>
I have this code that I am trying to use, but keep getting just an empty result:
SELECT
XMLCol.ReportEntry.query('Active_Status').value('.', 'VARCHAR(20)') AS ActiveStatus
FROM
(SELECT
CAST(XMLCol AS XML)
FROM
OPENROWSET(BULK '\\afcn2011\root\DATA\VisualCron\Employee Export\EmployeeExport.xml', SINGLE_BLOB) AS T(XMLCol)
) AS T(XMLCol)
CROSS APPLY
XMLCol.nodes('Report_Data/Report_Entry') AS XMLCol(ReportEntry);
You need to respect and include the XML namespace defined in your document.
Try something like this:
-- define the namespace and give it a prefix - here "wd"
;WITH XMLNAMESPACES ('urn:com.workday.report/Worker_Details_-_EXPORT_-_Workplace' as wd)
SELECT
-- you need to include namespace prefix when referring to the XML element
-- also: is "VARCHAR(20)" really the best datatype?? Looks more like "INT" to me ...
XMLCol.ReportEntry.value('(wd:Active_Status/text())[1]', 'VARCHAR(20)') AS ActiveStatus
FROM
(SELECT
CAST(XMLCol AS XML)
FROM
OPENROWSET(BULK '\\afcn2011\root\DATA\VisualCron\Employee Export\EmployeeExport.xml', SINGLE_BLOB) AS T(XMLCol)
) AS T(XMLCol)
CROSS APPLY
-- you need to include namespace prefix in your XPath expression
XMLCol.nodes('/wd:Report_Data/wd:Report_Entry') AS XMLCol(ReportEntry);

Parse selected values from XML in SQL

I have the following XML file (I put here only a short scheme), it's a SEPA XML bank statement. I'm not familiar with parsing XML files, my next move will be inserting and comparing to data stored in the SQL databases for error checks. Sadly, I know what to do next, don't know how to make progress with my first step. All I need is to create a table to select values of 2 attributes from a file stored at a particular place
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<GrpHdr>
..........
</GrpHdr>
<Stmt>
<Ntry>
<Amt Ccy="EUR">RequestedAmount1</Amt>
<AddtlNtryInf>RequestedInfo1</AddtlNtryInf>
</Ntry>
<Ntry>
<Amt Ccy="EUR">RequestedAmount2</Amt>
<AddtlNtryInf>RequestedInfo2</AddtlNtryInf>
</Ntry>
<Ntry>
<Amt Ccy="EUR">RequestedAmount3</Amt>
<AddtlNtryInf>RequestedInfo3</AddtlNtryInf>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>
If the XML structure was simpler, for example like this...
<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<Ntry>
<Amt Ccy="EUR">RequestedAmount1</Amt>
<AddtlNtryInf>RequestedInfo1</AddtlNtryInf>
</Ntry>
<Ntry>
<Amt Ccy="EUR">RequestedAmount2</Amt>
<AddtlNtryInf>RequestedInfo2</AddtlNtryInf>
</Ntry>
<Ntry>
<Amt Ccy="EUR">RequestedAmount3</Amt>
<AddtlNtryInf>RequestedInfo3</AddtlNtryInf>
</Ntry>
...then I'd use this query to select requested attributes Amt and AddtlNtryInf and it works perfectly
SELECT
MY_XML.Ntry.query('Amt').value('.', 'NVARCHAR(255)') AS Amt,
MY_XML.Ntry.query('AddtlNtryInf').value('.', 'NVARCHAR(255)') AS AddtlNtryInf
FROM (SELECT CAST(MY_XML AS xml)
FROM OPENROWSET(BULK 'C:\tmp\TestSqlSimple.xml', SINGLE_BLOB) AS T(MY_XML)) AS T(MY_XML)
CROSS APPLY MY_XML.nodes('Ntry') AS MY_XML (Ntry);
But don't know how to deal with that more complicated one. I've tried something like this and several similar attempts but I failed because it doesn't select anything, the result is nothing
SELECT
MY_XML.Ntry.query('Amt').value('.', 'NVARCHAR(255)') AS Amt,
MY_XML.Ntry.query('AddtlNtryInf').value('.', 'NVARCHAR(255)') AS AddtlNtryInf
FROM (SELECT CAST(MY_XML AS xml)
FROM OPENROWSET(BULK 'C:\tmp\TestSqlSimple.xml', SINGLE_BLOB) AS T(MY_XML)) AS T(MY_XML)
CROSS APPLY MY_XML.nodes('/Document/BkToCstmrStmt/Stmt/Ntry') AS MY_XML (Ntry);
Can't figure out what to do with that CROSS APPLY. Thank you very much for any suggestions or improvements, you're doing a great job
You were almost there.
(1) The XML file has a default namespace, and it needs a special treatment via XMLNAMESPACES clause.
(2) The Amt element probably has a numeric value so you could use DECIMAL(x,y) data type. But I kept the NVARCHAR(255) to match the obfuscated XML file example.
(3) The SQL below is using .value() method without unnecessary .query() method.
(4) It is a good practice to use elementName/text() technique for performance reasons. It is MS SQL Server specific peculiarity.
SQL
-- DDL and sample data population, start
DECLARE #tbl TABLE (
ID INT IDENTITY PRIMARY KEY,
Amt NVARCHAR(255),
AddtlNtryInf NVARCHAR(255)
);
-- DDL and sample data population, end
;WITH XMLNAMESPACES (DEFAULT 'urn:iso:std:iso:20022:tech:xsd:camt.053.001.02')
, XmlFile (xmlData) AS
(
SELECT TRY_CAST(BulkColumn AS XML)
FROM OPENROWSET(BULK 'e:\Temp\TestSqlSimple.xml', CODEPAGE = '65001', SINGLE_BLOB) AS x
)
INSERT INTO #tbl (Amt, AddtlNtryInf)
SELECT c.value('(Amt/text())[1]', 'NVARCHAR(255)') AS Amt
, c.value('(AddtlNtryInf/text())[1]', 'NVARCHAR(255)') AS AddtlNtryInf
FROM XmlFile CROSS APPLY xmlData.nodes('/Document/BkToCstmrStmt/Stmt/Ntry') AS t(c);
-- test
SELECT * FROM #tbl;
Something like this:
declare #doc xml = '<?xml version="1.0" encoding="utf-8" standalone="yes"?>
<Document xmlns="urn:iso:std:iso:20022:tech:xsd:camt.053.001.02">
<BkToCstmrStmt>
<GrpHdr>
..........
</GrpHdr>
<Stmt>
<Ntry>
<Amt Ccy="EUR">RequestedAmount1</Amt>
<AddtlNtryInf>RequestedInfo1</AddtlNtryInf>
</Ntry>
<Ntry>
<Amt Ccy="EUR">RequestedAmount2</Amt>
<AddtlNtryInf>RequestedInfo2</AddtlNtryInf>
</Ntry>
<Ntry>
<Amt Ccy="EUR">RequestedAmount3</Amt>
<AddtlNtryInf>RequestedInfo3</AddtlNtryInf>
</Ntry>
</Stmt>
</BkToCstmrStmt>
</Document>';
with xmlnamespaces (DEFAULT 'urn:iso:std:iso:20022:tech:xsd:camt.053.001.02')
select s.Stmt.value('(GrpHdr)[1]', 'varchar(200)') GrpHdr,
n.Ntry.value('(Amt)[1]', 'varchar(200)') Amt,
n.Ntry.value('(AddtlNtryInf)[1]', 'varchar(200)') AddtlNtryInf
from #doc.nodes('/Document/BkToCstmrStmt') s(Stmt)
outer apply s.Stmt.nodes('Stmt/Ntry') n(Ntry)

Parse saved xml from table MSSQL server

I have a table with a column named xml. Table is text type but contains xml responses. I need 2 values from this column:
PL81300032102 from <ie801:Traderid>
Some Company sp. z o.o. from <ie801:TraderName>.
It is possible in SQL Server using a query?
<?xml version="1.0" encoding="UTF-8"?><EMCSToTrader xmlns="urn:publicid:-:PL:GOV:MF:EMCS:PHASE3:EMCS-TRADER:REQUEST:V1.00" xmlns:ie801="urn:publicid:-:EC:DGTAXUD:EMCS:PHASE3:IE801:V1.51" xmlns:tms="urn:publicid:-:EC:DGTAXUD:EMCS:PHASE3:TMS:V1.51" xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
<Message><ie801:IE801>
<ie801:Header>
<tms:MessageSender>NDEA.PL</tms:MessageSender>
<tms:MessageRecipient>PL61300032004</tms:MessageRecipient>
<tms:DateOfPreparation>2018-07-17</tms:DateOfPreparation>
<tms:TimeOfPreparation>11:16:44.631</tms:TimeOfPreparation>
<tms:MessageIdentifier>PL#IE801#69474394</tms:MessageIdentifier>
</ie801:Header>
<ie801:Body>
<ie801:EADContainer>
<ie801:ConsigneeTrader language="pl">
<ie801:Traderid>PL81300032102</ie801:Traderid>
<ie801:TraderName>Some Company sp. z o.o.</ie801:TraderName> <...>
Table structure:
I was able to convert text data to xml type using:
SELECT TOP (10) * FROM (
SELECT CAST([xml] AS XML) AS xmlcontent
FROM [emcskomunikaty]
) det
Now trying to get value from xml.
I suppose you can do this:
SELECT
xmldata.value('declare namespace ns1="urn:publicid:-:EC:DGTAXUD:EMCS:PHASE3:IE801:V1.51"; (//ns1:Traderid)[1]', 'VARCHAR(100)') AS Traderid,
xmldata.value('declare namespace ns1="urn:publicid:-:EC:DGTAXUD:EMCS:PHASE3:IE801:V1.51"; (//ns1:TraderName)[1]', 'VARCHAR(100)') AS TraderName
FROM #t
CROSS APPLY (SELECT CAST(xml AS XML)) AS CA(xmldata)
The only tricky part here is handling the namespaces. If you choose ignore namespaces then just use //*:Traderid.

issue with xml parsing data to sql query

I am working on SQL query where i need to parse xml to sql .Query is working for other xml but not working for below xml. query i am using is as below
DECLARE #xmldata XML
SET #xmldata =
N'<SearchProductsResponse xmlns="http://api.abc.com/">
<productItems>
<productItem id="5d0ee86d84bcc5edef43236d61419a59">
<trackingLinks>
<trackingLink adspaceId="100">
<ppv>
abc.com
</ppv>
<ppc>
abc.com
</ppc>
</trackingLink>
</trackingLinks>
</productItem>
</productItems>
</SearchProductsResponse>';
select
t1.c.value('#id', 'varchar(300)') as itemid,
c.c.value('#id', 'int') as adspaceId
from
#xmldata.nodes('*:SearchProductsResponse/*:productItems/*:productItem') as t1(c)
OUTER APPLY t1.c.nodes('*:trackingLinks/*:trackingLink') as c(c)
Output I am getting is
itemid adspaceId
5d0ee86d84bcc5edef43236d61419a59 NULL
But I should return 100 instead of NULL . This query is working for other XML but don't know whats wrong with this XML. I have double check XML and query nothing different from other XML. Hope I am not missing some silly mistake
Change the attribute selection for your adspaceId column to #adspaceId.
select
t1.c.value('#id', 'varchar(300)') as itemid,
c.c.value('#adspaceId', 'int') as adspaceId
from #xmldata.nodes('*:SearchProductsResponse/*:productItems/*:productItem') as t1(c)
OUTER APPLY t1.c.nodes('*:trackingLinks/*:trackingLink') as c(c)