How to add additional XML node on top of my SQL generated XML - sql

I have generated XML from a SQL Server FOR XML PATH statement as shown here:
USE MySQLDB
SELECT *
FROM BillTable
FOR XML PATH ('BillAdd'), ROOT ('BillAddRq')
And this is the result:
<BillAddRq>
<BillAdd>
<TxnID>2432-1071510295</TxnID>
<TimeCreated>2003-12-16T01:44:55</TimeCreated>
<TimeModified>2015-12-15T22:38:33</TimeModified>
<EditSequence>1450190313</EditSequence>
<TxnNumber>413</TxnNumber>
<VendorRef_ListID>E0000-933272656</VendorRef_ListID>
<VendorRef_FullName>Timberloft Lumber</VendorRef_FullName>
<APAccountRef_ListID>C0000-933270541</APAccountRef_ListID>
<APAccountRef_FullName>Accounts Payable</APAccountRef_FullName>
<TxnDate>2016-12-01T00:00:00</TxnDate>
<DueDate>2017-12-31T00:00:00</DueDate>
<AmountDue>80.50000</AmountDue>
<TermsRef_ListID>50000-933272659</TermsRef_ListID>
<TermsRef_FullName>1% 10 Net 30</TermsRef_FullName>
<IsPaid>0</IsPaid>
</BillAdd>
<BillAdd>
<TxnID>243A-1071510389</TxnID>
<TimeCreated>2003-12-16T01:46:29</TimeCreated>
<TimeModified>2015-12-15T22:38:33</TimeModified>
<EditSequence>1450190313</EditSequence>
<TxnNumber>414</TxnNumber>
<VendorRef_ListID>C0000-933272656</VendorRef_ListID>
<VendorRef_FullName>Perry Windows & Doors</VendorRef_FullName>
<APAccountRef_ListID>C0000-933270541</APAccountRef_ListID>
<APAccountRef_FullName>Accounts Payable</APAccountRef_FullName>
<TxnDate>2016-12-02T00:00:00</TxnDate>
<DueDate>2018-01-01T00:00:00</DueDate>
<AmountDue>50.00000</AmountDue>
<TermsRef_ListID>10000-933272658</TermsRef_ListID>
<TermsRef_FullName>Net 30</TermsRef_FullName>
<IsPaid>0</IsPaid>
</BillAdd>
</BillAddRq>
Now, I'd like to encapsulate the above with these nodes:
<?xml version="1.0" encoding="utf-8"?>
<?qbxml version="15.0"?>
<QBXML>
<QBXMLMsgsRq onError="stopOnError">
//above generated xml//
</QBXMLMsgsRq>
</QBXML>
How will I achieve this in a SQL Query I created above?
I am new to SQL Server and XML. I am trying to generate this XML directly from my database and vice versa to make it more efficient and faster — let my SQL directly communicate with XML.
ATTEMPT 1:
USE MySQLDB;
GO
DECLARE #myDoc XML;
SET #myDoc = '<QBXML>
<QBXMLMsgsRq onError="stopOnError">
</QBXMLMsgsRq>
</QBXML>';
SET #myDoc.modify('
insert
-- instead of inserting string here.. I would like to insert here the query I made above
into (/QBXML/QBXMLMsgsRq)[1]');
SELECT #myDoc;
ATTEMPT 2:
USE MySQLDB;
GO
DECLARE #myDoc XML;
SET #myDoc = '<QBXML>
<QBXMLMsgsRq onError="stopOnError">
</QBXMLMsgsRq>
</QBXML>';
DECLARE #qry XML;
SET #qry = (SELECT * FROM BillTable FOR XML PATH ('BillAdd'), ROOT ('BillAddRq'));
-- SELECT #qry;
SET #myDoc.modify('insert #qry
into (/QBXML/QBXMLMsgsRq)[1]');
SELECT #myDoc;

There are many ways to construct your XML result, consider the following three alternatives...
Use XML.modify() to insert the BillTable XML into an XML scalar variable (which includes the ?qbxml XML processing instruction):
declare #BillTableXml xml = (
select *
from BillTable
for xml path('BillAdd'), root('BillAddRq')
);
declare #myDoc xml = '<?xml version="1.0" encoding="utf-8"?>
<?qbxml version="15.0"?>
<QBXML>
<QBXMLMsgsRq onError="stopOnError">
</QBXMLMsgsRq>
</QBXML>';
set #myDoc.modify('
insert sql:variable("#BillTableXml")
into (/QBXML/QBXMLMsgsRq)[1]
');
select #myDoc as Result;
Use a nested query to construct the entire XML result (which does not, however, include the ?qbxml XML processing instruction):
select
'stopOnError' as [QBXML/QBXMLMsgsRq/#onError],
(
select *
from BillTable
for xml path('BillAdd'), root('BillAddRq'), type
) as [QBXML/QBXMLMsgsRq]
for xml path('');
Or use an XQuery to construct the entire XML result (which also includes the ?qbxml XML processing instruction):
select BillTableXml.query('
<?qbxml version="15.0"?>,
<QBXML>
<QBXMLMsgsRq onError="stopOnError">
{ /BillAddRq }
</QBXMLMsgsRq>
</QBXML>
') as Result
from (
select *
from BillTable
for xml path('BillAdd'), root('BillAddRq'), type
) Data (BillTableXml);

Related

SSMS Obfuscation of string

I need to create a script to obfuscate some data.
The string is quite long and I need to obfuscate only some parts of it.
In the table the records are similar to this:
<?xml version="1.0" encoding="UTF-8"?><CONTRACT><IBC IBC_REF="f45f1231234ae5ac2easdasdfde5dfd" IBC_TYPE="I" TELEPHONE_1="1111111" TELEPHONE_2="11111111" MOBILE_PHONE="11111111" E_MAIL="asdasdasd#hotmail.com" SOLICITATION_MAIL="0" ARREARS_MAIL="1" MAIL_REDIRECTED="0" TITLE="Mrs" SURNAME_REGISTERED_NAME="Assadasd"
And it needs to become like this:
<?xml version="1.0" encoding="UTF-8"?><CONTRACT><IBC IBC_REF="f45f1231234ae5ac2easdasdfde5dfd" IBC_TYPE="I" TELEPHONE_1="Telephone-1" TELEPHONE_2="Telephone-2" MOBILE_PHONE="MobilePhone" E_MAIL="email-1" SOLICITATION_MAIL="0" ARREARS_MAIL="1" MAIL_REDIRECTED="0" TITLE="Mrs" SURNAME_REGISTERED_NAME="Surname"
How can I update all the rows of the table and change only some of the strings by saving the other words?
Seemingly you are working with XML strings, so I would suggest to use the SQL Server XML functionalities. Following a short example:
DECLARE #input NVARCHAR(4000) = '<?xml version="1.0" encoding="UTF-8"?><CONTRACT><IBC IBC_REF="f45f1231234ae5ac2easdasdfde5dfd" IBC_TYPE="I" TELEPHONE_1="1111111" TELEPHONE_2="11111111" MOBILE_PHONE="11111111" E_MAIL="asdasdasd#hotmail.com" SOLICITATION_MAIL="0" ARREARS_MAIL="1" MAIL_REDIRECTED="0" TITLE="Mrs" SURNAME_REGISTERED_NAME="Assadasd" /></CONTRACT>';
DECLARE #x xml = CONVERT(xml, REPLACE(#input,'encoding="UTF-8"','encoding="UTF-16"'));
SELECT #x.value('(/CONTRACT/IBC/#TELEPHONE_1)[1]', 'nvarchar(100)');
DECLARE #y xml = #x
SET #y.modify('replace value of (/CONTRACT/IBC/#TELEPHONE_1)[1] with "TELEPHONE_1"');
SELECT #y.value('(/CONTRACT/IBC/#TELEPHONE_1)[1]', 'nvarchar(100)');
SELECT #y
DECLARE #output NVARCHAR(4000) = '<?xml version="1.0" encoding="UTF-8"?>' + CONVERT(NVARCHAR(4000), #y)
SELECT #output

Passing irregular xml file to the stored procedure

I have a sample xml as follows. I am trying to get all or specific data and then insert into the my sql table which has the same columns representing the values coming from xml. I looked through some solutions but the xml files are not formatted like I have in here. Can you help me?
<?xml version="1.0" encoding="UTF-8" standalone="true"?>
<VehicleStatusResponse xmlns:ns2=
"http://fms-standard.com/rfms/v1.0.0/xsd/common/position" xmlns="http://fms-standard.com/rfms/v1.0.0/xsd/status">
<VehicleStatus>
<VIN>VF254ANA735752628</VIN>
<TriggerType>TIMER</TriggerType>
<CreatedDateTime>2014-09-08T09:30:20</CreatedDateTime>
<ReceivedDateTime>2014-09-08T09:30:57</ReceivedDateTime>
<GNSSPosition>
<ns2:Latitude>49.18557</ns2:Latitude>
<ns2:Longitude>11.18557</ns2:Longitude>
<ns2:Heading>33</ns2:Heading>
<ns2:Altitude>500</ns2:Altitude>
<ns2:Speed>16.4</ns2:Speed>
<ns2:PositionDateTime>2014-09-08T09:30:20</ns2:PositionDateTime>
</GNSSPosition>
<WheelBasedSpeed>16.07</WheelBasedSpeed>
<TachographSpeed>15.83</TachographSpeed>
<HRTotalVehicleDistance>817.5</HRTotalVehicleDistance>
<EngineTotalFuelUsed>575</EngineTotalFuelUsed>
<FuelLevel1>83</FuelLevel1>
<CatalystFuelLevel>88.48</CatalystFuelLevel>
<GrossCombinationVehicleWeight>10000</GrossCombinationVehicleWeight>
</VehicleStatus>
</VehicleStatusResponse>
You can use an XML type and XML methods if you remove or modify the declaration. The SQL Server XML type only supports UCS-2 encoding and doesn't recognize "standalone". The example below uses string manipulation to tweak the declaration. You'll need to change the data types according to your actual column types and should specify an explicit column list on the INSERT statement. I omitted that in this example only because I didn't want to assume your actual table columns matched the element names in the XML.
DECLARE #xml xml;
DECLARE #xmlString nvarchar(MAX) = N'<?xml version="1.0" encoding="UTF-8" standalone="true"?>
<VehicleStatusResponse xmlns:ns2=
"http://fms-standard.com/rfms/v1.0.0/xsd/common/position" xmlns="http://fms-standard.com/rfms/v1.0.0/xsd/status">
<VehicleStatus>
<VIN>VF254ANA735752628</VIN>
<TriggerType>TIMER</TriggerType>
<CreatedDateTime>2014-09-08T09:30:20</CreatedDateTime>
<ReceivedDateTime>2014-09-08T09:30:57</ReceivedDateTime>
<GNSSPosition>
<ns2:Latitude>49.18557</ns2:Latitude>
<ns2:Longitude>11.18557</ns2:Longitude>
<ns2:Heading>33</ns2:Heading>
<ns2:Altitude>500</ns2:Altitude>
<ns2:Speed>16.4</ns2:Speed>
<ns2:PositionDateTime>2014-09-08T09:30:20</ns2:PositionDateTime>
</GNSSPosition>
<WheelBasedSpeed>16.07</WheelBasedSpeed>
<TachographSpeed>15.83</TachographSpeed>
<HRTotalVehicleDistance>817.5</HRTotalVehicleDistance>
<EngineTotalFuelUsed>575</EngineTotalFuelUsed>
<FuelLevel1>83</FuelLevel1>
<CatalystFuelLevel>88.48</CatalystFuelLevel>
<GrossCombinationVehicleWeight>10000</GrossCombinationVehicleWeight>
</VehicleStatus>
</VehicleStatusResponse>';
SET #xmlString = REPLACE(#xmlString, 'encoding="UTF-8"', 'encoding="UCS-2"');
SET #xmlString = REPLACE(#xmlString, 'standalone="true"', '');
SET #xml = #xmlString;
WITH XMLNAMESPACES (
DEFAULT 'http://fms-standard.com/rfms/v1.0.0/xsd/status'
,'http://fms-standard.com/rfms/v1.0.0/xsd/common/position' AS ns2
)
INSERT INTO dbo.YourTable
SELECT
#xml.value('(/VehicleStatusResponse/VehicleStatus/VIN)[1]', 'varchar(50)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/TriggerType)[1]', 'varchar(50)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/CreatedDateTime)[1]', 'datetime2(3)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/ReceivedDateTime)[1]', 'datetime2(3)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/GNSSPosition/ns2:Latitude)[1]', 'decimal(8,5)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/GNSSPosition/ns2:Longitude)[1]', 'decimal(8,5)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/GNSSPosition/ns2:Heading)[1]', 'int')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/GNSSPosition/ns2:Altitude)[1]', 'int')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/GNSSPosition/ns2:Speed)[1]', 'decimal(8,3)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/GNSSPosition/ns2:PositionDateTime)[1]', 'datetime2(3)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/WheelBasedSpeed)[1]', 'decimal(8,3)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/TachographSpeed)[1]', 'decimal(8,3)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/HRTotalVehicleDistance)[1]', 'decimal(8,3)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/EngineTotalFuelUsed)[1]', 'int')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/CatalystFuelLevel)[1]', 'decimal(8,3)')
, #xml.value('(/VehicleStatusResponse/VehicleStatus/GrossCombinationVehicleWeight)[1]', 'int');
First of all you need to get your value into a declare variable of type XML or into an XML-typed data table column. As your XML contains namespaces you have to declare them in a WITH XMLNAMESPACES first. You might use wildcard syntax (*:), but its better to be as specific as possible.
The .nodes() call navigates to the Level of <VehicleStatus>. All elements below are simply 1:1 and easy to read...
You can try it like this:
DECLARE #xml XML=
N'<VehicleStatusResponse xmlns:ns2=
"http://fms-standard.com/rfms/v1.0.0/xsd/common/position" xmlns="http://fms-standard.com/rfms/v1.0.0/xsd/status">
<VehicleStatus>
<VIN>VF254ANA735752628</VIN>
<TriggerType>TIMER</TriggerType>
<CreatedDateTime>2014-09-08T09:30:20</CreatedDateTime>
<ReceivedDateTime>2014-09-08T09:30:57</ReceivedDateTime>
<GNSSPosition>
<ns2:Latitude>49.18557</ns2:Latitude>
<ns2:Longitude>11.18557</ns2:Longitude>
<ns2:Heading>33</ns2:Heading>
<ns2:Altitude>500</ns2:Altitude>
<ns2:Speed>16.4</ns2:Speed>
<ns2:PositionDateTime>2014-09-08T09:30:20</ns2:PositionDateTime>
</GNSSPosition>
<WheelBasedSpeed>16.07</WheelBasedSpeed>
<TachographSpeed>15.83</TachographSpeed>
<HRTotalVehicleDistance>817.5</HRTotalVehicleDistance>
<EngineTotalFuelUsed>575</EngineTotalFuelUsed>
<FuelLevel1>83</FuelLevel1>
<CatalystFuelLevel>88.48</CatalystFuelLevel>
<GrossCombinationVehicleWeight>10000</GrossCombinationVehicleWeight>
</VehicleStatus>
</VehicleStatusResponse>';
--This is the query
WITH XMLNAMESPACES(DEFAULT 'http://fms-standard.com/rfms/v1.0.0/xsd/status'
,'http://fms-standard.com/rfms/v1.0.0/xsd/common/position' AS ns2)
SELECT vs.value('VIN[1]','nvarchar(max)') AS VehicleStatus_VIN
,vs.value('TriggerType[1]','nvarchar(max)') AS VehicleStatus_TriggerType
,vs.value('CreatedDateTime[1]','datetime') AS VehicleStatus_CreatedDateTime
,vs.value('ReceivedDateTime[1]','datetime') AS VehicleStatus_ReceivedDateTime
,vs.value('(GNSSPosition/ns2:Latitude)[1]','decimal(14,6)') AS VehicleStatus_GNSSPosition_Latitude
,vs.value('(GNSSPosition/ns2:Longitude)[1]','decimal(14,6)') AS VehicleStatus_GNSSPosition_Longitude
/*other columns follow the same pattern*/
FROM #xml.nodes('/VehicleStatusResponse/VehicleStatus') AS A(vs)
update: insert into a table
Easiest was to wrap this call as CTE like
WITH XMLNAMESPACES(...)
,DerivedTableCTE AS
(
The query here
)
INSERT INTO YourTable (col1, col2, col3, ...)
SELECT col1, col2, col3, ...
FROM DerivedTableCTE

Parse the XML in SQL Server

<ROOT>
<arn>arn001</arn>
<arn>arn002</arn>
</ROOT>
Tried the following code though
SELECT
ARN.value('(//arn/text())[1]','VARCHAR(100)') AS arns --TAG
FROM
#xml.nodes('/ROOT')AS TEMPTABLE(ARN)
It returns only first value
Try this way :
declare #xml xml = '<ROOT>
<arn>arn001</arn>
<arn>arn002</arn>
</ROOT>'
SELECT
X.value('.','VARCHAR(100)') AS arns
FROM
#xml.nodes('/ROOT/arn')AS T(X)
SQL Fiddle

Getting the value of dc:creator using SQL XML

I am unsure how to get the value of dc:creator from an RSS-feed using SQL.
This is my xml/rss-feed:
<rss xmlns:dc="http://purl.org/dc/elements/1.1/" version="2.0">
<channel>
<title>Foobar RSS</title>
<link>http://www.foobar.com/</link>
<description>RSS feed</description>
<language>en</language>
<ttl>15</ttl>
<item>
<title>This is my title</title>
<link>http://www.foobar.com/link/blabla</link>
<description>Bla..bla..bla..</description>
<dc:creator>John Doe</dc:creator>
<guid isPermaLink="false">00082EA751F1D905DE00E7CFA2417DA9</guid>
<pubDate>Wed, 26 Oct 2011 00:00:00 +0200</pubDate>
</item>
</channel>
</rss>
In my SQL I use something like this to get the values - e.g for pubDate I use something like this:
DECLARE #xml XML
SET #xml = cast('my rss feed here' AS xml)
SELECT
convert(datetime,substring(T.nref.value('pubDate[1]','nvarchar(100)'),6,20)) as pubdate,
FROM #xml.nodes('//item') AS T(nref)
This works fine, but when I am trying to get dc:creator value 'John Doe', the following just gives me an error:
SELECT
T.nref.value('dc:creator','nvarchar(100)') as creator
FROM #xml.nodes('//item') AS T(nref)
error:
XQuery [value()]: The name "dc" does not denote a namespace.
I need to be able to select multiple columns from the rss-feed. Can anybody provide a solution or direction to get the value of dc:creator?
I have another question - how would you construct the code if you are doing it in a sub select?
E.g.
INSERT INTO RSSResults (ID, pubDate)
SELECT #ID, tbl.pubDate FROM (
;WITH XMLNAMESPACES('http://purl.org/dc/elements/1.1/' AS dc)
SELECT
RSS.Item.value('(dc:creator)[1]', 'nvarchar(100)') as pubDate
FROM
#xml.nodes('/rss/channel/item') as RSS(Item)) AS tbl
The code breaks at ";WITH XMLNAMESPACES". Is it possible to include the namespace directly in the statement somehow?
Try something like this:
DECLARE #xml XML
SET #xml = cast('my rss feed here' AS xml)
;WITH XMLNAMESPACES('http://purl.org/dc/elements/1.1/' AS dc)
SELECT
#xml.value('(rss/channel/item/dc:creator)[1]', 'nvarchar(100)')
If you need to catch multiple items - try this:
DECLARE #xml XML
SET #xml = cast('my rss feed here' AS xml)
;WITH XMLNAMESPACES('http://purl.org/dc/elements/1.1/' AS dc)
SELECT
RSS.Item.value('(dc:creator)[1]', 'nvarchar(100)')
FROM
#xml.nodes('/rss/channel/item') as RSS(Item)

adding encoding information to the result of FOR XML [duplicate]

This question already has answers here:
SQL Server FOR XML Enclosing Element?
(2 answers)
Closed 7 years ago.
I have a Script, which returns a XML using FOR XML in SQL 2008. Is there any way to add the version and encoding information in the beginning of the output. Eventually, i am planning to save the output in a file.
For example, right now my output looks like this
<Agents>
<Agent id="1">
<Name>Mike</Name>
<Location>Sanfrancisco</Location>
</Agent>
<Agent id="2">
<Name>John</Name>
<Location>NY</Location>
</Agent>
</Agents>
I would like to append the line <?xml version="1.0" encoding="UTF-8"?> in the beginning of the Xml output
So i want the output something like
<?xml version="1.0" encoding="UTF-8"?>
<Agents>
<Agent id="1">
<Name>Mike</Name>
<Location>Sanfrancisco</Location>
</Agent>
<Agent id="2">
<Name>John</Name>
<Location>NY</Location>
</Agent>
As #gbn points out in another answer and on another question, "the XML data is stored internally as ucs-2", and SQL Server doesn't include it when producing the data. However, you can convert the XML to a string and append the XML declaration at the beginning manually. However, simply using UTF-8 in the declaration would be inaccurate. The Unicode string which SQL produces is in UCS-2. For example, this will fail:
SELECT CONVERT(xml,N'<?xml version="1.0" encoding="UTF-8"?>' + CONVERT(NVARCHAR(MAX),CONVERT(XML,N'<x>' + NCHAR(10176) + N'</x>')));
with error:
Msg 9402, Level 16, State 1, Line 1 XML parsing: line 1, character 38,
unable to switch the encoding
This, on the other hand, will work as expected:
SELECT CONVERT(xml,N'<?xml version="1.0" encoding="UCS-2"?>' + CONVERT(NVARCHAR(MAX),CONVERT(XML,N'<x>' + NCHAR(10176) + N'</x>')));
Here is code which will produce the full, declaration-laden XML string you seek for your example data:
DECLARE #Agents TABLE
(
AgentID int,
AgentName nvarchar(50),
AgentLocation nvarchar(100)
);
INSERT INTO #Agents (AgentID, AgentName, AgentLocation) VALUES (1, N'Mike', N'Sanfrancisco');
INSERT INTO #Agents (AgentID, AgentName, AgentLocation) VALUES (2, N'John', N'NY');
WITH BaseData AS
(
SELECT
(
SELECT
AgentID AS '#id',
AgentName AS 'Name',
AgentLocation AS 'Location'
FROM #Agents
FOR XML PATH('Agent'), ROOT('Agents'), TYPE
) AS AgentXML
), FullStringTable AS
(
SELECT
*,
'<?xml version="1.0" encoding="UCS-2"?>' +
CONVERT(nvarchar(max),AgentXML) AS FullString
FROM BaseData
)
SELECT
AgentXML AS OriginalXML,
FullString,
CONVERT(xml,FullString) AS FullStringConvertedToXML
FROM FullStringTable;
SQL Server internally always uses utf-16 ucs-2 so you could just append it like we did. That is, SQL Server would never generate anything with "utf-8".
Edit: after some digging:
http://www.devnewsgroups.net/group/microsoft.public.sqlserver.xml/topic60022.aspx
http://forums.asp.net/t/1455808.aspx
If you will not be attempting to manipulate the results as TSQL XML then the simplest thing to do will be create a varchar with the string you wish to append then add the XML to it using a CAST statemnt to convert the XML to varchar.
declare #testXML as XML
declare #testPrefix as varchar(255)
set #testPrefix = '<?xml version="1.0" encoding="UTF-8"?>'
set #testXML = '<Agents> <Agent id="1"> <Name>Mike</Name> <Location>Sanfrancisco</Location> </Agent> <Agent id="2"> <Name>John</Name> <Location>NY</Location> </Agent></Agents>'
select #testPrefix
Select #testXML
select #testPrefix + CAST(#testXML as varchar(max))