I need to re-write an old DTS/SQL2000 process that exports xml data stored as a varchar from a table to an xml file only to read it back and populate some tables such as contract/customer/business activity. I want to query the xml column directly instead but always get Blank or NULL depending on the method I use and I am having a really having a hard time. My latest quick version based on another answer is....
SELECT
XML_MESSAGE.value('(Transaction_Id/text())[1]','varchar(100)') AS TransactionID
FROM dbo.XML_REPOSITORY t
OUTER APPLY t.XML_MESSAGE.nodes('Contract_Interface') AS bm(XMLData)
--OUTER APPLY bm.XML_MESSAGE.nodes('food') AS f(XMLData)
However, I have created an Schema collection based on the old xsd files for each collection (contract/customer etc) so the real question is how can I use that to extract the relevant data? A cut down version of the xml is below..
<Contract_Interface xmlns="http://company.com/backoffice/types/common" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://comapny.com/backoffice/types/common http://dataplace/xmlschemas/Policy_Transaction_Interface_1_1_9.xsd">
<Transaction_Id>48244272268</Transaction_Id>
<Source_System_Cd>SystemName</Source_System_Cd>
<Business_Activity>
<Contract_Id>169929</Contract_Id>
<Business_Activity_Type_Cd>01</Business_Activity_Type_Cd>
<Business_Activity_Type_Desc>New Binder</Business_Activity_Type_Desc>
<Out_Of_Sequence_Ind>N</Out_Of_Sequence_Ind>
<Effective_Dt>4/18/2021 12:00:00 AM</Effective_Dt>
<Expiration_Dt>4/18/2022 12:00:00 AM</Expiration_Dt>
<Premium_Amt>123678.00</Premium_Amt>
<Contract>
<Customer>
<Customer_Id>45678</Customer_Id>
<ODS_Customer_Id>6789</ODS_Customer_Id>
<Insured_Info_Overriden_Cd>N</Insured_Info_Overriden_Cd>
<Insured_Info_Overriden_Desc>No</Insured_Info_Overriden_Desc>
<Insured_Nm>Make Money PLC</Insured_Nm>
<Insured_Address>
<Address_Line_1>Alex Jones Terrace</Address_Line_1>
<Address_Line_2 />
<City_Nm>Houston</City_Nm>
<State_Cd>TX</State_Cd>
<State_Desc>Texas</State_Desc>
<Postal_Cd>77002</Postal_Cd>
<Country_Cd>US</Country_Cd>
<Country_Desc>United States</Country_Desc>
</Insured_Address>
</Customer>
<Producer>
<Producer_Id>33333</Producer_Id>
<ODS_Producer_Id>1234</ODS_Producer_Id>
<Producer_No>1234</Producer_No>
<Producer_Nm>Brokerage Limited</Producer_Nm>
<Billing_Address_Overridden_Ind>false</Billing_Address_Overridden_Ind>
<Producer_Billing_Address>
<Address_Line_1>Suite 1000</Address_Line_1>
<Address_Line_2>Free Road</Address_Line_2>
<City_Nm>Hamilton</City_Nm>
<State_Cd />
<State_Desc />
<Postal_Cd>HM11</Postal_Cd>
<Country_Cd>BM</Country_Cd>
<Country_Desc>Bermuda</Country_Desc>
</Producer_Billing_Address>
</Producer>
</Contract>
</Business_Activity>
</Contract_Interface>
As I mentioned, you need to define your Namespace. This uses a variable, instead of your table, but gives you the overlying idea:
DECLARE #XML xml = '<Contract_Interface xmlns="http://company.com/backoffice/types/common" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://comapny.com/backoffice/types/common http://dataplace/xmlschemas/Policy_Transaction_Interface_1_1_9.xsd">
<Transaction_Id>48244272268</Transaction_Id>
<Source_System_Cd>SystemName</Source_System_Cd>
<Business_Activity>
<Contract_Id>169929</Contract_Id>
<Business_Activity_Type_Cd>01</Business_Activity_Type_Cd>
<Business_Activity_Type_Desc>New Binder</Business_Activity_Type_Desc>
<Out_Of_Sequence_Ind>N</Out_Of_Sequence_Ind>
<Effective_Dt>4/18/2021 12:00:00 AM</Effective_Dt>
<Expiration_Dt>4/18/2022 12:00:00 AM</Expiration_Dt>
<Premium_Amt>123678.00</Premium_Amt>
<Contract>
<Customer>
<Customer_Id>45678</Customer_Id>
<ODS_Customer_Id>6789</ODS_Customer_Id>
<Insured_Info_Overriden_Cd>N</Insured_Info_Overriden_Cd>
<Insured_Info_Overriden_Desc>No</Insured_Info_Overriden_Desc>
<Insured_Nm>Make Money PLC</Insured_Nm>
<Insured_Address>
<Address_Line_1>Alex Jones Terrace</Address_Line_1>
<Address_Line_2 />
<City_Nm>Houston</City_Nm>
<State_Cd>TX</State_Cd>
<State_Desc>Texas</State_Desc>
<Postal_Cd>77002</Postal_Cd>
<Country_Cd>US</Country_Cd>
<Country_Desc>United States</Country_Desc>
</Insured_Address>
</Customer>
<Producer>
<Producer_Id>33333</Producer_Id>
<ODS_Producer_Id>1234</ODS_Producer_Id>
<Producer_No>1234</Producer_No>
<Producer_Nm>Brokerage Limited</Producer_Nm>
<Billing_Address_Overridden_Ind>false</Billing_Address_Overridden_Ind>
<Producer_Billing_Address>
<Address_Line_1>Suite 1000</Address_Line_1>
<Address_Line_2>Free Road</Address_Line_2>
<City_Nm>Hamilton</City_Nm>
<State_Cd />
<State_Desc />
<Postal_Cd>HM11</Postal_Cd>
<Country_Cd>BM</Country_Cd>
<Country_Desc>Bermuda</Country_Desc>
</Producer_Billing_Address>
</Producer>
</Contract>
</Business_Activity>
</Contract_Interface>';
WITH XMLNAMESPACES(DEFAULT 'http://company.com/backoffice/types/common')
SELECT bm.XMLData.value('(Transaction_Id/text())[1]','varchar(100)') AS TransactionID --Should this not be a int or bigint?
FROM #XML.nodes('Contract_Interface') AS bm(XMLData);
Related
How to fetch a value from a column which contains XML in SQL Server?
below is my sample XML column value and the id's can be swap anytime(101,100) or (201,100,101,321).
<Questions>
<item id="101">Yes</item>
<item id="100">No</item>
</Questions>
I want to fetch a value based on Id. Like fetching Yes from id=101.
Sample code much appreciated.
I tried with below sample, but Unable to retrieve value "Yes"
select Y.value('#item[1]','varchar[3]') as valT from tbl_storeXML s cross apply s.Questions.nodes('Questions/item') as X(Y) where e.empId=256 and Y.value('#id','int')=101
Please help on this.
Ps. It's not a home work, I am learning handling xml in sql server.
Use of the value is not done correct, you do:
Y.value('#id','int')
This should be: Y.value('(#id)[1]','int')
round braces around #id, see: docs: value() Method
and Y.value('item[1]','varchar[3]').
This should be: Y.value('(#item)[1]','varchar(3)').
The # is removed because item is not an attribute
varchar should have round braces, not square braces.
Your try, after changes will become:
select
Y.value('(item)[1]','varchar(3)') as valT
from tbl_storeXML s
cross apply s.Questions.nodes('Questions/item') as X(Y)
where e.empId=256 and Y.value('(#id)','int')=101
This is not tested, because I do not have those tables. (I do think Y.value('(item)[1]','varchar(3)') might need to be written as Y.value('(.)[1]','varchar(3)') )
But the same approach can be seen in this DBFIDDLE
DECLARE #xml XML = '<Questions>
<item id="101">Yes</item>
<item id="100">No</item>
</Questions>';
select
X.y.value('(#id)[1]','VARCHAR(20)') id,
X.y.value('(.)[1]','VARCHAR(20)') value
from #xml.nodes('Questions/item') as X(y);
output:
id
value
101
Yes
100
No
I have an XML column in a table which i am trying to parse out values from to flat table structure.
I am trying to input the XML here but stackoverflow ses it as code and when i try and format as code it still won't accept it.
I can't even get data from "Header" level.
<RequestMessage xmlns="http://iec.ch/TC57/2011/schema/message" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:noNamespaceSchemaLocation="Message.xsd">
<Header>
<Verb>created</Verb>
<Noun>MeterReadings</Noun>
<Timestamp>2021-03-08T00:57:18+01:00</Timestamp>
<Source>Ipsum Lorum</Source>
<AsyncReplyFlag>true</AsyncReplyFlag>
<AckRequired>true</AckRequired>
<MessageID>Ipsum Lorum</MessageID>
<CorrelationID />
</Header>
<Payload>
<MeterReadings xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#" xmlns="http://iec.ch/TC57/2011/MeterReadings#">
<MeterReading>
<IntervalBlocks>
<IntervalReadings>
<timeStamp>2021-03-07T01:00:00+01:00</timeStamp>
<value>480.196</value>
<ReadingQualities>
<ReadingQualityType ref="3.0.0" />
</ReadingQualities>
</IntervalReadings>
<IntervalReadings>
<ReadingType ref="11.0.7.3.1.2.12.1.1.0.0.0.0.101.0.3.72.0" />
</IntervalReadings>
</IntervalBlocks>
<Meter>
<mRID>0000000000000</mRID>
<status>
<remark>Ipsum Lorum</remark>
<value>ESP</value>
</status>
</Meter>
<UsagePoint>
<mRID>73599900000000</mRID>
</UsagePoint>
</MeterReading>
</MeterReadings>
</Payload>
</RequestMessage>
I am not able to parse it and i have tried using examples from other threads. I am trying to not use OPENXML solution because requires DECLARE and executing the built in procedure for clearing the XML from memmory periodically. I am trying to use the OUTER APPLY solution.
Like Shugos solution in How to parse XML data in SQL server table or Query XML with nested nodes on Cross Apply.
It doesn't work.
It returns null for the timestamp column.
select
t.file_created_time
,c.value('(Timestamp)[1]','varchar(max)') as timestamp
from load.t t
OUTER APPLY t.xml_data.nodes('RequestMessage/Header') as m(c)
Please try the following solution.
Starting from SQL Server 2005 onwards, it is better to use XQuery language, based on the w3c standards, while dealing with the XML data type.
Microsoft proprietary OPENXML and its companions sp_xml_preparedocument and sp_xml_removedocument are kept just for backward compatibility with the obsolete SQL Server 2000. Their use is diminished just to very few fringe cases.
I had to comment out the following tag <!--<IntervalReadings>--> to make your XML well-formed.
XML Header fragment has a default namespace:
xmlns="http://iec.ch/TC57/2011/schema/message"
XML Payload fragment has its own two additional namespaces:
xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#"
xmlns="http://iec.ch/TC57/2011/MeterReadings#"
Namespaces should be taken into account.
Check it out below.
SQL
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, xml_data XML);
INSERT INTO #tbl (xml_data) VALUES
(N'<RequestMessage xmlns="http://iec.ch/TC57/2011/schema/message"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:noNamespaceSchemaLocation="Message.xsd">
<Header>
<Verb>created</Verb>
<Noun>MeterReadings</Noun>
<Timestamp>2021-03-08T00:57:18+01:00</Timestamp>
<Source>Ipsum Lorum</Source>
<AsyncReplyFlag>true</AsyncReplyFlag>
<AckRequired>true</AckRequired>
<MessageID>Ipsum Lorum</MessageID>
<CorrelationID/>
</Header>
<Payload>
<MeterReadings xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#"
xmlns="http://iec.ch/TC57/2011/MeterReadings#">
<MeterReading>
<IntervalBlocks>
<IntervalReadings>
<timeStamp>2021-03-07T01:00:00+01:00</timeStamp>
<value>480.196</value>
<ReadingQualities>
<ReadingQualityType ref="3.0.0"/>
</ReadingQualities>
</IntervalReadings>
<!--<IntervalReadings>-->
<ReadingType ref="11.0.7.3.1.2.12.1.1.0.0.0.0.101.0.3.72.0"/>
</IntervalBlocks>
<Meter>
<mRID>0000000000000</mRID>
<status>
<remark>Ipsum Lorum</remark>
<value>ESP</value>
</status>
</Meter>
<UsagePoint>
<mRID>73599900000000</mRID>
</UsagePoint>
</MeterReading>
</MeterReadings>
</Payload>
</RequestMessage>');
-- DDL and sample data population, end
WITH XMLNAMESPACES(DEFAULT 'http://iec.ch/TC57/2011/schema/message')
SELECT id
, c.value('(Noun/text())[1]','VARCHAR(30)') AS Noun
, c.value('(Timestamp/text())[1]','DATETIMEOFFSET(0)') AS [timestamp]
FROM #tbl
CROSS APPLY xml_data.nodes('/RequestMessage/Header') AS t(c);
Output
+----+---------------+----------------------------+
| id | Noun | timestamp |
+----+---------------+----------------------------+
| 1 | MeterReadings | 2021-03-08 00:57:18 +01:00 |
+----+---------------+----------------------------+
You need to respect and include the XML namespace in your XML document in your XQuery!
<RequestMessage xmlns="http://iec.ch/TC57/2011/schema/message"
**********************************************
Try something like this:
WITH XMLNAMESPACES(DEFAULT N'http://iec.ch/TC57/2011/schema/message')
SELECT
t.id,
c.value('(Timestamp)[1]','varchar(max)') as timestamp
FROM
load.t t
CROSS APPLY
t.xml_data.nodes('RequestMessage/Header') AS m(c)
Also when trying to run this on my SQL Server, I get an error that the XML as shown is malformed.....
UPDATE:
If you need to also access bits in the Payload section - you need to also respect that XML namespace there:
<MeterReadings xmlns:MeterReadings="http://iec.ch/TC57/2011/MeterReadings#"
xmlns="http://iec.ch/TC57/2011/MeterReadings#">
***********************************************
Try this:
WITH XMLNAMESPACES(N'http://iec.ch/TC57/2011/schema/message' as hdr,
N'http://iec.ch/TC57/2011/MeterReadings#' as mr)
SELECT
t.id,
c.value('(hdr:Timestamp)[1]', 'varchar(50)') AS timestamp,
col.value('(mr:MeterReading/mr:IntervalBlocks/mr:IntervalReadings/mr:timeStamp)[1]', 'varchar(50)') AS MeterReadingsTimestamp
FROM
load.t t
CROSS APPLY
t.xml_data.nodes('/hdr:RequestMessage/hdr:Header') AS m(c)
CROSS APPLY
t.xml_data.nodes('/hdr:RequestMessage/hdr:Payload/mr:MeterReadings') AS mr(col)
I have a SQL Server database table with a column called XML that contains XML data which is structured like this:
<Item xmlns:i="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://test/data">
<Roots>
<Root>
<Name>Field Name</Name>
<Value>Field Value</Value>
</Root>
<Root>
<Name>Field Name</Name>
<Value>Field Value</Value>
</Root>
</Roots>
I want to use T-SQL to get the Value where Name = Total. I have tried the following but it isn't returning any data:
SELECT [XML]
FROM [BusinessAccount]
WHERE [XML].value('(/Root/Name)[13]', 'VARCHAR(MAX)') LIKE '%Total%'
If anyone could tell me where I've gone wrong?
You are missing the required WITH XMLNAMESPACES for your XML and the path is incorrect.
If you want to bring back rows where the 13th element consists of the text Total you can use the below.
WITH XMLNAMESPACES (DEFAULT 'http://test/data')
SELECT [XML]
FROM [BusinessAccount]
WHERE 1 = [XML].exist('(/Item/Roots/Root/Name)[13][text() = "Total"]')
Otherwise you can add the WITH XMLNAMESPACES to your original query and fix the path there too.
You need to specify namespaces. You can then match <Name> and <Value> pairs and extract the contents of <Value> like so:
SELECT NameNode.value('declare namespace x="http://test/data"; (../x:Value)[1]', 'varchar(100)')
FROM [BusinessAccount]
CROSS APPLY [XML].nodes('declare namespace x="http://test/data"; //x:Root/x:Name') AS n(NameNode)
WHERE NameNode.value('.', 'varchar(100)') = 'Total'
Demo on db<>fiddle
I am attempting to extract information from XML stored in a CLOB column. I've searched the forums and thus far have been unable to get the data to pull as needed. I have a basic understanding of SQL but this is beyond me.
The XML is similar to the following:
<?xml version="1.0" encoding="UTF-8"?>
<Response>
<Header>
<OrderNum value="12354321"/>
<ExtractDate value="11-30-2012"/>
<RType value="Status"/>
<Company value="Company"/>
</Header>
<Body>
<Status>
<Order>
<ActivityType value="ValidateRequest"/>
<EndUser>
<Name value="Schmo, Joe"/>
<Address>
<SANO value="12345"/>
<SASN value="Mickey Mouse"/>
<SATH value="Lane"/>
<SASS value="N"/>
<City value="Orlando"/>
<State value="FL"/>
<Zip value="34786"/>
<Number value="5550000"/>
</Address>
</EndUser>
<COS value="1"/>
<TOS value="3"/>
<MainNumber value="5550000"/>
</Order>
<ErrorCode value="400"/>
<ErrorMessage value="RECEIVED"/>
</Status>
</Body>
</Response>
I want to get the values under "Address".
I've tried the following but it returns "NULL".
SELECT EXTRACTVALUE(XMLTYPE(RESPONSE_CLOB),'/Response/Body/Status/Order/EndUser/Address/SANO') AS SANO
FROM RESPONSE_TABLE
WHERE ROWNUM < 2
I am trying to get it so I can pull the "12345" assigned as "value" in "SANO" (ultimately getting the value for other fields, but want to at least get the one working first).
You're currently retrieving the text value of the node, but 12345 is the value attribute of the element rather than its text content. So you would need to use the #attribute syntax, i.e.:
SELECT EXTRACTVALUE(XMLTYPE(RESPONSE_CLOB),'/Response/Body/Status/Order/EndUser/Address/SANO/#value') AS SANO
FROM RESPONSE_TABLE
WHERE ROWNUM < 2;
SANO
--------------------
12345
But extractvalue is deprecated; assuming you're on a recent version of Oracle it would be better to use an XMLQuery:
SELECT XMLQUERY(
'/Response/Body/Status/Order/EndUser/Address/SANO/#value'
PASSING XMLTYPE(RESPONSE_CLOB)
RETURNING CONTENT
) AS SANO
FROM RESPONSE_TABLE
WHERE ROWNUM < 2;
You may find it even easier to use an XMLTable - necessary if an XML document has multiple Address nodes, but even with just one pulling the values out as columns is less repetitive, and it makes it easier to retrieve suitable data types:
select x.*
from response_table rt
cross join xmltable(
'/Response/Body/Status/Order/EndUser/Address'
passing xmltype(rt.response_clob)
columns sano number path 'SANO/#value',
sasn varchar2(30) path 'SASN/#value',
sath varchar2(10) path 'SATH/#value'
-- etc.
) x
where rownum < 2;
SANO SASN SATH
-------------------- ------------------------------ ----------
12345 Mickey Mouse Lane
Read more about using these functions to query XML data.
I'm very very new to XQUERY/XPATH :) so I could very well be going about this the wrong way. I have a customer object serialized and stored in a database column in the following format.
<Customer xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
<Addresses>
<AddressBlock>
<AddressLine1>1234 SomeStreet Ave.</AddressLine1>
<City>SomeCity</City>
<State>SomeState</State>
<Zipcode>SomeZip</Zipcode>
</AddressBlock>
<AddressBlock>
<AddressLine1>5678 SomeOtherStreet Ave.</AddressLine1>
<City>SomeOtherCity</City>
<State>SomeOtherState</State>
<Zipcode>SomeOtherZip</Zipcode>
</AddressBlock>
</Addresses>
</Customer>
I'm looking for a way to select this record if addressline1 and city in the same addressblock contains certain keywords. I have the following statement that almost does what I'm looking for.
select *
from users
where [UserData].exist('/Customer/Addresses/AddressBlock/AddressLine1/text()[contains(upper-case(.),""SOMESTREET"")]')=1
and [UserData].exist('/Customer/Addresses/AddressBlock/City/text()[contains(upper-case(.),""SOMECITY"")]')=1"
My only problem is this statment will also return the record if the first addressblock contains the addressline1 and the second addressblock contains the city.
You have to test both conditions in the same XQuery.
select *
from users
where [UserData].exist('/Customer/Addresses/AddressBlock
[contains(upper-case(AddressLine1[1]),"SOMESTREET") and
contains(upper-case(City[1]),"SOMECITY")]')=1