XML Data into Row Selection - sql

I have a single field in a DB which contains columns of data, but in an continuous string of XML.
I want to extract the data from the XML, and return as normal columns.
There could be multiple rows of data within this single field.
Here are 2 real examples of data in this field
Example 1:
<DocumentElement>
<AuditTrailDetail>
<TableName>order_header_total</TableName>
<RowID>837</RowID>
<AuditType>Edit</AuditType>
<FieldName>oht_foreign_net</FieldName>
<ValueFrom>65.1600</ValueFrom>
<ValueTo>115.1600</ValueTo>
<EntityName />
<DisplayName />
<Identifier />
</AuditTrailDetail>
<AuditTrailDetail>
<TableName>order_line_item</TableName>
<RowID>2442</RowID>
<AuditType>Edit</AuditType>
<FieldName>oli_qty_required</FieldName>
<ValueFrom>1.0000</ValueFrom>
<ValueTo>2.0000</ValueTo>
<EntityName />
<DisplayName />
<Identifier>61 - test</Identifier>
</AuditTrailDetail>
</DocumentElement>
Example 2:
<DocumentElement>
<AuditTrailDetail>
<TableName>order_line_item</TableName>
<RowID>2446</RowID>
<AuditType>Edit</AuditType>
<FieldName>oli_description</FieldName>
<ValueFrom>2 Ply Tissue Masks</ValueFrom>
<ValueTo>2 Ply Tissue Masksdd</ValueTo>
<EntityName />
<DisplayName />
<Identifier>D/D170 - 2 Ply Tissue Masksdd</Identifier>
</AuditTrailDetail>
</DocumentElement>
For Example 1, I'd like to return 2 rows as below:
For Example 2 there is one row, and I'd like it as below:
The fields in the table are:
owat_id
owat_audit_type
owat_record_id
owat_record_type
owat_datetime
owat_ud_id
owat_details
The XML data is within owat_details.
Assume the table name is Audit_Trail.
How can I do this directly in the SQL query?

Thank you to Johnie Karr
I have got this working using XQuery.
I have to do 2 queries and union them together to get the 2 separate elements in one field as separate rows. I'm sure there is a cleaner solution, but this works.
Thank you very much :)
SELECT
AuditTrail.owat_record_type,
AuditTrail.owat_audit_type,
AuditTrail.owat_datetime as 'Audit Date',
user_detail.ud_username as 'User',
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/TableName)[1]','varchar(50)') AS TableName,
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/RowID)[1]','varchar(50)') AS RowID,
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/FieldName)[1]','varchar(50)') AS FieldName,
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/ValueFrom)[1]','varchar(50)') AS ValueFrom,
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/ValueTo)[1]','varchar(50)') AS ValueTo,
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/Identifier)[1]','varchar(50)') AS Identifier
FROM
(select
orderwise_audit_trail.owat_id,
orderwise_audit_trail.owat_audit_type,
orderwise_audit_trail.owat_record_id,
orderwise_audit_trail.owat_record_type,
orderwise_audit_trail.owat_datetime,
orderwise_audit_trail.owat_ud_id,
convert(xml, orderwise_audit_trail.owat_details) as owat_details
from
orderwise_audit_trail) as AuditTrail
left join user_detail on user_detail.ud_id = AuditTrail.owat_ud_id
Where AuditTrail.owat_details is not NULL
and (AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/TableName)[1]','varchar(50)')) is not null
UNION ALL
SELECT
AuditTrail.owat_record_type,
AuditTrail.owat_audit_type,
AuditTrail.owat_datetime as 'Audit Date',
user_detail.ud_username as 'User',
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/TableName)[2]','varchar(50)') AS TableName,
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/RowID)[2]','varchar(50)') AS RowID,
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/FieldName)[2]','varchar(50)') AS FieldName,
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/ValueFrom)[2]','varchar(50)') AS ValueFrom,
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/ValueTo)[2]','varchar(50)') AS ValueTo,
AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/Identifier)[2]','varchar(50)') AS Identifier
FROM
(select
orderwise_audit_trail.owat_id,
orderwise_audit_trail.owat_audit_type,
orderwise_audit_trail.owat_record_id,
orderwise_audit_trail.owat_record_type,
orderwise_audit_trail.owat_datetime,
orderwise_audit_trail.owat_ud_id,
convert(xml, orderwise_audit_trail.owat_details) as owat_details
from
orderwise_audit_trail) as AuditTrail
left join user_detail on user_detail.ud_id = AuditTrail.owat_ud_id
Where AuditTrail.owat_details is not NULL
and (AuditTrail.owat_details.value('(/DocumentElement/AuditTrailDetail/TableName)[2]','varchar(50)')) is not null
order by 1, 2, 3, 4

Related

How to get a specific XML format from SQL server 2014 (example provided)

Ive been banging my head against a wall with this for ages - and i've finally caved.
Can any SQL / XML experts please take a look at the below and tell me the best way to get the required XML format using FOR XML (raw? / Auto? / Path?) please?
Thanks for your help
Test data
CREATE TABLE #CustomerData(
[Customer ID] int
, IsCustomerID bit DEFAULT 1
, Amount1 float DEFAULT 0
, Amount2 float DEFAULT 0
)
INSERT #CustomerData
SELECT *
FROM (
VALUES (12345, 1, 50, 75),
(12444, 1, 100, 100),
(12455, 1, 25, 65)
) zz ([Customer ID], IsCustomerID, Amount1, Amount2)
Ive tried various combinations of FOR XML RAW / FOR XML PATH and FOR XML AUTO, but nothing quite matches.
<Root>
<Customers>
<Customer Prefix="Loan">
<Property Name="Customer ID" Value="REF_1234" IsCustomerId="True" />
<Property Name="Amount1" Value="10" />
<Property Name="Amount2" Value="15" />
</Customer>
</Customers>
</Root>
"Loan" is a hardcoded text flag to be applied to all entries from the above table, although there is no specific column for it.
"Customer ID"
"IsCustomerID"
"Amount1"
"Amount2"
Are all fields in a single table
A few things:
You do not specify where the value of Prefix comes from in your result, but for the query structure this doesn't matter.
FOR XML PATH allows you to conjure nested elements and attributes with specific syntax ('#' for properties) and specifying TYPE to insert the subqueries as typed XML.
T-SQL cannot dynamically pivot columns to rows (only statically), so you can't (easily) write a query to produce every individual column in its own child element except by writing out the columns explicitly. This is only a problem if you really do need dynamic output (i.e. the query must continue to include all columns without rewrites even if people add columns later).
Because of the downright peculiar syntax T-SQL uses to insert FLOAT and REAL values into XML (always using scientific notation) it's usually a good idea to either not use FLOAT/REAL at all but specific integral types (with CONVERT), or use FORMAT to produce a friendlier value. Likewise, a BIT value will always be output as 1 or 0, so if you need other values you'll need to provide for that yourself.
So:
SELECT (
SELECT
"#Prefix" = 'Loan',
(SELECT
"#Name" = 'Customer ID',
"#Value" = [Customer ID],
"#IsCustomerID" = CASE IsCustomerId WHEN 1 THEN 'True' ELSE 'False' END
FOR XML PATH('Property'), TYPE
),
(SELECT "#Name" = 'Amount1', "#Value" = FORMAT(Amount1, 'G') FOR XML PATH('Property'), TYPE),
(SELECT "#Name" = 'Amount2', "#Value" = FORMAT(Amount2, 'G') FOR XML PATH('Property'), TYPE)
FROM #CustomerData
FOR XML PATH ('Customer'), ROOT('Customers'), TYPE
)
FOR XML PATH('Root')
It is possible to factor out the repeated mention of Property by using a subquery that uses UNION ALL:
SELECT (
SELECT
"#Prefix" = 'Loan',
(
SELECT * FROM (
SELECT
"#Name" = 'Customer ID',
"#Value" = [Customer ID],
"#IsCustomerID" = CASE IsCustomerId WHEN 1 THEN 'True' ELSE 'False' END
UNION ALL
SELECT "#Name" = 'Amount1', "#Value" = FORMAT(Amount1, 'G'), NULL
UNION ALL
SELECT "#Name" = 'Amount2', "#Value" = FORMAT(Amount2, 'G'), NULL
) _
FOR XML PATH('Property'), TYPE
)
FROM #CustomerData
FOR XML PATH ('Customer'), ROOT('Customers'), TYPE
)
FOR XML PATH('Root')
I can see either of these forms being more or less readable/maintainable depending on what's likely to change.

SELECT only specific value from XML column in SQL

Running the following TSQL
SELECT Name FROM Category
returns these results as the column in SQL is of xml data type.
1 <locale en-US="Abstract" />
2 <locale en-US="African" />
3 <locale en-US="Americana" />
I'd like to get a result set like:
1 Abstract
2 African
3 Americana
How can I do this?
You can try to use the .value function
SELECT Name.value('(/locale/#en-US)[1]','varchar(20)')
from Category
sqffiddle

Get SQL Server column names with values

I am trying to create a query that allows me to have the column name next to its column values that it finds.
As an example:
SELECT
*
FROM
INFORMATION_SCHEMA.COLUMNS
WHERE
TABLE_NAME = 'saverTbl'
The above returns all of the column names in that table like this:
id, Link_userTblID, type, environment, vendor, theGUID, theGUID, etc etc...
Now what I need is to get the output from the table name next to the value. This is my query to just get my values from the table:
SELECT
*
FROM
saverTbl
WHERE
LINK_userTblID = #val1
The above returns the row that matches the LINK_userTblID like so:
32, 1, 'Blah', 'something', 'Adobe', 546656156-45332-54616516-4515, etc etc..
Now putting that all together (which is what this question is all about):
id: 32,
LINK_userTblID: 1,
type: Blah,
environment: something,
vendor: Adobe,
theGUID: 546656156-45332-54616516-4515,
etc etc.....
Pretty much needing the output in a json format but the column name matching up with the columns value.
Try this:
SELECT * FROM saverTbl
WHERE LINK_userTblID = #val1
FOR JSON PATH
Assuming, this is meant for one single row you can try this:
WITH cte AS
(
SELECT
(SELECT TOP 1 * FROM sys.objects FOR XML PATH('row'),TYPE) AS TheXml
)
SELECT TheElement.value('local-name(.)','nvarchar(max)')
+ ': '
+ TheElement.value('text()[1]','nvarchar(max)') AS YourOutput
FROM cte
CROSS APPLY TheXml.nodes('/row/*') AS A(TheElement);
The result:
YourOutput
---------------
name: sysrscols
object_id: 3
schema_id: 4
parent_object_id: 0
type: S
type_desc: SYSTEM_TABLE
create_date: 2012-02-10T20:15:58.693
modify_date: 2012-02-10T20:15:58.700
is_ms_shipped: 1
is_published: 0
is_schema_published: 0
XML in connection with XQuery and XPath is a very mighty toolset to solve rather generic problems. The cte builds an XML which looks like this:
<row>
<name>sysrscols</name>
<object_id>3</object_id>
<schema_id>4</schema_id>
<parent_object_id>0</parent_object_id>
<type>S </type>
<type_desc>SYSTEM_TABLE</type_desc>
<create_date>2012-02-10T20:15:58.693</create_date>
<modify_date>2012-02-10T20:15:58.700</modify_date>
<is_ms_shipped>1</is_ms_shipped>
<is_published>0</is_published>
<is_schema_published>0</is_schema_published>
</row>
The call to /row/* retrieves all nodes below <row> as derived table. The rest is rather easy XQuery.

how to get all the fields in sql server 2008

I want to get all the fields from the table which i select in the query even if the fields contains no value(the value null or empty). the following is the query i have written.
SELECT (
SELECT T1.[CarrierCode_Destination] AS '#CarrierCode_Destination',
T1.[CarrierCode_Destination_Address] AS
'#CarrierCode_Destination_Address',
T1.[CarrierCode_Destination_Address1] AS
'#CarrierCode_Destination_Address1',
T1.[CarrierCode_Destination_Address2] AS
'#CarrierCode_Destination_Address2',
T1.[CIMtrek_RegContact] AS '#CIMtrek_RegContact',
T1.[CIMtrek_CarrierContact] AS '#CIMtrek_CarrierContact',
[T1].[CIMtrek_AdditionalContacts] AS
'#CIMtrek_AdditionalContacts'
FROM (
SELECT CD.[CIMtrek_DestinationName]
CarrierCode_Destination,
CD.[CIMtrek_DestinationAdd]
CarrierCode_Destination_Address,
CD.[CIMtrek_DestinationAdd_1]
CarrierCode_Destination_Address1,
CD.[CIMtrek_DestinationAdd_2]
CarrierCode_Destination_Address2,
CD.[CIMtrek_RegContact] CIMtrek_RegContact,
CD.[CIMtrek_CarrierContact] CIMtrek_CarrierContact,
CD.[CIMtrek_AdditionalContacts]
CIMtrek_AdditionalContacts
FROM CIMtrek_SystemTable_DatawareHouse CD
WHERE LEN(
LTRIM(
RTRIM(ISNULL(LTRIM(RTRIM(CD.[CIMtrek_DestinationName])), ''))
)
) != 0
) AS T1
FOR XML PATH('Record'),
TYPE
) FOR XML PATH('CarrierCode_Destination'),
TYPE
the results i get is
<CarrierCode_Destination>
<Record CarrierCode_Destination="8S - San Fran" CarrierCode_Destination_Address="SAN FRANCISCO - Redistribution" CarrierCode_Destination_Address1="4025 Whipple Road, " CarrierCode_Destination_Address2="Union City CA 94587" CIMtrek_RegContact="RDC San Francisco" />
<Record CarrierCode_Destination="8G - St Louis" CarrierCode_Destination_Address="ST. LOUIS - Redistribution" CarrierCode_Destination_Address1="126 Enterprise Drive, " CarrierCode_Destination_Address2="Wentzville MO 63385" CIMtrek_RegContact="RDC St Louis" />
<Record CarrierCode_Destination="V8 PHO/CYPR/CUST/TL  " />
</CarrierCode_Destination>
but i want all the fields which are selected in the query. If you see the result it gives me the fields which are having value and omits the fields which don't have values.
how to do this, Please help.
Best Regards
If you use isnull() statements for each column you are selecting and convert null values to empty strings it should force the attributes to be generated for all columns. You may or may not also need to add rtrim() statements like I show below depending on if you have a datatype that is going to return a fixed size string:
SELECT (
SELECT rtrim(isnull(T1.[CarrierCode_Destination],'')) AS '#CarrierCode_Destination',
rtrim(isnull(T1.[CarrierCode_Destination_Address],'')) AS
'#CarrierCode_Destination_Address',
rtrim(IsNull(T1.[CarrierCode_Destination_Address1],'')) AS
'#CarrierCode_Destination_Address1',
rtrim(IsNull(T1.[CarrierCode_Destination_Address2],'')) AS
'#CarrierCode_Destination_Address2',
rtrim(IsNull(T1.[CIMtrek_RegContact],'')) AS '#CIMtrek_RegContact',
rtrim(IsNull(T1.[CIMtrek_CarrierContact],'')) AS '#CIMtrek_CarrierContact',
rtrim(IsNull([T1].[CIMtrek_AdditionalContacts],'')) AS
'#CIMtrek_AdditionalContacts'
FROM (
SELECT CD.[CIMtrek_DestinationName]
CarrierCode_Destination,
CD.[CIMtrek_DestinationAdd]
CarrierCode_Destination_Address,
CD.[CIMtrek_DestinationAdd_1]
CarrierCode_Destination_Address1,
CD.[CIMtrek_DestinationAdd_2]
CarrierCode_Destination_Address2,
CD.[CIMtrek_RegContact] CIMtrek_RegContact,
CD.[CIMtrek_CarrierContact] CIMtrek_CarrierContact,
CD.[CIMtrek_AdditionalContacts]
CIMtrek_AdditionalContacts
FROM CIMtrek_SystemTable_DatawareHouse CD
WHERE LEN(
LTRIM(
RTRIM(ISNULL(LTRIM(RTRIM(CD.[CIMtrek_DestinationName])), ''))
)
) != 0
) AS T1
FOR XML PATH('Record'),
TYPE
) FOR XML PATH('CarrierCode_Destination'),
TYPE
That worked for me when I tested it out but if you are also having issues with empty values not generating attributes you can first force all the empty values to be returned as NULL from the select against your original data table using NULLIF()
SELECT (
SELECT rtrim(isnull(T1.[CarrierCode_Destination],'')) AS '#CarrierCode_Destination',
rtrim(isnull(T1.[CarrierCode_Destination_Address],'')) AS
'#CarrierCode_Destination_Address',
rtrim(IsNull(T1.[CarrierCode_Destination_Address1],'')) AS
'#CarrierCode_Destination_Address1',
rtrim(IsNull(T1.[CarrierCode_Destination_Address2],'')) AS
'#CarrierCode_Destination_Address2',
rtrim(IsNull(T1.[CIMtrek_RegContact],'')) AS '#CIMtrek_RegContact',
rtrim(IsNull(T1.[CIMtrek_CarrierContact],'')) AS '#CIMtrek_CarrierContact',
rtrim(IsNull([T1].[CIMtrek_AdditionalContacts],'')) AS
'#CIMtrek_AdditionalContacts'
FROM (
SELECT NullIF(CD.[CIMtrek_DestinationName],'')
CarrierCode_Destination,
NullIF(CD.[CIMtrek_DestinationAdd],'')
CarrierCode_Destination_Address,
NullIF(CD.[CIMtrek_DestinationAdd_1],'')
CarrierCode_Destination_Address1,
NullIF(CD.[CIMtrek_DestinationAdd_2],'')
CarrierCode_Destination_Address2,
NullIF(CD.[CIMtrek_RegContact],'') CIMtrek_RegContact,
NullIF(CD.[CIMtrek_CarrierContact],'') CIMtrek_CarrierContact,
NullIF(CD.[CIMtrek_AdditionalContacts],'')
CIMtrek_AdditionalContacts
FROM CIMtrek_SystemTable_DatawareHouse CD
WHERE LEN(
LTRIM(
RTRIM(ISNULL(LTRIM(RTRIM(CD.[CIMtrek_DestinationName])), ''))
)
) != 0
) AS T1
FOR XML PATH('Record'),
TYPE
) FOR XML PATH('CarrierCode_Destination'),
TYPE
I normally use the following syntax:
SELECT *
FROM TABLE_NAME
WHERE FIELD_NAME = "CRITERIA"

Check if a value exists in a collection stored in XML data type column

I have an XML data type column called "tags".
In that, I am storing a collection, like so:
<ArrayOfString>
<string>personal</string>
<string>travel</string>
<string>gadgets</string>
<string>parenting</string>
</ArrayOfString>
I want to select all the rows, that have one of the values that I am looking for: for example, I want to select all rows in the table that have a tag "travel".
I know that this works, if I know the index of the value I am looking for:
select * from posts
where tags.value('(/ArrayOfString/string)[1]', 'nvarchar(1000)') = 'travel'
but this query works only if the tag "travel" is the 2nd item in the nodes. How do I check if a value exists, irrespective of the position it is in?
select *
from tags
where tags.exist('/ArrayOfString/string[. = "travel"]') = 1
Or like this if you want to check against a variable.
declare #Val varchar(10)
set #Val = 'travel'
select *
from tags
where tags.exist('/ArrayOfString/string[. = sql:variable("#Val")]') = 1
You can try something like this:
SELECT
*
FROM
dbo.Posts
WHERE
tags.exist('/ArrayOfString/string/text()[. = "travel"]') = 1
This will list all the rows that have "travel" in one of the strings in your XML