How to insert NULL into SQL Server DATE field *from XML* - sql

I've got some XML that I'm trying to insert into a Microsoft SQL Server database using their XML datatype functions.
One of the table fields is a nullable DATE column. If the node is missing, then it's inserted as NULL which is great. However, if the node is present but empty <LastDay/> when running the XPath query, it interprets the value from the empty node as an empty string '' instead of NULL. So when looking at the table results, it casts the date to 1900-01-01 by default.
I would like for empty nodes to also be inserted as NULL instead of the default empty string '' or 1900-01-01. How can I get it to insert NULL instead?
CREATE TABLE myxml
(
"id" INT,
"name" NVARCHAR(100),
"company" NVARCHAR(100),
"lastday" DATE
);
DECLARE #xml XML =
'<?xml version="1.0" encoding="UTF-8"?>
<Data xmlns="http://example.com" xmlns:dmd="http://example.com/data-metadata">
<Company dmd:name="Adventure Works Ltd.">
<Employee id="1">
<Name>John Doe</Name>
<LastDay>2016-08-01</LastDay>
</Employee>
<Employee id="2">
<Name>Jane Doe</Name>
</Employee>
</Company>
<Company dmd:name="StackUnderflow">
<Employee id="3">
<Name>Jeff Puckett</Name>
<LastDay/>
</Employee>
<Employee id="4">
<Name>Ill Gates</Name>
</Employee>
</Company>
</Data>';
WITH XMLNAMESPACES (DEFAULT 'http://example.com', 'http://example.com/data-metadata' as dmd)
INSERT INTO myxml (id,name,company,lastday)
SELECT
t.c.value('#id', 'INT' ),
t.c.value('Name[1]', 'VARCHAR(100)' ),
t.c.value('../#dmd:name','VARCHAR(100)' ),
t.c.value('LastDay[1]', 'DATE' )
FROM #xml.nodes('/Data/Company/Employee') t(c)
This produces:
id name company lastday
------------------------------------------------
1 John Doe Adventure Works Ltd. 2016-08-01
2 Jane Doe Adventure Works Ltd. NULL
3 Jeff Puckett StackUnderflow 1900-01-01
4 Ill Gates StackUnderflow NULL
I am trying to achieve:
id name company lastday
------------------------------------------------
1 John Doe Adventure Works Ltd. 2016-08-01
2 Jane Doe Adventure Works Ltd. NULL
3 Jeff Puckett StackUnderflow NULL
4 Ill Gates StackUnderflow NULL

You have to use NULLIF function to avoid default values popping out from XML selection.
Returns a null value if the two specified expressions are equal.
Your query will be changed as below:
SELECT
t.c.value('#id', 'INT' ),
t.c.value('Name[1]','VARCHAR(100)' ),
t.c.value('../#dmd:name', 'VARCHAR(100)' ),
NULLIF(t.c.value('LastDay[1]', 'DATE' ),'')
FROM #xml.nodes('/Data/Company/Employee') t(c)
For more information on NULLIF, please check this MSDN page.

Besides techspider's very good answer I'd like to show another approach:
Doing .nodes() on Company and CROSS APPLY .nodes() on Employee allows a cleaner XPath navigation and avoids the backward navigation you are using by ../#dmd.name. In your case this is just for info probably, but good to consider: If there was a company without any Employee you would skip the whole company otherwise... (My code would skip as well due to the CROSS APPLY, but you could use OUTER APPLY).
And to your actual question: Using the internal cast as xs:date will do the logic within the XQuery and should be faster...
WITH XMLNAMESPACES (DEFAULT 'http://example.com', 'http://example.com/data-metadata' as dmd)
INSERT INTO myxml (id,name,company,lastday)
SELECT
e.value('#id', 'INT' ),
e.value('Name[1]', 'VARCHAR(100)' ),
c.value('#dmd:name', 'VARCHAR(100)' ),
e.value('let $x:=LastDay[1] return $x cast as xs:date?','DATE' )
FROM #xml.nodes('/Data/Company') AS A(c)
CROSS APPLY c.nodes('Employee') AS B(e)

Related

SQL / XPATH - How to select all possible elements that have the same name into separate rows/columns

I need to select all the elements under a specific node, all the elements have the same name.
Tables
Lets say I have 2 tables in my database;
[Access_Groups], This table contains all the groups that can access my application
[User_Accounts], This table contains user details and an XML file that contains their access groups
I want to see which users have access groups in their XML file, that match the groups in my [Access_Groups] table
XML File
The XML files in [User_Accounts] look like this:
<Profile>
<Name>John Smith</Name>
<Role>Developer</Role>
</Profile>
<Groups>
<String>Group_1</String>
<String>Group_2</String>
<String>Group_3</String>
<String>Group_4</String>
<String>Group_5</String>
<String>Group_6</String>
</Groups>
Query
If i run this following query:
SELECT [XML].value('(//Groups)[1]', 'varchar(max)') AS 'XML'
FROM [User_Accounts]
I will get the following result:
XML
Group_1Group_2Group_3Group_4Group_5Group_6
This is of no use to me as i cannot JOIN this table to [Access_Groups]
If I run this query instead:
SELECT
[XML].value('(//Groups/*)[1]', 'varchar(max)') AS 'XMl_1',
[XML].value('(//Groups/*)[2]', 'varchar(max)') AS 'XMl_2',
[XML].value('(//Groups/*)[3]', 'varchar(max)') AS 'XMl_3',
[XML].value('(//Groups/*)[4]', 'varchar(max)') AS 'XMl_4',
[XML].value('(//Groups/*)[5]', 'varchar(max)') AS 'XMl_5',
[XML].value('(//Groups/*)[6]', 'varchar(max)') AS 'XMl_6'
FROM [User_Accounts]
I will get the following result:
XML_1
XML_2
XML_3
XML_4
XML_5
XML_6
Group_1
Group_2
Group_3
Group_4
Group_5
Group_6
I could JOIN this results set to [Access_Groups], However this is no use to me as i have to define every single column. Some users may have up to 100 groups.
Solution
Is there no way i can do something like this?:
SELECT [Name],
[XML].value('(//Groups)[*]', 'varchar(max)') AS 'XML'
FROM [User_Accounts]
To get a result set like this:
Name
XML
John Smith
Group_1
John Smith
Group_2
John Smith
Group_3
John Smith
Group_4
John Smith
Group_5
John Smith
Group_6
I could then SELECT FROM that results set WHERE IN (SELECT * FROM [ACCESS_GROUPS]) to determine if this user had access to my application
Obviously though:
SELECT [Name],
[XML].value('(//Groups)[*]', 'varchar(max)') AS 'XML'
FROM [User_Accounts]
This does not work!
Has anyone got any idea of how i could compare all the 'String' nodes for each user to my [Access_Groups] table?
Thanks!
This is my first post, so it might be a bit rubbish
This is possible, you need to use nodes() to expand your XML into rows, e.g.
DECLARE #User_Accounts TABLE ([XML] XML);
INSERT #User_Accounts ([XML]) VALUES('<Profile>
<Name>John Smith</Name>
<Role>Developer</Role>
</Profile>
<Groups>
<String>Group_1</String>
<String>Group_2</String>
<String>Group_3</String>
<String>Group_4</String>
<String>Group_5</String>
<String>Group_6</String>
</Groups>');
SELECT Name = ua.[XML].value('(Profile/Name/text())[1]', 'VARCHAR(100)'),
Role = ua.[XML].value('(Profile/Role/text())[1]', 'VARCHAR(100)'),
GroupName = g.x.value('text()[1]', 'VARCHAR(100)')
FROM #User_Accounts AS ua
CROSS APPLY ua.[XML].nodes('/Groups/String') AS g (x);
Example on db<>fiddle

XML to SQL Server Parsing Issue

I'm parsing below XML and trying to fetch all the attributes/values of node.
declare #XBL xml='
<Root>
<Department>
<Employees>
<Employee type="temp">
Jason
</Employee>
<Employee type="perm">
Roy
</Employee>
</Employees>
</Department>
<Department>
<Employees >
<Employee type="temp2">
Kevin
</Employee>
</Employees>
</Department>
</Root>'
SELECT
[Type] = XC.value('(#type)[1]', 'varchar(25)'),
[Name] = XC.value('(../Employee)[1]' , 'varchar(30)')
FROM
#XBL.nodes('Root/Department/Employees/Employee') AS XTbl(XC)
Output of above query gives me all the attributes but with first value only(Jason).
Type Name
temp Jason
perm Jason
temp2 Kevin
Expected Output:
Type Name
temp Jason
perm Roy
temp2 Kevin
This should be what you're after:
SELECT XBL.E.value('#type','varchar(25)') AS [Type],
XBL.E.value('(./text())[1]','varchar(30)') AS [Name]
FROM #XBL.nodes('Root/Department/Employees/Employee') XBL(E);
Note the use of /text() as well. When returning data from inside a node, adding /text() actually improves the performance of the query.
Edit: Also, based on your sample xml, the value returned for [Name] would actually be '{Line break} Jason{Line break}' (Obviously replace the line break with the actual character). Is that what you intend, or do you want the whitespace and line breaks/carriage returns removed as well?
You're selecting the first Employee child of the parent Department:
[Name] = XC.value('(../Employee)[1]' , 'varchar(30)'
^^^^^^^^^^^^^^^^
To select the current Employee, use:
[Name] = XC.value('(.)[1]' , 'varchar(30)')
^^^^^^
Example at SQL Fiddle.

How to get value from ntext (in xml format) column in sql

I have a column in my SQL database that is called Triggers_xml_data and its type is ntext. The column is in a xml format and I am trying to get a value from a certain part of the xml. I seen an example of this being done without a column like this:
declare #fileContent xml
set #fileContent ='<my:Header>
<my:Requestor>Mehrlein, Roswitha</my:Requestor>
<my:RequestorUserName>SJM\MehrlR01</my:RequestorUserName>
<my:RequestorEmail>RMehrlein#SJM.com</my:RequestorEmail>
<my:HRContact>Roswita Mehrlein, Beatrice Porta</my:HRContact>
<my:Entity>SJM Germany</my:Entity>
<my:Department>HR/Administration</my:Department>
<my:PositionTitle>Sales Representative</my:PositionTitle>
<my:JobDescription>x0lGQRQAAAABAAAAAAAAAAAeAQAyAAAAVgBAAAAA=</my:JobDescription>
<my:PositionDepartment>Sales</my:PositionDepartment>'
 
;WITH XMLNAMESPACES ('http://schemas.microsoft.com/office/infopath/2003/myXSD/2005-08-29T12-58-51' as my)
select #fileContent.value('(//my:PositionDepartment)[1]', 'varchar(255)')
But I want to select my column like this:
Declare #filevalue xml
select de.triggers_xml_data
from dbo.DEPLOYMENT_ENVIRONMENT as de
But this is not working and I tried to use this #filecontent.value('(//value)[1]','varchar(255)') and making it equal the column value, I have tried casting it but I can't find a way to do this. Is this possible?
When I do this:
SELECT
CAST(
REPLACE(CAST(de.TRIGGERS_XML_DATA AS VARCHAR(MAX)), 'encoding="utf-16"', '')
AS XML).value('(triggers/triggerDefinition/config/item/value)[1]', 'NVARCHAR(max)') as Item, de.ENVIRONMENT_ID
from dbo.DEPLOYMENT_ENVIRONMENT as de
where de.ENVIRONMENT_ID = 19234819
I am getting a null value returned.
Here is an example of what my xml could look like:
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<configuration xml:space="preserve">
<triggers>
<defined>true</defined>
<triggerDefinition>
<id>1</id>
<name>After successful deployment</name>
<userDescription/>
<isEnabled>true</isEnabled>
<pluginKey>com.atlassian.bamboo.triggers.atlassian-bamboo-triggers:afterSuccessfulDeployment</pluginKey>
<triggeringRepositories/>
<config>
<item>
<key>deployment.trigger.afterSuccessfulDeployment.triggeringEnvironmentId</key>
<value>19234819</value>
</item>
</config>
</triggerDefinition>
</triggers>
<bambooDelimiterParsingDisabled>true</bambooDelimiterParsingDisabled>
</configuration>
The XML, as you posted it, is not valid. Your code example does not work... It is not allowed to use a namespace prefix without a namespace declaration. Furthermore your example misses the closing Header-tag...
I corrected this...
DECLARE #yourTbl TABLE(ID INT, YourXML NTEXT);
INSERT INTO #yourTbl VALUES
(1,N'<my:Header xmlns:my="DummyUrl">
<my:Requestor>Mehrlein, Roswitha</my:Requestor>
<my:RequestorUserName>SJM\MehrlR01</my:RequestorUserName>
<my:RequestorEmail>RMehrlein#SJM.com</my:RequestorEmail>
<my:HRContact>Roswita Mehrlein, Beatrice Porta</my:HRContact>
<my:Entity>SJM Germany</my:Entity>
<my:Department>HR/Administration</my:Department>
<my:PositionTitle>Sales Representative</my:PositionTitle>
<my:JobDescription>x0lGQRQAAAABAAAAAAAAAAAeAQAyAAAAVgBAAAAA=</my:JobDescription>
<my:PositionDepartment>Sales</my:PositionDepartment>
</my:Header>');
--Lazy approach
SELECT ID
,CAST(CAST(YourXml AS NVARCHAR(MAX)) AS XML).value(N'(//*:PositionDepartment)[1]','nvarchar(max)')
FROM #yourTbl;
--explicit approach
WITH XMLNAMESPACES('DummyUrl' AS my)
SELECT ID
,CAST(CAST(YourXml AS NVARCHAR(MAX)) AS XML).value(N'(/my:Header/my:PositionDepartment)[1]','nvarchar(max)')
FROM #yourTbl
Some Background
If possible you should not store XML in other format than XML and further more one should avoid NTEXT, as it is depricated since SS2005!.
You have to cast NTEXT to NVARCHAR(MAX) first, than cast this to XML. The second will break, if the XML is not valid. That means: If the XML is really the way you posted it, this cannot work!
UPDATE: String-based approach, if XML does not work
If you cannot cast this to XML you might try this
--String based
WITH Casted AS
(
SELECT ID
,CAST(YourXML AS NVARCHAR(MAX)) AS TheXmlAsString
FROM #yourTbl
)
,WithPosition AS
(
SELECT Casted.*
,CHARINDEX(N'<my:PositionDepartment>',TheXmlAsString) + LEN(N'<my:PositionDepartment>') AS FirstLetter
FROM Casted
)
SELECT ID
,SUBSTRING(TheXmlAsString,FirstLetter,CHARINDEX('<',TheXmlAsString,FirstLetter)-FirstLetter)
FROM WithPosition
UPDATE 2
According to your edit the following returns a NULL value. This is good, because it shows, that the cast was successfull.
SELECT
CAST(
REPLACE(CAST(de.TRIGGERS_XML_DATA AS VARCHAR(MAX)), 'encoding="utf-16"', '')
AS XML).value('(triggers/triggerDefinition/config/item/value)[1]',
'NVARCHAR(max)') as Item, de.ENVIRONMENT_ID
from dbo.DEPLOYMENT_ENVIRONMENT as de
where de.ENVIRONMENT_ID = 19234819
Try this (skip namespace with wildcard):
SELECT
CAST(
REPLACE(CAST(de.TRIGGERS_XML_DATA AS VARCHAR(MAX)), 'encoding="utf-16"', '')
AS XML).value('(*:triggers/*:triggerDefinition/*:config/*:item/*:value)[1]', 'NVARCHAR(max)') as Item, de.ENVIRONMENT_ID
from dbo.DEPLOYMENT_ENVIRONMENT as de
where de.ENVIRONMENT_ID = 19234819
And this should be even better:
SELECT
CAST(CAST(de.TRIGGERS_XML_DATA AS NVARCHAR(MAX)) AS XML).value('(*:triggers/*:triggerDefinition/*:config/*:item/*:value)[1]', 'NVARCHAR(max)') as Item, de.ENVIRONMENT_ID
from dbo.DEPLOYMENT_ENVIRONMENT as de
where de.ENVIRONMENT_ID = 19234819
UPDATE 3
I'd rather cut away the full declaration. Your posted example would go like this
DECLARE #DEPLOYMENT_ENVIRONMENT TABLE(ENVIRONMENT_ID INT, TRIGGERS_XML_DATA NTEXT);
INSERT INTO #DEPLOYMENT_ENVIRONMENT VALUES
(19234819,N'<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<configuration xml:space="preserve">
<triggers>
<defined>true</defined>
<triggerDefinition>
<id>1</id>
<name>After successful deployment</name>
<userDescription/>
<isEnabled>true</isEnabled>
<pluginKey>com.atlassian.bamboo.triggers.atlassian-bamboo-triggers:afterSuccessfulDeployment</pluginKey>
<triggeringRepositories/>
<config>
<item>
<key>deployment.trigger.afterSuccessfulDeployment.triggeringEnvironmentId</key>
<value>19234819</value>
</item>
</config>
</triggerDefinition>
</triggers>
<bambooDelimiterParsingDisabled>true</bambooDelimiterParsingDisabled>
</configuration>');
WITH Casted AS
(
SELECT CAST(de.TRIGGERS_XML_DATA AS NVARCHAR(MAX)) AS XmlAsSting
FROM #DEPLOYMENT_ENVIRONMENT as de
where de.ENVIRONMENT_ID = 19234819
)
SELECT CAST(SUBSTRING(XmlAsSting,CHARINDEX('?>',XmlAsSting)+2,8000) AS XML).value('(/*:configuration/*:triggers/*:triggerDefinition/*:config/*:item/*:value)[1]', 'NVARCHAR(max)') as Item
FROM Casted;

Empty XML node causes Error converting data type varchar to numeric

How do I simply insert NULL if the node is empty or absent?
CREATE TABLE myxml (
"hours" DECIMAL(11,2)
);
DECLARE #xml XML =
'<?xml version="1.0" encoding="UTF-8"?>
<Data>
<Employee>
<NUMHOURS>0.5</NUMHOURS>
</Employee>
<Employee/>
<Employee>
<NUMHOURS>5</NUMHOURS>
</Employee>
<Employee>
<NUMHOURS/>
</Employee>
</Data>';
INSERT INTO myxml ("hours")
SELECT
t.c.value('NUMHOURS[1]','DECIMAL(11,2)' )
FROM #xml.nodes('/Data/Employee') t(c)
The empty node <NUMHOURS/> causes:
Error converting data type nvarchar to numeric.
I have tried:
NULLIF(t.c.value('NUMHOURS[1]','DECIMAL(11,2)' ),'')
but that seems to get processed after the fact resulting in the same error.
You could provide a default value for invalid numbers:
select case
when ISNUMERIC(t.c.value('NUMHOURS[1]', 'nvarchar(100)')) <> 1 then NULL
else t.c.value('NUMHOURS[1]', 'decimal(11,2)')
end
from #xml.nodes('/Data/Employee') t(c)
There might be a better approach: (should be at least faster with many rows...):
SELECT t.c.value('let $x:=NUMHOURS[1] return $x cast as xs:decimal?','decimal(11,2)')
FROM #xml.nodes('/Data/Employee') t(c);
The cast as xs:decimal? will cast the value to decimal if possible, or return null. Thus the final 'decimal(11,2)' has no type conversion issue any more...

Parsing nested XML into SQL table

What would be the right way to parse the following XML block into SQL Server table according to desired layout (below)? Is it possible to do it with a single SELECT statement, without UNION or a loop? Any takers? Thanks in advance.
Input XML:
<ObjectData>
<Parameter1>some value</Parameter1>
<Parameter2>other value</Parameter2>
<Dates>
<dateTime>2011-02-01T00:00:00</dateTime>
<dateTime>2011-03-01T00:00:00</dateTime>
<dateTime>2011-04-01T00:00:00</dateTime>
</Dates>
<Values>
<double>0.019974</double>
<double>0.005395</double>
<double>0.004854</double>
</Values>
<Description>
<string>this is row 1</string>
<string>this is row 2</string>
<string>this is row 3</string>
</Values>
</ObjectData>
Desired table output:
Parameter1 Parameter2 Dates Values Description
Some value Other value 2011-02-01 00:00:00.0 0.019974 this is row 1
Some value Other value 2011-03-01 00:00:00.0 0.005395 this is row 2
Some value Other value 2011-04-01 00:00:00.0 0.004854 this is row 3
I am after an SELECT SQL statement using OPENXML or xml.nodes() functionality. For example, the following SELECT statement results in production between Values and Dates (that is all permutations of Values and Dates), which is something I want to avoid.
SELECT
doc.col.value('Parameter1[1]', 'varchar(20)') Parameter1,
doc.col.value('Parameter2[1]', 'varchar(20)') Parameter2,
doc1.col.value('.', 'datetime') Dates ,
doc2.col.value('.', 'float') [Values]
FROM
#xml.nodes('/ObjectData') doc(col),
#xml.nodes('/ObjectData/Dates/dateTime') doc1(col),
#xml.nodes('/ObjectData/Values/double') doc2(col);
You can make use of a numbers table to pick the first, second, third etc row from the child elements. In this query I have limited the rows returned to the number if dates provided. If there are more values or descriptions than dates you have to modify the join to take that into account.
declare #XML xml = '
<ObjectData>
<Parameter1>some value</Parameter1>
<Parameter2>other value</Parameter2>
<Dates>
<dateTime>2011-02-01T00:00:00</dateTime>
<dateTime>2011-03-01T00:00:00</dateTime>
<dateTime>2011-04-01T00:00:00</dateTime>
</Dates>
<Values>
<double>0.019974</double>
<double>0.005395</double>
<double>0.004854</double>
</Values>
<Description>
<string>this is row 1</string>
<string>this is row 2</string>
<string>this is row 3</string>
</Description>
</ObjectData>'
;with Numbers as
(
select number
from master..spt_values
where type = 'P'
)
select T.N.value('Parameter1[1]', 'varchar(50)') as Parameter1,
T.N.value('Parameter2[1]', 'varchar(50)') as Parameter2,
T.N.value('(Dates/dateTime[position()=sql:column("N.Number")])[1]', 'datetime') as Dates,
T.N.value('(Values/double[position()=sql:column("N.Number")])[1]', 'float') as [Values],
T.N.value('(Description/string[position()=sql:column("N.Number")])[1]', 'varchar(max)') as [Description]
from #XML.nodes('/ObjectData') as T(N)
cross join Numbers as N
where N.number between 1 and (T.N.value('count(Dates/dateTime)', 'int'))
Use the OPENXML function. It is a rowset provider (it returns the set of rows parsed from the XML) and thus can be utilized in SELECT or INSERT like:
INSERT INTO table SELECT * FROM OPENXML(source, rowpattern, flags)
Please see the first example in the documentation link for clarity.
Typically, if you wanted to parse XML, you'd do it a programming language like Perl, Python, Java or C# that a) has an XML DOM, and b) can communicate with a relational database.
Here's a short article that shows you some of the basics of reading and writing XML in C# ... and even has an example of how to create an XML document from a SQL query (in one line!):
http://www.c-sharpcorner.com/uploadfile/mahesh/readwritexmltutmellli2111282005041517am/readwritexmltutmellli21.aspx