I am reading data from XML into a table. When I do select from the table, the table is empty.
SET #INPUTXML = CAST(#Attribute AS XML)
EXEC Sp_xml_preparedocument #TestDoc OUTPUT, #INPUTXML
SELECT Row_Number() OVER (ORDER BY Name) AS Row, *
INTO #tData
FROM OPENXML(#TestDoc, N'/DocumentElement/dtData')
WITH (
ID VARCHAR(100) './ID'
, Name VARCHAR(100) './Name'
, Value VARCHAR(max) './Value'
, Column VARCHAR(100) './Column'
)
EXEC Sp_xml_removedocument #TestDoc
Below are my questions:
select * from #tData is empty table. Why is data not getting populated?
What does Sp_xml_preparedocument do? When I print #TestDoc, it gives me a number
What is Sp_xml_removedocument ?
To answer your questions though.
#tData is empty because your SELECT statement returned no data. A SELECT...INTO statement will still create the table, even if the SELECT returns no rows. Why your SELECT is returning no data is impossible for us to say, because we have no sample data. If you remove the INTO clause you will see that no rows are returned, so you need to fix your SELECT, FROM, etc. but that brings on to my statement in a minute (about using XQUERY)
sp_xml_preparedocument (Transact-SQL) explains better than I could. Really though, you shouldn't be using it anymore, as it was used to read XML back in SQL Server 2000 (maybe 2005) and prior. Certainly SQL Server 2008 supported XQUERY, which you must be at least using if you are using SSMS 2014. To quote the opening statement of the documentation though:
Reads the XML text provided as input, parses the text by using the MSXML parser (Msxmlsql.dll), and provides the parsed document in a state ready for consumption. This parsed document is a tree representation of the various nodes in the XML document: elements, attributes, text, comments, and so on.
sp_xml_removedocument (Transact-SQL), but again, you should be using XQUERY.
Removes the internal representation of the XML document specified by the document handle and invalidates the document handle.
Related
Using Microsoft SQL Server 2019.
I have two columns, one text representing some xml, another varbinary(max) representing already compressed xml, that I need to compress.
Please assume I cannot change the source data, but conversions can be made as necessary in the code.
I'd like to compress the text column, and initially it works fine, but if I try to save it into a temp table to be used further along in the process I get weird characters like ‹ or tŠÌK'À3û€Í‚;jw. Again, the first temp table I make stores it just fine, I can select the initial table and it displays compressed correctly. But if I need to pull it into a secondary temp table or variable from there it turns into a mess.
I've tried converting into several different formats, converting later in the process, and bringing in the source data for the column at the very last stage, but my end goal is to populate a variable that will be converted into JSON, and it always ends up weird there as well. i just need the compressed version of the columns do display properly when viewing the json variable I've made.
Any suggestions on how to tackle this?
Collation issue?
This smells of collation issue. tempdb is actually its own database with its own default collation and other settings.
In one database with default CollationA you call COMPRESS(NvarcharData) and that produces some VARBINARY.
In other database (tempdb) with default CollationB you call CONVERT(NVARCHAR(MAX), DECOMPRESS(CompressedData)). Now, what happens under the hood is:
CompressedData gets decompressed into VARBINARY representing NvarcharData in CollationA
that VARBINARY is converted to NVARCHAR assuming the binary data represents NVARCHAR data in CollationB, which is not true!
Try to be more explicit (collation, data type) with conversions between XML, VARBINARY and (N)VARCHAR.
Double compression?
I have also noticed "representing already compressed xml, that I need to compress". If you are doublecompressing, maybe you forgot to doubledecompress?
Example?
You are sadly missing an example, but I have produced minimal example of converting between XML and compressed data that works for me.
BEGIN TRANSACTION
GO
CREATE TABLE dbo.XmlData_Base (
PrimaryKey INTEGER NOT NULL IDENTITY(1, 1),
XmlCompressed VARBINARY(MAX) NULL
);
GO
CREATE OR ALTER VIEW dbo.XmlData
WITH SCHEMABINDING
AS
SELECT
BASE.PrimaryKey,
CONVERT(XML, DECOMPRESS(BASE.XmlCompressed)) AS XmlData
FROM
dbo.XmlData_Base AS BASE;
GO
CREATE OR ALTER TRIGGER dbo.TR_XmlData_instead_I
ON dbo.XmlData
INSTEAD OF INSERT
AS
BEGIN
INSERT INTO dbo.XmlData_Base
(XmlCompressed)
SELECT
COMPRESS(CONVERT(VARBINARY(MAX), I.XmlData))
FROM
Inserted AS I;
END;
GO
CREATE OR ALTER TRIGGER dbo.TR_XmlData_instead_U
ON dbo.XmlData
INSTEAD OF UPDATE
AS
BEGIN
UPDATE BASE
SET
BASE.XmlCompressed = COMPRESS(CONVERT(VARBINARY(MAX), I.XmlData))
FROM
dbo.XmlData_Base AS BASE
JOIN Inserted AS I ON I.PrimaryKey = BASE.PrimaryKey;
END;
GO
INSERT INTO dbo.XmlData
(XmlData)
VALUES
(CONVERT(XML, N'<this><I>I call upon thee!</I></this>'));
SELECT
*
FROM
dbo.XmlData;
SELECT
PrimaryKey,
XmlCompressed,
CONVERT(XML, DECOMPRESS(XmlCompressed))
FROM
dbo.XmlData_Base;
UPDATE dbo.XmlData
SET
XmlData = CONVERT(XML, N'<that><I>I call upon thee!</I></that>');
SELECT
*
FROM
dbo.XmlData;
SELECT
PrimaryKey,
XmlCompressed,
CONVERT(XML, DECOMPRESS(XmlCompressed))
FROM
dbo.XmlData_Base;
GO
ROLLBACK TRANSACTION;
I'm using SQL Server 2019 - v15.0.4123.1.
This is the sample query (from this question):
DECLARE #tbl TABLE (ID INT IDENTITY PRIMARY KEY, city VARCHAR(30))
INSERT INTO #tbl (city)
VALUES ('Miami'), ('Orlando');
SELECT
'SIN_OPE' AS [#cod_1],
'08' AS [#cod_2],
'12' AS [#num_reg],
'yyyyyyyyyyyyyyyyyyy.xsd' AS [#xsi:noNamespaceSchemaLocation],
(SELECT *
FROM #tbl
FOR XML PATH('r'), TYPE)
FOR XML PATH('root'), TYPE, ELEMENTS XSINIL;
The result of the query is:
I just want to save/export this file as an XML file, automatically (with a stored procedure) to a folder in my PC. I've found many possible solutions but they are pretty old and doesn't work with my environment. I still can't understand if it's possible to export XML from SQL Server. I would like to avoid the manual passage "Query - Results to - Results to file"
Any advice?
Thank you
On you SQL Server Management Tool go to Query --> Result --> Result to file and then execute your query.
once you execute the query it will open a window & give you an option to save the file, there you can save it as xml.
DECLARE #result NVARCHAR(max);
SET #result = (SELECT * FROM table
FOR JSON AUTO, ROOT('Data'))
SELECT #result;
This returns a json string of ~43000 characters, with some results truncated.
SET #result = (SELECT * FROM table
FOR JSON AUTO, ROOT('Data'))
This returns a json string of ~2000 characters. Is there any way to prevent any truncation? Even when dealing with some bigdata and the string is millions and millions of characters?
I didn't find and 'official' answer, but it seems that this is an error with the new 'FOR JSON' statement which is splitting the result in lines 2033 characters long.
As recommended here the best option so far is to iterate through the results concatenating the returned rows:
string result = "";
while (reader.Read())
{
result += Convert.ToString(reader[0]);
}
BTW, it seems that the latest versions of SSMS are already applying some kind of workaround like this to present the result in a single row.
I was able to get the full, non-truncated string by using print instead of select in SQL Server 2017 (version 14.0.2027):
DECLARE #result NVARCHAR(max);
SET #result = (SELECT * FROM table
FOR JSON AUTO, ROOT('Data'))
PRINT #result;
Another option would be to download and use Azure Data Studio which I think is a multi-platform re-write of SSMS (similar to how Visual Studio was re-written as VS Code). It seems to spit out the entire, non-truncated json string as expected out of the box!
This will also work if you insert into a temp table - not presenting does not apply the truncate of SSMS.
Might be usefull if you need to calculate several values.
declare #json table (j nvarchar(max));
insert into #json select * from(select* from Table where Criteria1 for json auto)a(j)
insert into #json select * from(select* from Table where Criteria2 for json auto)a(j)
select * from #json
I know this is an old thread but I have had success with this issue by sending the result to an XML variable. The advantage of using an XML variable is that the size is not stated as character length but by size of the string in memory, which can be changed in the options. Therefore Brad C's response would now look like...
DECLARE #result XML
SET #result = (SELECT * FROM table
FOR JSON AUTO, ROOT('Data'))
SELECT #result
or...
PRINT #result;
Here is the answer to JSON truncation:
SQL divides the JSON result into chunks 2k in size (at least my SQL 2016 installation does), one chunk in the first column of each row in the result set. To get the entire result, your client code has to loop through the result set and concatenate the first column of each record. When you've gotten to the end of the rows, voila, your entire JSON result is retrieved, uncut.
When I first encountered the truncation problem I was baffled, and wrote off FOR JSON for several years as an unserious feature suited only to the smallest of datasets. I learned that I need to read the entire recordset only from the FOR XML documentation, and never actually saw it mentioned in the FOR JSON docs.
The easiest workaround to avoid the truncation is to wrap the query in another select:
select (
<your query> FOR JSON PATH [or FOR JSON AUTO]
) as json
We've seen similar issues in SSMS, without using a variable SSMS truncates at 2033.
With a variable the query actually works OK when you use an nvarcahr(max) variable, but it truncates the output in the query results view at 43697.
A possible solution I've tested is outputting Query results to a file, using BCP:
bcp "DECLARE #result NVARCHAR(max); SET #result = (SELECT * FROM table FOR JSON AUTO, ROOT('Data')); SELECT #result as Result;" queryout "D:\tmp\exportOutput.txt" -S SQL_SERVER_NAME -T -w
See BCP docs for specifying server name\instance and authentication options
It's difficult to determine exactly what the problem you're having without posting the data, but I had a similar problem when I was attempting to export a query in JSON format. The solution that worked for me was to go to Query/Query Options/Results/Text/Set "Maximum number of characters displayed in each column:" to 8192 (max value AFAIK).
This probably won't help much with your first query, but that potentially could be broken into smaller queries and executed successfully. I would anticipate that you could effectively run your second query after changing that setting.
If your datalength is less than 65535 then you should use the suggestion of #dfundako who commented in the first post:
Try going to Tools, Options, Query Results, SQL Server, Results to Grid, and set Non-XML data to the max amount (I think 65535)
In my case the datalength was 21k characters so after exporting to grid I copied the value and it was fine, not truncated. Still it doesn't solve the the issue for those with bigger amount of data.
Try Visual Studio Code with Microsoft SQL extension. I got 6800 characters of JSON without truncation. It seems SSMS truncates results.
I am trying to flatten the XML into SQL table using below code. The input datatable (#incomingTable) contains 10k un typed XMLs the query takes 7 sec to return the output. When I checked the Execution Plan found most of the time is spent on "Table Vaued Functions (XML Reader with XPATH filter)" step. My guess is this step refers to Value method in query.
The value() method uses the Transact-SQL CONVERT operator implicitly and tries to convert the result of the XQuery expression, to the corresponding SQL type specified by Transact-SQL conversion.
Questions:Is there any other XML method to retrieve element/attribute value without data type converting . because I want the data as string anyhow. Helps me compare the results for two approaches.
Is there any other way to optimize this query?
select
sqlXml.value('#ID', 'varchar(50)') as XMLFieldName,
sqlXml.value('#TS', 'varchar(50)') as XMLTSValue,
sqlXml.value('.','varchar(800)') as XMLFieldValue
from #incomingTable
cross apply playfieldvalues.nodes('/PlayAttributes/PlayFields/PlayField') as
XMLData(sqlXml)
Try to use OPENXML:
DECLARE #idoc int;
EXEC sp_xml_preparedocument #idoc OUTPUT, #incomingTable;
SELECT *
FROM OPENXML (#idoc, '/PlayAttributes/PlayFields/PlayField',1)
WITH (XMLFieldName varchar(50) '#ID',
XMLTSValue varchar(50) '#TS',
XMLFieldValue varchar(800) '.');
EXEC sp_xml_removedocument #idoc;
OPENXML allows accessing XML data as if it were a relational recordset. It provides a tabular (rowset) view of in-memory representation of an XML document. Technically, OPENXML is a rowset provider similar to a table or a view; hence it can be used wherever a table or a view is used. For instance, you can use OPENXML with SELECT or SELECT INTO statements, to work on an XML document that is retained in memory.
Source
I am developing an sql server 2012 based application.
I am a newbie to SQl server. But the requirement is to use it.
One of the table I am using contains an XML Datatype column. However the data containing in that column may vary as per the xml element. The only thing in common is the root: For instance this is a sample data:
<Tags>
<key1>Value1</key1>
<key2>Value2</key2>
<key3>Value3</key3>
<key4>Value4</key4>
<key5>Value5</key5>
<key6>Value6</key6>
</Tags>
What I want to do is to query the whole table and fetch records that will match a specific key and a specific values sent by the user.
Please assist me.
Well it sounds like you need to use some variables to build an XQuery, yes? So, assuming you build a stored procedure or something which takes a pair of string arguments for key and value you could use the following example I've knocked up in SQL Fiddle so you can try it out.
DECLARE #key nvarchar(20)
DECLARE #value nvarchar(20)
SET #key = N'key5'
SET #value = N'Value5'
SELECT
TagValue = T1.xmlcol.value('(/Tags/*[local-name()=sql:variable("#key")])[1]', 'varchar(10)')
FROM
dbo.T1
WHERE
T1.xmlcol.exist('/Tags/*[local-name()=sql:variable("#key")][text() = "Value5"]') = 1