In SQL Server 2005 I am trying to query a varchar(MAX) column which has some rows with text data that exceed the 8192. Yet, In Management Studio I have under Tools --> Options --> Query Results --> Results to Text --> Max numbers of characters displayed in each column = 8192, which is a maximum. Accordingly, it seems the truncation on these rows occurs only due to the limitation imposed by text output.
The only thing I see to get around this is to use a SUBSTRING function to grab say the first 8000 chars, then the next 8000 chars etc. etc. But this is ugly and error prone.
I should mention that SSIS and BCP are not options for me.
Does anyone have a better suggestion? Thanks!
You can export the data to a flat file which will not be truncated. To do this:
Right click the Database
Click Tasks -> Export Data
Select your Data Source (defaults should be fine)
Choose "Flat File Destination" for the Destination type.
Pick a file name for the output.
On the "Specify Table Copy or Query", choose "Write a query to specify the data to transfer"
Paste in your query
Remaining steps should be self explanatory. This will output the file to text and you can open it in your favorite text editor.
I also use XML but a slightly different method that gets around most of the issues with XML entitisation.
declare #VeryLongText nvarchar(max) = '';
SELECT top 100 #VeryLongText = #VeryLongText + '
' + OBJECT_DEFINITION(object_id)
FROM sys.all_objects
WHERE type='P' and is_ms_shipped=1
SELECT LEN(#VeryLongText)
SELECT #VeryLongText AS [processing-instruction(x)] FOR XML PATH('')
PRINT #VeryLongText /*WILL be truncated*/
Make sure that the "XML data" limit in SSMS is set sufficiently high!
Did you try this simple solution? Only 2 clicks away!
At the query window,
set query options to "Results to Grid", run your query
Right click on the results tab at the grid corner, save results as any files
You will get all the text you want to see in the file!!! I can see 130,556 characters for my result of a varchar(MAX) field
My solution was a bit round-about but got me there (as long as the output is less than 65535 characters):
In SQL Management Studio, set the limit for grid results to 65535 (Tools > Options > Query Results > SQL Server > Results to Grid > Non XML data)
Run the query, output to grid
Right-click the results, choose "Save Results As..." and save the results to a file
Open the file in notepad or similar to get the output
UPDATE: To demonstrate that this works, here's some SQL that selects a single 100,000 character column. If I save the grid output to a csv file, all 100,000 characters are there with no truncation.
DECLARE #test nvarchar(MAX), #i int, #line nvarchar(100)
SET #test = ''; SET #i = 100
WHILE #i < 100000
BEGIN
SET #test = #test + STUFF(REPLICATE('_', 98) + CHAR(13) + CHAR(10), 1, LEN(CAST(#i AS nvarchar)), CAST(#i AS nvarchar))
SET #i = #i + 100
END
SELECT #test
Notes:
It doesn't seem to make any difference what the character length setting is, as I orignally thought.
I'm using SQL 2008 R2 (both the server and Management Studio)
It doesn't seem to make a difference if the long column is stored in a local variable (as in this example), or selected from an actual table
I ran in to this trying to export XML. This is the solution I used:
Select the Result to Grid option, right click the link that shows up in the Results pane, then select Save Results As, choose the All Files file type, give the file a name and click Save. All the xml data is saved correctly to a file.
I'm using SSMS 10, and I could't get Torre's solution to work. The export wizard kept thinking the input column was an image:
The data type for "input column "XML_F52E2B61-18A1-11d1-B105-00805F49916B" (26)" is DT_IMAGE
In SSMS if you select data from a row it is limited to a small number of characters, but if you Edit data from a row, the full value will be there. It might not always be there but if you ctrl-a, ctrl-c then past it in an editor it will all be there.
If given a choice I would have the query return the data as "For XML Auto" or "For XML Raw" or "For XML explicit" that way the limitations are much higher and you can do much more with the outputed results.
I usually use XML to get huge debug string as output (using test harness from Luke):
declare #test nvarchar(max), #i int, #line nvarchar(100)
set #test = ''; set #i = 100
while #i < 100000
begin
set #test = #test + stuff(replicate('_', 98) + char(13) + char(10), 1, len(cast(#i as nvarchar)), cast(#i as nvarchar))
set #i = #i + 100
end
-- ctrl+d for "results to grid" then click the xml output
--select cast('<root>' + #test + '</root>' as xml)
-- revised
select #test for xml path(''), type;
Another workaround , use HeidiSql for this tricky queries. It does not have the limits in the field lenght.
The truncation you are talking about only happens in Management Studio. If you pull the column into another app, it will not be truncated.
There's no way you're using Query Analyzer to talk to SQL Server 2005. Do you mean Management Studio?
Related
We have an extremely large nvarchar(max) field that contains html. Within this html is an img tag.
Example:
<img style="float:right" src="data:image/png;base64,/9j/4AAQSkZJRgABAQEBLAEsAAD/7gAOQW....
The length of this column is 1645151, although what is being replace is a bit less than this, but not a lot.
What we are trying to do, is a replace in SQL on the column:
declare #url varchar(50) = 'myimageurl';
UPDATE table SET field =
CAST(REPLACE(CAST(field as NVARCHAR(MAX)),#source,'#url') AS NVARCHAR(MAX))
Where #source, is the above image bytes as string, which are assigned to an nvarchar(max) variable before running the replace. and dest is the url of an image, rather than the images bytes as string.
Although I still get the message string or binary data would be truncated.
Does anyone know if this is possible in SQL to replace strings as large as this.
I had the same error, but on a different function.
The fault was that my pattern has longer than my expression, which means that your search pattern will be truncated.
I hope this helps someone.
Also, make sure you put pattern and expression in the right location of your function.
Instead of doing the replace, can you rebuilt the entire field by parsing out the rest of the img tag?
Something like:
declare #Field nvarchar(max) = '<img style="float:right" src="data:image/png;base64,/9j/4AAQSkZJRgA....BAQEBLAEsAAD/7gAOQW" />'
declare #Source nvarchar(max) = 'data:image/png;base64,/9j/4AAQSkZJRgA....BAQEBLAEsAAD/7gAOQW'
declare #URL nvarchar(max) = 'www.img.img/img.png'
declare #Chars int = 20
select left(#Field,patindex('%' + left(#Source,#Chars) + '%', #Field) - 1) as HTMLStart
,#URL as ImgURL
,right(#Field,len(#Field) - patindex('%' + right(#Source,#Chars) + '%', #Field) - #Chars + 1) as HTMLEnd
If you were wanting to run this on a whole dataset at once, you would simply need to look for the src="data:image/png;base64, element and work backwards from there using a similar methodology to the above. Depends on how you are identifying which binary data to replace and what to replace it with.
I had data in Excel like
7540006
7540447
But when I import the data into SQL Server, it is saved as
7.54001e+006
7.54045e+006
So now when I try to convert it to back to original state, it's not ending up with the correct value.
I have tried following queries for conversion
declare #a varchar(40)
set #a = '7.54001e+006'
declare #b decimal(27, 12)
SELECT #b = CONVERT(REAL, #a, 2)
SELECT LTRIM(RTRIM(str(#a))), LTRIM(STR(#a))
SELECT CAST('7.54001e+006' as REAL)
and the output I am getting is addition of 3 to original value for all methods
i.e.
7540010
7540050
How do I convert it back to original state ??
Try the following query which gives the exact value
select CAST(CAST('7.54001e+006' AS FLOAT) AS bigint)
All data is stored as Unicode string 255 (DT_WSTR) in excel.
Read excel data as Unicode form. then do conversion in ssis or database using.
SELECT CAST('7.54001e+006' as REAL)
In excel data source >> connection manager >>Data access mode
== QUERY
SELECT * FROM [SheetName$]
when you will save in database that time convert value into toString() like
Convert.toString(7540006)
then it will save original value in database.
Using ssms 2012 to query a 2008R2 instance, I am trying to get a plan for a specific query using the DMVs like this:
SELECT t.text
, p.query_plan
from sys.dm_exec_query_stats qs
cross join sys.dm_exec_sql_text(qs.sql_handle) t
cross join sys.dm_exec_text_query_plan(qs.plan_handle,0,-1) p
where t.text like ...
The query plan column is getting cut off after 43,679 characters. Ending with
< /Outp
instead of
< /ShowPlanXML>
I tested this with a smaller query and the whole text was returned. The query in question is not that complex, but has a lot of columns, which may be making it a bit more verbose. Also, the value returned is not a link to the plan but just the XML in text form.
Is there a limit to what is stored in plan cache or I am doing something wrong in SSMS that it is not returning the value as a plan link in the column?
Even if the bug Aaron mentions in comments is in play here, you should be able to get around it with an SSMS tweak. You can directly cast the result of your query into the xml data type, and then return it that way.
If you're returning the XML in a grid view, go to Tools/Options/Query Results/SQL Server/Results to Grid and see what the setting for Maximum Characters Retrieved for XML data is, and bump it up to "Unlimited". This should allow you to circumvent the varchar limit.
Whoops! Wrong DMV. I needed to use dm_exec_query_plan not dm_exec_text_query_plan. That solved it - thanks for the replies.
Posting another answer as this may help someone else out in the future. There are a few approaches that I've found to work around the 43,679 character limitation in outputs displayed in Grid View as specific to this question. These approaches also work if your plan exceeds the SQL XML Data Type limit of 128 nested nodes.
The first, and arguably easiest option, is to fire up PowerShell ISE and follow the instructions outlined on this blog post by Patrick Keisler. Paste his script into the ISE editor, adjust the OFFSET values (recommended), output paths/names, and then run everything to generate the final sqlplan file.
If you don't want to use PowerShell, a kludgy TSQL method I cobbled together can also be used, as follows:
-- Quick and dirty script to output large execution plans from cache
-- Be sure to replace Plan Handle and Offset Values Below
DECLARE #query_plan_nvarmax NVARCHAR(MAX), #len_out INT, #sub_str INT = 0, #sub_end INT = 43679
DECLARE #full_query_plan TABLE
(
line INT IDENTITY(1,1),
qp_line NVARCHAR(MAX)
)
SELECT #query_plan_nvarmax = query_plan,
#len_out = LEN(query_plan)--, CAST(query_plan AS XML) as xml_query_plan
-- Usage: Get Query Offset Values From sys.dm_exec_query_stats DMV
-- sys.dm_exec_text_query_plan(plan_handle, OFFSET_START|DEFAULT, OFFSET_END|DEFAULT)
FROM sys.dm_exec_text_query_plan(0x050005005EDA4857307D56540300000001000000000000000000000000000000000000000000000000000000, 10078, 83616)
WHILE #sub_str < #len_out
BEGIN
INSERT INTO #full_query_plan (qp_line)
SELECT SUBSTRING(#query_plan_nvarmax, #sub_str, #sub_end)
SET #sub_str = #sub_end
SET #sub_end = #sub_end + 43679
IF #sub_end > #len_out
SET #sub_end = #len_out
END
-- Save Output of qp_line column to text editor and remove newline characters \r\n
-- I prefer Notepad++, but any editor will suffice then save output as a .sqlplan and open in SSMS
SELECT *
FROM #full_query_plan
Take note, you'll need to edit the output of the query in an external text editor to remove the newline characters \r\n and save the result as a .sqlplan file.
If you have 100 occurrences of "PRINT" in your stored procedure, is there a nice way to turn them all on/off when debugging an non-debugging?
I could add a variable #isdebug = 1
and later on, do something like
IF #isdebug = 1 PRINT #yourvar
and then just set the #isdebug to 0 or 1 depending on what you need.
Is there a nicer way to do this?
Nope, that is also what I have in procs
IF #debug = 1
BEGIN
print 'Something'
--or insert into a log table if you need the rows of a temp table
--or the results of a calculation
END
An expansion of this idea is to setup up a controlling string. This giving us more options rather than either on or off.
For example:
Stored procedure parameter declaration
(#Debug varchar(5), -- use bit wise control for debugging, currently 5 levels.
Simple substring to drive a testing/debug block.
if substring(#Debug, 1, 1) = '1'
For a test run you could enter '10001' for #Debug so that "level" 1 debugs show (ie initialisations) and only "level" 5 debugs show for the sub-section of code you are testing/debugging.
You could expand on this idea by using number values (ie 2-9) too. But that might be too over the top.
I've got a stored procedure I use to insert data from a csv. The data itself is a mix of types, some test, some dates, and some money fields. I need to guarantee that this data gets saved, even if it's formatted wrong, so, I'm saving them all to varchars. Later, once the data's been validated and checked off on, it will be moved to another table with proper datatypes.
When I do the insert into the first table, I'd like to do a check that sets a flag (bit column) on the row if it needs attention. For instance, if what should be a money number has letters in it, I need to flag that row and add the column name in an extra errormsg field I've got. I can then use that flag to find and highlight for the users in the interface the fields they need to edit.
The date parameters seem to be easy, I can just use IF ISDATE(#mydate) = '0' to test if that parameter could be converted from varchar to datetime. But, I can't seem to find an ISMONEY(), or anything that's remotely equivalent.
Does anyone know what to call to test if the contents of a varchar can legitimately be converted to money?
EDIT:
I haven't tested it yet, but what do you think of a function like this?:
CREATE FUNCTION CheckIsMoney
(
#chkCol varchar(512)
)
RETURNS bit
AS
BEGIN
-- Declare the return variable here
DECLARE #retVal bit
SET #chkCol = REPLACE(#chkCol, '$', '');
SET #chkCol = REPLACE(#chkCol, ',', '');
IF (ISNUMERIC(#chkCOl + 'e0') = '1')
SET #retVal = '1'
ELSE
SET #retVal = '0'
RETURN #retVal
END
GO
Update
Just finished testing the above code, and it works!
money is decimal in effect, so you test this way
Don't use ISNUMERIC out of the box though: it's unreliable. Use this:
ISNUMERIC(MyCOl + 'e0')
Note, if you have 6 decimal places then it will be lost on conversion to money
Other question with more info why: How to determine the field value which can not convert to (decimal, float,int) in SQL Server
Edit:
Can do it in one line if you want
ISNUMERIC(REPLACE(REPLACE(#chkCOl, '$', ''), ',', '') + 'e0')