I researched this and found that a text column in SQL Server can store a lot more than 8000 characters. But when I run the following insert in the text column, it only inserts 8000 characters:
UPDATE a
SET [File] = b.Header + CHAR(13) + CHAR(10) + d.Detail + c.Trailer + CHAR(13) + CHAR(10) + CHAR(26)
FROM Summary a
JOIN #Header b ON b.SummaryId = a.SummaryId
JOIN #Trailer c ON c.SummaryId = a.SummaryId
JOIN #Detail d ON d.SummaryId = a.SummaryId
WHERE
a.SummaryId = #SummaryId
I am trying to generate a fixed width flat file and every row should be 3900 characters long, and they are in the respective temp tables. But when I do the insert in the permanent table, the Trailer data gets truncated.
I am adding char(10) + char(13) to add carriage return and line feed and char(26) for end of file, and it seems like they are adding characters to the fixed width layout.
According to http://msdn.microsoft.com/en-us/library/ms187993.aspx TEXT fields are deprecated. Use VARCHAR(MAX) fields instead. They should support 2GB in text.
The problem with your code is not the data type of the field that you store the value in, it's the type of the value that you put together to store in it.
The type of b.Header is not text but varchar, which is used as type for the whole expression. When the strings are concatenated, the result will be truncated to fit in a varchar value.
If you cast the first string to text, the whole expression gets that type, and can become longer than 8000 characters:
SET [File] = cast(b.Header as text) + CHAR(13) + CHAR(10) + d.Detail + c.Trailer + CHAR(13) + CHAR(10) + CHAR(26)
Naturally you should transition into using the new type varchar(max) instead of text, but that is not the reason for your problem.
Your source fields aren't VARCHAR(MAX), so there is an 8000 character limit when concatenating them together, you can fix this by casting the first source field in the concat list as VARCHAR(MAX):
UPDATE a
SET [File] = CAST(b.Header AS VARCHAR(MAX)) + CHAR(13) + CHAR(10) + d.Detail + c.Trailer + CHAR(13) + CHAR(10) + CHAR(26)
FROM Summary a
JOIN #Header b ON b.SummaryId = a.SummaryId
JOIN #Trailer c ON c.SummaryId = a.SummaryId
JOIN #Detail d ON d.SummaryId = a.SummaryId
WHERE a.SummaryId = #SummaryId
If you concat a thousand VARCHAR(25) fields together, the length of the resulting string would be 8000, as that's the limit of the VARCHAR() type when supplied a numeric length. VARCHAR(MAX) does not share this limit, but the concat list inherets the type of the first string supplied. It's an interesting behavior, but that's how it works.
TEXT is deprecated - don't use it! Use VARCHAR(MAX) instead!
I think you need to explicitly cast all columns that you use in your UPDATE statement to VARCHAR(MAX) in order for this to work:
UPDATE a
SET [File] = CAST(b.Header AS VARCHAR(MAX)) + CHAR(13) + CHAR(10) +
CAST(d.Detail AS VARCHAR(MAX)) + CAST(c.Trailer AS VARCHAR(MAX))) + CHAR(13) + CHAR(10) + CHAR(26)
FROM Summary a
JOIN #Header b ON b.SummaryId = a.SummaryId
JOIN #Trailer c ON c.SummaryId = a.SummaryId
JOIN #Detail d ON d.SummaryId = a.SummaryId
WHERE
a.SummaryId = #SummaryId
Related
I am trying to add spaces in specific places of my string if it's < 143(the normal length of a row Regist) , if it's less of 143 then take the difference and add that quantity of ' ' Spaces. The update doesn't work, when I execute, it appears like affect rows but when i look for the LEN of the row that change it doesn't do anything. I am using this:
UPDATE TABLE
SET Regist = (SUBSTRING(Regist,1,(57 - (143 - LEN(Regist)))) +
REPLICATE(' ',(143 - LEN(Regist))) + SUBSTRING(Regist,(58 - (143 - LEN(Regist))),(LEN(Regist) - (58 - (143 - LEN(Regist))))))
WHERE
(LEN(Regist) - 143) < 0
AND Name = 'SQL SERVER V2008'
Example:
[Name]
Excel
--This row has length 142 (including the spaces after v2008)
[Regist]
ABCDEF12345678910111213411121341SQL SERVER V2008 A1111111111111 1111111111111111111111111111111111111111111111111111111111111111111111
So, i did that update to add a space (Just 1 in this case 143 - 142) in the place 56 (because i know that the name finish in place 57 but the row length is less 143, then 57 - (143-142 ) = 56 and that is the position of where is going to be the space.
LEN won't be a good tell of this. It doesn't count trailing blank space.
Returns the number of characters of the specified string expression, excluding trailing blanks.
Documentation
You could do something like:
SELECT LEN(Regist + 'x') - 1
Which would then count your blanks. (Note, you should do this for your length checks in your update too. Otherwise you may be adding way more blank space then you intend, especially if you run multiple times.)
Edit:
Try this:
DECLARE #SEARCHSTRING VARCHAR(100) = 'SQL SERVER V2008'
SELECT REGIST,
LEFT(REGIST,CHARINDEX(#SEARCHSTRING,REGIST,1) + LEN(#SEARCHSTRING) -1) + REPLICATE(' ',143 - LEN(REGIST)) + RIGHT(REGIST,143 - LEN(LEFT(REGIST,CHARINDEX(#SEARCHSTRING,REGIST,1) + LEN(#SEARCHSTRING) -1)) - (143 - LEN(REGIST)))
FROM yourTable
WHERE Name = #SearchString
AND LEN(REGIST) < 143
--UPDATE yourTable
--SET Regist = LEFT(REGIST,CHARINDEX(#SEARCHSTRING,REGIST,1) + LEN(#SEARCHSTRING) -1) + REPLICATE(' ',143 - LEN(REGIST)) + RIGHT(REGIST,143 - LEN(LEFT(REGIST,CHARINDEX(#SEARCHSTRING,REGIST,1) + LEN(#SEARCHSTRING) -1)) - (143 - LEN(REGIST)))
--WHERE Name = #SEARCHSTRING --Or whatever conditions you want
--AND LEN(Regist) < 143
Here is a breakdown you can run to see what's happening:
DECLARE #REGIST VARCHAR(200) = 'ABCDEF12345678910111213411121341SQL SERVER V2008 A1111111111111 1111111111111111111111111111111111111111111111111111111111111111111111'
DECLARE #REGISTLENGTH INT = LEN(#REGIST)
DECLARE #NUMSPACES INT = 143 - LEN(#REGIST)
DECLARE #SEARCHSTRING VARCHAR(100) = 'SQL SERVER V2008'
DECLARE #LOCATION INT = CHARINDEX(#SEARCHSTRING,#REGIST,1) + LEN(#SEARCHSTRING) -1
DECLARE #LEFTPART VARCHAR(200) = LEFT(#REGIST,#LOCATION)
DECLARE #RIGHTPART VARCHAR(200) = RIGHT(#REGIST,143 - LEN(#LEFTPART) - #NUMSPACES)
DECLARE #NEWREGIST VARCHAR(200) = #LEFTPART + REPLICATE(' ',#NUMSPACES) + #RIGHTPART
DECLARE #NEWREGISTLENGTH INT = LEN(#NEWREGIST)
PRINT 'Original string: ' + #REGIST
PRINT 'Original length: ' + CAST(#REGISTLENGTH AS VARCHAR(10))
PRINT 'Spaces to add: ' + CAST(#NUMSPACES AS VARCHAR(10))
PRINT 'Searchstring: ' + #SEARCHSTRING
PRINT 'Left of searchstring: ' + #LEFTPART
PRINT 'Right of searchstring: ' + #RIGHTPART
PRINT 'New string: ' + #NEWREGIST
PRINT 'New string length: ' + CAST(#NEWREGISTLENGTH AS VARCHAR(10))
If you take the final #NEWREGIST line and break it's parts back down, you'll be left with the update above.
The main goal here was is to make the select/update more scale-able. You can change #SEARCHSTRING to anything you want and this will add the correct number of spaces after that string.
Try it like this...
UPDATE yt SET
yt.Regist = LEFT(yt.Regist + REPLICATE(' ', 143), 143)
FROM
dbo.YourTable yt
WHERE
yt.name = 'Excel'
AND LEN(yt.Regist) < 143;
I have a large table of data where some of my columns contain line breaks. I would like to remove them and replace them with some spaces instead.
Can anybody tell me how to do this in SQL Server?
Thanks in advance
SELECT REPLACE(REPLACE(#str, CHAR(13), ''), CHAR(10), '')
This should work, depending on how the line breaks are encoded:
update t
set col = replace(col, '
', ' ')
where col like '%
%';
That is, in SQL Server, a string can contain a new line character.
#Gordon's answer should work, but in case you're not sure how your line breaks are encoded, you can use the ascii function to return the character value. For example:
declare #entry varchar(50) =
'Before break
after break'
declare #max int = len(#entry)
; with CTE as (
select 1 as id
, substring(#entry, 1, 1) as chrctr
, ascii(substring(#entry, 1, 1)) as code
union all
select id + 1
, substring(#entry, ID + 1, 1)
, ascii(substring(#entry, ID + 1, 1))
from CTE
where ID <= #max)
select chrctr, code from cte
print replace(replace(#entry, char(13) , ' '), char(10) , ' ')
Depending where your text is coming from, there are different encodings for a line break. In my test string I put the most common.
First I replace all CHAR(10) (Line feed) with CHAR(13) (Carriage return), then all doubled CRs to one CR and finally all CRs to the wanted replace (you want a blank, I put a dot for better visability:
Attention: Switch the output to "text", otherwise you wont see any linebreaks...
DECLARE #text VARCHAR(100)='test single 10' + CHAR(10) + 'test 13 and 10' + CHAR(13) + CHAR(10) + 'test single 13' + CHAR(13) + 'end of test';
SELECT #text
DECLARE #ReplChar CHAR='.';
SELECT REPLACE(REPLACE(REPLACE(#text,CHAR(10),CHAR(13)),CHAR(13)+CHAR(13),CHAR(13)),CHAR(13),#ReplChar);
I have the same issue, means I have a column having values with line breaks in it. I use the query
update `your_table_name` set your_column_name = REPLACE(your_column_name,'\n','')
And this resolves my issue :)
Basically '\n' is the character for Enter key or line break and in this query, I have replaced it with no space (which I want)
Keep Learning :)
zain
I'm dumbfounded that this question has not been asked meaningfully already. How does one go about creating an equivalent function in SQL like LTRIM or RTRIM for carriage returns and line feeds ONLY at the start or end of a string.
Obviously REPLACE(REPLACE(#MyString,char(10),''),char(13),'') removes ALL carriage returns and new line feeds. Which is NOT what I'm looking for. I just want to remove leading or trailing ones.
Find the first character that is not CHAR(13) or CHAR(10) and subtract its position from the string's length.
LTRIM()
SELECT RIGHT(#MyString,LEN(#MyString)-PATINDEX('%[^'+CHAR(13)+CHAR(10)+']%',#MyString)+1)
RTRIM()
SELECT LEFT(#MyString,LEN(#MyString)-PATINDEX('%[^'+CHAR(13)+CHAR(10)+']%',REVERSE(#MyString))+1)
Following functions are enhanced types of trim functions you can use. Copied from sqlauthority.com
These functions remove trailing spaces, leading spaces, white space, tabs, carriage returns, line feeds etc.
Trim Left
CREATE FUNCTION dbo.LTrimX(#str VARCHAR(MAX)) RETURNS VARCHAR(MAX)
AS
BEGIN
DECLARE #trimchars VARCHAR(10)
SET #trimchars = CHAR(9)+CHAR(10)+CHAR(13)+CHAR(32)
IF #str LIKE '[' + #trimchars + ']%' SET #str = SUBSTRING(#str, PATINDEX('%[^' + #trimchars + ']%', #str), LEN(#str))
RETURN #str
END
Trim Right
CREATE FUNCTION dbo.RTrimX(#str VARCHAR(MAX)) RETURNS VARCHAR(MAX)
AS
BEGIN
DECLARE #trimchars VARCHAR(10)
SET #trimchars = CHAR(9)+CHAR(10)+CHAR(13)+CHAR(32)
IF #str LIKE '%[' + #trimchars + ']'
SET #str = REVERSE(dbo.LTrimX(REVERSE(#str)))
RETURN #str
END
Trim both Left and Right
CREATE FUNCTION dbo.TrimX(#str VARCHAR(MAX)) RETURNS VARCHAR(MAX)
AS
BEGIN
RETURN dbo.LTrimX(dbo.RTrimX(#str))
END
Using function
SELECT dbo.TRIMX(#MyString)
If you do use these functions you might also consider changing from varchar to nvarchar to support more encodings.
In SQL Server 2017 you can use the TRIM function to remove specific characters from beginning and end, in one go:
WITH testdata(str) AS (
SELECT CHAR(13) + CHAR(10) + ' test ' + CHAR(13) + CHAR(10)
)
SELECT
str,
TRIM(CHAR(13) + CHAR(10) + CHAR(9) + ' ' FROM str) AS [trim cr/lf/tab/space],
TRIM(CHAR(13) + CHAR(10) FROM str) AS [trim cr/lf],
TRIM(' ' FROM str) AS [trim space]
FROM testdata
Result:
Note that the last example (trim space) does nothing as expected since the spaces are in the middle.
Here's an example you may run:
I decided to cast the results as an Xml value, so when you click on it, you will be able to view the Carriage Returns.
DECLARE #CRLF Char(2) = (CHAR(0x0D) + CHAR(0x0A))
DECLARE #String VarChar(MAX) = #CRLF + #CRLF + ' Hello' + #CRLF + 'World ' + #CRLF + #CRLF
--Unmodified String:
SELECT CAST(#String as Xml)[Unmodified]
--Remove Trailing Whitespace (including Spaces).
SELECT CAST(LEFT(#String, LEN(REPLACE(#String, #CRLF, ' '))) as Xml)[RemoveTrailingWhitespace]
--Remove Leading Whitespace (including Spaces).
SELECT CAST(RIGHT(#String, LEN(REVERSE(REPLACE(#String, #CRLF, ' ')))) as Xml)[RemoveLeadingWhitespace]
--Remove Leading & Trailing Whitespace (including Spaces).
SELECT CAST(SUBSTRING(#String, LEN(REPLACE(#String, ' ', '_')) - LEN(REVERSE(REPLACE(#String, #CRLF, ' '))) + 1, LEN(LTRIM(RTRIM(REPLACE(#String, #CRLF, ' '))))) as Xml)[RemoveAllWhitespace]
--Remove Only Leading and Trailing CR/LF's (while still preserving all other Whitespace - including Spaces). - 04/06/2016 - MCR.
SELECT CAST(SUBSTRING(#String, PATINDEX('%[^'+CHAR(13)+CHAR(10)+']%',#String), LEN(REPLACE(#String, ' ', '_')) - PATINDEX('%[^'+CHAR(13)+CHAR(10)+']%',#String) + 1 - PATINDEX('%[^'+CHAR(13)+CHAR(10)+']%', REVERSE(#String)) + 1) as Xml)[RemoveLeadingAndTrailingCRLFsOnly]
Remember to remove the Cast-to-Xml, as this was done just as a Proof-of-Concept to show it works.
How is this better than the currently Accepted Answer?
At first glance this may appear to use more Functions than the Accepted Answer.
However, this is not the case.
If you combine both approaches listed in the Accepted Answer (to remove both Trailing and Leading whitespace), you will either have to make two passes updating the Record, or copy all of one Logic into the other (everywhere #String is listed), which would cause way more function calls and become even more difficult to read.
I was stuck using Microsoft SQL Server 2008 R2 and so basing my functions on #sqluser's answer I came up with the below. This will return an empty string if the string only contains the characters to be trimmed.
The bit that threw me was the pattern for PATINDEX must be included between % characters, which for a while I was thinking of as the same wildcard in a LIKE statement but which I now believe is just the syntax to denote a pattern, though I may be wrong!
CREATE FUNCTION [dbo].[ExtendedLTRIM](#string_to_trim VARCHAR(MAX))
RETURNS VARCHAR(MAX)
AS
BEGIN
DECLARE #tab CHAR(1) = CHAR(9);
DECLARE #line_feed CHAR(1) = CHAR(10);
DECLARE #carriage_return CHAR(1) = CHAR(13);
DECLARE #space CHAR(1) = CHAR(32);
DECLARE #characters_to_trim VARCHAR(10)
SET #characters_to_trim = #tab + #line_feed + #carriage_return + #space
IF #string_to_trim LIKE '[' + #characters_to_trim + ']%'
BEGIN
DECLARE #first_non_trim_character INT = PATINDEX('%[^' + #characters_to_trim + ']%', #string_to_trim);
IF #first_non_trim_character = 0 RETURN '';
RETURN SUBSTRING(#string_to_trim, #first_non_trim_character, 8000)
END
RETURN #string_to_trim
END
GO
To trim characters from a pre-defined list you'll want to create the following UDF (should work in 2008R2 and above).
Handles both sides in a single pass and doesn't care if it's a CRLF, LFCR (yep, seen that abomination more than once), bare LF or a bunch of spaces.
is easy to extend to e.g. add additional parameters to do LTRIM/RTRIM only, or a full purge (that last bit is simpler to do in 2017 by incorporating STRING_AGG, but perfectly doable in 2008R2); as a matter of fact this is a simplified version of something I use to do all those things. If anybody is interested then let me know and I'll update:
CREATE FUNCTION fnTrimHarder
(
#String VARCHAR(MAX)
)
RETURNS VARCHAR(MAX)
AS
BEGIN
DECLARE
#Start INT,
#Len INT,
#Chars CHAR(5) = CONCAT(
CHAR(9), -- TAB
CHAR(10), -- LF
CHAR(13), -- CR
' '
), -- List of invalid characters
#Return VARCHAR(MAX) = '';
IF #String NOT LIKE '%[^' + #Chars + ']%' -- If string contains only invalid characters
OR COALESCE(#String, '') = '' -- Optional addition for NULL handling
RETURN #Return
ELSE
BEGIN -- Create a "table" of characters with ordinals, calculate the start of string and its length, then return the substring
WITH CTE AS (
SELECT 1 AS n
UNION ALL
SELECT n + 1
FROM CTE
WHERE n < LEN(#String)
)
SELECT
#Start = MIN(n),
#Len = 1 + MAX(n) - MIN(n)
FROM CTE
WHERE SUBSTRING(#String, n, 1) NOT LIKE '[' + #Chars + ']';
SET #Return = SUBSTRING(#String, #Start, #Len)
END
RETURN #Return
END
GO
Below is part of a SQL triggered that uses a subquery to return a value in the column TableString. The subquery will return the first two value and the "marker" for Fname, however nothing after that (not even the semicolon). The TableString column is an nvarchar that is set to 255 characters and captures all similar data during an insert or an update.
INSERT INTO TransactionLog (TransactionDate, Operator, TableName, Action
, TableString, UserId)
SELECT LastChangeDate
, 'Op'
, #tableName
, #action
, CAST('sNum:' + CAST(sNumber as nvarchar(10)) + ' entType:' + EntityType
+ ' Fname:' + ISNULL(FirstName, 'NULL')
+ ' Lname:' + ISNULL(LastName, 'NULL')
+ ' suff:' + ISNULL(NameSuffix, 'NULL')
+ ' corpName:' + ISNULL(CorporateName, 'NULL' )
+ ' ctrlId:' + ISNULL(CAST(ControlId as nvarchar(3)), 'NULL')
AS nvarchar(30)) as TableString
, LastChangeOperator
FROM deleted
Returned values in TableString:
sNum:1000024 entType:S Fname
This has nothing to do with ISNULL but rather, is the result of a truncation of data.
Pay attention to:
.. AS nvarchar(30)
That is, much more data is being omitted. (The output does not even contain "Fname:" in this case.)
As a side note, his return character count is 28 because Nvarchar(30) means, store 30 bytes and Unicode (N) is two bytes a letter, even if you are storing non-Unicode. The number in the varchar declare is not the number of characters, its the number of bytes to use. It really gets people when they use Nvarchar.
I have a table (locations) that has a field called Point (geometry). I wrote a query that passes the top and bottom latitude coordinates and the bottom and top longitude coordinates. I want to retrieve all records that are within the area of the coordinates I pass the stored procedure. When I run this it returns zero records even though I know there is a record that matches the criteria. Any ideas what I might be doing wrong?
DECLARE #categoryid AS int,#leftlong AS float,#rightlong AS float,#toplat AS float,#bottomlat AS float
DECLARE #searcharea geometry, #polygon AS varchar(500);
SET #leftlong = -85.605469
SET #toplat = 42.303468
SET #rightlong = -85.594912
SET #bottomlat = 42.297564
SET #polygon = CAST(#leftlong AS varchar(20)) + ' ' + CAST(#toplat AS varchar(20)) + ',' +
CAST(#leftlong AS varchar(20)) + ' ' + cast(#bottomlat AS varchar(20)) + ',' +
cast(#rightlong AS varchar(20)) + ' ' + cast(#bottomlat AS varchar(20)) + ',' +
cast(#rightlong AS varchar(20)) + ' ' + cast(#toplat AS varchar(20)) + ',' +
CAST(#leftlong AS varchar(20)) + ' ' + CAST(#toplat AS varchar(20))
SET #searcharea = geometry::STGeomFromText('POLYGON ((' + #polygon + '))', 0);
SELECT *
FROM locations l
WHERE l.point.STWithin(#searcharea) = 1
There are two likely potential sources of the problem:
Usage of geometry instead of geography type. When working with latitude and longitude data, you generally want to use the geography type in order to do calculations on the ellipsoid instead of on the plane. These calculations can different considerably - this is a lot of information available on this distinction in various articles such as this whitepaper.
Mismatching SRIDs - is the srid of the data in your locations table also 0 for all rows? If these do not match between your data and your #searcharea, no results will be returned.