In my SQL tables I have text which has hidden characters which is only visible when I copy and paste it in notepad++.
How to find those rows which has hidden characters using SQL Server queries?
I have tried comparing the lengths using datalength and len
it did not work.
DATALENGTH(name) AS BinaryLength != LEN(name)
I want the row which has hidden characters.
On the assumption that this is being caused by control characters. Some of which are invisible. But also include tabs, newlines and spaces. An example to illustrate and how to get them to appear.
--DROP TABLE #SillyTemp
DECLARE #InvisibleChar1 NCHAR(1) = NCHAR(28), #InvisibleChar2 NCHAR(1) = NCHAR(30), #NonControlChar NCHAR(1) = NCHAR(33);
DECLARE #InputString NVARCHAR(500) = N'Some |' + #InvisibleChar1 +'| random string |' + #InvisibleChar2 + '|' + '; Thank god Finally a normal character |' + #NonControlChar + '|';
SELECT #InputString AS OhNoInvisibleCharacters
DECLARE #ControlCharRange NVARCHAR(50) = N'%[' + NCHAR(1) + '-' + NCHAR(31) + ']%';
CREATE TABLE #SillyTemp
(
input nvarchar(500)
)
INSERT INTO #SillyTemp(input)
VALUES (#InputString),(N'A normal string')
SELECT #ControlCharRange;
SELECT input FROM #SillyTemp AS #SI WHERE input LIKE #ControlCharRange;
This produces 3 results. A string with invisiblechars within them like such:
Some || random string ||; Thank god Finally a normal character |!|
Note, the are actually invisible inside SQL. But stackoverflow shows them as such. The output in SQL Server is simply.
Some || random string ||; Thank god Finally a normal character |!|
But these characters still have a corresponding (N)CHAR(X) value. (N)CHAR(0) is a NULL character and is highly unlikely to be in a string, in my setup to detect them it also provides some problems in building a range. (N)CHAR(32) is the ' ' space character.
The way the [X-Y] string operator works is also based on the (N)CHAR numbers. Therefore we can make a range of [NCHAR(1)-NCHAR(31)]
The last select goes through the temporary table, one which has invisible characters. Since we're looking for any NCHARS between 1 and 31, only those with invisible characters (and often invalid characters or tabs/newlines) satisfy the where condition. Thus only they get returned. In this case only the 'faulty' string gets returned in my select statement.
Declare #Temp1 as nvarchar(max) = '[10004120][1100][10033583][1005]'
Declare #Temp2 as nvarchar(max) = '[10004120][1100]'
If #Temp1 like #Temp2 + '%'
Print 'Yup'
Why does this not work? I don't get the "yup" message.
This doesn't work because the brackets in the string have a special function in a LIKE statement - items between the brackets constitute a set of values that the singular character at the specified position matches. Your original pattern looks for a 1,0,4, or 2 followed by a 1 or an 0. To make this work, you should have a pattern like this:
Declare #Temp2 as nvarchar(max) = '[[]10004120][[]1100]'
You have the like backwards. The pattern is the second operand. The logic should be:
#temp1 like #temp2 + '%'
At least in MS Sql Server, the square brackets have meaning in a LIKE.
They are a bit similar to a character class in regex.
For example:
WHERE col LIKE '[0-9]%' will find col values that start with a digit.
WHERE col LIKE '%[^0-9]' will find col values that end with anything but a digit.
So something as a LIKE '[10004120][1100]%' is equivalent to a LIKE '[0-24][01]%' and will actually match with strings like '11X', '40Y'.
But in this case you don't have to use LIKE.
Since the first string just has to start with the second string.
IF #Temp2 = LEFT(#Temp1, LEN(#Temp2))
PRINT 'yup'
And to check if the second string is a part of the first string.
You could use CHARINDEX. Which will return 1 if it's found at the start.
IF CHARINDEX(#Temp2, #Temp1) > 0
BEGIN
PRINT 'yup'
END
Rather than add a specific pattern for an open bracket, I would use the optional ESCAPE clause for LIKE expressions:
Declare #Temp1 as nvarchar(max) = '[10004120][1100][10033583][1005]'
Declare #Temp2 as nvarchar(max) = '[10004120][1100]'
If #Temp1 LIKE REPLACE(#Temp2, '[', '\[') + '%' ESCAPE '\'
Print 'Yup'
To me that's cleaner and more explicit about what you're trying to accomplish. Obviously, you've got to pick an escape character that's guaranteed not to be in your data or otherwise use #Temp1 LIKE REPLACE(REPLACE(#Temp2, '\', '\\'), '[', '\[') + '%' ESCAPE '\' to escape embedded escape characters first.
why does this work,
select*
from some_table
WHERE some_column_name like '%i%'
and not this?
select*
from some_table
WHERE
some_column_name like (select ''''+'%' +value +'%' + '''' as val
from [dbo].[fn_Split](' i this is a test testing Chinese undefined',' ')
where idx = 0)
I am trying to search for individual words instead of the whole phrase, the split function above will split the string on space characters and plug the results into a table with two columns, idx and value.
the LIKE operator takes a string for an argument. It cannot be used on a table, which I assume your function returns.
I think what you want to do is JOIN to the function, and then check where LIKE fn.Value:
select *
from some_table t
INNER JOIN (select value as val
from [dbo].[fn_Split](' i this is a test testing Chinese undefined',' ')
where idx = 0) f
ON t.some_column_name like '%'+f.val+'%'
If your subquery is guaranteed to only return one result, you could try putting the modulo symbols around it instead of inside it:
LIKE '%' + (YourSubQuery) + '%'
One possible reason is because you are appending single quotes onto the beginning and end of the string, and none of the values actually store single quotes in the string.
Another reason is might not work is because the subquery returns more than one row or zero rows. The function fn_split() is your own function, so I don't know what it returns. You have a subquery in a context where it can return at most one row and one column. That is called a scalar subquery. If the subquery returns more than one row, you will get an error. If the subquery returns no rows -- for instance, if idx starts counting at 1 rather than 0 -- then it will return NULL which fails the test.
If you want to find a match this way, I would recommend exists:
select t.*
from some_table t
where exists (select 1 as val
from [dbo].[fn_Split](' i this is a test testing Chinese undefined',' ') s
where s.idx = 0 and
t.some_column_name like '%' + value + '%'
);
The results of your sub-query is a literal string. The % symbol isn't seen as a wildcard. Also, does your functions return multiple rows? If so, LIKE operator can only evaluate a single value.
If your functions does return a single value, I would suggest looking into using Dynamic SQL. Something like the following:
DECLARE #SQL VARCHAR(MAX), #WildCard VARCHAR(MAX)
SELECT #WildCard = '%' + value + '%'
FROM [dbo].[fn_Split](' i this is a test testing Chinese undefined',' ')
WHERE idx = 0
SET #SQL = 'SELECT * FROM some_table WHERE some_column_name like ''' + #WildCardWildCard + ''''
EXEC(#SQL)
I have a vairable
DECLARE #AssignOn nvarchar(20)='0,2,5'
I want to check a condition like this
DECLARE #index int
SET DATEFIRST 7
SELECT #index=DATEPART(DW, GETDATE())-1
IF(CONVERT(nvarchar(2),#index) IN #AssignOn)
IN cannot be used here . Any other methods to do this INLINE
You can use CharIndex to find if you have a match. It returns a non zero value if the first string appears in the second.
IF(CHARINDEX(CONVERT(nvarchar(2),#index), #AssignOn) > 0)
The easiest way to do this is to search for the substring ',needle,' in the csv list string. However, this doesn't work correctly for the first and last elements. This can be overcome by concatenating a comma onto each side of the csv list string.
An example in SQL might be:
SELECT
CHARINDEX(','+ NEEDLE +',', ','+ HAYSTACK +',')
FROM table;
Or using LIKE:
SELECT *
FROM table
WHERE ','+ HAYSTACK +',' LIKE '%,'+ NEEDLE +',';
IF CHARINDEX(','+CONVERT(nvarchar(2),#index)+',', ','+#AssignOn+',') <> 0
As you actually define the values in the code you could instead;
DECLARE #AssignOn TABLE (value int)
INSERT #AssignOn VALUES (0),(2),(5)
... #index IN (SELECT value FROM #AssignOn)
I have a table in a SQL Server database with an NTEXT column. This column may contain data that is enclosed with double quotes. When I query for this column, I want to remove these leading and trailing quotes.
For example:
"this is a test message"
should become
this is a test message
I know of the LTRIM and RTRIM functions but these workl only for spaces. Any suggestions on which functions I can use to achieve this.
I have just tested this code in MS SQL 2008 and validated it.
Remove left-most quote:
UPDATE MyTable
SET FieldName = SUBSTRING(FieldName, 2, LEN(FieldName))
WHERE LEFT(FieldName, 1) = '"'
Remove right-most quote: (Revised to avoid error from implicit type conversion to int)
UPDATE MyTable
SET FieldName = SUBSTRING(FieldName, 1, LEN(FieldName)-1)
WHERE RIGHT(FieldName, 1) = '"'
I thought this is a simpler script if you want to remove all quotes
UPDATE Table_Name
SET col_name = REPLACE(col_name, '"', '')
You can simply use the "Replace" function in SQL Server.
like this ::
select REPLACE('this is a test message','"','')
note: second parameter here is "double quotes" inside two single quotes and third parameter is simply a combination of two single quotes. The idea here is to replace the double quotes with a blank.
Very simple and easy to execute !
My solution is to use the difference in the the column values length compared the same column length but with the double quotes replaced with spaces and trimmed in order to calculate the start and length values as parameters in a SUBSTRING function.
The advantage of doing it this way is that you can remove any leading or trailing character even if it occurs multiple times whilst leaving any characters that are contained within the text.
Here is my answer with some test data:
SELECT
x AS before
,SUBSTRING(x
,LEN(x) - (LEN(LTRIM(REPLACE(x, '"', ' ')) + '|') - 1) + 1 --start_pos
,LEN(LTRIM(REPLACE(x, '"', ' '))) --length
) AS after
FROM
(
SELECT 'test' AS x UNION ALL
SELECT '"' AS x UNION ALL
SELECT '"test' AS x UNION ALL
SELECT 'test"' AS x UNION ALL
SELECT '"test"' AS x UNION ALL
SELECT '""test' AS x UNION ALL
SELECT 'test""' AS x UNION ALL
SELECT '""test""' AS x UNION ALL
SELECT '"te"st"' AS x UNION ALL
SELECT 'te"st' AS x
) a
Which produces the following results:
before after
-----------------
test test
"
"test test
test" test
"test" test
""test test
test"" test
""test"" test
"te"st" te"st
te"st te"st
One thing to note that when getting the length I only need to use LTRIM and not LTRIM and RTRIM combined, this is because the LEN function does not count trailing spaces.
I know this is an older question post, but my daughter came to me with the question, and referenced this page as having possible answers. Given that she's hunting an answer for this, it's a safe assumption others might still be as well.
All are great approaches, and as with everything there's about as many way to skin a cat as there are cats to skin.
If you're looking for a left trim and a right trim of a character or string, and your trailing character/string is uniform in length, here's my suggestion:
SELECT SUBSTRING(ColName,VAR, LEN(ColName)-VAR)
Or in this question...
SELECT SUBSTRING('"this is a test message"',2, LEN('"this is a test message"')-2)
With this, you simply adjust the SUBSTRING starting point (2), and LEN position (-2) to whatever value you need to remove from your string.
It's non-iterative and doesn't require explicit case testing and above all it's inline all of which make for a cleaner execution plan.
The following script removes quotation marks only from around the column value if table is called [Messages] and the column is called [Description].
-- If the content is in the form of "anything" (LIKE '"%"')
-- Then take the whole text without the first and last characters
-- (from the 2nd character and the LEN([Description]) - 2th character)
UPDATE [Messages]
SET [Description] = SUBSTRING([Description], 2, LEN([Description]) - 2)
WHERE [Description] LIKE '"%"'
You can use following query which worked for me-
For updating-
UPDATE table SET colName= REPLACE(LTRIM(RTRIM(REPLACE(colName, '"', ''))), '', '"') WHERE...
For selecting-
SELECT REPLACE(LTRIM(RTRIM(REPLACE(colName, '"', ''))), '', '"') FROM TableName
you could replace the quotes with an empty string...
SELECT AllRemoved = REPLACE(CAST(MyColumn AS varchar(max)), '"', ''),
LeadingAndTrailingRemoved = CASE
WHEN MyTest like '"%"' THEN SUBSTRING(Mytest, 2, LEN(CAST(MyTest AS nvarchar(max)))-2)
ELSE MyTest
END
FROM MyTable
Some UDFs for re-usability.
Left Trimming by character (any number)
CREATE FUNCTION [dbo].[LTRIMCHAR] (#Input NVARCHAR(max), #TrimChar CHAR(1) = ',')
RETURNS NVARCHAR(max)
AS
BEGIN
RETURN REPLACE(REPLACE(LTRIM(REPLACE(REPLACE(#Input,' ','¦'), #TrimChar, ' ')), ' ', #TrimChar),'¦',' ')
END
Right Trimming by character (any number)
CREATE FUNCTION [dbo].[RTRIMCHAR] (#Input NVARCHAR(max), #TrimChar CHAR(1) = ',')
RETURNS NVARCHAR(max)
AS
BEGIN
RETURN REPLACE(REPLACE(RTRIM(REPLACE(REPLACE(#Input,' ','¦'), #TrimChar, ' ')), ' ', #TrimChar),'¦',' ')
END
Note the dummy character '¦' (Alt+0166) cannot be present in the data (you may wish to test your input string, first, if unsure or use a different character).
To remove both quotes you could do this
SUBSTRING(fieldName, 2, lEN(fieldName) - 2)
you can either assign or project the resulting value
You can use TRIM('"' FROM '"this "is" a test"') which returns: this "is" a test
CREATE FUNCTION dbo.TRIM(#String VARCHAR(MAX), #Char varchar(5))
RETURNS VARCHAR(MAX)
BEGIN
RETURN SUBSTRING(#String,PATINDEX('%[^' + #Char + ' ]%',#String)
,(DATALENGTH(#String)+2 - (PATINDEX('%[^' + #Char + ' ]%'
,REVERSE(#String)) + PATINDEX('%[^' + #Char + ' ]%',#String)
)))
END
GO
Select dbo.TRIM('"this is a test message"','"')
Reference : http://raresql.com/2013/05/20/sql-server-trim-how-to-remove-leading-and-trailing-charactersspaces-from-string/
I use this:
UPDATE DataImport
SET PRIO =
CASE WHEN LEN(PRIO) < 2
THEN
(CASE PRIO WHEN '""' THEN '' ELSE PRIO END)
ELSE REPLACE(PRIO, '"' + SUBSTRING(PRIO, 2, LEN(PRIO) - 2) + '"',
SUBSTRING(PRIO, 2, LEN(PRIO) - 2))
END
Try this:
SELECT left(right(cast(SampleText as nVarchar),LEN(cast(sampleText as nVarchar))-1),LEN(cast(sampleText as nVarchar))-2)
FROM TableName