Search and Replace Byte Order Mark In Sql Server - sql

i have 10 table and more than 10000 record that contain 
how can search  and replace this in DB?
since the  equal 0xEF,0xBB,0xBF how can search this?
i use this code
WITH foo(myvarbincolumn) AS
(
SELECT text from BPM_Letters
)
SELECT *
FROM foo
WHERE CONVERT(VARCHAR(max), myvarbincolumn) COLLATE Arabic_CI_AS
LIKE '%' + CONVERT(NVARCHAR(max), CHAR(0xEF)+CHAR(0xBB)+CHAR(0xBF)) + '%'
I found this code in stackoverflow but it s incomplete.
script of BPM_Letters
this code not find any record!
please help me

I wrote a query to find that weird character via the query below:
SELECT cast(LEFT(text,1) AS VARBINARY(MAX)) from BPM_Letters
and the result was 0xFFFE. So I wrote this query and it worked perfectly:
UPDATE BPM_Letters Set text=REPLACE(text,0xFFFE,'');

What about this CTE:
StripBOM AS
(
SELECT CASE
WHEN LEFT(text,3) = 0xEFBBBF
THEN CONVERT(varbinary(max),SUBSTRING(text, 4, LEN(text)))
ELSE text
END AS text
FROM BPM_Letters
)
It should provide you with a new table where all BOM characters have been stripped off.
P.S. This code assumes 'text' field is of type varbinary.

Here's a simpler answer that builds on the other ones:
UPDATE BPM_Letters SET text=substr(text, 4) WHERE left(text, 3) = 0xEFBBBF;
I've tested this, and it works.

Related

What is the best way to return results similar to the input?

In my database I am selecting from a field containing Dillon Brookfield. I tried searching for brookfield with the following statement thinking LIKE would select the data, but it didn't work.
SELECT *
FROM [Reports]
WHERE ([VisitorName] LIKE '%' + #VisitorName + '%'
OR [PlateNumber] = #PlateNumber)
I record people who have been banned from a property along with their plate (if known), but if their name is not put in exactly it doesn't return the value. What would be the best way to return similar results?
This is more like a comment, but there is code inside and the comment only allows one #.
What data type is your #PlateNumber? Is it CHAR or VARCHAR? it can make a difference. Like these tests.
DECLARE #char NCHAR(20) = 'space';
SELECT '%'+#char+ '%',IIF('space' Like '%'+#char+ '%', 1,0);
DECLARE #varchar NVARCHAR(20) = 'space';
SELECT '%'+#varchar+ '%',IIF('space' Like '%'+#varchar+ '%', 1,0);
The first SELECT will give you
'%space %',0. -- there will be 15 blank character between space and %
The second SELECT will give you
'%space%', 1.
Create fulltext index. Search names/addresses and other similar fields with fulltext search - it'll be faster and you can find all names - Dillon Brookfield vs Brookfield Dillon.

T-SQL Substring - Last 3 Characters

Using T-SQL, how would I go about getting the last 3 characters of a varchar column?
So the column text is IDS_ENUM_Change_262147_190 and I need 190
SELECT RIGHT(column, 3)
That's all you need.
You can also do LEFT() in the same way.
Bear in mind if you are using this in a WHERE clause that the RIGHT() can't use any indexes.
You can use either way:
SELECT RIGHT(RTRIM(columnName), 3)
OR
SELECT SUBSTRING(columnName, LEN(columnName)-2, 3)
Because more ways to think about it are always good:
select reverse(substring(reverse(columnName), 1, 3))
declare #newdata varchar(30)
set #newdata='IDS_ENUM_Change_262147_190'
select REVERSE(substring(reverse(#newdata),0,charindex('_',reverse(#newdata))))
=== Explanation ===
I found it easier to read written like this:
SELECT
REVERSE( --4.
SUBSTRING( -- 3.
REVERSE(<field_name>),
0,
CHARINDEX( -- 2.
'<your char of choice>',
REVERSE(<field_name>) -- 1.
)
)
)
FROM
<table_name>
Reverse the text
Look for the first occurrence of a specif char (i.e. first occurrence FROM END of text). Gets the index of this char
Looks at the reversed text again. searches from index 0 to index of your char. This gives the string you are looking for, but in reverse
Reversed the reversed string to give you your desired substring
if you want to specifically find strings which ends with desired characters then this would help you...
select * from tablename where col_name like '%190'

T-SQL Split String Like Clause

I have declare #a varchar(100) = 'abc bcd cde def'. What I need is to select from a table where a column is like 'abc' or 'bcd' or 'cde' or 'def'. I can use a split function and a while to get what I want, but somewhere I saw a smart solution using replace or something similar and I just can't remember it.
I know I can use an xml variable, and parse it that way. However, the value is part of a large procedure, and the best way for me is to use it in string form.
I know I can solve this by building a dynamic sql query, but that is not an option in the domain I'm working in.
Damn, I just can remember the solution. Its a hack, a little dirty trick that do the job.
Anyways, I ll use the code bellow (Im over SQL Server 2008), is it a good idea? I prefer it over the dirty split. Is it more performatic?
declare #w varchar(100) = 'some word'
declare #f xml
set #f = '<word>' + replace(#w, ' ', '</word><word>') + '</word>'
select
template.item.value('.', 'varchar(100)') as word
from #f.nodes('/word') template(item)
Use a function to split the individual items into a table, one record per item. Then you simply join to that table.
insert into #FilterTable (filters)
select Items from dbo.Split(#YourFilterString)
select *
from YourTable yt
join #FilterTable f on f.filters = yt.YourColumn
Of course my example is using equality. It gets more complicated if you truly intend to use "like" with wildcards.
In tsql you can use a pattern col like '[abcd]'
http://msdn.microsoft.com/en-us/library/ms179859.aspx
For matching multiple words (not letter) and without dynamic SQL, you'll have to get the values into a temp table. For a split function try this page http://www.sommarskog.se/arrays-in-sql-2005.html#iterative and look at the List of Strings function iter_charlist_to_table.
Or maybe you are thinking of this little trick Parameterize an SQL IN clause from the SO CEO.
for 4 sections max
WHERE
PARSENAME(REPLACE(#a, ' ', '.'), 1) = 'xxx'
OR
PARSENAME(REPLACE(#a, ' ', '.'), 2) = 'xxx'
OR
PARSENAME(REPLACE(#a, ' ', '.'), 3) = 'xxx'
OR
PARSENAME(REPLACE(#a, ' ', '.'), 4) = 'xxx'

How to check if a string is a uniqueidentifier?

Is there an equivalent to IsDate or IsNumeric for uniqueidentifier (SQL Server)?
Or is there anything equivalent to (C#) TryParse?
Otherwise I'll have to write my own function, but I want to make sure I'm not reinventing the wheel.
The scenario I'm trying to cover is the following:
SELECT something FROM table WHERE IsUniqueidentifier(column) = 1
SQL Server 2012 makes this all much easier with TRY_CONVERT(UNIQUEIDENTIFIER, expression)
SELECT something
FROM your_table
WHERE TRY_CONVERT(UNIQUEIDENTIFIER, your_column) IS NOT NULL;
For prior versions of SQL Server, the existing answers miss a few points that mean they may either not match strings that SQL Server will in fact cast to UNIQUEIDENTIFIER without complaint or may still end up causing invalid cast errors.
SQL Server accepts GUIDs either wrapped in {} or without this.
Additionally it ignores extraneous characters at the end of the string. Both SELECT CAST('{5D944516-98E6-44C5-849F-9C277833C01B}ssssssssss' as uniqueidentifier) and SELECT CAST('5D944516-98E6-44C5-849F-9C277833C01BXXXXXXXXXXXXXXXXXXXXXXXXXXXXX' as uniqueidentifier) succeed for instance.
Under most default collations the LIKE '[a-zA-Z0-9]' will end up matching characters such as À or Ë
Finally if casting rows in a result to uniqueidentifier it is important to put the cast attempt in a case expression as the cast may occur before the rows are filtered by the WHERE.
So (borrowing #r0d30b0y's idea) a slightly more robust version might be
;WITH T(C)
AS (SELECT '5D944516-98E6-44C5-849F-9C277833C01B'
UNION ALL
SELECT '{5D944516-98E6-44C5-849F-9C277833C01B}'
UNION ALL
SELECT '5D944516-98E6-44C5-849F-9C277833C01BXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
UNION ALL
SELECT '{5D944516-98E6-44C5-849F-9C277833C01B}ssssssssss'
UNION ALL
SELECT 'ÀD944516-98E6-44C5-849F-9C277833C01B'
UNION ALL
SELECT 'fish')
SELECT CASE
WHEN C LIKE expression + '%'
OR C LIKE '{' + expression + '}%' THEN CAST(C AS UNIQUEIDENTIFIER)
END
FROM T
CROSS APPLY (SELECT REPLACE('00000000-0000-0000-0000-000000000000', '0', '[0-9a-fA-F]') COLLATE Latin1_General_BIN) C2(expression)
WHERE C LIKE expression + '%'
OR C LIKE '{' + expression + '}%'
Not mine, found this online... thought i'd share.
SELECT 1 WHERE #StringToCompare LIKE
REPLACE('00000000-0000-0000-0000-000000000000', '0', '[0-9a-fA-F]');
SELECT something
FROM table1
WHERE column1 LIKE '[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]-[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]-[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]-[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]-[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]';
UPDATE:
...but I much prefer the approach in the answer by #r0d30b0y:
SELECT something
FROM table1
WHERE column1 LIKE REPLACE('00000000-0000-0000-0000-000000000000', '0', '[0-9a-fA-F]');
I am not aware of anything that you could use "out of the box" - you'll have to write this on your own, I'm afraid.
If you can: try to write this inside a C# library and deploy it into SQL Server as a SQL-CLR assembly - then you could use things like Guid.TryParse() which is certainly much easier to use than anything in T-SQL....
A variant of r0d30b0y answer is to use PATINDEX to find within a string...
PATINDEX('%'+REPLACE('00000000-0000-0000-0000-000000000000', '0', '[0-9a-fA-F]')+'%',#StringToCompare) > 0
Had to use to find Guids within a URL string..
HTH
Dave
Like to keep it simple. A GUID has four - in it even, if is just a string
WHERE column like '%-%-%-%-%'
Though an older post, just a thought for a quick test ...
SELECT [A].[INPUT],
CAST([A].[INPUT] AS [UNIQUEIDENTIFIER])
FROM (
SELECT '5D944516-98E6-44C5-849F-9C277833C01B' Collate Latin1_General_100_BIN AS [INPUT]
UNION ALL
SELECT '{5D944516-98E6-44C5-849F-9C277833C01B}'
UNION ALL
SELECT '5D944516-98E6-44C5-849F-9C277833C01BXXXXXXXXXXXXXXXXXXXXXXXXXXXXX'
UNION ALL
SELECT '{5D944516-98E6-44C5-849F-9C277833C01B}ssssssssss'
UNION ALL
SELECT 'ÀD944516-98E6-44C5-849F-9C277833C01B'
UNION ALL
SELECT 'fish'
) [A]
WHERE PATINDEX('[^0-9A-F-{}]%', [A].[INPUT]) = 0
This is a function based on the concept of some earlier comments. This function is very fast.
CREATE FUNCTION [dbo].[IsGuid] (#input varchar(50))
RETURNS bit AS
BEGIN
RETURN
case when #input like '[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]-[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]-[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]-[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]-[0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F][0-9a-fA-F]'
then 1 else 0 end
END
GO
/*
Usage:
select [dbo].[IsGuid]('123') -- Returns 0
select [dbo].[IsGuid]('ebd8aebd-7ea3-439d-a7bc-e009dee0eae0') -- Returns 1
select * from SomeTable where dbo.IsGuid(TableField) = 0 -- Returns table with all non convertable items!
*/
DECLARE #guid_string nvarchar(256) = 'ACE79678-61D1-46E6-93EC-893AD559CC78'
SELECT
CASE WHEN #guid_string LIKE '________-____-____-____-____________'
THEN CONVERT(uniqueidentifier, #guid_string)
ELSE NULL
END
You can write your own UDF. This is a simple approximation to avoid the use of a SQL-CLR assembly.
CREATE FUNCTION dbo.isuniqueidentifier (#ui varchar(50))
RETURNS bit AS
BEGIN
RETURN case when
substring(#ui,9,1)='-' and
substring(#ui,14,1)='-' and
substring(#ui,19,1)='-' and
substring(#ui,24,1)='-' and
len(#ui) = 36 then 1 else 0 end
END
GO
You can then improve it to check if it´s just about HEX values.
I use :
ISNULL(convert(nvarchar(50), userID), 'NULL') = 'NULL'
I had some Test users that were generated with AutoFixture, which uses GUIDs by default for generated fields. My FirstName fields for the users that I need to delete are GUIDs or uniqueidentifiers. That's how I ended up here.
I was able to cobble together some of your answers into this.
SELECT UserId FROM [Membership].[UserInfo] Where TRY_CONVERT(uniqueidentifier, FirstName) is not null
Use RLIKE for MYSQL
SELECT 1 WHERE #StringToCompare
RLIKE REPLACE('00000000-0000-0000-0000-000000000000', '0', '[0-9a-fA-F]');
In a simplest scenario. When you sure that given string can`t contain 4 '-' signs.
SELECT * FROM City WHERE Name LIKE('%-%-%-%-%')
In BigQuery you can use
SELECT *
FROM table
WHERE
REGEXP_CONTAINS(uuid, REPLACE('^00000000-0000-0000-0000-000000000000$', '0', '[0-9a-fA-F]'))

SQL strip text and convert to integer

In my database (SQL 2005) I have a field which holds a comment but in the comment I have an id and I would like to strip out just the id, and IF possible convert it to an int:
activation successful of id 1010101
The line above is the exact structure of the data in the db field.
And no I don't want to do this in the code of the application, I actually don't want to touch it, just in case you were wondering ;-)
This should do the trick:
SELECT SUBSTRING(column, PATINDEX('%[0-9]%', column), 999)
FROM table
Based on your sample data, this that there is only one occurence of an integer in the string and that it is at the end.
I don't have a means to test it at the moment, but:
select convert(int, substring(fieldName, len('activation successful of id '), len(fieldName) - len('activation successful of id '))) from tableName
Would you be open to writing a bit of code? One option, create a CLR User Defined function, then use Regex. You can find more details here. This will handle complex strings.
If your above line is always formatted as 'activation successful of id #######', with your number at the end of the field, then:
declare #myColumn varchar(100)
set #myColumn = 'activation successful of id 1010102'
SELECT
#myColumn as [OriginalColumn]
, CONVERT(int, REVERSE(LEFT(REVERSE(#myColumn), CHARINDEX(' ', REVERSE(#myColumn))))) as [DesiredColumn]
Will give you:
OriginalColumn DesiredColumn
---------------------------------------- -------------
activation successful of id 1010102 1010102
(1 row(s) affected)
select cast(right(column_name,charindex(' ',reverse(column_name))) as int)
CAST(REVERSE(LEFT(REVERSE(#Test),CHARINDEX(' ',REVERSE(#Test))-1)) AS INTEGER)
-- Test table, you will probably use some query
DECLARE #testTable TABLE(comment VARCHAR(255))
INSERT INTO #testTable(comment)
VALUES ('activation successful of id 1010101')
-- Use Charindex to find "id " then isolate the numeric part
-- Finally check to make sure the number is numeric before converting
SELECT CASE WHEN ISNUMERIC(JUSTNUMBER)=1 THEN CAST(JUSTNUMBER AS INTEGER) ELSE -1 END
FROM (
select right(comment, len(comment) - charindex('id ', comment)-2) as justnumber
from #testtable) TT
I would also add that this approach is more set based and hence more efficient for a bunch of data values. But it is super easy to do it just for one value as a variable. Instead of using the column comment you can use a variable like #chvComment.
If the comment string is EXACTLY like that you can use replace.
select replace(comment_col, 'activation successful of id ', '') as id from ....
It almost certainly won't be though - what about unsuccessful Activations?
You might end up with nested replace statements
select replace(replace(comment_col, 'activation not successful of id ', ''), 'activation successful of id ', '') as id from ....
[sorry can't tell from this edit screen if that's entirely valid sql]
That starts to get messy; you might consider creating a function and putting the replace statements in that.
If this is a one off job, it won't really matter. You could also use a regex, but that's quite slow (and in any case mean you now have 2 problems).