SQL replace statement too slow - sql

Question

I have a replace statement that does something like this:
SELECT Distinct Forenames, Surname, dbUSNs.DateOfBirth, Datasetname,
dbUSNs.MoPIGrade, SourceAddress, VRM, URNs
FROM Person
WHERE ( Replace(Replace(Replace(Replace(Replace(Replace(Replace
(Replace(Replace(Replace(Replace(Replace(Replace(Replace
(Replace(Replace(Replace(Replace(Replace(Replace(Replace
(Replace(Replace(Replace(Replace
(Surname,'/',''''),'?',''''),'',''''),'^',''''),'{',''''),'}',''''),
'[',''''),']',''''),';',''''),'$',''''),'=',''''),'*',''''),
'#',''''),'|',''''),'&',''''),'#',''''),'\',''''),'<',''''),
'>',''''),'(',''''),')',''''),'+',''''),',',''''),'.',''''),
' ','''') LIKE 'OREILLY%')
Therefore even though OReilly is passed, O'Reilly will be found. However, this is too slow. Is there a better way of approaching it?

Answer 1

The problem isn't that REPLACE is "too slow", but that using it at all makes that part of the query unsargable, meaning that it can't use an index.
Wikipedia: Sargable
Basically you've forced a tablescan / indexscan, from top to bottom. On top of that you have the overhead of REPLACE.
If you want this query to run fast, I would instead do one of the following:
Create an additional column containing a searchable text version of the Surname
Create an indexed, materialized view with those REPLACE functions

Answer 2

If you want to simply remove all special characters it's easier to specify the valid characters and use a function to perform the cleansing.
This shows you how to clean the string to alphanumeric characters and spaces '%[^a-z0-9 ]%'
DECLARE #Temp nvarchar(max) ='O''Rielly la/.das.d,as/.d,a/.da.sdo23eu89038 !£$$'
SELECT #Temp
DECLARE #KeepValues AS VARCHAR(50) = '%[^a-z0-9 ]%'
WHILE PatIndex(#KeepValues, #Temp) > 0
SET #Temp = Stuff(#Temp, PatIndex(#KeepValues, #Temp), 1, '')
SELECT #Temp
Which would return: ORielly ladasdasdadasdo23eu89038
So you can write a function:
CREATE FUNCTION [dbo].[RemoveNonAlphaCharacters](#Temp VARCHAR(1000))
RETURNS VARCHAR(1000)
AS
BEGIN
DECLARE #KeepValues AS VARCHAR(50) = '%[^a-z0-9 ]%'
WHILE PatIndex(#KeepValues, #Temp) > 0
SET #Temp = Stuff(#Temp, PatIndex(#KeepValues, #Temp), 1, '')
RETURN #Temp
END
Then simply call it like so:
SELECT *
FROM Person
WHERE [dbo].[RemoveNonAlphaCharacters](Surname) LIKE 'OREILLY%'
If you don't want spaces, just change it to: '%[^a-z0-9]%'

Answer 3

Try this:
Create a function to split:
create function [dbo].[Split](#String varchar(8000), #Delimiter char(1))
returns #temptable TABLE (items varchar(8000))
as
begin
declare #idx int
declare #slice varchar(8000)
select #idx = 1
if len(#String)<1 or #String is null return
while #idx!= 0
begin
set #idx = charindex(#Delimiter,#String)
if #idx!=0
set #slice = left(#String,#idx - 1)
else
set #slice = #String
if(len(#slice)>0)
insert into #temptable(Items) values(#slice)
set #String = right(#String,len(#String) - #idx)
if len(#String) = 0 break
end
return
end
Use this in you where clause:
WHERE ((REPLACE(Surname, items, '') FROM dbo.Split('/,?,^,{,},[,],;,$,=,*,#,|,&,#,\,<,>,(,),+,.')) LIKE 'OREILLY%')

Answer 4

General approach - yes.
Make another field (NameNormalized)
Run a trigger that sets the field whenever the name is updated.
Then you can run a search on that field (which can have an index).
Bascially the whole replace orgy makes the whole thing non-indexable, so the better approach is to store the normalized value and allow fast lookups.
Oh, and evaluate whether the distinct is needed - that is another hugh slowdown.

Answer 5

SELECT Distinct Forenames, Surname, dbUSNs.DateOfBirth, Datasetname,
dbUSNs.MoPIGrade, SourceAddress, VRM, URNs
FROM Person
WHERE Surname LIKE 'O[/?^{}[];$=*#|#\<>()+.]R[/?^{}[];$=*#|#\<>()+.]E[/?^{}[];$=*#|#\<>()+.]I[/?^{}[];$=*#|#\<>()+.]L[/?^{}[];$=*#|#\<>()+.]L[/?^{}[];$=*#|#\<>()+.]Y%')

Answer 6

if you want to remove all special characters just using SUB STRING and While
DECLARE #str VARCHAR(100),#Len INT,#Pos INT = 1,#char char(1),#results varchar(100)
SET #str = 'O''Rielly la/.das.d,as/.d,a/.da.sdo23eu89038 !£$$'
SET #Len = LEN(#str)
Set #results = ''
WHILE #Pos < #Len
BEGIN
SET #char = SUBSTRING(#str,#Pos,1)
IF #char like '[a-z0-9]' or #char = ' '
BEGIN
SET #results = #results + #char
END
SET #Pos = #Pos + 1
END
select #results