Ignoring specific letters to find match in SQL Query - sql

Question

I want to query a table for all the values that are on a list on another table to find matches, but I know that some of the values in either table may be typed in incorrectly. One table may have '10Hf7K8' and another table may have '1OHf7K8' but I still want them to match.
Another example, if one table has 'STOP' but I know that in myTable, some of fields may say '5T0P' or 'ST0P' or '5TOP'. I want those to come up as results too. The same thing may occur for '2' and 'Z' if I want 'ZEPT' and '2EPT' to match.
So if I know to account for inconsistencies between '0' and 'O', '5' and 'S' and 'Z' and '2', and knowing that they will be in the same spot, but I do not know where exactly they will be in the word or how many letters the word will have, is it possible to make a query ignoring those letters?
Additional Information: These values are hundreds of serial keys that I have no way of confirming which is correct version between the two tables. I should not have used actual words for my example, these values can be any combination of letters and numbers in any order. There is no distinct pattern that I can hard code.
SOLUTION: Goat CO, Learning, and user3216429's answers contained the solution I needed. I was able to find matching values while keeping the underlying data.

Answer 1

Cleaning data is preferable, but could use nested REPLACE() statements if you can't alter the underlying data:
SELECT *
FROM Table1 a
JOIN Table2 b
ON REPLACE(REPLACE(REPLACE(a.field1,'2','Z'),'5','S'),'0','O') = REPLACE(REPLACE(REPLACE(b.field1,'2','Z'),'5','S'),'0','O')
Cleansing the data could be the same nested replace statement:
ALTER TABLE Table1 ADD cleanfield VARCHAR(25)
UPDATE Table1
SET cleanfield = REPLACE(REPLACE(REPLACE(dirtyfield,'2','Z'),'5','S'),'0','O')
Then you'd be able to join the tables on the clean field.

Answer 2

what you can and should do is to clean your data, replace all these 2,0,5 with Z,O and S.
But if you want to try some other solution, then you can try something like this
select case when
REPLACE(REPLACE(REPLACE('stop','0','o'),'5','s'),'2','Z') = REPLACE(REPLACE(REPLACE('5t0p','0','o'),'5','s'),'2','Z') then 1 else 2 end

Answer 3

Like previously said, if you have time, clean up the data.
If not, SQL SERVER supplies two string functions that might help.
The example is from my blog article. http://craftydba.com/?p=5211
The SOUNDEX() function turns a word into a 4 character value. The DIFFERENCE() function tells you how close two words are.
Your example seems to be one word. You might want to use a calculated column and index it so that the where clause is SARGABLE.
If you are using paragraphs, use a standard split function to turn your text paragraph into words. Use these functions to search the data. However, this will result in a non-SARGABLE expression.
-- Example returns 4, words are very close
select
soundex('Dog') as word_val1,
soundex('Dogs') as word_val2,
difference('Dog', 'Dogs') as how_close
-- Example returns 0, words are very different
select
soundex('Rattle-Snake') as word_val1,
soundex('Mongoose') as word_val2,
difference('Rattle-Snake', 'Mongoose') as how_close
output:
word_val1 word_val2 how_close
--------- --------- -----------
D200 D200 4
word_val1 word_val2 how_close
--------- --------- -----------
R340 M522 0
Last but not least, you an also look into FULL text indexing for speed. This requires some extra overhead (FTI structure and process to update FTI).
http://craftydba.com/?p=1421

Answer 4

select REPLACE(REPLACE( REPLACE([column_name],'O','0'),'Z','2'),'5','S')
from [table_name]

Answer 5

1) To filter out all rows containing all forms of STOP word (STOP, 5TOP, ST0P, 5T0P) you could use following query based on LIKE:
SELECT *
FROM (
SELECT 1, 'CocoJambo' UNION ALL
SELECT 2, '5T0P' UNION ALL
SELECT 3, ' 5TOP ' UNION ALL
SELECT 4, ' ST0P ' UNION ALL
SELECT 5, ' STOP ' UNION ALL
SELECT 6, 'ZTOP'
) x (ID, ColA)
WHERE x.ColA LIKE '%[5S]T[0O]P%';
Output:
ID ColA
----------- ---------
2 5T0P
3 5TOP
4 ST0P
5 STOP
2) Regarding your question:
For every table
first I would try to build a table with all patterns for every word and for every pattern I would store the proper/accurate word,
then I would try to replace every occurrence of pattern with the proper word
After this prepossessing of these two tables I will try to match both tables.
This script will replace only the first occurrence of pattern
SELECT x.*, oa.*,
CASE
WHEN oa.PatIx > 0 THEN STUFF( x.ColA , oa.PatIx , LEN(oa.Word), oa.Word )
ELSE x.ColA
END AS NewColA
FROM (
SELECT 1, 'CocoJambo' UNION ALL
SELECT 2, '5T0P' UNION ALL
SELECT 3, ' 5TOP ' UNION ALL
SELECT 4, ' ST0P ' UNION ALL
SELECT 5, ' STOP jambo jumbo 5TOP bOb ' UNION ALL
SELECT 6, 'ZTOP'
) x (ID, ColA)
OUTER APPLY (
SELECT *
FROM (
SELECT w.WordPattern, w.Word, PATINDEX( w.WordPattern , x.ColA ) AS PatIx
FROM #Words w
) y
WHERE y.PatIx > 0
) oa
Output:
ID ColA WordPattern Word PatIx NewColA
-- ----------------------------- ------------ ---- ----- ----------------------------
1 CocoJambo %b[o0]% bob 8 CocoJambob
2 5T0P %[5S]T[0O]P% STOP 1 STOP
3 5TOP %[5S]T[0O]P% STOP 2 STOP
4 ST0P %[5S]T[0O]P% STOP 3 STOP
5 STOP jambo jumbo 5TOP bOb %[5S]T[0O]P% STOP 4 STOP jambo jumbo 5TOP bOb
5 STOP jambo jumbo 5TOP bOb %b[o0]% bob 12 STOP jambobjumbo 5TOP bOb
6 ZTOP NULL NULL NULL ZTOP
Note: this solution it's just a proof of concept. It needs development.
Or you could try this solution which replaces all wrong words with the proper form:
CREATE TABLE dbo.Words ( Id INT IDENTITY PRIMARY KEY, WordSource NVARCHAR(50) NOT NULL, Word NVARCHAR(50) NOT NULL );
INSERT dbo.Words ( WordSource , Word ) VALUES ( N'5T0P' , N'STOP' );
INSERT dbo.Words ( WordSource , Word ) VALUES ( N'5TOP' , N'STOP' );
INSERT dbo.Words ( WordSource , Word ) VALUES ( N'ST0P' , N'STOP' );
INSERT dbo.Words ( WordSource , Word ) VALUES ( N'b0b' , N'bob' );
INSERT dbo.Words ( WordSource , Word ) VALUES ( N'bOb' , N'bob' );
GO
CREATE FUNCTION dbo.ReplaceWords (#ColA NVARCHAR(4000), #Num INT)
RETURNS TABLE
AS
RETURN
WITH CteRecursive
AS
(
SELECT w.Id, w.WordSource, w.Word, REPLACE(#ColA, w.WordSource, w.Word) AS NewColA
FROM dbo.Words w
WHERE w.Id = 1
UNION ALL
SELECT w.Id, w.WordSource, w.Word, REPLACE(prev.NewColA, w.WordSource, w.Word) AS NewColA
FROM CteRecursive prev INNER JOIN dbo.Words w ON prev.Id + 1 = w.Id
WHERE prev.Id + 1 <= #Num
)
SELECT r.NewColA
FROM CteRecursive r
WHERE r.Id = #Num
GO
-- Testing
SELECT * FROM dbo.ReplaceWords(N' ST0P jambo 5TOP bOb jumbo ', 5) f;
Output
NewColA
----------------------------
STOP jambo STOP bob jumbo
You can use previous function to replace all wrong words within every table and then you can compare both tables:
DECLARE #Num INT;
SET #Num = (SELECT COUNT(*) FROM dbo.Words);
SELECT x.*, rpl.NewColA
FROM (
SELECT 1, N'CocoJambo' UNION ALL
SELECT 2, N'5T0P' UNION ALL
SELECT 3, N' 5TOP ' UNION ALL
SELECT 4, N' ST0P ' UNION ALL
SELECT 5, N' STOP jambo jumbo 5TOP bOb ' UNION ALL
SELECT 6, N'ZTOP' UNION ALL
SELECT 7, N'' UNION ALL
SELECT 8, NULL
) x (ID, ColA)
OUTER APPLY dbo.ReplaceWords(x.ColA, #Num) rpl
Output:
ID ColA NewColA
-- ----------------------------- ----------------------------
1 CocoJambo CocoJambo
2 5T0P STOP
3 5TOP STOP
4 ST0P STOP
5 STOP jambo jumbo 5TOP bOb STOP jambo jumbo STOP bob
6 ZTOP ZTOP
7
8 NULL NULL