Looping through groups of records - sql

Question

SQL Server 2014, I have a table with a number of rows for example 15, 5 have a groupid column of 736881 and 10 have a group id column 3084235. What I want to do is process each group of records in turn and load the results in to a table.
I have written the code to do this but I think I am not setting the loopcounter incorrectly set as I keep getting the groupid of records 736881 loaded twice.
I cant't currently post the test data due to containing personal information but if the mistake is not obvious I will try and create some dummy data.
SELECT #LoopCounter = min(rowfilter) , #maxrowfilter = max(rowfilter)
FROM peops6
WHILE ( #LoopCounter IS NOT NULL
AND #LoopCounter <= #maxrowfilter)
begin
declare #customer_dist as Table (
[id] [int] NOT NULL,
[First_Name] [varchar](50) NULL,
[Last_Name] [varchar](50) NULL,
[DoB] [date] NULL,
[post_code] [varchar](50) NULL,
[mobile] [varchar](50) NULL,
[Email] [varchar](100) NULL );
INSERT INTO #customer_dist (id, First_Name, Last_Name, DoB, post_code, mobile, Email)
select id, first_name, last_name, dob, postcode, mobile_phone, email from peops6 where rowfilter = #LoopCounter
insert into results
SELECT result.* ,
[dbo].GetPercentageOfTwoStringMatching(result.DoB, d.DoB) [DOB%match] ,
[dbo].GetPercentageOfTwoStringMatching(result.post_code, d.post_code) [post_code%match] ,
[dbo].GetPercentageOfTwoStringMatching(result.mobile, d.mobile) [mobile%match] ,
[dbo].GetPercentageOfTwoStringMatching(result.Email, d.Email) [email%match]
FROM ( SELECT ( SELECT MIN(id)
FROM #customer_dist AS sq
WHERE sq.First_Name = cd.First_Name
AND sq.Last_Name = cd.Last_Name
AND ( sq.DoB = cd.DoB
OR sq.mobile = cd.mobile
OR sq.Email = cd.Email
OR sq.post_code = cd.post_code )) nid ,
*
FROM #customer_dist AS cd ) AS result
INNER JOIN #customer_dist d ON result.nid = d.id order by 1, 2 asc;
SELECT #LoopCounter = min(rowfilter) FROM peops6
WHERE rowfilter > #LoopCounter
end

Answer 1

You need to truncate your table variable (#customer_dist) at the end of the loop:
....
-- Add this
TRUNCATE TABLE #customer_dist
SELECT #LoopCounter = min(rowfilter) FROM peops6
WHERE rowfilter > #LoopCounter
end
See: https://social.msdn.microsoft.com/Forums/sqlserver/en-US/42ef20dc-7ad8-44f7-b676-a4596fc0d593/declaring-a-table-variable-inside-a-loop-does-not-delete-the-previous-data?forum=transactsql

Answer 2

I am not sure you need a LOOP like using a SQL Cursor to fulfill this task
Please check following SQL statement where I used multiple CTE expressions
with customer_dist as (
select
rowfilter,
id, first_name, last_name, dob, postcode, mobile_phone, email
from peops6
), result as (
SELECT
(
SELECT
MIN(id)
FROM customer_dist AS sq
WHERE
sq.rowfilter = cd.rowfilter
AND sq.First_Name = cd.First_Name
AND sq.Last_Name = cd.Last_Name
AND (sq.DoB = cd.DoB OR sq.mobile_phone = cd.mobile_phone OR sq.Email = cd.Email OR sq.postcode = cd.postcode )
) nid,
*
FROM customer_dist AS cd
)
SELECT
result.* ,
[dbo].edit_distance(result.DoB, d.DoB) [DOB%match] ,
[dbo].edit_distance(result.postcode, d.postcode) [post_code%match] ,
[dbo].edit_distance(result.mobile_phone, d.mobile_phone) [mobile%match] ,
[dbo].edit_distance(result.Email, d.Email) [email%match]
FROM result
INNER JOIN customer_dist d
ON result.nid = d.id
order by 1, 2 asc;
Please note, I used my fuzzy string matching Levenshtein Distance Algorithm in this sample instead of your function
And the outcome is as follows
Only you need to add the INSERT statement just before the last SELECT statement
Hope it is useful