Large range duplicate removal from another sheet - vba

Question

The object is to remove all the rows in sheet1 column A if they exist in the list in sheet2 column A.
Both columns only contain numbers.
Sheet one column A may contain duplicates which is fine if they are not on the list in sheet2.
One option that I'm not familiar with and might be missing out on is Autofilter.
The code executes on a small data range 100 to 1000 but I have many books with over 1,000,000 records to clean up and anything over 10,000 brings Excel to not responding and freezes up indefinitely.
Sub remDupesfromTwoWs()
With Application
.EnableEvents = False
CalcMode = .Calculation
.Calculation = xlCalculationManual
.ScreenUpdating = False
End With
' set range to be searched
Dim masterRecordRange As Range ' declare an unallocated array.
Set masterRecordRange = Range("Sheet1!A2:A316730") ' masterRecordRange is now an allocated array
' store sheet2 column A as searchfor array
Dim unwantedRecords() As Variant ' declare an unallocated array.
unwantedRecords = Range("Sheet2!A1:A282393") ' unwantedRecords is now an allocated array
' foreach masterRecord loop to search masterRecordRange for match in unwantedRecords
Dim i As Double
Dim delRange As Range
Set delRange = Range("A" & ActiveSheet.Rows.Count)
'go through all rows starting at last row
For i = masterRecordRange.Rows.Count To 1 Step -1
' loop through unwantedRecords check each offset
For Each findMe In unwantedRecords
'If StrComp(cell, findMe, 1) = 0 Then not as fast
' unwantedRecord found
If Cells(i, 1).Value = findMe Then
Set delRange = Union(delRange, Range("A" & i))
'MsgBox i
Exit For
End If
Next findMe
Next i
'remove them all in one shot
delRange.EntireRow.Delete
With Application
.EnableEvents = True
CalcMode = .Calculation
.Calculation = xlCalculationAutomatic
.ScreenUpdating = True
End With
'possibly count and display quantity found
MsgBox "finally done!"
End Sub

Answer 1

It is very slow to walk through a range one cell at a time because there is a large overhead on each call to Cells. So you should get both ranges into variant arrays, then compare them to build up another array of matches which you would then write back to the worksheet and use Autofilter to select the rows to delete.
Here is a blog post on various methods of comparing lists:
VBA Comparing lists shootout
The fastest method is to use either a Dictionary or a collection. You should be able to adapt the code to do what you want.

Answer 2

Have you ever tried Range.Find:
Sub TestIt()
Dim ws1 As Worksheet, ws2 As Worksheet
Dim LastRow As Long, DestLast As Long, CurRow As Long
Set ws1 = Sheets("Sheet1")
Set ws2 = Sheets("Sheet2")
LastRow = ws1.Range("A" & Rows.Count).End(xlUp).Row
DestLast = ws2.Range("A" & Rows.Count).End(xlUp).Row
For CurRow = LastRow to 2 Step -1 'Must go backwards because you are deleting rows
If Not ws2.Range("A2:A" & DestLast).Find(ws1.Range("A" & CurRow).Value, LookIn:=xlValues, LookAt:=xlWhole) is Nothing Then
Range("A" & CurRow).EntireRow.Delete xlShiftUp
End If
Next CurRow
End Sub