Excel VBA memory usage - vba

Question

I'm trying to copy a large number of lines (20k to 65k) into new workbooks, and for some reason, assigning the value of the range I'm copying uses more memory than using the copy/paste buffer, which doesn't make any sense to me, unless I'm somehow doing this wrong.
This is the original code:
Public Const FIRSTSHEETTAB As String = "Sheet1"
' <snip>
Dim last_row As Long
Dim num_files As Long
Dim ps_rng As Range
' <snip>
Dim i As Long
Dim new_book As Workbook
Dim start_row As Long
Dim end_row
start_row = 2
For i = 1 To num_files
Set new_book = Workbooks.Add
end_row = start_row + max_lines - 1
If end_row > last_row Then
end_row = last_row
End If
With new_book
.Windows(1).Caption = "PS Upload " & i
With .Worksheets(FIRSTSHEETTAB)
.Range("1:1").Value2 = ps_rng.Range("1:1").Value2
.Range("2:" & max_lines).Value2 = ps_rng.Range(CStr(start_row) & ":" & CStr(end_row)).Value2
End With
End With
start_row = end_row + 1
Next i
And what I had to do to get this working was change .Range("2:" & max_lines).Value2 = ps_rng.Range(CStr(start_row) & ":" & CStr(end_row)).Value2 to the following:
ps_rng.Range(CStr(start_row) & ":" & CStr(end_row)).Copy
.Range("2:" & max_lines).PasteSpecial
And I don't understand why this works where as the former code runs out of memory. I'd much rather not have to overwrite whatever is in the copy/paste buffer if I can help it.
What's causing just the simple assignment to run out of memory?

Answer 1

When you use Copy, Excel is smart enough only to copy the used part of the Copy range.
Eg. see below: granted this is looking at the "Text" version of what's on the clipboard, but that's pretty much what you're getting when you PasteSpecial
Sub Tester()
ActiveSheet.Cells.ClearContents
ActiveSheet.UsedRange 'reset sheet
CheckCopy '>> 1
ActiveSheet.Range("A1:J1").Value = "x"
CheckCopy '>> 10
ActiveSheet.Range("XFD1").Value = "x"
CheckCopy '>> 16384
ActiveSheet.Range("XFD1").ClearContents
CheckCopy '>> 16384
ActiveSheet.UsedRange 'reset sheet
CheckCopy '>> 10
End Sub
Sub CheckCopy()
Dim d As New DataObject, s As String
ActiveSheet.Rows(1).Copy
d.GetFromClipboard
s = d.GetText
Debug.Print "#Cols: " & IIf(Len(s) = 0, 0, UBound(Split(s, vbTab)) + 1)
End Sub
You don't get this optimization when you directly assign Value between two large ranges.

Answer 2

Because you are assigning the value of 16.384 x 65.000 = 1.064.960.000 cells simultaneously and this is too much for Excel to cope with.
A better approach would be to restrict the desired range to copy using the last column that does have values. I do not recommend utilizing the UsedRange property because it does give some unwanted results sometimes when a cell far away was edited sometime ago.
Below there is a code example:
Public Const FIRSTSHEETTAB As String = "Sheet1"
' <snip>
Dim last_row As Long
Dim num_files As Long
Dim ps_rng As Range
' <snip>
Dim i As Long
Dim new_book As Workbook
Dim start_row As Long
Dim end_row
start_row = 2
'Obtaining last column of the desired range
lastColumn = ps_rng.Cells(1, ps_rng.Columns.Count).End(xlToLeft).Column
For i = 1 To num_files
Set new_book = Workbooks.Add
end_row = start_row + max_lines - 1
If end_row > last_row Then
end_row = last_row
End If
With new_book
.Windows(1).Caption = "PS Upload " & i
With .Worksheets(FIRSTSHEETTAB)
.Range(.Cells(1, 1), .Cells(1, lastColumn)).Value2 = ps_rng.Range(ps_rng.Cells(1, 1), ps_rng.Cells(1, lastColumn)).Value2
.Range(.Cells(2, 1), .Cells(max_lines, lastColumn)).Value2 = ps_rng.Range(ps_rng.Cells(start_row, 1), ps_rng.Cells(end_row, lastColumn)).Value2
End With
End With
start_row = end_row + 1
Next i