Storing/Saving a dictionary even after excel is closed - vba

I am having a dictionary, that holds userkey -> username references. (I am using it to store the user name after looking it up in the windows directory based on the current userkey, because I assume it is a very slow process and want to improve the performance)
If I got it right on my searching, my dictionary gets completely cleared when I reopen the excel file, correct?
So I want to save it to one of the sheets, where I want to recreate it from on the next session. (one column should hold the userkey, the other the name).
My code runs, but does not write any values in the fields:
'will store the values on the rule sheets in row 4 following, columns BA and BB
Sub SaveDictToRulesSheet(dict As Object)
'startrow of list on excel sheet
startrow = 4
Dim i As Integer
i = 0
ActiveSheet.Name = "Rules"
For Each key In dict.Keys
Worksheets("Rules").Cells(startrow + i, "BA").Value = key
Worksheets("Rules").Cells(startrow + i, "BB").Value = dict(key)
i = i + 1
Next key
i = 0
End Sub
Any help is greatly appreciated.

So I want to save it to one of the sheets, where I want to recreate it from on the next session. (one column should hold the userkey, the other the name).
Well that part seems fairly simple. What's a little confusing is where you read in your dict. You refer to it, but it's unclear to me where the values are being loaded in. I'm going to show you how I would do it. Hopefully that helps and I've understood the issue properly.
Write your dictionary columns to a blank / current workbook and save. Then create a new sub that operates something like this:
Sub Retrieve_Dict()
Set wbkCSV = Workbooks.Open("Template.xlsx")
Set wshCSV = wbkCSV.Worksheets("Rules")
Set dict = CreateObject("Scripting.Dictionary")
numrows = application.worksheetfunction.counta(wshCSV.Columns(27)) - 5
numcols = 2
set wshRange = wshCSV.Range("BA5").Resize(numrows,numcols)
tempArray = wshRange.value
for i = 1 to ubound(tempArray) ' Read rows, columns, send to dict.
dict.key(tempArray(i, 1)) = tempArray(i, 2)' read values.
Next i
tempArray = Process(dict) ' Func. updating dictionary values.
wshRange.value = tempArray
wbkCSV.Close (True)
End Sub
Of course, you can make the above sub a function if you instead open the workbook outside, then pass the worksheet. The function could return as an Object / Scripting.Dictionary depending on your binding.
Also, note, I may have gotten the offset / row count wrong. But the general principle should apply, I think.

The code bellow:
TestDictionaryOps() - tests writing and reading from sheet
DictionaryToRange() - writes dictionary to sheet
DictionaryFromRange() - reads dictionary from sheet
Paste it in a new standard module, and run it on a new sheet (Sheet4)
Option Explicit
Public Sub TestDictionaryOps()
Dim d As Dictionary
Set d = New Dictionary
d("1") = "a"
d("2") = "b"
d("3") = "c"
DictionaryToRange d, Sheet4
Set d = DictionaryFromRange(Sheet4)
If Not d Is Nothing Then MsgBox "Total Dictionary items: " & d.Count
End Sub
Public Sub DictionaryToRange(ByRef d As Dictionary, _
ByRef ws As Worksheet, _
Optional ByVal startCol As Long = 1)
If Not d Is Nothing And Not ws Is Nothing And startCol > 0 Then
Dim cnt As Long, rng1 As Range, rng2 As Range
cnt = d.Count
If cnt > 0 Then
Set rng1 = ws.Range(ws.Cells(1, startCol + 0), ws.Cells(cnt, startCol + 0))
Set rng2 = ws.Range(ws.Cells(1, startCol + 1), ws.Cells(cnt, startCol + 1))
rng1 = Application.Transpose(d.Keys) 'write all keys to column 1
rng2 = Application.Transpose(d.Items) 'write all items to column 2
Else
MsgBox "Empty Dictionary"
End If
Else
MsgBox "Missing Dictionary or WorkSheet"
End If
End Sub
Public Function DictionaryFromRange(ByRef ws As Worksheet, _
Optional ByVal startCol As Long = 1) As Dictionary
If Not ws Is Nothing And startCol > 0 Then
Dim d As Dictionary, cnt As Long, vArr As Variant, i As Long
Set d = New Dictionary
cnt = ws.UsedRange.Columns(startCol).Cells.Count
vArr = ws.Range(ws.Cells(1, startCol), ws.Cells(cnt, startCol + 1)).Value2
For i = 1 To cnt
d(vArr(i, startCol)) = vArr(i, startCol + 1)
Next
Set DictionaryFromRange = d
Else
MsgBox "Missing WorkSheet"
End If
End Function
Early binding (fast): VBA Editor -> Tools -> References -> Add Microsoft Scripting Runtime
Late binding (slow): CreateObject("Scripting.Dictionary")

Related

VBA solution for very slow Countif formula

I have a solution that works in Excel with a countif formula (with the help of another Stackoverflow user).
Essentially what this countif formula does is count the first instance of an ID that exclusively exists with the classification type "DC". For example, as you can see in my snippet, 2232 is marked with as it is only exists with the classification "DC". Whilst in the case of 2240 it is marked as 0 as there are multiple classifications possible.
The formula in column D is the following:
=IF(IF(B2<>"DC",0,AND(COUNTIF(C$2:C$28,C2)=COUNTIF(A$2:A$28,A2),COUNTIF(A$2:A2,A2)=1)),1,0)
The problem that I am experiencing is that this is an extremely slow formula to process for Excel -- it takes roughly ~10-15 mins to complete. The database that I am running this on contains of roughly 150k~ lines.
I was wondering if it was possible to do this same process in VBA, but a lot faster and more efficient than the current processing time.
So I am using the following piece of VBA code to try to recreate the same results:
Sub MarkUniqueID()
Dim Ary As Variant, Nary As Variant
Dim r As Long
With ThisWorkbook.Sheets("sheet1")
Ary = .Range("A2", .Range("A" & Rows.Count).End(xlUp)).Value2
End With
ReDim Nary(1 To UBound(Ary), 1 To 1)
With CreateObject("scripting.dictionary")
For r = 1 To UBound(Ary)
If Not .Exists(Ary(r, 1)) Then
.Add Ary(r, 1), Nothing
Nary(r, 1) = 1
Else
Nary(r, 1) = 0
End If
Next r
End With
ThisWorkbook.Sheets("sheet1").Range("E2").Resize(r).Value = Nary
End Sub
Which runs the process much smoother it takes only a few ~seconds of my original time, however, I am not sure how I can add one more criteria into my array (i.e. only exclusively consider "DC"), as now the results are not what I want (see below).
Any pointers would be much appreciated!
You can use another dictionary to track which ID's should be excluded:
Sub MarkUniqueID()
Dim Ary As Variant, Nary() As Long, cls, id, k
Dim r As Long, dictIn As Object, dictOut As Object
Dim ws As Worksheet
Set dictIn = CreateObject("scripting.dictionary")
Set dictOut = CreateObject("scripting.dictionary")
Set ws = ThisWorkbook.Sheets("sheet1")
'pick up the classification and ID
Ary = ws.Range("B2:C" & ws.Cells(ws.Rows.Count, "B").End(xlUp).Row).Value
ReDim Nary(1 To UBound(Ary), 1 To 1)
For r = 1 To UBound(Ary, 1)
cls = Ary(r, 1)
id = CStr(Ary(r, 2))
If cls = "DC" Then
If Not dictIn.exists(id) Then dictIn.Add id, r
Else
If Not dictOut.exists(id) Then dictOut.Add id, True
End If
Next r
For Each k In dictIn
If Not dictOut.exists(k) Then Nary(dictIn(k), 1) = 1
Next k
ws.Range("E2").Resize(UBound(Nary, 1)).Value = Nary
End Sub

Making an Associative Table of Unique Identifiers

I'm trying to create an associative table on a sheet that is pulling in data from a different sheet. By associative I mean, if the data is changed in the source data sheet, it would be reflected on the new sheet. I also want to only have the new sheet's table to be contingent on having a certain unique value. In my case, I want to pull up information related to a part number. The original source data will have many rows that contain the same part number, but I only care to display one of them.
This is what I have so far:
Function IsInArray(stringToBeFound As String, arr As Variant) As Boolean
IsInArray = (UBound(Filter(arr, stringToBeFound)) > -1)
End Function
Dim ref() As Variant
Dim row As Integer
row = 92
Worksheets("Part Tracking Scorecard").Activate
While Cells(row, 6).Value:
If IsInArray(Cells(row, 6).Value, ref) Then
row = row + 1
ElseIf Not IsInArray(Cells(row, 6).Value, ref) Then
ReDim Preserve ref(1 To UBound(ref) + 1) As Variant
ref(UBound(ref)) = Cells(row, 6).Value
Worksheets("Unique Parts").Activate
?????
row = row + 1
To satisfy my condition to only showcase the unique part numbers, I initialized an empty array called "ref". Then, as I iterate through the source sheet, I would check if the part number was in ref with the function "IsInArray". If it was in it, it would move onto the next row, if it wasn't add the part number into the empty array and move to the next row.
The portion with the "????" is where I'm having most of my issue trying to figure out. That part is supposed to be where I make the new table with the date from the unique part number. The very simple and tedious thing I could do is make some loop to run through the columns of the rows and put in a vlookup function. I was wondering if there may be a more robust or more elegant way in doing this.
You've had the right reflex tyring to define an array to stock your values. Here are a few tips of how I would get around to doing it (not perfect, but it should help you out):
Function IsInArray(stringToBeFound As String, arr As Variant) As Boolean
IsInArray = (UBound(Filter(arr, stringToBeFound)) > -1)
End Function
Dim Source as Worksheets
Set Source = Worksheets("Part Tracking Scoreboard")
Dim ref1(), ref2() As Variant
Dim row, index, index2 As Integer
row = 92
ref1 = Source.Range(Worksheets(Source.Cells(row,1), Source.Cells(lastrow, last column))
'Start by placing your ENTIRE source sheet in ref1, if your source sheet is big, this will help you win A LOT of time during the looping phase. Notice how I start from row 92 seeing as this is where you started your loop
'lastrow and lastcolumn represent the position of the last cell in your source file
For index = row to lastrow
If Not IsInArray(ref1(row, 6).Value, ref2) Then
ref2(index) = ref1(index) 'copy the entire row from source to ref2
Next index
Dim NewFile as Worksheet
Set Newfile = Sheets("NewSheetName")
Dim ref2dimension_x, ref2dimension_y as Integer 'find dimensions of ref2 array
ref2dimension_x= UBound(ref2, 1) - LBound(ref2, 1) + 1
ref2dimension_y = UBound(ref2, 2) - LBound(ref2, 2) + 1
For index = 2 to ref2dimension_x 'go through entire new sheet and set values
For index2 = 1 to ref2dimension_y
NewFile.Cells(index, index2).Value = ref2(index - 1, index2)
Next index2
Next index
ref1() = nothing
ref2() = nothing 'free up the space occupied by these arrays
I was not sure about what you were trying to do exactly during the else loop. If you intention is to copy the entire row, this should work. If you want to copy only specific data from the source sheet, you will need to find the indexes of the corresponding columns (hardcode them if they are not going to budge, or use a loop to find them through string comparison otherwise).
This solution combines some macros that I use frequently (so even if you don't use them now, they might be helpful in the future). It won't work if the data in the unique table needs to be "live", but if it'd be sufficient for it to be updated whenever the workbook is opened/closed (or on demand), this is a lot less complicated than the array version.
Basically you just:
Copy the main/unduplicated table to a new sheet
Remove duplicates by part number
Remove unnecessary columns from unduplicated table (if applicable)
I'm assuming that your source data is in a formal Excel Table (ListObject). Just swap out "PartTable" for whatever your actual table is called.
Sub makeUniqueTable()
Application.ScreenUpdating = False
Dim MainWS As Worksheet
Set MainWS = ThisWorkbook.Sheets("Part Tracking Scorecard")
Dim UniqueWS As Worksheet
Set UniqueWS = ThisWorkbook.Sheets("Unique Parts")
UniqueWS.Cells.Clear
Call cloneTable(MainWS.ListObjects("PartTable"), "UniquePartTable", UniqueWS)
Dim UniquePartTable As ListObject
Set UniquePartTable = UniqueWS.ListObjects("UniquePartTable")
Call removeDuplicates(UniquePartTable, "Part Number")
'Optional: remove unnecessary columns by listing columns to be deleted...
'Call deleteColumns(UniquePartTable, Array("Unnecessary Column 1", "Unnecessary Column 2"))
'...or kept:
'Call deleteColumns(UniquePartTable, Array("Part Number", "Manufacturer", "Product Description"), True)
Application.ScreenUpdating = True
End Sub
Sub cloneTable(tbl As ListObject, newName As String, Optional newWS As Worksheet = Nothing)
'Copies a table (tbl) to a new worksheet (newWS) and gives it a name (newName)
'If there is any data in newWS, the new table will be added to the right of the used range
'If newWS is omitted, new table will be added to same worksheet as original table
Dim ws As Worksheet
Dim lastColumn As Long
Dim newRng As Range
Dim newTbl As ListObject
If newWS Is Nothing Then
Set ws = tbl.Parent
lastColumn = ws.Cells(1, ws.Columns.Count).End(xlToLeft).Column
Set newRng = ws.Range(ws.Cells(1, lastColumn + 2), ws.Cells(1 + tbl.ListRows.Count, lastColumn + tbl.ListColumns.Count + 1))
Else
Set ws = newWS
If ws.ListObjects.Count > 0 Then
lastColumn = ws.Cells(1, ws.Columns.Count).End(xlToLeft).Column
Set newRng = ws.Range(ws.Cells(1, lastColumn + 2), ws.Cells(1 + tbl.ListRows.Count, lastColumn + tbl.ListColumns.Count + 1))
Else
Set newRng = ws.Range(ws.Cells(1, 1), ws.Cells(1 + tbl.ListRows.Count, tbl.ListColumns.Count))
End If
End If
tbl.Range.Copy
newRng.PasteSpecial Paste:=xlPasteValuesAndNumberFormats
Application.CutCopyMode = False
Set newTbl = ws.ListObjects.Add(xlSrcRange, newRng, , xlYes)
newTbl.Name = newName
End Sub
Sub removeDuplicates(tbl As ListObject, Optional colName As Variant = "")
'Removes duplicates from a table (tbl) based on column header names (colName()) provided by user
'If no column names are provided, duplicates will be removed based on all columns in table
Dim i As Long
Dim j As Long
If Not IsArray(colName) Then
If colName = "" Then
ReDim colNumArr(0 To tbl.ListColumns.Count - 1) As Variant
For i = 0 To tbl.ListColumns.Count - 1
colNumArr(i) = tbl.ListColumns(i + 1).Range.Column
Next
Else
ReDim colNumArr(0 To 0) As Variant
colNumArr(0) = tbl.ListColumns(colName).Range.Column
End If
Else
ReDim colNumArr(0 To UBound(colName) - LBound(colName)) As Variant
j = 0
For i = LBound(colName) To UBound(colName)
colNumArr(j) = tbl.ListColumns(colName(i)).Range.Column
j = j + 1
Next
End If
tbl.Range.removeDuplicates Columns:=(colNumArr), Header:=xlYes
End Sub
Sub deleteColumns(tbl As ListObject, ByVal colName As Variant, Optional invert As Boolean = False, Optional sheetCol As Boolean = True)
'Deletes column(s) from sheet based on header names (colName) from a table (tbl)
'Will result in error if provided column contains multiple tables
'colName can be a String or an array of Strings
'Inverted mode deletes all columns *except* those in colName
Dim i As Long
Dim j As Long
Dim x As Boolean
If Not IsArray(colName) Then
tempStr = colName
ReDim colName(1 To 1) As String
colName(1) = tempStr
End If
If invert = False Then
For i = LBound(colName) To UBound(colName)
If sheetCol = True Then
tbl.Parent.Columns(tbl.ListColumns(colName(i)).Range.Column).Delete
Else
tbl.ListColumns(colName(i)).Delete
End If
Next
Else
For i = tbl.ListColumns.Count To 1 Step -1
x = False
For j = LBound(colName) To UBound(colName)
If tbl.HeaderRowRange(i).Value = colName(j) Then
x = True
Exit For
End If
Next
If x = False Then
If sheetCol = True Then
tbl.Parent.Columns(tbl.ListColumns(i).Range.Column).Delete
Else
tbl.ListColumns(i).Delete
End If
End If
Next
End If
End Sub

Gather data tidy in Excel using VBA

What the case is:
So I got a "results sample" in excel format that needs filtering and reshaping to look nice. It is a result that will be not identical all the time but it follows similar rules. I have to filter it further and make it a little more tidy. I have figured out the filtering part, but I am not sure how to sort the remaining data, in a tidy way.
What the situation is:
There are six columns involved.
Notice: Real deal is not THAT simple, but what I need can be demonstrated using such a simple example and then I can manage more complex stuff myself I suppose.
For our example we use columns from B to G
The data are set as pairs of a "title" and a value.
For instance, if you look the first example picture I provide, The first detais the pair B3 and C3.
As you can see, looking at the same picture, D3 and E3 is an empty pair.
Same goes for D4 - E4 and F4 - G4 and so on until a last one at B11 - C11.
Starting data example:
[
What I want to achieve:
I would like, using Visual Basic for Applications, to sort the data, starting from let's say for our example B3 (see second picture) and fill three SETS of two columns, (BC, DE, FG) if there are no data inside those cells.
Notice: If a cell like D3 is null then SURELY E3 will be null too so there can be just only one check. I mean we can check either value columns or title columns.
Notice2: The B,D,F or C,E,G columns DON'T have to be sorted. I just want all the not-null values of B,D,F and their respective values from C,E,G gathered together neat so printing will not need 30 pages but just a few (too many spaces between is causing it and I try to automate the cleanup)
Here's something to start with. The first double loop populates a VBA Collection with Range variables that refer to the Cells that contain the titles.
The associated values are obtained by using an offset. The middle double loop performs a bubble sort on the latter (highly inefficient - you might want to replace it with something else). The next if statement creates a 2nd sheet if it doesn't exist on which to write out the results (last loop).
Option Explicit
Sub GatherData()
Dim lastRow As Integer, lastCol As Integer
Dim r As Integer, c As Integer
Dim vals As Collection
Set vals = New Collection
With Sheets(1)
lastCol = .UsedRange.Columns(.UsedRange.Columns.Count).Column
lastRow = .UsedRange.Rows(.UsedRange.Rows.Count).row
For c = 1 To lastCol Step 2
For r = 1 To lastRow
If (Trim(Cells(r, c).Value) <> "") Then
vals.Add .Cells(r, c)
End If
Next
Next
End With
' Bubble Sort
Dim i As Integer, j As Integer
Dim vTemp As Range
For i = 1 To vals.Count - 1
For j = i + 1 To vals.Count
If vals(i).Value > vals(j).Value Then
Set vTemp = vals(j)
vals.Remove j
vals.Add vTemp, vTemp, i
End If
Next j
Next i
Dim sht2 As Worksheet
If ThisWorkbook.Worksheets.Count = 1 Then
Set sht2 = ThisWorkbook.Sheets.Add(After:=ThisWorkbook.Sheets(1))
Else
Set sht2 = Worksheets(2)
End If
With sht2
r = 3
c = 2
For i = 1 To vals.Count
.Cells(r, c).Value = vals(i).Value
.Cells(r, c + 1).Value = vals(i).Offset(, 1).Value
c = c + 2
If c = 8 Then
r = r + 1
c = 2
End If
Next
End With
End Sub
Here is a method using the Dictionary object. I use early binding which requires setting a reference to Microsoft Scripting Runtime. If you are going to be distributing this, you might want to convert this to late-binding.
We assume that your data is properly formed as you show it above. In other words, all the titles are in even numbered columns; and the results are in the adjacent cell.
We create the dictionary using the Title as the Key, and the adjacent cell value for the Dictionary item.
We collect the information
Transfer the Keys to a VBA array and sort alphabetically
create a "Results Array" and populate it in order
write the results to a worksheet.
I will leave formatting and header generation to you.
By the way, there is a constant in the code for the number of Title/Value pair columns. I have set it to 3, but you can vary that.
Enjoy
Option Explicit
Option Compare Text 'If you want the sorting to be case INsensitive
'set reference to Microsoft Scripting Runtime
Sub TidyData()
'Assume Titles are in even numbered columns
'Assume want ColPairs pairs of columns for output
'Use dictionary with Title as key, and Value as the item
Dim dctTidy As Dictionary
Dim arrKeys As Variant
Dim wsSrc As Worksheet, wsRes As Worksheet, rRes As Range
Dim vSrc As Variant, vRes As Variant
Dim LastRow As Long, LastCol As Long
Dim I As Long, J As Long, K As Long, L As Long
Dim V As Variant
'in Results
Const ColPairs As Long = 3
'Set Source and results worksheet and range
Set wsSrc = Worksheets("sheet1")
Set wsRes = Worksheets("sheet2")
Set rRes = wsRes.Cells(1, 2)
'Read source data into variant array
With wsSrc.Cells
LastRow = .Find(what:="*", after:=.Item(1, 1), _
LookIn:=xlValues, searchorder:=xlByRows, searchdirection:=xlPrevious).Row
LastCol = .Find(what:="*", after:=.Item(1, 1), _
LookIn:=xlValues, searchorder:=xlByColumns, searchdirection:=xlPrevious).Column
vSrc = .Range(.Cells(1, 1), .Cells(LastRow, LastCol))
End With
'Collect the data into a dictionary
Set dctTidy = New Dictionary
For I = 1 To UBound(vSrc, 1)
For J = 2 To UBound(vSrc, 2) Step 2
If vSrc(I, J) <> "" Then _
dctTidy.Add Key:=vSrc(I, J), Item:=vSrc(I, J + 1)
Next J
Next I
'For this purpose, we can do a simple sort on the dictionary keys,
' and then create our results array in the sorted order.
arrKeys = dctTidy.Keys
Quick_Sort arrKeys, LBound(arrKeys), UBound(arrKeys)
'Create results array
ReDim vRes(1 To WorksheetFunction.RoundUp(dctTidy.Count / ColPairs, 0), 1 To ColPairs * 2)
I = 0
J = 0
For Each V In arrKeys
K = Int(I / ColPairs) + 1
L = (J Mod ColPairs) * 2 + 1
vRes(K, L) = V
vRes(K, L + 1) = dctTidy(V)
I = I + 1
J = J + 1
Next V
'write the results
Set rRes = rRes.Resize(UBound(vRes, 1), UBound(vRes, 2))
With rRes
.Worksheet.Cells.Clear
.Value = vRes
.HorizontalAlignment = xlCenter
End With
End Sub
Sub Quick_Sort(ByRef SortArray As Variant, ByVal first As Long, ByVal last As Long)
Dim Low As Long, High As Long
Dim Temp As Variant, List_Separator As Variant
Low = first
High = last
List_Separator = SortArray((first + last) / 2)
Do
Do While (SortArray(Low) < List_Separator)
Low = Low + 1
Loop
Do While (SortArray(High) > List_Separator)
High = High - 1
Loop
If (Low <= High) Then
Temp = SortArray(Low)
SortArray(Low) = SortArray(High)
SortArray(High) = Temp
Low = Low + 1
High = High - 1
End If
Loop While (Low <= High)
If (first < High) Then Quick_Sort SortArray, first, High
If (Low < last) Then Quick_Sort SortArray, Low, last
End Sub
Assuming we got all variables set and initialized properly, in this example:
Sheets("sheetname").Select ' because stupid things can happen...
For i = 3 To 13
Let newrangeT = "B" & i '
Let newrangeV = "C" & i '
If Sheets("sheetname").Range(newrangeV) <> "" Then
values(Position) = Sheets("sheetname").Range(newrangeV)
titles(Position) = Sheets("sheetname").Range(newrangeT)
Position = Position + 1
Else
' Don't do anything if the fields are null
End If
Next i
Sheets("sheetname").Range("B1:G13").Clear
' We then get each data from the arrays with a For loop.
' We set a columnset variable to 1.
' We set a currentrow variable to 3.
' If columnset is 1 data will enter in B and C and columnset = columnset +1
' Then if columnset is 2 we set data to DE and columnset = columnset +1
' But if columnset is 2we set data to FG and columnset = 1 and currentrow = currentrow +1
' Iterating the arrays will result in a neat setting of the data, but it will add zeros for all the nulls. Thus we need an If statement that will exclude that values checking the TITLE array (that should contain a title instead). if the value is not 0 then... we run what I describe, otherwise we do nothing.
Putting the data in the array is half of the trick.
Then we clear the area.
We set two string variables to declare ranges (actually cell reference) for every cell iterated in the loop. Here I demonstrated only for column set B,C
but we have to do the same for the rest of the columns.
The If statement here checks for null. You might have different needs, so changing the if statement changes the filtering. Here I check if the cells are not null. If the cells of column C contain data, put those data in values array and the respective B data on titles array but where? Position starts as 1 and we then iterate it +1 each time it adds something.
You can set data from an array using this command:
' current_row is set to the first row of the spreadsheet we wanna fill.
Sheets("sheetname").Select ' because stupid things can happen...
newrangeV = "C" & current_row
Sheets("sheetname").Range(newrangeV) = values(j)
The rest is just putting things together.
In any case, I wanna thank both of the people involved in this question, because I might didn't got the solution, but I got an idea of how to do other stuff, like accidentally learning something new. Cheers.

Transfer Data by matching Header

I have a Backend raw output data set, which consists of multiple columns with some of them being empty except of the header.
I would like to transfer this data into another worksheet, let's call it Backend - processed. In this worksheet, I would prepare a header row, which consists of some of the headers included in the original data set. There won't be any new headers in the processed worksheet (so basically headers(processed) is a subset of headers(raw output)).
Once, I used to solve this problem with a function (Index & Match), but with growing raw data sets, this became suboptimal from a performance perspective.
Since then, I've been reading up on VBA codes and this is what I came up with until now:
Sub test()
Dim r As Range, c As Range, msg As String
With Sheets("Backend - raw").Range("4:4").CurrentRegion
For Each r In Sheets("Backend - processed").Range("b7:t7")
Set c = .Rows(1).Find(r.Value, , , xlWhole, , 0)
If Not c Is Nothing Then
.Columns(c.Column).Copy
r.PasteSpecial xlPasteValues
Else
msg = msg & vbLf & r.Value
End If
Next
Application.CutCopyMode = False
End With
End Sub
The Range 4:4 is where the headers of the original raw data output are found. Range b7:t7 is where the headers of the processed data table are found.
Being a total beginner at VBA, I'm quite happy that it works, but still think that there is a huge margin for improvement:
1) It's still pretty slow, taking about 10 seconds to complete 40x500 arrays.
2) I don't know how to make it stop looking for the next header, if the last header was blank (end of range b7:t7)
3) I'm very much open to new/better approaches to tackle this issue.
Searching, copying and pasting can be quite time consuming affairs. You'd probably be better off reading the headers just once into some kind of stored list (a Collection would work well for you because it could store the column number as its value and the header text as its key.
Given that you are only copying and pasting values (ie you don't need to pass cell formatting into your processed sheet) then reading the values into an array and then writing that array to the sheet will be quicker.
The code below is an example of that, but I'm sure with more thought it could be made even quicker (for example by discarding a header from the collection once it's been used, or not having to find the last row number for every individual column).
Dim rawSht As Worksheet
Dim procSht As Worksheet
Dim headers As Collection
Dim c As Integer
Dim v As Variant
Set rawSht = ThisWorkbook.Worksheets("Backend - raw")
Set procSht = ThisWorkbook.Worksheets("Backend - processed")
Set headers = New Collection
For c = 1 To rawSht.Cells(4, Columns.Count).End(xlToLeft).Column
headers.Add c, rawSht.Cells(4, c).Text
Next
For c = 2 To 20
rawCol = headers(procSht.Cells(7, c).Text)
v = rawSht.Range(rawSht.Cells(5, rawCol), rawSht.Cells(Rows.Count, rawCol).End(xlUp)).Value2
procSht.Cells(8, c).Resize(UBound(v, 1)).Value = v
Next
This is using arrays (40 cols x 1000 rows in 0.03125 sec)
Option Explicit
Sub testArr()
Const HDR1 As Long = 4 'header row on sheet 1
Const HDR2 As Long = 7 'header row on sheet 2
Dim ws1 As Worksheet, ur1 As Range, vr1 As Variant, c1 As Long, c2 As Long, r As Long
Dim ws2 As Worksheet, ur2 As Range, vr2 As Variant, msg As String, t As Double
t = Timer
Set ws1 = Worksheets("Backend - raw")
Set ws2 = Worksheets("Backend - processed")
Set ur1 = ws1.UsedRange
Set ur2 = ws2.UsedRange.Rows(ws2.UsedRange.Row - HDR2 + 1)
Set ur2 = ur2.Resize(ur1.Row + ur1.Rows.Count - HDR1 + 1)
vr1 = ur1 'copy from Range to array
vr2 = ur2
For c1 = 1 To UBound(vr1, 2)
For c2 = 1 To UBound(vr2, 2)
If vr1(1, c1) = vr2(1, c2) Then
For r = 2 To UBound(vr1, 1)
vr2(r, c2) = vr1(r, c1)
Next
Exit For
Else
msg = msg & vbLf & vr1(HDR1, c1)
End If
Next
Next
ur2 = vr2 'copy from array back to Range
Debug.Print "testArr duration: " & Timer - t & " sec"
End Sub

How to compare two columns in different sheets

I have one excel file with multiple sheets.
I need to compare two sheets (1) TotalList and (2) cList with more than 25 columns, in these two sheets columns are same.
On cList the starting row is 3
On TotalList the starting row is 5
Now, I have to compare the E & F columns from cList, with TotalList E & F columns, if it is not found then add the entire row at the end of TotalList sheet and highlight with Yellow.
Public Function compare()
Dim LoopRang As Range
Dim FoundRang As Range
Dim ColNam
Dim TotRows As Long
LeaData = "Shhet2"
ConsolData = "Sheet1"
TotRows = Worksheets(LeaData).Range("D65536").End(xlUp).Row
TotRows1 = Worksheets(ConsolData).Range("D65536").End(xlUp).Row
'TotRows = ThisWorkbook.Sheets(LeaData).UsedRange.Rows.Count
ColNam = "$F$3:$F" & TotRows
ColNam1 = "$F$5:$F" & TotRows1
For Each LoopRang In Sheets(LeaData).Range(ColNam)
Set FoundRang = Sheets(ConsolData).Range(ColNam1).Find(LoopRang, lookat:=xlWhole)
For Each FoundRang In Sheets(ConsolData).Range(ColNam1)
If FoundRang & FoundRang.Offset(0, -1) <> LoopRang & LoopRang.Offset(0, -1) Then
TotRows = Worksheets(ConsolData).Range("D65536").End(xlUp).Row
ThisWorkbook.Worksheets(LeaData).Rows(LoopRang.Row).Copy ThisWorkbook.Worksheets(ConsolData).Rows(TotRows + 1)
ThisWorkbook.Worksheets(ConsolData).Rows(TotRows + 1).Interior.Color = vbYellow
GoTo NextLine
End If
Next FoundRang
NextLine:
Next LoopRang
End Function
Please help with the VBA code.
Thanks in advance...
First I am going to give some general coding hints:
set Option Explicit ON. This is done through Tools > Options >
Editor (tab) > Require Variable Declaration . Now you HAVE to
declare all variables before you use them.
always declare a variables type when you declare it. If you are unsure about what to sue or if it can take different types (not advisable!!) use Variable.
Use a standard naming convention for all your variables. Mine is a string starts with str and a double with dbl a range with r, etc.. So strTest, dblProfit and rOriginal. Also give your variables MEANINGFUL names!
Give your Excel spreadsheets meanigful names or captions (caption is what you see in excel, name is the name you can directly refer to in VBA). Avoid using the caption, but refer to the name instead, as users can change the caption easily but the name only if they open the VBA window.
Ok so here is how a comparison between two tables can be done with your code as starting point:
Option Explicit
Public Function Compare()
Dim rOriginal As Range 'row records in the lookup sheet (cList = Sheet2)
Dim rFind As Range 'row record in the target sheet (TotalList = Sheet1)
Dim rTableOriginal As Range 'row records in the lookup sheet (cList = Sheet2)
Dim rTableFind As Range 'row record in the target sheet (TotalList = Sheet1)
Dim shOriginal As Worksheet
Dim shFind As Worksheet
Dim booFound As Boolean
'Initiate all used objects and variables
Set shOriginal = ThisWorkbook.Sheets("Sheet2")
Set shFind = ThisWorkbook.Sheets("Sheet1")
Set rTableOriginal = shOriginal.Range(shOriginal.Rows(3), shOriginal.Rows(shOriginal.Rows.Count).End(xlUp))
Set rTableFind = shFind.Range(shFind.Rows(5), shFind.Rows(shFind.Rows.Count).End(xlUp))
booFound = False
For Each rOriginal In rTableOriginal.Rows
booFound = False
For Each rFind In rTableFind.Rows
'Check if the E and F column contain the same information
If rOriginal.Cells(1, 5) = rFind.Cells(1, 5) And rOriginal.Cells(1, 6) = rFind.Cells(1, 6) Then
'The record is found so we can search for the next one
booFound = True
GoTo FindNextOriginal 'Alternatively use Exit For
End If
Next rFind
'In case the code is extended I always use a boolean and an If statement to make sure we cannot
'by accident end up in this copy-paste-apply_yellow part!!
If Not booFound Then
'If not found then copy form the Original sheet ...
rOriginal.Copy
'... paste on the Find sheet and apply the Yellow interior color
With rTableFind.Rows(rTableFind.Rows.Count + 1)
.PasteSpecial
.Interior.Color = vbYellow
End With
'Extend the range so we add another record at the bottom again
Set rTableFind = shFind.Range(rTableFind, rTableFind.Rows(rTableFind.Rows.Count + 1))
End If
FindNextOriginal:
Next rOriginal
End Function