UDF with Intersect runs slow - vba

Question

So I am creating a function to replace some manual index/match formulas. Note that this function works, but my problem is with speed. So I have a PivotTable with 6 columns and approx. 200.000 rows. I want this to find the value (and I don't use the pivotfunctions, meaning that this is just a table in pivot format) I found that this runs faster than having it in a regular data table. Both would be imported from a SQL table.
A single piece of this formula runs instantly, but the performance slows down when I have a few hundreds in the same sheet.
So any ideas on how to speed this up?
Function getnum2(ByVal Comp As String, Period As String, Measure As String, Optional BU As String, _
Optional Country As String, Optional Table As String, Optional TableSheet As String) As Double
Dim pTable As PivotTable, wTableSheet As Worksheet
If BU = "" Then
BU = "Group"
End If
If Country = "" Then
Country = "Total"
End If
If TableSheet = "" Then
Set wTableSheet = Worksheets("Data")
Else
Set wTableSheet = Worksheets(TableSheet)
End If
If Table = "" Then
Set pTable = wTableSheet.PivotTables("PivotTable1")
Else
Set pTable = wTableSheet.PivotTables(Table)
End If
'Find match
If Intersect(pTable.PivotFields("Bank").PivotItems(Comp).DataRange.EntireRow, _
pTable.PivotFields("Date").PivotItems(Period).DataRange.EntireRow, _
pTable.PivotFields("Business Unit").PivotItems(BU).DataRange.EntireRow, _
pTable.PivotFields("Country").PivotItems(Country).DataRange.EntireRow, _
pTable.PivotFields("Name").PivotItems(Measure).DataRange) Is Nothing Then
getnum2 = "No match"
ElseIf Intersect(pTable.PivotFields("Bank").PivotItems(Comp).DataRange.EntireRow, _
pTable.PivotFields("Date").PivotItems(Period).DataRange.EntireRow, _
pTable.PivotFields("Business Unit").PivotItems(BU).DataRange.EntireRow, _
pTable.PivotFields("Country").PivotItems(Country).DataRange.EntireRow, _
pTable.PivotFields("Name").PivotItems(Measure).DataRange).Count > 1 Then
getnum2 = "More than 1 match"
Else
getnum2 = Intersect(pTable.PivotFields("Bank").PivotItems(Comp).DataRange.EntireRow, _
pTable.PivotFields("Date").PivotItems(Period).DataRange.EntireRow, _
pTable.PivotFields("Business Unit").PivotItems(BU).DataRange.EntireRow, _
pTable.PivotFields("Country").PivotItems(Country).DataRange.EntireRow, _
pTable.PivotFields("Name").PivotItems(Measure).DataRange)
End If
End Function

Answer 1

Rather than calling the function three times, you could use a variable:
Function getnum2(ByVal Comp As String, Period As String, Measure As String, Optional BU As String, _
Optional Country As String, Optional Table As String, Optional TableSheet As String) As Double
Dim pTable As PivotTable, wTableSheet As Worksheet
Dim rgResult as Range
If BU = "" Then
BU = "Group"
End If
If Country = "" Then
Country = "Total"
End If
If TableSheet = "" Then
Set wTableSheet = Worksheets("Data")
Else
Set wTableSheet = Worksheets(TableSheet)
End If
If Table = "" Then
Set pTable = wTableSheet.PivotTables("PivotTable1")
Else
Set pTable = wTableSheet.PivotTables(Table)
End If
'Find match
Set rgResult = Intersect(pTable.PivotFields("Bank").PivotItems(Comp).DataRange.EntireRow, _
pTable.PivotFields("Date").PivotItems(Period).DataRange.EntireRow, _
pTable.PivotFields("Business Unit").PivotItems(BU).DataRange.EntireRow, _
pTable.PivotFields("Country").PivotItems(Country).DataRange.EntireRow, _
pTable.PivotFields("Name").PivotItems(Measure).DataRange)
if rgResult Is Nothing Then
getnum2 = "No match"
ElseIf rgResult.Count > 1 Then
getnum2 = "More than 1 match"
Else
getnum2 = rgResult.Value
End If
End Function

Answer 2

One very simple way to achieve this is by using two PivotTables.
In PivotTable 1, put all fields but the numeric one you want to
return in the ROWS area, and put the field that you want to return in
the VALUES area with aggregation set to COUNT.
In PivotTable 2, put all fields but the numeric one you want to
return in the ROWS area, and put the field that you want to return in
the VALUES area with aggregation set to SUM or MIN or MAX (It doesn't
matter which).
Then you can use a paramatized GETPIVOTDATA function to check PivotTable 1 to see if the thing you're looking up is unique (i.e. COUNT = 1) and if so, then look up the SUM/MIN/MAX of that item in PivotTable2. Given the item is unique, then the SUM/MIN/MAX is only operating on one number, and so does nothing to it.
Here's how that looks, using simplified data:
I've added conditional formatting to the two Pivots to highlight multiple occurances where we want to return the text 'Multiple Items', and as you can see, the formula that is populating column 6 of the Lookup table is only returning unique items as per your requirements.
Here's the formula, using Table notation as my Lookup range has been turned into an Excel Table:
=IF(GETPIVOTDATA("6",$A$3,"1",[#1],"2",[#2],"3",[#3],"4",[#4],"5",[#5])=1,GETPIVOTDATA("6",$H$3,"1",[#1],"2",[#2],"3",[#3],"4",[#4],"5",[#5]),"Multiple Items")
If I randomise the input cells in the Lookup table, you can see what happens when some items aren't in the PivotTable:
This approach works because the field you want to return is a numeric one, meaning you can add it to the VALUES pane of the Pivot. But you could still use this to return strings, by adding a unique ID to the source data, such as the row number, and putting that in the VALUES field, then retrieving it with the double GETPIVOTDATA lookup and using it to retrieve the associated string in the source data.
Another approach is to simply concatenate your columns into a primary key using a suitable delimiter such as the pipe character | and then use that as your lookup key. If you did a binary search on sorted data, this would be lightning fast. (I discuss this at http://dailydoseofexcel.com/archives/2015/04/23/how-much-faster-is-the-double-vlookup-trick/ ). The down side is that you wouldn't be warned in the event that there were multiple items. But it would be possible to do a second lookup using the match position returned by the first, to see if you get another result, and if so then return "Multiple Items". This would be super-fast.

Answer 3

Here's the fastest way to do this: using Binary Match on a sorted lookup table.
On the left I have 5 columns x 1048575 rows of random numbers between 1 and 10. These have been concatenated in column G to make a non-unique key, and then sorted ascending on that key.
(Because the concatenated key is text, it gets sorted alphabetical from left to right, which is why 1|1|1|1|10 appears between 1|1|1|1|1 and 1|1|1|1|2)
I gave the data in Column G the named range of Concat to simplify the formula. My lookup formula in J2 returns the row number of the lookup item if and only if that item is unique to the dataset. The formula is:
=IF(OR( AND(INDEX(Concat,MATCH(I2,Concat,1))=I2, MATCH(I2,Concat,1)=1), AND(INDEX(Concat,MATCH(I2,Concat,1))=I2,INDEX(Concat,MATCH(I2,Concat,1)-1)<>I2)),MATCH(I2,Concat,1),NA())
This executes in 0.01 milliseconds for one instance, for a lookup table of 1048576 rows. My double GETPIVOTDATA approach above took 6 milliseconds. So there you have it: a complex formula that gives a 600 times efficiency boost.
I can explain the formula later in need, but note that some of the complexity is due to the edge case where you may have a unique item appearing in row 1. If I leave out that edge case, then the formula is as follows:
=IF( AND(INDEX(Concat,MATCH(I3,Concat,1))=I3,INDEX(Concat,MATCH(I3,Concat,1)-1)<>I3),MATCH(I3,Concat,1),NA())