Optimise excel VBA code - combine resident address - vba

I have done the following 2 VBA code in excel. Main purpose is to combine multiple address rows into a single line. Problem is it takes forever to run. Is there anyway I can optimise it?
The data is as such, there is a case# for each of the customer address. The customer address can be split into multiple rows. Example: "Address row 1 - Block 56", "Address row 2 - Parry Avenue", "address row 3 - Postal code". There is a blank space between each new address.
My purpose is to combine the address into a single line, and remove the empty rows in between the case numbers eg "Block 56 Parry Avenue Postal code". There are approx 26K case numbers.
Sub test()
Dim l As Long
Dim lEnd As Long
Dim wks As Worksheet
Dim temp As String
Application.EnableEvents = False
Application.ScreenUpdating = False
Set wks = Sheets("data")
wks.Activate
lEnd = ActiveSheet.UsedRange.Rows.Count
For l = 3 To lEnd
If Not IsEmpty(Cells(l, 1)) Then
Do Until IsEmpty(Cells(l + 1, 4))
temp = Cells(l, 4).Value & " " & Cells(l + 1, 4).Value
Cells(l, 4).Value = temp
Cells(l + 1, 4).EntireRow.Delete
Loop
Else: Cells(l, 1).EntireRow.Delete
Do Until IsEmpty(Cells(l + 1, 4))
temp = Cells(l, 4).Value & " " & Cells(l + 1, 4).Value
Cells(l, 4).Value = temp
Cells(l + 1, 4).EntireRow.Delete
Loop
End If
Next l
End Sub
and the 2nd code I tried
Sub transformdata()
'
Dim temp As String
Application.ScreenUpdating = False
Application.Calculation = xlCalculationManual
Range("A3").Select
Do Until IsEmpty(ActiveCell) And IsEmpty(ActiveCell.Offset(1, 0))
Do Until IsEmpty(ActiveCell.Offset(1, 3))
temp = ActiveCell.Offset(, 3).Value & " " & ActiveCell.Offset(1, 3).Value
ActiveCell.Offset(, 3).Value = temp
ActiveCell.Offset(1, 3).EntireRow.Delete
Loop
ActiveCell.Offset(1, 0).EntireRow.Delete
ActiveCell.Offset(1, 0).Select
Loop
Application.ScreenUpdating = True
Application.Calculation = xlCalculationAutomatic
End Sub

Change the line lEnd = ActiveSheet.UsedRange.Rows.Count. Incorrect way of finding last row. You may want to see This
To delete rows where Cells(l, 1) is empty, use Autofilter. See This
Do not delete rows in a straight loop. Use a reverse loop. Or what you could do is identify the cells that you want to delete in a loop and then delete them in one go after the loop. You may want to see This
Here is a basic example.
Let's say your worksheet looks like this
If you run this code
Sub test()
Dim wks As Worksheet
Dim lRow As Long, i As Long
Dim temp As String
Application.ScreenUpdating = False
Set wks = Sheets("data")
With wks
'~~> Find Last Row
lRow = .Range("C" & .Rows.Count).End(xlUp).Row
For i = lRow To 2 Step -1
If Len(Trim(.Range("C" & i).Value)) <> 0 Then
If temp = "" Then
temp = .Range("C" & i).Value
Else
temp = .Range("C" & i).Value & "," & temp
End If
Else
.Range("D" & i + 1).Value = temp
temp = ""
End If
Next i
End With
End Sub
You will get this output
Now simply run the autofilter to delete the rows where Col D is empty :) I have already give you the link above for the same.

The code below will copy all the data into an array, consolidate it, and add it to a new worksheet. You'll need to make COLUMNCOUNT = the number of columns that contain data.
Sub TransformData2()
Const COLUMNCOUNT = 4
Dim SourceData, NewData
Dim count As Long, x1 As Long, x2 As Long, y As Long
SourceData = Range("A" & Range("D" & Rows.count).End(xlUp).Row, Cells(3, COLUMNCOUNT))
For x1 = 1 To UBound(SourceData, 1)
count = count + 1
If count = 1 Then
ReDim NewData(1 To 4, 1 To count)
Else
ReDim Preserve NewData(1 To 4, 1 To count)
End If
For y = 1 To UBound(SourceData, 2)
NewData(y, count) = SourceData(x1, y)
Next
x2 = x1 + 1
Do
NewData(4, count) = NewData(4, count) & " " & SourceData(x2, 4)
x2 = x2 + 1
If x2 > UBound(SourceData, 1) Then Exit Do
Loop Until IsEmpty(SourceData(x2, 4))
x1 = x2
Next
ThisWorkbook.Worksheets.Add
Range("A1").Resize(UBound(NewData, 2), UBound(NewData, 1)).Value = WorksheetFunction.Transpose(NewData)
End Sub

Related

Speed Up Matching program in Excel VBA

I am writing a VBA code on excel using loops to go through 10000+ lines.
Here is an example of the table
And here is the code I wrote :
Sub Find_Matches()
Dim wb As Workbook
Dim xrow As Long
Set wb = ActiveWorkbook
wb.Worksheets("Data").Activate
tCnt = Sheets("Data").UsedRange.Rows.Count
Dim e, f, a, j, h As Range
xrow = 2
Application.ScreenUpdating = False
Application.Calculation = xlManual
For xrow = 2 To tCnt Step 1
Set e = Range("E" & xrow)
Set f = e.Offset(0, 1)
Set a = e.Offset(0, -4)
Set j = e.Offset(0, 5)
Set h = e.Offset(0, 3)
For Each Cell In Range("E2:E" & tCnt)
If Cell.Value = e.Value Then
If Cell.Offset(0, 1).Value = f.Value Then
If Cell.Offset(0, -4).Value = a.Value Then
If Cell.Offset(0, 5).Value = j.Value Then
If Cell.Offset(0, 3).Value = h.Value Then
If (e.Offset(0, 7).Value) + (Cell.Offset(0, 7).Value) = 0 Then
Cell.EntireRow.Interior.Color = vbYellow
e.EntireRow.Interior.Color = vbYellow
End If
End If
End If
End If
End If
End If
Next
Next
End Sub
As you can imagine, this is taking a lot of time to go through 10000+ lines and I would like to find a faster solution. There must be a method I don't think to avoid the over looping
Here are the condition :
For each line, if another line anywhere in the file has the exact same
:
Buyer ID (col. E)
`# purchased (col. F)
Product ID (col.A)
Payment (col. J)
Date purchased (col. H)
Then, if the SUM of the Amount (col. L) the those two matching line is
0, then color both rows in yellow.
Note that extra columns are present and not being compared (eg- col. B) but are still important for the document and cannot be deleted to ease the process.
Running the previous code, in my example, row 2 & 5 get highlighted :
This is using nested dictionaries and arrays to check all conditions
Timer with my test data: Rows: 100,001; Dupes: 70,000 - Time: 14.217 sec
Option Explicit
Public Sub FindMatches()
Const E = 5, F = 6, A = 1, J = 10, H = 8, L = 12
Dim ur As Range, x As Variant, ub As Long, d As Object, found As Object
Set ur = ThisWorkbook.Worksheets("Data").UsedRange
x = ur
Set d = CreateObject("Scripting.Dictionary")
Set found = CreateObject("Scripting.Dictionary")
Dim r As Long, rId As String, itm As Variant, dupeRows As Object
For r = ur.Row To ur.Rows.Count
rId = x(r, E) & x(r, F) & x(r, A) & x(r, J) & x(r, H)
If Not d.Exists(rId) Then
Set dupeRows = CreateObject("Scripting.Dictionary")
dupeRows(r) = 0
Set d(rId) = dupeRows
Else
For Each itm In d(rId)
If x(r, L) + x(itm, L) = 0 Then
found(r) = 0
found(itm) = 0
End If
Next
End If
Next
Application.ScreenUpdating = False
For Each itm In found
ur.Range("A" & itm).EntireRow.Interior.Color = vbYellow
Next
Application.ScreenUpdating = True
End Sub
Before
After
I suggest a different approach altogether: add a temporary column to your data that contains a concatenation of each cell in the row. This way, you have:
A|B|C|D|E
1|Mr. Smith|500|A|1Mr. Smith500A
Then use Excel's conditional formatting on the temporary column, highlighting duplicate values. There you have your duplicated rows. Now it's only a matter of using a filter to check which ones have amounts equal to zero.
You can use the CONCATENATE function; it requires you to specify each cell separately and you can't use a range, but in your case (comparing only some of the columns) it seems like a good fit.
Maciej's answer is easy to implement (if you can add columns to your data without interrupting anything), and I would recommend it if possible.
However, for the sake of answering your question, I will contribute a VBA solution as well. I tested it on dataset that is a bit smaller than yours, but I think it will work for you. Note that you might have to tweak it a little (which row you start on, table name, etc) to fit your workbook.
Most notably, the segment commented with "Helper column" is something you most likely will have to adjust - currently, it compares every cell between A and H for the current row, which is something you may or may not want.
I've tried to include a little commentary in the code, but it's not much. The primary change is that I'm using in-memory processing of an array rather than iterating over a worksheet range (which for larger datasets should be exponentially faster).
Option Base 1
Option Explicit
' Uses ref Microsoft Scripting Runtime
Sub Find_Matches()
Dim wb As Workbook, ws As Worksheet
Dim xrow As Long, tCnt As Long
Dim e As Range, f As Range, a As Range, j As Range, h As Range
Dim sheetArr() As Variant, arr() As Variant
Dim colorTheseYellow As New Dictionary, colorResults() As String, dictItem As Variant
Dim arrSize As Long, i As Long, k As Long
Dim c As Variant
Set wb = ThisWorkbook
Set ws = wb.Worksheets("Data")
ws.Activate
tCnt = ws.UsedRange.Rows.Count
xrow = 2
Application.ScreenUpdating = False
Application.Calculation = xlManual
' Read range into an array so we process in-memory
sheetArr = ws.Range("A2:H" & tCnt)
arrSize = UBound(sheetArr, 1)
' Build new arr with "helper column"
ReDim arr(1 To arrSize, 1 To 9)
For i = 1 To arrSize
For k = 1 To 8
arr(i, k) = sheetArr(i, k)
arr(i, 9) = CStr(arr(i, 9)) & CStr(arr(i, k)) ' "Helper column"
Next k
Next i
' Iterate over array & build collection to indicate yellow lines
For i = LBound(arr, 1) To UBound(arr, 1)
If Not colorTheseYellow.Exists(i) Then colorResults = Split(ReturnLines(arr(i, 9), arr), ";")
For Each c In colorResults
If Not colorTheseYellow.Exists(CLng(c)) Then colorTheseYellow.Add CLng(c), CLng(c)
Next c
Next i
' Enact row colors
For Each dictItem In colorTheseYellow
'Debug.Print "dict: "; dictItem
If dictItem <> 0 Then ws.ListObjects(1).ListRows(CLng(dictItem)).Range.Interior.Color = vbYellow
Next dictItem
End Sub
Function ReturnLines(ByVal s As String, ByRef arr() As Variant) As String
' Returns a "Index;Index" string indicating the index/indices where the second, third, etc. instance(s) of s was found
' Returns "0;0" if 1 or fewer matches
Dim i As Long
Dim j As Long
Dim tmp As String
ReturnLines = 0
j = 0
tmp = "0"
'Debug.Print "arg: " & s
For i = LBound(arr, 1) To UBound(arr, 1)
If arr(i, 9) = s Then
j = j + 1
'Debug.Print "arr: " & arr(i, 9)
'Debug.Print "ReturnLine: " & i
tmp = tmp & ";" & CStr(i)
End If
Next i
'If Left(tmp, 1) = ";" Then tmp = Mid(tmp, 2, Len(tmp) - 1)
'Debug.Print "tmp: " & tmp
If j >= 2 Then
ReturnLines = tmp
Else
ReturnLines = "0;0"
End If
End Function
On my simple dataset, it yields this result (marked excellently with freehand-drawn color indicators):
Thanks everybody for your answers,
Paul Bica's solution actually worked and I am using a version of this code now.
But, just to animate the debate, I think I also found another way around my first code, inspired by Maciej's idea of concatenating the cells and using CStr to compare the values and, of course Vegard's in-memory processing by using arrays instead of going through the workbook :
Sub Find_MatchesStr()
Dim AmountArr(300) As Variant
Dim rowArr(300) As Variant
Dim ws As Worksheet
Dim wb As Workbook
Set ws = ThisWorkbook.Sheets("Data")
ws.Activate
Range("A1").Select
rCnt = ws.Cells.SpecialCells(xlCellTypeLastCell).Row
For i = 2 To rCnt
If i = rCnt Then
Exit For
Else
intCnt = 0
strA = ws.Cells(i, 1).Value
strE = ws.Cells(i, 5).Value
strF = ws.Cells(i, 6).Value
strH = ws.Cells(i, 8).Value
strL = ws.Cells(i, 10).Value
For j = i To rCnt - 1
strSearchA = ws.Cells(j, 1).Value
strSearchE = ws.Cells(j, 5).Value
strSearchF = ws.Cells(j, 6).Value
strSearchH = ws.Cells(j, 8).Value
strSearchL = ws.Cells(j, 10).Value
If CStr(strE) = CStr(strSearchE) And CStr(strA) = CStr(strSearchA) And CStr(strF) = CStr(strSearchF) And CStr(strH) = CStr(strSearchH) And CStr(strL) = CStr(strSearchL) Then
AmountArr(k) = ws.Cells(j, 12).Value
rowArr(k) = j
intCnt = intCnt + 1
k = k + 1
Else
Exit For
End If
Next
strSum = 0
For s = 0 To UBound(AmountArr)
If AmountArr(s) <> "" Then
strSum = strSum + AmountArr(s)
Else
Exit For
End If
Next
strAppenRow = ""
For b = 0 To UBound(rowArr)
If rowArr(b) <> "" Then
strAppenRow = strAppenRow & "" & rowArr(b) & "," & AmountArr(b) & ","
Else
Exit For
End If
Next
If intCnt = 1 Then
Else
If strSum = 0 Then
For rn = 0 To UBound(rowArr)
If rowArr(rn) <> "" Then
Let rRange = rowArr(rn) & ":" & rowArr(rn)
Rows(rRange).Select
Selection.Interior.Color = vbYellow
Else
Exit For
End If
Next
Else
strvar = ""
strvar = Split(strAppenRow, ",")
For ik = 1 To UBound(strvar)
If strvar(ik) <> "" Then
strVal = CDbl(strvar(ik))
For ik1 = ik To UBound(strvar)
If strvar(ik1) <> "" Then
strVal1 = CDbl(strvar(ik1))
If strVal1 + strVal = 0 Then
Let sRange1 = strvar(ik - 1) & ":" & strvar(ik - 1)
Rows(sRange1).Select
Selection.Interior.Color = vbYellow
Let sRange = strvar(ik1 - 1) & ":" & strvar(ik1 - 1)
Rows(sRange).Select
Selection.Interior.Color = vbYellow
End If
Else
Exit For
End If
ik1 = ik1 + 1
Next
Else
Exit For
End If
ik = ik + 1
Next
End If
End If
i = i + (intCnt - 1)
k = 0
Erase AmountArr
Erase rowArr
End If
Next
Range("A1").Select
End Sub
I still have some mistakes (rows not higlighted when they should be), the above code is not perfect, but I thought it'd be OK to give you an idea of where I was going before Paul Bica's solution came in.
Thanks again !
If your data is only till column L, then use below code, I found it is taking less time to run....
Sub Duplicates()
Application.ScreenUpdating = False
Dim i As Long, lrow As Long
lrow = Cells(Rows.Count, 1).End(xlUp).Row
Range("O2") = "=A2&E2&F2&J2&L2"
Range("P2") = "=COUNTIF(O:O,O2)"
Range("O2:P" & lrow).FillDown
Range("O2:O" & lrow).Copy
Range("O2:O" & lrow).PasteSpecial xlPasteValues
Application.CutCopyMode = False
For i = 1 To lrow
If Cells(i, 16) = 2 Then
Cells(i, 16).EntireRow.Interior.Color = vbYellow
End If
Next
Application.ScreenUpdating = True
Range("O:P").Delete
Range("A1").Select
MsgBox "Done"
End Sub

Adding a character at the end of text & increment letter A to B

What I need help on is to copy the previous cells text into the cell below it and add the letter A at the end of it i.e. before VP0007 after VP0007A. This should continue until all the blank cells have been incremented and it reaches the next VP0008.
Please see the images. I apologise if I am not too clear.
             Before:                         After:
      
Right now I have the following code:
ActiveCell.Offset(1, 0).Select
Letter = "A"
Letters = Chr(Asc(Letter) + 1)
Number = ActiveCell.Offset(-1, 0).Value
If ActiveCell.Value = Number & Letter _ Then
ActiveCell.Offset(1, 0).Select.Value Number & Number
Else
ActiveCell.Value = Number & Letters
End If
Loop Until ActiveCell.Offset(1, 0).Value <> ""
Try this short sub procedure.
Sub fillSubseries()
Dim i As Long, a As Long, str As String
With Worksheets("sheet4")
For i = 2 To .Cells(.Rows.Count, "A").End(xlUp).Row
If IsEmpty(.Cells(i, "A")) Then
.Cells(i, "A") = str & Chr(a)
a = a + 1
Else
a = 65
str = .Cells(i, "A").Value2
End If
Next i
End With
End Sub
Try using the below code
LastRow = ActiveSheet.Range("A" & Rows.Count).End(xlUp).Row
Letter = "A"
For iLoop = 2 To LastRow
If ActiveSheet.Range("A" & iLoop) = "" Then
iValue = ActiveSheet.Range("A" & iLoop - 1)
iiLoop = iLoop
Do
If ActiveSheet.Range("A" & iiLoop) = "" Then
ActiveSheet.Range("A" & iiLoop) = iValue & Letter
Letter = Chr(Asc(Letter) + 1)
Else
Letter = "A"
Exit Do
End If
iiLoop = iiLoop + 1
Loop
iLoop = iiLoop - 1
End If
Next
This code should handle cases where you have more than 26 blank rows and increment past the letter "Z".
Sub FillBlanks()
Dim lastRow As Long, cnt As Long, i As Long
Dim prevItem As String
Dim ws As Worksheet
Set ws = ActiveSheet
lastRow = ws.Cells(Rows.Count, 1).End(xlUp).Row
prevItem = ws.Cells(2, 1).Value
cnt = 0
For i = 2 To lastRow
If ws.Cells(i, 1) = "" Then
cnt = cnt + 1
ws.Cells(i, 1).Value = prevItem & Split(Cells(1, cnt).Address(True, False), "$")(0)
Else
prevItem = ws.Cells(i, 1)
cnt = 0
End If
Next i
End Sub
Alternate:
Sub tgr()
Dim ws As Worksheet
Dim aData As Variant
Dim sTemp As String
Dim sLetter As String
Dim i As Long, j As Long
Set ws = ActiveWorkbook.ActiveSheet
With ws.Range("A2", ws.Cells(ws.Rows.Count, "A").End(xlUp))
If .Row < 2 Then Exit Sub 'No data
aData = .Value
End With
For i = LBound(aData, 1) To UBound(aData, 1)
If Len(Trim(aData(i, 1))) > 0 Then
sTemp = Trim(aData(i, 1))
j = 0
Else
j = j + 1
sLetter = Replace(ws.Cells(1, j).Address(0, 0), 1, vbNullString)
aData(i, 1) = sTemp & sLetter
End If
Next i
ws.Range("A2").Resize(UBound(aData, 1)).Value = aData
End Sub
If you need a pure formula solution, you may try below steps (The first row of your data should be A2, not A1):
First we need a dummy column in order to fill in the blank rows. Use below formula on B2 and copy it down through the last row of column A:
=IF(A2<>"",A2,B1)
Then we will create the final values on column C. Add below formula to C2 and copy down:
=IF(A2<>"",A2,IF(ISNUMBER(VALUE(RIGHT(C1,1)))=TRUE,C1&"A",B2&CHAR(CODE(RIGHT(C1,1))+1)))
Basically we first filled in the blank rows with repeating values on column B. Then copied Col:A value to Col:C if Col:A is not blank. If Col:A is blank and upper row (Col:C) value's last character is numeric we add "A" to that value. If the last character is a letter than we concatenate the next letter with Col:B value.
You should have something like below, when everything is OK:

Excel vlookup to get all occurrence cell details?

I am doing automation on matching Data form row Data1 to Data 2,
I was done by looping statement but the problem is take much time, when number of row increase
For that reason i planed do by vlookup, In vlookup only return first occurrence cell but i need to find all match cell and highlighted matched row ,which i show in figure.
Working with cells directly reduces the code performance. Try to set Data1 and Data2 to arrays and work with arrays.
Something like this:
With ActiveSheet
arr = .Range(.[A2], .Cells(.Rows.Count, "A").End(xlUp)).Value
arr2 = .Range(.[D2], .Cells(.Rows.Count, "D").End(xlUp)).Value
For i& = 1 To UBound(arr)
For j& = 1 To UBound(arr2)
If arr(i, 1) = arr2(j) Then
...
End If
Next j
Next i
End With
Hope you are looking for this
Sub testvlookup()
Dim lastrow, lastrowdata, incre, i, j As Long
lastrow = Range("A" & Rows.Count).End(xlUp).Row
lastrowdata = Range("D" & Rows.Count).End(xlUp).Row
incre = 6
For i = 2 To lastrow
For j = 2 To lastrowdata
If Range("A" & i).Value = Range("D" & j).Value Then
Range("D" & j, "G" & j).Interior.ColorIndex = incre
End If
Next j
incre = incre + 1
Next i
End Sub
I don't see the point why it should be to slow for many rows, it would be good to have more informations about that.
I would do it like the others, takes ~1 sec with 100000 comparisons.
Dim i As Integer
Dim b As Integer
i = 1
While i < 20000
Range("A1:A5").Copy Range(Cells(i, 4), Cells(i + 5, 4))
i = i + 5
Wend
MsgBox ("hi")
i = 1
While i < 7
b = 3
While b < 20000
If Cells(i, 1).Value = Cells(b, 4).Value Then
Cells(b, 4).Interior.ColorIndex = i
End If
b = b + 1
Wend
i = i + 1
Wend
End Sub

Single function to write for all message id

Iam new to Excel VBA , I am started writing a code , which was executed fine, but I need a suggestion how to write a function where i dont need to write code for all "ID".
For example :
I have main works sheet having ID(1000x, 10000, 2000X,20000).
I want to search only ID with number not with alphabet, and compare it with another worksheet , having the same ID , if then get the corrosponding ID 3rd column data and conacdenate all them into main worksheet .
I have main worksheet ("Tabelle1")having all the ID(10000,20000) in Coloumn A ,I want the infomration of ID 10000 in column B of ID 10000. some times i have 10000 for four times . Want to paste infomration to another worksheet ("Test_2"), I want to collect all the 10000 and corrosponding data .
Sub Update()
If MsgBox("Are you sure that you wish to Update New Measurement ?", vbYesNo, "Confirm") = vbYes Then
Dim erow As Long, erow1 As Long, i As Long
erow1 = Sheets("Tabelle1").Cells(Rows.Count, 1).End(xlUp).Row
For i = 2 To erow1
If Sheets("Tabelle1").Cells(i, 2) <> "10000" Then
Sheets("Tabelle1").Range(Sheets("Tabelle1").Cells(i, 1), Sheets("Tabelle1").Cells(i, 2)).Copy
Sheets("Test_2").Activate
erow = Sheets("Test_2").Cells(Rows.Count, 1).End(xlUp).Offset(1, 0).Row
ActiveSheet.Paste Destination:=Sheets("Test_2").Range(Cells(erow, 1), Cells(erow, 2))
Sheets("Test_2").Activate
End If
Next i
Application.CutCopyMode = False
For i = 1 To erow
Totalstrings = Totalstrings & Cells(i, 2) & "" + vbCrLf
Next i
Totalstrings = Left(Totalstrings, Len(Totalstrings) - 1)
Range("C5") = Totalstrings
Range("C5").Select
Selection.Copy
Sheets("BSM_STF_iO").Select
Range("C5").Select
ActiveSheet.Paste
MsgBox "New measurements have been Updated !"
End If
End Sub
Example
In BSM:STM:IO
A B
ID
1000X
10000
10001
...
in Tabelle1
B C
ID
1000 abc
1000 xyz
10001 lmn
2000 def
"
I want to compare only digit from"the "BSM:STM:Io" with "tabelle1". Example take the the first value 10000 from "BSM_STM_io" compare with tabele take the the value of corrosponding Coloumn "C" in "tablle1" and put it into single cell in 1000 of BSM_STM:Io
A , B , C are coloumn in the worksheet
enter image description here
Lets assume worksheet "BSM_STF_iO" contains the ID information in A column beginning with A2 and worksheet Tabelle1 contains the required concaetenation information in B Column beginning from B2 (ex: Column B: IDs, Column C: information to concaetenate). Below code will concaetenate the contents and write in BSM_STF_iO sheet.
Sub test1()
Worksheets("BSM_STF_iO").Select
LastRow = Range("A" & Rows.Count).End(xlUp).Row
For i = 2 To LastRow
a = onlyDigits(Range("A" & i).Value)
With Worksheets("Tabelle1")
destlastrow = .Range("B" & Rows.Count).End(xlUp).Row
For j = 2 To destlastrow
If a = Trim(.Range("B" & j).Value) Then
If out <> "" Then
out = out & ", " & .Range("C" & j).Value
Else
out = .Range("C" & j).Value
End If
End If
Next j
Cells(i, 2) = out
out = ""
End With
Next i
End Sub
and below function taken from How to find numbers from a string?
Function onlyDigits(s As String) As String
Dim retval As String
Dim i As Integer
retval = ""
For i = 1 To Len(s)
If Mid(s, i, 1) >= "0" And Mid(s, i, 1) <= "9" Then
retval = retval + Mid(s, i, 1)
End If
Next
onlyDigits = retval
End Function

Excel VBA Optimization - Transposing Data

I've receieved a report in a rolled up fashion in Excel that I need to flatten out in order to import it into Access. Here's a sample of the row:
What needs to happen is the Customer Account and Name need to be transposed to be adjacent to the Voucher line, and needs to be copied so each voucher line has this information. After the transformation, the data should look like this:
Customer Account | Name | Date | Voucher | Invoice | Transation Text | Currency
Note that the row starting with "USD" denotes the end of records for that customer.
I have successfully implemented the following code:
Sub Process_Transactions()
'turn off some Excel functionality so code runs faster
Application.ScreenUpdating = False
Application.Calculation = xlCalculationManual
Application.DisplayStatusBar = False
Application.EnableEvents = False
Dim i As Long
For i = 1 To 731055
'Move two columns in
ActiveCell.Offset(0, 2).Select
'Select the customer account and name
Range(ActiveCell, ActiveCell.Offset(1, 1)).Select
'Copy and paste it down two rows and over two columns
Selection.Cut
ActiveCell.Offset(2, -2).Select
ActiveSheet.Paste
'Hop up a couple rows and delete 3 rows before the data that are not useful
Rows(ActiveCell.Offset(-2).Row).Select
Selection.Delete Shift:=xlUp
Selection.Delete Shift:=xlUp
Selection.Delete Shift:=xlUp
'Select the next row
Rows(ActiveCell.Offset(1).Row).Select
'If the first record in the row is not "USD", then we have multiple rows for
'this customer
While (ActiveCell.Offset(0, 2) <> "USD")
'Copy and Paste the customer account and number for each
'transaction row
ActiveCell.Select
Range(ActiveCell.Offset(-1, 0), ActiveCell.Offset(-1, 1)).Select
Selection.Copy
ActiveCell.Offset(1, 0).Select
ActiveSheet.Paste
ActiveCell.Select
ActiveCell.Offset(1, 0).Select
Wend
'Delete the two rows after the data that we need
ActiveCell.Select
Rows(ActiveCell.Row).Select
Selection.Delete Shift:=xlUp
ActiveCell.Select
Rows(ActiveCell.Row).Select
Selection.Delete Shift:=xlUp
'Move to the next row to start over
ActiveCell.Select
Debug.Print "Current Row: " & i
Next i
'at the end, don't forget to restore the default behavior
'calculate the formulas
Application.Calculate
Application.ScreenUpdating = True
Application.Calculation = xlCalculationAutomatic
Application.DisplayStatusBar = True
Application.EnableEvents = True
End Sub
The problem is that the program is very slow. I let the code run for approximately 10 hours last night, and it only processed 33k. I've got roughly 1.5 mil records to process.
I realize that the technique I am using is actually moving the activecell around, so removing that would likely help. However, I am unsure how to proceed. If this is a lost cause and better suited for a .net implementation, feel free to suggest that.
Your code is jam-packed with Excel-VBA methods that are very inefficient! I'll take a few shots:
Don't use .Select and Selection.. That's super slow.
Why do this
Range(ActiveCell, ActiveCell.Offset(1, 1)).Select
Selection.Cut
when you can do this
Range(ActiveCell, ActiveCell.Offset(1, 1)).Cut
Also don't use ActiveCell to move around your sheet. Just do operations directly on whatever cell or row you need, e.g.
Sheet1.Cells(i,2).Copy
Sheet1.Cells(i,1).Paste
Actually, avoid copy/pasting altogether and just say
Sheet1.Cells(i,1).Value = Sheet1.Cells(i,2).Value
Avoid referring to the same object many times and use With instead. Here, Sheet1 is used twice, so you could write this:
With Sheet1
.Cells(i,1).Value = .Cells(i,2).Value
End With
The above are just examples that you will have to adjust to your circumstances, and there is more to optimise, but they'll get you started. Show us your code once you've cleaned it up, and more advice will come!
The fast way to do this would be to grab large chunks of data into a 2-D variant array
Dim varr as Variant
varr=Worksheets("Sheet1").Range("C5:G10005")
then loop on the array and create another variant 2-d array (varr2)second that looks the way you want it, then write the variant array to another worksheet:
Worksheets("Sheet2").Range("A2:G2")=varr2
You don't have to select a cell on every command you execute.
Here is a try:
Dim i As Long
'Suppose you want to start on cell A1
With ActiveSheet
For i = 1 To 731055
'Move two columns to the right and select the customer account and name
'.Range("C" & i & ":D" & i + 1).Cut
'Cut and paste it down two rows and over two columns
'.Range("A" & i + 2 & ":B" & i + 3).Paste
.Range("A" & i + 2 & ":B" & i + 3).Value = .Range("C" & i & ":D" & i + 1).Value
'Hop up a couple rows and delete 3 rows before the data that are not useful
.Range("A" & i & ":C" & i + 2).EntireRow.Delete
'If the first record in the row is not "USD", then we have multiple rows for
'this customer
While (.Range("C" & i + 1).Value <> "USD")
'Copy and Paste the customer account and number for each
'transaction row
'.Range("A" & i & ":B" & i).Copy
'.Range("A" & i + 1 & ":B" & i + 1).Paste
.Range("A" & i + 1 & ":B" & i + 1).Value = .Range("A" & i & ":B" & i).Value
i = i + 1
Wend
'Delete the two rows after the data that we need
.Range("A" & i + 1 & ":A" & i + 2).EntireRow.Delete
'Move to the next row to start over
Debug.Print "Current Row: " & i
Next i
End With
[edit] i changed a little bit my code to copy only the values (this will be much much faster) instead of copy/paste >> see if you really need to copy paste to keep format or so
[edit] Nick: There were a few numbers that were just a little off, so I've updated the answer to reflect these.
I also posted this on Twitter, and got the following from #VisBasApp:
Sub Process_TransactionsPAT()
Const COL_CUSTOMER_ACC As Long = 3
Const COL_CUSTOMER_NAME As Long = 4
Const COL_CUSTOMER_VOUCHER As Long = 4
Const COL_CUSTOMER_INVOICE As Long = 5
Const COL_CUSTOMER_TRANS As Long = 6
Const COL_CUSTOMER_CURR As Long = 7
Const COL_CUSTOMER_AMT_CUR As Long = 8
Const COL_CUSTOMER_BAL_CUR As Long = 9
Const COL_CUSTOMER_BAL As Long = 10
Const COL_CUSTOMER_DUE_DATE As Long = 11
Const COL_CUSTOMER_COL_CODE As Long = 12
Const TEXT_TO_CHECK As String = "Customer account"
Dim accNumber As Variant
Dim accName As String
Dim index As Long
Dim counter As Long
Dim originalData As Variant
Dim transferedData() As Variant
'turn off some Excel functionality so code runs faster
Application.ScreenUpdating = False
Application.Calculation = xlCalculationManual
Application.DisplayStatusBar = False
Application.EnableEvents = False
originalData = Range("A1:L720909")
counter = 0
For i = 1 To UBound(originalData, 1)
If originalData(i, COL_CUSTOMER_ACC) = TEXT_TO_CHECK Then
' go to the first row under the text 'Customer Account'
index = i + 1
' get name and account number
accNumber = originalData(index, COL_CUSTOMER_ACC)
accName = originalData(index, COL_CUSTOMER_NAME)
' go to the first row under the text 'Date'
index = index + 2
counter = counter + 1
While (UCase(originalData(index, COL_CUSTOMER_ACC)) <> "USD")
ReDim Preserve transferedData(1 To 12, 1 To counter)
transferedData(1, counter) = accNumber
transferedData(2, counter) = accName
transferedData(3, counter) = originalData(index, COL_CUSTOMER_ACC)
transferedData(4, counter) = originalData(index, COL_CUSTOMER_VOUCHER)
transferedData(5, counter) = originalData(index, COL_CUSTOMER_INVOICE)
transferedData(6, counter) = originalData(index, COL_CUSTOMER_TRANS)
transferedData(7, counter) = originalData(index, COL_CUSTOMER_CURR)
transferedData(8, counter) = originalData(index, COL_CUSTOMER_AMT_CUR)
transferedData(9, counter) = originalData(index, COL_CUSTOMER_BAL_CUR)
transferedData(10, counter) = originalData(index, COL_CUSTOMER_BAL)
transferedData(11, counter) = originalData(index, COL_CUSTOMER_DUE_DATE)
transferedData(12, counter) = originalData(index, COL_CUSTOMER_COL_CODE)
index = index + 1
counter = counter + 1
Wend
' it is not the best technique but for now it works
i = index + 1
counter = counter - 1
End If
Next i
' add data on a new sheet
Sheets.Add
Cells(1, 1) = "Customer Account"
Cells(1, 2) = "Name"
Cells(1, 3) = "Date"
Cells(1, 4) = "Voucher"
Cells(1, 5) = "Invoice"
Cells(1, 6) = "Transaction Left"
Cells(1, 7) = "Currency"
Cells(1, 8) = "Amount in currency"
Cells(1, 9) = "Balance in currency"
Cells(1, 10) = "Balance"
Cells(1, 11) = "Due Date"
Cells(1, 12) = "Collection letter code"
For i = 1 To UBound(transferedData, 2)
For j = 1 To UBound(transferedData, 1)
Cells(i + 1, j) = transferedData(j, i)
Next j
Next i
Columns.AutoFit
'at the end, don't forget to restore the default behavior
'calculate the formulas
Application.Calculate
Application.ScreenUpdating = True
Application.Calculation = xlCalculationAutomatic
Application.DisplayStatusBar = True
Application.EnableEvents = True
End Sub
This takes roughly 2 minutes to parse 750,000 records.
I would throw the data as-is on a database, and write a query to do that. I'll write a query and update the answer when I get home (I'm on my phone, its impossible to write SQL :)