Optimization for Excel - optimization

Here is the Situation: In my Excel sheet I had a column with entries in the form 1-name. I wanted to remove the numbers, taking into account that the number can also be double digit. This by itself was not a Problem and I got it working, just the Performance is so bad. As it is now my program needs about half a second per cell entry.
My question: How can I improve the performance?
Here is the code:
Sub remove_numbers()
Dim YDim As Long
Dim i As Integer, l As Integer
Dim val As String
Dim s As String
YDim = Cells(Rows.Count, 5).End(xlUp).Row
For i = 8 To YDim
val = Cells(i, 5)
l = Len(val)
s = Mid(val, 2, 1)
If s = "-" Then
val = Right(val, l - 2)
Else
val = Right(val, l - 3)
End If
Cells(i, 5).Value = val
Next i
End Sub

Instead of using 3 different functions: Len(), Mid(), Right() you could use a Split() function which would have been much more efficient in this case.
Try the below code
Sub remove_numbers()
Application.ScreenUpdating = False
Dim i As Long
For i = 8 To Cells(Rows.Count, 5).End(xlUp).Row
Cells(i, 5) = Split(Cells(i, 5), "-")(1)
Next i
Application.ScreenUpdating = True
End Sub

My suggestion:
Sub remove_numbers()
Dim i As Integer, values() As Variant
values = Range(Cells(8, 5), Cells(Rows.Count, 5).End(xlUp).Row).Value
For i = LBound(values) To UBound(values)
values(i, 1) = Mid(values(i, 1), IIf(Mid(values(i, 1), 2, 1) = "-", 2, 3))
Next
Range(Cells(8, 5), Cells(Rows.Count, 5).End(xlUp).Row).Value = values
End Sub
Optimizations:
Perform all calculations in memory and them update entire range: this is a HUGE performance gain;
Condensed multiple commands into a single command;
Replaced Right(x, Len(x)-n) with Mid(x, n).
EDIT:
As suggested by #Mehow, you may also gain some performance using
values(i, 1) = Split(values(i, 1), "-", 2)(1)
instead of values(i, 1) = Mid(values(i, 1), IIf(Mid(values(i, 1), 2, 1) = "-", 2, 3))

You should manipulate the whole range values as an array and work directly with it in memory.
Something like :
Dim valuesOfRangeToModify() As Variant
Set valuesOfRangeToModify = Range(Cells(8, 5), Cells(Rows.Count, 5).End(xlUp)).Value
For Each cell In valuesOfRangeToModify
cell = ... // remove numbers
Next
Range(Cells(8, 5), Cells(Rows.Count, 5).End(xlUp)).Value = valuesOfRangeToModify
My VB is quite old so it probably has syntax errors but you get the idea.
This should give a huge boost.
For reference, here is an article full of interesting advices, see point #4 for more explanation of the solution given above :
http://www.soa.org/news-and-publications/newsletters/compact/2012/january/com-2012-iss42-roper.aspx

Also do not operate one cell at a time. Create a range of cells and transfer them into an array for processing. In the end the array can be used to replace the cells.
To tweak the answer from #mehow
Sub remove_numbers()
Dim i As Long, N as Long, r as Range
Set r = Range("B3") ' Whatever is the first cell in the column
N = Range(r, r.End(xlDown)).Rows.Count 'Count the rows in the column
Set r = r.Resize(N,1) ' Expand the range with all the cells
Dim values() as Variant
values = r.Value ' Collect all the values from the sheet
For i=1 to N
values(i,1) = Split( values(i,1), "-")(1)
Next i
r.Value = values 'Replace values to the sheet
End Sub
To make it more general you can add an argument to the procedure to pass a reference to the first cell in the column, like Sub remove_numbers(ByRef r as Range). There is no need to de-activate the screen as there is only one write operation in the end and you want the screen to update after that.

Related

For loop while copy and pasting specific columns

I need a loop that will match and select different columns (not in sequential order) and paste them to another sheet all whilst keeping the condition in check. It would also be ideal if when the values get pasted that the formatting for the cell is not carried over, just the value.
Below is the code I am currently using:
Sub Test()
Application.ScreenUpdating = False
Sheets("DATA").Select
lr = Range("B" & Rows.Count).End(xlUp).Row
Range("P3").Select
For i = 3 To lr
If Cells(i, 2) <> "" Then Range(Cells(i, 7), Cells(i, 16), Cells(i, 26)).Copy
Sheets("Sheet2").Range("A" & Rows.Count).End(xlUp).Offset(1, 0)
Next
Application.ScreenUpdating = True
End Sub
The problem is declaring the columns I want the loop to paste. I need the loop to run through the 16th column, check empty values, and then paste the index/matched value in the rows of columns 7,16,and 26 (so not in sequential order).. Any help would be appreciated.
The next code has to do what I understood you need. Please check it and confirm this aspect. It is very fast, working only in memory...
Sub PastingNextPage()
Dim sh As Worksheet, sh1 As Worksheet, arrIn As Variant, arrOut() As Variant
Dim lastRowIn As Long, lastRowOut As Long, nonEmpt As Long, rngP As Range, nrEl As Long
Dim i As Long, j As Long, P As Long
Set sh = Sheets("DATA"): lastRowIn = sh.Range("P" & sh.Rows.count).End(xlUp).Row
Set sh1 = Sheets("Sheet2"): lastRowOut = sh1.Range("A" & sh1.Rows.count).End(xlUp).Row + 1
arrIn = sh.Range("G2:Z" & lastRowIn).Value
nrEl = lastRowIn - Application.WorksheetFunction.CountIf(sh.Range("P2:P" & lastRowIn), "") - 2
P = 10 'column P:P number in the range starting with G:G column
ReDim arrOut(nrEl, 3) 'redim the array to keep the collected values
For i = 1 To lastRowIn - 1
If arrIn(i, P) <> "" Then
arrOut(j, 0) = arrIn(i, 1): arrOut(j, 1) = arrIn(i, P): arrOut(j, 2) = arrIn(i, 20)
j = j + 1
End If
Next i
sh1.Range(sh1.Cells(lastRowOut, "A"), sh1.Cells(lastRowOut + nrEl, "C")).Value = arrOut
End Sub
It does not select anything, you can run it activating any of the two involved sheets. I would recommend to be in "Sheet2" and see the result. If you want to repeat the test, its result will be added after the previous testing resulted rows...
If something unclear or not doing what you need, do not hesitate to ask for clarifications.

Paste one array (single dimension) with >100K item into a excel range

I posted the same question on StackOverflow thread
but I think here is the correct place to ask (if is not right, admin please to remove it).Every day I need to format date imported from AS400 (data, time,..).
Usualy (for some thousands of record) I use this code.
Public Sub Cfn_FormatDate(control As IRibbonControl)
Application.ScreenUpdating = False
Dim UR As Long, X As Long
Dim MyCol As Integer
MyCol = ActiveCell.Column
UR = Cells(Rows.Count, MyCol).End(xlUp).Row
For X = 2 To UR
If Not IsDate(Cells(X, MyCol)) Then
Select Case Len(Cells(X, MyCol))
Case 8
Cells(X, MyCol) = DateSerial(Left(Cells(X, MyCol), 4), Mid(Cells(X, MyCol), 5, 2), Right(Cells(X, MyCol), 2))
Case 6
Cells(X, MyCol) = DateSerial(Left(Cells(X, MyCol), 2), Mid(Cells(X, MyCol), 3, 2), Right(Cells(X, MyCol), 2))
End Select
End If
Next X
Columns(MyCol).NumberFormat = "DD/MM/YYYY;#"
Columns(MyCol).EntireColumn.AutoFit
Application.ScreenUpdating = True
End Sub
but if the records are are many more, the code posted code is not performing.
(ex 70K records were formatted / pasted in 18 seconds)
so I thought to using variables in an array and I wrote this code:
Sub ConvDate(c As Integer)
Application.ScreenUpdating = False
Dim lrw As Long, i As Long
Dim ArrVal As Variant
lrw = ActiveSheet().Range(Cells(1, c)).End(xlDown).Row
ReDim ArrVal(2 To lrw)
For i = 2 To lrw
If IsDate(Cells(i, c)) Then
ArrVal(i) = Cells(i, c)
Else
Select Case Len(Cells(i, c)) ' to check YYYYMMDD or YYMMDD
Case 8
ArrVal(i) = DateSerial(Left(Cells(i, c), 4), Mid(Cells(i, c), 5, 2), Right(Cells(i, c), 2))
Case 6
ArrVal(i) = DateSerial(Left(Cells(i, c), 2), Mid(Cells(i, c), 3, 2), Right(Cells(i, c), 2))
End Select
End If
NextX:
Next i
Range(Cells(2, c), Cells(lrw, c)) = ArrVal
Columns(c).NumberFormat = "DD/MM/YYYY;#"
Columns(c).EntireColumn.AutoFit
Application.ScreenUpdating = True
End Sub
It not work, all the cells (in the range) have the same result (Cells(2, c)).
a guy suggested me to change the code like:
ActiveSheet.Range(Cells(2, c), Cells(lrw, c)).Value = WorksheetFunction.Transpose(ArrVal)
this change is limiting , over 65536 records I get an error (runtime 13, type mismatch)
Ok, to summarise all the answers and comments:
As you have indicated in your question and as user85489 alludes, reading the values into an array, manipulating that same array, and writing it back to the sheet is vastly quicker than lopping cell by cell.
If you have an array whose 'row' dimension is not going to change. Then it might be fair to say that you're better off declaring a 2 dimensional array of size (1 to rows, 1 to columns). This way you can avoid having to transpose a 1 dimensional array at all.
Because as Gareth points out, Transpose() is limited to 65536 elements in a dimension.
Putting it all together, then, skeleton code for your post could be this:
Sub ConvertDates(colIndex As Long)
Dim v As Variant
Dim firstCell As Range
Dim lastCell As Range
Dim fullRange As Range
Dim i As Long
Dim dd As Integer
Dim mm As Integer
Dim yy As Integer
Dim dat As Date
'Define the range
With ThisWorkbook.Worksheets("Sheet1")
Set firstCell = .Cells(2, colIndex)
Set lastCell = .Cells(.Rows.Count, colIndex).End(xlUp)
Set fullRange = .Range(firstCell, lastCell)
End With
'Read the values into an array
v = fullRange.Value
'Convert the text values to dates
For i = 1 To UBound(v, 1)
If Not IsDate(v(i, 1)) Then
If Len(v(i, 1)) = 6 Then v(i, 1) = "20" & v(i, 1)
yy = CInt(Left(v(i, 1), 4))
mm = CInt(Mid(v(i, 1), 5, 2))
dd = CInt(Right(v(i, 1), 2))
dat = DateSerial(yy, mm, dd)
v(i, 1) = dat
End If
Next
'Write the revised array and format range
With fullRange
.NumberFormat = "DD/MM/YYYY;#"
.Value = v
.EntireColumn.AutoFit
End With
End Sub
You have come across the 32 bit limitation of the function Transpose which truncates your array to 65536.
You can use loop statement to populate the cells, else if you want to do it directly then define your array ArrVal like:
Redim ArrVal(1,Lrw) as variant
Flood the array with values and then, Offload it like
Range(Cells(2, c), Cells(lrw, c)) = ArrVal
Hopefully you get rid of the same value errors.

How do I transform a matrix of data on the spreadsheet into a single column using VBA?

Say, my original data block is worksheets(1).range("A1:C100"), and I'd like to stack the columns of this block into a single column, that is, I first put first column, then the second column goes below, and finally the third column. In the end, I should have a single column, say being put in worksheets(2).range("A1:A300"). I wonder if there's any smart and fast algorithm to achieve this?
Without VBA, In Sheet2 cell A1 enter:
=OFFSET(Sheet1!$A$1,MOD(ROWS($1:1)-1,100),ROUNDUP(ROWS($1:1)/100,0)-1,)
and copy down.
and with VBA
Sub copy_table_to_column()
Dim s As String
s = "=OFFSET(Sheet1!$A$1,MOD(ROWS($1:1)-1,100),ROUNDUP(ROWS($1:1)/100,0)-1,)"
With Worksheets("Sheet2").Range("A1:A300")
.Formula = s
.Value = .Value
End With
End Sub
There might be a better way, but I usually do it with an Offset
I=0
For Each A in Worksheets(1).Range("A1:A100").Cells
Worksheets(2).Range("A1").Offset(I,0) = A.Value
I = I + 1
Next
For Each B in Worksheets(1).Range("B1:B100").Cells
Worksheets(2).Range("A1").Offset(I,0) = B.Value
I = I + 1
Next
For Each C in Worksheets(1).Range("C1:C100").Cells
Worksheets(2).Range("A1").Offset(I,0) = C.Value
I = I + 1
Next
This might be good enough for you...
Hope it helps.
Option Explicit
'Define the test function...
Sub test()
Dim vData As Variant
Dim r As Range
Set r = Sheet1.Range("A1:C100")
vData = ConcatinateColumns(r)
End Sub
'Define the function to concatinate columns.
Public Function ConcatinateColumns(ByVal Data As Range)
Dim vTemp As Variant
Dim i As Integer, j As Long, k As Long
'Get the data for each cell to a variant.
vTemp = Data.Value
ReDim vData(1 To (UBound(vTemp, 1) - LBound(vTemp, 1) + 1) * (UBound(vTemp, 2) - LBound(vTemp, 2) + 1), 1 To 1) As Variant
For i = LBound(vTemp, 2) To UBound(vTemp, 2)
For j = LBound(vTemp, 1) To UBound(vTemp, 1)
k = k + 1
vData(k, LBound(vData, 1)) = vTemp(j, i)
Next
Next
ConcatinateColumns = vData
End Function

Macro/VBA: Clear cells in a row based on values in a column, and loop through entire column

I'm trying to write a macro in excel that will identify the first value in a row (A2) and then search the rest of the row to clear any cell with a greater value (C2:DGA2). I'd like to set this up such that the program loops through every row in the column (A2:A400), and clears the corresponding values.
I tried using the following code, which I modified from another post:
Sub clear_cell()
Dim v
v = Excel.ThisWorkbook.Sheets("TOP LINE").Range("B2").Value
Dim Arr() As Variant
Arr = Sheet1.Range("C2:DGJ2")
Dim r, c As Long
For r = 1 To UBound(Arr, 1)
For c = 1 To UBound(Arr, 2)
If Arr(r, c) > v Then
Arr(r, c) = ""
End If
Next c
Next r
Sheet1.Range("C2:DGJ2") = Arr
End Sub
I modified it to fit my needs, but it only works for the first row. I need some help getting it to loop through every row in the first column.
Thank you for the help.
I'm trying to write a macro in excel that will identify the first value in a row (A2) and then search the rest of the row to clear any cell with a greater value (C2:DGA2).
From the above statement, I am assuming that all ranges are in the same sheet. Your code works for me if I make a few changes. See this
Sub clear_cell()
Dim i As Long, j As Long
Dim Arr
'~~> Set Range here
Arr = Sheet1.Range("A2:DGJ400").Value
For i = 1 To UBound(Arr, 1)
For j = 2 To UBound(Arr, 2)
If Arr(i, j) > Arr(i, 1) Then
Arr(i, j) = ""
End If
Next j
Next i
'~~> Write back to the sheet
Sheet1.Range("A2:DGJ400") = Arr
End Sub
give this a try:
Sub clear_cell()
x = 2
Do While x <= 400
Y = Range(Cells(x, 2), Cells(x, 2)).Value
If Y < 100 Then Range(Cells(x, 2), Cells(x, 2)).FormulaR1C1 = ""
x = x + 1
Loop
End Sub
The 2 is the column range, in this case B. Good Luck.

VBA script to count string, insert rows, copy row, split cell

The department that provides me a spreadsheet to be used in my database now includes multiple text in a cell. In order to link to that data I have to turn it into multiple rows. Example: LC123/LC463/LC9846 needs to have the entire row copied with just one "LC" string in each row-
cell1 cell2 LC123
cell1 cell2 LC463
cell1 cell2 LC9846
I tried these two subroutines but obviously it failed
Sub InSert_Row()
Dim j As Long
j = InputBox(=SUM(LEN(ActiveCell)-LEN(SUBSTITUTE(ActiveCell,"LC",""))-1)
ActiveCell.EntireRow.Copy
ActiveCell.Offset(j).EntireRow.Insert Shift:=xlDown
End Sub
Sub SplitAndTranspose()
Dim N() As String
N = Split(ActiveCell, Chr(10))
ActiveCell.Resize(UBound(N) + 1) = WorksheetFunction.Transpose(N)
End Sub
The 2nd subroutine will split and copy but it doesn't insert rows, it writes over the rows below it.
'In memory' method
Inserting rows as necessary would be perhaps the most simple to understand, but the performance of making thousands of seperate row inserts would not be good. This would be fine for a one off (perhaps you only need a one-off) and should only take a minute or two to run but I thought what the heck and so wrote an approach that splits the data in memory using a collection and arrays. It will run in the order of seconds.
I have commented what it is doing.
Sub ProcessData()
Dim c As Collection
Dim arr, recordVector
Dim i As Long, j As Long
Dim rng As Range
Dim part, parts
'replace with your code to assign the right range etc
Set rng = ActiveSheet.UsedRange
j = 3 'replace with right column index, or work it out using Range.Find etc
arr = rng.Value 'load the data
'Process the data adding additional rows etc
Set c = New Collection
For i = 1 To UBound(arr, 1)
parts = Split(arr(i, j), "/") 'split the data based on "/"
For Each part In parts 'loop through each "LC" thing
recordVector = getVector(arr, i) 'get the row data
recordVector(j) = part 'replace the "LC" thing
c.Add recordVector 'add it to our results collection
Next part
Next i
'Prepare to dump the data back to the worksheet
rng.Clear
With rng.Parent
.Range( _
rng.Cells(1, 1), _
rng.Cells(1, 1).Offset(c.Count - 1, UBound(arr, 2) - 1)) _
.Value = getCollectionOfVectorsToArray(c)
End With
End Sub
'Helper method to return a vector representing our row data
Private Function getVector(dataArray, dataRecordIndex As Long)
Dim j As Long, tmpArr
ReDim tmpArr(LBound(dataArray, 2) To UBound(dataArray, 2))
For j = LBound(tmpArr) To UBound(tmpArr)
tmpArr(j) = dataArray(dataRecordIndex, j)
Next j
getVector = tmpArr
End Function
'Helper method to return an array from a collection of vectors
Function getCollectionOfVectorsToArray(c As Collection)
Dim i As Long, j As Long, arr
ReDim arr(1 To c.Count, LBound(c(1), 1) To UBound(c(1), 1))
For i = 1 To c.Count
For j = LBound(arr, 2) To UBound(arr, 2)
arr(i, j) = c(i)(j)
Next j
Next i
getCollectionOfVectorsToArray = arr
End Function
Edit:
Alternative "Range Insert" method.
It will be slower (although I made the number of discrete insert and copy operations be based on original row count, not some recursive sweep so it is not too bad) but is simpler to understand and so to perhaps tweak if needed. It should run in the order of a couple of minutes.
Sub ProcessData_RangeMethod()
Dim rng As Range
Dim colIndex As Long
Dim parts
Dim currRowIndex As Long
'replace with your code to assign the right range etc
Set rng = ActiveSheet.UsedRange
colIndex = 3 'replace with right column index, or work it out using Range.Find etc
Application.ScreenUpdating = False
Application.Calculation = xlCalculationManual
currRowIndex = 1
Do Until currRowIndex > rng.Rows.Count
parts = Split(rng.Cells(currRowIndex, colIndex), "/")
If UBound(parts) > 0 Then
rng.Range(rng.Cells(currRowIndex + 1, 1), rng.Cells(currRowIndex + UBound(parts), rng.Columns.Count)).Insert xlShiftDown
rng.Rows(currRowIndex).Copy rng.Range(rng.Cells(currRowIndex + 1, 1), rng.Cells(currRowIndex + UBound(parts), rng.Columns.Count))
rng.Range(rng.Cells(currRowIndex, colIndex), rng.Cells(currRowIndex + UBound(parts), colIndex)).Value = Application.Transpose(parts)
End If
currRowIndex = currRowIndex + 1 + UBound(parts)
Loop
Application.ScreenUpdating = True
Application.Calculation = xlCalculationAutomatic
End Sub