Iterating 100 cells takes too long - vba

In my excel VBA code, I need to move some data from a range to another sheet.
As of now, I'm iterating through the range and copying the values like this:
For offset = 0 To 101
ActiveWorkbook.Sheets(Sheet).Range("C3").offset(offset, 0).Value = ActiveSheet.Range("D4").offset(offset, 0).Value
Next offset
However, it takes almost a minute to iterate and copy the values for the 100 cells.
Would I be better off using Copy-Paste programatically, or is there a way to copy for the entire range at once? Something like:
ActiveWorkbook.Sheets(Sheet).Range("C3:C102").Value = ActiveSheet.Range("D4:D104").Value

You can read the entire range at once into a Variant array, and then write it back to another range. This is also quick, flickerless, and has the added bonus that you can code some operations on the data if you are so inclined.
Dim varDummy As Variant
varDummy = ActiveSheet.Range("D4:D104")
' Can insert code to do stuff with varDummy here
Workbook.Sheets(Sheet).Range("C3:C103") = varDummy
This I learned the hard way: Avoid Copy/Paste if at all possible! Copy and Paste use the clipboard. Other programs may read from / write to the clipboard while your code is running, which will cause wild, unpredictable results.
Also, it's generally a good idea to minimize the number of interactions between VBA and Excel, because they are slow. Having such interactions in a loop is multiply slow.

So, silly me did not try before posting here. Apparently, I can move data for an entire range this way:
Workbook.Sheets(Sheet).Range("C3:C102").Value = ActiveSheet.Range("D4:D104").Value
Its as fast as copy-paste without the switching of sheets.
Iterating through the range using a for loop takes about 45s for 100 cells, while the above two options are instant.

You can speed up code and stop flickering with:
Application.ScreenUpdating = False
'YOUR CODE
Application.ScreenUpdating = True
More: http://www.ozgrid.com/VBA/excel-macro-screen-flicker.htm

Columns("A:Z").Select
Selection.Copy
Sheets("Sheet2").Select
Range("A1").Select
ActiveSheet.Paste
That will copy columns A to Z from Sheet 1 to Sheet 2. This was generated by recording the macro. You can also apply it to ranges with something like this:
Range("D4:G14").Select
Selection.Copy
Sheets("Sheet2").Select
Range("D4").Select
ActiveSheet.Paste
Is this something like what you're after?
If you need anything specific and you can do it manually (e.g. copy and paste), record the macro to get the VBA code for it.

Copy and pasting has a decent amount of overhead in VBA, as does dealing with ranges like that. Its been a while since I have done VBA but if I recall correctly the fastest way to do something like this is to write the values you want into an array and then use the Resize function. So something like this:
Option Base 0
Dim firstrow as integer
Dim lastrow as integer
Dim valuesArray() as Long
Dim i as integer
//Set firstrow and lastrow however you deem appropriate
...
//Subtracing first row from last row gets you the needed size of the 0 based array
ReDim valuesArray(lastrow-firstrow)
for int i = 0 to (lastrow-firstrow)
valuesArray(i)=Cells(i+firstrow, COLUMNNUMBER).value
next i
Of course replace COLUMNNUMBER with whatever column it is you are iterating over. This should fill your array with your desired values. Then pick your destination cell and use Resize to put the values in. So if your destination cell is D4:
Range("D4").Resize(UBound(valuesArray)+1, 0).value = valuesArray
That write all the values in the array starting at D4 and going down to as many cells are in the array. Slightly more complicated but if you are going for speed I don't think I have ever come up with anything faster. Also I did this off the top of my head so please test and make sure that you don't cut off a cell here and there.

That OZGrid page has very useful info - http://www.ozgrid.com/VBA/SpeedingUpVBACode.htm
In my case, I need the formatting to be copied as well so I have been using this:
Sheet1.Range("A1:A200").Copy Destination:=Sheet2.Range("B1")
but was still having very slow execution - to the point of locking up the application - I finally found the problem - at some point in the past a number of empty text boxes got into my page - and while they were copied each time my code ran they were not erased by my code to clear the working area. The result was something like 4,500 empty text boxes - each of which was copy and pasted by even the code above.
If you use Edit - Go To... - Click on Special - then choose Objects - and you don't see anything that is good - if you see a bunch of objects that you were not aware of on your page that is not good.

Related

Excel macro - how to break wrapped text into rows for merged columns?

I have to import data from PDF to SAS, and one step involves converting the PDF data to excel spreadsheet before converting to text for simpler SAS import. Usually the PDF data converts fine into excel, with few errors. As I am trying to import older data, it is getting quite messy and some of the rows get wrapped in a single cell. I am trying to figure out if there is a macro possible which can help me fix this error in sheets without too much manual manipulation. I have never programmed in VBA before so I am quite new to excel macros.
Here is the example of messy data:
Here is the example of normal data:
(*Note the data values in the two images are different, just for example formatting)
I have tried working on a macro. For this, I copy the messy data into another sheet, and run the macro which outputs corrected data on a separate sheet, and then i copy the corrected data over the messy one in the original spreadsheet.
After trying to code the macro, I was unable to figure out how to tell excel to take the data in columns C,D,E,F which are all merged into one cell and break that wrapped text, and so on for other merged columns (as shown in messy data image).
Here is my current code that I got after watching some tutorials:
Sub Split_Text_to_Rows()
Dim splitVals1 As Variant
Dim splitVals2 As Variant
Dim totalVals As Long
Set sh1 = ThisWorkbook.Sheets(2)
Set sh2 = ThisWorkbook.Sheets(3)
sh2.Cells.Clear
lrow1 = sh1.Range("A65356").End(xlUp).Row
For j = 1 To lrow1
splitVals1 = Split(sh1.Cells(j, 1), Chr(10))
splitVals2 = Split(sh1.Cells(j, 2), Chr(10))
For i = LBound(splitVals1) To UBound(splitVals1)
lrow2 = sh2.Range("A65356").End(xlUp).Row
sh2.Cells(lrow2 + 1, 1) = splitVals1(i)
Next i
For k = LBound(splitVals2) To UBound(splitVals2)
lrow3 = sh2.Range("B65356").End(xlUp).Row
sh2.Cells(lrow3 + 1, 2) = splitVals2(k)
Next k
Next j
End Sub
As you can see, my code is also quite messy. Although, I got the code to work for columns A and B, when I get to column C - "Motor Vehicle Theft" and so on, I am not sure how to separate that wrapped text since they are merged in columns C,D,E,F. I would also like to keep the columns I to Q as two merged rows even after macro splits 1 row into 2 (shown in normal data image) and then continue splitting cells till column Z.
Any tips would be helpful! Please let me know if more information or clarification is needed.
I often find that the best approach is to first paste the data into Word, do some clean-up there, format it as a Word table, and then transfer it into Excel. The reason is that Word has very powerful find/replace features which allow you to quickly convert a mess into something sensible. Since you didn't provide example data I could paste in, I randomly found a pdf on the web to show one approach. The key in this case was noticing that each column begins with a space followed by a digit. So I did a search for " ^#" (a space followed by 'any digit') and replaced it by "^t" (tab character). Next, I used Word's 'Convert to Table' feature, and after that the data table is ready for pasting into Excel.

Range SpecialCells ClearContents clears whole sheet instead

I have a sheet in Excel 2010 which is setup as a pseudo form (I didn't create it, I'm just trying to fix it) so formatting suggests that the user can only enter in certain cells. Depending on certain functionality these areas need to be reset, i.e. cleared although formulae and standard/conditional formatting need to be kept. I have defined each of these cells/ranges as named ranges so I can easily loop through them using the following code: -
Public Sub ResetDetailSheet()
Dim nm As Name
With ThisWorkbook
For Each nm In .Names
If Left(nm.Name, 9) = "nmrDetail" Then
Range(nm.Name).SpecialCells(xlCellTypeConstants).ClearContents
End If
Next
End With
End Sub
For some reason instead of clearing the constants from the specific range it is clearing constants from the entire sheet so I am losing all titles/headings. Formulae and standard/conditional formatting are staying as expected.
What am I doing wrong?!?!
As a test using the immediate window I tried clearing a specific cell, e.g.
Range("G7").SpecialCells(xlCellTypeConstants).ClearContents
But this still cleared all constants from the entire sheet.
What am I missing? I don't understand. Maybe I'm being dumb.
Sorry, I can't upload an example. This place is pretty locked down.
Range({any single cell}).SpecialCells({whatever}) seems to work off the entire sheet.
Range({more than one cell}).SpecialCells({whatever}) seems to work off the specified cells.
So, make sure your range has more than a single cell before you clear it - if the range is only a single cell, then check if it .HasFormula; if that's the case then its .Value isn't a constant:
With ThisWorkbook
For Each nm In .Names
If Left(nm.Name, 9) = "nmrDetail" Then
If nm.RefersToRange.Count > 1 Then
nm.RefersToRange.SpecialCells(xlCellTypeConstants).ClearContents
ElseIf Not nm.RefersToRange.HasFormula Then
nm.RefersToRange.ClearContents
End If
End If
Next
End With
Note that I'm using Name.RefersToRange instead of fetching the range by name off the active sheet.

Excel running slowly when Excel the active window

I couldn't find the answer to this issue anywhere, so I do hope you guys can help me. My excel macro goes through a couple iterations of data. It autofilters a source file, takes out information, works with the data, and does so again for about 50 times - once per person. Here's some code of what I mean, all the individual submethods work just fine and are pretty damn fast:
For j = 1 To names.Count
'filter the source by name, generate sheet
FilterName names(j)
'prepare data with the necessary dates
FillMasterDates dates(), j
Dim i As Long
Dim ending As Long
ending = Sheets("Daten").Rows.End(xlDown).Row
Dim cellvalue As String
'check dates, etc
For i = 1 To ending
cellvalue = Sheets("Daten").Cells(i, 1)
If cellvalue = "" Then
Exit For
End If
ColorCell (i)
FilterDate CStr(dates(i)), names(j)
Next i
'user data has been successfully gathered, copy over to final sheet
FillColumns j
Next j
The whole code takes about 4~ seconds to run (given that I have about 2000 rows and I create a new sheet for 50~ people), which is fine. The baffling thing is that when Excel stays my active window despite using Application.ScreenUpdating = False (earlier in the macro, but still active at this point), the necessary time to run the macro goes up to a staggering 25~ seconds. Same input, same output. So to put it simply - run macro, tab out of excel - macro needs about 4-5 seconds to run. run macro but stay in excel and wait - 25 seconds.
I've tried Application.WindowState = Application.WindowState, ActiveWindow.SmallScroll, DoEvents, Application.CalculateFull(). I tried different calculation settings, but I do not really use any of the formula calculations innate to Excel - I have to use Excel as an interface because the source file is an *.xls file and the final output has to remain in this format.
If you need me to provide more code snippets to make sense of it, ask away. I've been stumped for a good two days now.
You could always try a couple more lines to disable the calculations and alerts etc.
Application.ScreenUpdating = false
Application.Calculations = xlManual
Application.DisplayAlerts = False
However if you really want to bypass all the background nonsense excel seems to go through dont access the sheet directly through a loop, this concept maybe tricky if your not used to it but its worth every bit, and will speed up your code so fast you will wonder why you never did it in the first place.
I dont have your code so ill just give an example of how it works
Dim RangeArray as Variant 'This will store your range as a values array
RangeArray = Sheet1.Range("A1:G100000").Value 'this will put the entire ranges values into the array
If Not IsArray(RangeArray) Then ExitSub 'If your range is only 1 cell it will not create an array so be careful, handle this as needed
'This Array always starts lowerbound 1, RangeArray(1,1) = First Cell
Now with this you can loop through your data and manipulate and modify the array just like you would with a cell or a range except there is no overhead, its just values and not objects .
Once you have done what you need all you need to do then is put the values back into the sheets range
Sheet1.Range("A1:G100000").value = RangeArray
And thats it, very simple and very effective, and this transfer from array to range is immediate no matter how big it is.
Just let me know if this helps
Thanks
Paul S
---------------NEW MESSAGE-------------------
You could try something which maybe a little excessive and risky, if your only getting this problem while the window is active and displayed how about making it invisible, the problem is if your code fails and you fail to trap an error it will remain invisible until you goto taskmanager and close it there.
Application.Visible = false
This should deactivate the window too (although i have never tested that)
this should simulate you hiding the window and just bring it back when your code has finished..
---------------NEW MESSAGE-------------------
Application.Windowstate = xlMinimized
This should do the trick :D, should have mentioned this first haha
I also just saw that you tried something similar, but the code is incorrect there, try this one

Making a formula a VBA Macro

I got the following code
=LEFT(A2, MIN(ROW(INDIRECT("1:"&LEN(A2)))+(((CODE(MID(UPPER(A2),
ROW(INDIRECT("1:"&LEN(A2))), 1))>64)*(CODE(MID(UPPER(A2),
ROW(INDIRECT("1:"&LEN(A2))), 1))<91))+
((CODE(MID(A2, ROW(INDIRECT("1:"&LEN(A2))), 1))>47)*
(CODE(MID(A2, ROW(INDIRECT("1:"&LEN(A2))), 1))<58)))*1E+99)-1)
I have this code and a few others, but how can I make it into a macro applicable to my entire workbook? I know its probably the same as a macro in terms of time, but I eventually want to loop it throughout a directory and would help automate a process. Is there a way to make this a macro for my workbook?
The crudest quickest way would be something like this:
Range("J2:J5000").Formula = "=LEFT(A2, MIN(ROW(INDIRECT(""1:""&LEN(A2)))+(((CODE(MID(UPPER(A2), ROW(INDIRECT(""1:""&LEN(A2))), 1))>64)*(CODE(MID(UPPER(A2), ROW(INDIRECT(""1:""&LEN(A2))), 1))<91))+((CODE(MID(A2, ROW(INDIRECT(""1:""&LEN(A2))), 1))>47)*(CODE(MID(A2, ROW(INDIRECT(""1:""&LEN(A2))), 1))<58)))*1E+99)-1)"
Which will put your exact formula in the range (and update itself according to the row reference). Obviously the reference to column J can be changed and the 5000 can be made dynamic using rows.count).end(xlup).row but without knowing which columns to play with I just had to take a stab at a crude solution.
However depending on what your "symbols" could be a solution using the split command would most likely be the better choice. Can you post more direction on this? Then I can edit this answer and add a code solution in for that for you.
Also include some sample data and expected results, maybe 10 rows worth to give a good set for testing
For an example of how the split command works select one of the cells with data in it that you need to split on the underscore and go to the debug window in the VBE (CTRL-G) and enter this (including the question mark) then press enter.
?split(Activecell.text,"_")(0)
Now update the 0 to 1 and press enter. This will show you how this command works, it splits a string to an array based on the delimiter you give it.
EDIT:
This code will do what you want, Notice how Split is being used.
Function GetFirstPart(SplitString As String)
Dim PosibleSplits As Variant, X As Long
PossibleSplits = Array("_", "+", "-")
For X = LBound(PossibleSplits) To UBound(PossibleSplits)
If Len(SplitString) <> Len(Split(SplitString, PossibleSplits(X))(0)) Then
GetFirstPart = Split(SplitString, PossibleSplits(X))(0)
Exit For
End If
Next
End Function
Use it by pasting the code into a module then in your sheet use it the same as any other formula =GetFirstPart(A1) where A1 has the string to split, drag down as far as your data goes.
You can add other delimiters in this line PossibleSplits = Array("_", "+", "-")

Why is my conditional format offset when added by VBA?

I was trying to add conditional formats like this:
If expression =($G5<>"") then make set interior green, use this for $A$5:$H$25.
Tried this, worked fine, as expected, then tried to adapt this as VBA-Code with following code, which is working, but not as expected:
With ActiveSheet.UsedRange.Offset(1)
.FormatConditions.Delete
'set used row range to green interior color, if "Erledigt Datum" is not empty
With .FormatConditions.Add(Type:=xlExpression, _
Formula1:="=($" & cstrDefaultProgressColumn & _
.row & "<>"""")")
.Interior.ColorIndex = 4
End With
End With
The Problem is, .row is providing the right row while in debug, however my added conditional-formula seems to be one or more rows off - depending on my solution for setting the row. So I am ending up with a conditional formatting, which has an offset to the row, which should have been formatted.
In the dialog it is then =($G6<>"") or G3 or G100310 or something like this. But not my desired G5.
Setting the row has to be dynamicall, because this is used to setup conditional formats on different worksheets, which can have their data starting at different rows.
I was suspecting my With arrangement, but it did not fix this problem.
edit: To be more specific, this is NOT a UsedRange problem, having the same trouble with this:
Dim rngData As Range
Set rngData = ActiveSheet.Range("A:H") 'ActiveSheet.UsedRange.Offset(1)
rngData.FormatConditions.Delete
With rngData.FormatConditions.Add(Type:=xlExpression, _
Formula1:="=($" & cstrDefaultProgressColumn & _
1 & "<>"""")")
.Interior.ColorIndex = 4
End With
My Data looks like this:
1 -> empty cells
2 -> empty cells
3 -> empty cells
4 -> TitleCols -> A;B;C;...;H
5 -> Data to TitleCols
. .
. .
. .
25
When I execute this edited code on Excel 2007 and lookup the formula in the conditional dialog it is =($G1048571<>"") - it should be =($G1<>""), then everything works fine.
Whats even more strange - this is an edited version of a fine working code, which used to add conditional formats for each row. But then I realized, that it's possible to write an expression, which formats a whole row or parts of it - thought this would be adapted in a minute, and now this ^^
edit: Additional task informations
I use conditional formatting here, because this functions shall setup a table to react on user input. So, if properly setup and a user edits some cell in my conditionalized column of this tabel, the corresponding row will turn green for the used range of rows.
Now, because there might be rows before the main header-row and there might be a various number of data-columns, and also the targeted column may change, I do of course use some specific informations.
To keep them minimal, I do use NamedRanges to determine the correct offset and to determine the correct DefaultProgessColumn.
GetTitleRow is used to determine the header-row by NamedRange or header-contents.
With ActiveSheet.UsedRange.Offset(GetTitleRow(ActiveSheet.UsedRange) - _
ActiveSheet.UsedRange.Rows(1).row + 1)
Corrected my Formula1, because I found the construct before not well formed.
Formula1:="=(" & Cells(.row, _
Range(strMatchCol1).Column).Address(RowAbsolute:=False) & _
"<>"""")"
strMatchCol1 - is the name of a range.
Got it, lol. Set the ActiveCell before doing the grunt work...
ActiveSheet.Range("A1").Activate
Excel is pulling its automagic range adjusting which is throwing off the formula when the FromatCondition is added.
The reason that Conditional Formatting and Data Validation exhibit this strange behavior is because the formulas they use are outside the normal calculation chain. They have to be so that you can refer to the active cell in the formula. If you're in G1, you can't type =G1="" because you'll create a circular reference. But in CF or DV, you can type that formula. Those formulas are disassociated with the current cell unlike real formulas.
When you enter a CF formula, it's always relative to the active cell. If, in CF, you make a formula
=ISBLANK($G2)
and you're in A5, Excel converts it to
=ISBLANK(R[-3]C7)
and when that gets put into the CF, it ends up being relative to the cell it's applied to. So in row 2, the formula comes out to
=ISBLANK($G655536)
(for Excel 2003). It offsets -3 rows and that wraps to the bottom of the spreadsheet.
You can use Application.ConvertFormula to make the formula relative to some other cell. If I'm in row 5 and the start of my range is in row 2, I make the formula relative to row 8. That way the R[-3] will put the formula in A5 as $G5 (three rows up from A8).
Sub test()
Dim cstrDefaultProgressColumn As String
Dim sFormula As String
cstrDefaultProgressColumn = "$G"
With ActiveSheet.UsedRange.Offset(1)
.FormatConditions.Delete
'set used row range to green interior color, if "Erledigt Datum" is not empty
'Build formula
sFormula = "=ISBLANK(" & cstrDefaultProgressColumn & .Row & ")"
'convert to r1c1
sFormula = Application.ConvertFormula(sFormula, xlA1, xlR1C1)
'convert to a1 and make relative
sFormula = Application.ConvertFormula(sFormula, xlR1C1, xlA1, , ActiveCell.Offset(ActiveCell.Row - .Cells(1).Row))
With .FormatConditions.Add(Type:=xlExpression, _
Formula1:=sFormula)
.Interior.ColorIndex = 4
End With
End With
End Sub
I only offset .Cells(1) row-wise because the column is absolute in this example. If both row and column are relative in your CF formula, you need more offsetting. Also, this only works if the active cell is below the first cell in your range. To make it more general purpose, you would have to determine where the activecell is relative to the range and offset appropriately. If the offset put you above row 1, you would need to code it so that it referred to a cell nearer the bottom of the total number of rows for your version of Excel.
If you thought selecting was a bit of a kludge, I'm sure you'll agree that this is worse. Even though I abhor unnecessary Selecting and Activating, Conditional Formatting and Data Validation are two places where it's a necessary evil.
A brief example:
Sub Format_Range()
Dim oRange As Range
Dim iRange_Rows As Integer
Dim iCnt As Integer
'First, create a named range manually in Excel (eg. "FORMAT_RANGE")
'In your case that would be range "$A$5:$H$25".
'You only need to do this once,
'through VBA you can afterwards dynamically adapt size + location at any time.
'If you don't feel comfortable with that, you can create headers
'and look for the headers dynamically in the sheet to retrieve
'their position dynamically too.
'Setting this range makes it independent
'from which sheet in the workbook is active
'No unnecessary .Activate is needed and certainly no hard coded "A1" cell.
'(which makes it more potentially subject to bugs later on)
Set oRange = ThisWorkbook.Names("FORMAT_RANGE").RefersToRange
iRange_Rows = oRange.Rows.Count
For iCnt = 1 To iRange_Rows
If oRange(iCnt, 1) <> oRange(iCnt, 2) Then
oRange(iCnt, 2).Interior.ColorIndex = 4
End If
Next iCnt
End Sub
Regarding my comments given on the other reply:
If you have to do this for many rows, it is definitely faster to load the the entire range into memory (an array) and check the conditions within the array, after which you do the writing on those cells that need to be written (formatted).
I could agree that this technique is not "necessary" in this case - however it is good practise because it is flexible for many (any type of) customizations afterwards and easier to debug (using the immediate / locals / watches window).
I'm not a fan of Offset although I don't state it doesn't work as it should and in some limited scenarios I could say that the chance for problems "could" be small: I experienced that some business users tend to use it constantly (here offset +3, there offset -3, then again -2, etc...); although it is easy to write, I can tell you it is hell to revise. It is also very often subject to bugs when changes are made by end users.
I am very much "for" the use of headers (although I'm also a fan of reducing database capabilities for Excel, because for many it results in avoiding Access), because it will allow you very much flexibility. Even when I used columns 1 and 2; better is it to retrieve the column nr dynamically based on the location of the named range of the header. If then another column is inserted, no bugs will appear.
Last but not least, it may sound exaggerated, but the last time, I used a class module with properties and functions to perform all retrievals of potential data within each sheet dynamically, perform checks on all bugs I could think of and some additional functions to execute specific tasks.
So if you need many types of data from a specific sheet, you can instantiate that class and have all the data at your disposal, accessible through defined functions. I haven't noticed anyone doing it so far, but it gives you few trouble despite a little bit more work (you can use the same principles again over and over).
Now I don't think that this is what you need; but there may come a day that you need to make large tools for end users who don't know how it works but will complain a lot about things because of something they might have done themselves (even when it's not your "fault"); it's good to keep this in mind.