Excel, VBA: Clear memory - vba

I have a macro for Excel, nothing too fancy, no objects, just plain old cycles with formulas. Basically it's this:
I cycle through rows in one column, assigning pretty big SUMPRODUCT functions to the cells without executing them. Once I cycled through the row I do Range.Value = Range.Value twice to execute the formulas and to save result as value. Then I go for the next column.
With each column memory used by Excel increases significantly and after the macro is done and I save the file - the memory is still used (if I can trust Task Manager). After I close and reopen the file - all memory is, of course, freed and I can continue to work. I've reached a point where I can't process all the columns I need on a 32 bit Excel due to memory limit.
I was wondering, maybe there's some trick I could implement between the columns to make Excel forget whatever it keeps remembering? I don't really care if it will hit performance (in case it has cached anything useful for further calculations), I just need to keep memory usage from growing too huge to process. I googled a bit but all the advises are related to setting big objects to nothing - I don't really set much of anything to anything, just cycle through, so that probably won't apply to my case.
Code is like this (don't have it on me right now, but it's a general simplified version of what is in it):
for i = 1 to 12
ThisWorkbook.ActiveWorksheet.Range(Cells(1,i),Cells(x,i)).Font.Color = vbWhite
for m = 1 to x
ThisWorkbook.ActiveWorksheet.Cells(m, i).Formula = "'=SUMPRODUCT(blah)"
next m
ThisWorkbook.ActiveWorksheet.Range(Cells(1,i),Cells(x,i)).Value = ThisWorkbook.ActiveWorksheet.Range(Cells(1,i),Cells(x,i)).Value
ThisWorkbook.ActiveWorksheet.Range(Cells(1,i),Cells(x,i)).Value = ThisWorkbook.ActiveWorksheet.Range(Cells(1,i),Cells(x,i)).Value
ThisWorkbook.ActiveWorksheet.Range(Cells(1,i),Cells(x,i)).Font.Color = vbBlack
next i
Basically, I color them white so I don't see the messy function text, add function, execute it, save as values, color text black.
Important addition: I use SUMPRODUCT to sum some cells from closed files.

This isn't really enough code to help you improve the efficiency. I can point out a few tips for you now though.
The first thing to remember is to switch application screen updating to false at the start of the macro.
Application.ScreenUpdating = False
then switch it back just before the end of the macro code
Application.ScreenUpdating = True
Changing the font color is unnecessary now. Your screen will be locked for refreshing while the macro is being executed so you will not be seeing the changes until the macro has finished working.
Consider, disabling Events too.
Application.EnableEvents = False and Application.EnableEvents = true same idea as with the screen updating.
The second thing is to make use of Evaluate() function. Please google it and read about how efficient that function can be.
Instead of putting a function into cell and then doing .Value = .Value you can just Evaluate() an expression and return the result straight to the cell which is much faster!
So you could do something like
for i = 1 to 12
ThisWorkbook.ActiveWorksheet.Range(Cells(1,i),Cells(x,i)).Value = Evaluate("=SUM(blah)")
next i
I hope you get the idea.
Here are a few links with tips on how to speed up your macro:
http://msdn.microsoft.com/en-us/library/office/ff193019.aspx
http://fastexcel.wordpress.com/2011/11/02/evaluate-functions-and-formulas-fun-how-to-make-excels-evaluate-method-twice-as-fast/
http://www.soa.org/news-and-publications/newsletters/compact/2012/january/com-2012-iss42-roper.aspx
http://xlvba.fr.yuku.com/topic/278/The-Evaluate-Method#.UmZP6Pmkp8s

I know this is an old question but a way that seems to work is saving the Workbook. Run ThisWorkbook.Save every once in a while and if you monitor EXCEL in the Task Manager you will notice that it clears up a significant amount of memory.
It seems as though Excel may be caching certain function calls or something and saving clears this cache.

Related

VBA Calculations Turning Back On Mid Execution

I'm running a large macro that includes various sub queries and functions.
Despite having the standard code to turn off calculations while the macro is running, somehow calculations are turned back on during the code execution.
Application.ScreenUpdating = False
Application.Calculation = xlCalculationManual
I've added the above code the beginning of each major sub but still have the same issue. There's one point in the macro where I calculate the application to update a worksheet stored value, but I can't imagine that's what's causing it.
application.Calculate
Trying to find if there's a work around.

Excel running slowly when Excel the active window

I couldn't find the answer to this issue anywhere, so I do hope you guys can help me. My excel macro goes through a couple iterations of data. It autofilters a source file, takes out information, works with the data, and does so again for about 50 times - once per person. Here's some code of what I mean, all the individual submethods work just fine and are pretty damn fast:
For j = 1 To names.Count
'filter the source by name, generate sheet
FilterName names(j)
'prepare data with the necessary dates
FillMasterDates dates(), j
Dim i As Long
Dim ending As Long
ending = Sheets("Daten").Rows.End(xlDown).Row
Dim cellvalue As String
'check dates, etc
For i = 1 To ending
cellvalue = Sheets("Daten").Cells(i, 1)
If cellvalue = "" Then
Exit For
End If
ColorCell (i)
FilterDate CStr(dates(i)), names(j)
Next i
'user data has been successfully gathered, copy over to final sheet
FillColumns j
Next j
The whole code takes about 4~ seconds to run (given that I have about 2000 rows and I create a new sheet for 50~ people), which is fine. The baffling thing is that when Excel stays my active window despite using Application.ScreenUpdating = False (earlier in the macro, but still active at this point), the necessary time to run the macro goes up to a staggering 25~ seconds. Same input, same output. So to put it simply - run macro, tab out of excel - macro needs about 4-5 seconds to run. run macro but stay in excel and wait - 25 seconds.
I've tried Application.WindowState = Application.WindowState, ActiveWindow.SmallScroll, DoEvents, Application.CalculateFull(). I tried different calculation settings, but I do not really use any of the formula calculations innate to Excel - I have to use Excel as an interface because the source file is an *.xls file and the final output has to remain in this format.
If you need me to provide more code snippets to make sense of it, ask away. I've been stumped for a good two days now.
You could always try a couple more lines to disable the calculations and alerts etc.
Application.ScreenUpdating = false
Application.Calculations = xlManual
Application.DisplayAlerts = False
However if you really want to bypass all the background nonsense excel seems to go through dont access the sheet directly through a loop, this concept maybe tricky if your not used to it but its worth every bit, and will speed up your code so fast you will wonder why you never did it in the first place.
I dont have your code so ill just give an example of how it works
Dim RangeArray as Variant 'This will store your range as a values array
RangeArray = Sheet1.Range("A1:G100000").Value 'this will put the entire ranges values into the array
If Not IsArray(RangeArray) Then ExitSub 'If your range is only 1 cell it will not create an array so be careful, handle this as needed
'This Array always starts lowerbound 1, RangeArray(1,1) = First Cell
Now with this you can loop through your data and manipulate and modify the array just like you would with a cell or a range except there is no overhead, its just values and not objects .
Once you have done what you need all you need to do then is put the values back into the sheets range
Sheet1.Range("A1:G100000").value = RangeArray
And thats it, very simple and very effective, and this transfer from array to range is immediate no matter how big it is.
Just let me know if this helps
Thanks
Paul S
---------------NEW MESSAGE-------------------
You could try something which maybe a little excessive and risky, if your only getting this problem while the window is active and displayed how about making it invisible, the problem is if your code fails and you fail to trap an error it will remain invisible until you goto taskmanager and close it there.
Application.Visible = false
This should deactivate the window too (although i have never tested that)
this should simulate you hiding the window and just bring it back when your code has finished..
---------------NEW MESSAGE-------------------
Application.Windowstate = xlMinimized
This should do the trick :D, should have mentioned this first haha
I also just saw that you tried something similar, but the code is incorrect there, try this one

Running a VBA Excel Macro and getting screen flickering even after rightly setting Application.ScreenUpdating to True and False

I have written a macro which is quite big and has several Sub functions. The macro pulls external data and perform Calculations on them and produces results on the sheet along with a graph.
The problem is that the macro runs absolutely fine with smaller data. But when I pull a databank of say 1500 entries, the screen starts flickering and the program hangs. It doesn't even give any error message.
Had it been any faults in loop, the program shouldn't have worked for 200 entries. But it does for a lot of datas except the very long ones. Also the size of arrays used are set up to 2000, and Application.ScreenUpdating has been turned False at the start and True at the end in every Sub.
Another important information as to how the program works:
The codes are connected to a Userform. So if one optionbutton and another entry is true, a particular process (Sub) is happening one at a time. So for each sub I have given Application.ScreenUpdating as False and True. Technically out of 15 subs, perhaps only 5 are running at a time in a module. I have kept every case in the same module as it is easier to handle that way. However, few subs run at a time.

Unused UDF being called when doing CalculateFullRebuild

I have some User Defined Functions in an Excel book. I used them for a while but, after a while, I deleted the calls to these functions from the cells because I found a better way to accomplish the same task (I didn't delete the function definition itself in the VBA editor). So, these functions are no longer being called neither in the book nor from any VBA code, I checked it using a search to be 100% sure.
Now I'm doing some review on my code and I noticed something strange: in a Sub procedure in the same workbook (which has nothing to do with these functions) I call Application.CalculateFullRebuild. When this happens those UDF get called, I can see it by setting a break point inside the UDF.
I'd like to know why is it happening and what can be done to avoid it, as it is slowing that Sub unnecessarily.
Thanks!
Application.CalculateFullRebuild MSDN reference has this to say:
The CalculateFullRebuild method is similar to re-entering all formulas. ... [When run] a full calculation of the data in all open workbooks is performed and the dependencies are rebuilt.
Further MSDN reference states:
Causes Excel to rebuild the dependency tree and the calculation chain
This means that any UDFs in the module code or sheet code will be recalculated because Excel is rebuilding and testing functions for dependency and use in the calculation chain.
If you are looking for a way to simply manually calculate the existing formulas in the sheet via your Sub, you can use 'Application.Calculate' (MSDN):
Application.Calculate 'for all open Sheets
Sheets("Name of Sheet").Calculate 'Specific Sheet
Sheets("Name of Sheet").Range("Name of Range").Calculate 'Specific Range
The system is working as it should. Consider:
Function qwerty() As String
qwerty = "qwerty"
MsgBox "XX"
End Function
It is non-Volatile and has no arguments. It will be calculated at the time it is entered in a worksheet cell. Application.Calculate may cause it to be calculated once, however:
Sub ytrewq()
Application.CalculateFullRebuild
End Sub
will cause the UDF to be re-calculated each time ytrewq is run.
To the moment my approach has been commenting all the code inside the UDF with two objectives: increasing speed on one side and checking if any side effect happened on the other side. To the moment, I have not observed any side effect, so more to the point that they are not being used anywhere.
Right now the application I'm developing is working quite well, but I'll try the solutions you're proposing just out of curiosity. By bets are on either it's being used somewhere hidden and forgoten or simply that I have some rubbish inide the workbook structure that is not getting cleaned.
Thanks!
Update
Tried again the next day and those UDF are no longer being called. Thus, I'll have to assume that something odd was going on with Excel that went away when I restarted it.
Anyway, thanks a lot for the Application.Caller thing, which I didn't know about.

Iterating 100 cells takes too long

In my excel VBA code, I need to move some data from a range to another sheet.
As of now, I'm iterating through the range and copying the values like this:
For offset = 0 To 101
ActiveWorkbook.Sheets(Sheet).Range("C3").offset(offset, 0).Value = ActiveSheet.Range("D4").offset(offset, 0).Value
Next offset
However, it takes almost a minute to iterate and copy the values for the 100 cells.
Would I be better off using Copy-Paste programatically, or is there a way to copy for the entire range at once? Something like:
ActiveWorkbook.Sheets(Sheet).Range("C3:C102").Value = ActiveSheet.Range("D4:D104").Value
You can read the entire range at once into a Variant array, and then write it back to another range. This is also quick, flickerless, and has the added bonus that you can code some operations on the data if you are so inclined.
Dim varDummy As Variant
varDummy = ActiveSheet.Range("D4:D104")
' Can insert code to do stuff with varDummy here
Workbook.Sheets(Sheet).Range("C3:C103") = varDummy
This I learned the hard way: Avoid Copy/Paste if at all possible! Copy and Paste use the clipboard. Other programs may read from / write to the clipboard while your code is running, which will cause wild, unpredictable results.
Also, it's generally a good idea to minimize the number of interactions between VBA and Excel, because they are slow. Having such interactions in a loop is multiply slow.
So, silly me did not try before posting here. Apparently, I can move data for an entire range this way:
Workbook.Sheets(Sheet).Range("C3:C102").Value = ActiveSheet.Range("D4:D104").Value
Its as fast as copy-paste without the switching of sheets.
Iterating through the range using a for loop takes about 45s for 100 cells, while the above two options are instant.
You can speed up code and stop flickering with:
Application.ScreenUpdating = False
'YOUR CODE
Application.ScreenUpdating = True
More: http://www.ozgrid.com/VBA/excel-macro-screen-flicker.htm
Columns("A:Z").Select
Selection.Copy
Sheets("Sheet2").Select
Range("A1").Select
ActiveSheet.Paste
That will copy columns A to Z from Sheet 1 to Sheet 2. This was generated by recording the macro. You can also apply it to ranges with something like this:
Range("D4:G14").Select
Selection.Copy
Sheets("Sheet2").Select
Range("D4").Select
ActiveSheet.Paste
Is this something like what you're after?
If you need anything specific and you can do it manually (e.g. copy and paste), record the macro to get the VBA code for it.
Copy and pasting has a decent amount of overhead in VBA, as does dealing with ranges like that. Its been a while since I have done VBA but if I recall correctly the fastest way to do something like this is to write the values you want into an array and then use the Resize function. So something like this:
Option Base 0
Dim firstrow as integer
Dim lastrow as integer
Dim valuesArray() as Long
Dim i as integer
//Set firstrow and lastrow however you deem appropriate
...
//Subtracing first row from last row gets you the needed size of the 0 based array
ReDim valuesArray(lastrow-firstrow)
for int i = 0 to (lastrow-firstrow)
valuesArray(i)=Cells(i+firstrow, COLUMNNUMBER).value
next i
Of course replace COLUMNNUMBER with whatever column it is you are iterating over. This should fill your array with your desired values. Then pick your destination cell and use Resize to put the values in. So if your destination cell is D4:
Range("D4").Resize(UBound(valuesArray)+1, 0).value = valuesArray
That write all the values in the array starting at D4 and going down to as many cells are in the array. Slightly more complicated but if you are going for speed I don't think I have ever come up with anything faster. Also I did this off the top of my head so please test and make sure that you don't cut off a cell here and there.
That OZGrid page has very useful info - http://www.ozgrid.com/VBA/SpeedingUpVBACode.htm
In my case, I need the formatting to be copied as well so I have been using this:
Sheet1.Range("A1:A200").Copy Destination:=Sheet2.Range("B1")
but was still having very slow execution - to the point of locking up the application - I finally found the problem - at some point in the past a number of empty text boxes got into my page - and while they were copied each time my code ran they were not erased by my code to clear the working area. The result was something like 4,500 empty text boxes - each of which was copy and pasted by even the code above.
If you use Edit - Go To... - Click on Special - then choose Objects - and you don't see anything that is good - if you see a bunch of objects that you were not aware of on your page that is not good.