I am working with a huge excel file that is updated with a set of macros. In the excel file there are also a large number of graphs to ensure easy output checks.
However, when I re-calculate the workbook it is extremely slow.
My question is: Do these graphs contribute to slowing down the calculation of the model? If so, is there a quick VBA way to only update graphs at the end of the overall calculation?
Without seeing your workbook this is hard to answer.
Most likely, it is not the charts (is that what you call "graphs") that are slowing down the recalc, but inefficient formulas.
Check the chart data sources. If they point to worksheet cells, then all is good. If they point to named ranges / named formulas, then check what these formulas are.
Recalculation is affected by
volatile formulas like Today(), Now(), Indirect(), Offset() and a few others
inefficient formulas that needlessly repeat calculations that have already been performed, typically done in running totals
And example of this would be
=Sum(A$1:A2) copied down, like in this screenshot
In each row, the calculation starts in row 1 and goes down to the current row. This is a waste of effort.
A much more efficient formula is in column C, where just the value from the row above is added to the value of the current row.
=SUM(C1,A2)
These details can make a heck of a difference.
For more information you may want to refer to Charles Williams' site http://www.decisionmodels.com/calcsecrets.htm and the pages linked from there.
It's a complex subject and can probably not be addressed in a simple answer to a seemingly simple question.
Related
I have two spread sheets, Sheet1 & Sheet2.
Sheet2 will be a paste of a lot of information received from a third party with a unique ID in column B, I’m currently using vlookup but the spread sheet is getting very large and taking a long time to load up and save.
In sheet 1 I'd like to use a macro to insert the vlookup result from a MACRO matching the unique ID in column B and displaying the value of column c from Sheet2.
Hope this makes sense, thank you!!
Tried a lot:
Sub UPDATE()
Dim s1 As Worksheet, s2 As Worksheet
Set s1 = Sheets("Sheet1")
Set s2 = Sheets("Sheet2")
v1 = s1.Range("A2")
v2 = s1.Range("B2")
s2.Activate
For Each r In Intersect(ActiveSheet.UsedRange, Range("B:B"))
If r.Value = v1 Then
r.Offset(0, 1).Value = v2
End If
Next
End Sub
If you can post a non-question in the question field, then can I post a non-answer in the answer field?
The help you need right now is not the help you kinda, sorta, but not really asked for. You just don't know it yet.
If I stop judging your intent between the lines and ignore your insulting response to a genuinely helpful comment. If I take the time and energy to focus on your problem and semantically transform your demand into a question. Then you still will not get what you asked for.
I shall demonstrate.
A large workbook with 2 sheets loads and saves slowly. Presumably because of its size and numerous vloookup functions. To improve performance you want a macro on sheet 1 to “insert the vlookup result from a MACRO” on sheet 2.
This may surprise you. If a formula is already optimized then replacing it with another formula will not improve performance. Even if that formula is a vlookup and even if that vlookup is placed by a MACRO. In fact, it will be slower because you're still calculating the vloookups and now you've added the overhead of running the macro on sheet 1 and the MACRO on sheet 2. Seems to me like you might get a performance boost by changing your calculation settings to calculate before save and do not refresh or calculate when opening. That won't improve your save time but opening and working with document should be faster.
That's an answer you didn't ask for but it's weak and really doesn't do much to get to the root. I'm really trying to help so I won't stop yet I'll dig through the code you shared and focus on the pieces where you think the problems are. Oh, you didn't say where they could be. No big deal, it's a short block and I'm almost VBA literate, I'm sure I won't have any issues fixing bugged code as I step through it in my head without access to test data or your runtime environment. Piece of cake!
Umm… what are you trying to do? I think I understand what you said you want; but I don't see how you actually went about doing it. I'll work with what you gave me but it isn't much so I hope you don't have high expectations.
Sheet 1 and 2 are set. 2 is activated and this sub starts working with it; therefore, this must be the sheet 2 MACRO. How does this interact with the sheet 1 macro? Are you certain there's no problem with that one?
Hrmmmm, I've read a little about the speed of intersection. If i recall correctly, it's supposed to be quite fast, that might be a good choice. I'll have to remember that. Uh, B:B, the whole column? You can often gain performance by narrowing constraints, but I am not certain in this case. Maybe intersection is a of short circuiting every row that follows, i should look into that.
I have to assume v1 and v2 are variants because they aren't declared and no information is given regarding option explicit or publicly scoped variables. You have to be careful variants because they can change type and cause you to throw errors all day. Especially if your code isn't clear enough to identify them on sight. Case and point, you assign 2 variants equal to a range, is this intentional or did you forget the word set? Further down you assign the value of a range to the same variant leaving me to guess about your intent.
Does your data use strings? If it does then value2 should be faster than value.
I'm confused about your loop I don't know why you have it or why you're do what you do. You said you have vlookup results pasted in B - so it makes sense that you read it. But why in a loop? You calculated a valid range, why not write if to an array in one step instead of potentially hundreds or thousands? Why are you writing to column C in sheet 2? I would have expected sheet1.
Wait, that's it? That's all your code does? They're must be something else because there is no way for me to get from where you are to where you said you want to be without acting like your personal code writing service.
So there it is. In detail. A genuine attempt to give you what you asked for. How much closer to your goal are you now that I've spent more than 2 hours offering my hand?
Probably not much.
Unsurprisingly the most helpful piece of information, the one that will get you the closest to your goal, was given to you in the first comment and it reflects poorly on you to respond in the manner you did.
From my perspective, it looks like you haven't figured out what you need to know. I'd start with with option explicit. Then transferring data between sheets. Then using ranges with arrays. That should get you to a point where we can actually help you. That's 3 topics and 1 of them is a 20 second read.
Good luck, and try to lose the attitude.
sheet 1 have 400 columns and data row upto 10000 rows, i want to compare with sheet 2 it has 600 columns means with additional columns in different order.
Compare sheet1 columns data with sheet2( as different column order) and highlight difference in sheet 2 and put difference in sheet 3 with highlighting the cell for mismatch, im new to VBA need your support,
you can highlight any one column as a primary column in sheet 1
Instead of hoping someone will write your code or teach you from scratch, you might have better luck with putting some research into how to do this with Excel's built-in tools. Therefore, my answer is "don't use VBA" — organizing data is what Excel is meant for and there is plenty of built-in functionality that you're probably not aware of.
First, there's Spreadsheet Compare (available in certain versions), which compares two workbooks (or two versions of the same workbook) and helps you see and organize differences between them. You can also identify potential issues, such as changes in formulas or calculations, or manually entered values.
Also built-in, see Consolidate data in multiple worksheets, allowing you to summarize and report results from separate worksheets into a single document. The sheets can be in the same workbook, or in other workbooks. When you consolidate data, you assemble data so that you can more easily update and aggregate as necessary.
If the columns are in different orders between the worksheets and that causes an issue for you (or for the built-in tools) is a problem, a simple fix is to simply sort the columns alphabetically on both sheets. In order to do this you just need to choose Left-to-Right in the Sort Options.
I'm sure there are other relevant features I'm not thinking of; take a look though "all" the commands available on your version's ribbon to see if there are other commands you're not aware of.
There are also a number of worksheet functions that could also help with a process like this. Exactly which ones depends on your needs (impossible for others to advise on without knowing the details of your current organization method).
The Insert Function Dialog
Off the top of my head, VLOOKUP, HLOOKUP, INDEX, MATCH, FIND, MID, LEFT and RIGHT could all potentially be beneficial to this task. (And still, no VBA required.) Find out more on those at the official Categorized List of Excel Functions, and also see the Lookup Functions section specifically.
Finally, there are a number of free or paid third-party add-ins specifically for Comparing & consolidating worksheets. For example, here is a lengthy list of the comparison functionality offered by DiffEngineX.
It's very common for Excel users to have a task at hand and assume that it's necessary to dive into VBA, without realizing Excel already provides the functionality they need. As a rule of thumb, ask yourself "Is this unique, or is this something that someone might have needed to do before?" If its not unique, chances are it's either already built-in, or there's an existing solution somewhere online.
this one's a bit of a painful one so thank you for your help and patience with me.
We have an Excel spreadsheet that we use as a master file for our website products. As such there are quite a few sheets and quite a few products on each running along side some macros to provide some extra functionality (turning entered data into HTML for product page, etc).
My issue is that one of our most used spreadsheets has become a trouble in that it has some phantom formatting all the way down to the millionth-and-something row and all the way across, causing the last cell to be the very last cell possible.
The issue that has finally popped up as a result is that we can no longer move rows in, out or around the sheet (a required functionality) as it results in an 'out of resources error'.
I've tried:
Highlight all rows below used range to right-click> delete - Results in runtime error (from macro)
Highlighting large chunks of rows and using Clear All - Resulted in the 38MB file bloating to 380MB
Deleting a chunk of rows at a time - Maxed out at 1,000 before it caused Excel to crash
Moving to new spreadsheet - Broke all our macros (which I did not write and am not proficient enough to fix on a new sheet)
Disabling macros and trying the above options, only marginally more efficient but still out of resources
I'm at my wits end on this one and, while we can continue with most day-to-day functions, we will soon be completely unable to use this particular sheet as we need it at all.
I'm wondering if there might be a way to run a VBA script to remove these rows, potentially one by one? I've tried running a short script that went something like rows[960,1000000].Delete (forgive my terrible VBA markup), but this also resulted in not enough resources errors.
I'm wondering if there's anything like:
row = 960;
while(row<=1048576){row.Delete};
Continuing, the runtime error debug points me to the below if statement within the macro:
If Target.Count > 1 Then Exit Sub
Where Target is the variable passed to the sub.
Which strikes me as very odd because my (limited) understanding of VBA and IF's in general simply recognizes that 'if my selection is larger than 1 (row?), do not run this code..
Thanks again in advance.
Use this method only if you don't have any links into or out of the sheet that will get broken. Also might have Sql connections that might get broken. Might need to disable macros. There are many possible problems with this approach. Use at your own risk.
Note the exact "Name" and "(Name)" of the sheet; Look in the VBA code window at the properties for the sheet. "Name" is the name displayed on the worksheet tab. "(Name)" is the code name visible only in the properties window.
Make a list of range names on the sheet.
Copy the data to a new sheet.
Copy any macros to the new sheet.
Delete the old sheet.
Rename the "Name" and "(Name)" of the new sheet the same as the old one.
Recreate range names.
A better method if you don't have too many formats:
Disable macros and set calculation to manual. This avoids recalculating while doing your delete operation.
Select entire sheet and clear formats.
Delete all rows below your data.
Redo your formatting. Select entire column (not just used area) to apply format if applicable.
It is important to remove formatting on the entire sheet from A1 to the end. Otherwise you'll get the bloat you mentioned. Just that step may solve your problem. If not then proceed with removing all the rows below the data. This should not cause file size bloat.
Background
I have an extremely large data table that takes up to 12 hours to run for around 1 million input scenarios on a high-end 64bit machine. The scenarios are based on a number of discrete Excel models, that are then fed into a financial model for detailed calculations
To improve the process, I am looking to test and compare the speeds of:
The current manual process
Using VBA to refresh the Data Table (with Calculation, ScreenUpdating etc off)
Running a VBS to refresh the Data Table in a invisible Excel instance
So, I am looking for the best approach to programmatically manage a Data Table
Update: using code in (2) and (3) did not provide a benefit on testing a simple example with a workbook with a single large data table
Rather surprisingly there seems to be very little - possibly no - direct support in VBA for Data Tables
My current knowledge and literature search
QueryTable BeforeRefresh and AfterRefresh Events can be added with this class module code. Intellisense doesn't provide this as an option for Data Tables
Individual PivotTables and QuertyTables can be accessed like so ActiveWorkbookk.Sheets(1).QueryTables(1). Not so Data Tables
Eliminating all other Data Tables and then running a RefreshAll was suggested in this MrExcel thread as a workaround.
The workaround is certainly do-able as I only have a single Data Table, but I'd prefer a direct approach if one exists.
Yes, I'm sticking to Excel :)
Please do not suggest other tools for this approach, both the input models and the overarching model that uses the data table are
part of a well established ongoing process that will stay Excel based,
have been professionally audited,
have been streamlined and optimised by some experience Excel designers
I was simply curious if there was a way to tweak the process by refreshing a specific data table with code, which my initial test results above have concluded no to.
So, you are looking for the best approach to programmatically manage a Data Table.
Well, Excel 2013 does record a macro for me when I manually create a data table, it goes
Selection.Table ColumnInput:=Range("G4")
The signature is
Range.Table(RowInput as Range, ColumnInput as Range) as Boolean
which is documented in Range.Table Method. The Range.Table() function seems to always return true.
This is the only way to create data tables using VBA. But that's all there is to data tables anyway.
AFAIK there is no class or object for data tables, so there is no dt.refresh() or similar method. And there is no collection of data tables you could query. You have to refresh the sheet or recreate the table with Range.Table().
There is a DataTable Interface, but it is related to charts and has nothing to do with Range.Table().
As you mention, you should turn off the usual suspects, i.e.
Application.ScreenUpdating = False
Application.DisplayStatusBar = False
Application.Calculation = xlCalculationManual
Application.EnableEvents = False
Try to have as little formulas in your workbook. Remove all formulas not related to the cells you base the data table on. Remove any intermediate results. Best have one cell with one, possibly big, formula.
Example: G4 is your ColumnInput, and it contains =2*G3, with G3 containing =G1+G2,
then better put =2*(G1+G2) into G4.
You may have 6 cores in your high end machine. Divide your scenarios into 6 chunks and
have 6 Excel instances calculate them in parallel.
I have two workbooks with identical sheets and I need to test whether the data they are getting (from different sources) is identical or within a certain threshold. This, I am already able to do fine. I create a third workbook which calculates the difference between the two.
However, the issue is that one workbook updates seconds before the second which means that if a cell gets two quick updates my calculations would lag behind.
So what I was thinking is that I make a note of the cell value in workbook 1 (the faster updating workbook) and if at anytime up to x seconds after workbook 2 cell has the same value as noted, they are good.
...but how would I go about this, is VBA even the best tool for this?
Any ideas?
using timestamps and vba - or maybe even conditional formatting and some good formulas - could solve this for sure.
However, without anything to work on, it is quite a big step to present you a solution.
Basically you would just create timestamps (internally as variables, or during the import process in cells) and then compare your values, after your threshold ran out. If the values of your compared cells still match, then it's ok.
However, this is all quite dependant on how you solved your dataimput and comparison.
My basic feeling, after I red your question was: "why wouldn't you just decrease your update rate of your calculation?"
Another idea: just use an indicator cell, switching between 1 and 0 or so, to indicate, if an update already happend - so, if you compare, you compare value+indicator. which is basically using timestamps without time.