Excel VBA: Memory Exceeds Limits on Second Import - vba

I am using 32-bit 2013 Excel with VBA extensively. I have disabled hardware graphics acceleration and COM add-ins, yet I still struggle with the following problem:
I am importing the contents of another large workbook with formatting on the cells but with no formulas (~3mb Excel file) into the problematic Excel workbook. On the first attempt - when the contents have not been imported yet - the import succeeds. I am importing the content via VBA similar to the following code:
Application.Workbooks(F_Home).Activate
Workbooks(F_Home).Sheets(Sheet1).Visible = xlSheetVisible
...
Application.Workbooks(F_Source).Activate
Workbooks(F_Source).Sheets(S_Source).Cells.Copy Destination:=Workbooks(F_Home).Sheets(Sheet1).Cells
Application.Workbooks(F_Home).Activate
F_Home is the problematic Excel workbook and F_Source is the workbook with the content we are importing in. When doing this the first time it works, and then I save the file and close out of it, and reopen the file, and try this a second time. On the second time we attempt to import the contents (when the contents already imported) the F_Home workbook crashes with the Out of Memory Error (There isn't enough memory to complete this action...) on the line that copies F_Source contents to F_Home.
Using Process Explorer I've found that the Excel process usually runs around 600mb - 700mb in virtual memory size, but when we run the VBA script to import the contents a second time, the virtual memory size suddenly jumps to 4gb (this does not happen on the first time around, which stays at the 600mb - 700mb range). How should I fix this? I cannot do a workaround such as saving the file before importing the contents a second time, because the timestamp on the file is used and saving the file through VBA will confuse some users.
Thank you for your help.

Copy and pasting the whole sheet is not a wise approach. Let's assume you have data from "A1:Z99999". You can do this which is going to be much faster.
Set S_Range = Workbooks(F_Source).Sheets(S_Source).Range("A1:Z99999")
Set H_Range = Workbooks(F_Home).Sheets(Sheet1).Range("A1:Z99999")
H_Range.Value = S_Range.Value
Also read why you should avoid selecting and activating in vba.

Related

Does openpyxl preserve cell color?

I'm creating an Excel file from the template. So I expect the formatting of the template to be preserved. However it seems saving of the workbook to new file looses some of formatting (at least cell color).
Original file looks like that:
I do then following:
import openpyxl
wb = openpyxl.open('c:\\temp\\test_templ.xlsx')
wb.save('c:\\temp\\test.xlsx')
Resulting file is 9KB smaller than original and looks like this:
Is there any way to save the Excel file with keeping the formatting?
Yes, it does. The problem I have met was abnormally complex Excel file with thousands of styles. So it seems some of them weren't properly read by openpyxl and hence the problem I had. But if you start with the clean slate and add necessary formatting openpyxl does the job just fine.

Excel Data Validations corrupt worksheets (when copying?)

Edit:
I further looked into this issue and have now a minimal file in which I can reproduce the error/bug. All Add-Ins are deactivated and I use only 4 lines of VBA-Code:
Private Sub Worksheet_BeforeDoubleClick(ByVal Target As Range, Cancel As Boolean)
Tabelle15.Range(Target, Target.Offset(1, 0)).EntireRow.Copy
Target.Offset(2, 0).EntireRow.Insert
End Sub
The file has 2 sheets with formatting, data validations, outlines (3 levels) and some data on it.
What I do to corrupt the files:
Open the file
Click 3 times on Outline-Level 2 (--> hiding some rows)
Click 3 times on Outline-Level 3 (--> unhiding the rows)
Doubleclick on a cell to copy two rows via the VBA-Code
Close the file
Repeat until file is corrupt (sometimes it needs 10 or more loops, sometimes the file is corrupt after 1 loop)
The sheet in the file which gets corrupted doesn't have any code in it and is not activated in these loops. Here is a before and after of the corrupted data validation:
Before corruption:
<dataValidation type="list" allowBlank="1" showInputMessage="1" showErrorMessage="1" sqref="D11 D13 D15 D9">
After corruption:
<dataValidation type="list" allowBlank="1" showInputMessage="1" showErrorMessage="1" sqref="D11:A11 1:A90 A16384:D4294967295 D9">
I have still no idea why this happens. It would be interesting if anyone has the same issue or knows of a workaround so the corruption doesnt happen. (I tried the obvious Application.Screenupdating, Application.Wait,... )
End of Edit ---
Unfortunately I cannot say when this problem occurs, because I only get the resulting corrupt workbooks.
We have written a COM-AddIn for Excel in VB.net which does all sort of things (i.e. copying cells and sheets in specific structures, data connections to an SQL-DB, saving the workbook ...)
Now I get some workbooks from users of the AddIn which are corrupted. The problem is the users are working with the files and don't have a problem. Only after saving, closing and reopening the file Excel shows the question if the file should be repaired. Depending on the file type (we use *.xlsx and *.xlsb) Excel "repairs" the file and removes all formating from 1 sheet - in *.xlsb it doesn't repair and open at all.
After going through the xml-Sheet files I found that the corruption occurs in the dataValidation-Tags. Not in one specific data-validation sometimes in one sometimes in another.
The corrupted dataValidation-tags look like this:
<dataValidation type="list" allowBlank="1" showInputMessage="1" showErrorMessage="1" sqref="S26:26 1:A523 A16384:S4294967295 S32"><formula1>"Item1,Item2,Item3"</formula1></dataValidation>
Repairing the data validation or the files is not the problem. By now I can recover my workbooks just fine. My main problem is that I can't find any lead to where this problem comes from.
The most operations which happen on the sheets are copying and inserting (not pasting) of entire rows. On one sheet which gets corrupted sometimes, there is only "copying from" and no pasting or inserting on the sheet. It also seems that general performance of the file impacts the frequency of the problem occuring.
Since I can't give any code where this corruption occurs I don't expect to find a solution here, but perhaps someone had a similar problem and has a hint which shows me the right direction.
Additional infos: The problem occurs on different machines. I did not manage to reproduce the corruption process. As far as I know the problem only occured on Excel 2016 but I can't rule out that earlier versions corrupt the files just the same. I checked (at least on some machines) that all updates for Office and Windows are installed. The formulas in the data validations have (way) less than 255 characters. On the corrupted sheets there is no data connection to a database.
Any idea or hint is appreciated!

Excel sheet Deletes the formulas present in the sheet when I open it. How to avoid this?

I'm uploading an excel file that contains sheets, to my server which encodes to base 64 so I decode it as required and process it by adding data in sheet 5 as column1 and column2 with certain number of rows. At the time of uploading, this sheet has some specific formulas on sheet 5 that makes changes in other sheets. So on opening the file which I send as response after editing from server, There comes this prompt that reads
"Excel Found unreadable content in 'MyDownloadedExcelData.xlsx'. Do you want to recover the contents of this workbook?If you trust the source of this workbook, click Yes', with Yes and no buttons
and when I click on yes and open the sheet, all the formulas are deleted.
I see something like
Excel was able to open the file by repairing ot removing the unreadable content.
Removed Records :Formula from /xl/calcChain.xml Part
Repaired Records : Cell Information from /xl/worksheets/sheet1.xml part etc
So, How do I make sure my formulas in the sheet are retained?
Using VBA you could have an on close event that pastes values and an on open event that recreates the formulas. Your file would essentially save with static data, but then be used with functions intact.
If this solution is of interest I can help provide some coding framework.

Excel VBA: Resetting spreadsheet count

I have a excel VBA macro that dynamically generates and deletes spreadsheets based on user input. However, when I open the VBA IDE, it seems that although I am naming my spreadsheets in the subs that create/delete them, the overall count is still increasing.
For example, depending on how far into execution my program is, under the "Microsoft Excel Objects" folder in my current project, the spreadsheets in the current workbook could look something like
Sheet101(Sheet3)
Sheet103(Sheet2)
Sheet104(Sheet1)
Or
Sheet81(Inputs)
Sheet83(Date Adjustment Interpolation)
Sheet84(Pricing)
Sheet85(Comparison)
No matter if I delete the rest of them and add one, it still picks up where the last highest one left off.
I don't know how many times this macro will be run and I'd feel a lot better about putting it out there if I could reset this annoying tally on the number of spreadsheets that have ever been generated, since I don't know for sure where excel will cut me off. Plus it's just annoying.
My Question:
I would like to know how to alter that spreadsheet number, or at least what the relevant object is for doing so.
Thanks!
Thanks to #dijkay s suggestion on code names, I've found some code to accomplish this.
ThisWorkbook.VBProject.VBComponents("Sheet1").name = "test"
Will change the code name of Sheet1 to test, so in the Excel Objects folder, it will appear as test(Sheet1) for example.
This option, however, requires messing around with some trust/security settings in each individual excel client running the macro, which is unsuitable for my purposes, unfortunately. You can also change the value manually by changing the (Name) property directly in the IDE through the properties window.
here are some ideas you can try...
Sheets(x).Name = "Sheet" & x
or (assuming in this example, 'Sheet3' doesn't already exist:
Set Sheet3 = sheets.Add
Sheet3.name = "Sheet3"
This is more cleanup than re-setting
cheers,
Micéal

vb.NET SaveAs not saving all Excel data

I have a very strange issue that I cannot seem to find an answer to online.
I have a VB.NET application that creates an Excel of data (roughly 42,542 rows in total) and the saves the file to a folder location & opens it on screen for the user.
The onscreen version & folder version is only showing 16,372 rows of data like it is being cut off.
When I go through debug I can see all the rows are being added & if I save manually in debug all the rows save. Some data seems to get lost on the system save.
I am taking data from 4 record sets & writing each set one after the other with specific headers for each block on the Excel sheet.
My save line is:
xlWBook.SaveAs(Filename:=sFileName, FileFormat:=Excel.XlFileFormat.xlExcel7)
Would anyone please have any ideas as to what this might be?
Older version of Excel only support 16,384 rows per worksheet. You are saving as Excel7 (which is Excel 95) and has this limitation:
See here for a summary of sizes per version:
https://superuser.com/questions/366468/what-is-the-maximum-allowed-rows-in-a-microsoft-excel-xls-or-xlsx
Change your code to another format, See here for all the allowed formats: XlFileFormat Enumeration
However the file format is actually an optional argument in the SaveAs method, so you could leave it off altogether: "For an existing file, the default format is the last file format specified; for a new file, the default is the format of the version of Excel being used."
Source: WorkBook.SaveAs Method