VBA App Crashes w/o Error Message - Works when stepping through program - vba
I have an excel application that often, but not always crashes, when run normally. In case you set a breakpoint and step through the program, it never fails. Likewise if you set breakpoints at strategic places, and then continues executing it generally also works well.
The issue appears to be related to opening a file, copying a large amount of data, and then closing the file. I am however unsure where the program actually crashes. Tips for debugging / methods for finding where the error occurs in the code would be most appreciated.
I have assumed this is due to either a race condition or memory problems, but unsure exactly what would cause either of those errors. Race condition seems more likely though, as pausing or stepping through application shouldn't help with memory issues. If race condition is the cause of the problem, is there a better solution than letting the application sleep/wait at certain points? How do I identify the points where I would need to sleep/wait?
EDIT: When running the application normally it seems to run longer than you would expect, then just closes without any error message. I am running Excel 2013 (32bit) on Win 10.
I considered data being saved to cliboard being the issue, and added
Application.CutCopyMode = False
after each paste, this did not resolve the issue though.
I am supressing alerts and screen updating, i.e.
Application.DisplayAlerts = False
Application.ScreenUpdating = False
but commenting out these settings, still causes application to crash.
EDIT2: Adding the code where the crash occurs. Errors appears to occur somewhere in ReadInAndCopyFiles.
Sub ReadInFiles(wb As Workbook, FolderPath As String, FileName As String)
Dim CurrentWeekDate As Date
Dim TempDate As Date
Dim TempFilePath As String
Dim DataFileName As String
Dim OpenDialog As Office.FileDialog
Dim DateString As String
Dim SheetNameArray As Variant
'Initialization
CurrentWeekDate = wb.Worksheets("Config").Range("EndOfWeekDate").Value
ChDir (FolderPath)
If FileName = "Weekly utilization" Then
SheetNameArray = Array("WeeklyUtilization_CW", "WeeklyUtilization_CW-1", "WeeklyUtilization_CW-2", "WeeklyUtilization_CW-3")
Else
SheetNameArray = Array("Charged Hours", "ChargedHours_CW-1", "ChargedHours_CW-2", "ChargedHours_CW-3")
End If
'Current Week
TempFilePath = FolderPath + FileName + ".xlsx"
ReadInAndCopyFile TempFilePath, CStr(SheetNameArray(0)), "Find " & FileName
'Current Week -1
TempDate = DateAdd("d", -7, CurrentWeekDate)
DateString = Format(TempDate, "yy-mm-dd")
TempFilePath = FolderPath + "Archives\" + FileName + " " + DateString + ".xlsx"
ReadInAndCopyFile TempFilePath, CStr(SheetNameArray(1)), "Find " & FileName & " -1"
'Current Week -2
TempDate = DateAdd("d", -14, CurrentWeekDate)
DateString = Format(TempDate, "yy-mm-dd")
TempFilePath = FolderPath + "Archives\" + FileName + " " + DateString + ".xlsx"
ReadInAndCopyFile TempFilePath, CStr(SheetNameArray(2)), "Find " & FileName & " -2"
'Current Week -3
TempDate = DateAdd("d", -21, CurrentWeekDate)
DateString = Format(TempDate, "yy-mm-dd")
TempFilePath = FolderPath + "Archives\" + FileName + " " + DateString + ".xlsx"
ReadInAndCopyFile TempFilePath, CStr(SheetNameArray(3)), "Find " & FileName & " -3"
End Sub
Sub ReadInAndCopyFile(TempFilePath As String, TargetSheetName As String, CustomMessage As String)
Dim DataFileName As String
Dim SourceWb, wb As Workbook
Dim ws As Worksheet
Dim LastRow, LastColumn, StartRow, TargetLastRow As Variant
Dim OpenDialog As Office.FileDialog
Set wb = ActiveWorkbook
DataFileName = Dir(TempFilePath)
If Not DataFileName <> "" Then
MsgBox CustomMessage
Set OpenDialog = Application.FileDialog(msoFileDialogFilePicker)
OpenDialog.Filters.Clear
OpenDialog.Filters.Add "Excel Files", "*.xlsx"
OpenDialog.AllowMultiSelect = False
OpenDialog.Show
TempFilePath = OpenDialog.SelectedItems(1)
End If
Workbooks.Open FileName:=TempFilePath, UpdateLinks:=False
Set SourceWb = ActiveWorkbook
'Determine where to start pasting, and if header should be included or not
If (wb.Worksheets(TargetSheetName).Cells(Rows.Count, 1).End(xlUp).Row = 1) Then
StartRow = 1
Else
StartRow = wb.Worksheets(TargetSheetName).Cells(Rows.Count, 1).End(xlUp).Row + 1
End If
'Copy First Sheet
LastRow = SourceWb.Worksheets("Sheet1").Cells(Rows.Count, 1).End(xlUp).Row
'Dont copy any data if blank
If LastRow <> 1 Then
LastColumn = SourceWb.Worksheets("Sheet1").Cells(1, Columns.Count).End(xlToLeft).Column
If StartRow = 1 Then
Range(SourceWb.Worksheets("Sheet1").Cells(1, 1), SourceWb.Worksheets("Sheet1").Cells(LastRow, LastColumn)).Copy
Else
Range(SourceWb.Worksheets("Sheet1").Cells(2, 1), SourceWb.Worksheets("Sheet1").Cells(LastRow, LastColumn)).Copy
End If
wb.Worksheets(TargetSheetName).Range("A" + CStr(StartRow)).PasteSpecial Paste:=xlPasteValues, Operation:=xlNone, SkipBlanks:=False, Transpose:=False
Application.CutCopyMode = False
TargetLastRow = wb.Worksheets(TargetSheetName).Cells(Rows.Count, 1).End(xlUp).Row
End If
'Copy Second Sheet
LastRow = SourceWb.Worksheets("Sheet2").Cells(Rows.Count, 1).End(xlUp).Row
'Dont copy any data if blank
If LastRow <> 1 Then
LastColumn = SourceWb.Worksheets("Sheet2").Cells(1, Columns.Count).End(xlToLeft).Column
'Copy from row 2 to avoid copying headers again
Range(SourceWb.Worksheets("Sheet2").Cells(2, 1), SourceWb.Worksheets("Sheet2").Cells(LastRow, LastColumn)).Copy
wb.Worksheets(TargetSheetName).Range("A" + CStr(TargetLastRow + 1)).PasteSpecial Paste:=xlPasteValues, Operation:=xlNone, SkipBlanks:=False, Transpose:=False
Application.CutCopyMode = False
End If
SourceWb.Close SaveChanges:=False
End Sub
I suspect this bit
Dim OpenDialog As Office.FileDialog
Set wb = ActiveWorkbook
DataFileName = Dir(TempFilePath)
If Not DataFileName <> "" Then
MsgBox CustomMessage
Set OpenDialog = Application.FileDialog(msoFileDialogFilePicker)
OpenDialog.Filters.Clear
OpenDialog.Filters.Add "Excel Files", "*.xlsx"
OpenDialog.AllowMultiSelect = False
OpenDialog.Show
TempFilePath = OpenDialog.SelectedItems(1)
End If
Replace with this
Dim s
Set wb = ActiveWorkbook
datafilename = Dir(tempfilepath)
If datafilename = "" Then
s = Application.GetOpenFilename("*.xlsx,Excel Files", 1, "Select File", , False)
If Not s = False Then
tempfilepath = s
End If
End If
I wasa able to resolve the issue by adding Application.Wait in two places in the code for the sub ReadInAndCopyFile.
'Firstplace
Workbooks.Open FileName:=TempFilePath, UpdateLinks:=False
Application.Wait (Now + TimeValue("0:00:10"))
Set SourceWb = ActiveWorkbook
'Second place
Application.Wait (Now + TimeValue("0:00:10"))
SourceWb.Close SaveChanges:=False
The placement is only due to where I assumed the errors were occurring. It is entirely possible that only one Wait would be enough, and that a shorter wait would be ok. I may do further experimenting later, but for now it is enough that it is running.
Happy to hear if anyone has better or faster methods for resolving this, as this methods as a significant amount of time to the total running time.
Related
Excel VBA - Running a Macro against all files in a folder [duplicate]
I have a folder where I receive 1000+ excel files on daily bases they all are same format and structure. What I want to do is run a macro on all 100+ files on daily bases ? Is there way to automate this ? So I can keep running that same macro on 1000+ files daily.
Assuming that you put your files in "Files" directory relative to your master workbook your code might look like this: Sub ProcessFiles() Dim Filename, Pathname As String Dim wb As Workbook Pathname = ActiveWorkbook.Path & "\Files\" Filename = Dir(Pathname & "*.xls") Do While Filename <> "" Set wb = Workbooks.Open(Pathname & Filename) DoWork wb wb.Close SaveChanges:=True Filename = Dir() Loop End Sub Sub DoWork(wb As Workbook) With wb 'Do your work here .Worksheets(1).Range("A1").Value = "Hello World!" End With End Sub In this example DoWork() is your macro that you apply to all of your files. Make sure that you do all your processing in your macro is always in the context of the wb (currently opened workbook). Disclaimer: all possible error handling skipped for brevity.
A part of the question might be how do I run this on 1000 files?... Do I have to add this macro to all 1000 workbooks? One way to do this is to add your macro's centrally to the file PERSONAL.XLSB (sometimes the extension might be different). This file will be loaded in the background every time you start Excel and makes your macro's available at any time. Initially the PERSONAL.XLSB file will NOT be there. To automatically create this file, just start recording a "dummy" macro (with the record button on the left-bottom of a spreadsheet) and select "Personal Macro Workbook" to store it in. After recording your macro, you can open the VBA editor with Alt+F11 and you will see the PERSONAL.XLSB file with the "dummy" recorded macro. I use this file to store loads of general macro's which are always available, independent of which .xlsx file I have open. I have added these macro's to my own menu ribbon. One disadvantage of this common macro file is that if you launch more than one instance of Excel, you will get an error message that the PERSONAL.XLSB file is already in use by Excel instance Nr. 1. This is no problem as long as you do not add new macro's at this moment.
Thank you very much for this Sub ProcessFiles() Dim Filename, Pathname As String Dim wb As Workbook Pathname = ActiveWorkbook.Path & "C:\Users\jkatanan\Desktop\20170206Glidepath\V37\" Filename = Dir(Pathname & "*.xls") Do While Filename <> "" Set wb = Workbooks.Open(Pathname & Filename) BSAQmacro wb wb.Close SaveChanges:=True Filename = Dir() Loop End Sub Sub DoWork(wb As Workbook) With wb 'Do your work here .Worksheets(1).Range("A1").Value = "Hello World!" End With End Sub
Sub ProcessFiles() Dim Filename, Pathname As String Dim wb As Workbook Pathname = ActiveWorkbook.Path & "\C:\Users\20098323\Desktop\EXCL\" Filename = Dir(Pathname & "*.xlsx") Do While Filename <> "" Set wb = Workbooks.Open(Pathname & Filename) DoWork wb wb.Close SaveChanges:=True Filename = Dir() Loop End Sub Sub DoWork(wb As Workbook) With wb 'Do your work here .Worksheets(1).Range("A1").Value = "Hello World!" End With End Sub While running this code its showing bad file name or number. i have stored my all file in ("\C:\Users\20098323\Desktop\EXCL\") EXCL folder
Instead of passing the values to DoWork one can also run the jobs in Processfiles(). Sub ProcessFiles() Dim Filename, Pathname As String Dim wb1 As Workbook Dim wb2 As Workbook Dim Sheet As Worksheet Dim PasteStart As Range Dim Counter As Integer Set wb1 = ActiveWorkbook Set PasteStart = [RRimport!A1] Pathname = ActiveWorkbook.Path & "\For Macro to run\" Filename = Dir(Pathname & "*.xls") Do While Filename <> "" Set wb2 = Workbooks.Open(Pathname & Filename) For Each Sheet In wb2.Sheets With Sheet.UsedRange .Copy PasteStart Set PasteStart = PasteStart.Offset(.Rows.Count) End With Next Sheet wb2.Close Filename = Dir() Loop End Sub
This isn't an exact answer to the question, since I was just trying to run a single script on any file that I opened and couldn't get that to work, but I thought this might help others like me. It worked once I moved the code into a Module in the Visual Basic for Applications dialog box (go to "Insert" then "Module"). Once I added my VBA code to a module, I was able to open any other file in Excel (even a CSV file) and go to Macros, and run the Macro from the other file (that contains the Module with the code) on the file that I had open.
Thanks Peterm!! Actually, I did my macro using exactly the same code you posted (process_fiels and dowork). It worked brilliant!! (before my question) Each of my 1000 workbooks has 84 worksheets. My own macro (which finally works!) splits each workbook into 85 different files (the original + a short version of each worksheet saved as an individual file). That leaves me with 1000 files + 1000x85 in the same folder, and that would be really hard to sort out. What I really need is for Process_Files to take the first file, create a folder with the name of the first file, move the first file to the folder with ist name, then run my macro (in the folder named after the first file in the newly created folder...), go back and take the second file, create a folder with the name of the second file, move the second file to the folder with ist name, then run my macro (in the folder named after the second file in the newly created folder...), etc... At the end, I should have moved all files into folders with the same name as the files, and the contents of the original \Files\ folder would be 1000 folders with the name of the original files, containgin the original files + 84 files which my own macro already does. Maybe it is easier with the code: Sub ProcessFiles() Dim Filename, Pathname As String Dim wb As Workbook Pathname = ActiveWorkbook.Path & "\Files\" Filename = Dir(Pathname & "*.xls") Do While Filename <> "" (Here, it should read the file name, create a folder with the file name, move the file into this newly created folder) Set wb = Workbooks.Open(Pathname & Filename) <- open file, just as is. DoWork wb <- do my macro,just as is wb.Close SaveChanges:=False <- not save, to keep the original file (go back to the original \Files\ folder) Filename = Dir() <- Next file, just as is Loop End Sub Sub DoWork(wb As Workbook) With wb MyMacro End With End Sub Many thanks, this site is great! __________________edit, the macro now works _________________________ As you can see, I am no VBA expert, but the macro finally works. The code is not neat at all, I am no SW programmer. Here it is, it might help some one some day. Sub ProcessFiles_All() Dim Filename, Pathname, NewPath, FileSource, FileDestination As String Dim wb As Workbook Pathname = ActiveWorkbook.Path & "\Files\" Filename = Dir(Pathname & "*.csv") Do While Filename <> "" NewPath = Pathname & Left(Filename, 34) & "\" On Error Resume Next MkDir (NewPath) On Error GoTo 0 Set wb = Workbooks.Open(Pathname & Filename) DoWorkPlease wb ' <------------ It is important to say please!! On Error Resume Next wb.Close SaveChanges:=False if Err.Number <> 0 then ‘Error handler needed here End if Filename = Dir() Loop End Sub Sub DoWorkPlease(wb As Workbook) With wb ‘ Since my application has over 1800 cells for each column and it is time consuming ‘ I use a “testing mode” were I only play with 18 values. Dim TestingMode As Integer Dim ThisRange(1 To 4) As Variant TestingMode = 0 If TestingMode = 1 Then ThisRange(1) = "B2:CG18" ThisRange(2) = "CT2:CT18" ThisRange(3) = "CH2:CN18" ThisRange(4) = "CN2:CS18" Rows("19:18201").Select Selection.Delete Shift:=xlUp End If If TestingMode = 0 Then ThisRange(1) = "B2:CG18201" ThisRange(2) = "CT2:CT18201" ThisRange(3) = "CH2:CN18201" ThisRange(4) = "CN2:CS18201" End If ‘ speed up the macro, turn off updating and alerts Application.ScreenUpdating = False Application.DisplayAlerts = False ‘ Here is my code that manipulates the cell values from digits (values read by sensors need to be “translated” into real world values. Code not here actually. ‘Then I copy the whole thing into just numbers, there are no longer formulas, easier to work this way. '_____________________________________ 'Get just values - no more formulas Sheets.Add After:=Sheets(Sheets.Count) Sheets("Sheet1").Select Columns("A:CT").Select Selection.Copy Sheets("Sheet2").Select Columns("A:A").Select Selection.PasteSpecial Paste:=xlPasteValues, Operation:=xlNone, SkipBlanks _ :=False, Transpose:=False Application.CutCopyMode = False Selection.NumberFormat = "0" With Selection .HorizontalAlignment = xlCenter .VerticalAlignment = xlBottom .WrapText = False .Orientation = 0 .AddIndent = False .IndentLevel = 0 .ShrinkToFit = False .ReadingOrder = xlContext .MergeCells = False End With ‘ Then I save this new workbook into a folder with its own name (and under the folder \FILES\ '_____________________________________ 'Save the work under its own folder Dim CleanName, CleanPath, CleanNewName As Variant CleanPath = ActiveWorkbook.Path CleanName = ActiveWorkbook.Name CleanName = Left(CleanName, 34) ‘I take out the extension CleanPath = CleanPath + "\" + CleanName CleanNewName = CleanPath + "\" + CleanName CleanNewName = CleanNewName + "_clean.csv" ‘ and I add “clean” to have a different name now. On Error Resume Next ActiveWorkbook.SaveAs Filename:=CleanNewName, FileFormat:=xlCSV, CreateBackup:=False ‘If there is an error I create an empty folder with the name of the file to know which file needs rework. If Err.Number <> 0 Then MkDir (CleanPath + "_error_" + CleanName) End If 'Resume Next ActiveSheet.Move _ After:=ActiveWorkbook.Sheets(1) ‘ Then I split the workbook into individual files with the data I need for individual sensors. ‘ Here are the individual ranges I need for each file. Since I have over 1000 files, it is worth the effort. '_______________ the Split!!______________________________ Dim Col(1 To 98) As Variant Col(1) = "A:A,B:B,CH:CH,CN:CN,CT:CT" Col(2) = "A:A,C:C,CH:CH,CN:CN,CT:CT" Col(3) = "A:A,D:D,CH:CH,CN:CN,CT:CT" Col(4) = "A:A,E:E,CH:CH,CN:CN,CT:CT" Col(5) = "A:A,F:F,CH:CH,CN:CN,CT:CT" Col(6) = "A:A,G:G,CH:CH,CN:CN,CT:CT" Col(7) = "A:A,H:H,CH:CH,CN:CN,CT:CT" Col(8) = "A:A,I:I,CH:CH,CN:CN,CT:CT" Col(9) = "A:A,J:J,CH:CH,CN:CN,CT:CT" Col(10) = "A:A,K:K,CH:CH,CN:CN,CT:CT" Col(11) = "A:A,L:L,CH:CH,CN:CN,CT:CT" Col(12) = "A:A,M:M,CH:CH,CN:CN,CT:CT" Col(13) = "A:A,N:N,CH:CH,CN:CN,CT:CT" Col(14) = "A:A,O:O,CH:CH,CN:CN,CT:CT" Col(15) = "A:A,P:P,CI:CI,CO:CO,CT:CT" Col(16) = "A:A,Q:Q,CI:CI,CO:CO,CT:CT" Col(17) = "A:A,R:R,CI:CI,CO:CO,CT:CT" Col(18) = "A:A,S:S,CI:CI,CO:CO,CT:CT" Col(19) = "A:A,T:T,CI:CI,CO:CO,CT:CT" Col(20) = "A:A,U:U,CI:CI,CO:CO,CT:CT" Col(21) = "A:A,V:V,CI:CI,CO:CO,CT:CT" Col(22) = "A:A,W:W,CI:CI,CO:CO,CT:CT" Col(23) = "A:A,X:X,CI:CI,CO:CO,CT:CT" Col(24) = "A:A,Y:Y,CI:CI,CO:CO,CT:CT" Col(25) = "A:A,Z:Z,CI:CI,CO:CO,CT:CT" Col(26) = "A:A,AA:AA,CI:CI,CO:CO,CT:CT" Col(27) = "A:A,AB:AB,CI:CI,CO:CO,CT:CT" Col(28) = "A:A,AC:AC,CI:CI,CO:CO,CT:CT" Col(29) = "A:A,AD:AD,CJ:CJ,CP:CP,CT:CT" Col(30) = "A:A,AE:AE,CJ:CJ,CP:CP,CT:CT" Col(31) = "A:A,AF:AF,CJ:CJ,CP:CP,CT:CT" Col(32) = "A:A,AG:AG,CJ:CJ,CP:CP,CT:CT" Col(33) = "A:A,AH:AH,CJ:CJ,CP:CP,CT:CT" Col(34) = "A:A,AI:AI,CJ:CJ,CP:CP,CT:CT" Col(35) = "A:A,AJ:AJ,CJ:CJ,CP:CP,CT:CT" Col(36) = "A:A,AK:AK,CJ:CJ,CP:CP,CT:CT" Col(37) = "A:A,AL:AL,CJ:CJ,CP:CP,CT:CT" Col(38) = "A:A,AM:AM,CJ:CJ,CP:CP,CT:CT" Col(39) = "A:A,AN:AN,CJ:CJ,CP:CP,CT:CT" Col(40) = "A:A,AO:AO,CJ:CJ,CP:CP,CT:CT" Col(41) = "A:A,AP:AP,CJ:CJ,CP:CP,CT:CT" Col(42) = "A:A,AQ:AQ,CJ:CJ,CP:CP,CT:CT" Col(43) = "A:A,AR:AR,CK:CK,CQ:CQ,CT:CT" Col(44) = "A:A,AS:AS,CK:CK,CQ:CQ,CT:CT" Col(45) = "A:A,AT:AT,CK:CK,CQ:CQ,CT:CT" Col(46) = "A:A,AU:AU,CK:CK,CQ:CQ,CT:CT" Col(47) = "A:A,AV:AV,CK:CK,CQ:CQ,CT:CT" Col(48) = "A:A,AW:AW,CK:CK,CQ:CQ,CT:CT" Col(49) = "A:A,AX:AX,CK:CK,CQ:CQ,CT:CT" Col(50) = "A:A,AY:AY,CK:CK,CQ:CQ,CT:CT" Col(51) = "A:A,AZ:AZ,CK:CK,CQ:CQ,CT:CT" Col(52) = "A:A,BA:BA,CK:CK,CQ:CQ,CT:CT" Col(53) = "A:A,BB:BB,CK:CK,CQ:CQ,CT:CT" Col(54) = "A:A,BC:BC,CK:CK,CQ:CQ,CT:CT" Col(55) = "A:A,BD:BD,CK:CK,CQ:CQ,CT:CT" Col(56) = "A:A,BE:BE,CK:CK,CQ:CQ,CT:CT" Col(57) = "A:A,BF:BF,CL:CL,CR:CR,CT:CT" Col(58) = "A:A,BG:BG,CL:CL,CR:CR,CT:CT" Col(59) = "A:A,BH:BH,CL:CL,CR:CR,CT:CT" Col(60) = "A:A,BI:BI,CL:CL,CR:CR,CT:CT" Col(61) = "A:A,BJ:BJ,CL:CL,CR:CR,CT:CT" Col(62) = "A:A,BK:BK,CL:CL,CR:CR,CT:CT" Col(63) = "A:A,BL:BL,CL:CL,CR:CR,CT:CT" Col(64) = "A:A,BM:BM,CL:CL,CR:CR,CT:CT" Col(65) = "A:A,BN:BN,CL:CL,CR:CR,CT:CT" Col(66) = "A:A,BO:BO,CL:CL,CR:CR,CT:CT" Col(67) = "A:A,BP:BP,CL:CL,CR:CR,CT:CT" Col(68) = "A:A,BQ:BQ,CL:CL,CR:CR,CT:CT" Col(69) = "A:A,BR:BR,CL:CL,CR:CR,CT:CT" Col(70) = "A:A,BS:BS,CL:CL,CR:CR,CT:CT" Col(71) = "A:A,BT:BT,CM:CM,CS:CS,CT:CT" Col(72) = "A:A,BU:BU,CM:CM,CS:CS,CT:CT" Col(73) = "A:A,BV:BV,CM:CM,CS:CS,CT:CT" Col(74) = "A:A,BW:BW,CM:CM,CS:CS,CT:CT" Col(75) = "A:A,BX:BX,CM:CM,CS:CS,CT:CT" Col(76) = "A:A,BY:BY,CM:CM,CS:CS,CT:CT" Col(77) = "A:A,BZ:BZ,CM:CM,CS:CS,CT:CT" Col(78) = "A:A,CA:CA,CM:CM,CS:CS,CT:CT" Col(79) = "A:A,CB:CB,CM:CM,CS:CS,CT:CT" Col(80) = "A:A,CC:CC,CM:CM,CS:CS,CT:CT" Col(81) = "A:A,CD:CD,CM:CM,CS:CS,CT:CT" Col(82) = "A:A,CE:CE,CM:CM,CS:CS,CT:CT" Col(83) = "A:A,CF:CF,CM:CM,CS:CS,CT:CT" Col(84) = "A:A,CG:CG,CM:CM,CS:CS,CT:CT" ‘ I want to split 84 new files, so for testing I use only 1, and for the real thing I go with 84 Dim CounterMode As Integer If TestingMode = 1 Then CounterMode = 1 Else CounterMode = 84 For i = 1 To CounterMode ‘ this code takes the columns need, and paste it into a new workbook. Sheets("Sheet1").Select Cells.Select Selection.ClearContents Range("A1").Activate Sheets(2).Select Range(Col(i)).Select Selection.Copy Sheets("Sheet1").Select ActiveSheet.Paste Application.CutCopyMode = False With Selection .HorizontalAlignment = xlCenter .VerticalAlignment = xlBottom .WrapText = False .Orientation = 0 .AddIndent = False .IndentLevel = 0 .ShrinkToFit = False .ReadingOrder = xlContext .MergeCells = False End With Columns("A:E").EntireColumn.AutoFit ‘ Save the individual file '_____________save the work________________ Dim ThePath, TheName, TheSwitch As String ThePath = ActiveWorkbook.Path + “\” TheName = Left(ActiveWorkbook.Name, 34) ‘ take out the extension from the name ThePath = ThePath + TheName TheSwitch = Cells(3, 2) ‘ In Cell (3,2) I have the name of the individual name, so I added to the file name. TheName = ThePath + "_" + TheSwitch + ".xls" Range("A1").Select Sheets("Sheet1").Select Sheets("Sheet1").Copy Dim SheetName As Variant ‘ I name Sheets(1) as Sheet1, since the original sheet has the name and date of the test. ‘ I do this to have the same name on all file in order to do a plot, then I rename the sheet with the ‘ original name SheetName = ActiveSheet.Name ActiveWorkbook.Sheets(1).Name = "Sheet1" ‘ here is the plot Columns("A:E").EntireColumn.AutoFit Columns("B:E").Select ActiveSheet.Shapes.AddChart.Select ActiveChart.SetSourceData Source:=Range("'Sheet1'!$B:$E") ActiveChart.ChartType = xlXYScatterLinesNoMarkers ActiveWorkbook.Sheets(1).Name = SheetName ‘save On Error Resume Next ActiveWorkbook.SaveAs Filename:=TheName, FileFormat:=56, CreateBackup:=False If Err.Number <> 0 Then MkDir (ThePath + "_error_" + TheName) End If ActiveWorkbook.Close Next i '____________________That was the Split__________________________________ ' Turn on screenupdating: Application.ScreenUpdating = True Application.DisplayAlerts = True Range("A1").Select End With End Sub
Making excel macro for file scanning more stable
I was curious if anybody could provide suggestions on how I can make an excel macro more stable. The macro prompts the user for a path to a folder containing files to scan. The macro then iterates for every file in this folder. It opens the excel file, scans Column D for the word fail, then copies that row of data to the data sheet in the excel file where this macro is programmed. For the most part the macro runs perfectly but sometimes I get run time errors or 'excel has stopped working' errors. I can scan through 5000+ files at a time and the macro takes a while to run. Any suggestions would be appreciated. Thanks! Sub findFail() Dim pathInput As String 'path to file Dim path As String 'path to file after being validated Dim fileNames As String 'path to test file Dim book As Workbook 'file being tested Dim sheet As Worksheet 'sheet writting data to Dim sh As Worksheet 'worksheet being tested Dim dataBook As Workbook 'where data is recorded Dim row As Long 'row to start writting data in Dim numTests As Long 'number of files tested Dim j As Long 'counter for number of files tested Dim i As Long 'row currently being tested Dim lastRow As Long 'last row used Dim startTime As Double 'time when program started Dim minsElapsed As Double 'time it took program to end Application.ScreenUpdating = False j = 0 i = 1 row = 2 Set dataBook = ActiveWorkbook Set sheet = Worksheets("Data") sheet.Range("A2:i1000").Clear startTime = Timer '-----Prompt for Path----- pathInput = InputBox(Prompt:="Enter path to files. It must have a \ after folder name.", _ Title:="Single Report", _ Default:="C:\Folder\") If pathInput = "C:\Folder\" Or pathInput = vbNullString Then 'check to make sure path was inputed MsgBox ("Please enter a valid file path and try again.") Exit Sub Else path = pathInput 'path = "C:\Temp\212458481\" ' Path for file location fileNames = Dir(path & "*.xls") 'for xl2007 & "*.xls?" on windows '-----begin testing----- Do While fileNames <> "" 'Loop until filename is blank Set book = Workbooks.Open(path & fileNames) Set sh = book.Worksheets(1) lastRow = sh.UsedRange.Rows(sh.UsedRange.Rows.Count).row If sh.Cells(lastRow, 2).Value - sh.Cells(1, 2).Value >= 0.08333333 Then Do While sh.Range("D" & i).Value <> "" 'loop untile there are no rows left to test If sh.Range("D" & i).Value = "Fail" Then 'record values if test result is false sheet.Range("A" & row).Value = book.Name sheet.Range("B" & row).Value = Format(sh.Range("B" & i).Value - sh.Range("B1").Value, "h:mm:ss") sheet.Range("C" & row).Value = sh.Range("A" & i).Value sheet.Range("D" & row).Value = Format(sh.Range("B" & i).Value, "h:mm:ss") sheet.Range("E" & row).Value = sh.Range("C" & i).Value sheet.Range("F" & row).Value = sh.Range("D" & i).Value sheet.Range("G" & row).Value = sh.Range("E" & i).Value sheet.Range("H" & row).Value = sh.Range("F" & i).Value sheet.Range("I" & row).Value = sh.Range("G" & i).Value row = row + 1 Exit Do End If i = i + 1 Loop j = j + 1 dataBook.Sheets("Summary").Cells(2, 1).Value = j End If book.Close SaveChanges:=False fileNames = Dir() i = 1 Loop numTests = j Worksheets("Summary").Cells(2, "A").Value = numTests minsElapsed = Timer - startTime Worksheets("Summary").Cells(2, "B").Value = Format(minsElapsed / 86400, "hh:mm:ss") End If End Sub
Without the same dataset as you we, can not definitively supply an answer but I can recommend the below which is related to the error you are seeing. Try freeing/destroying the references to book and sh. You have a loop that sets them:- Do While fileNames <> "" 'Loop until filename is blank Set book = Workbooks.Open(path & fileNames) Set sh = book.Worksheets(1) However the end of the loop does not clear them, ideally it should look as below:- Set sh = Nothing Set book = Nothing Loop This is a better way to handle resources and should improve memory usage. As a poor example, without it your code is saying, sh equals this, now it equals this instead, now it equals this instead, now it equals this instead, etc... You end up with the previous reference that was subsequently overwritten being a sort of orphaned object that is holding some space in memory.
Depending on your case, you may use the following to make it faster -by turning off excel processes that you don't really need at the time of your macro execution- Sub ExcelBusy() With Excel.Application .Cursor = xlWait .ScreenUpdating = False .DisplayAlerts = False .StatusBar = False .Calculation = xlCalculationManual .EnableEvents = False End With End Sub In your sub Dim startTime As Double 'time when program started Dim minsElapsed As Double 'time it took program to end Call ExcelBusy ... As a comment, you never set back screenupdating to true in your sub, that may lead to strange behavior in excel, you should turn everything to default after you are done with your stuff. OT: Some processes can't be optimized any further -sometimes-, by what you are saying -scanning over 5k files?- surely it's going to take a time, you need to work in how to communicate the user that is going to take a while instead -perhaps an application status bar message or a user form showing process?-.
Wait for big files to open in Excel
I've been trying to loop over a bunch of big .csv files in VBA. Each of them is approximately 50MB. At every iteration I open a new CSV to manipulate data but when the .csv is opening there is a downloading message saying that the file is opening and the progress bar always gets stuck at some point while the VBA is waiting for it to finish. Actually the .csv is opened because if I click "cancel" on the progress bar the code continues running well but I have to do a manual action at every iteration. My guess is that VBA goes to the next step while the file is not opened or something like that so maybe if I do a Sleep or something like that it could work but what I tried did not work for now. (I already tried Application.EnableEvents = False). Here is my code: Sub GetOptions() Application.DisplayAlerts = False Application.EnableEvents = False Set Dates = Sheets("Dates") Set Res = Sheets("Options") Dim dateToday As Date ETF = "SPY" nrows = Dates.Cells(Rows.Count, 1).End(xlUp).Row For i = 708 To nrows If Dates.Cells(i, 2).Value = "B" Then dateToday = Dates.Cells(i, 1).Value dateYear = Year(dateToday) stringOpening = "P:\Options Database\CSV\" & dateYear & "\bb_" & dateYear & "_" & GetMonth(dateToday) & "\bb_options_" & Format(dateToday, "yyyymmdd") & ".csv" Workbooks.Open stringOpening, UpdateLinks:=0, ReadOnly:=True Set Options = Workbooks("bb_options_" & Format(dateToday, "yyyymmdd")).Sheets(1) Do things... Workbooks("bb_options_" & Format(dateToday, "yyyymmdd")).Close SaveChanges:=False End If Next i End Sub
A trick would be : to open them as Read/Write files, wait for the Write status which indicates that it is fully opened set back the file to Read Only This code loops until the file goes into a Write status : Sub myWaitForFileOpen() Dim wb As Workbook Set wb = Application.Workbooks.Open("C:\File.xls") Do Until wb.ReadOnly = False wb.Close Application.Wait Now + TimeValue("00:00:01") Set wb = Application.Workbooks.Open("C:\File.xls") Loop 'Then the code that needs that Workbook open here! 'Or Call That other macro here! End Sub Here is your full code, that will open the CSV in Read/Write until it is fully loaded and then put it back to read only : Sub GetOptions() Dim wB As Workbook Application.DisplayAlerts = False Application.EnableEvents = False Set Dates = Sheets("Dates") Set Res = Sheets("Options") Dim dateToday As Date ETF = "SPY" nrows = Dates.Cells(Rows.Count, 1).End(xlUp).Row For i = 708 To nrows If Dates.Cells(i, 2).Value = "B" Then dateToday = Dates.Cells(i, 1).Value dateYear = Year(dateToday) stringOpening = "P:\Options Database\CSV\" & dateYear & "\bb_" & dateYear & "_" & GetMonth(dateToday) & "\bb_options_" & Format(dateToday, "yyyymmdd") & ".csv" Set wB = Workbooks.Open(stringOpening, UpdateLinks:=0, ReadOnly:=False) Do Until wB.ReadOnly = False wB.Close Application.Wait Now + TimeValue("00:00:01") Set wB = Application.Workbooks.Open("C:\My Files\AAA.xls") Loop wB.ReadOnly = True Set Options = wB.Sheets(1) Do 'things... Loop wB.Close SaveChanges:=False End If Next i End Sub
If you want to open the file and use it immediately Excel might give an error because Excel activates file opening process and goes to execute next statement. A quick and dirty workaround for not very long files is to introduce an extra code that is not related to a file thus keeping Excel busy while file is going through the opening process.
Create new worksheet if does not exist, rename based on cell value, then reference that worksheet
I have 2 workbooks one has the vba (MainWb), the other is just a template (TempWb) that the code paste values and formulas from the mainworkbook. The TempWb only has one blank sheet named graphs. The code needs to open the xltx file (TempWb), add a sheet and rename based on value in a certain cell on the MainWb (if it does not already exist) and then to reference that new sheet in the copy values process from the MainWb. I tried recording a macro but it didn't really help. I have researched and put some stuff together but not sure if it fits and works. Any suggestions would be appreciated. This is what I have so far. Option Explicit Sub ExportSave() Dim Alpha As Workbook 'Template Dim Omega As Worksheet 'Template Dim wbMain As Workbook 'Main Export file Dim FileTL As String 'Test location Dim FilePath As String 'File save path Dim FileProject As String 'Project information Dim FileTimeDate As String 'Export Date and Time Dim FileD As String 'Drawing Number Dim FileCopyPath As String 'FileCopy save path Dim FPATH As String 'File Search Path Dim Extract As Workbook 'File Extract Data Dim locs, loc 'Location Search Dim intLast As Long 'EmptyCell Search Dim intNext As Long 'EmptyCell Seach Dim rngDest As Range 'Paste Value Range Dim Shtname1 As String 'Part Platform Dim Shtname2 As String 'Part Drawing Number Dim Shtname3 As String 'Part Info Dim rep As Long With Range("H30000") .Value = Format(Now, "mmm-dd-yy hh-mm-ss AM/PM") End With FilePath = "C:\Users\aholiday\Desktop\FRF_Data_Macro_Insert_Test" FileCopyPath = "C:\Users\aholiday\Desktop\Backup" FileTL = Sheets("Sheet1").Range("A1").Text FileProject = Sheets("Sheet1").Range("E2").Text FileTimeDate = Sheets("Sheet1").Range("H30000").Text FileD = Sheets("Sheet1").Range("E3").Text FPATH = "C:\Users\aholiday\Desktop\FRF_Data_Macro_Insert_Test\" Shtname1 = wbMain.Sheets("Sheet1").Range("E2") Shtname2 = wbMain.Sheets("Sheet1").Range("E3") Shtname3 = wbMain.Sheets("Sheet1").Range("E4") Select Case Range("A1").Value Case "Single Test Location" Case "Location 1" Application.DisplayAlerts = False Set wbMain = Workbooks("FRF Data Export Graphs.xlsm") wbMain.Sheets("Sheet1").Copy ActiveWorkbook.SaveAs Filename:=FileCopyPath & "\" & FileProject & Space(1) & FileD & Space(1) & FileTL & Space(1) & FileTimeDate & ".xlsx", FileFormat:=xlOpenXMLWorkbook ActiveWorkbook.Close False Set Alpha = Workbooks.Open("\\plymshare01\Public\Holiday\FRF Projects\Templates\FRF Data Graphs.xltx") For rep = 1 To (Worksheets.Count) If LCase(Sheets(rep)).Name = LCase(Shtname1 & Space(1) & Shtname2 & Space(1) & Shtname3) Then MsgBox "This Sheet already exists" Exit Sub End If Next Sheets.Add after:=Sheets(Sheets.Count) Sheets(ActiveSheet.Name).Name = Shtname1 & Space(1) & Shtname2 & Space(1) & Shtname3 Set Omega = Workbooks(ActiveWorkbook.Name).Sheets("ActiveWorksheet.Name") locs = Array("FRF Data Export Graphs.xlsm") 'set the first data block destination Set rngDest = Omega.Cells(3, 1).Resize(30000, 3) For Each loc In locs Set Extract = Workbooks.Open(Filename:=FPATH & loc, ReadOnly:=True) rngDest.Value = Extract.Sheets("Sheet1").Range("A4:D25602").Value Extract.Close False Set rngDest = rngDest.Offset(0, 4) 'move over to the right 4 cols Next loc With ActiveWorksheet.Range("D3:D25603").Formula = "=SQRT((B3)^2+(C3)^2)" ActiveWorkbook.Charts.Add ActiveChart.ChartType = xlXYScatterLines ActiveChart.SetSourceData Source:=Sheets("Graphs").Range("A3:D7"), PlotBy:=xlRows ActiveChart.Location Where:=xlLocationAsNewSheet, Name:=Shtname2 With ActiveChart .HasTitle = True .ChartTitle.Characters.Text = Shtname2 .Axes(xlCategory, xlPrimary).HasTitle = True .Axes(xlCategory, xlPrimary).AxisTitle.Characters.Text = "Hz" .Axes(xlValue, xlPrimary).HasTitle = True .Axes(xlValue, xlPrimary).AxisTitle.Characters.Text = "Blank" End With Application.ScreenUpdating = True Case "Location 2" Case "Location 3" Case "Location 4" Case Else MsgBox "Export Failed!" End Select Application.DisplayAlerts = True End Sub Run-time error '91' Object variable or With block not set code lines Shtname1 = wbMain.Sheets("Sheet1").Range("E2") Shtname2 = wbMain.Sheets("Sheet1").Range("E3") Shtname3 = wbMain.Sheets("Sheet1").Range("E4") This is supposed to tell the code what to name the new created sheet Fixed: Moved under Set = wbMain = Workbooks("FRF Data Export Graphs.xlsm") New Error: Object doesnt support this property or method code If LCase(Sheets(rep)).Name = LCase(Shtname1 & Space(1) & Shtname2 & Space(1) & Shtname3) Then
A few things could be happening here Shtname1 = wbMain.Sheets("Sheet1").Range("E2") You are trying to access three objects and set a third. This means wbMain needs to be set and Sheets("Sheet1") need to be set and Range("E2") needs to exist. You also, because you are setting Shtname1 as a string I'd be explicit about what value you want to go in there. Shtname1 = wbMain.Sheets("Sheet1").Range("E2").Value So with the breakpoint on that line and the locals window open (View > Locals Window) make sure everything is set. If it's not it needs to be. One of those values is not set. If you do infact Set wbMain = Workbooks("FRF Data Export Graphs.xlsm") but it is in a different module or a different sub and wbMain is redeclared at the top of this sub these statements are in totally different contexts. The first wbMain is a different variable basically.
Copy data from closed workbook based on variable user defined path
I have exhausted my search capabilities looking for a solution to this. Here is an outline of what I would like to do: User opens macro-enabled Excel file Immediate prompt displays for user to enter or select file path of desired workbooks. They will need to select two files, and the file names may not be consistent After entering the file locations, the first worksheet from the first file selection will be copied to the first worksheet of the macro-enabled workbook, and the first worksheet of the second file selection will be copied to the second worksheet of the macro-enabled workbook. I've come across some references to ADO, but I am really not familiar with that yet. Edit: I have found a code to import data from a closed file. I will need to tweak the range to return the variable results. Private Function GetValue(path, file, sheet, ref) path = "C:\Users\crathbun\Desktop" file = "test.xlsx" sheet = "Sheet1" ref = "A1:R30" ' Retrieves a value from a closed workbook Dim arg As String ' Make sure the file exists If Right(path, 1) <> "\" Then path = path & "\" If Dir(path & file) = "" Then GetValue = "File Not Found" Exit Function End If ' Create the argument arg = "'" & path & "[" & file & "]" & sheet & "'!" & _ Range(ref).Range("A1").Address(, , xlR1C1) ' Execute an XLM macro GetValue = ExecuteExcel4Macro(arg) End Function Sub TestGetValue() path = "C:\Users\crathbun\Desktop" file = "test" sheet = "Sheet1" Application.ScreenUpdating = False For r = 1 To 30 For C = 1 To 18 a = Cells(r, C).Address Cells(r, C) = GetValue(path, file, sheet, a) Next C Next r Application.ScreenUpdating = True End Sub Now, I need a command button or userform that will immediately prompt the user to define a file path, and import the data from that file.
I don't mind if the files are opened during process. I just didn't want the user to have to open the files individually. I just need them to be able to select or navigate to the desired files Here is a basic code. This code asks user to select two files and then imports the relevant sheet into the current workbook. I have given two options. Take your pick :) TRIED AND TESTED OPTION 1 (Import the Sheets directly instead of copying into sheet1 and 2) Option Explicit Sub Sample() Dim wb1 As Workbook, wb2 As Workbook Dim Ret1, Ret2 Set wb1 = ActiveWorkbook '~~> Get the first File Ret1 = Application.GetOpenFilename("Excel Files (*.xls*), *.xls*", _ , "Please select first file") If Ret1 = False Then Exit Sub '~~> Get the 2nd File Ret2 = Application.GetOpenFilename("Excel Files (*.xls*), *.xls*", _ , "Please select Second file") If Ret2 = False Then Exit Sub Set wb2 = Workbooks.Open(Ret1) wb2.Sheets(1).Copy Before:=wb1.Sheets(1) ActiveSheet.Name = "Blah Blah 1" wb2.Close SaveChanges:=False Set wb2 = Workbooks.Open(Ret2) wb2.Sheets(1).Copy After:=wb1.Sheets(1) ActiveSheet.Name = "Blah Blah 2" wb2.Close SaveChanges:=False Set wb2 = Nothing Set wb1 = Nothing End Sub OPTION 2 (Import the Sheets contents into sheet1 and 2) Option Explicit Sub Sample() Dim wb1 As Workbook, wb2 As Workbook Dim Ret1, Ret2 Set wb1 = ActiveWorkbook '~~> Get the first File Ret1 = Application.GetOpenFilename("Excel Files (*.xls*), *.xls*", _ , "Please select first file") If Ret1 = False Then Exit Sub '~~> Get the 2nd File Ret2 = Application.GetOpenFilename("Excel Files (*.xls*), *.xls*", _ , "Please select Second file") If Ret2 = False Then Exit Sub Set wb2 = Workbooks.Open(Ret1) wb2.Sheets(1).Cells.Copy wb1.Sheets(1).Cells wb2.Close SaveChanges:=False Set wb2 = Workbooks.Open(Ret2) wb2.Sheets(1).Cells.Copy wb1.Sheets(2).Cells wb2.Close SaveChanges:=False Set wb2 = Nothing Set wb1 = Nothing End Sub
The function below reads data from a closed Excel file and returns the result in an array. It loses formatting, formulas etc. You might want to call the isArrayEmpty function (at the bottom) in your main code to test that the function returned something. Public Function getDataFromClosedExcelFile(parExcelFileName As String, parSheetName As String) As Variant 'see http://www.ozgrid.com/forum/showthread.php?t=19559 'returns an array (1 to nRows, 1 to nCols) which should be tested with isArrayEmpty in the calling function Dim locConnection As New ADODB.Connection Dim locRst As New ADODB.Recordset Dim locConnectionString As String Dim locQuery As String Dim locCols As Variant Dim locResult As Variant Dim i As Long Dim j As Long On Error GoTo error_handler locConnectionString = "Provider=Microsoft.Jet.OLEDB.4.0;" _ & "Data Source=" & parExcelFileName & ";" _ & "Extended Properties=""Excel 8.0;HDR=YES"";" locQuery = "SELECT * FROM [" & parSheetName & "$]" locConnection.Open ConnectionString:=locConnectionString locRst.Open Source:=locQuery, ActiveConnection:=locConnection If locRst.EOF Then 'Empty sheet or only one row ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' '''''' FIX: an empty sheet returns "F1" '''''' http://support.microsoft.com/kb/318373 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' If locRst.Fields.Count = 1 And locRst.Fields(0).Name = "F1" Then Exit Function 'Empty sheet ReDim locResult(1 To 1, 1 To locRst.Fields.Count) As Variant For i = 1 To locRst.Fields.Count locResult(1, i) = locRst.Fields(i - 1).Name Next i Else locCols = locRst.GetRows ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' '''''' FIX: an empty sheet returns "F1" '''''' http://support.microsoft.com/kb/318373 ''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''' If locRst.Fields.Count = 1 And locRst.Fields(0).Name = "F1" And UBound(locCols, 2) = 0 And locCols(0, 0) = "" Then Exit Function 'Empty sheet ReDim locResult(1 To UBound(locCols, 2) + 2, 1 To UBound(locCols, 1) + 1) As Variant If locRst.Fields.Count <> UBound(locCols, 1) + 1 Then Exit Function 'Not supposed to happen For j = 1 To UBound(locResult, 2) locResult(1, j) = locRst.Fields(j - 1).Name Next j For i = 2 To UBound(locResult, 1) For j = 1 To UBound(locResult, 2) locResult(i, j) = locCols(j - 1, i - 2) Next j Next i End If locRst.Close locConnection.Close Set locRst = Nothing Set locConnection = Nothing getDataFromClosedExcelFile = locResult Exit Function error_handler: 'Wrong file name, sheet name, or other errors... 'Errors (#N/A, etc) on the sheet should be replaced by Null but should not raise an error If locRst.State = ADODB.adStateOpen Then locRst.Close If locConnection.State = ADODB.adStateOpen Then locConnection.Close Set locRst = Nothing Set locConnection = Nothing End Function Public Function isArrayEmpty(parArray As Variant) As Boolean 'Returns false if not an array or dynamic array that has not been initialised (ReDim) or has been erased (Erase) If IsArray(parArray) = False Then isArrayEmpty = True On Error Resume Next If UBound(parArray) < LBound(parArray) Then isArrayEmpty = True: Exit Function Else: isArrayEmpty = False End Function Sample use: Sub test() Dim data As Variant data = getDataFromClosedExcelFile("myFile.xls", "Sheet1") If Not isArrayEmpty(data) Then 'Copies content on active sheet ActiveSheet.Cells(1,1).Resize(UBound(data,1), UBound(data,2)) = data End If End Sub