I've done some google searching and looked around stackoverflow but haven't come across something which resolves the below.
Basically I have around 46 workbooks and worksheets which I need to merge into one for analysis purposes by another software.
wbBank.Sheets(c).Copy Before:=wbStructure.Worksheets(wbStructure.Sheets.Count)
For some of these worksheets in 'wbStructure', I need to do a copy functionality (per ISO currency or country) which means that from 46 respective worksheets, I end up with around 1300 worksheets for this particular workbook that is ending up consuming too much memory.
For each worksheet I also do the following:
Insert Rows / Columns depending on parameters set
go through each row and columns and put in descriptions
The functionality of all of this used to work correctly, however I had to change the functionality recently to keep the same excel object rather than opening a new Excel instance due to compatibility issues between Excel 2016 and Excel 2010.
Dim oXLApp As Object
Set oXLApp = GetObject(,"Excel.Application")
The reason I need to use is that I need to get data from the current Excel document which contains the macro and copy paste this data through the other application. when I tried placing data in another object and then pasting the data, the formatting is lost and creates an issue.
Are there any pointers one can give me to reduce the memory consumption which is resulting in unusual behavior; randomly crashes throughout, issues of "'Not enough system resources to display correctly" etc. The final workbook is substantially large but it has to remain so due to this being an input for another system
Any feedback is highly appreciated.
Related
I'm trying to run a duplicate check In which varying data is pulled from a website and compared to a master list, the master list being stored in Excel. The information from the website is read from a table in which has line breaks. These breaks are translated over to the data collection they are initially stored in. Some of the data from the website us eventually written to the master list in Excel. So when I read the master list back into Blue Prism to run a duplicate check, the rows that have line breaks are written into a collection as multiple rows (ex. I should have on 7 rows in my collections but am getting 42). Since the rows are not EXACTLY the same between the 2 collections, when it runs the automation does not recognize the duplicates.
The easiest way to solve this would be if I could make the collection rows have no line breaks as soon as the data is read. I've attempted to use the calculation stage to do so with no luck. I'm not sure if it is actually possible to do this, but would appreciate any direction.
Record an Excel macro to do the data sorting/cleaning in Excel (possibly Text To Columns, etc..) and then include the running of the macro as part of your Blue Prism process by using an action stage and the MS Excel VBO - Run Macro. Get the process to create an Excel instance (and create a handle data item from that stage), then use Open Workbook (whatever workbook you store your Macro in) and then use the MS Excel VBO - Run Macro (use the same handle created earlier and type in the name of the "macro").
It sounds like what is happening is that the MS Excel VBO is grabbing the data from the Excel Worksheet wholesale.
This is to say that it's accessing your Worksheet table, copying the cell values BUT not the cell formatting data, and then dumping the values into a BP collection.
Since it did not bring along any of the original cell formatting data to reference when it went to populate the collection it's just breaking up the values based on crturn/line breaks. Thus, your collection is organized based on that, and not on the original Worksheet cell.
So, with that said, on to a solution!
Solution 1
Brute force the organization of the incoming Excel cell data to the collection by looping over the Excel Worksheet cell-by-cell.
Run a loop, and in that loop have BP go into the Excel Worksheet and grab the first populated cell it comes across. Run a formatting/cleanup Calculation stage over the data. Dump the cell value into a single collection field.
Repeat.
This is...inelegant, expensive at best, and not at all recommended for any medium to large dataset. But it's definitely the best way to do string manipulation and value comparisons before it hits your collection. Since it sounds like your using a Master template then you as-well know what the expected format of your data should be.
This method will enable you implement Trim(), Concat(), or Split() in a Calculation stage to better organize your incoming data before you dump it into a collection.
This is also basically what I think you're already trying to do, but cell-by-cell instead of Worksheet row-by-row or table-by-table.
Solution 2
Clean up the table data you grab from the website before you dump it into the Excel Worksheet.
This is basically Solution 1, but in reverse. Simply format/cleanup your data before it hits you Excel Worksheet.
I'm not sure this is any better than Solution 1, but, you know, it's something...
Solution 3
Format the cell data IN the MS Excel Worksheet itself.
Basically rearrange the cells and cell data in the Excel Worksheet into a more predictable format by using the Split, Trim, Merge, or other actions included in the MS Excel VBO. You can also do this using the Data - OLEDB utility object, but that requires some pretty solid understanding of SQL syntax.
This would look like this using the MS Excel VBO:
Grab the Excel Worksheet data wholesale and dump into a collection
Count the rows/fields of the collection
Is that number consistent with the desired/expected format of your data?
If not, have the bot go back into the Excel Worksheet and reformat the cells by removing any carriage returns/line breaks/whatever else
Repeat.
However, I'm always reluctant to reformat any original source, as it's then hard to figure out what wrong and where it went wrong when you've changed the original structure of your data. So it's best to always make a copy of the Worksheet before you make any manipulation.
Unfortunately I don't have access to my BP environment at the moment or I'd provide you with the act object actions you'd need to do any of this, my bad. Once I do I'll update this answer.
I have an 11 sheet excel workbook that both a coworker and I need to update at the same time. It has a large amount of formulas, some macros, and conditional formatting. We both need to enter data into different areas and if we could work at the same time, we could get much more done in a smaller time frame.
My questions are:
Should something with a lot of formulas and formatting be subject to a shared workbook?
Can I reverse the process?
Since we are using this just to enter numbers, could we enter the numbers and then unshare the document when we need to properly update the form?
I have a system where several "little Indians" put their stats into their own spread sheet and those stats are written to a delimited text file, so when the "Chief Indian" opens his workbook he can press a button to go import all of those text file stats. Perhaps you can try something like that. I've found that using the shared workbook function is a problem because someone will eventually come along and try to change something on a sheet and mess up all the formula addresses.
I used the delimited text files because our shared server was 500 miles away and opening 18 different excel workbooks to gather data took about 30 minutes, while opening 18 small text files only took about a minute. But if your shared server is local then you can also open the other workbooks and gather their data fairly quickly.
I disabled the "X" close button and added a button on the "little Indians" sheet that closes their workbook and writes the text file. If a text file has already been written for that date it will ask if they want to add the new data to the old or replace the old data with this new data. Of course they can cancel also. It's a lot of extra coding but it prevents "little Indians" from making changes that destroys the formula addresses.
Current Situation: I am consolidating a data package that receives multiple inputs from programs that roll into mine. Essentially, each program provides me their set of data and I am responsible for accurately consolidating data into one Excel workbook in a certain format. I have pushed out a template so that all the data comes back in the same format, but now I am having trouble quickly consolidating besides using "copy and paste".
Solutions I've looked into: I've looked into using connections to see if can pull in the data I need through a connected file however it imports the table in a different format and does not transfer correctly into the consolidation template that I have set up. I tried doing external source referencing to pull in data that way, but again the formatting is off. I know there are ways to just pull in the workbook, but I need data from specific points in these program workbooks that I can slot into my consolidated workbook. In most cases, copy and pasting would work, but I have over hundreds of programs that flow into this data sheet so any help would be appreciated!
How about setting the destination range to the values of the source range?
e.g.:
rngTarget.Resize(rngSource.Rows.Count, rngSource.Columns.Count).Value = rngSource.Value
this one's a bit of a painful one so thank you for your help and patience with me.
We have an Excel spreadsheet that we use as a master file for our website products. As such there are quite a few sheets and quite a few products on each running along side some macros to provide some extra functionality (turning entered data into HTML for product page, etc).
My issue is that one of our most used spreadsheets has become a trouble in that it has some phantom formatting all the way down to the millionth-and-something row and all the way across, causing the last cell to be the very last cell possible.
The issue that has finally popped up as a result is that we can no longer move rows in, out or around the sheet (a required functionality) as it results in an 'out of resources error'.
I've tried:
Highlight all rows below used range to right-click> delete - Results in runtime error (from macro)
Highlighting large chunks of rows and using Clear All - Resulted in the 38MB file bloating to 380MB
Deleting a chunk of rows at a time - Maxed out at 1,000 before it caused Excel to crash
Moving to new spreadsheet - Broke all our macros (which I did not write and am not proficient enough to fix on a new sheet)
Disabling macros and trying the above options, only marginally more efficient but still out of resources
I'm at my wits end on this one and, while we can continue with most day-to-day functions, we will soon be completely unable to use this particular sheet as we need it at all.
I'm wondering if there might be a way to run a VBA script to remove these rows, potentially one by one? I've tried running a short script that went something like rows[960,1000000].Delete (forgive my terrible VBA markup), but this also resulted in not enough resources errors.
I'm wondering if there's anything like:
row = 960;
while(row<=1048576){row.Delete};
Continuing, the runtime error debug points me to the below if statement within the macro:
If Target.Count > 1 Then Exit Sub
Where Target is the variable passed to the sub.
Which strikes me as very odd because my (limited) understanding of VBA and IF's in general simply recognizes that 'if my selection is larger than 1 (row?), do not run this code..
Thanks again in advance.
Use this method only if you don't have any links into or out of the sheet that will get broken. Also might have Sql connections that might get broken. Might need to disable macros. There are many possible problems with this approach. Use at your own risk.
Note the exact "Name" and "(Name)" of the sheet; Look in the VBA code window at the properties for the sheet. "Name" is the name displayed on the worksheet tab. "(Name)" is the code name visible only in the properties window.
Make a list of range names on the sheet.
Copy the data to a new sheet.
Copy any macros to the new sheet.
Delete the old sheet.
Rename the "Name" and "(Name)" of the new sheet the same as the old one.
Recreate range names.
A better method if you don't have too many formats:
Disable macros and set calculation to manual. This avoids recalculating while doing your delete operation.
Select entire sheet and clear formats.
Delete all rows below your data.
Redo your formatting. Select entire column (not just used area) to apply format if applicable.
It is important to remove formatting on the entire sheet from A1 to the end. Otherwise you'll get the bloat you mentioned. Just that step may solve your problem. If not then proceed with removing all the rows below the data. This should not cause file size bloat.
I have an Excel workbook with a number of features:
One main user-facing sheet
One summary sheet based on the user-facing sheet's data
A number of graphs based on the user-facing sheet's data (as in, the type of graphs with a separate tab for them, rather than objects within a worksheet - I'm not sure if they have a special name or special properties)
A series of 'background' worksheets that calculate the values for the user-facing sheet
A macro to allow the user to sort the user-sheet by any column they wish, which is referenced in the user-facing sheet's Worksheet_SelectionChange event
However, for distribution I'd like to cull the sheets for simplicity (and file size - the entire data query is included on one of the sheets). I still need to calculate the values for the user-facing sheet, but it's only done once per dataset, so that can quite happily be copied as formatting then values.
The trouble, however, is transferring the dependent sheet, graphs and macros across to a new workbook so that instead of referencing the old workbook, they reference the new versions of the sheet. Ideally I'd like to do this with VBA or something, but my Google searches thus far don't seem to have turned up much of relevance.
Does anyone know how to do this?
I'm not sure I entirely understand your question, but what I think that you want to do is to create a new version of your workbook, with less worksheets in it??
To do that in VBA, this code snippet is a good place to start:
Sub Macro1()
ActiveWorkbook.SaveAs Filename:= _
[your path here]\Book2.xls", FileFormat:=xlNormal, _
Sheets("Sheet2").Select
ActiveWindow.SelectedSheets.Delete
End Sub
Creating a copy of the entire workbook, and then deleting what you don't need will make sure that the links do not reference the old workbook.
Once you have created the new workbook, with links intact, it is then pretty easy to remove all the things (calculations, etc.) that you don't need.