this one's a bit of a painful one so thank you for your help and patience with me.
We have an Excel spreadsheet that we use as a master file for our website products. As such there are quite a few sheets and quite a few products on each running along side some macros to provide some extra functionality (turning entered data into HTML for product page, etc).
My issue is that one of our most used spreadsheets has become a trouble in that it has some phantom formatting all the way down to the millionth-and-something row and all the way across, causing the last cell to be the very last cell possible.
The issue that has finally popped up as a result is that we can no longer move rows in, out or around the sheet (a required functionality) as it results in an 'out of resources error'.
I've tried:
Highlight all rows below used range to right-click> delete - Results in runtime error (from macro)
Highlighting large chunks of rows and using Clear All - Resulted in the 38MB file bloating to 380MB
Deleting a chunk of rows at a time - Maxed out at 1,000 before it caused Excel to crash
Moving to new spreadsheet - Broke all our macros (which I did not write and am not proficient enough to fix on a new sheet)
Disabling macros and trying the above options, only marginally more efficient but still out of resources
I'm at my wits end on this one and, while we can continue with most day-to-day functions, we will soon be completely unable to use this particular sheet as we need it at all.
I'm wondering if there might be a way to run a VBA script to remove these rows, potentially one by one? I've tried running a short script that went something like rows[960,1000000].Delete (forgive my terrible VBA markup), but this also resulted in not enough resources errors.
I'm wondering if there's anything like:
row = 960;
while(row<=1048576){row.Delete};
Continuing, the runtime error debug points me to the below if statement within the macro:
If Target.Count > 1 Then Exit Sub
Where Target is the variable passed to the sub.
Which strikes me as very odd because my (limited) understanding of VBA and IF's in general simply recognizes that 'if my selection is larger than 1 (row?), do not run this code..
Thanks again in advance.
Use this method only if you don't have any links into or out of the sheet that will get broken. Also might have Sql connections that might get broken. Might need to disable macros. There are many possible problems with this approach. Use at your own risk.
Note the exact "Name" and "(Name)" of the sheet; Look in the VBA code window at the properties for the sheet. "Name" is the name displayed on the worksheet tab. "(Name)" is the code name visible only in the properties window.
Make a list of range names on the sheet.
Copy the data to a new sheet.
Copy any macros to the new sheet.
Delete the old sheet.
Rename the "Name" and "(Name)" of the new sheet the same as the old one.
Recreate range names.
A better method if you don't have too many formats:
Disable macros and set calculation to manual. This avoids recalculating while doing your delete operation.
Select entire sheet and clear formats.
Delete all rows below your data.
Redo your formatting. Select entire column (not just used area) to apply format if applicable.
It is important to remove formatting on the entire sheet from A1 to the end. Otherwise you'll get the bloat you mentioned. Just that step may solve your problem. If not then proceed with removing all the rows below the data. This should not cause file size bloat.
Related
I am working with a huge excel file that is updated with a set of macros. In the excel file there are also a large number of graphs to ensure easy output checks.
However, when I re-calculate the workbook it is extremely slow.
My question is: Do these graphs contribute to slowing down the calculation of the model? If so, is there a quick VBA way to only update graphs at the end of the overall calculation?
Without seeing your workbook this is hard to answer.
Most likely, it is not the charts (is that what you call "graphs") that are slowing down the recalc, but inefficient formulas.
Check the chart data sources. If they point to worksheet cells, then all is good. If they point to named ranges / named formulas, then check what these formulas are.
Recalculation is affected by
volatile formulas like Today(), Now(), Indirect(), Offset() and a few others
inefficient formulas that needlessly repeat calculations that have already been performed, typically done in running totals
And example of this would be
=Sum(A$1:A2) copied down, like in this screenshot
In each row, the calculation starts in row 1 and goes down to the current row. This is a waste of effort.
A much more efficient formula is in column C, where just the value from the row above is added to the value of the current row.
=SUM(C1,A2)
These details can make a heck of a difference.
For more information you may want to refer to Charles Williams' site http://www.decisionmodels.com/calcsecrets.htm and the pages linked from there.
It's a complex subject and can probably not be addressed in a simple answer to a seemingly simple question.
I'm trying to run a duplicate check In which varying data is pulled from a website and compared to a master list, the master list being stored in Excel. The information from the website is read from a table in which has line breaks. These breaks are translated over to the data collection they are initially stored in. Some of the data from the website us eventually written to the master list in Excel. So when I read the master list back into Blue Prism to run a duplicate check, the rows that have line breaks are written into a collection as multiple rows (ex. I should have on 7 rows in my collections but am getting 42). Since the rows are not EXACTLY the same between the 2 collections, when it runs the automation does not recognize the duplicates.
The easiest way to solve this would be if I could make the collection rows have no line breaks as soon as the data is read. I've attempted to use the calculation stage to do so with no luck. I'm not sure if it is actually possible to do this, but would appreciate any direction.
Record an Excel macro to do the data sorting/cleaning in Excel (possibly Text To Columns, etc..) and then include the running of the macro as part of your Blue Prism process by using an action stage and the MS Excel VBO - Run Macro. Get the process to create an Excel instance (and create a handle data item from that stage), then use Open Workbook (whatever workbook you store your Macro in) and then use the MS Excel VBO - Run Macro (use the same handle created earlier and type in the name of the "macro").
It sounds like what is happening is that the MS Excel VBO is grabbing the data from the Excel Worksheet wholesale.
This is to say that it's accessing your Worksheet table, copying the cell values BUT not the cell formatting data, and then dumping the values into a BP collection.
Since it did not bring along any of the original cell formatting data to reference when it went to populate the collection it's just breaking up the values based on crturn/line breaks. Thus, your collection is organized based on that, and not on the original Worksheet cell.
So, with that said, on to a solution!
Solution 1
Brute force the organization of the incoming Excel cell data to the collection by looping over the Excel Worksheet cell-by-cell.
Run a loop, and in that loop have BP go into the Excel Worksheet and grab the first populated cell it comes across. Run a formatting/cleanup Calculation stage over the data. Dump the cell value into a single collection field.
Repeat.
This is...inelegant, expensive at best, and not at all recommended for any medium to large dataset. But it's definitely the best way to do string manipulation and value comparisons before it hits your collection. Since it sounds like your using a Master template then you as-well know what the expected format of your data should be.
This method will enable you implement Trim(), Concat(), or Split() in a Calculation stage to better organize your incoming data before you dump it into a collection.
This is also basically what I think you're already trying to do, but cell-by-cell instead of Worksheet row-by-row or table-by-table.
Solution 2
Clean up the table data you grab from the website before you dump it into the Excel Worksheet.
This is basically Solution 1, but in reverse. Simply format/cleanup your data before it hits you Excel Worksheet.
I'm not sure this is any better than Solution 1, but, you know, it's something...
Solution 3
Format the cell data IN the MS Excel Worksheet itself.
Basically rearrange the cells and cell data in the Excel Worksheet into a more predictable format by using the Split, Trim, Merge, or other actions included in the MS Excel VBO. You can also do this using the Data - OLEDB utility object, but that requires some pretty solid understanding of SQL syntax.
This would look like this using the MS Excel VBO:
Grab the Excel Worksheet data wholesale and dump into a collection
Count the rows/fields of the collection
Is that number consistent with the desired/expected format of your data?
If not, have the bot go back into the Excel Worksheet and reformat the cells by removing any carriage returns/line breaks/whatever else
Repeat.
However, I'm always reluctant to reformat any original source, as it's then hard to figure out what wrong and where it went wrong when you've changed the original structure of your data. So it's best to always make a copy of the Worksheet before you make any manipulation.
Unfortunately I don't have access to my BP environment at the moment or I'd provide you with the act object actions you'd need to do any of this, my bad. Once I do I'll update this answer.
I have a very specific question that I am looking for help in answering. I have been researching for hours and I feel like I am not able to find what I am looking for. Below is a quick overview of the criteria that my document must follow:
I am using Excel 2013
I will just be using rows for data input (instead of an excel object
table).
The very top/first row will act as my "column header".
This top row will have AutoFilter enabled.
THE DOCUMENT MUST BE PROTECTED (a must-have)!
I will be using VBA code
Now, the final issue I am having with finishing this document are the last two criteria points that I must have:
The first/top row (column headers) must NOT BE EDITABLE.
Each column must be able to SORT AND FILTER.
Now, in a perfect world, I would just "Lock Cells" for the entire first row that acts as my column headers and when I protect the worksheet I would make sure to check the "Sort" and "Use AutoFilter" boxes.
However, this option does not work because there seems to be an issue when I try to sort the data. If I just filter the data there is no problem, but when I try to sort a column in ascending/descending order I will get an error informing me that I can't sort locked cells while in Protected mode. This is because when excel uses the Sort function, it counts the header as part of the data that is being sorted (I found this out through my research) even though I really just want the data below it to be sorted.
I have been trying to brainstorm on how to get past this issue as well as researching different methods, and I am having trouble coming to a final conclusion. However I have narrowed it down to 2 possible solutions:
I want to be able to keep the cells in the first row officially unlocked to allow the AutoFilter's sort command to work as intended, but make it "behave" like the cells are locked when a user tries to make changes to it (AKA, make the entire row un-editable or un-selectable).
The other option would be to keep the first row locked but somehow have an event in VBA that can tell when a user tries to "Sort" the column, which will then temporarily unprotect the worksheet, follow through with the intended sort command, then protect the worksheet again (apparently though, upon my research there is no such event that can trigger off the AutoFilter's sort command alone).
These 2 solutions are the most logical I can think of based off my research, but if someone out there is an Excel genius and knows another way I am open to suggestions.
Thanks in advance for your help/suggestions,
Travo
Consider using two header rows. The top row would be protected and the second row would facilitate filtering and sorting of the data in rows 3 and below.
We have a lot of automated spreadsheets that extract data, calculate forecasts, publish web pages, etc. It's a really messy system, and I'd love to redo it (with, say, a real web service), but we don't have permission to do that.
We run Excel 2007, with most of the spreadsheets converted to .xlsms.
Anyway, for some reason unknown to me, certain spreadsheets will format certain numeric cells as dates at seemingly random times. As you might imagine, the problem is difficult to track down, and usually I only find out when someone has written a nasty email about our data reading 3-Feb-1901 when it should read 400 (apparently Excel thinks there was a leap year in 1900).
I've explicitly set the cells in question to numeric format several times, only to find them filled with dates a week later.
I've scanned the macros (many are in the ancient Excel 4.0 macro language), they appear to be clean of any formatting changes. All copy/pastes are done as values and do not preserve source cell formatting.
There are no conditional formatting rules in the sheets in question.
Adding a little VBA to format the cells as desired in the Auto_Open subroutine appears to work around the issue.
Is this fix good enough, or in a week will I find the date format is appearing mid-calculation? If not, what is the root cause and how I can fix it?
One thing: I just read from this source that Excel '07 may spontaneously change Normal-styled cells to the format [$-409]m/d/yy h:mm AM/PM;#. Some of my cells that have changed were definitely styled Normal, but evidently not all.
I'm assuming the cells in question are the receipients of copied/pasted data, or external/queried data?
Whenever you add pasted data to a cell in excel, it runs througha routine to determine the best format for anything that's listed as "general" format, even if the source is "general" or "number". Sometimes even setting the destination cells to "number" format does not clear up this issue. But I've found the only way to to get this to work the way you need it to is to explicitly declare the format (in the macro) to the destination range in question before the paste/update/refresh operation.
I also think there's a way to disable date recognition within your VBA, but I can't remember the exact code. Try recording a macro while doing manually importing or performing your update and you should be able to use what comes up.
Edit: just did a test run, and after your connection string and destination add:
.WebDisableDateRecognition = True
That should definitely do the trick if you're pulling data in from an external source and not copy/pasting.
Just in case this helps, i had an issue where excel was auto formating my info pulled from a webpage into DATE format, to fix, in the import window, click options and UNCHECK the auto date formating option, after that, my data imported perfect!
We have are relatively simple Reporting Services report that our users commonly export to Excel. I've noticed that the files produced by the Excel export seem unusually large. If I open one of these files and just click save, without making any changes, the file size reduces to about half of it's previous size. Has anyone else run into this and is there a known workaround?
You've mentioned that the report is relatively simple, but this is important to check. The export to Excel will go to extraordinary lengths to try and maintain how your report looks.
If you have lots of different borders or colours (particularly if different formatting is determined by the data in your report) this will bloat the file.
Also check if many columns with very small and unusual sizes are created in the exported worksheet. The export does this to try and match alignment in Excel with the original report.
Try recreating your report as a basic table with no formatting or headers/footers and see if you can reproduce the problem. If Excel's behaviour is acceptable then add each piece of formatting back until it goes awry. Please let us know what you find.
I don't have an immediate solution, but a common problem in Excel is files bloating because one/some/all of the worksheets have saved all 64K rows instead of the ones being used. The fix in Excel is to select all the lower rows not being used, and delete them, then save the spreadsheet, close and reopen. Therefore, I'd pursue the angle of extra rows being saved in the export, and see if there is a way to keep this from happening.
What tool are you using when exporting to Excel?
I have also managed to reduce # of rows in my Excel worksheet by copying it to another worksheet, then deleting the original sheet.
You could also try copying only the data in your worksheet, and paste it into a new Excel Workbook (file).
I had the issue where the exported Excel files took and extremely long to open and they would stop responding every time you clicked on a cell.
Also, extra and merged columns would appear in exported excel files.
The fix was to make sure my header text boxes lined up with the beginning and end of columns in the data table just below it. Once both were aligned, there were no more extra columns in the exported excel spreadsheets and the performance was back to normal.
Here's the reference that helped me understand the issue:
http://www.codegur.press/12747988/issue-report-export-to-excel-in-rdlc-report
May be I am answering your question very late. Here's the solution for exporting to CSV.
You need to give the a name that you want to see as a column header for the field (not the column name) in the designer.
By default all the text headers are exported as a separate columns along with the table columns and make sure that you name the Design name in the properties with the name you want to see.
The other important thing to note about the option DataElementOutput which is set to Auto meaning it will be exported. You can change that if you don't want it to be exported.
The last but not least thing ... after you export the data looks messed up. You need select the whole first column and go to the Data tab - > convert text to column -> use the delimiter as comma and say Finish. That should solve your issue.