I coded a VBA script in Excel which adds new data into a Datasheet with previous information. Before doing that, the new data is copied into a provisional Datasheet. To prevent duplicates, I create an additional column and do a VLOOKUP of IDs. If the ID from the new imported data is already in the Datasheet with the old data, this row is marked as duplicated and will be deleted. The "non-duplicated rows" are then copied into the final Datasheet, where all the data is stored.
Right now I use a column reference (A:A) in the VLOOKUP and I don´t know if maybe this is the reason why the VBA script needs every day more resources and time to run. When I coded for the first time, I did the test with no more than 4,000 rows in the original Datasheet and 4,000 rows in the imported data. The macro was done after 90 seconds. Right now, it needs more than 5 minutes and the Datasheet with data is just 40,000 rows large, while the new data is always around 4,000 rows.
Should I dynamically reference the range in the VLOOKUP instead of using A:A or it doesn't matter in terms of speed?
As mentioned in my comment, there certainly is a way to accomplish this task using VBA, but sometimes the simpliest solution is best. I would reccomend added all 40K records each time and using the "Remove Duplicates" function under the "Data" ribbon using the column that holds your unique value.
Related
We keep our orders & production in Excel. An order workbook looks something like this:
Main Order XXXX
Cabinet
Solid Surface
Diewall
Custom
Metal
Finishing
Moulding
Adaptable
On our Main Order tab, we have a table with all the items on the order. What I need, is a way to automate copying certain rows to certain sheets. We have column on the Main Order that can be used to state which sheet that row needs to go to. Also worth noting, we sometimes have to ADD new sheets if we need more items that need to go on 2 separate Custom tabs for example (but this could apply to any of the sheet names other than Main Order). We do that by copying the applicable sheet and renaming it 'Custom1', 'Custom2'. It's the same format for each sheet - a number after the name of the sheet, no space. Right now, we are doing this all manually and it's very time consuming. Would it be possible to make the script create the new sheet based on what is in the Main Order? Meaning, if we put 'Custom1' in the Main Order table, can the script create the sheet called 'Custom1', copying the 'Custom' sheet and then adding the rows from the Main Order?
I'm not sure if it matters, but the Main Order sheet table starts at Row 18 and is variable in length depending on the order. There's another table below that, but that doesn't need to be copied anywhere. The tables where that rows need to be copied on the other sheets all start at Row 13 (that's the header row) and we can add however many columns we need.
I will admit I'm very new to scripts and VBA, but I have a very basic understanding (I think anyway). I feel like what I'm looking for should be possible, I just don't know enough to figure it out on my own at this point.
I found a script that copies rows to a table, but I need it to be more dynamic. I don't want to have to create a script for each sheet, especially since new sheets can be created as stated above. Below is the script I already have, and I confirmed it worked for the 1 sheet ('Adaptable'). I'm hoping someone can help me make this more 'generic' if that makes sense? So in our Main Order, I want to be able to put in a column the sheet name that the row needs to be copied to and then have it automatically copied, or if the user needs to click 'Run Script' or whatever, that would still be much easier and faster than what we're currently doing.
My problem is as follows, I have a workbook with (to begin with) 2 worksheets, the first (called WIP) acts as a form of data entry, each new occurrence of the BoM requires an insertion of 4 columns which happens to the left of the existing columns. At the same time a new worksheet is created based on a copy of the existing second worksheet (called FitOut) which pulls various bits of a data from the first worksheet based mainly upon the version of the BoM selected and the supplier referenced.
Of course adding new columns to the WIP sheet causes the functions, arrays and formulas in the sheets to automatically update, I had used a quick workaround by using some code to hold and then paste the new occurrences data into the worksheet which is created at the start of the macro, however the formulas have become slightly complex (due to the need to look for the previous 4 occurrences and return values based on specific cell locations) that the 256 character limit has been completely shot ( I think I'm over 800 on some bits).
I've very limited as to the layout of the WIP sheet, and the sheet needs to be fairly idiot proof (hence macros, buttons etc) but it needs to run well...
ANy and all suggestions/help would be much appreciated.
I have put an example of the formula I am trying to use, if it can be condensed further pleas let me know:
=IFERROR(IF($C$1='WIP'!$U$1,(INDEX('WIP'!$A$1:$X$2500,SMALL(IF('WIP'!$U$1:$U$2500=$A$1,ROW('WIP'!$U$1:$U$2500)),ROW(6:6)),COLUMN('WIP'!$C:$C))),(IF($C$1='WIP'!$X$1,(INDEX('WIP'!$A$1:$AA$2500,SMALL(IF('WIP'!$X$1:$X$2500=$A$1,ROW('WIP'!$X$1:$X$2500)),ROW(6:6)),COLUMN('WIP'!$C:$C))),(IF($C$1='WIP'!$AA$1,(INDEX('WIP'!$A$1:$AD$2500,SMALL(IF('WIP'!$AA$1:$AA$2500=$A$1,ROW('WIP'!$AA$1:$AA$2500)),ROW(6:6)),COLUMN('WIP'!$C:$C))),(INDEX('WIP'!$A$1:$AG$2500,SMALL(IF('WIP'!$AD$1:$AD$2500=$A$1,ROW('WIP'!$AD$1:$AD$2500)),ROW(7:7)),COLUMN('WIP'!$C:$C)))))))),"")
In my current Database I have a table whose data is manually entered or comes in an excel sheet every week. Before we had the "manual entry option", the table would be dropped and replaced by the excel version.
Now because there is data that only exists in the original table this can not be done.
I'm trying to find a way to update the original table with changes and additions from the (excel) table while preserving all rows not in the new sheet.
I've been attempting to simply use an insert query and an update query /but/ I can't find a way to detect changes in a record.
Any suggestions? I can provide the current sql if you'd find that helpful.
Based on what I have read so far, I think I can offer some suggestions:
It appears you have control of the MS Access. I would suggest adding a field to your data table called "source". Modify your form in the access database to store something like "m" for manual entry in the source field. When you import the excel, store an "e" for excel in the field.
You would need to do a one time scrub of the data to mark existing records as manual entries or excel entries. There are a couple of ways you can do it through automation/queries that I can explain in detail if you want.
Once past these steps, your excel process is fairly simple. You can delete all records with source = "e" and then do a full excel import. Manual records would remain unchanged.
This concept will allow you to add new sources and codes and allow you to handle each differently if needed. You just need to spend some time cleaning up your old data. I think you will find it worth it in the end.
Good Luck.
I have a excel sheet which we may keep adding rows/ deleting them.
And I have an average value present in some cell.I would want the excel formula to identify if there is text in another column to average the columns
So now if I insert another row, I have to manually update the average formula.
Is there a way to have a formula which check if column A is not empty, it should consider the data in column G for the average
There's a lot of approaches to this. My current favourite is a CELL:INDEX(...) expression. For instance, to find the last populated cell in the first continuously populated range between B1 and B5000, I would use (probably as a named range) $B$1:INDEX($B$1:$B$500,MATCH(TRUE, $B$1:$B$500="", 0)-1).
This approach is great because it's non volatile, so it shouldn't bog your worksheet down. It might be vulnerable to $B$500 gradually shrinking if you're only ever deleting rows, though. Alternatives are referencing the whole column ($C:$C), but that's usually dog slow in modern excel, or using OFFSET which never shrinks, but is volatile.
I have a Worksheet with 10 columns and data range from A1:J55. Col A has the invoice # and rest of the columns have other demographic data. Goal is to type the invoice number on a cell and display all the rows matching the invoice number from col A.
Besides auto filter function, the only thing comes to my mind is VBA. Please advice what is the best way to get the data. Thanks for your help in advance.
Alright, I'm pretty proud of this one. Again avoiding VBA, this one uses the volatile formula OFFSET to keep moving its VLOOKUP search down the table until it's found all matches. Just make sure you paste enough rows of the formula that if there are many matches, there's room for all of them to appear. If you put a border around your match area then it would be clear if you ever ran out of room and needed to copy down the formula some more.
Again, in the main section, it's just a single formula (using index):
=IFERROR(INDEX($A$1:$J$200,$M3,MATCH(N$2,$A$1:$J$1,0)),"")
This gets to be so simple because the hard work of the lookup is done by an initial column which looks up the next row that matches the invoice number. It has the formula:
=IFERROR(MATCH($L$2,OFFSET($A$1:$A$200,M2,0),0)+M2," ")
Here is the working example that goes with those formulas:
Let me know if you need any further description of how it works, but it mostly uses the same rules as above so that it's robust in copying and moving around.
I've uploaded the Excel file so you can play with it, but everything you need to reproduce this feature should be in this solution.
Google Docs - Click link and hit Ctrl+S to download and open in Excel.
A popular solution to this problem is a simple VLookup. Lookup the invoice the user types in on the table A1:J55, and then return an adjascent column's data.
Here's an example of it working:
The formula in the highlighted cell is:
=VLOOKUP($L3,$A:$J,MATCH(N$2,$1:$1,0),FALSE)
What's nice about this formula is you only need to type it once and then you can copy it across and it'll automatically pick out the correct column of the table (that's the match part). The rest is very simple:
The first part says lookup value $L3 (the invoice number typed in),
The second part says look it up in range $A:$J (which is where your table is located). I've shown how you can select the entire columns $A:$J so that you can add and remove data without worrying about adjustin the range in your lookups. (Excel takes care of optimizing the formula so that unused cells aren't checked)
The third part picks the column from which the resulting data will be drawn once a matching row is found.
The FALSE part is an indication that the invoice number must match exactly (no approximate matching allowed)
The $ signs ensure that fixed ranges like the location of your source table ($A:$J) and your lookup value ($L3) don't get automatically changed as you copy the formula across for multiple columns.
The formula is pretty easy to adapt if you want to move around your table and the area where you do your lookup. Here's an example:
Bonus
If you want to add a little spiff, you can add a dropdown to the Invoice # field so that the user gets auto-completion and the option to browse existing values like so: