Copy/Paste rows to matching named sheet - vba

I have a worksheet "List" which has rows of data that I need to copy to other worksheets. In column "J" of "List", there is a name (Matthew, Mark, Linda, etc.) that designates who's data that row is.
Each of those names (22 in all) has a matching spreadsheet with the same name. I want all rows that say "Linda" in column "J" to paste to worksheet "Linda", all rows with "Matthew" to paste to worksheet "Matthew", etc.
I have some code below, which mostly works, but I'd have to rewrite it for all 22 names/sheets.
Is there a way to loop through all the sheets, pasting the rows with matching names? Also, the code below works really slowly, and I'm using data sets with anywhere from 200 to 60,000 rows that need sorted and pasted, which means that if its slow on a small data set like the one I'm currently working on, and only for one sheet, it's going to be glacially slow for the big data sets.
Sub CopyMatch()
Dim c As Range
Dim j As Integer
Dim Source As Worksheet
Dim Target As Worksheet
Set Source = Worksheets("List")
Set Target = Worksheets("Linda")
j = 4 ' Start copying to row 1 in target sheet
For Each c In Source.Range("J4:J1000") ' Do 1000 rows
If c = "Linda" Then
Source.Rows(c.Row).Copy Target.Rows(j)
j = j + 1
End If
Next c
End Sub

Unless you've turned calculation off somewhere we can't see here, then every time you copy a row, Excel is recalculating - even if your sheets contain no formulas.
If you're not doing so already, simply putting:
application.calculation=xlcalculationmanual
before you start your loop and:
application.calculation=xlcalculationautomatic
after exiting the loop will massively speed up your loop. For extra swank, you can use a variable to store the calculation setting before you turn it off and restore that setting at the end, e.g.
dim lCalc as long
lCalc = application.calculation
application.calculation = xlcalculationmanual
for ... next goes here
application.calculation = lCalc
Also consider other settings, e.g.: application.screenupdating=False|True.
Sort the data by the name you're selecting on, then by any other sorts you want. That way you can skip through any size sheet in 22 steps (since you say you have 22 names).
How you copy the data depends on preference and how much data there is. Copying one row at a time is economical on memory and pretty much guaranteed to work, but is slower. Or you can identify the top and bottom rows of each person's data and copy the whole block as a single range, at the risk of exceeding the memory available on large blocks in large sheets.
Assuming the value in your name column, for the range you're checking, is always one of the 22 names, then if you've sorted first by that column you can use the value in that column to determine the destination, e.g.:
dim sTarget as string
dim rng as range
sTarget = ""
For Each c In Source.Range("J4:J1000") ' Do 1000 rows
if c <> "" then ' skip empty rows
if c <> sTarget then ' new name block
sTarget = c
Set Target = Worksheets(c)
set rng = Target.cells(Target.rows.count, 10).end(xlup) ' 10="J"
j = rng.row + 1 ' first row below last name pasted
end if
Source.Rows(c.Row).Copy Target.Rows(j)
j = j + 1
end if
Next
This is economical of memory because you're going row by row, but still reasonably fast because you're only recalculating Target and resetting j when the name changes.

you could use:
Dictionary object to quickly build the list of unique names out of column J names
AutoFilter() method of Range object for filtering on each name:
as follows
Option Explicit
Sub CopyMatch()
Dim c As Range, namesRng As Range
Dim name As Variant
With Worksheets("List") '<--| reference "List" worskheet
Set namesRng = .Range("J4", .Cells(.Rows.count, "J").End(xlUp)) '<--| set the range of "names" in column "J" starting from row 4 down to last not empty row
End With
With CreateObject("Scripting.Dictionary") '<--| instance a 'Dictionary' object
For Each c In namesRng.SpecialCells(xlCellTypeConstants, xlTextValues) '<--| loop through "names" range cells with text content only
.item(c.Value) = c.Value '<--| build the unique list of names using dictionary key
Next
Set namesRng = namesRng.Resize(namesRng.Rows.count + 1).Offset(-1) '<--| resize the range of "names" to have a "header" cell (not a name to filter on) in the first row
For Each name In .Keys '<--| loop through dictionary keys, i.e. the unique names list
FilterNameAndCopyToWorksheet namesRng, name '<--| filter on current name and copy to corresponding worksheet
Next
End With '<--| release the 'Dictionary' object
End Sub
Sub FilterNameAndCopyToWorksheet(rangeToFilter As Range, nameToFilter As Variant)
Dim destsht As Worksheet
Set destsht = Worksheets(nameToFilter) '<--| set the worksheet object corresponding to passed name
With rangeToFilter
.AutoFilter Field:=1, Criteria1:=nameToFilter
Intersect(.Parent.UsedRange, .Resize(.Rows.count - 1).Offset(1).SpecialCells(xlCellTypeVisible).EntireRow).Copy destsht.Cells(destsht.Rows.count, "J").End(xlUp)
.Parent.AutoFilterMode = False
End With
End Sub

Related

VBA: Deleting particular columns in a range

I'm working on a project and I need a program that will delete the empty columns (other than the header) in columns A through F and column J. I'd like it to loop through Range("A10:F10000") and Range("J:J") and delete the columns from cell A10, B10, etc and down, then shift the remaining data left. I was working with this, that affects the whole worksheet instead of just a range:
ecl = Sheet1.Cells.SpecialCells(xlCellTypeLastCell).Column
For cl = ecl To 1 Step -1
cnt = Application.WorksheetFunction.CountA(Sheet1.Columns(cl))
If cnt = 1 Then
Sheet1.Columns(cl).Delete
End If
Next
I usually can use formulas and am not super familiar with VBA, but for this project it has to be a macro. I hope this makes sense and would appreciate any advise!
you could use this:
Dim cl As Range, colsToDelete As Range
With ActiveSheet 'reference wanted sheet
Set colsToDelete = .UsedRange.Columns(.UsedRange.Columns.Count).Offset(, 1).Resize(1, 1) ' initialize 'colsToDelete' to a "dummy" range certainly out of relevant one
With Intersect(.Range("A10:J" & .UsedRange.Rows(.UsedRange.Rows.Count).Row), .Range("A:F, J:J")) ' reference referenced sheet range in column A to F and G form row 10 down to referenced sheet last not empty cell row
For Each cl In .Columns ' lop through referenced range columns
If Application.WorksheetFunction.CountA(cl) = 1 Then Set colsToDelete = Union(colsToDelete, cl) 'if current column is empty (i.e. only header in row 10) then add it to the colsToDelete range
Next
Set colsToDelete = Intersect(colsToDelete, .Cells) ' get rid of the "dummmy" range
End With
End With
If Not colsToDelete Is Nothing Then colsToDelete.EntireColumn.Delete ' if any range to delete, do it!

Looping through filter criteria for a column and performing analysis on visible data to copy to new sheet - VBA Macro

Effectively what I am trying to accomplish is to filter a database (all in Sheet 1) by names, which are given in column A, and then perform various data analysis on the now filtered visible data, and then copy that data to a new sheet in a given cell.
For example, filter the data in Sheet 1 by the name 'Smith' in column A and then let's say sum all of the visible data in column B and print that to cell C3 on Sheet 2. The more advanced data analysis I am sure I can tackle on my own, just want to get rolling here and I am definitely new to VBA macro coding. I have created all of these databases using Python.
The last piece of this, would be being able to loop through all of the filter criteria in column A (which I will not know before-hand and may be anywhere from 10-20 names.
Here is the code I am working with (there are likely some syntax errors in here as well):
Option Explicit
Sub Data()
Dim playername As String
Dim team As String
Dim numFilters As Integer
Dim hits As Integer
Dim src As Worksheet
Dim tgt As Worksheet
Dim filterRange As Range
Dim copyRange As Range
Dim lastRow As Long
Dim i As Integer
team = ThisWorkbook.Sheets(1).Name
numFilters = ActiveSheet.AutoFilter.Filters.Count ' I want this to capture the num of filter criteria for column A
For i = 1 To numFilters
playername = Sheets(team).Filter(i) ' This would be the filter criteria for the given iteration
ActiveSheet.Range("$A$1:$AN$5000").AutoFilter field:=1, Criteria1:=playername
' Create new sheet with name of person
Sheets.Add After:=ActiveSheet
ActiveSheet.Select
ActiveSheet.Name = playername
Set tgt = ThisWorkbook.Sheets(i + 1)
' Perform data analysis (e.g. sum column B of filtered data)
src.AutoFilterMode = False
' Find the last row with data in column A
lastRow = src.Range("A" & src.Rows.Count).End(xlUp).Row
' The range that we are auto-filtering (all columns)
Set filterRange = src.Range("A1:AN" & lastRow)
' Set the range to start in row 2 to prevent copying the header
Set copyRange = src.Range("B2:B" & lastRow)
' Copy the sum of column B to our target cell on the sheet corresponding to this iteration
Application.WorksheetFunction.Sum(copyRange.SpecialCells(xlCellTypeVisible)).Copy tgt.Range("A1")
Next i
End Sub
This is currently failing on the Application.WorksheetFunction.Sum line with the error 'Invalid qualifier'. Thanks for any help and please let me know if something needs clarified.

Extract Row Locations to Use as Reference

I populated an excel sheet with the locations of blank cells in my sheet using suggestions from this post. So I have a Column A filled with locations in the following format
$X$1 or $X2:$X$4.
What I am trying to do is use those row numbers from the column explain above to populate a separate column. I want to use the row numbers as a reference in what to populate for the column. So a Column B looking something like
=$B$1 or =$B$2:$B$4 (took 1 and 2-4 and used it as row number for reference call)
Both columns are referencing a different sheet so please excuse any column naming.
I'm not sure if this is going to require VBA or if I can get away with just using a formula, I expect VBA due to desired specifics. I've looked at post like this and this. But neither of these fully encompass what I'm looking for. Especially since I want it to express all the contents in a $B$2:$B$4 case.
My intuition on how to solve this problem tells me, parse the string from Column A for the 1st number then check if it's the end of the string. If it is, feed it to the reference that populates Column B, if not then find the 2nd number and go through a loop that populates the cell (would prefer to keep all the content in one cell in this case) with each value for each reverence.
i.e.
=$B2
=$B3
=$B4
My question is how do I go about this? How do I parse the string? How do I generate the loop that will go through the necessary steps? Such as using the number as a reference to pull information from a different column and feed it neatly into yet another column.
If (for example) you have an address of $X2:$X$4 then
Dim rng As Range
Set rng = yourSheetReference.Range("$X2:$X$4")
If you want to map that to the same rows but column B then
Set rng = rng.Entirerow.Columns(2)
will do that. note: it's not so clear from your question whether you're mapping X>>B or B>>X.
Once you have the range you want you can loop over it:
For Each c in rng.Cells
'do something with cell "c"
next c
Something like this should work for you:
Sub Tester()
Dim shtSrc As Worksheet, c As Range, rng As Range, c2, v, sep
Set shtSrc = ThisWorkbook.Worksheets("Sheet1") '<< source data sheet
Set c = ActiveSheet.Range("A2") '<<range addresses start here
'process addresses until ColA is empty
Do While c.Value <> ""
'translate range to (eg) Column X
Set rng = shtSrc.Range(c.Value).EntireRow.Columns(24)
sep = ""
v = ""
'build the value from the range
For Each c2 In rng.Cells
v = v & sep & c2.Value
sep = vbLf
Next c2
c.Offset(0, 1) = v '<< populate in colB
Loop
End Sub
Try this code:
Sub Test()
Dim fRng As Range ' the cell that has the formula
Set fRng = Worksheets("sheet1").Range("A1")
Dim tWS As Worksheet 'the worksheet that has the values you want to get
Set tWS = Worksheets("sheet2")
Dim r As Range
For Each r In Range(fRng.Formula).Rows
'Debug.Print r.Row ' this is the rows numbers
Debug.Print tWS.Cells(r.Row, "N").Value 'N is the column name
Next
End Sub

Speed up macro for large files (over 90000 rows, 236 columns)

I wrote a macro that compares the columns B, which contains file numbers, in two worksheets. There are three possibilities: the file number exists in both columns, the file number exists only in the first column and the file number exists only in the second column. If e.g. the file number exists in both columns, the macro should copy/paste the entire row to another sheet. Same for the other two scenario's.
My code work perfect for a small file (around 500 rows, 236 columns), but for the large files it doesn't work. It takes way too long, and at the end it just crashes. I already tried the usual tricks to speed up the macro.
Option Explicit
Sub CopyPasteWorksheets()
Dim wbDec As Workbook, wbJune As Workbook, wbAnalysis As Workbook
Dim wsDec As Worksheet, wsJune As Worksheet
Dim PresPres As Worksheet, PresAbs As Worksheet, AbsPres As Worksheet
'Stop screen from updating to speed things up
Application.ScreenUpdating = False
Application.EnableEvents = False
'Add 3 new worksheets. They each represent a different category, namely the one with already existing insurances, one with new insurances
'and one with the insurances that are closed due to mortality, lapse or maturity. Add two (temporary) worksheets to paste the databases.
Worksheets.Add().Name = "PresPres"
Worksheets.Add().Name = "PresAbs"
Worksheets.Add().Name = "AbsPres"
Worksheets.Add().Name = "DataDec"
Worksheets.Add().Name = "DataJune"
'Define the active workbook
Set wbAnalysis = ThisWorkbook
'Define the first database. Copy/paste the sheet and close them afterwards.
Set wbDec = Workbooks.Open(Filename:="F:\Risk_Management_2\Embedded_Value\2015\20151231\Data\DLL\Master Scala\Extract.xlsx")
wbDec.Sheets("SCALA").Range("A1").CurrentRegion.Copy
wbAnalysis.Sheets("DataDec").Range("A1").PasteSpecial xlPasteValues
wbDec.Close
'We have to do the same for the other database. We cannot do it at the same time, because both files have the same name,
'and can't be opened at the same time.
Set wbJune = Workbooks.Open(Filename:="F:\Risk_Management_2\Embedded_Value\2016\20160630\Data\DLL\Master Scala\extract.xlsx")
wbJune.Sheets("SCALA").Range("A1").CurrentRegion.Copy
wbAnalysis.Sheets("DataJune").Range("A1").PasteSpecial xlPasteValues
wbJune.Close
Application.ScreenUpdating = True
Application.EnableEvents = True
End Sub
Sub Compare()
Dim DataDec As Worksheet, DataJune As Worksheet
Dim lastRowDec As Long
Dim lastRowJune As Long
Dim lastRowPresAbs As Long
Dim lastRowPresPres As Long
Dim lastRowAbsPres As Long
Dim foundTrue As Boolean
Dim i As Long, j As Long, k As Long, l As Long
'Define the last row of the different sheets
lastRowDec = Sheets("DataDec").Cells(Sheets("DataDec").Rows.Count, "B").End(xlUp).Row
lastRowJune = Sheets("DataJune").Cells(Sheets ("DataJune").Rows.Count, "B").End(xlUp).Row
lastRowPresAbs = Sheets("PresAbs").Cells(Sheets("PresAbs").Rows.Count, "B").End(xlUp).Row
lastRowPresPres = Sheets("PresPres").Cells(Sheets ("PresPres").Rows.Count, "B").End(xlUp).Row
lastRowAbsPres = Sheets("AbsPres").Cells(Sheets("AbsPres").Rows.Count, "B").End(xlUp).Row
'Compare the file numbers in column B of both sheets. If they are the same, copy/paste the entire row to sheet PresPres,
'if they are not, copy/paste the entire row to sheet PresAbs.
For i = 1 To lastRowDec
foundTrue = False
For j = 1 To lastRowJune
If Sheets("DataDec").Cells(i, 1).Value = Sheets("DataJune").Cells(j, 1).Value Then
foundTrue = True
Sheets("PresPres").Rows(lastRowPresPres + 1) = Sheets("DataDec").Rows(i)
lastRowPresPres = lastRowPresPres + 1
Exit For
End If
Next j
If Not foundTrue Then
Sheets("DataDec").Rows(i).Copy Destination:= _
Sheets("PresAbs").Rows(lastRowPresAbs + 1)
lastRowPresAbs = lastRowPresAbs + 1
End If
Next i
'Look if there are file numbers that are only present in June's database. If so, copy/paste entire row to sheet AbsPres.
For k = 1 To lastRowJune
foundTrue = False
For l = 1 To lastRowDec
If Sheets("DataJune").Cells(k, 1).Value = Sheets("DataDec").Cells(l, 1).Value Then
foundTrue = True
Exit For
End If
Next l
If Not foundTrue Then
Sheets("DataJune").Rows(k).Copy Destination:= _
Sheets("AbsPres").Rows(lastRowAbsPres + 1)
lastRowAbsPres = lastRowAbsPres + 1
End If
Next k
'Stop screen from updating to speed things up.
Application.ScreenUpdating = True
Application.EnableEvents = True
End Sub
I've added some comments to explain what I'm trying to do. I'm relatively new to VBA so I believe I'm not coding very efficient.
Could someone have a look and try to make it work?
Basically what your are doing is comparing 2 column of elements, you want to know when:
an element is in both columns
an element is only in the first column
an element is only in the second column
To do that, your solution do:
For each element in column 1,
Find if there is this element in column 2
If found, it is in both, if not, it's just in 1
Continue to next element in column 1
Do quite the same with the element of the column 2
So basically, your examining column 2 for each element of column 1
And the same for the column 1 with the element of column 2
if we consider n the length of column1 and m the length of column2.
That is roughly 2*m*n comparison.
That's a lot !
My solution:
You are looking for numbers in column B.
Therefore you can sorted both sheet base on the value in column B
Then you can:
Create counter1 and counter2 referring to the current row in sheet1 and sheet2
Compare the value of sheet1.Value('B' + counter1) to sheet2.Value('B' + counter2)
Then you have 3 choice :
a) That is the same value, then copy the line in the right file and increments both counter
b) Value from sheet1 is greater, then you will never find the value from sheet2 in sheet1. So copy the line of sheet2 in the right file and increment only the counter2
c) The opposite
Do that until counter1 or counter2 is at the end.
As it is possible that both won't be at the end at the same time, you will have to copy the remaining lines in the right file as they will never be in the "finished" sheet.
With that solution, you will only read each "column" once ! So roughly about m+n comparison :)
You win a lot of time :)
With M=n=90 000:
you have a solution with about m*n=8 100 000 000 comparison
the other solution is just about 180 000 comparison
This should be the fastest approach as copying all data at once is much faster than copying it by row.
Select both columns > Home tab > Conditional Formatting > Highlight Cell Rules > Duplicate Values...
Now you need a filter from Data > Filter, but for that you will need to insert a header row above the numbers. After you have the filter, you can click on the second column filter and Filter by Color. Now you can copy the visible cells to wherever you copy the duplicates. I recommend sorting by color too before copying as copying one contiguous area should be a bit faster.
You can use the same method for the other two cases by filtering the columns with Filter by Color > No Fill.
Before you Record Macro of the process you can select View tab > Macros > Use Relative References.
Edit
I think I misunderstood the question. This method needs both columns to be next to each other, so if they are in separate sheets you can copy and insert them in column A. You can hide the column after the filter is applied. Then you can delete the column and header rows if needed.
Similar approach without conditional formatting is to inset a column with a helper function that checks if the id exists in the other sheet, but I think it will be a bit slower. For example:
= CountIf( Sheet2!A1:A1234, B2 )
I received an answer to my question on the Mr. Excel forum:
http://www.mrexcel.com/forum/excel-questions/963415-visual-basic-applications-speed-up-macro-large-file.html
Thanks for your answers!

How to loop a dynamic range and copy select information within that range to another sheet

I have already created a VBA script that is about 160 lines long, which produces the report that you see below.
Without using cell references (because the date ranges will change each time I run this) I now need to take the users ID, name, total hours, total break, overtime 1, and overtime 2 and copy this data into sheet 2.
Any suggestions as to how I can structure a VBA script to search row B until a blank is found, when a blank is found, copy the values from column J, K, L, M on that row, and on the row above copy value C - now paste these values on sheet 2. - Continue this process until you find two consecutive blanks or the end of the data...
Even if you can suggest a different way to tackle this problem than the logic I have assumed above it would be greatly appreciated. I can share the whole code if you are interested and show you the data I began with.
Thank you in advance,
J
As discussed, here's my approach. All the details are in the code's comments so make sure you read them.
Sub GetUserNameTotals()
Dim ShTarget As Worksheet: Set ShTarget = ThisWorkbook.Sheets("Sheet1")
Dim ShPaste As Worksheet: Set ShPaste = ThisWorkbook.Sheets("Sheet2")
Dim RngTarget As Range: Set RngTarget = ShTarget.UsedRange
Dim RngTargetVisible As Range, CellRef As Range, ColRef As Range, RngNames As Range
Dim ColIDIndex As Long: ColIDIndex = Application.Match("ID", RngTarget.Rows(1), 0)
Dim LRow As Long: LRow = RngTarget.SpecialCells(xlCellTypeLastCell).Row
'Turn off AutoFilter to avoid errors.
ShTarget.AutoFilterMode = False
'Logic: Apply filter on the UserName column, selecting blanks. We then get two essential ranges.
'RngTargetVisible is the visible range of stats. ColRef is the visible first column of stats.
With RngTarget
.AutoFilter Field:=ColIDIndex, Criteria1:="=", Operator:=xlFilterValues, VisibleDropDown:=True
Set RngTargetVisible = .Range("J2:M" & LRow).SpecialCells(xlCellTypeVisible)
Set ColRef = .Range("J2:J" & LRow).SpecialCells(xlCellTypeVisible)
End With
'Logic: For each cell in the first column of stats, let's get its offset one cell above
'and 7 cells to the left. This method is not necessary. Simply assigning ColRef to Column C's
'visible cells and changing below to CellRef.Offset(-1,0) is alright. I chose this way so it's
'easier to visualize the approach. RngNames is a consolidation of the cells with ranges, which we'll
'copy first before the stats.
For Each CellRef In ColRef
If RngNames Is Nothing Then
Set RngNames = CellRef.Offset(-1, -7)
Else
Set RngNames = Union(RngNames, CellRef.Offset(-1, -7))
End If
Next CellRef
'Copy the names first, then RngTargetVisible, which are the total stats. Copying headers is up
'to you. Of course, modify as necessary.
RngNames.Copy ShPaste.Range("A1")
RngTargetVisible.Copy ShPaste.Range("B1")
End Sub
Screenshots:
Set-up:
Result:
Demo video here:
Using Filters and Visible Cells
Let us know if this helps.