VBA: Copy specific range from multiple workbooks into one worksheet - vba

I have a folder with lots (hundreds) of locked .xls files.
I need to copy a specific range from one of the worksheets in each file into one big worksheet, which would be my data file for future analysis.
I tried to write a macro for this, but keep getting errors.
Please help me debug what I wrote:
Sub ProcessFiles()
' declarations & definitions
Dim Pathname As String
Dim Filename As String
Dim sourceWB As Workbook
Dim targetWB As Workbook
targetWB = ActiveWorkbook
Pathname = ActiveWorkbook.Path & "\Files\"
Filename = Dir(Pathname & "*.xls")
' loop through all files in folder
Do While Filename <> ""
Set sourceWB = Workbooks.Open(Pathname & Filename)
' unlock worksheets
sourceWB.Sheets(4).Visible = True
sourceWB.Sheets(4).Unprotect Password:="Password"
sourceWB.Sheets(2).Unprotect Password:="Password"
' create new worksheet
sourceWB.Sheets.Add After:=8
' copy required cells to new sheets
sourceWB.Sheets(2).Range("A14:FM663").Copy Destination:=sourceWB.Sheets(9).Range("C2")
' fill columns for all rows
sourceWB.Sheets(9).Range("A2:A663").Value = sourceWB.Name
sourceWB.Sheets(9).Range("B2:B663").Value = Worksheets(4).Range("C13").Value
'move AuxSheet to taget workbook
sourceWB.Sheets(9).Move Before:=Workbooks(targetWB).Sheets(1)
'add to full data worksheet
targetWB.Sheets(1).Range("A2:FO651").Copy Destination:=sourceWB.Sheets(2).Rows("3:" & Worksheets("Sheet2").UsedRange.Rows.Count)
'close file and repeat
sourceWB.Close SaveChanges:=False
Filename = Dir()
Loop
' save result
targetWB.Save
End Sub

Just to give you an idea of how tasks like this can be handled way more efficient... consider the following that I always use for tasks like this:
Option Explicit
' 1. Add reference to Microsoft Scripting Runtime and Access Data Objects Library via Extras>References
Sub ProcessFiles()
Dim strCon As String
Dim strSQL As String
Dim fso As New Scripting.FileSystemObject
Dim myfile As file
With ThisWorkbook
' 2. empty your outputsheet
.Sheets("out").Cells.Clear
' 3. loop the files in your folder
For Each myfile In fso.GetFolder(.Path & Application.PathSeparator & "Files").Files
' 3.1. no proper way to filter files like in Dir(), but we want to use the file objects
If myfile.Name Like "*.xls" Then
' 3.1.1. Construct the connection string, the only variable part is myfile.Path
strCon = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" & myfile.Path & ";Extended Properties='Excel 8.0;HDR=YES';"
' 3.1.2. Construct the SQL String. Luckily, you already know where your data is
strSQL = "SELECT '" & myfile.Name & "' AS WorkbookName, * FROM [sheetData$A1:C5], (SELECT TOP 1 * FROM [sheetSchool$C12:C13])"
' 3.1.3. Call the get-data sub from below
GetData .Sheets("out"), strCon, strSQL
End If
Next myfile
End With
End Sub
Sub GetData(ByRef wsOut As Variant, strCon As String, strSQL As String)
Dim i As Integer
On Error GoTo skpError
Application.ScreenUpdating = False
' Create a new database connection
Dim objCon As New ADODB.Connection
With objCon
.ConnectionString = strCon
.Open
End With
' Create a new database command
Dim objCmd As New ADODB.Command
With objCmd
.ActiveConnection = objCon
.CommandType = adCmdText
.CommandText = strSQL
Debug.Print .CommandText
End With
' Create a new recordset
Dim objRS
Set objRS = New ADODB.Recordset
With objRS
.ActiveConnection = objCon
.Open objCmd
End With
' Print your FieldNames, in case they're not already there
With wsOut
If wsOut.Cells(1, 1).Value = vbNullString Then
For i = 1 To objRS.Fields.Count
.Cells(1, i).Value = _
objRS.Fields(i - 1).Name
Next i
End If
' Output your data - pretty ugly, but reliable
.Range("A1048576").End(xlUp).Offset(1, 0).CopyFromRecordset (objRS)
End With
skpNoError:
Application.ScreenUpdating = True
Exit Sub
skpError:
MsgBox "Error #" & Err & vbNewLine & Error, vbCritical
GoTo skpNoError
End Sub
Notes: (why use something like this?)
protected and hidden worksheets shouldn't be a problem with this. For protected workbooks, a password parameter can be added to the connection string
this will be considerably faster for a large number of files than opening, editing, copying would be. If you feel fancy, you can further speed things up by moving stuff from the GetData-Sub to ProcessFiles, so they won't get called repeatedly.
you use a database language for querying data instead of some clumsy copy/paste mechanism.
Edit:
Edited my code, for me this works with the example you gave.
From what i gather, you only got protected Worksheets, not a password-protected Workbook - therefore there's no need to unhide or unprotect your worksheets
adjust the line strSQL = "SELECT '" & myfile.Name & "' AS WorkbookName, * FROM [sheetData$A1:C5], (SELECT TOP 1 * FROM [sheetSchool$C12:C13])" to contain your actual Sheets(2) and Sheets(4) names

Related

Last Header cell not copied by using ADO to read and write data in Excel workbooks?

I use below code to copy data from closed workbook ("Sheet1") using ADO to read and write data in Excel workbooks .
the data copied successfully as my specified requirements except Last Header cell.
I tried to change HDR=NO to HDR=Yes in ADO connection , But the same problem.
As always: great thanks for your help.
Public Sub GetData(SourceFile As Variant, SourceSheet As String, _
SourceRange As String, TargetRange As Range, Header As Boolean, UseHeaderRow As Boolean)
Dim rsCon As Object, rsData As Object
Dim szConnect As String, szSQL As String
Dim lCount As Long
szConnect = "Provider=Microsoft.ACE.OLEDB.12.0;" & _
"Data Source=" & SourceFile & ";" & _
"Extended Properties=""Excel 12.0;HDR=NO"";"
If SourceSheet = "" Then 'Workbook level name
szSQL = "SELECT * FROM " & SourceRange$ & ";"
Else
' worksheet level name or range
szSQL = "SELECT * FROM [" & SourceSheet$ & "$" & SourceRange$ & "];"
End If
On Error GoTo SomethingWrong
Set rsCon = CreateObject("ADODB.Connection")
Set rsData = CreateObject("ADODB.Recordset")
rsCon.Open szConnect
rsData.Open szSQL, rsCon, 0, 1, 1
If Not rsData.EOF Then ' Check to make sure we received data and copy the data
If Header = False Then
TargetRange.Cells(1, 1).CopyFromRecordset rsData
Else
End If
Else: MsgBox "No records returned from : " & SourceFile, vbCritical
End If
rsData.Close ' Clean up our Recordset object.
Set rsData = Nothing
rsCon.Close
Set rsCon = Nothing
Exit Sub
SomethingWrong:
MsgBox "The file name, Sheet name or Range is invalid of : " & SourceFile, _
vbExclamation, "Error"
On Error GoTo 0
End Sub
Sub GetData_Example4() 'Select one file with GetOpenFilenamewhere
Dim SaveDriveDir As String, MyPath As String
Dim FName As Variant
SaveDriveDir = CurDir
MyPath = Application.DefaultFilePath
ChDrive MyPath
ChDir MyPath
FName = Application.GetOpenFilename(filefilter:="Excel Files, *.xl*")
If FName = False Then
'do nothing
Else
GetData FName, "Sheet1", "A1:D5", Sheets("Sheet1").Range("A1"), False, False
End If
ChDrive SaveDriveDir
ChDir SaveDriveDir
End Sub
That header is likely missing because ADO has decided that column is numeric and so the header gets auto-converted to null because it's not numeric. You're telling ADO that row1 is part of the data when you use HDR=No.
You can try moving it's position in the source data and it should still show that behavior.
You really don't want ADO to treat your headers like they're part of your dataset, so you need to either skip them in your SQL (by excluding the header row from the range you supply) or use HDR=Yes in the connection.
If using HDR=Yes then you'll need to add some code to your sub to read each field name in the recordset and populate a header row on the results sheet before using CopyFromRecordSet.

Read CSV/Excel file into array

I am trying to make a macro which copies emails when I receive them, and saves them in specific windows folders on a network drive based on the domain name.
The list of domains I have will be large and subject to change by users without coding experience, so I am looking to develop a text, CSV, or excel file that someone can update which lists my company's relationship to them (client, vendor, sub-contractor, etc...) and their name (both of which controls the file path), the domain name (#example.com).
I think I can figure out how to do most of that (a clever combination of nested if and for statements), but I can't figure out how to read the file into an array, and my google-fu has failed me.
I don't think it really helps, but here is the code that I shamelessly copied from the web and am planning to work off of.
Option Explicit
Private WithEvents InboxItems As Outlook.Items
Sub Application_Startup()
Dim xNameSpace As Outlook.NameSpace
Set xNameSpace = Outlook.Application.Session
Set InboxItems = xNameSpace.GetDefaultFolder(olFolderInbox).Items
End Sub
Private Sub InboxItems_ItemAdd(ByVal objItem As Object)
Dim FSO
Dim xMailItem As Outlook.MailItem
Dim xFilePath As String
Dim xRegEx
Dim xFileName As String
Dim SenderAddress As String
On Error Resume Next
' Define SenderAddress as sender's email address or domain
xFilePath = PathCreator(SenderAddress)
Set FSO = CreateObject("Scripting.FileSystemObject")
If FSO.FolderExists(xFilePath) = False Then
FSO.CreateFolder (xFilePath)
End If
Set xRegEx = CreateObject("vbscript.regexp")
xRegEx.Global = True
xRegEx.IgnoreCase = False
xRegEx.Pattern = "\||\/|\<|\>|""|:|\*|\\|\?"
If objItem.Class = olMail Then
Set xMailItem = objItem
xFileName = xRegEx.Replace(xMailItem.Subject, "")
xMailItem.SaveAs xFilePath & "\" & xFileName & ".html", olHTML
End If
Exit Sub
End Sub
Function PathCreator(SenderAddress)
' [needs to read the file and create the path based on the values]
End Function
You can use ADODB to connect to the source file, and read it into a 2-dimensional array. Add a reference to Microsoft ActiveX Data Objects from Tools -> References.... For example, if you want to use an Excel file:
Dim excelPath As String
excelPath = "C:\path\to\excel\file.xlsx" ' Replace with the path to the Excel file
Dim connectionString As String
connectionString = _
"Provider=Microsoft.ACE.OLEDB.12.0;" & _
"Data Source=""" & excelPath & """;" & _
"Extended Properties=""Excel 12.0;HDR=Yes"""
'This assumes the Excel file contains column headers -- HDR=Yes
Dim sql As String
sql = "SELECT Relationship, LastName, FirstName, DomainName FROM [Sheet1$]"
'Assumes the relevant worksheet is named Sheet1
'Also assumes the first row of the sheet has the following labels: Relationship, LastName, FirstName, Domain (in no particular order)
Dim rs As New ADODB.Recordset
rs.Open sql, connectionString
Dim arr As Variant
arr = rs.GetRows 'Puts the data from the recordset into an array
rs.Close
Set rs = Nothing
Dim row As Variant, column As Variant
For row = 0 To UBound(arr, 2)
For column = 0 To UBound(arr, 1)
Debug.Print arr(column, row)
Next
Next
Using a text file or CSV is just a matter of slightly changing the connection string and the SQL. But I think using an Excel file will force the users to keep the data in columns, where in a CSV users would have to insert field- and row-separators manually; the same for any other text format -- users would have to remember the format's rules and apply them correctly.
But I question if an array is the best data structure for you to use; in this case you could use the recordset directly. In order to make sure the file is not held open, you could use a disconnected recordset. (If your intention is to find the appropriate domain name and use that to get other details, then I would suggest you load the data from a recordset into a Scripting.Dictionary.)
Also note that you probably only need to load the data from the file once, unless you expect it to change while the code is running.
I would write something like this
Dim rs As ADODB.Recordset
Function PathCreator(SenderAddress) As String
If rs Is Nothing Then
Dim excelPath As String
excelPath = "C:\path\to\excel\file.xlsx" ' Replace with the path to the Excel file
Dim connectionString As String
connectionString = _
"Provider=Microsoft.ACE.OLEDB.12.0;" & _
"Data Source=""" & excelPath & """;" & _
"Extended Properties=""Excel 12.0;HDR=Yes"""
Dim sql As String
sql = "SELECT Relationship, LastName, FirstName, DomainName FROM [Sheet1$]"
Set rs As New ADODB.Recordset
rs.CursorLocation = adUseClient
rs.CursorType = adOpenStatic
rs.Open sql, connectionString, adOpenStatic, adLockBatchOptimistic
'Disconnect the recordset
rs.ActiveConnection = Nothing
'Now the data will still be available as long as the code is running
'But the connection to the Excel file will be closed
End If
'build the path here, using the recordset fields
PathCreator = rs!Relationship & "_" & rs!LastName & "_" & rs!FirstName & "_" & rs!Domain
End Function
NB. By the same token, you can add a reference to Microsoft Scripting Runtime; then you can write the code that uses the FileSystemObject as follows:
Dim FSO As New Scripting.FileSystemObject
If Not FSO.FolderExists(xFilePath) Then
FSO.CreateFolder xFilePath
End If
and with a reference to the Microsoft VBScript Regular Expressions 5.5 libary:
Set xRegEx As New VBScript_RegExp_55.RegExp
xRegEx.Global = True
xRegEx.IgnoreCase = False
xRegEx.Pattern = "\||\/|\<|\>|""|:|\*|\\|\?"
If objItem.Class = olMail Then
Set xMailItem = objItem
xFileName = xRegEx.Replace(xMailItem.Subject, "")
xMailItem.SaveAs xFilePath & "\" & xFileName & ".html", olHTML
End If

Reference a cell if the sheet contains a certain string using VBA

the code below copies "ADXL364" sheet in my active worksheet, but is there way that I can copy the sheet if it contains "XL364" or "364"
if I put asterisk 'C:\data[adxl364.xls]*ADXL364_QC'!A1 in my code it does not work.
Sub GetRange()
With Range("A:Z")
.Formula = "=If('C:\data\[adxl364.xls]ADXL364_QC'!A1 > 0,'C:\data\[adxl364.xls]ADXL364_QC'!A1,Text(,))"
.Formula = .Value
End With
End Sub
the long code will be getting the location of file from the user then copying a worksheet that contains ADXL364 or XL364
With ActiveWorkbook
Sheets.Add.Name = "Flow_table"
Application.EnableEvents = False
TP_location = Left(TextBox1.Value, InStrRev(TextBox1.Value, "\"))
TP_filename = Right(TextBox1.Value, Len(TextBox1.Value) - InStrRev(TextBox1.Value, "\"))
TP_filename = "[" & TP_filename & "]"
TP_formula = "'" & TP_location & TP_filename & TextBox2.Value & "'!A1"
getcellvalue = "=if(" & TP_formula & ">0," & TP_formula & "," & """"")"
With Range("A:Z")
.Formula = getcellvalue
.Formula = .Value
End With
Sheets.Add.Name = "Job_lists"
End With
Unload UserForm2
End Sub
An ugly, but possible, way would be with a brute force error trapping technique.
However, a more elegant solution might be to use ADO. You could for example run two 'queries': the first on the table schema which would give you your sheet names in the specified file, and the second on the found sheet name. This would produce a RecordSet containing the data of your closed sheet which can be written directly to a Range using the .CopyFromRecordset method. Of course, you could just run the first query to find your sheet name and move on as you have in your posted code.
The example below shows the code for the two queries. It's all late bound so you needn't reference the ADO library but I'll leave that decision to you. I've put a few constants at the top of the module which might need changing depending on which version of Excel you have. You'll also need to write your own error handling (especially to close the connection) but, again, I'll leave that one for you.
Option Explicit
Private Const SCHEMA_TABLES As Integer = 20
Private Const OPEN_FORWARD_ONLY As Integer = 0
Private Const LOCK_READ_ONLY As Integer = 1
Private Const CMD_TEXT As Long = 1
Private Const PROVIDER As String = "Microsoft.ACE.OLEDB.12.0"
Private Const XL_PROP As String = """Excel 12.0;HDR=No"""
Private Const SHEETS_FIELD_NAME As String = "TABLE_NAME"
Public Sub AcquireData()
Dim fPath As String
Dim fName As String
Dim key As String
Dim addr As String
Dim oConn As Object
Dim oRS As Object
Dim connString As String
Dim sql As String
Dim found As Boolean
Dim sheetField As String
'Define the path and file name
fPath = "C:\Users\User\Documents\StackOverflow"
fName = "closed_book.xlsx"
'Define the search key
key = "XL364"
'Define the address of closed worksheet
'If reading one cell then use [address:address], eg "A1:A1"
addr = "A1:E5"
'Late bind the ADO objects
Set oConn = CreateObject("ADODB.Connection")
Set oRS = CreateObject("ADODB.Recordset")
'Open conection
connString = "Provider=" & PROVIDER & ";" & _
"Data Source=" & fPath & "\" & fName & ";" & _
"Extended Properties=" & XL_PROP & ";"
oConn.Open connString
'Search for the sheet name containing your key
'in the tables (ie sheets) schema
found = False
oRS.Open oConn.OpenSchema(SCHEMA_TABLES)
Do While Not oRS.EOF
sheetField = oRS.Fields(SHEETS_FIELD_NAME).Value
If InStr(sheetField, key) > 0 Then
found = True
Exit Do
End If
oRS.MoveNext
Loop
oRS.Close
'Read the target data
If found Then
sql = "SELECT * FROM [" & _
sheetField & addr & "];"
oRS.Open sql, oConn, OPEN_FORWARD_ONLY, LOCK_READ_ONLY, CMD_TEXT
'Write the data to your worksheet
If Not oRS.EOF Then
ThisWorkbook.Worksheets("Sheet1").Range("A1") _
.CopyFromRecordset oRS
End If
End If
'Housekeeping
oRS.Close
Set oRS = Nothing
oConn.Close
Set oConn = Nothing
End Sub
You can test if the text "XL364" is in the sheet name by looping through each sheet and using the InStr (in string) function. e.g.:
For Each ws in Workbooks.Open(filepathStringFromUserInput)
If InStr(1, ws.Name, "XL364") > 0 Then
MsgBox "hi"
'Set hwSheet = ws
End If
Next ws
With hwSheet
'do some code eg:
.Range("A1").Value = "Hi"
End With

How can I query specific columns in a VBA macro used to transfer data across workbooks?

I'm using a Excel macro to transfer data between works with ADO by following these guidelines. Currently, I've set up the code to search for a keyword in column A of the source file. Once it finds the keyword, it will copy data from that entire row. However, I only need data from columns G-I, and cannot find the information to condense the data selection.
Public Sub MoveData()
'defines the project name as a variable
Dim fileName As String
fileName = Worksheets("Cover").Range("B5").Value
'defines the path
Dim path As String
path = "C:\Users\(user)\Documents\(folder)\" & fileName & ".csv"
'defines the two workbooks that the data will move between
Dim currentWB As Workbook
Set currentWB = ThisWorkbook
Dim openWB As Workbook
Set openWB = Workbooks.Open(path)
Dim openWs As Worksheet
Set openWs = openWB.Sheets(fileName)
'connects using ADODB to transfer the data
Dim cn As ADODB.Connection
Set cn = New ADODB.Connection
With cn
.Provider = "Microsoft.ACE.OLEDB.12.0"
.ConnectionString = "Data Source=" & path & ";" & _
"Extended Properties=Excel 12.0 Xml;"
.Open
End With
'selects the first column to be read and sorted
Dim subCell As Range
Dim myRange As Range
Set myRange = Range("A1:A500")
Dim cmdOpen As Boolean
cmdOpen = False
For Each subCell In myRange
'searches for the column markups
If subCell Like "*COLUMN*" Then
strQuery = "SELECT * FROM [" & fileName & "$] Where Subject = '" & subCell.Value & "'"
Set cmd = New ADODB.Command
With cmd
.ActiveConnection = cn
.CommandText = strQuery
End With
Dim rst As New ADODB.Recordset
With rst
If cmdOpen = False Then
.Open cmd
cmdOpen = True
End If
End With
currentWB.Worksheets("Cols").Range("B7:D7").CopyFromRecordset rst
End If
Next subCell
openWB.Close
End Sub
This strQuery = "SELECT * FROM [" & fileName & "$] Where Subject = '" & subCell.Value & "'" is the line I am referring to. It selects the row of data where the keyword is found. I want to limit this selection to columns G-I. Any advice on how I could accomplish this would be greatly appreciated.

Reading a workbooks without opening it with ADO

After this question: get value/charts in another workbooks without opening it
I have coded this:
Sub test()
Dim oConn As New ADODB.Connection
Dim rst As New ADODB.Recordset
oConn.Provider = "Microsoft.Jet.OLEDB.4.0"
oConn.Properties("Extended Properties").Value = "Excel 8.0"
oConn.Open "C:\Workbook1.xlsm"
rst.Open "SELECT * FROM [A1:A2];", oConn, adOpenStatic
rst.MoveFirst
MsgBox rst.Fields(0)
rst.Close
oConn.Close
End Sub
For the moment my goal is to get the value in the cell A1 of the sheet 1 of workbook1.xlsm.
I've encountered two problems.
When the workbook1 is not opened I got a
Run time error '-214767259 (80004005)': Automation error Unspecified Error on the line oConn.Open "C:\Workbook1.xlsm`
This is annoying because I want to work without opening the workbooks. It works well when the workbook is open.
Second problem: I can't manage to only get a single cell value. I've tried to input only [A1] in rst.open but it doesn't work. How can I get a unique cell value with its address ? with its name ?
If you don't mind I'll provide you a bit different attempt to get your data. The difference is the way you connect with you database (excel sheet). However, you could possibly incorporate some important elements into your code. So, check comments inside the code below.
Sub Closed_excel_workbook()
Dim myConnection As String
Dim myRecordset As ADODB.Recordset
Dim mySQL As String
'connection string parameters
'CHANGE PATH TO YOUR CLOSED WORKBOOK
myConnection = "Provider=Microsoft.ACE.OLEDB.12.0;" & _
"Data Source=" & ThisWorkbook.Path & "\Dane\BazaDanych.xlsx;" & _
"Extended Properties=Excel 12.0"
'here is important, YOU CAN'T MISS SHEET NAME
mySQL = "SELECT * FROM [ARKUSZ1$a1:a2]"
'different way of getting data from excel sheet
Set myRecordset = New ADODB.Recordset
myRecordset.Open mySQL, myConnection, adOpenUnspecified, adLockUnspecified
'let's clear sheet before pasting data
'REMOVE IF NOT NEEDED
ActiveSheet.Cells.Clear
'HERE WE PASTING DATA WE HAVE RETRIEVED
ActiveSheet.Range("A2").CopyFromRecordset myRecordset
'OPTIONAL, IF REQUIRED YOU CAN ADD COLUMNS NAMES
Dim cell As Range, i!
With ActiveSheet.Range("A1").CurrentRegion
For i = 0 To myRecordset.Fields.Count - 1
.Cells(1, i + 1).Value = myRecordset.Fields(i).Name
Next i
.EntireColumn.AutoFit
End With
End Sub
My solution:
Function GetValue()
Path = "C:\Path\"
File = "Doc.xlsm"
Sheet = "Sheet_name"
Ref = "D4"
'Retrieves a value from a closed workbook
Dim Arg As String
'Make sure the file exists
If Right(Path, 1) <> "\" Then Path = Path & "\"
If Dir(Path & File) = "" Then
GetValue = "File not Found"
Exit Function
End If
'Create the argument
Arg = "'" & Path & "[" & File & "]" & CStr(Sheet) & "'!" & Range(Ref).Range("A1").Address(, , xlR1C1)
'Check the value
MsgBox Arg
'Execute XML
GetValue = ExecuteExcel4Macro(Arg)
End Function
It has the advantage of not using complex adodb connection, but may be less powerfull.