I have written a macro which downloads data from an online directory. The directory contains a file for each date:
29062016.csv
28062016.csv
27062016.csv
.. etc
My macro keeps my spreadsheet up to date with this data and downloads a file if it detects it is missing and copies it all to a single worksheet. So far so good.
I am using the following code within a loop to download each day that a user wishes to update (along with some control variables which keep count of rows).
Dim thisWb, downloadWb As Workbook
Set thisWb = ActiveWorkbook
Set downloadWb = Workbooks.Open("ftp://myusername:mypassword#address.com/directory/" & dateToDownload &".csv")
downloadWb.Worksheets(1).Range("A1:M" & lastRow).Copy Destination:=thisWb.Worksheets("Data").Range("A" & CStr(rows))
downloadWb.Close
My issue is this method is slow as excel has to individually open each file then copy paste, especially when a few days haven't been updated.
I can ask my client to change their data feed so I was wondering if anyone could recommend a method to speed up data download, either by changing the feed or my code? Would it be faster to download and update from a single file containing all data (may become very large over time)? Different format .txt maybe? Would a http request be more efficient?
Thanks
Related
I have experience with Excel but not much with access. I am trying to create a "Database" where bunch of similar data from many workbooks is kept together in one file in MS-Access.
All the workbooks are made up of many worksheets, but all of them include a "Data" worksheet which data gets inserted into with macros from all the other worksheets in the same workbook.
This data sheet is the same for all the workbooks, made up of headers starting in cell A1 and all the way to BL1. Data gets inserted starting with A3 (so A3 to BL3). Data is inserted with a macro as a whole row(A3-BL3 all together).
CURRENTLY:
The code I am currently using is below, which transfers the whole worksheet set up on a timer(every 12 hours), but I am trying to find a better solution to achieve my goal.
PROBLEMS CURRENTLY:
1) It is too slow, I have over 10,000 rows of data, and trasnferring the whole worksheet every 12 hours is not the fastest or best method (I really dont like doing it this way).
2) Excel workbook is always open and in use, so when timer kicks in and it is running the Excel-to-Access Macro this could cause confusion to the user and/or crash(I do not even want the user to be aware of this access database)
GOAL:
Have this macro run from Excel, every time new data row is created in the Data worksheet(A6-BL6), and have it inserted into the access database table without deleting anything else(basically inserting into access 1 row every time new row of data is created in the Data worksheet in excel. I want it to push old data down in access so everything is kept and nothing is overwritten or lost.
It is crucial I run the macro from excel and not access(dont even want access to be accessible or opened by other user just want data stored there)
Excel and Access are both 2016.
If anyone could please help and/or give me suggestions I would appreciate it, have done some research but haven't found anything that solved my problem completely.
Option Explicit
Sub AccImport()
Application.OnTime Now + TimeValue("12:00"), "AccImport"
Dim acc As New Access.Application
acc.OpenCurrentDatabase "C:\Users\yilly1\Desktop\Database.accdb"
acc.DoCmd.TransferSpreadsheet _
TransferType:=acImport, _
SpreadSheetType:=acSpreadsheetTypeExcel12Xml, _
TableName:="Workbook1", _
Filename:=Application.ActiveWorkbook.FullName, _
HasFieldNames:=True, _
Range:="Data1$A1:BL60000"
acc.CloseCurrentDatabase
acc.Quit
Set acc = Nothing
End Sub
I believe my problem is rather simple: I have a workbook and I'm using it to get some data from another software (SAP). When I export the data from the software, it automatically opens a .xlsx file, then what I'd need to do is copy some of the data from this file, paste on my original workbook, and then close the file. The section of my code which is giving me an error is this one:
fileName = "temp1.xlsx"
Set wbBasis = Workbooks(fileName)
This happens because the "temp1.xlsx" file that was opened by the SAP software is in another instance of excel, so my code isn't able to find it.
What I need to know is basically this. How to properly reference this "temp1.xlsx" workbook on my original code so that I'm able to edit it, copy stuff from it, close it, etc.
I've found some similar topics to my problem, like the two I'm listing down here, but couldn't adapt the solutions to my situation, and that's why I'm posting this new one.
Having multiple excel instances launched, how can I get the Application Object for all of them?
Finding a workbook in one of multiple Excel instances
Thank you in advance.
You don't need multiple instances of Excel, and you don't need the Excel file to be open in order to get information from it, either with VBA, or even with regular "linked formulas".
For example:
=SUM([C:\yourPath\Filename.xlsx]SheetName!A1:C25)
...returns the sum of cells A1:C25 on a worksheet named Sheetname in an Excel file located at C:\yourPath\Filename.xlsx.
Or you can import all the data directly with Get External Data.
See:
MSDN : Create an external reference (link) to a cell range in another workbook
MSDN : Connect to another workbook
...for more information and examples, search Google for "get data from another Excel workbook".
I am trying to automate a script that I use for PowerShell. When I get a request, instead of editing the script line by line with required information, I would like to create an Excel spreadsheet, input the data required, and populate my script with those variables in the correct location.
From a high level view with limited programming experience, this does not seem overly complicated but I cannot figure it out.
I am able to do bits and pieces of what I am trying to accomplish, but not everything together. Below is step by step what I want to accomplish.
Input data into Cells B10 through B16
Store each cell into an individual variable
Hit the command button which will take the variables and print/overwrite a file already created.
I have a specific file path that I need to use.
Here is the code that I have already, that I see will copy the cells and open my PowerShell module.
Public Sub CommandButton1_Click()
Dim fileName As Variant
Dim Azure As Workbook
Dim wsSource As Worksheet
Set Azure = ActiveWorkbook
Set wsSource = ActiveSheet
'Copy range on original sheet
wsSource.Range("B10:B16").Copy
fileName = Shell("powershell_ise.exe ""c:\Azure\AZURE_1.ps1""", vbNormalFocus)
End Sub
I can see that the above code doesn't accomplish what I need to do. However, I can see that I can use the button, copy the cells, and open the required file.
What I need to do is take each cell and place at a certain spot in my script and overwrite the previous file.
I have an ETL file that has suddenly stopped working on the ShowAllData command. In previous loops, the data IS being filtered.
Givens:
wkstSourceSheet is established in another module as the sheet that contains oSourceTable which is the originating table of data and/or scrubbed data to be uploaded. The steps are contained in functions, grouped logically by module. All local controls are contained in the Local_Controls module.
Loop Steps
Remove any Sorts & Filters on Data (it is on 2nd loop when this fails)
Apply new Sorts & Filters
Save Destination File if one is open and destination file is different for this loop
Open Destination File if needed
Clear applicable data from Destination
Paste new data to Destination
Clear some variables
Loop
Code
If wkstSourceSheet.FilterMode = True Then
wkstSourceSheet.ShowAllData
End If
I've been able to get a workaround by replacing it with this
If wkstSourceSheet.FilterMode = True Then
oSoureSheet.Range.Autofilter
oSoureSheet.Range.Autofilter
End If
However, I'm trying to figure out why it would fail when the data is clearly filtered.
(I would write this as a comment if I could... but I don't have enough reputation yet)
Could you be having a problem with the two different types of Autofilters that Excel has?
One for simple Worksheet ranges, the other for ListObjects (Excel "tables").
If there was manual user interaction with the worksheet (not just your code running), a user might have started the filters in two different ways...
I need to save several xlsm files as CSV in order to import them in other programs (such as R etc.). Well I've written the necessary import routines in all the other programs and they work very nicely with case 1:
Case 1: Manually saving an xlsm as CSV
If I use this option and manually save each file as CSV and click YES on all prompts, I get a .csv file which looks very similar to a normal Excel file when opened again within excel. It is a standard column view, and nothing comma separated etc. (maybe it is, but it doesn't look that way..)
Case 2: Saving an xlsm from VBA as CSV
Well here I get a completely different file when opened again in Excel. This one looks like a "real" csv file with all values being comma separated.
My questions are:
1. Why is there any difference?
2. How can I programmatically reach Case 2 from VBA? Or is that impossible?
If 2 is impossible I have to rewrite my import code routines to handle the "normal" csv file...not very difficult but still a lot of work and I'm really wondering why there even is a difference..
Q1: I don't think there is a difference, at least not in an example I pulled together. Q2: Try this out:
I've got 3 example XLSM files in C:\stack\folder1 as pictured below:
Each file has a single data sheet, which we'll turn into CSVs:
I'm sure your routine is much more complicated, but to test the CSV output I'm just going to loop through the files and save each as xlCSV:
Option Explicit
Sub TestCSVOutput()
Dim DataBook As Workbook
Dim DataSheet As Worksheet
Dim FilePaths(3) As String
Dim FileIdx As Long
'set up file paths for test
FilePaths(1) = "C:\stack\folder1\test_file_01.xlsm"
FilePaths(2) = "C:\stack\folder1\test_file_02.xlsm"
FilePaths(3) = "C:\stack\folder1\test_file_03.xlsm"
'loop through array and save each file as a CSV
Application.DisplayAlerts = False
For FileIdx = 1 To UBound(FilePaths)
Set DataBook = Workbooks.Open(FilePaths(FileIdx))
Set DataSheet = DataBook.ActiveSheet
DataBook.SaveAs FileFormat:=xlCSV '<~~ the save step
DataBook.Close
Next FileIdx
Application.DisplayAlerts = True
End Sub
Once the script completes, I end up with three CSV files:
Each file is comma-delimited when opened in a text editor: