excel to access selective row transfer instead of the whole spreadsheet - vba

I have experience with Excel but not much with access. I am trying to create a "Database" where bunch of similar data from many workbooks is kept together in one file in MS-Access.
All the workbooks are made up of many worksheets, but all of them include a "Data" worksheet which data gets inserted into with macros from all the other worksheets in the same workbook.
This data sheet is the same for all the workbooks, made up of headers starting in cell A1 and all the way to BL1. Data gets inserted starting with A3 (so A3 to BL3). Data is inserted with a macro as a whole row(A3-BL3 all together).
CURRENTLY:
The code I am currently using is below, which transfers the whole worksheet set up on a timer(every 12 hours), but I am trying to find a better solution to achieve my goal.
PROBLEMS CURRENTLY:
1) It is too slow, I have over 10,000 rows of data, and trasnferring the whole worksheet every 12 hours is not the fastest or best method (I really dont like doing it this way).
2) Excel workbook is always open and in use, so when timer kicks in and it is running the Excel-to-Access Macro this could cause confusion to the user and/or crash(I do not even want the user to be aware of this access database)
GOAL:
Have this macro run from Excel, every time new data row is created in the Data worksheet(A6-BL6), and have it inserted into the access database table without deleting anything else(basically inserting into access 1 row every time new row of data is created in the Data worksheet in excel. I want it to push old data down in access so everything is kept and nothing is overwritten or lost.
It is crucial I run the macro from excel and not access(dont even want access to be accessible or opened by other user just want data stored there)
Excel and Access are both 2016.
If anyone could please help and/or give me suggestions I would appreciate it, have done some research but haven't found anything that solved my problem completely.
Option Explicit
Sub AccImport()
Application.OnTime Now + TimeValue("12:00"), "AccImport"
Dim acc As New Access.Application
acc.OpenCurrentDatabase "C:\Users\yilly1\Desktop\Database.accdb"
acc.DoCmd.TransferSpreadsheet _
TransferType:=acImport, _
SpreadSheetType:=acSpreadsheetTypeExcel12Xml, _
TableName:="Workbook1", _
Filename:=Application.ActiveWorkbook.FullName, _
HasFieldNames:=True, _
Range:="Data1$A1:BL60000"
acc.CloseCurrentDatabase
acc.Quit
Set acc = Nothing
End Sub

Related

Automatic refreshment of Excel DDE RTD

I'm subscribed to a RTD financial application with the option to export real time data to Excel via DDE. So, using the DDE copy/paste app option, I've created a Excel table with real time DDE financial data. Cells actually contains DDE formulas. Now I open, refresh and save table manually, but what I need is a solution that will open, refresh the data, save and close this excel table automatically.
I didn't now how to attach the file, so here is a screenshot of it.
My Excel DDE file
Since I'm new to vba and vbscript, I was searching internet for a solution but without any success, especially not for automatic update (refreshment) of the DDE data.
What I've been doing for the past few days, was searching, trying and customising different kind of vba macros and vbscripts, and today I'm completely lost and confused.
I have tried bunch of suggested solution, even yours
Refresh data and exit with saving Macro Excel
and On workbook open, Excel Macro to refresh all data connections sheets and pivot tables and then export the pivot to csv but in my case they don't work.
Any solution, reference, code example will be greatly appreciated.
Kindly try the below one and
be careful when using
Application.Calculation = xlCalculationManual
Application.Calculation = xlCalculationAutomatic
this sets the entire excel application to calculate formula's either automatically or manually. If you use
Application.Calculation = xlCalculationManual
you'll notice your automatic formulas no longer work.
Thanks for the reply #mulla, but this doesn't work. In vbscript I get runtime error and in vba nothing happens (does not affect refreshment). The same thing is for the Worksheet.Calculate.
Than I recorded a macro upon manual update of the links (data) in order to inspect how Excel proceed update and this actually works, but not all the time. If I open Workbook manually and I keep it opened than whole export works perfectly since the data is constantly refreshed (not what I need). In case when the Workbook is opened via vbscript only (what I need), as you can see in the output below some of the values are exported successfully (for the faster refreshment I believe) and some are #N/A (for the slower refreshment). All the data is refreshed every second, except upon opening the file where successfully exported values are refreshed instantly and #N/A values need few seconds (2-3s) to be refreshed.
Recorder macro uses ActiveWorkbook.UpdateLink as follows (the code below is for the first row, but the same logic is used and for the rest of the rows):
Sub PriceUpdate()
ActiveWorkbook.UpdateLinks = xlUpdateLinksAlways
ActiveWorkbook.UpdateLink Name:= _
"vegadde|VEGA!897789,148,1#""1,\""12,0,0\"""",1", Type:=xlOLELinks
ActiveWorkbook.UpdateLink Name:= _
"vegadde|VEGA!897789,148,1#""1,\""12,0,0\"""",3", Type:=xlOLELinks
ActiveWorkbook.UpdateLink Name:= _
"vegadde|VEGA!897789,148,1#""1,\""12,0,0\"""",4", Type:=xlOLELinks
...
End Sub
Then I use vbscript to call the macro and export data in CSV comma delimited file with timestamp name:
Option Explicit
Dim objExcel, objBook, objSheet
Set objExcel = CreateObject("Excel.Application")
Set objBook = objExcel.Workbooks.Open ("d:\exptest\exptest.xlsm", 0, False)
'Set objSheet = objBook.Worksheets.Item(1)
objExcel.DisplayAlerts = False
objExcel.Run "PriceUpdate"
WScript.Sleep 5000 ' Delay in order to update links
objBook.SaveAs "d:\exptest\" & Year(Now) & "." & Month(Now) & "." & Day(Now) & "_" & Hour(Now) & "-" & Minute(Now) & ".txt",6
objBook.Close False
objExcel.DisplayAlerts = True
objExcel.Quit
'Set objSheet = Nothing
Set objBook = Nothing
Set objExcel = Nothing
And what I get is the following uotput:
Inst,Price,Datetime
USD,1.1015,7/22/2016 12:48
GBP,#N/A,#N/A
CHF,#N/A,#N/A
SEK,9.4962,7/22/2016 12:48
NOK,#N/A,#N/A
JPY,#N/A,#N/A
DKK,#N/A,#N/A
CAD,#N/A,#N/A
AUD,1.47395,7/22/2016 12:48
RUB,71.0082,7/22/2016 12:48
I'm struggling to learn Excel logic behind different updates in order to solve my problem and to get desired solution, but with no success.
And this is only a test file with 22 links. The real one has 482 links that need to be updated.
Marjan. Not sure if this is still a problem for you, but I ran into a similar problem with refreshing RTD data in a closed Excel workbook and seem to have found a solution. As you mentioned, when you open the workbook, the data refreshes automatically (with some of the more complex data points taking 2-3 sec). The problem is, you don't want to have to constantly manually open a workbook to refresh data. And I've found that the data fails to refresh with methods like RTD.RefreshData called to a closed workbook.
So to fix this, I created a vba module that simply opens the workbook, refreshes it, pauses for a second (you may need 2-3 more seconds for all data to update) and then saves and closes the workbook. This method is called recursively. Essentially this does the work of constantly opening and closing the workbook, while delaying briefly so that the RTD data can refresh. So far, this is able to provide refreshed RTD data successfully (although there is an inherent slight delay of a few seconds). Here's the basis of my code:
Sub refreshXLS()
Path = "workbook.xlsm" 'the workbook path you want to refresh
Workbooks.Open Path
ActiveWorkbook.RefreshAll
Application.OnTime Now() + TimeValue("00:00:01"), "closeActive"
End Sub
Sub closeActive()
ActiveWorkbook.Save
ActiveWorkbook.Close
refreshXLS
End Sub
Hope this helps if you still wish to solve this problem!

Fastest Way To Download Excel Data from server by VBA Macro

I have written a macro which downloads data from an online directory. The directory contains a file for each date:
29062016.csv
28062016.csv
27062016.csv
.. etc
My macro keeps my spreadsheet up to date with this data and downloads a file if it detects it is missing and copies it all to a single worksheet. So far so good.
I am using the following code within a loop to download each day that a user wishes to update (along with some control variables which keep count of rows).
Dim thisWb, downloadWb As Workbook
Set thisWb = ActiveWorkbook
Set downloadWb = Workbooks.Open("ftp://myusername:mypassword#address.com/directory/" & dateToDownload &".csv")
downloadWb.Worksheets(1).Range("A1:M" & lastRow).Copy Destination:=thisWb.Worksheets("Data").Range("A" & CStr(rows))
downloadWb.Close
My issue is this method is slow as excel has to individually open each file then copy paste, especially when a few days haven't been updated.
I can ask my client to change their data feed so I was wondering if anyone could recommend a method to speed up data download, either by changing the feed or my code? Would it be faster to download and update from a single file containing all data (may become very large over time)? Different format .txt maybe? Would a http request be more efficient?
Thanks

ShowAllData fails when excel data set is filtered

I have an ETL file that has suddenly stopped working on the ShowAllData command. In previous loops, the data IS being filtered.
Givens:
wkstSourceSheet is established in another module as the sheet that contains oSourceTable which is the originating table of data and/or scrubbed data to be uploaded. The steps are contained in functions, grouped logically by module. All local controls are contained in the Local_Controls module.
Loop Steps
Remove any Sorts & Filters on Data (it is on 2nd loop when this fails)
Apply new Sorts & Filters
Save Destination File if one is open and destination file is different for this loop
Open Destination File if needed
Clear applicable data from Destination
Paste new data to Destination
Clear some variables
Loop
Code
If wkstSourceSheet.FilterMode = True Then
wkstSourceSheet.ShowAllData
End If
I've been able to get a workaround by replacing it with this
If wkstSourceSheet.FilterMode = True Then
oSoureSheet.Range.Autofilter
oSoureSheet.Range.Autofilter
End If
However, I'm trying to figure out why it would fail when the data is clearly filtered.
(I would write this as a comment if I could... but I don't have enough reputation yet)
Could you be having a problem with the two different types of Autofilters that Excel has?
One for simple Worksheet ranges, the other for ListObjects (Excel "tables").
If there was manual user interaction with the worksheet (not just your code running), a user might have started the filters in two different ways...

Excel VBA - Code to move or copy worksheets without warnings

I am trying to create a macro that would act the same as right clicking a workbook tab, selecting move or copy, checking the copy option, selecting another open workbook and clicking ok but without the warnings. I found the code to disable warning and I was able to record a macro that does what I want but I don't know how to make it request which open workbook to copy to.
In short how do I make the following code work where WorksheetIWantToCopy is the one the user currently has selected and OpenWorkbookIWantToCopyToo.xlsx is a workbook to be selected by the user out of a list of open workbooks.
Application.DisplayAlerts = False
Sheets("**WorksheetIWantToCopy**").Select
Sheets("**WorksheetIWantToCopy**").Copy Before:=Workbooks( _
"**OpenWorkbookIWantToCopyToo.xlsx**").Sheets(1)
I appreciate any information anyone can provide. My team greatly appreciates your support (we currently have to hit ok on 25 warnings due to conflicts we don't really care about). Thx!
If the worksheet you want to copy will always be the active sheet then you can use ActiveSheet.
As for letting the user select a workbook, it can be as simple as using the InputBox.
Public Function getWorkbookName() As String
Dim i As Integer, sListOfWbks As String, sRsp As String
' build list of workbooks
For i = 1 To Workbooks.Count
sListOfWbks = sWbkList & vbCrLf & i & " - " & Workbooks(i).Name
Next i
sRsp = InputBox("Select workbook." & vbCrLf & sListOfWbks)
If IsNumeric(sRsp) Then
getWorkbookName = Workbooks(CInt(sRsp)).Name
Else
' user pressed cancel or entered invalid text
getWorkbookName = ""
End If
End Function
This basic example will of course list all workbooks, including hidden add-ins and the workbook you are moving away from.
This needs to be said before anything else: always, always, ALWAYS make use of .Copy instead of .Move when automatically shuffling excel workbooks with VBA. Move has inherent risks because it is a modification of the other file, and if your code misbehaves then you could lose all of the data you're working with.
First of all, know which workbook is which, with no ambiguity:
Dim wkbkDestination, wkbkTemporary As Workbook
Set wkbkDestination = Workbooks("OpenWorkbookIWantToCopyTo.xlsx")
Set wkbkTemporary = Workbooks.Open("WorkbookIWantToCopy.xlsx")
Next, Copy your desired tab to your destination workbook, rename the new tab to prevent errors, and close the second workbook, without saving.
wkbkTemporary.Worksheets("WorksheetIWantToCopy").Copy Before:=wkbkDestination.Worksheets(1)
wkbkDestination.Worksheets(1).Name = "WorkbookIWantToCopy"
wkbkTemporary.Close SaveChanges = False
Naturally, depending on the exact controls you intend to use, there are lots of ways this code could be implemented. From your description it is somewhat unclear what exact problem you're trying to solve and whether this is a one-off event you're trying to accomplish for a given list of files, or whether this function is to be used on an ongoing basis.

How to speed up copying cells in Excel vba

I have this Excel VB code, and everytime after it executes this line, it pauses for half a second:
Worksheets(ws.Name).Range("A" & i & ":G" & i).Copy _
Destination:=Worksheets("Sheet1").Range("A" & emptyCell)
Can someone tell me what it is doing, and how I can prevent it from taking so long?
I have Microsoft Excel 2007 on Windows XP Pro
Update: If I paste it by hand, it waits about the same amount of time.
It all depends on the specs of the hardware (hard drive, memory, network connection, ect). If its running on a network, that will be the biggest bottleneck. There is nothing inherently wrong with the code.
You can also try speeding up the routine by turning the workbooks calculation to manual, and turning off screen updating. If the calculation is automatic it will calculate on every copy call.
It's copying a range of cells from the worksheet assigned to the variable ws and pasting it to Sheet1.
The range it is copying is from A:G in the row number assigned to the variable i.
It is pasting the range into column A in Sheet1 in the row number assigned to the variable emptyCell.
You can prevent it by prefixing each line with an apostrophe.
However, you may first wish to explore WHY the code is doing that!