Download to file and readystate = 4 - vba

I'm trying to write a code to download a very large file that, depending on bandwidth, may take 30 minutes to download. I have a very basic script now, that typically terminates before the file is completely downloaded. Is there a way to use readystate, or something similar, to make VBA allow the entire file to download before moving on?
Here's the code:
Sub Download()
Dim strURL As String
Dim strPath As String
'~~> URL of the Path
strURL = "http://www.aeronav.faa.gov/upload_313-/terminal/DDTPPE_201612.zip"
'~~> Destination for the file
strPath = "c:\Users\username\Desktop\WebTest\database.zip"
Ret = URLDownloadToFile(0, strURL, strPath, 0, 0)
End Sub
Thanks!

You can put your URL in a cell and run the script below.
Private Declare Function URLDownloadToFile Lib "urlmon" Alias _
"URLDownloadToFileA" (ByVal pCaller As Long, ByVal szURL As String, ByVal _
szFileName As String, ByVal dwReserved As Long, ByVal lpfnCB As Long) As Long
Sub DownloadFilefromWeb()
Dim strSavePath As String
Dim URL As String, ext As String
Dim buf, ret As Long
URL = Worksheets("Sheet1").Range("A2").Value
buf = Split(URL, ".")
ext = buf(UBound(buf))
strSavePath = "C:\Users\your_path_here\" & "DownloadedFile." & ext
ret = URLDownloadToFile(0, URL, strSavePath, 0, 0)
If ret = 0 Then
MsgBox "Download has been succeed!"
Else
MsgBox "Error"
End If
End Sub
That's if you want to loop through a range with many URLs. If you want to download just one, try it this way.
Declare Function URLDownloadToFileA Lib "urlmon" _
(ByVal pCaller As Long, _
ByVal szURL As String, _
ByVal szFileName As String, _
ByVal dwReserved As Long, _
ByVal lpfnCB As Long) As Long
Sub ExampleDownload()
Dim IExpl As Object
Set IExpl = CreateObject("InternetExplorer.Application")
With IExpl
.Navigate "http://www.bom.mu/?id=80277" 'You need to change this for a variable and loop
Do Until .Readystate = 4: Loop ' Allow page to load
'Code below to find correct href link in page based on text
For Each lnk In IExpl.Application.Document.Links
If lnk.outertext = "Click Here to Open or Right Click to Download." Then Exit For
Debug.Print lnk.outertext
Next
End With
SuccessfulDownload = URLDownloadToFileA(0, lnk.href, "C:\myfilename.zip", 0, 0)
Set IExpl = Nothing
End Sub

Or, try R, which is blazing fast!! In order to get your data to download and uncompress, you need to set mode="wb"
download.file("...",temp, mode="wb")
unzip(temp, "gbr_Country_en_csv_v2.csv")
dd <- read.table("gbr_Country_en_csv_v2.csv", sep=",",skip=2, header=T)
Then, simply read the CSV from your Excel tool.

Related

download html web page or scrape text from it

I am struggling to download full html code from below page. I was using URLDownloadToFileA and MSXML2.XMLHTTP60 methods and none of them downloads full code. Part of the webpage is missing - table on the bottom with "DEBTI...." in not included in the code. My aim is to get these "DEBTI..." strings and I want to avoid selenium if possible. None of below codes work for me, but when I just ctrl+s (save) page from the browser everything is correctly mentioned in the code. Any suggestions? Thanks!
Sub Get_Data()
Dim xmlhttp As New MSXML2.XMLHTTP60, myurl As String
myurl = "https://ec.europa.eu/taxation_customs/dds2/ebti/ebti_consultation.jsp?Lang=en&Lang=en&refcountry=&reference=&valstartdate1=&valstartdate=&valstartdateto1=&valstartdateto=&valenddate1=&valenddate=&valenddateto1=&valenddateto=&suppldate1=&suppldate=&nomenc=2309&nomencto=&keywordsearch1=&keywordsearch=&specialkeyword=&keywordmatchrule=OR&excludekeywordsearch1=&excludekeywordsearch=&excludespecialkeyword=&descript=&orderby=0&Expand=true&offset=1&viewVal=Thumbnail&isVisitedRef=false&allRecords=0&showProgressBar=true" 'replace with your URL
xmlhttp.Open "GET", myurl, False
xmlhttp.Send
Open "C:\Desktop\test.txt" For Output As #1
Print #1, (xmlhttp.responseText)
Close #1
End Sub
Private Declare PtrSafe Function URLDownloadToFile Lib "urlmon" _
Alias "URLDownloadToFileA" (ByVal pCaller As Long, ByVal szURL As String, _
ByVal szFileName As String, ByVal dwReserved As Long, ByVal lpfnCB As Long) As Long
Function DownloadFile(URL As String, LocalFilename As String) As Boolean
Dim lngRetVal As Long
lngRetVal = URLDownloadToFile(0, URL, LocalFilename, 0, 0)
If lngRetVal = 0 Then DownloadFile = True
End Function
Sub test()
DownloadFile "https://ec.europa.eu/taxation_customs/dds2/ebti/ebti_consultation.jsp?Lang=en&Lang=en&refcountry=&reference=&valstartdate1=&valstartdate=&valstartdateto1=&valstartdateto=&valenddate1=&valenddate=&valenddateto1=&valenddateto=&suppldate1=&suppldate=&nomenc=8418&nomencto=&keywordsearch1=&keywordsearch=&specialkeyword=&keywordmatchrule=OR&excludekeywordsearch1=&excludekeywordsearch=&excludespecialkeyword=&descript=&orderby=0&Expand=true&offset=1&viewVal=List&isVisitedRef=false&allRecords=0&showProgressBar=true", "C:\Desktop\abc.html"
End Sub
web:
https://ec.europa.eu/taxation_customs/dds2/ebti/ebti_consultation.jsp?Lang=en&Lang=en&refcountry=&reference=&valstartdate1=&valstartdate=&valstartdateto1=&valstartdateto=&valenddate1=&valenddate=&valenddateto1=&valenddateto=&suppldate1=&suppldate=&nomenc=8418&nomencto=&keywordsearch1=&keywordsearch=&specialkeyword=&keywordmatchrule=OR&excludekeywordsearch1=&excludekeywordsearch=&excludespecialkeyword=&descript=&orderby=0&Expand=true&offset=1&viewVal=List&isVisitedRef=false&allRecords=0&showProgressBar=true

Download file from website hyperlink in outlook

So I used the following code to open a Hyperlink from an email. This hyperlink opens the webpage and opens the download window to choose where to download a CSV and with what name (all of this is in Chrome). I want to be able to choose where said file will be downloaded and with what name. I would really appreciate the help :)
Private Declare PtrSafe Function ShellExecute _
Lib "shell32.dll" Alias "ShellExecuteA" ( _
ByVal hWnd As Long, _
ByVal Operation As String, _
ByVal Filename As String, _
Optional ByVal Parameters As String, _
Optional ByVal Directory As String, _
Optional ByVal WindowStyle As Long = vbMinimizedFocus _
) As Long
Public Sub OpenLinks(olMail As Outlook.MailItem)
Dim Reg1 As RegExp
Dim M1 As MatchCollection
Dim M As Match
Dim strURL As String
Dim lSuccess As Long
Set Reg1 = New RegExp
With Reg1
.Pattern = "(https?[:]//([0-9a-z=\?:/\.&-^!#$%;_])*)>"
.Global = False
.IgnoreCase = True
End With
If Reg1.Test(olMail.Body) Then
Set M1 = Reg1.Execute(olMail.Body)
For Each M In M1
strURL = M.SubMatches(0)
Debug.Print strURL
lSuccess = ShellExecute(0, "Open", strURL)
Next
End If
Set Reg1 = Nothing
Set oApp = Nothing
End Sub
I've looked in other sites, but couldn't find anything similar.
You can choose one of the following ways:
Use Windows API, see the URLDownloadToFile function:
Private Declare Function URLDownloadToFile Lib "urlmon" Alias "URLDownloadToFileA" (ByVal pCaller As Long, ByVal szURL As String, ByVal szFileName As String, ByVal dwReserved As Long, ByVal lpfnCB As Long) As Long
Public Function DownloadFile(URL As String, LocalFilename As String) As Boolean
Dim lngRetVal As Long
lngRetVal = URLDownloadToFile(0, URL, LocalFilename, 0, 0)
If lngRetVal = 0 Then
If Dir(LocalFileName) <> vbNullString Then
DownloadFile = True
End If
End If
End Function
Private Sub Form_Load()
If Not DownloadFile("http://www.test.come", "c:\\file.doc") Then
MsgBox "Unable to download the file, or the source URL doesn't exist."
End If
End Sub
Click buttons programmatically using Windows API functions, see VBA - Go to website and download file from save prompt for more infromation.

vba code for printout option is not working for me

I have error while trying to give printout action, if any solution provided will be helpful...
Dim objTextStream
Dim fso As Object
Set fso = CreateObject("Scripting.FileSystemObject")
Set objTextStream = fso.opentextfile("E:\NewFile.prn")`
objTextStream.PrintOut(1)
// not executing
Debug error
Object doesn't support this property or method
There is no print method in a TextStream object (this is what opentextfile returns).
To print a file, you have to use a Windows API function apiShellExecute.
This is copied from http://www.dbforums.com/showthread.php?1066955-How-do-i-print-a-txt-file-with-VBA (but untested):
Place the following in the declaration section of a module (at the top) if you place it in a standard module, change Private to Public:
Private Declare Function apiShellExecute Lib "shell32.dll" _
Alias "ShellExecuteA" _
(ByVal hwnd As Long, _
ByVal lpOperation As String, _
ByVal lpFile As String, _
ByVal lpParameters As String, _
ByVal lpDirectory As String, _
ByVal nShowCmd As Long) _
As Long
Then in the routine where you need to print, call it with:
lngReturn = apiShellExecute(hWndAccessApp, "print", strFileAndPath, vbNullString, vbNullString, 0)
where strFileAndPath is your filename (eg "E:\NewFile.prn")
You can loop through each line of the read file like this. You can then either debug.print that line or write it to a separate write file as shown.
Const ForReading = 1, ForWriting = 2
Dim FSO, FileIn, FileOut As Object
Dim strTmp As String
Set FSO = CreateObject("Scripting.FileSystemObject")
Set FileIn = FSO.OpenTextFile("E:\NewFile.prn", ForReading)
Set FileOut = FSO.OpenTextFile("E:\NewWriteFile.prn", ForWriting, True)
Do Until FileIn.AtEndOfStream
strTmp = FileIn.ReadLine
FileOut.WriteLine strTmp
Loop
FileIn.Close
FileOut.Close

How can I create text files with special characters in their filenames

Demonstration of my problem
Open a new Excel workbook and save these symbols 設計師協會 to cell [A1]
insert the following VBA code somewhere in the editor (Alt+F11)
execute it line per line (F8)
Sub test()
strCRLF = StrConv(vbCrLf, vbUnicode)
strSpecialchars = StrConv(Cells(1, 1), vbUnicode)
strFilename = "c:\test.txt"
Open strFilename For Output As #1
Print #1, strSpecialchars & strCRLF;
Close #1
End Sub
You will get a textfile which contains the chinese characters from [A1]. This proofs that VBA is able to handle unicode characters if you know the trick with adding StrConv(vbCrLf, vbUnicode)
Now try the same for strFilename = "C:\" & strSpecialchars & ".txt". You will get an error that you can't create a file with this filename. Of course you can't use the same trick adding a new line since its a filename.
How can I create text files with special characters in their filenames using VBA?
Is there a work-around or am I doing something wrong?
Note
I'm using Windows 7 x64. I'm able to create text files with special characters manually
I found a second method using FileSystemObject. But I hope I could avoid setting a reference to the VB script run-time library
Value retrieved from the cell is already in Unicode.
StrConv(vbUnicode) gives you "double unicode" which is broken because it went through a conversion using the current system codepage.
Then the Print command converts it back to "single unicode", again using the current system codepage. Don't do this. You're not saving unicode, you're saving invalid something that may only appear valid on your particular computer under your current settings.
If you want to output Unicode data (that is, avoid the default VB mechanism of auto-converting output text from Unicode to ANSI), you have several options.
The easiest is using FileSystemObject without trying to invent anything about unicode conversions:
With CreateObject("Scripting.FileSystemObject")
With .CreateTextFile("C:\" & Cells(1).Value & ".txt", , True)
.Write Cells(1).Value
.Close
End With
End With
Note the last parameter that controls Unicode.
If you don't want that, you can declare CreateFileW and WriteFile functions:
Private Declare Function CreateFileW Lib "kernel32.dll" (ByVal lpFileName As Long, ByVal dwDesiredAccess As Long, ByVal dwShareMode As Long, ByRef lpSecurityAttributes As Any, ByVal dwCreationDisposition As Long, ByVal dwFlagsAndAttributes As Long, ByVal hTemplateFile As Long) As Long
Private Declare Function CloseHandle Lib "kernel32.dll" (ByVal hObject As Long) As Long
Private Declare Function WriteFile Lib "kernel32.dll" (ByVal hFile As Long, ByRef lpBuffer As Any, ByVal nNumberOfBytesToWrite As Long, ByRef lpNumberOfBytesWritten As Long, ByRef lpOverlapped As Any) As Long
Private Const CREATE_ALWAYS As Long = 2
Private Const GENERIC_WRITE As Long = &H40000000
Dim hFile As Long
hFile = CreateFileW(StrPtr("C:\" & Cells(1).Value & ".txt"), GENERIC_WRITE, 0, ByVal 0&, CREATE_ALWAYS, 0, 0)
Dim val As String
val = Cells(1).Value
WriteFile hFile, &HFEFF, 2, 0, ByVal 0& 'Unicode byte order mark (not required, but to please Notepad)
WriteFile hFile, ByVal StrPtr(val), Len(val) * 2, 0, ByVal 0&
CloseHandle hFile
You are on the right track with the FileSystemObject. As Morbo mentioned you can late bind this so no reference is set. The FSO has a CreateTextFile function which can be set in unicode so the characters will appear as '??????' in VBA but will write correctly to the filename. Note the second parameter of the CreateTextFile function specifies a unicode string for the filename. The following will do the trick for you:
Sub test()
Dim strCRLF As String, strSpecialchars As String, strFilename As String
Dim oFSO As Object, oFile As Object
strCRLF = StrConv(vbCrLf, vbUnicode)
strSpecialchars = StrConv(Cells(1, 1), vbUnicode)
strFilename = "C:\" & Cells(1, 1).Value & ".txt"
Set oFSO = CreateObject("Scripting.FileSystemObject")
Set oFile = oFSO.CreateTextFile(strFilename, , True)
oFile.Write strSpecialchars & strCRLF
oFile.Close
Set oFile = Nothing
Set oFSO = Nothing
End Sub

Open an html page in default browser with VBA?

How do I open an HTML page in the default browser with VBA? I know it's something like:
Shell "http://myHtmlPage.com"
But I think I have to reference the program which will open the page.
You can use the Windows API function ShellExecute to do so:
Option Explicit
Private Declare Function ShellExecute _
Lib "shell32.dll" Alias "ShellExecuteA" ( _
ByVal hWnd As Long, _
ByVal Operation As String, _
ByVal Filename As String, _
Optional ByVal Parameters As String, _
Optional ByVal Directory As String, _
Optional ByVal WindowStyle As Long = vbMinimizedFocus _
) As Long
Public Sub OpenUrl()
Dim lSuccess As Long
lSuccess = ShellExecute(0, "Open", "www.google.com")
End Sub
As given in comment, to make it work in 64-bit, you need add PtrSafe in the Private Declare Line as shown below:
Private Declare PtrSafe Function ShellExecute _
Just a short remark concerning security: If the URL comes from user input make sure to strictly validate that input as ShellExecute would execute any command with the user's permissions, also a format c: would be executed if the user is an administrator.
You can even say:
FollowHyperlink "www.google.com"
If you get Automation Error then use http://:
ThisWorkbook.FollowHyperlink("http://www.google.com")
If you want a more robust solution with ShellExecute that will open ANY file, folder or URL using the default OS associated program to do so, here is a function taken from http://access.mvps.org/access/api/api0018.htm:
'************ Code Start **********
' This code was originally written by Dev Ashish.
' It is not to be altered or distributed,
' except as part of an application.
' You are free to use it in any application,
' provided the copyright notice is left unchanged.
'
' Code Courtesy of
' Dev Ashish
'
Private Declare Function apiShellExecute Lib "shell32.dll" _
Alias "ShellExecuteA" _
(ByVal hwnd As Long, _
ByVal lpOperation As String, _
ByVal lpFile As String, _
ByVal lpParameters As String, _
ByVal lpDirectory As String, _
ByVal nShowCmd As Long) _
As Long
'***App Window Constants***
Public Const WIN_NORMAL = 1 'Open Normal
Public Const WIN_MAX = 3 'Open Maximized
Public Const WIN_MIN = 2 'Open Minimized
'***Error Codes***
Private Const ERROR_SUCCESS = 32&
Private Const ERROR_NO_ASSOC = 31&
Private Const ERROR_OUT_OF_MEM = 0&
Private Const ERROR_FILE_NOT_FOUND = 2&
Private Const ERROR_PATH_NOT_FOUND = 3&
Private Const ERROR_BAD_FORMAT = 11&
'***************Usage Examples***********************
'Open a folder: ?fHandleFile("C:\TEMP\",WIN_NORMAL)
'Call Email app: ?fHandleFile("mailto:dash10#hotmail.com",WIN_NORMAL)
'Open URL: ?fHandleFile("http://home.att.net/~dashish", WIN_NORMAL)
'Handle Unknown extensions (call Open With Dialog):
' ?fHandleFile("C:\TEMP\TestThis",Win_Normal)
'Start Access instance:
' ?fHandleFile("I:\mdbs\CodeNStuff.mdb", Win_NORMAL)
'****************************************************
Function fHandleFile(stFile As String, lShowHow As Long)
Dim lRet As Long, varTaskID As Variant
Dim stRet As String
'First try ShellExecute
lRet = apiShellExecute(hWndAccessApp, vbNullString, _
stFile, vbNullString, vbNullString, lShowHow)
If lRet > ERROR_SUCCESS Then
stRet = vbNullString
lRet = -1
Else
Select Case lRet
Case ERROR_NO_ASSOC:
'Try the OpenWith dialog
varTaskID = Shell("rundll32.exe shell32.dll,OpenAs_RunDLL " _
& stFile, WIN_NORMAL)
lRet = (varTaskID <> 0)
Case ERROR_OUT_OF_MEM:
stRet = "Error: Out of Memory/Resources. Couldn't Execute!"
Case ERROR_FILE_NOT_FOUND:
stRet = "Error: File not found. Couldn't Execute!"
Case ERROR_PATH_NOT_FOUND:
stRet = "Error: Path not found. Couldn't Execute!"
Case ERROR_BAD_FORMAT:
stRet = "Error: Bad File Format. Couldn't Execute!"
Case Else:
End Select
End If
fHandleFile = lRet & _
IIf(stRet = "", vbNullString, ", " & stRet)
End Function
'************ Code End **********
Just put this into a separate module and call fHandleFile() with the right parameters.
I find the most simple is
shell "explorer.exe URL"
This also works to open local folders.
You need to call ShellExecute.