Alternative to URLDownloadtofile when automating IE with VBA - vba

Question

I have been using InternetExplorer.application with Excel VBA for quite a while with few issues. One problem I have is downloading a file from website. I can get as far as having the "Open/Save As" buttons appear but that is where I am stuck.
I've tried using URLDownloadToFile and it does not seem to work through the same session as the InternetExplorer.application objects that I have. It usually returns the HTML text for a webpage stating that authentication is required. If I have multiple browsers open and some of the old ones are already authenticated then it will download the file most of the time.
Is there a way to download the file using the InternetExplorer.application object itself? If not, is there some way I can associate the URLDownloadtofile function with the object that is already authenticated and logged into the website?
EDIT:
The code I've been using is:
IE2.navigate ("https://...")
strURL = "https://..."
strPath = "c:\..."
Ret = URLDownloadToFile(0, strURL, strPath, 0, 0)
I've also tried:
Do While IE2.Readystate <> 4
DoEvents
Loop
SendKeys "%S"
IE2.ExecWB OLECMDID_SAVEAS, OLECMDEXECOPT_DODEFAULT
And:
Dim Report As Variant
Report = Application.GetSaveAsFilename("c:\...", "Excel Files (*.xls), *.xls")
No success in any of these, except for the first one which sometimes saves the actual file, but sometimes saves the website that states the authentication error.
Thanks,
Dave

Answer 1

I have managed to solve similar issue with some JavaScript.
The first step is to make JavaScript download the content of the file into a binary array (it doesn't require another authentication once the user is already authenticated).
Then, I needed to pass this binary array back to VBA. I didn't know the other way, so I print the content of this array into a temporary DIV element (with JavaScript) as a string and then read it with VBA and convert it back to binary array.
Finally, I re-created the file from the given binary array by using ADODB.Stream class.
The time required to download a single file grows geometrically with the size of this file. Therefore, this method is not suitable for large files (> 3MB), since it tooks more than 5 minutes then to download a single file.
Below is the code to do that:
'Parameters:
' * ie - reference to the instance of Internet Explorer, where the user is already authenticated.
' * sourceUrl - URL to the file to be downloaded.
' * destinationPath - where the file should be saved.
'Be aware that the extension of the file given in [destinationPath] parameter must be
'consistent with the format of file being downloaded. Otherwise the function below will
'crash on the line: [.SaveToFile destinationPath, 2]
Public Function saveFile(ie As Object, sourceUrl As String, destinationPath As String)
Dim binData() As Byte
Dim stream As Object
'------------------------------------------------------------------------------------
binData = getDataAsBinaryArray(ie, sourceUrl)
Set stream = VBA.CreateObject("ADODB.Stream")
With stream
.Type = 1
.Open
.write binData
.SaveToFile destinationPath, 2
End With
End Function
Private Function getDataAsBinaryArray(Window As Object, Path As String) As Byte()
Const TEMP_DIV_ID As String = "div_binary_transfer"
'---------------------------------------------------------------------------------------------
Dim strArray() As String
Dim resultDiv As Object
Dim binAsString As String
Dim offset As Integer
Dim i As Long
Dim binArray() As Byte
'---------------------------------------------------------------------------------------------
'Execute JavaScript code created automatically by function [createJsScript] in
'the given Internet Explorer window.
Call Window.Document.parentWindow.execScript(createJsScript(TEMP_DIV_ID, Path), "JavaScript")
'Find the DIV with the given id, read its content to variable [binAsString]
'and then convert it to array strArray - it is declared as String()
'in order to make it possible to use function [VBA.Split].
Set resultDiv = Window.Document.GetElementById(TEMP_DIV_ID)
binAsString = VBA.Left(resultDiv.innerhtml, VBA.Len(resultDiv.innerhtml) - 1)
strArray = VBA.Split(binAsString, ";")
'Convert the strings from the [strArray] back to bytes.
offset = LBound(strArray)
ReDim binArray(0 To (UBound(strArray) - LBound(strArray)))
For i = LBound(binArray) To UBound(binArray)
binArray(i) = VBA.CByte(strArray(i + offset))
Next i
getDataAsBinaryArray = binArray
End Function
'Function to generate JavaScript code doing three tasks:
' - downloading the file with given URL into binary array,
' - creating temporary DIV with id equal to [divId] parameter,
' - writing the content of binary array into this DIV.
Private Function createJsScript(divId As String, url As String) As String
createJsScript = "(function saveBinaryData(){" & vbCrLf & _
"//Create div for holding binary array." & vbCrLf & _
"var d = document.createElement('div');" & vbCrLf & _
"d.id = '" & divId & "';" & vbCrLf & _
"d.style.visibility = 'hidden';" & vbCrLf & _
"document.body.appendChild(d);" & vbCrLf & _
"var req = null;" & vbCrLf & _
"try { req = new XMLHttpRequest(); } catch(e) {}" & vbCrLf & _
"if (!req) try { req = new ActiveXObject('Msxml2.XMLHTTP'); } catch(e) {}" & vbCrLf & _
"if (!req) try { req = new ActiveXObject('Microsoft.XMLHTTP'); } catch(e) {}" & vbCrLf & _
"req.open('GET', '" & url & "', false);" & vbCrLf & _
"req.overrideMimeType('text/plain; charset=x-user-defined');" & vbCrLf & _
"req.send(null);" & vbCrLf & _
"var filestream = req.responseText;" & vbCrLf & _
"var binStream = '';" & vbCrLf & _
"var abyte;" & vbCrLf & _
"for (i = 0; i < filestream.length; i++){" & vbCrLf & _
" abyte = filestream.charCodeAt(i) & 0xff;" & vbCrLf & _
" binStream += (abyte + ';');" & vbCrLf & _
"}" & vbCrLf & _
"d.innerHTML = binStream;" & vbCrLf & _
"})();"
End Function

Answer 2

How about something like this?
Public Sub OpenWebXLS()
' *************************************************
' Define Workbook and Worksheet Variables
' *************************************************
Dim wkbMyWorkbook As Workbook
Dim wkbWebWorkbook As Workbook
Dim wksWebWorkSheet As Worksheet
Set wkbMyWorkbook = ActiveWorkbook
' *************************************************
' Open The Web Workbook
' *************************************************
Workbooks.Open ("http://www.sportsbookreviewsonline.com/scoresoddsarchives/nba/nba%20odds%202015-16.xlsx")
' *************************************************
' Set the Web Workbook and Worksheet Variables
' *************************************************
Set wkbWebWorkbook = ActiveWorkbook
Set wksWebWorkSheet = ActiveSheet
' *************************************************
' Copy The Web Worksheet To My Workbook and Rename
' *************************************************
wksWebWorkSheet.Copy After:=wkbMyWorkbook.Sheets(Sheets.Count)
wkbMyWorkbook.Sheets(ActiveSheet.Name).Name = "MyNewWebSheet"
' *************************************************
' Close the Web Workbook
' *************************************************
wkbMyWorkbook.Activate
wkbWebWorkbook.Close
End Sub