I want to read web pages using Excel VBA. How do I carry out this task? Is it even possible?
Coming from Excel 2003, yes this is possible - you may use the SHDocVw.InternetExplorer and MSHTML.HTMLDocument objects to call a web page, gain control over and interact with the DOM object. After creating references to Microsoft HTML Object Library (...\system32\MSHTML.TLB) and Microsoft Internet Control (...\system32\ieframe.dll) you can play with the following example:
Sub Test()
Dim Browser As SHDocVw.InternetExplorer
Dim HTMLDoc As MSHTML.HTMLDocument
Set Browser = New SHDocVw.InternetExplorer ' create a browser
Browser.Visible = True ' make it visible
Application.StatusBar = ".... opening page"
Browser.navigate "http://www.google.com" ' navigate to page
WaitForBrowser Browser, 10 ' wait for completion or timeout
Application.StatusBar = "gaining control over DOM object"
Set HTMLDoc = Browser.document ' load the DOM object
WaitForBrowser Browser, 10
' at this point you can start working with the DOM object. Usefull functions are
' With HTMLDoc
' .getElementByID(string)
' .getElementsByTagName(string)
' .getElementsByName(string)
' .getAttribute(string)
' .setAttribute string, string .... to change field values, click a button etc.
' End With
Application.StatusBar = "" ' time to clean up
Browser.Quit
Set HTMLDoc = Nothing
Set Browser = Nothing
End Sub
Sub WaitForBrowser(Browser As SHDocVw.InternetExplorer, Optional TimeOut As Single = 10)
Dim MyTime As Single
MyTime = Timer
Do While Browser.Busy Or (Timer <= MyTime + TimeOut)
DoEvents
Loop
' if browser still busy after timeout, give up
If Browser.Busy Then
MsgBox "I waited for " & Timer - MyTime & " seconds, but browser still busy" & vbCrLf & _
"I give up now!"
End
End If
End Sub
You can use VBA to automate IE (plenty of examples via Google) or you can fetch the page directly using an instance of MSHTTP (ditto plenty of examples on the web). Which is best for your needs will depend on exactly what you want to do. Hard to say more without more detailed requirements.
Related
I inherited this VBA script from my predecessor. It works fine for me in Excel 2013 up until recently when I was told I may need to work from home. Come to find out, the Office 2016 environment of my newly accessed VPN desktop does not like this script. I keep getting "The remote server machine is unknown or unavailable" when it reaches .ReadyState <> READYSTATE_COMPLETE.
The navigation did not fail as I can see the window where it successfully navigated to the URL and I can interact with it correctly. The strange thing is if I change the URL to "www.google.com" I get a valid ready state result.
I also need to figure out how to late bind the Shell Windows so it will work with both the v15 and v16 libraries simultaneously.
The intent of this script is to automate a process that
1. Opens an internal database at DBurl via web interface
2. Manipulates and runs a java script located on the web page
3. Close the browser window without closing any other browser windows
This could be modified for someone else's use by looking for a page element, such as a search box or specific button on a page, and interacting with it.
Edit:
Additional testing has revealed that a pause at and skipping the Do While loop and resuming at IETab1 = SWs.Count results in this script working in Office 2016. The only issue, then, is without the loop, the page isn't yet ready for the next step when the script tries to run the interaction. A wait for 5 seconds in place of the loop band-aid's this issue. Finding why the .ReadyState won't read will fix this issue.
Declare PtrSafe Function apiShowWindow Lib "user32" Alias "ShowWindow" _
(ByVal hwnd As Long, ByVal nCmdShow As Long) As Long
Sub OpenWebDB()
Dim ieApp As Object
Dim SWs As ShellWindows
Dim IETab1 As Integer
Dim JScript As String
Dim CurrentWindow As Object
Dim DBurl As String
Dim tNow As Date, tOut As Date
DBurl = "My.Database.url"
Set SWs = New ShellWindows
tNow = Now
tOut = tNow + TimeValue("00:00:15")
If ieApp Is Nothing Then
Set ieApp = CreateObject("InternetExplorer.Application")
With ieApp
.Navigate DBurl
Do While tNow < tOut And .ReadyState <> READYSTATE_COMPLETE
DoEvents
tNow = Now
Loop
IETab1 = SWs.Count
End With
End If
If Not tNow < tOut Then GoTo DBFail
On Error GoTo DBFail
Set CurrentWindow = SWs.Item(IETab1 - 1).Document.parentWindow
JScript = "javascript: DoSomething"
Call CurrentWindow.execScript(JScript)
On Error GoTo 0
SWs.Item(IETab1 - 1).Quit
Set ieApp = Nothing
Set SWs = Nothing
Exit Sub
DBFail:
MsgBox (DBurl & vbCrLf & "took too long to connect or failed to load correctly." & vbCrLf & _
"Please notify the Database manager if this issue continues."), vbCritical, "DB Error"
SWs.Item(IETab1 - 1).Quit
Set ieApp = Nothing
Set SWs = Nothing
End Sub
Try to remove the tNow < tOut from the Do While condition. Or, using the While statement to wait page complete:
While IE.ReadyState <> 4
DoEvents
Wend
The intent of this script is to automate a process that
1. Opens an internal database at DBurl via web interface
2. Manipulates and runs a java script located on the web page
3. Close the browser window without closing any other browser windows
Besides, according to the intent of the script, I suggest you could refer the following code (it could loop through the tabs, and close specific tab according the title):
Sub TestClose()
Dim IE As Object, Data As Object
Dim ticket As String
Dim my_url As String, my_title As String
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Visible = True
.Navigate "https://www.microsoft.com/en-sg/" '1st tab
.Navigate "https://www.bing.com", CLng(2048) '2nd
.Navigate "https://www.google.com", CLng(2048) '3rd
While IE.ReadyState <> 4
DoEvents
Wend
'wait some time to let page load
Application.Wait (Now + TimeValue("0:00:05"))
'get the opened windows
Set objShell = CreateObject("Shell.Application")
IE_count = objShell.Windows.Count
'loop through the window and find the tab
For x = 0 To (IE_count - 1)
On Error Resume Next
'get the location and title
my_url = objShell.Windows(x).Document.Location
my_title = objShell.Windows(x).Document.Title
'debug to check the value
Debug.Print x
Debug.Print my_title
'find the special tab based on the title.
If my_title Like "Bing" & "*" Then
Set IE = objShell.Windows(x)
IE.Quit 'call the Quit method to close the tab.
Exit For 'exit the for loop
Else
End If
Next
End With
Set IE = Nothing
End Sub
Thanks in advance for your answers.
I have a little Excel document that is made so that the naïve user can enter web pages in an Excel sheet, hit a button, and play the videos from that page in their browser, in full screen, and automatically loop the videos without any further user interaction. It basically creates a slide show of videos.
I originally made it for YouTube and it works fine there. I'm now trying to expand it to use another site. It works as planned but needs an extra step.
Whereas YouTube was made with a Full Screen mode that I can access programatically, this website has embedded videos. (An example: https://www.sharecare.com/video/health-topics-a-z/copd/what-can-i-do-to-prevent-my-copd-from-getting-worse).
You can see in the code that I open IE in full screen mode (which it does) but that's the full web page (header, side banner etc.). I want the video from that page to be the only element, full screen.
If I physically go into the page I can select for the video to play full screen. I've tried searching for various ways to do this, but most of the posts are for something else or how to get a video to play inside Excel rather than what I'm doing.
Sub StartLooping()
Dim IEapp As Object
Dim VidAddr1, VidAddr2 As String
Dim AddrStrStart, AddrStrEnd As Long
Dim AddrFudge1, AddrFudge2 As Integer
Dim TimeStart, DurMin, DurSec, DurTot As Single
Dim LRAll, LRVid, LRMin, LRSec, LRVidB, LRMinB, LRSecB As Integer
Dim I As Integer
Application.EnableCancelKey = xlErrorHandler
On Error GoTo ErrorHandle
'Review Sheet
LRVid = Cells(Rows.Count, "D").End(xlUp).Row
LRMin = Cells(Rows.Count, "E").End(xlUp).Row
LRSec = Cells(Rows.Count, "F").End(xlUp).Row
LRVidB = Cells(Rows.Count, "I").End(xlUp).Row
LRMinB = Cells(Rows.Count, "J").End(xlUp).Row
LRSecB = Cells(Rows.Count, "K").End(xlUp).Row
LRAll = Cells(Rows.Count, "S").End(xlUp).Row
If LRVid <> LRMin Then
MsgBox "You have to include video address and how long (the minutes and the seconds - use 0 if needed)"
Exit Sub
End If
If LRVid <> LRSec Then
MsgBox "You have to include video address and how long (the minutes and the seconds - use 0 if needed)"
Exit Sub
End If
If LRVidB <> LRMinB Then
MsgBox "You have to include video address and how long (the minutes and the seconds - use 0 if needed)"
Exit Sub
End If
If LRVidB <> LRSecB Then
MsgBox "You have to include video address and how long (the minutes and the seconds - use 0 if needed)"
Exit Sub
End If
'Start of For-Next Loop
For I = 20 To LRAll
'Set Addr
'VidAddr1
If Len(Range("S" & I).Text) = 0 Then
Exit Sub
Else
VidAddr1 = Range("S" & I).Text
End If
VidAddr2 = VidAddr1
'Set Timer
TimeStart = Timer 'Start time
DurMin = Range("T" & I).Value
DurSec = Range("U" & I).Value
DurTot = (DurMin * 60) + DurSec
'Open the web page
Set IEapp = CreateObject("Internetexplorer.Application") 'Set IEapp = InternetExplorer
With IEapp
.Silent = True 'No Pop-ups
.Visible = True
.FullScreen = True
.Navigate VidAddr2 'Load web page
'Keep it open for the duration
Do While Timer < (TimeStart + DurTot)
'Check for Esc - refers to a public function
If KeyDown(vbKeyEscape) Then
IEapp.Quit
Set IEapp = Nothing
Exit Sub
End If
Loop
'Close the page
IEapp.Quit
Set IEapp = Nothing
End With
If I = LRAll Then I = 19
Next I
ErrorHandle:
MsgBox Err.Number & " " & Err.Description
Exit Sub
End Sub
I copied the code. It works fine. It's just that extra bit of "Oh, I did that, here's how to go about it" that I need.
The browser used is IE so I can keep it simple, but if this were possible in another common browser that would be good to know.
Here's the second set that I tried today (9/12)
Dim IEapp As Object
Dim IEAppColl As HTMLButtonElement
'Open doc
Set IEapp = CreateObject("Internetexplorer.Application") 'Set IEapp = InternetExplorer
With IEapp
.Silent = True 'No Pop-ups
.Visible = True
'.FullScreen = True
.navigate "https://www.sharecare.com/video/health-topics-a-z/copd/got-copd-ask-your-doctor-about-vitamin-d"
Do While .readyState < 4 Or .Busy
Loop
Set IEAppColl = IEapp.Document.getElementsByTagName("BUTTON")
If IEAppColl.Name = "Fullscreen" Then
IEAppColl.Click
End If
End With
For the example COPD page given this works with Selenium basic. You install from here and then go VBE > Tools > References > and add a reference to Selenium Type Library. You can also use an IEDriver to work with InternetExplorer rather than Chrome (which uses ChromeDriver).
Option Explicit
Public Sub PlayFullScreen()
Dim d As WebDriver, t As Date, ele As Object
Const MAX_WAIT_SEC As Long = 10
Set d = New ChromeDriver
Const URL = "https://www.sharecare.com/video/health-topics-a-z/copd/what-can-i-do-to-prevent-my-copd-from-getting-worse"
With d
.Start "Chrome"
.get URL
Do
DoEvents
On Error Resume Next
Set ele = .FindElementByCss("[title='Accept Cookies']")
If Timer - t > MAX_WAIT_SEC Then Exit Do
On Error GoTo 0
Loop While ele Is Nothing
Application.Wait Now + TimeSerial(0, 0, 1)
If Not ele Is Nothing Then ele.Click
.FindElementByCss("#myExperience").Click
Application.Wait Now + TimeSerial(0, 0, 1)
.FindElementByCss("[Title=Fullscreen]", timeout:=7000).Click
Stop '<==Delete me later
.Quit
End With
End Sub
My suggestion is to, instead of using a browser to play video, use a video player.
Under "More Controls" you should have a "Windows Media Player", and likely others, depending on what you have installed.
For example, I've used the VLC control on Access forms. When you install VLC it automatically adds the control to Office (so I assume Office has to be installed first.)
Here's a tutorial I found online:
Link: How to Play Video on Access Form
Random Tip Time:
It can be tricky to Google about a website because, for example, using the search term *YouTube* in a Google search results in a list of YouTube content (videos).
Exclude results from a specific site with Google's site: and - operators like:
"microsoft access" form play youtube video -site:youtube.com
...which is how I found the tutorial above, and several others.
Using the search term access can also be tricky to search for since it's such a common word, which is why I'll often enclose it in quotes like "MS Access" or "Microsoft Access" (like above), which makes Google search for those words in that order.
(More Google tips)
I'm currently trying to automate my IE at work with VBA and Excel, because we have some repetitive tasks to complete in an SAP System which is called within the Internet Explorer.
So far I am able to start IE and navigate to the starting page of our System.
I looked on the source code from the Sap System to search for an id from a div (lets say its 'SampleID'). But if I try to click it via VBA I only get returned 'nothing'. So I tried to print out the source code via VBA Debug.Print to look at the problem. And here's the thing:
The source code that gets printed is completely different from the Website's Source Code. In the printed code for example is a iframe that I can't find in the website source code. So that's the reason I can't find my div and any other objects, because they are not in the code that VBA is searching through.
Can anyone tell me what's going wrong? Why are the source codes completely different?
Sub stgGo(){
Dim i As Long
Dim IE As InternetExplorerMedium
Dim objElement As Object
Dim objCollection As Object
Set IE = New InternetExplorer
IE.Visible = True
IE.Navigate ("URL")
'waiting 10 seconds to be really sure javascript is ready
Application.Wait (Now + TimeValue("0:00:10"))
'printing out the false source code
Debug.Print strHTML = myIE.Document.body.innerHTML
Set objCollection = IE.document.getElementsByTagName("div")
i = 0
While i < objCollection.Length
If objCollection(i).ID = "SampleID" Then
objCollection(i).Click
End If
i = i + 1
Wend
End Sub
in my Access & VBA application i Have to get and manage a web page of a web order application. Normally, I use the solution indicated in this post, which normally works perfectly:
VBA Open web page, login and get the opened page
But, now I have another old web application to connect with my Access software. In this web application, after login, the web page closes itself and open an other browser session with the first user page. In other words:
1) The user fill the login form with the user credentials
2) After login, the login web page closes the browser
3) An other browser session opens with the web app control panel
If I use the linked solution, I see an error. How can I get the new web page, in the new browser session, which opens it self after login?
Thanks!
EDIT
I have tried the following code (my purpose is set a text value in the web page). But, very strange, it works only if I put a text box in my code: maybe the code must wait some event! How can I solve it?
Dim SWs As SHDocVw.ShellWindows, vIE As SHDocVw.InternetExplorer
Dim winShell As Shell
Dim dt As Date
dt = DateAdd("s", 10, DateTime.Now)
Dim ieA As InternetExplorer
Dim IeDocA As HTMLDocument
Do While dt > DateTime.Now
Set winShell = New Shell
For Each ieA In winShell.Windows
If ieA.LocationURL = sURL Then
ieA.Visible = True
ieA.Silent = True
Do While ieA.Busy: DoEvents: Loop
Do Until ieA.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
Set IeDocA = ieA.Document
//only with this msgbox it works, how can I do this w/out it?
MsgBox IeDocA.title
With IeDocA
.getElementsByName("text").value="something"
End With
Set winShell = Nothing
Exit Do
End If
Next ieA
Set winShell = Nothing
DoEvents
Loop
You can Look for the URL in the window. You will need a reference to Microsoft Shell Controls and Automation.
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
'Desc: The Function gets the Internet Explorer window that has the current
' URL from the sURL Parameter. The Function Timesout after 30 seconds
'Input parameters:
'String sURL - The URL to look for
'Output parameters:
'InternetExplorer ie - the Internet Explorer window holding the webpage
'Result: returns the the Internet Explorer window holding the webpage
''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''''
Public Function GetWebPage(sUrl As String) As InternetExplorer
Dim winShell As Shell
Dim dt As Date
dt = DateAdd("s", 30, DateTime.Now)
Dim ie As InternetExplorer
Do While dt > DateTime.Now
Set winShell = New Shell
'loop through the windows and check the internet explorer windows
For Each ie In winShell.Windows
'check the URL
If ie.LocationURL = sUrl Then
'set some properties
ie.Visible = True
ie.Silent = True
Set GetWebPage = ie
'loop while IE is busy
Do While ie.Busy
DoEvents
Loop
Set winShell = Nothing
Exit Do
End If
Next ie
Set winShell = Nothing
DoEvents
Loop
End Function
I have a vba code which can upload the data from an excel sheet to a website. However, the code works fine in Win7 System and IE browser 8,but it does not work on a win8 IE browser 11.
Here are part of the code:
Dim objIE As SHDocVw.InternetExplorer
Dim htmlDoc As MSHTML.HTMLDocument
Dim htmlFrame As MSHTML.HTMLFrameElement
Dim frame As HTMLIFrame
Dim htmlElement As HTMLDTElement
Dim myDoc As Object
Set curSheet = ActiveWorkbook.ActiveSheet
Set oShApp = CreateObject("Shell.Application")
For Each oWin In oShApp.Windows
If oWin.Name = "Windows Internet Explorer" Then
Set IE = oWin
Exit For
End If
Next
If IE Is Nothing Then
MsgBox ("Please sign into Avocado, then re-run this macro")
Set IE = New InternetExplorerMedium
IE.Visible = True
IE.navigate "https://www.google.com"
Exit Sub
End If
Sheets("Prepare").Select
fPathName = Cells(5, 5)
Call MakeFolders(fPathName)
Call MakeFolders2(fPathName)
Call MakeFolders3(fPathName)
'fFileName = fPathName & "\*.xls"
fFileName = Dir(fPathName & "\*.xls")
The code runs in a loop when enters the statement : "If IE Is Nothing Then"
Even when the google site is opened, the program still keeps prompting out the msgBox, and reopen the website again and again, and it never executes to the last part "Sheets("Prepare").Select". I am very confused because it works perfect in the IE 8 browsers. I am wondering if there is any difference between IE11 and IE8 in terms of vba IE function.
Please take a look up it and give me some ideas on this, your help is greatly appreciated. Thank you very much.
I ran into the identical problem and found, that the window name of the shell.application has changed in IE 11. Earlier versions had the name "Windows Internet Explorer", but IE 11 uses only "Internet Explorer"
If you change your if condition accordingly, it will work again ...
I think you are using an old version of internet explorer. Use the code I provided bellow and see if it works. Please make sure you have the following references added to your project:
Microsoft HTML Object Library
Microsoft Internet Controls.
If you are not sure how to add the references to your code check out this link.External References
Sub Test()
Dim objIE As InternetExplorer
Dim htmlDoc As HTMLDocument
Dim htmlFrame As HTMLFrameElement
Dim frame As HTMLIFrame
Dim htmlElement As HTMLDTElement
Dim myDoc As Object
Dim curSheet As Worksheet
' Set the variables
Set curSheet = ActiveWorkbook.ActiveSheet
Set objIE = New InternetExplorer
' Make the browser visible and navigate
With objIE
.Visible = True
.navigate "https://www.google.com"
End With
WaitForInternetToLoad objIE
Sheets("Prepare").Select
fPathName = Cells(5, 5).Value
Call MakeFolders(fPathName)
Call MakeFolders2(fPathName)
Call MakeFolders3(fPathName)
'fFileName = fPathName & "\*.xls"
fFileName = Dir(fPathName & "\*.xls")
End Sub
Sub WaitForInternetToLoad(ByRef ie As InternetExplorer)
Do
dovents
Loop While Not ie.readyState = READYSTATE_COMPLETE
End Sub
Extra Information.
Hi I see you are having some trouble with this and I want to extend my help.
Let's start by the fact that you are using two external libraries in your code; and you should have references set to them for the code to work properly. See my picture I have highlighted the libraries yellow.
What are the differences:
Microsoft Internet Controls
Is the one that takes care of the internet object. It creates an internet explorer application and it navigates to certain links. Once it finishes navigating to the url then " internet object" has a "document". This object cannot do anything else.
Microsoft HTML Object Library
This library takes care of the Html document. And as you probably guessed you will assign the "document" from the previous object (internet explorer) to a HTML document variable and then you can do further manipulation.