How to automate a dynamically changing web page using Excel VBA? - vba

Question

I have been trying to automate a web page since two weeks but I could not proceed further after 3rd page.
First I'm logging into login page by giving credentials and then I would click a link from 2nd page. Until this point I'm fine; but after that again I need to click another link from the 3rd page that I'm not able to, Even I was not able to read the proper innerhtml of that particular page. The innerhtmal varies from the source code of that page. Using the source code I have taken the id/name to get the element but no use. The problem I'm seeing is the DOCUMENT object is not taking the inner details of 3rd page. When I tried to print the links of that page it printed me some common links in that page which would be available in all the pages instead of printing all the links in that particular page. I guess this might happen because the page frame varies with respect to the FromDate & ToDate. Pardon me if I'm wrong. Do we need to change every time the "ie.document" object with respect to the navigation of web page? Because I think it sticks with the same when the page loaded 1st time.
Below is my code:
Public Sub Test ()
Dim shellWins As ShellWindows
Dim ie As InternetExplorer
Dim doc As HTMLDocument
Dim frm As HTMLFrameElement
Dim frms As HTMLElementCollection
Dim strSQL As String
Dim Login As Boolean
strSQL = "https://website.com"
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
.Navigate strSQL
Do Until .ReadyState = 4: DoEvents: Loop
Set doc = ie.document
Dim link As Object
For Each link In doc.Links
'Debug.Print link.innerText
If link.innerText = "Click Here" Then
link.Click
Exit For
End If
Next link
Do While ie.Busy: DoEvents: Loop
Login_Pane:
For Each link In doc.Links
If link.innerText = "Leave & Attendance" Then
'Debug.Print doc.body.innerHTML
link.Click
Login = True
Exit For
End If
Next link
If Login <> True Then
Application.Wait (Now + TimeValue("00:00:02"))
Application.SendKeys "<USERNAME>", True
Application.SendKeys "{TAB}"
Application.Wait (Now + TimeValue("00:00:02"))
Application.SendKeys "<PASSWORD>", True
Application.SendKeys "{ENTER}"
GoTo Login_Pane
End If
Do While ie.Busy: DoEvents: Loop
Dim link As Object
For Each link In doc.Links
Debug.Print link.innerText
' Above line code should print all the links in that page_
_but unfortunatly it is not displaying as it is in the source code.
' instead printing half of the links which are commonly_ _available in all pages.
' This page has three frames
Next link
End With
'IE.Quit
End Sub
i'm unable to post the image of that page to make you understand more, Anyways i'll try my best.
when i use this below code i can only able to get the links from the upper portion of the page.
Set doc = ie.document
Dim text As Object
For Each text In doc.Links
Debug.Print text.innerText
Next text
Below to that portion of the page i have option to enter FromDate & ToDate, by giving dates to this textboxes i'll be able to see the details according to the dates (by default page displayes the details from 1st of the curent month to the current date of the month).
So, here i'm not getting the links/or other details. And i think the details of this sections are not stored in the ie.document object.
And this particular section alone has different URL from the main page.
Thanks.

Answer 1

A couple of thoughts:
For a page that dynamically loads you need to use Application.Wait (5 seconds or so) instead of Do Until .ReadyState = 4: DoEvents: Loop. The latter does not work if you have javascript being executed.
Using SendKeys should always be avoided as it is not robust.Inspect the element with a DOM explorer to get the ID or name.