How can I import data from an ambiguous URL? - vba

Question

This is almost working!! But not quite!
If link.innerHTML Like "*Upload Questionnaire*" Then
link.Click
Set objIE = CreateObject("InternetExplorer.Application")
objIE.Navigate strURL
Do While objIE.ReadyState <> 4 And objIE.Busy
DoEvents
Loop
Set XMLHTTP = CreateObject("MSXML2.XMLHTTP")
XMLHTTP.Open "GET", strSQL, False
XMLHTTP.send
Set html = CreateObject("htmlfile")
html.body.innerHTML = XMLHTTP.responseText
Set tbl = html.getElementsByTagName("Table")
Set tr_coll = tbl(0).getElementsByTagName("TR")
For Each tr In tr_coll
j = 1
Set td_col = tr.getElementsByTagName("TD")
For Each td In td_col
Cells(row + 1, j).Value = td.innerText
j = j + 1
Next
row = row + 1
Next
End If
For one thing, the code doesn't pause and wait for the browser to finish loading.
Do While objIE.ReadyState <> 4 And objIE.Busy
DoEvents
Loop
Also, I'm feeding in a Parent URL, like this:
strSQL = "https://blah_blah_blah_CampaignID=" & cell.Value
The line below doesn't work.
XMLHTTP.Open "GET", strSQL, False
Somehow I need to pass in the Child URL that opens from the Parent URL. This is the parent URL: strSQL = "https://blah_blah_blah_CampaignID=" & cell.Value
When: link.Click
runs then the Child URL opens, but I don't know how to reference the Child URL. How can I do that?!
Thanks in advance!

Answer 1

Assuming that the link is contained within the class, rather than looping through all a elements within the document itself, use getElementsByClassName() to grab ever element in your target class, and then loop through those.
Something like:
For each classElement in IE.document.getElementsByClassName("thClass")
For each linkElement in classElement.getElementsByTagName("a")
'check for innerHTML, etc.
Next
Next
If the class is within the link, you can just flip the loops.
This should work, for example:
Public Sub LookADemo()
Dim ie As InternetExplorer, doc As HTMLDocument
Dim thisClass As IHTMLElement2, thisLink As IHTMLElement
Set ie = New InternetExplorer
ie.navigate "https://www.google.com"
Do
DoEvents
Loop Until (ie.ReadyState >= READYSTATE_INTERACTIVE) And (ie.Busy = False)
Set doc = ie.Document
For Each thisClass In doc.getElementsByClassName("hp")
For Each thisLink In thisClass.getElementsByTagName("a")
Debug.Print thisLink.InnerText
Next
Next
ie.Quit
End Sub