VBA webpage scrape cannot fill search box, potential security issue..? - vba

I have the following abridged code to navigate to a webpage and fill a search box. However, I cannot understand how to fill the search box, given the classname is embedded in an input element. Please let me know if you have any insight. Thanks.
Dim ie, results As Object
Dim pagenumber, pagetotal, r, s As Long
Dim finrawebpage As HTMLDocument
Dim searchterm As HTMLElementCollection
Dim i As Integer
Set ie = CreateObject("InternetExplorer.application")
brokersearch = InputBox("ENTER BROKER NAME OR CRD#")
'firmsearch = InputBox("ENTER FIRM NAME OR CRD#")
'geosearch = InputBox("ENTER CITY, STATE, OR ZIP")
Application.StatusBar = "LOADING FINRA SEARCH"
With ie
.Visible = True
.navigate "https://brokercheck.finra.org/"
Do While .busy Or _
.readystate <> 4
DoEvents
Loop
Set finrawebpage = ie.document
'Set searchterm = finrawebpage.getElementsByTagName("input")
'i = 0
'While i < searchterm.Length
'If searchterm(i).Type = "text" Then
'searchterm(i).Value = brokersearch
'Set searchterm = finrawebpage.getElementsByClassName("ng-pristine ng-scope ng-empty ng-valid ng-valid-required ng-touched")
'If searchterm.Length > 0 Then
'searchterm(0).Value = brokersearch
'End If
finrawebpage.getElementsByClassName("ng-pristine ng-scope ng-empty ng-valid ng-valid-required ng-touched").Item.Value = brokersearch
'finrawebpage.getElementsByClassName("searchAutoContainer flex-auto").Item.innertext = brokersearch
'finrawebpage.getElementsByClassName("md-raised md-primary md-hue-2 md-button md-ink-ripple").Item.Click
'Wend
End With
End Sub

It looks like you were on the right track based on the commented lines in your code. Try:
Do While i < ie.document.getElementsByTagName("input").length
If ie.document.getElementsByTagName("input")(i).getAttribute("placeholder") = "Name or CRD#" Then
ie.document.getElementsByTagName("input")(i).value = brokersearch
End If
Loop
This will eliminate the need to hard code the position of the element which has a good chance of breaking if they update the page frequently. That isn't to say keying off the placeholder is perfect, but your options are limited based on what I saw on the page.

Related

Excel VBA cannot get element and input data a website

I am trying to write a VBA script to input keywords (For expamle: Amuse) into the following website's textbox "Map Positioning" and click "go" to search the place automatically.
CentaMap
Here is the html script I found for the textbox
<INPUT onkeyup=searchBoxTextChanged(this.value); id=qbyid style="FONT-SIZE: 10pt" maxLength=60 name=q autocomplete="off">
Knowing that the normal way to do is to use get element by id then input such data in it. However I am being stuck as I cannot get the textbox element with the following codes:
Sub SubCentalineAutomation()
Dim myIE As InternetExplorer
Const url As String = "http://hk.centamap.com/gc/home.aspx?lg=en"
Set myIE = New InternetExplorer
myIE.navigate (url)
Do While myIE.readyState <> 4
DoEvents
Loop
myIE.Visible = True
myIE.document.getElementsByName("q")(0).Value = "Amuse"
End Sub
I tried to replace the codes by using getElementsById("qbyid") instead however VBA cannot find the element too.
Can anybody help on this?
Needs Reference to
Microsoft Internet Controls
Microsoft HTML Object Library
Sub SubCentalineAutomation()
Dim myIE As InternetExplorer
Dim frame As MSHTML.HTMLFrameElement
Dim inp As MSHTML.HTMLInputElement
Const url As String = "http://hk.centamap.com/gc/home.aspx?lg=en"
Set myIE = New InternetExplorer
myIE.navigate (url)
Do While myIE.readyState <> 4
DoEvents
Loop
myIE.Visible = True
Set frame = myIE.document.getElementsByName("search")(0)
Set inp = frame.contentDocument.getElementsByName("q")(0)
inp.Value = "Amaze"
End Sub
try getElementById("qbyid"), not getElementsById("qbyid")
If you simply want the suggestion, without the map navigation you can use:
Option Explicit
Public Sub GetInfo()
Dim sResponse As String, html As New HTMLDocument, searchTerm As String
searchTerm = "Amuse"
With CreateObject("MSXML2.XMLHTTP")
.Open "GET", "http://hk.centamap.com/gsearch/paddresssearch1.aspx?lg=en&search=" & searchTerm & "&ck=gbase&ft2=", False
.send
sResponse = StrConv(.responseBody, vbUnicode)
sResponse = Mid$(sResponse, InStr(1, sResponse, "<!DOCTYPE "))
With html
.body.innerHTML = sResponse
Debug.Print html.querySelector("a").innerText
End With
End With
End Sub
If you want to enter a value and then navigate to the first returned suggestion:
Option Explicit
Public Sub SubCentalineAutomation()
Dim myIE As InternetExplorer, html As New MSHTML.HTMLDocument, frame As MSHTML.HTMLFrameElement, form As MSHTML.HTMLFormElement
Const URL As String = "http://hk.centamap.com/gc/home.aspx?lg=en"
Set myIE = New InternetExplorer
With myIE
.navigate URL
.Visible = True
While .Busy Or .readyState < 4: DoEvents: Wend
Set html = .document
Set frame = html.getElementsByTagName("frame")(1)
Set form = frame.contentDocument.getElementsByTagName("form")(0)
form.getElementsByTagName("input")(1).Value = "Amuse"
form.getElementsByTagName("input")(2).Click
While .Busy Or .readyState < 4: DoEvents: Wend
Set frame = .document.getElementsByTagName("frame")(4)
Set frame = frame.contentDocument.getElementsByTagName("iframe")(0)
frame.contentDocument.getElementsByTagName("table")(0).getElementsByTagName("a")(0).Click
'.Quit '<== Remember to quit application
End With
End Sub

Using VBA to click a button

I am trying to pull information from a website, but cannot for the life of me figure out how to click the button. I've got the code to input the information. This is the html for the website . Any help would be appreciated.
<p role="button" tabindex="0" class="fmtbutton" onkeypress="finddistancebetweenaandb(document.forms['inp']['pointa'].value,document.forms['inp']['pointb'].value);" onclick="finddistancebetweenaandb(document.forms['inp']['pointa'].value,document.forms['inp']['pointb'].value);"> Show </p>
Here is the full code that I have so far.
Sub RoundedRectangle1_Click()
Dim eRow As Long
Dim ele As Object
Dim objIE As Object
Set objIE = CreateObject("InternetExplorer.Application")
FranchiseAddress = Range("B2").Value
Movefrom = Range("B3").Value
moveto = Range("B4").Value
With objIE
.Visible = True
.navigate "https://www.freemaptools.com/how-far-is-it-between.htm"
Do While .Busy Or _
.readyState <> 4
DoEvents
Loop
Set d = .Document.getElementsByName("onoffswitch")
d.Item(0).Click
Set f = .Document.getElementsByName("pointa")
f.Item(0).Value = FranchiseAddress
Set a = .Document.getElementsByName("pointb")
a.Item(0).Value = Movefrom
' This is where I'm Stuck'
End With
End Sub
I have tried multiple solutions found on this site for many questions, but none of them seem to work.
Thank you.
add this to your code
Dim aaa As Object
Set aaa = .Document.getElementsByClassName("fmtbutton") ' this returns 4 objects, your button is the first one
aaa(0).Click
You can be a little more elegant and target the attribute with a CSS selector:
.document.querySelector("[role=button]").Click
The attribute = value selector, [role=button], looks for an element with attribute role whose value is button. There is only one on the page so no loop needed.

VBA IE change dropdown value

Tried the below code for following URL Scripture look up. Please how to change drop down value from WEB to RV1909?
Dim Doc As HTMLDocument
Set Doc = IEApp.document
TestV2 = ""
TestV3 = ""
TestV2 = Doc.getElementsByClassName("app-list text-list")(0).innerText
Debug.Print "4b of 5: " & TestV2
IEApp.Doc.getElementsByClassName("app-list text-list").selectedIndex = 1
IEApp.Doc.getElementsByClassName("app-list text-list").FireEvent ("onchange")
TestV3 = Doc.getElementsByClassName("app-list text-list")(0).innerText
Debug.Print "4c of 5: " & TestV3
Tried many approaches from other posts, the following does not work:
IEApp.Doc.getElementsByClassName("app-list text-list")(0).innerHTML = "RV1909"
Here is the screenshot of Chrome Inspector:
In this case it's not enough just change div.app-list.text-list element innerText, as you can see that element is simple div, but not even ul. It should be changed by scripts which are called on click events. So, first of all you need to click on div to display the entire list, and then click on the RV1909 item. The below example shows how that could be done:
Sub Test()
Dim oIE As Object
Dim oDiv As Object
Set oIE = CreateObject("InternetExplorer.Application")
With oIE
.Visible = True
.Navigate "http://ebible.org/study/"
Do While .readyState <> 4 Or .Busy
DoEvents
Loop
With .Document
Do While .readyState <> "complete"
DoEvents
Loop
Set oDiv = .getElementsByClassName("app-list text-list")(0)
oDiv.Click
Do While .readyState <> "complete"
DoEvents
Loop
For Each oNode In .getElementsByClassName("text-chooser-main")(0).getElementsByClassName("text-chooser-abbr")
If oNode.innerText = "RV1909" Then
oNode.Click
Do While oDiv.innerText <> "RV1909"
DoEvents
Loop
Exit For
End If
Next
End With
End With
End Sub
Change the html div element <div class=app-list text-list>WEB</div> to <div class=app-list text-list>RV1909</div>

VBA code not selecting a value from a dropdown that has a class but no id

I have some code which is meant to open an IE page and fill in the first drop-down to show "caravan". However my code is not selecting caravan from the drop-down.
The website is:
https://insurance.qbe.com.au/portal/caravan/form/estimate
When I right-click to inspect the source code of the drop-down I get directed to the following:
When I expand the source code below this, I see the following:
As you can see, the values associated with the drop-down are well below the source code I was initially directed to.
Here is the code I've been using to try and select 'caravan':
Sub GetQuote()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.navigate ("https://insurance.qbe.com.au/portal/caravan/form/estimate")
IE.Visible = True
Do
DoEvents
Loop Until IE.readystate = 4
Dim lists, l
Set lists = IE.document.getElementsByTagName("select")(0)
For Each l In lists
If InStr(1, l.className, "form-control ng-pristine ng-untouched ng-valid", 1) Then
l.SelectedIndex = 0
End If
Next
Application.Wait (Now + TimeValue("00:00:03"))
End Sub
With the above code I was trying to select 'caravan' by referencing the very last bit of the source code I pasted above (i.e. by getting all select tags and then checking all items within the tag until an item whose class name contains "form-control ng-pristine ng-untouched ng-valid" is found, and then selecting index 0 for the value associated with this item. And this obviously didn't work - so here's other code I have tried, again with no luck (it's a slight modification of the code above, by simply referencing the source code that I was initially directed to when I inspected the drop-down, and that would be the very first source code I pasted):
For Each l In lists
If InStr(1, l.className, "ui-select-search ui-select-toggle ng-pristine ng-valid ng-touched", 1) Then
l.SelectedIndex = 0
Does this offer any help, not sure if it's your solution, but added here for formatting.
Sub test()
Dim ie As SHDocVw.InternetExplorer
Dim htmlDoc As MSHTML.HTMLDocument
Dim inputCollection As MSHTML.IHTMLElementCollection
Dim inputElement As MSHTML.HTMLInputElement
Set ie = New SHDocVw.InternetExplorer
ie.navigate ("https://insurance.qbe.com.au/portal/caravan/form/estimate")
Do While ie.Busy Or ie.readyState <> READYSTATE_COMPLETE
DoEvents
Loop
ie.Visible = 1
Set htmlDoc = ie.document
Set inputCollection = htmlDoc.getElementsByTagName("INput")
For Each inputElement In inputCollection
Debug.Print inputElement.className
Next inputElement
' or like this
Set inputElement = htmlDoc.getElementsByClassName("ui-select-search ui-select-toggle ng-pristine ng-valid ng-touched")(0)
End Sub

I need to find and press a button on a webpage using VBA

I have written a macro that will open a webpage, find the spots I need to put the data into, and then I want the macro to hit a prefill button, then hit Ok.
The page source code for the button is:
<input type="text" size="30" maxlength="40" name="name" id="name">
<input type="button" value="Prefill" onclick="prefill()">
I've been searching for answers all week and I think I have a basic understanding of how this is supposed to work by running a loop to search for it, but I'm having no luck in my endeavor of actually getting this to work.
Can someone show me the loop that will search my page for this button?
Thank you in advance.
Requested code so far
Private Sub Populate_Click()
Dim i As Long
Dim IE As Object
Dim objElement As Object
Dim objCollection As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.Navigate "website" 'make sure you've logged into the page
Do
DoEvents
Loop Until IE.READYSTATE = 3
Do
DoEvents
Loop Until IE.READYSTATE = 4
ActiveSheet.EnableCalculation = False
ActiveSheet.EnableCalculation = True
Call IE.document.getelementbyid("name").SetAttribute("value", ActiveSheet.Range("b2").Value)
Call IE.document.getelementbyid("aw_login").SetAttribute("value", ActiveSheet.Range("a2").Value)
Set objCollection = IE.document.getElementsByTagName("input")
i = 0
While i < objCollection.Length
If objCollection(i).Type = "button" And _
objCollection(i).Name = "Prefill" Then
Set objElement = objCollection(i)
End If
i = i + 1
Wend
objElement.Click
End Sub
Looks like you are pretty close, this is what I have used to do something similiar
With ie.document
Set elems = .getelementsbytagname("input")
For Each e In elems
If (e.getattribute("className") = "saveComment") Then
e.Click
Exit For
End If
Next e
End With
You will probably just have to change the if statement.
I also notice that your code refers to .Name = "Prefill" but your html snippet refers to .value = "Prefill"