VBA Internet Explorer dropdown list trouble - vba

I'm trying to scrape some data using VBA via Excel. I'm pretty rusty with VBA but I'm confident in my google-fu, 5+ hours in and no luck yet. I have seen many ways to work with drop-down menus but none seem to work with this one. I have successfully navigated to https://www.kbb.com/used-cars/ but I cannot work with the "year" drop-down. Maybe I am not using the correct id/name? Any advice is appreciated, let me know if I left any important info out.
Code:
Sub WebForumEntry()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Top = 0
IE.Left = 0
IE.Width = 800
IE.Height = 600
IE.AddressBar = 0
IE.StatusBar = 0
IE.Toolbar = 0
IE.Navigate "http://www.kbb.com"
IE.Visible = True
Do
DoEvents
Loop Until IE.ReadyState = 4
'Click on the "Used Car Values" link
Set AllHyperlinks = IE.document.getelementsbytagname("A")
For Each hyper_link In AllHyperlinks
If hyper_link.innertext = "Used Car Prices" Then
hyper_link.Click
Exit For
End If
Next
'Select model year"
Set e = IE.document.getElementbyid("yearDropdown")
Dim o
For Each o In e.Options
If o.Value = "2012" Then
o.Selected = True
Exit For
End If
Next
End Sub

In the HTML there are no Option tags for you to select:
<div class="form-group">
<select id="yearDropdown" name="yearId" class="dropdown-one year-dropdown" data-default-option="Year" data-preselected-value=""></select>
</div>
The years are generated dynamically depending on whether you choose new/ used etc. So you can't select something that doesn't exist - perhaps you can directly set the value e.g.:
IE.document.getElementbyid("yearDropdown").value = "2012"
Edit
Note there is an issue in your original code:
'Click on the "Used Car Values" link
Set AllHyperlinks = IE.document.getelementsbytagname("A")
For Each hyper_link In AllHyperlinks
If hyper_link.innertext = "Used Car Prices" Then
hyper_link.Click
Exit For
End If
Next
'Select model year"
Set e = IE.document.getElementbyid("yearDropdown") '<-- error here if page not loaded per hyper_link.Click
When you call hyper_link.Click it will load another page (where the dropdowns are) and then per the Exit For the next command to run is Set e = IE... which will try and access an element that doesn't exist yet because the page is still loading per the hyper_link.Click. To work-around this you can just put in a pause like below:
Application.Wait Now + TimeValue("0:00:04")
It's a bit of a hack - and there may be better ways to solve it - but it will allow you to set the dropdown per my first suggestion :
Full code (re-write of yours):
Option Explicit
Sub WebForumEntry()
Dim IE As Object
Dim AllHyperLinks As Object
Dim hyper_link As Object
Dim e As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Top = 0
IE.Left = 0
IE.Width = 800
IE.Height = 600
IE.AddressBar = 0
IE.StatusBar = 0
IE.Toolbar = 0
IE.Navigate "http://www.kbb.com"
IE.Visible = True
' wait for page to load
Do
DoEvents
Loop Until IE.ReadyState = 4
'Click on the "Used Car Values" link
Set AllHyperLinks = IE.document.getelementsbytagname("A")
For Each hyper_link In AllHyperLinks
If hyper_link.innertext = "Used Car Prices" Then
hyper_link.Click
Exit For
End If
Next
'wait 4 seconds - so page fully loads
Application.Wait Now + TimeValue("0:00:04")
'Select model year"
Set e = IE.document.getElementbyid("yearDropdown")
' set a year
e.Value = "2016"
End Sub

...try and access an element that doesn't exist yet because the page is still loading...just put in a pause
Fantastic, thank you Robin!

Related

Firing events associated with input box

I am having trouble firing events associated with an input box. These are Event listeners in the "Events" window of "DOM Explorer" when I inspect the relevant text box. They are however not listed in the HTML of the input box itself.
Of the several events listeners listed (e.g. "input", "blur", "paste") they all fire the same javascript function "x". Whilst my code does insert the text, the field does not validate on pressing the submit button and highlights red. A manual entry or a cut and paste will solve this.
I tried .fireEvent ; .dispatchEvent and even attempted to call the javascript directly. It is a public website and the address is in the code. I include below the code to fill in the first 3 fields.
Below is the HTML of the field:
<input class="office-form-question-textbox office-form-textfield-input form-control office-form-theme-focus-border border-no-radius" aria-labelledby="QuestionId_r57b9be42eed24e63b4d2af31c178f530 QuestionInfo_r57b9be42eed24e63b4d2af31c178f530" spellcheck="true" maxlength="4000" placeholder="Enter your answer">
The screenshot of the DOM explorer window:
Private Sub Test()
myAddress = "https://forms.office.com/Pages/ResponsePage.aspx?id=q4GdsF1ZzU2rJGpWVuGNiROj17Ac1mtIt6V80jIX_PFUMEs1VTZaQ0VRV1NNNVI4Vzg3V0RMOE83TS4u"
'create new instance of IE.
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.navigate myAddress
'wait for loading
Do While IE.Busy = True Or IE.readyState <> 4: DoEvents: Loop
WasteTime (0.1) 'Custom function to pause for 0.1 sec
Dim myData(1 To 3, 1 To 3) As String
myData(1, 1) = "Member Surname"
myData(2, 1) = "QuestionId_r57b9be42eed24e63b4d2af31c178f530"
myData(3, 1) = "Chowdhary"
myData(1, 2) = "Member Forename"
myData(2, 2) = "QuestionId_rb88bf6d86f284dbaa14e5f3f27ee9f5b"
myData(3, 2) = "Saqib"
myData(1, 3) = "Member DOB"
myData(2, 3) = "QuestionId_r38d6e5656a6b4fbf86f586d10cbd8cb6"
myData(3, 3) = "19/04/1971"
Set objItem = IE.document.getElementsByTagName("input")
x = 0
Do While x < objItem.Length
Set ele = objItem(x)
mySearch = ele.outerHTML
For c = 1 To UBound(myData, 2)
If InStr(mySearch, myData(2, c)) > 0 Then
ele.Focus
ele.Value = myData(3, c)
Set event_onChange = IE.document.createEvent("HTMLEvents")
event_onChange.initEvent "paste", True, False
ele.dispatchEvent event_onChange 'This does not fire event
'ele.FireEvent ("onpaste") '- This did not work either
'IE.document.parentWindow.execScript code:="x(c)" ' Tried to directly call the javascript function
Exit For
End If
Next c
x = x + 1
Loop
End sub
The event in the search box can be fired with SendKeys. You can simulate user input using SendKeys to set value of the input boxes then click Submit to submit the form.
You can refer to the code snippet below. I have tested and it works:
Set objItem = IE.document.getElementsByTagName("input")(0)
objItem.Focus
SendKeys ("Chowdhary")
Application.Wait (Now + TimeValue("00:00:02"))
IE.document.getElementsByTagName("button")(4).Click
So I did finally get the code to work based on the same .initEvent and .dispatchEvent methods. I just needed to introduce a short 0.1 sec pause (using a custom "Wastetime" function) to allow the element to fully load. For interest the changed relevent code snippet is:
Set objItem = IE.Document.getElementsByTagName("input")
x = 0
Do While x < objItem.Length
Set ele = objItem(x)
mySearch = ele.outerHTML ' Could just enumerate the input indexes in order
For c = 1 To UBound(myInternalArray, 2)
If InStr(mySearch, myInternalArray(2, c)) > 0 Then
If myInternalArray(3, c) = "click" Then
ele.Click
WasteTime (0.1) 'Time for form to expand
Exit For
Else
ele.Value = myInternalArray(3, c)
ele.Focus
Set event_onInput = IE.Document.createEvent("HTMLEvents")
event_onInput.initEvent "input", True, False
ele.dispatchEvent event_onInput
Exit For
End If
End If
Next c
x = x + 1
Loop

Excel VBA loop to input data in webpage

I have gone through many articles and able to input data in to webpage from excel. Now I am trying to use a loop because I have to entry data for 10 to 50 times in the same page.
Here is the code
Sub vfcp()
Set objIE = CreateObject("InternetExplorer.Application")
objIE.Top = 100
objIE.Left = 100
objIE.Width = 800
objIE.Height = 800
objIE.AddressBar = 1
objIE.StatusBar = 1
objIE.Toolbar = 1
objIE.Visible = True
Dim Input
Range("A2").Select
objIE.Navigate ("https://www.askebsa.dol.gov/VFCPCalculator/WebCalculator.aspx")
Do Until Selection.Offset(0, 0).Value = ""
If objIE.ReadyState = 4 Then
Input = Selection.Offset(0, 2).Value
objIE.document.GetElementByID("_ctl0_MainContent_txtPrincipal").Value = Input
objIE.document.GetElementByID("_ctl0_MainContent_cmdCalculate").Click
Selection.Offset(1, 0).Select
End If
Loop
End Sub
The problems in this code is, I have a list of 10 date. the loop inputs the last date of the list. I am trying to input the first date and click submit, then wait for the page to load and input the second date, and stop the loop if there is an empty cell.
Where is the code to enter the dates? You cannot click without them.
"then wait for the page to load and input the second date"
you can achieve this by inserting the following between your End If and Loop lines.
'Allows IE to load
While objIE.ReadyState <> 4
DoEvents
Wend
This shouldn't be too hard. I went to that URL, hit F12 to see the code behind the site, and started searching for the appropriate ID names ('Principal:', 'Loss Date:', etc). That's about it. Run the script to see a working version. Oh, yeah, set a reference to 'Microsoft Internet Controls' before you run the code.
Sub TryThis()
Dim ie As SHDocVw.InternetExplorer
Set ie = New SHDocVw.InternetExplorer
ie.Visible = True
ie.Navigate "https://www.askebsa.dol.gov/VFCPCalculator/WebCalculator.aspx"
Do
DoEvents
Loop Until ie.readystate = 4
'ie READYSTATE has 5 different status codes, here we are using status 4:
Call ie.Document.GetElementByID("_ctl0_MainContent_txtPrincipal").SetAttribute("value", "100000")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtLossDateMonth").SetAttribute("value", "01")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtLossDateDay").SetAttribute("value", "01")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtLossDateYear").SetAttribute("value", "2016")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtRecoveryDateMonth").SetAttribute("value", "01")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtRecoveryDateDay").SetAttribute("value", "01")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtRecoveryDateYear").SetAttribute("value", "2017")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtFinalPaymentMonth").SetAttribute("value", "12")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtFinalPaymentDay").SetAttribute("value", "30")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtFinalPaymentYear").SetAttribute("value", "2017")
'to find the "<input……/>" tag hyper link and to click the button
Set AllInputs = ie.Document.getelementsbytagname("input") 'you can use any tagname as you wish
For Each hyper_link In AllInputs
If hyper_link.Name = "_ctl0:MainContent:cmdCalculate" Then 'you can use .name, .id etc
hyper_link.Click
Exit For
End If
Next
Do
DoEvents
Loop Until ie.readystate = 3
Do
DoEvents
Loop Until ie.readystate = 4
End Sub
If you want to scrape the results, refresh the URL and do the same as I described above: F12, search for the relevant IDs and/or names, and pull those into a Worksheet in Excel. I'll leave it to you to figure out that part. It will be a good learning experience!!!

Various errors with VBA Loop to pull HTML data

I've been getting various errors with the below VBA code (most recent error is Run-time error '70': permission denied). Basically the code/worksheet connects to an intranet IE database of customers, searches customer activity and imports any activity to the worksheet (will eventually use the activity for reporting). Here's where I run into the errors, depending on the length of time I'm searching I sometimes have multiple pages of activity to pull which requires clicking the "next" button and pull the data from each page until there is no longer a "next" button (no more activity). The loop I have set up will pull from the first page, click the "next" button then sometimes pull from the second sheet but then it trips the error. So I think the error has something to do with the loading of the pages but I've added pauses to allow for loading but still run into the same errors. I'm really stuck on this and unfortunately I can't move forward with the project until I can solve this issue.
Here is the code snippet:
Dim TDelements As IHTMLElementCollection
Dim TDelement As HTMLTableCell
Dim r As Long, i As Long
Dim e As Object
Set TDelements = IE.document.getElementsByTagName("tr")
r = 0
For i = 1 To 1
Application.Wait Now + TimeValue("00:00:03")
For Each TDelement In TDelements
If TDelement.className = "searchActivityResultsContent" Then
Sheet1.Range("E1").Offset(r, 0).Value = TDelement.ChildNodes(8).innerText
r = r + 1
ElseIf TDelement.className = "searchActivityResultsContent" Then
Sheet1.Range("E1").Offset(r, 0).Value = TDelement.ChildNodes(8).innerText
r = r + 1
End If
Next
Application.Wait Now + TimeValue("00:00:02")
Set elems = IE.document.getElementsByTagName("input")
For Each e In elems
If e.Value = "Next Results" Then
e.Click
i = 0
Exit For
End If
Next e
Next i
Do Until Not IE.Busy And IE.readyState = 4
DoEvents
Loop
IE.Quit
End Sub
Any help/suggestions would be very much appreciated. Thank you!
Looking at your code:
'getting any TD elements here
Set TDelements = IE.document.getElementsByTagName("tr")
'waiting here....
Application.Wait Now + TimeValue("00:00:03")
'now trying to use items in TDelements...
For Each TDelement In TDelements
'...
Next
Are you waiting for the page to load when you use Application.Wait ?
If Yes then you should know that TDelements isn't dynamic - it won't update itself as new TD elements are loaded: it's just a snapshot of the elements which were present when you called getElementsByTagName("tr"). So call that after the wait.

Select an item in an IE combobox

I've got the correct code to navigate to a website where I want to start populating the dropdown lists and fields etc. The code I've been trying, following #Bond 's suggestion, is:
Sub GetQuote()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.navigate ("website")
IE.Visible = True
Do
DoEvents
Loop Until IE.readystate = 4
Dim e
Set e = IE.document.getElementsByClassname("id of button")(1)
e.Click
Application.Wait (Now + TimeValue("00:00:02"))
Do
DoEvents
Loop Until IE.readystate = 4
Dim z As Object
Set z = IE.document.getElementbyid("vehicleYearOfManufactureList")
z.SelectedIndex = 4
End Sub
This used to populate the first dropdown with the value 2012 but now I just get a run-time error where object is required. Could it be because previously the link that is clicked on used to open in the same and current window, and now it's somehow opening an entirely new window? I'm just baffled by how the same code works and then doesn't work, assuming no changes have been made to the source code it references.
The line that is highlighted in the debug of this error is: Set z = IE.document.getElementbyid("vehicleYearOfManufactureList"). As per the comment I made below, it's a "run-time 424: object" required error
You can do this a few ways. If you know the index of the item, you can set it directly using the <select> element's SelectedIndex property. In your case, "2012" is the 4th item in the dropdown:
Set e = IE.document.getElementbyid("vehicleYearOfManufactureList")
e.SelectedIndex = 4
Alternatively, you can iterate all of the <option> elements in the dropdown to find the value you're looking for, then set its Selected property to True:
Set e = IE.document.getElementbyid("vehicleYearOfManufactureList")
Dim o
For Each o In e.Options
If o.Value = "2012" Then
o.Selected = True
Exit For
End If
Next

UPS Automated tracking clicking a button

I want to visit webpage. Then enter a number in test box. I was able to do that.
after that I want to click on the Track Button. Any idea how can I click through vba?
I already tried below commands. Any idea how can i find id of the Track button?
ie.document.getElementByName("track.x").Click
ie.document.getElementByClass("button btnGroup6 arrowIconRight").Click
Lets say we enter the tracking number "1ZW2E2360449018801" and once I click that button, a new webpage opens. I want to click on "shipment Progress" bar and copy the table that appears. any suggestions?
This will get it done..
Sub test()
' open IE, navigate to the website of interest and loop until fully loaded
Set ie = CreateObject("InternetExplorer.Application")
my_url = "http://wwwapps.ups.com/WebTracking/track?loc=en_US"
With ie
.Visible = True
.navigate my_url
.Top = 50
.Left = 530
.Height = 400
.Width = 400
Do Until Not ie.Busy And ie.readyState = 4
DoEvents
Loop
End With
' Enter a value in the "Number" text box
ie.Document.getElementById("trackNums").Value = "1ZW2E2360449018801"
' Click the "Submit" button
Set Results = ie.Document.getElementsByTagName("input")
For Each itm In Results
If InStr(1, itm.outerhtml, "Track", vbTextCompare) > 0 Then
itm.Click
exit for
End If
Next
' Click the "Shipment Progress" button
Set Results = ie.Document.getElementsByTagName("h4")
For Each itm In Results
If InStr(1, itm.outerhtml, "Shipment Progress", vbTextCompare) > 0 Then
itm.Click
exit for
End If
Next
' Get the text from the "Shipment Progress Table" and assign to a variable
tbltxt = ie.Document.getElementsByTagName("table")(4).innertext
' process the text as desired
End Sub
Something like this should work:
ie.document.getElementsByName("track.x")(0).Click
Note it's getElement*s*ByName, so it returns a collection, not a single element.