I have gone through many articles and able to input data in to webpage from excel. Now I am trying to use a loop because I have to entry data for 10 to 50 times in the same page.
Here is the code
Sub vfcp()
Set objIE = CreateObject("InternetExplorer.Application")
objIE.Top = 100
objIE.Left = 100
objIE.Width = 800
objIE.Height = 800
objIE.AddressBar = 1
objIE.StatusBar = 1
objIE.Toolbar = 1
objIE.Visible = True
Dim Input
Range("A2").Select
objIE.Navigate ("https://www.askebsa.dol.gov/VFCPCalculator/WebCalculator.aspx")
Do Until Selection.Offset(0, 0).Value = ""
If objIE.ReadyState = 4 Then
Input = Selection.Offset(0, 2).Value
objIE.document.GetElementByID("_ctl0_MainContent_txtPrincipal").Value = Input
objIE.document.GetElementByID("_ctl0_MainContent_cmdCalculate").Click
Selection.Offset(1, 0).Select
End If
Loop
End Sub
The problems in this code is, I have a list of 10 date. the loop inputs the last date of the list. I am trying to input the first date and click submit, then wait for the page to load and input the second date, and stop the loop if there is an empty cell.
Where is the code to enter the dates? You cannot click without them.
"then wait for the page to load and input the second date"
you can achieve this by inserting the following between your End If and Loop lines.
'Allows IE to load
While objIE.ReadyState <> 4
DoEvents
Wend
This shouldn't be too hard. I went to that URL, hit F12 to see the code behind the site, and started searching for the appropriate ID names ('Principal:', 'Loss Date:', etc). That's about it. Run the script to see a working version. Oh, yeah, set a reference to 'Microsoft Internet Controls' before you run the code.
Sub TryThis()
Dim ie As SHDocVw.InternetExplorer
Set ie = New SHDocVw.InternetExplorer
ie.Visible = True
ie.Navigate "https://www.askebsa.dol.gov/VFCPCalculator/WebCalculator.aspx"
Do
DoEvents
Loop Until ie.readystate = 4
'ie READYSTATE has 5 different status codes, here we are using status 4:
Call ie.Document.GetElementByID("_ctl0_MainContent_txtPrincipal").SetAttribute("value", "100000")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtLossDateMonth").SetAttribute("value", "01")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtLossDateDay").SetAttribute("value", "01")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtLossDateYear").SetAttribute("value", "2016")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtRecoveryDateMonth").SetAttribute("value", "01")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtRecoveryDateDay").SetAttribute("value", "01")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtRecoveryDateYear").SetAttribute("value", "2017")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtFinalPaymentMonth").SetAttribute("value", "12")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtFinalPaymentDay").SetAttribute("value", "30")
Call ie.Document.GetElementByID("_ctl0_MainContent_txtFinalPaymentYear").SetAttribute("value", "2017")
'to find the "<input……/>" tag hyper link and to click the button
Set AllInputs = ie.Document.getelementsbytagname("input") 'you can use any tagname as you wish
For Each hyper_link In AllInputs
If hyper_link.Name = "_ctl0:MainContent:cmdCalculate" Then 'you can use .name, .id etc
hyper_link.Click
Exit For
End If
Next
Do
DoEvents
Loop Until ie.readystate = 3
Do
DoEvents
Loop Until ie.readystate = 4
End Sub
If you want to scrape the results, refresh the URL and do the same as I described above: F12, search for the relevant IDs and/or names, and pull those into a Worksheet in Excel. I'll leave it to you to figure out that part. It will be a good learning experience!!!
Related
I am familiar with StackOverflow but have just recently signed up. I am trying to search a Hotel on google and return the address in Excel using VBA. Below is a photo of what Information I am trying to return from Google. From my research, I was able to find a VBA that allowed me to return the Results stats.
Would it be possible to modify my code and return the box at the top of my google search?
I would really appreciate your help! Below is the VBA I am using to return search results.
Sample Image - Red Roof Inn & Address
Sub SearchGoogle()
Dim ie As Object
Dim form As Variant
Dim button As Variant
Dim LR As Integer
Dim var As String
Dim var1 As Object
LR = Cells(Rows.Count, 1).End(xlUp).Row
For x = 2 To LR
var = Cells(x, 1).Value
Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
With ie
.Visible = True
.navigate "http://www.google.co.in"
While Not .readyState = READYSTATE_COMPLETE
Wend
End With
'Wait some to time for loading the page
While ie.Busy
DoEvents
Wend
Application.Wait (Now + TimeValue("0:00:02"))
ie.document.getElementById("lst-ib").Value = var
'Here we are clicking on search Button
Set form = ie.document.getElementsByTagName("form")
Application.Wait (Now + TimeValue("0:00:02"))
Set button = form(0).onsubmit
form(0).submit
'wait for page to load
While ie.Busy
DoEvents
Wend
Application.Wait (Now + TimeValue("0:00:02"))
Set var1 = ie.document.getElementById("resultStats")
Cells(x, 2).Value = var1.innerText
ie.Quit
Set ie = Nothing
Next x
End Sub
Right now your code loads the page and then loads the value of the resultStats element.
So the section of your code that you will need to alter is:
Set var1 = ie.document.getElementById("resultStats")
Cells(x, 2).Value = var1.innerText
The first step to your problem is to understand the DOM of the HTML page you are attempting to use, in this case Google. I would suggest using a browser to navigate the DOM as it would give you a good idea of what the whole page is doing.
If you are aiming to do this on a macro basis you will need a path through the DOM that will always take you where you want to go. I would suggest having two pages with different searches open so that you can check you hypothesis as you go.
For example the boxes that you refer to seem to be located in a class called kp-header from knowing this you can build out your path through the DOM to return the text value displayed on screen. Again you will need to do your own investigations to find the best stating point for your search as kp-header was just the first potently helpful result I could find.
Although please note that depending on the speed you are loading these webpages you may hit a limit from google as they discourage scraping. What would be a better option to avoid these limits and to avoid yourself having to investigate all of google's DOM would be to try and incorporate one of google's API's
I'm trying to scrape some data using VBA via Excel. I'm pretty rusty with VBA but I'm confident in my google-fu, 5+ hours in and no luck yet. I have seen many ways to work with drop-down menus but none seem to work with this one. I have successfully navigated to https://www.kbb.com/used-cars/ but I cannot work with the "year" drop-down. Maybe I am not using the correct id/name? Any advice is appreciated, let me know if I left any important info out.
Code:
Sub WebForumEntry()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Top = 0
IE.Left = 0
IE.Width = 800
IE.Height = 600
IE.AddressBar = 0
IE.StatusBar = 0
IE.Toolbar = 0
IE.Navigate "http://www.kbb.com"
IE.Visible = True
Do
DoEvents
Loop Until IE.ReadyState = 4
'Click on the "Used Car Values" link
Set AllHyperlinks = IE.document.getelementsbytagname("A")
For Each hyper_link In AllHyperlinks
If hyper_link.innertext = "Used Car Prices" Then
hyper_link.Click
Exit For
End If
Next
'Select model year"
Set e = IE.document.getElementbyid("yearDropdown")
Dim o
For Each o In e.Options
If o.Value = "2012" Then
o.Selected = True
Exit For
End If
Next
End Sub
In the HTML there are no Option tags for you to select:
<div class="form-group">
<select id="yearDropdown" name="yearId" class="dropdown-one year-dropdown" data-default-option="Year" data-preselected-value=""></select>
</div>
The years are generated dynamically depending on whether you choose new/ used etc. So you can't select something that doesn't exist - perhaps you can directly set the value e.g.:
IE.document.getElementbyid("yearDropdown").value = "2012"
Edit
Note there is an issue in your original code:
'Click on the "Used Car Values" link
Set AllHyperlinks = IE.document.getelementsbytagname("A")
For Each hyper_link In AllHyperlinks
If hyper_link.innertext = "Used Car Prices" Then
hyper_link.Click
Exit For
End If
Next
'Select model year"
Set e = IE.document.getElementbyid("yearDropdown") '<-- error here if page not loaded per hyper_link.Click
When you call hyper_link.Click it will load another page (where the dropdowns are) and then per the Exit For the next command to run is Set e = IE... which will try and access an element that doesn't exist yet because the page is still loading per the hyper_link.Click. To work-around this you can just put in a pause like below:
Application.Wait Now + TimeValue("0:00:04")
It's a bit of a hack - and there may be better ways to solve it - but it will allow you to set the dropdown per my first suggestion :
Full code (re-write of yours):
Option Explicit
Sub WebForumEntry()
Dim IE As Object
Dim AllHyperLinks As Object
Dim hyper_link As Object
Dim e As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Top = 0
IE.Left = 0
IE.Width = 800
IE.Height = 600
IE.AddressBar = 0
IE.StatusBar = 0
IE.Toolbar = 0
IE.Navigate "http://www.kbb.com"
IE.Visible = True
' wait for page to load
Do
DoEvents
Loop Until IE.ReadyState = 4
'Click on the "Used Car Values" link
Set AllHyperLinks = IE.document.getelementsbytagname("A")
For Each hyper_link In AllHyperLinks
If hyper_link.innertext = "Used Car Prices" Then
hyper_link.Click
Exit For
End If
Next
'wait 4 seconds - so page fully loads
Application.Wait Now + TimeValue("0:00:04")
'Select model year"
Set e = IE.document.getElementbyid("yearDropdown")
' set a year
e.Value = "2016"
End Sub
...try and access an element that doesn't exist yet because the page is still loading...just put in a pause
Fantastic, thank you Robin!
Sub Get_Data()
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
ie.Navigate "http://www.scramble.nl/military-database/usaf"
Do While ie.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
SendKeys "03-3114"
SendKeys "{ENTER}"
End Sub
The code below searches for keyboard typed value 03-3114 and gets a data in the table. If I 'd like to search for value which is already in cell A1 and scrape values from table for "Code, Type, CN, Unit" in cell range ("B1:E1") what should I do?
You are using SendKeys which are highly unreliable :) Why not find the name of the textbox and the search button and directly interact with it as shown below?
Sub Get_Data()
Dim ie As Object, objInputs As Object
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
ie.Navigate "http://www.scramble.nl/military-database/usaf"
Do While ie.readystate <> 4: DoEvents: Loop
'~~> Get the ID of the textbox where you want to output
ie.Document.getElementById("serial").Value = "03-3114"
'~~> Here we try to identify the search button and click it
Set objInputs = ie.Document.getElementsByTagName("input")
For Each ele In objInputs
If ele.Name Like "sbm" Then
ele.Click
Exit For
End If
Next
End Sub
Note: To understand how I got the names serial and sbm, refer to the explanation given just above the image below.
The code below searches for keyboard typed value 03-3114 and gets a data in the table. If I 'd like to search for value which is already in cell A1 and scrape values from table for "Code, Type, CN, Unit" in cell range ("B1:E1") what should I do?
Directly put the value from A1 in lieu of the hardcoded value
ie.Document.getElementById("serial").Value = Sheets("Sheet1").Range("A1").Value
To get the values from the table, identify the elements of the table by right clicking on it in the browser and clicking on "Inspect/Inspect Element(In Chrome it is just Inspect)" as shown below.
I can give you the code but I want you to do it yourself. If you are still stuck then update the question with the code that you tried and then we will take it from there.
Interesting read: html parsing of cricinfo scorecards
I've been getting various errors with the below VBA code (most recent error is Run-time error '70': permission denied). Basically the code/worksheet connects to an intranet IE database of customers, searches customer activity and imports any activity to the worksheet (will eventually use the activity for reporting). Here's where I run into the errors, depending on the length of time I'm searching I sometimes have multiple pages of activity to pull which requires clicking the "next" button and pull the data from each page until there is no longer a "next" button (no more activity). The loop I have set up will pull from the first page, click the "next" button then sometimes pull from the second sheet but then it trips the error. So I think the error has something to do with the loading of the pages but I've added pauses to allow for loading but still run into the same errors. I'm really stuck on this and unfortunately I can't move forward with the project until I can solve this issue.
Here is the code snippet:
Dim TDelements As IHTMLElementCollection
Dim TDelement As HTMLTableCell
Dim r As Long, i As Long
Dim e As Object
Set TDelements = IE.document.getElementsByTagName("tr")
r = 0
For i = 1 To 1
Application.Wait Now + TimeValue("00:00:03")
For Each TDelement In TDelements
If TDelement.className = "searchActivityResultsContent" Then
Sheet1.Range("E1").Offset(r, 0).Value = TDelement.ChildNodes(8).innerText
r = r + 1
ElseIf TDelement.className = "searchActivityResultsContent" Then
Sheet1.Range("E1").Offset(r, 0).Value = TDelement.ChildNodes(8).innerText
r = r + 1
End If
Next
Application.Wait Now + TimeValue("00:00:02")
Set elems = IE.document.getElementsByTagName("input")
For Each e In elems
If e.Value = "Next Results" Then
e.Click
i = 0
Exit For
End If
Next e
Next i
Do Until Not IE.Busy And IE.readyState = 4
DoEvents
Loop
IE.Quit
End Sub
Any help/suggestions would be very much appreciated. Thank you!
Looking at your code:
'getting any TD elements here
Set TDelements = IE.document.getElementsByTagName("tr")
'waiting here....
Application.Wait Now + TimeValue("00:00:03")
'now trying to use items in TDelements...
For Each TDelement In TDelements
'...
Next
Are you waiting for the page to load when you use Application.Wait ?
If Yes then you should know that TDelements isn't dynamic - it won't update itself as new TD elements are loaded: it's just a snapshot of the elements which were present when you called getElementsByTagName("tr"). So call that after the wait.
I want to visit webpage. Then enter a number in test box. I was able to do that.
after that I want to click on the Track Button. Any idea how can I click through vba?
I already tried below commands. Any idea how can i find id of the Track button?
ie.document.getElementByName("track.x").Click
ie.document.getElementByClass("button btnGroup6 arrowIconRight").Click
Lets say we enter the tracking number "1ZW2E2360449018801" and once I click that button, a new webpage opens. I want to click on "shipment Progress" bar and copy the table that appears. any suggestions?
This will get it done..
Sub test()
' open IE, navigate to the website of interest and loop until fully loaded
Set ie = CreateObject("InternetExplorer.Application")
my_url = "http://wwwapps.ups.com/WebTracking/track?loc=en_US"
With ie
.Visible = True
.navigate my_url
.Top = 50
.Left = 530
.Height = 400
.Width = 400
Do Until Not ie.Busy And ie.readyState = 4
DoEvents
Loop
End With
' Enter a value in the "Number" text box
ie.Document.getElementById("trackNums").Value = "1ZW2E2360449018801"
' Click the "Submit" button
Set Results = ie.Document.getElementsByTagName("input")
For Each itm In Results
If InStr(1, itm.outerhtml, "Track", vbTextCompare) > 0 Then
itm.Click
exit for
End If
Next
' Click the "Shipment Progress" button
Set Results = ie.Document.getElementsByTagName("h4")
For Each itm In Results
If InStr(1, itm.outerhtml, "Shipment Progress", vbTextCompare) > 0 Then
itm.Click
exit for
End If
Next
' Get the text from the "Shipment Progress Table" and assign to a variable
tbltxt = ie.Document.getElementsByTagName("table")(4).innertext
' process the text as desired
End Sub
Something like this should work:
ie.document.getElementsByName("track.x")(0).Click
Note it's getElement*s*ByName, so it returns a collection, not a single element.