Unable to get data as tabular format - vba

I've written a script in vba using IE to get the data from a webpage. The data are not stored within any table, I meant there is no table,tr or td tag. However, they look like to be in a tabular format. You can see the below image for clarity.
What I've tried so far can get the data in a single line like:
$4,085
$1,620
$1,435
$35
$1,125
$905
How I wish to get them is like:
$4,085 $1,620
$1,435 $35
$1,125 $905
In other languages there is an option for list comprehension using which I can handle it in a single line of code but in case of vba I get stuck.
html elements within which the data are (It's just a chunk of the whole):
<ul id="tco_detail_data">
<li>
<ul class="list-title">
<li class="first"> </li>
<li>Year 1</li>
<li>Year 2</li>
<li>Year 3</li>
<li>Year 4</li>
<li>Year 5</li>
<li class="last">5 Yr Total</li>
</ul>
</li>
<hr class="loose-dotted">
<li class="first">
<ul class="first">
<li class="first">Depreciation</li>
<li>$4,085</li>
<li>$1,620</li>
<li>$1,425</li>
<li>$1,263</li>
<li>$1,133</li>
<li class="last">$9,526</li>
</ul>
</li>
</ul>
The data look like in that page:
This is what I've attempted so far:
Sub Get_Information()
Dim IE As New InternetExplorer, HTML As HTMLDocument
Dim post As Object
With IE
.Visible = False
.Navigate "https://www.edmunds.com/ford/escape/2017/cost-to-own/?zip=43215"
While .Busy = True Or .ReadyState < 4: DoEvents: Wend
Set HTML = .Document
End With
Application.Wait Now + TimeValue("00:00:05") 'waiting for the items to be available
For Each post In HTML.getElementById("tco_detail_data").getElementsByTagName("li")
Debug.Print post.innerText
Next post
IE.Quit
End Sub
Reference to add to the library to execute the above script:
Microsoft Internet Controls
Microsoft HTML Object Library

This works using CSS selector. Updated to remove explicit wait.
The selector is:
#tco_detail_data > li
Which is the li within id of tco_detail_data
Which looks like the following sample results from webpage using CSS query
Code:
Option Explicit
Public Sub Get_Information()
Dim IE As New InternetExplorer
With IE
.Visible = False
.navigate "https://www.edmunds.com/ford/escape/2017/cost-to-own/?zip=43215"
While .Busy = True Or .readyState < 4: DoEvents: Wend
End With
Dim a As Object, exitTime As Date
exitTime = Now + TimeSerial(0, 0, 5)
Do
DoEvents
On Error Resume Next
Set a = IE.document.querySelectorAll("#tco_detail_data")
On Error GoTo 0
If Now > exitTime Then Exit Do
Loop While a Is Nothing
If a Is Nothing Then Exit Sub
Dim resultsNodeList As Object, i As Long, arr() As String
Set resultsNodeList = HTML.querySelectorAll("#tco_detail_data > li")
With ActiveSheet
For i = 0 To 9
arr = Split(resultsNodeList(i).innerText, Chr$(10))
.Cells(i + 1, 1).Resize(1, UBound(arr) + 1).Value = arr
Next
End With
IE.Quit
End Sub
Result in sheet
Additional info:
The array part is because resultsNodeList(i).innerText returns as a "stacked string" - i.e. with line breaks in between; See image below. I split on those, to produce an array, which I then write out to the sheet. The array is 0 based, so I have to add 1 to be able to populate the range properly.

Apart from what QHarr has already shown, there is another way the same goal can be achieved:
Sub Get_Information()
Dim IE As New InternetExplorer, HTML As HTMLDocument
Dim posts As Object, post As Object, oitem As Object
Dim R&, C&, B As Boolean
With IE
.Visible = False
.Navigate "https://www.edmunds.com/ford/escape/2017/cost-to-own/?zip=43215"
Do While .Busy = True Or .ReadyState <> 4: DoEvents: Loop
Set HTML = .Document
End With
''no hardcoded delay is required. The following line should take care of that
Do: Set oitem = HTML.getElementById("tco_detail_data"): DoEvents: Loop While oitem Is Nothing
For Each posts In oitem.getElementsByTagName("li")
C = 1: B = False
For Each post In posts.getElementsByTagName("li")
Cells(R + 1, C).Value = post.innerText
C = C + 1: B = True
Next post
If B Then R = R + 1
Next posts
IE.Quit
End Sub

Related

Clicking an Angular (ng-click) search button in IE using excel VBA

I am new to VBA. I am trying to click on the "GO" search button after I logged in to a secured website.
I tried many ways to click on the button, but I got no luck.
Here is the inspect element.
<button class="button primary pull-right" ng-click="search()" ng-disabled="searchForm.$invalid">
Go <i class="glyphicon glyphicon-chevron-right"></i>
</button>
Here is what I have tried.
Method 1
Set HTMLDoc = HTMLDoc.class("button primary pull-right").document
Set button = HTMLDoc.getElementsByTagName("BUTTON")(0) 'first button
button.Click
For i = 1 To 5
button.Click
DoEvents
Next
Method 2
For Each hyper_link In allhyperlinks
If hyper_link.class = "button primary pull-right" Then
hyper_link.Click
Exit For
End If
Next hyper_link
Method 3
Set allhyperlinks = IE.document.getElementsByTagName("button")
For Each hyper_link In allhyperlinks
If hyper_link.getAttribute("class") = "search" Then
hyper_link.Click
Exit For
End If
Next
Method 4
Dim oHTML_Element As IHTMLElement
Dim oBrowser As InternetExplorer
Dim ie As Variant
For Each oHTML_Element In ie.document.getElementsByName("button")
If oHTML_Element.className = "button primary pull-right" Then
oHTML_Element.Click
End If
Next
Method 5
Set ie = CreateObject("InternetExplorer.Application")
With ie
Do Until .readyState = 4
DoEvents
Loop
Application.Wait (Now + TimeValue("0:00:15"))
ie.document.getElementsByClassName("button primary pull-right").Click
Application.Wait (Now + TimeValue("0:00:1"))
End With
Method 6
Set ie = CreateObject("InternetExplorer.Application")
ie.document.getElementByClassName("button primary pull-right")(0).Click
Any help would be greatly appreciated.
Thank you!
I presume you have not given url because it is beyond a login. That makes solving ten times harder. Nevertheless some points.
ng-Click means its running Angular.
Sometimes it is required to make the control take the focus before simulating the Click event. Website does not recognize my inputs [how to fire IE dom event manually from VBA]
I'd use .querySelector() or .querySelectorAll() to acquire the button because I can use the path at the bottom of the Chrome Development Tools Window. VBA - Webscraping - jQuery selectors available with MSHTML's querySelector and querySelectorAll
Try this code
Sub Test()
Dim ie As Object
Dim e As Object
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
.navigate "www.jetblue.com"
Do Until .readyState = 4: DoEvents: Loop
For Each e In .document.getElementsByTagName("input")
If e.ID = "email_field" Then
e.Value = "your email"
ElseIf e.ID = "password_field" Then
e.Value = "your password"
End If
Next e
For Each e In .document.getElementsByTagName("input")
If e.ID = "signin_btn" And e.Type = "submit" Then e.Click: Exit For
Next e
End With
End Sub

Using VBA to click a button

I am trying to pull information from a website, but cannot for the life of me figure out how to click the button. I've got the code to input the information. This is the html for the website . Any help would be appreciated.
<p role="button" tabindex="0" class="fmtbutton" onkeypress="finddistancebetweenaandb(document.forms['inp']['pointa'].value,document.forms['inp']['pointb'].value);" onclick="finddistancebetweenaandb(document.forms['inp']['pointa'].value,document.forms['inp']['pointb'].value);"> Show </p>
Here is the full code that I have so far.
Sub RoundedRectangle1_Click()
Dim eRow As Long
Dim ele As Object
Dim objIE As Object
Set objIE = CreateObject("InternetExplorer.Application")
FranchiseAddress = Range("B2").Value
Movefrom = Range("B3").Value
moveto = Range("B4").Value
With objIE
.Visible = True
.navigate "https://www.freemaptools.com/how-far-is-it-between.htm"
Do While .Busy Or _
.readyState <> 4
DoEvents
Loop
Set d = .Document.getElementsByName("onoffswitch")
d.Item(0).Click
Set f = .Document.getElementsByName("pointa")
f.Item(0).Value = FranchiseAddress
Set a = .Document.getElementsByName("pointb")
a.Item(0).Value = Movefrom
' This is where I'm Stuck'
End With
End Sub
I have tried multiple solutions found on this site for many questions, but none of them seem to work.
Thank you.
add this to your code
Dim aaa As Object
Set aaa = .Document.getElementsByClassName("fmtbutton") ' this returns 4 objects, your button is the first one
aaa(0).Click
You can be a little more elegant and target the attribute with a CSS selector:
.document.querySelector("[role=button]").Click
The attribute = value selector, [role=button], looks for an element with attribute role whose value is button. There is only one on the page so no loop needed.

VBA IE change dropdown value

Tried the below code for following URL Scripture look up. Please how to change drop down value from WEB to RV1909?
Dim Doc As HTMLDocument
Set Doc = IEApp.document
TestV2 = ""
TestV3 = ""
TestV2 = Doc.getElementsByClassName("app-list text-list")(0).innerText
Debug.Print "4b of 5: " & TestV2
IEApp.Doc.getElementsByClassName("app-list text-list").selectedIndex = 1
IEApp.Doc.getElementsByClassName("app-list text-list").FireEvent ("onchange")
TestV3 = Doc.getElementsByClassName("app-list text-list")(0).innerText
Debug.Print "4c of 5: " & TestV3
Tried many approaches from other posts, the following does not work:
IEApp.Doc.getElementsByClassName("app-list text-list")(0).innerHTML = "RV1909"
Here is the screenshot of Chrome Inspector:
In this case it's not enough just change div.app-list.text-list element innerText, as you can see that element is simple div, but not even ul. It should be changed by scripts which are called on click events. So, first of all you need to click on div to display the entire list, and then click on the RV1909 item. The below example shows how that could be done:
Sub Test()
Dim oIE As Object
Dim oDiv As Object
Set oIE = CreateObject("InternetExplorer.Application")
With oIE
.Visible = True
.Navigate "http://ebible.org/study/"
Do While .readyState <> 4 Or .Busy
DoEvents
Loop
With .Document
Do While .readyState <> "complete"
DoEvents
Loop
Set oDiv = .getElementsByClassName("app-list text-list")(0)
oDiv.Click
Do While .readyState <> "complete"
DoEvents
Loop
For Each oNode In .getElementsByClassName("text-chooser-main")(0).getElementsByClassName("text-chooser-abbr")
If oNode.innerText = "RV1909" Then
oNode.Click
Do While oDiv.innerText <> "RV1909"
DoEvents
Loop
Exit For
End If
Next
End With
End With
End Sub
Change the html div element <div class=app-list text-list>WEB</div> to <div class=app-list text-list>RV1909</div>

How to sift through the tag name for image from the pasted elements

So far, while writing code in VBE to make a parser I have used the "img" tag and the "src" attribute to scrape an image but I stumbled trying to go through the portion I'm pasting below. Can't filter the portion I need to use in my code to parse an Image.
Set topics = html.getElementsByClassName("card card-lg")
For i = 0 To topics.Length - 1
Set topic = topics(i)
Cells(x, 1).Value = topic.getElementsByClassName("wine-card__image-wrapper")(0).getElementsByTagName("img")(0).src
x = x + 1
Next i
And a sample of the HTML I'm working with:
<div class="wine-card__image-wrapper">
<a href="/wineries/tschida/wines/angerhof-eiswein-gruner-veltliner-2012">
<figure class="wine-card__image" style="background-image: url(//images.vivino.com/thumbs/qlER3oggQVKh1FZn7YGxZg_375x500.jpg)">
<div class="image-inner"></div>
</figure>
</a>
</div>
You can access the "style" attribute I believe, so
Sub t()
Dim ie As SHDocVw.InternetExplorer
Dim d As MSHTML.HTMLDocument
Dim dv As MSHTML.HTMLDivElement
Dim ha As MSHTML.IHTMLElement
Set ie = New SHDocVw.InternetExplorer
ie.Visible = True
ie.navigate "https://www.vivino.com/explore?e=eJzLLbI11jNVy83MswWSiRW2RgZqyZW26Ulq5SXRsbaGAKA_Cdk%3D"
While ie.Busy Or ie.readyState <> READYSTATE_COMPLETE: DoEvents: Wend
Set d = ie.document
Set e = d.getElementsByClassName("wine-card__image-wrapper")(0)
Set ha = e.Children(0).Children(0)
Debug.Print ha.Style.backgroundImage
End Sub

VBA code not selecting a value from a dropdown that has a class but no id

I have some code which is meant to open an IE page and fill in the first drop-down to show "caravan". However my code is not selecting caravan from the drop-down.
The website is:
https://insurance.qbe.com.au/portal/caravan/form/estimate
When I right-click to inspect the source code of the drop-down I get directed to the following:
When I expand the source code below this, I see the following:
As you can see, the values associated with the drop-down are well below the source code I was initially directed to.
Here is the code I've been using to try and select 'caravan':
Sub GetQuote()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.navigate ("https://insurance.qbe.com.au/portal/caravan/form/estimate")
IE.Visible = True
Do
DoEvents
Loop Until IE.readystate = 4
Dim lists, l
Set lists = IE.document.getElementsByTagName("select")(0)
For Each l In lists
If InStr(1, l.className, "form-control ng-pristine ng-untouched ng-valid", 1) Then
l.SelectedIndex = 0
End If
Next
Application.Wait (Now + TimeValue("00:00:03"))
End Sub
With the above code I was trying to select 'caravan' by referencing the very last bit of the source code I pasted above (i.e. by getting all select tags and then checking all items within the tag until an item whose class name contains "form-control ng-pristine ng-untouched ng-valid" is found, and then selecting index 0 for the value associated with this item. And this obviously didn't work - so here's other code I have tried, again with no luck (it's a slight modification of the code above, by simply referencing the source code that I was initially directed to when I inspected the drop-down, and that would be the very first source code I pasted):
For Each l In lists
If InStr(1, l.className, "ui-select-search ui-select-toggle ng-pristine ng-valid ng-touched", 1) Then
l.SelectedIndex = 0
Does this offer any help, not sure if it's your solution, but added here for formatting.
Sub test()
Dim ie As SHDocVw.InternetExplorer
Dim htmlDoc As MSHTML.HTMLDocument
Dim inputCollection As MSHTML.IHTMLElementCollection
Dim inputElement As MSHTML.HTMLInputElement
Set ie = New SHDocVw.InternetExplorer
ie.navigate ("https://insurance.qbe.com.au/portal/caravan/form/estimate")
Do While ie.Busy Or ie.readyState <> READYSTATE_COMPLETE
DoEvents
Loop
ie.Visible = 1
Set htmlDoc = ie.document
Set inputCollection = htmlDoc.getElementsByTagName("INput")
For Each inputElement In inputCollection
Debug.Print inputElement.className
Next inputElement
' or like this
Set inputElement = htmlDoc.getElementsByClassName("ui-select-search ui-select-toggle ng-pristine ng-valid ng-touched")(0)
End Sub