VBA Webscrape not picking up elmenents; pick up frames/tables? - vba

Tried asking this question. Didn't get many answers. Can't install things onto my work computer. https://stackoverflow.com/questions/29805065/vba-webscrape-not-picking-up-elements
Want to scrape a morningstar page into Excel with the code below. Problem is, it doesn't feed any real elements/data back. I actually just want the Dividend and cap gain distribution table really from that link I put into my_Page.
This is usually easiest way, but an entire page scrape way, AND Excel-->Data-->From Web DON'T work.
I've tried to use get elements by tag name and class before, but I failed at being able to do it in this case.This might be the way to go... Once again, just want that Dividend and Cap Gain distribution table. Not seeing any results in via the Debug.print
Working code below, just need to parse into excel. Updated attempt below:
Sub Macro1()
Dim IE As New InternetExplorer
IE.Visible = True
IE.navigate "http://quotes.morningstar.com/fund/fundquote/f?&t=ANNPX&culture=en_us&platform=RET&viewId1=2046632524&viewId2=3141452350&viewId3=3475652630"
Do
DoEvents
Loop Until IE.readyState = READYSTATE_COMPLETE
Dim doc As HTMLDocument
Set doc = IE.document
'For Each Table In doc.getElementsByClassName("gr_table_b1")
'For Each td In Table.getElementsByTagName("tr")
On Error Resume Next
For Each td In doc.getElementsByClassName("gr_table_row4")
Debug.Print td.Cells(5).innerText
'Debug.Print td.Cells(1).innerText
Next td
'Next Table
'IE.Quit
'Application.EnableEvents = True
End Sub

The content in question is contained within an iframe. You can see this by right clicking on that section of the sebsite, and selecting Inspect element. Looking up the tree, you'll see an iframe tag, containing the url of data. You should try to find that element, and extract that url (which is generated with js) and then open that page.

No frame to worry about. You only need the table id.
Webpage view:
Print out from code:
VBA:
Option Explicit
Public Sub GetDivAndCapTable()
Dim ie As New InternetExplorer, hTable As HTMLTable
Const URL = "http://quotes.morningstar.com/fund/fundquote/f?&t=ANNPX&culture=en_us&platform=RET&viewId1=2046632524&viewId2=3141452350&viewId3=3475652630"
Application.ScreenUpdating = False
With ie
.Visible = True
.navigate URL
While .Busy Or .READYSTATE < 4: DoEvents: Wend
Set hTable = .document.getElementById("DividendAndCaptical")
WriteTable hTable, 1
Application.ScreenUpdating = True
.Quit
End With
End Sub
Public Sub WriteTable(ByVal hTable As HTMLTable, Optional ByVal startRow As Long = 1, Optional ByVal ws As Worksheet)
If ws Is Nothing Then Set ws = ActiveSheet
Dim tSection As Object, tRow As Object, tCell As Object, tr As Object, td As Object, R As Long, C As Long, tBody As Object
R = startRow
With ws
Dim headers As Object, header As Object, columnCounter As Long
Set headers = hTable.getElementsByTagName("th")
For Each header In headers
columnCounter = columnCounter + 1
.Cells(startRow, columnCounter) = header.innerText
Next header
startRow = startRow + 1
Set tBody = hTable.getElementsByTagName("tbody")
For Each tSection In tBody 'HTMLTableSection
Set tRow = tSection.getElementsByTagName("tr") 'HTMLTableRow
For Each tr In tRow
Set tCell = tr.getElementsByTagName("td")
C = 1
For Each td In tCell 'DispHTMLElementCollection
.Cells(R, C).Value = td.innerText 'HTMLTableCell
C = C + 1
Next td
R = R + 1
Next tr
Next tSection
End With
End Sub

Related

I'm having trouble scraping this

I'm trying to understand why my references arent working well to scrape this data.
Here is the site as an example:
http://quote.morningstar.ca/Quicktakes/Financials/is.aspx?t=GNTX&region=USA&culture=en-CA&ops=clear
And as a target:
<div id="data_i6" class="rf_crow"><div id="Y_1" class="pos column6Width_noChart116px" style="overflow:hidden;white-space: nowrap;" rawvalue="741131269">741</div><div id="Y_2" class="pos column6Width_noChart116px" style="overflow:hidden;white-space: nowrap;" rawvalue="836611464">837</div><div id="Y_3" class="pos column6Width_noChart116px" style="overflow:hidden;white-space: nowrap;" rawvalue="939841654">940</div><div id="Y_4" class="pos column6Width_noChart116px" style="overflow:hidden;white-space: nowrap;" rawvalue="1010472512">1,010</div><div id="Y_5" class="pos column6Width_noChart116px" style="overflow:hidden;white-space: nowrap;" rawvalue="1100344312">1,100</div><div id="Y_6" class="pos column6Width_noChart116px" style="overflow:hidden;white-space: nowrap;" rawvalue="1115401551">1,115</div></div>
What I need to extract is the actual value in rawvalue="741131269" and the following is what I've gotten to work so far.
'Cells(1, 1) = Document.getElementsByClassName("rf_crow")'returns the rows of data into one cell
'Cells(1, 1) = Document.getElementById("Y_1").innerText 'returns the text for the year
'Cells(1, 1) = Document.getElementById("data_i1").innerText 'returns to first row of data
I know the above doesn't return what I want, because the comment tells you what it extracts into Excel. The sub-element doesn't seem to work as it does in other macros I've built. I thought something like this would work:
Cells(1, 1) = Document.getElementById("Y_1").getAttribute("rawvalue")
but that doesn't work, also, I tried:
Cells(1, 1) = Document.getElementById("data_i6").getElementById("Y_1").innertext
and that doesn't work either.
The solution is very easy. Just call it using it's attribute which is `rawvalue.
This is how you can go:
Using Hardcoded delay and for loop to check the availability of the desired value:
Sub GetValue()
Dim IE As New InternetExplorer, HTML As HTMLDocument, post As Object, elem As Object
With IE
.Visible = True
.Navigate "http://quote.morningstar.ca/Quicktakes/Financials/is.aspx?t=GNTX&region=USA&culture=en-CA&ops=clear"
While .Busy = True Or .ReadyState < 4: DoEvents: Wend
Set HTML = .Document
End With
''using hardcoded delay
Application.Wait Now + TimeValue("00:00:05")
For Each elem In HTML.getElementsByTagName("div")
If elem.innerText = "741" Then MsgBox elem.getAttribute("rawvalue"): Exit For
Next elem
End Sub
Using Explicit Wait:
Sub GetValue()
Dim IE As New InternetExplorer, HTML As HTMLDocument, post As Object
With IE
.Visible = True
.Navigate "http://quote.morningstar.ca/Quicktakes/Financials/is.aspx?t=GNTX&region=USA&culture=en-CA&ops=clear"
While .Busy = True Or .ReadyState < 4: DoEvents: Wend
Set HTML = .Document
End With
Do: Set post = HTML.querySelector("#data_i6 #Y_1"): DoEvents: Loop While post Is Nothing
MsgBox post.getAttribute("rawvalue")
End Sub
Output at this moment:
741131269
The following should illuminate some of the problems you were having.
.querySelectorAll
The exact element you mention is the second index returned by .querySelectorAll method of .document using the CSS selector #Y_1. The # means Id.
From that webpage it returns the following (sample shown - not all):
From the above you can see the string you want is returned by the index of 2 in the result.
querySelectorAll with Id? Isn't Id a unique identifier for a single element?
This Id, unexpectedly, is not unique to a single element on the page. It occurs a whopping 27 times:
This means you can use the .querySelectorAll method to return a nodeList of all matching items and take the item at index 2 to get your result.
Note:
If you want the long number next to rawvalue, 741131269, then parse the outerHTML of the returned element.
Debug.Print Replace(Split(Split(a.item(2).outerHTML, "rawvalue=")(1), ">")(0), Chr$(34), vbNullString)
.querySelector
Alternatively, you can target the id which is specific data_i6 with
.document.querySelector("#data_i6")
This CSS selector (#data_i6) returns the entire row as it has each year within. If using .querySelector you will only get the first item anyway which is year 1.
You can be more specific with the CSS selector and add the additional year Id to get just the year of interest:
#data_i6 #Y_1
Code: (querySelector method commented out next to querySelectorAll)
Option Explicit
Public Sub Get_Information()
Dim IE As New InternetExplorer
With IE
.Visible = True
.navigate "http://quote.morningstar.ca/Quicktakes/Financials/is.aspx?t=GNTX&region=USA&culture=en-CA&ops=clear"
While .Busy = True Or .readyState < 4: DoEvents: Wend
Dim a As Object, exitTime As Date
exitTime = Now + TimeSerial(0, 0, 2)
Do
DoEvents
On Error Resume Next
Set a = .document.querySelectorAll("#Y_1") ' .document.querySelector("#data_i6 #Y_1")
On Error GoTo 0
If Now > exitTime Then Exit Do
Loop While a Is Nothing
If a Is Nothing Then Exit Sub
Debug.Print Split(Split(a.item(2).innerText, "rawvalue=")(0), ">")(0) 'Split(Split(a.innerText, "rawvalue=")(0), ">")(0)
Debug.Print Replace(Split(Split(a.item(2).outerHTML, "rawvalue=")(1), ">")(0), Chr$(34), vbNullString) 'Replace(Split(Split(a.outerHTML, "rawvalue=")(1), ">")(0), Chr$(34), vbNullString)
.Quit
End With
End Sub
Try and declare "objCollection" as an object, strValue as string, and in the code below, replace in the first line the name of the http-request you declared:
Document.body.innerHTML = YourHTTPRequest.responseText
Set objCollection = Document.getElementsByClassName("rf_crow")
For Each objElement In objCollection
If objElement.ID = "Y_1" Then
strValue = objElement.getAttribute("rawvalue")
Exit For
End If
Next
Cells(1, 1) = strValue
Does this work for you?
Sub web_table_option_two()
Dim HTMLDoc As New HTMLDocument
Dim objTable As Object
Dim lRow As Long
Dim lngTable As Long
Dim lngRow As Long
Dim lngCol As Long
Dim ActRw As Long
Dim objIE As InternetExplorer
Set objIE = New InternetExplorer
objIE.Navigate "http://quote.morningstar.ca/Quicktakes/Financials/is.aspx?t=GNTX&region=USA&culture=en-CA&ops=clear"
Do Until objIE.ReadyState = 4 And Not objIE.Busy
DoEvents
Loop
Application.Wait (Now + TimeValue("0:00:03")) 'wait for java script to load
HTMLDoc.body.innerHTML = objIE.Document.body.innerHTML
With HTMLDoc.body
Set objTable = .getElementsByTagName("table")
For lngTable = 0 To objTable.Length - 1
For lngRow = 0 To objTable(lngTable).Rows.Length - 1
For lngCol = 0 To objTable(lngTable).Rows(lngRow).Cells.Length - 1
ThisWorkbook.Sheets("Sheet1").Cells(ActRw + lngRow + 1, lngCol + 1) = objTable(lngTable).Rows(lngRow).Cells(lngCol).innerText
Next lngCol
Next lngRow
ActRw = ActRw + objTable(lngTable).Rows.Length + 1
Next lngTable
End With
objIE.Quit
End Sub

Excel VBA-Run-time error 438. Object doesn't support this property or method. When trying to copy website table to excel

I am trying to extract table from an internal website to excel by giving some input parameters. Everything works until it refreshes the website data with my inputs. The part I get the run-time error 438 is marked (For r = 1 To elemCollection.Rows.Length - 1). I also tried to load the data from website to excel using web query and the table wasn't showing up on my excel spreadsheet. "It gives the following error-This page might not function correctly because either your browser does not support scripts or active scripting is disabled. Your browser does not support scripts or has been configured not to allow scripts. The report viewer web control http handler has not been registered in the application's web config file."
Wondering if this has anything to do with permissions.
VBA code below:
Option Explicit
Sub Macro1()
Dim IE As Object, obj As Object
Dim StartDate As Object
Dim EndDate As Object
Dim myState As String
Dim r As Integer, c As Integer, t As Integer
Dim elemCollection As Object, curHTMLRow As Object
Dim wb As Workbook
Dim ws As Worksheet
Dim objCollection As Object
Dim objElement As Object
Dim i As Long
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.navigate ("http://internalwebsite_SSRSReport")
' we ensure that the web page downloads completely before we fill the form automatically
While IE.ReadyState <> 4: DoEvents: Wend
IE.Document.All.Item("ctl31_ctl03_txtValue").InnerText = Format("7/1/2016", "m/d/yyyy")
IE.Document.All.Item("ctl31_ctl05_txtValue").InnerText = Format("7/31/2016", "m/d/yyyy")
Wait 2
IE.Document.getElementsByName("ctl31_ctl04_divDropDown").Item.Click
Wait 2
' accessing the button
IE.Document.getElementsByName("ctl31_ctl04_ctl00").Item.Click
Wait 2
' again ensuring that the web page loads completely before we start scraping data
While IE.busy: DoEvents: Wend
Wait 2
'Clearing any unnecessary or old data in Sheet1
ThisWorkbook.Sheets("Sheet1").Activate
Range("A1:K500").ClearContents
Set elemCollection = IE.Document.getelementbyId("ctl31_ctl09_ReportArea")
'error here
For r = 1 To elemCollection.Rows.Length - 1
Set curHTMLRow = elemCollection.Rows(r)
For c = 0 To curHTMLRow.Cells.Length - 1
Cells(r + 1, c + 1) = curHTMLRow.Cells(c).InnerText
Next
Next
' cleaning up memory
IE.Quit
Set IE = Nothing
End Sub
Private Sub Wait(ByVal nSec As Long)
nSec = nSec + Timer
While nSec > Timer
DoEvents
Wend
End Sub
Below is some code that should be able to grab the data from the HTML table from the SSRS report and extract it to Excel.
Basically the code will iterate through all the TRs and TDs in the Table Element, and output the InnerText to Excel. If you are moving a lot of data over, please consider writing to an array, then doing the write all at once by setting to an equally sized range object.
I also cleaned up the code, mostly removing variables that were not referenced and reduced some of the lines by combining some statements together
Option Explicit
Public Sub GetSSRSData()
On Error GoTo errhand:
Application.ScreenUpdating = False
Dim IE As Object: Set IE = CreateObject("InternetExplorer.Application")
Dim TR_Elements As Object
Dim TR As Object ' Table Row
Dim TD_Elements As Object
Dim TD As Object ' Table Data
Dim RowNumb As Integer
Dim Columns As Integer
Dim ColumnNumb As Integer
With IE
.Visible = True
.Navigate ("http://internalwebsite_SSRSReport")
While .ReadyState <> 4: DoEvents: Wend ' Wait for page load
'Fill the form out with dates
.Document.All.Item("ctl31_ctl03_txtValue").InnerText = Format("7/1/2016", "m/d/yyyy")
.Document.All.Item("ctl31_ctl05_txtValue").InnerText = Format("7/31/2016", "m/d/yyyy")
Wait 2
'Click the DropDown
.Document.getElementsByName("ctl31_ctl04_divDropDown").Item.Click
Wait 2
' Click the other button
.Document.getElementsByName("ctl31_ctl04_ctl00").Item.Click
End With
Wait 2
While IE.busy: DoEvents: Wend ' Wait for page load
Wait 2
'Clearing any unnecessary or old data in Sheet1
Sheets("Sheet1").Range("A1:K500").ClearContents
Set TR_Elements = IE.Document.getelementbyId("ctl31_ctl09_ReportArea").getElementsByTagName("tr")
RowNumb = 1
ColumnNumb = 1
'Tables usually consists of TR (Table Rows), and -
'TD (Table Data)
For Each TR In TR_Elements
Set TD_Elements = TR.getElementsByTagName("td")
ColumnNumb = 1
For Each TD In TD_Elements
'Consider using an array to save the values to memory if there is going
'to be a lot of data to be moved over
ActiveSheet.Cells(RowNumb, ColumnNumb).Value = TD.InnerText
ColumnNumb = ColumnNumb + 1
Next
RowNumb = RowNumb + 1
Next
' cleaning up memory
IE.Quit
Set IE = Nothing
Set TD_Elements = Nothing
Set TR_Elements = Nothing
Set TD = Nothing
Set TR = Nothing
Application.ScreenUpdating = True
errhand:
Application.ScreenUpdating = True
End Sub
Private Sub Wait(ByVal nSec As Long)
nSec = nSec + Timer
While nSec > Timer
DoEvents
Wend
End Sub

Extract URL from Internet popup window?

I have the following code that I am using to extract data from a sports website. My issue is that I am unable to locate the url for a pop-up window on this website - therefore, I am not sure how I could extract data from this window. The pop-window can be accessed by clicking the blue icon next to a players name, and the data I need is on the second tab of the pop-up window.
Sub Extract_goals()
Dim url As String, links_count As Integer
Dim i As Integer, j As Integer, row As Integer
Dim XMLHTTP As Object, html As Object
Dim tr_coll As Object, tr As Object
Dim td_coll As Object, td As Object
links_count = 40
For i = 1 To links_count
url = "http://fantasy.premierleague.com/stats/elements/?stat_filter=goals_scored&element_filter=0&page=" & i & ""
Set XMLHTTP = CreateObject("MSXML2.XMLHTTP")
XMLHTTP.Open "GET", url, False
XMLHTTP.send
Set html = CreateObject("htmlfile")
html.body.innerHTML = XMLHTTP.ResponseText
Set tbl = html.getelementsbytagname("Table")
Set tr_coll = tbl(0).getelementsbytagname("TR")
For Each tr In tr_coll
j = 1
Set td_col = tr.getelementsbytagname("TD")
For Each td In td_col
Cells(row + 1, j).Value = td.innerText
j = j + 1
Next
row = row + 1
Next
Next
End Sub
Any help with this is appreciated.
Thanks,
Shahid
I used a different method to get this (by creating an Internet Explorer object), because I could not get it to work exactly the same way using the MSXML2.XMLHTTP object.
I got as far as finding the URL of the popup window, but have yet to discover how to pull data from that window. If I have more time, I will play more, but perhaps this gets you over the hump and you can figure out the rest.
Sub Extract_goals2()
Dim ie As Object
Dim doc As Object
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
links_count = 40
For i = 1 To links_count
.navigate "http://fantasy.premierleague.com/stats/elements/?stat_filter=goals_scored&element_filter=0&page=" & i & ""
Do
DoEvents
Loop Until Not .busy Or .readyState <> 4
Set doc = .document
Dim tbl As Object
Set tbl = doc.getelementsbytagname("Table")
Dim tr_coll As Object
Set tr_coll = tbl(0).getelementsbytagname("TR")
For Each tr In tr_coll
j = 1
Set td_col = tr.getelementsbytagname("TD")
For Each td In td_col
If j = 2 Then 'only do this on 2nd table column
Set td_a = td.getelementsbytagname("a")
Debug.Print td_a(o).href 'this will provide the exact URL
td_a(o).Click 'this will actually open the pop-up box
'my thoughts were then to work with the elements in this URL to extract what you need
Else
Cells(row + 1, j).Value = td.innerText
End If
j = j + 1
Next
row = row + 1
Next
Next
End With
End Sub

Excel VBA Macro: Scraping data from site table that spans multiple pages

Thanks in advance for the help. I'm running Windows 8.1, I have the latest IE / Chrome browsers, and the latest Excel. I'm trying to write an Excel Macro that pulls data from StackOverflow (https://stackoverflow.com/tags). Specifically, I'm trying to pull the date (that the macro is run), the tag names, the # of tags, and the brief description of what the tag is. I have it working for the first page of the table, but not for the rest (there are 1132 pages at the moment). Right now, it overwrites the data everytime I run the macro, and I'm not sure how to make it look for the next empty cell before running.. Lastly, I'm trying to make it run automatically once per week.
I'd much appreciate any help here. Problems are:
Pulling data from the web table beyond the first page
Making it scrape data to the next empty row rather than overwriting
Making the Macro run automatically once per week
Code (so far) is below. Thanks!
Enum READYSTATE
READYSTATE_UNINITIALIZED = 0
READYSTATE_LOADING = 1
READYSTATE_LOADED = 2
READYSTATE_INTERACTIVE = 3
READYSTATE_COMPLETE = 4
End Enum
Sub ImportStackOverflowData()
'to refer to the running copy of Internet Explorer
Dim ie As InternetExplorer
'to refer to the HTML document returned
Dim html As HTMLDocument
'open Internet Explorer in memory, and go to website
Set ie = New InternetExplorer
ie.Visible = False
ie.navigate "http://stackoverflow.com/tags"
'Wait until IE is done loading page
Do While ie.READYSTATE <> READYSTATE_COMPLETE
Application.StatusBar = "Trying to go to StackOverflow ..."
DoEvents
Loop
'show text of HTML document returned
Set html = ie.document
'close down IE and reset status bar
Set ie = Nothing
Application.StatusBar = ""
'clear old data out and put titles in
'Cells.Clear
'put heading across the top of row 3
Range("A3").Value = "Date Pulled"
Range("B3").Value = "Keyword"
Range("C3").Value = "# Of Tags"
'Range("C3").Value = "Asked This Week"
Range("D3").Value = "Description"
Dim TagList As IHTMLElement
Dim Tags As IHTMLElementCollection
Dim Tag As IHTMLElement
Dim RowNumber As Long
Dim TagFields As IHTMLElementCollection
Dim TagField As IHTMLElement
Dim Keyword As String
Dim NumberOfTags As String
'Dim AskedThisWeek As String
Dim TagDescription As String
'Dim QuestionFieldLinks As IHTMLElementCollection
Dim TodaysDate As Date
Set TagList = html.getElementById("tags-browser")
Set Tags = html.getElementsByClassName("tag-cell")
RowNumber = 4
For Each Tag In Tags
'if this is the tag containing the details, process it
If Tag.className = "tag-cell" Then
'get a list of all of the parts of this question,
'and loop over them
Set TagFields = Tag.all
For Each TagField In TagFields
'if this is the keyword, store it
If TagField.className = "post-tag" Then
'store the text value
Keyword = TagField.innerText
Cells(RowNumber, 2).Value = TagField.innerText
End If
If TagField.className = "item-multiplier-count" Then
'store the integer for number of tags
NumberOfTags = TagField.innerText
'NumberOfTags = Replace(NumberOfTags, "x", "")
Cells(RowNumber, 3).Value = Trim(NumberOfTags)
End If
If TagField.className = "excerpt" Then
Description = TagField.innerText
Cells(RowNumber, 4).Value = TagField.innerText
End If
TodaysDate = Format(Now, "MM/dd/yy")
Cells(RowNumber, 1).Value = TodaysDate
Next TagField
'go on to next row of worksheet
RowNumber = RowNumber + 1
End If
Next
Set html = Nothing
'do some final formatting
Range("A3").CurrentRegion.WrapText = False
Range("A3").CurrentRegion.EntireColumn.AutoFit
Range("A1:C1").EntireColumn.HorizontalAlignment = xlCenter
Range("A1:D1").Merge
Range("A1").Value = "StackOverflow Tag Trends"
Range("A1").Font.Bold = True
Application.StatusBar = ""
MsgBox "Done!"
End Sub
There's no need to scrape Stack Overflow when they make the underlying data available to you through things like the Data Explorer. Using this query in the Data Explorer should get you the results you need:
select t.TagName, t.Count, p.Body
from Tags t inner join Posts p
on t.ExcerptPostId = p.Id
order by t.count desc;
The permalink to that query is here and the "Download CSV" option which appears after the query runs is probably the easiest way to get the data into Excel. If you wanted to automate that part of things, the direct link to the CSV download of results is here
You can improve this to parse out exact elements but it loops all the pages and grabs all the tag info (everything next to a tag)
Option Explicit
Public Sub ImportStackOverflowData()
Dim ie As New InternetExplorer, html As HTMLDocument
Application.ScreenUpdating = False
With ie
.Visible = True
.navigate "https://stackoverflow.com/tags"
While .Busy Or .READYSTATE < 4: DoEvents: Wend
Set html = .document
Dim numPages As Long, i As Long, info As Object, item As Object, counter As Long
numPages = html.querySelector(".page-numbers.dots ~ a").innerText
For i = 1 To 2 ' numPages ''<==1 to 2 for testing; use to numPages
DoEvents
Set info = html.getElementById("tags_list")
For Each item In info.getElementsByClassName("grid-layout--cell tag-cell")
counter = counter + 1
Cells(counter, 1) = item.innerText
Next item
html.querySelector(".page-numbers.next").Click
While .Busy Or .READYSTATE < 4: DoEvents: Wend
Set html = .document
Next i
Application.ScreenUpdating = True
.Quit '<== Remember to quit application
End With
End Sub
I'm not making use of the DOM, but I find it very easy to get around just searching between known tags. If ever the expressions you are looking for are too common just tweak the code a bit so that it looks for a string after a string).
An example:
Public Sub ZipLookUp()
Dim URL As String, xmlHTTP As Object, html As Object, htmlResponse As String
Dim SStr As String, EStr As String, EndS As Integer, StartS As Integer
Dim Zip4Digit As String
URL = "https://tools.usps.com/go/ZipLookupResultsAction!input.action?resultMode=1&companyName=&address1=1642+Harmon+Street&address2=&city=Berkeley&state=CA&urbanCode=&postalCode=&zip=94703"
Set xmlHTTP = CreateObject("MSXML2.XMLHTTP")
xmlHTTP.Open "GET", URL, False
On Error GoTo NoConnect
xmlHTTP.send
On Error GoTo 0
Set html = CreateObject("htmlfile")
htmlResponse = xmlHTTP.ResponseText
If htmlResponse = Null Then
MsgBox ("Aborted Run - HTML response was null")
Application.ScreenUpdating = True
GoTo End_Prog
End If
'Searching for a string within 2 strings
SStr = "<span class=""address1 range"">" ' first string
EStr = "</span><br />" ' second string
StartS = InStr(1, htmlResponse, SStr, vbTextCompare) + Len(SStr)
EndS = InStr(StartS, htmlResponse, EStr, vbTextCompare)
Zip4Digit = Left(Mid(htmlResponse, StartS, EndS - StartS), 4)
MsgBox Zip4Digit
GoTo End_Prog
NoConnect:
If Err = -2147467259 Or Err = -2146697211 Then MsgBox "Error - No Connection": GoTo End_Prog 'MsgBox Err & ": " & Error(Err)
End_Prog:
End Sub

if statement with getelementbyclass

First off, I am relatively new to programming. I have been making myself familiar primarily with VBA. I have asked questions here before and so far the responses have been very helpful so I am trying my luck again.
I am going to this site: http://www.otcmarkets.com/stock-screener
I want my script to say something like:
if ElementClassName("listingOdd").innertext = "USA, NV"
then GetElementsbyClassName("listingOddlistingOdd whiteSpaceNormal").innertext
I would also like it to loop through and click the next button which is behind this html until it is completed :
a href="javascript:void(null);">next & gt;/a
Any help is welcomed, I have been searching through past questions and watching tutorials and would not be asking if I was not seriously banging my head on this one. Thanks a lot in advance
This is some code that I have tried, with the following error: "Object doesnt support this method"
There is a lot commented out from trial and error
Sub t()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = False
IE.navigate "http://www.otcmarkets.com/stock-screener"
Do
DoEvents
Loop Until IE.readyState = READYSTATE_COMPLETE
Dim Doc As HTMLDocument
Set Doc = IE.document
Dim sDD As String
Dim lnk As Variant
sDD = Trim(Doc.GetElementbyClassName("listingOddlistingOdd whiteSpaceNormal").innerText)
'If lnk.classame.innerText = "USA, NV" Then
If lnk.innerText = "USA, NV" Then
'If ElementClassName("listingOdd")(0).innerText = "USA, NV" Then
'sDD
'sDD = Trim(Doc.GetElementbyClassName("listingOddlistingOdd whiteSpaceNormal").innerText)
MsgBox sDD
End If
End Sub
Sorry for the spaghetti code, like I said I am new
I'm not really clear on exactly what you're trying to get (and why not just use the handy "download all" link at the top of the page?)
EDITED - place cell values on worksheet
EDIT2 - clicking the "next" link.
EDIT3 - looped
This kind of thing is a total rabbit hole though.
Sub Tester()
Dim doc As Object
Dim IE As Object, nxt
Dim rng As Range, x As Integer
Set rng = ThisWorkbook.Sheets("sheet1").Range("A2")
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.navigate "http://www.otcmarkets.com/stock-screener"
WaitForLoad IE
Set nxt = GetNext(IE)
Do While Not nxt Is Nothing
ExtractTableContent IE, rng
nxt.Click
WaitForLoad IE
Set nxt = GetNext(IE)
Loop
End Sub
Sub WaitForLoad(IE As Object)
Do: DoEvents: Loop Until IE.readyState = 4 'READYSTATE_COMPLETE
Application.Wait Now + TimeSerial(0, 0, 3)
End Sub
'copy table content
Sub ExtractTableContent(ByRef IE As Object, ByRef rng As Range)
Dim tableDiv As Object, r As Object, rw As Object, x As Long
Set tableDiv = IE.document.getelementbyid("stockScreenerResults")
Set r = tableDiv.getelementsbytagname("table")(0).Rows
For Each rw In r
For x = 1 To rw.Cells.Length
rng.Offset(0, x - 1).Value = rw.Cells(x - 1).innerText
Next x
Set rng = rng.Offset(1, 0)
Next rw
End Sub
'find the link which takes you to the next page...
Function GetNext(IE As Object) As Object
Dim links, l As Object, rv As Object, cn As Object
Set links = IE.document.getelementsbytagname("a")
For Each l In links
If l.innerText Like "*next*" Then
Set rv = l
Exit For
End If
Next l
Set GetNext = rv
End Function