First off, I am relatively new to programming. I have been making myself familiar primarily with VBA. I have asked questions here before and so far the responses have been very helpful so I am trying my luck again.
I am going to this site: http://www.otcmarkets.com/stock-screener
I want my script to say something like:
if ElementClassName("listingOdd").innertext = "USA, NV"
then GetElementsbyClassName("listingOddlistingOdd whiteSpaceNormal").innertext
I would also like it to loop through and click the next button which is behind this html until it is completed :
a href="javascript:void(null);">next & gt;/a
Any help is welcomed, I have been searching through past questions and watching tutorials and would not be asking if I was not seriously banging my head on this one. Thanks a lot in advance
This is some code that I have tried, with the following error: "Object doesnt support this method"
There is a lot commented out from trial and error
Sub t()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = False
IE.navigate "http://www.otcmarkets.com/stock-screener"
Do
DoEvents
Loop Until IE.readyState = READYSTATE_COMPLETE
Dim Doc As HTMLDocument
Set Doc = IE.document
Dim sDD As String
Dim lnk As Variant
sDD = Trim(Doc.GetElementbyClassName("listingOddlistingOdd whiteSpaceNormal").innerText)
'If lnk.classame.innerText = "USA, NV" Then
If lnk.innerText = "USA, NV" Then
'If ElementClassName("listingOdd")(0).innerText = "USA, NV" Then
'sDD
'sDD = Trim(Doc.GetElementbyClassName("listingOddlistingOdd whiteSpaceNormal").innerText)
MsgBox sDD
End If
End Sub
Sorry for the spaghetti code, like I said I am new
I'm not really clear on exactly what you're trying to get (and why not just use the handy "download all" link at the top of the page?)
EDITED - place cell values on worksheet
EDIT2 - clicking the "next" link.
EDIT3 - looped
This kind of thing is a total rabbit hole though.
Sub Tester()
Dim doc As Object
Dim IE As Object, nxt
Dim rng As Range, x As Integer
Set rng = ThisWorkbook.Sheets("sheet1").Range("A2")
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.navigate "http://www.otcmarkets.com/stock-screener"
WaitForLoad IE
Set nxt = GetNext(IE)
Do While Not nxt Is Nothing
ExtractTableContent IE, rng
nxt.Click
WaitForLoad IE
Set nxt = GetNext(IE)
Loop
End Sub
Sub WaitForLoad(IE As Object)
Do: DoEvents: Loop Until IE.readyState = 4 'READYSTATE_COMPLETE
Application.Wait Now + TimeSerial(0, 0, 3)
End Sub
'copy table content
Sub ExtractTableContent(ByRef IE As Object, ByRef rng As Range)
Dim tableDiv As Object, r As Object, rw As Object, x As Long
Set tableDiv = IE.document.getelementbyid("stockScreenerResults")
Set r = tableDiv.getelementsbytagname("table")(0).Rows
For Each rw In r
For x = 1 To rw.Cells.Length
rng.Offset(0, x - 1).Value = rw.Cells(x - 1).innerText
Next x
Set rng = rng.Offset(1, 0)
Next rw
End Sub
'find the link which takes you to the next page...
Function GetNext(IE As Object) As Object
Dim links, l As Object, rv As Object, cn As Object
Set links = IE.document.getelementsbytagname("a")
For Each l In links
If l.innerText Like "*next*" Then
Set rv = l
Exit For
End If
Next l
Set GetNext = rv
End Function
Related
So I'm new to vba and i am try to get price (i tried everyting my know). The macro is :
Sub Deneme()
Dim objIE As InternetExplorer
Dim Prc1 As String
Set objIE = New InternetExplorer
Dim Search_Terms() As Variant
Dim CopiedData() As Variant
Dim y As Integer
objIE.Visible = False
Search_Terms = Application.Transpose(ActiveSheet.Range("A2:A169").Value)
ReDim CopiedData(LBound(Search_Terms) To UBound(Search_Terms))
y = 2
For a = LBound(Search_Terms) To UBound(Search_Terms)
objIE.navigate "https://steamcommunity.com/market/listings/578080/" & Search_Terms(a)
Do: DoEvents: Loop Until objIE.readyState = 4
Prc1 = objIE.document.getElementsByClassName("market_commodity_orders_table")(4).getElementsByTagName("tr")(1).textContent '<----- the problem is here
ActiveSheet.Range("D" & y).Value = Prc1
y = y + 1
Next
objIE.Quit
End Sub
The website is THIS and I am trying to get this value:
Mostly error is :
Run-time error '91':
Object variable or With block variable not set.
And Debug is :
objIE.document.getElementsByClassName("market_commodity_orders_table")(4).getElementsByTagName("tr")(1).textContent
In the process of me testing my new code for you, I realized that you have other issues other than the class name you were attempting to use not existing.
The other issue is that the document loads before some of the other resources - this is likely due to the fact that this site updates the price every second (and therefore the price is not initially loaded in the objIE.Document object).
To get around this, I've added a couple of loops to wait for your object to become available. This should work for you.
Sub Deneme()
Dim objIE As InternetExplorer
Dim Prc1 As String
Set objIE = New InternetExplorer
Dim Search_Terms() As Variant
Dim CopiedData() As Variant
Dim y As Integer
Dim elemObj As Object
objIE.Visible = False
Search_Terms = Application.Transpose(ActiveSheet.Range("A2:A169").Value)
ReDim CopiedData(LBound(Search_Terms) To UBound(Search_Terms))
y = 2
For a = LBound(Search_Terms) To UBound(Search_Terms)
objIE.navigate "https://steamcommunity.com/market/listings/578080/" & Search_Terms(a)
Do: DoEvents: Loop Until objIE.readyState = 4
Do While Prc1 = ""
Do While elemObj Is Nothing
Set elemObj = objIE.document.getElementById("market_commodity_buyrequests")
Set elemObj = elemObj.getElementsByClassName("market_commodity_orders_header_promote")(1)
Loop
Prc1 = elemObj.innerText
Loop
ActiveSheet.Range("D" & y).Value = Prc1
Set elemObj = Nothing
Prc1 = vbNullString
y = y + 1
Next
objIE.Quit
End Sub
There are 2 issues in your code …
There is no class called market_commodity_orders_table
item counting starts with 0 so the 4ᵗʰ item is item no 3.
You can use this:
Prc1 = objIE.document.getElementsByClassName("market_commodity_orders_header_promote").Item(3).innerText
Let us try it in a slightly different manner. If you have IE9 or later then the following code should work for you flawlessly. I used .querySelector() here. Give this a shot and find the price you are after.
Sub GetPrice()
Const URL As String = "https://steamcommunity.com/market/listings/578080/PLAYERUNKNOWN's%20Bandana"
Dim HTML As HTMLDocument, post As Object
With New InternetExplorer
.Visible = True
.navigate URL
While .Busy = True Or .readyState < 4: DoEvents: Wend
Set HTML = .document
Do: Set post = HTML.querySelector("#market_commodity_buyrequests .market_commodity_orders_header_promote:nth-of-type(2)"): DoEvents: Loop While post Is Nothing
[A1] = post.innerText
.Quit
End With
End Sub
Reference to add to the library:
Microsoft Internet Controls
Microsoft HTML Object Library
I have the below code, wherein I'm trying to open a series of urls and pull in the data from each url (example: http://apps.mohltc.ca/ltchomes/detail.php?id=2588&lang=en). Of most interest to me would be those labeled as "Local Health Integration Network", "Licensee" and "Licensed Beds".
As it stands, I'm trying to just pull in all elements with tag name "p" and deal with the data scrubbing later on. My code currently pulls in "[object HTML Paragraph Element]" instead of the array that I'm hoping for. Can someone explain why this is?
Sub ImportLicenseeData()
Dim ie As Object
Dim LH As Object
Dim r As Integer
Set ie = CreateObject("InternetExplorer.Application")
For r = 4 To 10
With ie
ie.Visible = False
ie.Navigate Cells(r, "H").Value
Do While (ie.Busy Or ie.ReadyState <> 4): DoEvents: Loop
Set Doc = ie.Document
Set LH = Doc.getElementsByTagName("p")
End With
Worksheets("Sheet1").Range("J" & r).Value = LH
Next r
End Sub
Any help is appreciated.
Dim LH As IHTMLElementCollection
Dim htmlEle1 as IHTMLElement
It requires Microsoft HTML Object Library reference. Then you can interact with elements of LH collection (it's not an array) like this:
Set LH = Doc.getElementsByTagName("p")
For Each htmlEle1 in LH
Debug.Print htmlEle1.innerText
Next htmlEle1
Thanks for the help everyone. I wasn't too familiar with handling the HTML Elements, so I ended up going with a different approach. Appreciate the feedback regardless.
via http://www.ozgrid.com/forum/showthread.php?t=178150
Sub RetrieveHTML()
Dim rngSelect As Range
Dim sURL As String
Set rngSelect = Range("H8", Range("H8").End(xlDown))
Debug.Print rngSelect.Address
Set ie = CreateObject("InternetExplorer.Application")
For Each c In rngSelect
sURL = c.Value
With ie
.Visible = False
.Navigate sURL
Do Until .ReadyState = 4
DoEvents
Loop
Do While .Busy: DoEvents: Loop
Range(c.Address).Offset(0, 1).Value = ie.Document.DocumentElement.outerHTML
End With
Next c
End Sub
Please see the code below. I am compiling a list of unusual currency pairings in excel and I wish to scrape this data with VBA. I only want to insert the value itself into the cell. Does anyone know where I am going wrong here? I am getting a 'Run-time error '91': object variable or With block variable not set'. I'm relatively new to VBA and i've put a lot a deal of thought into this.
Sub ie_open()
Dim wb As Workbook
Dim ws As Worksheet
Dim TxtRng As Range
Dim ie As Object
Set ie = CreateObject("INTERNETEXPLORER.APPLICATION")
ie.NAVIGATE "http://www.barchart.com/quotes/forex/British_Pound/Costa_Rican_Colon/%5EGBPCRC"
ie.Visible = True
While ie.ReadyState <> 4
DoEvents
Wend
Set wb = ActiveWorkbook
Set ws = wb.Sheets("Test Sheet")
Set TxtRng = ws.Range("A1")
TxtRng.Value = ie.document.getelementsbyname("divQuotePage").Item.innertext
End Sub
This is the data which I am trying to scrape:
Thanks.
I'm not that accomplished at web scraping, but that kind of error often means that what you are looking for isn't there. In particular, I don't see divQuotePage in the screen shot you provided.
But if you want the quote (793.19) you could do something like:
Dim V As Variant
Set V = ie.document.getelementbyid("dtaLast")
TxtRng = V.innertext
This will work.
Sub Test()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Visible = True
.Navigate "http://www.barchart.com/quotes/forex/British_Pound/Costa_Rican_Colon/%5EGBPCRC" ' should work for any URL
Do Until .ReadyState = 4: DoEvents: Loop
x = .document.body.innertext
y = InStr(1, x, "Last Price")
Z = Mid(x, y, 19)
Range("A1").Value = Trim(Z)
.Quit
End With
End Sub
You can target that element with a CSS selector of div.pricechangerow > span.last-change;
which can be simplified to .last-change.
The "." means class and you can retrieve this specific item with
Debug.Print ie.document.querySelector.querySelector(".last-change").innerText
That is for the website's current incarnation at 2018-06-30
Tried asking this question. Didn't get many answers. Can't install things onto my work computer. https://stackoverflow.com/questions/29805065/vba-webscrape-not-picking-up-elements
Want to scrape a morningstar page into Excel with the code below. Problem is, it doesn't feed any real elements/data back. I actually just want the Dividend and cap gain distribution table really from that link I put into my_Page.
This is usually easiest way, but an entire page scrape way, AND Excel-->Data-->From Web DON'T work.
I've tried to use get elements by tag name and class before, but I failed at being able to do it in this case.This might be the way to go... Once again, just want that Dividend and Cap Gain distribution table. Not seeing any results in via the Debug.print
Working code below, just need to parse into excel. Updated attempt below:
Sub Macro1()
Dim IE As New InternetExplorer
IE.Visible = True
IE.navigate "http://quotes.morningstar.com/fund/fundquote/f?&t=ANNPX&culture=en_us&platform=RET&viewId1=2046632524&viewId2=3141452350&viewId3=3475652630"
Do
DoEvents
Loop Until IE.readyState = READYSTATE_COMPLETE
Dim doc As HTMLDocument
Set doc = IE.document
'For Each Table In doc.getElementsByClassName("gr_table_b1")
'For Each td In Table.getElementsByTagName("tr")
On Error Resume Next
For Each td In doc.getElementsByClassName("gr_table_row4")
Debug.Print td.Cells(5).innerText
'Debug.Print td.Cells(1).innerText
Next td
'Next Table
'IE.Quit
'Application.EnableEvents = True
End Sub
The content in question is contained within an iframe. You can see this by right clicking on that section of the sebsite, and selecting Inspect element. Looking up the tree, you'll see an iframe tag, containing the url of data. You should try to find that element, and extract that url (which is generated with js) and then open that page.
No frame to worry about. You only need the table id.
Webpage view:
Print out from code:
VBA:
Option Explicit
Public Sub GetDivAndCapTable()
Dim ie As New InternetExplorer, hTable As HTMLTable
Const URL = "http://quotes.morningstar.com/fund/fundquote/f?&t=ANNPX&culture=en_us&platform=RET&viewId1=2046632524&viewId2=3141452350&viewId3=3475652630"
Application.ScreenUpdating = False
With ie
.Visible = True
.navigate URL
While .Busy Or .READYSTATE < 4: DoEvents: Wend
Set hTable = .document.getElementById("DividendAndCaptical")
WriteTable hTable, 1
Application.ScreenUpdating = True
.Quit
End With
End Sub
Public Sub WriteTable(ByVal hTable As HTMLTable, Optional ByVal startRow As Long = 1, Optional ByVal ws As Worksheet)
If ws Is Nothing Then Set ws = ActiveSheet
Dim tSection As Object, tRow As Object, tCell As Object, tr As Object, td As Object, R As Long, C As Long, tBody As Object
R = startRow
With ws
Dim headers As Object, header As Object, columnCounter As Long
Set headers = hTable.getElementsByTagName("th")
For Each header In headers
columnCounter = columnCounter + 1
.Cells(startRow, columnCounter) = header.innerText
Next header
startRow = startRow + 1
Set tBody = hTable.getElementsByTagName("tbody")
For Each tSection In tBody 'HTMLTableSection
Set tRow = tSection.getElementsByTagName("tr") 'HTMLTableRow
For Each tr In tRow
Set tCell = tr.getElementsByTagName("td")
C = 1
For Each td In tCell 'DispHTMLElementCollection
.Cells(R, C).Value = td.innerText 'HTMLTableCell
C = C + 1
Next td
R = R + 1
Next tr
Next tSection
End With
End Sub
I have multiple urls stored in Excel sheet. I want to Get data reside within particular div tag. For One Website it works fine
Sub Cityline()
Dim IE As Object
Set IE = CreateObject("Internetexplorer.application")
IE.Visible = True
IE.navigate "http://Someurl.com/bla/bla/bla"
Do While IE.busy
DoEvents
Loop
Do
DoEvents
Dim Doc As Object
Set Doc = IE.Document
Dim workout As String
workout = Doc.getElementsByClassName("CLASS_NAME_OF_DATA")(0).innertext
Range("A2") = workout
Loop
End Sub
I used Below code for loop Through all urls but its not working
Sub GetData()
Dim oHtm As Object: Set oHtm = CreateObject("HTMLFile")
Dim req As Object: Set req = CreateObject("msxml2.xmlhttp")
Dim oRow As Object
Dim oCell As Range
Dim url As String
Dim y As Long, x As Long
x = 1
For Each oCell In Sheets("sheet1").Range("A2:A340")
req.Open "GET", oCell.Offset(, 1).Value, False
req.send
With oHtm
.body.innerhtml = req.responsetext
With .getelementsbytagname("table")(1)
With Sheets(1)
.Cells(x, 1).Value = oCell.Offset(, -1).Value
.Cells(x, 2).Value = oCell.Value
End With
y = 3
For Each oRow In .Rows
Sheets(1).Cells(x, y).Value = oRow.Cells(1).innertext
y = y + 1
Next oRow
End With
End With
x = x + 1
Next oCell
End Sub
But its not working
can any one suggest me where i went wrong ?
I used Fetching Data from multiple URLs but it doesn't works for me.
Please guide me how to get data from all urls at a Time
I'm new to SO, so apologies to the mods if this should be in comments (I couldn't get it to fit).
I agree with Silver's comments, but I thought I'd suggest a different approach that might help. If you have URLs in a column of cells, you could create a custom VBA function that will extract the relevant data out of the HTML. Just use this function in the cells to the right of your URL to return the relevant data from the HTML. An example is this:
Public Function GetHTMLData(SiteURL As String, FieldSearch As String) As String
Dim IE As Object
Dim BodyHTML As String
Dim FieldStart As Integer
Dim FieldEnd As Integer
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Navigate SiteURL
Do While .Busy Or .ReadyState <> 4
DoEvents
Loop
BodyHTML = IIf(StrComp(.Document.Title, "Cannot find server", vbTextCompare) = 0, _
vbNullString, .Document.body.innerhtml)
FieldStart = InStr(1, BodyHTML, FieldSearch) + Len(FieldSearch) + 12
FieldEnd = InStr(FieldStart, BodyHTML, "<")
GetHTMLData = Mid(BodyHTML, FieldStart, FieldEnd - FieldStart)
.Quit
End With
Set IE = Nothing
End Function
The function above has 2 input parameters: the URL and a string that will be searched for within the HTML. It will then return a string from within the HTML, starting from 12 characters after the searched parameter and ending at the following '<' within the HTML.
Hope that helps.