I am confused by this object required error VBA - vba

I have been working on this VBA Code for a long time. The goal is to copy something from my excel document, search for it online, and then pull back something from the html code. I keep getting an error code that says "Object Required" and sometimes it says "Object variable or With block variable not set." It is all focused on the line "set elementTWO = elementONE.Item(i).innerText
I have tried deleting the word "Set" I have tried changing elementTWO to a string. Another really weird piece is the For...Next Loop won't let me do "exit for." It returns an error. I have tried a few other things to no avail. Any help is greatly appreciated
Option Explicit
Option Compare Text
Public Enum READYSTATE
READYSTATE_UNINITIALIZED = 0
READYSTATE_LOADING = 1
READYSTATE_LOADED = 2
READYSTATE_INTERACTIVE = 3
READYSTATE_COMPLETE = 4
End Enum
Sub GetCategory()
Dim RowNo, ColNo, i As Integer
Dim Parent, Item, URL1, URL2, URL3 As String
Dim objHTML As Object
Dim elementONE As Object
Dim elementTWO As String
RowNo = 3
ColNo = 5
URL1 = "http://www.infores.com/public/us/knowledgegroup/resources/resources.pli?defaultDataType=&pageid=validatorresults&upc1="
URL2 = "&upc2="
URL3 = "&submitupc=find+it%21"
Dim ie As Object
Set ie = CreateObject("InternetExplorer.Application")
With Worksheets(1)
While RowNo <= 5
Parent = Cells(RowNo, ColNo)
Item = Cells(RowNo, ColNo + 1)
With ie
.navigate URL1 & Parent & URL2 & Item & URL3
.Visible = False
'Delay while IE loads
Do While (ie.Busy Or ie.READYSTATE <> READYSTATE.READYSTATE_COMPLETE)
DoEvents
Loop
'Put html code in document object
Set objHTML = .document
DoEvents
End With
Set elementONE = objHTML.getElementsByTagName("TD") 'Break Down HTML code
For i = 1 To elementONE.Length
elementTWO = elementONE.Item(i).innerText
If elementTWO = "Description" Then 'Find the Category
Cells(RowNo, ColNo + 4) = elementONE.Item(i + 1).innerText 'Put Category into excel
End If
Next i
DoEvents
ie.Quit
RowNo = RowNo + 1
Wend
End With
End Sub

this I think is what you want, it processes the loop but then fails during it. you can atleast debug it and see why:
Option Explicit
Option Compare Text
Public Enum READYSTATE
READYSTATE_UNINITIALIZED = 0
READYSTATE_LOADING = 1
READYSTATE_LOADED = 2
READYSTATE_INTERACTIVE = 3
READYSTATE_COMPLETE = 4
End Enum
Sub GetCategory()
Dim RowNo, ColNo, i As Integer
Dim Parent, Item, URL1, URL2, URL3 As String
Dim objHTML As Object, elementONE As Object, elementTWO As String
RowNo = 3
ColNo = 5
URL1 = "http://www.infores.com/public/us/knowledgegroup/resources/resources.pli?defaultDataType=&pageid=validatorresults&upc1="
URL2 = "&upc2="
URL3 = "&submitupc=find+it%21"
Dim ie As Object
'Set ie = CreateObject("InternetExplorer.Application")
With Worksheets(1)
While RowNo <= 5
Parent = Cells(RowNo, ColNo)
Item = Cells(RowNo, ColNo + 1)
Set ie = CreateObject("InternetExplorer.Application")
With ie
.navigate URL1 & Parent & URL2 & Item & URL3
.Visible = False
'Delay while IE loads
Do While (ie.Busy Or ie.READYSTATE <> READYSTATE.READYSTATE_COMPLETE)
DoEvents
Loop
'Put html code in document object
Set objHTML = .document
DoEvents
End With
Set elementONE = objHTML.getElementsByTagName("TD") 'Break Down HTML code
For i = 0 To elementONE.Length - 1
elementTWO = elementONE(i).innerText
If elementTWO = "Description" Then 'Find the Category
Cells(RowNo, ColNo + 4) = elementONE(i + 1).innerText 'Put Category into excel
End If
Next i
DoEvents
ie.Quit
RowNo = RowNo + 1
Wend
End With
End Sub

Related

How to get first 50 post url from instagram

I want to get the first 50 post for any public instagram url. To get the first 12 posts which are visible by default is easy and below code is doing that just fine but I am not sure how to access other urls which is visible when you hit "load more" button at the bottom and scroll further down for more. Is this possible? Can anyone help me with this?
Sub getData()
Dim urL As String, instaID As String, totalPost As Long, fRow As Long
Dim ie, var1, var2, a
urL = "https://www.instagram.com/nike/"
OUT.Range("A2:A" & OUT.Rows.Count).Clear
Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
fRow = 2
'----------------------
With OUT
ie.navigate urL
'Busy
Do While ie.Busy
Application.Wait DateAdd("s", 1, Now)
Loop
Set var1 = ie.document.getElementsByclassname("_mck9w _gvoze _f2mse")
Do While IsObject(var1) = False
Application.Wait Now + TimeValue("00:00:03")
Set var1 = ie.document.getElementsByclassname("_mck9w _gvoze _f2mse")
Loop
Set var2 = var1(0).document.getElementsBytagname("a")
For Each a In var2
If Trim(a.href) <> "" And a.parentelement.classname = "_mck9w _gvoze _f2mse" Then
.Range("A" & fRow) = a.href
Debug.Print a.innertext & " " & a.parentelement.classname
fRow = fRow + 1
End If
Next a
End With
'------------------
ie.Quit
Set ie = Nothing
Application.ScreenUpdating = True
End Sub
try this
Sub getData()
Dim urL As String, instaID As String, totalPost As Long, fRow As Long
Dim ie As Object, var1 As Object, a As Variant
urL = "https://www.instagram.com/nike/"
Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
ie.navigate urL
Do While ie.Busy Or ie.readystate <> 4
DoEvents
Loop
Do
Set var1 = ie.Document.getElementsByclassname("_1cr2e _epyes") ' wait for "load more"
Loop While var1 Is Nothing
var1(0).Click ' click "load more"
Do While ie.Busy Or ie.readystate <> 4 ' not sure if this is needed
DoEvents
Loop
For a = 1 To 4
ie.Document.parentWindow.scrollBy 0, 60000 ' roll to bottom
Application.Wait Now + TimeValue("00:00:01") ' wait for "fill in"
Next a
Set var1 = ie.Document.getElementsBytagname("a")
Range("A2:A" & Rows.count).Clear
fRow = 2
For Each a In var1
If Trim(a.href) <> "" And a.parentelement.classname = "_mck9w _gvoze _f2mse" Then
Range("A" & fRow) = a.href
Debug.Print a.innertext & " " & a.parentelement.classname
fRow = fRow + 1
End If
Next a
' ------------------
ie.Quit
Set ie = Nothing
End Sub

I'm trying to import innertext from a td-class. Getting lots of extra text

I'm trying to get the innertext of 'wfm-bodyText'. So, I want to loop through items on a webpage and import, in this case, 'H. Validate Gamer Correct'
I thought the script below would work, but it actually gives me all kinds of extra text!!
Public Sub CopyFromURL()
Dim IE As InternetExplorer, doc As HTMLDocument
Dim thisClass As IHTMLElement2, thisLink As IHTMLElement
Dim rng As Range, cell As Range
Const READYSTATE_COMPLETE As Integer = 4
Dim TR_col As Object, TR As Object
Dim TD_col As Object, TD As Object
Dim row As Long, col As Long
row = 1
'Set rng = Range("A1:A5")
'For Each cell In rng
Set IE = CreateObject("InternetExplorer.Application")
IE.Navigate URL
Do While (IE.Busy Or IE.ReadyState <> READYSTATE_COMPLETE)
DoEvents
Loop
Set TR_col = IE.Document.getElementsByTagName("TR")
For Each TR In TR_col
Set TD_col = TR.getElementsByTagName("TD")
row = ActiveSheet.Cells(ActiveSheet.Rows.Count, "A").End(xlUp).row
col = 1
For Each TD In TD_col
Cells(row, col) = TD.innerText
col = col + 1
Next
row = row + 1
Next
'Next cell
IE.Quit
End Sub
The URL is behind a firewall. I can't share it.
Again, below is the structure of the page. I want to copy the 'E. Verify phone number if applicable' into my sheet.
I tried the script below in an attempt to get every element from the page.
Sub DumpData()
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
URL = "http://www.SharePoint.aspx"
'Wait for site to fully load
IE.Navigate2 URL
Do While IE.Busy = True
DoEvents
Loop
RowCount = 1
With Sheets("Sheet1")
.Cells.ClearContents
RowCount = 1
For Each itm In IE.document.all
.Range("A" & RowCount) = itm.tagname
.Range("B" & RowCount) = itm.ID
.Range("C" & RowCount) = itm.classname
.Range("D" & RowCount) = Left(itm.innertext, 1024)
RowCount = RowCount + 1
Next itm
End With
End Sub
I still can't tell where the relevant data is coming from.

VBA to Excel how do i get data name and phone numbers instead of hyperlinks?

Sub GetAllLinks()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
url_name = "http://www.trovanumeri.com/?azione=cerca&cerca=portoscuso"
If url_name = "" Then Exit Sub
IE.navigate url_name
Do
DoEvents
Loop Until IE.readyState = 4
'<a href="http://google.com">Click Here
Set AllHyperLinks = IE.document.getElementsByTagName("A")
Foglio1.ListBox1.Clear
'this is the code
For Each hyper_link In AllHyperLinks
Foglio1.ListBox1.AddItem hyper_link
Next
MsgBox "Done!"
End Sub
Take a look at the below example, showing how to retrieve the data from via IE automation and DOM processing:
Option Explicit
Sub GetData()
Dim oIE As Object
Dim lCurRow As Long
Dim lResultIndex As Long
Dim sUrl As String
Dim oTable0 As Object
Dim oTable1 As Object
Dim oTable2 As Object
Dim oTable3 As Object
Dim sTbl3Text As String
Dim cAncorNodes As Object
Dim oAncorNode As Variant
Dim sRowText As String
Dim oRowNode As Object
Dim aData() As Variant
Set oIE = CreateObject("InternetExplorer.Application")
With oIE
.Visible = True
Sheets(1).Cells.Delete
lCurRow = 1 ' Worksheet rows counter
lResultIndex = 0 ' Search result index counter
Do
' Navigate to the page
sUrl = "http://www.trovanumeri.com/?azione=cerca&cerca=cagliari&da=" & lResultIndex
.Navigate sUrl
' Wait IE
Do While .ReadyState < 3 Or .Busy
DoEvents
Loop
' Wait Document
Do Until .Document.ReadyState = "complete"
DoEvents
Loop
' Retrieve target tables
Set oTable0 = .Document.getElementsByTagName("table")(0)
Set oTable1 = oTable0.getElementsByTagName("table")(1)
Set oTable2 = oTable1.getElementsByTagName("table")(3)
' Get and process ancor nodes
Set cAncorNodes = oTable2.getElementsByTagName("a")
For Each oAncorNode In cAncorNodes
With CreateObject("Scripting.Dictionary")
' Add .href to result
.Add .Count, oAncorNode.href
' Get ancor's parent row
Set oRowNode = oAncorNode.ParentNode.ParentNode.ParentNode
Do
' Add nonemtpy row to result
sRowText = Trim(Replace(oRowNode.innerText, vbCrLf, ""))
If sRowText <> "" Then .Add .Count, sRowText
' If last row then exit
If IsNull(oRowNode.nextElementSibling) Then Exit Do
' Proceed with next row
Set oRowNode = oRowNode.nextElementSibling
' If net row contains oAncorNode then exit
If oRowNode.getElementsByTagName("a").Length > 0 Then Exit Do
DoEvents
Loop
' Get results as array
aData = .Items
End With
' Output array to worksheet row
With Sheets(1).Cells(lCurRow, 1)
.Resize(1, UBound(aData) + 1) = aData
.Select
End With
lCurRow = lCurRow + 1
DoEvents
Next
' Get table containing 'Next' button
Set oTable3 = oTable0.getElementsByTagName("table")(7)
sTbl3Text = oTable3.innerText
' If no 'Next' button then exit
If InStr(sTbl3Text, "Avanti >>") = 0 Then Exit Do
lResultIndex = lResultIndex + 10
DoEvents
Loop
.Quit
End With
End Sub

Scraping from VBA (very close to working!!)

I have been playing around with a scraper for consumer stocks and I can scrape data from the main page of items but once i start using the second, thid parges,
Sub asosdesc2()
Const READYSTATE_COMPLETE = 4
Dim j As Integer
Dim ie As InternetExplorer
Dim Doc As IHTMLDocument
Dim xcolElements As IHTMLElementCollection
Dim ell As IHTMLElement
Dim pn As Integer
j = 1
Set ie = CreateObject("InternetExplorer.Application")
For pn = 1 To 1
ie.Visible = False
ie.Navigate "http://www.asos.com/Men/Sale/Accessories/Cat/pgecategory.aspx?cid=2097&CTARef=shop|sale|cat|accessories#parentID=-1&pge=0&pgeSize=36&sort=-1"
Do
DoEvents
Loop Until Not ie.Busy And ie.readyState = READYSTATE_COMPLETE
Set Doc = ie.Document
'Set xcolElements = Doc.getElementsByClassName("description")
'Here lies the problem. Description works perflectly, price doesn't
Set xcolElements = Doc.getElementsByClassName("desc")
For Each ell In xcolElements
On Error GoTo Skip
Range("B" & j).Value = ell.innerText
j = j + 1
Skip:
Next ell
On Error GoTo 0
Next pn
ie.Quit
Set el = Nothing
Set xcolElements = Nothing
Set Doc = Nothing
Set ie = Nothing
End Sub
I am trying to return the description, the price and the HREF code to identify the item
Instead of looping through the collection of "desc" anchors (which only contain the product description), I suggest you loop through their parent "items" collection. That way you can reference your anchors' siblings, which contain price etc.:
Sub asosdesc2()
Const READYSTATE_COMPLETE = 4
Dim j As Integer
Dim ie As InternetExplorer
Dim Doc As IHTMLDocument
Dim itemsColl As IHTMLElementCollection
Dim itemsEle As IHTMLElement
Dim itemsAttributesColl As IHTMLElementCollection
Dim itemsAttributesEle As IHTMLElement
Dim pn As Integer
On Error GoTo 0
j = 1
Set ie = CreateObject("InternetExplorer.Application")
For pn = 1 To 1
ie.Visible = True
ie.Navigate "http://www.asos.com/Men/Sale/Accessories/Cat/pgecategory.aspx?cid=2097&CTARef=shop|sale|cat|accessories#parentID=-1&pge=0&pgeSize=36&sort=-1"
Do
DoEvents
Loop Until Not ie.Busy And ie.readyState = READYSTATE_COMPLETE
Set Doc = ie.Document
Set itemsColl = Doc.getElementById("items").Children
For Each itemsEle In itemsColl
On Error GoTo Skip
Set itemsAttributesColl = itemsEle.Children
For Each itemsAttributesEle In itemsAttributesColl
If itemsAttributesEle.className = "desc" Then
Range("B" & j).Value = itemsAttributesEle.innerText 'Description
Range("E" & j).Value = itemsAttributesEle.getAttribute("href") 'Link
End If
If itemsAttributesEle.className = "productprice" Then
Range("C" & j).Value = itemsAttributesEle.Children(0).innerText 'recRP
Range("D" & j).Value = itemsAttributesEle.Children(1).innerText 'Outlet current price
End If
Next itemsAttributesEle
j = j + 1
Skip:
Next itemsEle
On Error GoTo 0
Next pn
ie.Quit
Set itemsColl = Nothing
Set itemsEle = Nothing
Set itemsAttributesColl = Nothing
Set itemsAttributesEle = Nothing
Set Doc = Nothing
Set ie = Nothing
End Sub

Scraping from VBA - scraper already works on some data but not others

I have used code from this website to pull data from property site Foxtons, the code is as follows
Sub foxtons()
Const READYSTATE_COMPLETE = 4
Dim j
Dim xcolElements
Dim Doc, ell, ie
Dim pn
j = 1
For pn = 1 To 10
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = False
ie.Navigate "http://www.foxtons.co.uk/search?current_page=" & pn & "&location_ids=288&search_form=map&search_type=SS&sold=1&submit_type=search"
Do
DoEvents
Loop Until Not ie.Busy And ie.readyState = READYSTATE_COMPLETE
Set Doc = ie.Document
Set xcolElements = Doc.getElementsByClassName("description")
'Here lies the problem. Description works perflectly, price doesn't
For Each ell In xcolElements
Range("B" & j).Value = ell.FirstChild.Data
j = j + 1
Next
Next pn
ie.Quit
Set el = Nothing
Set xcolElements = Nothing
Set Doc = Nothing
Set ie = Nothing
End Sub
The above code works and should pull in 600 property descriptions into column B. The problem arises when I switch "descriptions" to price. Here I receive an error message yet this is probably much more important for me
Use this code to retrieve the price elements:
Private Sub CommandButton1_Click()
Const READYSTATE_COMPLETE = 4
Dim j As Integer
Dim ie As InternetExplorer
Dim Doc As IHTMLDocument
Dim xcolElements As IHTMLElementCollection
Dim ell As IHTMLElement
Dim pn As Integer
j = 1
Set ie = CreateObject("InternetExplorer.Application")
For pn = 1 To 10
ie.Visible = True
ie.Navigate "http://www.foxtons.co.uk/search?current_page=" & pn & "&location_ids=288&search_form=map&search_type=SS&sold=1&submit_type=search"
Do
DoEvents
Loop Until Not ie.Busy And ie.readyState = READYSTATE_COMPLETE
Set Doc = ie.Document
'Set xcolElements = Doc.getElementsByClassName("description")
'Here lies the problem. Description works perflectly, price doesn't
Set xcolElements = Doc.getElementsByClassName("price")
For Each ell In xcolElements
On Error GoTo Skip
Range("B" & j).Value = ell.Children(0).Children(1).innerText
j = j + 1
Skip:
Next ell
On Error Goto 0
Next pn
ie.Quit
Set el = Nothing
Set xcolElements = Nothing
Set Doc = Nothing
Set ie = Nothing
End Sub
Good luck finding your new home :)