Hi I need help with this code I'm trying to extract data from this page https://finance.yahoo.com/quote/ADM.L/balance-sheet?p=ADM.L ,
but the problem is page is by default set to annual but I need quarterly values of total assets and total liabilities.
This code runs but most of the time it is picking annual values. Please suggest something what can I do.
Private Sub CommandButton3_Click()
'
Dim ie As Object
Set Rng = Range("A2:A50")
Set Row = Range(Rng.Offset(1, 0), Rng.Offset(1, 0).End(xlDown))
Set ie = CreateObject("InternetExplorer.Application")
With ie
'.Visible = False
For Each Row In Rng
.navigate "https://finance.yahoo.com/quote/" & Range("A" & Row.Row).Value & "/balance-sheet?p=" & Range("A" & Row.Row).Value
'Application.Wait (Now + TimeValue("0:00:02"))
While ie.readyState <> 4
Wend
Do While ie.Busy: DoEvents: Loop
Dim doc As HTMLDocument
Set doc = ie.document
doc.getElementsByClassName("P(0px) M(0px) C($actionBlue) Bd(0px) O(n)")(2).Click
Do While ie.Busy: DoEvents: Loop
Application.Wait (Now + TimeValue("0:00:05"))
Range("D" & Row.Row).Value = doc.getElementsByClassName("Fw(b) Fz(s) Ta(end)")(4).innerText
Range("E" & Row.Row).Value = doc.getElementsByClassName("Fw(b) Fz(s) Ta(end)")(12).innerText
Range("F" & Row.Row).Value = doc.getElementsByClassName("C($gray) Ta(end)")(0).innerText
Next Row
End With
ie.Quit
'
End Sub
This should be a good start for you to get going.
Sub DownloadData()
Set ie = CreateObject("InternetExplorer.application")
With ie
.Visible = True
.navigate "https://finance.yahoo.com/quote/ADM.L/balance-sheet?p=ADM.L"
' Wait for the page to fully load; you can't do anything if the page is not fully loaded
Do While .Busy Or _
.readyState <> 4
DoEvents
Loop
Set e = ie.Document.GetElementsByClassName("Fz(s) Fw(500) D(ib) Pend(15px) H(18px) C($finDarkLink):h Mend(15px)")(1)
e.Click
' Wait for the page to fully load; you can't do anything if the page is not fully loaded
Do While .Busy Or _
.readyState <> 4
DoEvents
Loop
End With
End Sub
Basically, 'Annual' is the default and you have to click the 'Quarterly' link, to get the quarterly data displayed. I believe yahoo used to have 2 different URLs for Annual and Quarterly. Now, apparently, they give you 2 links to click to toggle back and forth between the 2 frequencies of financial statements.
Do find below on the code fix. Do take note that Yahoo has updated the classname i.e. from "P(0px) M(0px) C($actionBlue) Bd(0px) O(n)" ==> "P(0px) M(0px) C($c-fuji-blue-1-b) Bd(0px) O(n)".
Private Sub CommandButton3_Click()
Dim ie As Object
Set Rng = Range("A2:A50")
Set row = Range(Rng.Offset(1, 0), Rng.Offset(1, 0).End(xlDown))
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
For Each row In Rng
.navigate "https://finance.yahoo.com/quote/" & Range("A" & row.row).Value & "/balance-sheet?p=" & Range("A" & row.row).Value
While ie.readyState <> 4
Wend
Do While ie.Busy: DoEvents: Loop
Dim doc As HTMLDocument
Set doc = ie.document
Set element = doc.getElementsByClassName("P(0px) M(0px) C($c-fuji-blue-1-b) Bd(0px) O(n)")(2)
element.Click
Do While ie.Busy: DoEvents: Loop
Range("D" & row.row).Value = doc.getElementsByClassName("Fw(b) Fz(s) Ta(end)")(4).innerText
Range("E" & row.row).Value = doc.getElementsByClassName("Fw(b) Fz(s) Ta(end)")(12).innerText
Range("F" & row.row).Value = doc.getElementsByClassName("C($gray) Ta(end)")(0).innerText
Next row
End With
ie.Quit
End Sub
It is not possible to extract Quarterly data from Yahoo using VBA.
Annual data can be extracted, but not quarterly.
Related
I am working on a website data extractor. I have two worksheets one for input and other for output, which looks like this..
In the first sheet the cell contains the URL needed to extract data. I am trying this URL
https://www.propertyguru.com.sg/singapore-property-listing/property-for-sale?limit=30&market=residential&property_type_code%5B%5D=4S&property_type=H&freetext=Yishun
I have written this macro..
Sub extractTablesData()
Dim IE As Object, obj As Object
Dim str, e As String
Dim pgf, pgt, pg As Integer
Dim ele, Results As Object
Dim add, size, cno, price, inurl, sp, sp1 As String
Dim isheet, rts As Worksheet
Dim LastRow As Long
Dim pgno As Variant
Set IE = CreateObject("InternetExplorer.Application")
Set isheet = Worksheets("InputSheet")
Set rts = Worksheets("Results")
URL = isheet.Cells(3, 2)
RowCount = 1
rts.Range("A" & RowCount) = "Address"
rts.Range("B" & RowCount) = "Size"
rts.Range("C" & RowCount) = "Contact Number"
rts.Range("D" & RowCount) = "Price"
rts.Range("E" & RowCount) = "Url"
LastRow = rts.Cells(Rows.Count, 2).End(xlUp).Row
'RowCount = LastRow
With IE
.Visible = True
.navigate (URL)
DoEvents
Do While IE.Busy Or IE.readyState <> 4
Loop
'Application.Wait (Now + #12:00:05 AM#)
For Each Results In .document.all
Select Case Results.className
Case "title search-title"
str = Results.innerText
str1 = Split(str, " ")
str = CInt(str1(0))
End Select
If Results.className = "btn btn-main-inverted dropdown-toggle" And InStr(1, Results.Title, " page") > 2 Then
str2 = Results.Title
str1 = Split(str2, " ")
str2 = CInt(str1(0))
End If
Next
If str2 = 0 Then
pgno = CVErr(xlErrDiv0)
Else
pgno = WorksheetFunction.RoundUp(str / str2, 0)
End If
End With
IE.Quit
Set IE = Nothing
UrlS = Split(URL, "?")
Url1 = UrlS(0)
Url2 = "?" & UrlS(1)
For i = 1 To pgno
Set IE = CreateObject("InternetExplorer.Application")
URL = Url1 & "/" & i & Url2
With IE
.Visible = True
.navigate (URL)
DoEvents
Do While IE.Busy Or IE.readyState <> 4
Loop
'Application.Wait (Now + #12:00:08 AM#)
For Each ele In .document.all
Select Case ele.className
Case "listing-img-a"
inurl = ele.href
rts.Cells(LastRow + 1, 5) = inurl
Case "listing-location"
LastRow = LastRow + 1
add = ele.innerText
rts.Cells(LastRow, 1) = add
Case "lst-sizes"
sp = Split(ele.innerText, " ·")
size = sp(0)
rts.Cells(LastRow, 2) = size
Case "pgicon pgicon-phone js-agent-phone-number" ' btn-tools" 'pgicon pgicon-phone js-agent-phone-number" 'agent-phone-number"
rts.Cells(LastRow, 3) = ele.innerText
Case "listing-price"
price = ele.innerText
rts.Cells(LastRow, 4) = price
End Select
Next
LastRow = rts.Cells(Rows.Count, 2).End(xlUp).Row
rts.Activate
rts.Range("A" & LastRow).Select
End With
IE.Quit
Set IE = Nothing
Application.Wait (Now + #12:00:04 AM#)
Next i
MsgBox "Success"
End Sub
When I run this macro I am getting the error
Type Miss Match
When I debug it highlights the code
For i = 1 To pgno
Set IE = CreateObject("InternetExplorer.Application") URL = Url1 & "/" & i & Url2
With IE .Visible = True .navigate (URL)
I have tried my best to figure it out but could not understand where the problem is. Please help me to make correction..
It is also not getting the whole records on the link. This link contains more than 200 Records as per page is 30 records.
You can rely on implicit conversion and use the following. Assuming all pages do have numbering. You might want to improve error handling. I default to page numbers = 1 if the penultimate li CSS selector fails, otherwise it attempts to get the last page number before the ">"
Refer to my prior answer to your related question which shows you how to more effiently scrape the info off the page.
Sample code to show function being used:
Option Explicit
Public Sub GetListings()
Dim IE As New InternetExplorer, pgno As Long
With IE
.Visible = True
.navigate "https://www.propertyguru.com.sg/singapore-property-listing/property-for-sale?limit=30&market=residential&property_type_code%5B%5D=4S&property_type=H&freetext=Yishun", False
While .Busy Or .readyState < 4: DoEvents: Wend
pgno = GetNumberOfPages(.document)
End With
End Sub
Public Function GetNumberOfPages(ByVal doc As HTMLDocument) As Long
On Error GoTo errhand:
GetNumberOfPages = doc.querySelector(".listing-pagination li:nth-last-child(2)").innerText
Exit Function
errhand:
If Err.Number <> 0 Then GetNumberOfPages = 1
End Function
The below code opens an instance of InternetExplorer and downloads odds. It works fine but occasionally a pop-up window appears which causes the code to not work. Any help on how to navigate the below pop-up (i.e. click 'continue to oddschecker') when the pop-up does appear?
<a class="continue beta-callout js-close-class" onclick='s_objectID="javascript:void(0)_9";return this.s_oc?this.s_oc(e):true' href="javascript:void(0)">Continue to Oddschecker</a>
Full code:
Sub Oddschecker()
Dim ie, wp As Object
Dim i As Integer
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = False
ie.Navigate "https://www.oddschecker.com/horse-racing/racing-coupon"
Do While ie.Busy
DoEvents
Loop
Do While ie.ReadyState <> 4
DoEvents
Loop
Set wp = ie.Document
'Application.ActiveSheet.UsedRange.ClearContents
Application.Worksheets("sheet1").UsedRange.ClearContents
i = 2
For Each rw In wp.getElementsByTagName("table")(0).getElementsByTagName("tr")
If rw.className = "date" Then
Worksheets("sheet1").Range("A1") = rw.innerText
ElseIf rw.className = "fixture-name" Then
i = i + 1
Worksheets("sheet1").Range("A" & i) = rw.getElementsByTagName("td")(0).innerText
i = i + 1
ElseIf rw.className = "coupons-table-row match-on" Then
For Each od In rw.getElementsByTagName("p")
If InStr(od.innerText, "(") <> 0 Then
Worksheets("sheet1").Range("A" & i) = Trim(Left(od.innerText, InStr(od.innerText, "(") - 1))
np = Trim(Right(od.innerText, Len(od.innerText) - InStr(od.innerText, "(")))
Worksheets("sheet1").Range("B" & i) = Left(np, Len(np) - 1)
i = i + 1
Else
Worksheets("sheet1").Range("A" & i) = Trim(od.innerText)
i = i + 1
End If
Next od
End If
Next rw
ie.Quit
Range("A1:B" & i).WrapText = False
Columns("A:B").EntireColumn.AutoFit
Set wp = Nothing
Set ie = Nothing
End Sub
If you wish to continue with that page (navigating to that popup page), you can try like:
Dim HTML As HTMLDocument, addcheck As Object
While IE.Busy = True Or IE.readyState < 4: DoEvents: Wend ''(You can write it the way you feel comfortable)
Set HTML = IE.document ''place this line after the prevous line
Set addcheck = HTML.querySelector("#promo-modal a.continue")
If Not addcheck Is Nothing Then
addcheck.Click
End If
But, that is not a good idea cause it will lead you to some page where you might need to do some activity to get back on this data ridden page.
I suppose you should get rid of that popup blocker by ticking the cross button located on the top right area and continue to do what you are doing:
Dim HTML As HTMLDocument, addcheck As Object
While IE.Busy = True Or IE.readyState < 4: DoEvents: Wend ''(You can write it the way you feel comfortable)
Set HTML = IE.document ''place this line after the prevous line
Set addcheck = HTML.querySelector("#promo-modal span[title='Close")
If Not addcheck Is Nothing Then
addcheck.Click
End If
If I didn't understand what your intention was, do let me know. Thanks.
I have created Macro which gives me all URLs present on any webpages.
We just need to provide the URL and it gives us the all links present in that webpage and paste it in one column
Private Sub CommandButton4_Click()
'We refer to an active copy of Internet Explorer
Dim ie As InternetExplorer
'code to refer to the HTML document returned
Dim html As HTMLDocument
Dim ElementCol As Object
Dim Link As Object
Dim erow As Long
Application.ScreenUpdating = False
'open Internet Explorer and go to website
Set ie = New InternetExplorer
ie.Visible = True
ie.navigate Cells(1, 1)
'Wait until IE is done loading page
Do While ie.READYSTATE <> READYSTATE_COMPLETE
Application.StatusBar = "Trying to go to website…"
DoEvents
Loop
Set html = ie.document
'Display text of HTML document returned in a cell
'Range("A1") = html.DocumentElement.innerHTML
Set ElementCol = html.getElementsByTagName("a")
For Each Link In ElementCol
erow = Worksheets("Sheet4").Cells(Rows.Count, 1).End(xlUp).Offset(1, 0).Row
Cells(erow, 1).Value = Link
Cells(erow, 1).Columns.AutoFit
Next
'close down IE, reset status bar & turn on screenupdating
'Set ie = Nothing
Application.StatusBar = ""
Application.ScreenUpdating = True
ie.Quit
ActiveSheet.Range("$A$1:$A$2752").removeDuplicates Columns:=1, Header:=xlNo
End Sub
Now can anyone will help me to create macro to find particular text from all these URLs present in column and if that text is present then in next column it should print text "text found".
Example if we search text "New" then it should print text "Text found" in next column of the URL.
Thank you.
The key would be the function Instr, if it finds the string "New", it returns the position where it begins, otherwise it returns 0.
i=1
do until trim(Cells(i,1).Value) = vbNullString
if instr(Cells(i,1).Value,"New") then
Cells(i,2).value="Text found"
end if
i=i+1
loop
Similar to above.
Dim a As Variant
a = 2
While Cells(a, 1) <> "" And Cells(a + 1, 1) <> ""
If InStr(Cells(a, 1), "new") = 0 Then
Else
Cells(a, 2) = "Text Found"
End If
a = a + 1
Wend
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 8 years ago.
Improve this question
Thank you in advance for the help.
When I run tickers through the code it stops. This is pulling mutual fund data, so if you want to test the code yourself...I would Use(INDZX, CULAX, ABRZX, TAGBX, PRPFX (Don't use these Mutual funds, they are no good; just for an example)). I literally have to sit by my computer and erase the tickers where the data has already been pulled over so that it can start over again; very time consuming.
Can one of you please help me out.
Let me know if you have further questions on this.
Just to add when it completely breaks, and look at the debug, it highlights the "Do While IE.readystate<> 4: DoEvents: Loop
The other issue I am having is that when there are no tickers left, the code continues to run.
Sub upDown()
Dim IE As Object, Doc As Object, lastRow As Long, tblTR As Object, tblTD As Object,
strCode As String
lastRow = Range("H65000").End(xlUp).Row
Set IE = CreateObject("internetexplorer.application")
IE.Visible = True
last_row = Sheets("Tickers").Range("H1").End(xlDown).Row
ini_row_dest = 1
Sheets("upDown").Select
Sheets("upDown").Range("A1:m10000").ClearContents
Application.ScreenUpdating = True
For i = 1 To lastRow
Application.StatusBar = "Updating upDown" & i & "/" & last_row
row_dest = ini_row_dest + (i - 1)
strCode = "Tickers" ' Range("A" & i).value
list_symbol = Sheets("Tickers").Range("h" & i)
IE.navigate "http://performance.morningstar.com/fund/ratings-risk.action?t=" & list_symbol
Do While IE.readystate <> 4: DoEvents: Loop
Set Doc = CreateObject("htmlfile")
Set Doc = IE.document
tryAgain:
Set tblTR = Doc.getelementbyid("div_upDownsidecapture").getelementsbytagname("tr")(3)
If tblTR Is Nothing Then GoTo tryAgain
On Error Resume Next
j = 2
For Each tblTD In tblTR.getelementsbytagname("td")
tdVal = Split(tblTD.innerText, vbCrLf)
Cells(i, j) = tdVal(0)
Cells(i, j + 1) = tdVal(1)
j = j + 2
Next
Sheets("upDown").Range("A" & row_dest).Value = list_symbol
Next i
Range("A3").Select
Application.StatusBar = False
Application.Calculation = xlAutomatic
End Sub
From your description, when it's 'stuck' you press CTRL-Break, and it stops at
Do While IE.readystate<> 4: DoEvents: Loop
This means that IE is busy. You should probably work out why. What happens if you switch to the IE window? Maybe it has a popup? It's entirely likely that morningstar.com has detected that you are scraping data and is halting it. Normally you need to pay some kind of a subscription to get this kind of thing.
Anyway what you could do is put in a 'watchdog' that detects this state and tries to recover. Here is some code below but it is basically a hack and I don't quite understand how your row index is meant to work. The code below uses Goto which is just a lazy way of doing things but it is certainly no worse than the existing code.
Anyway try it and see. What you might find is that the IE.Quit line might prompt you to close IE, but at least it can restart from where it failed and you don't need to clear the tickers out and start again.
An alternative solution might be to save the half finished workbook and alter the code to pick up from where it left off based on which tickers have data and which don't
Sub upDown()
Dim IE As Object, Doc As Object, lastRow As Long, tblTR As Object, tblTD As Object,
strCode As String
Dim iWatchDog as Integer
iWatchDog = 1
lastRow = Range("H65000").End(xlUp).Row
ini_row_dest = 1
Sheets("upDown").Select
Sheets("upDown").Range("A1:m10000").ClearContents
Start:
Set IE = CreateObject("internetexplorer.application")
IE.Visible = True
last_row = Sheets("Tickers").Range("H1").End(xlDown).Row
Application.ScreenUpdating = True
For i = 1 To lastRow
Application.StatusBar = "Updating upDown" & i & "/" & last_row
row_dest = ini_row_dest + (i - 1)
strCode = "Tickers" ' Range("A" & i).value
list_symbol = Sheets("Tickers").Range("h" & i)
IE.navigate "http://performance.morningstar.com/fund/ratings-risk.action?t=" & list_symbol
Do While IE.readystate <> 4
DoEvents
DoEvents
DoEvents
DoEvents
DoEvents
iWatchDog = iWatchDog + 1
If iWatchDog >= 10000 Then
Application.StatusBar = "Stuck - resetting"
iWatchDog = 1
IE.Stop
IE.Quit
Set IE = Nothing
DoEvents
DoEvents
DoEvents
DoEvents
Goto Start
End If
Loop
Set Doc = CreateObject("htmlfile")
Set Doc = IE.document
tryAgain:
Set tblTR = Doc.getelementbyid("div_upDownsidecapture").getelementsbytagname("tr")(3)
If tblTR Is Nothing Then GoTo tryAgain
On Error Resume Next
j = 2
For Each tblTD In tblTR.getelementsbytagname("td")
tdVal = Split(tblTD.innerText, vbCrLf)
Cells(i, j) = tdVal(0)
Cells(i, j + 1) = tdVal(1)
j = j + 2
Next
Sheets("upDown").Range("A" & row_dest).Value = list_symbol
Next i
Range("A3").Select
Application.StatusBar = False
Application.Calculation = xlAutomatic
End Sub
Where is this 3,800 lines of ticker data eventually going? into a database or is it fed into another Excel sheet?
So I'm looking to dump a bunch of URLs from a webpage into excel as a list. I was previously dumping the items into a listbox, but I have found that listboxes are quite difficult to work with!
Once I have collected the URLs into a column in excel, I want excel to click on each link and find the email address that is on the page. Here is the coding that I currently have...
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
url_name = Sheet1.Range("A2")
If url_name = "" Then Exit Sub
IE.Navigate (url_name)
Do
DoEvents
Loop Until IE.ReadyState = 4
Set AllHyperLinks = IE.Document.GetElementsByTagName("A")
For Each hyper_link In AllHyperLinks
Range("x":"F").Value = hyper_link
This is all I have so far! I'm not sure how to complete the loop! I want the code to paste every new URL that it finds on the page in the next empty row in column F.
You can complete the loop in this way:
Dim IE As Object, LR As Long, i As Long
LR = Sheet1.Range("A" & Sheet1.Rows.Count).End(xlUp).Row
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
For i = 2 To LR
url_name = Sheet1.Range("A" & i).Value
If url_name = "" Then Exit Sub
IE.Navigate (url_name)
Do
DoEvents
Loop Until IE.ReadyState = 4
Set AllHyperLinks = IE.Document.GetElementsByTagName("A")
For Each hyper_link In AllHyperLinks
Range("x":"F").Value = hyper_link
Next hyper_link
Next i
Please note that if you have large set of data, this is going to take a LOOOONGGGG time.