IE button click with VBA - vba

Can't get the "GO" button to click via VBA on this site: https://finra-markets.morningstar.com/BondCenter/BondDetail.jsp?ticker=C631551&symbol=RDS4242315
Will eventually want to loop code. Should be simple...just can't get this one.
Sub Macro1()
'we define the essential variables
Dim ie As Object
Dim acct
Dim button
Set Rng = Range("B4:B4")
Set Row = Range(Rng.Offset(1, 0), Rng.Offset(1, 0).End(xlDown))
For Each Row In Rng
'add the "Microsoft Internet Controls" reference in your VBA Project indirectly
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = True
.navigate ("https://finra-markets.morningstar.com/BondCenter/BondDetail.jsp?ticker=C631551&symbol=RDS4242315")
While ie.ReadyState <> 4
DoEvents
Wend
Set Cusip = .document.getElementById("ms-finra-autocomplete-box") 'id of the username control (HTML Control)
Cusip.Value = Range("B" & Row.Row).Value
ie.document.getElementsByTagName("submit").Click
End With
Next Row
End Sub

The tag name of your button is not "submit" but "INPUT" ... "submit" is the type.
But watch out, there are more INPUT elements, so your getElementsBy... will return a collection and you need to further dig to find the correct one, e.g. by checking a significant attribute.
Example
' ...
Set ECol = ie.document.getElementsByTagName("input")
For Each IFld In ECol
If IFld.getAttribute("class") = "button_blue autocomplete-go" Then
IFld.Click
Exit For
End If
Next IFld
' ...

Related

Can't assign text to cell with Excel VBA

I'm trying to scrape zip codes from Google. I've been trying to put innertext into a cell, but I think I may be getting a variable mismatch on 2nd to last line.
'This Must go at the top of your module. It's used to set IE as the active window
Sub Automate_IE_Enter_Data()
'This will load a webpage in IE
Dim i As Long
Dim URL As String
Dim IE As Object
Dim objElement As Object
Dim objCollection As Object
Dim HWNDSrc As Long
Dim adds As Variant, add As Variant
Dim addt As String
'Create InternetExplorer Object
Set IE = CreateObject("InternetExplorer.Application")
'Set IE.Visible = True to make IE visible, or False for IE to run in the background
IE.Visible = True
'Define URL
URL = "https://www.google.com/search?ei=djKhW7nELYqs8AO96baoAw&q=1000 Westover Rd kansas city, Mo"
'Navigate to URL
IE.Navigate URL
' Statusbar let's user know website is loading
Application.StatusBar = URL & " is loading. Please wait..."
' Wait while IE loading...
'IE ReadyState = 4 signifies the webpage has loaded (the first loop is set to avoid inadvertantly skipping over the second loop)
Do While IE.ReadyState = 4: DoEvents: Loop
Do Until IE.ReadyState = 4: DoEvents: Loop
'Webpage Loaded
Application.StatusBar = URL & " Loaded"
'Get Window ID for IE so we can set it as activate window
HWNDSrc = IE.Hwnd
'Set IE as Active Window
'SetForegroundWindow HWNDSrc
Debug.Print "ihgc"
'Unload IE
endmacro:
Set adds = IE.Document.getElementsbyClassName("desktop-title-subcontent")
For Each add In adds
Debug.Print add.innertext
Next
Cells(2, f).Value = add.innertext
End Sub
Couple of things. First and foremost, your loop is unnecessary. I ran your code, and there's nothing to loop. Even if it was necessary, it's being used improperly.
So, in assuming that you in fact do not need a For...Next loop, then you can use the index number of 0 for your collection of IE.Document.getElementsbyClassName("desktop-title-subcontent"), then set your cell reference equal to the innerText property of that collection item.
This brings me to the next issue, your cell reference. Cells(2, f), the f is not a declared variable. If you where actually wanting to use the column "F", then you need to enclose 'F' in double quotes:
Cells(2, "F") or use the column's index of 6, Cells(2, 6)
So, replace this entire portion:
Set adds = IE.Document.getElementsbyClassName("desktop-title-subcontent")
For Each add In adds
Debug.Print add.innertext
Next
Cells(2, f).Value = add.innertext
with this:
Cells(2, "F").Value = IE.Document.getElementsByClassName _
("desktop-title-subcontent")(0).innerText
OPTIONAL
And lastly, I would look into using Early Binding over late binding. It has many advantages, with a possible notable speed improvement.
You would need to set a reference to Microsoft Internet Controls and declare IE as type InternetExplorer vs Object. But that's not going to make or break your code.

Crawler & Scraper using excel vba

I am trying to crawl in an intranet URL, so I can get the excel automatically select one of the options from a dropdown menu, then enter a value in a text box, then click on Find to get redirected to another page, where I want to get a value copy to another worksheet in the same workbook, I have created the below, but the code is not working, saying object required. :(
Sub Test()
Dim rng As Range
Set rng = Sheets("sheet1").Range("A1", Sheets("sheet1").Cells.Range("A1").End(xlDown))
Set ie = CreateObject("InternetExplorer.application")
ie.Visible = True
ie.Navigate ("https://gcd.ad.plc.cwintra.com/GCD_live/login/login.asp")
Do
If ie.ReadyState = 4 Then
ie.Visible = False
Exit Do
Else
DoEvents
End If
Loop
ie.Document.forms(0).all("txtUsername").Value = ""
ie.Document.forms(0).all("txtPassword").Value = ""
ie.Document.forms(0).submit
ie.Visible = True
Appliction.Wait (Now + TimeValue("00:00:02"))
DoEvents
For Each cell In rng
ie.Navigate ("https://gcd.ad.plc.cwintra.com/GCD_live/search.asp")
DoEvents
ie.Document.getElementById("cboFieldName").selectedIndex = 6
ie.Document.getElementById("txtFieldValue").Select
SendKeys (cell.Value)
DoEvents
ie.Document.getElementById("cmdFind").Click
Next cell
End Sub

how to continue VBA code after opening a new web page

I'm new to creating VBA code and I'm slowly getting a basic understanding of it, however I'm unable to pass this point of my project without assistance. I have the code below and runs great up until I need to continue the code with the new page that opens. I have no idea on how to be able to continue the code and the plan is to be able to click on the odds comparison tab and extract data from that page. Any assistance would be much appreciated.
Sub odd_comparison()
Dim objIE As InternetExplorer
Dim ele As Object
Dim y As Integer
Set objIE = New InternetExplorer
objIE.Visible = True
objIE.navigate "http://www.flashscore.com/basketball/"
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
objIE.document.getElementById("fs").Children(0) _
.Children(2).Children(2).Children(0).Children(2).Click
End Sub
Try to make loop until the webpage ready as described in this and this answers (you know, replace WScript.Sleep with DoEvents for VBA).
Inspect the target element on the webpage with Developer Tools (using context menu or pressing F12). HTML content is as follows:
bwin.fr Odds
As you can see there is onclick attribute, and actually you can try to execute jscript code from it instead of invoking click method:
objIE.document.parentWindow.execScript "setNavigationCategory(4);pgenerate(true, 0,false,false,2); e_t.track_click('iframe-bookmark-click', 'odds');", "javascript"
Going further you can find the following spinner element, which appears for the short time while data is being loaded after the tab clicked:
<div id="preload" class="preload pvisit" style="display: none;"><span>Loading ...</span></div>
So you can detect when the data loading is completed by checking the visibility state:
Do Until objIE.document.getElementById("preload").style.display = "none"
DoEvents
Loop
The next step is extracting the data you need. You can get all tables from central block: .document.getElementById("fs").getElementsByTagName("table"), loop through tables and get all rows oTable.getElementsByTagName("tr"), and finally get all cells .getElementsByTagName("td") and innerText.
The below example shows how to extract all table data from the webpage odds comparison tab to Excel worksheet:
Option Explicit
Sub Test_Get_Data_www_flashscore_com()
Dim aData()
' clear sheet
Sheets(1).Cells.Delete
' retrieve content from web site, put into 2d array
aData = GetData()
' output array to sheet
Output Sheets(1).Cells(1, 1), aData
MsgBox "Completed"
End Sub
Function GetData()
Dim oIE As Object
Dim cTables As Object
Dim oTable As Object
Dim cRows As Object
Dim oRow As Object
Dim aItems()
Dim aRows()
Dim cCells As Object
Dim i As Long
Dim j As Long
Set oIE = CreateObject("InternetExplorer.Application")
With oIE
' navigate to target webpage
.Visible = True
.navigate "http://www.flashscore.com/basketball/"
' wait until webpage ready
Do While .Busy Or Not .readyState = 4: DoEvents: Loop
Do Until .document.readyState = "complete": DoEvents: Loop
Do While TypeName(.document.getElementById("fscon")) = "Null": DoEvents: Loop
' switch to odds tab
.document.parentWindow.execScript _
"setNavigationCategory(4);pgenerate(true, 0,false,false,2); e_t.track_click('iframe-bookmark-click', 'odds');", "javascript"
Do Until .document.getElementById("preload").Style.display = "none": DoEvents: Loop
' get all table nodes
Set cTables = .document.getElementById("fs").getElementsByTagName("table")
' put all rows into dictionary to compute total rows count
With CreateObject("Scripting.Dictionary")
' process all tables
For Each oTable In cTables
' get all row nodes within table
Set cRows = oTable.getElementsByTagName("tr")
' process all rows
For Each oRow In cRows
' put each row into dictionary
Set .Item(.Count) = oRow
Next
Next
' retrieve array from dictionary
aItems = .Items()
End With
' redim 1st dimension equal total rows count
ReDim aRows(1 To UBound(aItems) + 1, 1 To 1)
' process all rows
For i = 1 To UBound(aItems) + 1
Set oRow = aItems(i - 1)
' get all cell nodes within row
Set cCells = aItems(i - 1).getElementsByTagName("td")
' process all cells
For j = 1 To cCells.Length
' enlarge 2nd dimension if necessary
If UBound(aRows, 2) < j Then ReDim Preserve aRows(1 To UBound(aItems) + 1, 1 To j)
' put cell innertext into array
aRows(i, j) = Trim(cCells(j - 1).innerText)
DoEvents
Next
Next
.Quit
End With
' return populated array
GetData = aRows
End Function
Sub Output(objDstRng As Range, arrCells As Variant)
With objDstRng
.Parent.Select
With .Resize( _
UBound(arrCells, 1) - LBound(arrCells, 1) + 1, _
UBound(arrCells, 2) - LBound(arrCells, 2) + 1)
.NumberFormat = "#"
.Value = arrCells
.Columns.AutoFit
End With
End With
End Sub
Webpage odds comparison tab content for me is as follows:
It gives the output:

VBA to find text from webpages

I have created Macro which gives me all URLs present on any webpages.
We just need to provide the URL and it gives us the all links present in that webpage and paste it in one column
Private Sub CommandButton4_Click()
'We refer to an active copy of Internet Explorer
Dim ie As InternetExplorer
'code to refer to the HTML document returned
Dim html As HTMLDocument
Dim ElementCol As Object
Dim Link As Object
Dim erow As Long
Application.ScreenUpdating = False
'open Internet Explorer and go to website
Set ie = New InternetExplorer
ie.Visible = True
ie.navigate Cells(1, 1)
'Wait until IE is done loading page
Do While ie.READYSTATE <> READYSTATE_COMPLETE
Application.StatusBar = "Trying to go to website…"
DoEvents
Loop
Set html = ie.document
'Display text of HTML document returned in a cell
'Range("A1") = html.DocumentElement.innerHTML
Set ElementCol = html.getElementsByTagName("a")
For Each Link In ElementCol
erow = Worksheets("Sheet4").Cells(Rows.Count, 1).End(xlUp).Offset(1, 0).Row
Cells(erow, 1).Value = Link
Cells(erow, 1).Columns.AutoFit
Next
'close down IE, reset status bar & turn on screenupdating
'Set ie = Nothing
Application.StatusBar = ""
Application.ScreenUpdating = True
ie.Quit
ActiveSheet.Range("$A$1:$A$2752").removeDuplicates Columns:=1, Header:=xlNo
End Sub
Now can anyone will help me to create macro to find particular text from all these URLs present in column and if that text is present then in next column it should print text "text found".
Example if we search text "New" then it should print text "Text found" in next column of the URL.
Thank you.
The key would be the function Instr, if it finds the string "New", it returns the position where it begins, otherwise it returns 0.
i=1
do until trim(Cells(i,1).Value) = vbNullString
if instr(Cells(i,1).Value,"New") then
Cells(i,2).value="Text found"
end if
i=i+1
loop
Similar to above.
Dim a As Variant
a = 2
While Cells(a, 1) <> "" And Cells(a + 1, 1) <> ""
If InStr(Cells(a, 1), "new") = 0 Then
Else
Cells(a, 2) = "Text Found"
End If
a = a + 1
Wend

Webbrowser control in userform: how to wait for page to initialize

Task: using Excel VBA to navigate to a website, log in and go to an input page.
On that page, sequentially enter a series of values stored in a column in Sheet1.
What I've done so far:
I create a webbrowser control and navigate to the website and stop.
Then click on a button on Sheet1 with the macros that will do the inputting, stored in a module.
What's happening:
The control comes up nicely and navigates to the intended site. (this is the userform code)
Click on the button and it gets the userid and password from the spreadsheet, inputs them, clicks on the login button and all is well.
However, the next statement is:
Set inputfield = WebBrowser.objWebBrowser.Document.getElementById("ctl02_ctl03_ddlBus")
and inputfield comes up empty.
If I stop execution and step through it, it'll work.
I've tried Application.Wait; For x = 1 to 5000000; On Error Goto/Resume and keep trying, but nothings works.
I've also tried .NavigateComplete, .DocumentCompleted, as well as others, but I get errors saying member is not supported.
I am at my wits end - I'm just so close!! So far, I've spent more time on this that it will ever save, but now it's personal! Thanks for your help.
This is borrowed code from another site that initializes the control.
Private Sub UserForm_Initialize()
Dim a, c As Integer
With Me
.StartUpPosition = 0
.Top = 150
.Left = -700
End With
With Me.objWebBrowser
.Navigate2 "http://www.schoolbuscity.com/Mapnetweb_47/login.aspx"
.Visible = True
End With
End Sub
Private Sub GetSheets()
'this is my code
Dim inputfield As Object
Dim SendText As String
Dim NumberOfRoutes, r, errCount As Integer
errCount = 0
NumberOfRoutes = Range("NumberOfRoutes")
ReDim RouteNumbers(NumberOfRoutes) As String
For r = 1 To NumberOfRoutes
RouteNumbers(r) = Cells(r, 1).Value
Next r
' Sheets("Sheet1").Select
Range(Cells(5, 2), Cells(6, 2)).ClearContents ' this indicates success for the chosen cells
SendText = Range("userid").Value
Set inputfield = WebBrowser.objWebBrowser.Document.getElementById("Login1_UserName")
inputfield.Value = SendText
SendText = Range("password").Value
Set inputfield = WebBrowser.objWebBrowser.Document.getElementById("Login1_Password")
inputfield.Value = SendText
Set inputfield = WebBrowser.objWebBrowser.Document.getElementById("Login1_Login")
Application.Wait (Now + TimeValue("0:00:01"))
inputfield.Click
Application.Wait (Now + TimeValue("0:00:01")) ' I've tried waiting for up to 10 seconds
Set inputfield = Nothing
On Error GoTo TryAgain
For r = 5 To 6 'NumberOfRoutes ' just want to use 2 loops for testing
' this is where is fails, I believe, because the page is not initialized
' but if waiting is not the answer, then what is?
Set inputfield = WebBrowser.objWebBrowser.Document.getElementById("ctl02_ctl03_ddlBus")
inputfield.Value = RouteNumbers(r)
Set inputfield = WebBrowser.objWebBrowser.Document.getElementById("ctl02_ctl03_btnGo")
inputfield.Click
Application.Wait (Now + TimeValue("0:00:01"))
Cells(r, 2).Value = "Sent"
' WebBrowser.objWebBrowser.Document.Print
WebBrowser.objWebBrowser.GoBack
Next r
GoTo EndIt
TryAgain:
Set inputfield = Nothing
Set inputfield = WebBrowser.objWebBrowser.Document.getElementById("ctl02_ctl03_ddlBus")
errCount = errCount + 1
If errCount > 5 Then GoTo EndIt
Resume
EndIt:
If errCount > 0 Then
MsgBox "errCount= " + CStr(errCount)
Else
MsgBox "Did it"
End If
End Sub
This is how to waitbin vbscript.
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = 0
'must navigate to a local file to avoid security prompts
ie.Navigate2 "C:\Users\User\Desktop\Filter.html"
Do
wscript.sleep 100
Loop until ie.document.readystate = "complete"
examples in VBA:
Private Sub UserForm_Initialize()
Set ie = Me.WebBrowser1
ie.Navigate2 "about:blank"
Do Until ie.ReadyState = READYSTATE_COMPLETE
DoEvents
Loop
Set ie = Nothing
End Sub
Private Sub Conectar_Click()
Dim ie As Object
Set ie = Me.WebBrowser1
ie.Navigate2 "http://www.mytest.com"
Do Until ie.ReadyState = READYSTATE_COMPLETE
DoEvents
Loop
'different alternatives
'Dim inputfield As Object
'Set inputfield = ie.Document.getElementById("Login_tbLogin")
'inputfield.Value = "mylogin"
'Set inputfield = Nothing
'ie.Document.getElementById("Login_tbLogin").Value = "mylogin"
'ie.Document.All("Login_tbLogin").Focus
'ie.Document.All("Login_tbLogin").Value = "mylogin"
ie.Document.All.Item("Login_tbLogin").Value = "mylogin"
Set ie = Nothing
End Sub