I am looking to automate internet explorer using Excel VBA to extract football results from a website and am really struggling with getting the data to update when I change the dropdown value.
The website is: http://www.whoscored.com/Regions/250/Tournaments/30/Seasons/3871/Stages/8209/Fixtures/Europe-UEFA-Europa-League-2013-2014
I am looking to change the value of the 'stages' dropdown and scrape the match results.
My code works fine for opening IE, changing the value of the 'scrape' dropdown but I can't get the data to update. Whilst I am comfortable with VBA I know very little about HTML and Javascript although I can guess what some lines are doing. From what I can see there is javascript code that handles the change event, I just can't see how to get it to run - I have tried firing the 'onchange' event in my code as suggested from my searches but I can't get it to work.
This is the code I can see that controls the drop down (I have deleted a lot of the dropdown values for other dropdowns as it made this post even longer:
<div id="breadcrumb-nav">
.
.
<span><select id="stages" name="stages"><option selected="selected" value="/Regions/250/Tournaments/30/Seasons/3871/Stages/8209">Europa League Group Stages</option>
<option value="/Regions/250/Tournaments/30/Seasons/3871/Stages/7816">Europa League Qualification</option>
<option value="/Regions/250/Tournaments/30/Seasons/3871/Stages/8158">Europa League Grp. A</option>
<option value="/Regions/250/Tournaments/30/Seasons/3871/Stages/8159">Europa League Grp. B</option>
.
.
<option value="/Regions/250/Tournaments/30/Seasons/3871/Stages/8466">Europa League</option>
</select></span>
</div>
<script type="text/javascript">
$('#breadcrumb-nav select').change(function () {
NG.GA.trackEvent('BreadcrumbNav', this.id);
window.location.href = this.value;
// TODO: Disable all selects?
});
</script>
my code:
Sub ScrapeData()
Dim ie As InternetExplorer
Dim URL As String
URL = "http://www.whoscored.com/Regions/250/Tournaments/30/Seasons/3871/Stages/8466/Fixtures/Europe-UEFA-Europa-League-2013-2014"
Set ie = New InternetExplorer
ie.Visible = True
ie.navigate (URL)
Do
DoEvents
Loop Until ie.readyState = 4
SelectValue ie, "/Regions/250/Tournaments/30/Seasons/3871/Stages/7816"
SelectValue ie, "/Regions/250/Tournaments/30/Seasons/3871/Stages/8209"
End Sub
Sub SelectValue(ByVal ie As InternetExplorer, ByVal value As String)
Dim htmlDoc As HTMLDocument
Dim ddStages As HTMLSelectElement
Dim idBreadCrumb As Object
Set htmlDoc = ie.document
With ie.document
Set idBreadCrumb = .getelementbyid("breadcrumb-nav")
Set ddStages = .getelementbyid("stages")
End With
ddStages.value = value
ddStages.FireEvent ("onchange")
'fireevent on ddStages didn't work so tried here too
idBreadCrumb.FireEvent ("onchange")
Do
DoEvents
Loop Until ie.readyState = 4
End Sub
Any help would be really appreciated.
There must be some JavaScript executing on the event "the select element has changed its value". My suggestion, much easier than executing the JavaScript, is to just navigate the link (because what the JS does here is just changing the HTML page you are seeing, and not the elements within the same webpage).
So, for example, I would just replace this:
SelectValue ie, "/Regions/250/Tournaments/30/Seasons/3871/Stages/7816"
with this
ie.Navigate "http://www.whoscored.com/" & "/Regions/250/Tournaments/30/Seasons/3871/Stages/7816"
to get the exactly same result.
Related
First off, please forgive me for any mistakes in posting - this is my first time posting (although I'm a frequent visitor!
I'm working on a program to improve productivity within my workgroup. I am attempting to link an Access database to an internal web application (Pega based) that is very complex. Unfortunately, my knowledge of coding is very basic, and I do not have the available funding or bandwidth to obtain formal assistance. While I would love to use add-ins, I cannot, since my company will not allow it (I've asked, begged, pleaded, etc).
This is what the page of the program looks like. I first need to click the Work Manager Tab (WMT) blocked in GREEN, then I need to click the Get Next button (NGN) blocked in RED.
Steps I think need to happen
Bind the correct IE window session
Click on WMT
Click on NGN
I have been able use Access to bind the correct window and tab of IE (thank you to the person who provided the basic code for this!) but have not yet been able to decipher how to target the correct element on the page. The naming structure is a bit too much for my knowledge.
This is a screenshot of the HTML for the WMT (it won't copy it, at least not legibly, even after cleanup). I would love to be able to use the ID for the specific element, however, it's dynamic, and changes with each interaction, which is why I've been trying to target the static pieces, highlighted in green.
I haven't done much work with the second piece, however, This is the screenshot for that, with the button highlighted in yellow, if anyone wants to help there, too.
Thank you!
CODE FOR THE WINDOW BIND (works all the time so far!)
Sub C360WindowFind()
marker = 0
Set objShell = CreateObject("Shell.Application")
IE_count = objShell.Windows.Count
For x = 0 To (IE_count - 1)
On Error Resume Next
my_url = objShell.Windows(x).document.Location
my_title = objShell.Windows(x).document.Title
If my_title Like "Coverage User" & "*" Then
Set C360Window = objShell.Windows(x)
marker = 1
Exit For
Else
End If
Next
If marker = 0 Then
MsgBox ("C360 window is not found. Please ensure C360 is open in Internet Explorer and try again")
Else
'DO THE OTHER THINGS
End If
End Sub
CODE I'VE TRIED TO TARGET CORRECT TAB (Add in place of "DO THE OTHER THINGS" in previous code set)
FIRST TRY (does some crazy stuff that ends up making the program log out)
FindWorkManager1()
Dim C360doc As HTMLDocument
Dim Element
Dim ULTabName As String
Dim LIElement As String
Dim WMT_HREF As String
ULTabName = "yui-nav tab-ul tab-ul-t tab-ul-t-ns subTabsList"
ULElement = "getElementsByClassName"
LIElement = "getElementbyID"
WMT_HREF = "getElementsByTagName"
Set C360doc = C360Window.document
With C360doc.all
If element = ULElement("ULTabName").LIElement("Work Manager").innerText.WMT_HREF("a")(0) Then
element.Click
Else
MsgBox ("Please ensure Work Manager is open, then try again")
End If
End With
End Sub
SECOND TRY: Gets to the lines with stars and quits (tried a couple variations of this with same results)
FindWorkManager2()
Dim C360doc As HTMLDocument
Set C360doc = C360Window.document
TabTagCounter = 0
TabItemCounter = 0
With C360doc.all
Set Header = C360doc.getElementsByClassName("yui-nav tab-ul tab-ul-t tab-ul-t-ns subTabsList")
'Find the correct TabTag within Header
For Each TabTag In Header
If TabTag.tagName Like "UL" Then
TabTagCounter = 1
'If we've identified the correct TabTag, then look at the TabItem
'*************Not currently working beyond this point :(
If TabTagCounter = 1 Then
For Each TabItem In TabTag
If TabItem.innerText Like "*Work Manager*" Then
TabItemCounter = 1
End If
MsgBox ("try again")
Next
Else
End If
End If
Next
End With
End Sub
I think you should try to click the <a> tag as it's a link navigating to #Tab2. If the class yui-nav tab-ul tab-ul-t tab-ul-t-ns subTabsList is the only one, you could locate ul with this class name first, then locate the li and a with tag name.
As the real link is not accessible, so I make a test page like below:
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title></title>
</head>
<body>
<div class="tStrCntr">
<ul class="yui-nav tab-ul tab-ul-t tab-ul-t-ns subTabsList">
<li tabindex="-1"title="Work Manager"></li>
<li tabindex="0" title="Home" id="Tab2">
<a tabindex="-1" id="TABANCHOR" href="https://www.google.com">
<span style="display: inline-block">
<label>Work Manager</label>
</span>
</a>
</li>
</ul>
</div>
</body>
</html>
And the VB code to click Work Manager in the test page is like below, you could have a test:
Updated version:
Sub LOADIE()
Dim link As Object
Set ieA = CreateObject("InternetExplorer.Application")
ieA.Visible = True
ieA.navigate "http://the test web page"
Do Until ieA.readyState = 4
DoEvents
Loop
Set doc = ieA.Document
Set ElementCol = doc.getElementsByClassName("yui-nav tab-ul tab-ul-t tab-ul-t-ns subTabsList")(0).getElementsByTagName("li")
For Each link In ElementCol
If link.getAttribute("title") = "Work Manager" Then
link.getElementsByTagName("a").Item(0).Click
End If
Next link
End Sub
I am very new to VBA and had a question regarding how to click an href link in Internet Explorer. There are multiple href's on the source page. I have never encountered this and it has been giving me a hard time! I have looked on this website searching for answers but decided to ask here.
Below I have listed the code I have, up to the point where I encounter the problem, as well as the Source Code on Internet Explorer.
I commented out what I have tried and listed the error I received.
Code Below:
Sub ()
Dim i As Long
Dim URL As String
Dim IE As Object
Dim objElement As Object
Dim objCollection As Object
User = "User"
Pwd = "Pwd"
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
URL = "URL.com"
IE.Navigate URL
Do While IE.ReadyState <> 4
DoEvents
Loop
IE.Document.getElementById("txtUsername").Value = User
IE.Document.getElementById("txtPassword").Value = Pwd
IE.Document.getElementById("btnSubmit").Click
'IE.getElementByClassName("txtTerms").Click - Runtime Error 438
'IE.getElementByTagName("Claims Management").Click - Runtime Error 438
'Set HREF = IE.Document.getElementsByClassName("txtTerms")
'For Each HREF In IE.Document.getElementsByTagName("Claims").Click - No error occurs, nothing happens.
End Sub
Internet Explorer Source Code:
<table id="tblContent">
<tr>
<td class="txtTerms"><a href='href url 1'>Claims</a>
<br>Download<br>Create<br><a class='terms' href='href url 2'
target='terms'>Terms</a><br><br></td>
</tr>
My question would be, how to get VBA to click only on 'href url 1'?
Let me know if any additional information is needed. I apologize for my level of VBA but I am excited to learn more!
Thanks for the help!
In HTML, href is a property of the type <a> (link) which contains an absolute or relative path.
For example:
Questions
... will show as "Questions" and, if you click it, will bring you to www.stackoverflow.com/questions/. Note that "www.stackoverflow.com" has been added automatically since the path is relative.
Facebook
... will show as "Facebook" and, if you click it, will bring you to www.facebook.com. In this case, the path is absolute.
Although your HTML code is incomplete, I guess that all the links you want to navigate are contained in the table having id="tblContent". If that's the case, then you can get all the links (tagName == 'a') in that table and store the values in a collection:
Dim allHREFs As New Collection
Set allLinks = IE.Document.getElementById("tblContent").getElementsByTagName("a")
For Each link In allLinks
allHREFs.Add link.href
Next link
You can then decide to navigate them and do what you have to do one by one:
For j = 1 To allHREFs.Count
IE.Navigate URL + allHREFs(j) '<-- I'm assuming hrefs are relative.
'do your stuff here
Next href
I am using Excel VBA to try click a button on a site, here's the code from the site using inspect element:
<button class="_ah57t _84y62 _frcv2 _rmr7s">ClickHere</button>
And here's what i'm doing in VBA:
Sub testcode()
Dim ie As InternetExplorer
Dim html As HTMLDocument
Set ie = New InternetExplorer
ie.Visible = True
ie.Navigate "somesite.com"
Do While ie.READYSTATE <> READYSTATE_COMPLETE
DoEvents
Loop
Dim e
Set e = ie.Document.getElementsByClassName("_ah57t _84y62 _frcv2 _rmr7s")
e.Click
End Sub
Using the debug I found that the code seems to be storing something called "[object]" in the variable e and and then gives a Runtime error '438' when it gets to e.click. I have even tried using .Focus first, but get the same error. Any ideas?
The getElementsByClassName() function returns a collection not a single element. You need to specify an index on the returned collection in order to return a single element. If there is only one element within the class you can simply use:
ie.Document.getElementsByClassName("_ah57t _84y62 _frcv2 _rmr7s")(0).Click
The (0) specifies the index of the element within the collection returned from the class.
Its easy to tell whether a function returns a collection or single element:
getElementBy... - Returns a single element.
getElementsBy... - Returns a collection of elements.
I have created a ribbon for Powerpoint with visual studio XML ribbon. This ribbon has a button that, simplifying, does this:
opens an IE browser
search an element (hiddenfield) in the code by his id
get the value of this element
Print the value in the actual slide
It works correctly the first time I click the button of my ribbon, but it throws an Exception 0x800A01B6 the following times I click the button.
This is the code executed when I click the button:
Dim oType As Type = Type.GetTypeFromProgID("InternetExplorer.Application")
If oType IsNot Nothing Then
Dim ie As SHDocVw.InternetExplorer
ie = Nothing
ie = TryCast(Activator.CreateInstance(oType), SHDocVw.InternetExplorer)
If ie IsNot Nothing Then
Dim oEmpty As Object = [String].Empty
Dim oURL As Object = targetURL
ie.AddressBar = False
ie.MenuBar = False
ie.ToolBar = 0
ie.Visible = True
ie.Height = 800
ie.Width = 1100
ie.Navigate(oURL, oEmpty, oEmpty, oEmpty, oEmpty)
End If
Do While (ie.Busy Or ie.ReadyState <> READYSTATE.READYSTATE_COMPLETE)
Sleep(1000)
Application.DoEvents()
Loop
Sleep(10000) ' 10 seconds for testing purpose
Dim str As String = String.Empty
Dim hdnstring As HTMLInputElement = ie.Document.getElementById("hdnstring")
str = hdnstring.value
DoSomething(str)
ie.Quit()
ie = Nothing
End If
This is the code of the website that opens (targetURL), the code remains identical in every load and only the hidden value changes:
<html>
<body>
<form name="form1" id="form1">
<input type="hidden" name="hdnstring" id="hdnstring" value="Get This String" />
</form>
</body>
</html>
The second time (and following) I execute the function: the IE opens, the website fully loads, it waits 10 seconds and then I get an error in the line:
Dim hdnstring As HTMLInputElement = ie.Document.getElementById("hdnstring")
with Exception 0x800A01B6 message.
The most strange thing is that if I click viewsource in the IE contextual menu while the 10 seconds delay (the ones for testing purpose), it works perfect every time I click the button; but if I don't, the Exception 0x800A01B6 appers.
Any idea of what I'm doing wrong?
Error details image:
The type of the Document property is only resolved at run-time, so it's an Object until then. This is why calling any methods in it results in the so-called late binding - you do not yet know if the getElementById method exists or not, so that has to be determined a run-time.
You most likely get the error because the Document is not of the IHTMLDocument3 type, which is the only document type that includes the getElementById method.
What you can try is casting the Document to an IHTMLDocument3 interface. Since it inherits IHTMLDocument and IHTMLDocument2 you can cast between them even if the document is actually one of the earlier types.
DirectCast(ie.Document, IHTMLDocument3).getElementById("hdnstring")
I am trying to write VBA codes to pull the price of a product from a website. In order to this, I turned on the "Microsoft HTML Object Library" and "Microsoft Internet Controls" in VBA References. However, when I get up to the point to search the of the item that attaches the price, the codes failed. Appreciate if anyone can provide a solution for it.
Below is the link to the sample webpage that I would like to copy price from.
Link
Below is my initial codes:
Sub Update()
Dim IE As New InternetExplorer
IE.Visible = False
IE.navigate "http://www.chemistwarehouse.com.au/buy/36985/Reach-Dentotape-Waxed-20m"
Do
DoEvents
Loop Until IE.readyState = READYSTATE_COMPLETE
Dim Doc As HTMLDocument
Set Doc = IE.document
Dim getprice As String
getprice = Trim(Doc.getElementsByTagName("div class="Price" itemprop="price"").innerText)
Worksheets("Sheet1").Range(C1).Value = getprice
End Sub
The function getElementsByTagName() requires a tag name only as parameter:
e.g. getElementsByTagName("div")
Try getElementsByClassName() instead:
getprice = Trim(Doc.getElementsByClassName("Price").Item.innerText)
There were a few issues with the above code.
Issue 1
getprice = Trim(Doc.getElementsByTagName("div class="Price" itemprop="price"").innerText)
This:
div class="Price" itemprop="price" isn't a TagName. TagNames are things like Input, IMG, Anchors, etc. However, we can see the Class attribute for the price element you are interested in. We can change how we select this element by doing:
getprice = Trim(Doc.getElementsByClassName("Price")(0).innerText)
You may notice (0) at the end of the element selection. This is to indicate which element is being selected of the Price ClassName collection. getElementsByClassName returns multiple elements, the first element being 0.
Issue 2
Worksheets("Sheet1").Range(C1).Value = getprice
I don't C1 referenced anywhere. One way to reference a specific cell, is to use a String to represent the range. From your code this becomes:
Worksheets("Sheet1").Range("C1").Value = getprice