I'm very new to work with IE in vba so it is hard for me sometimes to rectify any mistake I make while writing any code to scrape data from web. I've written some code to click on each video links out of 20 links under the caption Microsoft computer training videos available in it's front page. I expect to click each links then navigate back and repeat the process until all the links are clicked. The class names and tag names I've defined in my scraper is accurate. All i need to do is perform the process in the right way. At this moment my scraper is clicking the last link of the 20 links and get stuck there where it reaches but don't navigate back.
This is what I've written so far.
Sub clicking_links()
Const surl As String = "http://www.wiseowl.co.uk/videos/"
Dim IE As New InternetExplorer, iedoc As HTMLDocument
Dim posts As Object
With IE
.Visible = True
.navigate surl
Do Until .readyState = READYSTATE_COMPLETE: Loop
Set iedoc = IE.document
End With
For Each posts In iedoc.getElementsByClassName("woVideoListDefaultSeriesTitle")
posts.getElementsByTagName("a")(0).Click
Next posts
End Sub
Try the following:
Sub clicking_links()
Const surl As String = "http://www.wiseowl.co.uk/videos/"
Dim newurl as String
Dim IE As New InternetExplorer, iedoc As HTMLDocument
Dim posts As Object
Dim t As Long, i As Long
With IE
.Visible = True
.navigate surl
Do Until .readyState = READYSTATE_COMPLETE: Loop
Set iedoc = .document
For Each posts In iedoc.getElementsByClassName("woVideoListDefaultSeriesTitle")
t = t + 1 'count the number of posts
Next posts
For i = 1 To t
Debug.Print i
newurl = iedoc.getElementsByClassName("woVideoListDefaultSeriesTitle")(i - 1).getElementsByTagName("a")(0).href
Debug.Print newurl
.navigate newurl
Do Until .readyState = READYSTATE_COMPLETE: Loop
Set iedoc = .document
'here do your stuff within the new url
.navigate surl 'back to old url
Do Until .readyState = READYSTATE_COMPLETE: Loop
Set iedoc = .document
Next i
End With
End Sub
Related
I need to webscrape a website that I actually have to type the keys to show the password form.
I've tried this code with no success (even using the right login)
Sub login()
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.navigate ("https://wsvr10.coamo.com.br/npl/#!/login")
While IE.readyState <> READYSTATE_COMPLETE
Wend
sng = Timer
Do While sng + 5 > Timer
Loop
Dim idoc As MSHTML.HTMLDocument
Set idoc = IE.document
idoc.all.cpf.Value = "11111111111"
I am trying to upload a file to a web page, the following are the steps I followed:
Open the web page http://www.htmlquick.com/reference/tags/input-file.html
Wait for until the page is getting loaded
In this webpage I am uploading the file into the first “Upload a File” browser.
Get the input element by tag name as “input”
Hit the “browse” button, since the paste potion is disabled.
Enter the file path in the “Choose File to Upload” window
Enter
After 5th step, I am not able to enter the file path in the “Choose File to Upload” window, looks like the macro is not supporting for this.
Here is my code :
Sub File_Test()
Dim HTMLDoc As MSHTML.HTMLDocument
Dim HTMLButtons As MSHTML.IHTMLElementCollection
Dim HTMLButton As MSHTML.IHTMLElement
Dim ie As Object
Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
ie.navigate "http://www.htmlquick.com/reference/tags/input-file.html"
Do While ie.readyState <> READYSTATE_COMPLETE
Loop
Set HTMLDoc = ie.document
Set HTMLButtons = HTMLDoc.getElementsByTagName("input")
For Each HTMLButton In HTMLButtons
If HTMLButton.Type = "file" Then
HTMLButton.Click
HTMLButton.Value = "C:\Documents\Test\Temp.txt"
Exit For
End If
Next
End Sub
And here's a screenshot:
Any suggestions?
===================================================
Here is another modified code but sill I am not able to enter the file name
Sub File_Test()
Dim HTMLDoc As MSHTML.HTMLDocument
Dim HTMLButtons As MSHTML.IHTMLElementCollection
Dim HTMLButton As MSHTML.IHTMLElement
Dim ie As Object
Dim WSshell
Set WSshell = CreateObject("WScript.Shell")
Set ie = CreateObject("internetexplorer.application")
ie.Visible = True
ie.navigate "http://www.htmlquick.com/reference/tags/input-file.html"
Do While ie.readyState <> READYSTATE_COMPLETE
Loop
Set HTMLDoc = ie.document
Set HTMLButtons = HTMLDoc.getElementsByTagName("input")
For Each HTMLButton In HTMLButtons
If HTMLButton.Type = "file" Then
HTMLButton.Click
With WSshell
Application.Wait (Now + TimeValue("0:00:10"))
.AppActivate "Choose File to Upload"
Application.Wait (Now + TimeValue("0:00:10"))
.SendKeys "C:\Documents\Test\Temp.txt"
Application.Wait (Now + TimeValue("0:00:10"))
.SendKeys "~" 'Enter
End With
Exit For
End If
Next
End Sub
Any idea? why sendkyes are not working?
I am trying to download some data from website which starts download upon clicking.
But this code is not working, Can anyone help.
Dim ie As InternetExplorer
Dim htmldoc As MSHTML.IHTMLDocument
Dim HTMLInput As MSHTML.IHTMLElementCollection
Dim HTMLAs As MSHTML.IHTMLElementCollection
Dim HTMLA As MSHTML.IHTMLElement
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
'Navigate to webpage
Dim ieURL As String: ieURL = "http://erldc.org/final-schedule.aspx"
ie.navigate ieURL
Do While ie.readyState <> READYSTATE_COMPLETE
Loop
Set htmldoc = ie.document
Set HTMLInput = htmldoc.getElementsByTagName("a")
For Each HTMLA In HTMLAs
Debug.Print HTMLA.getAttribute("classname"), HTMLA.getAttribute("href"), HTMLA.getAttribute("rel")
If HTMLA.getAttribute("href") = "javascript:__doPostBack('ctl00$ContentPlaceHolder1$Calendar1','6420')" Then
HTMLA.Click
Exit For
End If
Next HTMLA
End Sub
Try pausing the macro for a few seconds, maybe 5 seconds, after the Do While/Loop that checks for the ReadyState...
Dim sngFinish As Single
Dim intPauseTime As Integer
intPauseTime = 5 'in seconds
sngFinish = timer + intPauseTime
Do While timer < sngFinish
DoEvents
Loop
Also, I would suggest that you check the Busy state of Internet Explorer, in addition to the ReadyState, and add DoEvents...
Do While ie.Busy Or ie.readyState <> READYSTATE_COMPLETE
DoEvents
Loop
Hope this helps!
My code :
Sub login()
Dim IE As Object
Dim HTMLDoc As Object, HTMLDoc2 As Object
Dim objCollection As Object
Const navOpenInNewTab = &H800
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.Navigate "https:/com/1/19/login.esp"
Do While IE.Busy Or IE.ReadyState <> 4: Loop
Set HTMLDoc = IE.Document
With HTMLDoc
HTMLDoc.getElementById("USERNAME").Value = "xxxx" 'Entering credential
HTMLDoc.getElementById("PASSWORD").Value = "yyyyy"
End With
Set objCollection = IE.Document.getElementById("loginbutton")
objCollection.Click
'Second webpage
Do While IE.Busy Or IE.ReadyState <> 4: Loop ' opening the second webpage
Set HTMLDoc2 = IE.Document
With HTMLDoc2
**HTMLDoc2.getElementById("DEPARTMENTID").selectedindex = 1 'Drop down menu
HTMLDoc2.getElementById("DEPARTMENTID").FireEvent ("onchange")**
End With
Set objCollection = IE.Document.getElementById("loginbutton")
objCollection.Click
End Sub
Q)What code changes do I do to select Dwell_DF option Value 1567?
The above code gives run time error '424' : Object required.
HTMLDoc2.getElementById("DEPARTMENTID").selectedindex = 1 'Drop down menu
HTMLDoc2.getElementById("DEPARTMENTID").FireEvent ("onchange")
The above line give the error.
In the first webpage I fill the login credentials then in the next page is that of the image pasted with this post. Here I want to change the value in the drop down menu.
Give this a try. The value "1567" corresponds with the InnerText "Dwell_DF".
With HTMLDoc2
.getElementById("DEPARTMENTID").Focus
.getElementById("DEPARTMENTID").Value = "1567" 'You can also loop to find the text of the Option
.getElementById("DEPARTMENTID").FireEvent ("onchange")
End With
It should be something like this.
Sub passValueToComboBox1()
Dim ie As Object
Dim oHTML_Element As IHTMLElement
Set ie = CreateObject("InternetExplorer.Application")
ie.Visible = True
ie.navigate "http://your_URL_here.php"
While ie.Busy Or ie.readyState <> 4: DoEvents: Wend
Set oHTML_Element = ie.document.getElementsByName("selectedReportClass")(0)
If Not oHTML_Element Is Nothing Then oHTML_Element.Value = "FUBU7"
For Each oHTML_Element In ie.document.getElementsByTagName("input")
If oHTML_Element.Type = "submit" Then oHTML_Element.Click: Exit For
Next
End Sub
Check out the link below for some other ideas of how to programatically interact with a web site.
http://vbadud.blogspot.com/2009/08/how-to-login-to-website-using-vba.html
This is not for the first time I am mentioning this...but yeah I am a rookie in vba and in need of help...
I am having trouble in filling up a value in a box in webpage using vba excel.
The source code of the box is
<input
name="MultiRenameTable_OR:wt.epm.EPMDocument:3053059700_NEWNAME_JSID"
class="enabledfield"
id="MultiRenameTable_OR:wt.epm.EPMDocument:3053059700_NEWNAME_JSID"
style="WIDTH:98% onblur="setJSAttributeValue(this,'NEWNAME_JSID','MultiRenameTable', 'epmdoc_09876889_u46_drw',undefined)" type="text" size="25" ?=""
trlId="MultiRenameTable_epmdoc_09876889_u46_drw_NEWNAME_JSID"></input>
the code i wrote is
Sub xx()
Dim ie As Object
Dim doc As HTMLDocument
Dim lo As IHTMLElementCollection
Dim l As IHTMLElement
Set ie = CreateObject("internetexplorer.application")
ie.navigate "mysiteurl"
ie.Visible = True
Do While ie.Busy: DoEvents: Loop
Do While ie.readyState <> 4: DoEvents: Loop
Set doc = ie.document
doc.getElementById("MultiRenameTable_OR:wt.epm.EPMDocument3053059700_NEWNAME_JSID").Value = "xyz"
End Sub
It doesnt set the value.
Also i cannot use "name","ID","trlID","onblur" because they keep on changing each time i launch IE.
Please help me with this.
I'm guessing from the setJSAttributeValue that the name will always contain the substring "NEWNAME_JSID". Just grab all of the input elements, then loop through them looking for it in the name:
Sub xx()
Dim ie As Object
Dim doc As HTMLDocument
Dim lo As IHTMLElementCollection
Dim l As IHTMLElement
Set ie = CreateObject("internetexplorer.application")
ie.navigate "mysiteurl"
ie.Visible = True
Do While ie.Busy: DoEvents: Loop
Do While ie.readyState <> 4: DoEvents: Loop
Set doc = ie.document
Set lo = doc.getElementsByTagName("input")
For Each l In lo
If l.Name Like "*NEWNAME_JSID*" Then
'Do whatever.
Exit For
End If
Next l
End Sub