I'm trying to get data from a website which requires to log in a user and password. I've followed this tutorial and managed to log into the website, but for some reason it's not getting the table.
Here's the code:
Sub GetTable()
Dim ieApp As InternetExplorer
Dim ieDoc As Object
Dim ieTable As Object
Dim clip As DataObject
'create a new instance of ie
Set ieApp = New InternetExplorer
'you don’t need this, but it’s good for debugging
ieApp.Visible = True
'assume we’re not logged in and just go directly to the login page
ieApp.Navigate "https://accounts.google.com/ServiceLogin"
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
Set ieDoc = ieApp.Document
'fill in the login form –
With ieDoc.forms(0)
.Email.Value = "email#email.com"
.Passwd.Value = "password"
.submit
End With
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
'now that we’re in, go to the page we want
ieApp.Navigate "my-website.com"
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
'get the table based on the table's id
Set ieDoc = ieApp.Document
For i = 0 To (ieDoc.all.Length - 1)
'Only look at tables
If TypeName(ieDoc.all(i)) = “HTMLTable” Then
Set ieTable = ieDoc.all(i)
'I want to check the 3rd row (.Rows(2)) and will get an error if there
'are less than three rows.
If ieTable.Rows.Length > 2 Then
'Here’s the text in the first cell of the third row that tells me
'I have the right table
If ieTable.Rows(0).Cells(0).innertext = "Text" Then
'copy the tables html to the clipboard and paste to teh sheet
If Not ieTable Is Nothing Then
Set clip = New DataObject
clip.SetText "<html>" & ieTable.outerHTML & "</html>"
clip.PutInClipboard
Sheet1.Select
Sheet1.Range("A1").Select
Sheet1.PasteSpecial "Unicode Text"
End If
End If
End If
End If
Next i
'close 'er up
ieApp.Quit
Set ieApp = Nothing
End Sub
Assuming correctly marked with table tag you could have used the following to get the collection of tables which you can then loop through:
ieDoc.getElementsByTagName("table")
Related
I would like to import data from a password protected website(https://www.vesseltracker.com/fr/Ports/Home.html), i do have the username and the password.
I've tried this VBA code made by Dick in this website: http://dailydoseofexcel.com/archives/2006/11/29/html-tables/ but it doesn't work and get stuck every time i adjust it.
It get stuck here :
Sub GetTable()
Dim ieApp As InternetExplorer
Dim ieDoc As Object
Dim ieTable As Object
Dim clip As DataObject
'create a new instance of ie
Set ieApp = New InternetExplorer
'you don’t need this, but it’s good for debugging
ieApp.Visible = True
'assume we’re not logged in and just go directly to the login page
ieApp.Navigate "https://www.vesseltracker.com/fr/Home.html"
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
Set ieDoc = ieApp.Document
'fill in the login form – View Source from your browser to get the control names
With ieDoc.forms(0)
.UserName.Value = "username"
.Password.Value = "password"
.submit
End With
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
'now that we’re in, go to the page we want
ieApp.Navigate "https://www.vesseltracker.com/fr/Port/tangermed/Dashboard.html"
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
'get the table based on the table’s id
Set ieDoc = ieApp.Document
Set ieTable = ieDoc.all.Item
'copy the tables html to the clipboard and paste to teh sheet
If Not ieTable Is Nothing Then
Set clip = New DataObject
clip.SetText "" & ieTable.outerHTML & ""
clip.PutInClipboard
Sheet1.Select
Sheet1.Range("A1").Select
Sheet1.PasteSpecial "Unicode Text"
End If
'close 'er up
ieApp.Quit
Set ieApp = Nothing
End Sub
I really appreciate any help from you. Thank you.
The type DataObject is undefined because MSForms is not present in your references.
You could use late binding instead to set some text in the clipboard:
With VBA.CreateObject("new:{1C3B4210-F441-11CE-B9EA-00AA006B1A69}")
.SetText "1234"
.PutInClipboard
End With
You can add the required reference like this:
You need to include the MSForms library.
C:\Windows\system32\FM20.DLL
Easiest way is to add a userform to your project.
(Source)
Or you can add the library by going:
Tools > References > Browse > C:\Windows\system32\FM20.DLL
Need help, I have the below code that logs you in, moves to the right page, selects the table data and copies it, but the issue is there is a drop down list as shown with code, how do I get the code to select the "All" and how do I write that? I am willing to pay some to get this answer fast.
The data from web page for the drop down is here:
"<select name="flightrisk_tbl_length" aria-controls="flightrisk_tbl" class="form-control input-sm"><option value="5">5</option><option value="10">10</option><option value="25">25</option><option value="-1">All</option></select>"
Sub GetTable()
Dim ieApp As InternetExplorer
Dim ieDoc As Object
Dim ieTable As Object
Dim clip As DataObject
'create a new instance of ie
Set ieApp = New InternetExplorer
'you don’t need this, but it’s good for debugging
ieApp.Visible = True
'assume we’re not logged in and just go directly to the login page
ieApp.Navigate "xxxx"
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
Set ieDoc = ieApp.Document
'fill in the login form – View Source from your browser to get the control names
With ieDoc.forms(0)
.UserName.Value = "xxxx"
.Password.Value = "xxxx"
.submit
End With
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
'now that we’re in, go to the page we want
ieApp.Navigate "xxxx"
Do While ieApp.Busy: DoEvents: Loop
Do Until ieApp.ReadyState = READYSTATE_COMPLETE: DoEvents: Loop
'get the table based on the table’s id
Set ieDoc = ieApp.Document
Set ieTable = ieDoc.All.Item("flightrisk_tbl")
'copy the tables html to the clipboard and paste to the sheet
If Not ieTable Is Nothing Then
Set clip = New DataObject
clip.SetText "" & ieTable.outerHTML & ""
clip.PutInClipboard
Sheet12.Select
Sheet12.Range("A1").Select
Sheet12.PasteSpecial "Unicode Text"
End If
'close 'er up
ieApp.Quit
Set ieApp = Nothing
End Sub
You can use the GetElementsByName method to get the dropdown object:
Dim oDropDown as object
Set oDropDown = ieApp.GetElementsByName("flightrisk_tbl_length")(0) 'Use zero, because this returns an array of all elements with the same name.
Afterwards, you can use the innerHTML or use the GetElementsByClassName method to get everything from the option classes.
I am running into a bit of a problem. Normally when I pull a table I use the "data from web" tool in excel, however I now have quite a few places I need to pull data that first require me to enter a username and password. I figured out some code for that (though probably not the most elegant) but realized that once I get to my desired page I have no idea how to extract the table. Here is what I have so far.
Sub Login()
Sheets("IOL").Select
Set ie = CreateObject("InternetExplorer.application")
ie.Visible = True
ie.Navigate ("https://internalsite.company.com/secure/login" & ActiveCell)
Do
If ie.ReadyState = 4 Then
ie.Visible = True
Exit Do
Else
DoEvents
End If
Loop
ie.Document.forms(0).all("badgeBarcodeId").Value = "00000"
ie.Document.forms(0).submit
'used because it redirects to a new page after submitting and I couldn't figure out how to make it wait for the new page to load before proceeding.
Application.Wait (Now + TimeValue("0:00:02"))
ie.Document.forms(0).all("password").Value = "00000"
ie.Document.forms(0).submit
End Sub
After the login is accomplished I would like to go to http://internalsite.company.com/csv and import the csv directly into a sheet. Anytime I make a new connection it makes me log in again so I figure there has to be a way to extract the file without adding a new connection. I'm pretty new with more complex VBA so bear with me.
I was able to get this code to do the job, but it is more preferable to get the CSV directly instead of the table. Sometimes the table doesn't like to load.
Sub Login()
Dim clip As DataObject
Dim ieTable As Object
Set ie = CreateObject("InternetExplorer.application")
ie.Visible = True
ie.Navigate ("https://internalsite1.company.com/secure/login" & ActiveCell)
Do
If ie.ReadyState = 4 Then
ie.Visible = True
Exit Do
Else
DoEvents
End If
Loop
ie.Document.forms(0).all("badgeBarcodeId").Value = "00000"
ie.Document.forms(0).submit
Do While ie.Busy: DoEvents: Loop
Do Until ie.ReadyState = 4: DoEvents: Loop
ie.Document.forms(0).all("password").Value = "000000"
ie.Document.forms(0).submit
Do While ie.Busy: DoEvents: Loop
Do Until ie.ReadyState = 4: DoEvents: Loop
ie.Navigate "http://internalsite2.company.com/site/Inbound?filter=1To3Days"
Do While ie.Busy: DoEvents: Loop
Do Until ie.ReadyState = 4: DoEvents: Loop
Set ieTable = ie.Document.all.Item("DataTables_Table_0")
If Not ieTable Is Nothing Then
Set clip = New DataObject
clip.SetText "" & ieTable.outerHTML & ""
clip.PutInClipboard
Workbooks("Production Meeting Dashboard.xlsm").Activate
Sheets("IOL").Select
Range("A1").Select
ActiveSheet.PasteSpecial Format:="Unicode Text", link:=False, _
DisplayAsIcon:=False, NoHTMLFormatting:=True
End If
End Sub
I used the below code for loading the web site http://www.flashscore.com/soccer/england/premier-league/results/.
After I found and click on the "Show more matches" link, all the football matches are loaded in the browser.
The below code will give as results only the first half of matches, the events showed before pressing the "Show more matches" link.
My question is how can I list all the events URL adress?
Sub Test_Flashscore()
Dim URL As String
Dim ie As New InternetExplorer
Dim HTMLdoc As HTMLDocument
Dim dictObj As Object: Set dictObj = CreateObject("Scripting.Dictionary")
Dim tRowID As String
URL = "http://www.flashscore.com/soccer/england/premier-league/results/"
With ie
.navigate URL
.Visible = True
Do Until .readyState = READYSTATE_COMPLETE: DoEvents: Loop
Set HTMLdoc = .document
End With
For Each objLink In ie.document.getElementsByTagName("a")
If Left(objLink.innerText, 4) = "Show" Or Left(objLink.innerText, 4) = "Arat" Then
MsgBox "The link was founded!"
objLink.Click
Exit For
End If
Next objLink
With HTMLdoc
Set tblSet = .getElementById("fs-results")
Set mTbl = tblSet.getElementsByTagName("tbody")(0)
Set tRows = mTbl.getElementsByTagName("tr")
With dictObj
'If if value is not yet in dictionary, store it.
For Each tRow In tRows
'Remove the first four (4) characters.
tRowID = Mid(tRow.ID, 5)
If Not .Exists(tRowID) Then
.add tRowID, Empty
End If
Next tRow
End With
End With
i = 14
For Each Key In dictObj
ActiveSheet.Cells(i, 2) = "http://www.flashscore.com/" & Key & "/#match-summary"
i = i + 1
Next Key
Set ie = Nothing
MsgBox "Process Completed"
End Sub
You need to wait a little while for the rest of the content to load - clicking the link fires off a GET request to the server, so that needs to return content and the content needs to be rendered on the page before you can grab it.
Clicking on that link takes you to fixtures. You can replace all that before dictionary with
.navigate "https://www.flashscore.com/football/england/premier-league/fixtures/"
That is:
Option Explicit
Public Sub GetInfo()
Dim IE As New InternetExplorer
With IE
.Visible = True
.navigate "https://www.flashscore.com/football/england/premier-league/fixtures/"
While .Busy Or .readyState < 4: DoEvents: Wend
'other code...using dictionary
'.Quit
End With
End Sub
I am in need of assistance. I am trying to write a VBA script that would take the value in column A and place it on an online form in an input element with no ID but the name ("OldUrl"). Then the VBA script would take the value in the adjacent cell in column B and place that in the same form ("digiSHOP") in the input field named ("NewUrl").
The form is on a secure server however I have gotten as far as the window pulling up and the form selected. I am having trouble finding a way to target the input field since they have no ID. Below is my code and thank you for your help.
Sub Redirect()
Dim IE As Object
Dim doc As Object
Dim form As Object
Dim OldURL As Object
Dim NewURL As Object
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Visible = True
.Navigate "https://...."
Do Until .ReadyState = 4: DoEvents: Loop
Set doc = IE.Document
Set form = doc.forms("digiSHOP")
Set OldURL = doc.getElementById("OldUrl")'Error occurs here. Element has no ID
OldURL.Value = Range("A2")
Set NewURL = doc.getElementById("NewUrl")
NewURL.Value = Range("B2")
form.submit
Do Until .ReadyState = 4: DoEvents: Loop
Do While .Busy: DoEvents: Loop
End With
End Sub
Also I wasn't sure how to target the entire column and loop it therefore the Value is set to the cell A2. This was more to test the script.
Sub Redirect()
Dim IE As Object
Dim doc As Object
Set IE = CreateObject("InternetExplorer.Application")
With IE
.Visible = True
.Navigate "https://...."
Do Until .ReadyState = 4: DoEvents: Loop
With .Document.forms("digiSHOP")
.elements("OldUrl").Value = Range("A2")
.elements("NewUrl").Value = Range("B2")
.submit
End With
Do Until .ReadyState = 4: DoEvents: Loop
Do While .Busy: DoEvents: Loop
End With
End Sub