How to import multiple pages into Excel from ESPN - vba

Question

I need to import data from ESPN Fantasy Football into Excel. This includes multiple pages and a web query does not work. The URL is http://games.espn.com/ffl/tools/projections?startIndex=0. When you click next, the URL becomes http://games.espn.com/ffl/tools/projections?startIndex=40 and 80 and so on. Basically, the startIndex increments to multiples of 40 when you click on the next page.
This is the code I've user so far modifying the code provided by #Dee and it seems to adding a new worksheet but has no data in it.
Private Const URL_TEMPLATE As String =
"URL;http://games.espn.com/ffl/tools/projections?startIndex={0}"
Private Const NUMBER_OF_PAGES As Byte = 7
Sub test()
Dim page As Byte
Dim queryTableObject As QueryTable
Dim url As String
For page = 1 To NUMBER_OF_PAGES
url = VBA.Strings.Replace(URL_TEMPLATE, "{0}", page * 40)
Set queryTableObject = ActiveSheet.QueryTables.Add(Connection:=url, Destination:=ThisWorkbook.Worksheets.Add.[a1])
queryTableObject.WebSelectionType = xlSpecifiedTables
queryTableObject.WebTables = "14"
queryTableObject.Refresh
Next page
End Sub
The end result should be pulling all the columns from each page into a tab in one Excel sheet. Is there a way to do this? Please assist.
Thanks,
J

Answer 1

The Id of the table is not 14 but playertable_0. If you need to paste all the data to one single sheet try to use the following modified code. Number of pages is here determined just by guessing. You will have to add some more code to prove if e.g. the Next element is present on the page and do it in do-loop or just simply guess the number of pages. HTH
Option Explicit
Private Const URL_TEMPLATE As String = _
"URL;http://games.espn.com/ffl/tools/projections?startIndex={0}"
Private Const NUMBER_OF_PAGES As Byte = 20
Private Const TableId As String = "playertable_0"
Sub test()
Dim page As Byte
Dim queryTableObject As QueryTable
Dim url As String
Dim ws As Worksheet
Dim target As Range
Dim lastRow As Long
Set ws = ThisWorkbook.Worksheets.Add
Set target = ws.[a1]
lastRow = 1
For page = 0 To NUMBER_OF_PAGES
url = VBA.Strings.Replace(URL_TEMPLATE, "{0}", page * 40)
Set queryTableObject = ActiveSheet.QueryTables.Add( _
Connection:=url, _
Destination:=target)
With queryTableObject
.BackgroundQuery = False ' Run the query synchronously
' to be able to compute the last row
.WebSelectionType = xlSpecifiedTables
.WebTables = TableId
.Refresh
End With
lastRow = ws.Cells.Find( _
what:="*", _
SearchDirection:=xlPrevious, _
SearchOrder:=xlByRows) _
.Row
Set target = ws.Cells(lastRow + 1, 1)
target.Select
Next page
End Sub

Answer 2

Give this a go. I hope it will fix all the issues you are having.
Sub Espn_Data()
Const URL = "http://games.espn.com/ffl/tools/projections?startIndex="
Dim http As New XMLHTTP60, html As New HTMLDocument, page As Long
Dim htmla As Object, tRow As Object, tCel As Object, ws As Worksheet
For page = 0 To 120 Step 40 '''input the highest number replacing 120 to get them all
With http
.Open "GET", URL & page, False
.send
html.body.innerHTML = .responseText
End With
With ThisWorkbook
Set ws = .Sheets.Add(After:=.Sheets(.Sheets.Count))
ws.Name = page
End With
Set htmla = html.getElementsByClassName("playerTableTable tableBody")(0)
For Each tRow In htmla.Rows
For Each tCel In tRow.Cells
c = c + 1: Cells(x + 1, c) = tCel.innerText
Next tCel
c = 0
x = x + 1
Next tRow
c = 0
x = 0
Next page
End Sub

Answer 3

The Web Query for ESPN doesn't work any longer with ESPN Fantasy Football... I don't know why as it only returns all of the page info except what is in the table... Don't insult my intelligence, I've tried all of the settings...
I have come to discover that Excel only allows IE 6 as a web browser as you can find this in the 'File/Options/Advanced/General' "Web Options" then Browsers and then scroll into Target Browsers.... There are your choices for Web Query to obtain Data.... ESPN does not support any of those browsers so like you, we're screwed until EXCEL updates our choices - Yes, I know... It's 2021 :(