Get table from website using VBA with dynamic web address - vba

I'm trying to insert a table from a website into excel, specifically off of WSJ futures. There's a calendar feature I can get around by changing the web address. I would like to make the website in the code a cell reference so I can make the address dynamic. Below is the code for the regular macro with no dynamic address. Suggestions on how to make this change? I've tried taking out the URL and making it a cell reference, but it did not work.
ActiveWorkbook.Queries.Add Name:=Range("A1"), Formula:= _
"let" & Chr(13) & "" & Chr(10) & " Source = Web.Page(Web.Contents(""http://www.wsj.com/mdc/public/page/2_3023-fut_metal-futures-20170901.html?mod=mdc_pastcalendar""))," & Chr(13) & "" & Chr(10) & " Data5 = Source{5}[Data]," & Chr(13) & "" & Chr(10) & " #""Changed Type"" = Table.TransformColumnTypes(Data5,{{""Column1"", type text}, {""Column2"", type text}, {""Column3"", type text}, {""Column4"", type text}, {""Column5"", type text}, {""Colu" & _
"mn6"", type text}, {""Column7"", type text}, {""Column8"", type text}, {""Column9"", type text}, {""Column10"", type text}})," & Chr(13) & "" & Chr(10) & " #""Removed Top Rows"" = Table.Skip(#""Changed Type"",1)," & Chr(13) & "" & Chr(10) & " #""Promoted Headers"" = Table.PromoteHeaders(#""Removed Top Rows"", [PromoteAllScalars=true])" & Chr(13) & "" & Chr(10) & "in" & Chr(13) & "" & Chr(10) & " #""Promoted Headers"""
ActiveWorkbook.Worksheets.Add
With ActiveSheet.ListObjects.Add(SourceType:=0, Source:= _
"OLEDB;Provider=Microsoft.Mashup.OleDb.1;Data Source=$Workbook$;Location=""Table 5"";Extended Properties=""""" _
, Destination:=Range("$A$1")).QueryTable
.CommandType = xlCmdSql
.CommandText = Array("SELECT * FROM [Table 5]")
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.PreserveColumnInfo = True
.ListObject.DisplayName = "Table_5"
.Refresh BackgroundQuery:=False
End With
End Sub

Related

How to turn the number inside Source{0}[Data] into a variable?

I have been test the following macro. It originates from a recorded macro by running QueryTable, and is intended to get data table for an appointed stock from a site.
Some information about the site for one stock is as follows.
stock code & structure of the tables
I modified the original macro, making the stock code, the table number and table date into arguments, so that a loop could be invoked to get tables with the assigned "table number/table date" for a whole list of stock codes.
Following is the modified macro.
Sub Fetch_Table(code As String, tableDate As String)
Dim sourceFullName As String
sourceFullName = "http://emweb.securities.eastmoney.com/PC_HSF10/BusinessAnalysis/Index?type=web&code=" & code
ActiveWorkbook.Queries.Add Name:="Table 0", Formula:= _
"let" & Chr(13) & "" & Chr(10) & " Source = Web.Page(Web.Contents(""" & sourceFullName & """))," & Chr(13) & "" & Chr(10) & " Data0 = Source{0}[Data]," & Chr(13) & "" & Chr(10) & " #""Changed Type"" = Table.TransformColumnTypes(Data0,{{""" & tableDate & """, type text},{""主营构成"", type text}, {""主营收入(元)"", type text}, {""收入比例"", Percentage.Type}, {""主营成本(元)"", type text}, {""成本比" & _
"例"", type text}, {""主营利润(元)"", type text}, {""利润比例"", type text}, {""毛利率(%)"", type text}})" & Chr(13) & "" & Chr(10) & "in" & Chr(13) & "" & Chr(10) & " #""Changed Type"""
With ActiveSheet.ListObjects.Add(SourceType:=0, Source:= _
"OLEDB;Provider=Microsoft.Mashup.OleDb.1;Data Source=$Workbook$;Location=""Table 0"";Extended Properties=""""" _
, Destination:=Range("$A$5")).QueryTable
.CommandType = xlCmdSql
.CommandText = Array("SELECT * FROM [Table 0]")
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.PreserveColumnInfo = True
.ListObject.DisplayName = "Table_0"
.Refresh BackgroundQuery:=False
End With
End Sub
The code As String, tableDate as String work well. And I call Fetch_Table(code,tableDate), with this two parameters passed.
Then I tried to turn the number in Source{0}[Data] in to a reference, which means I can choose which talbe to fetch, and make the number match the date of the table. So I tested the following code, which treated N as a local variable temporarily, before making it into a parameter. All the rest are same as the above well working macro.
Sub Fetch_Table(code As String, tableDate As String)
Dim sourceFullName As String
sourceFullName = "http://emweb.securities.eastmoney.com/PC_HSF10/BusinessAnalysis/Index?type=web&code=" & code
N = 0
ActiveWorkbook.Queries.Add Name:="Table 0", Formula:= _
"let" & Chr(13) & "" & Chr(10) & " Source = Web.Page(Web.Contents(""" & sourceFullName & """))," & Chr(13) & "" & Chr(10) & " Data0 = Source{N}[Data]," & Chr(13) & "" & Chr(10) & " #""Changed Type"" = Table.TransformColumnTypes(Data0,{{""" & tableDate & """, type text},{""主营构成"", type text}, {""主营收入(元)"", type text}, {""收入比例"", Percentage.Type}, {""主营成本(元)"", type text}, {""成本比" & _
"例"", type text}, {""主营利润(元)"", type text}, {""利润比例"", type text}, {""毛利率(%)"", type text}})" & Chr(13) & "" & Chr(10) & "in" & Chr(13) & "" & Chr(10) & " #""Changed Type"""
With ActiveSheet.ListObjects.Add(SourceType:=0, Source:= _
"OLEDB;Provider=Microsoft.Mashup.OleDb.1;Data Source=$Workbook$;Location=""Table 0"";Extended Properties=""""" _
, Destination:=Range("$A$5")).QueryTable
.CommandType = xlCmdSql
.CommandText = Array("SELECT * FROM [Table 0]")
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.PreserveColumnInfo = True
.ListObject.DisplayName = "Table_0"
.Refresh BackgroundQuery:=False
End With
End Sub
However, when I called this macro again, an error msgbox came out.
enter image description here
At first, I tried to explicited define the type of N as integer, and then I tried to change the type of N into string. But they all failed.
I assumed that the nubmer in Source{0}[Data] could be taken palce by a variable, and the prolem stemmed from syntax mistakes.
I don't know whether my assumption is right.

Concatenating variables into a string for a query url

I am trying to program a dynamic url that changes as the day changes. I can get the query to run if I hardcode the date into the string but it will not run when "todaysDate" is used at the end of the url. I looked in the locals window and the variable url returns the correct string needed to download the csv file that the query calls for.
'''
Sub historicalDataQuery(ByVal ticker As String)
Dim todaysDate As String
Dim oneYearAgo As String
Dim url As String
todaysDate = Format(Now, "YYYY-MM-DD")
oneYearAgo = Format(Now - 365, "YYYY-MM-DD")
url = "https://www.nasdaq.com/api/v1/historical/" & ticker & "/stocks/" & oneYearAgo & "/" & todaysDate
'On Error Resume Next
ActiveWorkbook.Queries.Add Name:="2020-02-23", Formula:= _
"let" & Chr(13) & "" & Chr(10) & " Source = Csv.Document(Web.Contents(url)),[Delimiter="","", Columns=6, Encoding=1252, QuoteStyle=QuoteStyle.None])," & Chr(13) & "" & Chr(10) & " #""Promoted Headers"" = Table.PromoteHeaders(Source, [PromoteAllScalars=true])," & Chr(13) & "" & Chr(10) & " #""Changed Type"" = Table.TransformColumnTypes(#""Promoted Headers"",{{""Date""," & _
" type date}, {"" Close/Last"", Currency.Type}, {"" Volume"", Int64.Type}, {"" Open"", Currency.Type}, {"" High"", Currency.Type}, {"" Low"", Currency.Type}})," & Chr(13) & "" & Chr(10) & " #""Removed Columns"" = Table.RemoveColumns(#""Changed Type"",{""Date"", "" Volume"", "" Open"", "" High"", "" Low""})" & Chr(13) & "" & Chr(10) & "in" & Chr(13) & "" & Chr(10) & " #""Removed Columns"""
Sheets.Add After:=ActiveSheet
With ActiveSheet.ListObjects.Add(SourceType:=0, Source:= _
"OLEDB;Provider=Microsoft.Mashup.OleDb.1;Data Source=$Workbook$;Location=2020-02-23;Extended Properties=""""" _
, Destination:=Range("$A$1")).QueryTable
.CommandType = xlCmdSql
.CommandText = Array("SELECT * FROM [2020-02-23]")
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.PreserveColumnInfo = True
.ListObject.DisplayName = ticker
.Refresh BackgroundQuery:=False
End With
end sub
for example, I know that this code works:
Source = Csv.Document(Web.Contents(""https://www.nasdaq.com/api/v1/historical/" & ticker & "/stocks/" oneYearAgo & "/2020-02-23"")
You had a typo in your query definition
I also refactored some of the code
Code:
Sub test()
historicalDataQuery "msft"
End Sub
Sub historicalDataQuery(ByVal ticker As String)
Dim todaysDate As String
Dim oneYearAgo As String
Dim url As String
Dim queryName As String
Dim queryString As String
todaysDate = Format(Now, "YYYY-MM-DD")
oneYearAgo = Format(Now - 365, "YYYY-MM-DD")
url = "https://www.nasdaq.com/api/v1/historical/" & ticker & "/stocks/" & oneYearAgo & "/" & todaysDate
queryName = ticker & todaysDate
If QueryExists(queryName, ThisWorkbook) Then
MsgBox "Query already exists"
Exit Sub
End If
queryString = "let" & Chr(13) & Chr(10) & _
" Source = Csv.Document(Web.Contents(""" & url & """),[Delimiter="","", Columns=6, Encoding=1252, QuoteStyle=QuoteStyle.None])," & Chr(13) & Chr(10) & _
" PromoteHeaders = Table.PromoteHeaders(Source, [PromoteAllScalars=true])," & Chr(13) & Chr(10) & _
" ChangeTypes = Table.TransformColumnTypes(PromoteHeaders,{{""Date"", type date}, {"" Close/Last"", Currency.Type}, {"" Volume"", Int64.Type}, {"" Open"", Currency.Type}, {"" High"", Currency.Type}, {"" Low"", Currency.Type}})," & Chr(13) & Chr(10) & _
" RemoveColumns = Table.RemoveColumns(ChangeTypes,{""Date"", "" Volume"", "" Open"", "" High"", "" Low""})" & Chr(13) & Chr(10) & _
"in" & Chr(13) & Chr(10) & _
" RemoveColumns"
ActiveWorkbook.Queries.Add Name:=queryName, Formula:=queryString
Sheets.Add After:=ActiveSheet
With ActiveSheet.ListObjects.Add(SourceType:=0, Source:= _
"OLEDB;Provider=Microsoft.Mashup.OleDb.1;Data Source=$Workbook$;Location=" & queryName & ";Extended Properties=""""" _
, Destination:=Range("$A$1")).QueryTable
.CommandType = xlCmdSql
.CommandText = Array("SELECT * FROM [" & queryName & "]")
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.PreserveColumnInfo = True
.ListObject.DisplayName = ticker
.Refresh BackgroundQuery:=False
End With
End Sub
Function QueryExists(q$, Optional wb As Workbook) As Boolean
' Credits: https://gallery.technet.microsoft.com/VBA-to-automate-Power-956a52d1
If wb Is Nothing Then Set wb = ActiveWorkbook
On Error Resume Next
QueryExists = CBool(Len(wb.Queries(q).Name))
On Error GoTo 0
End Function
Let me know if it works

Import text file from drop-down in Excel

I am trying to create a macro in Excel that imports a text file based on values in cells on a worksheet. One cell specifies the path of the directory, a second cell with a drop-down contains the file's name including extension.
I've tried to record a macro wherein I import a file from the directory in question, but I am having trouble changing it so that it uses the two cells with the path and file name instead of constant values.
The original recorded macro looks as follows:
Sub txt_import()
' txt_import Macro
ActiveWorkbook.Queries.Add Name:="testfile", Formula:= _
"let" & Chr(13) & "" & Chr(10) & " Source = Csv.Document(File.Contents(""D:\testpath\testfile.txt""),[Delimiter="";"", Columns=10, Encoding=1252, QuoteStyle=QuoteStyle.None])," & Chr(13) & "" & Chr(10) & " #""Change Type"" = Table.TransformColumnTypes(Source,{{""Column1"", type text}, {""Column2"", type text}, {""Column3"", type text}, {""Colu" & _
"mn4"", type text}, {""Column5"", type text}, {""Column6"", type text}, {""Column7"", type text}, {""Column8"", type text}, {""Column9"", type text}, {""Column10"", type text}})" & Chr(13) & "" & Chr(10) & "in" & Chr(13) & "" & Chr(10) & " #""Change Type"""
ActiveWorkbook.Worksheets.Add
With ActiveSheet.ListObjects.Add(SourceType:=0, Source:= _
"OLEDB;Provider=Microsoft.Mashup.OleDb.1;Data Source=$Workbook$;Location=testfile;Extended Properties=""""" _
, Destination:=Range("$A$1")).QueryTable
.CommandType = xlCmdSql
.CommandText = Array("SELECT * FROM [testfile]")
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.PreserveColumnInfo = True
.ListObject.DisplayName = "_testfile"
.Refresh BackgroundQuery:=False
End With
End Sub
The directory path is located on Sheet1 A1 and the drop-down list with the file names is in Sheet3 A2.
Any help is greatly appreciated!
You'll need to modify the Source query string, try:
Sub txt_import()
' txt_import Macro
Dim path as String, filename as String, fullFileName as String
path = ThisWorkbook.Worksheets("Sheet1").Range("A1").Value
filename = ThisWorkbook.Worksheets("Sheet3").Range("A2").Value
fullFileName = path & Application.PathSeparator & filename
ActiveWorkbook.Queries.Add Name:="testfile", Formula:= _
"let" & Chr(13) & "" & Chr(10) & " Source = Csv.Document(File.Contents(" & fullFileName & "),[Delimiter="";"", Columns=10, Encoding=1252, QuoteStyle=QuoteStyle.None])," & Chr(13) & "" & Chr(10) & " #""Change Type"" = Table.TransformColumnTypes(Source,{{""Column1"", type text}, {""Column2"", type text}, {""Column3"", type text}, {""Colu" & _
"mn4"", type text}, {""Column5"", type text}, {""Column6"", type text}, {""Column7"", type text}, {""Column8"", type text}, {""Column9"", type text}, {""Column10"", type text}})" & Chr(13) & "" & Chr(10) & "in" & Chr(13) & "" & Chr(10) & " #""Change Type"""

error 438 - ActiveWorkbook.queries

I have the following code in an Excel file:
Sub PullData()
ActiveWorkbook.Queries.Add Name:="Sheet1", Formula:= _
"let" & Chr(13) & "" & Chr(10) & " Source = Excel.Workbook(Web.Contents(""Link.xlsx""), null, true)," & Chr(13) & "" & Chr(10) & " Sheet1_Sheet = Source{[Item=""Sheet1"",Kind=""Sheet""]}[Data]," & Chr(13) & "" & Chr(10) & " #""Changed Type"" = Table.TransformColumnTypes(Sheet1_Sheet,{{""Column1"", type text}, {""Column2"", type text}})" & Chr(13) & "" & Chr(10) & "in" & Chr(13) & "" & Chr(10) & "" & _
" #""Changed Type"""
With ActiveSheet.ListObjects.Add(SourceType:=0, Source:= _
"OLEDB;Provider=Microsoft.Mashup.OleDb.1;Data Source=$Workbook$;Location=Sheet1;Extended Properties=""""" _
, Destination:=Range("$B$3")).QueryTable
.CommandType = xlCmdSql
.CommandText = Array("SELECT * FROM [Sheet1]")
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.PreserveColumnInfo = True
.ListObject.DisplayName = "Sheet1"
.Refresh BackgroundQuery:=False
End With End Sub
Sub ClearData() ClearData Macro
Columns("B:C").Select
Selection.ClearContents
ActiveWorkbook.Queries("Sheet1").Delete
End Sub
On my computer (and several other computers from my company) this file works without any problems. However, on some computers, I receive
"Run-time error: '438': Object doesn't support this property or
method"
in VBA, it highlights the row ActiveWorkbook.Queries("Sheet1").Delete.
To explain a bit how it should work, I have an Online Excel file that contains a list with entries. There are 2 buttons, first that pulls data from that file, second one that deletes the data. I would have used only one button, that would refresh the database, but, for some reason, I keep receiving an error related to a query that already exists.
I apologize if this was asked before, but I am just begining to work with VBA and any support is more than welcomed.
Apparently Workbook objects do not have a Queries collection. Did you maybe mean ActiveWorkbook.Worksheets("Sheet1").Delete?

Error when refreshing query

I'm recording a macro to automate the process of gathering P/E ratio information from a website for various stock tickers. The macro involves two web queries. I recorded them both the exact same way, and they both use the same URL. One of these queries works fine, and the other gives an
error 1004 "Application-defined or object-defined error"
I have pasted the code for that query below.
ActiveWorkbook.Queries.Add Name:="MSFT PE Ratio (TTM) Range, Past 5 Years", _
Formula:= _
"let" & Chr(13) & "" & Chr(10) & "
Source = Web.Page(Web.Contents(""https://ycharts.com/companies/" & ActiveCell.Value & "/pe_ratio""))," & Chr(13) & "" & Chr(10) & " Data3 = Source{3}[Data]," & Chr(13) & "" & Chr(10) & " #""Changed Type"" = Table.TransformColumnTypes(Data3,{{""Column1"", type text}, {""Column2"", type number}, {""Column3"", type date}})" & Chr(13) & "" & Chr(10) & "in" & Chr(13) & "" & Chr(10) & " #""Changed Type"""
ActiveWorkbook.Worksheets.Add
ActiveSheet.Name = "query2"
With ActiveSheet.ListObjects.Add(SourceType:=0, Source:=Array( _
"OLEDB;Provider=Microsoft.Mashup.OleDb.1;Data Source=$Workbook$;Location=""" & ActiveCell.Value & "PE Ratio (TTM) Range, Past 5 Years"";Extended Properties=" _
, """"""), Destination:=Range("$A$1")).QueryTable
.CommandType = xlCmdSql
.CommandText = Array( _
"SELECT * FROM [" & ActiveCell.Value & "PE Ratio (TTM) Range, Past 5 Years]")
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.PreserveColumnInfo = True
.ListObject.DisplayName = "_PE_Ratio__TTM__Range__Past_5_Years"
**.Refresh BackgroundQuery:=False** <-- this is where I get the error
End With
This is code for one web query and one database query. Not two webqueries.
Both querytypes can make use of parameters which you can attach to a cell and set to automatically refresh the querytable when that cell changes.
See: Webquery with parameters and http://dailydoseofexcel.com/archives/2004/12/13/parameters-in-excel-external-data-queries/