Get data from web query on drop down change in excel - vba

I had some exceptional requirement. That is i have set of value which will come in the drop down in an Excel cell. While changing the value it will hit the server and fetch data according the drop down selected in the excel.
Finally i have done with some simple steps. Please find it following.
Step 1:- Create your drop down list
Go to Data -> Data validation
Now you have created a drop down in F2 cell as follows:
Now on change we have to create a macro which will fetch data from server:
Step 2 :-
Now you have to click on View Code which will open a code window. There you have to paste the following code.
Private Sub Worksheet_Change(ByVal Target As Range)
'MsgBox "You just changed " & Target.Address
If Target.Address = "$F$2" Then
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;https://www.google.com/", Destination:= _
Range("A1"))
.Name = "MR07"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = True
.WebDisableRedirections = True
.Refresh BackgroundQuery:=False
End With
End If
End Sub
In the above code you can replace your own URL instead of https://www.google.com/
A1 is the starting row of output, which is coming from server. $F$2 is validating the changes coming from drop down. if you not put this condition for all cell changes it will call this function which is not expected.

Related

Web Scraping Data Destination

I am very new to using VBA in Excel. I have a list of hundreds of links that I want to scrape the data from (the links do not have nicely formatted tables, I have to scrape the raw data for what I need). I have a macro built that is working nicely, but the destination is not quite what I'm hoping for.
I want excel to read each url, and then dump the data in the next column over. BUT I want each set of data to appear directly below the previous. So I want all of the data from all of the URLs in the same column. Currently, my macro is putting the data from each URL into its own column.
Example:
My URLs are listed in each row separately in column A. The macro reads A1 and dumps the in Data B1. Then, it shifts that column to the right a bit (so it is now C1), and dumps the data from A2 into (the new) B2.
What I want it to do is read A1, and put the data in B1. Then, read A2 and put the data in B30 (if B29 was the last used row for the data from A1).
Hopefully this makes sense.
Here is the macro I currently have working:
Sub WebScraping()
Dim Erw, Frw, Lrw
Frw = 1
Lrw = Range("A" & Rows.Count).End(xlUp).Row
For Erw = Frw To Lrw
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;" & Range("A" & Erw).Value, Destination:=Range("B" & Erw))
.Name = ""
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
Next Erw
End Sub
And here are some examples of the URLs I am scraping:
http://www.washington.edu/students/timeschd/AUT2018/arctic.html
http://www.washington.edu/students/timeschd/AUT2018/hnrs.html
Thanks for any suggestions!
So the yellow URL is the first in the list, and the red URL is the second in the list. The yellow column to the right is where it is placing the data from the first URL, and the red column to the right is where it is placing the data from the second URL. But I want to it to first place the data from the first URL in Column B, then place the data from the second URL directly below that.
You need to write to the next available row in B (Or add some padding rows in between as well)
Option Explicit
Public Sub WebScraping()
Dim Erw As Long, Frw As Long, Lrw As Long, LrwB As Long
Frw = 1
Lrw = ActiveSheet.Range("A" & ActiveSheet.Rows.Count).End(xlUp).Row
For Erw = Frw To Lrw
LrwB = ActiveSheet.Range("B" & ActiveSheet.Rows.Count).End(xlUp).Row + 1
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;" & ActiveSheet.Range("A" & Erw).Value, Destination:=ActiveSheet.Range("B" & LrwB))
.Name = vbNullString
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
Next Erw
End Sub

Auto refresh Pivot tables on data change

I have a worksheet Data which web scrapes a data based on a dynamic link. There is another PivotTable with pivot tables based on the Data worksheet.
Data worksheet uses the following macro and clears the contents of the cells before web scraping new updated data. This data is updated every 1 minute.
I have the following code which will refresh the pivot tables on data update.
ThisWorkbook.Worksheets("PivotTable").PivotTables("PivotTable1").RefreshTable
Since the data takes about 20 seconds to complete updating, there is no data (as the cell contents are cleared first) for the pivot table to refresh. So, I get an error.
Data uses the following code to update data:
With ThisWorkbook.Worksheets("Data").QueryTables.Add(Connection:= _
"<URL redacted>", Destination:=ThisWorkbook.Worksheets("Data").Range("$A$1"))
.Name = "DataPull"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=True ' .Delete
End With
I have tried updating the .RefreshStyle = xlInsertDeleteCells to .RefreshStyle = xlOverwriteCells. But it overwrites the cells until the end of the rows of the new data. If new data (number of rows) is less than old data's rows, then the old data rows at the end are not deleted.
I only want the data from the latest update to be kept.
How do I auto refresh the pivot tables based on above conditions?
Just set .BackgroundQuery = False so that your query will be performed synchronously (meaning, it will wait for the data to be loaded before doing the pivot refresh).
Try using a do loop while to wait for the scraping to complete.
Do
Err.Clear
On Error Resume Next
Debug.Print Err.Number
ThisWorkbook.Worksheets("PivotTable").PivotTables("PivotTable1").RefreshTable
Debug.Print Err.Number
Loop While Err.Number > 0

Getting Worksheets renamed in excel after importing data from the web

I would like to download the daily prices of our stock exchange using the following code in vba. Although the code works i cant seem to be able to get the sheets renamed to the corresponding day when the price list was obtained.
Dim DownloadDay As Date
DownloadDay = #3/4/2014#
Do While DownloadDay < #4/4/2014#
ActiveWorkbook.Worksheets.Add
Call website(Format(DownloadDay, "YYYYMMDD"))
'INCREMENT THE DAY
Sheets.Add.Name = "DownloadDay"
DownloadDay = DownloadDay + 1
Loop
End Sub
Sub website(sDate As String)
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;http://live.mystocks.co.ke/price_list/" & DownloadDay & "/", Destination:=Range("$A$1"))
.Name = DownloadDay
'To rename each work sheet with the corresponding day'
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlTables
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
End Sub
This line of VBA will set the name of your worksheet:
Sheets("Sheet2").Name = "NewName"
You can not use "/" the character in the name though, and the variable is set to a date not a string.
Try this.
.name = CSTR(FORMAT(DownloadDay,"YYYYMMMDD"))

Using VBA to Grab Stock Info from the Web

I have been attempting to write a macro that grabs financial information from the internet and pastes it into my macro for further analysis. I have been trying to use a query from one of my existing connections (MSN stock quotes).
with my code (below) I have been able make the query pop up but what I can't figure out how to do is to enter anything into the box that pops up. What I'm basically looking for is how to (after the code I have listed below) tell excel to type in certain values into the box that pops up and click "OK" to run the query.
below is my code that initiates the query box asking for tickers
With ActiveSheet.QueryTables.Add(Connection:= _
"FINDER;C:\Program Files\Microsoft Office\Office12\QUERIES\MSN MoneyCentral Investor Stock Quotes.iqy" _
, Destination:=Range("$A$1"))
.Name = "MSN MoneyCentral Investor Stock Quotes"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = False
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingAll
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = True
.Refresh BackgroundQuery:=False
End With
Sean, have a looksie at VB.NET as opposed to VBA - you'll find it much more conducive to things like this. As for good libraries for financial data - check out this library:
https://code.google.com/p/yahoo-finance-managed/

Recorded Macro in Excel fails on run

Sub Macro3()
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;http://..." _
, Destination:=Range("Sheet6!$G$23"))
''// The line above fails with the error:
''// "Run-time error '-2147024809 (80070057)':
''// The destination range is not on the same worksheet
''// that the Query table is being created on."
.Name = _
"?tmp=toolbar_FlvTube_homepage&prt=..."
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
End Sub
The recorded macro fails as described in the comment.
You recorded the macro while Sheet 6 was active, but are now trying to run it on a different sheet. To run the macro for the current active sheet, simply change the code as follows:
Sub Macro3()
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;http://..." _
, Destination:=ActiveSheet.Range("$G$23"))
...
End With
End Sub
Edit: (in response to comment):
I need to be able to paste the results of the query to a different sheet then the active one since the macro can be run at any time and must be paste to the same location everytime. Perhaps there is a way to change your active sheet with code?
The error happens when the two sheets are different, so if you want the magic to happen on a particular sheet, you should specify that sheet instead of using ActiveSheet. The following code would always place the QueryTable on Sheet6:
Sub Macro3()
With Sheet6.QueryTables.Add(Connection:= _
"URL;http://..." _
, Destination:=Sheet6.Range("$G$23"))
...
End With
End Sub