I need to connect to a website, with credentials then click a link and get the table it brings to my excel sheet.
I'´m already logged in the site and with the following code I get the table updated but this only works while the excel"from web " connection is alive. I mean, first I have to update the DATA-FROM WEB connection refreshing credentials and then running the macro.
Though the web table is always reached and updated I can't get it pasted to the excel sheet if that connection was updated more that an hour ago.
Here is the code:
Sub Descargar_tabla()
Dim IE As Object
Set IE = CreateObject("WScript.Shell")
IE.Run ("""C:\Program Files (x86)\Opera\launcher.exe"" ""https://clientes.invertirenbolsa.com.ar/cart.aspx""")
With ActiveSheet.QueryTables.Add(Connection:="URL;https://clientes.invertirenbolsa.com.ar/cart.aspx", Destination:=Range("$L$40"))
' .Name = "cart"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = False
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlOverwriteCells
.SavePassword = True
.SaveData = True
.AdjustColumnWidth = False
.RefreshPeriod = 0
.WebSelectionType = xlSpecifiedTables
.WebFormatting = xlWebFormattingNone
.WebTables = "2"
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=True
Set IE = Nothing
End With
End Sub
What should I add to the code to use the website open session that I already have in the browser which it doesn't expire?
Related
As lacking of essential IT knowledge, I am looking for your kind help to make codes for getting data (final destination & ETA estimated time of arrival) from website tracking container numbers SEARATES (https://www.searates.com/container/tracking/) to Excel.
I have tried method of using GetElementbyID using IE Internet Explorer but this website browser is seems obsolete and useless.
This SERATES website offer API information but it is too complicated for me to understand.
I am enclosing container tracking list for your testing.
https://drive.google.com/file/d/1E0tuA4pYMDPZMgwjQC8dDs6w6BxqVdXK/view?usp=sharing
Hope someone could give me support on this project which is important for my job of logistics.
Thanks in advance.
How about recording a Macro, for starters.
Sub Macro1()
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;https://drive.google.com/file/d/1E0tuA4pYMDPZMgwjQC8dDs6w6BxqVdXK/view", _
Destination:=Range("$A$1"))
.CommandType = 0
.Name = "view"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
Range("A1").Select
Application.Left = 201.25
Application.Top = 93.25
End Sub
Notice, you have to be signed in. I don't have login creds for this link.
I have a connection that web scrapes based on a dynamic link. So, I cannot set a fixed connection. The following macro creates the connection and then updates the worksheet.
With ThisWorkbook.Worksheets("Data").QueryTables.Add(Connection:= _
"<URL redacted>", Destination:=ThisWorkbook.Worksheets("Data").Range("$A$1"))
.Name = "DataPull"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = False
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
.Delete
End With
This macro runs every 1 minute to update the data. So, this creates a new connection every time it runs. I do not want so many connections to exist as they won't be used anymore.
How do I delete the connection once web scraping is complete?
or Is there a way to set up a single connection that can modify itself based on a variable. The variable is the time intervals which vary based on current time.
I looked at this option
For Each qr In ThisWorkbook.Queries
qr.Delete
Next qr
But there are two other fixed connections which I don't want to delete.
The new connections that are created have the names Connection, Connection1, and so on. Is there a way to delete the connections based on name?
Let's say you want to delete all connections, except Connection1 and Connection2, try...
Dim Conn As WorkbookConnection
For Each Conn In ThisWorkbook.Connections
If Conn.Name <> "Connection1" And Conn.Name <> "Connection2" Then
Conn.Delete
End If
Next Conn
Hope this helps!
Extracting tabular form data from multiple webpages into an excel using VBA macros..!! Currently iam using below link but i could able to only one webpage in the code..i have list of ulr's to get data from...and it has to come in vertical..!! please suggest me.. :)
Sub INDEXdata()
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;http://recorder.maricopa.gov/recdocdata/GetRecDataDetail.aspx?rec=19770000007" _
, Destination:=Range("$A$1"))
.Name = "rec=19770000006"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlSpecifiedTables
.WebFormatting = xlWebFormattingNone
.WebTables = "2,3"
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
End Sub
OK, I don't know how much programming background you have, and I also don't know what parts of the code you posted are specific to that one source url and target location.
But, something like this might work. I made the assumption that the url, the destination, and the name would change for each page you wanted to pull from.
What I did was take the part of the code that looked like it would be true for all of the source pages and destinations, and put that in its own parameterized subroutine. The original routine IndexData just specifies the URL and the destination, and the name, for each copy operation.
Sub IndexData()
GetData("http://recorder.maricopa.gov/recdocdata/GetRecDataDetail.aspx?rec=19770000007" , _
"$A$1", _
"rec=19770000006")
GetData("http://somewhereelse.com/somedata.aspx?rec=12345", _
"$A$2", _
"rec=12345")
GetData("http://anotherurl.com/etc", _
"$A$3", _
"something")
End
Sub GetData(url as string, destination as string, name as string)
With ActiveSheet.QueryTables.Add(Connection:= _
"URL;" & url , Destination:=Range(destination))
.Name = name
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlSpecifiedTables
.WebFormatting = xlWebFormattingNone
.WebTables = "2,3"
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
End Sub
I'm doing a web scrape in VBA (see code below) to btc-e.com to fetch prices of some Cryptocurrency. When to it manually by going to data tab and then clicking on from web It works fine, but When I do it in the Macro I only get back "please wait..."
The page displays "please wait..." as it loads and the macro assumes that is the entire page.
I have been looking for a way to make the macro wait for the full page load and cant find anything.
Any help would be appreciated.
Thanks
With ActiveSheet.QueryTables.Add(connection:="URL;https://btc-e.com", _
Destination:=Range("$A$1"))
.Name = "btc-e"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = False ' was true
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingNone
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False ' True ' was false
.WebDisableRedirections = False 'True ' was false
.Refresh BackgroundQuery:=False
End With
You have to choose a specific table or area of the page you want scraped, otherwise it won't work. The reason is that you are automatically forwarded from the page you are trying to scrape to the page you can actually scrape.
When I chose to scrape the Sell Orders, this is the code I got from the macro recorder:
With ActiveSheet.QueryTables.Add(Connection:="URL;https://btc-e.com", _
Destination:=Range("$B$2"))
.Name = "btc-e"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = True
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlSpecifiedTables
.WebFormatting = xlWebFormattingNone
.WebTables = "3"
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = False
.Refresh BackgroundQuery:=False
End With
As you can see it takes the attribute ".WebTables" which chooses a specific portion of the site. You can choose the portion you want by activating the macro recorder, scraping it through the normal way, choosing the area you want and then looking at the value of WebTables in the resulting code.
Hope this helps!
I have been attempting to write a macro that grabs financial information from the internet and pastes it into my macro for further analysis. I have been trying to use a query from one of my existing connections (MSN stock quotes).
with my code (below) I have been able make the query pop up but what I can't figure out how to do is to enter anything into the box that pops up. What I'm basically looking for is how to (after the code I have listed below) tell excel to type in certain values into the box that pops up and click "OK" to run the query.
below is my code that initiates the query box asking for tickers
With ActiveSheet.QueryTables.Add(Connection:= _
"FINDER;C:\Program Files\Microsoft Office\Office12\QUERIES\MSN MoneyCentral Investor Stock Quotes.iqy" _
, Destination:=Range("$A$1"))
.Name = "MSN MoneyCentral Investor Stock Quotes"
.FieldNames = True
.RowNumbers = False
.FillAdjacentFormulas = False
.PreserveFormatting = False
.RefreshOnFileOpen = False
.BackgroundQuery = True
.RefreshStyle = xlInsertDeleteCells
.SavePassword = False
.SaveData = True
.AdjustColumnWidth = True
.RefreshPeriod = 0
.WebSelectionType = xlEntirePage
.WebFormatting = xlWebFormattingAll
.WebPreFormattedTextToColumns = True
.WebConsecutiveDelimitersAsOne = True
.WebSingleBlockTextImport = False
.WebDisableDateRecognition = False
.WebDisableRedirections = True
.Refresh BackgroundQuery:=False
End With
Sean, have a looksie at VB.NET as opposed to VBA - you'll find it much more conducive to things like this. As for good libraries for financial data - check out this library:
https://code.google.com/p/yahoo-finance-managed/