We have a Excel-List of URLs with a lot of parameters.
The problem is: The first time you follow a link, you get redirected to a ADFS-Login, which cuts some of the Parameters, since they have a maximum URL-length.
My question: Is there a possibility to tell excel (be it via VBA or default) to use an existing Session?
I tried some shennenigans, for example via Chrome: Find the Window handle for a Chrome Browser or to take an existing IE-Window: http://www.mrexcel.com/forum/excel-questions/553580-visual-basic-applications-macro-already-open-ie-window.html While I get an existing Window, it seems like it always gets redirected and the URL cut. Is there anyhow a possibility to make this?
Please try this and post feedback
Open Sheet1
In Column A, from row 2 create your list of URLS
Insert ActiveXControl Microsoft Web Browser WebBrowser1
Size the control to your needs
Insert Control Button outside the bounds of the browser
Change name of the button to NextButton
Open Code Editor (Alt+F11)
In Sheet1 place the below code
Dim currentURLRow As Integer ''Sheet level variable
Sub NextButton_Click()
On Error Resume Next
Dim url As String
''VBA evaluates second expression even when the first of OR is true. So on error resume next helps here
If currentURLRow = 0 Or Trim(Cells(currentURLRow, 1)) = "" Then
''First time or loop back
currentURLRow = 2
Else
currentURLRow = currentURLRow + 1
End If
On Error GoTo 0 ''reset error so you know of any (good) errors
url = Cells(currentURLRow, 1)
''Sheet1.WebBrowser1.Silent = True ''Uncomment this if you are seeing lot of script errors that you dont want to see
WebBrowser1.Navigate url
Debug.Print WebBrowser1.Document.body.InnerHTML ''' Here you can do magic if the urls you are navigating are serialisable to objects :)
End Sub
Now the first time you navigate to the site, you should be prompted for user name and password, on click of next, your session to saved.
Related
I'm using MSAccess VBA with Selenium VBA, Chromedriver
I am able to get to a website, login, find the button to click to start download, and get that download to save to the location I want.
I want to track the progress.
I have opened a new Chrome window (tab) and navigated to chrome:\\downloads\ and switched my driver window to that window.
I've used the following code I found on stack overflow, in a loop to monitor the progress.
downloadPercentage = wd.ExecuteScript("return document.querySelector('downloads-manager').shadowRoot.querySelector('#downloadsList downloads-item').shadowRoot.querySelector('#progress').value")
It returns this error ...cannot return properties of null...
My download is present on the page. I can get the file name.
When I open the developer tab on the downloads page and enter the same query selector path into the console I get the same thing. It returns null.
If I manually (or maybe even using VBA haven't tried) click the button for a second download, then all of a sudden that same code returns a value of 100. (It may catch it at a lower percent. The download is too fast for me to catch that in debug mode.)
What would cause the selector to not be present for one download, but then present for the next?
Here's the code that's in question.
Function getDownloadedFileName(wd as ChromeDriver) As String
Dim startTime As Date
'I'm using this method because opening a second ChromeDriver instance and going to the chrome://downloads/ page returns a clean slate (no downloads) and this method works for me.
wd.ExecuteScript ("window.open()")
wd.SwitchToNextWindow
wd.Get "chrome://downloads/"
startTime = Now()
Dim downloadPercentage As Integer
Do While DateDiff("s", startTime, Now()) < 120 And downloadPercentage < 100
'This is the line that returns the Javascript error ... Cannot read properties of null ...
downloadPercentage = wd.ExecuteScript("return document.querySelector('downloads-manager').shadowRoot.querySelector('#downloadsList downloads-item').shadowRoot.querySelector('#progress').value")
If (downloadPercentage = 100) Then
getDownLoadedFileName = wd.ExecuteScript("return document.querySelector('downloads-manager').shadowRoot.querySelector('#downloadsList downloads-item').shadowRoot.querySelector('div#content #file-link').text")
Exit Do
End If
Loop
wd.SwitchToPreviousWindow
End Function
I'll appreciate any help on this. Thanks!
In Access VBA I have written a routine (subform Click event) to display in a text box on the main form the contents of a field in the selected row of the subform, when the user clicks on a row in a subform.
I created a simple form/subform application to test the code and it works perfectly. However, when I port this code into the production application I am updating, I get error 13. My debug code shows that both variables (MyRsMF and MyFormMF.Recordset) are type 9 (Object). I have reduced my routine’s code to the sample below.
I accept that there is a difference between my test application environment and my production app. However, I have run out ideas and would be grateful for any suggestions as to how I can find the difference(s).
I am working with W10 and Access 365.
My code is:
Private Sub Form_Click()
'
Dim MyFormMF As Form
Dim MyRsMF As Recordset
' Get the Form's recordset
Set MyFormMF = Forms.TB.Form
Debug.Print VarType(MyRsMF) & vbTab & VarType(MyFormMF.Recordset)
Set MyRsMF = MyFormMF.Recordset <<----<<< Error 13 occurs here
Set MyFormMF = Nothing
Set MyRsMF = Nothing
End Sub
I'm a journalist. I spend countless hours copying brief passages from various webpages, and then pasting those passages - along with attribution to the websites I found them - into web-based articles.
For example, many of my articles have passages which look like this:
The Mexican finance minister wrote:
The euro exchange rate is, strictly speaking, too low for the German economy's competitive position.
I want to use VBA to do the following:
(1) When I highlight the text I want to copy - in the example "The euro exchange rate is, strictly speaking, too low for the German economy's competitive position"
(2) I would also automatically copy the url where the text comes from (in this case, http://www.cnbc.com/2017/02/06/german-finance-minister-agrees-euro-too-low-for-germany.html)
(3) So when I paste the text into my blog, it would automatically paste the text and ALSO the url. In other words, I would end up with what I wrote above.
I think the write script is IE.LocationURL to automatically determine the url I'm at. And I know how to launch Internet Explorer and navigate to a web page.
But I don't know how to put the script together.
Here's my attempt:
Const READYSTATE_COMPLETE = 4
' Declare Windows API function for setting active window
Declare Function SetForegroundWindow Lib "user32" _
Alias "SetForegroundWindow" (ByVal Hwnd As Long)As Long
' Declare Internet Explorer object
Dim IE As SHDocVw.InternetExplorer
Sub Main
' create instance of InternetExplorer
Set IE = New InternetExplorer
' using your newly created instance of Internet Explorer
With IE
SetForegroundWindow IE.HWND
.Visible = True
.Navigate2 "http://www.cnbc.com/2017/02/06/german-finance-minister-agrees-euro-too-low-for-germany.html"
' Wait until page we are navigating to is loaded
Do While .Busy
Loop
Do
Loop Until .ReadyState = READYSTATE_COMPLETE
On Error Resume Next
If Err Then
'Do Nothing
Else
'Copy the selected text
SendKeys "^c"
' Here's where I'm trying to copy the url
debug.print.IE.LocationURL
End With
End If
' Tidy Up
Set IE = Nothing
End Sub
I would then run another script to automatically log into my publishing platform and paste the copied text and url info. Ideally, it would be pasted in the format shown at the top of this post (with linked text and then indented quote).
But if I just have the copied text and url, that would still save me a lot of time.
I use Nuance Dragon Naturally Speaking to run my VBA scripts.
But I'm lost. Please help steer me in the right direction! Thanks!
UPDATE: I guess what I really need is a way to store the url as a string. I can then later write the string (and just paste the selected text the old-fashioned way, with control-v.)
So does anyone know how to read and store the url as a string or value?
Playing around with a bunch of possibilities, I think I finally created a script which works.
To do it, I
(1) first have to copy the url manually from the web page by highlighting the url and the copying it to clipboard using control-c on my keyboard;
(2) select (i.e. manually highlight with my keyboard) the text within the article which I want to copy.
Here's the script:
Sub Main
Dim MyData As DataObject
Dim strClip As String
Set MyData = New DataObject
MyData.GetFromClipboard
strClip = MyData.GetText
Wait 5
SendKeys "^c"
'I have code here to open up the webpage where I'm inserting the information
SendKeys "^v"
' The line above pastes the text I have selected
SendKeys strClip
' The line above sends the url which I previously saved
End Sub
I don't know if it's the most efficient or elegant solution, but it works.
I'm sure that this is a repeat post, but I've been unable to find exactly what I need. I currently live on a college campus where I need to enter a username and password to use the internet. I'm trying to automate the process (using .bat files) so I can run a server application on startup without ever pressing a key. Unfortunately, I keep getting an error at line 9 char 9: "Object does not support this property or method: 'getElementByID'" I tried replacing .getElementByID with .getElementByName, but it didn't make a difference.
Call Main
Function Main
Set IE = WScript.CreateObject("InternetExplorer.Application", "IE_")
IE.Visible = True
IE.Navigate "https://caserver.jbu.edu/auth/perfigo_weblogin.jsp"
Wait IE
With IE.Document
.getElementByID("username").value = "name"
.getElementByID("password").value = "password"
.getElementByID("tx_voputilities_pi1[sign_in]")(0).Submit
End With
End Function
Sub Wait(IE)
Do
WScript.Sleep 500
Loop While IE.ReadyState < 4 And IE.Busy
End Sub
I think that the problem has to do with the webpage. The "username" and "password" fields are elements inside a table. I couldn't find an example webpage with similar properties, so I'm at a loss. I'm not sure if the authentication page can be viewed from off-campus, so I attached a picture with some of the HTML (or at least the elements list)
My research so far:
VBScript to Launch a website login
IE 9 error getElementbyId: Object required
VBScript get contents of html table text input field
VBS website login script - "Object required" error
Like I said, I'm using batch files to start a computer. I wanted this to be a quick (and, yes, crude) solution, but there may be a better way to automate webpage logon. If there is a [quick] better way, feel free to point me in the correct direction.
Your problem may be that they don't use id, but name instead.
There is no function getElementByName, but there is getElementsByname
You could try
.getElementsByName("username")[0].value = "name"
.getElementsByName("password")[0].value = "password"
(I should probably note that this assumes the first element with these names is the one you are looking for - names do not have to be unique hence the array returned and the lack of a getElementByName function, whereas id is unique)
As a further note, your problem is not that the fields are in a table - getting an element from the DOM with either getElementById or getElementsByName do not take positioning into account, only that the element exists somewhere within the parent you are running the function on.
I am writing a macro that will scrape my company's internal SAP site for vendor information. For several reasons I have to use VBA to do so. However, I cannot figure out why I keep getting these three errors when I attempt to scrape the page. Is it possible that this has something to do with the UAC integrity model? Or is there something wrong with my code? Is it possible for a webpage using http can be handled differently in internet explorer? I am able to go to any webpage, even other internal webpages, and can scrape each of those just fine. But when i attempt to scrape the SAP page, i get these errors. The error descriptions and when they occur are:
800706B5 - The interface is unknown (occurs when I place breakpoints before running the offending code)
80004005 - Unspecified error (occurs when I don't place any errors and just let the macro run)
80010108 - The Object invoked has disconnected from its clients. (I can't seem to get a consistent occurrence of this error, it seems to happen around the time that something in excel is so corrupted that no page will load and i have to reinstall excel)
I have absolutely no idea what is going on. The Integrity page didn't make much sense to me, and all the research I found on this talked about connecting to databases and using ADO and COM references. However I am doing everything through Internet Explorer. Here is my relevant code below:
Private Sub runTest_Click()
ie.visible = True
doScrape
End Sub
'The code to run the module
Private Sub doTest()
Dim result As String
result = PageScraper.scrapeSAPPage("<some num>")
End Sub
PageScraper Module
Public Function scrapeSAPPage(num As Long) As String
'Predefined URL that appends num onto end to navigate to specific record in SAP
Dim url As String: url = "<url here>"
Dim ie as InternetExplorer
set ie = CreateObject("internetexplorer.application")
Dim doc as HTMLDocument
ie.navigate url 'Will always sucessfully open page, regardless of SAP or other
'pauses the exection of the code until the webpage has loaded
Do
'Will always fail on next line when attempting SAP site with error
If Not ie.Busy And ie.ReadyState = 4 Then
Application.Wait (Now + TimeValue("00:00:01"))
If Not ie.Busy And ie.ReadyState = 4 Then
Exit Do
End If
End If
DoEvents
Loop
Set doc = ie.document 'After implementation of Tim Williams changes, breaks here
'Scraping code here, not relevant
End Function
I am using IE9 and Excel 2010 on a Windows 7 machine. Any help or insight you can provide would be greatly appreciated. Thank you.
I do this type of scraping frequently and have found it very difficult to make IE automation work 100% reliably with errors like those you have found. As they are often timing issues it can be very frustrating to debug as they don't appear when you step through, only during live runs To minimize the errors I do the following:
Introduce more delays; ie.busy and ie.ReadyState don't necessarily give valid answers IMMEDIATELY after an ie.navigate, so introduce a short delay after ie.navigate. For things I'm loading 1 to 2 seconds normally but anything over 500ms seems to work.
Make sure IE is in a clean state by going ie.navigate "about:blank" before going to the target url.
After that you should have a valid IE object and you'll have to look at it to see what you've got inside. Generally I avoid trying to access the entire ie.document and instead use IE.document.all.tags("x") where 'x' is a suitable thing I'm looking for such as td or a.
However after all these improvements although they have increased my success rate I still have errors at random.
My real solution has been to abandon IE and instead do my work using xmlhttp.
If you are parsing out your data using text operations on the document then it will be a no-brainer to swap over. The xmlhttp object is MUCH more reliable. and you just get the "responsetext" to access the entire html of the document.
Here is a simplified version of what I'm using in production now for scraping, it's so reliable it runs overnight generating millions of rows without error.
Public Sub Main()
Dim obj As MSXML2.ServerXMLHTTP
Dim strData As String
Dim errCount As Integer
' create an xmlhttp object - you will need to reference to the MS XML HTTP library, any version will do
' but I'm using Microsoft XML, v6.0 (c:\windows\system32\msxml6.dll)
Set obj = New MSXML2.ServerXMLHTTP
' Get the url - I set the last param to Async=true so that it returns right away then lets me wait in
' code rather than trust it, but on an internal network "false" might be better for you.
obj.Open "GET", "http://www.google.com", True
obj.send ' this line actually does the HTTP GET
' Wait for a completion up to 10 seconds
errCount = 0
While obj.readyState < 4 And errCount < 10
DoEvents
obj.waitForResponse 1 ' this is an up-to-one-second delay
errCount = errCount + 1
Wend
If obj.readyState = 4 Then ' I do these on two
If obj.Status = 200 Then ' different lines to avoid certain error cases
strData = obj.responseText
End If
End If
obj.abort ' in real code I use some on error resume next, so at this point it is possible I have a failed
' get and so best to abort it before I try again
Debug.Print strData
End Sub
Hope that helps.