IE11 Automation via Excel VBA - company webpage - vba

First I'm very new to trying to automate IE via Excel VBA.
That being said, I'm working to automate a login to a company-specific webpage (only accessible to our employees). The goal is to automate the login (employee number, password and click Login). I find Firefox to be particularly helpful in identifying fields so that's what I'm using in the screenshot.
I found some code online to navigate to a webpage and enter something into a search box. I've modified that as follows (the included link is not real).
Finally to the issue. If I enter a webpage like www.google.com for example, all will execute fine. But when I change to my company link, the code freezes at the Do While and I get the error shown. So my question is why it works for a general webpage but not for my company specific one? If I comment-out that line, I still get the disconnected error when debugging. Assuming that issue is an easy one to resolve, have I also properly identified the field?
Hopefully I've included enough info for you. If not, please let me know what else may be required. Thanks in advance for your help!
Error
'start a new subroutine called SearchBot
Sub SearchBot()
'dimension (declare or set aside memory for) our variables
Dim objIE As InternetExplorer 'special object variable representing the IE browser
'initiating a new instance of Internet Explorer and asigning it to objIE
Set objIE = New InternetExplorer
'make IE browser visible (False would allow IE to run in the background)
objIE.Visible = True
'navigate IE to this web page (a pretty neat search engine really)
objIE.navigate "http://sampletext.asp"
'wait here a few seconds while the browser is busy
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
'enter value in the employee number box
objIE.document.getElementByName("txtEmployeeNum").Value = "123456"
Employee Number

Correct name of the method is getElementsByName.
You also want to operate on element of collection returned by this method, not whole collection. Using (0) index will allow to work on 1st element of collection.
Change:
objIE.document.getElementByName("txtEmployeeNum").Value = "123456"
to:
objIE.document.getElementsByName("txtEmployeeNum")(0).Value = "123456"
With such corrected code, you should step through code with F8 in VB Editor. For example by hovering over, see if objIE.Busy ever gets FALSE and especially if objIE.readyState ever reaches 4 - if only 3, try objIE.readyState < 3 instead.
EDIT:
Try replacing:
Dim objIE As InternetExplorer
Set objIE = New InternetExplorer
with:
Dim objIE As Object
Set objIE = GetObject("new:{D5E8041D-920F-45e9-B8FB-B1DEB82C6E5E}")
You may also need to change objIE.Navigate with objIE.Navigate2

Related

VBA Intenet Explorer issue with loading and achiving readystate specific webbsite

Dear Friends Vba Masters,
I have Issue connected with no possibility of achieved state "ready state" for my page. For others it is works for ex. google.com but I don`t know why for my page not. Strange things is that this page always load to view even in state .Visible = false , and object value od empty. I would like to create macro to login for this page and control some activities - on this page you can create some excel file to analysis. I will be very kind full. Mac.
Sub OpenKWM()
Dim URL As String
Dim IE As Object
Set IE = CreateObject("internetExplorer.application")
URL = "kwm.kromi.de"
With IE
.Navigate URL
.Visible = True
End With
Do: Loop Until IE.readyState = READYSTATE_COMPLETE
End Sub
Strange things is that this page always load to view even in state .Visible = false , and object value od empty.
If you check the relevant doc, you will find that this is actually a reasonable result. Whether the created Internet Explorer window is visible depends on whether you have called the Navigate method or the GoSearch method. When you call these methods, the window will become visible.
Edit:
After testing, I found that the cause of the problem may be Enable Protected Mode in IE settings. You can try to disable it: open IE -> Tools -> Internet Options -> security tab -> uncheck Option Enable Protected Mode. Like this:

GetElementbyID not showing for IE.Dcoument (VB.Net)

I'm trying to make a webscraping program in Visual Studio 2019 using VB.Net Framework but I'm running into an issue. I have the below code to locate an open IE window so that I can manipulate it. The issue I'm running into is when I use IE.Document. the list that appears does not show getelementsbyid. Here is a snippet of my code, it's able to find the appropriate IE window and bring it into focus but I need to be able to manipulate the elements on the page.
I have also added the Com References
Microsoft HTML Object Library
Microsoft Internet Control
Dim SWS as new SHDocVw.ShellWindows
Dim IE as SHDocVw.InternetExplorer
For Each IE in SWS
If IE.locationName = "XXXXXXXX" Then Exit For
NExt
'To get focus on IE window
IE.Visible = False
IE.Visible = True
IE.Document.
I would appreciate any insight since I'm at a loss as to why getelementbyid won't show.
As an alternative, you can try to refer to the code example below that may help you to get/ set values.
If you want to set data then refer to the line below.
IE.Document.All.Item("fname").Value = "ABC"
If you want to fetch data then refer to the line below.
Console.WriteLine(IE.Document.All.Item("fname").Value)

Create VBA that returns search top 5 search results in Google and microsoft edge

Below is the error I'm getting when entering the codeI'm new to VBA but I'd like to create a macro that returns the top 5 search results of the item listed in cell A15 using either google chrome or microsoft edge? I tried adding in the code below and got an error.
VBA can only automate the Internet Explorer browser. So if you want to use google chrome or Microsoft edge then this approach will not work for you.
Below is the sample code that populates Google search results to a worksheet.
Option Explicit
Public Sub GetLink()
Dim ie As New InternetExplorer
Dim url As String
url = "https://google.co.uk/search?q=" + Sheet1.Range("A2").Value
With ie
.Visible = True
.navigate url
While .Busy Or .readyState < 4: DoEvents: Wend
Sheet1.Range("B2").Value = .document.querySelector("#search div.r [href*=http]").href
Sheet1.Range("C2").Value = .document.querySelector("#search div.r [href*=http]").innerText
.Quit
End With
End Sub
Output:
Reference:
How to get the first search result link of a google search using VBA?
Below is another helpful thread. Its solution uses the XMLHTTP object.
Using VBA in Excel to Google Search in IE and return the hyperlink of the first result
Further, you can try to check the above example and try to modify the sample as per your own requirements.

Connect to data after having found an already open IE window using shell application

VBA code to interact with specific IE window that is already open
Above is a thread to find and go to an already open instance of IE using shell applications in VBA. AFTER I found the open IE instance I am looking for, I need to query the tables from that IE page without using it's URL. The reason that I cannot use it's URL is that this IE page is a generic 'result' page that opens in a separate window after doing a search on the main website, so if I use the URL of the result page, which is: https://a836-acris.nyc.gov/DS/DocumentSearch/BBLResult, it will return an error. Are there any other methods that allow querying tables without using URL connections, like a "getElements" for tables?
K.Davis, Tim William: you are correct in your assumptions. The first part of my code/project opens up a search page: objIE.navigate "https://a836-acris.nyc.gov/DS/DocumentSearch/BBL" and through it I submit a search form. The second part (outlined above in the first paragraph) opens up a result page (pop-up). I am trying to automate the retrieving of the tables from that page. I tried using QueryTables.Add method, the way I am familiar with to connect to the data/webpage requires an URL. If I use the URL from the result page it returns an error, thus I am looking for suggestions/help on how I could otherwise connect. That said I am able to retrieve elements of the page using 'getElements' method but not able to query tables. There are other ways to connect to the data source using the QueryTables.Add method, see, https://learn.microsoft.com/en-us/office/vba/api/excel.querytables.add but I am not familiar with these other methods. Hope this clarifies a bit.
I haven't experienced a problem with this as although you have an intermediate window the final IE window resolves to being the main IE window with focus. I was able to grab the results table with the following code using the indicated search parameters:
Option Explicit
Public Sub GetInfo()
Dim IE As New InternetExplorer
With IE
.Visible = True
.navigate "https://a836-acris.nyc.gov/DS/DocumentSearch/BBL"
While .Busy Or .readyState < 4: DoEvents: Wend
With .document
.querySelector("option[value='3']").Selected = True
.querySelector("[name=edt_block]").Value = 1
.querySelector("[name=edt_lot]").Value = "0000"
.querySelector("[name=Submit2]").Click
End With
While .Busy Or .readyState < 4: DoEvents: Wend
Dim hTable As HTMLTable
Set hTable = .document.getElementsByTagName("table")(6)
'do stuff with table
.Quit
End With
End Sub
You can copy a table via clipboard. Any tick windings appear in the right place but as empty icons.
For clipboard early bound go VBE > Tools > References > Microsoft-Forms 2.0 Object Library.
If you add a UserForm to your project, the library will get automatically added.
Dim clipboard As DataObject
Set clipboard = New DataObject
clipboard.SetText hTable.outerHTML
clipboard.PutInClipboard
ThisWorkbook.Worksheets("Sheet1").Cells(1, 1).PasteSpecial
Late bound use
Dim clipboard As Object
Set clipboard = GetObject("New:{1C3B4210-F441-11CE-B9EA-00AA006B1A69}")

Automation Errors: 800706B5, 80004005, 80010108 appear for internal SAP site scrape

I am writing a macro that will scrape my company's internal SAP site for vendor information. For several reasons I have to use VBA to do so. However, I cannot figure out why I keep getting these three errors when I attempt to scrape the page. Is it possible that this has something to do with the UAC integrity model? Or is there something wrong with my code? Is it possible for a webpage using http can be handled differently in internet explorer? I am able to go to any webpage, even other internal webpages, and can scrape each of those just fine. But when i attempt to scrape the SAP page, i get these errors. The error descriptions and when they occur are:
800706B5 - The interface is unknown (occurs when I place breakpoints before running the offending code)
80004005 - Unspecified error (occurs when I don't place any errors and just let the macro run)
80010108 - The Object invoked has disconnected from its clients. (I can't seem to get a consistent occurrence of this error, it seems to happen around the time that something in excel is so corrupted that no page will load and i have to reinstall excel)
I have absolutely no idea what is going on. The Integrity page didn't make much sense to me, and all the research I found on this talked about connecting to databases and using ADO and COM references. However I am doing everything through Internet Explorer. Here is my relevant code below:
Private Sub runTest_Click()
ie.visible = True
doScrape
End Sub
'The code to run the module
Private Sub doTest()
Dim result As String
result = PageScraper.scrapeSAPPage("<some num>")
End Sub
PageScraper Module
Public Function scrapeSAPPage(num As Long) As String
'Predefined URL that appends num onto end to navigate to specific record in SAP
Dim url As String: url = "<url here>"
Dim ie as InternetExplorer
set ie = CreateObject("internetexplorer.application")
Dim doc as HTMLDocument
ie.navigate url 'Will always sucessfully open page, regardless of SAP or other
'pauses the exection of the code until the webpage has loaded
Do
'Will always fail on next line when attempting SAP site with error
If Not ie.Busy And ie.ReadyState = 4 Then
Application.Wait (Now + TimeValue("00:00:01"))
If Not ie.Busy And ie.ReadyState = 4 Then
Exit Do
End If
End If
DoEvents
Loop
Set doc = ie.document 'After implementation of Tim Williams changes, breaks here
'Scraping code here, not relevant
End Function
I am using IE9 and Excel 2010 on a Windows 7 machine. Any help or insight you can provide would be greatly appreciated. Thank you.
I do this type of scraping frequently and have found it very difficult to make IE automation work 100% reliably with errors like those you have found. As they are often timing issues it can be very frustrating to debug as they don't appear when you step through, only during live runs To minimize the errors I do the following:
Introduce more delays; ie.busy and ie.ReadyState don't necessarily give valid answers IMMEDIATELY after an ie.navigate, so introduce a short delay after ie.navigate. For things I'm loading 1 to 2 seconds normally but anything over 500ms seems to work.
Make sure IE is in a clean state by going ie.navigate "about:blank" before going to the target url.
After that you should have a valid IE object and you'll have to look at it to see what you've got inside. Generally I avoid trying to access the entire ie.document and instead use IE.document.all.tags("x") where 'x' is a suitable thing I'm looking for such as td or a.
However after all these improvements although they have increased my success rate I still have errors at random.
My real solution has been to abandon IE and instead do my work using xmlhttp.
If you are parsing out your data using text operations on the document then it will be a no-brainer to swap over. The xmlhttp object is MUCH more reliable. and you just get the "responsetext" to access the entire html of the document.
Here is a simplified version of what I'm using in production now for scraping, it's so reliable it runs overnight generating millions of rows without error.
Public Sub Main()
Dim obj As MSXML2.ServerXMLHTTP
Dim strData As String
Dim errCount As Integer
' create an xmlhttp object - you will need to reference to the MS XML HTTP library, any version will do
' but I'm using Microsoft XML, v6.0 (c:\windows\system32\msxml6.dll)
Set obj = New MSXML2.ServerXMLHTTP
' Get the url - I set the last param to Async=true so that it returns right away then lets me wait in
' code rather than trust it, but on an internal network "false" might be better for you.
obj.Open "GET", "http://www.google.com", True
obj.send ' this line actually does the HTTP GET
' Wait for a completion up to 10 seconds
errCount = 0
While obj.readyState < 4 And errCount < 10
DoEvents
obj.waitForResponse 1 ' this is an up-to-one-second delay
errCount = errCount + 1
Wend
If obj.readyState = 4 Then ' I do these on two
If obj.Status = 200 Then ' different lines to avoid certain error cases
strData = obj.responseText
End If
End If
obj.abort ' in real code I use some on error resume next, so at this point it is possible I have a failed
' get and so best to abort it before I try again
Debug.Print strData
End Sub
Hope that helps.