I am trying to submit form details. I am unable to share HTML details so tried to explain below.
I have a menu control page https://www.abcmenu.aspx.
This page calls the url https://www.abc-employee.aspx using javascript:void(0), which bring a form in same page when I click on employee menu item.
However, the page does not refresh nor does another page load, and the URL in the address bar remains unchanged.
Here is a sample view of the website:
I need to fill the form details and hit submit button.
The below code gives run time error stating object required.
Set htmldoc = ie.document
Dim emp as mshtml.ihtmlinputelement
Set emp = htmldoc.getelementbyid("fld_emp")
emp.value = 357690
Dim subm as mshtml.ihtmlelememt
Set subm = htmldoc.getelementbyid("btnk_sub")
Subm.click
I tried to debug.print all elements under form tag, but it does not return the elements in the form.
When I execute the code, it returns only the main menu page details and not form elements.
Here is the code I tried to print HTML elements
Dim htmla as mshtml.ihtmlelement
Dim htmlas as mshtml.ihtmlelementcollection
For each htmla in htmlas
Debug.print htmla.innertext
Next htmla
Why am I not able to access HTML elements inside form that was opened in the main menu page?
If you are trying to access iframe elements on the ASP.Net web page using VBA then you can refer to the example below may help you to solve your issue.
Sub demo()
Dim URL As String
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
URL = "https://example.com"
IE.navigate URL
Do While IE.readyState = 4: DoEvents: Loop
Do Until IE.readyState = 4: DoEvents: Loop
Dim elemCollection As IHTMLElementCollection
Debug.Print (IE.document.getElementsByTagName("iframe")(0).contentDocument.getElementsByName("fname")(0).Value)
Set IE = Nothing
End Sub
Output:
Related
I've written a script in vba using IE to get the titles of different hotel names from a webpage. The hotel names traverse multiple pages through pagination.
My scraper can keep clicking on the next button successfully while parsing the titles from each page until ther is no more click left to perform. The parser is doing is job just perfect. All I wish to know is a simple logic I've asked below.
My question: How the content of each page is rightly coming through even when I didn't use this Set Htmldoc = IE.document line just after the .click? When a click is initiated, the scraper goes to a new page with new content. How come it gets updated with new content from each page as my defined do loop comes after with IE block?
This is the script:
Sub GetTitles()
Const Url As String = "https://www.tripadvisor.com/Hotels-g147237-Caribbean-Hotels.html"
Dim IE As New InternetExplorer, Htmldoc As HTMLDocument, post As Object, R&
With IE
.Visible = True
.navigate Url
While .Busy = True Or .readyState < 4: DoEvents: Wend
Set Htmldoc = .document
End With
Do
For Each post In Htmldoc.getElementsByClassName("listing") ''how this "Htmldoc" gets updated
With post.getElementsByClassName("property_title")
If .Length Then R = R + 1: Cells(R, 1) = .Item(0).innerText
End With
Next post
If Not Htmldoc.querySelector(".standard_pagination span[onclick*='pagination_next']") Is Nothing Then
Htmldoc.querySelector(".standard_pagination span[onclick*='pagination_next']").Click
Application.Wait Now + TimeValue("00:00:05")
''I didn't use anything like "Set Htmldoc = IE.document" but it still works flawlessly
Else:
Exit Do
End If
Loop
IE.Quit
End Sub
The script is not faulty. Though, you are using it without fully understanding is certainly troublesome.
When you do this Set Htmldoc = .document you are setting the IE's document for later use.
When you do this Htmldoc.querySelector(".standard_pagination span[onclick*='pagination_next']").Click javascript comes in play and updates the content of the page (i.e document).
You may believe that the document has changed but its only being updated. In reality,there is no navigation happening at all.
Add the following and see how the page/document remains the same, just the content changes.
'/ Url before Next button click
Debug.Print "Before Click " & Htmldoc.Url
Htmldoc.querySelector(".standard_pagination span[onclick*='pagination_next']").Click
'/ Url after Next button click
Debug.Print "After Click " & Htmldoc.Url
Since the document, once set remains the same and the updated content has same layout/DOM (that is how mostly programmers code, most likely all the pages are being rendered using a template) hence your code works perfectly fine. Net to net for your do loop, nothing changed.
Set Htmldoc = .document
gets a pointer to the DOM. When it changes the Htmldoc is pointing at the new content. No need to do a new Set Htmldoc
I've written a script in vba in combination with IE to click on some dots available on a map in a web page. When a dot is clicked, a small box containing relevant information pops up.
Link to that website
I would like to parse the content of each box. The content of that box can be found using class name contentPane. However, the main concern here is to generate each box by clicking on those dots. When a box shows up, it looks how you can see in the below image.
This is the script I've tried so far:
Sub HitDotOnAMap()
Const Url As String = "https://www.arcgis.com/apps/Embed/index.html?webmap=4712740e6d6747d18cffc6a5fa5988f8&extent=-141.1354,10.7295,-49.7292,57.6712&zoom=true&scale=true&search=true&searchextent=true&details=true&legend=true&active_panel=details&basemap_gallery=true&disable_scroll=true&theme=light"
Dim IE As New InternetExplorer, HTML As HTMLDocument
Dim post As Object, I&
With IE
.Visible = True
.navigate Url
While .Busy = True Or .readyState < 4: DoEvents: Wend
Set HTML = .document
End With
Application.Wait Now + TimeValue("00:0:07") ''the following line zooms in the slider
HTML.querySelector("#mapDiv_zoom_slider .esriSimpleSliderIncrementButton").Click
Application.Wait Now + TimeValue("00:0:04")
With HTML.querySelectorAll("[id^='NWQMC_VM_directory_'] circle")
For I = 0 To .Length - 1
.item(I).Focus
.item(I).Click
Application.Wait Now + TimeValue("00:0:03")
Set post = HTML.querySelector(".contentPane")
Debug.Print post.innerText
HTML.querySelector("[class$='close']").Click
Next I
End With
End Sub
when I execute the above script, it looks like it is running smoothly but nothing happens (I meant, no clicking) and it doesn't throw any error either. Finally it quits the browser gracefully.
This is how a box with information looks like when a dot gets clicked.
Although I've used hardcoded delay within my script, they can be fixed later as soon as the macro starts working.
Question: How can I click each of the dots on that map and collect the relevant information from the popped-up box? I only expect to have any solution using Internet Explorer
The data are not the main concern here. I would like to know how IE work in such cases so that I can deal with them in future cases. Any solution other than IE is not I'm looking for.
No need to click on each dots. Json file has all the details and you can extract as per your requirement.
Installation of JsonConverter
Download the latest release
Import JsonConverter.bas into your project (Open VBA Editor, Alt + F11; File > Import File)
Add Dictionary reference/class
For Windows-only, include a reference to "Microsoft Scripting Runtime"
For Windows and Mac, include VBA-Dictionary
References to be added
Download the sample file here.
Code:
Sub HitDotOnAMap()
Const Url As String = "https://www.arcgis.com/sharing/rest/content/items/4712740e6d6747d18cffc6a5fa5988f8/data?f=json"
Dim IE As New InternetExplorer, HTML As HTMLDocument
Dim post As Object, I&
Dim data As String, colObj As Object
With IE
.Visible = True
.navigate Url
While .Busy = True Or .readyState < 4: DoEvents: Wend
data = .document.body.innerHTML
data = Replace(Replace(data, "<pre>", ""), "</pre>", "")
End With
Dim JSON As Object
Set JSON = JsonConverter.ParseJson(data)
Set colObj = JSON("operationalLayers")(1)("featureCollection")("layers")(1)("featureSet")
For Each Item In colObj("features")
For j = 1 To Item("attributes").Count - 1
Debug.Print Item("attributes").Keys()(j), Item("attributes").Items()(j)
Next
Next
End Sub
Output
I'm creating a macro that will navigate to a login page, log in, navigate to another page and scrape data, and then loop through 100-200 more pages scraping data from each.
So far I've gotten it to the point of logging in, navigating to the second page, and scraping the first bit of data. But so far the only way I can get it to work is if the second page opens in a new window. Since I ultimately have to go through 100-200 pages, I'd rather not use a new window for each one.
For this example let's just say that the only data I'm trying to scrape is the page title.
Option Explicit
Sub admin_scraper()
Dim ie As Object
Dim doc As Object
' Get through log in page
Set ie = CreateObject("internetexplorer.application")
With ie
.navigate "http://example.com/login" 'Page title is "Page 1"
.Visible = True
End With
While ie.Busy Or ie.readyState <> 4
DoEvents
Wend
ie.document.forms(0).all("Username").Value = "user"
ie.document.forms(0).all("Password").Value = "abc123"
ie.document.forms(0).submit
'Navigate to second page and pull page title
Set ie = CreateObject("internetexplorer.application") '***Line in question
With ie
.navigate "http://example.com/Products" 'Page title is "Page 2"
.Visible = True
End With
While ie.Busy Or ie.readyState <> 4
DoEvents
Wend
Set doc = ie.document
Debug.Print doc.Title
End Sub
*** If I include this line the code works as expected (console prints "Page 2"), but it opens the second page in a new window. If I don't include this line, the second page opens smoothly in the same window, but the console prints "Page 1."
Any way I can get it to open each new page in the same window while making sure it pulls data from the new page? Or if it has to be in a new window, any way to automatically close the old window each time?
I'm trying to simulate the interaction with Google through the IE app and going through the DOM to get the classes I need and all is fine, stepping though the code, except the .Click action which doesn't cause a crash but it doesn't do anything (page doesn't navigate) - Code and screenshot of HTML below:
Option Explicit
Private Sub Test_Automation()
Dim ie, doc, eInput, eButton, eButtons As Object
Dim sURL, sTest As String
Set ie = CreateObject("internetexplorer.application")
sURL = "https://www.google.co.uk/?gfe_rd=cr&ei=IpDvWK72LsjCaJCbjKAL&gws_rd=ssl"
sTest = "Test"
With ie
.Visible = True
.Navigate sURL
End With
Do While ie.Busy Or ie.readyState <> 4
DoEvents
Loop
Set doc = ie.document
Set eInput = doc.getElementByid("lst-ib")
Set eButtons = doc.getElementsByTagName("input")
eInput.Value = sTest
For Each eButton In eButtons
If (eButton.getattribute("name") = "btnK") Then
eButton.Click
Exit For
End If
Next
End Sub
Any advise on what I'm doing wrong would be great!
You can get rid of your For...Next loop at the bottom and replace it with this to click the button:
doc.forms(0).submit
The 0 can be changed to another number (such as 1 or 2) to click on a different button. If there are multiple buttons on a page that can be clicked on it will just take some trial and error to find out which number matches the button you want to click.
I am a rookie in VBA excel.
There is a web page application in which
i need to click a button, the source of which is
<em class="x-btn-arow" unselectable="on">
<button class= x-btn-text" id="ext-gen7576" style="" type="button">Actions</button>
Sub xx()
Dim IE As Object
Dim doc As HTMLDocument
Dim l As IHTMLElement
Dim lo As IHTMLElementCollection
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.Navigate "http://theapplicationlink"
Do
DoEvents
Loop Until IE.ReadyState = 4
Set doc = IE.Document
Set lo = doc.getElementsByTagName("button")
For Each l In lo
If l.getAttribute("class") = "x-btn-text" Then
l.click
End If
Next
End Sub
it doesn't throw any error but it doesn't click the button.
I cannot use ID as it keeps on changing each time i launch the application.
Also the class and type is same for other buttons also.
Forgive me for any technical errors
Any help will be a huge favour here.
There is an id. Does it change completely or does part of it remain the same? If it were you could partial match on the bit that remain the same using a CSS selector.
That aside you could use:
objIE.document.querySelector("button[class*= x-btn-text]").Click
This uses a CSS selector to target the element of button[class*= x-btn-text]. Which will be the first element with button tag having attribute class with value containing x-btn-text.
"button" is not a HTML tag. use a "Tag". let me give you an example here. Replace the "strTagName" with a HTML tage that inlcudes the thing you want to click.
Dim objTag As Object
For Each objTag In objIE.document.getElementsByTagName(strTagName)
If InStr(objTag.outerHTML, "x-btn-text") > 0 Then
objTag.Click
Exit For
End If
Next