first time post here cause for the first time I couldn't find an answer on Stack.
The program I'm fiddling with goes to an API website - gets the passport key - and uses that to load the info in XML and then parses it. The code is very long so I'm only posting the bit that results in the error.
myURL = "https://api.smartbidnet.com/project?PassportKey=" & PassportKey & "&ResultType=xml" 'This is the API URL where we use our passport key to get our data
IE.Navigate myURL 'Go to Webpage
Dim XDoc As Object
Set XDoc = CreateObject("MSXML2.DOMDocument")
XDoc.async = False
XDoc.validateOnParse = False
XDoc.Load (ThisWorkbook.Path & myURL) 'Here we will load the XML info from the web page
Set lists = XDoc.DocumentElement 'lists will be used to access XML nodes
Set Projects = lists.SelectNodes("//root/projects")
Now heres the weird part:
If I open excel by selecting "Blank Workbook" the code works just fine from the editor.
If I save the blank workbook and then try to run it or If I simply right click the desktop and open a blank sheet rather than workbook and try to run it. the last line results in
Runtime 91 error
Any help is of course greatly appreciated.
The bit of code before the above is simply navigating to the site to grab the passport key for the API and build the myURL variable.
Related
I want to import the web page source code in excel what I see using View Page Source option in Chrome. But when I import it using below code, it doesn't import all content. The values that I'm looking for do not get displayed on web page.
I'm also unable to locate the element using getElementsByClassName or other methods.
Private Sub HTML_VBA_Excel()
Dim oXMLHTTP As Object
Dim sPageHTML As String
Dim sURL As String
'Change the URL before executing the code
sURL = "http://pntaconline/getPrDetails?entry=8923060"
'Extract data from website to Excel using VBA
Set oXMLHTTP = CreateObject("MSXML2.ServerXMLHTTP")
oXMLHTTP.Open "GET", sURL, False
oXMLHTTP.send
sPageHTML = oXMLHTTP.responseText
'Get webpage data into Excel
' If longer sourcecode mean, you need to save to a external text file or somewhere,
' since excel cell have some limits on storing max characters
ThisWorkbook.Sheets(1).Cells(1, 1) = sPageHTML
MsgBox "XMLHTML Fetch Completed"
End Sub
Data I want to import is IDs and Name:
So you need to understand the DOM in order to realize why this isnt loading everything.
XMLHTTP is going to load that specific resource you requested. A lot of web pages, sorry pretty much all web pages, load extra resources after the initial request is done.
If you're missing stuff, it's probably loaded on a different network request. So open up your DevTools in Chrome, make sure Network tab is recording, and watch how many network requests go in and out when you load your target page.
Essentially, this if you're using XMLHTTP, you'd have to simulate each of those to get the requests you want to scrape.
EDIT
So you're just kind of pasting the data response into Excel.
Better to create HTMLDocument variable then set the response from XMLHTTP to be the response like here: https://learn.microsoft.com/en-us/previous-versions/windows/desktop/ms762275(v=vs.85)
set xmlhttp = new ActiveXObject("Msxml2.XMLHTTP.3.0");
xmlhttp.open("GET", "http://localhost/books.xml", false);
xmlhttp.send();
Debug.print(xmlhttp.responseText);
Dim xString as String
xSring = xmlhttp.responseText
'search the xString variable
You can then split that response for the sheet or search it and extract the values in VBA memory, rather than print to the sheet.
You could also set the xString responseText as the innerHTML for a new HTMLDocument variable
Dim xHTML as HTMLDocument
Set xHTML.innertext = xString
I am very new to VBA and had a question regarding how to click an href link in Internet Explorer. There are multiple href's on the source page. I have never encountered this and it has been giving me a hard time! I have looked on this website searching for answers but decided to ask here.
Below I have listed the code I have, up to the point where I encounter the problem, as well as the Source Code on Internet Explorer.
I commented out what I have tried and listed the error I received.
Code Below:
Sub ()
Dim i As Long
Dim URL As String
Dim IE As Object
Dim objElement As Object
Dim objCollection As Object
User = "User"
Pwd = "Pwd"
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
URL = "URL.com"
IE.Navigate URL
Do While IE.ReadyState <> 4
DoEvents
Loop
IE.Document.getElementById("txtUsername").Value = User
IE.Document.getElementById("txtPassword").Value = Pwd
IE.Document.getElementById("btnSubmit").Click
'IE.getElementByClassName("txtTerms").Click - Runtime Error 438
'IE.getElementByTagName("Claims Management").Click - Runtime Error 438
'Set HREF = IE.Document.getElementsByClassName("txtTerms")
'For Each HREF In IE.Document.getElementsByTagName("Claims").Click - No error occurs, nothing happens.
End Sub
Internet Explorer Source Code:
<table id="tblContent">
<tr>
<td class="txtTerms"><a href='href url 1'>Claims</a>
<br>Download<br>Create<br><a class='terms' href='href url 2'
target='terms'>Terms</a><br><br></td>
</tr>
My question would be, how to get VBA to click only on 'href url 1'?
Let me know if any additional information is needed. I apologize for my level of VBA but I am excited to learn more!
Thanks for the help!
In HTML, href is a property of the type <a> (link) which contains an absolute or relative path.
For example:
Questions
... will show as "Questions" and, if you click it, will bring you to www.stackoverflow.com/questions/. Note that "www.stackoverflow.com" has been added automatically since the path is relative.
Facebook
... will show as "Facebook" and, if you click it, will bring you to www.facebook.com. In this case, the path is absolute.
Although your HTML code is incomplete, I guess that all the links you want to navigate are contained in the table having id="tblContent". If that's the case, then you can get all the links (tagName == 'a') in that table and store the values in a collection:
Dim allHREFs As New Collection
Set allLinks = IE.Document.getElementById("tblContent").getElementsByTagName("a")
For Each link In allLinks
allHREFs.Add link.href
Next link
You can then decide to navigate them and do what you have to do one by one:
For j = 1 To allHREFs.Count
IE.Navigate URL + allHREFs(j) '<-- I'm assuming hrefs are relative.
'do your stuff here
Next href
I'm hoping someone can help. I'm trying to speed up the process of filling a webform that must be completed dozens or hundreds of times with information stored in excel.
To do this, I need one button to open an IE window and navigate to a certain website's login page (I've figured this bit out). The user can then log in and navigate to the form that needs to be filled. Then, I'd like the user to be able to return to the excel page, click another button, which will automatically fill several drop downs and text boxes.
Within Excel, I already have some code to allow the user to search for the particular set of information that needs to go to the form, so all they should have to do is click the button to transfer it over. The first bit of the code is just this:
Public IE As Object
Public Sub OpenIE()
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.Navigate "loginpage url"
End Sub
Where I'm having trouble, however, is having a different function access the same IE window once the user has logged in and navigated to the form. Right now I've got this:
Sub FillMacro()
Dim sh As Object, oWin As Object
Set sh = CreateObject("Shell.Application")
For Each oWin In sh.Windows
If TypeName(oWin.document) = "HTMLDocument" Then
Set IE = oWin
Exit For
End If
Next
IE.Visible = True
Application.Wait (Now + TimeValue("00:00:01"))
IE.document.getElementById("idec").Value = "John"
Application.Wait (Now + TimeValue("00:00:01"))
IE.document.getElementById("idee").Value = "Smith"
End Sub
Most of that I've gotten from other posts on this forum, and while I'm a bit of a novice at this, the problem seems to be that for some reason VBA can't find the text boxes with the id of LastName or FirstName. What's more, the IE.Visible = True doesn't bring the IE window back to the foreground, so I'm trying to find the proper line to do that. When I try to run the code, I get an "Object Required" error at:
IE.document.getElementById("idec").Value = "John"
I've tried searching this site and will continue to look, but any help in the meantime would be greatly appreciated!
On the Internet Explorer page, here is the line for the first text box I'm trying to fill:
<input name="componentListPanel:componentListView:1:component:patientLastNameContainer:patientLastName" class="input300" id="idec" type="text" maxlength="60" value="">
Why not automate logging process as well? Login could be stored in Excel and its value read by macro from cell.
As Tim Williams suggests, if there is Iframe on website, use (for me it works only with contentwindow included):
IE.document.getElementById("iFrameIdHere").contentwindow.document.getElemeântById("idec").Value = "John"
Instead of Application.Wait use:
Do Until IE.ReadyState = 4 And IE.Busy = False
DoEvents
Loop
It will save you a lot of time when page loads fast and prevent errors when loading exceeds wait time. Use it ONLY after page reloads (meaning after navigating or anything what causes page reloads, especially .click on HTML elements.
Use early binding, it's a bit faster than creating objects. It can increase performance by a few percent based on page loading speed, the faster pages load, the bigger increase.
Set IE = New InternetExplorer
Finally, you can toggle loading pictures, depending on whether you need to download images from website.
Public Sub ShowPictures(ByVal EnabledStatus As Boolean)
Public ScrapingCancelled as Boolean
Dim obj_Shell
Dim v_Result As Variant
Set obj_Shell = CreateObject("WScript.Shell")
'Reads the registry key that determines whether 'Show pictures' Internet Explorer advanced setting is enabled
v_Result = obj_Shell.RegRead("HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\Display Inline Images")
Select Case v_Result
Case "yes" 'Pictures are displayed
If EnabledStatus = False Then _
obj_Shell.RegWrite "HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\Display Inline Images", "no", "REG_SZ"
Case "no" 'Pictures are not displayed
If EnabledStatus = True Then _
obj_Shell.RegWrite "HKEY_CURRENT_USER\Software\Microsoft\Internet Explorer\Main\Display Inline Images", "yes", "REG_SZ"
Case Else
ScrapingCancelled = True
End Select
End Sub
No images loaded:
ShowPictures (0)
Images loaded:
ShowPictures (1)
A good practice is to set value to 1 in the end of macro.
I am trying to login the mentioned website using vba but unable to get through.kindly assist. The website i am trying to logging contains a form where we are suppose to fill our credentials.though, the website is opening using the below enlisted code but nothing happens post that.
Dim HTMLDoc as HTMLDocument
Dim My Browser as Internet Explorer
Sub MYRED()
Dim MyHTML_Element As IHTMLElement
Dim MyURL as String
On Error GoTo Err_Clear
MyURL = "https://www.Markit.com"
Set MyBrowser = New InternetExplorerMedium
MyBrowser.Silent = True
MyBrowser.navigate MyURL`enter code here`
MyBrowser. Visible = True`enter code here`
Do
Loop Until MyBrowser.readyState = READYSTATE_COMPLETE
Set HTMLDoc = MyBrowser. Document
'Useridele = "input_5ac52c16-cbba-4fb4-b284-857fac1f55fd"
MyBrowser.document.getElementById ("input_5ac52c16-cbba-4fb4-b284- 857fac1f55fd").Value = "user#example.com"
'HTMLDoc.all.input_5ac52c16-cbba-4fb4-b284-857fac1f55fd.Value = "atul.sanwal#markit.com" 'Enter your email id here
passele = "input_b14b8d03-75b6-49b5-a61a-602672036046"
MyBrowser.document.getElementById("input_b14b8d03-75b6-49b5-a61a- 602672036046").Value = "***************"
'HTMLDoc.all.passele.Value = "12345678" 'Enter your password here
For Each My Html_Element in HTMLDoc.getElementsByTagName("input")
If MyHTML_Element.Type = "submit" Then MyHTML_Element.Click: Exit For
Next
Err_Clear:
If Err <> 0 Then
Err.Clear
Resume Next
End If
End Sub
As I can see this:
"input_5ac52c16-cbba-4fb4-b284-857fac1f55fd")
I'm guessing that this ID is generated every time you are making request
(yep, everytime I'm refreshing this site this element got new ID ), so
You cannot pass static ID as key to get elementById You should try other options to get this specific element You want to fill with data.
Maybe Your library supports gettibng by attribute so, You can get element with atribute
name='username'
One additional proposition :
You are using html to make request via web Interface, but You could try to ommit loging in just to make pure requests.
As I see when filling data and pushing Submit my browser is sending
POST request :
POST /home/UFEExternalSignOnServlet HTTP/1.1
Host: products.markit.com
I dont know how large is your knowledge about WebAPI and etc. but You can write to me if need more help
I am trying to use the following code to geocode a bunch of cities from this website: mygeoposition.com but there seems to be some sort of issue and the variable 'Lat' in the following code always returns empty:
Sub Code()
Dim IE As MSXML2.XMLHTTP60
Set IE = New MSXML2.XMLHTTP60
IE.Open "GET", "http://mygeoposition.com/?q=Chuo-ku, Osaka", False
IE.send
While IE.ReadyState <> 4
DoEvents
Wend
Dim HTMLDoc As MSHTML.HTMLDocument
Dim htmlBody As MSHTML.htmlBody
Set HTMLDoc = New MSHTML.HTMLDocument
Set htmlBody = HTMLDoc.body
htmlBody.innerHTML = IE.responseText
Lat = HTMLDoc.getElementById("geodata-lat").innerHTML
IE.abort
End Sub
I have another code that uses the browser to do the same thing and it works fine with that but it gets quite slow. When I use this code with MSXML, it just doesn't work. Apologies I am a newbie with using VBA for data extraction from website. Please help.
The response contains no content in the geodata-lat element. It appears that client side code is getting that data so your response only is looking at the html that the server generated. I tried this out myself and this is the section of the response you are looking for. You can see it is empty:
If you try an element that has content (geodata-kml-button), it does pull in a value ("Download KML file"). Ignore the ByteArrayToString() call, that was just me testing:
If they don't have a real API then I don't think you can get your data this way.