I'm using VBA to learn programming.
In the process of scraping web pages with createobjct function, I am seriously stuck with problems with the following message box.
I can't go forward with VBA without closing below box. Contents of message is simple, there is no data in that page. (It is EC site)
...
ie.Navigate strURL
ie.Visible = False
'Waiting with do while
Do While (ie.ReadyState <> READYSTATE_COMPLETE Or ie.Busy = True)
DoEvents
Loop
'2seconds wait more
Application.Wait (Now + TimeValue("00:00:02")) ' Here is the stop point with message box.
Is there a way to close the popup message box?
Related
Good afternoon all,
Please forgive me, I am new to VBA and have been practicing within my workbooks but for my work would be highly useful to branch out to Webscraping.
The following is my code.
Sub pullsalesdatafromonline()
Dim IE As Object
Dim doc As HTMLDocument
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.navigate "https://www.peoplestuffuk.com/WFMMCDPRD/Login.jsp"
Do While IE.Busy Or IE.readyState <> 4
Application.Wait DateAdd("s", 1, Now)
Loop
Set doc = IE.document
doc.getElementById("txtUserID").Click
doc.getElementById("txtUserID").Value = Usernameinserted
doc.getElementById("txtPassword").Click
doc.getElementById("txtPassword").Value = PasswordInserted
doc.getElementById("Login").Click
IE.navigate "https://www.peoplestuffuk.com/WFMMCDPRD/LoginSubmit.jsp"
IE.navigate "https://www.peoplestuffuk.com/WFMMCDPRD/ModuleSelection.jsp"
IE.navigate "https://www.peoplestuffuk.com/WFMMCDPRD/dashboard/main/dashboardmain.jsp"
IE.navigate "https://www.peoplestuffuk.com/WFMMCDPRD/rrd/matrixReport.jsp?fromPM=N"
End Sub
Here is the element that I am trying to scrape from;
<INPUT onclick=javaScript:submitForm() class=button-t type=button value=Login>
<SPAN id=Login_span class=buttonbox></SPAN>
I've looked at several other links that have looked into firing the .onclick event but, can't quite seem to figure it out.
The error in VBA highlights the:
IE.navigate "https://www.peoplestuffuk.com/WFMMCDPRD/ModuleSelection.jsp"
line. When my username and password enter, the page refreshes/ tries to exectute and then empties the Login and password info and leaves me on the Login page.
Would greatly appreciate someone's input.
Thank you
If you had a demo username and password to share then there was an efficient way of logging in using xhr. However, as for your current attempt concern, they are not the ids you used them within .getElementById(). Given that the following approach should log you in. Make sure to uncomment the delay, if the script fails to do the doing.
Sub pullsalesdatafromonline()
Dim IE As Object, doc As HTMLDocument
Set IE = CreateObject("InternetExplorer.Application")
IE.Visible = True
IE.navigate "https://www.peoplestuffuk.com/WFMMCDPRD/Login.jsp"
While IE.Busy Or IE.readyState < 4: DoEvents: Wend
Application.Wait Now + TimeValue("00:00:03")
Set doc = IE.document
doc.querySelector("[name='txtUserID']").innerText = "username"
doc.querySelector("[name='txtPassword']").innerText = "password"
' Application.Wait Now + TimeValue("00:00:03")
doc.querySelector("input[value='Login']").Click
End Sub
I thought I was well on my way to success. What I'm attempting to do is Load a URL in IE, input some data, then run a report.
To accomplish this, I've created an InternetExplorer object, used that object to turn on visibility, and navigate to my URL.
Upon getting to the page, I have to supply some data to let the report generator do what it needs to do, then press enter. Once I'm to this point I need to do some navigation to different URLs until I arrive at the report page. To accomplish this, I'm attempting to return to using my InternetExplore object (IE) to navigate, but any time I use IE.anything the 462 error pops.
My question is two fold:
The object was good to begin with, I have to assume it was the use of Application.Send or Application.Wait that has somehow broke my object, why?
What did I do wrong?
Suggestions on how to resolve this?
Dim text As String
Dim IE
Set IE = CreateObject("InternetExplorer.Application")
'
IE.Visible = True
IE.Navigate "http://~~~~~~~~~.net/reports/views/result/reportResult.faces"
'
text = "somedata"
.
Application.Wait (Now + TimeValue("00:00:02"))
Application.SendKeys (text), True
'
Application.Wait (Now + TimeValue("00:00:02"))
Application.SendKeys vbCrLf, True
'
Application.Wait (Now + TimeValue("00:00:02"))
IE.Navigate "http://~~~~~~~~~.net/reports/views/myMrs/myFavorites.faces"
The end product was completely different than where I was headed. Thanks to those who provided input, especially Sorceri who turned me on to focusing on the elements.
Below is the finished code segment (I have more things I do after this but this gets me where I needed to go).
Private Sub OpenReport()
Dim text As String
Dim x As Integer
Dim IE
Set IE = New InternetExplorerMedium
text = "somecode"
'Load link
IE.Visible = True
IE.Navigate "http://~~~~~~~~~/reports/views/result/reportResult.faces"
'Wait for page to load
Do While IE.Busy Or IE.ReadyState <> 4
DoEvents
Loop
'fill in the pcode
IE.Document.getelementbyid("code").Value = text
'Click Submit
IE.Document.getElementsByClassName("btn btn-lg btn-primary btn-block")(0).Click
'Wait for page to load
Do While IE.Busy Or IE.ReadyState <> 4
DoEvents
Loop
'Click on Favorites
IE.Document.getElementsByClassName("myMRSFavoritesLink")(0).Click
'Wait for page to load
Do While IE.Busy Or IE.ReadyState <> 4
DoEvents
Loop
'Click on report
IE.Document.getElementsByName("myFavoritesForm:treeTable_tableId:1:infoButton101644806j_id__v_19")(0).Click
'Wait for page to load
Do While IE.Busy Or IE.ReadyState <> 4
DoEvents
Loop
Click on execute report
IE.Document.getElementsByName("setParametersForm:startReportButton")(0).Click
End Sub
I'm one step away from success on a huge automation project. I've got a dynamic download automated all the way up to the IE11 Save As dialogue box. This is where I'm stuck. I need to 1) enter in a dynamic file name and save to a dynamic folder (all based on a cell value). 2) click Save while overriding any update dialogue boxes
Limitations; I cant download based on the URL if you were going to suggest that. I'm dealing with a secure site and a funky URL that can only be accessed via the For Next loop I have below. I'm limited to dealing with this SaveAs box as far as I'm aware.
Here's what I have so far;
Sub savedownload()
Dim objIE As InternetExplorer
Set objIE = New InternetExplorer
objIE.Visible = True
objIE.navigate Sheet3.Range("A1").Value ' this is where the dynamic URL is updated
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
objIE.document.getElementById("PrintFormat").Value = "Pdf"
Do While objIE.Busy = True Or objIE.readyState <> 4: DoEvents: Loop
Dim list As Object, item As Object
Set list = objIE.document.getElementsByTagName("a")
For Each item In list
If item.innerText = "Export" Then
item.Click
Exit For
End If
Next
Application.Wait (Now + TimeValue("0:00:05"))
Application.SendKeys "{TAB}", True
Application.SendKeys "{TAB}", True
Application.SendKeys "{DOWN}", True
Application.SendKeys "{DOWN}", True
Application.SendKeys "{RETURN}", True
'this is where the dialoge box comes up, but I cant seem to send a dynamic string of text to the file name/path field
End Sub
I have attached a screenshot as well. Would send keys work? Is there a way to call the SaveAs window and drop in a dynamic file name? I can throw a file name at it and get it to save with SendKeys, but I'm thinking there's a better way where I can send an entire dynamic String to the FileName field in the dialogue box.
Thanks for any suggestions.
Screenshot of IE11 SaveAs dialogue box
I'm creating a macro that will navigate to a login page, log in, navigate to another page and scrape data, and then loop through 100-200 more pages scraping data from each.
So far I've gotten it to the point of logging in, navigating to the second page, and scraping the first bit of data. But so far the only way I can get it to work is if the second page opens in a new window. Since I ultimately have to go through 100-200 pages, I'd rather not use a new window for each one.
For this example let's just say that the only data I'm trying to scrape is the page title.
Option Explicit
Sub admin_scraper()
Dim ie As Object
Dim doc As Object
' Get through log in page
Set ie = CreateObject("internetexplorer.application")
With ie
.navigate "http://example.com/login" 'Page title is "Page 1"
.Visible = True
End With
While ie.Busy Or ie.readyState <> 4
DoEvents
Wend
ie.document.forms(0).all("Username").Value = "user"
ie.document.forms(0).all("Password").Value = "abc123"
ie.document.forms(0).submit
'Navigate to second page and pull page title
Set ie = CreateObject("internetexplorer.application") '***Line in question
With ie
.navigate "http://example.com/Products" 'Page title is "Page 2"
.Visible = True
End With
While ie.Busy Or ie.readyState <> 4
DoEvents
Wend
Set doc = ie.document
Debug.Print doc.Title
End Sub
*** If I include this line the code works as expected (console prints "Page 2"), but it opens the second page in a new window. If I don't include this line, the second page opens smoothly in the same window, but the console prints "Page 1."
Any way I can get it to open each new page in the same window while making sure it pulls data from the new page? Or if it has to be in a new window, any way to automatically close the old window each time?
What I'm trying to do here is automate the opening of several helpdesk tickets simultaneously in separate IE tabs. I create a list of ticket numbers in Excel and then loop through the ticket numbers, opening each one.
My code seems to work fine when I open each one in a separate IE instance, but since I've tried to open them in separate tabs of one IE instance, I get an error on the second loop. Here is what I have so far:
Set Tickets = Sheet5.Range("a1", Range("a1").End(xlDown))
Set ie = New InternetExplorerMedium
ie.Visible = 1
apiShowWindow ie.hwnd, SW_MAXIMIZE
For Each Ticket In Tickets
If Ticket <> "" And Not Ticket Like "IM*" And Not Ticket Like "ARS*" And Not Ticket Like "C*" Then
'Load Mantis Page
If Tabbed = False Then
ie.Navigate "http://URL"
Else:
ie.Navigate "http://URL", CLng(2048)
End If
Do
DoEvents
Loop Until ie.ReadyState = 4
'LoginCheck
Set LoginExists = ie.document.getElementById("username")
If LoginExists Is Nothing Then
GoTo SearchForTicket
Else: GoTo Login
End If
Login:
Call ie.document.getElementById("username").SetAttribute("value", "xx")
Call ie.document.getElementById("password").SetAttribute("value", "xx")
ie.document.getElementById("login_form").Submit
Do
DoEvents
Loop Until ie.ReadyState = 3
GoTo SearchForTicket
'Search for Mantis ticket
SearchForTicket:
Application.Wait (Now + TimeValue("0:00:03"))
ie.document.All("bug_id").Value = Ticket
Set AllButtons = ie.document.getElementsByTagName("input")
For Each Button In AllButtons
If Button.Value = "Jump" Then
Button.Click
Exit For
End If
Next
End If
Tabbed = True
Next
It works the first time around, and opens IE, navigates to the page and searches for the ticket. The second time around, it opens the new tab and navigates to the page, but when it tries to search for the second ticket, I get an error saying:
Object doesn't support this property or method"
On line:
ie.document.All("bug_id").Value = Ticket
I've been searching for an answer with no luck so far. Any help would be appreciated.