VBA Web scraping GetElementById - Select 1 Option drop down menu - vba

Issue Description:
I have a complex VBA code for web scraping - using it to open Web Page, update fields by using GetElementById and then click "Save" button on specific page to create a new record.
There is only one Element, where I need to select with VBA 1 of 7 options called "Full R" from the list menu, but after I inspected Element on web page I'm not sure how to achieve it, as there are no Options available in Inspected Element (see screenshot below).
Here is the Element Inspected:
My Element id is "TakeoverCorporateActionType" and on web page I can choose from 7 different options when I click on dropbox menu
My goal is always to select same option "Full R" from 7 Options, but i'm not entirely sure how to do it.
There is a full private_sub I'm running as part of more complex code (believe this will enough to understand
Private Sub UpdateFXDATA(client As String, url As String, postNow As Boolean, finalRate As Boolean)
Dim editUrl As String, caUrl As String
Dim element As Variant
editUrl = "MY URL" 'hidden from security reasons
caUrl = "My URL 2" 'hidden from security reasons
lastRow = ws.Range("E12389").End(xlUp).Row
Set ie = GrabIE(ie, url)
Do While ie Is Nothing
Application.Wait (DateAdd("s", 1, Now))
MsgBox "please login to " & url & " in Internet Explorer."
Loop
For Each c In ws.Range("K3:K" & lastRow)
Application.Goto c
If c.Offset(0, 3).Value <> "Data Not Found" Then
On Error GoTo 0
Else
ie.Navigate url & editUrl
waitIE ie
Application.Wait (DateAdd("s", 1, Now))
ie.Document.getelementbyid("TakeoverCorporateActionType").Value = "Full Redemption"
ie.Document.getelementbyid("ExDate").Value = c.Offset(0, -4).Value
ie.Document.getelementbyid("ExDatecutoff1").Value = c.Offset(0, -4).Value
ie.Document.getelementbyid("RecordDate").Value = c.Offset(0, -4).Value
ie.Document.getelementbyid("ActionRequiredDate").Value = c.Offset(0, -4).Value
ie.Document.getelementbyid("InstrumentCode").Value = c.Value
ie.Document.getelementbyid("UnitOfOldStock").Value = 1
ie.Document.getelementbyid("NewShares").Value = 0
ie.Document.getelementbyid("ExternalEventIdentifierGenericOption1").Value = c.Offset(0, -2).Value
ie.Document.getelementbyid("Compulsory").Value = 1
If c.Offset(0, 2).Value = "" Then
ie.Document.getelementbyid("CashAmount").Value = 0.001
Else
ie.Document.getelementbyid("CashAmount").Value = c.Offset(0, 2).Value
End If
End If
Application.Wait (DateAdd("s", 0.5, Now))
ie.Navigate ("javascript:btnSave_Click();")
Application.Wait (DateAdd("s", 1, Now)) 'added additional time to prevent fail
Next c
The Part of the code focused on my problem relates to this line:
ie.Document.getelementbyid("TakeoverCorporateActionType").Value = "Full Redemption"

Problem resolved. Issue was on my side, when I inspect Element, there were no options visible in the code, while when I opened whole view-source I could see them and saw that option names have different names, that's why my code couldn't work.
So this code works fine:
ie.Document.getelementbyid("TakeoverCorporateActionType").Value = "FullRedemption"

Related

excel vba to send mail from excel sheet

i am trying to send mails from excel sheet, my vba is not working
error i am getting is "object doesn't suppoer this property or method" for all getelement statement.
I have data in excel sheet like recieptent, subject, message.
code i am using is,
Sub Macro2()
Do Until IsEmpty(ActiveCell)
URL = Selection.Value 'https://mail.google.com/mail/u/0/?ui=2&view=cm&fs=1&tf=1
Set ie = CreateObject("InternetExplorer.Application")
With ie
.Visible = 1
.navigate URL
While .Busy Or .readyState <> 4
DoEvents
Wend
End With
Dim Doc As HTMLDocument
Set Doc = ie.document
ie.document.getElementByName(":oa").Value = ActiveCell.Offset(0, 1).Value 'reciever_mail_id
ie.document.getElementById(":op").Value = ActiveCell.Offset(0, 2).Value 'subject
ie.document.getElementById(":nq").Value = ActiveCell.Offset(0, 3).Value 'message
ie.document.getElementById(":p0").Click 'submit button
ActiveCell.Offset(0, 4).Value = "mail sent"
ActiveCell.Offset(1, 0).Select
ie.Quit
Loop
End Sub
*my gmail id is already logged in, so no need to sign in.
**i am beginner in VBA, need help from expert.
GetElementByName does not exist.
You want to use GetElementsByName which will return an HTMLCollection rather than a simple HTMLElement.
So if there is only one element with that name write this : ie.document.getElementsByName(":oa")(0).Value

Excel VBA to Enter Data Online With IE and Loop Through Rows

I'm trying to use Excel VBA to pull the info from columns A2-D2 and enter it into the web site and then click the "Next" button. The code below is what I have so far which works fine for entering the info found on row 2 only.
I'm hoping to achieve that IE opens a new window, enters the values in cells A2 through D2, clicks the "Next" button, and then loops to open another new IE window and enters the values in cells A3 through D3 until it hits an empty cell.
Here are the current numbers that I'm using for testing, http://imgur.com/a/88XEF.
Thanks in advance for any suggestions.
Sub FillInternetForm()
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
'create new instance of IE. use reference to return current open IE if
'you want to use open IE window. Easiest way I know of is via title bar.
IE.Navigate "https://mygift.giftcardmall.com/Card/Login?returnURL=Transactions"
'go to web page listed inside quotes
IE.Visible = True
While IE.busy
DoEvents 'wait until IE is done loading page.
Wend
'pause if needed
Application.Wait Now + TimeValue("00:00:02")
IE.Document.All("CardNumber").Value = ThisWorkbook.Sheets("sheet1").Range("a2")
IE.Document.All("ExpirationMonth").Value = ThisWorkbook.Sheets("sheet1").Range("b2")
IE.Document.All("ExpirationYear").Value = ThisWorkbook.Sheets("sheet1").Range("c2")
IE.Document.All("SecurityCode").Value = ThisWorkbook.Sheets("sheet1").Range("d2")
'presses the next button
Set tags = IE.Document.GetElementsByTagname("Input")
For Each tagx In tags
If tagx.Value = "Next" Then
tagx.Click
Exit For
End If
Next
End Sub
You just need to wrap your code in a loop..
Dim i as integer
i = 2
do while (ThisWorkbook.Sheets("sheet1").cells(i, 1).value <> "")
'your code from Application.wait line to end of next button click
i = i + 1
loop
This assumes an empty row can be identified by column A being empty. You could change the condition on the while loop if this assumption is bad
sorry had to right a new answer because the formatting was going weird
ok makes sense, you just need to move the start of your loop further up the code so it encases the creation of the IE object and the navigation, you should also close the IE window before opening a new one:
move the chunk:
Dim i as integer
i = 2
do while (ThisWorkbook.Sheets("sheet1").cells(i, 1).value <> "")
right up to the top right after:
Sub FillInternetForm()
Add the following lines after the line "Next" right at the bottom but still enclosed by the loop
IE.Quit
Set IE = Nothing
I was able to accomplish what I wanted with the following. Thanks to those that commented.
Sub FillInternetForm()
Range("A2").Select
Do Until IsEmpty(ActiveCell)
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
'create new instance of IE. use reference to return current open IE if
'you want to use open IE window. Easiest way I know of is via title bar.
IE.Navigate "https://mygift.giftcardmall.com/Card/Login?returnURL=Transactions"
'go to web page listed inside quotes
IE.Visible = True
While IE.busy
DoEvents 'wait until IE is done loading page.
Wend
'pause if needed
Application.Wait Now + TimeValue("00:00:02")
IE.Document.All("CardNumber").Value = ActiveCell.Value
ActiveCell.Offset(0, 1).Select
IE.Document.All("ExpirationMonth").Value = ActiveCell.Value
ActiveCell.Offset(0, 1).Select
IE.Document.All("ExpirationYear").Value = ActiveCell.Value
ActiveCell.Offset(0, 1).Select
IE.Document.All("SecurityCode").Value = ActiveCell.Value
ActiveCell.Offset(1, -3).Select
'presses the next button
Set tags = IE.Document.GetElementsByTagname("Input")
For Each tagx In tags
If tagx.Value = "Next" Then
tagx.Click
Exit For
End If
Next
Loop
End Sub

VBA to find text from webpages

I have created Macro which gives me all URLs present on any webpages.
We just need to provide the URL and it gives us the all links present in that webpage and paste it in one column
Private Sub CommandButton4_Click()
'We refer to an active copy of Internet Explorer
Dim ie As InternetExplorer
'code to refer to the HTML document returned
Dim html As HTMLDocument
Dim ElementCol As Object
Dim Link As Object
Dim erow As Long
Application.ScreenUpdating = False
'open Internet Explorer and go to website
Set ie = New InternetExplorer
ie.Visible = True
ie.navigate Cells(1, 1)
'Wait until IE is done loading page
Do While ie.READYSTATE <> READYSTATE_COMPLETE
Application.StatusBar = "Trying to go to website…"
DoEvents
Loop
Set html = ie.document
'Display text of HTML document returned in a cell
'Range("A1") = html.DocumentElement.innerHTML
Set ElementCol = html.getElementsByTagName("a")
For Each Link In ElementCol
erow = Worksheets("Sheet4").Cells(Rows.Count, 1).End(xlUp).Offset(1, 0).Row
Cells(erow, 1).Value = Link
Cells(erow, 1).Columns.AutoFit
Next
'close down IE, reset status bar & turn on screenupdating
'Set ie = Nothing
Application.StatusBar = ""
Application.ScreenUpdating = True
ie.Quit
ActiveSheet.Range("$A$1:$A$2752").removeDuplicates Columns:=1, Header:=xlNo
End Sub
Now can anyone will help me to create macro to find particular text from all these URLs present in column and if that text is present then in next column it should print text "text found".
Example if we search text "New" then it should print text "Text found" in next column of the URL.
Thank you.
The key would be the function Instr, if it finds the string "New", it returns the position where it begins, otherwise it returns 0.
i=1
do until trim(Cells(i,1).Value) = vbNullString
if instr(Cells(i,1).Value,"New") then
Cells(i,2).value="Text found"
end if
i=i+1
loop
Similar to above.
Dim a As Variant
a = 2
While Cells(a, 1) <> "" And Cells(a + 1, 1) <> ""
If InStr(Cells(a, 1), "new") = 0 Then
Else
Cells(a, 2) = "Text Found"
End If
a = a + 1
Wend

Copy url from browser into Excel

My company uses short URLs for marketing campaigns, which redirect to URLs with tracking. For example:
example.com/campaign
will redirect to:
example.com/utm_source=one&utm_medium=two&utm_term=three&utm_content=four&utm_campaign=five
I have an excel file with a list of short URLs, but the tracking URLs are not listed. The current process is to manually open each link and copy the URL.
Is there a way to copy the tracking URLs into excel using VBA?
So far I've explored the following:
Dim ie As Object
Set ie = CreateObject("Internetexplorer.Application")
ie.Navigate Cells(1, 1)
But I can't find a way to then copy the new URL back into excel.
I've also tried:
Dim link As Hyperlink
Cells(1, 1).Select
Set link = ActiveSheet.Hyperlinks.Add(Anchor:=Cells(1, 1), Address:=Cells(1, 1))
link.Follow
Cells(1, 2) = link.Address
However this just (unsurprisingly) brings back the original hyperlink.
How can I bring back the tracking URL? Ideally without having to install any other software (as my company's IT department is a nightmare) but if that's the only option then so be it.
I believe this will do what you want:
Set webClient = CreateObject("WinHttp.WinHttpRequest.5.1")
webClient.Option(6) = False
webClient.Open "GET", Cells(1, 1), False
webClient.send ("")
Cells(1, 2) = webClient.GetResponseHeader("Location")
I got the idea from this other answer.
What you are trying to do is get the location you're redirecting to.
Redirects follow a standard process described here... and I'm just grabbing the appropriate header from the response.
You want the .LocationURL property
Const READYSTATE_COMPLETE As Long = 4
Dim IE As Object
Set IE = CreateObject("InternetExplorer.Application")
IE.Navigate Cells(1, 1).Value
'// Wait for page to finish loading
While IE.Busy Or IE.ReadyState <> READYSTATE_COMPLETE
DoEvents
Wend
Cells(1, 2).Value = IE.LocationURL

Using VBA to retrieve information from google, trying to set up a stop when google doesn't work as expected

Sub test()
Dim IE As New InternetExplorer
Dim city$, state$
Dim i As Integer
Dim dd As Variant
Dim doc As HTMLDocument 'variable for document or data which need to be extracted out of webpage
Set doc = IE.document
i = 2 'start row
'Setting the variables
city = Cells(i, 1).Value 'City variable for search
state = Cells(i, 2).Value 'State variable for search
Do Until city = ""
city = Cells(i, 1).Value 'City variable for search
state = Cells(i, 2).Value 'State variable for search
'Search google for state and county
URL = "www.google.com/?safe=active&ssui=on#q=" + city + "+" + state + "+county&safe=active&ssui=on"
IE.navigate URL
IE.Visible = False
Do
DoEvents
Loop Until IE.readyState = READYSTATE_COMPLETE
Application.Wait (Now() + TimeValue("00:00:005")) ' For internal page refresh or loading
dd = doc.getElementsByClassName("_eF")(0).innerText
' Now, trim the name to take the State out
dd = Left(dd, WorksheetFunction.Search(",", dd) - 1)
'set county value
Cells(i, 3).Value = dd
'MsgBox dd
i = i + 1
'Setting the next variables
city = Cells(i, 1).Value 'City variable for search
state = Cells(i, 2).Value 'State variable for search
Loop
MsgBox "The macro has finished running"
End Sub
So far this is what I've got thanks to the help of user #BruceWayne
However sometimes google doesn't prompt up the necessary information if the town doesn't have a county or if it is too obscure. I need the macro to record this and keep going. However, every time I try to make an if statement or a fail check the macro stops and prompts the debug window because it is trying to retrieve doc.getElementsByClassName("_eF")(0).innerText.
What can I do to check if doc.getElementsByClassName("_eF")(0).innerText is there and if it is not to record this and keep going?
Thanks!
The methods of retrieving elements from a collection is a zero based index but the .Length property is one based. In other words, the first in the collection of a single element is at position 0 but the length is 1.
if cbool(doc.getElementsByClassName("_eF").LENGTH) then _
dd = doc.getElementsByClassName("_eF")(0).innerText
I just dealt with this problem recently. I wrote a Function to test if the element I wanted was on the screen. The function is below. I am sure you can modify it for your needs.
Function ScreenTest(oDoc as Object, sLookUp As String) As Boolean
'tests to see if a particular screen is loaded by passing the element name associated with what you would expect to see on the screen
'pass oDoc has your IE document
Dim sTest As String
On Error Resume Next
sTest = oDoc.getElementsByName(sLookUp).item(0).Value
If Err.Number <> 0 Then ScreenTest = False Else: ScreenTest = True
On Error GoTo 0
End Function
To put this in your code, do this:
If ScreenTest(doc, "_eF") Then
dd = doc.getElementsByClassName("_eF")(0).innerText
' Now, trim the name to take the State out
dd = Left(dd, WorksheetFunction.Search(",", dd) - 1)
'set county value
Cells(i, 3).Value = dd
'MsgBox dd
i = i + 1
End If
'... continue with rest of code