I am fairly new to web scraping and I started with Selenium in VBA/Excel using Chrome. My target elements are the menu items.
In some websites I can find the elements but I can't get their "href". Here is the destination website. I want the "href" of the menu items in the right side of the page.
This is what I tried:
' geting the menus, working well:
Set mnus = bot1.FindElementsByClass("topmenu")
MenuCounter = 0
For Each mnu In mnus
'getting the href, fails
lnk = mnu.Attribute("href")
I also tried some other ways but no success.
This is a screen shot of the inspect:
Note that I don't only want the href for this specific element (whose href is "art"). I also want the href of other equivalent menu items (except for the first item).
The href value is inside anchor tag Not li element.You need to target anchor tag.Use following css selector.
Set mnus = bot1.FindElementsByCssSelector("li.topmenu >a[href]")
MenuCounter = 0
For Each mnu In mnus
lnk = mnu.Attribute("href")
Or To get all menus link try this.
To get all menus link
Set mnus = bot1.FindElementsByCssSelector("ul.topmenu a[href]")
MenuCounter = 0
For Each mnu In mnus
lnk = mnu.Attribute("href")
Related
I am using the below mentioned code for automation of Edge Browser
The code is working fine except for "If then Else" block.
The complete script is as follows
Dim Obj As New WebDriver
' search for company - SF 1cr and above
Sub EdgeAutoSF1CRA0()
Set Obj = New Selenium.EdgeDriver
Dim ele As WebElement
Dim By As New Selenium.By
Obj.SetCapability "ms:edgeOptions", "{""excludeSwitches"":[""enable-automation""]}"
Obj.Start "edge", ""
Obj.Get "https://*********************"
Obj.Window.Maximize
Obj.FindElementByName("croreAccount").SendKeys ("Search")
Obj.FindElementByXPath("//*[#id='loadSuitFiledDataSearchAction']/div[1]/div[3]/div[4]/img").Click
Obj.FindElementById("borrowerName").SendKeys (ThisWorkbook.Sheets("Sheet1").Range("C5").Value)
Obj.FindElementByXPath("//*[#id='search-button']/ul/li[1]/div/input").Click
Obj.Wait 30000
If Obj.FindElementByCss("#downloadReport").Attribute("Style" = "display") = "none" Then
Obj.FindElementByXPath("//*[#id='three-icons']/ul/li[3]/a/div").Click
Else
Obj.FindElementByXPath("//*[#id='downloadReport']/div").Click
End If
End Sub
In the If then Else statement I want to search for the style attribute of the id "downloadReport" for "display :none"
The code on website is < a href="downloadStatusReport" id="downloadReport" style="display: none;"><div class="download-icon">Download</div></a>
However, code always evaluate the statement as False and proceeds to execute the command "Obj.FindElementByXPath("//*[#id='downloadReport']/div").Click"
The attribute name is style, not capitalized Style.
You can construct the locator so that the desired style value will be a part of locator itself.
You can use FindElements instead of FindElement so it will return you a list of matching elements so as if such element found it will return a non-empty list, otherwise it would be an empty list. With this you can check if returned list is empty or not, as following:
If Not IsEmpty(Obj.FindElementsByCss("[id='downloadReport'][style*='none']")) Then
Obj.FindElementByXPath("//*[#id='three-icons']/ul/li[3]/a/div").Click
Else
Obj.FindElementByXPath("//*[#id='downloadReport']/div").Click
End If
In short, there is no merit in locating an element with style attribute set as display: none;. Even through you locate the element, you won't be able to click on it. So the If-Then-Else logic won't work as intended.
Instead find the desired element which is visible and enabled element, so you can invoke click on it.
I'm trying to enter text into a field (the subject field in the image) in a section using Selenium .
I've tried locating by Xpath , ID and a few others but it looks like maybe I need to switch context to the section. I've tried the following, errors are in comments after lines.
from selenium.webdriver import Firefox
from selenium.webdriver.firefox.options import Options
opts = Options()
browser = Firefox(options=opts)
browser.get('https://www.linkedin.com/feed/')
sign_in = '/html/body/div[1]/main/p/a'
browser.find_element_by_xpath(sign_in).click()
email = '//*[#id="username"]'
browser.find_element_by_xpath(email).send_keys(my_email)
pword = '//*[#id="password"]'
browser.find_element_by_xpath(pword).send_keys(my_pword)
signin = '/html/body/div/main/div[2]/div[1]/form/div[3]/button'
browser.find_element_by_xpath(signin).click()
search = '/html/body/div[8]/header/div[2]/div/div/div[1]/div[2]/input'
name = 'John McCain'
browser.find_element_by_xpath(search).send_keys(name+"\n")#click()
#click on first result
first_result = '/html/body/div[8]/div[3]/div/div[1]/div/div[1]/main/div/div/div[1]/div/div/div/div[2]/div[1]/div[1]/span/div/span[1]/span/a/span/span[1]'
browser.find_element_by_xpath(first_result).click()
#hit message button
msg_btn = '/html/body/div[8]/div[3]/div/div/div/div/div[2]/div/div/main/div/div[1]/section/div[2]/div[1]/div[2]/div/div/div[2]/a'
browser.find_element_by_xpath(msg_btn).click()
sleep(10)
## find subject box in section
section_class = '/html/body/div[3]/section'
browser.find_element_by_xpath(section_class) # no such element
browser.switch_to().frame('/html/body/div[3]/section') # no such frame
subject = '//*[#id="compose-form-subject-ember156"]'
browser.find_element_by_xpath(subject).click() # no such element
compose_class = 'compose-form__subject-field'
browser.find_element_by_class_name(compose_class) # no such class
id = 'compose-form-subject-ember156'
browser.find_element_by_id(id) # no such element
css_selector= 'compose-form-subject-ember156'
browser.find_element_by_css_selector(css_selector) # no such element
wind = '//*[#id="artdeco-hoverable-outlet__message-overlay"]
browser.find_element_by_xpath(wind) #no such element
A figure showing the developer info for the text box in question is attached.
How do I locate the text box and send keys to it? I'm new to selenium but have gotten thru login and basic navigation to this point.
I've put the page source (as seen by the Selenium browser object at this point) here.
The page source (as seen when I click in the browser window and hit 'copy page source') is here .
Despite the window in focus being the one I wanted it seems like the browser object saw things differently . Using
window_after = browser.window_handles[1]
browser.switch_to_window(window_after)
allowed me to find the element using an Xpath.
I need some help in selecting a div form many div's that for every session number of div's are changing.
For example:
one time a have the div in that position(absolute path): /html/body/div[97]
other time in that position (absolute path):/html/body/div[160]
and so on...
At a moment only one div is active and the other div's are hidden.
I attached a picture to show the code.
I try the xpath below but doesn't work,I get the error "no such element: Unable to locate element ...
driver.findElement(By.xpath(".//*[#class=\'ui-selectmenu-menu ui-selectmenu-open\']/ul/li[1]")).click();
Picture with html code is here:
XPath is great if you expect the target element to be in the exact location every time the page is displayed. However, CSS is the way to go if the content is constantly changing.
In the example below I have found the DIV and the Element that you highlighted in your screen capture.
WebElement targetElementDiv = null;
WebElement targetElement = null;
targetElementDiv = driver.findElement(By.cssSelector("[class='ui-selectmenu-menu ui-selectmenu-open']"));
if (targetElementDiv != null) {
targetElement = targetElementDiv.findElement(By.cssSelector("[class='ui-menu-item ui-state-focus'])");
}
I am using the method FindElementsByCssSelector like
var iframes = RemoteWebDriver.FindElementsByCssSelector("iframe");
and that gives me two iframes.
If I write
$('iframe');
in the browser development console, it displays only one.
In the DOM, one iframe is inside another like:
iframe
html
body
form
div
table
tbody
tr
td
iframe
Locate the firs frame and use it to locate the other one
IWebElement firstIframe = RemoteWebDriver.FindElementByCssSelector("iframe");
RemoteWebDriver.SwitchTo().Frame(firstIframe);
IWebElement secondIframe = iframe.FindElementByCssSelector("iframe");
RemoteWebDriver.SwitchTo().Frame(secondIframe);
I'm currently working on a program that will average the prices from a searched item on Amazon.
I have button on the program that when pressed, prints out the HTML source code into a richtextbox and then finds the specific div within the source code.
My only problem right now is having it print out the money amount after each div.
Is there any way to do this?
You can use HTMLAgilityPack, one of the best HTML parsing libraries to ease this task.
An example.
Examples:
Assuming div's id is divPrice
HtmlDocument doc = new HtmlDocument();
doc.LoadHtml(yourHTML);
HtmlNode priceNode = doc.GetElementbyId("divPrice");
string price = priceNode.InnerText;
Let's say div has no id but a css class cssPrice, then you can query it with XPath:
HtmlNode priceNode = doc.DocumentNode.SelectSingleNode("//div[#class='cssPrice']");
OR
HtmlNodeCollection nodes = doc.DocumentNode.SelectNodes("//div[#class='cssPrice']");
foreach (HtmlNode node in nodes) {
string nodeText = node.InnerText;
}