I installed Chrome addon "Selectors Hub".
I opened site: spicejet.com
I choose some random city of "from" dropdown.
With help of "Selectors Hub" chrome addon, I grabbed the Xpath code of
that city:
//div[#class='css-1dbjc4n r-14lw9ot r-z2wwpe r-vgw6uq r-156q2ks r-urutk0 r-8uuktl r-136ojw6']//div[11]
While validating this Xpath code in console, it shows 0 matches.
This website is built on ReactJs as front-end, if I am not wrong, and finding the elements of a ReactJs website is a bit challenging; adding to that, if you rely on some locator finding tool, it's gets more difficult. It's always better you build your own locator strategy than rely on tools, especially for websites built with React, Vue, etc.
Having said that, the strategy here is to find the relatively narrowed down relative locator, and then since you are looking for a random selection of city, collect all the cities first, then apply random to it. Here is what it figured:
I collected cities, but along with it came some unwanted items (courtesy my relative locator), and I check them and if they are picked up, I pass them out, and only when a city is selected by random, I click on it. Check this code:
import random
driver.get("https://www.spicejet.com/")
time.sleep(10)
driver.find_element(By.XPATH, "//div[#data-testid='to-testID-destination']").click()
time.sleep(2)
cities = driver.find_elements(By.XPATH, "//div[#data-testid='to-testID-destination']//div[#data-focusable='true']")
print(len(cities))
x = random.choice(cities)
if x.text in ['To', 'India', 'International']:
pass
else:
print(x.text)
x.click()
time.sleep(5)
driver.quit()
Output:
Pakyong
Pakyong Airport
PYG
Process finished with exit code 0
Related
Based on the Selenium documentation the find element by css selector syntax is element = driver.find_element_by_css_selector('#foo') but the example shows there is a.nav before the # sign ‘(a.nav#home)’ which based on this website is HTML tag.
In another part of the Selenium documentation the css_selector even doesn't have the # sign: ele = driver.find_element(By.CSS_SELECTOR, 'h1')
Questions:
Which syntax is correct? with or without the HTML tag? with or without the # sign?
In Visual Studio Code I used these syntaxes to find the search boxes or sign-in boxes. It worked in this website but didn't work in this website. Could you help me find the search box using css_selector in this website?
Here is an example of my scripts:
from selenium import webdriver
from selenium.webdriver.common.by import By
try:
driver = webdriver.Chrome()
driver.get("https://www.arizonarealestate.com")
searchBox = driver.find_element(By.CSS_SELECTOR, "#input[placeholder='Enter city, address, neighborhood, zip, or MLS #']")
searchBox = driver.find_element(By.CSS_SELECTOR, "input#input[placeholder='Enter city, address, neighborhood, zip, or MLS #']")
searchBox.send_keys("Some text")
searchBtn = driver.find_element(By.CSS_SELECTOR, "button#.btn.btn-primary.btn-lg.btn-block.js-qs-btn").click()
finally:
#print("============ Done!")
driver.quit()
Generally speaking a css selector is just a string with some specific syntax and is not really defined by the selenium WebDriver itself.
You should have a look at the MDN description of css selectors.
In your question you specifically seem to have question on where to specify the id selector specified with the # character. This selector should actually only be used just by itself as all id's in a page should be unique and therefore no other information is needed.
In your example, the css selector #input[placeholder=...'] selector would select an element with an id equal to input.
If you intended selecting an input tag with a specific placeholder you should omit the #.
im trying to find an element with dinamic values , for example <span class="ms-Button-label label-175" id="id__177">Save</span> in inspect element, the id and class values tend to change for every refresh, how can i in this case find the element in selenium? i tried troguht xpath but seems doesnt work because can not find the path, i was thinking to find "Save" world torught always find by xpath but actually i dont know if im doing well : driver.find_element_by_xpath(//span(#.... but then? how can insert element if it changes everytime? thanks!
Something like this may work:
driver.find_element_by_xpath('//span[text()="Save"]')
But this will fail, if there is more than one button with text "Save" on the page.
In that case you may try to find some specific outer element (div, form, etc.) which does not change and contains the button. Then find the button inside of it.
With few requests with driver:
specific_div = driver.find_element_by_id("my_specific_div")
button = specific_div.find_element_by_tag_name("span") # e.g. there is only one span in that div
Or with more specific xpath:
button = driver.find_element_by_xpath('//div[#class="some-specific-class"]/span[text()="Save"]')
If needed, search for more nested elements before the button, so you can get more narrow search field.
More examples in the docs.
Note : I'm experienced in python however just starting out in selenium and webscraping. Please excuse if this is a bad question or if my fundamentals in selenium seem amiss. I could not find an answer in hours of searching hence i ask here
Goal: To extract the "About the Business" information found in Yelp pages of businesses
Some pages have their about the business information within a Read More button based popup (eg : https://www.yelp.com/biz/and-pizza-bethesda-bethesda)
Some pages do not have their business information in a Read More button based popup (eg : https://www.yelp.com/biz/pneuma-fashions-upper-marlboro-3 )
Problem: Unable to navigate to the About the Business popup that appears after clicking the Read More button and extract the text present in it.
Attempts as of now: From googling I had found explanations on how to handle alert popups or window popups. However the code doesnt work. The popup that emerges when clicking Read More button does not cause change in window_handles
import re
# getting all sections of the page
result=driver.find_elements_by_tag_name("section")
About = None
for sec in result:
if sec.text.startswith("About the Business"):
# this pertains only to the About the business section
main_page=driver.current_window_handle
print(main_page) # Returns the current handle
sec.find_element_by_tag_name("button").click()
popup=None
for handle in driver.window_handles: # is an iterable with only one handle
# The only handle present is the main_page handle
print(handle)
if handle!=main_page:
popup = handle
print(popup) # returns None
driver.switch_to.window(popup) # Throws error because popup=None
# THE FOLLOWING SECTION IS NOT EXECUTED BECAUSE OF THE ERROR ABOVE
#////////////////////////////////////////////////////
button_contents=driver.find_elements_by_tag_name("p")
for b in button_contents:
print(b.text) # intended to print text contents
close=driver.find_element_by_tag_name("button")
close.click()
driver.switch_to.window(main_page)
Please help
Thank you to everyone who reads this question and provides advice and answers
That is a custom pop-up so you won't need to switch to it. I suggest to study about getting relative xpath . Use loop to navigate to your urls and include below code
driver.get(your_URL)
readMoreBtnXpath= "//h4[text()='About the Business']/ancestor::section//button"
aboutTheBusinessSec = "//h4[text()='About the Business']/ancestor::section"
fromTheBusinessSec = "((//h2[text()='From the business']/parent::div/following-sibling::div//div)[5]/div)[last()]/preceding-sibling::div"
try:
driver.find_element(By.XPATH, readMoreBtnXpath).click()
button_contents = driver.find_elements(By.XPATH, fromTheBusinessSec)
for b in button_contents:
print(b.text)
except:
print(driver.find_element(By.XPATH, aboutTheBusinessSec).text)
One thing that u should know is that the pop-up is not displayed in a new window. It is instead displayed in the same page itself. Here is the complete code to extract the text from the pop-up:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://www.yelp.com/biz/and-pizza-bethesda-bethesda')
try:
driver.find_element_by_xpath('//*[#id="wrap"]/div[3]/div/div[4]/div/div/div[2]/div/div/div[1]/div/div[1]/section[5]/div[2]/button').click()
p1 = driver.find_element_by_xpath('//*[#id="modal-portal-container"]/div[2]/div/div/div/div[2]/div/div[2]/div/div[2]/div/div/div[1]/p').text
p2 = driver.find_element_by_xpath('//*[#id="modal-portal-container"]/div[2]/div/div/div/div[2]/div/div[2]/div/div[2]/div/div/div[2]/p[2]').text
print("Specialties --",p1)
print("History --",p2)
except:
print('Read more button not found')
Output:
Specialties -- Award-winning pizza: Named one of Fast Company's "World's Most Innovative Companies" in 2018, third-place in the Washington Post Express's of "Best Fast Casual" in 2018, third place in the Washington City Paper's "Best Gluten-Free Menu" in 2018 and won its "Best Pizza in D.C." in 2017, 11th on TripAdvisor's "Best Fast Casual Restaurants -- United States" in 2018.
History -- Since 2012, we've built pizza shops with an edge to their craft pies, beverages and shop design, created an environment where ALL of our Tribe can thrive, supported our local communities and now we'll text you back, if you want. Started with a pizza shop. Became a culture. That's &pizza.
Edit:
Since this doesn't work with this website, replace the first find_element_by_xpath with:
driver.find_element_by_xpath("//div[#class='lemon--div__373c0__1mboc border-color--default__373c0__3-ifU']/button[.='Read more']").click()
This works for both the websites.
I am trying to get the job description for job search page indeed.com This is how it looks like
Provide technical leadership around
QA
automation to IT teams. Work with various team to promote
QA
processes, practices and standardization....
Any idea how can I get that description? I tried the following:
//span[contains(#class,'summary')]
That does not give me the text description. Should I xpath or is there any other solution? Thanks in advance for your time.
This XPath are correct.
//span[contains(#class,'summary')]
//span[#class='summary']
I'm a Python guy, But I translated it to Java. You can do:
element = driver.findElement(By.name("summary"));
element = driver.findElement(By.className("summary"));
element = driver.findElement(By.cssSelector('span[class="summary"]');
And remember that If you want the element text, every element has the method .getText(), the find* functions only retrieve the element/s.
Double check you were not using driver.findElements(By.xpath()) in plural. In that case you should first retrieve the individual elements. Then access to the .getText() method.
description = driver.findElement(By.className("summary")).getText();
System.out.print(description);
Alternatively you could do:
description = driver.findElement(By.className("summary"));
description_text = description.getAttribute("innerHTML");
System.out.print(description_text);
If your problem is that your element is not visible or reachable (stale). Then you can use javascript.
element = driver.executeScript("return document.querySelector('span[class=\"summary\"]');");
For more reference:
https://seleniumhq.github.io/selenium/docs/api/java/org/openqa/selenium/WebElement.html
I have tried a lot in finding out how measure the coordinate of an element in a web page in different browser.But I could not find any solution.
Is there any other tool that can measure the position of an element in various browsers???
In AutoIt you can use the following code to get screen coordinates (in my example for displaying a tool tip as an overlay on an Internet Explorer):
$oIE = _IECreate("http://...URL...")
$username = _IEFormElementGetObjByName(_IEFormGetObjByName($oIE, "loginform"), "username")
ToolTip("Login", _IEPropertyGet($username, "screenx"), _IEPropertyGet($username, "screeny"))
_IEAction($username, "focus")
Alternatively you can use _IEGetObjById($oIE, "mx77") to get an object reference. Or run through all all elemnts by tag name as shown here.
Instead of getting the absolute screen position, you can get the In-Browser-Position, using
browserx and browsery.