So, I'm trying to creade a code to automatic fill some research queries at my work, to save time and to exercise Python (I've only been trying python for a week). While using Selenium, I can't seen to get the Xpath to work properly, especially on the last Three Items. I don't know where I got the /table or the /div/div on Xpath, But as I was editing the code it seems that only the one with /table works. I don't know why I can't get it on the page anymore or why it was even there, since i'm doing the same method.
I'm actually having trouble on the last three buttons, I can't seem to know where and why. Those are the ones that I was having trouble, the first one works but the last one doesnt, when I try to extract the full xpath I get the div/div but when i use the /table/tbody it works for the first of these buttons but not for the latter ones. Can someone shed a light on what i'm missing.
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[4]/table/tbody/tr[1]/td[1]/div/input').click()
time.sleep(1)
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[4]/div/div[2]/div[2]/input').click()
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[4]/table/tbody/tr[2]/td[2]/div/input').click()
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/div[2]/button').click()
THE FULL CODE:
from selenium import webdriver
navegador = webdriver.Chrome("chromedriver.exe")
navegador.get("https://pesquisasatisfacao.saude.rj.gov.br/Unidade/criar-emergencia/5955637")
time.sleep(1)
# Caracteristicas paciente.
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[1]/div/div[1]/div[1]/input').click()
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[1]/div/div[2]/div/div[3]/input').click()
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[1]/div/div[3]/div/div[1]/input').click()
import time
time.sleep(1)
navegador.execute_script("window.scrollTo(0, 1000);")
time.sleep(1)
#Atenção e respeito dos profissionais
#/html/body/section/section[2]/div[1]/form/fieldset[2]/table/tbody/tr[1]/td[1]/div/input
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[2]/table/tbody/tr[1]/td[1]/div/input').click()
#Tempo de espera
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[2]/table/tbody/tr[2]/td[1]/div/input').click()
#Limpeza da unidade
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[2]/table/tbody/tr[3]/td[1]/div/input').click()
#Avaliação das instalaçoes
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[2]/table/tbody/tr[4]/td[1]/div/input').click()
#atendimento da recepção
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[2]/table/tbody/tr[5]/td[1]/div/input').click()
#Classificação de Risco
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[2]/table/tbody/tr[6]/td[1]/div/input').click()
#Atendimento da Enfermagem
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[2]/table/tbody/tr[7]/td[1]/div/input').click()
#Atendimento Médico
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[2]/table/tbody/tr[8]/td[1]/div/input').click()
#Exames em geral
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[2]/table/tbody/tr[9]/td[1]/div/input').click()
navegador.execute_script("window.scrollTo(0, 1200);")
time.sleep(3)
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[4]/div/div[1]/div[1]/input').click()
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[4]/table/tbody/tr[1]/td[1]/div/input').click()
time.sleep(1)
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[4]/div/div[2]/div[2]/input').click()
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/fieldset[4]/table/tbody/tr[2]/td[2]/div/input').click()
navegador.find_element_by_xpath('/html/body/section/section[2]/div[1]/form/div[2]/button').click()
#/html/body/section/section[2]/div[1]/form/fieldset[3]/div[1]/div[1]/input
#/html/body/section/section[2]/div[1]/form/fieldset[3]/div[1]/div[1]/input
#/html/body/section/section[2]/div[1]/form/fieldset[3]/div[1]/div[1]/label```
Related
I try to get the xpath for the following button on instagram making an automate unfollowing soft. enter image description here
I found it just like this:
driver.find_element_by_xpath('//div[#class="qF0y9 Igw0E rBNOH YBx95 ybXk5 _4EzTm soMvl "]').click()
But i want to itterate over all ,,Following" Buttons , but like this is stuck at the first one!
This is my Code:
fBody = driver.find_element_by_xpath("//div[#class='isgrP']")
for i in range(1, 1500):
driver.find_element_by_xpath('//div[#class=" qF0y9 Igw0E rBNOH YBx95 ybXk5 _4EzTm soMvl "]').click()
#driver.find_element_by_xpath("//*[text()='Following']").click()
print("Am apasat follow")
sleep(5)
driver.find_element_by_xpath('//button[#class="aOOlW -Cab_ "]').click()
sleep(5)
driver.execute_script('arguments[0].scrollTop = arguments[0].scrollTop + arguments[0].offsetHeight;', fBody)
print("Ma bag la somn 1 min")
sleep(2)
print("salut")
Selenium does Not "like" empty or white spaces in the attributes.
I suggest using a CSS selector and using *= in order to find text contains:
driver.find_element_by_CSS('//div[class*="qF0y9"][class*="Igw0E"]').click();
Avoid using white or empty spaces and, underscores (_) and hyphens (-) for the element's attributes.
I think the classes on the elements change as yours do not match with mine. Here is a more generic XPath that matches the "following" button.
//div//button[div[text()='Following']
When using this in a test I found it instantly failing unless I surrounded it with an explicit wait condition.
wait.until(ExpectedConditions.elementToBeClickable(By.xpath("//div//button[div[text()='Following']"))).click();
Ill post my example when Instagram stops giving me connectivity issues.
I installed Chrome addon "Selectors Hub".
I opened site: spicejet.com
I choose some random city of "from" dropdown.
With help of "Selectors Hub" chrome addon, I grabbed the Xpath code of
that city:
//div[#class='css-1dbjc4n r-14lw9ot r-z2wwpe r-vgw6uq r-156q2ks r-urutk0 r-8uuktl r-136ojw6']//div[11]
While validating this Xpath code in console, it shows 0 matches.
This website is built on ReactJs as front-end, if I am not wrong, and finding the elements of a ReactJs website is a bit challenging; adding to that, if you rely on some locator finding tool, it's gets more difficult. It's always better you build your own locator strategy than rely on tools, especially for websites built with React, Vue, etc.
Having said that, the strategy here is to find the relatively narrowed down relative locator, and then since you are looking for a random selection of city, collect all the cities first, then apply random to it. Here is what it figured:
I collected cities, but along with it came some unwanted items (courtesy my relative locator), and I check them and if they are picked up, I pass them out, and only when a city is selected by random, I click on it. Check this code:
import random
driver.get("https://www.spicejet.com/")
time.sleep(10)
driver.find_element(By.XPATH, "//div[#data-testid='to-testID-destination']").click()
time.sleep(2)
cities = driver.find_elements(By.XPATH, "//div[#data-testid='to-testID-destination']//div[#data-focusable='true']")
print(len(cities))
x = random.choice(cities)
if x.text in ['To', 'India', 'International']:
pass
else:
print(x.text)
x.click()
time.sleep(5)
driver.quit()
Output:
Pakyong
Pakyong Airport
PYG
Process finished with exit code 0
Note : I'm experienced in python however just starting out in selenium and webscraping. Please excuse if this is a bad question or if my fundamentals in selenium seem amiss. I could not find an answer in hours of searching hence i ask here
Goal: To extract the "About the Business" information found in Yelp pages of businesses
Some pages have their about the business information within a Read More button based popup (eg : https://www.yelp.com/biz/and-pizza-bethesda-bethesda)
Some pages do not have their business information in a Read More button based popup (eg : https://www.yelp.com/biz/pneuma-fashions-upper-marlboro-3 )
Problem: Unable to navigate to the About the Business popup that appears after clicking the Read More button and extract the text present in it.
Attempts as of now: From googling I had found explanations on how to handle alert popups or window popups. However the code doesnt work. The popup that emerges when clicking Read More button does not cause change in window_handles
import re
# getting all sections of the page
result=driver.find_elements_by_tag_name("section")
About = None
for sec in result:
if sec.text.startswith("About the Business"):
# this pertains only to the About the business section
main_page=driver.current_window_handle
print(main_page) # Returns the current handle
sec.find_element_by_tag_name("button").click()
popup=None
for handle in driver.window_handles: # is an iterable with only one handle
# The only handle present is the main_page handle
print(handle)
if handle!=main_page:
popup = handle
print(popup) # returns None
driver.switch_to.window(popup) # Throws error because popup=None
# THE FOLLOWING SECTION IS NOT EXECUTED BECAUSE OF THE ERROR ABOVE
#////////////////////////////////////////////////////
button_contents=driver.find_elements_by_tag_name("p")
for b in button_contents:
print(b.text) # intended to print text contents
close=driver.find_element_by_tag_name("button")
close.click()
driver.switch_to.window(main_page)
Please help
Thank you to everyone who reads this question and provides advice and answers
That is a custom pop-up so you won't need to switch to it. I suggest to study about getting relative xpath . Use loop to navigate to your urls and include below code
driver.get(your_URL)
readMoreBtnXpath= "//h4[text()='About the Business']/ancestor::section//button"
aboutTheBusinessSec = "//h4[text()='About the Business']/ancestor::section"
fromTheBusinessSec = "((//h2[text()='From the business']/parent::div/following-sibling::div//div)[5]/div)[last()]/preceding-sibling::div"
try:
driver.find_element(By.XPATH, readMoreBtnXpath).click()
button_contents = driver.find_elements(By.XPATH, fromTheBusinessSec)
for b in button_contents:
print(b.text)
except:
print(driver.find_element(By.XPATH, aboutTheBusinessSec).text)
One thing that u should know is that the pop-up is not displayed in a new window. It is instead displayed in the same page itself. Here is the complete code to extract the text from the pop-up:
from selenium import webdriver
driver = webdriver.Chrome()
driver.get('https://www.yelp.com/biz/and-pizza-bethesda-bethesda')
try:
driver.find_element_by_xpath('//*[#id="wrap"]/div[3]/div/div[4]/div/div/div[2]/div/div/div[1]/div/div[1]/section[5]/div[2]/button').click()
p1 = driver.find_element_by_xpath('//*[#id="modal-portal-container"]/div[2]/div/div/div/div[2]/div/div[2]/div/div[2]/div/div/div[1]/p').text
p2 = driver.find_element_by_xpath('//*[#id="modal-portal-container"]/div[2]/div/div/div/div[2]/div/div[2]/div/div[2]/div/div/div[2]/p[2]').text
print("Specialties --",p1)
print("History --",p2)
except:
print('Read more button not found')
Output:
Specialties -- Award-winning pizza: Named one of Fast Company's "World's Most Innovative Companies" in 2018, third-place in the Washington Post Express's of "Best Fast Casual" in 2018, third place in the Washington City Paper's "Best Gluten-Free Menu" in 2018 and won its "Best Pizza in D.C." in 2017, 11th on TripAdvisor's "Best Fast Casual Restaurants -- United States" in 2018.
History -- Since 2012, we've built pizza shops with an edge to their craft pies, beverages and shop design, created an environment where ALL of our Tribe can thrive, supported our local communities and now we'll text you back, if you want. Started with a pizza shop. Became a culture. That's &pizza.
Edit:
Since this doesn't work with this website, replace the first find_element_by_xpath with:
driver.find_element_by_xpath("//div[#class='lemon--div__373c0__1mboc border-color--default__373c0__3-ifU']/button[.='Read more']").click()
This works for both the websites.
I am having issues with selenium using the next xpath "./preceding-sibling::text()"
I don't understand why, my first thought was that IE wasn't supporting that xpath statement but it didnt work on chrome neither.
What I am trying to do is to verify if a radio button have a certain text "near" it. For example if a radio button is like this
<div>
<label> Yes <input name='radioBtn'></input></label>
<label> No <input name='radioBtn'></input></label>
</div>
This is a simplified scenario where I need to check the "yes" radio button, so what I am doing is to search for the radiobutton and check if it preceding-sibling::text(), but selenium is cant find any element. I know the Xpath works because I test it on chrome developer tools and it returns the text "Yes".
I can't do something like 'input[1]' because I can't be sure that the first one will be always that have the Yes text.
Any idea why isn't this working on selenium? and if there is any work around?
I got to a work around but is kind of specific to the problem. But let's answer the questions 1 at the time.
Any idea why isn't this working on selenium?
It's not working because selenium don't support text elements, so even when selenium find the element it cant map it to a selenium object, i didn't see it because my code hided the selenium exception. The Exception throw is the next one:
An unhandled exception of type
'OpenQA.Selenium.InvalidSelectorException' occurred in WebDriver.dll
Additional information: invalid selector: The result of the xpath
expression "./preceding-sibling::text()" is: [object Text]. It should
be an element
Is there any work around?
Yes it is. What I did was to run a java script using the selenium IJavaScriptExecutor. With the script I revised the preceding sibling text and return it as a string so if the text was equal to the thing I was looking for (i.e Yes) trhat means that is the radio button I was looking for.
The code looks is similar to this (it can have some sintax errors since I didn't copied from the source):
string script = "function GetPrecedingText(node){"+
"var result = document.evaluate('./preceding-sibling::text()[1]', node, null, XPathResult.STRING_TYPE, null);"+
"return (result==null) ? '' : result.stringValue.trim();}"+
"return GetPrecedingText(arguments[0])";
string result = ((driver as IJavaScriptExecutor).ExecuteScript(script, SeleniumElement)).toString();
Hope someone can find this useful :) and thanks for all that tried to help me :)
I am testing a scenario for the site "https://www.freecrm.com/index.html"
login credentials [ john2013 / john2013 ]
Scenario :
1 open the site https://www.freecrm.com/index.html
2 login with valid credentials
3 click on the "New Contacts" link
4 Add new contacts
using Selenium ide i am able to login and click on the "New Contact" link , but when i am trying to do the same thing using Webdriver [ java] i am not able to click the "New Contact" link
the code i have written is given below
driver.findElement(By.name("username")).clear()
driver.findElement(By.name("username")).sendKeys("john2013");
driver.findElement(By.name("password")).clear();
driver.findElement(By.name("password")).sendKeys("john2013");
driver.findElement(By.cssSelector("input[type=\"image\"]")).click();
The code up to the above is working fine but clicking the "New Contact" link
driver.findElement(By.xpath("//div[#class='noprint']/span[#class='headertext']/a[3]/")).click();
is not working though the same xpath is working in IDE.
i have tried with expected condition option , sleep but nothing is working.
can any one help me in this regard.
The problem is that your page uses a frameset. You should tell your webdriver which frame to use when it will search for your element. So try this:
driver.findElement(By.name("username")).clear()
driver.findElement(By.name("username")).sendKeys("john2013");
driver.findElement(By.name("password")).clear();
driver.findElement(By.name("password")).sendKeys("john2013");
driver.findElement(By.cssSelector("input[type=\"image\"]")).click();
//switch the driver to use one of the frames on your page. Potentially wait for a bit till the page is loaded
driver.switchTo().frame("mainpanel");
driver.findElement(By.xpath("//*[text()='New Contact']")).click();
Please change the xpath of link to
xpath=//a[contains(text(),'<whatevertext>')]
Incase if the same text link is already present in some part o the page we can proceed with occurences. Example if it is repeating second time:
xpath=(//a[contains(text(),'<whatevertext>')])[2]
If Nothing is working the best way is through java script executor.
First get the webelement of the link.
Then use the following code"
webelement=driver.findElement(by.xpath("=//a[contains(text(),'')]
"))
(JavaScriptExecutor)driver.executescript("argument[0].click();",webelement)
Actually Speaking //div[#class='noprint']/span[#class='headertext']/a[3]/ is absolute XPath.
as it ends with a[3] the position will vary always so it is advisable to use relative XPath/CSS
Suggested CSS:css=.noprint > .headertext > a.classname
Another Suggested CSS: css=.noprint > .headertext > a[attribute='value']
So we can use either of the above format.