I am trying to click an element for one website.
The source is:
<dd data-value="33" data-text="" data-target="brand">奥迪</dd>
It is supposed to be a dropdown button that clickable, but I've tried different methods:
find_elements_by_css_selector
find_elements_by_xpath that I just copied from google
But it turns out that the element the selenium found is not clickable. I am curious I can still click it.
The website is https://www.autohome.com.cn/beijing/ (you probably need to translate it into English).
.
I would really appreciate any help on this.
Furtherly, as I am going to click all car brands here for scraping, I hope the methods can be generalized to other cars such as Alfa Romeo listed in the figure.
Thank you so much!
Wrong element to open it up.
<div class="option option-brand">
Just showed an example in Python.
wait = WebDriverWait(driver, 10)
driver.get("https://www.autohome.com.cn/beijing/")
wait.until(EC.element_to_be_clickable((By.CSS_SELECTOR,"div.option.option-brand"))).click()
name='奥迪'
wait.until(EC.element_to_be_clickable((By.XPATH,"//dd[.='{}']".format(name)))).click()
Import
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Related
Before this, I have automated a login into the website howe page. I was stuck trying to make it so that it clicks on the image to navigate to another webpage.
shop_button = driver.find_element(By.XPATH, "/html/body/footer/div/div[2]/a/img")
shop_button.click()
This is the code for the image hyperlink
You can use the below xpath
//p[text()='Shop']//preceding-sibling::img
this should locate the img node, however p tag which has Shop as a text has to be unique in nature.
If it's unique, you can click on it like:
Code trial 1:
time.sleep(5)
driver.find_element(By.XPATH, "//p[text()='Shop']//preceding-sibling::img").click()
Code trial 2:
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//p[text()='Shop']//preceding-sibling::img"))).click()
Imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
PS: time.sleep(5) is just for visualization purpose, you should not use it ideally if code 2 works fine.
You want to redirect to another webpage by clicking on a image. According to the code snippet, I can see that Image tag is inside the tag, you can click on that.
Can you try this ?
shop_button = driver.find_element(By.XPATH,"/html/body/footer/div/div[2]/a")
shop_button.click()
Do let me know if this works.
My first question was related to this. In the meantime I have found a less efficient workaround. I'd still like automatically download these files.
I would like my script to click on the download icon, click on the Download XLS icon that follows of the page below. The green arrow is where I'd like to click.
The id of the highchart changes every time the page loads. I've tried to fix it it with code sample:
def unityengine_data():
driver.get("https://steamdb.info/tech/Engine/Unity/")
wait
WebDriverWait(driver,30).until(EC.visibility_of_element_located((By.XPATH, "div[contains(#id,'highcharts-')]//*[local-name() = 'svg']/*[name()='g'][6]/*[name()='g']/*[name()='image']"))).click
WebDriverWait(driver, 30).until(EC.visibility_of_element_located((By.XPATH, "div[contains(#id,'highcharts-')]//**[local-name() = 'div']/*[name()='ul'][6]/*[name()='li'][2]"))).click
os.rename("downloaded file path", "new file path")
return "Unity Engine apps data successfully downloaded"
Website image
Website after first click
I hope you can help.
You are simply missing // here at this line
WebDriverWait(driver,30).until(EC.visibility_of_element_located((By.XPATH, "div[contains(#id,'highcharts-')]//*[local-name() = 'svg']/*[name()='g'][6]/*[name()='g']/*[name()='image']"))).click
also, you should ideally create object of WebDriverWait once and reuse it whenever you want.
See the below code, I have improved one locator as well.
Code:
driver.maximize_window()
wait = WebDriverWait(driver, 30)
driver.get("https://steamdb.info/tech/Engine/Unity/")
wait.until(EC.visibility_of_element_located((By.XPATH, "//div[contains(#id,'highcharts-')]//*[local-name() = 'svg']/*[name()='g'][6]/*[name()='g']/*[name()='image']"))).click()
wait.until(EC.visibility_of_element_located((By.XPATH, "//li[text()='Download XLS']"))).click()
Imports:
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
You were almost there. You just need three minor adjustments and one optimization as follows:
You need to add the leading // for an effective xpath.
click is a method so you need to click()
The xpath for the highchart image can be optimized a bit.
The element with text Download XLS is within local-name space, hence you don't need [local-name() = 'div']
Your effective code block will be:
driver.get("https://steamdb.info/tech/Engine/Unity/")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//div[contains(#id,'highcharts-')]//*[local-name() = 'svg']//*[name()='g']//*[name()='image']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//ul[#class='highcharts-menu']//li[text()='Download XLS']"))).click()
Browser Snapshot;
References
You can find a couple of relevant detailed discussions in:
How to click on SVG elements using XPath and Selenium WebDriver through Java
I am new to using the selenium module. I have started with some simple tutorials which go ok until I get a popup.
Because the popup does not have an x, I am not able to apply other advice I have found online.
How to close pop up window in Selenium
However I have tried to inspect the code of the popup window and I can see that maybe I have a couple of options, close by referencing the link text 'Accept Cookies', or close by the button id which is "onetrust-accept-btn-handler"
This is the code I have so far.
import selenium
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
import time
PATH = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)
driver.get("https://www.reed.co.uk/jobs/senior-insight-analyst/42347955")
driver.implicitly_wait(10)
link = driver.find_element_by_link_text("onetrust-accept-btn-handler")
link.click()
Trying
link = driver.find_element_by_link_text("onetrust-accept-btn-handler")
and
link = driver.find_element_by_link_text("Accept cookies")
Result in errors
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"link text","selector":"onetrust-accept-btn-handler"}
or
NoSuchElementException: Message: no such element: Unable to locate element: {"method":"link text","selector":"Accept cookies"}
I am wondering if there is any better command than link text if the button id is known, or am I missing another step in the process because it is a pop up? Any help appreciated. Thank you.
Please use the explicit wait so that your popup window can come up and your selenium script can detect the element and click on it.
Use the below code -
WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.ID, 'onetrust-accept-btn-handler'))).click()
Mark it as answer if it resolves your problem.
Thank you Swaroop Humane and Dev for your answers. The answer above works, I also needed to add in three more lines of import code before the solution managed to click the Accept cookies button in the pop up. https://selenium-python.readthedocs.io/waits.html
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
Final code below.
import selenium
from selenium.webdriver.common.by import By
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
import time
PATH = "C:\Program Files (x86)\chromedriver.exe"
driver = webdriver.Chrome(PATH)
driver.get("https://www.reed.co.uk/jobs/senior-insight-analyst/42347955")
#driver.implicitly_wait(10)
#link = driver.find_element_by_link_text("onetrust-accept-btn-handler")
#link.click()
WebDriverWait(driver, 15).until(EC.element_to_be_clickable((By.ID, 'onetrust-accept-btn-handler'))).click()
Then this happened.
I probably need to figure out a way to run selenium while I am signed into this webservice, rather than let it run a new browser every time. Any ideas?
I can't find the correct xpath to use for retweet and like on this page: https://twitter.com/snowfulls/status/1198269659465818115
Also, I need help finding the xpath for the second retweet button that comes up to confirm the retweet.
Is there a way to find xpaths automatically?
To answer your first question -- no, there is not a way to find XPaths automatically unless you use some kind of scanner tool. These XPaths are not always accurate though. The best approach is to use an XPath browser extension helper that will allow you to test out XPath expressions on a page in real-time. That is what I have used to help develop my solution.
To click the "Like" button on a tweet, you can use the following code:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# ensure the above references are added to use WebDriverWait correctly
# wait for the element to exist
like_button = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//div[#aria-label='Like']")))
# click the like button
like_button.click()
To click the retweet button, similarly:
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
# ensure the above references are added to use WebDriverWait correctly
# wait for the element to exist
retweet_button = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//div[#aria-label='Retweet']")))
# click the retweet button
retweet_button.click()
# now, confirm the retweet:
retweet_confirm = WebDriverWait(driver, 10).until(EC.presence_of_element_located((By.XPATH, "//div[#data-testid='retweetConfirm']")))
# click the retweet confirm button
retweet_confirm.click()
The above solution uses a few simple extensions of Selenium library -- mainly, WebDriverWait and ExpectedConditions class. WebDriverWait allows us to wait up to a specified time for a condition to occur. This goes hand-in-hand with ExpectedConditions class, which measures the status of elements on the page to confirm whether or not a WebElement meets a certain condition.
So, WebDriverWait(driver, 10).until(EC.presence_of_element_located means "Wait up to 10 seconds for the presence of a WebElement" -- this WebElement then gets specified in the locator strategy, By.XPath, "....".
Hope this explanation helps a bit.
I am stuck and I would love your help. I am not a Python genius, so apologies for the language. I need to click on a button (Export) on this website https://www.fec.gov/data/filings/?data_type=processed&committee_id=C00097485. The button should drive to the bottom of the page where a link to an Excel file appears. Now, I have used this code:
text="//button[#type='button' and contains(.,'Export')]"
driver = webdriver.Firefox()
driver.get("https://www.fec.gov/data/filings/data_type=processed&committee_id=C00142711")
time.sleep(5)
button=driver.find_element_by_xpath(text)
button.click
The script runs fine, no error messages. The website appears, but the 'click' doesnt take place.
I also tried:1) the "wait driver until element is clickable", 2) the ActionChain to move the cursor, 3) to substitute click with sendKeys.
There is no Iframe. I tried also on Chrome. I am using a pc with Windows 10.
What am I doing wrong??? Considering that with other websites, the click function works perfectly fine!
Click is a method so it should be button.click(). You are missing parens.
Also, it would be better if you used WebDriverWait instead of .sleep(), e.g. button = WebDriverWait(driver, 20).until( EC.element_to_be_clickable((By.XPATH, text)));
As per the url the button with text as Export is a JavaScript enabled element, so you need to induce _WebDriverWait_for the element to be clickable and you can use the following solution:
Code Block:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get("https://www.fec.gov/data/filings/?data_type=processed&committee_id=C00097485")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "button.js-export.button.button--cta.button--export"))).click()
Browser Snapshot: