How to accept span button using Selenium? - selenium

Question

I'm trying to accept cookies with Selenium, but the accept button is not found. I am not familiar with Selenium and I don't know how to debug. For instance, if I try to accept cookies from webrankinfo.com.
This is my code:
driver = webdriver.Chrome("chromedriver")
driver.get("https://www.webrankinfo.com")
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[normalize-space()='Tout accepter et continuer']"))).click()
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//span[contains(., 'Tout accepter et continuer')]"))).click()
Whatever the selected option (Xpath or CSS), the button is not found. What is the solution?
Moreover, is there any option to debug Xpath directly on my Web Browser like Chrome or Firefox?
EDIT :
Because most people can't see the cookie banner, I share the original cookie framework used. I think you can't see banner because it's not obligated when you're not in Europe.
You can force banners to appear by going to https://www.consentframework.com/#/ and clicking on Gérer vos préférences (green button) on top right.

Answer 1

I can help you using a CSS selector, I've managed to locate the element and here's what you can utilise:
#sd-cmp .sd-cmp-3V2Vm span.sd-cmp-3cRQ2
#sd-cmp is the unique ID for the cookie banner
.sd-cmp-3V2Vm is a row class containing the Accept all/do not accept/Set your choices buttons. I've used this because it seems you have an older version of this row class (sd-cmp-25TOo) containing the exact same button locators in the DOM but are no longer visible on the UI - this is likely the cause of the problems as it's finding these hidden ones first
span.sd-cmp-3cRQ2 is specifically the Accept all button
Screenshot - Accept All selected

Answer 2

I see no any "accept cookies" button appearing on that web page, so I can't help with providing locator for it.
As about debugging the locators with Chrome of Firefox:
You can press F12 on your browser, it will open the Developer Tools.
Then select Elements tab there.
Then you can locate any element with the arrow in the upper-left corner there.
It will present all the attributes of the selected element.
Control+F on that view will allow you to search elements on that page with XPath or CSS Selector.
See also here or any other resource about locating web elements with Developer Tools.

Answer 3

As Prophet, no cookie alert appears for me, so I can't help you locate the button. Although, I can help you trying to make the cookie alert not appear at all.
Some websites will only require cookies for certain browsers or browser versions. This way, changing the user agent of the WebDriver may cause the alert to disappear. Your code would be like this:
options = webdriver.ChromeOptions()
user_agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36'
options.add_argument('user-agent={0}'.format(user_agent))
driver = webdriver.Chrome("chromedriver")
driver.get("https://www.webrankinfo.com")
That user agent is just an example, maybe it won't work but others will. You're able to find older versions of Chrome here.
What may be happening, also, is that the website is detecting an automate tool. It would explain why the cookie alert doesn't show up for us. To prevent the detection, you'll have to add some option arguments to WebDriver:
options = webdriver.ChromeOptions()
options.add_argument('--no-sandbox')
options.add_argument('--allow-running-insecure-content')
options.add_argument('--ignore-certificate-errors')
options.add_argument('-allow-insecure-localhost')
options.add_argument('--remote-debugging-port=9222')
options.add_argument('--disable-gpu')
options.add_argument('--start-maximized')
options.add_experimental_option("useAutomationExtension", False)
options.add_experimental_option("excludeSwitches",["enable-automation"])
user_agent = 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/94.0.4606.81 Safari/537.36'
options.add_argument('user-agent={0}'.format(user_agent))
driver = webdriver.Chrome("chromedriver")
driver.get("https://www.webrankinfo.com")
Some of these arguments may be useless. So you can test them one by one and let on your code only the usefull ones.

Answer 4

As I seen the cookie info thing loads later than the page it self so use wait before the actual click on the accept button.
WebDriverWait wait = new WebDriverWait(driver, Duration.ofSeconds(10));
wait.until(ExpectedConditions.elementClickable( insert your element here ));
Then another suggestion for the xpath, if you want to find a span by the text inside it
//span[text()='Accept all']
and I think the correct form for your second attempt would be
//span[text()[contains(.,'Accept all')]]