I am making a simple crawler that can open a site and when a pop up appears, it should close it. but the following command isn't working.
from selenium import webdriver
browser = webdriver.Chrome(executable_path=r"C:\Program Files\chromedriver.exe")
url = "https://www.bnbaccessories.com/"
browser.get(url)
alert = webdriver.switch_to_alert().dismiss()
innerHTML = browser.execute_script("return document.body.innerHTML")
browser.implicitly_wait(50)
browser.close()
Use this
alert = browser.switch_to.alert.dismiss()
instead
webdriver.switch_to_alert().dismiss()
driver instance name is browser not webdriver
Related
I am using selenium/webdriver to testing a web on Chrome,
I want to trace the network activity that happens after I click on all buttons, each clicking opens a new tab(i could not change anything to the buttons for it control by compressed javascript),
i tried Chrome Dev Tools: How to trace network for a link that opens a new tab? but it did not match my expect.
Below is a mock example, in the example i want to capture the new tab request "https://cdn.bootcss.com/jquery/3.2.1/jquery.min.js" but failed
(my actual scenario is that all web page opened in a android app, each click create a new tab/window)
import json
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
caps = {
"browserName": "chrome",
'goog:loggingPrefs': {'performance': 'ALL'}
}
options = Options()
options.add_experimental_option("w3c",False)
driver = webdriver.Chrome(desired_capabilities=caps, options=options)
# access a link
driver.get("https://www.google.com/")
# add a link for open new tab(just mock)
driver.execute_script('a = window.document.createElement("a");a.id="newtab";a.href="https://cdn.bootcss.com/jquery/3.2.1/jquery.min.js";a.target="_blank";a.text="33333";window.document.body.appendChild(a);')
time.sleep(5)
# click a button/link which open a new tab
element = driver.find_element_by_id('newtab')
driver.execute_script("arguments[0].click();", element)
time.sleep(3)
wins = driver.window_handles
driver.switch_to.window(wins[-1])
performance_log = driver.get_log('performance')
for packet in performance_log:
message = json.loads(packet.get('message')).get('message')
if message.get('method') != 'Network.responseReceived':
continue
requestId = message.get('params').get('requestId')
url = message.get('params').get('response').get('url')
try:
resp = driver.execute_cdp_cmd('Network.getResponseBody', {'requestId': requestId})
except BaseException as e:
resp = "error"
print("\n===============")
print(url)
# print(resp)
I am very new.
I was trying to make an auto mail sender for practice.
It opens website but not the login button.
There is nothing happen after it opened.
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.action_chains import ActionChains
import time
options = webdriver.ChromeOptions()
driver = webdriver.Chrome(
executable_path='/Users/aidencho/practice/chromedriver', options = options)
# open a website
url = 'https://naver.com'
driver.get(url)
# driver.maximize_window()
action = ActionChains(driver)
driver.find_element_by_css_selector('.link_login').click()
# driver.find_element_by_css_selector("#account > a").click()
# driver.find_element_by_class_name('.account > a').click()
One more thing.
I saw someone doing this, and there was a completed sentence for driver.find_element_by_css_selector part even he typed only driver.find.
Why not me?
Would there be a setting problem?
enter image description here
find_element_by_css_selector is deprecated. Please use find_element(by=By.CSS_SELECTOR, value=css_selector) instead.
driver.find_element(by="css selector", value='.link_login')
I Have this login page, so after pass keys and clicking login button, the webpage should redirect me to a new page with my session, and sometimes it does, but in most case, the driver.title by example, still having the title of the Authentication page and obiously this prevent me to find elements of the page that I'm looking for.
I Already try to driver.get(correct url) but didn't work.
Here are my WebDriver's options.
options = Options()
options.page_load_strategy = 'eager'
options.binary_location = '/opt/headless-chromium'
options.add_argument('--headless')
options.add_argument('--no-sandbox')
options.add_argument('--single-process')
options.add_argument('--disable-dev-shm-usage')
driver = webdriver.Chrome('/opt/chromedriver', options=options)
driver.implicitly_wait(60)
options.page_load_strategy = 'eager'
Remove this, this will not wait for page load to finish . So you might be trying to get title before the page is loaded
I try to open a Tor Browser using Selenium on Ubuntu 18. I have tried lots of examples but with no success.
proxyIP = "127.0.0.1"
proxyPort = "9050"
profileTor = '/etc/tor/' # torrc
binary = os.path.expanduser("~/.local/share/torbrowser/tbb/x86_64/tor-browser_en-US/Browser/firefox")
firefox_binary = FirefoxBinary(binary)
firefox_profile = FirefoxProfile(profileTor)
proxy_address = "127.0.0.1:9050"
proxy = Proxy({
'proxyType': ProxyType.MANUAL,
'httpProxy': proxy_address,
})
driver = webdriver.Firefox(firefox_binary = firefox_binary,firefox_profile=firefox_profile, proxy = proxy)
A blank Tor Browser window opens but after a while I get an error as:
selenium.common.exceptions.WebDriverException: Message: connection refused.
I have also tried an alternative to firefox binary the:
start-tor-browser
which opens a working Tor Browser and showing some index.
The script however stops and I cannot visit another page using Selenium unless I do it manually.
I have also tried the:
profile.default
as some examples suggest but I get an error:
Unable to start Tor. The torrc file is missing and could not be created.
To open a Tor Browser using Selenium you can start the Tor daemon first and then open the Tor Browser and you can use the following solution:
Sample WindowsOS style Code Block:
from selenium import webdriver
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
import os
torexe = os.popen(r'C:\Users\AtechM_03\Desktop\Tor Browser\Browser\TorBrowser\Tor\tor.exe')
profile = FirefoxProfile(r'C:\Users\AtechM_03\Desktop\Tor Browser\Browser\TorBrowser\Data\Browser\profile.default')
profile.set_preference('network.proxy.type', 1)
profile.set_preference('network.proxy.socks', '127.0.0.1')
profile.set_preference('network.proxy.socks_port', 9050)
profile.set_preference("network.proxy.socks_remote_dns", False)
profile.update_preferences()
driver = webdriver.Firefox(firefox_profile= profile, executable_path=r'C:\Utility\BrowserDrivers\geckodriver.exe')
driver.get("http://check.torproject.org")
I'm trying to set MutationObserver for observing page mutation while loading.
In order to do that, MutationObserver should be configured before page loading.
With selenium-chromedriver, couldn't find the way to inject JS for such purpose.
I know chrome extension can do that but extensions won't work on headless mode.
That's the problem.
It's possible via the DevTool API by calling Page.addScriptToEvaluateOnNewDocument
from selenium import webdriver
from selenium.webdriver.remote.webdriver import WebDriver
import json
def send(driver, cmd, params={}):
resource = "/session/%s/chromium/send_command_and_get_result" % driver.session_id
url = driver.command_executor._url + resource
body = json.dumps({'cmd': cmd, 'params': params})
response = driver.command_executor._request('POST', url, body)
if response['status']:
raise Exception(response.get('value'))
return response.get('value')
def add_script(driver, script):
send(driver, "Page.addScriptToEvaluateOnNewDocument", {"source": script})
WebDriver.add_script = add_script
# launch Chrome
driver = webdriver.Chrome()
# add a script which will be executed when the page starts loading
driver.add_script("""
if (window.self === window.top) { // if main document
console.log('add script');
}
""")
# load a page
driver.get("https://stackoverflow.com/questions")
We can now use execute_cdp_cmd(cmd, cmd_args) to execute Chrome Devtools Protocol command in Selenium
from selenium import webdriver
driver = webdriver.Chrome()
driver.execute_cdp_cmd(
"Page.addScriptToEvaluateOnNewDocument",
{
"source": """// Your JavaScript here"""
}
)
driver.get("https://stackoverflow.com")
driver.quit()
The argument for "source" is just a string. So you can actually write your script in a .js file (for syntax highlighting) and read it using Python