I am running Ubuntu 14.04, Firefox 49.0.2, Python 3.4.3 & Selenium 3.0.1
I want to use Selenium to automate some browser functions, not to do any web site testing. How can I can I modify the simple login script below to use the instance of Firefox running on my desktop instead of opening a new Firefox window?
# login_yahoo_mail.py
from selenium import webdriver
from selenium.webdriver.common.keys import Keys
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
driver = webdriver.Firefox()
driver.get('http://mail.yahoo.com/?.intl=us')
enter_email = driver.find_element_by_id('login-username')
enter_email.clear()
enter_email.send_keys('cleanman2#yahoo.com')
next_button = driver.find_element_by_id('login-signin')
next_button.click()
enter_password = WebDriverWait(driver,10).until(EC.visibility_of_element_located((By.ID, 'login-passwd')))
enter_password.send_keys('dumba$$yahoo!^!&')
signin_button = driver.find_element_by_id('login-signin')
signin_button.click()
Thanks, Jim
It is possible with 'Selenium 4'.
Each window has a unique identifier which remains persistent in a single session. You can get the window handle of the current window by using:
driver.current_window_handle
driver = webdriver.Firefox()
# Store the ID of the original window
original_window = driver.current_window_handle
# Opens a new tab and switches to new tab
driver.switch_to.new_window('tab')
# Opens a new window and switches to new window
driver.switch_to.new_window('window')
#Close the tab or window
driver.close()
#Switch back to the old tab or window
driver.switch_to.window(original_window)
I haven't try it yet but since Webdriver doesn't know where the OS focus is, you can find the way you want with those options.
More: Windows and tabs handle
Related
So I'm trying to scrape some forecasting data from a website periodically, and ideally I would like for it to happen in the background. I had a look at some documentation and came up with the following code:
from selenium import webdriver
from selenium.webdriver.common.by import By
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.chrome.options import Options
options = webdriver.ChromeOptions()
options.add_argument("--headless")
driver = webdriver.Chrome(options = options)
options.add_argument("disable-infobars")
options.add_argument("--disable-extensions")
driver = webdriver.Chrome()
driver.get('https://www.windguru.cz/53')
WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#forecasts-page")))
\#Scraping block of code goes here
driver.quit()
I think the following line is over-riding the --headless argument but i'm not sure.
WebDriverWait(driver, 5).until(EC.visibility_of_element_located((By.CSS_SELECTOR, "#forecasts-page")))
The reason I have it added in the first place is that the website I'm scraping isn't just static html (have a look for yourself, link is in code). I think there's some js that prompts the forecast data to load, so I need to wait a bit and make sure before the script starts scraping the dom.
Any idea how I can achieve this and run the browser in headless mode?
The line that's opening the chrome is this
driver = webdriver.Chrome()
You can do away with this and code should still work the same.
I have successfully installed Selenium in Databricks and can import the Python selenium and webdriver. On my local computer once I run the Selenium get command a separate browser windows open where I can see what Selenium is doing.
However, when running the same script on Databricks there is unfortunately no window opening. I am wondering if this is at all possible. I found some options like
browser.execute_script('''window.open("http://bings.com","_blank");''')
or
add_experimental_option
but these options did not work.
I am currently using the following options on Databricks:
from selenium import webdriver
from webdriver_manager.chrome import ChromeDriverManager
from webdriver_manager.utils import ChromeType
import pandas as pd
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument('--no-sandbox')
chrome_options.add_argument('--headless')
chrome_options.add_argument('--disable-dev-shm-usage')
# installed driver
chrome_driver = "/tmp/chromedriver1/chromedriver"
driver = webdriver.Chrome(executable_path=chrome_driver, options=chrome_options)
Has someone an idea how this could work?
Thanks in advance!
No it won't open since you are using headless browser. See the below line from your code :-
chrome_options.add_argument('--headless')
Having said that, it will execute the Python-Selenium bindings instruction, you have. You can try to print the page title like this :
driver = webdriver.Chrome(executable_path=chrome_driver, options=chrome_options)
driver.get("https://www.google.com")
print(driver.title)
should print google.com title in the console.
I am using Python's Selenium Webdriver with Firefox, and trying to
navigate to a webpage that embeds a frame
open the devtools menu (Ctrl+Shift+K in Firefox)
select this other iframe from the iframe context picker exposed by devtools.
Question:
How to achieve this?
What I've tried:
My current script reads
#!/usr/bin/env python
from selenium.webdriver import Firefox, DesiredCapabilities, FirefoxProfile
from selenium.webdriver.common.by import By
from selenium.webdriver.firefox.options import Options
import time
import traceback
import os
options = Options()
options.add_argument("-devtools")
webdriver = Firefox(options=options)
webdriver.get(<url that will embed the iframe>)
try:
<do a bunch of stuff that will trigger the appearance of the iframe>
except:
pass
try:
time.sleep(3)
with webdriver.context(webdriver.CONTEXT_CHROME):
ifr =
webdriver.find_element_by_class_name("devtools-toolbox-bottom-iframe")
except Exception as error:
traceback.print_exc()
This doesn't complain, because it does find an element with class devtools-toolbox-bottom-iframe. My problem is that I do not know what to do with it afterwards to access the iframe-selection menu underneath. That element has a tag of browser, i.e. looks like this:
<browser type="content" flex="1" class="devtools-toolbox-bottom-iframe" height="250" aria-label="Developer Tools" src="about:devtools-toolbox" session_id="910"/>
This old answer of mine no longer works, because back then this same element was an iframe and I could simply switch to it with
webdriver.switch_to.frame(<iframe>)
I have also tried clicking it (with ifr.click()) to no effect: I see nothing happening in the Firefox window that the script opens (no errors either though).
When using Selenium to create a Chrome app, chromedriver.exe still in background when user close the tab manually. Is there any way to fix this?
I don't want to use driver.quit() in my code, because user need to do something in this chrome app.
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--app=https://www.jianshu.com/sign_in')
driver = webdriver.Chrome(".\\chromedriver.exe", options=chrome_options)
driver.find_element_by_xpath('//*[#id=\"session_email_or_mobile_number\"]').send_keys('aaaa')
driver.find_element_by_xpath('//*[#id=\"session_password\"]').send_keys('bbbbbbbbbb')
I think we don't have a direct solution to close the chromedriver.exe when user manually close the chrome instance launched by selenium. Alternatively you can save the below command as bat and then double click each time you close the chrome tab manually.
taskkill /IM chromedriver.exe /F
`from selenium import webdriver
from selenium.webdriver.chrome.options import Options
chrome_options = Options()
chrome_options.add_argument('--app=https://www.jianshu.com/sign_in')
driver = webdriver.Chrome(".\\chromedriver.exe", options=chrome_options)
driver.find_element_by_xpath('//*
[#id=\"session_email_or_mobile_number\"]').send_keys('aaaa')
driver.find_element_by_xpath('//*[#id=\"session_password\"]').send_keys('bbbbbbbbbb')
driver.close()
///just add driver.close() to close the chrome browser after the test
I need to download a xls file from a website using chrome and selenium. There are multiple websites I need to go and so I need to open new tabs. However, when I open the second tab, I cannot download the file I need. Below are simple version my code. Image that I have just download some file from one tab and then open a new one using window.open():
import time
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.common.by import By
from selenium.webdriver.support import expected_conditions as EC
options = webdriver.ChromeOptions()
prefs = {'download.default_directory' : SAVE_PATH, "download.prompt_for_download": False}
options.add_experimental_option('prefs', prefs)
driver = webdriver.Chrome(executable_path = DRIVE_PATH, chrome_options = options)
driver.execute_script("window.open('https://www.fhfa.gov/DataTools/Downloads/Pages/House-Price-Index-Datasets.aspx#mpo');")
time.sleep(5)
driver.switch_to.window(driver.window_handles[1])
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.XPATH, "//*[#id='WebPartWPQ2']/div[1]/table[3]/tbody/tr[2]/td[2]/p/a"))).click()
Without opening new tab, I could download the file successfully. But after opening new tab, chrome tells me "Fail - Download error". Something wrong with my code?
MacOS, Chrome Version 76.0.3809.100, ChromeDriver version 75.0.3770.140 in download success in both ways.
To locate download link better to use css-selectors below, you find more information about locator strategies here
WebDriverWait(driver, 20).until(EC.element_to_be_clickable((By.CSS_SELECTOR, "a[href*='HPI_PO_summary.xls']"))).click()
Faster way is to use requests to download files from https://www.fhfa.gov/, here's example:
import requests
import os
file_name = "HPI_PO_summary.xls"
response = requests.get(f'https://www.fhfa.gov/DataTools/Downloads/Documents/HPI/{file_name}')
with open(os.path.join(SAVE_PATH, file_name), 'wb') as f:
f.write(response.content)
Answering my question in here:
It seems the issue is in the SAVE_PATH. Initially my SAVE_PATH was:
r"C:\Users\hw\Desktop\myfile\"
And for some reason it works (based on the answer here) if I add one more slash to the end of the path:
r"C:\Users\hw\Desktop\myfile\\"