Selenium requires a driver to interface with the chosen browser. Firefox, for example, requires geckodriver, which needs to be installed before the below examples can be run.
after installing chrome driver for selenium in path directory
from time import sleep
from selenium import webdriver
# this is tested on Firefox or you can use "webdriver.Chrome()"
browser = webdriver.Firefox()
browser.get(‘https://www.facebook.com/')
sleep(5)
browser.close()
This error message...
SyntaxError: invalid character in identifier
...implies that there is a syntax error at line 5 i.e. in the url string you have passed through get() method.
As #cruisepandey correctly pointed out, you need to pass the url_string within single quotes i.e. '...':
browser.get('https://www.facebook.com/')
or within double quotes i.e. "...":
browser.get("https://www.facebook.com/")
Related
Running Selenium with Chrome Webdriver; my program worked for 4 months until Edge suddenly stopped working, switched to Chrome at behest of another [far more capable] scripter
Now Chrome doesn't stay open and won't run my program! I literally ONLY changed which webdriver I use for this code; from Edge --> Chrome! It worked fine before
driver = webdriver.Chrome('C:\path..')
df = pd.read_excel('my_workbook...')
while variable_in_df <= X:
driver.get(f'URL...')
(((gets CSS element whatever)))
return (CSS element from webpage)
The erros are:
Traceback (most recent call last):
driver = launchBrowser()
driver = webdriver.Chrome(executable_path=PATH);
super(WebDriver, self).__init__(DesiredCapabilities.CHROME['browserName'], "goog",
RemoteWebDriver.__init__(
self.start_session(capabilities, browser_profile)
response = self.execute(Command.NEW_SESSION, parameters)
self.error_handler.check_response(response)
Another error says ChromeDriver is version 106 but wehnever I download Chrome it says it's 105. wghat??
You need to match the driver to the version of the browser you are using. It may be that your script stopped working with MS Edge because your browser updated and the webdriver didn't match the newer version of the browser anymore. You will need to periodically update the webdriver to match the browser version.
In the case of Chrome not working, it again sounds like a driver version mismatch. It sounds like your browser is on version 105, but you have the 106 driver. I suggest checking the version of Chrome you have installed on your machine and downloading the matching driver from here:
https://chromedriver.chromium.org/downloads
Alternatively, you can switch back to Edge and get the matching driver here:
https://developer.microsoft.com/en-us/microsoft-edge/tools/webdriver/
You can check the version in most browsers under Help > About or similar from the main menu.
I hope this helps.
To overcome this problem, use WebDriverManager: refer - https://bonigarcia.dev/webdrivermanager/ and https://github.com/bonigarcia/webdrivermanager.
To install: pip install webdriver-manager
Then, in the code, add the below lines:
For Chrome:
from selenium.webdriver.chrome.service import Service as ChromeService
from webdriver_manager.chrome import ChromeDriverManager
driver = webdriver.Chrome(service=ChromeService(ChromeDriverManager().install()))
For Edge:
from selenium.webdriver.edge.service import Service as EdgeService
from webdriver_manager.microsoft import EdgeChromiumDriverManager
driver = webdriver.Edge(service=EdgeService(EdgeChromiumDriverManager().install()))
I'm new at Python and I'm trying to use selenium webdriver to do web scraping from a webpage. When I run my code, I have not problems in obtaining the results I need, but when someone else tries to run the code from an executable it shows the error: selenium.common.exceptions.webdriverexception: message: 'chromedriver.exe' unexpectedly exited.status code was: 1. The path where I'm saving chromedriver.exe is a public repository. Can someone help me with this please? Here is the piece of code I'm using:
from selenium import webdriver
url= "https://www.byma.com.ar/obligaciones-negociables/"
driver = webdriver.Chrome(executable_path=r'\\path\\chromedriver.exe')
driver.implicitly_wait(30)
driver.get(url)
time.sleep(2)
You do not need to setup a driver like this :
driver = webdriver.Chrome(executable_path=r'\\path\\chromedriver.exe')
do this :
This is a pre - requisite
Installation
pip install chromedriver-autoinstaller
Usage:
Just type import chromedriver_autoinstaller in the module you want to use chromedriver.
Example
from selenium import webdriver
import chromedriver_autoinstaller
chromedriver_autoinstaller.install() # Check if the current version of chromedriver exists
# and if it doesn't exist, download it automatically,
# then add chromedriver to path
driver = webdriver.Chrome()
driver.get("http://www.python.org")
assert "Python" in driver.title
and the export this changes to your executables
When I am trying to use Selenium I am getting an error as:
nvalidArgumentException: Message: invalid argument: user data directory is already in use, please specify a unique value for --user-data-dir argument, or don't use --user-data-dir
my code :
driver = webdriver.Chrome('chromedriver\chromedriver.exe')
# request url
driver.get('https://jamalon.com/ar/best-seller-books-2019')
#give it some time
sleep(2)
## retrive , download html page
html = driver.page_source
#close
driver.close()
i try to do same solution here but i get same error :(
InvalidArgumentException: Message: invalid argument: user data directory is already in use error using --user-data-dir to start Chrome using Selenium
First if you're using Pycharm then keep your chromedriver.exe in the location where your current python file is located.
Then use the below code:
from selenium import webdriver
import time
driver = webdriver.Chrome()
# request url
driver.get('https://jamalon.com/ar/best-seller-books-2019')
#give it some time
time.sleep(2)
## retrive , download html page
html = driver.page_source
print(html)
#close
driver.close()
Output:
First, machine and package specs:
I am running:
ChromeDriver version 75.0.3770.140
Selenium: version '3.141.0'
WSL (linux subsystem) of windows 10
I am trying to run a chromebrowser through selenium. I found: these commands, to use selenium through google chrome.
I have a test directory, with only the chromedriver binary file, and the script, in it. The location of the directory is: /home/kela/test_dir/
I ran the code:
import selenium
from selenium import webdriver
from bs4 import BeautifulSoup
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
options = Options()
options.binary_location='/home/kela/test_dir/chromedriver'
driver = webdriver.Chrome(chrome_options = options,executable_path='/home/kela/test_dir/chromedriver')
The output from this code is:
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: No matching capabilities found
Can anyone explain why I need capabilities when the same script works for others without capabilities? I did try adding:
chrome_options.add_argument('--headless')
chrome_options.add_argument('--no-sandbox')
but I got the same error. So I'm not sure what capabilities I need to add (considering it works for others without it?)
Edit 1: Addressing DebanjanB's comments below:
Chromedriver is in the expected location. I am using windows 10. From here, the expected location is C:\Program Files (x86)\Google\Chrome\Application\chrome.exe; and this is where it is on my machine (I copied and pasted this location from the chrome Properties table).
ChromeDriver is having executable permission for non-root users.
I definitely have Google Chrome v75.0 installed (I can see that the Product version 75.0.3770.100)
I am running the script as a non-root user, as my bash command line ends with a $ and not # (i.e kela:~/test_dir$ and not kela:~/test_dir#)
Edit 2: Based on DebanjanB's answer below, I am very close to having it working, but just not quite.
The code:
import selenium
from selenium import webdriver
from bs4 import BeautifulSoup
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.firefox_binary import FirefoxBinary
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location='/c/Program Files (x86)/Google/Chrome/Application/chrome.exe'
driver = webdriver.Chrome(options=options)
driver.get('http://google.com/')
Produces a dialog box that reads:
Google Chrome cannot read and write to it's data directory: /tmp/.com/google.Chrom.gyw63s
So then I double checked my Chrome permissions and I should be able to write to Chrome:
Also, I can see that /tmp/ has a bunch of .com dirs in it:
.com.google.Chrome.4jnWme/ .com.google.Chrome.FdNyKP/ .com.google.Chrome.VAcWMQ/ .com.google.Chrome.ZbkRx0/ .com.google.Chrome.iRrceF/
.com.google.Chrome.A2QHHB/ .com.google.Chrome.G7Y51c/ .com.google.Chrome.WD8BtK/ .com.google.Chrome.cItmhA/ .com.google.Chrome.pm28hN/
However, since that seemed to be more of a warning than an error, I clicked 'ok' to close the dialog box, and a new tab does open in the browser; but the URL is just 'data:,'. The same thing happens if I remove the line 'driver.get('http://google.com')' from the script, so I know the warning/issue is with the line:
driver = webdriver.Chrome(chrome_options = options,executable_path='/home/kela/test_dir/chromedriver')
For example, from here, I tried adding:
options.add_argument('--profile-directory=Default')
But the same warning pops up.
Edit 3:
As edit 3 was starting to veer into a different question than specifically being addressed here, I started a new question here.
This error message...
selenium.common.exceptions.SessionNotCreatedException: Message: session not created: No matching capabilities found
...implies that the ChromeDriver was unable to initiate/spawn a new WebBrowser i.e. Chrome Browser session.
binary_location
binary_location set/get(s) the location of the Chrome (executable) binary and is defined as:
def binary_location(self, value):
"""
Allows you to set where the chromium binary lives
:Args:
- value: path to the Chromium binary
"""
self._binary_location = value
So as per your code trials, options.binary_location='/home/kela/test_dir/chromedriver' is incorrect.
Solution
If Chrome is installed at the default location, you can safely remove this property. Incase Chrome is installed at a customized location you need to use the options.binary_location property to point to the Chrome installation.
You can find a detailed discussion in Selenium: WebDriverException:Chrome failed to start: crashed as google-chrome is no longer running so ChromeDriver is assuming that Chrome has crashed
Effectively, you code block will be:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
options = Options()
options.binary_location=r'C:\Program Files (x86)\Google\Chrome\Application\chrome.exe'
driver = webdriver.Chrome(options=options, executable_path='/home/kela/test_dir/chromedriver.exe')
driver.get('http://google.com/')
Additionally, ensure the following:
ChromeDriver is having executable permission for non-root users.
As you are using ChromeDriver v75.0 ensure that you have the recommended version of the Google Chrome v75.0 as:
---------ChromeDriver 75.0.3770.8 (2019-04-29)---------
Supports Chrome version 75
Execute the Selenium Test as non-root user.
When i tried to take screenshot of a webpage using selenium in python, i get error message selenium.common.exceptions.TimeoutException: Message: timeout: Timed out receiving message from renderer: 10.000.
Code:
from selenium import webdriver
from selenium.webdriver.chrome.options import Options
indi_url = 'http://www.google.com'
options = Options()
options.add_argument("disable-infobars")
options.add_argument("--start-maximized")
options.add_argument("--disable-popup-blocking")
options.add_argument("disable-popup-blocking")
options.add_argument("--disable")
driver = webdriver.Chrome(options=options)
driver.get(indi_url)
driver.implicitly_wait(30)
driver.save_screenshot("new.png")
Error message:
I'm using Chrome version 73, chromedriver version 73.
Note: code was working fine (ie.screenshot)in lower version of chrome and chrome driver.
Help me out in fixing this issue for new version of chrome driver.
Thanks in advance
As the error shows, your filename for screenshot does not match the template extensions .png
Here is an example how to make a screenshot.
Java:
File scrFile = ((TakesScreenshot)driver).getScreenshotAs(OutputType.FILE);
FileUtils.copyFile(scrFile, new File(".\\Screenshots\\example_screenshot.png"));
Python:
driver.save_screenshot("screenshot.png")
This error message...
UserWarning: name used for saved screenshot does not match file type. It should end with a .png extension
"type. It should end with a .png extension", UserWarning)
...implies that the Selenium-Python client encountered an issue while invoking get_screenshot_as_file() method.
get_screenshot_as_file()
get_screenshot_as_file() saves a screenshot of the current window to a PNG image file. Returns False if there is any IOError, else returns True. Use full paths in your filename.
Args:
filename: The full path you wish to save your screenshot to. This should end with a .png extension.
Usage:
driver.get_screenshot_as_file('/Screenshots/foo.png')
Defination:
if not filename.lower().endswith('.png'):
warnings.warn("name used for saved screenshot does not match file "
"type. It should end with a `.png` extension", UserWarning)
png = self.get_screenshot_as_png()
try:
with open(filename, 'wb') as f:
f.write(png)
except IOError:
return False
finally:
del png
return True
Analysis
As per the snapshot of the error stack trace:
You have used the command as:
driver.get_screenshot_as_file('new.jpeg')
The issues were:
The filename didn't end with .png
The desired full path of your filename wasn't provided.
Even if you desire to use save_screenshot() this method in-turn invokes get_screenshot_as_file(filename)
Solution
Create a directory within your project as Screenshots and provide the absolute path of the filename you desire for the screenshot while invoking either of the methods as follows:
driver.get_screenshot_as_file("./Screenshots/YakeshrajM.png")
driver.save_screenshot("./Screenshots/YakeshrajM.png")
Update
Currently GAed Chrome v73 have some issues and you may like to downgrade to Chrome v72. You can find a couple of relevant discussions in:
Getting Timed out receiving message from renderer: 600.000 When we execute selenium scripts using Jenkins windows service mode
Timed out receiving message from renderer: 10.000 while capturing screenshot using chromedriver and chrome through Jenkins on Windows