I get the following error:
File "C:\Users\ryanv\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.9_qbz5n2kfra8p0\LocalCache\local-packages\Python39\site-packages\selenium\webdriver\firefox\service.py", line 44, in __init__
log_file = open(log_path, "a+") if log_path is not None and log_path != "" else None
PermissionError: [Errno 13] Permission denied: 'geckodriver.log'
when I'm trying to run
browser = webdriver.Firefox()
Could this be because firefox is in the pythonfoundation folder and I have another folder where python is?
I have tried adding geckodriver to path as well as downloading it and putting it into my other python folder however nothing works.
in your cmd line type: where python
and put the gecko file in that folder where python is located
Related
Apologies I am quite new to all this so trying to figure out if I am doing the right thing. I am running Ubuntu 22.04 and Selenium 4.8
I installed latest Geckodriver from Mozilla GitHub, converted it into executable and then moved it to /usr/local/bin/
I then downloaded webdriver manager and tried to run the following snippet for Firefox:
# selenium 4 - updating geckodriver for firefox
from selenium import webdriver
from selenium.webdriver.firefox.service import Service as FirefoxService
from webdriver_manager.firefox import GeckoDriverManager
driver = webdriver.Firefox(service=FirefoxService(GeckoDriverManager().install()))
When I run that, it starts downloading: [WDM] - Downloading: 19.0kB [00:00, 13.9MB/s] and then it fails I get following error: selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 127
full error:
[WDM] - Downloading: 19.0kB [00:00, 13.9MB/s]
Traceback (most recent call last):
File "/home/gizmo/PycharmProject/PythonProjects/web/firefox_selenium.py", line 6, in <module>
driver = webdriver.Firefox(service=Service(GeckoDriverManager().install()))
File "/home/gizmo/PycharmProject/PythonProjects/venv/lib/python3.10/site-packages/selenium/webdriver/firefox/webdriver.py", line 196, in __init__
super().__init__(command_executor=executor, options=options, keep_alive=True)
File "/home/gizmo/PycharmProject/PythonProjects/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 286, in __init__
self.start_session(capabilities, browser_profile)
File "/home/gizmo/PycharmProject/PythonProjects/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 378, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/home/gizmo/PycharmProject/PythonProjects/venv/lib/python3.10/site-packages/selenium/webdriver/remote/webdriver.py", line 440, in execute
self.error_handler.check_response(response)
File "/home/gizmo/PycharmProject/PythonProjects/venv/lib/python3.10/site-packages/selenium/webdriver/remote/errorhandler.py", line 245, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.WebDriverException: Message: Process unexpectedly closed with status 127
Where am I going wrong?
Goal: tried to get the webdriver manager to update my geckodriver for Firefox - ran it as a test to see if it is all setup correctly.
Pre-requisites
Ensure that:
Selenium is upgraded to v4.0.0 or above
pip3 install -U selenium
Webdriver Manager for Python is installed
pip3 install webdriver-manager
You can find a detailed discussion on installing Webdriver Manager for Python in ModuleNotFoundError: No module named 'webdriver_manager' error even after installing webdrivermanager
You don't have to cast to FirefoxService instance and can use the Service instance. You can use:
from selenium import webdriver
from selenium.webdriver.firefox.service import Service
from webdriver_manager.firefox import GeckoDriverManager
driver = webdriver.Firefox(service=Service(GeckoDriverManager().install()))
Note: Ensure that your firefox is updated to the latest version.
I have Anaconda with python 3.8 on macOS Big Sur. The python script works just fine within PyCharm or interactively inside a shell script :
/Users/nicholaskalita/opt/anaconda3/bin/python3.8 /Users/nicholaskalita/PycharmProjects/CrpytoScrape/CMCScrape.py
The shell script need to be launched regularly, which is where the trouble begins. launchd starts it (crontab doesn't seem to work on MacOs but that's another story) as root but the python script fails with
Traceback (most recent call last):
File "/Users/nicholaskalita/PycharmProjects/CrpytoScrape/CMCScrape.py", line 241, in
dframe.to_csv(FilePath+NQuotes, index=False)
File "/Users/nicholaskalita/opt/anaconda3/lib/python3.8/site-packages/pandas/core/generic.py", line 3170, in to_csv
formatter.save()
File "/Users/nicholaskalita/opt/anaconda3/lib/python3.8/site-packages/pandas/io/formats/csvs.py", line 185, in save
f, handles = get_handle(
File "/Users/nicholaskalita/opt/anaconda3/lib/python3.8/site-packages/pandas/io/common.py", line 493, in get_handle
f = open(path_or_buf, mode, encoding=encoding, errors=errors, newline="")
PermissionError: [Errno 1] Operation not permitted: '/usr/local/ ...
The destination directory happens to be on network drive but neither applying chmod 777 to it nor moving to a local disk solves the problem.
After clearing out the following directories this problem vanished
~/Library/LaunchAgents
/Library/LaunchAgents
/Library/LaunchDaemons
/Library/StartupItems
I'm trying to follow Automate the Boring Stuff and am trying to do the selenium work. I’m running Linux using Mu. Here's the code I'm trying to run
from selenium import webdriver
chromedriver = “/lorenmspeer/Downloads/chromedriver.exe”
browser = webdriver.Chrome(executable_path=chromedriver)
url = “https://www.duckduckgo.com”
browser.get(url)
I’m getting this error:
Traceback (most recent call last):
File “/home/lorenmspeer/.local/share/mu/mu_venv-38-20210401-164244/lib/python3.8/site-packages/selenium/webdriver/common/service.py”, line 72, in start
self.process = subprocess.Popen(cmd, env=self.env,
File “/usr/lib/python3.8/subprocess.py”, line 854, in init
self._execute_child(args, executable, preexec_fn, close_fds,
File “/usr/lib/python3.8/subprocess.py”, line 1702, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: ‘/lorenmspeer/Downloads/chromedriver.exe’
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File “/home/lorenmspeer/mu_code/seleniumTest22.py”, line 5, in
browser = webdriver.Chrome(executable_path=chromedriver)
File “/home/lorenmspeer/.local/share/mu/mu_venv-38-20210401-164244/lib/python3.8/site-packages/selenium/webdriver/chrome/webdriver.py”, line 73, in init
self.service.start()
File “/home/lorenmspeer/.local/share/mu/mu_venv-38-20210401-164244/lib/python3.8/site-packages/selenium/webdriver/common/service.py”, line 81, in start
raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: ‘chromedriver.exe’ executable needs to be in PATH. Please see https://sites.google.com/a/chromium.org/chromedriver/home
To check where Chrome driver is installed on Linux run whereis chromedriver. Then, put this path to Chrome("your_path")
In my project it is specified in the folder I defined:
driver = webdriver.Chrome("/usr/lib/chromium-browser/chromedriver")
It should not have .exe resolution
For more info check docs:
https://chromedriver.chromium.org/getting-started
driver = webdriver.Chrome('/path/to/chromedriver') # Optional argument, if not specified will search path.
I found a solution. First I had to make the file executable:
chmod +x chromedriver
Then I moved it to PATH
sudo mv chromedriver /usr/local/bin/
This made the code launch the browser.
I am trying to scrape a web site using Selenium, gecko driver and firefox. If I execute the script as root or as an ordinary user, it is working fine. But when I run it as apache (which is how it will be running in production), it is failing. Versions -
Mozilla Firefox 72.0
geckodriver 0.26.0
I have set log level to trace. Entries appearing in geckolog
1579004036460 mozrunner::runner INFO Running command: "/bin/firefox" "-marionette" "--disable-extensions" "--disable-infobars" "--disable-images" "--headless" "--disable-gpu" "--no-sandbox" "--disable-setuid-sandbox" "-foreground" "-no-remote" "-profile" "/tmp/rust_mozprofileKkQRyT"
1579004036461 geckodriver::marionette DEBUG Waiting 60s to connect to browser on 127.0.0.1:37317
*** You are running in headless mode.
(firefox:21013): dconf-CRITICAL **: 12:14:56.520: unable to create directory '/usr/share/httpd/.cache/dconf': Permission denied. dconf will not work properly.
(firefox:21013): dconf-CRITICAL **: 12:14:56.521: unable to create directory '/usr/share/httpd/.cache/dconf': Permission denied. dconf will not work properly.
(firefox:21013): dconf-CRITICAL **: 12:14:56.521: unable to create directory '/usr/share/httpd/.cache/dconf': Permission denied. dconf will not work properly.
(firefox:21013): dconf-CRITICAL **: 12:14:56.521: unable to create directory '/usr/share/httpd/.cache/dconf': Permission denied. dconf will not work properly.
(firefox:21013): dconf-CRITICAL **: 12:14:56.524: unable to create directory '/usr/share/httpd/.cache/dconf': Permission denied. dconf will not work properly.
1579004096533 mozrunner::runner DEBUG Killing process 21013
1579004096537 webdriver::server DEBUG <- 500 Internal Server Error {"value":{"error":"timeout","message":"connection refused","stacktrace":""}}
In the session where I am executing the script, this is the output
Traceback (most recent call last):
File "/var/www/html/***.py", line 36, in <module>
scrap_price()
File "/var/www/html/***.py", line 26, in scrap_price
driver = webdriver.Firefox(firefox_binary=binary, executable_path=gecko, options=options,firefox_profile=firefoxProfile, service_log_path='/tmp/geckodriver.log')
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/firefox/webdriver.py", line 174, in __init__
keep_alive=True)
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 157, in __init__
self.start_session(capabilities, browser_profile)
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 252, in start_session
response = self.execute(Command.NEW_SESSION, parameters)
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/webdriver.py", line 321, in execute
self.error_handler.check_response(response)
File "/usr/local/lib/python3.5/site-packages/selenium/webdriver/remote/errorhandler.py", line 242, in check_response
raise exception_class(message, screen, stacktrace)
selenium.common.exceptions.TimeoutException: Message: connection refused
Relevant sections of the code
options = Options()
options.add_argument('--disable-extensions')
options.add_argument('--disable-infobars')
options.add_argument('--disable-images')
options.add_argument('--headless')
options.add_argument('--disable-gpu')
options.add_argument('--no-sandbox')
options.add_argument('--disable-setuid-sandbox')
options.log.level = 'trace'
gecko = os.path.normpath(os.path.join(os.path.dirname(__file__), 'geckodriver'))
binary = FirefoxBinary('/bin/firefox')
firefoxProfile = webdriver.FirefoxProfile()
firefoxProfile.set_preference('permissions.default.stylesheet', 2)
firefoxProfile.set_preference('permissions.default.image', 2)
firefoxProfile.set_preference('dom.ipc.plugins.enabled.libflashplayer.so','false')
driver = webdriver.Firefox(firefox_binary=binary, executable_path=gecko, options=options,firefox_profile=firefoxPro
file, service_log_path='/tmp/geckodriver.log')
driver.get(urlpage)
OS is CentOS Linux release 7.6 with SELinux enabled. But sealert does not show any errors related to this.
I am using Selenium and Virtualenv in Ubuntu14.04 to scrape some code on webpage. After making a virtualenv directory, while activating virtualenv using
source bin/activate,
I installed selenium, headless firefox, pyvirtualdisplay and firefox geckodriver.
I installed headless firefox according to here.
And I also Found out that there are different python2.7 folders at
"/usr/lib/python2.7/" and "~/ladder_selenium/lib/python2.7" (ladder_selenium is the Virtualenv directory)
and after I ran a python code which contains
from selenium import webdriver
, the error below happens.
Traceback (most recent call last):
File "predict.py", line 12, in <module>
driver = webdriver.Firefox()
File "/home/heyjude/ladder_selenium/local/lib/python2.7/site-packages/selenium/webdriver/f
self.service.start()
File "/home/heyjude/ladder_selenium/local/lib/python2.7/site-packages/selenium/webdriver/c
stdout=self.log_file, stderr=self.log_file)
File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 8] Exec format error
Could you tell me what to fix?
EDIT for the COMMENT BELOW:
(ladder_selenium) heyjude#ladder-selenium:~$ export DISPLAY=:99
(ladder_selenium) heyjude#ladder-selenium:~$ firefox
XPCOMGlueLoad error for file /opt/firefox/libmozgtk.so:
libgtk-3.so.0: cannot open shared object file: No such file or directory
Couldn't load XPCOM.