I'm running Selenium with Firefox browser driver on a Linux server. The root/tmp folder is kept flooding with temporary files generate by selenium, even though I have used driver.quit() after each driver call. Those temp files has names like tmp04kyw42n, Temp_xxx.
Here is my code:
opts = FirefoxOptions()
opts.add_argument("--headless")
driver = webdriver.Firefox(firefox_driver_path, options=opts)
driver.get(url)
...
WebDriverWait(driver, 2).until(EC.element_to_be_clickable((...)).click()
...
driver.quit()
Are those temp files keep generating because of the click in the middle?
My environment:
OS: Ubuntu 18.04
Browser: Firefox
Browser version: 80
Browser Driver version: GeckoDriver 0.29
selenium package 3.8.0
Could anyone tell me how to stop generating those temp files in root/tmp, or how to redirect the location where those temporary file are generated to another folder? I'll consider the later one as a secondary but acceptable solution, because I can delete all content in the new tmp folder whenever I want. Thanks in advance!
Related
I built a web scraper with Selenium using Geckodriver for Linux64. The program clicks on a download link for an Excel file and downloads this file (automatically, without asking to save). This works fine on my 64-bit Ubuntu laptop.
My goal is to run this program on my Synology NAS server:
>uname -a
Linux name_of_nas 4.4.59+ #25556 SMP Thu Mar 4 17:52:53 CST 2021 aarch64 GNU/Linux synology_rtd1296_ds218play
The problem I encountered is that Geckodriver acts as a link between Selenium and the Firefox webbrowser. This would mean that I have to have the Firefox webbrowser installed on my NAS. I don't see how this can be achieved. The Geckodriver I installed on my NAS can be found here. This Geckodriver seems to be working, because the error I get when running my program is:
RuntimeError: Could not find firefox in your system PATH. Please specify the firefox binary location or install firefox
Other versions of the Geckodriver would give me the error:
OSError: [Errno 8] Exec format error: 'geckodriver'
Is there a way to download Firefox on my NAS to get this working? Or perhaps there is another way to achieve my goal?
Code of my test web scraper:
url = 'https://file-examples.com/index.php/sample-documents-download/sample-xls-download/'
from selenium import webdriver
from selenium.webdriver.firefox.options import Options
from selenium.webdriver.firefox.firefox_profile import FirefoxProfile
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
options = Options()
options.add_argument('--headless')
profile = FirefoxProfile()
profile.set_preference("browser.download.dir", "/volume1/NAS storage/Jens/NAS_programs/qteam_scraper/data/")
profile.set_preference("browser.download.folderList", 2)
profile.set_preference("browser.helperApps.alwaysAsk.force", False)
profile.set_preference("browser.download.manager.showWhenStarting", False)
profile.set_preference("browser.helperApps.neverAsk.saveToDisk", "application/vnd.ms-excel")
cap = DesiredCapabilities().FIREFOX
cap["marionette"] = False
# Problem: geckodriver is a link between selenium and your firefox browser -> I need to have firefox installed on NAS server
driver = webdriver.Firefox(firefox_profile=profile, options=options, capabilities=cap) # Automatically uses geckodriver in /usr/local/bin/
driver.get(url)
driver.find_element_by_xpath('/html/body/div[1]/main/section/div/div[2]/div/div/table/tbody/tr[1]/td[5]/a[1]').click()
driver.quit()
I have just installed Selenium Runner on my Mac (Mojave) but I will also later be running it on CentOS 7. Is there any way I can tell my process the path of the chromedriver when I run it from a command line? Currently, I'm running it like so ...
/usr/local/bin/selenium-side-runner /tmp/0ba4e59f-53d5-43ff-b7ff-127499868cf3.side
but getting this error
● Test suite failed to run
The ChromeDriver could not be found on the current PATH. Please download the latest version of the ChromeDriver from http://chromedriver.storage.googleapis.com/index.html and ensure it can be found on your PATH.
After reading the insturxctions here -- https://www.seleniumhq.org/selenium-ide/docs/en/introduction/command-line-runner/ , there is much talk about adding the path to the driver in the PATH environment variable. However, I'm running my script from a separate process that' deson't have access to my PATH, so I'd like a bit more control over how to tell Selnium Runner where the driver is.
Edit: Using Tarun's solution I get the following strange result ...
localhost:selenium davea$ selenium-side-runner -c "chromeOptions.binary='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'" myTestFile.side
info: Running myTestFile.side
FAIL ./DefaultSuite.test.js
● Test suite failed to run
TypeError: Target browser must be a string, but is <undefined>; did you forget to call forBrowser()?
Edit 2:
Contents of my ".side.yml" file and path to chromedriver ...
localhost:selenium davea$ cat .side.yml
capabilities:
browserName: 'chrome'
chromeOptions:
binary: '/Users/davea/Documents/workspace/starter_project/selenium/chromedriver_mac'
firefoxOptions:
binary: '/Users/davea/Documents/workspace/starter_project/selenium/geckodriver_mac'
localhost:selenium davea$ ls -al /Users/davea/Documents/workspace/starter_project/selenium/chromedriver_mac
-rwxr-xr-x 1 davea staff 14994520 Jun 11 19:42 /Users/davea/Documents/workspace/starter_project/selenium/chromedriver_mac
Yet I still get the same result complaining about driver can't be found ...
localhost:selenium davea$ selenium-side-runner myTestSpike.side
info: Running myTestSpike.side
FAIL ./DefaultSuite.test.js
● Test suite failed to run
The ChromeDriver could not be found on the current PATH. Please download the latest version of the ChromeDriver from http://chromedriver.storage.googleapis.com/index.html and ensure it can be found on your PATH.
If you see their readme they do specify how to do the same
https://www.npmjs.com/package/selenium-side-runner
selenium-side-runner -c "chromeOptions.binary='/Applications/Google Chrome.app/Contents/MacOS/Google Chrome'"
You can even create a .side.yml and save the options there
capabilities:
browserName: 'chrome'
chromeOptions:
binary: '/path/to/chromedriver'
See my updates in the another question related to this
How do I run my selenium-side-runner to execute my test against Firefox?
You can try ChromeOptions class. You can create an instance of ChromeOptions, which has convenient methods for setting ChromeDriver-specific capabilities.
// Create ChromeOptions instance
ChromeOptions options = new ChromeOptions();
// Set your custom path of the chrome driver to the options
options.setBinary("/path/to/chrome/binary");
// Pass the options object to the ChromeDriver instance
ChromeDriver driver = new ChromeDriver(options);
Since Selenium version 3.6.0, the ChromeOptions class in Java also implements the Capabilities interface, allowing you to specify other WebDriver capabilities not specific to ChromeDriver.
ChromeOptions options = new ChromeOptions();
// Add the WebDriver proxy capability.
Proxy proxy = new Proxy();
proxy.setHttpProxy("myhttpproxy:3337");
options.setCapability("proxy", proxy);
// Add a ChromeDriver-specific capability.
options.addExtensions(new File("/path/to/extension.crx"));
ChromeDriver driver = new ChromeDriver(options);
Please go through this link for much more options Capabilities & ChromeOptions
Refer this link Command line runner.
Chrome specific capabilities
If you have Chrome installed in a non-standard location on your machine you can specify the path so ChromeDriver knows where to look.
selenium-side-runner -c "chromeOptions.binary='/path/to/non-standard/Chrome/install'"
With Chrome specific capabilities you can also run the tests headlessly.
selenium-side-runner -c "chromeOptions.args=[disable-infobars, headless]"
If you are running in a shell you can set an env variable which will be passed into the process your run. So just do this:
env "webdriver.chrome.driver=/path/to/chromedriver" /usr/local/bin/selenium-side-runner /tmp/0ba4e59f-53d5-43ff-b7ff-127499868cf3.side
I have the following setup:
A Selenium server hub running at "http://localhost:hubPortNum" (a service with the Jar file selenium-server-standalone-3.141.5.jar with parameter -role hub).
A Selenium Node at running "http://localhost:nodePortNum' (the service with Jar file with parameters: -Dwebdriver.chrome.driver=ChromeWebdriverPath -role node -port :nodePortNum).
I checked the URL for the hub and node instances to be sure they are working.
Whenever I try to create Remote Webdriver via Python script:
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
desiredCapabilities = DesiredCapabilities.CHROME.copy()
chromeOptionsRemote = webdriver.ChromeOptions()
chromeOptionsRemote.add_argument("--start-maximized")
chromeOptionsRemote.add_argument("--disable-session-crashed-bubble")
initRemoteDriver = webdriver.Remote(options=chromeOptionsRemote, command_executor='http://127.0.0.1:<nodePortNum>/wd/hub', desired_capabilities=desiredCapabilities)
print(initRemoteDriver.current_url)
The last line does print the current URL(which is "data:,"), that means Webdriver is created.
But the browser does not open on my local machine, that is it is running in the background and I don't know how to make it visible although it has worked in the past.
The troubleshooting steps I have made:
Reinstall latest selenium python package.
Re-Download latest Selenium server jar file.
Updating chrome.
adding chromeOptionsRemote.add_argument("--no-sandbox")
Making sure local Webdriver does open:
That is the line:
self.localDriver = webdriver.Chrome(options=chromeOptionsLocal,
desired_capabilities=desiredCapabilities)
does open the browser locally(the Chromedriver is in the path).
After I made these troubleshooting steps, I have tried the same configuration on a remote server and got the same result(browser not visible), so I think this is probably by design.
what configuration should I create for the browser to be visible?
Any help would be appreciated.
I was running the jar file by Always-Up: https://www.coretechnologies.com/products/AlwaysUp/
The problem was related to session 0 isolation: https://stackoverflow.com/a/26752251/2710840
in order to not run the application under session 0, I have enabled the Autologon feature:
defined the user under the application run as my user:
and executed the application from the context menu with the option to: "restart in this session"
I have some selenium tests written in java and built using maven. The pom file includes the jbehave and selenium libraries. It uses firefox browser. So I installed firefox on linux and included the path in my .bashrc. If I do a mvn clean install manually , it works fine.
But if I try the same thing using Jenkins ( am trying to automate the tests), it is throwing errors:
[ERROR] FATAL ERROR
[INFO] ------------------------------------------------------------------------
[INFO] null
Cannot find firefox binary in PATH. Make sure firefox is installed. OS appears to be: LINUX
Caused by: org.openqa.selenium.WebDriverException: Cannot find firefox binary in PATH. Make sure firefox is installed. OS appears to be: LINUX
I tried doing an echo of PATH within Jenkins and it did not show the path of firefox. So I also did an export PATH with the new path of firefox included and then called mvn clean install in Jenkins. Even though the PATH now showed the path of firefox in jenkins console output, it still throws the same error.
What is that I am missing?
Three things to check:
1) as #shawnzhu said, check whether you have installed firefox properly?
2) To provide Binary path to your driver, you need to set system property. Use below code for selenium with java:
File firefoxPathBinary = new File("path/to/your//firefox-bin");
System.setProperty("webdriver.firefox.bin", firefoxPathBinary.getAbsolutePath());
driver = new FirefoxDriver();
3) For Ubuntu, path to your firefox executable could be - usr/lib/firefox/firefox-bin and for Mac it could be /Applications/Firefox.app/Contents/MacOS/firefox-bin
You probably need to link the executable:
sudo unlink /usr/bin/firefoxsudo ln -s /path/to/new/firefox/executable /usr/bin/firefox
Here's simple batch file I wrote to start the node for selenium grid
set webdriver.ie.driver=C:\selenium-server\IEDriverServer.exe
echo %webdriver.ie.driver%
set webdriver.chrome.driver=C:\selenium-server\chromedriver.exe
echo %webdriver.chrome.driver%
java -jar selenium-server-standalone-2.32.0.jar -role hub
Yes both drivers exist in that directory and I've even added that directory to my System's Path variable. When I try to create a remote web driver like thus:
Platform platform = Platform.WINDOWS;
desiredCapabilities =new DesiredCapabilities("internet explorer", "9.0", platform);
driver = new RemoteWebDriver(new URL(gridUrl), desiredCapabilities);
I'm still getting an exception stating:
Exception: The path to the driver executable must be set by the webdriver.ie.driver system property; for more information, see htt
/p/selenium/wiki/InternetExplorerDriver. The latest version can be downloaded from http://code.google.com/p/selenium/downloads/list
I'm using the latest IEDriver and chromedriver and selenium server 2.32.0 (which I believe is the latest one too).
Running on windows 7. I've tried both the 32 and 64 bit drivers for IE. Get the same problem with the chrome driver.
Adding this option at the end of the command to start my node got everything working. Is this just poorly documented? did I miss something obvious? or is there still something weird going on?
-Dwebdriver.ie.driver=C:\selenium-server\IEDriverServer.exe