selenium-webdriver script isn't finding local file - selenium

I am unable to get a nodejs script loading Chrome to load a local file.
It works on 18.04 but not 22.04.
Has there been some significant change that would affect local file loading syntax, or is there something wrong in my code?
const { Builder } = require('selenium-webdriver')
async function start() {
const chrome = require('selenium-webdriver/chrome')
const options = new chrome.Options()
options.addArguments('--disable-dev-shm-usage')
options.addArguments('--no-sandbox')
const driver = new Builder()
.forBrowser('chrome')
.setChromeOptions(options)
.build()
await driver.get('file://' + __dirname + '/myfile.html')
await driver.sleep(10000)
const text = await driver.executeScript('return document.documentElement.innerText')
console.log(text)
driver.quit()
}
start()
The result is:
Your file couldn’t be accessed
It may have been moved, edited or deleted.
ERR_FILE_NOT_FOUND
I can confirm that myfile.html is definitely present.
Using console.log to show the file argument value shows it is the same for both older and newer Ubuntu.
Changing the driver.get argument to a website, e.g. https://www.google.com/ correctly shows the webpage content in the output.
The local file code fails as above using:
Ubuntu 22.04.1 LTS
Node v16.15.0
chromium-browser Chromium 108.0.5359.71 snap
It works fine on:
Ubuntu 18.04.6 LTS
Node v16.15.0
Chromium 107.0.5304.87 Built on Ubuntu , running on Ubuntu 18.04

This seems to be because the default Ubuntu 22.04 Chrome package is a snap package that limits local file access to /home/ .
Switching to the .deb distribution from Google solves the issue. The Chromedriver also needs to match.
# Chrome browser - .deb package, not the standard OS Snap which limits access to only /home/
# See: https://askubuntu.com/questions/1184357/why-cant-chromium-suddenly-access-any-partition-except-for-home
# See: https://www.ubuntuupdates.org/ppa/google_chrome?dist=stable
wget -q -O - https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add -
sh -c 'echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" >> /etc/apt/sources.list.d/google.list'
apt-get update
apt-get install google-chrome-stable
# Chromedriver, matching version number
# See: https://skolo.online/documents/webscrapping/#pre-requisites
# See step 3 at: https://tecadmin.net/setup-selenium-chromedriver-on-ubuntu/
# See: https://sites.google.com/chromium.org/driver/
#!# NB This will become out of sync if apt-get updates google-chrome-stable
google-chrome --version
wget https://chromedriver.storage.googleapis.com/108.0.5359.71/chromedriver_linux64.zip
unzip chromedriver_linux64.zip
sudo mv chromedriver /usr/bin/chromedriver
sudo chown root:root /usr/bin/chromedriver
sudo chmod +x /usr/bin/chromedriver
rm chromedriver_linux64.zip

Related

Chrome driver version 92 not running tests

I am having issues to run the version 92 with the browser open, in headless mode is working fine.
I am currently running my tests on a docker container. The installation of the chrome-driver follows:
RUN curl -s https://dl.google.com/linux/linux_signing_key.pub | apt-key add - \
&& echo "deb [arch=amd64] http://dl.google.com/linux/chrome/deb/ stable main" > /etc/apt/sources.list.d/google-chrome.list && \
apt-get update && \
apt-get install -y \
xvfb \
google-chrome-stable=92.\* \
unzip
RUN curl -s -o /tmp/chromedriver.zip "https://chromedriver.storage.googleapis.com/$(curl -s https://chromedriver.storage.googleapis.com/LATEST_RELEASE_92)/chromedriver_linux64.zip" && \
unzip /tmp/chromedriver.zip chromedriver -d /usr/bin/ && \
chmod +x /usr/bin/chromedriver
after executing my command to run the tests, the browser open but nothing happens.
I get stuck on data; on the address bar, and page keeps loading forever.
I'm currently using the chrome options:
chrome_options = webdriver.ChromeOptions()
chrome_options.add_argument("--no-sandbox")
chrome_options.add_argument("--incognito")
chrome_options.add_argument("--disable-dev-shm-usage")
Looks like it was related to an old Nvidia driver running on Linux:
You may need to run with --disable-gpu on Linux with NVIDIA driver older than 295.20
Source: chromium documentation
So, adding chrome_options.add_argument("--disable-gpu") solved my problem.

Jenkins: ChromeDriver update PATH from shell script and use new version

I am looking for a way to update the PATH in Jenkins for running Selenium tests with PyTest.
I need to run the latest version of chromedriver but due to an infrastructure deficiency (our base image is running Debian and the latest available version is 73 and I need to be running at least 83).
There is already a version of chromedriver installed on the image at /usr/bin and I need to be able to point to a different version
The jenkins chromedriver plugin appears that it will just use the latest available version for debian, which doesn't help me at all.
Until I have time to address the systemic issue, I'd like to just install chromedriver and update PATH - because Selenium requires chromedriver on the PATH
It seemed, for ease of use, that https://pypi.org/project/chromedriver-binary/ was a good solution - it installs just fine and the shell script chromedriver-path echoes the location, so I could just update path as the documentation shows: PATH=$PATH:chromedriver-path
This doesn't seem to jive in Jenkins - PATH is not updated
stages {
stage('build'){
steps {
withCredentials([...]) {
sh """
alias python=python3.8
python -m venv --system-site-packages venv # only for jenkins
python -u setup.py
. venv/bin/activate
which chromedriver #/usr/bin/chromedriver
chromedriver-path #path/to/python/lib/python3.8/site-packages/chromedriver_binary
export PATH=$PATH:`chromedriver-path`
which chromedriver #/usr/bin/chromedriver
"""
sh "python -m pytest"
}
}
}
}
I Have looked at the withEnv() option and environment{} step but I'm not sure how to access that binary and update PATH once chromedriver-binary has been set - because it appears that environment{} would not have access to shell scripts that are installed in the individual steps.
Any tips would be greatly appreciated
The issue may actually be in Jenkinsfile declaration.
Try using sh with single quotes '''. Also binaries are searched in directories defined in PATH from left to right, so to override system PATH, you must put your directory on the beginning, not on the end.
If I alter your code snippet:
stages {
stage('build'){
steps {
withCredentials([...]) {
sh '''
alias python=python3.8
python -m venv --system-site-packages venv # only for jenkins
python -u setup.py
. venv/bin/activate
which chromedriver #/usr/bin/chromedriver
chromedriver-path #path/to/python/lib/python3.8/site-packages/chromedriver_binary
export PATH=$(chromedriver-path):$PATH
echo $PATH # just to check the output, your path should be on the beginning
which chromedriver # this should now find the proper chromedriver
'''
sh "python -m pytest"
}
}
}
}

Why can I access my Apache default page ONLY when I go in my container's bash?

First of all, I would like to say that I'm new to Docker and all that is around it.
I have been wanting to make a container where I have Apache, php and Firebird installed. So far, so good ; everything seems to work and I can get my default page when I type in my Internet search bar my ip address and :8080. I do so by first starting my container like this :
docker run -p 8080:80 -d apps
Where "apps" is the name of my container.
I have achieved this with my Dockerfile, which looks like this (it might be a bit messy, still learning the good practices !) :
# Download of base image - ubuntu 20.04
FROM ubuntu:20.04
# Updating/upgrading
RUN apt-get update -y && apt-get upgrade -y
# Installing apache2, php and firebird with modules
RUN DEBIAN_FRONTEND="noninteractive" apt-get install apache2 php libapache2-mod-php -y && \
apt-get install php-curl php-gd php-intl php-json php-mbstring php-xml php-zip -y && \
DEBIAN_FRONTEND="noninteractive" apt-get install firebird3.0-server -y && apt-get install firebird->
# Start up apache in foreground by default
CMD /usr/sbin/apache2 -D FOREGROUND
ENTRYPOINT service apache2 restart && /bin/bash
# Expose apache
EXPOSE 80
Now, my idea was to export this container to another computer and try the same thing. I have followed a few tutorials and got to import my container on the new machine. My problem here is that somehow, the command I previously used doesn't work ; it shows me this error :
docker: Error response from daemon: No command specified.
See 'docker run --help'.
Which is odd, because it works just fine on the other machine. I also did this command, WHICH WORKS :
docker run -i -t -p 8080:80 apps /bin/bash
This one works alright, but I don't want to have to access the bash everytime I want my Apache page to load. I would want my container to run without me having to get in my container, if that makes sense.
In my opinion, it probably comes from the fact that I only loaded the container, and not the image used to build it (maybe a bad practice? Couldn't find anything about it on google).
Here is my setup just in case ---
On the first machine (which is the one where I created the image and the container) :
Ubuntu 20.04 LTS
Apache/2.4.41
Docker 19.03.8
On the other machine which I'm trying to make my container work :
Ubuntu 18.04 LTS
Apache/2.4.29
Docker 19.03.6
Thank you for your patience and time !
apps is your docker image, if you want to give name for your container you can specify --name in the run command ie,
docker run --name container_name -p 8080:80 -d apps
You can use sudo docker save -o apps.tar apps to create a tar file of the image
then change the root permission of the tar file sudo chmod 777 apps.tar
Copy this tar file to the other system you want to try, then
sudo docker load --input apps.tar
This will load the image, then you can use the previous command to start the container
docker run -p 8080:80 -d apps
Where "apps" is the name of my container. <- This statement is incorrect and perhaps the misunderstood concept that leads you to the problem.
apps is the name of the image, not the name of the container. On the host on which you can run the container, you must have built that image from the Dockerfile that you shared using the command:
docker build -t apps .
Copy the Dockerfile on the host where you cannot run the container, built the image in-there as well and try again running the container.

Unable to open X display when trying to run google-chrome on Centos (Rhel 7.5)

I need to run Google Chrome remotely on a virtual machine using SSH. I do not want xforwarding - I want to utilize the GPU available on the vm. When I try running google-chrome I get following error:
[19615:19615:0219/152933.751028:ERROR:browser_main_loop.cc(1512)] Unable to open X display.
I've tried to setting my DISPLAY env value to various values:
export DISPLAY=localhost:0.0
export DISPLAY=127.0.0.1:0.0
export DISPLAY=:0.0
I've also tried replacing 0.0 in abowe examples with different values.
I have ForwardX11 no in /etc/ssh/sshd_config
I tried setting up target like this:
systemctl isolate multi-user.target
When I try to run sudo lshw -C display i get folowing output:
*-display
description: VGA compatible controller
product: Hyper-V virtual VGA
vendor: Microsoft Corporation
physical id: 8
bus info: pci#0000:00:08.0
version: 00
width: 32 bits
clock: 33MHz
capabilities: vga_controller bus_master rom
configuration: driver=hyperv_fb latency=0
resources: irq:11 memory:f8000000-fbffffff
*-display UNCLAIMED
description: VGA compatible controller
product: GM204GL [Tesla M60]
vendor: NVIDIA Corporation
physical id: 1
version: a1
width: 64 bits
clock: 33MHz
capabilities: pm msi pciexpress vga_controller bus_master cap_list
configuration: latency=0
resources: iomemory:f0-ef iomemory:f0-ef memory:41000000-41ffffff memory:fe0000000-fefffffff memory:ff0000000-ff1ffffff
I've tried to update my gpu drivers by:
wget https://www.nvidia.com/content/DriverDownload-March2009/confirmation.php?url=/tesla/375.66/nvidia-diag-driver-local-repo-rhel7-375.66-1.x86_64.rpm
yum -y install nvidia-diag-driver-local-repo-rhel7-375.66-1.x86_64.rpm
But after that I still see UNCLIMED next to my NVIDIA gpu.
Aby ideas?
You can try with Xvfb. it does not require additional hardware.
Install Xvfb if you didn't install it yet and do the following steps.
sudo apt-get install -y xvfb
Dependencies to make "headless" chrome/selenium work:
sudo apt-get -y install xorg xvfb gtk2-engines-pixbuf
sudo apt-get -y install dbus-x11 xfonts-base xfonts-100dpi xfonts-75dpi xfonts-cyrillic xfonts-scalable
Optional but nifty: For capturing screenshots of Xvfb display:
sudo apt-get -y install imagemagick x11-apps
Make sure that Xvfb starts every time the box/vm is booted:
Xvfb -ac :99 -screen 0 1280x1024x16 &
export DISPLAY=:99
Run Google Chrome
google-chrome
Okay guys. I found my problem after 2 hours of going crazy. My box was configured correctly. What you can NOT do, is ssh from one box, to another box, to this box and expect X11 forwarding to play nicely. Without tearing apart the entire network, I found that if I shelled over from the MAIN box to this box ( no double or triple ssh'ing) chrome comes right up as a regular user using CLI. So it was a matter of multiple shells from multiple boxes that made the display say it was set to NOTHING! Setting the display manually only complicates the problems. Once I shelled directly over to this box from the main outside box, my display was set to 10:0, which is first instance in my configuration. Don't make this mistake, you will waste valuable time.
FWIW, I ran into this when using SSH to log into a Selenium chrome node in a Docker compose stack. Chrome would launch if I invoked it as root with sudo -u seluser google-chrome, but not if I logged in as seluser. The trick turned out to be that root had DISPLAY set to :99:0, and seluser didn't have it set at all. If I set it explicitly (either from a seluser shell or from the docker compose exec command line) it worked.
$ docker-compose exec -u seluser \
selenium-chrome \ # or whatever your service is called
/bin/bash
seluser#c02cda62b751:/$ export DISPLAY=:99:0
seluser#c02cda62b751:/$ google-chrome http://app.test:3000/home
or
$ docker-compose exec -u seluser -e DISPLAY=:99:0 \
selenium-chrome \
google-chrome http://app.test:3000/home
That :99.0 is undocumented, though, so if this isn't working, you might try checking root's DISPLAY value with:
docker-compose exec -u root selenium-chrome bash -c 'echo "${DISPLAY}"'
I faced the same issue with WSL and Ubuntu. I have unininstalled/Reset the ubuntu. After that, I have executed the below command
wsl --set-default-version 2
then I installed again the Ubuntu, I didn't get --no-sandbox issue or any issue.
Hope this will use for someone.

How to implement chromedriver in selenium in Linux platform

Can somebody tell me how to use the Chrome driver in Selenium for Linux platform?
I have my chrome driver location at username/home/chromedriver.
My code is:
System.setProperty("webdriver.chrome.driver", "/home/username/ChromeDriver/chrome‌​driver");
driver = new ChromeDriver();
driver.get("facebook.com");
The error I am getting is:
org.openqa.selenium.WebDriverException: Unable to either launch or
connect to Chrome. Please check that ChromeDriver is up-to-date.
Using Chrome binary at: /opt/google/chrome/google-chrome
(WARNING: The server did not provide any stacktrace information)
From [the official documentation](https://github.com/SeleniumHQ/selenium/wiki/ChromeDriver:
Requirements
The ChromeDriver controls the browser using Chrome's automation proxy
framework.
The server expects you to have Chrome installed in the default
location for each system:
OS Expected Location of Chrome
-------------------------------------
Linux /usr/bin/google-chrome
Mac /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome
Windows XP %HOMEPATH%\Local Settings\Application Data\Google\Chrome\Application\chrome.exe
Windows Vista C:\Users\%USERNAME%\AppData\Local\Google\Chrome\Application\chrome.exe
For Linux systems, the ChromeDriver expects /usr/bin/google-chrome to be a symlink to the actual Chrome binary. See also the section on overriding the Chrome binary location.
Getting Started
To get set up, first
download the
appropriate prebuilt server. Make sure the server can be located on
your PATH or specify its location via the webdriver.chrome.driver
system property. Finally, all you need to do is create a new
ChromeDriver instance:
WebDriver driver = new ChromeDriver();
driver.get("http://www.google.com");
Therefore, download the version of chromedriver you need, unzip it somewhere on your PATH (or specify the path to it via a system property), then run the driver.
We have installed Successfully
sudo apt-get install unzip
wget -N http://chromedriver.storage.googleapis.com/2.10/chromedriver_linux64.zip -P ~/Downloads
unzip ~/Downloads/chromedriver_linux64.zip -d ~/Downloads
chmod +x ~/Downloads/chromedriver
sudo mv -f ~/Downloads/chromedriver /usr/local/share/chromedriver
Change the directory to /usr/bin/chromedriver
sudo ln -s /usr/local/share/chromedriver /usr/local/bin/chromedriver
sudo ln -s /usr/local/share/chromedriver /usr/bin/chromedriver
Now run the script and add the following in the environment file.
Capybara.register_driver :chrome do |app| client = Selenium::WebDriver::Remote::Http::Default.new Capybara::Selenium::Driver.new(app, :browser => :chrome, :http_client => client) end
Capybara.javascript_driver = :chrome
Note : Change the chrome driver version according to your operating system type like 32 bit or 64 bit.
Here is a complete script for Linux 18.04 to install Google Chrome and the chrome driver. It should automatically adjust to collect the correct chrome driver for the browser.
#!/usr/bin/env bash
# install the latest version of Chrome and the Chrome Driver
apt-get update && apt-get install -y libnss3-dev
version=$(curl http://chromedriver.storage.googleapis.com/LATEST_RELEASE)
wget -N http://chromedriver.storage.googleapis.com/${version}/chromedriver_linux64.zip
unzip chromedriver_linux64.zip -d /usr/local/bin
chmod +x /usr/local/bin/chromedriver
wget https://dl.google.com/linux/direct/google-chrome-stable_current_amd64.deb
dpkg -i google-chrome-stable_current_amd64.deb; apt-get -fy install
For me worked with these commands:
Unziped the file -> unzip -q chromedriver_linux64.zip
Force the copy to the directory 'usr/bin' -> sudo mv -f chromedriver /usr/bin
The selenium code was something like that.
System.setProperty("webdriver.chrome.driver","/usr/bin/chromedriver");
WebDriver driver = new ChromeDriver();
driver.get("https://mvnrepository.com");
driver.close();
You can see small example from this example
For linux, i downlaod chrome driver and keep as system path variable(or put in exist path folder). And from code i use following ways (add property and initiate with path of chrome driver)
System.setProperty("webdriver.chrome.driver", "/usr/local/bin/chromedriver");
ChromeDriverService service = new ChromeDriverService.Builder()
.usingDriverExecutable(new File("/usr/local/bin/chromedriver"))
.usingAnyFreePort()
.build();
try {
service.start();
} catch (IOException e) {
e.printStackTrace();
}
return new RemoteWebDriver(service.getUrl(), DesiredCapabilities.chrome());