Unable to run tests against Safari 11 remotely (ssh, CI) - ssh

I'm having troubles running Ruby/RSpec tests against Safari 11 when I'm trying to run tests via ssh manually or via Jenkins (where machine where Safari exists is remote slave).
When executing tests, I'm getting following error:
Selenium::WebDriver::Error::WebDriverError:
unable to connect to safaridriver 127.0.0.1:7050
What is weird is that I'm able to run tests when I'm logged to the Mac machine directly. This leads me to the conclusion that there could be some permission which, by default, disables execution from ssh session but not sure why?
Also, to my knowledge, Safari Driver is part of Safari 10+ and as such is not installed anymore as extension
Update: I've found out that safaridriver executable that should be spawned by tests cannot be spawned from some reason when I login via ssh.
Example:
/usr/bin/safaridriver -p 7050
It will just terminate with non-zero exit code while running same command directly on machine will run safaridriver in foreground. Since I expect Jenkins to be running test job on this machine, my idea to overcome this issue would be to launch safaridriver on this machine (by cron or launchctl) and then use this instance to connect to it with my tests. However, so far, I was not able to make my tests (Selenium/Capybara) re-use existing safaridriver instead of always trying to spawn new one on different port. Any idea on this would also be greatly appreciated.
This is my environment:
OS: MacOS Sierra 10.12.6
Browser: Safari 11.0.2
Thanks in advance

There is a possible workaround to do this. First you need to create a Automator Workflow or Apple Script that launches the SafariDriver
Then save this as a application. Let assume we name it SafariDriver7050.
Then from the SSH session you need to execute
open /Applications/SafariDriver7050.app
This will actually launch SafariDriver in the logged in session and it should work for you.
The caveat being shutting it down, you will need to first kill the SafariDriver7050 app and then you need to kill the safaridriver process. The order matters, else it will create a error dialog on UI
Edit-1:
As you suggested, it would be even easier to do this, when you wrap your test as a app and then it will be automatically be able to launch SafariDriver without any issues. The key to issue is using open command in a SSH session
Edit-2
Why does SafariDriver not work in SSH? Well if you look at the linux counterpart
In case of linux we can use the DISPLAY environment variable to launch an app in an existing display or we can use something like XVFB to launch the browser in a virtual display. That is the concept that most frameworks use in case of linux machines.
But Mac doesn't have such kind of feature, which is why this workaround is needed. Now why it doesn't have that, I am not sure. There may be some other workaround that I may not be aware of, so anyone who has valuable info, can help improve this part of the answer

For my case:
On OSX host side was created Automator app as described above and modified a little bit:
security unlock-keychain -p your_host_password /Users/$USER/Library/Keychains/login.keychain-db
safaridriver --enable
safaridriver -p 7050
On CCI/Jenkins side:
open /Applications/StartSafariDriver7050.app/
run pytest cmd
osascript -e 'quit app "StartSafariDriver7050"'
pkill safaridriver
In Python:
def safaridriver():
return webdriver.Safari(desired_capabilities=Caps.SAFARI, port=7050)

Related

Selenium works when started locally, but not remotely

I'm hoping this is a simple issue to solve:
I have a dedicated machine that runs selenium with chromedriver. I can physically walk to the machine and type the following in a local terminal:
java -Dwebdriver.chrome.driver=chromedriver238 -jar selenium-server-3.11.jar
I can then go to any other machine on the network and kick off testing (in my case, codeception on jenkins) and it will sucessfully run my tests in chrome without a hitch.
However, if I try to run the above command without physically being at the selenium machine (ex, trying to ssh into the machine to run the command or making this command run as part of a supervisor process) then selenium has trouble starting the instance of chrome.
Has anyone run into this scenario before? Is there a way I can give 'interactive' access to the background script to allow it to open stuff on my screen?
Because when you physically walk to the machine and run the command. the selenium can get the desktop/window interface.
When use SSH, selenium lost the desktop/window interface, selenium have to start chrome under headless model, but your code not config to run as headless model. So the conflict come out.
you need to config your code to run with headless model.
I'm to getting the same scenario, when i try to start the selenium code on the machine A it works yet when i try to start selenium and run the scripts from machine B
(remotely connecting to machine A using ssh [ssh test#machine-a]) chrome instance doesn't start/launch and driver is getting null.
logs :
2020-02-27 12:04:27,319 INFO [LogWritter] Exception in beforeclass chrome driver instantiation , driver is null
2020-02-27 12:04:27,493 INFO [LogWritter] Exception in getting screenshot---java.awt.AWTException: headless environment
Any help/suggestion would be great !

Timeout when running xcodebuild tests under Xcode 6 via SSH

I seem to be having issues with integrating Xcode6 with jenkins, I currently have this setup and working with Xcode 5.
With xcode 6 running remotely via SSH the simulator time-out, when I run locally it succeeds.
Command
xcodebuild -workspace PROJECTNAME.xcworkspace -scheme BGO_Tests -destination 'platform=iOS Simulator,name=iPhone 5s' -derivedDataPath ./Build clean test
2014-08-19 10:46:36.591 xcodebuild[33966:381f] iPhoneSimulator: Timed out waiting 120 seconds for >simulator to boot, current state is 1.
Testing failed:
Test target BGO_Tests encountered an error (Timed out waiting 120 seconds for simulator to boot, current state is 1
Tested with recent Xcode 6 beta 6
Note: the device names changed in Xcode 7, so you no longer specify them using iPhone 5 (9.1 Simulator) but rather iPhone 5 (9.1).
Use xcrun instruments -s to get the current list of devices and then you can pre-launch it using:
xcrun instruments -w "iPhone 5 (9.1)" || echo "(Pre)Launched the simulator."
Prelaunching
I got to a point where what I proposed down there wasn't working anymore. In addition to making the changes mentioned here, you need to launch the simulator xcodebuild is expecting BEFORE xcodebuild is ran:
# First get the UDID you need
xcrun instruments -s
# Then launch it
open -a "iOS Simulator" --args -CurrentDeviceUDID <sim device UDID>
# and wait some time....
sleep 5
# Then launch your unit tests
xcodebuild [...] -destination 'platform=iOS Simulator,name=<device name matching the UDID>'
Old post
This bug is fixed in Xcode 6.3 and above. If you are experiencing similar problems in newer Xcode, it's likely another bug.
Apple follow up regarding Bug ID# 18001199:
The context provided by LaunchDaemons is not supported for running GUI
applications. The SSH service, and the default setup for Jenkins, are
both implemented as LaunchDaemons. In earlier versions of Xcode 5
xcodebuild could run tests on the iOS simulator in this context, but
that was never a supported configuration, and as you have noted that
is no longer working as of Xcode 6.
Unlike LaunchDaemons, LaunchAgents provide a context where you can run
GUI applications - if the user is logged in at the time, with a window
server / Aqua session. Converting your Jenkins configuration from
being a LaunchDaemon to being a LaunchAgent would avoid the reported
issue. You can also use launchd for running tests on the iOS simulator
from a SSH session, either by crafting a LaunchAgent and manually
loading / starting that, or by using "launchctl submit”.
Ok, after some more digging around the comments around here (many thanks to Opal), I found out that launching the slave via JNLP instead works.
As many people mentioned, it is not currently possible to run the unit test over SSH, so you might want to turn towards the JNLP agent for now until Apple fixes it.
If connecting with JNLP still does not solve it, try the solution mentioned in this comment.
i.e.: Run these on command line:
DevToolsSecurity -enable
sudo dscl . -append /Groups/_developer GroupMembership "user-that-runs-the-sim"
security authorizationdb write system.privilege.taskport is-developer
See References here and here.
I've recently found out that if you install a new version of Xcode and do not launch it. The simulator might start timing out again. To solve this, I've had to manually launch Xcode, and install the additional tools it requested.
I ended up solving this on Xcode 5 by doing the steps here, essentially running:
sudo security authorizationdb write system.privilege.taskport allow
This will eliminate one class of these authentication popups. You’ll also need to run:
sudo DevToolsSecurity -enable
However, once I upgraded to Xcode 6, I now get an infinite hang when trying to run xcodebuild tests over SSH. They continue to run just fine as long as I'm logged into the console, and running them from the keyboard.
I ran into the same issue. My working theory is that SSH on OSX is started as a LaunchDaemon, and LaunchDaemons are not allowed to present a UI; Reference.
I was able to work around the issue by using Java Web Start to launch the Jenkins slave. I then installed the Jenkins slave as a launchd service.
Unfortunately the Jenkins slave then installs itself as a -you've guessed it- LaunchDaemon, leading to the exact same problem of not being able to launch the tests; Reference.
I worked around that issue by moving the Jenkins Slave LaunchDaemon plist and jar files in /System/Library/LaunchDaemons into ~/Library/LaunchAgents, and updated the paths inside the plist file.
That finally allowed me to run XCode6 (Beta6) tests on an OSX jenkins slave.
I finally managed to find a good simple solution. JNLP was causing numerous issues with our jenkins server.
Workaround for SSH timeout via https://corner.squareup.com/2015/07/ios-build-infrastructure.html
"Mavericks (10.9) and Yosemite (10.10) determine if a process can access accessibility hooks via the parentage of the accessing process. By putting launchd in the list of allowed processes, processes launched via SSH or Jenkins have access to the accessibility hooks across the system. To do this you can modify the TCC database, per this gist. A reboot is required to make the change take effect."
#!/bin/bash
# This will add lauchd to the list of allowed processes for accessibility access
sudo sqlite3 /Library/Application\ Support/com.apple.TCC/TCC.db "INSERT or REPLACE INTO access VALUES('kTCCServiceAccessibility','/sbin/launchd',1,1,1,NULL)"
# This outputs the rows in the TCC database
sudo sqlite3 /Library/Application\ Support/com.apple.TCC/TCC.db 'select * from access'
echo "Restart is required for these changes to take effect"
Update 8/02/2016
This is now fixed in Xcode 7.2.1 ("Command line tool ‘xcodebuild test’ will no longer time out waiting for Simulator.app to launch")
I've seen this error before, one possibility is that since you probably downloaded the Xcode6 Beta from the internet (not the appstore as its not available yet), the machine you are trying to run it on will show a pop up asking you if you really want to open this app as its from the internet.
The same will happen when xcodebuild tries to launch the iPhone simulator app.
One thing you might want to try is to share screen with the machine and click "Open" in that pop up.
If that still doesn't work, I would try to:
Reset the Content & Settings of the simulator
Reboot the machine and make sure no simulator is running on start up (you can just choose not to re-open any app when restarting)

How to setup webdiver to perform actions in background?

I am currently working in Automation QA. Our project using Java: jbehave, thucydides, selenium. We are working in Ubuntu OS.
Every time I run my tests, I have to stop doing everything(coding, browsing the web), because every click or any action of selenium is taking driver window(we are using chrome driver) in focus(makes window active) and if i do something i mess up the test. Our team is wasting a lot of time because of this.
We have tried Phantomjs, it doesn't work correctly, cannot find elements etc... And also to run tests in virtual machine using vagrant isn't working for us(for company specific reasons).
It is fine if Chrome driver will take over active window when it starts, as long as it would continue performing different actions in background. Strange, but in my previous project(windows, c#, mstest) chromedriver would behave exactly like this without any additional setup.
Please help, I know few other teams having this issue. Seems like this issue only on Mac and linux.
After having this issue for a long time I have finally found a good solution. This issue still remains on ChromeDriver level, so if there are any updates on this side let me know.
The easiest way to run tests in Ubuntu in a background is using VNC Server. Here are the steps you should follow:
Install vnc server from ubuntu terminal:
sudo apt-get install vnc4server
Start server with any number(I use 7). First time doing it enter password for vnc server(Remember it!).
vncserver :7
To start server in full screen add "geometry" to last command with your screen resolution for example:
vncserver :7 -geometry 1920x1080
Download some VNC Viewer. I use Real VNC.
Go to Real vnc viewer, start new connection and enter:
VNC Server: localhost:7
Encryption: Let VNC Server choose
Press connect and enter your vnc server password(the one from step 2).
New window should be opened, open new terminal in it.
Run your test from terminal. I use java maven project so for me it works like this. Navigate to project folder and run: mvn clean install
If you want simply to run test in hidden mode using only terminal(without opening vncviewer) use this command:
cd path/to/project
xterm -display localhost:7 -e mvn clean install
If you want to stop server:
vncserver -kill :7
Hope this will help many of you.
For me (I had the same problem in my Grails project's functional test using Chrome; not with Firefox) works this set up:
ChromeOptions options = new ChromeOptions();
options.addArguments("--start-maximized --disable-popup-blocking");
new ChromeDriver(options);
Regards

Selenium with Xvfb and Jenkins throws TimeoutException

I'm trying to setup automatic Selenium tests with Jenkins on a Linux server.
The problem is that all tests fail with:
org.openqa.selenium.TimeoutException: Timed out after 60 seconds waiting for visibility of element located by By.linkText
Each test fails with the fist element expected.
Jenkins has the Xvfb plugin installed, and from the console messages it seems to work:
Xvfb starting$ /usr/bin/Xvfb :1 -screen 0 1024x768x24
I tried to increase the timeout, but it seems its not that the problem.
The tests run fine on windows and on a linux system with display.
The problem is I don't have direct access to the server, so I'm trying to explore all possibilities before I make requests to the sys admins.
To run the tests are used two projects, both seem to deploy fine, but when the tests start its like one project (the backend) is not deployed and some URLs are not found, from here the timeout exception. I'm not very sure how Jenkins behaves in this case, and if it starts one project at a time.
Any ideas? Thanks a lot!
To check if the application under test is running when tests try to access it, you can connect to the test machine via ssh, and take screenshot at the moment when the test runs:
xwd -root | convert xwd:- capture.png
http://inspirated.com/2007/04/02/howto-use-xwd-for-screenshots

Selenium server cannot be started inside Bamboo

I am trying to configure a Selenium testing through Bamboo. I am able to run Selenium scripts directly from command line, however it always fails if I run it through Bamboo remote agent. The error is:
: Could not start Selenium session: Failed to start new browser session: Error while launching browser
at com.thoughtworks.selenium.DefaultSelenium.start(DefaultSelenium.java:107)
I took some advice from the web, and checked "Allow service to interact with desktop" for Bamboo remote agent service. However, it does not work.
Anyone have other suggestions?
You must run Bamboo in Console Mode for this to work. Also, the build machine desktop must be active - and not covered or minimized if you want mouse simulations to work. I know right!