I am running a Selenium test suite on multiple browsers on a remote Apple Mini. I schedule these tests with Bamboo.
When running the tests on Chrome, they are extremely slow and some actions timeout, causing the tests to fail.
More interestingly, this only happens when Bamboo kicks off the test. When I kick off the tests from my local computer, they run fine.
I have read that Chromedriver performs poorly sometimes when automatically detecting proxy settings. I have addressed this but still the performance is poor.
Has something like this happened to others and does anyone know what could cause something like this?
Something similar happens to me.
The root cause can be in the network connection between the Bamboo server (where the project is built) and the Selenium server (which probably runs on another machine).
When you kick off tests suite from your local machine, you probably have the project build and Selenium running at the same place. So, the commands sent to Selenium do not have to pass the network stage and there are almost no delays.
Try to eliminate the slow connection or investigate if it is possible to build and launch your tests on the machine which runs the Selenium server.
I had this when running tests on Team City with multiple browsers on a remote Mac Mini.
After much head scratching we stumble upon the fact that tests sped up if we remotely connected or connected a monitor to the Mac Mini.
We can only assume that the Mac Mini is smart enough to reduce resource to GUI activities if it knows there is no mechanism to attach a GUI.
Therefore, we purchased a very cheap monitor and left it attached to the MAc Mini, and all started running normally
Related
I have some problem with Jenkins and Selenium.
I have a Jenkins Master and some Build Slaves, installed as Windows Services on Windows Server 2016 machines, in which I perform build jobs. In some of these jobs, I perform test with Selenium and I need to take some screenshots of these tests, but all the screenshots that I got are black.
Reading on the web, I found that many people are facing this kind issue. But I also found this, https://www.coretechnologies.com/blog/windows-services/interact-with-desktop/, in which is reported that it's not allowed for Windows Services to interact with desktop.
So, the question is: it's possible to make interact Jenkins Slave Service with the desktop in order to take some screenshot of it?
I am running an Azure Pipelines (DevOps) pipeline that builds my automation test solution into a dll, and loads that dll to get all of the integration automation tests.
I am using the default cloud "Host" agents to run the automation. However, the drivers throw tons of errors. I recognize the errors as the same ones that I encountered when I set up automation via Jenkins -- when I was running Jenkins as a service.
To fix the issue in Jenkins, I stopped running it as a Windows service, and started launching it as a java -jar Jenkins.war command in Powershell. That allowed the selenium drivers to launch browsers as normal on the VM's desktop, rather than running the browser instances and drivers completely in memory.
So, when it ran the browser windows and drivers completely in memory, I received the same driver errors as I am now encounting on the Host agents in Azure Pipelines. The problem is that I cannot find how to make the Agents run the tests on the VM desktop. I don't have any idea where to start on solving this problem, and can't seem to find any info online about this.
Edit: To be as clear as possible. The ONLY error occurring is "$ is not defined". This is because the DOM does not load properly because the browser executables and driver executables are running completely in memory, headless. I need to be able to run the executables in a rendered part of the host machines. So, what I mean is, if I were to remote desktop into this machine, I would actively see browser windows rendering, and automation running. Right now though, if I were to remote into this VM, I would see chrome.exe, and chromedriver.exe in TaskManager, but nothing would be visible on the desktop.
I found the answer.
https://learn.microsoft.com/en-us/azure/devops/pipelines/test/ui-testing-considerations?view=vsts&tabs=mstest#visible-ui-mode
You must manually configure the hosts to run Selenium non-headless. This will prevent everything from running as a service, which is purely in memory, with no UI components. This is the problem I mentioned that Jenkins encountered when running Jenkins as a service.
The fix for Jenkins running automation with a GUI is the same for the Hosted agents in Azure pipelines, ultimately. It's just a manual configuration. This is done via a plugin and build step in pipelines.
I've been working with Selenium for a few years already.
I started with some little stuff in Java and in my previous job I did a project using C# bindings with SpecFlow framework, page objects model, I dealt with complex locators, some JavascriptExecutor even some browsermobproxy work, bottom line I have some experience with Selenium.
Still there is something that is not clear to me.
Is Selenium expected to work properly when there is no "interactive session" into the machine that is running the code ?
Let's say that I connect into remote desktop to a machine in the cloud (Let's say Amazon or Azure), I develop a script and schedule a windows job to run it on the next 10 minutes, then I disconnect from the remote desktop session, the machine is on but no user is connected to it in remote desktop.
Will the script work ? Or depends what does it do ? Might some of the actions not work (Script might include changing the window size, sending keystrokes both through selenium sendkeys and by OS level actions) ? Can we ensure that any script that we developed and works OK while we are connected to the machine will also work when I'm not ?
I hope the question is clear, if not I can maybe explain further
Thanks !!!
Yes, Selenium can run on the cloud machine even though you are not connected to it. It should work without any issue.
As per my knowledge cloud machines are protected by firewall which blocks almost all the ports. You may need to get permission to use the default selenium ports like 4444 or 5555.
Also, may need to increase wait time because the cloud machines are slow in performance compared to normal physical desktop.
Usually cloud machines are linux based, you need to consider the environment as well.
The issue is that Sikuli's image recognition capabilities only work when the target of the Sikuli tests is in the foreground and it has full access to the mouse. In its current configuration, Jenkins projects are never visible from the desktop of the windows slave node.
The way my current automated testing suite is setup right now, about 30 tests are written using only selenium. In the final two tests, selenium starts a webdriver (currently chromedriver but can be firefox or IE) and navigates to the right pages and Sikuli does its thing from there. Both use the java language bindings.
Currently Sikuli appears to be installed correctly, but when it is asked to click through a list of screenRegions supposedly populated by matching the desktop with a target image, it immediately throws a NPE.
How do I change my Jenkins setup on my windows 7 slave to allow the browser to be launched in the foreground, as it is when i run it from my windows 7 development machine?
So you've got a development machine where Sikuli works, and a test machine where Jenkins runs Sikuli and it doesn't work.
I had this same setup (with the Sikuli tester/Jenkins slave running Windows 7). What worked for me was to have the test machine run a VNC server, log in as a the tester account over VNC, and launch the Jenkins slave as a regular application and not a service. I used UltraVNC server.
As far as I could tell, the problem stems from Windows' security measures, which are designed to prevent a remote user from controlling your machine without your presence.
If you run Jenkins as a service, it doesn't get a real desktop allocated to it. The script will run, but (as you're seeing) Sikuli won't actually be able to find anything because there's nothing for it to look at. (Selenium is inspecting the page content programmatically, so it doesn't mind that the page is not actually displayed on any screen.)
You can also just run Jenkins and leave yourself logged in, and skip the VNC server. I wanted to run the machine without a monitor or keyboard and still have control, though. If that's your goal as well, you can't use Remote Desktop, because it allocates a new desktop on connection and then destroys it when you disconnect. (So Jenkins will work when you're watching and fail when you leave for the night. Very frustrating!)
As a side benefit, if you do it this way you can connect over VNC and watch Jenkins as it's running the test.
make the slave machine remote desktop to itself.
run the jenkins salve as program in that session
cmd /c start java -jar slave.jar -jnlpUrl http....slave-agent.jnlp -secret ... (find it in your jenkins master (shown when the slave service is down)
set it all to run at salve win startup.
I am automating web-application tests that are driven by Selenium architecture in Windows VMs, connecting via Windows Remote Desktop. It became known to me that Flash/SWF and other in-browser elements behave differently if the VM window is opened/shown (all is cool) and when the window is closed (or maybe even minimized). Which means that browsers/plugins engage some optimization if no one watches the screen (I might be wrong with that - if so - please correct me).
As a (working) workaround, installing VNC (and logging into it at least once before running the tests) helps. From that moment noone needs to "watch" the monitor.
I think that the same functionality/system calls is used by (for example) movie players that cancel the screensaver during the film watching.
The question is if any special daemon was written for those purposes, so I won't need to connect to those machines by VNC, but only to run the daemon.
Update: http://www.jddesign.co.uk/products/freeware/freeware_programs.htm - meanwhile found this utility, checking if it will do the job. Advices are still welcomed.
Update N2: the utility above doesn't do the job.
What you experience is the standard render optimization: The OS will ignore all rendering commands if the window isn't visible.
My guess is that VM tools "minimize" the whole desktop to save memory as long as no one is connected.
On Unix, I'd write a small script which runs the VNC viewer when the tests are started.
[EDIT] What you're looking for is a kind of /dev/null device for VNC connections.
Here is a solution that might work: Download the sources for the TightVNC Java Viewer (license: GPL) and remove all the rendering code.
That should allow you to start a headless VNC client for your tests.