I am looking for ways to set up like a central 'hub' for Selenium in my work, allowing anyone to access in within the company. For example, Tester A writes test scripts, the Person B can run without having to manually copy over the test scripts to their local workstation)
So far, I've only thought of installing Selenium in a VM which will then execute as per normal. But if I run Selenium Grid, it will run VMs within VM (?). My only concern with VMs is that it'd run slowly.
If anyone can think of a better solution or recommendation please do give me some advice. Thank you in advance.
One idea. You can create an infrastructure combining Jenkins/Selenium/Amazon.
The following is my solution from another post.
You can do it with a grid.
First of all you need to create a Selenium hub with an EC2 ubuntu 14.04 AMI without UI and link it as a jenkins slave to your Jenkins master. Or as directly a master. What you want. Only command line. Download Selenium Server standalone. (be careful on downloading the version. If you Download the Selenium3Beta, things could change). Here you can configure the HUB. You can also add the Selenium Hub as a service and configure to run automatically at server start. its important that you open the Selenium default port (or the one that you configured) so the nodes can connect to it. You can do that on the Amazon EC2 console when you have created your instance. You just need to add a security group with an inbound rule for TCP in the port you want for the IPs you want.
Then, you can create a Windows server 2012 instance server (for example, that's what I did), and do the same process. Download the same version for Selenium and the chromedriver (there is no need to download any firefoxdriver for Selenium versions before Selenium3). Generate a txt file and prepare the Selenium command to link to the HUB as a NODE. And convert it to *.bat in order to execute it. If you want to run the bat at start you can create a service with the task scheduler or use NSSM (https://nssm.cc/). Don't forget to add the rules to the security groups for this machine too!
Next, create the Jenkins server. You can use the Selenium Hub as the Jenkins master or as a slave.
Last step is configuring a job to be run in the Jenkins-Selenium machine. This job needs to be linked to your code repository (git, mercurial...) Using the parametrized build plugion for jenkins you can tell that job to pull the revision you want (where every developer can pull the revision with the new changes and new tests) and run the Selenium tests in that build with the current breanch/revision and against one unique selenium. You can use ANT or Maven to run the Selenium tests in Jenkins.
May be it's complicated to understand because there are so many concepts here but it's robust and it works fine!
If you have doubts, tell me!
If Internet Explorer is not one of the browsers on which you must run your automation tests, I would recommend that you consider docker selenium.
Selenium is providing pre-configured docker images for both Selenium Hub and Node ( refer here for more information ). For making use of docker selenium all you need to do is find a machine (preferably unix machine), install docker on it by following instructions detailed here and then start the hub and node by starting off those containers. In the case of docker you can literally transform a VM (or) a physical machine into a VM farm and yet not have to worry about slowness etc., because I believe docker is optimised for these and it runs your VM as a process.
Resorting to using Amazon cloud for running your selenium nodes is all fine, but if you have corporate policies that prevent in-coming traffic from the internet into your intranet region, then I am not sure how far Amazon cloud would be useful.
Also remember that Jenkins is not something that is absolutely required but is more of a good to have part in the setup because it would let anyone run their tests from a web UI. This will however require that all your tests are checked-in and made available in a central version control system in your organization.
PS : The reason why called out Internet Explorer as an exception is because IE runs only on windows and there are no docker images (yet) for windows. All the docker images are UNIX based images.
Related
I have some problem with Jenkins and Selenium.
I have a Jenkins Master and some Build Slaves, installed as Windows Services on Windows Server 2016 machines, in which I perform build jobs. In some of these jobs, I perform test with Selenium and I need to take some screenshots of these tests, but all the screenshots that I got are black.
Reading on the web, I found that many people are facing this kind issue. But I also found this, https://www.coretechnologies.com/blog/windows-services/interact-with-desktop/, in which is reported that it's not allowed for Windows Services to interact with desktop.
So, the question is: it's possible to make interact Jenkins Slave Service with the desktop in order to take some screenshot of it?
I am running an Azure Pipelines (DevOps) pipeline that builds my automation test solution into a dll, and loads that dll to get all of the integration automation tests.
I am using the default cloud "Host" agents to run the automation. However, the drivers throw tons of errors. I recognize the errors as the same ones that I encountered when I set up automation via Jenkins -- when I was running Jenkins as a service.
To fix the issue in Jenkins, I stopped running it as a Windows service, and started launching it as a java -jar Jenkins.war command in Powershell. That allowed the selenium drivers to launch browsers as normal on the VM's desktop, rather than running the browser instances and drivers completely in memory.
So, when it ran the browser windows and drivers completely in memory, I received the same driver errors as I am now encounting on the Host agents in Azure Pipelines. The problem is that I cannot find how to make the Agents run the tests on the VM desktop. I don't have any idea where to start on solving this problem, and can't seem to find any info online about this.
Edit: To be as clear as possible. The ONLY error occurring is "$ is not defined". This is because the DOM does not load properly because the browser executables and driver executables are running completely in memory, headless. I need to be able to run the executables in a rendered part of the host machines. So, what I mean is, if I were to remote desktop into this machine, I would actively see browser windows rendering, and automation running. Right now though, if I were to remote into this VM, I would see chrome.exe, and chromedriver.exe in TaskManager, but nothing would be visible on the desktop.
I found the answer.
https://learn.microsoft.com/en-us/azure/devops/pipelines/test/ui-testing-considerations?view=vsts&tabs=mstest#visible-ui-mode
You must manually configure the hosts to run Selenium non-headless. This will prevent everything from running as a service, which is purely in memory, with no UI components. This is the problem I mentioned that Jenkins encountered when running Jenkins as a service.
The fix for Jenkins running automation with a GUI is the same for the Hosted agents in Azure pipelines, ultimately. It's just a manual configuration. This is done via a plugin and build step in pipelines.
So that we may perform front-to-back web UI testing, we are using Selenium and ChromeDriver to automate page loads/interaction as part of our testing pack.
This is behaving as expected during developer testing (on a developer's local machine), but we are struggling to perform these checks as part of our continuous integration build.
Our server plant is *NIX based, and all of our CI infrastructure runs on these machines. So that we may test Chrome under Windows (our delivery mechanism), we have configured a Selenium Grid. When the CI tests run, they access the grid, in order to locate a Windows node to run the tests on.
We have had a Windows desktop provisioned solely for the purpose of running these tests. This contains our standard enterprise build of Windows 7. This machine will be periodically rebooted in-line with the IT department's update policy.
In an effort to ensure the Selenium server is always running, we have added the Selenium Server (running in "node" mode) as a Windows service. The selenium Server is configured to start-up ChromeDriver to invoke the simulated user-interaction.
However, when running the tests from CI they fail due to timeout. Our working theory is, the system user that is running the service cannot create interactive windows. A web search has raised reference to the "Session 0" problem, but with little to no constructive advice on how to move forward.
Starting the Selenium Server process manually from an interactive session is not a viable solution, as this is leading to brittle tests - which are failing due to an infrastructure problem, rather than a genuine test regression.
How can we have an instance of Selenium Server started via a Windows Service whenever the system reboots, that is capable of launching Chrome instances?
It could be easily done with NSSM.
Installation of services looks like these:
nssm install seleniumhub java -jar C:\selenium\selenium-server-standalone-2.45.0.jar -role hub -hubConfig C:\selenium\hub.json
nssm install seleniumnode java -jar C:\selenium\selenium-server-standalone-2.45.0.jar -role node -nodeConfig C:\selenium\node.json
It provides easily way to remove service if needed:
nssm remove seleniumnode confirm
Add destination to nssm to your PATH variable and run from console as admin
UPDATE April 2021
NSSM is not supported for more than 3 years. So please consider other options like winsw or any other. WinSW does the same job as NSSM and allows to keep run configuration in xml.
You cannot run Selenium Grid as a windows service ever since Windows Vista. Microsoft calls it "Session 0 Isolation". You could do it in Windows 2000 or XP but since the time that Vista came out, Microsoft no longer will let Grid interact with the desktop (or any other UI programs for that matter). Regardless of the fact that you still see that "interact with desktop" checkbox, it is a red herring. So, you MUST run Selenium Grid in the foreground on that server in order for it to get access to the session. If it is running Windows Server, you could in theory have multiple sessions and leave Grid running in the foreground on one of the non-zero user sessions.
Right now you can't help it - it used to work fine in session 0 but for the past few days after chrome update only works for interactive sessions.
Related bugs:
https://code.google.com/p/selenium/issues/detail?id=8029
https://code.google.com/p/chromium/issues/detail?id=422218
My preferred solution to this problem (and my default choice for running Selenium Grid as a service) is to use a simple tool called AlwaysUp. It has a free 30 day trial to try it out.
What to do:
Download AlwaysUp
Configure AlwaysUp to start the Selenium Grid node on startup
Configure AlwaysUp to run the Selenium node as a specific user (not the default System user)
This way the the node will run as a service, survive machine restarts and work with the latest version Chrome.
If the user account you use to login to the machine is different from the user account you specify to run the node as a service then you will not see the browsers pop up on the desktop as they are running in a different user session. The end result is that it is almost identical to running as a normal service but gets round the Session 0 issue.
Yeah, you should use NSSM. Important is, that you add your windows account in the "Log on" tab, or any other valid account. If you run your node with the "Local System account" option, you will get the session 0 problem. With a normal user session, the nodes run smoothly invisible in the background :)
we don't use selenium GRID, we were disappointed with its stability. We use a "Jenkins Grid", that is jenkins slaves nodes on various servers.
The slaves are services with the interact with desktop flag. They run as services with NSSM, and the SERVICE_INTERACTIVE_PROCESS flag. Making sure that NoInteractiveProcess is set to 1 (cf https://learn.microsoft.com/en-us/windows/desktop/services/interactive-services).
We don't have the fancy features of the grid (that is balancing according the browser types slots). Instead, we have Jenkins balancing the test jobs using a slave node or another.
Initially we did not use the interact with desktop flag, having browsers to run without "real" display, but the behavior was not very stable (especially with resize commands).
Hope this helps.
As I explained on this thread, I found that using a small paid tool FireDaemon Pro saved me a lot of time from trying to configure NSSM and other free tools.
It works well in the background, and restarts Selenium along with the server, which was my main requirement for running Selenium Standalone Server as a Windows Service.
This free tool would probably do it:
http://yajsw.sourceforge.net/
For that to work, you need a wrapper.conf file and a script to run the YAJSW wrapper. I takes time to read the documentation, but it is a free solution.
I wrote an example shared here, that installs JBoss7 as a Windows service.
Of course, you can simplify my example by a lot.
Apologies, I'm still fairly new to selenium so please bear with me as I explain this.
Currently my selenium tests are running on a remote machine but no window opens when I'm remotely logged into that machine!
My setup is:
Remote machine has 2 admin users with Selenium Grid2 and a node as a windows service.
The machine is running windows server 2012 with the services having Interactive services enabled.
I am using Selenium 2.42.1 with IEDriver version 2.42.0
Tests are being built and run remotely on our build server.
I think that's everything, if there's any more information then please let me know as I'd really like to know why I cannot view my tests running.
Just for clarification, the tests are running and they successfully pass or fail where necessary, but I just can't see it if I'm remote logged into the machine.
UPDATE
There has been some interesting progress on this issue but still no resolution.
I decided to try and run a node from command line and funny enough the tests run with no problems and the browser window is displayed.
So if anyone has any idea why a browser window would appear when running a node from command line but not when running a node as a service would be great.
I'm using java service wrapper to create my service and im using the same nodeconfig.json when running the node.
This can happen when you are running the node as a windows service. There is an option in service to say whether the job should run in background or interact with desktop. Check that and you should see the windows.
I have a jenkins agent set up on window 7 and a jenkins server on Linux. I am running GUI testing on the windows agent. It runs fine if I have a remote desktop connection connected to it, but fails otherwise. I found this link, Jenkins on Windows and GUI Tests without RDC
But the solution provided there is pretty vague.. It seems like the only solution is to somehow make Jenkins server to have a remote desktop connection open at all times. But I can't find such an option to do so.
Could anyone please clearly teach me how to solve this issue?
Much appreciated!
Your slave machines have to be at a desktop before the test can run
properly. We had the same problem.
Solution was to have the test machine start up and auto-logon to the
desktop. To ensure that the test would ONLY start after the desktop
was available, we added a scheduled task, set to run at user login,
that would launch the Jenkins slave via Java Web Start. That way,
Jenkins would only see the slave once the desktop was running. After
that, everything worked fine.
This is the winning answer to the question you linked to and it is very clear on what to do. The whole setup is outside of Jenkins. Jason Swager discribed on how he automated a user logging into a windows desktop machine and then starting the Jenkins slave in the user session.
And now Step by step:
1. make sure you have a GUI evailable
Solution was to have the test machine start up and auto-logon to the desktop
Configure a standard Windows desktop to login a specific user automatically when windows start. This way nobody needs to physically log into the desktop. (see How to turn on automatic logon in Windows 7)
2. start Jenkins slave
You need to start the Jenkins slave within this user setting. Otherwise, the Jenkins slave won't have access to the Windows UI components (or in other words can not interact with the desktop).
To ensure that the test would ONLY start after the desktop
was available, we added a scheduled task, set to run at user login,
that would launch the Jenkins slave via Java Web Start.
So you have to create a scheduled task and configure it to start your Jenkins client using Java Web Start.
3. use it
That way, Jenkins would only see the slave once the desktop was running. After
that, everything worked fine.
When the slave is online, you can use it to run your UI tests.
To solve it, set Windows Auto-Logon as I explain here:
https://serverfault.com/questions/269832/windows-server-2008-automatic-user-logon-on-power-on/606130#606130
Then create a startup batch for Jenkins slave (place it in Jenkins directory), which will launch it's console on desktop, and will allow GUI jobs to run:
java -jar slave.jar -jnlpUrl http://{Your Jenkins Server}:8080/computer/{Your Jenkins Node}/slave-agent.jnlp
(slave.jar can be downloaded from http://{Your Jenkins Server}:8080/jnlpJars/slave.jar)
EDIT :
If you're getting black screenshots (when using Selenium for example), create a batch file that disconnects Remote Desktop, instead of closing the RDP session with the regular X button:
%windir%\system32\tscon.exe %SESSIONNAME% /dest:console
The following thing did the trick for me:
In Jenkins execute windows shell command:
cmdkey /generic:TERMSRV/<servername> /user:<username> /pass:<password>
mstsc /v:<servername> /w:<width> /h:<height>
cd <path to your pom.xml>
<maven command>(e.g. mvn test -Dfiles_to_run=groupLaunch.xml
cmdkey /delete:TERMSRV/<servername>
It creates real mstsc session (Win-to-Win) with specified width and height in your virtual mstsc session (Jenkins-to-Win) powered by Jenkins.
I tried the solutions provided here but nothing worked for me. At the end, I came up with a workaround.
I opened an RDP connection to the VM in a different VM (VM2).
I left the first connection open inside VM2 and disconnected from it.
It worked but that implies having two virtual machines available.
You still need to use RDP but in my case, we can use loopback of the RDP in same VM.
The procedure:
In a VM, create two different accounts, and create Jenkins slave for both of account.
Now you'll have two Jenkins slave for two accounts in one VM
slave 1 - account 1
slave 2 - account 2
Enable multiple RDP follow guide
https://www.serverwatch.com/server-tutorials/how-to-enable-concurrent-remote-desktop-sessions-in-windows.html
In slave 2(with account 2), run rdp command to connect to slave 1(with account 1), like following
Start /b "" "C:\RDP\rdp.exe" /v:127.0.0.2 /domain:\ /u:admin /p:xxxx /fullscreen /w:1920 /h:1200
127.0.0.2 is very important, it's a loopback connection for RDP
Put above command into Jenkins job with the name such as "OpenRDP_ToVMXXX, and then you can run any test on slave 1 with GUI enabled, enjoy.
As the solutions above seemed a bit overkill, I used this approach:
Disable Jenkins service
Start Jenkins from command line using java -jar C:\Program Files (x86)\Jenkins\jenkins.war
For some reason I had to install all plugins and everything when starting it this way, so I recommend creating a backup, there is a plugin for that. Good luck. :)