Trying to enable Xvfb in Jenkins declarative pipeline to be able to run Selenium headless tests from the pipeline definition.
Have been able to run Selenium tests in a standard Jenkins (Linux) job. That's fine, i.e. Xvfb can be enabled under build (after plugin install) in the Jenkins job and then can Python virtual env be setup and Selenium tests executed from shell.
But I want to have a pipeline scope/setup. But in pipeline type jobs the Xvfb doesn't show up. And I haven't been able to find an answer if and how it can be enabled from the declarative pipeline code itself. Is it possible?
Is there any workaround?
Yes you can, every job which is pipeline have a link on left side of job page "Pipeline Syntax" when you go there it will help you a lot for some non obvious cases. So, for your case:
Related
I'm trying to use these all things together to run parallel tests in a headless chrome:
Docker, Selenium, Pytest
However, I'm wondering where it makes sense to run the parallel part of the system?
Docker can do this (using selenium grid). Both these can be used to run parallel (and distributed) selenium tests. e.g.
https://github.com/elgalu/docker-selenium
https://github.com/zalando/zalenium
Also Pytest has its own way of running parallel tests (using pytest-xdist) e.g.
http://pytest.org/dev/xdist.html
Would it be easier to run 10 parallel pytest-xdist than running 10 docker containers?
I would be grateful to find out the advantages/disadvantages are for each.
Also, any idea how to use these things together? Information on this seems really sparse.
You create as much as you need/want containers then you will let know xdist IPs of containers and if you need UI tests then pytest has pytest-splinter and if you need bdd scenarios you can use pytest-bdd.
However, I'm wondering where it makes sense to run the parallel part of the system?
Each part will contribute for the parallelism to happen.
You need Selenium HUB to orchestrate available browsers to run a test. You can have n browser running in headless mode, each one isolated in its own container.
Would it be easier to run 10 parallel pytest-xdist than running 10 docker containers?
Pytest will parallelize the test execution for you, bur not the instantiation and orchestration to the available browsers.
Summarizing:
Problem: You need to run UI (Selenium) tests in parallel. You will need N amount of browsers available to run this test.
Solution: You can start N nodes of headless chrome from docker.
problem: You have 10 different connection options to give to your drivers in your tests.
Solution: Start selenium hub and let it manage these for you. So you have to concern with only one connection point and then it will give you the browser that is free to run that test.
Problem: You run you tests now and only one browser is being used.
Solution: Use xdist to instruct pytest to run X amount of tests per time. X in this case can match with N number of browser available.
I put together a machine (Windows Server 2012R2) for POC reasons where a Jenkins installed and it executes Selenium UI tests using msunit
But, when I log in the server where the Jenkins runs and I watch what happens during CI build (compile and test execution) I can't see that the browser (Firefox) starts automatically, however, the test results and the logs show that a browser was executed.
My question is that, what the is happening when my tests are executed by Jenkins? If I execute the command which from visual studio on the same machine then I can see that Firefox starts, does what is programmed in the tests and the results are in the result.trx.Can I somehow set up Jenkins the way the browser really executed (I can believe it when I see it :)
In Jenkins when you run selenium test cases, they are executed in the background by default.
Your Jenkins might be configured to run those test cases in some video buffer(usually it happens on Linux but can also be configured on Windows) or in a headless state.
As your question, if you are using MSTest which basically used to convert the test cases result from trx to JMX format but also can be used to run selenium tests. when you run the same in Jenkins it will run in background on any slave or on master.
is it possible to run selenium script in server without using eclipse or command prompt.for every fixed time intervals my selenium script has to run automatically in backend with out any browser opening
This can be split into two questions;
Can I schedule a Selenium test?
Yes, there are many options including using a CI tool such as Jenkins, Hudson, Team City,etc. Alternatively you could write a shell script to run the command line, and schedule this to run every x hours.
Can I run Selenium without opening a browser?
Yes. As mentioned before you could go headless, but depending on complexity of your application and test, you may find headless is not suitable. However, you can use RemoteWebdriver to launch a browser on a different machine but the test would still be run on your back end, it would just be the browse launched remotely. See https://code.google.com/p/selenium/wiki/Grid2 and https://code.google.com/p/selenium/wiki/RemoteWebDriverServer
I'm trying to setup automatic Selenium tests with Jenkins on a Linux server.
The problem is that all tests fail with:
org.openqa.selenium.TimeoutException: Timed out after 60 seconds waiting for visibility of element located by By.linkText
Each test fails with the fist element expected.
Jenkins has the Xvfb plugin installed, and from the console messages it seems to work:
Xvfb starting$ /usr/bin/Xvfb :1 -screen 0 1024x768x24
I tried to increase the timeout, but it seems its not that the problem.
The tests run fine on windows and on a linux system with display.
The problem is I don't have direct access to the server, so I'm trying to explore all possibilities before I make requests to the sys admins.
To run the tests are used two projects, both seem to deploy fine, but when the tests start its like one project (the backend) is not deployed and some URLs are not found, from here the timeout exception. I'm not very sure how Jenkins behaves in this case, and if it starts one project at a time.
Any ideas? Thanks a lot!
To check if the application under test is running when tests try to access it, you can connect to the test machine via ssh, and take screenshot at the moment when the test runs:
xwd -root | convert xwd:- capture.png
http://inspirated.com/2007/04/02/howto-use-xwd-for-screenshots
Right now I am using Hudson CI to launch browser in xvnc and run my tests through Webdriver (pre 2.0)
Should I use Selenium remote webdriver instead? is that more efficient than hudson ci?
I want to run my tests on Amazon....are there any selenium remote webdriver server or Selenium Grid plugin that can already integrate with Amazon? Should I pursue Grid or remotedriver server?
My ultimate goal is to be able to launch multiple tests parallel on Selenium on Amazon and be able to quickly scale up and down.
Right now I pay $100/month for server that has hudson CI on it....but I never end up using the whole capacity....
Grid or the Remote Server does not replace a CI tool like Hudson. They will not schedule jobs, parallelize your tests, report results (with history) or send email/IM notifications when builds fail.
So you would still have Hudson (or similar) for all those things, possibly using Grid or the Remote Server to decouple the machine(s) running the browser(s) from the one running the tests.
Selenium Grid will allow you launch multiple tests in parallel. And with the right configuration you can even run the tests of different browsers in parallel.
Selenium grid is still using selenium 1.x remote control drivers. So if you already have your tests written in webdriver code, then you might not be able to use it with current Selenium grid. There is a new version of selenium grid - Grid 2.0 that will be out soon. That will be support webdriver as well.