Phpunit and Jenkins - selenium

I have a PHP project written in PHPUnit using Selenium.
The project is structured as below:
PHPProjectName
Source Files
(doesn't contain anything)
Selenium Test Files
contains all my selenium test php files - extending the class PHPUnit_Extensions_SeleniumTestCase
Include Path
c:\program files\PHP
c:\program files\PHP\PEAR\PHPUnit
I then run start the Selenium server manually by running java -jar selenium-server-standalone-2.24.1.jar
The php script to execute all my selenium test php files works fine.
But now I want to use Jenkins as a test management tool to build and execute my PHPunit tests in this folder. I guess the steps are:
Install Jenkins
Write a build script for the PHPunit tests
Execute the build script through Jenkins
Are the steps correct? Has anyone done or know how to set this up?
Thanks very much,

I have done this many times with various platforms. Your steps are generally correct and should work, however managing the server is not always so simple. The Selenium RC server gets unstable if left open for too long, so you will have to manage it somehow.
You could set up a second Jenkins job which runs once or twice a day to reset your server. The better option however would be to write a simple test framework which closes any open servers and then launches a new server instance before running the tests. You could also use a cron job to reset the server of course, but if you have Jenkins installed it will be easier to do this via a jenkins job.
The best option of course is to switch to Webdriver, but that could take some work depending on how complex your tests are.

We have a similar setup to what you describe. We have Jenkins run a job to restart the Selenium server periodically:
#!/bin/bash
# startselenium.sh: Start Selenium up and also start headless screen.
Xvfb :99 -ac &
export DISPLAY=:99
java -jar /opt/selenium/selenium-server-standalone-2.19.0.jar &
Sebastian Bergmann maintains a bunch of templates for using Jenkins with PHP here:
http://jenkins-php.org/
Included is the necessary Ant script to run PHPUnit (which is really simple and just calls PHPUnit):
<target name="phpunit" description="Run unit tests with PHPUnit">
<exec executable="phpunit" failonerror="true"/>
</target>
And the necessary 'phpunit.xml' file:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="tests/bootstrap.php"
backupGlobals="false"
backupStaticAttributes="false"
strict="true"
verbose="true">
<testsuites>
<testsuite name="ProjectName">
<directory suffix="Test.php">tests/unit/</directory>
<directory suffix="Test.php">tests/integration/</directory>
</testsuite>
</testsuites>
<logging>
<log type="coverage-html" target="build/coverage" title="BankAccount"
charset="UTF-8" yui="true" highlight="true"
lowUpperBound="35" highLowerBound="70"/>
<log type="coverage-clover" target="build/logs/clover.xml"/>
<log type="junit" target="build/logs/junit.xml" logIncompleteSkipped="false"/>
</logging>
<filter>
<whitelist addUncoveredFilesFromWhitelist="true">
<directory suffix=".php">src</directory>
<exclude>
<file>src/bootstrap.php</file>
</exclude>
</whitelist>
</filter>
</phpunit>

You have to install the selenium plugin in jenkins, then a selenium server will automatically start on jenkins, which will create a hub. Now on the client you have to start a node which connects to this hub.
Note: The jenkins selenium server is always the same version as the selenium plugin from jenkins. So if the selenium plugins name is
selenium plugin 3.1.0 then it runs on selenium server 3.1.0.
After installing the jenkins selenium plugin, then you can find a new option for selenium grid, click on it and you will get more informations:
Now you have to start a jenkins selenium standalone server like this:
Windows (create a .bat file with the following content and execute it, change relevant parts accordingly):
start java -jar -Dwebdriver.gecko.driver="C:\Webdrivers\GeckoDriver\geckodriver.exe" -Dwebdriver.chrome.driver="C:\Webdrivers\ChromeDriver\chromedriver.exe" selenium-server-standalone-<VERSION>.jar -role node -hub http://<YOUR_JENKINS_MACHINE_IP>:<PORT>/grid/register
In my case, I used:
start java -jar -Dwebdriver.gecko.driver="C:\Webdrivers\GeckoDriver\geckodriver.exe" -Dwebdriver.chrome.driver="C:\Webdrivers\ChromeDriver\chromedriver.exe" selenium-server-standalone-3.1.0.jar -role node -hub http://172.25.201.100:4444/grid/register
Make sure to correct the paths to geckodriver and chromedriver to their actual location.
Now the node should connect to the hub and you can start your tests.
More infos:
https://github.com/SeleniumHQ/selenium/wiki/Grid2

Related

Selenium IDE 2.9.1 - Can I run tests from a command line?

I would like to add a command line start for Selenium IDE test scripts to our build process so some of the tests are run immediately after the latest code is built for our products.
I have been looking and everything I find seems to relate to older versions. I can't find anything that looks up to date.
Can I run tests that I have recorded in Selenium IDE 2.9.1 from the command line - and by extension, a batch file to run multiple tests.
Is there documentation that I did not find? Can someone direct me to it?
Thanks.
You didnt specify an operating system but i think you mean windows, and yes, you can run selenium IDE test suites from powershell or CMD on windows.
You will need Java installed on your machine, and also the Selenium Server.jar file which should have come with your installation of Selenium, if not you can download from the Selenium Site.
Then, create a test suite in IDE and save it somewhere on your machine. (you can only run test suites, not testcases - i wont explain the difference between the two here)
Then in powershell or CMD you run this command, you will need to change it depending on your installation directories.
C:\Java\JDK1.8.0_66-X64\bin\java -jar C:\Selenium\Selenium-Server\selenium-server.jar -htmlSuite *firefox "baseurl" "C:\temp\testsuitename" "C:\TEMP\logs" -timeout 0060 -trustAllSSLCertificates
I will break this down for you -
C:\JDK1.8.0_66-X64\bin\java -jar - this is your Java installation directory (mine is Java JDK 1.8)
C:\Selenium\Selenium-Server\selenium-server.jar - This is the location of your Selenium .jar file.
-htmlSuite *firefox These are .jar file options to tell the jar file you are running it from a command line and to use firefox
"baseurl" In here, you should enter your base url which would normally be at the top of the IDE gui. eg, www.google.com
"C:\temp\testsuitename" The full path where you have saved your test suite
"C:\TEMP\logs" Running from command line produces a html log, advise where you want this placed
-timeout 0060 -trustAllSSLCertificates There are additional options, of which there are many. In this case the total time the command will run before it times out is 60 seconds, and it will ignore all those untrusted certificate errors that firefox often gets (where you need to keep adding exceptions etc).
Hope i could help!

Jenkins & TestNG start browsers

Is it possible to make Jenkins use actual browser instead of headless browser? I a running some tests written in TestNG (using Selenium webdriver). When I run the testng.xml file in Eclipse, the browser starts and the tests run. But when I use Jenkins and run the tests with maven, it doesn't start any browsers.
If your jenkins is hosted in a Windows machine, there are some special configurations you should know about services that are allow to use the interface.
By the way, the easiest way to see the browsers running is starting jenkins using the command line:
java -jar jenkins.war
In linux you could use the same command or use xvfb plugin to run browsers in background.
Hope helps
In addition to this, the main reason for not launching the browser is JNLP (java network launch protocol) , when we execute the war we can interact with the desktop applications.
Using Selenium Grid will allow you to execute the test on Jenkins but open the browser on a remote slave.
To achieve this you need to create an instance RemoteWebdriver than ChromeDriver, IEDriver etc I
For linux. If jenkins is running as a daemon, you could specify active display to connect to and run your browser on it. Check what display you could connect to:
ps e | grep -Po " DISPLAY=[\.0-9A-Za-z:]* " | sort -u
My output is:
DISPLAY=:10.0
DISPLAY=:2
DISPLAY=:2.0
Then go to your jenkins project -> Configure -> Build and add the next string above your main build configuration through "Add build step -> Execute shell"
/bin/bash -c "export DISPLAY=:10"
Edited: I've encountered the issue again recently. To resolve it:
I've given for the jenkins' user ability to interact with the desktop of my current user:
xhost +si:localuser:jenkins
so if I connect to my linux system through ssh using jenkins' user credentials, export display of my current user (export DISPLAY=:10) and run, for example google-chrome or firefox inside of putty's terminal, they are launching on my current user's desktop.
After this I've checked If I could start "mvn test" command inside of workspace/MyTests folder from my putty so it will start browser and execute tests.
At the end I've created simple script in the root of my current user:
vi ~/.startup.sh
#!/bin/bash
xhost +si:localuser:jenkins
and added it to my Xfce4 GUI: Application -> Settings -> Session and Startup -> Application Autostart, specifying command field as:
sh -c $HOME/.startup.sh
It's because of this script should work only when desktop is loaded to share my current user's display with jenkins' user. After reboot and connecting to this server through RDC desktop loads with xhost command applied. And after this jenkins could interract with desktop even when I close the RDP connection but leaving current user's session alive.
I've removed Build step in my jenkins' project configuration that was stated "Invoke top-level Maven targets". It could not start my browsers.
I've changed my "Add build step -> Execute shell" to:
export DISPLAY=:10
cd /var/lib/jenkins/workspace/MyTests
mvn test
I've tried grid also, turning selenium-server-standalone -hub and -node into daemons. But it was slower than launching browsers with WebDriver in the such way.

Running Selenium tests in Jenkins

I have recorded some simple selenium tests by Selenium IDE. Now I want to run those tests in Jenkins.
Which plugin to Jenkins do I need to do that? And how to run the tests step by step? Help is appreciated.
you can use recorded selenium IDE script and selenium-server.jar file to run it from Jenkins
Here is steps:
Go to SeleniumHQ page and download Selenium Server file
Eg: selenium-server-standalone-2.33.0.jar
Repair html test suite Use Selenium IDE to record then save as
html test case and test suite then put them in a same folder
eg: TestCase.html, TestSuite.html
In jenkins
Plugin Seleniumhq
Configure Selenium runner file
Manage Jenkins > Configure System > Selenium Remote Control: htmlSuite Runner = path to file u have download in step 1
Configure Job to run
In Build field click " Add build step" then select "seleniumhq htmlSuite Run"
browser: *firefox or *iexploer ....
startURL: http://www.google.com or ...
suiteFile: Input absolutely path to TestSuite.html file saved in step 2
resultFile: Input absolutely path to a file that results will be saved
Hope this help!
I did the same but the following error occurred:
Unable to find the HTML runner. This is normally because you have not downloaded
or made available the 'selenium-leg-rc' jar on the CLASSPATH. Your test will
not be run.
Download the Selenium HTML Runner from http://www.seleniumhq.org/download/ and
use that in place of the selenium-server-standalone.jar for the simplest way of
running your HTML suite.

Selenium-PHPUnit, Selenium Grid

Before posting this I checked a lot on this site and on google to figured out my problem.
I am testing my web application which is principally coded in PHP using Symfony2 framework.
I am using Selenium to do my functional tests. All I want to do for the moment is to run my functional tests in parallel on my local machine using Selenium Grid. What I do is recording the test on Selenium IDE and export the test case in phpunit format. I tried to use selenium grid but my phpunit tests are still running sequentially.
What I did:
1) java -jar selenium-server-standalone-2.24.1.jar -role hub
2) java -jar selenium-server-standalone-2.24.1.jar -role node -hub http://localhost:4444/grid/register -browser "browserName=firefox,maxInstances=2,maxSession=2"
3) ant
There is in my build.xml a phpunit target:
<target name="phpunit" description="Run unit tests">
<exec executable="phpunit" failonerror="true"/>
</target>
In my phpunit.xml this part of code is present:
<testsuites>
<testsuite name="LoginSuite">
<file suffix="Test.php">../../src/Tests/FunctionalTests/LoginSuite_testLoginTest.php</file>
</testsuite>
</testsuites>
And my LoginSuite_testLoginTest.php looks like this:
<?php
namespace Tests\FunctionalTests;
use Tests\FunctionalTests\SetUpTest;
class LoginSuite_testLoginTest extends SetUpTest
{
public function testLogin()
{
$this->open("/home");
$this->click("link=Login");
$this->type("id=username", "test.user#gmail.com");
$this->type("id=password", "test");
$this->click("id=_submit");
$this->waitForPageToLoad("30000");
}
public function testLogin2()
{
$this->open("/home");
$this->click("link=Login");
$this->type("id=username", "test.user2#gmail.com");
$this->type("id=password", "test");
$this->click("id=_submit");
$this->waitForPageToLoad("30000");
}
}
?>
At the third step when I launch ant command I am getting a jetty error 500 Problem accessing /selenium-server/driver/
If instead of doing:
java -jar selenium-server-standalone-2.24.1.jar -role node -hub http://localhost:4444 /grid/register -browser "browserName=firefox,maxInstances=2,maxSession=2"
I do the same command without -browser informations it launches my tests but not in parallel..., so strange.
I saw that to launch phpunit tests in parallel we have to create our own script to do it. So in this case do I need selenium grid or not?? I am very confused. Thanks for your help.
There is a parallel wrapper for phpunit command line https://github.com/siivonen/parallel-phpunit so you don't need to write it. You need to have Selenium Hub (with some nodes connected to it) running somewhere. Then you run your tests pointing to that location and the Hub will proxy the calls to free nodes.
We use parallel-phpunit and Selenium Grid with four nodes connected to a hub. Our CI runs 30 minutes of Selenium tests in 3 minutes.
There could be different issues. One is, while running pre recorded steps, you should add seleniumProtocol=Selenium in "browser" argument, because by default it will be the webdriver.

I want to run Selenium test case file from command line

I made then saved a test case with the Firefox extension "Selenium IDE".
Now I want to use command line to run this exported html file.
I try to follow this how-do-i-launch-the-selenium-ide-from-the-command-line-with-a-specific-test-case but it doesn't work.
Please help me.
You will need the Selenium RC which you can get from:
http://seleniumhq.org/download/
And Java 1.5 or higher (Download Java here)
1) Install Java
2) Unpack Selenium RC.
3) Open a cmd.exe window and go to the directory containing the Selenium Server (selenium-remote-control-1.0.1\selenium-server-1.0.1)
4) Run the command below:
java -jar selenium-server.jar -htmlSuite "*firefox" "http://10.8.100.106" "C:\mytestsuite\mytestsuite.html" "C:\mytestsuite\results.html"
This should run your test suite in Firefox and write the results to the html file. Obviously you will need to change the "http://10.8.100.106" argument to your own server (this might just be localhost / 127.0.0.1)
It is possible to run individual test cases using Selenese Runner. You can specify a single test case file or a test suite as the unit to run.
We should execute the SeleniumRC in using following command;
java -jar filename.jar
ex:
java -jar program1.jar
the program1 consist of the followings are:
program1.class file
Resource library file such as SeleniumRC Server.jar and Selenium Java client.jar file
This method is applicable for SeleniumRC execution. We can directly create the program1.jar file from eclipse using
File->Export.
Here is an article that explains you step-by-step process of how to run Selenium RC application in Java.
Create a Java Selenium RC test script and executing the script
I have needed to do this before, and used the following:
An Ant Build (complex)
Creating a test runner class(a part of junit framework)class.
Most commonly we would run into build path errors while trying to run from cmd.
If you want to run it from command prompt you may consider writing your selenium test in python.
Make sure you have python installed if you are on windows. Mac will have python by default.
Running test from CMD is quite easy.
Follow below steps
1- Go to home directory and Set class path
Home Directory > set classpath=Home Directory\bin; and press enter
Home Directory > set classpath=Home Directory\lib*; and press enter
2-Home-directory > java org,testng.TestNG testng.xml testng2.xml testng2.xml and hit enter
I have documented all steps here. Hope it will help. Cheers
1) Running from CMD
java -cp "C:\ProjectX\Mortgage\bin;C:\Selenium_latest\selenium2.49.1\*;C:\Selenium_latest\selenium-2.49.1\libs\*" org.testng.TestNG C:\ProjectX\Mortgage\testng.xml
Run above command in C:\ProjectX\Mortgage
2) Create batch file name runner.bat
SET projectLocation=C:\ProjectX\Mortgage
CD %projectLocation%
SET classpath=%projectLocation%\bin;C:\Selenium_latest\selenium-2.49.1\*;C:\Selenium_latest\selenium-2.49.1\libs\*
java org.testng.TestNG %projectLocation%\testng.xml
PAUSE
3) Run the batch file by double clicking on it.
To be able to run in Chrome browser, you can use *chrome option instead of *firefox like below
java -jar selenium-server.jar -htmlSuite "*chrome" "http://localhost" "C:\testsuite\testsuite.html" "C:\testsuite\results.html"
Other browsers list include:
*firefox
*mock
*firefoxproxy
*pifirefox
*chrome
*iexploreproxy
*iexplore
*firefox3
*safariproxy
*googlechrome
*konqueror
*firefox2
*safari
*piiexplore
*firefoxchrome
*opera
*iehta
*custom
on session null