I want to run all the scripts one by one sequentially. I've created a suite file and included scripts in suite. When I run a suite, the scripts run in parallel in multiple browsers. I would like to run them one after the another in a single browser.
You can run a suite file from a single browser by changing the Threads in the ant target.
<target name="runbrowsertests">
<sahi suite="../userdata/scripts/demo/demo.suite"
browserType="firefox"
baseurl="http://sahi.co.in/demo/"
sahihost="localhost"
sahiport="9999"
failureproperty="sahi.failed"
haltonfailure="false"
threads="1"
>
<report type="html" />
</sahi>
</target>
If it still doesn't work, edit browser_types.xml (Click "configure" link on dashboard). Change <capacity> to 1 for the browser that you want to run the tests with. Restart Sahi.
Related
I have multiple test_*.js files underneath a /testcafe/tests/folder. When executing TestCafe from commandline OR via Visual Studio, only ONE file is ever executed....why would that be?
tests (folder)
test_somefilename.js
test_someOtherfilename.js
test_onlyfileexecuted.js
test_anotherfilename.js
When running the following command, only 1 of the files is ever executed:
testcafe <pathToTestFolder>/tests/*.js
this will only ever execute the file named test_onlyfileexecuted.js
WHY? I can manually type in the command to execute every single file and TestCafe works perfectly, but when doing glob pattern, that is the only tests found/executed....So Confused
Just run testcafe {browser} <pathToTestFolder>/tests/ (no *.js) and Testcafe will automatically pick up all the tests in that directory
As I am implementing an automated way to GUI test our webapplication with selenium I ran into some issues.
I am using selenese-runner to execute our Selenium test suites, created with Selenium IDE as a post build action in Jenkins.
This works perfeclty fine, as the build fails when something is wrong, and the build succeeds if all tests are passed. And the results are stored on a per build basis as HTML files, generated be selenese-runner.
My problem is however, that I seem to be unable to find a way, how to display these results in the respective jenkins build.
Does anyone have an idea how to solve this issue. Or maybe I am on the wrong path at all?
Your help is highly appreciated!
I believe the JUnit plugin should do what you want, but it doesn't work for me.
My config uses this shell script to run the tests (you can see the names of all my test suites):
/usr/bin/Xvfb &
export DISPLAY=localhost:0.0
cd ${WORKSPACE}
java -jar ./test/selenium/bin/selenese-runner.jar --baseurl http://${testenvironment} --screenshot-on-fail ./seleniumResults/ --html-result ./seleniumResults/ ./test/selenium/Search_TestSuite.html ./test/selenium/Admin_RegisteredUser_Suite.html ./test/selenium/Admin_InternalUser_Suite.html ./test/selenium/PortfolioAgency_Suite.html ./test/selenium/FOAdmin_Suite.html ./test/selenium/PublicWebsite_Suite.html ./test/selenium/SystemAdmin_Content_Suite.html ./test/selenium/SystemAdmin_MetaData_Suite.html
killall Xvfb
And I can see the result of the most recent test (you can see the name of my jenkins task folder)
http://<JENKINS.MY.COMPANY>/job/seleniumRegressionTest/ws/seleniumResults/index.html
Earlier tests are all saved on the Jenkins server, so I can view them if I need to.
I have selenium test script written on one of my local machine and it works fine on it using testng.xml
I have copied the same script to different local machine and am trying to execute it using testng.xml by right clicking on testng.xml and selecting run as testng suite but nothing happens.
No error is shown neither script is executed or browser is instantiated.
There is no error in the project. I have added all the requred jar files.
Can any one help me with any pointer.
<?xml version="1.0" encoding="UTF-8"?>
<suite name="XXXXX">
<test name="testScripts">
<parameter name="superAdmin" value="superAdmin"/>
<parameter name="participant" value="participant"/>
<classes>
<class name="testScripts.ParticipantSide" />
</classes>
</test>
Running testng.xml from run as > configuration and specifying the suite, runs the scripts
It is running by manually specifying the suite to execute. It is still not running directly via path right click on testng.xml > Run As > TestNg Suite
Try to add the full path of your class as local machine is different. Try to build and compile or clean the entire source code and try again. Because testNG.xml seems to be the correct one.
Facing this very similar issue after I updated TestNG yesterday. In my case, I haven't copied Selenium Suite from anywhere, it was built from scratch on the very same machine that I am using at the moment.
But after I updated TestNG yesterday, I now have to go through Run > Run Configuration > path\to\testng.xml (in Suite field) and then Run it.
It doesn't work if I simply right click on testng.xml and do Run As > TestNG Suite
I have a PHP project written in PHPUnit using Selenium.
The project is structured as below:
PHPProjectName
Source Files
(doesn't contain anything)
Selenium Test Files
contains all my selenium test php files - extending the class PHPUnit_Extensions_SeleniumTestCase
Include Path
c:\program files\PHP
c:\program files\PHP\PEAR\PHPUnit
I then run start the Selenium server manually by running java -jar selenium-server-standalone-2.24.1.jar
The php script to execute all my selenium test php files works fine.
But now I want to use Jenkins as a test management tool to build and execute my PHPunit tests in this folder. I guess the steps are:
Install Jenkins
Write a build script for the PHPunit tests
Execute the build script through Jenkins
Are the steps correct? Has anyone done or know how to set this up?
Thanks very much,
I have done this many times with various platforms. Your steps are generally correct and should work, however managing the server is not always so simple. The Selenium RC server gets unstable if left open for too long, so you will have to manage it somehow.
You could set up a second Jenkins job which runs once or twice a day to reset your server. The better option however would be to write a simple test framework which closes any open servers and then launches a new server instance before running the tests. You could also use a cron job to reset the server of course, but if you have Jenkins installed it will be easier to do this via a jenkins job.
The best option of course is to switch to Webdriver, but that could take some work depending on how complex your tests are.
We have a similar setup to what you describe. We have Jenkins run a job to restart the Selenium server periodically:
#!/bin/bash
# startselenium.sh: Start Selenium up and also start headless screen.
Xvfb :99 -ac &
export DISPLAY=:99
java -jar /opt/selenium/selenium-server-standalone-2.19.0.jar &
Sebastian Bergmann maintains a bunch of templates for using Jenkins with PHP here:
http://jenkins-php.org/
Included is the necessary Ant script to run PHPUnit (which is really simple and just calls PHPUnit):
<target name="phpunit" description="Run unit tests with PHPUnit">
<exec executable="phpunit" failonerror="true"/>
</target>
And the necessary 'phpunit.xml' file:
<?xml version="1.0" encoding="UTF-8"?>
<phpunit bootstrap="tests/bootstrap.php"
backupGlobals="false"
backupStaticAttributes="false"
strict="true"
verbose="true">
<testsuites>
<testsuite name="ProjectName">
<directory suffix="Test.php">tests/unit/</directory>
<directory suffix="Test.php">tests/integration/</directory>
</testsuite>
</testsuites>
<logging>
<log type="coverage-html" target="build/coverage" title="BankAccount"
charset="UTF-8" yui="true" highlight="true"
lowUpperBound="35" highLowerBound="70"/>
<log type="coverage-clover" target="build/logs/clover.xml"/>
<log type="junit" target="build/logs/junit.xml" logIncompleteSkipped="false"/>
</logging>
<filter>
<whitelist addUncoveredFilesFromWhitelist="true">
<directory suffix=".php">src</directory>
<exclude>
<file>src/bootstrap.php</file>
</exclude>
</whitelist>
</filter>
</phpunit>
You have to install the selenium plugin in jenkins, then a selenium server will automatically start on jenkins, which will create a hub. Now on the client you have to start a node which connects to this hub.
Note: The jenkins selenium server is always the same version as the selenium plugin from jenkins. So if the selenium plugins name is
selenium plugin 3.1.0 then it runs on selenium server 3.1.0.
After installing the jenkins selenium plugin, then you can find a new option for selenium grid, click on it and you will get more informations:
Now you have to start a jenkins selenium standalone server like this:
Windows (create a .bat file with the following content and execute it, change relevant parts accordingly):
start java -jar -Dwebdriver.gecko.driver="C:\Webdrivers\GeckoDriver\geckodriver.exe" -Dwebdriver.chrome.driver="C:\Webdrivers\ChromeDriver\chromedriver.exe" selenium-server-standalone-<VERSION>.jar -role node -hub http://<YOUR_JENKINS_MACHINE_IP>:<PORT>/grid/register
In my case, I used:
start java -jar -Dwebdriver.gecko.driver="C:\Webdrivers\GeckoDriver\geckodriver.exe" -Dwebdriver.chrome.driver="C:\Webdrivers\ChromeDriver\chromedriver.exe" selenium-server-standalone-3.1.0.jar -role node -hub http://172.25.201.100:4444/grid/register
Make sure to correct the paths to geckodriver and chromedriver to their actual location.
Now the node should connect to the hub and you can start your tests.
More infos:
https://github.com/SeleniumHQ/selenium/wiki/Grid2
nunit tests fails when run through cc.net saying process timeout. Process has been killed
All works fine when through nUNit or VS.
Also cc.net will then show the results of previous build even if the build is a clean one.
Any help plz.
The default timeout is 600 seconds. If your tests start to exceed that the build will fail with no indication. You may need to up the timeouts for your cc.net nunit task
If you are seeing the results from a previous build, it is probably because you are not deleting the results from your previous build.
For example, my NUnit test results are written to files with the name {foo}-results.xml:
<publishers>
<merge>
<files>
<file>bin\debug\*-results.xml</file>
</files>
</merge>
</publishers>
In my tasks, I have a step in my build file that deletes the entire "bin\debug" directory so that my results are always the current ones.
One possibility is that you have a permission issue. CruiseControl is perhaps running under a service account and has different permissions than your user account (which I'm assuming you use to manually run the tests.) Try logging into the machine as the service account, then see if you can run the unit tests through VS or NUnit.
I've seen this happen if a test has an assertion, e.g. Debug.Assert(something here). When this happens to me in CC.Net, the CC.Net build pops up a message box for the assertion. Since no one closes out the message box on the build server, the NUnit test times out.