Issue running Selenium IDE test suite using Selenium Standalone server - selenium

I am try to follow following references to run the selenium IDE test suites on standalone server . My ultimate purpose is to run headless data-driven tests on ubuntu.
Selenium RC with DataDriven
Running Tests from command line
Selenium Headless tests on Ubuntu
So after looking into these tutorials I have setup very simple test suite and trying to run from command line e.g
java -jar ~/selenium/selenium-server-standalone-2.39.0.jar -userExtensions user-extensions.js -htmlSuite "*firefox" http://google.com ./suite.html ./out/firefox-results.html
Output
org.openqa.grid.selenium.GridLauncher main
INFO: Launching a standalone server
10:16:54.164 INFO - Java: Apple Inc. 20.65-b04-462
10:16:54.164 INFO - OS: Mac OS X 10.6.8 x86_64
10:16:54.178 INFO - v2.39.0, with Core v2.39.0. Built from revision ff23eac
10:16:54.250 INFO - Default driver org.openqa.selenium.ie.InternetExplorerDriver registration is skipped: registration capabilities Capabilities [{platform=WINDOWS, ensureCleanSession=true, browserName=internet explorer, version=}] does not match with current platform: MAC
10:16:54.336 INFO - RemoteWebDriver instances should connect to: http://127.0.0.1:4444/wd/hub
10:16:54.338 INFO - Version Jetty/5.1.x
10:16:54.339 INFO - Started HttpContext[/selenium-server/driver,/selenium-server/driver]
10:16:54.340 INFO - Started HttpContext[/selenium-server,/selenium-server]
10:16:54.340 INFO - Started HttpContext[/,/]
10:16:54.406 INFO - Started org.openqa.jetty.jetty.servlet.ServletHandler#5ff06dc3
10:16:54.407 INFO - Started HttpContext[/wd,/wd]
10:16:54.413 INFO - Started SocketListener on 0.0.0.0:4444
10:16:54.414 INFO - Started org.openqa.jetty.jetty.Server#4ca31e1b
jar:file:/Users/myname/selenium/selenium-server-standalone-2.39.0.jar!/customProfileDirCUSTFFCHROME
10:16:54.580 INFO - Preparing Firefox profile...
10:16:55.946 INFO - Launching Firefox...
10:16:57.246 INFO - Checking Resource aliases
It does open two firefox windows and hangs there after INFO - Checking Resource aliases
It does not matter If I skip -userExtensions user-extensions.js,it gives same output , show two popup windows and hangs there. The firefox popup window shows test suite loaded successfully , but not command is executed further.
I wonder if I need to run/configure sth else
Example Test Suite and Test Case:
suite.html
case_a.html
Command
java -jar ~/selenium/selenium-server-standalone-2.39.0.jar -htmlSuite "*firefox" http://www.google.com suite.html firefox-results.html
Tested Under:
A) OS: Mac OS X 10.6.8 x86_64 /java version "1.6.0_65"
B) OS: Windows 7 / java version : 1.7.0_02
Hacks Tried without any luck:
A) Removing xml tag
B) .html file extension

Since this question got much attention and needed a proper solution, which I found this way:
First of all we need proper addons to setup headless tests
Getting the right addons
I am already using following selenium IDE addons
datadriven v0.2
flowcontrol v08
includecommand 1.3
What I need is respective selenium addons which work with selenium core server for which I need
datadriven_v0.2-core.js
flowcontrol ( goto_sel08.js)
includecommand ( includeCommand_2.3.js) origional source
since openqa.org does not maintain these addons anymore (probably broken or so) You have to use wayback machine to get js files ( no luck with zip attachments). e g flowcontrol in wayback machine
Since I have done this already and created a gist with some fixes
datadriven_v0.2-core.js
flowcontrol ( goto_sel08.js)
includeCommand_2.3.js
user-extension.js ( all three of above combined)
Setting up Headless Tests
a) install xvfb ( X Vritual Frame Buffer and firefox )
sudo apt-get update && sudo apt-get install -y xfonts-100dpi xfonts-75dpi xfonts-scalable xfonts-cyrillic xvfb x11-apps firefox
b) create xvfb init script /var/init.d/xvfb
if [ -z "$1" ]; then
echo "`basename $0` {start|stop}"
exit
fi
case "$1" in
start)
/usr/bin/Xvfb :99 -ac -screen 0 1024x768x8 &
;;
stop)
killall Xvfb
;;
esac
c) add to startup script:
sudo update-rc.d xvfb defaults
sudo chmod +x /etc/init.d/xvfb
d) start the xvfb
/etc/init.d/xvfb start
e) Setup selenium Tests (you can put test anywhere)
sudo mkdir /usr/local/SeleniumTests && cd /usr/local/SeleniumTests
checkout/copy your html tests to /usr/local/SeleniumTests
f) create alias to test directory (config file 'selenium_alias')
(only allow from localhost)
<IfModule alias_module>
Alias /SeleniumTests/ /usr/local/SeleniumTests/
</IfModule>
<Directory "/usr/local/SeleniumTests/">
Options Indexes MultiViews FollowSymLinks
AllowOverride None
Order deny,allow
Deny from all
Allow from 127.0.0.1
</Directory>
sudo mv SeleniumTests/selenium_alias /etc/apache2/sites-avaiable
sudo a2ensite selenium
sudo service apache2 restart
g) update selenium tests data
(/usr/local/SeleniumTests/tests/data)
e.g change test username/password and baseUrl to test app (which I have in xml file)
h) Download Seleninum Server jar file
sudo mkdir /var/lib/selenium/
sudo wget http://selenium.googlecode.com/files/selenium-server-standalone-2.39.0.jar -o /var/lib/selenium/selenium-server.jar
i) Run the headless /Selenese Tests
export DISPLAY=":99" && java -jar /var/lib/selenium/selenium-server.jar -htmlSuite *firefox https://localhost "/usr/local/SeleniumTests/tests/my_app_smoke_testing_suite.html" "firefox-results.html" -trustAllSSLCertificates -ensureCleanSession -port 5561 -userExtensions /usr/local/SeleniumTests/addons/user-extensions.js
Note I have my selenium addons setup under /usr/local/SeleniumTests/addons
Some of the the steps might not be desirable in all situations.

Try this:
<Selenium server location path>java -jar <selenium-server name>.jar –htmlSuite "*<browser_name>" "<Url of base website>" "<Path of html_suite>" "<Path to store reports>"
selenium-server name :- selenium-server-standalone-2.xx.xx was used.
browser_name :- "iexplore" for Internet Explorer/ "firefox" for
Mozilla Firefox
Url of base website :- Base URL.
Path of html_suite :- The path of test suite you used to save file
(E.g.:"C:\TestSuite.htm", assuming "C" drive as location you used
to save test suite).
Path to store reports :-The path you want to save your test result
into (E.g.:"C:\TestResult.htm", assuming "C" drive as location you
used to save test result).

Some commands are not working in Firefox:
The user extension goto_sel_ide that defines the While and Goto
commands does not work with Selenium RC. Include the user extension
goto_sel08.js instead.
From: https://wiki.mozilla.org/Running_IDE_scripts_with_Selenium_RC
I think you are following these steps from this page:
Download goto_sel08.js
Download the 1.3 version of the includeCommand4IDE extension.
Download the most recent version (0.2) of the datadriven.js extension.
Merged these files to new user-extensions.js file. This order is critical!
create an xml file which must contain all the data–input and output–for a single test case.
Utilize your .xml file
Use Selenium-IDE or an editor to create an HTML test suite that includes your data-driven test as one of the tests to be executed.
Execute a command line:
-jar selenium-server.jar
-userExtensions user-extensions.js
-htmlSuite
"*chrome"
"`<base URL>`"
"`<Selenium test suite file>`"
"`<results log file>`"
-timeout nnnn
I don't post all steps. I think these steps are important for create data-driven testing from a Selenium-RC command line. Please check all steps!
Perhaps its working on Ubuntu OS, if its works on Windows. Two popup window is weird for me, but may be not is problem. I can only offer check these points:
create suite.html with firefox selenium IDE
try to use *chrome in command line.
Please check order of scripts in user-extensions.js file.
I hope it helps!

Related

How to configure Geckodriver for Seleniumhq plugin for Jenkins

I am currently working on configuring Jenkins to run selenium tests.
I am using Seleniumhq plugin and the Selenium HTML runner in the Selenium RC slot in the Jenkins configuration*.
In the particular job, I have a shell script running export PATH=$PATH:/path/to/geckodriver.exe to add it to the path followed by the SeleniumHQ htmlSuite Run step.
When it runs with this configuration, the job fails with the following errors message:
Started by timer
Building in workspace /var/lib/jenkins/jobs/MYJOB-selenium-ide-test/workspace
[workspace] $ /bin/sh -xe /tmp/hudson2672803749243149546.sh
+ export PATH=/var/lib/jenkins/tools/hudson.model.JDK/Oracle_8u45/bin:/var/lib/jenkins/tools/hudson.model.JDK/Oracle_8u45/bin:/var/lib/jenkins/.rvm/gems/ruby-2.2.2/bin:/var/lib/jenkins/.rvm/gems/ruby-2.2.2#global/bin:/var/lib/jenkins/.rvm/rubies/ruby-2.2.2/bin:/usr/local/bin:/usr/bin:/bin:/usr/local/games:/usr/games:/var/lib/jenkins/.rvm/bin:/path/to/geckodriver.exe
java -jar /var/lib/jenkins/users/me/selenium-html-runner-3.0.1.jar -htmlSuite *firefox example.com /var/lib/jenkins/jobs/MYJOB-selenium-ide-test/workspace/suite1/JenkinsTestSuite /var/lib/jenkins/jobs/MYJOB-selenium-ide-test/workspace/result.html
[workspace] $ java -jar /var/lib/jenkins/users/ME/selenium-html-runner-3.0.1.jar -htmlSuite *firefox example.com /var/lib/jenkins/jobs/myjob-selenium-ide-test/workspace/suite1/JenkinsTestSuite /var/lib/jenkins/jobs/MYJOB-selenium-ide-test/workspace/result.html
Multi-window mode is longer used as an option and will be ignored.
Dec 14, 2016 6:31:00 PM org.openqa.selenium.server.htmlrunner.HTMLLauncher mainInt
WARNING: Test of browser failed: *firefox
java.lang.IllegalStateException: The path to the driver executable must be set by the webdriver.gecko.driver system property; for more information, see https://github.com/mozilla/geckodriver. The latest version can be downloaded from https://github.com/mozilla/geckodriver/releases
at com.google.common.base.Preconditions.checkState(Preconditions.java:199)
at org.openqa.selenium.remote.service.DriverService.findExecutable(DriverService.java:109)
at org.openqa.selenium.firefox.GeckoDriverService.access$000(GeckoDriverService.java:37)
at org.openqa.selenium.firefox.GeckoDriverService$Builder.findDefaultExecutable(GeckoDriverService.java:95)
at org.openqa.selenium.remote.service.DriverService$Builder.build(DriverService.java:296)
at org.openqa.selenium.firefox.FirefoxDriver.createCommandExecutor(FirefoxDriver.java:277)
at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:247)
at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:242)
at org.openqa.selenium.firefox.FirefoxDriver.<init>(FirefoxDriver.java:135)
at org.openqa.selenium.server.htmlrunner.HTMLLauncher.createDriver(HTMLLauncher.java:285)
at org.openqa.selenium.server.htmlrunner.HTMLLauncher.runHTMLSuite(HTMLLauncher.java:107)
at org.openqa.selenium.server.htmlrunner.HTMLLauncher.mainInt(HTMLLauncher.java:245)
at org.openqa.selenium.server.htmlrunner.HTMLLauncher.main(HTMLLauncher.java:273)
Publishing Selenium report...
ERROR: Test reports were found but none of them are new. Did tests run?
For example, /var/lib/jenkins/jobs/myjob-selenium-ide-test/workspace/suite1/results.html is 7 days 1 hr old
Build step 'Publish Selenium Report' changed build result to FAILURE
Finished: FAILURE
I can see that the issue is that the line System.setProperty("webdriver.gecko.driver","path/to/geckodriver.exe"); needs to be run. How do I properly configure this to happen? (I have tried to have it run as a Groovy build step, but this failed).
*(I have also attempted replacing it with an older version of the standalone server)

Selenium IDE 2.9.1 - Can I run tests from a command line?

I would like to add a command line start for Selenium IDE test scripts to our build process so some of the tests are run immediately after the latest code is built for our products.
I have been looking and everything I find seems to relate to older versions. I can't find anything that looks up to date.
Can I run tests that I have recorded in Selenium IDE 2.9.1 from the command line - and by extension, a batch file to run multiple tests.
Is there documentation that I did not find? Can someone direct me to it?
Thanks.
You didnt specify an operating system but i think you mean windows, and yes, you can run selenium IDE test suites from powershell or CMD on windows.
You will need Java installed on your machine, and also the Selenium Server.jar file which should have come with your installation of Selenium, if not you can download from the Selenium Site.
Then, create a test suite in IDE and save it somewhere on your machine. (you can only run test suites, not testcases - i wont explain the difference between the two here)
Then in powershell or CMD you run this command, you will need to change it depending on your installation directories.
C:\Java\JDK1.8.0_66-X64\bin\java -jar C:\Selenium\Selenium-Server\selenium-server.jar -htmlSuite *firefox "baseurl" "C:\temp\testsuitename" "C:\TEMP\logs" -timeout 0060 -trustAllSSLCertificates
I will break this down for you -
C:\JDK1.8.0_66-X64\bin\java -jar - this is your Java installation directory (mine is Java JDK 1.8)
C:\Selenium\Selenium-Server\selenium-server.jar - This is the location of your Selenium .jar file.
-htmlSuite *firefox These are .jar file options to tell the jar file you are running it from a command line and to use firefox
"baseurl" In here, you should enter your base url which would normally be at the top of the IDE gui. eg, www.google.com
"C:\temp\testsuitename" The full path where you have saved your test suite
"C:\TEMP\logs" Running from command line produces a html log, advise where you want this placed
-timeout 0060 -trustAllSSLCertificates There are additional options, of which there are many. In this case the total time the command will run before it times out is 60 seconds, and it will ignore all those untrusted certificate errors that firefox often gets (where you need to keep adding exceptions etc).
Hope i could help!

Can Travis-CI run Codeception tests?

I'm creating my tests (though I'm a beginner, learning) using Codeception. This includes acceptance and unit tests for now.
I want to add my repo to Travis CI so I can automate testing process after each commit and put build-status tag.
I would like to ask;
Can Travis-CI run codeception tests?
Can Travis-CI run codeception acceptance tests emulating browser?
If both answers are no, is there any other CI tool which can?
Thank you.
Yes, it is possible to run Codeception tests, including acceptance tests that run using WebDriver, on Travis CI.
It is possible to run your tests with a real browser on Travis, but it is easiest to use a headless browser, since Travis is running on a headless machine. PhantomJS is perfect for this, and it comes pre-installed with Travis CI's build bootstrap.
To run the tests with PhantomJS, you'll need to configure the WebDriver module like this in your .yml Codeception configuration file:
modules:
config:
WPWebDriver:
url: 'http://127.0.0.1:8888'
browser: phantomjs
The URL is important. I have found that attempting to use localhost instead of 127.0.0.1 will not work. Also, if you accidentally leave out the http://, that won't work either. You can use most any 8*** port, since most of them are open, but of course you'll need to have a web server running on that port to serve your static files or run your PHP application. The easiest way to do this, I find, is to use PHP's built-in webserver.
Your .travis.yml file might look something like this:
# Travis CI configuration file.
language: php
php:
- 5.6
- 7.0
before_script:
# Start up a web server.
- php -S 127.0.0.1:8888 -t /path/to/web/root >/dev/null 2>&1 &
# Start up the webdriver.
- phantomjs --webdriver=4444 >/dev/null 2>&1 &
# Install Codeception.
# Doing this last gives the webdriver and server time to start up.
- composer install --prefer-source
script:
- vendor/bin/codecept run
You will of course need to add Codeception to your project's composer.json file:
composer require --dev codeception/codeception
You'll also need to change path/to/web/root above to the path to the directory where you want the server's document root to be.
If you'd like to see a working demo running WebDriver tests against WordPress, you can check out this GitHub repo.
I'd think that it can be done, but gluing everything tohether is not going to be for the faint of heart. Reason why I think it can be done is that codeception, itself, is ci-ed on Travis. See https://travis-ci.org/Codeception/Codeception. I'd contact the people at codeception and ask for their thoughts.
Or you can take a peek at how they do it in the build logs, such as:
https://travis-ci.org/Codeception/Codeception/jobs/14432638
Looks like they're running headless with a downloaded standalone selenium server.
Travis-ci have some information on how to run GUI tests. In particular, they allow you to use a sauce labs account and run distributed selenium tests from there.
I ran into this problem today and I solved it by adding Codeception to my composer.json:
"require-dev": {
"codeception/codeception": "^2.1"
},
and referring to it on my .travis.yml:
install:
- composer self-update
- composer install
before_script:
- #Code that creates and seeds my database and so on
script: php vendor/codeception/codeception/codecept run

selenium standalone server log console is blank

My problem is: selenium standalone server log console shows blank
Steps to reproduce the problem:
Install selenium standalone-2.25.0.jar
Have test suites in same folder
Put in command to run the test through console
java -jar <location_of_selenium>/selenium-server-standalone-2.25.0.jar -htmlSuite *firefox <website URL> <location_of_selenium>/Test_suite1/Test_suite1_ts <location_of_selenium>/Test_suite1_results.html
Expected output:
There are errors in my test and it should show in the Selenium log console as well as create a log file in my selenium folder.
Platform and other Details:
Selenium version: 2.25.0
OS: Ubuntu
Browser: FireFox
Browser version: 7
Pls look how this problem I solved:
java -jar selenium-server-standalone-2.25.0.jar -htmlSuite *firefox http://libo.ru/ c:\temp\selenium\juzsay\testSute1.html testresult1.html
where:
website URL = http://libo.ru/
location_of_selenium = C:\temp\selenium\juzsay\testSute1.html (build with Selenium IDE)
location_of_selenium = testresult1.html (file with test result, in selenium-server-standalone folder)
Some remarks:
1) You should run cmd and open folder where your selenium-server-standalone-2.25.0.jar file located in my situation is C:\tools\selenium2.0\
2) for help also user this command -h
Test suite names need to have .html extension in order for the logs to work
side note: for selenium to run all your tests and not just 1, you must add .html to your test case files as well.
SOLUTION: Rename the suite file to Test_suite1_ts.html

I want to run Selenium test case file from command line

I made then saved a test case with the Firefox extension "Selenium IDE".
Now I want to use command line to run this exported html file.
I try to follow this how-do-i-launch-the-selenium-ide-from-the-command-line-with-a-specific-test-case but it doesn't work.
Please help me.
You will need the Selenium RC which you can get from:
http://seleniumhq.org/download/
And Java 1.5 or higher (Download Java here)
1) Install Java
2) Unpack Selenium RC.
3) Open a cmd.exe window and go to the directory containing the Selenium Server (selenium-remote-control-1.0.1\selenium-server-1.0.1)
4) Run the command below:
java -jar selenium-server.jar -htmlSuite "*firefox" "http://10.8.100.106" "C:\mytestsuite\mytestsuite.html" "C:\mytestsuite\results.html"
This should run your test suite in Firefox and write the results to the html file. Obviously you will need to change the "http://10.8.100.106" argument to your own server (this might just be localhost / 127.0.0.1)
It is possible to run individual test cases using Selenese Runner. You can specify a single test case file or a test suite as the unit to run.
We should execute the SeleniumRC in using following command;
java -jar filename.jar
ex:
java -jar program1.jar
the program1 consist of the followings are:
program1.class file
Resource library file such as SeleniumRC Server.jar and Selenium Java client.jar file
This method is applicable for SeleniumRC execution. We can directly create the program1.jar file from eclipse using
File->Export.
Here is an article that explains you step-by-step process of how to run Selenium RC application in Java.
Create a Java Selenium RC test script and executing the script
I have needed to do this before, and used the following:
An Ant Build (complex)
Creating a test runner class(a part of junit framework)class.
Most commonly we would run into build path errors while trying to run from cmd.
If you want to run it from command prompt you may consider writing your selenium test in python.
Make sure you have python installed if you are on windows. Mac will have python by default.
Running test from CMD is quite easy.
Follow below steps
1- Go to home directory and Set class path
Home Directory > set classpath=Home Directory\bin; and press enter
Home Directory > set classpath=Home Directory\lib*; and press enter
2-Home-directory > java org,testng.TestNG testng.xml testng2.xml testng2.xml and hit enter
I have documented all steps here. Hope it will help. Cheers
1) Running from CMD
java -cp "C:\ProjectX\Mortgage\bin;C:\Selenium_latest\selenium2.49.1\*;C:\Selenium_latest\selenium-2.49.1\libs\*" org.testng.TestNG C:\ProjectX\Mortgage\testng.xml
Run above command in C:\ProjectX\Mortgage
2) Create batch file name runner.bat
SET projectLocation=C:\ProjectX\Mortgage
CD %projectLocation%
SET classpath=%projectLocation%\bin;C:\Selenium_latest\selenium-2.49.1\*;C:\Selenium_latest\selenium-2.49.1\libs\*
java org.testng.TestNG %projectLocation%\testng.xml
PAUSE
3) Run the batch file by double clicking on it.
To be able to run in Chrome browser, you can use *chrome option instead of *firefox like below
java -jar selenium-server.jar -htmlSuite "*chrome" "http://localhost" "C:\testsuite\testsuite.html" "C:\testsuite\results.html"
Other browsers list include:
*firefox
*mock
*firefoxproxy
*pifirefox
*chrome
*iexploreproxy
*iexplore
*firefox3
*safariproxy
*googlechrome
*konqueror
*firefox2
*safari
*piiexplore
*firefoxchrome
*opera
*iehta
*custom
on session null