I would like to run my protractor test on different environments such as
testing it on local environment,
testing it on test environment,
testing it on production
environment and so on at the same time and using the same browser example chrome.
So in this case my base URL would change for every environment: When I run the test I would like to run it parallel on all the different environments.
baseUrl:'localhost:8080'
baseUrl:'tst.company.com'
baseUrl:'prod.company.com'
etc
and browser remains the same
multiCapabilities:[
{ 'browsername':'chrome',
'chromeOptions':{
'binary': 'drive:pathToChrome',
'args':'[]'
'extensions':[]
}
}]
Any one knows how to, on this cases.
Thanks
I would approach this with a task manager: grunt and grunt-parallel.
Create 3 separate grunt task configurations with different baseUrl settings (you would need grunt-protractor-runner package installed).
Related
I succeeded to run cucumber scenario's in parallel but only on one browsertype(chrome or firefox). So first I run my tests on chrome. When tests finish I start a second test run on firefox.
Is it possible to run cucumber scenarios in parallel on different browsertypes at same time?
See cucumber bdd documentation how to achieve parallel execution of scenarios at https://cucumber.io/docs/guides/parallel-execution/
I use testNG as testrunner!
Thanks a lot for your responses!
You are running a your tests against a matrix of browsers. Typically this matrix configured in CI and provided to the test execution via environment variables. For example using Gitlab CI Matrix:
test:
stage: test
script:
- mvn test
parallel:
matrix:
- OS: Windows
OS_VERSION: 10
BROWSER: [Chrome, Firefox, Edge]
- OS: OS X
OS_VERSION: Big Sur
BROWSER: [Chrome, Firefox, Edge, Safari]
You then create the web driver in the before hook using the environment variables.
#Before
public void before(Scenario scenario){
String os = System.getenv("OS");
String osVersion = System.getenv("OS_VERSION");
String browser = System.getenv("BROWSER");
driver = createDriver(os, osVersion, browser);
}
You could also use Maven Profiles or Gradle Tasks to define these different sets of environment variables.
However key is to let these jobs in parallel on your CI system by starting multiple JVMs rather then only in Cucumber by starting multiple threads.
See the following answer Is it posible to run feature cucumber in parallel in different browser
This link at https://github.com/prashant-ramcharan/courgette-jvm-selenium explains how to achieve by using courgette-jvm(extension on cucumber-jvm).
How can I maintain separate data files for different environments such as development, staging and production environment in Behat with Mink Extension.
Example : login credentials, order numbers , shipping address etc?
You can use different configurations in different environments in several different ways (each detailed here):
You can use a different behat config file per environment and use them with the --config flag. For example, behat --config dev-config.yml.
You can use environment variables to set parameters that differ, for example (taken from the documentation): export BEHAT_PARAMS='{"extensions" : {"Behat\\MinkExtension" : {"base_url" : "https://www.example.com/"}}}'
Ideally, you would not run behat in production. Tests like these should confirm behavior in development and staging. By them time your code is in production, these behaviors should be well established.
As I am implementing an automated way to GUI test our webapplication with selenium I ran into some issues.
I am using selenese-runner to execute our Selenium test suites, created with Selenium IDE as a post build action in Jenkins.
This works perfeclty fine, as the build fails when something is wrong, and the build succeeds if all tests are passed. And the results are stored on a per build basis as HTML files, generated be selenese-runner.
My problem is however, that I seem to be unable to find a way, how to display these results in the respective jenkins build.
Does anyone have an idea how to solve this issue. Or maybe I am on the wrong path at all?
Your help is highly appreciated!
I believe the JUnit plugin should do what you want, but it doesn't work for me.
My config uses this shell script to run the tests (you can see the names of all my test suites):
/usr/bin/Xvfb &
export DISPLAY=localhost:0.0
cd ${WORKSPACE}
java -jar ./test/selenium/bin/selenese-runner.jar --baseurl http://${testenvironment} --screenshot-on-fail ./seleniumResults/ --html-result ./seleniumResults/ ./test/selenium/Search_TestSuite.html ./test/selenium/Admin_RegisteredUser_Suite.html ./test/selenium/Admin_InternalUser_Suite.html ./test/selenium/PortfolioAgency_Suite.html ./test/selenium/FOAdmin_Suite.html ./test/selenium/PublicWebsite_Suite.html ./test/selenium/SystemAdmin_Content_Suite.html ./test/selenium/SystemAdmin_MetaData_Suite.html
killall Xvfb
And I can see the result of the most recent test (you can see the name of my jenkins task folder)
http://<JENKINS.MY.COMPANY>/job/seleniumRegressionTest/ws/seleniumResults/index.html
Earlier tests are all saved on the Jenkins server, so I can view them if I need to.
I'm creating my tests (though I'm a beginner, learning) using Codeception. This includes acceptance and unit tests for now.
I want to add my repo to Travis CI so I can automate testing process after each commit and put build-status tag.
I would like to ask;
Can Travis-CI run codeception tests?
Can Travis-CI run codeception acceptance tests emulating browser?
If both answers are no, is there any other CI tool which can?
Thank you.
Yes, it is possible to run Codeception tests, including acceptance tests that run using WebDriver, on Travis CI.
It is possible to run your tests with a real browser on Travis, but it is easiest to use a headless browser, since Travis is running on a headless machine. PhantomJS is perfect for this, and it comes pre-installed with Travis CI's build bootstrap.
To run the tests with PhantomJS, you'll need to configure the WebDriver module like this in your .yml Codeception configuration file:
modules:
config:
WPWebDriver:
url: 'http://127.0.0.1:8888'
browser: phantomjs
The URL is important. I have found that attempting to use localhost instead of 127.0.0.1 will not work. Also, if you accidentally leave out the http://, that won't work either. You can use most any 8*** port, since most of them are open, but of course you'll need to have a web server running on that port to serve your static files or run your PHP application. The easiest way to do this, I find, is to use PHP's built-in webserver.
Your .travis.yml file might look something like this:
# Travis CI configuration file.
language: php
php:
- 5.6
- 7.0
before_script:
# Start up a web server.
- php -S 127.0.0.1:8888 -t /path/to/web/root >/dev/null 2>&1 &
# Start up the webdriver.
- phantomjs --webdriver=4444 >/dev/null 2>&1 &
# Install Codeception.
# Doing this last gives the webdriver and server time to start up.
- composer install --prefer-source
script:
- vendor/bin/codecept run
You will of course need to add Codeception to your project's composer.json file:
composer require --dev codeception/codeception
You'll also need to change path/to/web/root above to the path to the directory where you want the server's document root to be.
If you'd like to see a working demo running WebDriver tests against WordPress, you can check out this GitHub repo.
I'd think that it can be done, but gluing everything tohether is not going to be for the faint of heart. Reason why I think it can be done is that codeception, itself, is ci-ed on Travis. See https://travis-ci.org/Codeception/Codeception. I'd contact the people at codeception and ask for their thoughts.
Or you can take a peek at how they do it in the build logs, such as:
https://travis-ci.org/Codeception/Codeception/jobs/14432638
Looks like they're running headless with a downloaded standalone selenium server.
Travis-ci have some information on how to run GUI tests. In particular, they allow you to use a sauce labs account and run distributed selenium tests from there.
I ran into this problem today and I solved it by adding Codeception to my composer.json:
"require-dev": {
"codeception/codeception": "^2.1"
},
and referring to it on my .travis.yml:
install:
- composer self-update
- composer install
before_script:
- #Code that creates and seeds my database and so on
script: php vendor/codeception/codeception/codecept run
I'm having a hard time trying to grasp some concepts on selenium Grid/RC. What I need is to provide specific environments (ie6-on-xp, ie7-on-xp, etc) to the tests. For what I've been reading, the browser line in grid_configuration.yml do not make any reference of what version of MSIE or Firefox I'm running. So I can't get my head around in which form I can tell Grid/RC that I want some specific browsers and the path to run them (how RC knows which exe to run?)
Second, I'd like to run portable versions of those browsers. I've only seen that specified in the tests, and not in the RC's command line to run them. That is the way to do it, per test?
I will answer your question by breaking up the info that you need
What I need is to provide specific
environments (ie6-on-xp, ie7-on-xp,
etc) to the tests.
Well since you can't have multiple IE instances on the same machine, I know there are apps that allow you to do that but in my experience they cause more issues than solving them. Ideally you want different machines to run the tests. By doing this you are also setting up a selenium farm for your devs to use because they can target a test at a specific instance. So setting up Grid as an Infrastructure is a good step.
For what I've been reading, the
browser line in grid_configuration.yml
do not make any reference of what
version of MSIE or Firefox I'm
running. So I can't get my head around
in which form I can tell Grid/RC that
I want some specific browsers and the
path to run them (how RC knows which
exe to run?)
The YAML just lets you know what the grid can handle. When starting up the grid you need to make sure that you pass in similar configurations. Think of Se:GRID like you would Se:RC except you don't care where the RC server is because you speak to a central place that works the rest out for you.
If you need it to run tests against a specific items then you will need to handle this in your test setup. There is a common misconception that all tests will run the same in every single browser. This will happen if you never rely on XPath or CSS selectors in your tests because browsers always handle this slightly differently and the slight differences can lead to flaky tests which should always be avoided.
One way to specify which browser to use for a test is to have a central configuration file. In C# this would be using the app.config that has a collection for each browser and doing
Config
<Firefox>
<addKey browserVersion='3.5.6' OS='WindowsXP'>
</Firefox>
Central Config Class looking inside 1 element
public class BoothElement : ConfigurationElement
{
[ConfigurationProperty("browserVersion", DefaultValue = "", IsKey = true, IsRequired = true)]
public string browserVersion
{
get
{
return ((string)(base["browserVersion"]));
}
set
{
base["browserVersion"] = value;
}
}
Tests
selenium = new DefaultSelenium(HubPort, HubPort, browserVersion, SUTServer);
selenium.Open("/test.htm");
//Rest of the test
In python you could create an Array in a module that you include on all your tests
include.py
hubServer = 'hub'
hubPort = 5555
sut = 'http://serverUnderTest'
firefox = [hubServer,hubPort,"\*chrome",sut]
iexplore = [hubServer,hubPort,"\*iehta",sut]
test.py
sel = selenium(firefox)
sel.open("/test.html")
#rest of the test
When using Selenium Grid try thinking of it more as a test infrastructure help framework and hopefully that will help you a little more.
Second, I'd like to run portable
versions of those browsers. I've only
seen that specified in the tests, and
not in the RC's command line to run
them. That is the way to do it, per
test?
I have never tried to get Selenium to work on mobile browsers and don't think it would work to well, however with Selenium 2 which is currently in alpha there is android support for testing apps.
EDIT FROM COMMENT
- name: "Firefox on OS X"
browser: "*firefox"
- name: "Firefox on Linux"
browser: "*firefox"
- name: "IE on Windows"
browser: "*iehta"
- name: "Safari on OS X"
browser: "*safari"
So say we have the above setup, according to the YAML file we have a number of different *firefox instances. So to call those different ones in our tests our browser setup command would look like
selenium.Start(hubHost, hubPort, "Firefox on Linux", "http://serverUnderTest");
or selenium.Start(hubHost, hubPort, "Firefox on OS X", "http://serverUnderTest");
The hub will translate that into *firefox for you. I prefer having very granular names for my environment instead of the usual *firefox so that if there is a failure its easier to spot where it was and on which specific browser.
Virtual machines can be very handy for setting up "inexpensive" mules in the Selenium Grid farm.