We have around 400+ TestCafe js files and we are running these tests nightly through our CI/CD pipeline. Our problem right now is that, it consumes a lot of hours before it completes the test. What we are planning to do is to split up those tests maybe at least 4. Now my question is: How can we run the TestCafe in 4 instances with each instance runs the split tests?
For example:
Instance 1 - runs Tests #1 - #100
Instance 2 - runs Tests #101 - #200
Instance 3 - runs Tests #201 - #300
Instance 4 - runs Tests #301 - #400
You can use fixture or test metadata and hen execute a different command in each instance.
For example you use fixture metadata:
fixture `My fixture`
.meta('instance', '1')
// and somewhere else:
fixture `My fixture`
.meta('instance', '2')
then to run only tests within fixtures with a specific metadata:
$ testcafe chrome my-tests --fixture-meta instance=1
Or you can just group those fixture files in different directories and then run tests from only a specific directory:
$ testcafe chrome ./Tests/instance-one/*.js
I think I've found a solution to my own problem. I've installed this package called npm-run-all
I added the split tests in the package.json
"test:split1": "testcafe chromium src-compact/SplitTest1.js --skip-js-errors",
"test:split2": "testcafe chromium src-compact/SplitTest2.js --skip-js-errors",
"test:parallel": "npm-run-all -p test:split1 test:split2"
And then execute it by running this command:
npm run test:parallel
Related
I would like to see a better way to run tests sequentially in Testcafe one browser, I have more than 30 tests.
Currently I'm using numbers for each test:
List item
test1register.ts,
test2login.ts,
Just pass the directory in command line, it will run tests sequentially.
testcafe chrome ./tests/
I have a .side file generated by the Selenium IDE, which I need to run on CI using Jenkins.
I am running it as a build step with the following shell command:
selenium-side-runner /path/to/file.ide
The problem arises due to the fact that no matter if the selenium test fails, Jenkins always shows is as success.
In this thread it's suggested to upload the file as generic, but still, the commands to execute it are missing
How to upload a generic file into a Jenkins job?
I've found a possible solution to it on this posts, but I would appreciate having a cleaner way to solve this instead of parsing the results checking for errors.
How to mark a build unstable in Jenkins when running shell scripts
Is there a plugin able to run selenium .side files on Jenkins and this one showing the success/failures of the test?
You can generate a Junit test report file and then use the Jenkins Junit plugin after your tests execution.
selenium-side-runner --output-directory=results --output-format=junit
# Outputs results in `junit` frormat in `./results/projectName.xml'
Check the official documentation for more details.
I configured my tests to run in parallel:
In my build.gradle:
test{
maxParallelForks 2
forkEvery 0
}
./gradlew build --max-workers=8 --parallel
But additionally I would like to know which executor is running particular test.
Is it possible to log such information?
I've tried to use --debug option but I doesn't appear there.
the worker id is stored as system property and you can access it via within your test named System.getProperty('org.gradle.test.worker') from within your test.
I would like to run my protractor test on different environments such as
testing it on local environment,
testing it on test environment,
testing it on production
environment and so on at the same time and using the same browser example chrome.
So in this case my base URL would change for every environment: When I run the test I would like to run it parallel on all the different environments.
baseUrl:'localhost:8080'
baseUrl:'tst.company.com'
baseUrl:'prod.company.com'
etc
and browser remains the same
multiCapabilities:[
{ 'browsername':'chrome',
'chromeOptions':{
'binary': 'drive:pathToChrome',
'args':'[]'
'extensions':[]
}
}]
Any one knows how to, on this cases.
Thanks
I would approach this with a task manager: grunt and grunt-parallel.
Create 3 separate grunt task configurations with different baseUrl settings (you would need grunt-protractor-runner package installed).
I'm creating my tests (though I'm a beginner, learning) using Codeception. This includes acceptance and unit tests for now.
I want to add my repo to Travis CI so I can automate testing process after each commit and put build-status tag.
I would like to ask;
Can Travis-CI run codeception tests?
Can Travis-CI run codeception acceptance tests emulating browser?
If both answers are no, is there any other CI tool which can?
Thank you.
Yes, it is possible to run Codeception tests, including acceptance tests that run using WebDriver, on Travis CI.
It is possible to run your tests with a real browser on Travis, but it is easiest to use a headless browser, since Travis is running on a headless machine. PhantomJS is perfect for this, and it comes pre-installed with Travis CI's build bootstrap.
To run the tests with PhantomJS, you'll need to configure the WebDriver module like this in your .yml Codeception configuration file:
modules:
config:
WPWebDriver:
url: 'http://127.0.0.1:8888'
browser: phantomjs
The URL is important. I have found that attempting to use localhost instead of 127.0.0.1 will not work. Also, if you accidentally leave out the http://, that won't work either. You can use most any 8*** port, since most of them are open, but of course you'll need to have a web server running on that port to serve your static files or run your PHP application. The easiest way to do this, I find, is to use PHP's built-in webserver.
Your .travis.yml file might look something like this:
# Travis CI configuration file.
language: php
php:
- 5.6
- 7.0
before_script:
# Start up a web server.
- php -S 127.0.0.1:8888 -t /path/to/web/root >/dev/null 2>&1 &
# Start up the webdriver.
- phantomjs --webdriver=4444 >/dev/null 2>&1 &
# Install Codeception.
# Doing this last gives the webdriver and server time to start up.
- composer install --prefer-source
script:
- vendor/bin/codecept run
You will of course need to add Codeception to your project's composer.json file:
composer require --dev codeception/codeception
You'll also need to change path/to/web/root above to the path to the directory where you want the server's document root to be.
If you'd like to see a working demo running WebDriver tests against WordPress, you can check out this GitHub repo.
I'd think that it can be done, but gluing everything tohether is not going to be for the faint of heart. Reason why I think it can be done is that codeception, itself, is ci-ed on Travis. See https://travis-ci.org/Codeception/Codeception. I'd contact the people at codeception and ask for their thoughts.
Or you can take a peek at how they do it in the build logs, such as:
https://travis-ci.org/Codeception/Codeception/jobs/14432638
Looks like they're running headless with a downloaded standalone selenium server.
Travis-ci have some information on how to run GUI tests. In particular, they allow you to use a sauce labs account and run distributed selenium tests from there.
I ran into this problem today and I solved it by adding Codeception to my composer.json:
"require-dev": {
"codeception/codeception": "^2.1"
},
and referring to it on my .travis.yml:
install:
- composer self-update
- composer install
before_script:
- #Code that creates and seeds my database and so on
script: php vendor/codeception/codeception/codecept run