what I am trying to do is to set browser launched when calling cypress run having this environment variables:
"env": {
"TAGS": "#smoke",
"browser": "chrome"
For example when I am calling:
cypress run -- -e TAGS=$TAGS,-b $browser
cypress run -- -e TAGS=$TAGS,-b=$browser
cypress run -- -e TAGS=$TAGS,b $browser
Nothing happens and all the time the default Electron browser is called and the tags are working fine.
cypress run --browser chrome
Works fine but I want to do it via environment variables if possible, not sure if it is and if I am missing something.
Thank you for you attention and time helping out!
Related
I have an Nx project with Gitlab CI configured. My project has 2 kinds of E2E testing included - main project's and Storybook's ones. The tests themselves work fine, but I use cypress.io dashboard for E2E results gathering.
I configured the project to record them and it works, but I want them to be saved as one run, but with 2 groups x 2 browsers (Main/Storybook x Chrome/Firefox). Recording also works fine, cypress.io dashboard recognizes 2 browsers and groups correctly, but I only get results for one browser (there is one result for Chrome for Main and one for Firefox for Storybook - the second browsers are visible in filters, but results for them are empty - please, see the screenshots).
I tried some flags and configurations from https://docs.cypress.io/guides/guides/parallelization.html
Unfortunately, no matter what I try, there is always some kind of error like:
You passed the --parallel flag, but we do not parallelize tests across
different environments. This machine is sending different environment
parameters than the first machine that started this parallel run. The
existing run is: https://dashboard.cypress.io/projects/qwerty/runs/44
In order to run in parallel mode each machine must send identical
environment parameters such as:
specs
osName
osVersion
browserName
browserVersion (major) This machine sent the following parameters: { "osName": "linux", "osVersion": "Ubuntu - 20.04", "browserName":
"Firefox", "browserVersion": "80.0.1", "specs": [
"src/integration/app/app.spec.ts" ] } https://on.cypress.io/parallel-group-params-mismatch
Only one browser per group passes and the 2nd one gets the following error.
These are my scripts in package.json:
# 1st --parallel flag for nx and 2nd after -- for Cypress directly
# $CYPRESS_RECORD_KEY - key provided by cypress.io
# $CI_ID - <branch-name>-<commit-hash> - e.g. my-new-branch-qweqtwerwtreyzsxfc4123dxfv
"e2e:ci": "nx e2e main-e2e --prod --headless --parallel --record --key $CYPRESS_RECORD_KEY --ci-build-id $CI_ID --group MainWeb -- --parallel",
"e2e:ci:chrome": "yarn e2e:ci --browser=chrome",
"e2e:ci:firefox": "yarn e2e:ci --browser=firefox",
"e2e:storybook:ci": "nx e2e storybook-e2e --prod --headless --parallel --record --key $CYPRESS_RECORD_KEY --ci-build-id $CI_ID --group Storybook -- --parallel",
"e2e:storybook:ci:chrome": "yarn e2e:storybook:ci --browser=chrome",
"e2e:storybook:ci:firefox": "yarn e2e:storybook:ci --browser=firefox"
And here is my .gitlab-ci.yml (e2e stage part only):
E2E Main-Web - Chrome:
stage: e2e
script:
- yarn e2e:ci:chrome
E2E Main-Web - Firefox:
stage: e2e
script:
- yarn e2e:ci:firefox
E2E Storybook - Chrome:
stage: e2e
script:
- yarn e2e:storybook:ci:chrome
E2E Storybook - Firefox:
stage: e2e
script:
- yarn e2e:storybook:ci:firefox
I'm not sure what is wrong here. I tried so many configurations, flags, solutions (even parallelization on Gitlab), but always there is something wrong...
Of course I set cypress.io > Project Settings > Parallelization > Run Completion Delay for 60s.
I think the answer lies in the Cypress error you shared:
You passed the --parallel flag, but we do not parallelize tests across
different environments.
You are triggering two parallel runs, with each parallel run setting up two browsers. As the error says, this is not expected - each parallel run should have only one browser.
The solution in this case is easy: just configure four groups, one for each test and browser type.
This is how this could look:
"e2e:ci": "nx e2e main-e2e --prod --headless --parallel --record --key $CYPRESS_RECORD_KEY --ci-build-id $CI_ID -- --parallel",
"e2e:ci:chrome": "yarn e2e:ci --browser=chrome --group 'MainWeb Chrome' ",
"e2e:ci:firefox": "yarn e2e:ci --browser=firefox --group 'MainWeb Firefox' ",
"e2e:storybook:ci": "nx e2e storybook-e2e --prod --headless --parallel --record --key $CYPRESS_RECORD_KEY --ci-build-id $CI_ID -- --parallel",
"e2e:storybook:ci:chrome": "yarn e2e:storybook:ci --browser=chrome --group 'Storybook Chrome'",
"e2e:storybook:ci:firefox": "yarn e2e:storybook:ci --browser=firefox --group 'Storybook Firefox'"
I am planning to convert my testcafe console output to txt/html file with custom file name, I am using below line of script, but output file is not generating and no errors. I have installed the required report templates. Please guide me .
$ testcafe chrome tests/core/sample.test.js --reporter html:/file.html
$ testcafe chrome tests/core/sample.test.js --reporter list:/test.txt
Thanks
Ramesh D
console output to txt
Isn't this just easier?
$ testcafe chrome tests/core/sample.test.js > test_output.txt 2>&1
I mean you can use reporters, you can build your own reporter and spend hours on that, or if you really just need a file out of console output, this should be enough.
console output to html
I have a habit of using config files rather than command line options, so I'll show it on config files:
.testcaferc.json
{
"reporter": [
{
"name": "html",
"output": "Results/report.html"
}
]
}
First, of course, you need to install the appropriate npm package:
$ npm install --save-dev testcafe-reporter-html
If you insist on a command-line option, this testcafe docs should guide you. I believe this would be the right command:
$ testcafe chrome tests/core/sample.test.js -r html:Results/report.html
I am taking a swing at setting up a test suite for my company's web app. We use four environments at the time (Production, Regression, Staging, Development). I have environment variables setup in my cypress.json file but I would like to be able to switch my environment for example from regression to development and force cypress to change the baseURL to my new environment as well as point to a different cypress.json file that has development variables. The documentation around environments on cypress.io is a little confusing to me and I'm not sure where to start.
I have cypress running in different environments using package.json's scripts. You can pass in env vars before the cypress command. It would look something like:
"scripts": {
"cypress:open:dev": "CYPRESS_BASE_URL=http://localhost:3000 cypress open",
"cypress:open:prod": "CYPRESS_BASE_URL=http://mycompanydomain.com cypress open",
"cypress:run:dev": "CYPRESS_BASE_URL=http://localhost:3000 cypress run",
"cypress:run:prod": "CYPRESS_BASE_URL=http://mycompanydomain.com cypress run",
}
If you want to make 4 separate cypress.json files instead, you could have them all named according to environment and when you run an npm script that corresponds with that environment just copy it to be the main cypress.json when you run the tests.
Files:
./cypress.dev.json
./cypress.prod.json
./cypress.staging.json
./cypress.regression.json
npm scripts:
"scripts": {
"cypress:run:dev": "cp ./cypress.dev.json ./cypress.json; cypress run;"
}
Update:
I wrote this while cypress was still in beta. Using the config flag seems like a cleaner option:
https://docs.cypress.io/guides/guides/command-line.html#cypress-run
npm scripts:
"scripts": {
"cypress:run:dev": "cypress run -c cypress.dev.json;"
}
You can pass the config file to be used with --config-file param as:
Syntax:-
cypress open --config-file <config-file-name>
If you have different environment files then it should be as:
"scripts": {
"cypress:open:prod": "cypress open --config-file production-config.json",
"cypress:open:stag": "cypress open --config-file staging-config.json",
},
If you see above commands we are telling the cypress to use production-config.json file for prod environment and similarly staging-config.json for stag environment.
I'm using Codeception and I want to do Selenium tests.
Obviously, I need Selenium server running and I can spin it up with:
java -jar ./path/to/selenium/binary
However, I've used another testing framework in past which allowed me to specify this path in config file. Then, whenever I did something like codecept run, the framework automatically started selenium server and also shutted it down after all tests were completed.
Can I do this with Codeception? I tried to put exec() in _bootstrap file but it didn't work..
I'm using phantomjs - headless alternative to selenium. This extension starts phantomjs automatically with codecept run. Also phantomjs is faster than selenium.
You can write your own extension for running selenium - have a look at this file.
I start my tests with a bash script and then start and stop selenium via screen:
#!/usr/bin/env bash
# Start selenium in detached screen
screen -d -m -S "selenium" java -Dwebdriver.chrome.driver=./path/to/chromedriver -jar ./path/to/selenium/binary || true
./vendor/bin/codecept run -c codeception.yml --fail-fast --coverage --steps --coverage-html --html || error=true
# quit selenium
screen -S selenium -X quit || true
This might not fit the use-case but I can execute my code in my bash on Windows and Ubuntu as well as via my jenkins build process.
How can i run Ember-cli test in jenkins?
Currently to run the tests i added a build step "Execute shell" with the following in it:
ember test --silent --config-file ${WORKSPACE}/testem.json > ${WORKSPACE}/xunit-ember-dev.xml;
But it doesn't work, this is the output
<testsuite name="Testem Tests" tests="0" failures="0" timestamp="Thu Feb 12 2015 14:20:24 GMT+0100 (CET)" time="0">
</testsuite>
If I do the same manually in the workspace as jenkins user I got the expected results.
<testsuite name="Testem Tests" tests="70" failures="0" timestamp="Thu Feb 12 2015 15:06:40 GMT+0100 (CET)" time="15">
<testcase name="PhantomJS 1.9 Integration - Homepage: Should display the homepage"/>
<testcase name="PhantomJS 1.9 Integration - Profile: Should display the profile sections"/>
...
Every time I make Jenkins run the tests, he find no tests.
Thanks
Just add an Execute Shell step that runs:
npm run test > results.tap
This command tells npm to run a script called test and redirect the output to a file called results.tap. (The purpose of redirecting to results.tap is so you can take this file and hand it to Publish TAP Results post-build step and get pretty charts of test runs.)
In your package.json, you should have a block that looks like this:
"scripts": {
"start": "ember server",
"build": "ember build",
"test": "ember test"
},
(This is the default from ember-cli 1.13.8 and likely a few versions earlier than that.)
Using this approach, you don't need to have ember-cli globally installed on your build system: it will simply pick it up from the project's node_modules folder.
Caveat emptor: you'll likely find some blogs/forum posts that tell you to run npm run test | tee results.tap. This will run the tests, however, it will eat the return code. (If the test run fails, the npm process exits with a return value of 1, but since you're piping output to a second command (tee), the return code of that command is what jenkins sees. The net result is that jenkins will interpret test failures as success and keep on running build steps.)