TestCafe unable to use testController (t) outside of test run (e.g. as a conditional to skip a test) - testing

I'm trying to check which browser we're running tests on, and then skip a test/fixture based on the result (as mentioned in this TestCafe Issue).
import { t } from 'testcafe';
fixture `test`
.page('https://testcafe.devexpress.com')
if (t.browser.name.includes('Chrome')) {
test('is Chrome?', async () => {
console.log(t.browser.name);
await t.expect(t.browser.name.includes('Chrome').ok();
});
} else {
test.skip('is Chrome?')
};
Results in...
ERROR Cannot prepare tests due to an error.
Cannot implicitly resolve the test run in the context of which the test controller action should be executed. Use test function's 't' argument instead.
Is there any way I can call the testObject (t) outside of the test?

I don't have a solution to exactly your question. But I think it's better to do it slightly differently, so the outcome will be the same, but the means to achieve it will differ a bit. Let me explain.
Wrapping test cases in if statements is, in my opinion, not a good idea. It mostly clutters test files so you don't only see test or fixture at the left side, but also if statements that make you stop when reading such files. It presents more complexity when you just want to scan a test file quickly from top to bottom.
The solution could be you introduce meta data to your test cases (could work well with fixtures as well).
test
.meta({
author: 'pavelsaman',
creationDate: '16/12/2020',
browser: 'chrome'
})
('Test for Chrome', async t => {
// test steps
});
Then you can execute only tests for Chrome like so:
$ testcafe --test-meta browser=chrome chrome
That's very much the same as what you wanted to achieve with the condition, but the code is a bit more readable.
In case you want to execute tests for both chrome and firefox, you can execute more commands:
$ testcafe --test-meta browser=chrome chrome
$ testcafe --test-meta browser=firefox firefox
or:
$ testcafe --test-meta browser=chrome chrome && testcafe --test-meta browser=firefox firefox
If your tests are in a pipeline, it would probably be done in two steps.

The better solution, as mentioned in one of the comments in this question is to use the runner object in run your tests instead of the command line. Instead of passing the browser(s) as a CLI argument, you would pass it as an optional argument to a top-level script.
You would then read the browser variable from either the script parameter or the .testcaferc.json file.
You would need to tag all tests/fixtures with the browser(s) they apply to using meta data.
You then use the Runner.filter method to add a delegate that returns true if the browser in the meta data is equal to the browser variable in the top level script
var runner = testcafe.createRunner();
var browser = process.env.npm_package_config_browser || require("./testcaferc.json").browser;
var runner.filter((testName, fixtureName, fixturePath, testMeta, fixtureMeta) => {
return fixtureMeta.browser === browser || testMeta.browser === browser ;
}

Related

Using tags ( Smoke, regression) with TestCafe

Using testcafe grep patterns would partially solve our problem of using tags but it would still display those tags on the spec report ...!!!
Is there a way to include tags in the test/fixture names and use grep patterns but skip those tags to be displayed in the execution report ??
import { Selector } from 'testcafe';
fixture `Getting Started`
.page `http://devexpress.github.io/testcafe/example`;
test('My first test --tags {smoke, regression}', async t => {
// Test code
});
test('My Second test --tags {smoke}', async t => {
// Test code
});
test('My first test --tags {regression}', async t => {
// Test code
});
testcafe chrome test.js -F "smoke"
The above snippet would trigger the smoke only tests for me though but the report will display the test names along with those tags
Is there an alternative way to deal with tags or a solution to not display the tags in the test execution report?
It appears in a recent release (v0.23.1) of testcafe you can now filter with metadata via the commandline.
You can now run only those tests or fixtures whose metadata contains a specific set of values. Use the --test-meta and --fixture-meta flags to specify these values.
testcafe chrome my-tests --test-meta device=mobile,env=production
or
testcafe chrome my-tests --fixture-meta subsystem=payments,type=regression
Read more at https://devexpress.github.io/testcafe/blog/testcafe-v0-23-1-released.html
I think the best solution in this case is to use test/fixture metadata. Please refer the following article: http://devexpress.github.io/testcafe/documentation/test-api/test-code-structure.html#specifying-testing-metadata
For now, you can't filter by metadata, but this feature is in the pull request: https://github.com/DevExpress/testcafe/pull/2841. So, after this PR is merged, you will be able to add any metadata to tests and filter by this metadata in a command line.

Leadfoot + sauce: mapping a collection of elements using getAttr fails in mobile only

My use case varies for this, but in general i'm trying to collect a bunch of elements and then apply _.map() to each. The problem is that this series of .getAttribute() calls can cause a test that works locally to fail against a remote server like sauce/android.
One example: collecting all <div class='article'><a href='articles/{id}'> on a page and then getting the hrefs. it might look something like this, and this approach will work until i test on a mobile (android) sauce environment. then I get a timeout.
Is it possible this is an issue related to my android environment capabilities? To piling up so many requests? I've tried scaling my test down from using 75 articles to only 45 and i've upped the timeout to 60s and still the mobile test fails. locally with chromedriver is fine, chrome desktop + sauce is fine.
Not my actual test but an approximation of the code i'm talking about:
/// ... return this.remote
.findAllByTagName('div.article a')
.then(function (articles) {
var promises = articles.map(function(article) {
return article.getAttribute('href');
});
Promise.all(promises)
.then(function (hrefs) {
uniques = _.uniq(hrefs);
assert(hrefs.length === uniques.length);
});
});
Since you're seeing a timeout error, I'd suggest continuing to increase the test timeout until the test passes. The mobile testing environments on Sauce are both slower to initialize and slower to operate than the desktop environments, so it's quite possible that a test with many requests is simply very slow.
One way to speed things up would be to use an execute block to gather the references, like:
.then(function (articles) {
return this.parent.execute(function (articles) {
return articles.map(function (node) {
return node.getAttribute('href');
});
}, [ articles ]);
})
In the above snippet, the articles element array is passed as an argument to the execute block. The remote WebDriver will deserialize the element references into actual DOM elements that can be operated on in the execute code. This is significantly more efficient than using individual getAttribute requests for each element since only a single request will be made to the remote browser.

Protractor - How to separate each test to one file and separate variabiles

I have some komplex protractor test written but everything is in one file.
Where I'm on top of it loading all variabiles like:
var userLogin = "John";
and after that somewhere in code I use it together.
What I need to do is
1. Separate all variabiles to aditional file (some config file)
2. Each test to one file
1- I try to make config.js where I add all variabiles and i required it in protractor.conf.js it load correctly problem is that when i use any of this variabiles in some test it's not working (test fail with "userName is not defined")
I know there is a way where i requre config.file in each test script but that's really not best option in my eyes.
2- How can I know what I did in last script if it's separate, like for example how to know I am logged in?
Thanks.
There are multiple things you can make use of.
2) How can I know what I did in last script if it's separate, like for example how to know I am logged in?
This is where beforeEach(), afterEach() can help:
To help a test suite DRY up any duplicated setup and teardown code,
Jasmine provides the global beforeEach and afterEach functions. As the
name implies, the beforeEach function is called once before each spec
in the describe is run, and the afterEach function is called once
after each spec.
There are also beforeAll(), afterAll() available in jasmine 2, or via jasmine-beforeAll third-party for jasmine 1:
The beforeAll function is called only once before all the specs in
describe are run, and the afterAll function is called after all specs
finish. These functions can be used to speed up test suites with
expensive setup and teardown.
1) I try to make config.js where I add all variabiles and i required
it in protractor.conf.js it load correctly problem is that when i use
any of this variabiles in some test it's not working (test fail with
"userName is not defined") I know there is a way where i requre
config.file in each test script but that's really not best option in
my eyes.
One option which I've personally used would be to create a config.js file with all the reusable configuration variables you would need in multiple tests and require the file once - in the protractor config - then set it as a params configuration key value:
var config = require("./config.js");
exports.config = {
...
params: config,
...
};
where config.js is, for example:
var config;
config = {
user: {
login: "user",
password: "password"
}
};
module.exports = config;
Then, you would not need to require config.js in every test, but instead, you'll use browser.params. For example:
expect(browser.params.user.login).toEqual("user");
Also, if you need some sort of a global test preparation step, you can do it in onPrepare() function, see Setting Up the System Under Test. Example configuration that performs a "global" login step is available here.
And an another quick note: you can have custom globally defined variables (like built-in browser or protractor), set them using global in onPrepare. For example, I've defined protractor.ExpectedConditions as a custom global variable:
onPrepare: function () {
global.EC = protractor.ExpectedConditions;
}
Then, in tests, don't require anything, `EC variable would be available in the scope, e.g.:
browser.wait(EC.invisibilityOf(scope.page.dropdown), 5000)
Also, organizing your tests using "Page Object Pattern" would also help to solve the reusability and modularity problem.

Access window object / browser scope from protractor

I'm running tests with protractor, but it seems impossible to access the JS 'window' object. I even tried adding a tag in my html file that would contain something like
var a = window.location;
and then try expect(a) but I couldn't make it work, I always get undefined references...
How should I process to access variables that are in the browser scope ?
Assuming you are using a recent version of Protractor, let's say >= 1.1.0, hopefully >= 1.3.1
Attempting to access Browser side JS code directly from Protractor won't work because Protractor runs in NodeJS and every Browser side code is executed through Selenium JsonWireProtocol.
Without further detail, a working example:
browser.get('https://angularjs.org/');
One-liner promise that, as of today, resolves to '1.3.0-rc.3'
browser.executeScript('return window.angular.version.full;');
You can use it directly in an expect statement given Protractor's expect resolves promises for you:
expect(browser.executeScript('return window.angular.version.full;')).
toEqual('1.3.0-rc.3');
Longer example passing a function instead of a string plus without expect resolving the promise for you. i.e. for more control and for doing some fancy thing with the result.
browser.driver.executeScript(function() {
return window.angular.version.full;
}).then(function(result) {
console.log('NodeJS-side console log result: ' + result);
//=> NodeJS-side console log result: 1.3.0-rc.3
});

Asynchronous execution of test cases in JS test driver

We are having CI build servers where different JS TestCases are to be executed parallely (asynchronously) on the same instance of JS test driver server. The test cases are independent of each other and should not interfere with each other. Is this possible using JS test driver?
Currently, I have written two test cases, just to check if this is possible:
Test Case 1:
GreeterTest = TestCase("GreeterTest");
GreeterTest.prototype.testGreet = function() {
var greeter = new myapp.Greeter();
alert(“just hang in there”);
assertEquals("Hello World!", greeter.greet("World"));
};
Test Case2:
GreeterTest = TestCase("GreeterTest");
GreeterTest.prototype.testGreet = function() {
var greeter = new myapp.Greeter();
assertEquals("Hello World!", greeter.greet("World"));
};
I have added the alert statement in test case 1 to make sure that it hangs there.
I have started the JS test driver server with the following command:
java --jar JsTestDriver-1.3.5.jar --port 9877 --browser <path to browser exe>
I am starting the execution of both the test cases as follows:
java -jar JsTestDriver-1.3.5.jar --tests all --server http://localhost:9877
The Test Case 1 executes and hangs at alert statement. Test Case 2 fails with an exception (BrowserPanicException). The conf file is proper, as the second test case passes if executed by itself.
Is there any configuration changes required to make the second test case pass, while the first test case is still executing?
This issue is caused by the "app" (the jsTestDriver slave, running the tests) cannot really run them in parallel. This is because it's written in javascript, which is single threaded.
The implementation probably loops all tests and runs them one by one, thus, when an alert pops, the entire "app" is blocked, and cannot even report back to the jsTestDriver server, resulting in a timeout manifested in a BrowserPanicException.
Writing Async Tests won't help since the entire "app" is stuck.