Chai-As-Promised is eating assertion errors - selenium

I'm using chai-as-promised + mocha for writing some selenium-webdriver tests. Since webdriver extensively uses promises, I imagined it would be better if I used chai-as-promised for those type of tests.
The problem is that when the tests fail, the error is not being caught properly by mocha, and it just fails without outputting anything.
Sample code:
it 'tests log', (next) ->
log = #client.findElement(By.css("..."))
Q.all([
expect(log.getText()).to.eventually.equal("My Text")
expect(log.findElement(By.css(".image")).getAttribute('src'))
.to.eventually.equal("/test.png")
]).should.notify(next)
According to documented behaviour, chai-as-promised should pass the errors along to mocha when the expectation fails. Right?
As a variation,
I've also tried these, but to no avail:
#2
# same, no error on failure
it 'tests log', (next) ->
log = #client.findElement(By.css("..."))
Q.all([
expect(log.getText()).to.eventually.equal("My Text")
expect(log.findElement(By.css(".image")).getAttribute('src'))
.to.eventually.equal("/test.png")
]).should.notify(next)
#3
# same, no error shown on failure
it 'tests log', (next) ->
log = #client.findElement(By.css("..."))
expect(log.getText()).to.eventually.equal("My Text")
.then ->
expect(log.findElement(By.css(".image")).getAttribute('src'))
.to.eventually.equal("/test.png").should.notify(next)
#4
## DOES NOT EVEN PASS
it 'tests log', (next) ->
log = #client.findElement(By.css("..."))
Q.all([
expect(log.getText()).to.eventually.equal("My Text")
expect(log.findElement(By.css(".image")).getAttribute('src'))
.to.eventually.equal("/test.png")
])
.then ->
next()
.fail (err) ->
console.log(err)
.done()

I think you're having two different issues:
First, you need to return a promise at the end of your tests (e.g. Q.all([...]); in your first test should be return Q.all([...]).
Second, I found that chai-as-promised doesn't seem to work unless used with mocha-as-promised.
Here's an example of mocha-as-promised with two assertions that will cause the test to fail.
/* jshint node:true */
/* global describe:false */
/* global it:false */
'use strict';
var Q = require('q');
require('mocha-as-promised')();
var chai = require('chai');
var chaiAsPromised = require('chai-as-promised');
chai.use(chaiAsPromised);
var expect = chai.expect;
describe('the test suite', function() {
it('uses promises properly', function() {
var defer = Q.defer();
setTimeout(function() {
defer.resolve(2);
}, 1000);
return Q.all([
expect(0).equals(0),
// the next line will fail
expect(1).equals(0),
expect(defer.promise).to.eventually.equal(2),
// the next line will fail
expect(defer.promise).to.eventually.equal(0)
]);
});
});

Related

Truffle Chai Assertion Error with Truffle Unit Cases

Problem: facing an issue with the below Truffle test case while trying to Implement the ERC20 token in my contract.
contract("Token Test", async(accounts) => {
const[deployerAccount, recipient, anotherAccount] = accounts;
it("All tokens should be in deployer's account", async() => {
let instance = await Token.deployed();
let totalSupply = await instance.totalSupply();
// let balance = await instance.balanceOf(accounts[0]);
// assert.equal(balance.valueOf(), totalSupply.valueOf(), "The Balance Was not same");
expect(instance.balanceOf(deployerAccount)).to.eventually.be.a.bignumber.equal(totalSupply);
});
it("Is possible to send tokens between accounts", async() => {
const sendToken = 1;
let instance = await Token.deployed();
let totalSupply = await instance.totalSupply();
expect(instance.balanceOf(deployerAccount)).to.eventually.be.a.bignumber.equal(totalSupply);
expect(instance.transfer(recipient, sendToken)).to.eventually.be.fulfilled;
expect(instance.balanceOf(deployerAccount)).to.eventually.be.a.bignumber.equal(totalSupply.sub(new BN(sendToken)));
expect(instance.balanceOf(recipient)).to.eventually.be.a.bignumber.equal(new BN(sendToken));
});
it("Should not be able to send more token than available with owner", async() => {
let instance = await Token.deployed();
let balanceOfDeployer = await instance.balanceOf(deployerAccount);
console.log(balanceOfDeployer+ " : " + await instance.balanceOf(deployerAccount));
// expect(instance.transfer(recipient, new BN(balanceOfDeployer+3))).to.eventually.be.rejected;
expect(instance.balanceOf(deployerAccount)).to.eventually.be.a.bignumber.equal(balanceOfDeployer);
});
});
Description: When I am trying to execute test cases 2 and 3 at the same time it is failing with the below error:
"before all" hook: prepare suite for "Should not be able to send more token than available with owner":
Uncaught AssertionError: expected '1000000000' to equal '999999999'
+ expected - actual
-1000000000
+999999999
at /project_dir/node_modules/chai-as-promised/lib/chai-as-promised.js:302:22
at processTicksAndRejections (internal/process/task_queues.js:95:5)
UnhandledRejections detected
Promise {
<rejected> AssertionError: expected '1000000000' to equal '999999999'
at /project_dir/node_modules/chai-as-promised/lib/chai-as-promised.js:302:22
at processTicksAndRejections (internal/process/task_queues.js:95:5) {
showDiff: true,
actual: '1000000000',
expected: '999999999',
operator: 'strictEqual',
uncaught: true
}
However when I try to execute both test cases independently (commenting test case 2 keeping 3, and vice-versa). They work fine and are completed without any error.
Need help understanding what am I missing here and how to fix it.
Things I have tried:
Initially, I thought that this is might be happening due to state change in the variable in test case 2. So I moved test case 3 to a new contract(). But I was still facing the same issue. However, this should not be happening as contract() provides a clean room environment before executing every contract test.
P.S.: I am initiating the value of totalSupply to 1000000000 in the constructor of Contract.
Added await before every expect() where I was trying to communicate with the contract. This helped in resolving the issue.

How can i use soft assertion in Cypress

I have configured soft assert from npm (npm i soft-assert) and now my package.josn has "soft-assert": "^0.2.3"
i want to use function of Soft assert
softAssert(actual, expected, msg, ignoreKeys)
But don't know, what is the exact steps to use it
Example:
When i use soft assertion function in my code, getting following error.
If i use like this
cy.softAssert(10, 12, "expected actual mismatch and will execute next line") : not supported
or if i use different way like
softAssert(10, 12, "expected actual mismatch and will execute next line") : SoftAssert not defined
Can any one tell me how to use this 'softAssert' function in cypress code with some small example?
Now the problem I am facing
it('asserts and logs and fails', () => {
Cypress.softAssert(10, 12, "expected actual mismatch...");
cy.log("text")
Cypress.softAssertAll();
})
I need my code after soft assertion as cy.log("text") to be executed in the same 'it' block but the current test failing the whole 'it' block, without executing 'cy.log("text")' statement.
The soft assertion concept is pretty cool, and you can add it with minimal implementation to Cypress
const jsonAssertion = require("soft-assert")
it('asserts several times and only fails at the end', () => {
jsonAssertion.softAssert(10, 12, "expected actual mismatch");
// some more assertions, not causing a failure
jsonAssertion.softAssertAll(); // Now fail the test if above fails
})
To me, it would be nicer to see each soft assertion failure in the log, so it's possible to add custom commands to wrap the soft-assert functions
const jsonAssertion = require("soft-assert")
Cypress.Commands.add('softAssert', (actual, expected, message) => {
jsonAssertion.softAssert(actual, expected, message)
if (jsonAssertion.jsonDiffArray.length) {
jsonAssertion.jsonDiffArray.forEach(diff => {
const log = Cypress.log({
name: 'Soft assertion error',
displayName: 'softAssert',
message: diff.error.message
})
})
}
});
Cypress.Commands.add('softAssertAll', () => jsonAssertion.softAssertAll())
//-- all above can go into /cypress/support/index.js
//-- to save adding it to every test (runs once each test session)
it('asserts and logs but does not fail', () => {
cy.softAssert(10, 12, "expected actual mismatch...");
cy.log('text'); // this will run
})
it('asserts and logs and fails', () => {
cy.softAssert(10, 12, "expected actual mismatch...");
cy.log('text'); // this will run
cy.softAssertAll();
})

Possible to manipulate CasperJS assertions?

My CasperJS asserts seem to be overly strict. I have a function where I am trying to test the names of client logo images from an array, using Casperjs. However I do not seem to be able to use a variable from a forLoop in casperJS.
I understand there are probably hoisting issues that I am not accounting for, but this does not seem to be the primary problem. I have tried several things to resolve hoisting issues, such as immediately invoked functions, try catch blocks, and using ES6 term "Let" in my loop. None seem to work. Then I notice if I simply hard-code the string my variable should represent, and stick a console.log into my assert of a PASSING test, right before the return, the passing test fails.
Here is my failing code
var clients = 'https://www.google.com/';
var logoArray = ["images/logos/AC.png", "images/logos/Affiny.png", "images/logos/ffintus.png", "images/logos/agileAsset.png"]
function checkClientsArrayTest() {
casper.test.begin('The layout is as expected', 10, function suite(test) {
casper.start(clients, function () {
casper.then(function () {
for (var i = 0; i < logoArray.length; i++) {
try { throw i }
catch (ii) {
console.log(ii);
console.log(i);
test.assertEvalEquals(function () {
return document.querySelectorAll('div.client_logo a img')[ii].getAttribute('src')
.match(logoArray[ii]).toString();
}, logoArray[ii], 'Test searches for Client Logos in DOM.');
}
}
});
}).run(function () {
test.done();
});
});
}
If I change logoArray[ii] to a hardcoded string from the first index of the array, it passes. If I consolelog logoArray[ii], it seems to be what I expect. But if I pass a variable to the assert, or even stick a console.log inside of it, the test fails with the following
Running check for the layout of URL: https://www.google.com
0
0
FAIL Test searches for Client Logos in DOM.
type: assertEvalEquals
file: headlessTester.js
subject: null
fn: undefined
params: undefined
expected: "images/logos/AC.png"
Is this an issue of me getting hoisting wrong (shouldn't fail by sticking in a logger if this is the case afaik), or is this due to strictly structured asserts in CasperJS?

How to write custom messages on passed tests with jasmine and protractor?

I'm looking a way to add custom messages on succeeded tests with protractor and jasmine.
There're many ways to customize messages on failure, like jasmine-custom-message node package or simply:
expect(column.get(0)).toEqual("7", "This is not something I've expected");
But I haven't found a way to add custom messages on success. I'm using protractor-jasmine2-html-reporter to generate my reports, but successful tests only display a "passed" or "0 failure" messages and I'd like to be more explicit about what am I testing and why the tests passed.
You want to write a custom matcher for Jasmine. You want to include them in the protractor.conf.js file so they are accessible over the entire project
https://jasmine.github.io/2.0/custom_matcher.html
Here is an example of comparing numbers with the message printing off if there is an error. The error if you run this will be Expected 7 to be 8 (Expected Error). Note, this is including it in the spec using the beforeEach.
var customMatchers = {
toBeEqualWithMessage: function(util, customEqualityTesters) {
return {
compare: function(actual, expected, message) {
var result = {};
result.pass = util.equals(actual, expected, customEqualityTesters);
if (!result.pass) {
result.message = "Expected " + actual + " to be " + expected + " (" + message + ")";
}
return result;
}
};
}
};
describe('Custom Matcher', function(){
beforeEach(function() { jasmine.addMatchers(customMatchers);});
it('should use a matcher', function(){
expect(7).toBeEqualWithMessage(7,"No error because they match");
expect(7).not.toBeEqualWithMessage(8,"No error because of the 'not'");
expect(7).toBeEqualWithMessage(8,"Expected Error");
});
});
FYI: The doc has a result.message for a passing case, but I don't know where that is used, maybe a verbose print out.

Testing ember components with promises

I've hit a wall in testing an ember component.
I have component that has an action
pseudocode
actions: {
doSomething(){
this.set('showSpinner', true);
this.model.serverOperation().then(()=>{
this.set('showSpinner', false);
this.service.doSomething();
})
}
}
in my test, i would like to assert that showSpinner is true when the action is invoked, then becomes false after the promise resolves. So my test looks like (pseudocode, using ember-sinon-qunit):
const deferred = Ember.RSVP.defer();
const modelMock = this.mock(this.model);
const serviceMock = this.mock(this.service);
modelMock.expects('serverOperation').returns(deferred.promise);
serviceMock.expects('doSomething);
this.$('selector').click(); // invokes 'doSomething'
assert(this.showSpinner, true)
deferred.resolve();
assert(this.showSpinner, true)
the problem is, on the deferred.resolve() call, the last assert fires before the then method in the component.
Is there a canonical way to handle this sort of thing in ember?
I tried using async to test, but in that case a few interesting things happen:
I change the final assert to execute in a setTimeout, and then the assert on showSpinner passes, but
the test still exits before then runs (so before the timeout callback fires), and since we are using ember-sinon-qunit, the test harness fails on the serviceMock expectation.
Put the last assert in a then() after call to deferred.resolve()
deferred.resolve();
deferred.promise.then(()=> {
assert(this.showSpinner, true);
});