Is it possible to have several pm.test() inside a pm.test()? In such a way that if any of the inner pm.test() fails, the outer pm.test() will also fail. And if no inner pm.test() fails, the outer pm.test() passes.
My requirement is similar to how pm.expect() works. But i want to use pm.test() instead so i can the status of each test in the Test Results tab.
My code is below.
var response = pm.response.json()
pm.test("Test scenario 1", ()=> {
pm.test("Checking response code", ()=> {
pm.expect(response.code).to.eql(200);
});
pm.test("Checking response message", ()=> {
pm.expect(response.message).to.eql('OK');
});
//more pm test
});
Thanks a lot.
pm.test("Test scenario 1", ()=> {
pm.test("Checking response code", ()=> {
pm.expect(pm.response.code).to.eql(200);
});
pm.test("Checking response message", ()=> {
pm.expect(pm.response.status).to.eql('5OK');
});
//more pm test
});
you can have but the output will show this as three tests only and result of internal test won't affect the out side abstraction
so in short you can but it won't give the behavior as you except
By implementing some counter I was able to achieve what I want. I incremented the counter at the end of the inner pm.test(). If the pm.expect() fails, the counter is not incremented.
When all inner pm.test() are done, I compare the counter+1 with pm.test.index(). If they are equal, I do nothing (which basically passes the outer pm.test()). If they are not equal, I throw pm.expect.fail().
Note: pm.test.index() returns the number of tests executed at that point of time.
pm.test("To test able to do that", () => {
let numberOfPassedTest = 0;
pm.test("This test will pass", () => {
//put assertion here
pm.expect(200).to.eql(200)
numberOfPassedTest += 1;
});
pm.test("This test will fail", () => {
pm.expect(200).to.eql(201)
numberOfPassedTest += 1; // this line is not executed because the previous line (pm.expect()) failed.
});
if(pm.test.index() === numberOfPassedTest+1){
//do nothing. meaning no test failed and the whole pm.test() will be passed
} else {
pm.expect.fail("At least one of the tests failed. So this test case is marked as failed.");
}
});
Sample result with failure:
enter image description here
Related
I have a typical search where the user types some text in an input, async work is done and a table with the results is properly updated.
I have tests that must wait for this search step and then assert business rules regarding the results, like if the table records are eligible for edit.
Every time I ran a complete test battery (something like 80 test files), one or two of the tests involving that search inevitably fail. But if immediately after that, I run the same test alone, the test passes. It's excruciating and makes e2e testing in CI/CD pointless for the project.
I've read the Cypress documentation about flaky tests and searched questions in StackOverflow and GitHub with only complete failure. It's a drama.
Here is one of the tests:
import { searchList } from '../helpers';
import { createFluxoIniciado, randomFluxoNome } from './common';
import { fluxoSelectors } from './selectors';
describe('fluxos finish', () => {
it('can manually finish a fluxo INICIADO', () => {
// feed data to be searched
const fluxoNome = randomFluxoNome();
createFluxoIniciado({ fluxoNome });
// search
searchList(fluxoNome);
// do something with search results
fluxoSelectors.fluxos.view().click();
fluxoSelectors.finish().click();
fluxoSelectors.confirm().click();
// serach again
searchList(fluxoNome);
cy.contains('FINALIZADO');
});
});
The code in searchList is where trouble emerge sometimes. It uses the callback strategy recommended here. The code attempts to cause retries if not all rows have the searched text.
export function searchList (text) {
cy.get('#searchText')
.scrollIntoView()
.type(text)
.blur();
cy.get('tbody tr').should($trs => {
$trs.each((i, $tr) => {
expect($tr).to.contain(text);
});
}, { timeout: 15000 });
}
Here is an example of a test failure inside a run all test execution:
The problem is obviously caused by the async fetch between .blur() and testing the rows.
You are correctly trying to use Cypress retry with .should(callback), but if the callback is complex or there are multiple steps it may not retry the element that is changing (the table rows).
Ideally you want to keep the cy.get(...).should(...) as simple as possible, and start by testing that the table loading has completed.
// wait for expected number of rows
cy.get('tbody tr', {timeout: 15000}).should('have.length', 5)
cy.get('tbody tr').each($tr => {
expect($tr).to.contain(text);
})
But you have a randomizer there, so maybe it's not possible to test explicitly the row count.
Another approach, test the whole table for text (.contains() checks child text also)
// wait for text to appear somewhere
cy.get('tbody tr', {timeout: 15000}).should('contain', text)
cy.get('tbody tr').each($tr => {
expect($tr).to.contain(text);
})
You can also add an intercept between start and end of api call
export function searchList (text) {
cy.intercept('search/api/endpoint').as('search')
cy.get('#searchText')
.scrollIntoView()
.type(text)
.blur();
cy.wait('#search') // wait for api response
cy.get('tbody tr', {timeout: 15000}).should('contain', text)
cy.get('tbody tr').each($tr => {
expect($tr).to.contain(text);
})
}
I just noticed you have the {timeout} option on .should(), but that's the wrong place,
see Timeouts
cy.get('input', { timeout: 10000 }).should('have.value', '10')
// timeout here will be passed down to the '.should()'
// and it will retry for up to 10 secs
This may be successful
cy.get('tbody tr', { timeout: 15000 })
.should($trs => {
$trs.each((i, $tr) => {
expect($tr).to.contain(text);
});
})
I have a page build in vue.js where an API is called (using axios) getting a large json object (1k+).
While I am processing this object, I want to show user the progress of the operation - nothing fancy like progress bars, just a simple "Processed X from Y".
I have found example on how to do this in jquery of pure JS, but I can get this to work in vue.
Any help is appreciated.
Here is my skeleton code:
<div>Processed {{processed.current}} of {{processed.total}} records</div>
<script>
data() {
return {
progress:{
current:0,
total: 0
},
records: [],
};
},
mounted() {
this.getRecords();
},
methods: {
getRecords(){
axios({
method: "GET",
url: process.env.VUE_APP_REPORTING_API + "/Reports/orders",
headers: {
"content-type": "application/json",
Authorization: this.$cookie.get("wwa_token")
}
}).then(
result => {
this.progress.total = result.data.length;
//and here where the loop should happen, something like this
//obviously the below won't work :)
result.data.forEach(function(item) {
this.records.push(item);
this.progress.current++;
}
},
error => {
}
);
}
}
</script>
Well, the obvious problem with the posted code is that it has a syntax error (missing a closing parenthesis) and it establishes a new context. The code would (sort of) "work" if it was changed it to use arrow functions:
result.data.forEach(item => {
this.records.push(item);
this.progress.current++;
});
However, I don't think that's going to do what you want. The JavaScript code will process all the items before the user interface updates, so all the user would see would be "Processed N of N records". Even if you inserted a this.$forceUpdate() in the loop to make the interface update at each iteration, the changes would still be too fast for any user to see.
The real problem is that "processing" all the items only takes a few milliseconds. So it's always going to happen too quickly to show intermediate results.
If you're trying to show the progress of the AJAX request/response, that's an entirely different question which will require coordination between the client and the server. Search for HTTP chunked responses for a start.
I am writing test for Ember app written in Ember 1.6.
Inside a controller I have a function executed upon promise success:
var me = this;
function onSuccess(result) {
printSuccessMessage();
Ember.RSVP.all(promises).then(function(value) {
Ember.run.later(this, function() {
clearMessages();
}, 5000);
});
}
Then, inside the test, I am trying to assert that the success message appears:
fillIn('#MyInputField', 'Some text');
click('#MyButton');
andThen(function() {
strictEqual(find('[data-output="info-message"]').text().trim().indexOf('Done!') >= 0, true, 'Expected success message!');
});
But the problem is, that after the click, andThen is waiting for a run loop to finish. So after this click, andThen waits 5 seconds and then executes assertions.
In that moment clearMessages() is already executed, the message div is cleared, and the test fails.
Any idea how to assert that this message has certain text?
If you are willing to have a condition in your code, that checks whether or not Ember is in testing mode, you can toggle the Ember.testing value in your test, and then clear or not clear the message in your controller, based on that value. Your tests can then assert that the message is cleared in one instance, and showing in the other.
In the controller's onSuccess call, observe the Ember.testing condition:
onSuccess(message) {
this.printSuccessMessage(message);
if (Ember.testing) { // <-- HERE
// during testing
return; // don't clear the message and assert that it's there
} else {
// during dev, live in production, or Ember.testing === false
this.clearMessages(); // clear the message, and assert that it's gone
}
},
In the acceptance test for setting the message, since Ember.testing is true by default, the controller will not clear the message, and the following test will succeed:
test('setting the message', function(assert) {
visit('/messages');
fillIn('input.text-input', 'Some text');
click('button.clicker');
// while Ember.testing is `true` (default), do not remove message
andThen(() => {
assert.equal(find('div.info-message').text(),
'Done!',
'The message was set properly.');
});
});
In the test that follows, observe the toggling of false for Ember.testing, which will "emulate" live dev or production conditions for the controller. The controller will clear the message, as normal, and this test will also succeed:
test('clearing the message', function(assert) {
visit('/messages');
fillIn('input.text-input', 'Some text');
andThen(() => {
Ember.testing = false;
});
click('button.clicker');
// while Ember.testing is `false`, remove message, as normal, as in dev or prod
andThen(() => {
assert.equal(find('div.info-message').text(),
'',
'The message has been cleared.');
});
// reset Ember.testing to its default
andThen(() => {
Ember.testing = true;
});
});
Please note, Ember.testing is reset to its default value of true as soon as the false condition is no longer needed. This is important because Ember run loop behavior is different during testing by design.
In this solution, some code has been refactored, to isolate concerns and make it easier to unit test. Here's an Ember Twiddle to demonstrate, which was inspired, in part, by this article on Medium.
Related questions:
wait for element present without error
Nightwatchjs: how to check if element exists without creating an error/failure/exception
Selenium WebDriver : Wait for complex page with JavaScript(JS) to load
Whenever I load my web-page, I need to wait for ajax calls to complete. I currently do this with:
browser.waitForElementNotPresent(
"cssSelector_For_HTML_afterAJAXComplete",
10000 //timeout
);
This adds an extra assertion to my test though, and makes the test-count for the report artificially larger than necessary.
Any ideas?
Ok, I hacked something together. I'm basically sending javascript to the console in selenium using browser.execute(function, params, callback)
function waitForElement_NoPassFail(selector, timeout){
browser.execute(function(selector, timeout, done){
let intervalId;
var p1 = new Promise(function(resolve, reject){
intervalId = setInterval(function(){
let itemArray = document.querySelectorAll(selector);
if(itemArray.length == 1){
clearInterval(intervalId);
resolve(true); //element found
} else if(itemArray.length>1){
reject(false); //too many elements found, because of ambiguous selector
}
}, 100);
});
var p2 = new Promise(function(resolve, reject){
setTimeout(reject,timeout, false); //timeout reached
});
return Promise.race([p1, p2]).then(function(result){
done(result);
});
},
[selector, timeout],
function(result){
if(!result){
throw "Element: " + selector + " wasn't found after " + timout + " ms.";
} else {
console.log("Element found within timeout limits.") //doesn't trigger assert=ok
}
});
};
waitForElement_NoPassFail(
"cssSelector_that_Is_Valid_after_AjaxIsComplete",
10000 //timeout
);
This can be extended in various ways, for example to support xPath. You could use the nightwatch global variable for element checking frequency when waiting. Write a comment if you need the help.
This question already has answers here:
mocha pass variable to the next test
(3 answers)
Closed 7 years ago.
I'm learning how to write a NodeJS module that works against a vendor's REST API. The critical code for the module itself is written, but now I'm trying to learn how to test it all properly. Currently I'm using MochaJS and ChaiJS for the testing framework. In one test I create a user which returns a random ID, which I need to save. Then later I want to use said ID value and test the user deletion.
Here's the current code that doesn't work:
var names = require('./names.json');
var ids = [];
describe('users', function() {
describe('addUser', function (){
it('should create ' + names[0].firstname, function (done){
this.slow(3000); this.timeout(10000);
api.addUser(names[0],function(x){
x.should.have.property('id').with.length.of.at.least(2);
ids.push(x.id);
done();
});
});
it('should create ' + names[1].firstname, function (done){
this.slow(3000); this.timeout(10000);
api.addUser(names[1],function(x){
x.should.have.property('activated').and.equal(true);
ids.push(x.id);
done();
});
});
});
describe('deleteUser', function (){
for(var a=0;a<ids.length;a++){
it('should delete ' + ids[a], function (done){
api.deleteUser(ids[a],function(x){
x.should.have.property('id').and.equal(ids[a]);
done();
});
});
}
});
});
Even though ids is scoped far outside the testing, the values are not saved. Now I've read other comments on stack overflow about this where the responders basically say "don't re-use values...something something waterfall failure". Which I understand but to me, that's Expected Functionality (TM). If for any reason (either my code or the vendors API) there is a failure and I cannot create a user, then obviously I will not be able to delete a user.
I want to put all this into Travis CI, so I cannot expect a specific user will always be there to delete unless my test framework creates is. I also have a limited number of users on the vendors system, so I need to clean up my testing. There are also other use cases (such as modifying an existing user) that I want to test.
The quick answer is that your for loop never loops.
When the test file is parsed, the for loop (before any test has run and hence before you can push anything into ids) executes and because ids is empty, has no work to do.
To prove this, tweak your code to be:
describe('deleteUser', function (){
console.log("How many IDs?", id);
for(var a=0;a<ids.length;a++){
console.log("This will not be seen...");
it('should delete ' + ids[a], function (done){
api.deleteUser(ids[a],function(x){
x.should.have.property('id').and.equal(ids[a]);
done();
});
});
}
});
The absolute easiest way to fix this is to not loop over the IDs but to instead delete both users, one after the other and then check both were successful:
var names = require('./names.json');
var ids = [];
describe('users', function() {
describe('addUser', function (){
it('should create ' + names[0].firstname, function (done){
this.slow(3000); this.timeout(10000);
api.addUser(names[0],function(x){
x.should.have.property('id').with.length.of.at.least(2);
ids.push(x.id);
done();
});
});
it('should create ' + names[1].firstname, function (done){
this.slow(3000); this.timeout(10000);
api.addUser(names[1],function(x){
x.should.have.property('activated').and.equal(true);
ids.push(x.id);
done();
});
});
});
describe('deleteUser', function (){
it('should delete users', function (done){
api.deleteUser(ids[0],function(x){
x.should.have.property('id').and.equal(ids[0]);
api.deleteUser(ids[1],function(x){
x.should.have.property('id').and.equal(ids[1]);
done();
});
});
});
});
});
Untested and nowhere near great, but should work.