experiencing super slow tests with testcafe - testing

I'm currently experimenting testcafe for the first time, I use the testdriven.io course as an example, and I'm facing really slow tests.
For instance testcafe e2e/login.test.js which is basically a register and a login takes sometimes 6min to pass, which I found insanely slow compared to the 15s the author has in his course.
import {Selector} from 'testcafe';
const randomstring = require('randomstring');
const username = randomstring.generate();
const email = `${username}#test.com`;
const TEST_URL = process.env.TEST_URL;
fixture('/login').page(`${TEST_URL}/login`);
test(`should display the sign-in form`, async (t) => {
await t
.navigateTo(`${TEST_URL}/login`)
.expect(Selector('H1').withText('Login').exists).ok()
.expect(Selector('form').exists).ok()
});
test(`should allow a user to sign in`, async (t) => {
// register user
await t
.navigateTo(`${TEST_URL}/register`)
.typeText('input[name="username"]', username)
.typeText('input[name="email"]', email)
.typeText('input[name="password"]', 'test')
.click(Selector('input[type="submit"]'))
// log a user out
await t
.click(Selector('a').withText('Log Out'))
// log a user in
await t
.navigateTo(`${TEST_URL}/login`)
.typeText('input[name="email"]', email)
.typeText('input[name="password"]', 'test')
.click(Selector('input[type="submit"]'))
// assert user is redirected to '/'
// assert '/' is displayed properly
const tableRow = Selector('td').withText(username).parent();
await t
.expect(Selector('H1').withText('All Users').exists).ok()
.expect(tableRow.child().withText(username).exists).ok()
.expect(tableRow.child().withText(email).exists).ok()
.expect(Selector('a').withText('User Status').exists).ok()
.expect(Selector('a').withText('Log Out').exists).ok()
.expect(Selector('a').withText('Register').exists).notOk()
.expect(Selector('a').withText('Log In').exists).notOk()
// log a user out
await t
.click(Selector('a').withText('Log Out'))
// assert '/logout' is displayed properly
await t
.expect(Selector('p').withText('You are now logged out').exists).ok()
.expect(Selector('a').withText('User Status').exists).notOk()
.expect(Selector('a').withText('Log Out').exists).notOk()
.expect(Selector('a').withText('Register').exists).ok()
.expect(Selector('a').withText('Log In').exists).ok()
});
I tried with Firefox, same deal, albeit much faster, 44s.
(project-tOIAx_A8) ➜ testdriven-app git:(master) ✗ testcafe 'chromium' e2e/login.test.js
Using locally installed version of TestCafe.
(node:16048) ExperimentalWarning: The fs.promises API is experimental
Running tests in:
- Chrome 67.0.3396 / Linux 0.0.0
/login
✓ should display the sign in form
✓ should allow a user to sign in
2 passed (6m 15s)
I wish I could provide more info, went to the dev tools, inspected the console without noticing any error message except this one sometimes:
hammerhead.js:7 WebSocket connection to 'ws://192.168.1.195:38039/4wxhbvTF7!w!http%3A%2F%2F172.18.0.5/http://172.18.0.5/sockjs-node/444/2kjm15kn/websocket' failed: Error during WebSocket handshake: Unexpected response code: 400
, the waterfall looks like slow, is there a way to give a log report of it to help understand what cases this slowness ?
edit: I put a .debug() at beginning and at the end of the tests and save a Performance profile inside Firefox if that helps
edit2: this is a performance profile in chromium if that helps

Related

How do I check network response in playwright?

I want to test the network https status response with playwright.
test.only('Rights.2: Users without the required author role do not have access.', async ({page}) => {
await login(page, 'xxx', "password.xxx");
const search = await searchForProcess(page, `${process.process1.title}`);
await login(page, 'PortalUser');
const open = await openProcess(page, `${process.process1.title}`);
});
So I tried with this
expect(response.status()).toEqual(403);
But it won't work, because response is not defined. Thats curious, because Playwright does document the "response.status()" as a function.
Can somebody help?
You could try page.waitForResponse(). Official docs:
https://playwright.dev/docs/api/class-page#page-wait-for-response
Here is an example where we click a button and we want to monitor that certain API calls actually happen and that they have certain response statuses:
// Start waiting for all required API calls as we select company.
// We can accept only 200 but we could accept 403 like with /api/Environment/
await Promise.all([
page.waitForResponse(resp => resp.url().includes('/api/Environment/') && (resp.status() === 200 || resp.status() === 403)),
page.waitForResponse(resp => resp.url().includes('/api/Security/') && (resp.status() === 200)),
// We already started waiting before we perform the click that triggers the API calls. So now we just perform the click
page.locator('div[role="gridcell"]:has-text("text")').click()
]);
If you wish to validate other reponse data, see: https://playwright.dev/docs/api/class-response
If you want to monitor all network traffic, this official docs would be more suitable: https://playwright.dev/docs/network#network-events

chrome:headless (MacOS) results with ' 1) AssertionError: expected 'about:blank' to include $target page'

I am using TestCafe in combination with gherkinTestcafe (steps) / cucumber.
I am also using environment variables so that i can run my tests on 2 different environments.
My code is as follows, although through debugging, i don't believe this is something strictly code related, as much as it is related to:
chrome:headless
environment
version of chrome / MacOS
import Enviorments from "../../../../../../AEM_Engine/Enviorment/Enviorments";
import { Helper } from "../../../../../TestActions/Test_specific/Career_helper";
import {AddAuthCredentialsHook} from "../../../../../TestActions/BasicAuth";
const {Before, Given, Then} = require('cucumber');
let publisher = new Publish();
let aemEnv = new Enviorments();
let helper = new Helper;
let careersPage = '/career';
Before('#basicAuth', async testController => {
const addAuthCredentialsHook = new AddAuthCredentialsHook('$someUserName', '$somePassword');
await testController.addRequestHooks(addAuthCredentialsHook);
});
Before('#disableCookie', async testController => {
await testController.addRequestHooks(publisher.mockCookieResponse);
});
Given('I am at Careers page', async testController => {
await publisher.Navigate(testController, aemEnv.frontEndURL + careersPage);
await publisher.verifyURL(testController, aemEnv.frontEndURL + careersPage);
});
.
.
.
When i wait for the script to run i have
1) AssertionError: expected 'about:blank' to include $expectedPage
As i mentioned, i don't believe the problem is in the code. Even if i remove the step for verifying the current URL location, the test fails on the next step after.
Tests pass on
Chrome (with UI shell)
Other browsers (firefox, safari), headless or with UI shell
Second (staging) environment
When Tests are run and TestCafe starts, i get the following info
Running tests in:
- HeadlessChrome 99.0.4844 / Mac OS X 10.15.7
Feature: Careers Page Available
(node:87344) Warning: Setting the NODE_TLS_REJECT_UNAUTHORIZED environment variable to '0' makes TLS connections and HTTPS requests insecure by disabling certificate verification.
I tried re-installing some packages, re-writing some of the steps, adding some flags to clear cache, change chrome port or similar, but nothing worked.
Any thoughts on what might be causing this and how to solve it?

Testcafe Authentication issues

I have a Fixture which includes 2 Tests. Both tests accessing the same site with httpAuth().
The first request loads the page correctly and everything is ok. But the second test can't even load the page completely. A lot of requests are stuck on "Pending". On the IIS and also on our Proxy i see that every Request from Testcafe-Hammerhead is logged twice first with an 401 without credentials and then 200 with credentials. But when checking the log for the Second test for some request i only see the 401 Request without credentials.
After some minutes the test stops because of timeouts.
Here is a sample of the Test:
fixture("Getting Started").page(pageUrl).httpAuth(devAuth);
test("Test 1", async t => {
await t.maximizeWindow();
await t.debug();
var text = Selector("#content").find("h1");
var headerText = await text.innerText;
console.log("Found text: " +headerText);
await t.expect(headerText).eql("About our Team");
});
test("Test 2", async t => {
await t.maximizeWindow();
await t.debug();
var text = Selector("#content").find("h1");
var headerText = await text.innerText;
console.log("Found text: " +headerText);
await t.expect(headerText).eql("About our Team");
});
We found out that Testcafe opens for every request a new connection on our Proxy. This used all available connections and no new connections could be made which results that some requests are stuck in "Pending". That every request opens a new Connection on our Proxy is only via Testcafe. We will further investigate this.

How to login with HTTP basic authentication to ACCESS the website- Test Cafe with Gherkin and cucumber

All links, project name and company name has been modified and are not original.
We have a basic HTTP authentication popup that appears when we are accessing out test/staging environments.
I am wondering, how can I enter the login information into the popup login window or send an api request in advance to save the cookie?
Because otherwise, I can't run the automation on our test environment. When the test access the page, I see a white website showing "unauthorized" in small letters.
This is what the login prompt looks like when accessing the Test/Staging env
We are using following plugin: https://www.npmjs.com/package/gherkin-testcafe
What I am asking is very similar to this question: Testing http basic authentication with Cucumber and RSpec in Rails 3.2.3
I have tried adding a role and using TestCafe http-authentication.html
Here is what I have tested so far:
TestCafe trying to use role:
const { Role } = require('testcafe');
const regularAccUser = Role('https://website/login', async t => {
await t
.typeText('#login', 'username')
.typeText('#password', 'password')
.click('#sign-in');
});
Given("I am open Dealer's login page", async t => {
await t
.useRole(regularAccUser)
.navigateTo(`${url}/login`);
});
That gives me:
ERROR CompositeParserException: Parser errors:
(7:3): expected: #EOF, #Comment, #BackgroundLine, #TagLine, #ScenarioLine, #ScenarioOutlineLine, #Empty, got 'Correct action happens when user provide either wrong or correct login information'
at Function.Errors.CompositeParserException.create (/Users/dennis/Projects/src/company/project/node_modules/gherkin/lib/gherkin/errors.js:27:13)
at Parser.parse (/Users/dennis/Projects/src/company/project/node_modules/gherkin/lib/gherkin/parser.js:72:45)
at specFiles.forEach.specFile (/Users/dennis/Projects/src/company/project/node_modules/gherkin-testcafe/src/compiler.js:43:33)
at Array.forEach (<anonymous>)
at GherkinTestcafeCompiler.getTests (/Users/dennis/Projects/src/company/project/node_modules/gherkin-testcafe/src/compiler.js:42:20)
at getTests (/Users/dennis/Projects/src/company/project/node_modules/testcafe/src/runner/bootstrapper.js:79:47)
at Generator.next (<anonymous>)
at step (/Users/dennis/Projects/src/company/project/node_modules/babel-runtime/helpers/asyncToGenerator.js:17:30)
at /Users/dennis/Projects/src/company/project/node_modules/babel-runtime/helpers/asyncToGenerator.js:28:13
If I try using:
import {Role} from 'testcafe'
I get:
tests/ui/stepDefinitions/login/login.js:1
(function (exports, require, module, __filename, __dirname) { import { Role } from 'testcafe';
^
SyntaxError: Unexpected token {
Using TestCafe's HTTP Authentication:
Feature: Login with correct and wrong info functionallity
.page('website/login')
.httpAuth({
username: 'username',
password: 'password',
})
Correct action happens when user provide either wrong or correct login information
#loginFunc
Scenario: Should NOT be able to login without filling in any credentials
Given I am open Dealer's login page
When I am login in with "empty" and "empty"
Then I should NOT be able to press the login button
I am getting following:
Feature: Login with correct and wrong info functionallity
✖ Scenario: Should NOT be able to login without filling in any credentials
1) Cannot obtain information about the node because the specified selector does not match any node in the DOM tree.
  | Selector('button[data-qa="qa-login-submit-button"]')
 > | Selector('button[data-qa="qa-login-submit-button"]')
Browser: Chrome 72.0.3626 / Mac OS X 10.14.3
32 | }
33 |});
34 |
35 |Then("I should NOT be able to press the login button", async t => {
36 | await t
> 37 | .expect(submitButton.hasAttribute('disabled')).ok()
38 | .expect(h1.exists).eql(true);
39 |});
```
It is basically showing me a white screen and saying: "**unauthorized**" in small letters.
You don't need any post/get requests with the Basic HTTP. Instead, you can log in using the Role mechanism:
When you switch to a role for the first time, TestCafe internally creates a branch of this role for this particular test run. All cookies set by your further actions will be appended to this branch. This branch will be used whenever you switch back to this role from the same test run.

Ember-auth signin test fails with json

I am having some issues with testing my signin/signout and related features of my app. The app works, but the test fail.
For testing, I use a QUnit with testem (I also tried teaspoon)
test "after signin, should redirect user back to previous page", ->
visit '/library'
fillIn '.signin-email', 'example#example.com'
fillIn '.signin-password', 'examplepass'
click '.signin-btn'
andThen ->
equal(testing().path(), 'library', "Should redirect back to library (was #{testing().path()})")
After running the test, I get a failure:
(screenshot here )
Authentication: visiting restricted page as non authenticated user: after signin, should redirect user back to previous page (2, 0, 2)Rerun544 ms
{user_id: 2, auth_token: wVveiyDLuXBXu69pQ2XQwg}
Source:
at Test.QUnitAdapter.Test.Adapter.extend.exception (http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:50149:5)
at superWrapper [as exception] (http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:13374:16)
at Object.onerror (http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:50009:22)
at onerror (http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:20453:16)
at EventTarget.trigger (http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:20286:22)
at null.<anonymous> (http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:20439:14)
at EventTarget.trigger (http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:20286:22)
at http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:20588:17
Should redirect back to library (was signin)
Expected:
"library"
Result:
"signin"
Diff:
"library" "signin"
Source:
at http://localhost:7357/public/assets/spec/javascripts/integration/authentication_pages_spec.js.js:22:14
at andThen (http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:50258:20)
at http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:49817:21
at isolate (http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:49989:14)
at http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:49972:12
at invokeCallback (http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:20463:19)
at null.<anonymous> (http://localhost:7357/public/assets/application-aad0a1b2c887cc25124c361787446e83.js:20513:11)
Also, auth.coffee:
App.Auth = Em.Auth.extend
request: 'jquery'
response: 'json'
strategy: 'token'
session: 'cookie'
modules: [
'emberData',
'authRedirectable',
'actionRedirectable'
]
signInEndPoint: '/signin'
signOutEndPoint: '/signout'
tokenKey: 'auth_token'
tokenIdKey: 'user_id'
tokenLocation: 'param'
emberData:
userModel: 'user' # create user model on login
authRedirectable:
route: 'signin'
actionRedirectable:
signInRoute: 'library'
# signInSmart: true
# signInBlacklist: ['signin']
signOutRoute: 'index'
I am unable to find the source of the error, so maybe it is something to do with ember-auth. Any ideas would be very appreciated.
Update 1 [Jan 4th]:
I've written an additional test, which passes only halfway. The test is simpler than the previous in that it does not check a redirect, but only checks that the user name appears in the UI after signin.
test "after signin, TEST", ->
visit '/library'
fillIn '.signin-email', 'user#example.com'
fillIn '.signin-password', 'foobargaz'
click '.signin-btn'
andThen ->
ok exists('.menu-signout'), "signout button exists"
The assertions passes, but I get an additional error reporting the returned JSON as seen in this screenshot. The screenshot basically shows:
[Fail] {user_id: 2, auth_token: wVveiyDLuXBXu69pQ2XQwg}
[Pass] signout button exists
Additionally, I've also run the tests by mocking the ajax requests with mockjax, but with the same failure.
Third, I should note that I had to patch "ember-auth-request-jquery.js" to make it work with ember testing as suggested here
I'm pretty sure you're failing to wait on the first visit to happen, so here's how I read it (I'm no CS person)
You're telling Ember to go to library
Before being sure it's finished navigating you're trying to fill in 2 fields and click a button (all of which probably doesn't exist)
then you check to see if it's library, but while waiting after you thought you clicked, really the page finishes rendering the login page from the visit
Here's what js2coffe says it'd kind of look like (my main point is the then after the visit).
test "after signin, should redirect user back to previous page", ->
visit("/library").then ->
fillIn ".signin-email", "example#example.com"
fillIn ".signin-password", "examplepass"
click(".signin-btn").then ->
equal testing().path(), "library", "Should redirect back to library (was " + (testing().path()) + ")"
Update 1/4: Documentation changed on me
Now we move to educated guess time. Looking through the Ember-auth code it might not be creating any timers/promises that Ember is aware of, in affect Ember thinks it's finished the signin process immediately. So the click promise is resolved immediately and you run your test immediately (andThen waits on the global testing promise to resolve). To test the theory you can do some terrible timeout and see if it does indeed redirect after some time
test "after signin, should redirect user back to previous page", ->
visit "/library"
fillIn ".signin-email", "example#example.com"
fillIn ".signin-password", "examplepass"
click ".signin-btn"
stop()
Ember.run.later this, (->
start()
equal testing().path(), "library", "Should redirect back to library (was " + (testing().path()) + ")"
), 5000
It turns out my coffeescript was not the best in the world.
The module function in QUnit should NOT compile to:
module('Authentication: visiting restricted page as non authenticated user', function() {
return setup(function() {
return Em.run(App, App.advanceReadiness);
});
});
but to:
module('Authentication: visiting restricted page as non authenticated user', {
setup: function() {
Ember.run(App, App.advanceReadiness);
},
// This is also new
teardown: function() {
App.reset();
}
});
Additionally, in my spec_helper.coffee file I had something like this:
QUnit.testStart(function() {
// FIXME: this below made it fail every time
// Ember.run(function() {
// return App.reset();
// });
Ember.testing = true;
});
QUnit.testDone(function() {
Ember.testing = false;
});
QUnit.done(function() {
return Ember.run(function() {
return App.reset();
});
});
which seems to have caused some issues, so I just deleted it and the tests now pass.