Concept
I'm using the concept of reusing the authentication state via storageState file in Playwright.
Following is my code snippet spread across different files:
Code
playwright.config.json
import type { PlaywrightTestConfig } from '#playwright/test';
import { devices } from '#playwright/test';
const config: PlaywrightTestConfig = {
testDir: './e2e',
reporter: 'html',
globalSetup: require.resolve('./e2e/global-setup.ts'),
use: {
storageState: './e2e/authStorageState.json',
},
projects: [
{
name: 'firefox',
use: {
...devices['Desktop Firefox'],
},
},
{
name: 'webkit',
use: {
...devices['Desktop Safari'],
},
},
{
name: 'chromium',
use: {
...devices['Desktop Chrome'],
},
},
],
};
export default config;
global-setup.ts
require('dotenv').config();
import { firefox, FullConfig } from '#playwright/test';
import E2EConstants from './e2e.constants';
const authenticateUser = async () => {
const browser = await firefox.launch();
const page = await browser.newPage();
await page.goto(`${process.env.BASE_URL}/${E2EConstants.LoginPage.URL}`);
await page
.getByLabel(E2EConstants.LoginPage.LABEL.EMAIL)
.fill(process.env.TEST_ADMIN_USERNAME as string);
await page
.getByLabel(E2EConstants.LoginPage.LABEL.PASSWORD)
.fill(process.env.TEST_ADMIN_PASSWORD as string);
await page.getByText(E2EConstants.LoginPage.BUTTON).click();
await page.waitForLoadState('networkidle');
await page.context().storageState({ path: './e2e/authStorageState.json' });
await browser.close();
};
async function globalSetup(_: FullConfig) {
await authenticateUser();
}
export default globalSetup;
This sets all the cookies in the authStorageState.json file. However, some cookies have domain as .b.com and some of them have the domain as a.b.com.
example.spec.ts
require('dotenv').config();
import { test, expect } from '#playwright/test';
test('homepage has same link', async ({ page }) => {
await page.goto(process.env.TEST_URL as string);
await expect(page).toHaveURL(process.env.TEST_URL as string);
});
The TEST_URL is the URL with domain a.b.com that is behind authentication and is accessible only when the user is signed in.
Problem
When I run the tests, I see that the Chromium test fails but the Firefox and Webkit tests pass. The test is unable to sign the user in on Chromium, unlike on other browsers. This is because the auth-related cookies (belonging to a.b.com domain) are not set on Chromium but they are set on other browsers. However, the non-auth-related cookies (belonging to .b.com domain) are set properly on all browsers.
What I tried
I tried manually setting the url of the cookies saved in authStorageState.json file to https://a.b.com after deleting the domain and path keys for the auth-related cookies and then when I run the tests, the Chromium test also passes.
The secure key for all cookies, which were not set, had the value as false. I tried manually changing the "secure": true for all cookies which were not set. Note that the sameSite property is set to "None". This change made the Chromium test cases to pass.
The sameSite key for all cookies, which were not set, had the value "None" while having "secure": false property. I tried manually removing the "sameSite": "None" property for all cookies which were not set. Note that I did not change the secure property. This change made the Chromium test cases to pass.
[Note that all the above points were independently implemented as workarounds and were not done together.]
Requirement
However, because the authStorageState.json file is to be created by default on first sign-in, I want to persist the same cookies that appear on browser instead of manually manipulating them. How can I get my Chromium test cases to pass?
Doubts
What is the difference between providing url instead of domain + path for Chromium?
Why do the cookies with subdomain a.b.com require secure property to be set to true and the Chromium-based test cases pass after this change?
Is removing the "sameSite": "None" for all the cookies with subdomain a.b.com right and why do the Chromium-based test cases pass after this change?
Related
I'm trying to implemented login via API following Playwright's guidelines but somehow nothing seems to be working.
As a comparison I've built the same in Cypress and it works out of the box:
Context:
Playwright Version: 1.30
Operating System: Mac
Node.js version: v16.19.0
Browser: Chromium
I am unable to make a simple API login that works perfectly using Cypress instead. Let me share the 2 code snippets for comparison:
Simple test case:
API request to the login end-point - Auth token is retrieved
set the auth token as a cookie
navigate to a page that is accessible only if authenticated
Code Snippet
Cypress (working fine)
const body = {
username: 'username...',
password: 'password',
rememberMe: true,
};
describe('Login via API to management console', () => {
it('Login via API to management console', () => {
cy.request({
method: 'POST',
url: loginEndPoint,
headers: {
'Content-Type': 'application/json',
},
body,
}).then((response) => {
cy.setCookie('Authorization', `Token ${response.body.data.token}`);
});
cy.visit(`/management`);
});
});
Playwright (not working)
test('Login via API', async ({ browser }) => {
const context = await browser.newContext();
const page = await context.newPage();
const loginResponse = await context.request.post(`https://${process.env.MANAGEMENT_URL}/web/api/v2.1/users/login`, {
data: {
username: process.env.MANAGEMENT_USER,
password: process.env.MANAGEMENT_PASSWORD,
rememberMe: true,
}
});
const {
data: { token },
} = await loginResponse.body().then((b) => {
return JSON.parse(b.toString());
});
expect(token).toMatch(/^[a-z0-9]{80}$/)
await context.addCookies([{ name: 'Authorization', value: `Token ${token}`, path: '/', domain: `https://${process.env.MANAGEMENT_URL}` }]);
await page.goto(`https://${process.env.MANAGEMENT_URL}/management/`);
await expect(page).toHaveURL(/management/);
});
Describe the bug
Both scripts are successful at retrieving the authentication token but somehow either I'm doing something wrong with setting the cookie in Playwright or there is an issue. I'd assume the 2 scripts should be comparable.
Furthermore: I've tried to execute login via UI using global-setup, saving the storage-state, loading it before running the test and it fails also in this case... so there is something that is not setting properly the state in this case or the cookie in the previous one.
Not entirely sure why the cookie approach wasn’t working, perhaps the https:// part should be removed from the domain?
That being said, in Playwright you shouldn’t even need to do that especially within a single test, looking at the Playwright docs on signing in via the API and related page about the request context particularly under cookie management. The associated request and browser contexts share cookies, so once you complete the login request, the browser should already have the cookie state too and be logged in, so you should be able to just remove getting the token and adding the cookie. Or you can login with the API in the global setup even, as that doc showed. Just make sure in that case to save the storage state, and specify the same file in your config.
I see you tried the global setup approach (through the UI, but you can use the API since you have it), not sure what happened there. I would say to ensure that you specified the storageState in the config; I would be curious how you loaded it as mentioned, and if you’re still having problems maybe share the code you’re using for that piece?
Hope that helps or we can troubleshoot further!
I'm working on a Cypress test for the Polish Wikipedia plugin, and I have this code in my cypress test:
Cypress.Commands.overwrite('visit', (orig, path, options) => {
return orig(`https://pl.wikipedia.org/${path}`);
});
Cypress.Commands.add('login', (pass) => {
cy.visit('/w/index.php?title=Specjalna:Zaloguj');
cy.get('#wpName1').type('<username>');
cy.get('#wpPassword1').type(pass);
cy.get('#userloginForm form').submit();
});
Cypress.Commands.add('dark_vector_wait', (pass) => {
cy.get('.vector-menu-content-list > li:nth-child(7)', { timeout: 10000 }).should('be.visible');
});
And in my spec:
describe('dark vector test', () => {
beforeEach('login', () => {
cy.login(Cypress.env('WIKI_PASSWORD'));
});
it('test discussion', () => {
cy.visit('/wiki/Dyskusja_wikipedysty:<username>');
cy.dark_vector_wait();
cy.matchImageSnapshot('discussion');
});
it('testing-page page', () => {
cy.visit('/wiki/Wikipedysta:<username>/testing-page');
cy.dark_vector_wait();
cy.matchImageSnapshot('testing-page');
});
});
And the second test is failing because as soon as Cypress type the password it automatically submits a form so cy.get('#userloginForm form').submit(); is executing after Cypress visits the home page (default redirect) and fail to find a form.
What's wrong? Why does Cypress auto-submit a form after a second time? This is not what Wikipedia is doing since the login form doesn't have any JavaScript code and you need to click login to be able to login to Wikipedia.
EDIT:
I've tried to
Use BaseURL and remove overwrite of visit.
Add type('{enter}'), but this only shows an error: cy.type() failed because this element is detached from the DOM.
EDIT 2
This is the screenshot of the action taken by cypress, second test has a page load without {enter} action and without form submit.
The problem is in Cypress.Commands.overwrite('visit').
You pass the parameter with a leading slash '/wiki/Dyskusja_wikipedysty:<username>' but concatinate to base which also has a trailing slash https://pl.wikipedia.org/${path}, so now the full path is
https://pl.wikipedia.org//wiki/Dyskusja_wikipedysty:<username>
If you set baseUrl in configuration, Cypress sorts it out for you
cypress.config.js
const { defineConfig } = require('cypress')
module.exports = defineConfig({
e2e: {
setupNodeEvents(on, config) {
// implement node event listeners here
},
baseUrl: 'https://pl.wikipedia.org'
}
})
Then remove the Cypress.Commands.overwrite('visit').
With these changes, I was able to pass both tests.
Using cy.session()
The problem might be one specific to locality, I do not have any steps missing in the Cypress log.
You can try adding a session cache so that the first login is re-used.
Cypress.Commands.add('login', (pass) => {
cy.session('login', () => { // login occurs only once
// credentials are cached
cy.visit('/w/index.php?title=Specjalna:Zaloguj');
cy.get('#wpName1').type('Jack Bosko');
cy.get('#wpPassword1').type(pass);
cy.get('#userloginForm form').submit();
// for good measure, confirm login was successful
// by checking for your name on the page
cy.contains('span', 'Jack Bosko')
})
})
So the problem was the weird IME keyboard that is part of MediaWiki. I somehow got it enabled on my system even when I was not logged in. Maybe added globally with cookies or something.
I noticed that keyboard when I was asking questions on the MediaWiki support page.
This is not related to Cypress. I'm investigating why the keyboard is there, and why clean environment it.
I'm trying to figure out how to mock calls to the auth0 authentication backend when testing a next js app with React Testing Library. I'm using auth0/nextjs-auth0 to handle authentication. My intention is to use MSW to provide mocks for all API calls.
I followed this example in the nextjs docs next.js/examples/with-msw to set up mocks for both client and server API calls. All API calls generated by the auth0/nextjs-auth0 package ( /api/auth/login , /api/auth/callback , /api/auth/logout and /api/auth/me) received mock responses.
A mock response for /api/auth/me is shown below
import { rest } from 'msw';
export const handlers = [
// /api/auth/me
rest.get(/.*\/api\/auth\/me$/, (req, res, ctx) => {
return res(
ctx.status(200),
ctx.json({
user: { name: 'test', email: 'email#domain.com' },
}),
);
}),
];
The example setup works fine when I run the app in my browser. But when I run my test the mocks are not getting picked up.
An example test block looks like this
import React from 'react';
import {render , screen } from '#testing-library/react';
import Home from 'pages/index';
import App from 'pages/_app';
describe('Home', () => {
it('should render the loading screen', async () => {
render(<App Component={Home} />);
const loader = screen.getByTestId('loading-screen');
expect(loader).toBeInTheDocument();
});
});
I render the page inside the App component like this <App Component={Home} /> so that I will have access to the various contexts wrapping the pages.
I have spent about 2 days on this trying out various configurations and I still don't know what I might be doing wrong. Any and every help is appreciated.
This is probably resolved already for the author, but since I ran into the same issue and could not find useful documentation, this is how I solved it for end to end tests:
Overriding/configuring the API host.
The plan is to have the test runner start next.js as custom server and then having it respond to both the next.js, as API routes.
A requirements for this to work is to be able to specify the backend (host) the API is calling (via environment variables). Howerver, access to environment variables in Next.js is limited, I made this work using the publicRuntimeConfig setting in next.config.mjs. Within that file you can use runtime environment variables which then bind to the publicRuntimeConfig section of the configuration object.
/** #type {import('next').NextConfig} */
const nextConfig = {
(...)
publicRuntimeConfig: {
API_BASE_URL: process.env.API_BASE_URL,
API_BASE_PATH: process.env.API_BASE_PATH,
},
(...)
};
export default nextConfig;
Everywhere I reference the API, I use the publicRuntimeConfig to obtain these values, which gives me control over what exactly the (backend) is calling.
Allowing to control the hostname of the API at runtime allows me to change it to the local machines host and then intercept, and respond to the call with a fixture.
Configuring Playwright as the test runner.
My e2e test stack is based on Playwright, which has a playwright.config.ts file:
import type { PlaywrightTestConfig } from '#playwright/test';
const config: PlaywrightTestConfig = {
globalSetup: './playwright.setup.js',
testMatch: /.*\.e2e\.ts/,
};
export default config;
This calls another file playwright.setup.js which configures the actual tests and backend API mocks:
import {createServer} from 'http';
import {parse} from 'url';
import next from 'next';
import EndpointFixture from "./fixtures/endpoint.json";
// Config
const dev = process.env.NODE_ENV !== 'production';
const baseUrl = process?.env?.API_BASE_URL || 'localhost:3000';
// Context
const hostname = String(baseUrl.split(/:(?=\d)/)[0]).replace(/.+:\/\//, '');
const port = baseUrl.split(/:(?=\d)/)[1];
const app = next({dev, hostname, port});
const handle = app.getRequestHandler();
// Setup
export default async function playwrightSetup() {
const server = await createServer(async (request, response) => {
// Mock for a specific endpoint, responds with a fixture.
if(request.url.includes(`path/to/api/endpoint/${EndpointFixture[0].slug}`)) {
response.write(JSON.stringify(EndpointFixture[0]));
response.end();
return;
}
// Fallback for pai, notifies about missing mock.
else if(request.url.includes('path/to/api/')) {
console.log('(Backend) mock not implementeded', request.url);
return;
}
// Regular Next.js behaviour.
const parsedUrl = parse(request.url, true);
await handle(request, response, parsedUrl);
});
// Start listening on the configured port.
server.listen(port, (error) => {
console.error(error);
});
// Inject the hostname and port into the applications publicRuntimeConfig.
process.env.API_BASE_URL = `http://${hostname}:${port}`;
await app.prepare();
}
Using this kind of setup, the test runner should start a server which responds to both the routes defined by/in Next.js as well as the routes intentionally mocked (for the backend) allowing you to specify a fixture to respond with.
Final notes
Using the publicRuntimeConfig in combination with a custom Next.js servers allows you to have a relatively large amount of control about the calls that are being made on de backend, however, it does not necessarily intercept calls from the frontend, the existing frontend mocks might stil be necessary.
I want to save/persist/preserve a cookie or localStorage token that is set by a cy.request(), so that I don't have to use a custom command to login on every test. This should work for tokens like jwt (json web tokens) that are stored in the client's localStorage.
To update this thread, there is already a better solution available for preserving cookies (by #bkucera); but now there is a workaround available now to save and restore local storage between the tests (in case needed). I recently faced this issue; and found this solution working.
This solution is by using helper commands and consuming them inside the tests,
Inside - cypress/support/<some_command>.js
let LOCAL_STORAGE_MEMORY = {};
Cypress.Commands.add("saveLocalStorage", () => {
Object.keys(localStorage).forEach(key => {
LOCAL_STORAGE_MEMORY[key] = localStorage[key];
});
});
Cypress.Commands.add("restoreLocalStorage", () => {
Object.keys(LOCAL_STORAGE_MEMORY).forEach(key => {
localStorage.setItem(key, LOCAL_STORAGE_MEMORY[key]);
});
});
Then in test,
beforeEach(() => {
cy.restoreLocalStorage();
});
afterEach(() => {
cy.saveLocalStorage();
});
Reference: https://github.com/cypress-io/cypress/issues/461#issuecomment-392070888
From the Cypress docs
For persisting cookies: By default, Cypress automatically clears all cookies before each test to prevent state from building up.
You can configure specific cookies to be preserved across tests using the Cypress.Cookies api:
// now any cookie with the name 'session_id' will
// not be cleared before each test runs
Cypress.Cookies.defaults({
preserve: "session_id"
})
NOTE: Before Cypress v5.0 the configuration key is "whitelist", not "preserve".
For persisting localStorage: It's not built in ATM, but you can achieve it manually right now because the method thats clear local storage is publicly exposed as Cypress.LocalStorage.clear.
You can backup this method and override it based on the keys sent in.
const clear = Cypress.LocalStorage.clear
Cypress.LocalStorage.clear = function (keys, ls, rs) {
// do something with the keys here
if (keys) {
return clear.apply(this, arguments)
}
}
You can add your own login command to Cypress, and use the cypress-localstorage-commands package to persist localStorage between tests.
In support/commands:
import "cypress-localstorage-commands";
Cypress.Commands.add('loginAs', (UserEmail, UserPwd) => {
cy.request({
method: 'POST',
url: "/loginWithToken",
body: {
user: {
email: UserEmail,
password: UserPwd,
}
}
})
.its('body')
.then((body) => {
cy.setLocalStorage("accessToken", body.accessToken);
cy.setLocalStorage("refreshToken", body.refreshToken);
});
});
Inside your tests:
describe("when user FOO is logged in", ()=> {
before(() => {
cy.loginAs("foo#foo.com", "fooPassword");
cy.saveLocalStorage();
});
beforeEach(() => {
cy.visit("/your-private-page");
cy.restoreLocalStorage();
});
it('should exist accessToken in localStorage', () => {
cy.getLocalStorage("accessToken").should("exist");
});
it('should exist refreshToken in localStorage', () => {
cy.getLocalStorage("refreshToken").should("exist");
});
});
Here is the solution that worked for me:
Cypress.LocalStorage.clear = function (keys, ls, rs) {
return;
before(() => {
LocalStorage.clear();
Login();
})
Control of cookie clearing is supported by Cypress: https://docs.cypress.io/api/cypress-api/cookies.html
I'm not sure about local storage, but for cookies, I ended up doing the following to store all cookies between tests once.
beforeEach(function () {
cy.getCookies().then(cookies => {
const namesOfCookies = cookies.map(c => c.name)
Cypress.Cookies.preserveOnce(...namesOfCookies)
})
})
According to the documentation, Cypress.Cookies.defaults will maintain the changes for every test run after that. In my opinion, this is not ideal as this increases test suite coupling.
I added a more robust response in this Cypress issue: https://github.com/cypress-io/cypress/issues/959#issuecomment-828077512
I know this is an old question but wanted to share my solution either way in case someone needs it.
For keeping a google token cookie, there is a library called
cypress-social-login. It seems to have other OAuth providers as a milestone.
It's recommended by the cypress team and can be found on the cypress plugin page.
https://github.com/lirantal/cypress-social-logins
This Cypress library makes it possible to perform third-party logins
(think oauth) for services such as GitHub, Google or Facebook.
It does so by delegating the login process to a puppeteer flow that
performs the login and returns the cookies for the application under
test so they can be set by the calling Cypress flow for the duration
of the test.
I can see suggestions to use whitelist. But it does not seem to work during cypress run.
Tried below methods in before() and beforeEach() respectively:
Cypress.Cookies.defaults({
whitelist: "token"
})
and
Cypress.Cookies.preserveOnce('token');
But none seemed to work. But either method working fine while cypress open i.e. GUI mode. Any ideas where I am coming short?
2023 Updated on Cypress v12 or more:
Since Cypress Version 12 you can use the new cy.session()
it cache and restore cookies, localStorage, and sessionStorage (i.e. session data) in order to recreate a consistent browser context between tests.
Here's how to use it
// Caching session when logging in via page visit
cy.session(name, () => {
cy.visit('/login')
cy.get('[data-test=name]').type(name)
cy.get('[data-test=password]').type('s3cr3t')
cy.get('form').contains('Log In').click()
cy.url().should('contain', '/login-successful')
})
I'm running an AngularJS app and I'm using the grunt-contrib-connect plugin to host my stuff. I'd like some PDF files to be accessed in the browser (tested in all of them), but the pages stay blank. It does show the correct amount of pages, though. I've exported the PDF from a .doc-file using both Word and Pages, but the result stays the same.
I wrote this little middleware snippet to ensure the headers are set (and they are, according to Chrome's Network tab), but the pages stay blank. Any help?
In my Gruntfile;
grunt.initConfig({
settings: {}, // ...
connect: {
livereload: {
middleware: function(connect) {
var middlewares = [];
// Other middlewares
middlewares.push(function (req, res, next) {
if (~req.url.indexOf('.pdf')) {
res.setHeader('Content-type', 'application/pdf');
}
return next();
});
return middlewares;
}
}
}
});
It seems grunt-contrib-connect has problems serving binary files: https://github.com/gruntjs/grunt-contrib-connect/issues/142