In Playwright, can't use page.goto with headless webkit - webkit

I'm trying to write a test for a Drupal website in Playwright. I'm running on Ubuntu 20.04 via WSL2.
I set a base url (https://www.example.com) in my Playwright config file.
I have a few simple steps:
await this.page.goto('/user/logout');
await this.page.goto('/user/login');
await this.page.locator('input[name="name"]').fill('admin#example.com');
await this.page.locator('input[name="pass"]').fill('password');
await this.page.locator('input:has-text("Login")').click();
await this.page.waitForSelector('text="You are logged in"');
await this.page.goto('/admin/structure/webform/manage/myform', { waitUntil: 'networkIdle' });
And I wanted to test on a simulated iPhone, so I used this in my config file:
name: 'iPhone 6/7/8',
use: devices['iPhone 8'],
The problem is that in this iPhone 8 simulated device in headless mode only, the last goto fails (await this.page.goto('/admin/structure/webform/manage/myform')). It happens 100% of the time when I run the test.
The test works fine on webkit (desktop), chrome (desktop), firefox (desktop), and android (mobile). The test also works on iOS when not headless. And the same steps work on a real iOS device.
I needed to add networkIdle to the final step to get it to work on headless Chrome (both desktop and Android), so I'm guessing I might need to make another change to get it to work on iOS, but what?
When I watch the video of my test in action, when it gets to the final step, the screen starts flickering very rapidly for about 3 seconds and then cuts to black.
Additional info
Based on the list of Playwright devices, I copied the code for iPhone 8 and experimented to see what specifically causes the error.
As shown below, changing isMobile from true to false will allow the tests to pass. So the specific error seems to be related to whatever setting isMobile to true does.
projects: [
{
name: 'iPhone 6/7/8',
use: {
userAgent:
'Mozilla/5.0 (iPhone; CPU iPhone OS 11_0 like Mac OS X) AppleWebKit/604.1.38 (KHTML, like Gecko) Version/15.4 Mobile/15A372 Safari/604.1',
browserName: 'webkit',
viewport: {
width: 375,
height: 667,
},
deviceScaleFactor: 2,
isMobile: false,
hasTouch: true,
defaultBrowserType: 'webkit',
},
},
],
Playwright bug report filed here.

As described in the playwright issue, this behavior seems to be caused by a failure in the OS (in my case, Ubuntu running on WSL2).
This was solved by updating all packages in ubuntu; unfortunately, there were several packages and I'm not sure which was the exact fix, but the takeaways are:
Ubuntu and WSL2 sometimes have graphical issues that can interfere with playwright running tests.
If you start seeing issues, update ubuntu, not just package.json.

Related

Parsehub "This Stencil app is disabled for this browser."

I need to scrape some data from transfermarkt.com using parsehub, but when i try to load the website with parse hub I'm only met with:
This Stencil app is disabled for this browser.
Developers:
ES5 builds are disabled during development to take advantage of 2x faster build times.
Please see the example below or our config docs if you would like to develop on a browser that does not fully support ES2017 and custom elements.
Note that as of Stencil v2, ES5 builds and polyfills are disabled during production builds. You can enable these in your stencil.config.ts file.
When testing browsers it is recommended to always test in production mode, and ES5 builds should always be enabled during production builds.
This is only an experiment and if it slows down app development then we will revert this and enable ES5 builds during dev.
Enabling ES5 builds during development:
npm run dev --es5
For stencil-component-starter, use:
npm start --es5
Enabling full production builds during development:
npm run dev --prod
For stencil-component-starter, use:
npm start --prod
Current Browser's Support:
ES Module Imports: false
ES Dynamic Imports: false
Custom Elements: false
Shadow DOM: false
fetch: true
CSS Variables: true
Current Browser:
Mozilla/5.0 (Macintosh; Intel Mac OS X 10.13; rv:65.0) Gecko/20100101 Firefox/65.0
I tried following the steps to enable ES5, but it does not work.
If i go to the website on the standard firefox browser it works like normal, but not in parsehub
That message indicates that the browser does not support JS modules. Looking at parsehub's FAQ they are using Firefox v54 which was released in 2017 and does not support JS modules.
Starting with version 2 Stencil has changed legacy browser support to be opt-in.
If you have access to the source code you can add legacy browser support using the following config:
export const config: Config = {
buildEs5: 'prod',
extras: {
cssVarsShim: true,
__deprecated__dynamicImportShim: true,
shadowDomShim: true,
safari10: true,
scriptDataOpts: true,
appendChildSlotFix: false,
cloneNodeFix: false,
slotChildNodesFix: true,
}
};
The only other options are:
ask the site to add those config changes and publish a new version
use a different scraper with a more recent version

Disable file downloads Selenium Firefox

I'm building a webscraping app and I'm using Selenium with a Firefox driver to open my pages. Whenever it opens a link that leads to a download, my app stalls and hangs on the link for eternity.
I've tried looking for a solution, but they never mention disabling downloads. They only talk about enabling them, changing the download directory,...
Would it be possible to detect that the link is a download link and just skip it, or maybe to skip the link whenever it opens?
I had a slightly different problem and I solved it by changing Firefox capabilities.
Open about:config in Firefox to see list of them. Then you can pass them to webdriver through desired_capabilities.
caps = {'acceptInsecureCerts': True,
'browserName': 'firefox',
'marionette': True,
'moz:firefoxOptions': {
'args': ['--no-remote'],
'prefs': {
'browser.safebrowsing.downloads.enabled': False
}
},
driver = webdriver.Remote([...]
desired_capabilities=caps)

Nightwatch tests do not find elements in Headless chrome mode

I am trying to get my tests (written in Nightwatch.js) to run locally in headless chrome.
However, the tests fail since they are not able to find the elements in headless mode (they work without headless mode though).
If I check the failure screenshots I only get a white screen.
But if test checks for the "body" element, it actually pass. So I think the page is loaded, but maybe headless chrome, for some reason, cannot load the javascript? Later I wait for divs and buttons etc to be visible for several seconds, but it does not find them.
Do you have any ideas what could be wrong?
I have added the --headless and --disable-gpu flags in desiredCapabilities in the nightwatch config file.
So, as I said in my reply I had the same issue yesterday, I was just trying to do a dummy test on google homepage. This morning with a fresh brain I tried to tackle that. I had the brilliant idea to take a screenshot just before nightwatch fails to find an element.
It turns out the "normal" chrome had the home page in English, and the "headless" chrome had the homepage in french, for some reason.
I found this: https://bugs.chromium.org/p/chromedriver/issues/detail?id=1925 may be explaining why.
The workaround I found to always have the correct language is:
"chromeOptions" : {
"args" : ["--lang=fr", "--headless"]
}
I still have a hard time setting it in english (weirdly) but I hope this will save someone a couple of hours in the future
I think you should declare binary path in Nightwatch.js
If you are on linux please try this, it works perfectly for me :
"desiredCapabilities": {
"browserName": "chrome",
"javascriptEnabled": true,
"acceptSslCerts": true,
"chromeOptions": {
"args": [
"headless", "disable-gpu"
],
"binary": "/usr/bin/google-chrome"
}
}
If you are on Mac, replace your binary path, eg /Applications/Google\ Chrome.app/Contents/MacOS/Google\ Chrome

Test browser code with Intern through Grunt and Phantomjs

I have been playing with Intern and made my tests work in the browser. To better integrate with my current workflow I'm trying to run my tests through grunt with phantomjs. However all my attempts have failed. I've been looking at this question as a reference Intern WebDriver and PhantomJS but can't figure out all of the steps to make it work.
First of all: Is it even possible to run the tests through grunt and phantomjs?
A little bit of info:
I don't want/can't connect to Sauce Labs or a Selenium testing environment.
I want to test browser code while having jQuery as a shimmed dependency
I have Grunt 0.4.1 and phantomjs 1.9.1 installed
I'm not testing any ajax request (as the linked question is having problem with)
I'm not familiar with neither Selenium nor Sauce Lab
If it is possible to test through grunt and phanomjs, how would I go about doing it?
I guess that I have to start the GhostDriver
phantomjs --webdriver=8910
But then what are the important pieces of info in the Intern config to make this work?
define({
proxyPort: 9000,
proxyUrl: 'http://localhost:9000/',
environments: [
{
browserName: 'phantom',
version: '1.9.0',
platform: 'Linux'
}
],
webdriver: {
host: 'localhost',
port: 8910
},
maxConcurrency: 3,
useSauceConnect: false,
// ...
});
How does the environments browserName map to phantomjs? I've tried the browserNames 'phantom' as well as 'phanomjs' with different versions and platforms (running Mac 10.7)
My Gruntfile section for intern looks like:
intern: {
phantom: {
options: {
config: 'tests/intern',
reporters: ['webdriver']
}
}
}
Running this setup without any test-suite that includes browser code outputs
'ReferenceError: document is not defined' in 'node_modules/intern/lib/reporters/webdriver.js:41:19'
Adding browser tests gives 'ReferenceError: window is not defined' in 'src/vendor/jquery/jquery-1.9.1.js:9597:5'
Running it through node gives the same 'window is not defined' error.
node node_modules/intern/client.js config=tests/intern
What am I doing wrong/missing here?
There are two problems with your Gruntfile configuration:
The default run type for the Grunt task is 'client', which means it runs the Node.js client by default, not the test runner. You need to set runType: 'runner' in your Gruntfile options to run tests against your specified environments.
The webdriver reporter is for use in a browser client only and is specified automatically by the test runner when it is needed. You should typically never use it directly. The reporter you specify in the Gruntfile should be the one you want the test runner to use; the console default is usually appropriate.
These two things in combination mean that you’ve configured Intern to try to use a reporter that only works in a browser in conjunction with the test runner inside the Node.js client, hence the error. Once corrected, the remaining configuration should work properly to run on PhantomJS, assuming it is already running on localhost at port 8910.

E2E Testing for AngularJS App on another computer

I'm developing single page application on AngularJS for learning. My project is located on Apache HTTP Server on another computer, I use WinSCP synchronisation while developing so that it is always the last version of my work.
Halfway through (actually, when I has already finished the biggest part the application), I realized that I don't have any tests and I should learn how to test what I do not just manually. I decided to try writing E2E tests for my AngularJS application using Karma Test Runner.
I installed Karma via npm, initialized it (karma init test/karma.conf.js), but what happens now?
I tried karma start test/karma.conf.js it launches Chrome (as I stated in config) and says that
Karma - connected
Chrome 26.0 (Windows) is idle
even though in conf file there are specified my test file:
files = [
'test/first_test.js'
];
And that's what inside it:
describe('my app', function() {
browser().navigateTo('/');
it('should then be.', function() {
expect(browser().location().url()).toBe('/login');
});
});
Thanks in advance!
EDIT: I realized, it's not just 'Chrome is idle', there's also console log error:
Uncaught ReferenceError: browser is not defined
Any ideas? I'm so confused right now.
Browser is only defined inside of beforeEach. Try this:
describe('my app', function() {
beforeEach(function(){
browser().navigateTo('/');
});
it('should then be.', function() {
expect(browser().location().url()).toBe('/login');
});
});
Alright, looks like I solved it myself.
I should have added
ANGULAR_SCENARIO,
ANGULAR_SCENARIO_ADAPTER,
to the files property of karma config file. After that, I progressed a little bit, but still got a lot of troubles, but essentially the main was that I got error that resumeBootstrap was undefined. My app was using AngularJS 1.0.4, but it looks like Karma's Adapters are for 1.0.6+ only. Upgrading app to 1.0.6 helped with resumeBootstrap.
Regarding testing the app on external server:
proxies = {
'/': 'http://another.internal/app/'
};
and don't forget to change links to CSS and JS files in app's index.html from local (css/style.css) to web (//another.internal/app/css/style.css).