I'm evaluating TestCafe as a potential UI testing tool for our team. The target browser is Safari. Is that correct that headless tests cannot be run with Safari?
There is not a headless mode for Safari. It isn't unique to TestCafe; you won't be able to run in headless mode in Safari with any testing framework.
Source: https://discussions.apple.com/thread/251837694
I just updated to TestCafe v1.18.0 and I followed the upgrade guide. I am able to run tests against Chrome and Firefox, as the system correctly prompted me if I wanted to let TestCafe control these, but this is not happening for Safari, so I am unable to test against it. I don't see any way to manually add an entry in the Automation tab either.
Automation Tab entry for TestCafe Browser Tools
Edit: I managed to add it using the remote option for TestCafe and pasting the URL into Safari, but even after that, npx testcafe safari is unable to launch Safari. I'm using an M1 Mac and macOS Monterey 12.0.1
TestCafe 1.18.0 does have the Safari-related issue. It has already been fixed. Please update TestCafe to v1.18.1.
I am automating a simple login script through Browserstack using Selenium.
The script I have written works on chrome windows, chrome mac os, and Android Samsung. However, when I run the script through apple iPhone 11 or mac os safari I get an error message saying can not locate
driver.findElement(By.xpath("//*[contains(text(),'Log in')]")).click();
Apart from the device, I am automating nothing else changes.
What is the fix for this?
If the test returned "cannot find element" on the device, I'll recommend changing the element locator strategy.
You could otherwise also try to inspect the app against the same browser manually via Live: https://live.browserstack.com/
This could help you narrow down if the element locator is correct.
I'm looking at writing some E2E tests using Playwright. I can see that the library allows tests to be run against Chromium, Firefox and Webkit. My understanding is that Webkit is the underlying engine of Safari, and Chromium is the base of Chrome (and other browsers).
When it comes to testing against Webkit/Chromium, how close is it to testing against the user facing browsers which use the underlying engines? I'd imagine that there would be some quirks which the user can run in to whilst using Safari and it wouldn't be caught when running the tests on Webkit.
Thanks
Stock browsers like Google Chrome, Apple Safari embed rendering engines (Chromium, WebKit) and add stuff on top of them. In particular, they add proprietary media codecs, inject browser extensions, etc. They also add surrounding interfaces such as bookmarks sync. But they reuse the underlying web platform implementation.
Chromium
Chromium is the open source web platform implementation used by Google Chrome, Opera, Microsoft Edge and other browsers. It implements web specs, renders content, works with network, etc. Playwright uses a stock Chromium build that can be automated with the Playwright API for e2e testing.
For Google Chrome things are simple: Chromium is a safe target to test, modulo proprietary media codecs and DRM. You can point Playwright against stock Chrome Canary or Edge Canary to use proprietary media codecs.
WebKit
WebKit is the open source web platform implementation used by Apple Safari and Epiphany. As of June 2020, Playwright provides a WebKit build that can be automated with the Playwright API for e2e testing. Playwright WebKit works across all platforms (macOS, Linux, Windows), in both headless and headful modes.
When WebKit runs on macOS, it is a safe target to test Safari. WebKit on Linux and Windows differs from Apple Safari in the following ways: it uses a non-macOS network stack, uses non-Core Animation to composite scene and produce image raster. This means that screenshots on Linux and Windows will not perfectly match screenshots from macOS. In terms of the web platform, the same WebKit code will layout the page and run JavaScript—it will match how WebKit works in Safari.
To conclude, we consider the browsers provided by Playwright to be the best of what you can get for e2e testing. Running WebKit on a Linux CI would use a different compositing pipeline than macOS, but it can be used for e2e testing of web applications in WebKit-powered browsers.
I've found there are some differences between webkit and Safari depending on which version of the two you're testing against.
I was testing Safari 16.0, and for my purposes Playwright 1v17 seemed to match very well. Upgrading to Safari 16.2 didn't seem to affect that.
But upgrading Playwright to 1v28 though seemed to change the behaviour around getting selection bounding boxes. It's seems more correct, but it's no longer consistent with Safari 16.2.
I'm assuming that WebKit is a bit ahead of Safari, and at some point Safari will catch up, and we'll see they are behaving the same again, but so far that's not happened. (Or maybe it has already, I'm a couple of versions behind at this point).
My solution has been to use both the latest version of Playwright for testing Chromium and Firefox, but use the older version for testing Safari. I am also testing webkit from Playwright 1v28 for interest right now, but I expect Safari will start showing that behaviour at some point.
(NB: Getting the bounding boxes of selections gets different results from each browser, so it's not too surprising that it's changing in WebKit right now. The change seems to be closer to Chrome's behaviour but still not the same).
In case it helps anyone, in my package.json devDependencies I have
"playwright1v17": "npm:playwright#~1.17.2",
"playwright1v28": "npm:playwright#~1.28.1",
and then in the test script I do
import { chromium, webkit, firefox } from 'playwright1v28';
import { webkit as safari16v1 } from 'playwright1v17';
That lets me test both versions of webkit and compare, as well as Chromium and Firefox.
Can I run headless safari browser via windows?
I did not tried it yet but it's just I wanted to check if it's possible. If anyone have what all I need.
The short answer to this is no. The reason for this is an issue with Safari specifically, not Selenium. You can find an explanation on this relevant GitHub issue opened under the Selenium repo: https://github.com/SeleniumHQ/selenium/issues/5985
The GitHub issue is marked as closed by the Selenium developers, because Safari application itself does not support headless mode.