Strategy to use Selenium browser testing with Flutter Web apps [duplicate] - selenium

Question

we want to use Appium/Selenium to do automated testing on a Flutter application. Some elements do not have selectors when viewed in Selenium. In Android we just add ids onto every element and they appear in Appium. How do we do this in a flutter environment?

Answer 1

I found an approach with a workaround which then lets you use Selenium reasonably naturally with Flutter Web (although not working with headless browser)
You need to find the offset of window x y coordinates from screen x y coordiantes. I found this idea in another thread
pageCallibrator.html:
<script>
window.coordinates = [];
document.addEventListener('click', function() {
window.coordinates = [event.pageX, event.pageY];
});
</script>
Then in Selenium setup before running tests (Java example)
int windowScreenOffsetX = 0;
int windowScreenOffsetY = 0;
void callibrateXY(WebDriver driver) {
driver.get("http://localhost:8080/pageCallibrator.html"); //TODO adjust host
Dimension size = driver.manage().window().getSize();
int x = size.width / 2;
int y = size.height / 2;
clickMouseAtXY(x, y);
try {
Thread.sleep(500);
} catch (InterruptedException e) {
}
List<Object> coordinates = (List<Object>) ((JavascriptExecutor) driver).executeScript("return window.coordinates;");
windowScreenOffsetX = x - (int) (long) coordinates.get(0);
windowScreenOffsetY = y - (int) (long) coordinates.get(1);
}
Now in Selenium to press a Flutter button
WebElement continueToBankButtonElement = findElementWithText(driver, "My button text");
clickMouseAtElement(continueToBankButtonElement);
where you define
import org.openqa.selenium.*
Robot robot = new Robot();
Driver driver = new ChromeDriver(options); // TODO handler exceptions and options in a method
WebElement findElementWithText(WebDriver driver, String text) {
return driver.findElement(containsTextLocator(text));
}
By containsTextLocator(String text) {
return By.xpath("//*[contains(text(), '" + text + "')]");
}
void clickMouseAtElement(WebElement element) {
clickMouseAtXY(element.getLocation().getX() + element.getSize().width / 2, element.getLocation().getY() + element.getSize().height / 2);
}
void clickMouseAtXY(int x, int y) {
moveMouse(x, y);
robot.mousePress(InputEvent.BUTTON1_DOWN_MASK);
robot.mouseRelease(InputEvent.BUTTON1_DOWN_MASK);
}
/**
* #param x
* #param y
*/
protected void moveMouse(int x, int y) {
robot.mouseMove(x + windowScreenOffsetX, y + windowScreenOffsetY); // Offset of page from screen
}

Answer 2

Prior to this morning I knew nothing of Flutter. A few hours later and I can safely say "you don't." While Flutter makes developing an application quick and easy, it removes a lot of the control you have, including the level of customization you're looking for.
There are hits on this on official Flutter message boards dating back a year or two, but there were no answers.
You could attempt locating everything by text? Kluge, difficult or impossible to maintain, but likely your only option at this point.