Get the on-screen location of web page with Selenium WebDriver - selenium

Question

Is there a way to get the on-screen coordinates of HTML window (page body) with Selenium WebDriver?

Answer 1

Seen this a few times and haven't found an elegant solution from WebDriver yet (they have a param that looks to support in in their ILocatable settings but the method is not implemented yet).
What I do is use UIAutomation to get the windows AutomationElement and use a treewalker to find the actual object of the window - downside is I noticed the browsers occasionally update what their window is so the conditionals have to change every once in awhile to accommodate.
Here is some example code (I removed some company code here so it's more elegant on my end but this should work for C#)
public static Rectangle GetAbsCoordinates(this IWebElement element)
{
var driver = GetDriver(element);
var handle = GetIntPtrHandle(driver);
var ae = AutomationElement.FromHandle(handle);
AutomationElement doc = null;
var caps = ((RemoteWebDriver) driver).Capabilities;
var browserName = caps.BrowserName;
switch (browserName)
{
case "safari":
var conditions = (new AndCondition(new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Pane),
new PropertyCondition(AutomationElement.ClassNameProperty, "SearchableWebView")));
doc = ae.FindFirst(TreeScope.Descendants, conditions);
break;
case "firefox":
doc = ae.FindFirst(TreeScope.Descendants, new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Document));
break;
case "chrome":
doc = ae.FindFirst(TreeScope.Descendants, new PropertyCondition(AutomationElement.NameProperty, "Chrome Legacy Window"));
if (doc == null)
{
doc = ae.FindFirst(TreeScope.Descendants, new PropertyCondition(AutomationElement.NameProperty, "Google Chrome"));
if (doc == null)
throw new Exception("unable to find element containing browser window");
doc = doc.FindFirst(TreeScope.Descendants, new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Document));
}
break;
case "internet explorer":
doc = ae.FindFirst(TreeScope.Descendants, new AndCondition(new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Pane),
new PropertyCondition(AutomationElement.ClassNameProperty, "TabWindowClass")));
break;
}
if (doc == null)
throw new Exception("unable to find element containing browser window");
var iWinLeft = (int) doc.Current.BoundingRectangle.Left;
var iWinTop = (int)doc.Current.BoundingRectangle.Top;
var coords = ((ILocatable) element).Coordinates;
var rect = new Rectangle(iWinLeft + coords.LocationInDom.X, iWinTop + coords.LocationInDom.Y, element.Size.Width, element.Size.Height);
return rect;
}
public static IWebDriver GetDriver(this IWebElement e)
{
return ((IWrapsDriver)e).WrappedDriver;
}
public static IntPtr GetIntPtrHandle(this IWebDriver driver, int timeoutSeconds = Timeout)
{
var end = DateTime.Now.AddSeconds(timeoutSeconds);
while(DateTime.Now < end)
{
// Searching by AutomationElement is a bit faster (can filter by children only)
var ele = AutomationElement.RootElement;
foreach (AutomationElement child in ele.FindAll(TreeScope.Children, Condition.TrueCondition))
{
if (!child.Current.Name.Contains(driver.Title)) continue;
return new IntPtr(child.Current.NativeWindowHandle);;
}
}
return IntPtr.Zero;
}

Answer 2

The posted code by Zechtitus is amazing, I tried it under IE11 and Chrome Version 39.0.2171.95 m and it worked like a charm. Although I had to pass the real object of IWebDriver instead of using WrappedDriver because it doesn't work with Chrome. Just for your info, I have Win 7 ultimate x64 and using Selenium WebDriver 2.44. this is the code that I took it from Zechtitus and modified it:
public static Rectangle GetAbsCoordinates(IWebDriver driver, IWebElement element)
{
var handle = GetIntPtrHandle(driver);
var ae = AutomationElement.FromHandle(handle);
AutomationElement doc = null;
var caps = ((RemoteWebDriver)driver).Capabilities;
var browserName = caps.BrowserName;
switch (browserName)
{
case "safari":
var conditions = (new AndCondition(new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Pane),
new PropertyCondition(AutomationElement.ClassNameProperty, "SearchableWebView")));
doc = ae.FindFirst(TreeScope.Descendants, conditions);
break;
case "firefox":
doc = ae.FindFirst(TreeScope.Descendants, new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Document));
break;
case "chrome":
doc = ae.FindFirst(TreeScope.Descendants, new PropertyCondition(AutomationElement.NameProperty, "Chrome Legacy Window"));
if (doc == null)
{
doc = ae.FindFirst(TreeScope.Descendants, new PropertyCondition(AutomationElement.NameProperty, "Google Chrome"));
if (doc == null)
throw new Exception("unable to find element containing browser window");
doc = doc.FindFirst(TreeScope.Descendants, new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Document));
}
break;
case "internet explorer":
doc = ae.FindFirst(TreeScope.Descendants, new AndCondition(new PropertyCondition(AutomationElement.ControlTypeProperty, ControlType.Pane),
new PropertyCondition(AutomationElement.ClassNameProperty, "TabWindowClass")));
break;
}
if (doc == null)
throw new Exception("unable to find element containing browser window");
var iWinLeft = (int)doc.Current.BoundingRectangle.Left;
var iWinTop = (int)doc.Current.BoundingRectangle.Top;
var coords = ((ILocatable)element).Coordinates;
var rect = new Rectangle(iWinLeft + coords.LocationInDom.X, iWinTop + coords.LocationInDom.Y, element.Size.Width, element.Size.Height);
return rect;
}
public static IntPtr GetIntPtrHandle(this IWebDriver driver, int timeoutSeconds = 20)
{
var end = DateTime.Now.AddSeconds(timeoutSeconds);
while (DateTime.Now < end)
{
// Searching by AutomationElement is a bit faster (can filter by children only)
var ele = AutomationElement.RootElement;
foreach (AutomationElement child in ele.FindAll(TreeScope.Children, Condition.TrueCondition))
{
if (!child.Current.Name.Contains(driver.Title)) continue;
return new IntPtr(child.Current.NativeWindowHandle); ;
}
}
return IntPtr.Zero;
}
and I used it like this:
Rectangle recView = GetAbsCoordinates(MyWebDriverObj, myIWebElementObj);
the correct X, Y are then stored in recView.X and recView.Y
As I said, it's working for me for both IE11 and Chrome.
Good luck

Answer 3

hmmm, I cannot directly comment to the one user asking about chrome so I will have to add another comment here.
Basically for UIAutomation you will want to get your hands on a tool called inspect (comes free in the 8.1 SDK). Older tools like uispy would probably work as well.
Basically you would fire up chrome and then fire up the inspector tool - your going to look at the tree like structure and then navigate down to the document which contains the DOM. Turn on highlighting in the tool to make this easier.
Chrome is quite dynamic it seems in the layout of the tree controls - have had to modify it a few times to accomodate the control I am looking at. If your using a different version than I had - basically find the document window in the tree and take a look at all of the control patterns associated with it - this is what I am passing into the PropertyCondition for how to search for the control. Intellisense should bring you up different things to query for like AutomationElement.NameProperty. It the example I had - I noticed there is a difference between when I run chrome on a winXP machine vs a win8 machine... hence the checking for null.
Like I have said before - this is not elegant and would be awesome if it was inbuilt into Selenium (I imagine they have much better methods for determining the coords of the DOM area)... I think this will also be problematic for people moving to Selenium Grid (like I am looking at doing) - far as I know with using it I don't know if you can shuttle over a bunch of supporting dll to selenium to the remote machine... at least without a lot of hacks.
If it still doesn't work for you - give me a specific idea on the OS, Chrome version and I'll try to take a look and give exact Property match. Probably best though if you fiddle yourself as these things are not static unfortunately.

Answer 4

Yes. Its possible. With little trick. Find below my code to get on
screen top position of web element.
public static long getScrollYPosition() {
WebDriver driver = DriverFactory.getCurrentDriver();
JavascriptExecutor jse = (JavascriptExecutor) driver;
Long scrollYPos = (Long) jse.executeScript("return window.scrollY;");
return scrollYPos;
}
long scrollPosition = getScrollYPosition();
long elemYPositionOnScreen = (long) elem.getLocation().getY() - scrollPosition;

Answer 5

you can try in this way:
WebDriver driver=new FirefoxDriver();
driver.get("http://www.google.com");
JavascriptExecutor js=(JavascriptExecutor) driver;
Double i= (Double) js.executeScript("var element = document.getElementById('hplogo');var position = element.getBoundingClientRect();return position.left");
System.out.print(i);

Answer 6

I took a quick look at chrome and you may have better luck with the following.
doc = win.Find.ByConditions(new PropertyCondition(AutomationElement.ClassNameProperty, "Chrome_RenderWidgetHostHWND"));
I think that class name is consistent for chrome... seems to work on older and newer OS's for me - chrome version 34.0.1847.116m. Hope that helps.

Answer 7

This should work once it's supported:
WebElement htmlElement = driver.findElement(By.tagName("html"));
Point viewPortLocation = ((Locatable) htmlElement).getCoordinates().onScreen();
int x = viewPortLocation.getX();
int y = viewPortLocation.getY();
However right now it's throwing the following error:
java.lang.UnsupportedOperationException: Not supported yet.
at org.openqa.selenium.remote.RemoteWebElement$1.onScreen(RemoteWebElement.java:342)
(on org.seleniumhq.selenium:selenium-java:2.46.0)

Answer 8

I needed a this in Robot Framework and I was inspired by Jeyabal's solution, so here is an adaptation that works for me:
${verticalWindow}= Execute Javascript return window.scrollY;
${verticalElement} = Get Vertical Position /xpath
${hasScrolled} = Evaluate (${verticalElement} - ${verticalWindow}) == 0

Answer 9

Nothing from above worked for me. A workaround is to use window.innerHeight and window.innerWidth and work your way up from the the left, bottom corner. This assumes that the browser bottom border is almost 0 (no horizontal scrollbar or thick window decoration).
win_pos = selenium.get_window_position()
win_size = selenium.get_window_size()
win_bottom_y = win_pos['y'] + win_size['height']
# We assume viewport x == window x. For y coordinate we take the bottom
# of the browser and subtract the viewport height
viewport_height = selenium.execute_script('return window.innerHeight')
viewport_width = selenium.execute_script('return window.innerWidth')
viewport_y = win_bottom_y - viewport_height
This is not 100% accurate but it's a good workaround that can be tweaked for your case.

Answer 10

SOOO many factors have to be considered to get the element position relative to the screen. For the longest time I was using the UIAutomation code above, but UIAutomation is unreliable, it crashes or fails to find the browser (for some reason), with EdgeDriver, tabs crash consistently making getting the values via UIAutomation now as a 'fallback'.
That said, when it works, the answer is GOLDEN as to the on-screen coordinates of the HTML page.
However, something that always works is using javascript. So we calculate that first, then also attempt to call UIAutomation. If UIAutomation fails, we use this code's answer. If UIAutomation works, we use the UIAutomation values.
// use javascript to get our html document location, but it's off by 2 pixels compared to uiautomation. uiautomation, however, failed so often that is was unreliable
int outerHeight = Int32.Parse(BrowserHelper.ExecuteJavascript(browser, "return window.outerHeight"));
int innerHeight = Int32.Parse(BrowserHelper.ExecuteJavascript(browser, "return window.innerHeight"));
int outerWidth = Int32.Parse(BrowserHelper.ExecuteJavascript(browser, "return window.outerWidth"));
int innerWidth = Int32.Parse(BrowserHelper.ExecuteJavascript(browser, "return window.innerWidth"));
int browserNavHeight = outerHeight - innerHeight;
int browserNavWidth = outerWidth - innerWidth;
iWinLeft = browserNavWidth + 2;
iWinTop = browserNavHeight + 2;
`

Answer 11

Try this, I hope it will help you :
Rectangle rec = new Rectangle(element.getLocation(), element.getSize());