I have an online calculator that i want to automate some operations for, like subtraction, division, etc. but the thing is that there are no elements since it is a canvas app calculator (link below). I'm wondering how can i click the buttons so im able to automate some operations?
The online calculator im trying to automate:
https://www.online-calculator.com/full-screen-calculator/
The canvas HTML code
<canvas id="canvas" width="335" height="434" style="position: absolute; display: block; background-color: rgb(255, 255, 255); width: 167.939px; height: 217px;"></canvas>
My Selenium-Java code
driver.get("https://www.online-calculator.com/full-screen-calculator/");
driver.manage().window().maximize();
WebElement li = driver.findElements(By.tagName("iframe")).get(0);
driver.switchTo().frame(li);
WebElement canv = driver.findElements(By.tagName("canvas")).get(0);
System.out.println(canv.getSize());
System.out.println(canv.getLocation());
try {
Actions builder = new Actions(driver);
builder.moveToElement(canv, x, y);
builder.click();
builder.perform();
} catch (Exception e)
{
// do nothing
}
So as you see in above code, I still have to find the x and y of the operations i want to run. For ex, how can i find the coordinates of the buttons 10, 4 and the subtraction operation '-', if I want to calculate '10 - 4' and then validate that it is equal to 6. Any help would be highly appreciated. Thanks :)
Note: The canvas width, height and location will change if the window size changes ((Im thinking later of locking the window size so my tests are not flaky on different screens)).
The <canvas> element is within an <iframe>. So to invoke click() on the elements within the <canvas> you have to:
Induce WebDriverWait for the desired frame to be available and switch to it.
Induce WebDriverWait for the desired element to be clickable.
You can use the following solution:
Code Block:
driver.get("https://www.online-calculator.com/full-screen-calculator/")
new WebDriverWait(driver, 10).until(ExpectedConditions.frameToBeAvailableAndSwitchToIt(By.id("fullframe")));
WebElement canvas = new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.id("canvas")));
HTML5 Canvas
The element is only a container for graphics and is a rectangular area on an HTML page. By default, a canvas has no border and no content. However, an id attribute (to be referred to in a script), a width and height attribute are specified to define the size of the canvas. To add a border, the style attribute is used. An example:
<canvas id="myCanvas" width="200" height="100" style="border:1px solid #000000;"></canvas>
The HTML canvas is a two-dimensional grid. The upper-left corner of the canvas has the coordinates (0,0).
In the article Automated Testing of HTML5 Canvas Applications with Selenium WebDriver #Aaron Mulder mentions, interacting with the elements within <canvas> is possible using event support of the Actions Class API:
moveToElement(WebElement target, int xOffset, int yOffset): Moves the mouse to an offset from the top-left corner of the element.
new Actions(driver).moveToElement(canvas, xWithinCanvas, yWithinCanvas).click().perform();
But is not 100% reliable as, every mouse down, mouse up, or mouse click happens at the center of the element. So the code above produces a mouse move event to the provided coordinates (x,y), then a mouse move event to the center of the <canvas>, then a mouse down, mouse up, and click at the center of the <canvas>. That should have been fine for a <button> but is not worth for a <canvas>, where you want to be able to click at a specific location.
The workaround, is to dispatch synthesized mouse events using JavaScript as follows:
// pageX and pageY are offsets which you need to know through mouse coordinates.
((JavascriptExecutor)driver).executeScript("var evt = $.Event('click', { pageX: " + x +
", pageY: " + (y + 55) + " } );" +
"$('#myCanvas').trigger(evt);");
However, to click on the elements within the <canvas> you can be at ease using firefox as the mouse move event works well in Firefox and you can avoid using the mouse coordinates as the event processing as follows:
new Actions(driver).moveToElement(
canvas, xWithinCanvas, yWithinCanvas).perform();
((JavascriptExecutor)driver).executeScript("$('#canvas').click();");
This usecase
To automate a substruction operation e.g. 3-1= using Selenium you can use the following solution:
Code Block:
driver.get("https://www.online-calculator.com/full-screen-calculator/");
new WebDriverWait(driver, 10).until(ExpectedConditions.frameToBeAvailableAndSwitchToIt(By.id("fullframe")));
WebElement canvas = new WebDriverWait(driver, 10).until(ExpectedConditions.elementToBeClickable(By.id("canvas")));
//clicking on 3
new Actions(driver).moveToElement(canvas, 0, 0).moveByOffset(0,(255/6)*3).click().build().perform();
//clicking on the substract sign (-)
new Actions(driver).moveToElement(canvas, 0, 0).moveByOffset((174/5)*2,(255/6)*3).click().build().perform();
//clicking on 1
new Actions(driver).moveToElement(canvas, 0, 0).moveByOffset(-(174/5)*4,(255/6)*3).click().build().perform();
//clicking on equals to sign (=)
new Actions(driver).moveToElement(canvas, 0, 0).moveByOffset((174/5)*4,(255/6)*4).click().build().perform();
Execution Video:
Reference
You can find a couple of relevant detailed discussion in:
How to perform Mouse Wheel scrolling over HTML5 Canvas in Selenium?
I think you are facing difficulty to find the coordinates of the element. To find them easily will have some plugins for each browser. For chrome, you can use Page Ruler and for Firefox, you can use MeasureIt. By using these tools you can get the coordinates of the particular element. After that, you can easily click on those elements. Watch the following video for how to use MeasureIt in firefox (follow from 11:45 minutes).
Finding the coordinates of particular Element using Firefox MeasureIt plugin
And your other concern is how to find the coordinates of the size of the screen changes? normally will have some standard screen sizes for each monitor or laptop.
So first, get the size of the screen using selenium and then you can use if the condition for each screen size. Means, if the size is something like 800*1200 then use these coordinates else use some other coordinates for each screen size.
I have a Capybara/Cucumber test running in firefox that will not click an svg element. I have the equivalent test working on other elements that are of the same type, but Capybara is telling me this error for this specific element:
Element is not clickable at point (1179.5, 172.96665954589844). Other element would receive the click: <svg height="124" width="290"></svg> (Selenium::WebDriver::Error::UnknownError)
The click looks like:
find("#partner-profit-chart svg g.pie-slice._1").click
And the actual site is hosted here http://mrr.devtechlab.com/mrr-dashboard.html, the element that it won't click is the third pie chart on the right. I am able to click the other pie charts just fine, but somehow Selenium thinks it will click the SVG containing the element for only this chart???
EDIT:
Ended up manually clicking the d3 element using the following(jquery click doesn't work on d3 elements FYI):
execute_script(
%Q(
jQuery.fn.d3Click = function () {
this.each(function (i, e) {
var evt = new MouseEvent("click");
e.dispatchEvent(evt);
});
};
$("#partner-profit-chart svg g.pie-slice._1 path").d3Click();
)
)
Selenium attempts to click in the middle of the bounding box of an element. The issue here is that with a highly concave shape the center of the bounding box isn't actually in the element, and therefore the click is going through to the encapsulating svg element. Since this page is using jQuery, your best bet may just be to use #execute_script to find the element and trigger click on it.
I am unable to extract or get the text from the popup Window. I am using the below code:
Driver.findElement(By.xpath("xpath of popup")).getText();
It is a div image popup (if that helps). I want get the text out the window and print it in the console. When I use the above code it just passes the test and does not get the text from the image.
You can't simply get text from an image. To get text from an image, you need OCR (optical character recognition).
The following should give you a starting point:
URL url = new URL(imageUrl);
Image image = ImageIO.read(url);
String text = new OCR().recognizeCharacters((RenderedImage) image);
My original question was confusing, I rewrote it. Now it is much clearer.
Hello I am trying to navigate web page using HtmlUnit. I am trying to
go to url -> enter text in text box (in frame 1) -> Press Search Button (in frame 1) -> go to new frame (frame 2) that appeared as result of search.
When I write this code:
HtmlPage page = webClient.getPage("http://someurl.com");
HtmlPage toolbar = (HtmlPage) page.getFrameByName("toolbar").getEnclosedPage(); //GET TOOLBAR FRAME
HtmlTextInput searchBox = toolbar.getElementByName("query"); //GET TEXT BOX ELEMENT
searchBox.setValueAttribute("SOME STRING"); //FILL TEXT BOX ELEMENT WITH TEXT
List<HtmlElement> elements1 = (List<HtmlElement>) toolbar.getByXPath(<SOME XPATH>); //LIST OF A SINGLE ELEMENT WHICH IS THE SEARCH BUTTON
HtmlElement element2 = elements1.get(0); // GET SEARCH BUTTON ELEMENT FROM LIST
page = element2.click(); // CLICK SEARCH BUTTON AND LOAD NEW PAGE TO "page"
I am searching for "SOME STRING". When I do this, another page comes up (though does not refresh the page, the URL never refreshes) with an ADDITIONAL frame. So now, I click the search button but thew new page is just a webpage of the toolbar frame, not of the whole webpage.
I want to now access the additional frame. So how do I "back out" of the current frame and select the new frame?
I found out the answer my trial and error.
page = (HtmlPage) searchButton.click().getEnclosingWindow().getTopWindow().getEnclosedPage();
page = (HtmlPage) page.getFrameByName("Frame 1").getEnclosedPage();
page = (HtmlPage) page.getFrameByName("Frame 2").getEnclosedPage();
It works great, hope I can help some of you out.
I want to simulate a simple mouse click and drag within an svg element.
I managed to get the coordinates of my starting and ending point, both absolute (window coordinates) and relative to the encapsulating svg element.
Here is the code I am using to simulate the mouse:
Actions builder = new Actions(driver);
builder.moveToElement(area, xStart, yStart);
builder.clickAndHold();
builder.moveToElement(area, xStop, yStop);
builder.release();
Action setFilter = builder.build();
setFilter.perform();
Where area is a WebElement representing my svg and the coordinates are relative to that element. Note that:
area.getLocation(); // returns null
This made me wonder whether the webdriver is able to find that element at all. So I tried with absolute coordinates:
Actions builder = new Actions(driver);
builder.moveByOffset(chart.getLocation().x + xStart, chart.getLocation().y + yStart);
builder.clickAndHold();
builder.moveByOffset(xStop - xStart, yStop - yStart);
builder.release();
Action setFilter = builder.build();
setFilter.perform();
where chart is the div surrounding the svg element (note that the offset between the div position and the svg position is only 10 pixels and is not significant). That didn't work either and I also tried by relative position to the div but still no luck.
What am I doing wrong here?
I got a hack for it to work but it requires the webpage to be opened on the foreground. If you're doing anything else at the same time it may break the test and I cannot say if it would work if ran remotely.
Here's what it looks like:
Robot robert = new Robot();
robert.mouseMove(xStart, yStart);
// full click once to get focus on the window
robert.mousePress(MouseEvent.BUTTON1_MASK);
robert.mouseRelease(MouseEvent.BUTTON1_MASK);
// then set the filter
robert.mousePress(MouseEvent.BUTTON1_MASK);
robert.mouseMove(xStop, yStop);
robert.mouseRelease(MouseEvent.BUTTON1_MASK);