I have a UI where yFiles based graphs are used.
What would be the best automation tool suitable to automate and test the same
yFiles for HTML can be automated using the usual tools for web UI automation with some caveats (see below). Internally we use the Selenium WebDriver API, but some other approaches work as well.
The main requirement for an automation tool is that events are reproduced faithfully like they would be by normal interaction with a browser. yFiles can be very sensitive to the correct order of input events, since we use a state machine that can get out of sync if, e.g., a mouse-down event isn't followed by a mouse-up event. Some testing approaches don't bother raising the correct events (e.g. only raising a click event while disregarding the mouse-down and -up events) and this can sometimes cause yFiles to not work as expected.
Related
I was just curious about how does Uipath process render GUI to interact with various application in unattended mode without screen. I am trying to build my own RPA system for few specific use cases but I am stuck at running those process in unattended. Because to interact with application(click etc) it requires GUI to render.
Thanks
According to this article (and a little bit simplified) they either use the console session (which is a well-known solution / workaround) or they create RDP Sessions programmatically using the FreeRDP framework. (I have tried my luck with FreeRDP but most of it's features are disabled in corporate environments)
If you really want to dig in the whole thing, Microsoft provides a framework for implementing own Remoting Solutions. Theoretically you could implement your own protocol with lower security boundaries and by not destroying the GUI if the remote session is not active (disconnected but not closed)
It's based on the coordinates of the controls and the text they contain. It recognizes graphical objects by their platform-specific attributes. In very particular scenarios, where object recognition is not available such as with RDP, it uses image and OCR text-based automation.
I am working on implementing accessibility (for visually impaired individuals) for one of our web application. It need to be ARIA compliant. Right now we are testing our changes with screen reader manually.
For example we have Tree control in our application. I open NVDA screen reader and then navigate through my Tree Nodes. NVDA screen reader speaks out
Node XYZ expanded, (When I expand XYZ node with right arrow key)
Node XYZ collapsed, (When I collapse XYZ node with left arrow key)
Along with the voice it also write down this text.
But all this is manual. Now we want to setup automated test cases for the same so that any regression bugs can be caught by are test cases. Do there exist any such tool which we can use to automate our test cases. Any direction will be helpful.
PS: Just for a sake of comparison. We have nunit to write test cases for c# application. After writing test cases we integrate them into our build process. Any breaking change is caught when we run the build. I am looking for something similar to test out our aria compliance and screen reader's behavior with our web application.
I don't know of any existing tools for testing screen readers, however, there are accessibility APIs that test websites and web applications.
axe-core from Deque Systems is widely used and well-supported.
I wrote a python package to run automated web accessibility tests that uses axe-core and selenium.
While it isn't quite what you are looking for, it does cover about 60% of accessibility guidelines, including aria roles and attributes. It should help with determining screen reader usability.
You could integrate axe into C#, similar to my python package and the Java package, also created by Deque.
I hope this helps!
It sounds like you're already performing some pretty good manual accessibility testing against your web application, which no automated testing tool is going to be able to replicate completely. That said, if you're looking to take care of any low-hanging fruit with an automated solution, like Kimberly suggested, there are several automated accessibility testing tools out there that you can relatively easily integrate into your existing web application's testing framework that might help you.
One such tool is Continuum, which doesn't have a C#-based library offering at the moment, but could be used in a separate testing framework to be run against your web application after it has already been built. This may be preferable depending on your use case, as code linters for accessibility aren't perfect and are highly language-dependent, whereas testing the HTML of your web application more closely matches the screen reader use case you say you're trying to test for. You could even integrate Continuum into your existing CI/CD process to make sure your application is tested during development as opposed to afterwards, to reduce your manual accessibility testing load.
Continuum has a few sample projects to get you started, depending on your technologies of choice. Free versions are available at webaccessibility.com if you're interested. Most of them are Java- or JavaScript-based at the moment.
Appreciate this is quite an old question, but having explored this area a lot recently thought was worth updating with the state as of 2023 as there is now some progress in this space.
Current tooling available at time of writing (that I’m aware of, may not be exhaustive):
guidepup - NodeJS automation for VoiceOver and NVDA supporting all keyboard commands and getters for spoken phrases (disclaimer: I’m the author).
auto-vo - CLI for navigating sequentially through a page with VoiceOver and reporting the spoken phrases, also exports a separate Node module for some interactions with VoiceOver.
screen-reader-reader - NodeJS automation for VoiceOver and NVDA for starting, stopping, and getting spoken phrases.
web-test-runner-voiceover - NodeJS plugins for #web/test-runner to automate VoiceOver testing.
nvda-testing-driver - .NET automation for NVDA supporting all keyboard commands and getters for spoken phrases.
assistive-webdriver - NodeJS implementation of a Webdriver server that allows remote testing of screen readers (e.g. NVDA, JAWS) running in a VM.
As stated in other answers, there are also a number of static analysis tools such as axe, as well numerous browser extensions offering similar static analysis, and companies such as Assistiv Labs offering remote environment services to interact with screen readers manually (similar to SauceLabs/BrowserStack/etc. but for screen readers, magnification etc. - no affiliation and haven’t used services so can’t vouch, simply an observation).
Worth calling out that none of these cover the full range of a11y requirements - there is more to a11y than just screen readers. A combined/layer approach including automation, manual testing, and user testing likely preferred.
Context is provided in case anyone knows of an alternative way to solve the larger issue.
Problem Context
I am spearheading the development of a test automation framework for a web application which uses Web Components. This has presented a problem when testing in Internet Explorer, because Internet Explorer does not support Web Components natively; instead, a polyfill is used to provide this functionality.
A primary repercussion of this is that much of Selenium will not work as expected. It cannot 'see' the Shadow DOM in Internet Explorer the way it can in Firefox and Chrome.
The alternative is to write a test framework which provides an alternate mechanism for accessing elements via JavaScript - this allows elements to be located through the polyfill.
Our current implementation checks the WebDriver being used, and either uses the original Selenium implementation of a method (in the case of Chrome or Firefox), or our own alternative implementation (in the case of Internet Explorer).
This means that we want our implementation to be as close as possible to Selenium's implementation, at its core, browser-interacty, level.
Problem
I am trying to replicate the functionality of Actions.click(WebElement onElement) (source), in a simplified form (without following the Builder design pattern of the Actions class, and making assumptions that the click is with the left mouse button and no other keys (Ctrl, Shift, Alt) are being held down).
I want to find the core code which handles the click does (specifically in Chrome, Firefox, and Internet Explorer), so I can replicate it as closely as possible, however I've found myself lost in a deep pit of classes and interfaces...
A new ClickAction (source) is created (to later be performed). Performing this includes a 'click()' call on an instance of the Mouse interface (source) ... aaaaand I'm lost. I see from generated JavaDoc that this is implemented by either EventFiringMouse (source) or HtmlUnitMouse (source), but I'm not sure which one will be implemented. I made an assumption (with little basis) that HtmlUnitMouse would be used, which has led me further down the rabbit hole looking at HTMLUnit code from Gargoyle Software...
In short, I am totally lost.
Any guidance would be much appreciated :)
Research
I have found that I was incorrect in my assumption that HTMLUnit is used by Chrome, Firefox, and Internet Explorer. Documentation shows that RemoteWebDriver (source) is subclassed by ChromeDriver, FirefoxDriver, and InternetExplorerDriver.
The Essentials
The drivers for Chrome, Firefox, and Internet Explorer are all RemoteWebDrivers.
This means that any actions which Selenium performs are sent to the browser (the WebDriver), via an HttpRequest.
Once the request is received by the browser, it will perform the action as either a "native event" or synthetically. How a browser executes an action depends on the capabilities of the browser (and potentially a flag option).
"Native" events are OS-level events.
Actions executed synthetically are executed using JavaScript. "Automation Atoms" are used - as one infers from 'atom', they are small, simple functions to perform low-level actions.
References
RemoteWebDriver is subclassed by ChromeDriver, FirefoxDriver, InternetExplorerDriver, OperaDriver, and SafariDriver. (reference)
All implementations of WebDriver that communicate with the browser, or a RemoteWebDriver server shall use a common wire protocol. This wire protocol defines a RESTful web service using JSON over HTTP. (reference)
In WebDriver advanced user interactions are provided by either simulating the JavaScript events directly (i.e. synthetic events) or by letting the browser generate the JavaScript events (i.e. native events). Native events simulate the user interactions better whereas synthetic events are platform independent [...] Native events should be used whenever possible. (reference)
Browser Automation Atoms are building blocks intended to be used by Selenium implementations. By using the same pieces throughout the codebase, rather than reimplementing required functionality in multiple places, the project can reduce the number of bugs found, and can simplify the process of adding new functionality and drivers. (reference)
Automation Atoms
A summary of the available Automation Atoms
The raw JavaScript code for the Automation Atoms - this may serve as a useful starting point in developing simpler synthetic events, if necessary.
The wiki for the Selenium IE Driver states that it uses native events rather than JavaScript events to interact with the browser.
As the InternetExplorerDriver is Windows-only, it attempts to use
so-called "native", or OS-level events to perform mouse and keyboard
operations in the browser. This is in contrast to using simulated
JavaScript events for the same operations.
Except for clicking <option> elements, where it uses JavaScript.
The IE driver handles this one scenario by using the click()
Automation Atom, which essentially sets the .selected property of the
element and simulates the onChange event in JavaScript.
I am interested in automating repetitive data entry in some forms for a website I frequent. So far the tools I've looked up that would provide support for this in a headless fashion could be Selenium WebDriver and Mechanize.
My question is, is there a fundamental technical difference in using once versus the other? Selenium is mostly used for testing. I've also noticed some folks use it for doing exactly what I'm looking for, and that's automating data entry. Testing becomes a second benefit in that case.
Is there reasons to not use Selenium for what I want to do over Mechanize? Does it not matter and both of these tools will work?
I'm not asking which is better, I'm asking which is the right tool for the job. Perhaps I'm not understanding the premise behind the purpose of each tool.
These are completely different tools that somewhat "cross" in the web-scraping, web automation, automated data extraction scope.
mechanize is a mature and widely-used tool for programmatic web-browsing with a lot of built-in features, like cookie handing, browser history, form submissions. The key thing to understand here is that mechanize.Browser is not a real browser, it cannot execute and understand javascript, it cannot send asynchronous requests often needed to form a web page.
This is where selenium comes into play - it is a browser automation tool which is also widely used in web-scraping. selenium usually becomes a "fall-back" tool - when someone cannot web-scrape a site with mechanize or RoboBrowser or MechanicalSoup (note - another alternatives) because of, for instance, it's javascript "heaviness", the choice is usually selenium. With selenium you can also go headless, automating PhantomJS browser, or having a virtual display. As a commonly mentioned drawback, performance is often mentioned - with selenium you are working with a target site as a real user in a web browser, which is loading additional files needed to form a page, making XHR requests, rendering etc.
And this itself does not mean you should use selenium everywhere - choose the tool wisely, choose it because it fits the problem better, not because you are more familiar with an instrument.
Also note that you should, first, consider using an API (if provided by the target website) instead of going down to web-scraping. And, if it comes to it, be a good web-scraping citizen:
How to be a good citizen when crawling web sites?
Web scraping etiquette
What tools are useful for automating clicking through a windows form application? Is this even useful? I see the testers at my company doing this a great deal and it seems like a waste of time.
Check out https://github.com/TestStack/White and http://nunitforms.sourceforge.net/. We've used the White project with success.
Though they're mostly targeted at automating administration tasks or shortcuts for users, Autohotkey and AutoIT let you automate nearly anything you want as far as mouse/keyboard interaction.
Some of the mouse stuff can get tricky when the only way to really tell it what you want to click is an X,Y coordinate, but for automating entirely arbitrary tasks on a Windows machine, it does the trick.
Like I said, they're not necessarily intended for testing purposes, so they're not instrumented for unit test conventions. However, I use them all of the time to automate stuff that isn't testing related.
You can do it programmatically via the Microsoft UI Automation API. There's an MSDN Magazine article about it.
Integrates well with unit test frameworks. A better option than the coordinate-based script runners because you don't have to rewrite scripts when layouts change.
There's a couple out there. They all hook into the windows API to log item clicks, and then reproduce them to test.
We're now mostly web based (using WatiN), but we used to use Mercury Quicktest.
Don't use Quicktest, it's awful for a tremendously long list of reasons.
This is what i was looking for.
Check out http://www.codeplex.com/white and http://nunitforms.sourceforge.net/. We've used the White project with success.