I need to use TestCafe to perform actions within our Electron app, however I also need to perform actions on the Windows desktop (e.g. click taskbar, launch other apps).
Since TestCafe can only interact with what is within the browser window, I have attempted to use Appium to essential run TestCafe (calling the commandline from C#) and then performing the Windows desktop actions through WinAppDriver however this feels like a kludge.
Are there other methods of arranging this or a framework that would be more elegant?
Thanks for any insights
m
You are right - TestCafe can only interact with what is within the browser window, but not outside it.
You can also check out the testcafe-browser-provider-webdriverio plugin. Its author claims that this provider can connect to an Appium server.
I have successfully used robotjs to interact with the desktop (mouse move, click, etc.). This seems to be an excellent choice at the moment. Just install alongside TestCafe in your node project.
Related
I am working on an automation project, in which I need to capture the activities [ application launched, data entered, input type etc.] user performs on a desktop. I came across Microsoft UI Automation framework which so far works well for native windows based applications like MS Office, .NET apps etc. However I did not find any useful information / samples of capturing the information from different web browsers [Chrome is a must], Python apps, Java Apps etc. Can someone please confirm whether MS UI Automation Framework supports such apps. Any working example to extract user activities from these apps would be highly appreciated. Thanks.
Chrome only supports UI Automation for toolbars, tabs, menu, buttons around the web page. Everything that's rendered as a web page is not seen by UIA.
For the web page content, the easiest way is to use Selenium (driven by the ChromeDriver), which is kind of a de facto standard for browsers, and has nothing to do with UIA.
To test if an app supports UIA, and how far it does, it's very easy, just run UIA's Inspect tool and check the UI tree over that application.
Some additions to Simon's answer...
Chrome page content can be seen by UIA if you run chrome --force-renderer-accessibility. Only for existing Chrome process it won't work. Though user can create a new tab chrome://accessibility manually and enable UIA for all or some chosen pages. This method also works for AT-SPI accessibility technology on Linux. Of course, Selenium WebDriver is an industry standard here. But another way exists. Both Mozilla and IE support UIA by default.
Inspect.exe can be simply downloaded from this GitHub repo.
Regarding Java apps it depends on the app type. Your chances is about 50/50.
WxPython or PyQt5 are good for UIA. TkInter or Kivy apps are not.
P.S. There is an example how to drag a file from explorer.exe and drop to Google Drive in Chrome using Python library pywinauto.
I'm a bit late to the party..
But Chromes accessibility features are only activated once something tries to access it's accessibility.
If you call AccessibleObjectFromWindow ([DllImport("oleacc.dll")]) with the window handle an existing chrome window will have its accessibility activated (and you'll see the actual web page content in UIA!).
If the chrome window is opened after your app is running - Chrome pings open processes for any open accessibility apps... for that you use AccessibleObjectFromEvent and the event you're responding to comes from the windows pipeline: EVENT_SYSTEM_ALERT = 0x0002 .
The bottom line is - you have to tell chrome that there's something installed that wants to access it's web page content.
Oh! and your application has to be signed!! Unsigned apps won't be able to access web content - I think that's the same in firefox too.
I hope this helps someone in the future.
See:
https://www.chromium.org/developers/design-documents/accessibility
I am using Blueprism to launch a URL in browser to spy it. I am able to launch the URL using the application modeller, but not able to spy it. The "Identify" option itself does not appear in blue prism. What is the problem and what is the solution ?
In spying a web application, key things to note for your application modeler;
1. your application modeler must be a "Browser-Based Application"
2. Launching from an executable or attaching to one. (Assuming you are launching from an executable from your question)
3. Specify url.
From your question. you are able to launch the browser from the application modeler. the most possible problem is you missing step 1 as mentioned above. Confirm your application modeler is specified as "Browser-Based Application"
Check a few things : is your modeler attached to the browser, are you in the right mode (for ease of use I would recommend using IE in HTML mode), and if that doesn't work try accessibility mode.
Try these simple steps:
Close the browser application and let the Blue Prism open your browser.
Try testing your object in Object Studio.
Now open your application modeler and launch. Check and you will find the identify option appeared.
If you launch application from Google Chrome, then you can get this error because in Google Chrome it's not possible to Spy and launch the applications. Any RPA Tools will support only Internet Explorer
I want to write an acceptance test for my browser extension. I've tried to initiate an extension via selenium but I can't seem to access a content of popup. Can someone suggest how can I do it with selenium or any other way to write UI/acceptance tests for browser extensions? Thanks.
How can I access browser extension's popup using selenium?
It's not possible. Selenium supports interaction with web view only.
What you can do with Selenium and extension for sure is automatic installation: https://stackoverflow.com/a/16512012/2517622
You may try to use desktop automation tools (e.g. White on Windows platform) for clicking on extension popup but it's not that easy and it's not platform independent as Selenium.
Here is the workaround we came up. Unless someone posts here the "right" solution I will consider this as the best approach.
So eventually, our extension is an iframe which just loads a page content from our website + does some other neat stuff. We simply open that url in a new tab and do regular selenium tests.
Side not: we have considered to write a little javascript wrapper to be able to access ext via main window through javascript. E.g. there is some js in ext that listens to main window's events and perform certain actions. Tho, it is too much efforts and doesn't really sound like a proper acceptance test so we discarded this approach.
I have some scripts (AutoIt) browsing YouTube for list of trending videos etc. It involves no mouse clicks (just keystrokes). The script takes a long time to finish and I can't use my PC during this time (it needs the window activated to work on it).
Is there anything I can do about this? Can these scripts run from a server or some stuff like that?
I've run into a similar problem: got to run automation with AutoIt on a Windows Server and the whole thing had got to be headless. Using Remote Desktop simply didin't work because then I'd had to keep a client opened and maximized all the time.
Short solution: install a VNC server in the Windows Server, open a client from another computer, log in and close the client. As the AutoIt script was being started by a Jenkins job, before closing the client the Jenkins applet had to be started via web interface.
By the way, I've had this idea from this post: How to run remote headless GUI automation.
As Johannes said, AutoIt probably wouldn't be suitable (and likewise AutoHotkey), but you could check out the many GUI and web testing frameworks that exist for other languages. With some of those, you can run a "headless" browser (a program that navigates the web just like a browser, but has no visible window); or you can run a standard browser on a virtual display like a Xvfb X11 server. This would be easiest if the server (or wherever it's going to run) is running a Unix-like OS, but it may be possible with Windows too.
Selenium Webdriver seems to be a very popular choice for scripting and testing actual browsers. It's natively Java but has bindings for languages like Ruby. It can also hook into something called HtmlUnit, which is also Java; that's one of the more popular headless browsers. Another (a relative newcomer) is phantom.js, which is in Javascript but (again) has bindings for other languages.
As far as I know this will not work unless the user account is logged on. You could try to see if you could convert it to an exe and run this as a service, but even then I don't think this will work. Let me know if you found out!
You can either:
Hide your window (SetWindowState #SW_HIDE) or something like that...
and use ControlClicks (if the they are original controls!)
or
Hide your window and use SendControl
or
use SendKeepActive
or
use OLEObjects like ie.au3 for automation.
Good Luck
I'm executing selenium tests with testng, that are started on a remote system with Selenium RC via hudson (with ssh connection). The remote system is windows xp with MKS Toolkit installed, hence ssh. Tests are NOT executed as a windows service.
I've tried using both captureScreenshot and captureEntirePageScreenshot methods. The first one always produces a black image. The second one creates the correct screen shot but it only works on Firefox and our tests usually pass on Firefox and fail in other browsers, so it is crucial to capture screen shots for the other browsers (mainly IE and Safari). The tests are ran in parallel, with many browser windows open at the same time. I'm not certain if this is what's causing the problem. Any thoughts will be appreciated.
Unfortunately screenshots in Selenium have been problematic from the start in browsers that are not Firefox. This is something that we Selenium Developers have been working on for a while to correct.
The latest work has been updating Snapsie to work in IE. There is a blog post at http://blog.codecentric.de/en/2010/02/remote-screenshots-mit-selenium-und-dem-robot-framework/ that explains what has happened.
I have noticed that if the screen isn't active, i.e. the screensaver has kicked in, it can produce black screenshots.
Edit:
I just had a thought. You can always run Castro to video record your tests and then watch it play back. This is something SauceLabs use to run Selenium in the cloud.
Write a method for this and call that whereever you need to take the screenshot. Use the java.awt package which has been used in selenium. For example, check this site
After setting Windows Auto-Logon, and launching process not as Windows Service, I found how to solve the Remote Desktop with Black Screenshots problem of IEDriverServer.exe, by creating a batch file that disconnects RDP, instead of closing the RDP session with the regular X button:
%windir%\system32\tscon.exe %SESSIONNAME% /dest:console
See more details here:
https://stackoverflow.com/a/24529629/658497
(Although, I would prefer there was a way to run it as the default action, when terminating RDP session with X Windows button).