We're experiencing problems spying a Citrix window.
Our team is not able to spy the outline of the Citrix application window once it is launched. We tried full-screen, on the desktop, on the laptop, closing all other windows, and numerous other scenarios. I believe that it is because the sign-in is via internet explorer and once the Citrix application is launched, the application detaches from Blue Prism.
We were working to troubleshoot this issue and are trying to set up a separate business object for the Citrix window. Another problem is that we cannot launch the Citrix .exe file because it does not allow us to input any parameters to log-in that way. We’re now looking into saving the Citrix file as an .ica file (see https://support.citrix.com/article/CTX804493).
Does anyone have any suggestions?
You'll need to use Surface Automation techniques in order to interact with applications launched via Citrix. Essentially bitmap mapping.
If you are having difficulty attaching to the citrix window itself, try using a dedicated application like citrix receiver and have Blue Prism launch that application.
You could also try checking the windows process running (.exe) once citrix is launched via IE, and then try attaching to that process.
Good Luck.
Try using System Internals process monitor to make sure you attach the correct application, download it for free from Microsoft
When you work with Citrix it depends on the mode you are using. Some of the windows will actually be separate windows, use the Active Accessibility mode to spy them. Some other "windows" or tabs are only visual and you will need to use the Region mode to interact with them.
You should probably be aware that BP has a known issue not flushing all unused memory until the process is finished. If you have a large scale SA process that requires a lot of image processing, OCR etc. it should be a design consideration.
Related
I have got a vb.net windows service that runs a File System Watcher to monitor an xml file for changes, when it detects a change it will stop the screen saver running and what I would then like it to do is give one of the other running applications the focus.
I have successfully used Appactivate from a Windows Form App in the past but I think I am correct in saying this doesn't work with Windowless applications (I have tried it and it doesn't).
I have done some Googling but I can't seem to find any way of getting the focus when called from a service \ windowless app. Is this possible?
Thanks
what if you make it a single instance application? and then simply launch its .exe file? it might give focus to the other running application instead of opening a new instance.
I am working on an automation project, in which I need to capture the activities [ application launched, data entered, input type etc.] user performs on a desktop. I came across Microsoft UI Automation framework which so far works well for native windows based applications like MS Office, .NET apps etc. However I did not find any useful information / samples of capturing the information from different web browsers [Chrome is a must], Python apps, Java Apps etc. Can someone please confirm whether MS UI Automation Framework supports such apps. Any working example to extract user activities from these apps would be highly appreciated. Thanks.
Chrome only supports UI Automation for toolbars, tabs, menu, buttons around the web page. Everything that's rendered as a web page is not seen by UIA.
For the web page content, the easiest way is to use Selenium (driven by the ChromeDriver), which is kind of a de facto standard for browsers, and has nothing to do with UIA.
To test if an app supports UIA, and how far it does, it's very easy, just run UIA's Inspect tool and check the UI tree over that application.
Some additions to Simon's answer...
Chrome page content can be seen by UIA if you run chrome --force-renderer-accessibility. Only for existing Chrome process it won't work. Though user can create a new tab chrome://accessibility manually and enable UIA for all or some chosen pages. This method also works for AT-SPI accessibility technology on Linux. Of course, Selenium WebDriver is an industry standard here. But another way exists. Both Mozilla and IE support UIA by default.
Inspect.exe can be simply downloaded from this GitHub repo.
Regarding Java apps it depends on the app type. Your chances is about 50/50.
WxPython or PyQt5 are good for UIA. TkInter or Kivy apps are not.
P.S. There is an example how to drag a file from explorer.exe and drop to Google Drive in Chrome using Python library pywinauto.
I'm a bit late to the party..
But Chromes accessibility features are only activated once something tries to access it's accessibility.
If you call AccessibleObjectFromWindow ([DllImport("oleacc.dll")]) with the window handle an existing chrome window will have its accessibility activated (and you'll see the actual web page content in UIA!).
If the chrome window is opened after your app is running - Chrome pings open processes for any open accessibility apps... for that you use AccessibleObjectFromEvent and the event you're responding to comes from the windows pipeline: EVENT_SYSTEM_ALERT = 0x0002 .
The bottom line is - you have to tell chrome that there's something installed that wants to access it's web page content.
Oh! and your application has to be signed!! Unsigned apps won't be able to access web content - I think that's the same in firefox too.
I hope this helps someone in the future.
See:
https://www.chromium.org/developers/design-documents/accessibility
I have an application that is a Windows Service application. It currently reads an XML file for it's configuration and the code works off those settings.
I have created a separate project that is a Windows Forms application (typical windows app). This app is a GUI interface into the configuration settings (changed from XML to database, but that is relatively unimportant).
What I need to do is:
Integrate the 2 projects
Add a system tray icon to pop up the form
I'm unsure exactly how to proceed on this and wanted to get some advice before messing things up.
You can't integrate these two things (entirely). A service is designed to run without a GUI of any kind (there is a work around to allow desktop interaction but this is messy and clunky and will cause you more problems than it solves)
It sounds like you just need to use the code from the service in the forms application. This should be a simple copy and paste operation.
To show an icon in the system tray is pretty easy:
Create a program to run from the system tray
how to put an .net application in system tray when minimized?
Here's what I want to be able to do:
Run a program on my local computer which logs in to a citrix server (using citrix receiver or doing so in a similar way), on the server in the citrix session open a web browser, load a website, and then bring the html of that site back out of the citrix session and onto my local computer. Bascially I want to get data out of a citrix remote session.
How can I do this programmatically?
I'm fine with whatever programming language/modality you are comfortable in answering the question using.
I've looked a little into the citrix apis but while I find some things about logging in and even sending keystrokes and mouse clicks I found nothing about obtaining data. I could just log in and then use a program like wireshark to get the information, but I'm guessing it's all encrypted (plus then I wouldn't be doing my task all programmatically). I know of at least one open source program which seems to be able to replace a citrix reciever/client (openthinclient.org) but before I got digging through all its source code to try to answer my question I thought I'd ask here in case someone had an easier answer.
If all you want is to automate the task, is having the program act as a citrix client necessary?
I assume you don't have install privileges inside your citrix session, so are unable to install one of the many automation tools available (such as http://docs.seleniumhq.org/)?
Given the above...
If you have/allow java on your local machine, have a look at http://www.sikuli.org/
The main difference between this and other automation tools I've come across is that Sikuli uses the image on screen to navigate the gui, rather than grabbing calls to the widgets (which wont work in a citrix session).
So, assuming you can take a screenshot of your citrix session, it could be useful to you.
I've downloaded and installed the windows 8 consumer preview, and I'd like to figure out how to use the UI Automation API's to get data from metro style applications.
Background: I have a lot of automated tests which use the Microsoft UI Automation API's to interact with applications. The scripts are written in IronRuby against .NET 4
Initially, I'd simply like to check if the start menu is visible (and if so, close it)
According to Microsoft, the normal UI Automation API's should be able to access and control metro style applications, but the problem is, I can't figure out how my code can actually access the UI of metro applications? Our normal way of viewing UI automation data is using UISpy, however when I run UI spy under windows 8 and bring up a metro app, there is no mention of it... it seems like metro apps (including the start screen) are walled off from the desktop.
If I can't find the metro apps in UISpy, how can I find what their automation ID's are, and how could I find them from my automated test scripts?
A few resources on UI automation which works in Metro style apps:
Here is a sample demonstrating it.
Here is a blog post explaining it.
This is a forum thread on the subject.
There is nothing about the app that stops traditional UI automation from working correctly. You may have to get updated tools to view the Metro style apps. Some ship with the Consumer Preview. Look for inspect.exe.
UI Automation tools can automate Metro applications. Check RIATest for example.
Stating from version 6.0 a number of features in RIATest are specifically targeted at Metro style application automation, particularly:
Ability to stay on top of Metro UI to allow you to simultaneously see your Metro application and RIATest IDE and minimize switching from Desktop to Metro screens when automating a Metro application.
Recording of actions performed on native Windows GUI elements (including Metro GUI). The recorder analyses your script code and reuses your variable names to generate cleaner recorded scripts similar to how you would hand-code an automation script.
Seamless workaround for bugs like this in Windows UI Automation implementation in Metro UI.
Disclaimer: I work for Cogitek, the RIATest company.
WinRT (aka "Metro Style") applications run in an sandbox. You cannot pierce the walls between applications, including to Desktop applications. This includes network isolation (you cannot refer back to localhost for example except in a development environment). There are a couple of exceptions to this such as the Share and Search contacts which allow for very specific types of interactions between apps.
It is this sandbox that is preventing UISpy from seeing the executing WinRT application in the background. I'm not sure how UISpy could work around this issue without some kind of development environment exception to the sandbox (similar to the network isolation exception) that isn't available in the consumer preview.
I'm also unaware of any announcements regarding when/if the UI Automation API will be supported for WinRT applications at this time.