I'm writing an implementation of the windows UI automation client in VBA and that's going pretty well. But I would like to be able to draw a simple box around the UI element of another window. I'm aware of the inspect.exe application, but was wanting to recreate some of its functionality in VBA for instances where company policy restrictions prevent me from installing any 3rd party applications. Code I've written myself can be copy and pasted from a paste bin account.
Is there something in the windows API or a reference I can import that I could use to draw a box on top of another screen at specific coordinates for a second or two? I'm trying to limit this project to just things that come with an out of the box windows installation.
I need for get UIelements using "UIautomation" technology from another programms in windows OS.
Some programms has "chrome-like" browsers for content rendering.
The methods are get element (like c# AutomationElement.FromPoint / FindAll) return browser element with ClassName "Chrome_RenderWidgetHostHWND" and Name like a "Chrome Legacy Window". But i need the content elements in that browser.
I found out that, programm "Accessibility Insights" can do it!
Run application, "Microsoft Teams", for example
Run "Accessibility Insights"
Move mouse on "Microsoft Teams", we get "Chrome Legacy Window"
After any mouse click (left/right) on target programm ("Microsoft Teams") provoke rebuild UI tree
Then UIautomation technology can get all inner UI elements in browser.
"Accessibility Insights" source code is https://github.com/microsoft/accessibility-insights-windows
I studied it very carefully, but not found code, that provide rebuilding (or, initializing?) inner UI elements.
Any one can help me?
What i should to do for win my task - get inner UI elements in chrome-like browsers at another programm on windows OS?
If you've read to the end, thank you
Accessibility Insights for Windows doesn't interact directly with the UI Automation system. It builds on Axe.Windows (Source code), which queries and exposes the UI Automation properties. In the case you're describing, Axe.Windows is walking the UI Automation tree in response to a UIA_AutomationFocusChangedEventId message from the UI Automation framework. That notification gets handled by the onFocusChangedEventForSelectingElement method of FocusTracker.cs. Hopefully that will be enough to get you started.
--Dave Tryon (Accessibility Insights team)
I'm looking for some input on how feasible it would be to create a Chrome extension that, when activated via toolbar button, adds context menu commands that call procedures (or methods of a class) written in VB.net.
That's pretty much it. At this point, i'd be thrilled to see context menu items that simply display message boxes built in VB.net.
I'm starting with Chrome, but would anticipate replicating similar functionality in other browsers, starting with Edge or IE, then Firefox and possibly Safari. If it's easier to start with another widely used browser besides Chrome, i'd be fine with that.
I've done the obvious google searches, and I can't seem to find an easy-ish place to start. Any ideas?
--edit, just an additional note, the VB.net procedures would be contained in a dll, exe, or other container installed on the client side machine. I'm not trying to execute server side code
I'm designing an application on VB.net and I'm newbie as well.
I thought about the general architecture of the application and came up with the following idea.
the application is made of different modules
the user can run different modules in the same time
each module will run his main form in a different tab in tabcontrol
each module has his own modal forms, mode-less windows, messages, ... etc.
Before going through much development details, I started first with trying this design. Though, I couldn't first embed dynamically a form in tab during run time and even after some workarounds, I couldn't make the modules run perfectly in parallel. For example when I have a modal window displayed in a module, the whole application freezes while I expect only the related tab to freeze and be able to switch to the others to do some work.
Does anyone know how to make the tab contents completely separate and not have one freezes the other?
Thank you.
There's no need to have separate modules/forms to run more than one task simultaneously. The issue you are having with the UI freezing is you are running all operations on the primary thread. You need to multithread the application. Do a quick search on stackoverflow for multithreading vb.net. If you can't find anything let me know and I'll do the search and provide links.
I have a virtual machine (VMware) with Mercury Quick Test Professional 9.2 installed. I have a script to test an application, written in VB.NET using the Infragistics library.
If I access this virtual machine using my laptop (using Remote Desktop), everything works fine, the script completes without a problem. My laptop runs XP, with Windows Classic theme.
If I access this virtual machine using another machine (using Remote Desktop), the script starts fine, but stops halfway through, without no error message from QTP, nothing. This machine runs XP, with Windows Classic theme.
One difference between the two setups is the size of the screen, the laptop is 1920x1280, other machine 1280x1024.
The step where the script stops involves checking a checkbox within an UltraWinGrid. The checkbox itself is displayed, is on the screen in both cases.
Has anyone had this problem before, or have any idea why the behaviour is different between the two machines?
Thanks.
OK. I've found the problem. In fact, the script was failing silently because that's what the person who wrote the script told it to do. It couldn't validate something which was off screen, so the script failed.
The problem was the QTP definition of 'off screen'. I have two screens attached to my laptop, the screen for the laptop itself (1920x1200) and another screen (1280x1024). I connect to the VM for QTP using remote desktop, and it uses the settings of the screen for the laptop. This means that when I launch my QTP script, and move it to the other screen, it doesn't fit, so the screen is no longer maximized, and the object is partially off screen, so can't be found.
The fix is simple: in the Remote Desktop, use the Display tab, and set the size of the screen to a size to 1280x1024, and QTP doesn't have any more problems.
Voilà.
If you are not using Expert Mode, and / or are allowing QTP to do most of the work to create your repository objects, then yes it is referencing everything by pixels.
I create all of my repository objects by hand, viewing the source (in the case of automated web-application testing) and using the Object Spy for assistance where needed. I make a point to not have any positioning information as part of my object definition, for the very reason you are running in to.
For the parts of my web-app that interacted with Windows (opening a file to upload, etc.) the Object Spy was essential for the trial and error necessary to create a unique identifier for creating the repository object. But it can be done.
Ex1: File Browse Dialog
text = "Choose file"
nativeclass = #32770 (apparently some Windows VooDoo for a file open dialog?)
Ex2: Filename textbox in Browse dialog:
nativeclass = "Edit"
attached text "File &name:" (more Windows VooDoo? It woudn't work for me without the "&")
Ex3: Open Button in the dialog:
text = "&Open"
object class = "Button"
Good Luck!
Point of clarification: You mentioned that QTP stops with no error message. Does that also mean that the test results log file also has no error message? If the log has any information, that may be helpful in diagnosing the problem. Could you share the lines of code at the point where the script fails?
Also, remote desktop will resize the desktop on the remote machine. Although QTP scripts are not inherently coordinate based, individual statements can be coordinate-based relative to an object. The resolution could be an issue in that regard. For example, imagine you had a line like Button.Click(5, 150) recorded on a higher resolution machine. But if you attempted to play it back on a lower resolution machine, and the 150 is out of bounds of the object on the lower resolution, it could cause an issue.
QTP does not use screen coordinates except as a last resort, if the objects are identified as high level objects (SwfTable in this case) you should be OK, if however QTP doesn't recognise the object it falls back to WinObject and screen coordinates.
If you're using Infragistics then you should know that they extend QTP's support with their TestAdvantage product which will probably solve your issue.
Edit:
#MatthieuF said:
In fact, we use the Infragistics plugin for QTP, and we still have the problem
Can you give me an example of a line that fails?
A few things:
You should be able to debug on the VM easily - just wait for it to stop, go into your object repository, and see if it can identify the object. If no then use object spy to figure out what properties are different between the OSes. If there is a difference then you can always set that property to a regular expression and have it check for both possibilities.
Assuming that isn't the issue we've run into problems using remote desktop with QTP if the Remote window is closed or minimized. For us, it was an issue where the clipboard can not be changed when an RDP window isn't visible, but there could be other surprises when using QTP that way.