Accessibility Insights for Windows shows the following property:
ControlType Group(50026)
when I hover over a search box that is an <input /> html tag.
I used Google Chrome Version 80.0.3987.87 (Official Build) (64-bit).
How does AI4W determine the control type of the html tag?
The accessibility insights doc says the control type is a concept from the Microsoft UI automation accessibility framework. You can look up the 50026 id for Group on this reference page.
It is not yet clear to me how AI4W determined the exact control type. I suspect Chrome implements a Client-Side (Proxy) UI Automation Provider but I am not sure about it. If it does, how does the implementation deduce the control types?
Related questions:
How to know the ControlType of a UI Element from CurrentControlType property
Does Microsoft UI Automation Framework work with Chrome, Python and Java Apps?
As described on this Chrome accessibility page, Chrome exposes HTML and ARIA control types/roles through an accessibility API called IAccessible2.
As of Windows 10.0.1903, Windows translates IAccessible2 into UIA. I was unable to find any documentation which specifically describes the mapping of IAccessible and IAccessible2 roles to UIA control types, but it is probably very close to the mapping recommended by the W3C.
Accessibility Insights displays the UIA control type after Windows has converted the IAccessible2 data to UIA.
To add to Rob Gallo's answer:
To determine the exact accessibility provider used by the browser:
Start Accessibility Insights for Windows.
Select live inspect and point to element of interest.
Set the tool to always show following properties: FrameworkId and ProviderDescription (use the cogwheel in upper right of pane containing the Properties table).
Observe the value.
For example, this:
FrameworkId: Chrome
ProviderDescription: [pid:33748,providerId:0x0 Main(parent link):Microsoft: MSAA Proxy (IAccessible2) (unmanaged:UIAutomationCore.DLL)]
means that UIA is translating to its format the data exposed by Chrome as IAccessible2, as explained in UI Automation: An Incomplete Guide for UI builders – Part 1. There are more possible UIA providers even when using the same browser. Knowing the right UIA provider is required for root causing problems with the UIA hierarchy. See the linked article for details.
In addition, the HTML Accessibility API Mappings 1.0 spec section on accessibility APIs links to UI Automation for W3C Accessible Rich Internet Applications Specification which explains specified ControlTypes for given ARIA roles, which are defined in this spec (v1.1).
A good article providing additional context for this discussion: https://sarahmhigley.com/writing/roles-and-relationships/
Related
I am developing a Master-Detail application which should show an embedded PDF in its detail view. The project is based on UI5 version 1.48, so the new PDFViewer control can be used.
In desktop mode, everything works as expected:
But on a mobile screen, PDF is not showing properly:
The data source property binding at the PDFViewer is done by OData path.
<FlexBox id="fbPDFViewer"
direction="Column"
renderType="Div"
class="sapUiSmallMargin">
<PDFViewer id="idPDFViewer"
source="{myModel>/myPDFUrl}"
title=""
busy="true"
showDownloadButton="false"
height="700px"
width="95%"
loaded="onPDFLoaded"
errorMessage="{i18n>notFoundText}"
errorPlaceholderMessage="{i18n>notFoundText}">
</PDFViewer>
</FlexBox>
Even the SAP sample for embedded usage doesn't work on mobile devices.
According to the source code (1.48), the viewer renders its PDF document directly in the app / iframe only when ..:
The source is valid
It's coming from the same domain
It has a valid URI format
UI5 thinks the user is using a desktop computer (Device.system.desktop returns true) or the displayTypeapi, available since 1.58, is set to Embedded.
UI5 detects that the user agent (browser) has a certain type of PDF plugin enabled. Currently, there are no known mobile browsers which support displaying PDF documents directly in the page.
The steps 2 and 3 explain why the PDF is not rendered on a mobile device even if the source is valid. In such cases, I guess the only option is to provide a download option instead of trying to render the PDF forcefully in the mobile app.
You can use nabi.m.PDFViewer from the the nabi.m library. It works cross device, even on iOS (where you typically don't have the adobe PDF plugin). For details see https://github.com/nzamani/ui5-nabi-m
There you will find also how to try it out on localhost. I have also added some detailed instructions for deployment of the library to NW ABAP as well as SAP CP.
You can find live demos at http://ui5lab.io/browser/#/Samples/nabi.m/Sample/nabi.m.PDFViewer.PDFViewer and http://ui5lab.io/browser/#/Samples/nabi.m/Sample/nabi.m.PDFViewer.InPagePDFViewer
Open the two links on your iOS device or any other device to see how it looks like.
On mobile devices (smartphones and tablets), the PDF viewer control renders a toolbar with the title and a download icon, which behaves as a standard device/browser file link. We've made this note in Fiori Design Guidelines 1.48 and we'll add it to the SDK documentation.
(More in openui5 issue 1759 on GitHub)
Thank you for your responses and hints...
Installing an Adobe Acrobat viewer on the mobile devices was not an option here.
Source of the PDF had same domain like app itself and had a valid URI format too
Due to project timeline in meantime I decided to fall back to a solution that SAP provided with it's "Paystub" (Fiori 1.0). Therein they're using a 3rd-Party control for displaying PDF's which unfortunately has an overhead of code and implementations steps but it works (with minor issues only) perfect on desktop devices as well as on mobile devices.
So far, this is workaround solved my requirement.
Thanks.
I'm trying out the mobile pattern, and have been trying to crate my own custom pattern that is now supposedly supported in Worklight 6.1.
When I tried creating jquery UI pattern, several issues:
1. The rich page editor for the pattern.html does not display the jquery component correctly on the design page (e.g button is displayed as link).
2. When I added a new page (into a jquery hybrid app) based on the custom UI pattern, it does not create a new page. It only adds the content code into the index.html, and I had to create the page myself.
Is this the correct behaviour?
I'm also having difficulty in creating Dojo UI pattern… as there is no Dojo component available on the palette when I open up the dojo > pattern.html file.
Do I have to add the libraries and code manually (i.e. no Drag-and-Drop)?
Appreciate any pointers on this.
PS: I'm using Eclipse Juno R2
1) For jQuery based patterns you need to append a jQuery core file to the project besides the jQuery mobile ones, for example, append this one: http://code.jquery.com/jquery-1.10.2.js to your project, next to jquery mobile JS file. This is just because "UI Pattern" projects don't have this file available, but they need it to handle a proper preview.
2) For Dojo patterns, there is still no official support (for example Drag and drop), so even you can modify pattern.html to get some "insertable" code, you may still need some additional tuning to get a valid pattern.
I recently started writing LINQPad data context drivers for accessing various types of systems. Unfortunately every driver I write, uses the same "cog wheel" icon. This makes it harder to discern their type than if they'd have a type-specific icon.
Is there a way to provide a custom icon for a driver?
Joe wasn't kidding in his comment. The release notes for LINQPad Beta version 4.42.10 contain this:
You can now specify a custom icon when writing Data Context drivers. Just include two files in your .lpx package: Connection.png and FailedConnection.png (the latter is applied if a data context is unable to load).
What the release notes don't mention (but is obvious once you check the existing icons) is that the icons needs to be 16x16 pixels.
The jQuery VideoControl is documented here:
https://developers.google.com/tv/web/lib/jquery/#gtv.jq.VideoControl
It takes a set of VideoParms. That are not specified.
It is used in the sample templates on this page:
https://developers.google.com/tv/web/docs/gtv-templates
I've taken apart the templates, but I'm having difficulty creating a simple example of a standalone video control. In the templates, the sidenav and carousel code are tied to how the videocontrol works. There is a relationship between the keyController and behavior (behaviorZones) that I can not find an explanation for.
Is there a standalone example somewhere? What needs to be setup with keycontroller, css, and behavior to get this going?
I'm not aware that there is a standalone videocontrol.js sample but it should be possible to set it up. Most of the necessary pieces are inside videocontrol.js, in particular in the videocontrol's constructor and this function: gtv.jq.VideoControl.prototype.makeControl.
keycontroller mapping is used to handling key inputs and some of the CSS are dynamically controlled within JS so there might be errors if you just rip them apart straight from the Template. You'd have to debug it using a solid tool like Chrome Developer Tools. You can watch tutorials of this tool here: http://www.youtube.com/results?search_query=chrome+developer+tools&oq=chrome+develop&aq=0&aqi=g3g-m2&aql=&gs_sm=3&gs_upl=1972l4015l0l6142l14l11l0l0l0l0l207l1476l3.5.3l11l0
Otherwise please send your error traces and/or code snippet for debugging help.
S
Trying to get access to a page's DOM after rendering. I do not need to view the page and plan to apply this programmatically without any GUI or interaction.
The reason I am interested in post-rendering is that I want to know where objects appear. Some location information is coded in the HTML (e.g., via offsetLeft), but much is not. Also, Javascript can change the ultimate positioning. I want positions that are as close to what the user will see as possible.
I've looked into Chromium code and think there is a way to do this but there is not enough documentation to get started.
Putting it VERY simply I'd be interested in pseudo-code like this:
DOMRoot *r = new Page("http://stackoverflow.com")->getDom();
Any tips on starting points?
You should use the Web API wrapper that Chromium exposes; specifically, the WebDocument class contains the functionality that you need. You can call it like this:
WebFrame * mainFrame = webView->mainFrame();
WebDocument document = mainFrame->document();
WebElement docElement = document->docElement();
// Manipulate the DOM here using docElement
...
You can browse the source code for Chromium's Web API wrapper here. Although there's not much in the way of documentation, the header files are fairly well-commented and you can browse Chrome's source code to see the API in action.
It's difficult to get started using Chromium. I recommend looking at the test_shell application. Also, a framework like the Chromium Embedded Framework (CEF) simplifies the process of embedding Chromium in your application; I use CEF in my current project and I'm very satisfied with it.