UIA Automation

1. Overview #

Clicknium UIA automation is based on Microsoft UI Automation to support the recording function, generating locator, and all operations on UI elements.During recording, for the Windows applications, such as notepad, calculator, and ERP client applications, Clicknium will use UIA automation to record the element.

2. Locator attributes #

You can learn locator concept first. For UIA automation, the attributes are defined below:


3. Locator samples #

  • notepad ‘format’ menuitem
<Application processName="notepad" filePath="notepad.exe" version="1.3" />
<Uia className="Notepad" isDirectChild="True" name="Untitled - Notepad" role="window" />
<Uia accessKey="Alt" automationId="MenuBar" isDirectChild="True" role="menuBar" />
<Uia accessKey="Alt+o" isDirectChild="True" name="Format" role="menuItem" />
  • notepad ‘document’ area
<Application processName="notepad" filePath="notepad.exe" version="1.3" />
<Uia className="Notepad" isDirectChild="True" name="Untitled - Notepad" role="window" />
<Uia automationId="15" className="Edit" isDirectChild="True" role="document" />
  • calculator button ‘5’
<Application processName="ApplicationFrameHost" filePath="Calculator.exe" version="1.3" />
<Uia className="ApplicationFrameWindow" isDirectChild="True" name="计算器" role="window" />
<Uia className="Windows.UI.Core.CoreWindow" isDirectChild="True" name="计算器" role="window" />
<Uia className="LandmarkTarget" isDirectChild="True" role="group" />
<Uia automationId="NumberPad" className="NamedContainerAutomationPeer" isDirectChild="True" role="group" />
<Uia automationId="num5Button" className="Button" isDirectChild="True" />

4. UIA element properties #

By find_element on one UIA locator, you can get one UIA element, and properties of the element by get_property.
Clicknium UIA supports the following properties:

NameThe name of the user interface.
IsEnabledWhether the element is enabled in the user interface.
AccessKeyA string containing the access key character for the element.
AutomationIdA string containing the UI Automation identifier (ID) for the element.
BoundingRectangleThe coordinates of the rectangle that completely encloses the element.
ProcessIdThe process identifier (ID) of the element.
ItemTypeThe description of the item type.
IsPasswordWhether the UI Automation element contains protected content.
IsOffscreenWhether the UI Automation element is visible on the screen(true when the control is not visible; otherwise false).
AcceleratorKeyA string containing the accelerator key combinations for the element.
HelpTextThe help text of the element.
IsKeyboardFocusableWhether the UI Automation element can accept keyboard focus.
IsContentElementWhether the element is a content element.
IsControlElementWhether the element is viewed as a control.
HasKeyboardFocusWhether the element has keyboard focus.
FrameworkIdThe name of the UI framework, such as “Win32”, “WinForm”, or “DirectUI”. The default value is an empty string.
ControlTypeThe ControlType of the element.
What are your feelings
Updated on 30 November 2023