IA Automation

1. Overview #

Clicknium IA automation is based on Microsoft Active Accessibility(MSAA) to support the recording function, locators, and all operations on elements.
As MSAA is now considered a legacy API, we treat IA as a supplement of UIA, on some Electron&CEF applications or legacy applications, IA can be better. Please manually choose IA technology if you want to use IA during recording.

2. Locator attributes #

Learn locator concept first. For IA automation, the definition of the attributes is listed in the following:

NameequalscontainsstartWithendWithregex
NameYesYesYesYesYes
ClassNameYesYesYesYesYes
HelpTextYesYesYesYesYes
AccessKeyYesYesYesYesYes
DefaultActionYesYesYesYesYes
DescriptionYesYesYesYesYes
RoleYesNoNoNoNo
IsDirectChildYesNoNoNoNo
TagYesNoNoNoNo
IndexYesNoNoNoNo

3. Locator samples #

  • github desktop ‘open in visual studio code’ button
<Application processName="GitHubDesktop" filePath="GitHubDesktop.exe" version="1.3" />
<IA className="Chrome_WidgetWin_1" isDirectChild="True" name="GitHub Desktop" role="window" />
<IA role="document" />
<IA defaultAction="click" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" index="2" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" index="2" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" role="group" />
<IA className="Chrome_WidgetWin_1" defaultAction="press" name="Open in Visual Studio Code" />
  • github desktop commit ‘description` edit
<Application processName="GitHubDesktop" filePath="GitHubDesktop.exe" version="1.3" />
<IA className="Chrome_WidgetWin_1" isDirectChild="True" name="GitHub Desktop" role="window" />
<IA role="document" />
<IA defaultAction="click" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" index="2" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" index="2" isDirectChild="True" role="group" />
<IA defaultAction="click" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" name="Create commit" role="group" />
<IA defaultAction="click" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" role="group" />
<IA className="Chrome_WidgetWin_1" defaultAction="activate" name="Description" role="edit" />
  • github desktop ‘history’ tabitem
<Application processName="GitHubDesktop" filePath="GitHubDesktop.exe" version="1.3" />
<IA className="Chrome_WidgetWin_1" isDirectChild="True" name="GitHub Desktop" role="window" />
<IA role="document" />
<IA defaultAction="click" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" index="2" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" index="2" isDirectChild="True" role="group" />
<IA defaultAction="click" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" role="group" />
<IA defaultAction="click ancestor" isDirectChild="True" role="tab" />
<IA className="Chrome_WidgetWin_1" defaultAction="click" name="History" role="tabItem" />

4. IA element properties #

By find_element on one IA locator, you can get one IA element, and you can get properties of the element by get_property, Clicknium IA supports the following properties:

NameDescription
Namethe name of the user interface
IsEnabledwhether the element is enabled in the user interface
AccessKeya string containing the access key character for the element
BoundingRectanglethe coordinates of the rectangle that completely encloses the element
ProcessIdthe process identifier (ID) of the element
Descriptiondescription of the element
AcceleratorKeya string containing the accelerator key combinations for the element
HelpTextthe help text of the element
ControlTypethe ControlType of the element.
What are your feelings
Updated on 30 November 2023