appium-novawindows2-driver 1.1.21 → 1.1.22

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -1,716 +1,716 @@
1
- NovaWindows2 Driver
2
- ===================
3
-
4
- NovaWindows2 Driver is a custom Appium driver designed to tackle the limitations of existing Windows automation solutions like WinAppDriver. It supports testing Universal Windows Platform (UWP), Windows Forms (WinForms), Windows Presentation Foundation (WPF), and Classic Windows (Win32) apps on Windows 10 and later.
5
-
6
- Built to improve performance and reliability for traditional desktop applications, it offers:
7
- - **Faster XPath locator performance** — Reduces element lookup times, even in complex UIs.
8
- - **RawView element support** — Access elements typically hidden from the default ControlView/ContentView.
9
- - **Enhanced text input handling** — Fast text entry with support for various keyboard layouts.
10
- - **Platform-specific commands** — Supports direct window manipulation, advanced UI interactions, and more.
11
- - **Seamless Setup** — Designed to work without Developer Mode or additional software.
12
-
13
- ---
14
-
15
- ## 📑 Table of Contents
16
- - [Getting Started](#-getting-started)
17
- - [Configuration](#-configuration)
18
- - [Example Usage](#-example-usage)
19
- - [Key Features](#-key-features)
20
- - [Element Location](#element-location)
21
- - [Attribute Retrieval](#attribute-retrieval)
22
- - [PowerShell Execution](#powershell-execution)
23
- - [Platform-Specific Extensions](#-platform-specific-extensions)
24
- - [Mouse & Pointer](#mouse--pointer)
25
- - [Keyboard](#keyboard)
26
- - [Element Operations](#element-operations)
27
- - [Selection Management](#selection-management)
28
- - [Window Management](#window-management)
29
- - [System & State](#system--state)
30
- - [Development](#-development)
31
-
32
- ---
33
-
34
- ## 🚀 Getting Started
35
-
36
- ### Installation
37
- The driver is built for Appium 3. To install it, run:
38
- ```bash
39
- appium driver install --source=npm appium-novawindows2-driver
40
- ```
41
-
42
- ### Prerequisites
43
- - **Host OS**: Windows 10 or later (x64, ARM64 supported).
44
- - No Developer Mode or extra dependencies required.
45
-
46
- > [!NOTE]
47
- > **ARM Support**: The driver is compatible with ARM-based Windows. However, the **Screen Recording** feature is currently only supported on x64 due to `ffmpeg-static` limitations.
48
-
49
- ---
50
-
51
- ## ⚙️ Configuration
52
-
53
- NovaWindows2 Driver supports the following capabilities:
54
-
55
- | Capability Name | Description | Default | Example |
56
- | :--- | :--- | :--- | :--- |
57
- | `platformName` | Must be set to `Windows` (case-insensitive). | (Required) | `Windows` |
58
- | `automationName` | Must be set to `NovaWindows2` (case-insensitive). | (Required) | `NovaWindows2` |
59
- | `smoothPointerMove` | CSS-like easing function (including valid Bezier curve). This controls the smooth movement of the mouse for `delayBeforeClick` ms. | (None) | `ease-in`, `linear`, `ease`, `ease-in`, `ease-out`, `ease-in-out`, `cubic-bezier(0.42, 0, 0.58, 1)` |
60
- | `delayBeforeClick` | Time in milliseconds before a click is performed. | `0` | `500` |
61
- | `delayAfterClick` | Time in milliseconds after a click is performed. | `0` | `500` |
62
- | `appTopLevelWindow` | The handle of an existing application top-level window to attach to. It can be a number or string (not necessarily hexadecimal). | (None) | `12345`, `0x12345` |
63
- | `shouldCloseApp` | Whether to close the window of the application in test after the session finishes. | `true` | `false` |
64
- | `appArguments` | Optional string of arguments to pass to the app on launch. | (None) | `--debug` |
65
- | `appWorkingDir` | Optional working directory path for the application. | (None) | `C:\Temp` |
66
- | `prerun` | An object containing either `script` or `command` key. The value of each key must be a valid PowerShell script or command to be executed prior to the WinAppDriver session startup. See [Power Shell commands execution](#powershell-execution) for more details. | (None) | `{script: 'Get-Process outlook -ErrorAction SilentlyContinue'}` |
67
- | `postrun` | An object containing either `script` or `command` key. The value of each key must be a valid PowerShell script or command to be executed after WinAppDriver session is stopped. See [Power Shell commands execution](#powershell-execution) for more details. | (None) | `{command: '...'}` |
68
- | `isolatedScriptExecution` | Whether PowerShell scripts are executed in an isolated session. | `false` | `true` |
69
- | `powerShellCommandTimeout` | Timeout (ms) for PowerShell script execution. | `60000` | `30000` |
70
- | `convertAbsoluteXPathToRelativeFromElement` | Convert absolute XPath to relative when searching from an element. | `true` | `true` |
71
- | `includeContextElementInSearch` | Include the context element itself in the search. | `true` | `true` |
72
- | `releaseModifierKeys` | Whether to release modifier keys after `sendKeys`. | `true` | `true` |
73
- | `typeDelay` | Time in milliseconds to wait after inputting each character. Note that this delay does not apply to modifier keys (Shift, Ctrl, Alt, Win). | `0` | `100` |
74
- | `ms:waitForAppLaunch` | Time in seconds to wait after launching the app before searching for the root window. | `0` | `3` |
75
- | `ms:forcequit` | Whether to force-kill the application process on session deletion. | `false` | `true` |
76
-
77
- ---
78
-
79
-
80
-
81
- ## 💡 Example Usage
82
-
83
- Check out the [examples/refactor](examples/refactor) directory for comprehensive examples.
84
-
85
- ### Python (Appium-Python-Client)
86
- ```python
87
- from appium import webdriver
88
- from appium.options.windows import WindowsOptions
89
-
90
- options = WindowsOptions()
91
- options.app = 'C:\\Windows\\System32\\notepad.exe'
92
- options.automation_name = 'NovaWindows2'
93
-
94
- driver = webdriver.Remote('http://127.0.0.1:4723', options=options)
95
- # ... tests ...
96
- driver.quit()
97
- ```
98
-
99
- ---
100
-
101
- ## ✨ Key Features
102
-
103
- ### Element Location
104
- Appium Windows Driver supports the same location strategies [the WinAppDriver supports](https://github.com/microsoft/WinAppDriver/blob/master/Docs/AuthoringTestScripts.md#supported-locators-to-find-ui-elements), but also includes Windows UIAutomation conditions:
105
-
106
- | Name | Description | Example |
107
- | :--- | :--- | :--- |
108
- | `accessibility id` | This strategy is `AutomationId` attribute in inspect.exe | `CalculatorResults` |
109
- | `class name` | This strategy is `ClassName` attribute in inspect.exe | `TextBlock` |
110
- | `id` | This strategy is `RuntimeId` (decimal) attribute in inspect.exe | `42.333896.3.1` |
111
- | `name` | This strategy is `Name` attribute in inspect.exe | `Calculator` |
112
- | `tag name` | This strategy is `LocalizedControlType` (upper camel case) attribute in inspect.exe | `Text` |
113
- | `xpath` | Custom XPath 1.0 queries on any attribute exposed by inspect.exe. | `(//Button)[2]` |
114
- | `windows uiautomation` | UIAutomation conditions (C# or PowerShell syntax). | `new PropertyCondition(...)` |
115
-
116
- ### Attribute Retrieval
117
- Retrieve comprehensive details about UI elements using standard or bulk methods.
118
-
119
- #### Supported Attributes
120
- - **Standard UIA Properties**: `AutomationId`, `Name`, `ClassName`, `RuntimeId`, `ControlType`, `IsEnabled`, `IsOffscreen`, and all other direct UIA properties.
121
- - **Pattern-Specific Properties**: Use dot-notation (`Pattern.Property`) to access pattern members, for example:
122
- - `Value.Value`, `Value.IsReadOnly`
123
- - `Window.CanMaximize`, `Window.CanMinimize`, `Window.WindowVisualState`
124
- - `Toggle.ToggleState`
125
- - `ExpandCollapse.ExpandCollapseState`
126
- - `Selection.Selection`, `Selection.CanSelectMultiple`
127
- - *Includes support for 20+ UIA patterns.*
128
- - **Legacy Properties (MSAA Fallback)**:
129
- - **Shorthand Aliases**: `legacyname`, `legacyvalue`, `legacyrole`, `legacystate`, `legacydescription`, `legacyhelp`, `legacykeyboardshortcut`, `legacydefaultaction`, `legacychildid`.
130
- - **Dot-Notation**: `LegacyIAccessible.Name`, `LegacyIAccessible.Role`, etc.
131
- - **Special Keywords**:
132
- - `"all"`: Returns all attributes.
133
- - `"source"`: Returns the XML source for the element and its subtree.
134
- - **Enhanced Type Names**:
135
- - `ControlType` returns a human-readable short name (e.g., `Button`, `ListItem`, `Text`) instead of the UIA constant.
136
-
137
- ```python
138
- # List specific attribute
139
- value = element.get_attribute("Value.Value")
140
-
141
- # Get all attributes as a JSON string
142
- all_attributes = element.get_attribute("all")
143
-
144
- # Get XML source for this element and its subtree
145
- element_source = element.get_attribute("source")
146
- ```
147
-
148
- ### PowerShell Execution
149
- Execute internal PowerShell scripts or commands directly from your test. This requires the `power_shell` insecure feature to be enabled on the Appium server.
150
-
151
- It is possible to execute a single PowerShell command or a whole script. Note that `powerShell` is case-insensitive.
152
-
153
- ```python
154
- # Execute a command string
155
- driver.execute_script('powerShell', {'command': 'Get-Process Notepad'})
156
-
157
- # Execute a script string
158
- driver.execute_script('powerShell', {'script': '$p = Get-Process Notepad; $p.Kill();'})
159
-
160
- # Shorthand (executes as command/script depending on context)
161
- driver.execute_script('powerShell', 'Get-Process')
162
- ```
163
-
164
- ### Enhanced Text Input
165
- You can specify the delay directly within the text string using the `[delay:ms]` pattern. This overrides the session setting (set via `windows: typeDelay` or `typeDelay` capability) for that specific action.
166
-
167
- ```python
168
- driver.find_element(...).send_keys("[delay:500]Slow text")
169
- ```
170
-
171
- ---
172
-
173
- ## 🛠 Platform-Specific Extensions
174
-
175
- All extensions are invoked via `driver.executeScript("windows: <methodName>", ...args)`.
176
- Below are the detailed descriptions and arguments for each command.
177
-
178
- > **Note**
179
- > In most cases, commands can be used more intuitively by passing the element as the first argument (if required) and other parameters subsequently.
180
-
181
- ### Mouse & Pointer
182
-
183
- #### `windows: click`
184
- This is a shortcut for a single mouse click gesture.
185
-
186
- | Name | Type | Required | Description | Example |
187
- | :--- | :--- | :--- | :--- | :--- |
188
- | `elementId` | `string` | no | Hexadecimal identifier of the element to click on. If this parameter and coordinates are missing, the command uses the current cursor position. If `elementId` is present without coordinates, it clicks the center of the element. | `123e4567-e89b...` |
189
- | `x` | `number` | no | Integer horizontal coordinate of the click point. If omitted (along with `elementId`), the current cursor position is used. If `elementId` is present, `x` is an offset from the element's top-left corner. | `100` |
190
- | `y` | `number` | no | Integer vertical coordinate of the click point. If omitted (along with `elementId`), the current cursor position is used. If `elementId` is present, `y` is an offset from the element's top-left corner. | `100` |
191
- | `button` | `string` | no | Name of the mouse button to be clicked. Supported button names are: `left`, `middle`, `right`, `back`, `forward`. The default value is `left`. | `right` |
192
- | `modifierKeys` | `string[]` \| `string` | no | List of possible keys or a single key name to depress while the click is being performed. Supported key names are: `Shift`, `Ctrl`, `Alt`, `Win`. | `['ctrl', 'alt']` |
193
- | `durationMs` | `number` | no | The number of milliseconds to wait between pressing and releasing the mouse button. By default no delay is applied. | `500` |
194
- | `times` | `number` | no | How many times the click must be performed. One by default. | `2` |
195
- | `interClickDelayMs` | `number` | no | Duration of the pause between each click gesture. Only makes sense if `times` is greater than one. 100ms by default. | `10` |
196
-
197
- #### Usage
198
-
199
- **Scenario 1: Using Element ID (Clicks Center)**
200
- ```python
201
- driver.execute_script('windows: click', {
202
- 'elementId': element.id,
203
- 'button': 'right',
204
- 'times': 2
205
- })
206
- ```
207
-
208
- **Scenario 2: Using Absolute Coordinates**
209
- ```python
210
- driver.execute_script('windows: click', {
211
- 'x': 500,
212
- 'y': 300,
213
- 'button': 'left'
214
- })
215
- ```
216
-
217
- **Scenario 3: Using Element ID with Offset (Relative to Top-Left)**
218
- ```python
219
- driver.execute_script('windows: click', {
220
- 'elementId': element.id,
221
- 'x': 10, # 10px from the left edge of the element
222
- 'y': 10 # 10px from the top edge of the element
223
- })
224
- ```
225
-
226
- #### `windows: clickAndDrag`
227
- Performs a click-and-drag gesture.
228
-
229
- | Name | Type | Required | Description | Example |
230
- | :--- | :--- | :--- | :--- | :--- |
231
- | `startElementId` | `string` | no | Hexadecimal identifier of the start element. If this and start coordinates are missing, the current cursor position is used. | `123e4567-e89b...` |
232
- | `startX` | `number` | no | Horizontal coordinate of the start point. If omitted (along with `startElementId`), the current cursor position is used. | `100` |
233
- | `startY` | `number` | no | Vertical coordinate of the start point. If omitted (along with `startElementId`), the current cursor position is used. | `100` |
234
- | `endElementId` | `string` | no | Hexadecimal identifier of the end element. If this and end coordinates are missing, the current cursor position is used. | `123e4567-e89b...` |
235
- | `endX` | `number` | no | Horizontal coordinate of the end point. If omitted (along with `endElementId`), the current cursor position is used. | `200` |
236
- | `endY` | `number` | no | Vertical coordinate of the end point. If omitted (along with `endElementId`), the current cursor position is used. | `200` |
237
- | `durationMs` | `number` | no | Duration of the drag in milliseconds. Default is `1000`. | `2000` |
238
- | `button` | `string` | no | Mouse button to use (`left`, `right`, `middle`). Default is `left`. | `right` |
239
- | `smoothPointerMove` | `string` | no | CSS-like easing function (e.g., `linear`, `ease-out`) for smooth movement. Default is `None`. | `linear` |
240
- | `modifierKeys` | `string[]` \| `string` | no | Keys to hold during the drag. | `shift` |
241
-
242
- #### Usage
243
-
244
- **Scenario 1: Element to Element (Center to Center)**
245
- ```python
246
- driver.execute_script('windows: clickAndDrag', {
247
- 'startElementId': element1.id,
248
- 'endElementId': element2.id,
249
- 'durationMs': 2000
250
- })
251
- ```
252
-
253
- **Scenario 2: Absolute Coordinates**
254
- ```python
255
- driver.execute_script('windows: clickAndDrag', {
256
- 'startX': 100,
257
- 'startY': 100,
258
- 'endX': 500,
259
- 'endY': 500,
260
- 'smoothPointerMove': 'linear'
261
- })
262
- ```
263
-
264
- **Scenario 3: Element with Offset (Drag from specific point inside element)**
265
- ```python
266
- driver.execute_script('windows: clickAndDrag', {
267
- 'startElementId': element1.id,
268
- 'startX': 10, # Start 10px from left of element1
269
- 'startY': 10, # Start 10px from top of element1
270
- 'endElementId': element2.id,
271
- 'endX': 50, # End 50px from left of element2
272
- 'endY': 50 # End 50px from top of element2
273
- })
274
- ```
275
-
276
- #### `windows: scroll`
277
- This is a shortcut for a mouse wheel scroll gesture. The API is a thin wrapper over the SendInput WinApi call.
278
-
279
- | Name | Type | Required | Description | Example |
280
- | :--- | :--- | :--- | :--- | :--- |
281
- | `elementId` | `string` | no | Same as in `windows: click`. Falls back to the current cursor position if omitted along with coordinates. | `123e4567-e89b...` |
282
- | `x` | `number` | no | Same as in `windows: click`. Falls back to the current cursor position if omitted along with `elementId`. | `100` |
283
- | `y` | `number` | no | Same as in `windows: click`. Falls back to the current cursor position if omitted along with `elementId`. | `100` |
284
- | `deltaX` | `number` | no | The amount of horizontal wheel movement measured in wheel clicks. Positive = right, Negative = left. | `-5` |
285
- | `deltaY` | `number` | no | The amount of vertical wheel movement. Positive = forward (away), Negative = backward (toward). | `5` |
286
- | `modifierKeys` | `string[]` \| `string` | no | Same as in `windows: click`. | `win` |
287
-
288
- #### Usage
289
- ```python
290
- driver.execute_script('windows: scroll', {
291
- 'elementId': element.id,
292
- 'deltaY': -120, # Scroll down 3 lines
293
- 'modifierKeys': 'shift'
294
- })
295
- ```
296
-
297
- #### `windows: hover`
298
- This is a shortcut for a hover gesture.
299
-
300
- | Name | Type | Required | Description | Example |
301
- | :--- | :--- | :--- | :--- | :--- |
302
- | `startElementId` | `string` | no | Same as in `windows: click`. Falls back to current cursor position if omitted along with coordinates. | `123e4567-e89b...` |
303
- | `startX` | `number` | no | Same as in `windows: click`. Falls back to current cursor position if omitted along with `startElementId`. | `100` |
304
- | `startY` | `number` | no | Same as in `windows: click`. Falls back to current cursor position if omitted along with `startElementId`. | `100` |
305
- | `endElementId` | `string` | no | Same as in `windows: click`. | `123e4567-e89b...` |
306
- | `endX` | `number` | no | Same as in `windows: click`. | `200` |
307
- | `endY` | `number` | no | Same as in `windows: click`. | `200` |
308
- | `modifierKeys` | `string[]` \| `string` | no | Same as in `windows: click`. | `shift` |
309
- | `durationMs` | `number` | no | The number of milliseconds between moving the cursor from the starting to the ending hover point. 500ms by default. | `700` |
310
-
311
- #### Usage
312
-
313
- **Scenario 1: Element to Element (Center to Center)**
314
- ```python
315
- driver.execute_script('windows: hover', {
316
- 'startElementId': element1.id,
317
- 'endElementId': element2.id,
318
- 'durationMs': 1000
319
- })
320
- ```
321
-
322
- **Scenario 2: Absolute Coordinates**
323
- ```python
324
- driver.execute_script('windows: hover', {
325
- 'startX': 100,
326
- 'startY': 100,
327
- 'endX': 500,
328
- 'endY': 500
329
- })
330
- ```
331
-
332
- **Scenario 3: Element with Offset (Hover specific point)**
333
- ```python
334
- driver.execute_script('windows: hover', {
335
- 'startElementId': element1.id,
336
- 'startX': 5,
337
- 'startY': 5,
338
- 'endElementId': element2.id,
339
- 'endX': 5,
340
- 'endY': 5
341
- })
342
- ```
343
-
344
- ### Keyboard
345
-
346
- #### `windows: typeDelay`
347
- Sets the delay between key injections in milliseconds. This persistent setting applies to the entire session until changed.
348
-
349
- | Name | Type | Required | Description | Example |
350
- | :--- | :--- | :--- | :--- | :--- |
351
- | `delay` | `number` | yes | Delay in milliseconds. | `100` |
352
-
353
- #### Usage
354
- ```python
355
- driver.execute_script('windows: typeDelay', {'delay': 500})
356
- # or shorthand
357
- driver.execute_script('windows: typeDelay', '500')
358
- ```
359
-
360
- #### `windows: keys`
361
- This is a shortcut for a customized keyboard input. Selenium keys should also work as modifier keys.
362
-
363
- | Name | Type | Required | Description | Example |
364
- | :--- | :--- | :--- | :--- | :--- |
365
- | `actions` | `KeyAction[]` \| `KeyAction` | yes | One or more KeyAction dictionaries. | `[{'virtualKeyCode': 0x10, 'down': true}]` |
366
- | `forceUnicode` | `boolean` | no | Forces the characters to be sent as unicode characters. | `true` |
367
-
368
- ##### KeyAction Dictionary
369
-
370
- | Name | Type | Required | Description | Example |
371
- | :--- | :--- | :--- | :--- | :--- |
372
- | `pause` | `number` | no | Allows to set a delay in milliseconds between key input series. | `100` |
373
- | `text` | `string` | no | Non-empty string of Unicode text to type. | `Hello` |
374
- | `virtualKeyCode` | `number` | no | Valid virtual key code. | `0x10` |
375
- | `down` | `boolean` | no | If set to `true` then the corresponding key will be depressed, `false` - released. | `true` |
376
-
377
- #### Usage
378
- ```python
379
- driver.execute_script('windows: keys', {
380
- 'actions': [
381
- {'virtualKeyCode': 0x10, 'down': True}, # Shift Down
382
- {'text': 'Hello World'},
383
- {'virtualKeyCode': 0x10, 'down': False} # Shift Up
384
- ]
385
- })
386
- ```
387
-
388
- ### System & State
389
-
390
- #### `windows: setClipboard`
391
- Sets Windows clipboard content to the given text or a PNG image.
392
-
393
- | Name | Type | Required | Description | Example |
394
- | :--- | :--- | :--- | :--- | :--- |
395
- | `b64Content` | `string` | yes | Base64-encoded content of the clipboard to be set. | `QXBwaXVt` |
396
- | `contentType` | `string` | no | Set to `plaintext` (default) or `image`. | `image` |
397
-
398
- #### Usage
399
- ```python
400
- driver.execute_script('windows: setClipboard', {
401
- 'b64Content': 'SGVsbG8=', # "Hello" in Base64
402
- 'contentType': 'plaintext'
403
- })
404
- ```
405
-
406
- #### `windows: getClipboard`
407
- Retrieves Windows clipboard content.
408
-
409
- | Name | Type | Required | Description | Example |
410
- | :--- | :--- | :--- | :--- | :--- |
411
- | `contentType` | `string` | no | Set to `plaintext` (default) or `image`. | `image` |
412
-
413
- #### Usage
414
- ```python
415
- content = driver.execute_script('windows: getClipboard', {
416
- 'contentType': 'plaintext'
417
- })
418
- print(content)
419
- ```
420
-
421
- #### `windows: pushCacheRequest`
422
- This is an asynchronous function that sends cache requests based on specific conditions.
423
-
424
- | Name | Type | Required | Description | Example |
425
- | :--- | :--- | :--- | :--- | :--- |
426
- | `treeFilter` | `string` | yes | Defines the filter that is applied when walking the automation tree. | `RawView` |
427
- | `treeScope` | `string` | no | Defines the scope of the automation tree to be cached. | `SubTree` |
428
- | `automationElementMode` | `string` | no | Specifies the mode of automation element (e.g., `None`, `Full`). | `Full` |
429
-
430
- #### Usage
431
- ```python
432
- driver.execute_script('windows: pushCacheRequest', {
433
- 'treeFilter': 'RawView',
434
- 'treeScope': 'SubTree'
435
- })
436
- ```
437
-
438
- ### Element Operations
439
-
440
- #### `windows: invoke`
441
- Invokes a UI element pattern, simulating an interaction like clicking or activating the element.
442
-
443
- | Position | Type | Description | Example |
444
- | :--- | :--- | :--- | :--- |
445
- | 1 | `Element` | The UI element on which the `InvokePattern` is called. | `element` |
446
-
447
- #### Usage
448
- ```python
449
- driver.execute_script('windows: invoke', element)
450
- ```
451
-
452
- #### `windows: expand`
453
- Expands a UI element that supports the `ExpandPattern`.
454
-
455
- | Position | Type | Description | Example |
456
- | :--- | :--- | :--- | :--- |
457
- | 1 | `Element` | The UI element to expand. | `element` |
458
-
459
- #### Usage
460
- ```python
461
- driver.execute_script('windows: expand', element)
462
- ```
463
-
464
- #### `windows: collapse`
465
- Collapses a UI element that supports the `CollapsePattern`.
466
-
467
- | Position | Type | Description | Example |
468
- | :--- | :--- | :--- | :--- |
469
- | 1 | `Element` | The UI element to collapse. | `element` |
470
-
471
- #### Usage
472
- ```python
473
- driver.execute_script('windows: collapse', element)
474
- ```
475
-
476
- #### `windows: setValue`
477
- Sets the value of a UI element using the `ValuePattern`.
478
-
479
- | Position | Type | Description | Example |
480
- | :--- | :--- | :--- | :--- |
481
- | 1 | `Element` | The UI element whose value will be set. | `element` |
482
- | 2 | `string` | The value to be set. | `"new value"` |
483
-
484
- #### Usage
485
- ```python
486
- driver.execute_script('windows: setValue', element, 'New Value')
487
- ```
488
-
489
- #### `windows: getValue`
490
- Gets the current value of a UI element that supports the `ValuePattern`.
491
-
492
- | Position | Type | Description | Example |
493
- | :--- | :--- | :--- | :--- |
494
- | 1 | `Element` | The UI element from which to retrieve the value. | `element` |
495
-
496
- #### Usage
497
- ```python
498
- value = driver.execute_script('windows: getValue', element)
499
- ```
500
-
501
- #### `windows: scrollIntoView`
502
- Scrolls the UI element into view using the `ScrollItemPattern`.
503
-
504
- | Position | Type | Description | Example |
505
- | :--- | :--- | :--- | :--- |
506
- | 1 | `Element` | The UI element to bring into view. | `element` |
507
-
508
- #### Usage
509
- ```python
510
- driver.execute_script('windows: scrollIntoView', element)
511
- ```
512
-
513
- > **Note**
514
- > You can also use the standard JavaScript way:
515
- > ```python
516
- > driver.execute_script('arguments[0].scrollIntoView()', element)
517
- > ```
518
-
519
- #### `windows: toggle`
520
- Toggles a UI element’s state using the `TogglePattern`.
521
-
522
- | Position | Type | Description | Example |
523
- | :--- | :--- | :--- | :--- |
524
- | 1 | `Element` | The UI element to toggle. | `element` |
525
-
526
- #### Usage
527
- ```python
528
- driver.execute_script('windows: toggle', element)
529
- ```
530
-
531
- ### Selection Management
532
-
533
- #### `windows: select`
534
- Selects a UI element using the `SelectionPattern`.
535
-
536
- | Position | Type | Description | Example |
537
- | :--- | :--- | :--- | :--- |
538
- | 1 | `Element` | The UI element to select. | `element` |
539
-
540
- #### Usage
541
- ```python
542
- driver.execute_script('windows: select', element)
543
- ```
544
-
545
- #### `windows: addToSelection`
546
- Adds an element to the current selection on a UI element that supports the `SelectionPattern`.
547
-
548
- | Position | Type | Description | Example |
549
- | :--- | :--- | :--- | :--- |
550
- | 1 | `Element` | The UI element to add to the selection. | `element` |
551
-
552
- #### Usage
553
- ```python
554
- driver.execute_script('windows: addToSelection', element)
555
- ```
556
-
557
- #### `windows: removeFromSelection`
558
- Removes an element from the current selection on a UI element that supports the `SelectionPattern`.
559
-
560
- | Position | Type | Description | Example |
561
- | :--- | :--- | :--- | :--- |
562
- | 1 | `Element` | The UI element to remove from the selection. | `element` |
563
-
564
- #### Usage
565
- ```python
566
- driver.execute_script('windows: removeFromSelection', element)
567
- ```
568
-
569
- #### `windows: isMultiple`
570
- Checks if a UI element supports multiple selection using the `SelectionPattern`.
571
-
572
- | Position | Type | Description | Example |
573
- | :--- | :--- | :--- | :--- |
574
- | 1 | `Element` | The UI element to check. | `element` |
575
-
576
- #### Usage
577
- ```python
578
- is_multi = driver.execute_script('windows: isMultiple', element)
579
- ```
580
-
581
- #### `windows: selectedItem`
582
- Gets the selected item from a UI element that supports the `SelectionPattern`.
583
-
584
- | Position | Type | Description | Example |
585
- | :--- | :--- | :--- | :--- |
586
- | 1 | `Element` | The UI element from which to retrieve the selected item. | `element` |
587
-
588
- #### Usage
589
- ```python
590
- selected_el = driver.execute_script('windows: selectedItem', element)
591
- ```
592
-
593
- #### `windows: allSelectedItems`
594
- Gets all selected items from a UI element that supports the `SelectionPattern`.
595
-
596
- | Position | Type | Description | Example |
597
- | :--- | :--- | :--- | :--- |
598
- | 1 | `Element` | The UI element from which to retrieve all selected items. | `element` |
599
-
600
- #### Usage
601
- ```python
602
- selected_els = driver.execute_script('windows: allSelectedItems', element)
603
- ```
604
-
605
- ### Window Management
606
-
607
- #### `windows: maximize`
608
- Maximizes a window or UI element using the `WindowPattern`.
609
-
610
- | Position | Type | Description | Example |
611
- | :--- | :--- | :--- | :--- |
612
- | 1 | `Element` | The window or UI element to maximize. | `element` |
613
-
614
- #### Usage
615
- ```python
616
- driver.execute_script('windows: maximize', element)
617
- ```
618
-
619
- #### `windows: minimize`
620
- Minimizes a window or UI element using the `WindowPattern`.
621
-
622
- | Position | Type | Description | Example |
623
- | :--- | :--- | :--- | :--- |
624
- | 1 | `Element` | The window or UI element to minimize. | `element` |
625
-
626
- #### Usage
627
- ```python
628
- driver.execute_script('windows: minimize', element)
629
- ```
630
-
631
- #### `windows: restore`
632
- Restores a window or UI element to its normal state using the `WindowPattern`.
633
-
634
- | Position | Type | Description | Example |
635
- | :--- | :--- | :--- | :--- |
636
- | 1 | `Element` | The window or UI element to restore. | `element` |
637
-
638
- #### Usage
639
- ```python
640
- driver.execute_script('windows: restore', element)
641
- ```
642
-
643
- #### `windows: close`
644
- Closes a window or UI element using the `WindowPattern`.
645
-
646
- | Position | Type | Description | Example |
647
- | :--- | :--- | :--- | :--- |
648
- | 1 | `Element` | The window or UI element to close. | `element` |
649
-
650
- #### Usage
651
- ```python
652
- driver.execute_script('windows: close', element)
653
- ```
654
-
655
- #### `windows: setProcessForeground`
656
- Brings the main window of the specified process to the foreground.
657
-
658
- | Name | Type | Required | Description | Example |
659
- | :--- | :--- | :--- | :--- | :--- |
660
- | `process` | `string` | yes | The name of the process whose window should be brought to the foreground. | `notepad.exe` |
661
-
662
- #### Usage
663
- ```python
664
- driver.execute_script('windows: setProcessForeground', {
665
- 'process': 'notepad.exe'
666
- })
667
- ```
668
-
669
- #### `windows: setFocus`
670
- Sets focus to the specified UI element using UIAutomationElement's `SetFocus` method.
671
-
672
- | Position | Type | Description | Example |
673
- | :--- | :--- | :--- | :--- |
674
- | 1 | `Element` | The UI element to set focus on. | `element` |
675
-
676
- #### Usage
677
- ```python
678
- driver.execute_script('windows: setFocus', element)
679
- ```
680
-
681
- ### Screen Recording
682
-
683
- #### `windows: startRecordingScreen`
684
- Starts screen recording using ffmpeg.
685
-
686
- | Name | Type | Required | Description | Example |
687
- | :--- | :--- | :--- | :--- | :--- |
688
- | `fps` | `number` | no | Frames per second. Default is 15. | `15` |
689
- | `timeLimit` | `number` | no | Recording time limit in seconds. Default is 600. | `60` |
690
- | `preset` | `string` | no | ffmpeg preset. Default is `veryfast`. | `ultrafast` |
691
- | `captureCursor` | `boolean` | no | Whether to capture mouse cursor. | `true` |
692
- | `captureClicks` | `boolean` | no | Whether to capture mouse clicks. | `true` |
693
-
694
- > [!IMPORTANT]
695
- > This feature requires the optional recording stack to be installed separately:
696
- > `npm i -g asyncbox teen_process ffmpeg-static`
697
- > Recording is currently **not supported** on ARM architecture (no ARM build of `ffmpeg-static`).
698
-
699
- #### `windows: stopRecordingScreen`
700
- Stops screen recording and returns the video as base64 string.
701
-
702
- | Name | Type | Required | Description | Example |
703
- | :--- | :--- | :--- | :--- | :--- |
704
- | `remotePath` | `string` | no | Remote path to upload the video to. | `http://example.com/upload` |
705
-
706
- ---
707
-
708
- ## 🛠 Development
709
-
710
- Recommended VS Code plugin: [Comment tagged templates](https://marketplace.visualstudio.com/items?itemName=bierner.comment-tagged-templates) for syntax highlighting.
711
-
712
- ```bash
713
- npm install # Setup dependencies
714
- npm run lint # Code quality check
715
- npm run build # Transpile TypeScript to JS
716
- ```
1
+ NovaWindows2 Driver
2
+ ===================
3
+
4
+ NovaWindows2 Driver is a custom Appium driver designed to tackle the limitations of existing Windows automation solutions like WinAppDriver. It supports testing Universal Windows Platform (UWP), Windows Forms (WinForms), Windows Presentation Foundation (WPF), and Classic Windows (Win32) apps on Windows 10 and later.
5
+
6
+ Built to improve performance and reliability for traditional desktop applications, it offers:
7
+ - **Faster XPath locator performance** — Reduces element lookup times, even in complex UIs.
8
+ - **RawView element support** — Access elements typically hidden from the default ControlView/ContentView.
9
+ - **Enhanced text input handling** — Fast text entry with support for various keyboard layouts.
10
+ - **Platform-specific commands** — Supports direct window manipulation, advanced UI interactions, and more.
11
+ - **Seamless Setup** — Designed to work without Developer Mode or additional software.
12
+
13
+ ---
14
+
15
+ ## 📑 Table of Contents
16
+ - [Getting Started](#-getting-started)
17
+ - [Configuration](#-configuration)
18
+ - [Example Usage](#-example-usage)
19
+ - [Key Features](#-key-features)
20
+ - [Element Location](#element-location)
21
+ - [Attribute Retrieval](#attribute-retrieval)
22
+ - [PowerShell Execution](#powershell-execution)
23
+ - [Platform-Specific Extensions](#-platform-specific-extensions)
24
+ - [Mouse & Pointer](#mouse--pointer)
25
+ - [Keyboard](#keyboard)
26
+ - [Element Operations](#element-operations)
27
+ - [Selection Management](#selection-management)
28
+ - [Window Management](#window-management)
29
+ - [System & State](#system--state)
30
+ - [Development](#-development)
31
+
32
+ ---
33
+
34
+ ## 🚀 Getting Started
35
+
36
+ ### Installation
37
+ The driver is built for Appium 3. To install it, run:
38
+ ```bash
39
+ appium driver install --source=npm appium-novawindows2-driver
40
+ ```
41
+
42
+ ### Prerequisites
43
+ - **Host OS**: Windows 10 or later (x64, ARM64 supported).
44
+ - No Developer Mode or extra dependencies required.
45
+
46
+ > [!NOTE]
47
+ > **ARM Support**: The driver is compatible with ARM-based Windows. However, the **Screen Recording** feature is currently only supported on x64 due to `ffmpeg-static` limitations.
48
+
49
+ ---
50
+
51
+ ## ⚙️ Configuration
52
+
53
+ NovaWindows2 Driver supports the following capabilities:
54
+
55
+ | Capability Name | Description | Default | Example |
56
+ | :--- | :--- | :--- | :--- |
57
+ | `platformName` | Must be set to `Windows` (case-insensitive). | (Required) | `Windows` |
58
+ | `automationName` | Must be set to `NovaWindows2` (case-insensitive). | (Required) | `NovaWindows2` |
59
+ | `smoothPointerMove` | CSS-like easing function (including valid Bezier curve). This controls the smooth movement of the mouse for `delayBeforeClick` ms. | (None) | `ease-in`, `linear`, `ease`, `ease-in`, `ease-out`, `ease-in-out`, `cubic-bezier(0.42, 0, 0.58, 1)` |
60
+ | `delayBeforeClick` | Time in milliseconds before a click is performed. | `0` | `500` |
61
+ | `delayAfterClick` | Time in milliseconds after a click is performed. | `0` | `500` |
62
+ | `appTopLevelWindow` | The handle of an existing application top-level window to attach to. It can be a number or string (not necessarily hexadecimal). | (None) | `12345`, `0x12345` |
63
+ | `shouldCloseApp` | Whether to close the window of the application in test after the session finishes. | `true` | `false` |
64
+ | `appArguments` | Optional string of arguments to pass to the app on launch. | (None) | `--debug` |
65
+ | `appWorkingDir` | Optional working directory path for the application. | (None) | `C:\Temp` |
66
+ | `prerun` | An object containing either `script` or `command` key. The value of each key must be a valid PowerShell script or command to be executed prior to the WinAppDriver session startup. See [Power Shell commands execution](#powershell-execution) for more details. | (None) | `{script: 'Get-Process outlook -ErrorAction SilentlyContinue'}` |
67
+ | `postrun` | An object containing either `script` or `command` key. The value of each key must be a valid PowerShell script or command to be executed after WinAppDriver session is stopped. See [Power Shell commands execution](#powershell-execution) for more details. | (None) | `{command: '...'}` |
68
+ | `isolatedScriptExecution` | Whether PowerShell scripts are executed in an isolated session. | `false` | `true` |
69
+ | `powerShellCommandTimeout` | Timeout (ms) for PowerShell script execution. | `60000` | `30000` |
70
+ | `convertAbsoluteXPathToRelativeFromElement` | Convert absolute XPath to relative when searching from an element. | `true` | `true` |
71
+ | `includeContextElementInSearch` | Include the context element itself in the search. | `true` | `true` |
72
+ | `releaseModifierKeys` | Whether to release modifier keys after `sendKeys`. | `true` | `true` |
73
+ | `typeDelay` | Time in milliseconds to wait after inputting each character. Note that this delay does not apply to modifier keys (Shift, Ctrl, Alt, Win). | `0` | `100` |
74
+ | `ms:waitForAppLaunch` | Time in seconds to wait after launching the app before searching for the root window. | `0` | `3` |
75
+ | `ms:forcequit` | Whether to force-kill the application process on session deletion. | `false` | `true` |
76
+
77
+ ---
78
+
79
+
80
+
81
+ ## 💡 Example Usage
82
+
83
+ Check out the [examples/refactor](examples/refactor) directory for comprehensive examples.
84
+
85
+ ### Python (Appium-Python-Client)
86
+ ```python
87
+ from appium import webdriver
88
+ from appium.options.windows import WindowsOptions
89
+
90
+ options = WindowsOptions()
91
+ options.app = 'C:\\Windows\\System32\\notepad.exe'
92
+ options.automation_name = 'NovaWindows2'
93
+
94
+ driver = webdriver.Remote('http://127.0.0.1:4723', options=options)
95
+ # ... tests ...
96
+ driver.quit()
97
+ ```
98
+
99
+ ---
100
+
101
+ ## ✨ Key Features
102
+
103
+ ### Element Location
104
+ Appium Windows Driver supports the same location strategies [the WinAppDriver supports](https://github.com/microsoft/WinAppDriver/blob/master/Docs/AuthoringTestScripts.md#supported-locators-to-find-ui-elements), but also includes Windows UIAutomation conditions:
105
+
106
+ | Name | Description | Example |
107
+ | :--- | :--- | :--- |
108
+ | `accessibility id` | This strategy is `AutomationId` attribute in inspect.exe | `CalculatorResults` |
109
+ | `class name` | This strategy is `ClassName` attribute in inspect.exe | `TextBlock` |
110
+ | `id` | This strategy is `RuntimeId` (decimal) attribute in inspect.exe | `42.333896.3.1` |
111
+ | `name` | This strategy is `Name` attribute in inspect.exe | `Calculator` |
112
+ | `tag name` | This strategy is `LocalizedControlType` (upper camel case) attribute in inspect.exe | `Text` |
113
+ | `xpath` | Custom XPath 1.0 queries on any attribute exposed by inspect.exe. | `(//Button)[2]` |
114
+ | `windows uiautomation` | UIAutomation conditions (C# or PowerShell syntax). | `new PropertyCondition(...)` |
115
+
116
+ ### Attribute Retrieval
117
+ Retrieve comprehensive details about UI elements using standard or bulk methods.
118
+
119
+ #### Supported Attributes
120
+ - **Standard UIA Properties**: `AutomationId`, `Name`, `ClassName`, `RuntimeId`, `ControlType`, `IsEnabled`, `IsOffscreen`, and all other direct UIA properties.
121
+ - **Pattern-Specific Properties**: Use dot-notation (`Pattern.Property`) to access pattern members, for example:
122
+ - `Value.Value`, `Value.IsReadOnly`
123
+ - `Window.CanMaximize`, `Window.CanMinimize`, `Window.WindowVisualState`
124
+ - `Toggle.ToggleState`
125
+ - `ExpandCollapse.ExpandCollapseState`
126
+ - `Selection.Selection`, `Selection.CanSelectMultiple`
127
+ - *Includes support for 20+ UIA patterns.*
128
+ - **Legacy Properties (MSAA Fallback)**:
129
+ - **Shorthand Aliases**: `legacyname`, `legacyvalue`, `legacyrole`, `legacystate`, `legacydescription`, `legacyhelp`, `legacykeyboardshortcut`, `legacydefaultaction`, `legacychildid`.
130
+ - **Dot-Notation**: `LegacyIAccessible.Name`, `LegacyIAccessible.Role`, etc.
131
+ - **Special Keywords**:
132
+ - `"all"`: Returns all attributes.
133
+ - `"source"`: Returns the XML source for the element and its subtree.
134
+ - **Enhanced Type Names**:
135
+ - `ControlType` returns a human-readable short name (e.g., `Button`, `ListItem`, `Text`) instead of the UIA constant.
136
+
137
+ ```python
138
+ # List specific attribute
139
+ value = element.get_attribute("Value.Value")
140
+
141
+ # Get all attributes as a JSON string
142
+ all_attributes = element.get_attribute("all")
143
+
144
+ # Get XML source for this element and its subtree
145
+ element_source = element.get_attribute("source")
146
+ ```
147
+
148
+ ### PowerShell Execution
149
+ Execute internal PowerShell scripts or commands directly from your test. This requires the `power_shell` insecure feature to be enabled on the Appium server.
150
+
151
+ It is possible to execute a single PowerShell command or a whole script. Note that `powerShell` is case-insensitive.
152
+
153
+ ```python
154
+ # Execute a command string
155
+ driver.execute_script('powerShell', {'command': 'Get-Process Notepad'})
156
+
157
+ # Execute a script string
158
+ driver.execute_script('powerShell', {'script': '$p = Get-Process Notepad; $p.Kill();'})
159
+
160
+ # Shorthand (executes as command/script depending on context)
161
+ driver.execute_script('powerShell', 'Get-Process')
162
+ ```
163
+
164
+ ### Enhanced Text Input
165
+ You can specify the delay directly within the text string using the `[delay:ms]` pattern. This overrides the session setting (set via `windows: typeDelay` or `typeDelay` capability) for that specific action.
166
+
167
+ ```python
168
+ driver.find_element(...).send_keys("[delay:500]Slow text")
169
+ ```
170
+
171
+ ---
172
+
173
+ ## 🛠 Platform-Specific Extensions
174
+
175
+ All extensions are invoked via `driver.executeScript("windows: <methodName>", ...args)`.
176
+ Below are the detailed descriptions and arguments for each command.
177
+
178
+ > **Note**
179
+ > In most cases, commands can be used more intuitively by passing the element as the first argument (if required) and other parameters subsequently.
180
+
181
+ ### Mouse & Pointer
182
+
183
+ #### `windows: click`
184
+ This is a shortcut for a single mouse click gesture.
185
+
186
+ | Name | Type | Required | Description | Example |
187
+ | :--- | :--- | :--- | :--- | :--- |
188
+ | `elementId` | `string` | no | Hexadecimal identifier of the element to click on. If this parameter and coordinates are missing, the command uses the current cursor position. If `elementId` is present without coordinates, it clicks the center of the element. | `123e4567-e89b...` |
189
+ | `x` | `number` | no | Integer horizontal coordinate of the click point. If omitted (along with `elementId`), the current cursor position is used. If `elementId` is present, `x` is an offset from the element's top-left corner. | `100` |
190
+ | `y` | `number` | no | Integer vertical coordinate of the click point. If omitted (along with `elementId`), the current cursor position is used. If `elementId` is present, `y` is an offset from the element's top-left corner. | `100` |
191
+ | `button` | `string` | no | Name of the mouse button to be clicked. Supported button names are: `left`, `middle`, `right`, `back`, `forward`. The default value is `left`. | `right` |
192
+ | `modifierKeys` | `string[]` \| `string` | no | List of possible keys or a single key name to depress while the click is being performed. Supported key names are: `Shift`, `Ctrl`, `Alt`, `Win`. | `['ctrl', 'alt']` |
193
+ | `durationMs` | `number` | no | The number of milliseconds to wait between pressing and releasing the mouse button. By default no delay is applied. | `500` |
194
+ | `times` | `number` | no | How many times the click must be performed. One by default. | `2` |
195
+ | `interClickDelayMs` | `number` | no | Duration of the pause between each click gesture. Only makes sense if `times` is greater than one. 100ms by default. | `10` |
196
+
197
+ #### Usage
198
+
199
+ **Scenario 1: Using Element ID (Clicks Center)**
200
+ ```python
201
+ driver.execute_script('windows: click', {
202
+ 'elementId': element.id,
203
+ 'button': 'right',
204
+ 'times': 2
205
+ })
206
+ ```
207
+
208
+ **Scenario 2: Using Absolute Coordinates**
209
+ ```python
210
+ driver.execute_script('windows: click', {
211
+ 'x': 500,
212
+ 'y': 300,
213
+ 'button': 'left'
214
+ })
215
+ ```
216
+
217
+ **Scenario 3: Using Element ID with Offset (Relative to Top-Left)**
218
+ ```python
219
+ driver.execute_script('windows: click', {
220
+ 'elementId': element.id,
221
+ 'x': 10, # 10px from the left edge of the element
222
+ 'y': 10 # 10px from the top edge of the element
223
+ })
224
+ ```
225
+
226
+ #### `windows: clickAndDrag`
227
+ Performs a click-and-drag gesture.
228
+
229
+ | Name | Type | Required | Description | Example |
230
+ | :--- | :--- | :--- | :--- | :--- |
231
+ | `startElementId` | `string` | no | Hexadecimal identifier of the start element. If this and start coordinates are missing, the current cursor position is used. | `123e4567-e89b...` |
232
+ | `startX` | `number` | no | Horizontal coordinate of the start point. If omitted (along with `startElementId`), the current cursor position is used. | `100` |
233
+ | `startY` | `number` | no | Vertical coordinate of the start point. If omitted (along with `startElementId`), the current cursor position is used. | `100` |
234
+ | `endElementId` | `string` | no | Hexadecimal identifier of the end element. If this and end coordinates are missing, the current cursor position is used. | `123e4567-e89b...` |
235
+ | `endX` | `number` | no | Horizontal coordinate of the end point. If omitted (along with `endElementId`), the current cursor position is used. | `200` |
236
+ | `endY` | `number` | no | Vertical coordinate of the end point. If omitted (along with `endElementId`), the current cursor position is used. | `200` |
237
+ | `durationMs` | `number` | no | Duration of the drag in milliseconds. Default is `1000`. | `2000` |
238
+ | `button` | `string` | no | Mouse button to use (`left`, `right`, `middle`). Default is `left`. | `right` |
239
+ | `smoothPointerMove` | `string` | no | CSS-like easing function (e.g., `linear`, `ease-out`) for smooth movement. Default is `None`. | `linear` |
240
+ | `modifierKeys` | `string[]` \| `string` | no | Keys to hold during the drag. | `shift` |
241
+
242
+ #### Usage
243
+
244
+ **Scenario 1: Element to Element (Center to Center)**
245
+ ```python
246
+ driver.execute_script('windows: clickAndDrag', {
247
+ 'startElementId': element1.id,
248
+ 'endElementId': element2.id,
249
+ 'durationMs': 2000
250
+ })
251
+ ```
252
+
253
+ **Scenario 2: Absolute Coordinates**
254
+ ```python
255
+ driver.execute_script('windows: clickAndDrag', {
256
+ 'startX': 100,
257
+ 'startY': 100,
258
+ 'endX': 500,
259
+ 'endY': 500,
260
+ 'smoothPointerMove': 'linear'
261
+ })
262
+ ```
263
+
264
+ **Scenario 3: Element with Offset (Drag from specific point inside element)**
265
+ ```python
266
+ driver.execute_script('windows: clickAndDrag', {
267
+ 'startElementId': element1.id,
268
+ 'startX': 10, # Start 10px from left of element1
269
+ 'startY': 10, # Start 10px from top of element1
270
+ 'endElementId': element2.id,
271
+ 'endX': 50, # End 50px from left of element2
272
+ 'endY': 50 # End 50px from top of element2
273
+ })
274
+ ```
275
+
276
+ #### `windows: scroll`
277
+ This is a shortcut for a mouse wheel scroll gesture. The API is a thin wrapper over the SendInput WinApi call.
278
+
279
+ | Name | Type | Required | Description | Example |
280
+ | :--- | :--- | :--- | :--- | :--- |
281
+ | `elementId` | `string` | no | Same as in `windows: click`. Falls back to the current cursor position if omitted along with coordinates. | `123e4567-e89b...` |
282
+ | `x` | `number` | no | Same as in `windows: click`. Falls back to the current cursor position if omitted along with `elementId`. | `100` |
283
+ | `y` | `number` | no | Same as in `windows: click`. Falls back to the current cursor position if omitted along with `elementId`. | `100` |
284
+ | `deltaX` | `number` | no | The amount of horizontal wheel movement measured in wheel clicks. Positive = right, Negative = left. | `-5` |
285
+ | `deltaY` | `number` | no | The amount of vertical wheel movement. Positive = forward (away), Negative = backward (toward). | `5` |
286
+ | `modifierKeys` | `string[]` \| `string` | no | Same as in `windows: click`. | `win` |
287
+
288
+ #### Usage
289
+ ```python
290
+ driver.execute_script('windows: scroll', {
291
+ 'elementId': element.id,
292
+ 'deltaY': -120, # Scroll down 3 lines
293
+ 'modifierKeys': 'shift'
294
+ })
295
+ ```
296
+
297
+ #### `windows: hover`
298
+ This is a shortcut for a hover gesture.
299
+
300
+ | Name | Type | Required | Description | Example |
301
+ | :--- | :--- | :--- | :--- | :--- |
302
+ | `startElementId` | `string` | no | Same as in `windows: click`. Falls back to current cursor position if omitted along with coordinates. | `123e4567-e89b...` |
303
+ | `startX` | `number` | no | Same as in `windows: click`. Falls back to current cursor position if omitted along with `startElementId`. | `100` |
304
+ | `startY` | `number` | no | Same as in `windows: click`. Falls back to current cursor position if omitted along with `startElementId`. | `100` |
305
+ | `endElementId` | `string` | no | Same as in `windows: click`. | `123e4567-e89b...` |
306
+ | `endX` | `number` | no | Same as in `windows: click`. | `200` |
307
+ | `endY` | `number` | no | Same as in `windows: click`. | `200` |
308
+ | `modifierKeys` | `string[]` \| `string` | no | Same as in `windows: click`. | `shift` |
309
+ | `durationMs` | `number` | no | The number of milliseconds between moving the cursor from the starting to the ending hover point. 500ms by default. | `700` |
310
+
311
+ #### Usage
312
+
313
+ **Scenario 1: Element to Element (Center to Center)**
314
+ ```python
315
+ driver.execute_script('windows: hover', {
316
+ 'startElementId': element1.id,
317
+ 'endElementId': element2.id,
318
+ 'durationMs': 1000
319
+ })
320
+ ```
321
+
322
+ **Scenario 2: Absolute Coordinates**
323
+ ```python
324
+ driver.execute_script('windows: hover', {
325
+ 'startX': 100,
326
+ 'startY': 100,
327
+ 'endX': 500,
328
+ 'endY': 500
329
+ })
330
+ ```
331
+
332
+ **Scenario 3: Element with Offset (Hover specific point)**
333
+ ```python
334
+ driver.execute_script('windows: hover', {
335
+ 'startElementId': element1.id,
336
+ 'startX': 5,
337
+ 'startY': 5,
338
+ 'endElementId': element2.id,
339
+ 'endX': 5,
340
+ 'endY': 5
341
+ })
342
+ ```
343
+
344
+ ### Keyboard
345
+
346
+ #### `windows: typeDelay`
347
+ Sets the delay between key injections in milliseconds. This persistent setting applies to the entire session until changed.
348
+
349
+ | Name | Type | Required | Description | Example |
350
+ | :--- | :--- | :--- | :--- | :--- |
351
+ | `delay` | `number` | yes | Delay in milliseconds. | `100` |
352
+
353
+ #### Usage
354
+ ```python
355
+ driver.execute_script('windows: typeDelay', {'delay': 500})
356
+ # or shorthand
357
+ driver.execute_script('windows: typeDelay', '500')
358
+ ```
359
+
360
+ #### `windows: keys`
361
+ This is a shortcut for a customized keyboard input. Selenium keys should also work as modifier keys.
362
+
363
+ | Name | Type | Required | Description | Example |
364
+ | :--- | :--- | :--- | :--- | :--- |
365
+ | `actions` | `KeyAction[]` \| `KeyAction` | yes | One or more KeyAction dictionaries. | `[{'virtualKeyCode': 0x10, 'down': true}]` |
366
+ | `forceUnicode` | `boolean` | no | Forces the characters to be sent as unicode characters. | `true` |
367
+
368
+ ##### KeyAction Dictionary
369
+
370
+ | Name | Type | Required | Description | Example |
371
+ | :--- | :--- | :--- | :--- | :--- |
372
+ | `pause` | `number` | no | Allows to set a delay in milliseconds between key input series. | `100` |
373
+ | `text` | `string` | no | Non-empty string of Unicode text to type. | `Hello` |
374
+ | `virtualKeyCode` | `number` | no | Valid virtual key code. | `0x10` |
375
+ | `down` | `boolean` | no | If set to `true` then the corresponding key will be depressed, `false` - released. | `true` |
376
+
377
+ #### Usage
378
+ ```python
379
+ driver.execute_script('windows: keys', {
380
+ 'actions': [
381
+ {'virtualKeyCode': 0x10, 'down': True}, # Shift Down
382
+ {'text': 'Hello World'},
383
+ {'virtualKeyCode': 0x10, 'down': False} # Shift Up
384
+ ]
385
+ })
386
+ ```
387
+
388
+ ### System & State
389
+
390
+ #### `windows: setClipboard`
391
+ Sets Windows clipboard content to the given text or a PNG image.
392
+
393
+ | Name | Type | Required | Description | Example |
394
+ | :--- | :--- | :--- | :--- | :--- |
395
+ | `b64Content` | `string` | yes | Base64-encoded content of the clipboard to be set. | `QXBwaXVt` |
396
+ | `contentType` | `string` | no | Set to `plaintext` (default) or `image`. | `image` |
397
+
398
+ #### Usage
399
+ ```python
400
+ driver.execute_script('windows: setClipboard', {
401
+ 'b64Content': 'SGVsbG8=', # "Hello" in Base64
402
+ 'contentType': 'plaintext'
403
+ })
404
+ ```
405
+
406
+ #### `windows: getClipboard`
407
+ Retrieves Windows clipboard content.
408
+
409
+ | Name | Type | Required | Description | Example |
410
+ | :--- | :--- | :--- | :--- | :--- |
411
+ | `contentType` | `string` | no | Set to `plaintext` (default) or `image`. | `image` |
412
+
413
+ #### Usage
414
+ ```python
415
+ content = driver.execute_script('windows: getClipboard', {
416
+ 'contentType': 'plaintext'
417
+ })
418
+ print(content)
419
+ ```
420
+
421
+ #### `windows: pushCacheRequest`
422
+ This is an asynchronous function that sends cache requests based on specific conditions.
423
+
424
+ | Name | Type | Required | Description | Example |
425
+ | :--- | :--- | :--- | :--- | :--- |
426
+ | `treeFilter` | `string` | yes | Defines the filter that is applied when walking the automation tree. | `RawView` |
427
+ | `treeScope` | `string` | no | Defines the scope of the automation tree to be cached. | `SubTree` |
428
+ | `automationElementMode` | `string` | no | Specifies the mode of automation element (e.g., `None`, `Full`). | `Full` |
429
+
430
+ #### Usage
431
+ ```python
432
+ driver.execute_script('windows: pushCacheRequest', {
433
+ 'treeFilter': 'RawView',
434
+ 'treeScope': 'SubTree'
435
+ })
436
+ ```
437
+
438
+ ### Element Operations
439
+
440
+ #### `windows: invoke`
441
+ Invokes a UI element pattern, simulating an interaction like clicking or activating the element.
442
+
443
+ | Position | Type | Description | Example |
444
+ | :--- | :--- | :--- | :--- |
445
+ | 1 | `Element` | The UI element on which the `InvokePattern` is called. | `element` |
446
+
447
+ #### Usage
448
+ ```python
449
+ driver.execute_script('windows: invoke', element)
450
+ ```
451
+
452
+ #### `windows: expand`
453
+ Expands a UI element that supports the `ExpandPattern`.
454
+
455
+ | Position | Type | Description | Example |
456
+ | :--- | :--- | :--- | :--- |
457
+ | 1 | `Element` | The UI element to expand. | `element` |
458
+
459
+ #### Usage
460
+ ```python
461
+ driver.execute_script('windows: expand', element)
462
+ ```
463
+
464
+ #### `windows: collapse`
465
+ Collapses a UI element that supports the `CollapsePattern`.
466
+
467
+ | Position | Type | Description | Example |
468
+ | :--- | :--- | :--- | :--- |
469
+ | 1 | `Element` | The UI element to collapse. | `element` |
470
+
471
+ #### Usage
472
+ ```python
473
+ driver.execute_script('windows: collapse', element)
474
+ ```
475
+
476
+ #### `windows: setValue`
477
+ Sets the value of a UI element using the `ValuePattern`.
478
+
479
+ | Position | Type | Description | Example |
480
+ | :--- | :--- | :--- | :--- |
481
+ | 1 | `Element` | The UI element whose value will be set. | `element` |
482
+ | 2 | `string` | The value to be set. | `"new value"` |
483
+
484
+ #### Usage
485
+ ```python
486
+ driver.execute_script('windows: setValue', element, 'New Value')
487
+ ```
488
+
489
+ #### `windows: getValue`
490
+ Gets the current value of a UI element that supports the `ValuePattern`.
491
+
492
+ | Position | Type | Description | Example |
493
+ | :--- | :--- | :--- | :--- |
494
+ | 1 | `Element` | The UI element from which to retrieve the value. | `element` |
495
+
496
+ #### Usage
497
+ ```python
498
+ value = driver.execute_script('windows: getValue', element)
499
+ ```
500
+
501
+ #### `windows: scrollIntoView`
502
+ Scrolls the UI element into view using the `ScrollItemPattern`.
503
+
504
+ | Position | Type | Description | Example |
505
+ | :--- | :--- | :--- | :--- |
506
+ | 1 | `Element` | The UI element to bring into view. | `element` |
507
+
508
+ #### Usage
509
+ ```python
510
+ driver.execute_script('windows: scrollIntoView', element)
511
+ ```
512
+
513
+ > **Note**
514
+ > You can also use the standard JavaScript way:
515
+ > ```python
516
+ > driver.execute_script('arguments[0].scrollIntoView()', element)
517
+ > ```
518
+
519
+ #### `windows: toggle`
520
+ Toggles a UI element’s state using the `TogglePattern`.
521
+
522
+ | Position | Type | Description | Example |
523
+ | :--- | :--- | :--- | :--- |
524
+ | 1 | `Element` | The UI element to toggle. | `element` |
525
+
526
+ #### Usage
527
+ ```python
528
+ driver.execute_script('windows: toggle', element)
529
+ ```
530
+
531
+ ### Selection Management
532
+
533
+ #### `windows: select`
534
+ Selects a UI element using the `SelectionPattern`.
535
+
536
+ | Position | Type | Description | Example |
537
+ | :--- | :--- | :--- | :--- |
538
+ | 1 | `Element` | The UI element to select. | `element` |
539
+
540
+ #### Usage
541
+ ```python
542
+ driver.execute_script('windows: select', element)
543
+ ```
544
+
545
+ #### `windows: addToSelection`
546
+ Adds an element to the current selection on a UI element that supports the `SelectionPattern`.
547
+
548
+ | Position | Type | Description | Example |
549
+ | :--- | :--- | :--- | :--- |
550
+ | 1 | `Element` | The UI element to add to the selection. | `element` |
551
+
552
+ #### Usage
553
+ ```python
554
+ driver.execute_script('windows: addToSelection', element)
555
+ ```
556
+
557
+ #### `windows: removeFromSelection`
558
+ Removes an element from the current selection on a UI element that supports the `SelectionPattern`.
559
+
560
+ | Position | Type | Description | Example |
561
+ | :--- | :--- | :--- | :--- |
562
+ | 1 | `Element` | The UI element to remove from the selection. | `element` |
563
+
564
+ #### Usage
565
+ ```python
566
+ driver.execute_script('windows: removeFromSelection', element)
567
+ ```
568
+
569
+ #### `windows: isMultiple`
570
+ Checks if a UI element supports multiple selection using the `SelectionPattern`.
571
+
572
+ | Position | Type | Description | Example |
573
+ | :--- | :--- | :--- | :--- |
574
+ | 1 | `Element` | The UI element to check. | `element` |
575
+
576
+ #### Usage
577
+ ```python
578
+ is_multi = driver.execute_script('windows: isMultiple', element)
579
+ ```
580
+
581
+ #### `windows: selectedItem`
582
+ Gets the selected item from a UI element that supports the `SelectionPattern`.
583
+
584
+ | Position | Type | Description | Example |
585
+ | :--- | :--- | :--- | :--- |
586
+ | 1 | `Element` | The UI element from which to retrieve the selected item. | `element` |
587
+
588
+ #### Usage
589
+ ```python
590
+ selected_el = driver.execute_script('windows: selectedItem', element)
591
+ ```
592
+
593
+ #### `windows: allSelectedItems`
594
+ Gets all selected items from a UI element that supports the `SelectionPattern`.
595
+
596
+ | Position | Type | Description | Example |
597
+ | :--- | :--- | :--- | :--- |
598
+ | 1 | `Element` | The UI element from which to retrieve all selected items. | `element` |
599
+
600
+ #### Usage
601
+ ```python
602
+ selected_els = driver.execute_script('windows: allSelectedItems', element)
603
+ ```
604
+
605
+ ### Window Management
606
+
607
+ #### `windows: maximize`
608
+ Maximizes a window or UI element using the `WindowPattern`.
609
+
610
+ | Position | Type | Description | Example |
611
+ | :--- | :--- | :--- | :--- |
612
+ | 1 | `Element` | The window or UI element to maximize. | `element` |
613
+
614
+ #### Usage
615
+ ```python
616
+ driver.execute_script('windows: maximize', element)
617
+ ```
618
+
619
+ #### `windows: minimize`
620
+ Minimizes a window or UI element using the `WindowPattern`.
621
+
622
+ | Position | Type | Description | Example |
623
+ | :--- | :--- | :--- | :--- |
624
+ | 1 | `Element` | The window or UI element to minimize. | `element` |
625
+
626
+ #### Usage
627
+ ```python
628
+ driver.execute_script('windows: minimize', element)
629
+ ```
630
+
631
+ #### `windows: restore`
632
+ Restores a window or UI element to its normal state using the `WindowPattern`.
633
+
634
+ | Position | Type | Description | Example |
635
+ | :--- | :--- | :--- | :--- |
636
+ | 1 | `Element` | The window or UI element to restore. | `element` |
637
+
638
+ #### Usage
639
+ ```python
640
+ driver.execute_script('windows: restore', element)
641
+ ```
642
+
643
+ #### `windows: close`
644
+ Closes a window or UI element using the `WindowPattern`.
645
+
646
+ | Position | Type | Description | Example |
647
+ | :--- | :--- | :--- | :--- |
648
+ | 1 | `Element` | The window or UI element to close. | `element` |
649
+
650
+ #### Usage
651
+ ```python
652
+ driver.execute_script('windows: close', element)
653
+ ```
654
+
655
+ #### `windows: setProcessForeground`
656
+ Brings the main window of the specified process to the foreground.
657
+
658
+ | Name | Type | Required | Description | Example |
659
+ | :--- | :--- | :--- | :--- | :--- |
660
+ | `process` | `string` | yes | The name of the process whose window should be brought to the foreground. | `notepad.exe` |
661
+
662
+ #### Usage
663
+ ```python
664
+ driver.execute_script('windows: setProcessForeground', {
665
+ 'process': 'notepad.exe'
666
+ })
667
+ ```
668
+
669
+ #### `windows: setFocus`
670
+ Sets focus to the specified UI element using UIAutomationElement's `SetFocus` method.
671
+
672
+ | Position | Type | Description | Example |
673
+ | :--- | :--- | :--- | :--- |
674
+ | 1 | `Element` | The UI element to set focus on. | `element` |
675
+
676
+ #### Usage
677
+ ```python
678
+ driver.execute_script('windows: setFocus', element)
679
+ ```
680
+
681
+ ### Screen Recording
682
+
683
+ #### `windows: startRecordingScreen`
684
+ Starts screen recording using ffmpeg.
685
+
686
+ | Name | Type | Required | Description | Example |
687
+ | :--- | :--- | :--- | :--- | :--- |
688
+ | `fps` | `number` | no | Frames per second. Default is 15. | `15` |
689
+ | `timeLimit` | `number` | no | Recording time limit in seconds. Default is 600. | `60` |
690
+ | `preset` | `string` | no | ffmpeg preset. Default is `veryfast`. | `ultrafast` |
691
+ | `captureCursor` | `boolean` | no | Whether to capture mouse cursor. | `true` |
692
+ | `captureClicks` | `boolean` | no | Whether to capture mouse clicks. | `true` |
693
+
694
+ > [!IMPORTANT]
695
+ > This feature requires the optional recording stack to be installed separately:
696
+ > `npm i -g asyncbox teen_process ffmpeg-static`
697
+ > Recording is currently **not supported** on ARM architecture (no ARM build of `ffmpeg-static`).
698
+
699
+ #### `windows: stopRecordingScreen`
700
+ Stops screen recording and returns the video as base64 string.
701
+
702
+ | Name | Type | Required | Description | Example |
703
+ | :--- | :--- | :--- | :--- | :--- |
704
+ | `remotePath` | `string` | no | Remote path to upload the video to. | `http://example.com/upload` |
705
+
706
+ ---
707
+
708
+ ## 🛠 Development
709
+
710
+ Recommended VS Code plugin: [Comment tagged templates](https://marketplace.visualstudio.com/items?itemName=bierner.comment-tagged-templates) for syntax highlighting.
711
+
712
+ ```bash
713
+ npm install # Setup dependencies
714
+ npm run lint # Code quality check
715
+ npm run build # Transpile TypeScript to JS
716
+ ```