@desklink/axi 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md ADDED
@@ -0,0 +1,119 @@
1
+ <h1 align="center">@desklink/axi</h1>
2
+
3
+ <p align="center">
4
+ <a href="https://www.npmjs.com/package/@desklink/axi"><img alt="npm" src="https://img.shields.io/npm/v/@desklink/axi?style=flat" /></a>
5
+ <a href="https://github.com/umeranjum17/desklink/actions/workflows/ci.yml"><img alt="CI" src="https://img.shields.io/github/actions/workflow/status/umeranjum17/desklink/ci.yml?style=flat&branch=main" /></a>
6
+ <a href="https://github.com/umeranjum17/desklink/blob/main/LICENSE"><img alt="Apache 2.0" src="https://img.shields.io/badge/license-Apache--2.0-666?style=flat" /></a>
7
+ <img alt="Linux and macOS" src="https://img.shields.io/badge/Linux%20%7C%20macOS-111?style=flat" />
8
+ </p>
9
+
10
+ <p align="center">
11
+ <strong>See and drive a live desktop from an agent: text first, pixels on request.</strong><br/>
12
+ A desktop CLI for coding agents on the desklink engine. It answers with OCR text, refs to click and changed regions, and returns an image crop only when asked, so a turn costs text tokens rather than screenshots.
13
+ </p>
14
+
15
+ ## Install
16
+
17
+ Install from npm; the badge above tracks the registry version:
18
+
19
+ ```sh
20
+ npm install -g @desklink/axi
21
+ ```
22
+
23
+ Or build from the repository root with `npm install && npm run build`.
24
+ The engine is resolved by `@desklink/host`; see its [installation guide](../desktop-host/README.md#install-and-run)
25
+ for supported platforms and [macOS setup](../desktop-host/README.md#macos).
26
+ Set `DESKLINK_AXI_ENGINE` for a source build.
27
+
28
+ A Linux/macOS desktop AXI. It runs a private bridge per session, captures read-only frames through `@desklink/host` protocol v3, and sends input only through a WebRTC `control` channel. The bridge's Unix socket lives in `$XDG_RUNTIME_DIR/desklink-axi` (or the OS temp directory), with `DESKLINK_AXI_SESSION` selecting an isolated socket. It opens a separate engine session; don't drive the desktop while someone else is controlling it.
29
+
30
+ The engine is resolved by `@desklink/host`; set `DESKLINK_AXI_ENGINE` to an executable built from source if no packaged binary is installed. On macOS use `PATH=/opt/homebrew/bin:$PATH`, Rust >= 1.97.1, delete copied `*.tsbuildinfo` before rebuilding a moved worktree, and `DESKLINK_VPX_STATIC_DIR=/opt/homebrew` (the install **prefix**, not `/opt/homebrew/lib`). For macOS start the bridge from a stable signed app, not `desklink-axi start` in Terminal/SSH: `open -n /absolute/DesklinkHost.app --args axi-bridge /absolute/path/to/packages/axi/bin/desklink-axi.js --control`. Install or update that app using `packages/desktop-host/release/install-mac-app.sh`; grant Screen Recording and Accessibility to it once. An SSH-launched Node helper does not inherit those grants. OCR needs the optional `tesseract` executable (`brew install tesseract`, `sudo pacman -S tesseract` or `sudo apt install tesseract-ocr`); on macOS AXI also finds Homebrew's standard install paths when LaunchServices omits them from PATH. `look` works without it.
31
+
32
+ ```
33
+ desklink-axi start --control --source x11 --display :97
34
+ desklink-axi screen --query "Budget"
35
+ desklink-axi tree --query "Save"
36
+ desklink-axi type "hello" --into @a3.10
37
+ desklink-axi click @a3.7
38
+ desklink-axi marks
39
+ desklink-axi click 100,120
40
+ desklink-axi type "hello"
41
+ desklink-axi batch '[[{"verb":"snapshot","name":"ui"},{"verb":"click","args":["@ui.2"]},{"verb":"assert","args":["Saved"]}]]'
42
+ desklink-axi diff
43
+ desklink-axi look @r1
44
+ desklink-axi stop
45
+ ```
46
+
47
+ `health` reports capture state, latest frame sequence, and a stream-stop reason without OCR. `start` defaults to view-only; Linux uses the portal unless `--source x11 --display :N` is given, and macOS uses the main display unless `--source display --display <id>` is given. The portal can require a desktop consent prompt; its single-use restore token is kept in `$XDG_STATE_HOME/desklink-axi/portal-token` (otherwise `~/.local/state/desklink-axi/portal-token`), atomically replaced with mode 0600 after each grant. Set `DESKLINK_AXI_RESTORE_TOKEN_FILE` to use a different path. macOS Screen Recording and Accessibility belong to the app that launches the bridge and its engine child. Do not run it unattended against someone else's display. Input and clipboard read/write require `start --control`; view-only clipboard calls return a structured error naming that flag. `stop` releases held input. Actions acknowledge `input: applied` immediately, without waiting for a frame. Every action (`click`, `type`, `press`, `scroll`, `drag`) accepts `--wait settle|change|none|<milliseconds>`; frame timeout after applied input returns `input: applied; frame: timed out` with success status. `batch` takes a JSON array of 1–30 `[verb, ...args]` steps (click/type/press/scroll/wait/snapshot/assert/tree/marks), runs them in order in one socket round trip, stops on the first failed step and returns a compact per-step result. OCR refs survive advancing frame numbers only while their text and position still match a fresh crop; moved or ambiguous text requires `screen --query`. `screen --query` reuses a prior matching text location for a targeted crop before falling back to full OCR, and reports estimated text tokens. Start with `screen --query` or `--region` before `--full`. For web pages prefer DOM/browser automation when available; AXI is for native apps or whole-desktop tasks. `wait change` and `wait settle` block on engine frame signals rather than polling. `diff` reports repeatedly changing regions separately as `animating`; `diff --include-animating` includes them among ordinary changes. `look` writes a PNG crop. `--help` lists the commands and their flags. Errors are structured and the CLI does not prompt.
48
+ ## Browser lane (opt-in, loopback-only)
49
+
50
+ For web pages the browser lane gives semantic control through [playwright-core](https://www.npmjs.com/package/playwright-core) over CDP; the desktop lane stays for native apps and whole-desktop tasks. Nothing is attached implicitly:
51
+
52
+ ```
53
+ desklink-axi browser attach --cdp 127.0.0.1:9222 # a Chromium YOU started with --remote-debugging-port (loopback only)
54
+ desklink-axi browser launch --profile /tmp/task-profile # or: isolated task-owned Chromium, debug port on loopback only
55
+ desklink-axi browser tabs # 'selected' = this lane's page, 'front' = the browser's actual activated tab
56
+ desklink-axi browser snapshot # compact accessibility snapshot with [ref=eN] elements; no screenshots
57
+ desklink-axi browser click @e12 / fill @e5 "text" / upload @e9 /tmp/report.pdf / press Enter
58
+ desklink-axi browser select 1 / open <url> / navigate <url> / close 1 / detach
59
+ ```
60
+
61
+ Non-loopback endpoints are refused; `launch` renders on `$DISPLAY` (X11) so it never follows `WAYLAND_DISPLAY` to another desktop, and `--arg <chromium flag>` (repeatable) passes extra flags such as `--arg=--window-size=1280,640`. Snapshot refs are re-derived on every command and rejected as `stale-ref` when the page changed. `upload` only sets files on `<input type=file>`; for OS-level dialogs it returns `not-supported` and names the desktop lane (`desklink-axi click/type`), which is also the route for real native-dialog navigation — a DOM upload is not equivalent to completing a native file dialog. `detach` stops the browser only if this lane launched it; a passed `--profile` directory is kept, a lane-created temp profile is removed.
62
+
63
+ ## Batch outcomes
64
+
65
+ `batch` steps are `[verb, ...args]` arrays or `{verb, args?, name?, wait?}` objects with verbs `click, type, press, scroll, wait, snapshot, assert, tree, marks`. A named snapshot step binds its OCR items so later steps can target `@name.n`; `assert` steps make success depend on observed screen text; per-step lines separate `input: applied` (transport acknowledgement) from `effect: change observed | none within timeout`, and the footer says `effects: asserted[n]` or `effects: unverified (input acknowledgements only)` — an input acknowledgement is never a verified outcome, so wrong-focus typing or a no-op click fails the batch instead of reporting success.
66
+
67
+ ## Proofs and measurements
68
+
69
+ From the repository root with a task-built engine (`DESKLINK_AXI_ENGINE`) on Linux:
70
+
71
+ - `node packages/axi/test/smoke.mjs` — clean-tree rebuild of host+axi, then `batch` and `click` on a private Xvfb; asserts no task-owned survivors.
72
+ - `node packages/axi/test/flow.mjs` — full desktop proof (frame/damage, input, chords, batch outcomes, injected-failure cleanup).
73
+ - `node packages/axi/test/browser-flow.mjs` — the open/form/two-tab fixture through the browser lane with semantic refs on a private Xvfb; also proves selected-vs-front, stale-ref rejection, DOM upload and the not-supported fallback.
74
+ - `node packages/axi/test/bench.mjs [runs]` — comparative p50/p95 (default 20 measured runs after 3 warmups) for the browser lane vs the AXI fallback over the open/form/tabs/native fixtures; text-token proxy is `ceil(stdout chars/4)`, a proxy, not a tokenizer.
75
+
76
+ Measured on one private 1280x720 Xvfb with a task-owned Chromium 151 (`--window-position=0,0 --window-size=1280,640`, 20 measured runs per task after 3 warmups, CLI wall time per run; Chromium for Testing 151.0.7922.34, playwright-core 1.62.1):
77
+
78
+ | task | lane | p50 ms | p95 ms | calls/run | retries | text-token proxy/run | failures |
79
+ |---|---|---|---|---|---|---|---|
80
+ | native typing | axi | 57 | 68 | 1.0 | 0 | 22 | 1/23 |
81
+ | open + verify heading | browser | 532 | 558 | 2.0 | 0 | 149 | 0/23 |
82
+ | open + verify heading | axi | 6043 | 8659 | 2.0 | 0 | 107 | 0/23 |
83
+ | form fill+save+verify | browser | 1735 | 6082 | 4.0 | 0 | 290 | 0/23 |
84
+ | two tabs read both | browser | 5176 | 6007 | 17.0 | 0 | 900 | 0/23 |
85
+ | form fill+save+verify | axi | not completed — OCR word-target steps (`--into Name`, `click Save`) are flaky in the bench sequence; the same steps pass in isolation and the batch/assert machinery is proven in flow.mjs. Follow-up: stabilize, then re-measure. |
86
+ | two tabs read both | axi | not completed — same OCR word-target limitation on the popup-link click; works in isolation, not yet in the measured sequence. |
87
+
88
+ The browser lane completed every web task 23/23 with compact text snapshots; the desktop lane's OCR fallback stays reliable for the native proof and degrades on small web text, matching the two-lane ranking in the 2026-09-27 computer-use benchmark.
89
+
90
+ For a remote signed Mac bridge, start it once **inside the permission-granted app**, then forward its Unix socket over one persistent SSH connection: `ssh -N -L "$LOCAL_RUNTIME/desklink-axi/default.sock:$REMOTE_RUNTIME/desklink-axi/default.sock" mac-host`. Set `XDG_RUNTIME_DIR=$LOCAL_RUNTIME` on the client (create `$LOCAL_RUNTIME/desklink-axi` mode 0700 first), and run `desklink-axi screen`, `batch`, etc. locally. The SSH login does not launch the engine, and the forwarded socket grants control: use a private directory and close SSH when done. Do not run local `start` against a forwarded socket. Never swap and re-sign the executable per run. Update the complete app at its existing path with the same key using `packages/desktop-host/release/install-mac-app.sh`; verify grants after updating.
91
+
92
+ `desklink-axi setup hooks` adds a Claude `SessionStart` hook to `.claude/settings.local.json` in the current directory; it does not start capture.
93
+
94
+ For an isolated integration proof run `DESKLINK_AXI_ENGINE=/path/to/engine CARGO_TARGET_DIR=/tmp/task-owned-target node packages/axi/test/flow.mjs` from the repository root. It launches and kills its own Xvfb process by exact PID. `test/a11y-flow.mjs` is the accessibility + marks proof: it also starts a private D-Bus session bus, a private AT-SPI bus, and a GTK3 fixture app (all task-owned, private `XDG_RUNTIME_DIR`), asserts semantic-ref driving of form controls and a file dialog, and prints per-command p50/p95 with `--measure 20`. It skips itself when `busctl`/`dbus-daemon`/`at-spi2-core`/GTK are absent and never touches the user's accessibility bus.
95
+
96
+ To check the CLI against the AXI spec, run `axi-axi validate "node bin/desklink-axi.js" --dir .` from `packages/axi`.
97
+
98
+ Measured on a private 1280×720 Xvfb client: `diff → click 100,100 → look @r1 → type abc` returned 79 + 79 + 39 + 30 = **227 text tokens** (`@anthropic-ai/tokenizer` 0.0.4), plus ~5 tokens for the 64×64 crop. Four full screenshots would cost ~4×1,229 = **4,916 image tokens** under the same published width×height/750 estimate (~21× difference). This is a short synthetic task, not a production-workload benchmark; the tokenizer is legacy, so absolute counts are approximate.
99
+
100
+ ## Accessibility tree (`tree`) — Linux AT-SPI
101
+
102
+ `tree` reads the desktop's AT-SPI accessibility tree over D-Bus and returns bounded TOON rows of `ref,role,name,value,x,y,w,h,states`. It needs no `--control`; acting on refs does. Refs are generation-tagged (`@a<gen>.<n>`), stay usable across newer snapshots, and are revalidated at use time: a ref whose role, name, or screen extents changed — or whose object is gone — returns `stale-ref` instead of acting blind.
103
+
104
+ - Capability probe: the bus is resolved from `AT_SPI_BUS_ADDRESS`, else `org.a11y.Bus.GetAddress` on the session bus. Unavailable → structured `a11y-unavailable` error; every other command keeps working. macOS AX is **not implemented** and labeled as such.
105
+ - Values: text and value widgets expose their content in the `value` column (Text/Value interfaces), so a typed entry can be verified without pixels.
106
+ - States include `enabled`, `showing`, `focused`, `editable`, `checked`; acting on a disabled or hidden ref returns `not-actionable`.
107
+ - `click @a<gen>.<n>` performs the widget's own AT-SPI action (`input: applied (atspi action "Click")`); a node without an action interface falls back to a coordinate click at its center (`coordinate fallback`). `type "text" --into @a<n>` grabs AT-SPI focus first (`atspi focus`; coordinate fallback when the toolkit refuses). Raw screenshot/coordinate/OCR paths are unchanged.
108
+ - `tree --query <words>` filters by name/role/value; output is bounded at 40 rows unless `--full`; `--fields` selects columns. `tree` and `marks` are usable as `batch` steps.
109
+
110
+ Measured facts (private Xvfb + private bus, September 2026; `test/a11y-flow.mjs`):
111
+
112
+ - GTK3 (3.24, atk-bridge): full exposure. Named roles, bounds, states, values, and actions all work over D-Bus; semantic clicks carry real effects (fixture-verified file export). While a `GtkDialog.run()` modal loop is up, atk-bridge answers **no** AT-SPI method calls at all (measured 25 s timeouts) — dialogs must be read before they are opened or shown non-modally.
113
+ - Qt6 (6.11, QtAtSpi): registers the application on the a11y bus with duplicated registry entries (three roots for one app, two sharing one bus name), and its named form controls are visible to the walk (3/3 in the fixture); some roots reject the `GetChildCount` method (property-only), which the walk tolerates. Treat per-toolkit role spellings ("push button" vs "button") and duplicated roots as normal input.
114
+ - Wayland/portal: the a11y bus is display-protocol independent, but exposure through the compositor's portal/permission paths was **not measured** in this task (no owned Wayland fixture); scope before relying on it.
115
+ - Measured over 20 warm runs (1280×720 Xvfb, GTK3 fixture, private bus): `tree --full` p50 472 ms / p95 573 ms, semantic `click` p50 69 ms / p95 85 ms, `type --into` p50 69 ms / p95 83 ms, `marks` p50 657 ms / p95 774 ms, 0 stale-ref retries across the whole suite. Each `tree`/`marks` walk is bounded (~100–400 bounded busctl spawns, parallelized per level).
116
+
117
+ ## Marks (`marks`) — numbered visual fallback
118
+
119
+ `marks` overlays numbered boxes on a crop for actionable candidates — AT-SPI nodes with actions plus OCR text — and returns both the image path and a compact `mark,ref,label,role` map. Pick a control from the numbered image, then act on its ref; no pixel guessing. The cost line reports the crop's dimensions and estimated image tokens (`dimensions=1280x720 cost: ~1229 tokens`). Refs in the map expire exactly like `tree` refs: changed geometry or content returns `stale-ref` on use. `--region x,y,w,h` scopes the crop; `--out <path>` chooses the artifact path (default writes next to the session). The overlay drawing is a pure buffer op (`src/marks.ts`); no bundled detector.
@@ -0,0 +1,2 @@
1
+ #!/usr/bin/env node
2
+ import(new URL("../dist/index.js", import.meta.url).href);
package/dist/atspi.js ADDED
@@ -0,0 +1,267 @@
1
+ // Linux AT-SPI accessibility client over `busctl` (systemd's D-Bus client,
2
+ // present wherever systemd is). Zero new npm dependencies; every call is one
3
+ // bounded process spawn against the user's accessibility bus.
4
+ //
5
+ // Bus resolution order (matching libatspi itself):
6
+ // 1. AT_SPI_BUS_ADDRESS environment variable (headless/test cages)
7
+ // 2. org.a11y.Bus.GetAddress on the session bus (normal desktops)
8
+ // The X11 _AT_SPI_BUS root-window property fallback is not implemented; on
9
+ // desktops without a session bus there is no AXI session anyway.
10
+ //
11
+ // macOS: the AX API route is NOT implemented. Capability probing reports it
12
+ // as explicitly unavailable so callers can degrade to OCR/coordinates.
13
+ import { execFile } from 'node:child_process';
14
+ import { appendFileSync } from 'node:fs';
15
+ function a11yDebug(line) {
16
+ const path = process.env.DESKLINK_AXI_A11Y_DEBUG;
17
+ if (path) {
18
+ try {
19
+ appendFileSync(path, line + '\n');
20
+ }
21
+ catch { /* debug sink only */ }
22
+ }
23
+ }
24
+ export class A11yError extends Error {
25
+ code;
26
+ constructor(code, message) {
27
+ super(message);
28
+ this.code = code;
29
+ }
30
+ }
31
+ export const MACOS_A11Y_NOTICE = 'a11y-unavailable: macOS AX route is not implemented (planned; Linux AT-SPI first)';
32
+ const REGISTRY = 'org.a11y.atspi.Registry';
33
+ const ROOT = '/org/a11y/atspi/accessible/root';
34
+ const MAX_NODES = 400;
35
+ const MAX_DEPTH = 12;
36
+ // ATSPI state enum bits (at-spi-constants.h), only the ones we surface.
37
+ const STATE_BITS = [
38
+ [1, 'active'], [4, 'checked'], [6, 'defunct'], [7, 'editable'], [8, 'enabled'],
39
+ [11, 'focusable'], [12, 'focused'], [24, 'sensitive'], [25, 'showing'],
40
+ [26, 'single-line'], [30, 'visible'], [33, 'required'], [41, 'checkable'],
41
+ ];
42
+ export function decodeStates(words) {
43
+ const set = [];
44
+ for (const [bit, name] of STATE_BITS) {
45
+ const word = bit < 32 ? words[0] ?? 0 : words[1] ?? 0;
46
+ if (word & (1 << bit % 32))
47
+ set.push(name);
48
+ }
49
+ return set;
50
+ }
51
+ export function isActionable(node) {
52
+ return node.interfaces.includes('Action') && node.states.includes('enabled') && node.states.includes('showing');
53
+ }
54
+ export const axRefPattern = /^@a(\d+)\.(\d+)$/;
55
+ // busctl --json wraps each out parameter as {type,data}; the whole reply is a
56
+ // one-element array of the tuple. decode() unwraps variants and nesting.
57
+ export function decodeReply(json) {
58
+ if (!json || typeof json.type !== 'string')
59
+ return json;
60
+ if (json.type === 'v') {
61
+ const inner = Array.isArray(json.data) ? json.data[0] : json.data;
62
+ return decodeReply(inner);
63
+ }
64
+ if (Array.isArray(json.data))
65
+ return json.data.map(decodeReply);
66
+ return json.data;
67
+ }
68
+ export function replyPayload(json) {
69
+ const decoded = decodeReply(json);
70
+ return Array.isArray(decoded) ? decoded[0] : decoded;
71
+ }
72
+ export function parseActions(reply) {
73
+ // Modern a(sss) name/description/keybinding; tolerate legacy (uss).
74
+ const rows = Array.isArray(reply) ? reply : [];
75
+ return rows.map(row => {
76
+ const tuple = Array.isArray(row) ? row : [];
77
+ return typeof tuple[0] === 'number' ? String(tuple[1]) : String(tuple[0] ?? '');
78
+ });
79
+ }
80
+ export class A11yClient {
81
+ env;
82
+ address;
83
+ nextGen = 1;
84
+ constructor(env = process.env) {
85
+ this.env = env;
86
+ }
87
+ busctl(args, address, timeoutMs = 5000) {
88
+ return new Promise((resolve, reject) => {
89
+ execFile('busctl', ['--address', address, '--json=short', ...args], { encoding: 'utf8', timeout: timeoutMs, maxBuffer: 1 << 24 }, (error, stdout, stderr) => {
90
+ if (error) {
91
+ const detail = String(stderr || error.message).trim().split('\n').at(-1) ?? 'busctl failed';
92
+ reject(new A11yError(classifyBusctlError(detail), detail));
93
+ return;
94
+ }
95
+ try {
96
+ resolve(JSON.parse(stdout));
97
+ }
98
+ catch {
99
+ reject(new A11yError('a11y-error', `unparsable busctl output: ${stdout.slice(0, 120)}`));
100
+ }
101
+ });
102
+ });
103
+ }
104
+ call(dest, path, iface, method, signature = '', ...args) {
105
+ return this.ready().then(address => this.busctl(['call', dest, path, iface, method, ...(signature ? [signature, ...args] : [])], address).then(replyPayload));
106
+ }
107
+ async ready() {
108
+ if (this.address)
109
+ return this.address;
110
+ if (process.platform === 'darwin')
111
+ throw new A11yError('a11y-unavailable', MACOS_A11Y_NOTICE.replace('a11y-unavailable: ', ''));
112
+ const direct = this.env.AT_SPI_BUS_ADDRESS;
113
+ if (direct) {
114
+ this.address = direct;
115
+ return direct;
116
+ }
117
+ const session = this.env.DBUS_SESSION_BUS_ADDRESS ?? (this.env.XDG_RUNTIME_DIR ? `unix:path=${this.env.XDG_RUNTIME_DIR}/bus` : '');
118
+ if (!session)
119
+ throw new A11yError('a11y-unavailable', 'no D-Bus session bus; accessibility tree unavailable');
120
+ try {
121
+ const reply = await this.busctl(['call', 'org.a11y.Bus', '/org/a11y/bus', 'org.a11y.Bus', 'GetAddress'], session);
122
+ const payload = replyPayload(reply);
123
+ if (typeof payload !== 'string' || !payload)
124
+ throw new Error('empty address');
125
+ this.address = payload;
126
+ return payload;
127
+ }
128
+ catch (error) {
129
+ throw new A11yError('a11y-unavailable', `accessibility bus unreachable: ${error.message}; install at-spi2-core (running desktop sessions usually provide it)`);
130
+ }
131
+ }
132
+ reset() { this.address = undefined; }
133
+ async nodeFacts(bus, path) {
134
+ const logged = (error) => {
135
+ a11yDebug(`${bus} ${path}: ${error.message}`);
136
+ throw error;
137
+ };
138
+ const [role, states, extents, interfaces] = await Promise.all([
139
+ this.call(bus, path, 'org.a11y.atspi.Accessible', 'GetRoleName').catch(logged).catch(() => 'unknown'),
140
+ this.call(bus, path, 'org.a11y.atspi.Accessible', 'GetState').catch(logged).catch(() => []),
141
+ this.call(bus, path, 'org.a11y.atspi.Component', 'GetExtents', 'u', '0').catch(logged).catch(() => []),
142
+ this.call(bus, path, 'org.a11y.atspi.Accessible', 'GetInterfaces').catch(logged).catch(() => []),
143
+ ]);
144
+ const name = await this.call(bus, path, 'org.freedesktop.DBus.Properties', 'Get', 'ss', 'org.a11y.atspi.Accessible', 'Name').catch(() => '');
145
+ return {
146
+ name: typeof name === 'string' ? name : '',
147
+ role: typeof role === 'string' ? role : 'unknown',
148
+ states: decodeStates(Array.isArray(states) ? states.filter(n => typeof n === 'number') : []),
149
+ box: Array.isArray(extents) && extents.every(n => typeof n === 'number') && extents.length === 4 ? extents : [0, 0, 0, 0],
150
+ interfaces: Array.isArray(interfaces) ? interfaces.map(name => String(name).replace('org.a11y.atspi.', '')) : [],
151
+ };
152
+ }
153
+ async readValue(node) {
154
+ try {
155
+ if (node.interfaces.includes('Text')) {
156
+ const count = await this.call(node.bus, node.path, 'org.freedesktop.DBus.Properties', 'Get', 'ss', 'org.a11y.atspi.Text', 'CharacterCount');
157
+ const n = typeof count === 'number' ? count : 0;
158
+ if (n <= 0)
159
+ return '';
160
+ const text = await this.call(node.bus, node.path, 'org.a11y.atspi.Text', 'GetText', 'ii', '0', String(Math.min(n, 400)));
161
+ return typeof text === 'string' ? text : undefined;
162
+ }
163
+ if (node.interfaces.includes('Value')) {
164
+ const value = await this.call(node.bus, node.path, 'org.freedesktop.DBus.Properties', 'Get', 'ss', 'org.a11y.atspi.Value', 'Value');
165
+ return typeof value === 'number' ? String(value) : undefined;
166
+ }
167
+ }
168
+ catch (error) {
169
+ a11yDebug(`value ${node.path}: ${error.message}`);
170
+ return undefined;
171
+ }
172
+ return undefined;
173
+ }
174
+ async snapshot() {
175
+ const gen = this.nextGen++;
176
+ const roots = await this.call(REGISTRY, ROOT, 'org.a11y.atspi.Accessible', 'GetChildren');
177
+ const nodes = [];
178
+ const apps = [];
179
+ for (const [bus, path] of Array.isArray(roots) ? roots : []) {
180
+ apps.push(String(bus));
181
+ let frontier = [[String(bus), String(path), 0]];
182
+ while (frontier.length && nodes.length < MAX_NODES) {
183
+ const facts = await Promise.all(frontier.map(([b, p]) => this.nodeFacts(b, p).catch(() => undefined)));
184
+ const next = [];
185
+ for (let i = 0; i < frontier.length && nodes.length < MAX_NODES; i++) {
186
+ const [busName, nodePath, depth] = frontier[i];
187
+ const fact = facts[i];
188
+ if (!fact || fact.states.includes('defunct'))
189
+ continue;
190
+ const node = {
191
+ ref: `@a${gen}.${nodes.length + 1}`, bus: busName, path: nodePath,
192
+ role: fact.role, name: fact.name,
193
+ x: fact.box[0], y: fact.box[1], w: fact.box[2], h: fact.box[3],
194
+ states: fact.states, interfaces: fact.interfaces,
195
+ };
196
+ if (fact.interfaces.includes('Action')) {
197
+ try {
198
+ const actions = parseActions(await this.call(busName, nodePath, 'org.a11y.atspi.Action', 'GetActions'));
199
+ if (actions.length)
200
+ node.action = actions[0];
201
+ }
202
+ catch { /* action probe optional */ }
203
+ }
204
+ nodes.push(node);
205
+ if (depth < MAX_DEPTH) {
206
+ try {
207
+ const children = await this.call(busName, nodePath, 'org.a11y.atspi.Accessible', 'GetChildren');
208
+ for (const [childBus, childPath] of Array.isArray(children) ? children : []) {
209
+ next.push([String(childBus), String(childPath), depth + 1]);
210
+ }
211
+ }
212
+ catch { /* leaf */ }
213
+ }
214
+ }
215
+ frontier = next;
216
+ }
217
+ }
218
+ for (const node of nodes.slice(0, 100)) {
219
+ if (node.interfaces.includes('Text') || node.interfaces.includes('Value')) {
220
+ node.value = await this.readValue(node);
221
+ }
222
+ }
223
+ return { gen, nodes, apps };
224
+ }
225
+ query(nodes, words) {
226
+ const needle = words.toLowerCase();
227
+ return nodes.filter(node => node.name.toLowerCase().includes(needle) || node.role.toLowerCase().includes(needle) || node.value?.toLowerCase().includes(needle));
228
+ }
229
+ // Re-check a snapshot node against the live tree: identity (role, name,
230
+ // geometry) must still match, else the ref is stale.
231
+ async revalidate(node) {
232
+ let fact;
233
+ try {
234
+ fact = await this.nodeFacts(node.bus, node.path);
235
+ }
236
+ catch {
237
+ throw new A11yError('stale-ref', `${node.ref} is gone; run desklink-axi tree`);
238
+ }
239
+ const moved = fact.box.some((n, i) => n !== [node.x, node.y, node.w, node.h][i]);
240
+ if (fact.states.includes('defunct') || fact.name !== node.name || fact.role !== node.role || moved) {
241
+ throw new A11yError('stale-ref', `${node.ref} (${node.role} ${JSON.stringify(node.name)}) moved or changed; run desklink-axi tree`);
242
+ }
243
+ return { ...node, states: fact.states, interfaces: fact.interfaces };
244
+ }
245
+ async doAction(node) {
246
+ const actions = parseActions(await this.call(node.bus, node.path, 'org.a11y.atspi.Action', 'GetActions'));
247
+ if (!actions.length)
248
+ throw new A11yError('not-actionable', `${node.ref} exposes no AT-SPI action`);
249
+ const done = await this.call(node.bus, node.path, 'org.a11y.atspi.Action', 'DoAction', 'i', '0');
250
+ if (done !== true)
251
+ throw new A11yError('action-failed', `${node.ref} action ${JSON.stringify(actions[0])} was rejected`);
252
+ return actions[0];
253
+ }
254
+ async grabFocus(node) {
255
+ const done = await this.call(node.bus, node.path, 'org.a11y.atspi.Component', 'GrabFocus');
256
+ return done === true;
257
+ }
258
+ }
259
+ function classifyBusctlError(detail) {
260
+ if (/ServiceUnknown|not provided by any .service|Could not activate|Failed to activate/.test(detail))
261
+ return 'a11y-unavailable';
262
+ if (/No such file or directory|Failed to connect/.test(detail))
263
+ return 'a11y-unavailable';
264
+ if (/doesn't exist|Unknown interface|UnknownMethod/.test(detail))
265
+ return 'a11y-unsupported';
266
+ return 'a11y-error';
267
+ }