@ycheung5/pi-computer-use 0.0.0-stage → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (95) hide show
  1. package/GLOSSARY.md +38 -0
  2. package/LICENSE +21 -0
  3. package/README.md +165 -2
  4. package/assets/accessibility.cs +208 -0
  5. package/assets/browser-indicators.cs +37 -0
  6. package/assets/capture.cs +215 -0
  7. package/assets/clipboard.cs +197 -0
  8. package/assets/controls.cs +249 -0
  9. package/assets/indicator-preview.png +0 -0
  10. package/assets/indicators.cs +252 -0
  11. package/assets/input-timing.cs +42 -0
  12. package/assets/node-LICENSE.txt +2732 -0
  13. package/assets/ocr.cs +86 -0
  14. package/assets/ownership.cs +59 -0
  15. package/assets/pointer.cs +241 -0
  16. package/assets/puppeteer-LICENSE.txt +202 -0
  17. package/assets/readme-banner.svg +33 -0
  18. package/assets/stop.cs +197 -0
  19. package/assets/text.cs +279 -0
  20. package/assets/wgc.cs +167 -0
  21. package/assets/window-management.cs +217 -0
  22. package/assets/window-marker.cs +23 -0
  23. package/assets/window-references.cs +80 -0
  24. package/assets/windows.cs +127 -0
  25. package/dist/accessibility.d.ts +29 -0
  26. package/dist/accessibility.js +258 -0
  27. package/dist/browser-actions.d.ts +22 -0
  28. package/dist/browser-actions.js +246 -0
  29. package/dist/browser-indicators.d.ts +23 -0
  30. package/dist/browser-indicators.js +69 -0
  31. package/dist/browser-profiles.d.ts +10 -0
  32. package/dist/browser-profiles.js +174 -0
  33. package/dist/browser.d.ts +11 -0
  34. package/dist/browser.js +2071 -0
  35. package/dist/capture.d.ts +74 -0
  36. package/dist/capture.js +199 -0
  37. package/dist/client.d.ts +39 -0
  38. package/dist/client.js +384 -0
  39. package/dist/desktop.d.ts +20 -0
  40. package/dist/desktop.js +42 -0
  41. package/dist/extension.d.ts +4 -0
  42. package/dist/extension.js +409 -0
  43. package/dist/input-timing.d.ts +24 -0
  44. package/dist/input-timing.js +70 -0
  45. package/dist/launcher.d.ts +6 -0
  46. package/dist/launcher.js +85 -0
  47. package/dist/native.d.ts +11 -0
  48. package/dist/native.js +146 -0
  49. package/dist/ocr.d.ts +52 -0
  50. package/dist/ocr.js +246 -0
  51. package/dist/ownership.d.ts +4 -0
  52. package/dist/ownership.js +24 -0
  53. package/dist/pointer.d.ts +128 -0
  54. package/dist/pointer.js +198 -0
  55. package/dist/protocol.d.ts +15 -0
  56. package/dist/protocol.js +1 -0
  57. package/dist/runtime.d.ts +8 -0
  58. package/dist/runtime.js +72 -0
  59. package/dist/settings.d.ts +85 -0
  60. package/dist/settings.js +449 -0
  61. package/dist/stop.d.ts +24 -0
  62. package/dist/stop.js +156 -0
  63. package/dist/text.d.ts +120 -0
  64. package/dist/text.js +212 -0
  65. package/dist/waits.d.ts +19 -0
  66. package/dist/waits.js +262 -0
  67. package/dist/window-management.d.ts +25 -0
  68. package/dist/window-management.js +167 -0
  69. package/dist/worker.d.ts +1 -0
  70. package/dist/worker.js +553 -0
  71. package/docs/README.md +55 -0
  72. package/docs/adr/0001-preserve-reference-behavior.md +9 -0
  73. package/docs/adr/0002-window-lifetime-references.md +11 -0
  74. package/docs/adr/0003-exclusive-desktop-control.md +11 -0
  75. package/docs/adr/0004-pi-first-acceptance.md +9 -0
  76. package/docs/adr/0005-default-apps-and-workspace-reuse.md +7 -0
  77. package/docs/agents/domain.md +9 -0
  78. package/docs/agents/issue-tracker.md +10 -0
  79. package/docs/agents/triage-labels.md +9 -0
  80. package/docs/browser.md +113 -0
  81. package/docs/implementation-plan.md +153 -0
  82. package/docs/indicators.md +53 -0
  83. package/docs/native-interop-research.md +17 -0
  84. package/docs/ownership.md +25 -0
  85. package/docs/pi-extension-research.md +57 -0
  86. package/docs/pointer.md +77 -0
  87. package/docs/runtime.md +42 -0
  88. package/docs/settings.md +33 -0
  89. package/docs/spec.md +230 -0
  90. package/docs/stopping.md +83 -0
  91. package/docs/text.md +99 -0
  92. package/docs/tickets.md +85 -0
  93. package/docs/validation.md +1420 -0
  94. package/docs/windows.md +171 -0
  95. package/package.json +134 -4
@@ -0,0 +1,77 @@
1
+ # Guarded pointer actions
2
+
3
+ Native actions inherit [confirmed visible-window intent](windows.md#default-apps-and-workspaces) within the current operation after resolving the actual target root. Explicit `activate: false` and background dispatch/method preserve their existing behavior; passive observations remain passive. Supported #30 acceptance and remaining release owners are recorded in the [queue](tickets.md#execution-queue).
4
+
5
+ The candidate acquires [exclusive control](spec.md#exclusive-control) before physical pointer dispatch. Foreign injected or unconsumed interference still aborts. Reuse the accepted [#31 control/handback evidence](validation.md#control-interruption-acceptance).
6
+
7
+ `click`, `double_click`, `move`, `drag` and coordinate `scroll` use screen coordinates in physical desktop pixels. Convert image coordinates using the screenshot's `origin`, `imageScale` and `imageScaleY`; do not treat a resized PNG's pixels as desktop coordinates. Negative monitor coordinates are supported. Observed HWND/PID origins are scoped to the operation; source-window movement shifts coordinate points/the whole drag path by the observed delta. A remembered title resolving to a replacement window fails closed until fresh observation.
8
+
9
+ Auto left clicks on an observed element try native/UIA background behavior, then unconfirmed background messages. Explicit `dispatch: background` posts messages and never authorizes physical fallback. Auto double/right clicks and `dispatch: foreground` use physical input. Move retains the requested hover position and activates with explicit or inherited `activate: true`. Other physical actions return the pointer when it is still where automation left it; foreground restoration respects `activate: true`, user movement/focus changes and opened menus/dialogs. Target activation shares the existing native thread-attachment helper used by window management, without synthetic Alt/keyboard workarounds. Windows may refuse foreground activation/restoration; results report that refusal.
10
+
11
+ Before physical input, the worker arms the independent Esc listener and passes its operation-specific cancellation event and retained process identity to the native helper. The helper checks target HWND/PID, enabled/visible state, foreground, source identity and source-or-visible-same-process-popup hit testing immediately before dispatch. A covering window from the same process also fails the check. Held user buttons/modifiers refuse takeover. Drag validates screen bounds and the starting point against the source or its visible same-process menu/dialog, including popups outside the source frame and enabled modal dialogs whose source frame is disabled. It permits cross-window drops and destination foreground activation, keeps checking source identity, cancellation and pointer/user interference between steps, and releases its button in `finally` before any restoration/provider work. There is no automatic replay after uncertain dispatch, timeout or cancellation. Only a background `WCU_NEEDS_INPUT` result, generated before input, permits guarded invoke/element-scroll fallback.
12
+
13
+ A missing cleanup acknowledgment retains desktop ownership and blocks replacement work. Both bridge and outer-worker clients refuse force termination while input may be active; the flag clears only after the native button-up cleanup acknowledgment. A release failure remains closed rather than allowing another operation to send input. [Keyboard and clipboard actions](text.md) share this cleanup contract.
14
+
15
+ Confirmed activation and successful input/background operations remember the exact target HWND/PID for this operation. Later targetless observations and input use that identity after foreground is returned. Explicit targets, browser requests, element references and explicit desktop scope take precedence. Settlement clears the remembered target; a stale identity fails rather than guessing the user's current foreground. Screenshots retain their existing full-desktop versus window-capture semantics.
16
+
17
+ ## Single-click holds and deadlines
18
+
19
+ The source candidate adds optional `holdMs` and `timeoutMs` to single `click` for existing left/right/middle buttons. Omitted or zero hold releases immediately. Explicit timing uses native foreground dispatch, including auto element clicks; explicit `dispatch: background`, `activate: false` and browser targets refuse before input. With both fields omitted, existing routing and activation rules above apply. Double-click and drag do not accept timing fields.
20
+
21
+ Both request layers reuse the [keypress timing policy](text.md#keypress-holds-and-deadlines): positive timeout is authoritative, zero disables only the execution deadline, and omission adds an explicit hold to the ordinary budget. Safe whole nonnegative milliseconds have no product cap; invalid values and derived overflow refuse before input. Native waits use bounded interruptible intervals and check target, foreground, point, protection and pointer interference throughout a positive hold. Zero/omitted holds retain immediate release without a new post-down target check that could reject a dialog opened by the click.
22
+
23
+ Results report effective `timing` (`holdMs`, `timeoutMs`, `deadlinePolicy`), `inputDispatched`, interruption and `cleanupConfirmed`. Dispatch remains distinct from an observed application effect. Release runs before pointer/foreground restoration. Unconfirmed release retains ownership and blocks dependent work; timeout, cancellation and partial input never trigger replay.
24
+
25
+ The existing owned fixture prepares scoped acceptance with `npm run test:native:pointer:physical -- --list --case=timing` (safe planning only). The Pi-only `--case=timing` run requires a build, Windows Node and `PI_CU_DEDICATED_INPUT_DESKTOP=1` on the authorized idle test desktop. It observes actual button messages/timestamps and OS state for default click/restoration, zero hold, timed left/right/middle, a covering-target error, shorter deadline and cancellation of a representable very long hold with deadline disabled. Failure or unconfirmed cleanup retains the owner and fixture. This case does not rerun the old physical baseline or physical Esc demonstration.
26
+
27
+ Portable worker/peer checks verify schema, route refusals, two-layer timing forwarding, default/positive/disabled deadlines, long values, overflow, shutdown and ownership retention without sleeping for long requested durations. Native compilation and the scoped actual fixture effects passed; see the [composed receipt](validation.md#input-timing-and-modifier-tap-acceptance). Portable results alone do not establish physical input timing.
28
+
29
+ ## Validation
30
+
31
+ - `npm test`: worker argument validation and the public worker lifecycle contract that prevents retirement while cleanup is unconfirmed. These portable tests do not prove Windows input delivery.
32
+ - `npm run build` then `npm run test:native:pointer:guards`: dedicated passive fixture, background click/double click, last-target observation, stale/wrong-owner refusal, outside points and covered-point refusal. Sends no physical input. Uses the same Windows runtime setup as [runtime validation](runtime.md).
33
+ - `npm run test:native:controls`: background controls regression; unsupported invoke fallback is explicitly disabled so this suite stays free of physical input.
34
+ - `npm run test:native:pointer:physical -- --list --engine=pi`: safe WSL plan only; does not load a backend, inspect files, compile a fixture or contact Windows. Execution without `PI_CU_DEDICATED_INPUT_DESKTOP=1` refuses before those operations. Unknown options/engines fail rather than silently selecting a backend.
35
+ - [Dedicated Windows runner instructions](validation.md#dedicated-physical-input-runners): the existing runner now selects `--engine=pi` (default, registered Pi tool execution) or `--engine=reference --reference=<Windows-readable-reference-path>` (public MCP). Each invocation creates a fresh controls fixture and stops on failure; there is no input replay or method switch. It prepares scaled-point click, exact double-click count, actual hover, inner-panel scroll and drag-cancellation release checks. Fixture-owned counters/hover/scroll state and the OS left-button state supply independent outcomes; raw action/errors are retained as diagnostics without requiring Pi-specific method/restoration fields. The later [dedicated physical baseline](validation.md#dedicated-physical-comparison-evidence) records executed cases and retained failures.
36
+
37
+ Before release, complete the remaining display matrix under #6 (150%/200%, mixed DPI and topology changes), sharing evidence across pointer and capture owners. Reuse accepted #31 physical stop, suppression, held-input cleanup and handback results; an open pointer issue is not a reason to repeat them. Target replacement/inaccessible-provider gaps retain #28 ownership.
38
+
39
+ ## Corrective acceptance cases
40
+
41
+ The passive `test:native:pointer:guards` fixture covers background dispatch to an owned popup outside its source frame, stale/wrong-PID/foreign/covered-point and no-input refusals. Shared hit testing admits the source or a visible same-process owned/popup window, never an arbitrary other process or our indicator overlay.
42
+
43
+ The [2026-11-04 physical receipt](validation.md#scoped-pointer-and-restoration-acceptance) adds a disabled-parent popup and a real native menu outside the source frame, exact OS cursor restoration, intended foreground and a successful cross-process drop with destination activation while held. Use the existing runner's selected cases: `--case=owned-popup`, `--case=native-menu`, `--case=cross-app-drop` or passive `--case=cross-app-discovery`. They retain the opt-in gate and independent button-release requirement. Native menu reads use its observed HWND; the physical selection remains scoped to its owned source.
44
+
45
+ Restoration never overrides observed external interference or runs after unconfirmed key/button cleanup. Human handback reuses #31 and the operation-settlement path, which waits for pointer cleanup and releases control without scheduling delayed restoration. Universal app behavior and the remaining display matrix are not established by these scoped passes.
46
+
47
+ ## Paired comparison preparation (#30)
48
+
49
+ Historical preparation only. Under [ADR 0004](adr/0004-pi-first-acceptance.md), reference-engine runs and side-by-side comparisons are not release prerequisites; current #30 work validates Pi outcomes.
50
+
51
+ The original preparation validated portable CLI planning/refusal; the later dedicated session adds scoped physical results. The `--list` output distinguishes the implemented runner cases from pending coverage; listing is not a test pass. Same-application drag/drop and scoped foreground/restoration, same-process distractor and replacement cases have since passed paired dedicated-session tests. Later Pi-only popup/menu/cross-process-drop results are recorded above; accepted control evidence stays with #31 and display gaps with #6. No DSH comparison rerun is required. This historical paired preparation is not the current completion ledger; use the [queue](tickets.md#execution-queue).
52
+
53
+ The paired runner accepts DSH's single uniform screenshot scale and Pi's separate Y scale, retaining the raw capture metadata. Hover preparation explicitly activates the owned fixture and verifies foreground through the fixture's independent HWND observation; bounded read-only polling allows the event queue to update after the single move. These runner corrections require a new physical run and do not establish a product fix.
54
+
55
+ The legacy dedicated physical runner retains its client, no-expiry fixture and workspace when a pointer dispatch rejects or lacks an action-matched cleanup acknowledgment. Reference `activate_window` is guarded too because its fallback can send Alt input. Contradictory cleanup fields are unconfirmed. The drop runner also requires an explicit acknowledgment plus independently observed button release, and uses a referenced keepalive while retaining an uncertain owner. Reference results without that acknowledgment are inconclusive even when the fixture observed a drop and button-up; the earlier observed effects remain historical evidence. These retention checks have portable regressions but have not been validated by a new live failure run.
56
+
57
+ ## Drag failure reporting
58
+
59
+ The physical runner now waits for the drag and fixture mouse-down observation to settle, then records bounded read-only release observations **before** asserting the dispatch outcome or tearing down the fixture. A timeout, unexpected success/error or missing fixture reply retains its original evidence together with button-held state, release-counter progression and any observation error. The report distinguishes cancellation confirmation from release confirmation; a released button does not turn a timeout into successful cancellation. No automatic button release, new input or replay is performed by this reporting step. A held or unconfirmed state requires the coordinator to stop dependent physical tests and arrange separately verified cleanup.
60
+
61
+ An offline saved-evidence check is available through the existing runner CLI:
62
+
63
+ ```sh
64
+ npm run test:native:pointer:physical -- --report-drag=/path/to/drag-evidence.json
65
+ ```
66
+
67
+ It runs before Windows/desktop opt-in and starts no client, fixture or input. Save the emitted report's `evidence` object as the input. Portable regressions distinguish cancelled-and-released, timed-out-and-held, timed-out-but-released, absent mouse-down and unavailable release observation. These reporting checks do not rerun or repair DSH's recorded physical cancellation failure. Live use of the updated error path remains pending the next dedicated session.
68
+
69
+ The separately selected `--case=drag-drop` prepares a fresh owned Windows Forms drag source and drop target for each engine. The target counts an actual OLE drop carrying the fixture payload, independently of the engine's result; the fixture also reads the OS left-button state. A pass requires a successful dispatch, exactly one observed drop and a released button. For example, first inspect the plan with `npm run test:native:pointer:physical -- --list --engine=pi --case=drag-drop`; run each engine only under the dedicated Windows prerequisites above. The case never retries input. Cleanup is independent of task success: a fulfilled, action-matched response with explicit cleanup acknowledgment and an independently observed released button permits teardown even when the action failed. The task remains failed. A rejected/timed-out or unacknowledged dispatch, unreadable oracle or held button retains the owner; stop dependent tests and investigate. This fixture tests a same-application drop; cross-application and user-interference tests remain pending. Fresh Pi and DSH runs each recorded one drop and an independently observed released button; see [validation evidence](validation.md#successful-dragdrop-fixture-preflight).
70
+
71
+ The first Pi drop run stopped before input because neither fixture panel appeared in UIA, including raw-view observations after a delay. The fixture now uses standard button controls with the same drag/drop handlers. `--case=drop-discovery` is a Windows-only passive preflight: it creates a fresh owned fixture, records control and raw trees, and reports UIA discovery separately and checks fixture-owned source/target geometry against the independently observed window identity and bounds. It sends no pointer or keyboard input but creates a normal window that can activate, so the dedicated desktop opt-in remains required. The physical case uses freshly observed geometry and stops before input if its checks fail. Subsequent no-input preflights found both controls on each engine, followed by successful paired physical drops as recorded in the validation evidence.
72
+
73
+ The original fixture used a hidden Windows process launch. Separate passive probes found an OS-hidden fixture and observed source and target buttons after launching the fixture visibly. The runner now starts a visible fixture and requires OS window visibility and a root observation that is not offscreen before input; this still requires dedicated live preflight and does not establish a successful drop.
74
+
75
+ ## Native-menu comparison preparation
76
+
77
+ `test:reference:focus-foreground -- --plan --native-menu --provenance` lists a separate Win32 `TrackPopupMenu` fixture. On an explicitly authorized idle Windows desktop the same options, plus engine selection, invoke its owned button without physical input. A result requires a visible native menu window, its target GUI thread's menu-owner and menu-mode evidence, a matching menu item from the engine UIA find, and repeated foreground/menu observations. The UIA item lookup is not an independent client. Reads are successive, not atomic. Missing or ambiguous identity, unobserved UIA item or unavailable menu mode remain inconclusive; an unresolved action retains its owned processes. Before normal cleanup, the fixture must confirm a posted dismissal, a queued acknowledgement after the modal menu handler returned, and native menu mode/window closure; otherwise it retains the backend and fixture. The fixture is compiled and its offline planning/report checks run, but no live native-menu run has been performed; it does not establish cross-application menu or user-interference behavior.
@@ -0,0 +1,42 @@
1
+ # Worker runtime and validation
2
+
3
+ ## Current scope
4
+
5
+ The extension implements the desktop/browser tool catalog from both Windows and WSL2 Pi. Health checks the worker connection and reports runtime/settings/helper status; it does not by itself prove that every desktop capability or physical input path passed acceptance. See [release validation](validation.md) for capability-to-test mapping and pending manual checks.
6
+
7
+ ## Runtime provisioning
8
+
9
+ The native launcher downloads Windows Node **22.23.4** for x64 or arm64 into `%LOCALAPPDATA%\pi-computer-use\runtime`. SHA-256 values are pinned from the [official release checksums](https://nodejs.org/dist/v22.23.4/SHASUMS256.txt). Downloads are bounded, cancellation-aware and installed atomically only after verification. Cached executables are verified before execution. Node's [license](../assets/node-LICENSE.txt) is included with the package.
10
+
11
+ A checksum mismatch fails explicitly; remove only the indicated runtime cache and retry from a trusted connection. HTTP failures identify nodejs.org as the required endpoint. Missing compiled worker files instruct checkout users to build. Unsupported hosts/architectures and missing Windows data directories fail without trying a different system runtime.
12
+
13
+ From WSL2, a bounded fixed-variable `cmd.exe /u /d` query discovers the Windows data directory and architecture. UTF-16LE decoding preserves Unicode, and environment values are never interpolated into commands. Discovery is shared with personal settings and cached for subsequent launcher calls. `wslpath` translates the mounted runtime cache and the compiled worker location, including WSL UNC paths. The launcher executes that same private Windows Node directly with argument arrays, keeping protocol stdout separate from bootstrap diagnostics; it never sends a Linux PID to a Windows process API. Windows executable interop and `cmd.exe`/`wslpath` must be available; browser discovery and some native fixtures also use Windows PowerShell. A failed bootstrap reports `WSL_BOOTSTRAP_FAILED` with its cause.
14
+
15
+ The runtime and worker start on first tool execution, not extension registration. Restoring tool exposure reads personal settings at session startup; in WSL2 this performs the small shared environment lookup but does not provision or start the worker. The cross-host settings fixture measured 39 ms for this first lookup on the tested host (a prior PowerShell query took about 1.2 seconds); this is measured overhead, not a zero-cost startup claim. The worker stays available until session shutdown/reload. Startup provisioning is cancelled on shutdown, even if a provider fails to observe the abort signal. Worker timeouts reject pending work and retire the worker when ownership permits; a later call can start a replacement. Keypress, single-click and explicit SendInput text execution deadlines first request cooperative cancellation and retain the cleanup reply during an independent five-second grace. Optional hold/pacing and positive/disabled deadline policies cross both request layers; see [keypress timing](text.md#keypress-holds-and-deadlines), [text pacing](text.md#sendinput-pacing-and-deadlines) and [single-click timing](pointer.md#single-click-holds-and-deadlines). Worker shutdown also signals active operations when an execution deadline is disabled. Input is never automatically replayed.
16
+
17
+ ## Interfaces for implementation
18
+
19
+ The normal worker client interface is `execute(action, args, signal?, operationId?)` and `dispose()`. The launcher supplies an executable, argument array and optional working directory. Transport is one JSON request/response per line, with numeric request IDs and operation identity. Responses contain `result` with `ok`, `action` and action-specific evidence. Cancellation and stop controls carry the affected request/operation identity; input-state notices keep cleanup and retirement ordered. Stdout is protocol-only; diagnostics use stderr.
20
+
21
+ On stdin closure the worker announces private `shutdown_state` phases `started`, `complete` or `failed`. During retirement, the client gives announced resource cleanup its ordinary execution deadline; it does not wait a fixed delay after successful cleanup. Silent legacy peers retain the short process-exit grace; their retirement proves process exit, not resource-cleanup acknowledgement. Once cleanup is announced, a forced/early exit without completion or an explicit cleanup failure reports `WORKER_CLEANUP_UNCONFIRMED` and blocks replacement, including after repeated disposal or later process exit. Shutdown notices are accepted only from the current retiring child, and an out-of-order completion cannot acknowledge cleanup. A completed shutdown does not override an action's explicit retained-profile warning. Physical-input release remains independent and stronger: shutdown completion never clears an active input lease or permits killing its owner.
22
+
23
+ Desktop ownership, independent stopping and guarded input paths are integrated. Their completion/cleanup acknowledgments are part of the worker protocol; cancellation never authorizes replay or early ownership release. See [ownership](ownership.md), [stopping](stopping.md), [pointer](pointer.md) and [text](text.md) for their contracts and unrun physical acceptance cases. Tests remain at the agreed Pi tool and worker request/cancellation interfaces.
24
+
25
+ ## Verification
26
+
27
+ For live Windows tests from a WSL worktree, make dependencies readable by Windows Node before starting a fixture. An absolute Linux `node_modules` symlink can work in Linux while Windows cannot resolve `puppeteer-core` through it. Use a real worktree dependency installation or a Windows-readable staged copy, build the candidate, and verify dependency resolution from that exact location. Record staged source identity; a missing package in the test copy is a staging failure, not a product failure.
28
+
29
+ - `npm run check`: TypeScript, Biome and portable tests for tool execution, lazy reuse, disposal, startup cancellation, failed-start recovery, timeout non-replay and checksum rejection.
30
+ - `npm run build`: emits extension and worker modules; browser execution uses the declared puppeteer-core runtime dependency.
31
+ - `npm run test:package`: verifies the actual distribution includes compiled/native assets, runtime notices and a self-contained indexed documentation set.
32
+ - `npm run test:pi`: loads a clean production omit-peer install through real Pi CLI with a public offline fixture provider, then executes health/list and verifies settlement/shutdown. On WSL2 it exercises both WSL2 and native Windows Pi processes. See [validation scope](validation.md#reproduce-automated-checks).
33
+ - `npm run test:wsl`: after building, run from WSL2. It copies the package to disposable WSL and Windows-drive locations containing spaces, Unicode and a quote, then executes the default tool factory through the Windows worker. It also checks active transport cancellation, timeout cleanup and abrupt Linux-parent exit. Like the native smoke, it models Pi registration rather than running an LLM-driven Pi CLI session. It sends no physical input.
34
+ - `npm run test:native:lifetime:enumeration`: the real native enumeration callback on a verified unswitched owned desktop; a fixture removes its marker during the responsiveness read. No normal-desktop fallback. This does not certify interactive UIA/input.
35
+ - `npm run test:native:lifetime -- --reuse-only`: one empirical owned hidden-window probe, with two 30-second cycle limits, a 131072-entry history cap and a finite creation cap. A genuine full-HWND/PID collision and hidden-root public UIA binding are prerequisites to the reference/close checks; exhausted bounds exit 2 as unavailable, never pass. No global discovery/input or assumptions about handle bit layouts. Keep the shared live-test lock throughout; do not retry or enlarge limits to turn an unavailable environment into acceptance.
36
+ - `npm run test:native:lifetime -- --expiry-only`: the same scoped runner waits 305 seconds without refreshing references and proves actual five-minute expiration with unchanged fixture/worker/native-helper process identities. The fixture alone uses a six-minute safety timer; the default suite remains unchanged. Hold the shared live-test lock for this entire case.
37
+ - `npm run test:native:lifetime`: Windows-only owned-fixture window references, marker loss, two authorities/restart, composed captures/OCR/waits and provider dispatch guards; no global discovery/launch/desktop captures or physical input. See [acceptance limits](validation.md#window-lifetime-reference-acceptance).
38
+ - `npm run test:native`: after building, run with Windows Node in the package directory. It executes the default extension factory's health definition through the actual private Windows worker, checks PID reuse and verifies shutdown. Its minimal host harness models Pi registration; it is not an LLM-driven Pi CLI acceptance run.
39
+
40
+ Live evidence on 2026-10-29: Windows 11, x64, PowerShell 5.1; no Windows Node on PATH. A private bootstrap executable ran the compiled tool factory from a WSL UNC checkout, then the native launcher performed a cold checksum-verified download and returned health twice from one Windows worker. Shutdown and post-shutdown rejection passed. The test sent no desktop input. This initial smoke did not prove real Pi CLI execution or clean installation. Later real-CLI production-package evidence is recorded in [release validation](validation.md); arm64, clean-machine installation and full interactive acceptance remain separate release gates.
41
+
42
+ WSL2 live evidence on 2026-10-29: the health-tool smoke passed from both filesystem locations, preserving literal path characters. Repeated calls reused the Windows PID; disposal and abrupt parent exit left no worker process. A disposable Windows protocol peer proved active cancellation and timeout cleanup without input. Sandbox-restricted Windows launches require environment approval; the default sandbox `UtilBindVsockAnyPort` failure is an interop restriction, not an application failure. The later real Pi CLI runner verifies production-package health/list through both hosts. Clean-machine installation, full interactive workflows and physical input cancellation remain separate release checks.
@@ -0,0 +1,33 @@
1
+ # Personal settings and tool loading
2
+
3
+ ## Commands and storage
4
+
5
+ Use `/computer-settings` (or `edit`) to edit the personal settings JSON in Pi's editor. `show` reports the loaded values, notices and location; `set {"key":value}` merges a validated patch. Examples:
6
+
7
+ - `/computer-settings set {"toolPreset":"lite"}`
8
+ - `/computer-settings set {"toolPreset":"custom","enabledTools":["health","list_windows","find","screenshot","tools"]}`
9
+ - `/computer-settings set {"browserApp":"edge","browserVisible":false}`
10
+
11
+ The Windows-side file is `%LOCALAPPDATA%\pi-computer-use\settings.json`, shared across projects and native Windows/WSL2 hosts. A missing file uses explicit new-personal defaults. Malformed JSON, unreadable files or invalid numeric/text fields produce visible errors and block execution rather than pretend defaults were loaded. Unknown saved enum/switch choices are repaired in memory with notices; unknown tool names are dropped. User-entered values are validated strictly before atomic save. `PI_CU_SETTINGS_FILE` is an explicit alternate location used by disposable fixtures; ordinary users need no environment variable.
12
+
13
+ Settings writes are serialized behind active calls in each Windows worker and use a cross-process save lock plus atomic file replacement. Other workers apply the file at their next call boundary. Concurrent live saves receive `SETTINGS_BUSY` rather than overwrite each other silently. A settings change does not replay an action or clear a stopped operation. Session startup restores tool exposure by reading settings without starting the worker; WSL discovery is shared/cached with the launcher. See [runtime overhead](runtime.md#runtime-provisioning).
14
+
15
+ ## Presets and management
16
+
17
+ `toolPreset` supports `full`, `lite`, `lazy`, `custom` and legacy `auto`. Full allows/loads implemented tools. Lite allows the reference common set: list windows, find, click, scroll, text, keypress, OCR, screenshot, wait-for, launch and management. Lazy allows all tools but initially loads only management. Custom uses `enabledTools` as both allowlist and initial selection. Legacy `auto` resolves to full when all names are selected, otherwise custom; the explicit preset takes precedence over legacy `mode`.
18
+
19
+ `windows_computer_use_tools` accepts `operation: list | describe | enable | disable | reset` and optional `names`. Names may be short or fully prefixed. Describe requires one name; enable/disable require names; `['*']` is supported only alone for enable/disable; reset takes no names. Every name is resolved before mutation, so a typo cannot partially alter the selection. Reset restores the saved startup preset. Temporary loading changes are scoped to the worker and do not rewrite personal settings.
20
+
21
+ The canonical catalog includes all 24 automation names plus management, with separate `available`, `allowed` and `enabled` fields. Capabilities not implemented in this build are explicitly unavailable; enabling them fails. Pi exposure changes only this extension's actually registered tools and preserves unrelated tools. Execution checks the Windows-side policy independently, so invoking a hidden or settings-disabled tool directly is still refused. Custom selection can disable management itself; recover through `/computer-settings set {"toolPreset":"full"}`. Persisted selection errors can be corrected in the displayed settings file when it cannot be parsed.
22
+
23
+ ## Browser and presentation preferences
24
+
25
+ Ordinary app requests follow the [workspace policy](spec.md#default-apps-workspaces-and-foreground). Browser settings apply within an explicitly selected dedicated workspace: browserFirst prefers its protocol channel, browserVisible is true, browserApp uses Chrome/Edge default preference, and remembered sign-ins are off; screen/pointer/remote indicators on; Pi-branded one-line indicator text, top position, 800 ms idle delay; extra stop shortcut empty (Esc remains reserved). App/visibility/profile changes close an idle owned browser before the next call reconnects, so refresh tab/element references afterward. Browser preference never overrides ordinary Windows default-app/existing-window reuse. User-started Pi guidance respects disabled navigation.
26
+
27
+ During candidate [exclusive control](spec.md#exclusive-control), the local border is required regardless of optional presentation settings; the saved idle delay does not expire control or delay its release. Stop shortcuts and presentation use the independent helper described in [indicators](indicators.md). Remembered sign-in behavior belongs to the dedicated browser-profile slice. A saved preference alone is not proof that a later capability is implemented. `indicatorIdleMs` is 200–3000, text is a trimmed single line up to 120 characters, and resume/preview tokens are nonnegative safe integers. Extra shortcuts accept Ctrl/Alt/Shift/Win plus one supported letter, digit, function or navigation key; modifiers are canonicalized and Esc cannot be replaced.
28
+
29
+ ## Validation
30
+
31
+ `npm run check` includes worker policy and Pi command tests: persistence/reload, lazy/custom/full/lite transitions, invalid edits, repaired choices, malformed reads, all-or-nothing management, queued saves, disabled execution, and preservation of unrelated tools. Tests exercise the approved worker/Pi seams; they do not assert private implementation state.
32
+
33
+ After build, `npm run test:settings` runs from WSL2 and uses a disposable Windows-side settings file. A native Windows peer saves values that WSL Pi reads, then WSL's settings command saves values read by the Windows peer. It verifies lazy startup, tool denial, error visibility and literal Unicode/shell-character environment lookup. It never changes daily personal settings, browser profiles or physical input. Live evidence on 2026-10-29 passed on Windows 11 x64 with first shared settings discovery measured at 39 ms. The user confirmed the Pi terminal settings display, editor cancel/unchanged-save and reload checks on 2026-11-09; see the [acceptance receipt](validation.md#interactive-settings-and-reload-acceptance). The automated host contract and this human confirmation have distinct scopes.
package/docs/spec.md ADDED
@@ -0,0 +1,230 @@
1
+ # Pi Windows Computer Use
2
+
3
+ Status: agreed scope. Original product specification: [GitHub issue #1](https://github.com/ycheung5/pi-computer-use/issues/1). Version 0.1.0 is published and its product/release owners are completed. The post-release [input timing and modifier-tap specification](#input-timing-and-modifier-taps) is merged into main for 0.1.1; it is not part of published 0.1.0. [ADR 0004](adr/0004-pi-first-acceptance.md) supersedes mandatory DSH parity; historical alignment records are not current release gates.
4
+
5
+ ## Problem Statement
6
+
7
+ Pi users need to observe and operate Windows applications and browser pages without installing DSH or coordinating several Pi extensions. DSH supplied useful design context, but this project is judged on supported Pi outcomes and its Windows/WSL2 integration. The new project also needs a simple source layout and indexed documentation so contributors can find relevant context without reading the whole repository.
8
+
9
+ ## Solution
10
+
11
+ Deliver one installable Pi package with native computer-use tools, reliable supported desktop/browser behavior, machine-wide personal settings and package-managed Windows execution prerequisites. Support native Windows Pi and WSL2 Pi through one Windows-side automation worker. Preserve target safeguards, independent emergency stopping and honest verification while simplifying implementation where behavior stays intact.
12
+
13
+ ## User Stories
14
+
15
+ 1. As a Pi user, I want one package to install, so that I can use computer control without other Pi extensions or DSH.
16
+ 2. As a native Windows host user, I want desktop automation from default Pi, so that I can operate Windows applications directly.
17
+ 3. As a WSL2 host user, I want equivalent Windows desktop access, so that I can keep my Linux-based Pi workflow.
18
+ 4. As a user, I want required worker runtimes managed by the package, so that I need not install Windows Node separately.
19
+ 5. As a user, I want lazy worker startup, so that ordinary Pi usage avoids automation startup overhead.
20
+ 6. As a user, I want a reusable worker, so that subsequent calls avoid repeated startup costs.
21
+ 7. As a user, I want health diagnostics, so that I can identify unavailable prerequisites and connections.
22
+ 8. As an agent, I want to list windows, so that I can choose the intended application.
23
+ 9. As an agent, I want snapshots and accessibility trees, so that I can understand controls and their current state.
24
+ 10. As an agent, I want to find and inspect elements, so that I can target controls precisely.
25
+ 11. As an agent, I want screenshots with coordinate metadata, so that visual targeting maps correctly to the desktop.
26
+ 12. As an agent, I want OCR with text bounds, so that I can work with interfaces that expose little accessibility information.
27
+ 13. As an agent, I want click, double-click, move, drag and scroll actions, so that I can operate visual interfaces.
28
+ 14. As an agent, I want text entry and key chords, so that I can interact with applications accurately.
29
+ 15. As an agent, I want focus, invoke and value-setting actions, so that supported controls can be operated with less physical input.
30
+ 16. As an agent, I want application launch and window activation, movement and closure, so that I can manage a complete desktop workflow. Follow the [default-app and workspace policy](#default-apps-workspaces-and-foreground), preserving existing work.
31
+ 17. As an agent, I want explicit waits and observable conditions, so that I can verify outcomes without guessing.
32
+ 18. As a user, I want background actions preferred, so that automation interferes less with my work.
33
+ 19. As a user, I want input refused when another window covers the target, so that actions do not reach the wrong application.
34
+ 20. As a user, I want stale element and window identities rejected, so that changed interfaces require fresh observation.
35
+ 21. As a user, I want clipboard, pointer and foreground restoration where appropriate, so that automation leaves my working state intact.
36
+ 22. As a user, I want exclusive control during agent desktop input and explicit human handback, so that accidental input cannot disrupt automation and I can regain control with Esc.
37
+ 23. As a user, I want visible control indicators and a configurable stop shortcut, so that I know when physical input is in use.
38
+ 24. As a user, I want emergency stop independent of a busy worker or focused terminal, so that I can stop automation reliably.
39
+ 25. As a user, I want the stopped computer-use operation blocked from retrying, so that my stop request remains effective.
40
+ 26. As a user, I want a new user operation to permit resuming, so that stopping does not permanently disable computer use.
41
+ 27. As a user, I want competing sessions to receive a busy result, so that they cannot interleave desktop actions.
42
+ 28. As an agent, I want calls within the owning session ordered, so that observations and actions retain their intended sequence.
43
+ 29. As a user, I want browser automation without moving my physical pointer, so that web work can remain in the background.
44
+ 30. As a user, I want installed Chrome and Edge support and experimental installed Firefox-family support, so that I can use the installed supported browsers.
45
+ 31. As an agent, I want browser tabs, navigation, element actions and dialogs, so that I can complete supported web workflows.
46
+ 32. As an agent, I want page changes returned after browser actions, so that I can assess their effect.
47
+ 33. As a user, I want my Windows default browser and existing browser window reused with a new task tab, so that opening or using the browser fits my normal workspace and preserves existing tabs.
48
+ 34. As a user, I want an optional dedicated browser profile, including remembered sign-ins, when I explicitly choose a separate automation workspace.
49
+ 35. As a user, I want personal settings shared across projects and hosts, so that preferences stay consistent on my Windows machine.
50
+ 36. As a user, I want full, lite, lazy and custom tool selection, so that I can control available capabilities and model context.
51
+ 37. As a user, I want disabled tools rejected during execution, so that hiding tools is not the only enforcement.
52
+ 38. As an agent, I want dispatched input distinguished from observed success, so that I do not claim an unverified objective.
53
+ 39. As a user, I want cancellation, timeout and crash cleanup, so that keys, processes and desktop ownership are not abandoned.
54
+ 40. As a user, I want input never replayed automatically after an uncertain timeout, so that an action is not accidentally duplicated.
55
+ 41. As a maintainer, I want deterministic fixtures and failure coverage, so that regressions are reproducible.
56
+ 42. As a maintainer, I want integration tests through both hosts, so that worker-only success does not hide launch-path failures.
57
+ 43. As a user, I want Pi performance measurements and acceptance checks, so that improvements preserve reliability and useful responsiveness.
58
+ 44. As a contributor, I want a short documentation index and simple folders, so that I can find task-specific context with less searching.
59
+
60
+ 45. As a maintainer, I want fixes traced to observed Pi failures and evidence, so that symptom workarounds do not become permanent requirements.
61
+ 46. As a maintainer, I want supported Pi behavior and limitations explicit, so that design choices are distinguishable from defects.
62
+ 47. As a user, I want exact, responsive Pi text entry validated, so that pacing changes preserve correctness and focus without unnecessary delay.
63
+ 48. As a user, I want repeatable Pi tests automated, so that I do not have to paste diagnostic prompts manually.
64
+ 49. As an agent, I want detailed action guidance and useful failure evidence, so that I can choose supported recovery.
65
+ 50. As a user, I want ordinary tasks with controlled obstacles tested, so that tool success translates into useful task completion.
66
+ 51. As a user, I want offscreen controls, stale IDs and incomplete accessibility handled when supported, so that routine obstacles do not require unnecessary intervention.
67
+ 52. As a maintainer, I want direct-call reproduction of agent failures, so that tool, fixture, environment and reasoning problems are separated using evidence.
68
+ 53. As a user, I want my available Pi model configuration tested, so that acceptance reflects how I use the extension.
69
+ 54. As a maintainer, I want Pi failures and limitations recorded, so that unavailable or inconclusive cases do not become false passes.
70
+ 55. As a maintainer, I want failures retained and affected regressions rerun, so that a final pass does not hide previous defects.
71
+ 56. As a user, I want three consecutive reset successes for required task/recovery scenarios, so that a single lucky run is not sufficient.
72
+ 57. As a contributor, I want stale documentation replaced during each fix, so that agents receive one current answer rather than conflicting notes.
73
+ 58. As a user, I want correctness and fewer interventions prioritized over latency and tokens, so that optimization preserves reliable behavior.
74
+ 59. As a user, I want observed native targets bound to a window lifetime, so that expired or unverifiable observations are rejected instead of silently selecting a replacement with the same HWND and process ID.
75
+
76
+ ## Implementation Decisions
77
+
78
+ - Implement fresh TypeScript informed by reference version 0.5.3 at commit `6fb642129595a6357588c70812974951db304662`; neither copy its engine nor wrap its package. A small native Windows helper is permitted where necessary. Retain applicable notices if any third-party material is subsequently reused.
79
+ - Register native Pi tools directly. The initial release requires neither a custom MCP server nor another Pi extension.
80
+ - Retain the 24 implemented automation capabilities and tool-management entry. Define supported parameters, observations, failures and safeguards by this Pi specification; exact DSH behavior is not a release requirement. Preserve the declared browser/host support and experimental status.
81
+ - Native screenshots optionally accept `windowOnly: true` with window scope. This opt-in captures only the selected window through PrintWindow or HWND-bound Windows.Graphics.Capture, never screen-copy/desktop fallbacks. Hidden, minimized or unavailable surfaces fail without an image. Reject desktop scope and browser-tab combinations; preserve existing behavior when omitted or false. This option is specific to the screenshot tool and does not change snapshot/OCR capture contracts.
82
+ - Use an extension module for tools, lifecycle, commands and rendering; a launcher module for host differences; and a Windows worker with focused desktop, browser, settings and stop-control modules. Start with flat responsibility-named source files and introduce subdirectories only when implementation growth justifies them.
83
+ - The worker exposes request execution with cancellation and disposal through an internal transport. Native Windows launches it directly; WSL2 uses a thin launcher. Keep browser discovery, profiles, filesystem paths and native process management on Windows.
84
+ - Supply a private runtime or packaged worker. Start it on first use, reuse it, and clean up on session shutdown/reload. Packaging and native interop choices require early feasibility evidence.
85
+ - Enforce desktop ownership across native Windows and WSL2 sessions. Queue calls within the owner; reject competitors rather than retaining their stale requests. Recover ownership after proven owner failure without stealing it from a slow live owner.
86
+ - Track a computer-use operation across model turns. Use final Pi settlement for completion and idempotent session shutdown for cleanup. Do not mistake an individual model turn or low-level run end for a new user operation.
87
+ - Propagate cancellation, allow bounded key/button release cleanup, and keep emergency stopping independent of blocked automation. Never replay timed-out input automatically.
88
+ - Prefer supported background dispatch while preserving the requested [workspace and foreground](#default-apps-workspaces-and-foreground). A visible open/use request still brings its result forward. Apply the [exclusive-control contract](#exclusive-control) when physical desktop control is needed; preserve target, occlusion, lifetime and handback safeguards.
89
+ - Keep the Pi observation contract: desktop actions generally report dispatch; browser actions normally return page changes. Explicit waits/observations establish expected outcomes. Distinguish sent, observed, unconfirmed and inconclusive results.
90
+ - Preserve Chrome/Edge and experimental installed Firefox-family compatibility. Libraries are allowed where they reduce code without altering the agreed contracts; managed Firefox is not an accepted compatibility substitute.
91
+ - Follow the [default-app and workspace policy](#default-apps-workspaces-and-foreground). Keep dedicated browser profiles as an explicit alternative. Personal settings live on Windows and apply across projects through a Pi command; project overrides remain excluded initially.
92
+ - Show the accepted blue edge glow at approximately 1.5 times DSH thickness throughout exclusive control; its visibility follows the [control contract](#exclusive-control). Preserve non-activation, click-through, capture and per-monitor behavior.
93
+ - Preserve full/lite/lazy/custom tool selection and execution-time denial. Use Pi's own registration, exposure, rendering and lifecycle facilities where appropriate.
94
+ - Support a release-tested baseline of Windows 11, Windows PowerShell 5.1, WSL2 Ubuntu and pinned/documented Pi versions. Compatibility outside this baseline remains unverified until tested.
95
+ - Provide a small documentation index with task-specific section links. Requirements, delivery sequence, terminology and rationale each have a distinct source of truth; load only the documentation needed for the current task.
96
+
97
+ - Prioritize working Pi outcomes, simple maintainable design and evidence-backed repairs. DSH source can inform a diagnosis, but installing/running DSH, matching its implementation or proving manual parity is not required. [ADR 0004](adr/0004-pi-first-acceptance.md) records this change.
98
+ - The user approved stronger window-lifetime tracking beyond the reference's raw HWND/PID checks. Design and validate lifetime-bound window references under #28 before choosing the mechanism and caller interface. Explicitly distinguish compatibility selectors from stronger references; lost identity proof must not silently retarget a replacement or bypass Windows access restrictions.
99
+ - Change provisional workarounds only when Pi evidence justifies it. Exact text, useful responsiveness, preservation and cleanup are the criteria; finding DSH corruption causes or the minimum possible per-character delay is not a release prerequisite. Record uncertainty honestly.
100
+ - Extend existing Pi fixtures and host test infrastructure rather than introducing a new framework or public test-only API. Investigate supported host interfaces before promising unattended model-driven execution.
101
+ - Do not add a separate planner, elaborate recovery controller or fixed production retry/time budget. Runner watchdogs bound tests without altering production policy.
102
+ - Reconcile documentation during each investigation/fix: replace stale instructions, remove duplicate authority, retain concise historical evidence, and update the index/links. Requirements, sequencing, results, topic guidance and decision rationale retain their existing distinct owners.
103
+
104
+ ## Default apps, workspaces and foreground
105
+
106
+ This is the accepted target behavior under [ADR 0005](adr/0005-default-apps-and-workspace-reuse.md); delivery and remaining acceptance are tracked in #30, with editor focus in #26. Earlier dedicated-profile acceptance does not establish this new workflow.
107
+
108
+ - Resolve the Windows default handler for the requested URL/file type when no app is explicitly named. An explicit user choice takes precedence. Do not silently substitute another app because its automation channel is easier; report unavailable control and use supported native UI where feasible. Opening the default app and supporting its browser protocol are separate capabilities.
109
+ - Discover the intended existing app window first. Create and verify a new task tab or document through its supported interface, preserving existing tabs, documents and unsaved contents. Reuse that task workspace for subsequent steps. Open a new window when the user requests one or no suitable window exists. Ambiguous targets or an app lacking a safe separate workspace require an explicit limitation or clarification rather than overwriting current work.
110
+ - Browsers follow the same policy, including the user's existing ordinary browser window. Dedicated temporary/kept automation profiles remain an explicit alternative, not an automatic replacement for reuse. Native UI operation of an existing browser does not imply CDP/BiDi attachment to its running profile. Do not change personal browser launch flags or copy, delete or take ownership of its profile to obtain protocol access.
111
+ - For a visible open/use request, activate and independently verify the intended window/tab, then keep the task target foreground across related steps. Intentional app/dialog transitions follow the task. Do not bounce to the caller after every tool call or apply delayed restoration after human handback. Explicit background work preserves the user's foreground; completion/handback leaves the requested result visible unless the user asked to restore another window. Windows activation refusal remains an explicit outcome; unconfirmed focus must not authorize input.
112
+ - Preserve target/lifetime, stop, exclusive-control and uncertain-input safeguards. Extension shutdown/idle/profile cleanup may close only resources it owns; reusing a user's window does not grant permission to close that window or unrelated tabs/documents.
113
+
114
+ ## Exclusive control
115
+
116
+ Accepted product decision, 2026-11-01; scoped control (#31) and presentation/display (#20) acceptance are complete. Installation acceptance (#2) is complete for the user-approved first-release scope; final release reconciliation (#21) remains separate. [ADR 0003](adr/0003-exclusive-desktop-control.md) records the deliberate departure from reference behavior. Desktop ownership excludes competing agents; exclusive control additionally suppresses ordinary physical user input.
117
+
118
+ - The first release uses low-level Windows hooks at normal privileges. The user explicitly accepts Windows' ability to silently remove timed-out hooks: suppression and physical stop-key availability cannot be guaranteed after such removal. Stronger OS blocking, elevation and an isolated desktop are not first-release prerequisites. This scoped exception qualifies the suppression/stop guarantees below; detected protection loss still stops automation, and all host, cleanup and recovery acceptance remains required. See [ADR 0003](adr/0003-exclusive-desktop-control.md) for rationale.
119
+ - Before the first physical action, establish independent stop protection, physical-input suppression and the visible local blue border. Continue protection across tool gaps, observations and model thinking until human handback or final settlement. Quiet-only UIA/browser operations do not acquire exclusive control; mixed workflows retain it once acquired.
120
+ - During control, ordinary physical clicks, pointer movement and typing must neither reach applications nor cancel or modify agent actions. Agent-generated input remains available. Esc and the configured stop shortcut always stop the operation; their physical events, including chord modifiers, must not leak into the target or become an ordinary-interference failure. Agent-generated Escape remains usable without triggering emergency stop.
121
+ - For credentials or other human interaction, stop dispatch, settle in-flight input and restore appropriate state before releasing suppression and returning control. Automation stays paused until explicit user continuation, then revalidates targets and reacquires protection. Emergency stop retains the existing same-operation rejection and new-user-operation recovery rules.
122
+ - Completion, stop, cancellation, setup failure and owner/listener failure must release physical-input suppression and any agent-held keys/buttons without replay. If release cannot be verified, report uncertainty and prevent further automation; release of user control and release of an uncertain automation ownership lease are distinct obligations. Preserve Windows secure-attention access; locking or leaving the interactive desktop prevents further dispatch.
123
+ - First-release lock handling is a safety requirement: refuse input while locked and stop interrupted control with cleanup, or report uncertainty and block further automation. Automatic continuation, target survival across lock/unlock and seamless post-unlock task recovery are not release requirements. Any later user-requested work follows ordinary fresh observation and control acquisition; reuse the accepted handback evidence.
124
+ - The blue border is required locally while exclusive control is held; an optional indicator setting or idle timer cannot silently hide it or release protection while automation continues. During handback, show that control has returned. Own capture may briefly omit overlays without releasing protection; preserve remote/recording preferences where Windows supports them.
125
+ - Suppression is not a substitute for target identity, foreground, occlusion, clipboard preservation, bounded cleanup or uncertain-input non-replay. If the mechanism cannot establish protection on a supported host, refuse physical dispatch with a clear limitation rather than silently allow shared input.
126
+
127
+ ## Testing Decisions
128
+
129
+ - Reuse the agreed tool/worker execution seams and integrated Pi entry points. Test external behavior and observable guarantees, not private implementation shape. No additional public test-only interface is required by this spec.
130
+ - Reuse accepted evidence across issues. An open issue is not a reason to repeat an accepted check; rerun an affected check when relevant behavior or its evidence collector changes, or a recorded result is invalidated. Group remaining human-only checks into prepared sessions and distinguish new host/mechanism coverage from repeated coverage.
131
+ - Test extension behavior through native Pi tool calls, including images, settings/tool denial and lifecycle. Test worker behavior through its normal request/cancellation interface, and prove both host launch paths end to end.
132
+ - Test every capability for valid/invalid arguments, supported success, relevant failure conditions and cancellation. A successful mock response cannot prove actual UIA, OCR, capture or input behavior.
133
+ - Use the reference's isolated regression suites, dedicated UIA fixtures, local browser pages, capture/input contracts, lifecycle tests and benchmarks as prior art. Reuse existing project fixtures and evidence through the [current implementation phases](implementation-plan.md#detailed-milestones) and [validation catalog](validation.md#alignment-scenario-catalog).
134
+ - Use Pi tool checks and real-agent outcome/recovery tasks. Record the effective Pi model and host, and separate product, model, environment and fixture failures using evidence. Retain three consecutive reset successes for required Pi task/recovery scenarios, preserve failures and rerun affected regressions. Optimization follows demonstrated correctness.
135
+ - Separate portable deterministic tests, live Windows fixtures, and suites emitting physical input. The user selected scoped disposable tests in the current Windows session, with standing idle-test authorization. Keep input scoped to owned disposable targets, preserve unrelated applications/documents and profiles, and serialize desktop activity. Cases whose isolation or cleanup requirements cannot be met remain blocked.
136
+ - Cover stale targets, incomplete observations, focus/occlusion, clipboard restoration, DPI/coordinate mapping, browser navigation/profiles, competing sessions, crash recovery, timeout non-replay and emergency-stop behavior. Exclusive-control acceptance follows the [bounded delivery plan](implementation-plan.md#exclusive-control-delivery); retain historical shared-input outcomes as historical evidence.
137
+ - Require automated regression and integrated-host suites plus recorded manual checks for visual indicators, physical stop shortcuts, display changes and interactive sign-in. The user performs additional acceptance testing.
138
+ - Measure Pi cold startup, warm observations, input dispatch and representative workflows as needed to diagnose responsiveness or regressions. Do not require DSH timings or remove safeguards to improve speed.
139
+ - Complete the Pi coverage matrix and capability-to-test mapping before declaring release readiness. Automated tests are necessary but do not establish every possible application/state combination.
140
+ - On 2026-11-06 the user accepted #6 screenshot-mapping closeout based on actual 100%/150%/175% mixed-DPI and changed negative-origin arrangement evidence; unavailable 200% custom scaling is not a first-release test gate. Other unexercised display configurations remain unverified, not inferred passes or a promise of universal scaling support. Target/coordinate safeguards are unchanged. See [accepted scope](validation.md#screenshot-mapping-closeout).
141
+ - First-release acceptance permits the two bounded #28 cases that could not be established in the owned environment: genuine same-HWND/same-PID replacement and verified Windows UIPI denial. The user accepted these test limitations on 2026-11-05; neither is a passing test or a waiver of identity/access safeguards. See the [recorded disposition](validation.md#pi-targeting-and-accessibility-reconciliation).
142
+ - On 2026-11-09 the user accepted #2 installation closeout based on second-computer WSL installation and actual computer actions, plus prior package/native/WSL evidence. Fresh native-Windows installation, missing/corrupt prerequisite handling and shutdown in that clean environment remain accepted unverified first-release test limitations; runtime safeguards remain required. See [installation receipt](validation.md#second-computer-installation-report).
143
+
144
+ - Separate direct Pi tool checks on reset fixtures from real-agent outcome tasks. Agent tasks permit supported recovery within intended disposable targets; method-specific diagnostics retain their constraints. Explicit stops, disabled controls and uncertain-input non-replay remain binding.
145
+ - Test the user's GPT-6.1 Sol low-reasoning Pi setup and record its effective configuration. Reproduce failed traces through direct calls where possible; leave unsupported cause attribution unknown. DSH/OpenRouter credentials and DSH host/model/manual runs are not required.
146
+ - Gather each scenario's runnable baseline, diagnose its observed failures and make targeted repairs with affected regressions. Whole-suite baseline completion is not required before diagnosing a known case; final reconciliation still accounts for every required outcome. Continue independent cases after confirmed cleanup/reset; block dependent cases after emergency stop or uncertain cleanup. A historical DSH result is optional context, not a current acceptance dependency.
147
+ - Validate current documentation and remove contradictions before a phase closes. Three reset successes do not erase earlier failures or establish universal reliability; existing manual release gates still apply.
148
+
149
+ ## Out of Scope
150
+
151
+ - DSH integration, dependency on other Pi extensions and a standalone MCP interface.
152
+ - Copying the reference engine or substituting managed Firefox for installed-Firefox compatibility.
153
+ - Project-specific settings overrides, non-Windows desktop automation and claims of universal application coverage.
154
+ - Administrator/UAC bypass, bundled AI models or credentials, and automatic replay of uncertain input.
155
+ - Claims of full DSH parity. Release claims concern the supported Pi capabilities and documented limits; milestones are incremental development checkpoints.
156
+
157
+ ## Further Notes
158
+
159
+ Version 0.1.0 is published with its scoped acceptance and documented limitations. The input-timing change below is merged into main for 0.1.1 with scoped acceptance recorded. Follow the [current outcome map and queue](tickets.md#delivery-outcomes-and-ownership), [input-timing delivery plan](implementation-plan.md#input-timing-delivery) and [validation receipts](validation.md#npm-publication-receipt). Each runnable scenario proceeds through baseline, diagnosis, targeted repair and affected regressions; final reconciliation and optimization follow the required outcomes. Pi functional and preservation requirements remain binding; removed DSH comparison gates must not be reintroduced through historical notes.
160
+
161
+ Issue tracker: [ycheung5/pi-computer-use](https://github.com/ycheung5/pi-computer-use). See [tracker configuration](agents/issue-tracker.md) for operations and [triage labels](agents/triage-labels.md) for status vocabulary.
162
+
163
+ ## Input timing and modifier taps
164
+
165
+ Status: implemented, accepted and merged through PR #44 for 0.1.1, 2026-11-10; [GitHub specification #40](https://github.com/ycheung5/pi-computer-use/issues/40). The following template is the canonical first-change requirement set. See the [composed receipt](validation.md#input-timing-and-modifier-tap-acceptance). These new parameters are not in the unchanged published npm 0.1.0.
166
+
167
+ ### Problem Statement
168
+
169
+ Pi users cannot tap Ctrl, Alt or Shift alone, request a timed key/button press, or adjust the fixed SendInput typing delay. Long requested input durations also conflict with execution deadlines inside the Windows worker and at the Pi boundary. Agents need explicit per-call controls that preserve existing behavior when omitted and retain stopping, cleanup and non-replay guarantees. This addresses input flexibility; the cause of the user's whole-task slowness has not been measured.
170
+
171
+ ### Solution
172
+
173
+ Extend the existing keypress, click and text-entry tools with standalone generic modifier taps and opt-in per-call timing. Every press/hold completes and releases inside one tool call. Agents can choose a typing interval, hold duration and execution timeout, including disabling that execution timeout. Keep the existing Pi extension and Windows worker; add no new tool family, MCP server or persistent held-input state.
174
+
175
+ ### User Stories
176
+
177
+ 1. As an agent, I want to tap Ctrl alone, so that applications can respond to a standalone modifier.
178
+ 2. As an agent, I want to tap Alt or Shift alone, so that modifier-only interactions are available.
179
+ 3. As a user, I want existing shortcuts unchanged, so that upgrading preserves current workflows.
180
+ 4. As an agent, I want to hold a supported key or chord for a requested duration, so that I can perform a long press.
181
+ 5. As an agent, I want to hold a supported mouse button at the selected target, so that I can perform a long mouse press.
182
+ 6. As a user, I want each held input released before its call completes, so that the model cannot leave a key down while thinking.
183
+ 7. As an agent, I want an explicit SendInput typing interval, so that I can adapt pacing to an application's behavior.
184
+ 8. As a user, I want the existing 50 ms typing interval when omitted, so that the compatibility baseline remains available.
185
+ 9. As an agent, I want zero typing delay where appropriate, so that I can request unpaced dispatch explicitly.
186
+ 10. As a user, I want no arbitrary duration ceiling, so that the extension does not guess how long my task should take.
187
+ 11. As an agent, I want a per-call execution timeout, so that I can choose how long to wait for that operation.
188
+ 12. As an agent, I want zero timeout to disable the execution deadline, so that an intentional long operation is not cut short by that deadline.
189
+ 13. As an agent, I want default deadlines to accommodate explicit timing, so that accepted long-duration requests can finish.
190
+ 14. As a user, I want Esc and cancellation to interrupt waits, so that increasing durations does not prevent stopping.
191
+ 15. As a user, I want input released after errors, shutdown or cancellation, so that failures do not leave controls held.
192
+ 16. As a user, I want uncertain cleanup to retain ownership and block dependent work, so that subsequent input does not compound uncertainty.
193
+ 17. As an agent, I want unsupported method/parameter combinations rejected before input, so that my requested method is not silently changed.
194
+ 18. As an agent, I want actual timing settings and dispatch status reported, so that I can distinguish configuration from observed task success.
195
+ 19. As a user, I want partial or stopped input never automatically replayed, so that changing timing cannot duplicate actions.
196
+ 20. As a maintainer, I want native Windows and WSL2 callers to share the same contract, so that host routing does not change timing semantics.
197
+ 21. As a maintainer, I want focused regression and owned-fixture checks, so that this update does not reopen accepted release tests.
198
+ 22. As a user, I want broader keyboard/mouse coverage delivered separately, so that the first change stays small and reviewable.
199
+
200
+ ### Implementation Decisions
201
+
202
+ - Extend existing input schemas, route validation, Pi integration, request/deadline handling and native input execution. Preserve the current module responsibilities and independent emergency-stop helper. No new dependencies, public test API, general scheduler or persistent down/up API.
203
+ - Accept one standalone generic Ctrl/Control, Alt/Option or Shift modifier in keypress. Existing supported ordinary keys/chords remain supported. Multiple modifiers without a main key are not newly promised. Existing aliases and malformed/duplicate-key validation remain applicable.
204
+ - Add optional holdMs to keypress and single click only. Omission preserves current immediate release; zero explicitly requests immediate release. For a chord, hold its pressed main key and modifiers together, then release all held inputs. For a click, press the selected supported button at the resolved target, wait, and release. Existing targeting, foreground intent and restoration rules continue to apply. No new auto-repeat guarantee is implied by a held key.
205
+ - Add optional inputDelayMs to type_text only with explicitly selected sendinput. Omission preserves the existing 50 ms pacing. Zero requests no intentional inter-unit delay. Keep current Unicode/newline/tab handling; the unit is the actual dispatched UTF-16 input unit, with the existing newline normalization. This is not a promise of exact application processing time or universal correctness at lower intervals.
206
+ - Add optional timeoutMs to type_text, keypress and single click only. A positive value supplies the requested execution deadline; zero disables that call's execution deadline through all relevant input execution layers. An explicit shorter timeout is honored, not silently extended, and may end partial input with the existing cleanup/non-replay result.
207
+ - With timeoutMs omitted, ordinary calls retain current deadline behavior. When holdMs or inputDelayMs is explicitly supplied, derive sufficient default execution budgets from the requested hold/pacing duration plus existing execution overhead allowances. Both the native-helper request and outer Pi-worker request must accommodate it; an inner 40-second or outer 90-second timer must not silently override the accepted contract.
208
+ - Timing arguments are non-negative safe whole milliseconds, without arbitrary product ceilings. Reject negative, fractional, non-finite, unsafe or otherwise unrepresentable values and unrepresentable derived durations before input. Long waits must not wrap, clamp to a short timer, or accidentally mean infinity because of an underlying API integer limit. Zero has the parameter-specific meanings above.
209
+ - Disabling the execution deadline does not disable explicit cancellation, shutdown handling, emergency stopping, target/protection checks or cleanup acknowledgments. Existing startup, target-resolution and cleanup safeguards remain independent. Timing is best effort, not a hard real-time guarantee.
210
+ - Validate backend/method capability before sending input. The first slice requires native foreground timed holds and explicit native SendInput pacing. A selected browser/background/quiet route must either honor the requested semantics or return a clear unsupported-combination error before input. Do not silently ignore a parameter, switch an explicit method, or escalate an explicit background request. Existing auto routing remains unchanged when new parameters are omitted; its resolution with new parameters must honor them or refuse before dispatch. This slice does not promise universal browser/background long-press support.
211
+ - Preserve exclusive control, visible local border, target identity/focus/occlusion checks, protection during tool gaps, clipboard preservation and human handback. Cancellation/error/shutdown must release agent-held input, or explicitly report unconfirmed cleanup and block dependent work. No automatic replay or forced termination of an uncertain input owner.
212
+ - Report the effective requested timing/deadline policy, whether input was dispatched or interrupted, and existing cleanup confirmation. A successful dispatch does not establish an observed application effect. Existing result fields retain their meaning.
213
+ - Keep the current Pi package and tool names. The published 0.1.0 remains unchanged; implementation and any later publication are separate work.
214
+
215
+ ### Testing Decisions
216
+
217
+ - Reuse the confirmed public Pi-tool and worker request/cancellation interfaces as the highest practical seams. Extend existing schema, input-preparation, text, pointer and worker/transport tests; do not add a new framework or public timing-test interface.
218
+ - Test externally visible validation, routing, effects and cleanup results rather than private state. Cover modifier taps, existing chords, omitted/zero/positive timing, representable large values and derived overflow, invalid values, unsupported routes and no input before refusal.
219
+ - Use existing controllable worker peers and owned fixtures to test omitted/positive/disabled execution deadlines at both request layers, a deliberately shorter timeout, cancellation during a long requested interval and release/ownership uncertainty. Use short deterministic simulations for very long values and timer-overflow cases; tests must not sleep for the full requested durations.
220
+ - Extend the existing native text and pointer fixtures for one grouped scoped Windows acceptance: standalone modifier down/up events, a timed key/chord and mouse press/release, explicit typing intervals with exact Unicode readback, default pacing regression, and stop/error cleanup of newly held inputs. Observe events/effects independently of returned dispatch success. Reuse existing stop/control evidence; repeat a physical Esc demonstration only if the changed mechanism or a failure specifically requires it.
221
+ - Check relevant Pi/WSL2-to-Windows and native request forwarding so no layer drops a field or restores a hidden default deadline. Keep these checks scoped to changed behavior; no new paid-model reset series, DSH comparison or unrelated display/browser/profile acceptance.
222
+ - Run configured portable checks and actual package/documentation validation for implementation. Record unrun live prerequisites and limitations honestly. Passing mocks do not establish real Windows input effects; exact-text verification is required before claiming lower pacing correct.
223
+
224
+ ### Out of Scope
225
+
226
+ Broader named-key coverage, Windows keys, left/right modifiers, numpad/media keys, mouse side buttons and modifier-click/drag are deferred to the second change. Also excluded: raw virtual-key/scan-code input, holds across calls, separate key-down/up tools, double-click/drag timing controls, privileged or device-specific integrations, secure-desktop bypass, changing default typing pace, global timeout removal, a new MCP adapter, universal device/application coverage and diagnosing or claiming a fix for unmeasured whole-task latency.
227
+
228
+ ### Further Notes
229
+
230
+ The user confirmed this first-change scope after choosing a two-change delivery. Existing 0.1.0 acceptance remains closed and reusable. This spec describes the implementation merged into main for 0.1.1; the unchanged npm 0.1.0 does not accept the new timing parameters. Readable key names remain the agent-facing interface. Required input release and uncertain-cleanup safeguards qualify every duration/timeout option. Future broader input coverage must be specified and accepted separately rather than silently added here.
@@ -0,0 +1,83 @@
1
+ # Emergency stopping
2
+
3
+ The candidate implements [exclusive control](spec.md#exclusive-control) in the existing independent helper. The scoped native/WSL host, held-input, handback and interruption results complete #31; broader presentation and capability acceptance stays with its separate owners; see the [candidate results](validation.md#exclusive-control-candidate).
4
+
5
+ Use `/computer-stop` to stop the current computer-use operation without waiting for a tool response. Pi cancellation also stops the operation. Queued actions are cancelled, active work is signaled, and subsequent calls in that operation are rejected with `COMPUTER_USE_STOPPED`. A model turn, retry or custom continuation does not reset stopping; a genuine new user run does.
6
+
7
+ Operation settlement (`operation_end`) remains tracked but is exempt from stop-triggered cancellation. It must finish input cleanup, disarm protection and release ownership even when queued behind the cancelled action; cancelling settlement itself can strand the cleanup acknowledgment. The actual-worker regression queues settlement before stopping a preparation-blocked action and verifies ownership release without dispatching input.
8
+
9
+ ## Current Windows listener
10
+
11
+ A small separate Windows process installs a low-level keyboard hook. It is armed by the internal `prepare_input` request and guarded native mutation paths, including background element scrolling; quiet reads and browser work do not arm it. Background element scrolling retains the cleanup lease while a synchronous provider call is outstanding, but never announces physical input to the presentation. Esc signals an operation-specific manual-reset Windows event directly, then notifies the worker. The listener never calls UI Automation and can signal while a native provider is blocked. During exclusive control, keyboard/mouse hooks consume ordinary physical input before application delivery. Physical Esc and the configured shortcut signal stop and are consumed; pressed keys/buttons whose down events were swallowed have their later releases drained. The helper tracks physical modifiers itself because suppressed key-down events do not update ordinary Windows key state. Status exposes aggregate blocked-event and held-input counts, without key/text logs. It also supports the personal extra shortcut and ignores injected keyboard events, so agent Escape actions do not stop themselves. See [indicator lifetime and preview](indicators.md).
12
+
13
+ If the listener process exits, the transport notifies the worker immediately, including during model thinking; the operation is blocked before later input preparation. Explicit native protection-loss/stop results also stop the operation. Hook handles do not prove that Windows retained hooks after a silent timeout. The user accepted this [first-release platform limitation](spec.md#exclusive-control); suppression and physical stop keys are not guaranteed after silent removal. Actual-host and scoped cleanup acceptance is recorded in the [#31 evidence](validation.md#control-interruption-acceptance).
14
+
15
+ The explicit stop control bypasses the worker request queue. The worker blocks the operation before cleanup, signals its listener/event if armed, cancels active and queued requests and retires the read-only desktop bridge. Retirement waits for confirmed child exit; failure to confirm exit retains the child handle and sticky failure, blocks replacement work and is not permission to release ownership. Worker stdin closure also retains ownership if prior stop cleanup was unconfirmed. If a worker exits after reporting active input without a cleanup acknowledgement, its client rejects replacement work instead of starting a new worker; this fail-closed state requires external investigation. Final settlement waits for this stop cleanup before releasing desktop ownership. Cancellation is not an instruction to replay any action.
16
+
17
+ ## Current physical-input protocol
18
+
19
+ Pointer and keyboard actions use the cancellation event and report `inputReady: true` with an explicit `readyActions` list. See [pointer guards](pointer.md) and [text/clipboard cleanup](text.md) for pending physical acceptance. Each enabled action must:
20
+
21
+ 1. Arm the independent listener, confirm exclusive-control acquisition and the required border before physical dispatch, and check the operation's cancellation event.
22
+ 2. Check cancellation between input steps and releases every held key/button in `finally`.
23
+ 3. Confirm input cleanup and physical-control release before retiring a native input bridge or releasing desktop ownership; an unconfirmed release retains ownership.
24
+ 4. Handle listener failure and timeout without silently replaying input or claiming success.
25
+
26
+ Before physical dispatch, the worker now sends a private request-scoped preparation token and waits for the controlling client to acknowledge it. The client records the active-input lease before acknowledging; a missing or mismatched acknowledgement cannot start input. Stop or stdin closure while still waiting cancels preparation without dispatch. The worker clears the lease before dispatch only if preparation was cancelled; after dispatch, only the native cleanup acknowledgement clears it. This private worker/client protocol is not a public tool parameter. A raw older worker-protocol client that does not acknowledge `input_prepare` will stall before physical dispatch until it cancels or closes stdin; bundled clients use the updated acknowledgement. The real-worker protocol has deterministic coverage against disposable fake native processes; actual Windows input and crash behavior remain separate acceptance checks.
27
+
28
+ A blocked read-only UIA/capture process can be killed after a bounded grace period. Four [known-held cancellation/target-error cases](validation.md#known-held-input-release) now passed; this does not establish release after abrupt death of an input owner. Uncertain input retains its ownership and prevents replacement work.
29
+
30
+ ## Validation
31
+
32
+ - `npm test` covers a busy disposable protocol peer, queued cancellation, same-operation rejection, custom-continuation rejection, Pi abort, new-user recovery and confirmed retirement of an unresponsive peer.
33
+ - After `npm run build`, `npm run test:stop` verifies real Windows hook registration, an initially unsignaled kernel event, signaling through the normal stop control, a distinct unsignaled event for the next operation, and worker rejection/recovery and a deliberately invalid pointer target refusal. It sends no keys or pointer input.
34
+ - The existing paired accessibility provider subset (`test:accessibility:report -- /path/to/reference --providers`) independently confirms the identified reader is alive after provider entry and absent after cancellation, before releasing its owned provider, followed by fresh-read recovery. This is native read-process cleanup, not held-input cleanup; see [provider evidence](validation.md#accessibility-comparison-evidence).
35
+ - A dedicated physical Esc run stopped busy typing in the Pi worker fixture and independently observed released keys; actual native Pi-host Esc subsequently passed; remaining WSL-host and held-input/control cases stay with #31. DSH host stopping is removed. Hook registration and explicit signaling are not reported as a physical-key test.
36
+
37
+ The [passive operation baseline](validation.md#passive-operation-evidence) records current results and reference source comparison. User-reported earlier Esc/command checks do not replace a recorded busy-input/held-key acceptance run. A rejected promise, hook registration or `cleanupConfirmed` alone cannot establish actual key/button release.
38
+
39
+ ## Held-key acceptance boundary
40
+
41
+ Neither Pi nor DSH exposes a public key-down/hold/key-up operation; each `keypress` dispatches a complete chord. Observing a modifier up after cancellation does not establish that it was held when cancellation arrived. The legacy Pi-only long-typing fixture overwrote the original clipboard without restoring it and could retire an unsettled input owner; its runner now refuses execution before starting a backend. A Pi long-typing cancellation case needs an independently observed input-settlement boundary before retiring its worker or fixture: Pi abort rejection can precede worker cleanup. Reference-client limits are historical, not a current test dependency. If cleanup cannot be confirmed, retain the owned processes/files and report uncertainty instead of killing a possible input owner or replaying input. Native Pi-host Esc and the [actual WSL Pi configured shortcut](validation.md#wsl-pi-physical-shortcut) are accepted; reuse these results. See [dedicated session evidence](validation.md#dedicated-physical-comparison-evidence).
42
+
43
+ ### Known-held input cancellation and target failure
44
+
45
+ `npm run test:stop:held -- --plan` lists four cases without built modules, files or Windows access. After building, run the same command without `--plan` through Windows Node with `PI_CU_DEDICATED_INPUT_DESKTOP=1` on the authorized idle desktop, under the shared live-test lock. The existing disposable editor fixture adds a passive observer for tagged agent events: a later Alt-down observes Ctrl already held, or a later pointer move observes the left button already held. It then synchronously signals the normal cancellation event or invalidates its owned target. It never suppresses input, adds production delays or exposes a hold tool.
46
+
47
+ Each case requires the held-at-trigger receipt, expected cancellation/target error, native cleanup acknowledgment, independent Windows input release, operation settlement, released control/held counters and worker exit. Uncertain input/control cleanup requests normal stop and retains the owner. These four cases passed in the [scoped evidence](validation.md#known-held-input-release); they do not prove cleanup after abrupt input-owner death.
48
+
49
+ ### Clipboard-free manual worker fixture
50
+
51
+ After `npm run build`, run `npm run test:stop:busy -- esc` on a dedicated, unlocked Windows desktop with `PI_CU_DEDICATED_INPUT_DESKTOP=1`. The runner waits up to two minutes at a start gate: once the user is ready, send `start` followed by Enter to the runner's stdin. After it reports typing has started, physically press Esc once within 40 seconds. Use `shortcut` instead of `esc` in a separate fresh run to test the fixture's isolated `Ctrl+Alt+F9` setting. Optional indicators are disabled in fixture settings; the required local border still appears during control. Add `--interference` for the grouped check: after typing starts, physically type `abc`, move the mouse and click inside the disposable editor, then press the requested stop key. The fixture requires aggregate blocked keyboard/mouse/button observations and no unexpected characters. Do not interact outside the disposable window. The fixture never reads or changes the clipboard.
52
+
53
+ When stdout is buffered, set `PI_CU_STOP_STATUS_FILE` to a new, nonexistent file in a private Windows-readable directory before starting the runner. Its JSON `phase` changes from `setup` to `ready`, then `started`; monitor it independently and relay the key prompt when it reports `started`. The caller sends the stdin start command only after the user is ready. The runner refuses to overwrite an existing status file.
54
+
55
+ The runner observes the disposable editor, the worker stop event, rejection of the stopped operation, acknowledged operation settlement, stable partial text, absence of unexpected characters, and released mouse buttons, modifiers, Esc and F9. It checks both independent Windows key state and the listener's physical-held counts; OS state alone cannot prove release of swallowed key-down events. After settlement it polls key state for up to five seconds to allow the user to release the physical shortcut; keys still held after that period fail the check. A pass also requires the input result itself to report `COMPUTER_USE_STOPPED` with confirmed cleanup; a generic worker cancellation or other input error is inconclusive. A one-millisecond wait checks stopped-operation rejection and fresh-operation recovery without enumerating or binding unrelated windows. If input settlement cannot be confirmed, it retains the fixture, worker and temporary files rather than terminating a possible input owner; manual investigation is required. A timeout sends the normal stop control once and remains a failed physical-key test. These results cover the Pi worker and listener, not a real Pi session, DSH host, or cancellation while a modifier is known held. `npm run test:stop:busy:check` checks the outcome evaluator without desktop input.
56
+
57
+ ### Passive registered-command fixture
58
+
59
+ After building, `npm run test:stop:pi-host -- command` selects the native-Windows no-input case; `--command-host wsl` selects a WSL2 Pi host with the same real Windows worker. Add `--plan` to either selector for a side-effect-free summary. The case uses installed Pi 1.4.0 and an isolated scripted provider. It uses the existing runner, real RPC host, production registered `/computer-stop` command and Windows worker; only health and wait tools are permitted. Hold the shared live-test lock for the entire run. No editor, window discovery, UIA, screenshot, clipboard or physical key is used. This mode does not substitute for the physical fixture's prerequisites below.
60
+
61
+ The finite sequence requests one 30-second wait, establishes an actual received/pending request plus independently observed ownership, submits `/computer-stop` through RPC, requires the actual matching worker wait response to report `COMPUTER_USE_STOPPED` before its original 30-second budget (same-process monotonic timestamps), gates later turns on its successful ownership-release acknowledgment, checks aborted settlement and same-operation custom-continuation refusal, then permits one fresh ordinary-user short wait under a distinct operation. A passive metadata-only Node preload observes existing worker protocol frames without introducing a worker/proxy owner or starting stdin flow early. It never rewrites or substitutes responses. Malformed/missing observer proof, deadlines or uncertain cleanup retain the actual host and keeper, deny further tools and authorize no kill, next reset or deletion. Reports stay in the printed owned temporary directory; successful cleanup removes only the isolated settings/agent directory.
62
+
63
+ Portable forwarding/callback, observer-fault and owner/tool guard controls run in `npm run check`; the existing keeper's fake-process retention checks remain binding. One independently reviewed native-Windows run on `73d4162` passed actual worker cancellation and release before continuation; [scoped evidence](validation.md#passive-operation-evidence) preserves that exact source. A separately reviewed WSL2 run on `e09b12c` also passed, with its own Linux-host/Windows-worker identities and [paired command evidence](validation.md#passive-operation-evidence). The native result remains pinned to its earlier source; it was not rerun or relabeled as the WSL candidate. Host identity and cleanup stay separate: Linux `/proc` start/executable identities never enter Windows PID queries, which start only from the Windows worker’s real returned PID. WSL-only relative settings, shared mapped cwd and child-only `WSLENV` forwarding keep settings/logs owned; worker metadata must confirm the actual cwd/override. This case cannot prove physical Esc/custom shortcuts, held-key/button release, DSH-host or real-model behavior.
64
+
65
+ ### Scripted native Pi host fixture
66
+
67
+ `npm run test:stop:pi-host -- preflight` runs on native Windows with no physical-input opt-in. It uses the same RPC launcher and default extension as the guarded physical fixture. Its scoped provider permits only health, one fixed invalid HWND/PID click and a pending wait; the click must fail before dispatch. A worker-only preload records protocol/child-lifetime metadata without arguments, environment values or typed text. The runner independently identifies the Pi host, Windows worker, stop listener, native bridge and ownership helper by executable, parentage, birth and native assembly role. It submits the real `/computer-stop`, requires the matching worker wait cancellation and operation-release acknowledgment, then closes Pi stdin and verifies normal exit and absence of all recorded participants. Failure requests cooperative stop before retaining uncertain resources. This proves orderly cleanup of recorded participants, not crash containment or held-input release. Keep the shared live-test lock throughout; no settings or agent directory is removed until successful exit checks.
68
+
69
+ `preflight host-exit` uses the same no-input sequence and participant checks, then terminates only the retained Pi host after the invalid pointer request has confirmed cleanup. It observes the worker/helpers exiting without forcibly ending them. Its receipt distinguishes this failure-path cleanup from orderly Pi shutdown; it cannot prove cleanup while native input is held or authorize forced retirement of a native helper.
70
+
71
+ `npm run test:stop:pi-host -- esc` requires native Windows, `PI_CU_DEDICATED_INPUT_DESKTOP=1` and both fresh same-launcher preflights in the same invocation. Their source fingerprint covers source, native assets, built modules, tests and the installed Pi CLI; changes before dispatch invalidate the gate. The current physical run must independently identify its own participants, confirm its invalid-target setup refusal/cleanup, active protection and border, then authorize one finite typing request through a single-use private gate. Its provider denies other mutations, changed targets and retries. This replaces the earlier unconditional refusal with tested prerequisites; it does not require a kill-on-close job or claim universal descendant containment. `npm run test:stop:pi-host:check` validates the evidence evaluator and prerequisite refusal controls without desktop input.
72
+
73
+ The passive job supervisor is not itself a safe physical-input supervisor: it kills job members if the supervisor dies. The no-input [crash boundary fixture](text.md#isolated-clipboard-diagnostics) separately demonstrates that the killed synthetic worker's managed `finally` does not execute, although the normal control's does. Pi's input paths release held inputs in `finally`; the synthetic result is a reason to retain the physical gate, not an observation of an actual held key or button.
74
+
75
+ The separate `npm run test:stop:pi-host:job` prerequisite requires Windows Node, a built package and `PI_CU_PI_CLI` pointing to the installed Pi CLI. A test-only supervisor starts suspended, enters a Windows job and launches the native Pi CLI with an isolated scripted provider. It sends no physical input, exercises health, list-windows and a wait, observes the distinct Pi host, worker and a process named `windows` in the job, then requires the job to be empty. The process name alone does not establish its executable path or parentage. Its temporary stdout may contain a window title and is removed on completion. Its synthetic provider and passive launcher do not establish containment of the physical stop launcher or independently exclude escaped/external descendants. `npm run test:stop:pi-host:job:check` checks the evidence evaluator without Windows.
76
+
77
+ The same fixture now supports a WSL2 Pi host when launched from WSL, with Windows Node compiling the owned editor and the editor reporting its actual Windows PID. Linux host/interop identities stay in `/proc`; Windows inspection starts from the traced Windows worker. Observer/settings overrides are mapped and forwarded only to the child. Actual WSL Ctrl+Alt+F9 [passed](validation.md#wsl-pi-physical-shortcut); native Esc is reused.
78
+
79
+ The physical fixture uses the installed project Pi SDK through the shared launcher, isolated settings and the stdin/status-file coordination above. `shortcut` selects its configured physical key. It requires a matching worker stop event and canonical input cleanup response, operation-specific release acknowledgment, stable partial text with no unexpected characters, independent OS key/button release and a fresh health observation showing suppression and physical-held counts released. A stop key reaching the editor fails the check. SDK abort alone cannot pass or authorize retirement. Only after those receipts does it close Pi stdin, verify normal host exit and absence of all recorded participants, close the owned fixture and remove its temporary resources. Timeout/failure requests normal abort; uncertain cleanup retains actual owners and files. Native Esc and WSL physical-shortcut results retain their named scripted-provider scopes; known-held release is covered separately above; DSH and additional model-comparison runs are removed.
80
+
81
+ The existing `test:stop:pi-host -- handback` case writes one `x`, proves normal settlement/control release, then waits at a human gate for `abc`. Explicit continuation is another ordinary-user turn; a successful fresh snapshot must precede Ctrl+End/one appended `x`, producing exactly `xabcx`. Cleanup and exact operation-release acknowledgments are required before human invitation and retirement. This [WSL Pi case passed](validation.md#wsl-pi-human-handback-and-continuation). The [control interruption acceptance](validation.md#control-interruption-acceptance) completes #31 using the narrowed safety scope. Reuse locked-input refusals and this handback result; do not require target survival or seamless unlock recovery. No further human lock step is queued without a proven evidence gap.
82
+
83
+ `npm run test:stop -- --desktop-loss` runs with Windows Node under the shared live-test lock. It claims the production desktop mutex and compiles a disposable helper copy with only an availability-failure guard added to `InteractiveDesktop`; real OS checks and the native timer/stop/release paths remain intact. It injects no input and does not lock Windows. The available-state control, automatic stop/cancel/control release, same-operation refusal, disarm and helper exit passed. This complements the real locked-refusal, worker and held-input receipts; it does not claim an actual OS-lock-during-held-input run.