@ycheung5/pi-computer-use 0.0.0-stage → 0.1.0

package/docs/windows.md

@@ -0,0 +1,171 @@
+# Windows discovery
+
+## Screenshots
+
+`windows_computer_use_screenshot` accepts native window identity/title/process targeting, `scope: active_window | desktop`, `maxWidth` (0 means original size), and optional explicit activation. Capture does not resolve UIA elements. PNG data is returned as a Pi image block; text and rendering details contain metadata without duplicated base64.
+
+For native window capture, opt in to `windowOnly: true` to prohibit screen-copy and desktop fallback before acquisition. This mode uses only `PrintWindow` or lifetime-validated HWND-bound WGC; it can capture a covered window without reading the covering app. Hidden/minimized targets return `WINDOW_CAPTURE_UNAVAILABLE` without an image; failed WGC also returns failure without a fallback image. It rejects `scope: desktop` and `browserTab` combinations. Omitting the option or setting it false retains the capture chain below. The opt-in may need the slower render/WGC path even for a clear foreground window. Snapshot and OCR contracts are unchanged.
+
+Prepare the display checks with `npm run test:native:capture -- --display --plan` and `--control-capture --plan`; these plans require neither compiled code nor Windows. After building and verifying Windows-readable dependencies, use Windows Node to run the configured runner with `--display` or `--control-capture`, under the shared live-test lock. Both modes use disposable settings and strict owned-window screenshots. The control case briefly holds control without dispatching input and samples only guarded owned edge points. It proves the exercised render path, not every screen-copy/OCR/recording path. The commands record actual DPI and leave Windows display settings unchanged; [current evidence](validation.md#display-and-border-baseline) distinguishes 100% runs from pending higher/mixed-DPI setup.
+
+The capture chain prefers verified unobstructed foreground pixels, then `PrintWindow`. A failed or apparently blank render tries an isolated Windows.Graphics.Capture helper using HWND interop and a free-threaded frame pool. The helper has bounded frame/copy polling plus an external worker deadline, so unavailable WinRT metadata and stalled capture do not disable ordinary window enumeration. A final screen-region fallback is marked `occludedPossible: true`. Hidden/minimized/unavailable target surfaces return a desktop fallback marked `windowCaptureFailed: true`; callers must not treat it as the target's content. Capture errors and fallback warnings remain explicit.
+
+If a visible window's `PrintWindow` render fails and the following screen-region copy throws a Win32 error, capture can still pass its validated private target to WGC without a native image. Successful recovery returns the actual WGC PNG and retains `nativeCaptureError`; if WGC is unavailable, the result remains a failure with both stage diagnostics and no fabricated image. Identity loss, cancellation and unconfirmed cleanup do not authorize this handoff or a successful fallback.
+
+`bounds` and `origin` are physical screen coordinates, including negative virtual-desktop positions. `width`/`height` describe the actual PNG. Map image X using `origin.x + imageX / imageScale` and Y using `origin.y + imageY / imageScaleY`; a separate Y scale accounts for height rounding during resizing. Target identity and geometry are checked around capture. No tool automatically activates a window unless `activate: true` is requested.
+
+Run `npm run test:native:capture` with Windows Node after building. Its dedicated passive fixtures verify actual green pixels behind a magenta cover, PNG dimensions, coordinate mapping, wrong-owner refusal, a black-frame-triggered WGC fallback and explicit minimized-target metadata. Screenshots remain in test process memory; the suite sends no physical input. The existing paired runner's `--scenario monitor-placements` now covers primary, real negative-X and boundary placements on two 100% monitors, with native/scaled PNGs and independently measured physical DWM frame geometry, exact bounds/origin checks and documented reference scale precision; `--plan` is safe without a backend. See [focused mapping evidence](validation.md#current-negative-origin-screenshot-mapping). 150/200%, mixed-DPI transitions and changed display/primary topology still require suitable recorded validation rather than being inferred from resizing or window movement. The [paired capture/OCR baseline](validation.md#paired-capture-and-ocr-comparison) additionally compares the pinned reference through public interfaces and retains its minimized-caption and fallback differences; those findings do not authorize changing Pi to reproduce misleading capture evidence.
+
+WGC uses the documented [CreateForWindow interop](https://learn.microsoft.com/en-us/windows/win32/api/windows.graphics.capture.interop/nf-windows-graphics-capture-interop-igraphicscaptureiteminterop-createforwindow), [free-threaded frame pool](https://learn.microsoft.com/en-us/uwp/api/windows.graphics.capture.direct3d11captureframepool.createfreethreaded), and [memory-buffer byte access](https://devblogs.microsoft.com/oldnewthing/20240122-00/?p=109304). A composited frame can legitimately be black; only the unreliable PrintWindow path uses black-frame detection to choose a fallback.
+
+## Implemented scope
+
+`windows_computer_use_list_windows` reads native top-level windows without resolving UI Automation elements or sending physical input. Options match the reference: `includeInvisible` defaults to false and `maxWindows` defaults to 50, with an integer range of 1–200. The default excludes hidden and minimized windows.
+
+Results include a window ID with HWND/PID ownership, name, class, bounding box, enabled/offscreen/focus state, native source and bounded responsiveness information. `truncated` indicates the limit stopped enumeration. A denied responsiveness probe is reported as `responsive: null` and `responsiveness: access_denied`; an unsuccessful probe is not proof that an application is permanently hung. Enumeration rechecks owner PID before returning a record, but its result remains an observation rather than a guarantee that the window still exists at a later action.
+
+Worker request envelopes and listing arguments are validated at execution, independently of Pi's schema. Invalid requests with usable IDs receive failures; unidentifiable malformed requests produce stderr diagnostics without crashing later valid requests.
+
+## Native bridge
+
+TypeScript owns dispatch and lifecycle. A focused C# bridge performs Win32 enumeration on an MTA thread through a persistent JSON-line process. The package contains fresh bridge source, compiled lazily with the Windows .NET Framework64 compiler into a source-versioned cache below `%LOCALAPPDATA%\pi-computer-use\native`. No separately installed .NET SDK is required on the tested Windows 11 baseline. Missing compiler/data-directory and compilation failures are explicit; cache and compilation are not assumed to work on untested Windows configurations.
+
+Keeping the helper persistent gives subsequent accessibility slices a place for apartment-owned element references. Helper stdin closure ends it; worker disposal closes its transport. The native helper has an execution watchdog, and cancellation rejects a request without replaying it. [Desktop ownership](ownership.md), [stopping](stopping.md) and [guarded pointer actions](pointer.md) now extend this bridge; discovery itself sends no physical input.
+
+## Validation
+
+- `npm run check`: includes invalid listing options, malformed worker envelopes, continued health execution after errors and Pi tool registration/dispatch.
+- `npm run build`: produces the worker modules; package contents include the native bridge source.
+- `npm run test:native:windows`: run with Windows Node after building. Creates only dedicated passive fixture windows, tests visible/hidden filtering, identity metadata, bounds, responsiveness, limits and window disappearance, then closes fixtures and disposes the worker. It does not send keyboard/mouse input or modify normal applications.
+
+Live proof on 2026-10-29 used Windows 11 x64 through WSL interop and the private package-managed Node runtime. Both portable worker/Pi tests and the real fixture test passed. After integrating the WSL2 launcher, a Linux-side worker client also enumerated one native window through that launcher and disposed successfully; only counts were logged. Per-monitor DPI awareness is requested before enumeration; the later [focused mapping check](validation.md#current-negative-origin-screenshot-mapping) covers two-monitor/negative-X geometry at 100%, with other DPI and topology cases still open.
+
+## Default apps and workspaces
+
+`launch_app` accepts HTTP(S) URLs and existing fully qualified document paths in addition to explicit executables, shortcuts and installed app identities. `target: "browser"` selects the Windows HTTPS default browser without inventing a page URL. `resolveOnly: true` returns `verification: resolved`, `launched: false` and available handler metadata, without launching or enumerating new windows. For explicit apps the resolved target identifies the executable/app; associated URLs/documents include `handler` with association, executable, display name and ProgID. Metadata may be incomplete for packaged handlers and is not a window identity.
+
+Resolve the handler before selecting an intended existing window; create a separate task tab/document through its observed UI and preserve original work. An actual open is delegated once to Explorer's shell, allowing the selected default app to handle the URL/document. It is reported as dispatched, not proof of a new window/tab. Unknown associations refuse with `DEFAULT_APP_UNAVAILABLE` rather than using generic Open With or another browser. Blank default-browser launch requires an executable; an unavailable executable is an explicit limitation. Associated targets accept no command arguments/working directory; script/installer files and command/script-host handlers remain refused. Arbitrary protocol schemes remain unsupported. Resolution uses Microsoft's [AssocQueryStringW](https://learn.microsoft.com/en-us/windows/win32/api/shlwapi/nf-shlwapi-assocquerystringw), including current-user protocol association and unknown-handler refusal flags; no default association is changed.
+
+Confirmed native activation returns its lifetime reference and establishes visible intent for this computer-use operation. Related native actions resolve their actual root before inheriting that intent, including element-only actions; explicit `activate: false` or background dispatch/method takes precedence. Passive observations do not inherit activation. A helper restart or new operation does not inherit old intent. Existing target/lifetime and input safeguards still apply; activation refusal is reported before control dispatch. This is operation context in the existing native helper, not a separate planner or an additional UIA lookup.
+
+Run `npm run test:native:management -- --workspace` with Windows Node under the shared live-test lock after building. The owned fixture resolves the actual HTTPS default without launching it, uses a private GUID extension/ProgID to exercise Explorer document opening, verifies a new task tab and unchanged original document, switches to an owned caller to test foreground retention, checks explicit background preservation and a separate-window request, then verifies borrowed-window survival across worker shutdown. The private registration rolls back partial setup; fixture entry records its PID, and cleanup verifies executable identity before retirement. No real default associations, personal documents, clipboard or physical input are used. The existing `npm run test:native:chromium -- --workspace` additionally exercises native New Tab invocation and original-page/editor preservation in a normal window of the resolved Windows default Chromium browser, with a disposable profile and no protocol connection. [Evidence and remaining scope](validation.md#default-app-and-workspace-acceptance). The real-Pi companion `npm run test:pi:workspace` prints a safe plan. `--preflight` checks owned fixture readiness/cleanup without a model; `--run --runs 3` requires explicit model-usage authorization and unlocked Windows. See [actor evidence and acceptance limits](validation.md#ordinary-pi-workspace-actor-preparation). The same companion supports `--scenario ambiguous` and `--scenario unsafe-workspace` with owned native app fixtures; [each passed three independently reviewed agent resets](validation.md#workspace-decision-acceptance). Future refusal invocations permit one episode before semantic review; structured observation checks never automatically accept the agent’s decision.
+
+## Target resolution
+
+Listing establishes identity metadata; targeted observations and actions recheck that identity. The native accessibility fixtures exercise ambiguous titles, missing/wrong-owner targets and removed/unseen references through the public interface. The [paired accessibility baseline](validation.md#accessibility-comparison-evidence) compares these outcomes with DSH. Inaccessible/elevated providers and additional replacement cases remain release checks. No test-only target-resolution API is required. Browser-tab discovery is described in [browser control](browser.md).
+
+### Window lifetime references
+
+Native observations return an opaque `windowRef` bound to a unique private property on the observed window. Retain it when targeting later native reads/actions. Explicit HWND/PID/title selectors and `uia:hwnd` IDs remain weaker compatibility selection: those numbers alone cannot distinguish same-process handle reuse. A supplied reference is never discarded, restamped or converted into a weak selector after failure. Additional selectors must agree with it; desktop scope and browser-tab targets cannot use it.
+
+Enumeration binds before retaining window metadata and validates that same witness before emitting its row. Lost proof omits the unsafe row with explicit incomplete observation; the same pass never restamps it. Internal metadata-only enumeration does not bind.
+
+The persistent native bridge is the sole authority. Native-child restart, expiry or unknown references require fresh observation. A missing/mismatched marker means lost proof, including when HWND/PID still match; it does not prove the window disappeared. Window waits remain inconclusive in that case, including when a destroyed window leaves a requested title constraint unverifiable. References do not grant desktop ownership or bypass Windows UIPI. Marker acquisition failure preserves read metadata and `windowRefUnavailable`, without issuing actionable native control IDs.
+
+Issued native element IDs are opaque, immutable bindings to the observed root reference and cached UIA object; RuntimeId is comparison metadata. A fresh observation cannot overwrite an old ID with a replacement root. Responses carry the root token once rather than repeating it on every tree node. Browser IDs keep their existing behavior.
+
+Window captures, snapshot `captureWindow: true`, OCR/hit tests and guarded fallback input retain the originating reference. Snapshot's default image is separately labeled desktop evidence, without inheriting the tree's window token. WGC receives only a private stateless marker descriptor from the authority; it does not resolve public tokens. Implicit last targets preserve a known reference. Raw window waits and internal launch bookkeeping enumerate metadata without stamping unrelated windows; new-window launch metadata is explicitly unbound until selected/observed.
+
+Validation occurs at existing preparation/dispatch checks and between steps. If proof is lost after an action attempt, results retain `dispatchAttempted`, `effectsUncertain` and dispatch verification, refuse automatic fallback/replay, and preserve cleanup failures. A posted close retains its truthful sent receipt if the intended root is observed absent; it does not infer whether an application accepted closure or displayed a save prompt. Key/button release remains independent of target validity. Validation cannot atomically pin a foreign window through a later action or undo already-dispatched effects.
+
+Observations may write a small namespaced scalar window property without activating the app or editing its document. Expiry/disposal removes only the authority's own markers. Abrupt native-child exit may leave scalar residue until window destruction; new authorities never adopt it or sweep other sessions' properties. See [ADR 0002](adr/0002-window-lifetime-references.md) and [scoped acceptance](validation.md#window-lifetime-reference-acceptance). Actual identical-HWND reuse and verified UIPI denial remain unverified, with [user-accepted first-release test limitations](validation.md#pi-targeting-and-accessibility-reconciliation); different-handle replacement or synthetic failures are not substituted as passes.
+
+## Accessibility observations
+
+`accessibility_tree`, `snapshot`, `find` and `element_info` are registered Pi tools backed by a persistent MTA UIA bridge. The worker validates options before native execution. Tree traversal bounds depth, nodes and elapsed time; a synchronous provider can still block, so the native process has a 40-second watchdog and the extension permits the maximum 30-second traversal budget plus cleanup. Truncated or failed observations remain explicit.
+
+Like the reference, `includeOffscreen: false` filters the projected UIA view: an offscreen ancestor is omitted while its visible descendants remain discoverable. It does not prune that ancestor's entire subtree. The owned provider regression covers control/raw/content tree and find behavior; [Notepad evidence](validation.md#semantic-save-and-visible-descendant-discovery) records the real menu gap that motivated this correction.
+
+Issued native element references are held for five minutes, bound to their observed window lifetime and checked against the retained UIA object and current RuntimeId. Unseen, removed and wrong-owner references require fresh observation. Full detail can include a supported value, except password-field values are never read. Native window IDs also validate HWND/PID before UIA access. Title/PID selection refuses ambiguous windows; explicit handles are preferred.
+
+Chromium observations probe renderer accessibility and report `webContent` readiness. A first read can return `starting`; callers should observe again rather than infer that absent page controls do not exist. Unnamed browser layout panes do not consume logical depth. This has been exercised against installed Edge without a forced-accessibility launch flag; other Chromium integrations require their own release checks.
+
+`npm run test:native:accessibility` (Windows Node after build) opens dedicated passive forms and verifies structured trees, values/password protection, find, element inspection, title ambiguity, missing/wrong-owner targets, removed/unseen references and depth/node truncation. `npm run test:native:chromium` opens a disposable visible Chromium profile and local page, reads its page button after lazy initialization, and closes only that browser. Neither test sends physical input; they are live desktop tests, not portable substitutes.
+
+The existing `benchmark:accessibility` collector also supports `--native-chromium-only` (`--plan` lists without loading a backend). Its isolated Edge fixture now verifies native UIA logical focus, ValuePattern Unicode writing and InvokePattern activation against pinned DSH, with `fallbackType: false` / `fallbackClick: false`. The page independently reports its actual value, active control and click count; it never simulates interaction. Each engine gets a fresh profile/page. Both engines passed the scoped actions; unavailable foreground observations do not establish restoration. See [action evidence](validation.md#native-chromium-uia-action-acceptance). Genuine same-process HWND reuse, inaccessible targets and broader provider/action families remain open.
+
+Live evidence: Windows 11 x64, .NET Framework bridge, native form fixtures and installed Edge passed on 2026-10-29. Elevated/UAC target access remains a release case. Actual five-minute native root/element-reference expiry is covered by the optional [owned-fixture case](runtime.md#verification), with unchanged helper identities and no effects from the expired ID. Snapshot now includes a separate PNG by default, preserves explicit UIA failure when an image is attached, and honors `includeScreenshot: false`. Like the reference, `captureWindow: false` defaults to desktop pixels; `captureWindow: true` captures the selected window. Sparse empty-find OCR composition is implemented as described in [OCR and sparse accessibility fallback](#ocr-and-sparse-accessibility-fallback). Browser references route through the implemented [browser channel](browser.md); native UIA success does not establish browser-channel acceptance.
+
+## Background controls
+
+`focus`, `invoke`, `set_value` and element-targeted `scroll` operate only on fresh observed references and validate the requested owning window. Native Win32 buttons use background button messages, native edit fields use selection/replacement messages without clipboard or keyboard input, and scrolling prefers the nearest native scrollable control before UIA ScrollPattern. Element scroll uses Windows wheel-lines/page settings and the reference's midpoint-to-even rounding, vertical then horizontal dispatch, and 100-step line-mode cap; page mode retains the reference's uncapped count. The worker reuses its stop-event and cleanup lease without showing physical-control presentation. It checks stop between calls and after each provider return; an already-dispatched remote app call may settle after caller cancellation. Desktop ownership remains held until settlement/cleanup acknowledgment. Other controls use their supported UIA patterns. Expand/collapse is idempotent; scroll-into-view uses ScrollItemPattern.
+
+A shared native activity guard observes the whole call using mouse/keyboard hooks on its own message-pump thread. It excludes only the helper's tagged SendInput and expected pointer moves; intervening physical or foreign injected input prevents restoration. Foreground can return only when the exact intended root is still in front, activity is unchanged, and no new menu/dialog needs it. Another pre-existing top-level window in the same process is left selected. Background/keyboard calls never return a changed pointer just because it is inside a field: without proven movement ownership they leave it alone. Windows refusal and uncertainty are reported in metadata.
+
+Results report `verification: sent`, not task completion. Use an explicit observation/wait to verify effects. Read-only, disabled, stale and wrong-owner controls fail explicitly. The quiet native stage returns `WCU_NEEDS_INPUT` with `inputRequired: true` when a requested action needs a permitted fallback; the worker may then execute that fallback through the guarded-input path. Use `fallbackClick: false` or `fallbackType: false` where available to forbid it. Native `scroll` has no public no-fallback flag, so unrestricted passive comparisons defer it. The owned, separately gated background-scroll fixture checks the actual dispatch method and independent input state; a timeout, cancellation or partial dispatch never authorizes fallback. Guarded pointer behavior and outstanding physical acceptance are described in [pointer actions](pointer.md).
+
+Run `npm run test:native:nullable-uia -- --quiet-scroll` with Windows Node after build for the owned ScrollPattern cancellation case. It holds the first provider call, cancels through the public worker, confirms a competitor stays busy, then releases the provider and proves only that already-dispatched call settles before ownership release. It sends no physical input or clipboard actions; it cannot undo the in-flight app effect.
+
+Run `npm run test:native:controls` with Windows Node after build. It creates a dedicated form, proves Unicode native editing, button dispatch, focus, actual element-scroll state change, read-only refusal, unsupported-pattern/fallback distinction and wrong-owner/stale rejection through registered Pi tools, then closes the fixture. It sends no physical input. Expand/collapse across provider families and menu/dialog/user-interference restoration require additional release fixtures/manual cases; this test does not claim universal provider coverage.
+
+Operation identity reaches the native bridge; the operation target/focus contracts are described in [pointer](pointer.md) and [text](text.md) entry. The installed package requires every registered native capability source; a missing source fails clearly with `NATIVE_RESOURCE_MISSING` rather than silently building an obsolete partial bridge.
+
+## OCR and sparse accessibility fallback
+
+`windows_computer_use_ocr` captures through the same screenshot module, then recognizes that PNG in a separate WinRT OCR process. Options preserve the reference target/scope conventions, `maxWidth: 1920` (0–7680), optional literal `query`, `resolveControl: false` and `includeScreenshot: false`. Output lines/words and matched union boxes use physical screen coordinates, applying both horizontal and vertical image scales. Text matching ignores case and whitespace, supports contiguous CJK word runs, treats regex characters literally, and returns at most three line matches.
+
+OCR candidates have `source: ocr`, recognized text, bounds and centers—not invented control IDs. Only explicit `resolveControl` performs a UIA hit-test; it may describe an enclosing document rather than a semantic button. Hit-test failures stay explicit. A capture flagged as potentially occluded cannot establish target text absence or control identity. Missing query results are always inconclusive evidence about the actual interface. Minimized/hidden-target desktop fallbacks are refused as OCR target evidence.
+
+Empty `find` observations with sparse (at most 20 nodes) or incomplete UIA trees attach OCR candidates by default, except when `controlType` is specified or `ocrFallback: false`. The UIA result remains separate from OCR and retains UIA errors/truncation. OCR unavailability does not manufacture a matching element.
+
+The package compiles fresh OCR helper source against installed Windows WinMetadata and .NET Framework assemblies. Recognition uses user-profile OCR languages and reports the actual language plus available recognizers. Missing languages/metadata, image-size limits, native recognition failures, timeout and cancellation remain explicit. The OCR helper is isolated from UIA and retired after cancellation; requests are never replayed. Microsoft documents [Windows.Media.Ocr desktop support as requiring package identity](https://learn.microsoft.com/en-us/uwp/api/windows.media.ocr), although the unpackaged helper works on the tested Windows 11 machine. Clean-machine availability remains a release validation requirement, not a universal compatibility claim.
+
+Run `npm run test:native:ocr` using Windows Node after building. A dedicated custom-painted fixture proves actual recognized text, query normalization/literal behavior, three-result limits, image-to-screen mapping across scales, optional screenshots, explicit control lookup, sparse-find fallback and minimized-target refusal. It sends no physical input and does not capture normal applications. Portable contract tests cover validation, pre-cancelled calls and Pi registration. Live OCR evidence on 2026-10-29: Windows 11 x64 with profile recognizer `zh-Hant-HK` recognized fixture English text. The [focused failure run](validation.md#pi-ocr-failure-acceptance), `npm run test:native:ocr -- --failures`, covers missing resources, an injected null-engine outcome, actual profile-language reporting and cancellation after recognition dispatch with helper exit and fresh-operation recovery. It uses a disposable package copy and makes no installed-language changes. Actual clean-machine availability remains #2, mixed-DPI mapping #6; protected-image compatibility is not claimed. The [paired baseline](validation.md#paired-capture-and-ocr-comparison) now records native/scaled localization against independently painted ink, literal misses, clipped recognition and minimized results. Its measured offsets and recognition differences are diagnostics, not a claim of exact OCR localization across applications.
+
+The same Windows runner has an optional fixture-only association mode: set `PI_CU_OCR_VISUAL_ASSOCIATION=1` in the Windows Node process. Two native buttons expose blank accessible names while painting randomized APPROVE/IGNORE codes. The public OCR result is associated with an observed button ID through bounds; background invocation disables physical fallback, and private fixture counters verify APPROVE once and IGNORE unchanged. Default HELLO behavior is unchanged. The [direct preflight and scoped model evidence](validation.md#visual-ocr-association-preflight) distinguish OCR tool checks from actual visual-to-semantic invocation.
+
+## Explicit waits
+
+`windows_computer_use_wait` performs a cancellable delay (`milliseconds: 500`, integer 1–30000) and makes no verification claim. `wait_for` polls `window`, `element` or literal OCR `text` with `appear: true` by default. Window waits require explicit title/PID/HWND targeting; element/text waits require a nonempty query. Defaults are 5000 ms overall, 250 ms between observations, and a 1000 ms UIA traversal budget. Overall time is 100–60000 ms, interval 50–5000 ms, and reference depth/node/view options apply to element observations.
+
+A satisfied predicate returns `verification: observed`; a complete observation that did not satisfy it before the deadline returns `timeout`. Truncated, unavailable, timed-out or potentially occluded evidence returns `inconclusive`. UIA positive matches can establish presence even when another part of the tree is truncated; truncated absence cannot establish disappearance. Window existence uses native enumeration including invisible windows, bounded at 4096 entries; a truncated enumeration cannot prove absence. Target HWND/PID matching checks the current native values, subject to the window-lifetime limitation above.
+
+Text waits concern literal OCR output from a complete reliable capture, rather than universal truth about application text. OCR exposes `captureConclusive` separately from query-hit certainty so a reliable image without a recognized sequence can satisfy an explicit disappearance predicate, while potentially occluded or failed capture cannot. Native recognition failure never establishes absence. Polls avoid activating targets and disable UIA's OCR fallback for element predicates.
+
+Polling bounds each observation by the remaining deadline and retires a stuck native observation worker; a new provider call is not started with less than 100 ms remaining after a prior observation. Cancellation propagates through the common operation stop path. Valid explicit waits receive a bounded transport allowance (delay plus 2 seconds; condition timeout plus 10 seconds) rather than being cut off by a shorter ordinary request watchdog. Queue time is still outside execution time.
+
+Run `npm run test:native:waits` with Windows Node after building. Its passive fixture proves window existence/disappearance, actual control transitions, complete missing-condition timeout, truncated-absence refusal, OCR text appearance/disappearance and prompt cancellation. Portable worker tests cover argument bounds, plain waits, unavailable native observations and deadlines shorter than a valid explicit wait. Mixed-DPI/protected/occluded target cases remain covered by capture/OCR release validation rather than inferred from this passive fixture.
+
+## Launching and managing windows
+
+Pi guidance prefers reusing the intended existing application window after discovery. When a separate workspace is needed, prefer creating a tab/document through the observed application UI, preserving existing documents and unsaved work. Launch only when no suitable window exists or the user explicitly requests a new instance. This is agent guidance rather than a universal tab API: applications differ, and shell launch does not guarantee a new process, window or tab. Ambiguity and activation refusals still require resolution. Only explicitly selected dedicated browser-channel work uses its isolated profile/automation tabs; ordinary browser work follows the [default-app policy](#default-apps-and-workspaces).
+
+`launch_app` resolves a fully qualified executable/COM/shortcut path, a bare executable through PATH or registered App Paths, or an exact installed app display name/AppUserModelID. Its `arguments` and `workingDirectory` are passed as data to the Explorer desktop's ShellExecute automation object, not interpolated into a shell command. Launch does not create the application as a worker child, so the application survives tool completion and worker exit. There is no child-process fallback when the interactive Explorer desktop is unavailable, no requested elevation, and no automatic replay after uncertain dispatch.
+
+HTTP(S) URLs, the default-browser alias and associated documents follow the [handler rules above](#default-apps-and-workspaces). Explicit app-file launching retains exe/com/lnk support. Existing fully qualified working directories are required when supplied; installed-AppUserModelID and default-handler launches reject arguments/directories. Command interpreters and script hosts accept no arguments through this tool—use Pi's ordinary shell tool for commands. Shortcut target chains are inspected too, including embedded shortcut arguments, and cycles/deep chains are refused. `activate: false` requests no activation but an application may activate itself. `waitMs` defaults to 1500 (0–10000); new-window observations are not confirmed matches and may be incomplete. Launch reports `verification: dispatched`, never that the user's objective succeeded.
+
+`activate_window`, `move_window` and `close_window` require explicit HWND/PID, title or process targeting and use Win32 identity checks rather than UIA element resolution. Ambiguous titles/processes are refused; observed HWND/PID should be preferred. Activation restores a minimized target and requests foreground through native APIs without an Alt/keyboard workaround. If the initial request is refused, the helper attaches to the foreground input thread, raises the target with BringWindowToTop, rechecks window ownership, and requests foreground again. It verifies the resulting foreground HWND rather than treating the API call as task success. Unresponsive/inaccessible targets and Windows foreground refusal report `activated: false` and `unconfirmed`. Movement accepts physical screen X/Y, does not resize, and avoids activation unless explicitly requested. A successful move is dispatch evidence; observe fresh bounds before subsequent coordinate input. Close posts WM_CLOSE without activation, and an application may veto it or display a save prompt. Accepted posting is not proof that the window closed.
+
+When an activated accessibility observation or physical text or pointer input receives `TARGET_ACTIVATION_REFUSED`, `activationDiagnostic` records coarse foreground relationships and request outcomes when available. The relationships describe the observed window as the target, a target child, another window in the same process, another process, or unavailable; no window title or content is included. These sequential samples can change and do not establish why Windows refused activation. A refusal means activation was not confirmed; some read-only work may already have occurred. Do not infer that a later attempt will succeed.
+
+`npm run test:native:management` (Windows Node after build) compiles only a dedicated disposable fixture, launches it through the real Explorer broker, exits the first worker and proves the app survives, observes its new position through `list_windows`, exercises a close veto and subsequent closure, and rejects stale ownership and command-bearing shortcuts. A temporarily unresponsive fixture proves activation refusal remains honest. Pre-cancelled launch sends no dispatch. The suite sends no keyboard/mouse input; explicit fixture activation can change foreground briefly. It does not open normal user applications or browser profiles. Exact installed-app-name/AppUserModelID launching and elevated/foreground-lock combinations remain release validation cases; the fixture does not claim those environments were exercised.
+
+For #30, `npm run test:reference:management -- --plan --engine=pi` safely lists a prepared Pi run (the command name is historical; the DSH engine is optional and not an acceptance gate) without building, starting a backend or accessing Windows. Execution requires Windows Node, `PI_CU_WINDOW_MANAGEMENT_TEST=1`, an authorized desktop and `--reference=<Windows-readable-pinned-checkout>` for DSH. The runner uses the existing disposable management fixture, selects the Pi worker or reference MCP, captures action responses (window lists retain only the owned target), and independently queries window owner, foreground, bounds, process path and creation identity. The fixture records actual close-veto/close events in its private directory. Cleanup matches the fixture executable’s unique path and, when observed, its PID and creation identity; an uncertain controller, launch or fixture exit is reported and blocks claimed cleanup. No keyboard, mouse, clipboard or daily application is used, but activation can change the foreground. On 2026-10-30, both Pi and the pinned DSH reference passed all eight cases on separate fresh Windows fixtures, including independently confirmed foreground activation and fixture-observed veto/close; owned process and controller cleanup were confirmed. See [paired window-management evidence](validation.md#paired-window-management). The opt-in `--extended` plan adds a same-process distractor, foreground restoration after focusing an owned control, and stale HWND refusal after a same-title replacement appears. On 2026-10-30, both engines passed all 11 cases on separate fresh owned Windows fixtures, including these three extended cases. Fixture focus/close events and independent HWND/PID/bounds observations verified the results; both controllers and fixtures exited. That original subset did not cover every interference/restoration case; [later Pi acceptance](validation.md#scoped-pointer-and-restoration-acceptance) records the current supported scope. A remembered operation target or `activated: true` alone cannot establish foreground acceptance.
+
+The desktop broker follows Windows' documented [ShellExecute automation interface](https://learn.microsoft.com/en-us/windows/win32/shell/shell-shellexecute). Window operations use [SetWindowPos](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-setwindowpos) and [PostMessage](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-postmessagew); dispatch acceptance does not establish application completion.
+
+The control fixture also stages a package with a missing native source and checks its explicit failure. Its programmatic sibling-window case reports whether Windows allowed the transition; a refusal is recorded as unexercised, not evidence that physical user-interference acceptance passed. Physical suppression and explicit handback remain dedicated-desktop checks under the [control delivery plan](implementation-plan.md#exclusive-control-delivery); this older fixture does not establish them.
+
+## Unknown UIA control types
+
+Some Windows controls expose newer control types that the .NET Framework UIA client cannot name. These nodes retain their available properties and children with `controlType: "Unknown"` and `controlTypeUnavailable: true`. Observations report `unknownControlTypes`, remain inconclusive, and cannot establish absence. An unknown type does not make an observed element stale.
+
+Run `npm run test:native:nullable-uia` with Windows Node after building. Its passive custom-provider fixture checks snapshot retention, inspection, traversal beyond an unknown type, and inconclusive disappearance waits. It performs no activation, screenshots or physical input.
+
+## Locked-session refusals
+
+A locked Windows session returns an error containing `DESKTOP_SESSION_LOCKED` before foreground activation, physical pointer input or physical text/key input. The helper reads only the WTS session-state prefix, without copying account-name fields. Unknown/unavailable WTS state is not reported as locked; existing target, foreground and input guards remain active. Read-only snapshots/screenshots can still return background application data while Windows is locked, which does not prove that the app can receive physical input. Unlock Windows and refresh observations before testing interactive behavior.
+
+On Windows after building, `PI_CU_EXPECT_LOCKED_SESSION=1 npm run test:native:session:locked` checks the explicitly locked case against a disposable fixture. Set the environment variable using your shell's syntax. The runner does not lock/unlock the workstation. It checks activation, pointer and text refusals, cleanup, and a quiet fixture observation. If the initial activation does not report a locked session, it aborts before further actions. This is a lock-diagnosis test, not proof of unlocked activation or foreground restoration.
+
+The diagnosis uses Microsoft's documented [WTSINFOEX session flags](https://learn.microsoft.com/en-us/windows/win32/api/wtsapi32/ns-wtsapi32-wtsinfoex_level1_w). In the reproduced Windows 11 case, WTS reported locked and the foreground process was LockApp; input-thread attachment returned access denied. Unlocked-session refusals still require separate diagnosis.
+
+## Unlocked activation round trips
+
+On an unlocked Windows desktop after building, run `npm run test:native:activation` with Windows Node and `PI_CU_ACTIVATION_TEST=1`. It requires installed Chrome, uses a disposable native fixture and isolated visible Chrome profile, activates between them six times through one warm worker, and independently verifies foreground after each transition. One native hover must activate Chrome and increment a page movement counter. It attempts to return to the original foreground only if a fixture is still in front, then closes its own windows/profile. It does not type, click, use the clipboard, or attach to a daily profile. Do not interact during the test. This does not establish all pointer actions, automatic post-input restoration, user-interference behavior, or mixed-DPI acceptance.
+
+### Scoped visual agent check
+
+`npm run test:pi:visual` prints a plan without opening Windows or calling a model. After building, run `npm run test:pi:visual -- --run --runs 1` (up to three fresh resets). Use the live Windows prerequisites in [runtime verification](runtime.md#verification), an unlocked session and the same authenticated Pi setup used for ordinary work. Serialize it with other desktop checks; on WSL use `flock -w 120 /tmp/pi-cu-live-tests.lock npm run test:pi:visual -- --run --runs 1`.
+
+The existing OCR fixture opens a separate owned form; no personal app discovery, keyboard, pointer or clipboard input is permitted. The task gives an outcome without a tool sequence: read APPROVE's randomized code, invoke its associated public-observed control once and preserve IGNORE. Independent fixture IPC verifies exact code/counters and pinned process identity. Reports in the printed temporary directory include requested/effective model settings, source hashes, sequential image-free traces and strict worker/fixture retirement. Unknown cleanup, collector failure or a deadline retains the owner and denies further tool dispatch; investigate its owner journal before any cleanup or new reset. A known failed outcome stops the reset loop. This is a scoped A04 check, not full #35 or OCR-availability acceptance.

package/package.json

@@ -1,6 +1,136 @@
 {
   "name": "@ycheung5/pi-computer-use",
-  "version": "0.0.0-stage",
-  "stub": true,
-  "description": "Temporary package placeholder for staged publishing"
-}
+  "version": "0.1.0",
+  "description": "Native Pi tools for Windows computer use from Windows and WSL2",
+  "type": "module",
+  "license": "MIT",
+  "author": {
+    "name": "ycheung5",
+    "email": "yatfei9@gmail.com"
+  },
+  "publishConfig": {
+    "access": "public"
+  },
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/ycheung5/pi-computer-use.git"
+  },
+  "bugs": {
+    "url": "https://github.com/ycheung5/pi-computer-use/issues"
+  },
+  "homepage": "https://github.com/ycheung5/pi-computer-use#readme",
+  "keywords": [
+    "pi-package",
+    "computer-use",
+    "windows"
+  ],
+  "files": [
+    "dist",
+    "README.md",
+    "GLOSSARY.md",
+    "docs/",
+    "assets/"
+  ],
+  "pi": {
+    "extensions": [
+      "./dist/extension.js"
+    ]
+  },
+  "engines": {
+    "node": ">=22.19.0"
+  },
+  "scripts": {
+    "build": "tsc -p tsconfig.build.json",
+    "test": "node --import tsx --test tests/*.test.ts",
+    "typecheck": "tsc --noEmit",
+    "format": "biome format --write .",
+    "lint": "biome check .",
+    "check": "npm run typecheck && npm run lint && npm test",
+    "prepack": "npm run build",
+    "lint:fix": "biome check --write .",
+    "test:native": "node tests/native-health.mjs",
+    "test:native:windows": "node tests/native-windows.mjs",
+    "test:wsl": "node tests/wsl-health.mjs",
+    "test:native:accessibility": "node tests/native-accessibility.mjs",
+    "test:native:nullable-uia": "node tests/native-nullable-uia.mjs",
+    "test:native:chromium": "node tests/native-chromium-accessibility.mjs",
+    "test:ownership": "node tests/ownership-host.mjs",
+    "test:native:capture": "node tests/native-capture.mjs",
+    "test:reference:capture": "node tests/reference-capture.mjs",
+    "test:reference:capture:report": "node tests/reference-capture-report.mjs",
+    "test:native:ocr": "node tests/native-ocr.mjs",
+    "test:stop": "node tests/stop-host.mjs",
+    "test:stop:busy": "node tests/native-stop-busy.mjs",
+    "test:stop:held": "node tests/native-held-input.mjs",
+    "test:stop:busy:check": "node tests/stop-busy-check.mjs",
+    "test:stop:pi-host": "node tests/pi-host-stop.mjs",
+    "test:stop:pi-host:check": "node tests/pi-host-stop-outcome-check.mjs",
+    "test:stop:pi-host:job": "node tests/pi-host-job.mjs",
+    "test:stop:pi-host:job:check": "node tests/pi-host-job-check.mjs",
+    "test:browser": "node tests/browser-host.mjs",
+    "test:native:controls": "node tests/native-controls.mjs",
+    "test:native:waits": "node tests/native-waits.mjs",
+    "test:settings": "node tests/settings-host.mjs",
+    "test:native:management": "node tests/native-window-management.mjs",
+    "test:reference:focus-foreground": "node tests/foreground-focus-comparison.mjs",
+    "test:reference:management": "node tests/window-management-comparison.mjs",
+    "test:native:session:locked": "node tests/native-session-activation.mjs",
+    "test:native:activation": "node tests/native-activation-roundtrip.mjs",
+    "test:native:pointer:guards": "node tests/native-pointer-guards.mjs",
+    "test:native:pointer:physical": "node tests/native-pointer-physical.mjs",
+    "test:native:text": "node tests/native-text.mjs",
+    "test:native:text:physical": "node tests/native-text-physical.mjs",
+    "test:native:notepad:unicode": "node tests/native-notepad-unicode.mjs",
+    "test:native:clipboard:ole": "node tests/native-ole-clipboard.mjs",
+    "test:browser:actions": "node tests/browser-actions-host.mjs",
+    "test:browser:firefox:endpoint": "node tests/browser-firefox-endpoint.mjs",
+    "test:indicators": "node tests/indicators-host.mjs",
+    "test:browser:profiles": "node tests/browser-profiles-host.mjs",
+    "test:package": "node tests/package-host.mjs",
+    "test:pi": "node tests/pi-host.mjs",
+    "test:pi:browser": "node tests/pi-browser-host.mjs",
+    "test:pi:workspace": "node tests/pi-workspace-host.mjs",
+    "benchmark:reference": "node tests/benchmark-reference.mjs",
+    "test:benchmark:report": "node tests/benchmark-report.mjs",
+    "test:benchmark:stop": "node tests/benchmark-report.mjs --stop",
+    "test:parity": "node tests/parity-host.mjs",
+    "test:browser:foreground": "node tests/browser-foreground-host.mjs",
+    "test:reference:transport": "node tests/reference-client-check.mjs",
+    "test:dsh:host-preflight": "node tests/dsh-host-preflight.mjs",
+    "test:browser:comparison": "node tests/browser-comparison.mjs",
+    "test:browser:comparison:oracle": "node tests/browser-comparison-check.mjs",
+    "benchmark:accessibility": "node tests/accessibility-comparison.mjs",
+    "test:accessibility:report": "node tests/accessibility-comparison-check.mjs",
+    "test:accessibility:provider-walker": "node tests/provider-walker-probe.mjs",
+    "test:accessibility:provider-platform": "node tests/provider-platform-probe.mjs",
+    "test:accessibility:diagnostics": "node tests/accessibility-comparison-check.mjs --diagnostics",
+    "test:native:clipboard:plan": "node tests/native-ole-clipboard-check.mjs",
+    "test:native:ime:owner": "node tests/ime-owner.mjs",
+    "test:native:process-containment": "node tests/process-containment-check.mjs",
+    "test:native:owner-supervisor": "node tests/owner-supervisor-check.mjs",
+    "test:native:owner-backend-health": "node tests/owner-backend-health-check.mjs",
+    "test:native:owner-crash-boundary": "node tests/owner-crash-boundary-check.mjs",
+    "test:pi:visual": "node tests/pi-visual-host.mjs",
+    "test:native:lifetime": "node tests/native-window-lifetime.mjs",
+    "test:native:lifetime:enumeration": "node tests/native-window-enumeration-lifetime.mjs"
+  },
+  "peerDependencies": {
+    "@earendil-works/pi-coding-agent": "*",
+    "typebox": "*"
+  },
+  "devDependencies": {
+    "@biomejs/biome": "2.6.4",
+    "@earendil-works/pi-coding-agent": "1.4.0",
+    "@types/node": "22.19.19",
+    "tsx": "4.21.0",
+    "typebox": "1.3.27",
+    "typescript": "5.9.3"
+  },
+  "overrides": {
+    "undici": "8.12.2",
+    "esbuild": "0.28.1"
+  },
+  "dependencies": {
+    "puppeteer-core": "25.16.0"
+  }
+}