@ycheung5/pi-computer-use 0.0.0-stage → 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/GLOSSARY.md +38 -0
- package/LICENSE +21 -0
- package/README.md +165 -2
- package/assets/accessibility.cs +208 -0
- package/assets/browser-indicators.cs +37 -0
- package/assets/capture.cs +215 -0
- package/assets/clipboard.cs +197 -0
- package/assets/controls.cs +249 -0
- package/assets/indicator-preview.png +0 -0
- package/assets/indicators.cs +252 -0
- package/assets/node-LICENSE.txt +2732 -0
- package/assets/ocr.cs +86 -0
- package/assets/ownership.cs +59 -0
- package/assets/pointer.cs +229 -0
- package/assets/puppeteer-LICENSE.txt +202 -0
- package/assets/readme-banner.svg +33 -0
- package/assets/stop.cs +197 -0
- package/assets/text.cs +249 -0
- package/assets/wgc.cs +167 -0
- package/assets/window-management.cs +217 -0
- package/assets/window-marker.cs +23 -0
- package/assets/window-references.cs +80 -0
- package/assets/windows.cs +127 -0
- package/dist/accessibility.d.ts +29 -0
- package/dist/accessibility.js +258 -0
- package/dist/browser-actions.d.ts +21 -0
- package/dist/browser-actions.js +213 -0
- package/dist/browser-indicators.d.ts +23 -0
- package/dist/browser-indicators.js +69 -0
- package/dist/browser-profiles.d.ts +10 -0
- package/dist/browser-profiles.js +174 -0
- package/dist/browser.d.ts +11 -0
- package/dist/browser.js +2071 -0
- package/dist/capture.d.ts +74 -0
- package/dist/capture.js +199 -0
- package/dist/client.d.ts +38 -0
- package/dist/client.js +331 -0
- package/dist/desktop.d.ts +20 -0
- package/dist/desktop.js +42 -0
- package/dist/extension.d.ts +4 -0
- package/dist/extension.js +409 -0
- package/dist/launcher.d.ts +6 -0
- package/dist/launcher.js +85 -0
- package/dist/native.d.ts +11 -0
- package/dist/native.js +145 -0
- package/dist/ocr.d.ts +52 -0
- package/dist/ocr.js +246 -0
- package/dist/ownership.d.ts +4 -0
- package/dist/ownership.js +24 -0
- package/dist/pointer.d.ts +108 -0
- package/dist/pointer.js +174 -0
- package/dist/protocol.d.ts +15 -0
- package/dist/protocol.js +1 -0
- package/dist/runtime.d.ts +8 -0
- package/dist/runtime.js +72 -0
- package/dist/settings.d.ts +85 -0
- package/dist/settings.js +449 -0
- package/dist/stop.d.ts +24 -0
- package/dist/stop.js +156 -0
- package/dist/text.d.ts +100 -0
- package/dist/text.js +183 -0
- package/dist/waits.d.ts +19 -0
- package/dist/waits.js +262 -0
- package/dist/window-management.d.ts +25 -0
- package/dist/window-management.js +167 -0
- package/dist/worker.d.ts +1 -0
- package/dist/worker.js +540 -0
- package/docs/README.md +54 -0
- package/docs/adr/0001-preserve-reference-behavior.md +9 -0
- package/docs/adr/0002-window-lifetime-references.md +11 -0
- package/docs/adr/0003-exclusive-desktop-control.md +11 -0
- package/docs/adr/0004-pi-first-acceptance.md +9 -0
- package/docs/adr/0005-default-apps-and-workspace-reuse.md +7 -0
- package/docs/agents/domain.md +9 -0
- package/docs/agents/issue-tracker.md +10 -0
- package/docs/agents/triage-labels.md +9 -0
- package/docs/browser.md +113 -0
- package/docs/implementation-plan.md +139 -0
- package/docs/indicators.md +53 -0
- package/docs/native-interop-research.md +17 -0
- package/docs/ownership.md +25 -0
- package/docs/pi-extension-research.md +57 -0
- package/docs/pointer.md +65 -0
- package/docs/runtime.md +42 -0
- package/docs/settings.md +33 -0
- package/docs/spec.md +161 -0
- package/docs/stopping.md +83 -0
- package/docs/text.md +79 -0
- package/docs/tickets.md +84 -0
- package/docs/validation.md +1372 -0
- package/docs/windows.md +171 -0
- package/package.json +134 -4
package/docs/text.md
ADDED
|
@@ -0,0 +1,79 @@
|
|
|
1
|
+
# Text and key entry
|
|
2
|
+
|
|
3
|
+
The candidate acquires [exclusive control](spec.md#exclusive-control) before physical text entry. Suppressed physical events do not invalidate typing; foreign injected or unconsumed events still do. Pi chord/focus has [scoped acceptance](validation.md#pi-chords-and-editor-focus-acceptance); #31 control/handback and #27 clipboard outcomes retain their accepted receipts.
|
|
4
|
+
|
|
5
|
+
`type_text` accepts `text`, `method: auto | clipboard | sendinput | background`, `focusMode: preserve | main`, `restoreClipboard` (default true), and native target options. Browser options retain browser-first routing. `keypress` accepts a `keys` array with exactly one non-modifier key and optional Ctrl/Alt/Shift modifiers. Windows/Cmd/Meta keys are blocked; unsupported names/layout mappings fail without input. Explicit background keypress supports one printable character only.
|
|
6
|
+
|
|
7
|
+
Auto text first tries the focused native Edit/RichEdit control using bounded `EM_REPLACESEL`, preserving its selection. This quiet path handles Unicode without clipboard or physical keys; multiline text, main-focus selection and unsupported controls require physical entry. An uncertain edit-message result is never retried as clipboard input. Background WM_CHAR delivery is explicitly unverified. Read-only/disabled controls refuse mutation.
|
|
8
|
+
|
|
9
|
+
The operation's last target is reused after foreground restoration. Fresh Win32 focus takes precedence over remembered native focus. Only an agent-established focus is remembered; subsequent click/focus/target changes invalidate it. Remembered controls must retain their HWND/PID/root and unchanged input-history stamp. An intervening user input with unavailable fresh focus returns `TARGET_FOCUS_CHANGED`; it does not guess the old field. Main focus explicitly chooses the largest enabled, keyboard-focusable, on-screen writable Edit/Document control. After an explicit or remembered focus request, physical text entry waits up to one second for foreground and the requested field to agree. The wait observes without repeating focus or text, checks cancellation and identity, and refuses with `TARGET_FOCUS_CHANGED` if focus cannot be confirmed. An explicit field request cannot be replaced by remembered focus. Successful confirmation reports `focusConfirmed` and `inputFocusHandle`. Optional native edit `verifyValue` is capped at 200 characters and excludes password edits. Physical entry does not read password values.
|
|
10
|
+
|
|
11
|
+
## Physical entry and cleanup
|
|
12
|
+
|
|
13
|
+
Explicit `sendinput` text uses KEYEVENTF_UNICODE down/up pairs for ordinary text, Enter key pairs for LF or CR (CRLF becomes one Enter), and Tab key pairs for tabs. Whitespace-only strings are valid text. These keys retain the target application's normal Enter/Tab behavior. A cancellable 50 ms gap follows each dispatched unit: both unpaced delivery and a 10 ms gap were observed to corrupt appended text in modern Notepad despite successful dispatch. The pinned DSH reference instead batches Unicode events without this per-character delay. Pacing and control-character handling changed together in Pi, so the successful regression does not establish that 50 ms is necessary or isolate the cause. See the [Pi text validation scope](validation.md#unicode-mechanism-investigation). Minimum-delay discovery and DSH batching experiments are not release gates. This provisional compatibility workaround adds approximately 50 ms per unit plus platform scheduling overhead; it is not a universal guarantee about application processing. Dispatch remains unverified until a subsequent observation confirms the intended text. Successful results expose `inputDelayMs` so callers can identify the pacing actually used by the running helper. Default `auto` entry retains its existing quiet-edit/clipboard routing. Chords are mapped using the target thread's keyboard layout and use SendInput rather than an uninterruptible SendKeys call. The shared activity observer spans the whole call and distinguishes our tagged key events from intervening physical/foreign injected input; a later synthetic key cannot erase earlier intervention evidence. The helper rechecks target HWND/PID, foreground and cancellation before each character/key; text also checks focused control and user-held modifiers/buttons. Every held key/modifier is released in `finally`, even when its key-down result was uncertain. Partial text, timeout and cancellation never authorize automatic replay.
|
|
14
|
+
|
|
15
|
+
The independent stop event and retained-input acknowledgment from [pointer stopping](pointer.md) cover keyboard actions too. A timeout signals the operation to stop; missing release/clipboard acknowledgment retains ownership and blocks worker termination/replacement. `set_value` may escalate only an explicit background `WCU_NEEDS_INPUT`, after re-resolving the observed control, checking enabled/read-only state and focusing it; select-all and typing use the same guarded cleanup.
|
|
16
|
+
|
|
17
|
+
## Clipboard transaction
|
|
18
|
+
|
|
19
|
+
Clipboard entry uses a separate STA process with a hidden message-only owner window and message pumping. All UIA focus work finishes before clipboard replacement; restoration never calls application rendering providers. The snapshot phase duplicates each available public clipboard format, including native bitmap/metafile formats, using [OleDuplicateData](https://learn.microsoft.com/en-us/windows/win32/api/ole2/nf-ole2-oleduplicatedata). When OLE transport markers (`DataObject` / `Ole Private Data`) are present, the helper validates the advertised FORMATETC descriptors before omitting those process-bound markers. Public formats must be unindexed content, without a target device, and available in supported native storage. All public native payloads are still duplicated. Unmaterializable, genuinely indexed, device-specific or unsupported OLE data fails before replacement rather than falling back to text-only preservation. Snapshot/rendering has a five-second process deadline while mutation is still forbidden.
|
|
20
|
+
|
|
21
|
+
Only a completed snapshot permits replacement. The helper publishes zero-valued history/cloud exclusion formats before Unicode text, following [Microsoft's clipboard format contract](https://learn.microsoft.com/en-us/windows/win32/dataxchg/clipboard-formats#cloud-clipboard-and-clipboard-history-formats). Clipboard sequence checks protect an intervening user copy. Windows can also advance the sequence while synthesizing formats during paste; restoration across that change requires both the helper's hidden owner window and its unique transaction nonce. A different owner or missing nonce preserves the newer clipboard instead. Ctrl+V is followed by a cancellable bounded wait and native WM_NULL drain; a stalled target returns an uncertain-dispatch error. The original opaque snapshot is restored in `finally` before cleanup acknowledgment, or the explicit `restoreClipboard: false` result keeps the injected clipboard content. Cancellation/error always requests restoration. A newer user copy is preserved.
|
|
22
|
+
|
|
23
|
+
Cancellation, parent exit or a 30-second stalled transaction also triggers independent restoration. OpenClipboard acquisition has bounded retries. If another application indefinitely holds the clipboard or restoration fails, the helper retains duplicate originals, retries restoration and leaves cleanup/ownership unconfirmed. It is never force-killed after mutation or silently reported restored. This is a fail-closed recovery condition, not a guarantee that Windows will release another application's clipboard lock.
|
|
24
|
+
|
|
25
|
+
## Validation and scope
|
|
26
|
+
|
|
27
|
+
- `npm test`: normal worker validation for text options, malformed/ambiguous chords, Windows-key blocking and unsupported background chords, plus shared stop/non-replay lifecycle tests.
|
|
28
|
+
- `npm run test:native:text` after build: background Unicode/emoji, remembered target, WM_CHAR, read-only and wrong-target refusal on disposable forms. Native auto-edit checks use the backend's ordinary quiet-attempt protocol, so failure cannot escalate into physical input. This is backend-seam proof, not full Pi auto-escalation acceptance. The suite never reads or replaces the clipboard and sends no physical keys/mouse input.
|
|
29
|
+
- `PI_CU_DEDICATED_INPUT_DESKTOP=1 npm run test:native:text -- --chords` on an unlocked idle Windows desktop: native RichEdit chord effects from an unselected caret, navigation/selection/deletion, exact text, separate main/preserve editors and refusal checks. Uses only the owned fixture and never the clipboard; `-- --chords --plan` describes the case without loading a backend. [Evidence and limitations](validation.md#pi-chords-and-editor-focus-acceptance).
|
|
30
|
+
- The legacy `test:native:text:physical` runner refuses execution because it overwrote clipboard contents and could retire an unsettled input owner. Use the [existing OLE fixture](#ole-clipboard-regression) for its supported preservation cases; exclusive-control acceptance follows the delivery plan.
|
|
31
|
+
|
|
32
|
+
Supported paste, rich restoration, bounded clipboard-busy refusal, exclusion markers and copying across handback have [accepted #27 evidence](validation.md#pi-clipboard-contention-and-privacy-acceptance). [#26 chord/editor-focus](validation.md#pi-chords-and-editor-focus-acceptance), [#30 workspace/foreground](validation.md#workspace-decision-acceptance), and [#31 control/human handback](validation.md#control-interruption-acceptance) have their scoped receipts. These do not establish universal application shortcuts or IME behavior. Clipboard writes by other applications still require newer-copy protection; partial or stopped text is never automatically replayed. The [coverage plan](implementation-plan.md#coverage-matrix) remains authoritative for release checks.
|
|
33
|
+
|
|
34
|
+
## Owner-thread IMM observation
|
|
35
|
+
|
|
36
|
+
`npm run test:native:ime:owner -- --list` describes the passive fixture without running Windows. On native Windows, `npm run test:native:ime:owner -- --execute` creates only a hidden owned TextBox HWND and samples `ImmGetContext`/`ImmGetOpenStatus` on its creator thread and a second thread. It does not focus the control, change keyboard layout or send input. A Windows Framework compiler and Windows Node are required. Raw booleans and handle availability describe only this synthetic fixture; a raw false open result does not distinguish all API failure cases, and neither sample establishes the focused Notepad IME or TSF state. `--report=PATH` formats a saved fixture observation offline without changing its scope.
|
|
37
|
+
|
|
38
|
+
## Notepad Unicode regression
|
|
39
|
+
|
|
40
|
+
The runner requires fresh, independently observed window/process/thread identity, foreground, keyboard layout and focus within the UIA-observed editor HWND before Ctrl+End and again before test text. The editor HWND must be a distinct child of the owned root; a missing or virtual-only HWND stops the run as inconclusive. `--report-environment=/path/to/report.json` checks saved `{target, editor, observation}` evidence offline; a nonzero exit denotes incomplete evidence. The observation does not establish TSF/IME status or prove the target remains unchanged afterward. A subsequent paired Notepad append case validated these checks on both engines; Pi matched the expected text and the reference returned corrupted appended text. See [the scoped evidence](validation.md#unicode-append-with-verified-editor-focus).
|
|
41
|
+
|
|
42
|
+
List the six cases from WSL with `npm run test:native:notepad:unicode -- --list`; listing starts no backend or Windows application. On an authorized idle Windows desktop, build and run the default Pi engine with Windows Node and `PI_CU_NOTEPAD_INPUT_TEST=1`. Reuse the existing six-case and append passes; run only a missing or affected Pi case. The owned-document oracle compares exact UIA values and saved UTF-8 bytes, normalizing only line endings, and records method/timing plus observable editor/focus/keyboard context. Unknown IME/TSF state remains unknown; exhaustive instrumentation is not required. It saves/closes only owned test tabs after action-matched cleanup; uncertainty retains actual owners without replay. This does not by itself establish chord, handback or general activation acceptance. DSH runs are discarded under [ADR 0004](adr/0004-pi-first-acceptance.md); old paired receipts remain in validation.
|
|
43
|
+
|
|
44
|
+
## OLE clipboard regression
|
|
45
|
+
|
|
46
|
+
Preparation is portable and performs no clipboard or native actions:
|
|
47
|
+
|
|
48
|
+
```sh
|
|
49
|
+
npm run test:native:clipboard:plan
|
|
50
|
+
|
|
51
|
+
```
|
|
52
|
+
|
|
53
|
+
Use the existing runner with `--engine pi` (default). `--list-cases` and `--plan` report prepared cases without importing a backend or creating/launching Windows fixtures. Real execution requires Windows Node, a built Pi worker and `PI_CU_CLIPBOARD_INPUT_TEST=1` on an authorized idle desktop. Keep preservation/cleanup gates before clipboard mutation. The legacy reference-engine option is historical and is not required or scheduled for acceptance.
|
|
54
|
+
|
|
55
|
+
For a future explicitly enabled run, the fixture privately caches the original text/stream clipboard formats and restores only when it still owns the seeded clipboard; a genuinely intervening copy is preserved. Unsupported original fixture formats are refused before seeding. No clipboard payload is printed. The same fixture checks actual editor text independently against UIA, synthetic Unicode/HTML/binary preservation, indexed-OLE payload preservation and synthetic newer-copy preservation. An indexed attempt may refuse without input or report exact paste with preservation; the report records which occurred rather than requiring DSH to use Pi’s refusal policy. Raw tool results/errors and any optional cleanup diagnostics remain in the output; Pi-specific restoration labels/counts are not required from DSH. An explicitly unconfirmed cleanup blocks subsequent cases. Only owned fixture files/processes are eligible for cleanup; normal applications and documents are untouched.
|
|
56
|
+
|
|
57
|
+
Historical reference-client activation/cleanup constraints belong to the retained comparison evidence. Current Pi acceptance requires action-matched cleanup, independent clipboard/editor observations and retention of uncertain owners; no reference backend is needed.
|
|
58
|
+
|
|
59
|
+
The safe CLI checks establish preparation and gating only. Windows C# compile-only validation passed for the modified clipboard fixture without executing it or accessing the clipboard. Dedicated runs verified supported paste/restoration and newer-copy preservation for Pi; DSH lost the synthetic newer copy. A standard virtual-file case independently verified indexed data before and after each tool call: Pi refused safely and preserved it, while DSH reported dispatch but the indexed content was no longer retrievable. The later Pi locked-backend and privacy/handback cases complete #27's scoped outcomes. Universal formats, history-database/cross-device behavior and remote desktop behavior are not claimed. The removed delayed-provider experiment did not establish a backend timeout and is not part of acceptance.
|
|
60
|
+
|
|
61
|
+
Pi publishes zero-valued `ExcludeClipboardContentFromMonitorProcessing`, `CanIncludeInClipboardHistory` and `CanUploadToCloudClipboard` markers for temporary clipboard text. The paste consumer independently observed all three zero DWORD markers in the accepted privacy case; the actual backend-busy case also passed. These establish the supported Windows contract, not inspection of history contents or cross-device delivery.
|
|
62
|
+
|
|
63
|
+
Clipboard results expose `clipboardSnapshot` counts (`preservedFormats`, `oleMetadataExcluded`, `oleFormatsValidated`) and `clipboardRestoreOutcome`: `restored`, `newer-copy-preserved`, `kept`, or `unchanged` when no replacement was authorized. Cleanup confirmation alone is not a claim that the original clipboard was restored. OLE transport markers are regenerated by Windows/consumers as needed rather than restoring stale interface handles.
|
|
64
|
+
|
|
65
|
+
Physical comparison diagnostics can select one fresh Unicode case with `--case=3` (IDs 1–6); `--list --case=3` safely reports the selected seed and text. The opt-in runner also offers isolated, labeled `--diagnostic=pi-no-delay`, `--diagnostic=reference-keyup-scan`, and `--diagnostic=reference-pair-batches` variants, each changing one variable in a temporary copy and recording hashes. These are diagnostic experiments, not production settings. The read-only target-thread legacy context probe records raw API returns and leaves focused-context, open-state and TSF claims unknown unless independently established. A failed full run remains recorded; selecting a different case creates fresh state rather than retrying the failed action. The clipboard runner independently reads the indexed fixture payload before dispatch, so a malformed seed is distinguished from post-action corruption.
|
|
66
|
+
|
|
67
|
+
### Isolated clipboard diagnostics
|
|
68
|
+
|
|
69
|
+
Use `--case ole-preservation`, `--case indexed-refusal`, `--case virtual-file` or `--case newer-copy` with the existing runner to select one case. `--plan` still works from WSL without desktop resources or opt-in. Each executed case resets only its fixture editor, verifies it is blank, seeds its own clipboard prerequisite and checks that seed before the tool call. Thus newer-copy can start independently from a fresh fixture; it does not depend on a successful indexed case or reuse an uncertain prior action.
|
|
70
|
+
|
|
71
|
+
`--probe-indexed` is a fixture-only diagnostic behind the same `PI_CU_CLIPBOARD_INPUT_TEST=1` gate. It temporarily replaces the clipboard but sends no keyboard/pointer input and starts no Pi/DSH backend. It compares the existing indexed data object directly with the object returned by `OleGetClipboard`, recording Query/Get results and their FORMATETC enumerations, then restores the original clipboard according to fixture ownership. A direct pass plus clipboard-query failure isolates the boundary; it does not establish why that boundary failed or justify a production change. This diagnostic leaves the indexed format unchanged. Startup refusal reports only a fixed category and HRESULT, never original clipboard formats or values. Before a valid HWND is returned no seed/backend command is sent; an early-exited fixture receives no cleanup command. Replies and stuck-startup retirement are bounded, and cleanup failures do not replace the original diagnostic. The [Shell clipboard contract](https://learn.microsoft.com/en-us/windows/win32/shell/clipboard) documents indexed virtual-file `FileContents` with its descriptor as a separate supported design, which was tested separately as described in [the current evidence](validation.md#unicode-mechanism-probes-and-clipboard-startup--follow-up).
|
|
72
|
+
|
|
73
|
+
Portable selector/probe contracts were observed failing before these CLI options existed, then passing. The custom indexed fixture remains invalid through `OleGetClipboard`. The independently readable standard virtual-file fixture has since been executed against both engines; its results are recorded in the current evidence.
|
|
74
|
+
|
|
75
|
+
`--probe-locked` is a separate fixture-only prerequisite for future locked-clipboard comparisons. Its plan can be inspected on WSL with `npm run test:native:clipboard:ole -- --plan --probe-locked`. Execution uses the same dedicated-desktop opt-in, but does not start an engine, send input, read clipboard content or replace it. An owned fixture temporarily opens the clipboard; the OS-reported lock holder and a second process independently verify available → busy → available, and the clipboard sequence must stay unchanged. A fixture-side watchdog releases the lock after five seconds if the runner stops responding. This prepares a controllable contention fixture; it does not yet test Pi or DSH behavior while the clipboard is locked. On 2026-10-30, the dedicated fixture-only probe confirmed available → busy → available, matched the OS lock owner, left the clipboard sequence unchanged, and confirmed fixture cleanup. No clipboard contents were read or replaced and no Pi/DSH tool call was made. The later [actual backend-lock case](validation.md#pi-clipboard-contention-and-privacy-acceptance) passed; reuse that result.
|
|
76
|
+
|
|
77
|
+
Windows-only, no-input process probes support future timeout research: `npm run test:native:owner-supervisor` runs synthetic child processes, and `npm run test:native:owner-backend-health -- <reference-checkout>` calls only Pi and DSH health. Each starts its disposable runner suspended, assigns it to a Windows job before resuming, terminates the job on its test deadline and requires the job to report no active processes. The synthetic probe also checks that its descendant started and did not perform a delayed write. The health probe on a staged build returned healthy results and zero active job processes after retirement for both engines. The outer runner does not force a timeout: unconfirmed supervisor cleanup can wait indefinitely and retains its workspace. `npm run test:native:owner-crash-boundary` checks synthetic processes when their job owner is forcibly terminated: it verifies the two observed job members exit, while a separately brokered child outside the job survives and must be stopped separately. A normal synthetic control runs its managed `finally` block; the killed job worker entered its `try` block but did not run that `finally`. This establishes a containment and cleanup limit, not whether Pi or DSH uses such a broker or what happened to OS input state. These tests do not establish complete descendant containment, clipboard-lock continuity or physical input release. In particular, this supervisor is not a safe mechanism for terminating a possible physical-input owner and must not be used as a locked-clipboard acceptance run.
|
|
78
|
+
|
|
79
|
+
For the accepted Pi cases, use `--probe-locked-backend` or `--case privacy-handback` with the existing opt-in runner. Both privately preserve the original before using owned seeds. Warm setup precedes locking; success requires a real busy refusal before verified lock release, unchanged seed/editor and cleanup. A slow failure can encounter only disposable data even if the lock watchdog expires; uncertain cleanup blocks original restoration and teardown. Privacy observes only the temporary transaction markers; the handback case checks released control before a scripted application copy and a fresh-operation paste. No personal history or cross-device account is accessed.
|
package/docs/tickets.md
ADDED
|
@@ -0,0 +1,84 @@
|
|
|
1
|
+
# Implementation tickets
|
|
2
|
+
|
|
3
|
+
Requirements live in [spec](spec.md), delivery/coverage in [implementation-plan](implementation-plan.md), results in [validation](validation.md). This is the current Pi-first queue, reconciled on 2026-11-09 under [ADR 0004](adr/0004-pi-first-acceptance.md). DSH manual/host/model comparisons, credentials and parity benchmarks are discarded.
|
|
4
|
+
|
|
5
|
+
## Current investigation and acceptance work
|
|
6
|
+
|
|
7
|
+
### Current progress
|
|
8
|
+
|
|
9
|
+
| Measure | Status after scope reconciliation |
|
|
10
|
+
| --- | --- |
|
|
11
|
+
| Tool implementation | All 25 target tools have implementations. Release acceptance concerns Pi correctness, preservation and usability. |
|
|
12
|
+
| Completed issues | 24: #2, #3, #6, #7, #8, #14, #16, #17, #18, #19, #20, #24, #25, #26, #27, #28, #29, #30, #31, #32, #33, #34, #35, #36. Historical comparison completions retain their original evidence scope. |
|
|
13
|
+
| Superseded issues | 10: #4, #5, #9, #10, #11, #12, #13, #15, #23, #37. Superseded is not a completed product delivery. |
|
|
14
|
+
| Open issues | 2: release #21 and product parent #1. Installation #2 is complete with explicitly accepted first-release test limitations. |
|
|
15
|
+
| Accepted Pi outcomes | Windows/WSL2 packaged launch, ownership, waits, settings/tool selection, Chromium and experimental Firefox workflows; six browser/mixed agent scenarios each passed three reset runs. Native creation/preservation, replacement/partial recovery and composed continuation (#35), #31 control interruption/cleanup and human-confirmed GitHub sign-in persistence/removal also passed. |
|
|
16
|
+
| Latest recorded checks | Final release-audit checks: 187 passed, one historical-reference skip; 91-entry 0.1.0 archive, MIT/notices/version/development-file exclusion and documentation links passed. [Audit](validation.md#final-release-audit). Existing Windows/model/physical receipts retain their source scope; no release declared. |
|
|
17
|
+
|
|
18
|
+
The package is implemented; remaining work is targeted Pi acceptance and repair. An open issue does not reopen its accepted cases. No overall completion percentage is asserted, and no release is declared. The earlier development baseline and historical comparison receipts remain available through commits/archive tags.
|
|
19
|
+
|
|
20
|
+
### Delivery outcomes and ownership
|
|
21
|
+
|
|
22
|
+
| Owner | Accepted work | Remaining Pi outcome |
|
|
23
|
+
| --- | --- | --- |
|
|
24
|
+
| #2 — clean installation | Runtime/package provisioning and both current-machine Pi host paths passed; #3/#34 complete. User reports second-computer WSL installation and computer actions succeeded. | Completed on 2026-11-09: [second-computer WSL success and accepted limitations](validation.md#second-computer-installation-report). Fresh native-Windows/prerequisite-failure/clean-environment shutdown cases remain unverified, not blocking first release. |
|
|
25
|
+
| #6 — screenshot mapping | Capture/fallback, strict window-only refusal/recovery, actual 100%/150%/175% mixed-DPI and changed negative-origin mapping passed; restored display state recorded. | Completed for the user-accepted first-release scope; [closeout](validation.md#screenshot-mapping-closeout). 200% and other unexercised display variants remain unverified; no universal scaling claim. |
|
|
26
|
+
| #7 — OCR failures | Literal/CJK text, bounds and partial-result semantics passed in named cases. | Completed: missing-resource/null-engine refusals, actual profile-language reporting and cancellation after dispatch with helper exit/fresh-operation recovery; [scoped receipt](validation.md#pi-ocr-failure-acceptance). Clean-machine compatibility stays #2; display mapping stays #6. |
|
|
27
|
+
| #18 — remembered sign-in | Profile lifecycle evidence plus human-confirmed GitHub authentication across actual Pi restart and verified test-profile removal passed. | Completed; [scoped receipt](validation.md#manual-github-remembered-sign-in). |
|
|
28
|
+
| #20 — visible control | Appearance, lifecycle, human/control receipts, actual scaling/changed-layout strict capture and human-confirmed Snipping Tool recording-visible/hidden phases passed. | Completed for the composed first-release scope; [acceptance and limits](validation.md#snipping-tool-recording-acceptance). No universal recorder/remote/capture-route pixel claim. |
|
|
29
|
+
| #26 — chords/focus | Six exact-text cases, physical RichEdit chord/selection effects, distinct main/preserve destinations, refusal checks and reused #31 human-edit continuation passed. | Completed for the supported editor scope; [acceptance and limitations](validation.md#pi-chords-and-editor-focus-acceptance). Keep the provisional text delay; no universal shortcut/IME claim. |
|
|
30
|
+
| #27 — clipboard preservation | Rich/OLE preservation, unsupported virtual-file refusal and newer-copy preservation passed. | Completed: actual backend-busy refusal, observed exclusion markers and preserved application copy across released-control continuation; [scoped acceptance](validation.md#pi-clipboard-contention-and-privacy-acceptance). |
|
|
31
|
+
| #28 — targeting/UIA | Native/Chromium/provider/budget/scroll, lifetime and registered-tool/host receipts [reconciled](validation.md#pi-targeting-and-accessibility-reconciliation); no DSH rerun required. | Completed with user-accepted first-release test limitations on 2026-11-05: genuine same-HWND/PID reuse and verified UIPI denial remain unverified. Identity/access safeguards remain unchanged. |
|
|
32
|
+
| #30 — apps/foreground/restoration | Default-handler/native-app and ordinary Chrome reuse, original preservation, three ambiguity and three unavailable-workspace agent decisions, popup/native-menu/cross-process drop and exact cursor/foreground restoration passed. The one-pixel conversion defect is repaired. | Completed for its supported scope: [workspace decisions](validation.md#workspace-decision-acceptance), [pointer receipt](validation.md#scoped-pointer-and-restoration-acceptance). Preserve Windows activation-refusal limits; native create/recovery stays #35 and displays stay #6. |
|
|
33
|
+
| #31 — stop/control | Native Pi Esc, worker shortcut/suppression, native/WSL command stop, rejection/recovery and no-input host-exit cleanup passed; queued-settlement bug fixed. Four known-held Ctrl/button cancellation/target-error cases and actual WSL Pi Ctrl+Alt+F9 and human handback/resume also passed. | Completed: native availability-loss response and existing worker/held-input/host/handback results satisfy the narrowed scope; [acceptance](validation.md#control-interruption-acceptance). |
|
|
34
|
+
| #35 — native agent goals | Three preopened append/save tasks, three creation/preservation/save resets and three replacement/partial-observation recovery workflows have accepted independent evidence. [Creation](validation.md#native-document-workflow); [recovery and collector-failure disposition](validation.md#native-replacement-and-partial-observation-recovery-preparation). | Completed: three real-Pi continuation resets preserved the labelled stand-in edit after cancelled/retired setup. [Composed handback acceptance](validation.md#native-cancellation-and-handback-continuation) reuses #31 actual human/control proof; no new physical-control claim. |
|
|
35
|
+
| #36 — browser/mixed agent goals | Six named Pi scenarios passed three resets: form/delay; navigation/dialog/transfer; stale recovery; quiet mixed task; partial-observation recovery; held-navigation cancellation. | Completed for this scope. Shared physical control stays #31/#20; native creation/recovery stays #35. No DSH leg remains. |
|
|
36
|
+
| #21 under #1 — release | Capability mapping, existing checks and accepted scopes retained. | Reconcile the Pi owners above, known failures/limits, useful responsiveness, package/docs/version and final user acceptance; then publish. |
|
|
37
|
+
|
|
38
|
+
### Execution queue
|
|
39
|
+
|
|
40
|
+
Work on one bounded missing outcome at a time. Reuse accepted evidence across issues and diagnose an observed Pi failure before changing its mechanism.
|
|
41
|
+
|
|
42
|
+
| Order / owner | Next action | User involvement |
|
|
43
|
+
| --- | --- | --- |
|
|
44
|
+
| 1 — #21 under #1 | Agent-owned [release audit](validation.md#final-release-audit) is complete: package/license repair, coherent version/install docs, capability/failure/limit reconciliation and final checks/reviews. The user confirmed [interactive settings/editor/reload](validation.md#interactive-settings-and-reload-acceptance) on 2026-11-09. #2 is complete with accepted limitations. [0.1.0 release notes and scope](validation.md#first-release-preparation) are prepared; final aggregate approval and publication remain. | Final approval of the prepared release scope/notes; no more display-scaling/stop-key tests for closed issues, and no repeated guest request. |
|
|
45
|
+
|
|
46
|
+
Standing scoped idle-test authorization remains. Preserve unrelated applications, documents, clipboard and profiles, serialize desktop activity and request human participation only for genuine physical/private/visual steps. Unknown input cleanup retains actual owners and blocks dependent activity. No DSH/OpenRouter credentials, DSH installation or side-by-side tests are needed.
|
|
47
|
+
|
|
48
|
+
### Comparison and agent-ticket accounting
|
|
49
|
+
|
|
50
|
+
| Tickets | Disposition |
|
|
51
|
+
| --- | --- |
|
|
52
|
+
| #24/#25/#29/#32/#33/#34 | Previously completed baseline/audit scopes; retained evidence is useful context, not a future DSH test programme. |
|
|
53
|
+
| #36 | Completed on existing named Pi workflow evidence after dropping the DSH leg. This does not assert that its older quiet scenarios tested the later physical-control implementation. |
|
|
54
|
+
| #23/#37 | Superseded, not completed. Pi diagnosis stays with capability owners; native outcomes with #35; aggregate evidence/failure disposition and release reconciliation with #21 under #1. |
|
|
55
|
+
| #26/#27/#28/#30/#31 | Existing Pi behavior/correctness owners; comparison-only clauses removed. |
|
|
56
|
+
| #16/#17/#19/#14/#8/#3 | Completed product slices retain their accepted scope; remaining shared obligations stay with their current owners. |
|
|
57
|
+
|
|
58
|
+
### Cleanup dispositions
|
|
59
|
+
|
|
60
|
+
| Candidate | Disposition | Reason |
|
|
61
|
+
| --- | --- | --- |
|
|
62
|
+
| Repeated Pi host mocks and guard-probe setup | Consolidated into existing helpers | One host adapter retains both current tools and registration history, preserving duplicate-registration assertions. The same guard probe serves retained and running states. |
|
|
63
|
+
| Child processes used only to call the pure report verifier | Removed | Call the existing verifier directly with serialized report facts. Keep subprocesses where the behavior under test actually involves a process or host. |
|
|
64
|
+
| Duplicate browser budget checks and unreachable `set_value` validation | Removed | Existing routed action schemas already enforce the same or stricter limits; `set_value` always uses its action normalizer. Public worker argument checks preserve those contracts. |
|
|
65
|
+
| Repeated accepted evidence in this execution queue | Consolidated | Current status remains in the delivery map; source-scoped receipts remain in validation. The queue now states unfinished next actions. |
|
|
66
|
+
| Windows worker/native bridge/independent stop and ownership | Keep | These isolate host differences, native execution and stopping of busy work. Removing them would remove required behavior or move the same complexity into callers. |
|
|
67
|
+
| Process/job/supervisor diagnostic fixtures | Keep only for a named remaining Pi question | Remove a diagnostic when its question is resolved or a retained behavior-level case replaces it; size alone does not establish redundancy. |
|
|
68
|
+
| Browser engine-specific code | Keep concrete supported behavior | Chromium and Firefox are real variations, but an engine adapter must hide actual repeated differences. Do not add a class hierarchy or split files merely to move line counts. |
|
|
69
|
+
|
|
70
|
+
## Original issue disposition
|
|
71
|
+
|
|
72
|
+
The 2026-10-31 consolidation transferred source obligations rather than claiming them completed. On 2026-11-02 the user removed DSH manual/parity acceptance. #36 can close on its recorded Pi results; #23/#37 retire as duplicate/superseded umbrellas. Their remaining Pi obligations are owned below and in #21, not discarded.
|
|
73
|
+
|
|
74
|
+
| Superseded source | Current owner | Retained Pi scope |
|
|
75
|
+
| --- | --- | --- |
|
|
76
|
+
| #4/#5/#10 | #28 | Discovery, stale/wrong/inaccessible targets, native/Chromium observations and supported control/scroll behavior. |
|
|
77
|
+
| #9 | #31 | Stop protection, bounded cleanup, same-operation rejection/new-user recovery and genuine physical-stop evidence. |
|
|
78
|
+
| #11 | #30/#31 | Pointer geometry/restoration versus held-button release/cancellation/non-replay. |
|
|
79
|
+
| #12 | #26/#27/#31 | Text/chords/focus; clipboard/privacy/restoration; input cancellation/held-key release/non-replay. |
|
|
80
|
+
| #13 | #30 | Application/window management, targeting, lifecycle and failure behavior. |
|
|
81
|
+
| #15 | Completed #16 | Supported Chromium tool workflows; broader profile/sign-in completed in #18. |
|
|
82
|
+
| #23/#37 | #1/#21 and existing capability owners | Current Pi failure diagnosis/targeted repairs with the owning issue; aggregate capability/agent evidence, model/host identity, failure disposition and readiness with #21. |
|
|
83
|
+
|
|
84
|
+
Closed historical issues retain their original results with a scope-supersession notice. Do not reopen them to demand DSH comparisons. Close a Pi deliverable as completed only when its Pi criteria have evidence; use `not_planned` for superseded scope, preserving explicit ownership. No new umbrella, test framework or replacement ticket set is introduced.
|