pi-studio 0.9.62 → 0.10.0

package/ROADMAP.md

@@ -2,6 +2,10 @@
 
 This is the current planning source of truth for Studio work. `NEXT-STAGE.md` is historical; `TODO.md` tracks smaller backlog items that are not yet assigned to a release.
 
+## Current status (14 Oct 2026)
+
+`0.10.0` ships the buffer-first workspace as **dual editor mode** (Prompt and Document), on by default, with **single editor mode** one switch away (footer button, `/studio --single`). It went through real course use as private alpha candidates 49–69. The `PI_STUDIO_*` flags described further down are now development overrides only: when any is set it decides, and the mode switch is hidden. Fixes follow in `0.10.x`. The sections below are the history up to this point.
+
 ## Release approach
 
 Studio development follows two tracks:
@@ -21,17 +25,64 @@ Each release should finish with:
 
 Committing, publishing, pushing, and tagging remain explicit actions rather than automatic consequences of implementation or planning approval.
 
-## Agreed next sequence (2026-09-07)
+## Hosting candidate — 2026-09-27
+
+The approved alpha now also includes single-owner Document hosting behind recovery + switching + `PI_STUDIO_DOCUMENT_HOSTING=1`. Move, detached copy, registered blank/text-file editors, existing-owner reuse and atomic file replacement are distinct; Prompt stays anchored. Retained save/metadata decisions, permission checks and uncertain-operation backups remain required. See [README](README.md#additional-opt-in-prototype-single-owner-document-hosting) for deliberate entrypoint restrictions. All flags remain off by default.
+
+The source46 checkpoint passed998 full tests, TypeScript, all eight flag combinations and the bounded hosting workflows. Those are historical source-tree results, not a claim that the subsequent copy-warning fix or a new tarball is validated. Finish candidate-specific checks, dependency/content audits and private installation before any release decision. No feature expansion, publication, commit or normal-install change is authorized by this status update.
+
+## Earlier toolbar follow-up — 2026-09-23
+
+The first exact `0.10.0-alpha.1` artifact passed its fresh-install checks and was tried manually in an isolated runtime through Oliver's existing browser, without browser automation. He then approved a small UI cleanup: group **Add to Prompt → Selection / Whole document**, consolidate **Open / load** with explicit destinations/replacement descriptions, and remove permanent help and duplicate Run-target rows. Keep annotations, the fixed pair and all existing source/copy semantics. This follow-up is uncommitted; rebuild/recheck in new candidate-scoped evidence rather than changing the first tarball. No multiple-document or role-switching work, publication or normal-install change is authorized.
+
+## Previous checkpoint — 2026-09-22
+
+The opt-in **one Prompt ↔ one Document** runtime/tests are locally committed through `40d4462` on `feature/buffer-recovery-foundation`. Development remains in `pi-studio-buffer-dev`; the original checkout is frozen. Bounded local alpha preparation is approved, with unused version `0.10.0-alpha.1` selected after checking npm. The stable installation remains `0.9.60`. Documentation/package preparation is uncommitted; nothing from this branch has been published or enabled by default.
+
+Implemented scope now includes:
+
+- independent Prompt/Document editing, previews, temporary recovery and switching during an identified Run;
+- explicit selection and whole-document snapshot appends, without saving, switching or submitting;
+- guarded response/critique loading and a leading-only annotation header in opt-in Prompt, with copied body text left intact;
+- fixes for stale file-open/grant decisions, mixed-input append activation, history rollover and separate response/Working reading positions;
+- the latest fixes preserving an outstanding response-scroll reset across buffer switches and preventing a delayed numeric restore from being adopted by another view.
+
+Whole-document and composed-Prompt hands-on trials, prepared-payload checks and rendered-preview checks have been completed. Later review findings were reproduced and fixed in separate local checkpoints. Validation of that committed checkpoint includes **718 recorded full-suite tests** on the same runtime/test bytes, plus **202 focused tests and normal typecheck/static checks freshly passed on 2026-09-20**. The latest browser groups cover **41 cases in Brave**, and the same 41 cases across **29 Chrome-app checks plus 12 Chrome Headless Shell 153 checks**. Those Chrome results span two variants, not one full Chrome-app run. The earlier Safari smoke covers the foundation, not the current switcher. No fresh-package or current live-provider release sign-off is implied.
+
+The separate **immediate pane-pointer → view-change scroll race**, whose helper predates this branch, is fixed in approved local commit `40d4462`. Snapshots retain their DOM/view generation and, with recovery enabled, existing buffer/editor consent. Of 38 new VM cases, 29 failed before the fix and 9 controls passed; all 38 now pass. On 2026-09-21, dedicated **Chrome Headless Shell 153** confirmed both stale callback windows in default and two-buffer modes: the old client moves Working from 10259px to 0; the fixed client stays at 10259px. Settled/same-view controls and unaffected-source restoration pass. Timing uses trusted pane input plus deliberately scheduled programmatic view changes, not measured human timing. **240 focused tests, normal typecheck and syntax/whitespace/runtime-boundary checks** freshly pass. The full suite and supplemental check-JS were not rerun; the earlier supplemental baseline was 493 diagnostics, not a passing check.
+
+Oliver's reported **Working appearing empty until interacted with** remains unreproduced and is not claimed fixed. Native view/buffer returns retained Working content and visible geometry; inspected screenshots show content before interaction. Headless Shell did not enter a hidden state during tab switching, so background-tab/visible-browser painting remains unverified. No rendering/CSS workaround was added. All six test attempts, including two retained harness-calibration failures, had backup-first disposal and verified process/port cleanup; 25 samples stayed at normal pressure with no per-trial swap growth. Evidence: `context/pane-scroll-native-20260921_225607/`.
+
+## Next — bounded opt-in prerelease
+
+Follow [PRERELEASE-CHECKLIST.md](PRERELEASE-CHECKLIST.md) for the approved local `0.10.0-alpha.1` preparation. This is a way to try the tested fixed-slot workflow, not a claim that the full architecture below is complete. Eventual npm publication would use `next` and needs separate approval.
+
+1. The pane-scroll fix/tests are checkpointed. Keep the unreproduced blank-Working report open as a documented limitation, accepted for local preparation; publication still needs an explicit decision on the exact artifact and its limitations.
+2. Align the public documentation and validate one exact package candidate: full tests, static/dependency checks, package-content/byte audit, isolated fresh installation and a short credential-free workflow smoke. Record fresh results separately from the historical counts above.
+3. Keep recovery/switching/hosting opt-in and the normal `0.9.60` installation untouched. Further commits, publishing, pushing/tagging, live-provider trials, changing the installed source and default activation remain separate decisions.
+
+Use a dedicated test-only browser with resource monitoring; do not run browser trials alongside full tests/typecheck on a constrained host. The existing suite includes a browser-launching test, so an unconfigured `npm test` is not a browser-free check. Do not begin another broad review loop unless a concrete finding changes the release decision.
+
+After this bounded prerelease, the next proposed functional slice is explicit **Prompt-role switching between the existing buffers**, preserving the previous buffer. Prompt is a submission role an editable document can take; in-place annotation and making a detached prompt copy are distinct operations. This direction is not implemented or approved for execution. Keep annotation/comments in both roles and preserve editor space rather than adding toolbar rows. The separately approved third-flag scope above now covers single-owner Document handoff and independent editor hosts. Linked views, Prompt-role migration and parked prompts remain separate work, not additions to this alpha.
+
+## Agreed architecture sequence (2026-09-07)
 
 1. Ship **`0.9.60`** as the bounded networking/shortcut release. Do not start buffer architecture in this batch.
 2. After that release, implement **`0.10.0`** around an explicit active Prompt, independently editable documents, context transfer, and parked prompt drafts. The design is in [`BUFFER-DESIGN.md`](https://github.com/omaclaren/pi-studio/blob/main/BUFFER-DESIGN.md).
 3. Keep Derived REPL Transcript, editor-engine migration, and arbitrary pane arrangements outside both batches.
 
-Current status:
+### Historical implementation milestones through 2026-09-11
+
+The counts and pending steps below describe those earlier checkpoints; the current status and next actions are above.
 
 - `0.9.60` combines the networking fix with Follow activity, Annotate response, and view-preserving Load response shortcuts, plus accurate draft-replacement protection. The published networking RC remains unchanged.
 - The reporter has not replied to the RC invitation on [issue #4](https://github.com/omaclaren/pi-studio/issues/4) as of 2026-09-07. Exact `sbx` validation remains outstanding; the release proceeds with that limitation documented and the issue open for confirmation or a follow-up fix.
-- `0.10.0` remains design-only. Approval to release `0.9.60` does not authorize buffer implementation.
+- After releasing `0.9.60`, Oliver approved starting the `0.10.0` store/recovery foundation with the current UI unchanged. Stage 1a is implemented and tested. Stage 1b now has an isolated, development-flagged single-editor binding, acknowledged v2 server fallback, explicit recovery inspection/export/choice/retry, and locally completed callback/metadata ownership coverage. The isolated host matrix passes in Brave and Google Chrome, with the earlier targeted foundation matrix also passing Safari 26.5. Seven independent Codex safety findings and two later in-session Astra findings have been reproduced, fixed, and regression-tested. The latest fixes protect empty dirty-file comments-prompt replacement and prevent unread saved comments/scratchpads from being overwritten. All 432 tests and the expanded isolated Brave/Chrome checks pass. The current targeted Safari 26.5 smoke also passes after Allow Remote Automation was re-enabled. Oliver approved a local checkpoint commit on `feature/buffer-recovery-foundation` on 2026-09-09. Recovery remains opt-in; default activation, source switching and release still require separate explicit approval.
+
+- On 2026-09-09, after the local foundation/header checkpoints and visual guide, Oliver approved a small **Prompt ↔ one document** prototype. It is now implemented locally under a second `PI_STUDIO_BUFFER_SWITCHING=1` flag, requiring recovery and a full editable workspace. All **472 tests**, static checks, and isolated Brave/Chrome switching and foundation-only workflows pass. After the first hands-on trial, a fresh read-only review identified held-Enter submission and queued-Follow defects; both were reproduced and fixed with unit/native regressions. This increment is a local, unreleased checkpoint; installed Studio and defaults remain untouched. General tabs, context transfer and parked drafts are still future work.
+- On 2026-09-10, following the successful isolated hands-on/live-model trial, Oliver approved allowing existing-buffer selection during an identified direct Run. This follow-up is implemented as a local, unreleased checkpoint. It keeps local save/open/recovery locks, submitted-Prompt identity and global Stop; queues completion for Prompt, preserves Document's live preview and rejects stale failure settlement. A fresh read-only review found a queued response surviving an empty tree change and hidden activity ownership surviving a disabled policy. Both were reproduced, then fixed with separate approval and failing regressions first. All **500 tests** and isolated Brave/Chrome Run-switching/regression checks pass. These last two fixes used controlled browser events. The resulting checkpoint subsequently passed an isolated live-model hands-on trial; Oliver confirmed the interaction worked and approved backing up/stopping that trial. Further checkpoint/release/installation changes remain separate decisions.
+- Oliver then separately approved **Add selection to Prompt**, limited to Document editor selections. It is a separately approved local, unreleased checkpoint, with **522 tests /80 focused tests** and credential-free Brave/Chrome checks. The action appends a labelled in-memory snapshot without changing Document, replacing existing Prompt content, transferring metadata/resource authority, saving files or submitting. It stays in Document and can prepare the next draft during an identified Run. A fresh read-only review found a literal-label boundary defect and a held press surviving selection A→B→A. Both were reproduced, then fixed with separate approval and failing regressions first. The actual label failure is reachable through a trailing-backtick imported filename; the reviewer's leading-backtick image example did not reproduce. Nine fix cases and eleven existing selection groups per browser, plus ten prior compatibility runs, pass. Preview/companion copying and general tabs remain deferred; no provider trial or independent post-fix sign-off is claimed for this slice. The committed selection slice subsequently passed Oliver's credential-free hands-on trial; both drafts were backed up before stopping.
+- Oliver separately approved the scoped **Add document to Prompt** follow-up. It is a local, unreleased checkpoint, with **549 tests /107 focused tests** and twenty sequential credential-free Brave/Chrome runs passing. It adds the full in-memory Document from raw-editor view without requiring/changing selection, preserves Prompt text and both buffer owners, displays exact addition/capacity counts, and retains the identified-Run and local-operation safeguards. Eighteen new regressions failed before implementation. A fresh read-only review then found stale-open control state and cancelled-press issues; both were reproduced in Brave/Chrome and fixed with separate approval, adding nine regressions. Native overlap testing also covered completed Space-key cancellation. Click-only assistive activation was simulated, not tested through VoiceOver. At this checkpoint, post-fix review and a hands-on trial were still pending. Subsequent trials, reviews and fixes are reflected in the current checkpoint above; installation/default changes and release remain unapproved.
 
 ## 0.9.52 — Rendering and media consistency (shipped 2026-08-28)
 
@@ -163,17 +214,23 @@ The networking review and production lifecycle tests support the reported fix; a
 
 Do not include buffer tabs, context-transfer messages, recovery schema v2, or REPL transcript work in this release.
 
-## 0.10.0 — Active Prompt and editable document buffers (planned)
+## 0.10.0 — Active Prompt and editable document buffers
+
+The first opt-in prerelease is limited to the current two-buffer workflow. The wider architecture and acceptance scenarios remain the target for the full release; see the [prerelease checklist](PRERELEASE-CHECKLIST.md) for the smaller candidate's gates.
+
+The local foundation includes the browser-safe store/codec/migration modules plus a `PI_STUDIO_BUFFER_RECOVERY=1` single-editor/server integration, available through the explicit process-scoped recovery flag. With switching off, the default v1 recovery path, successful editor layout, Run's visible-text destination, and browser-opening actions remain unchanged. Native isolated production tests cover reload, file baselines, metadata persistence/ordering, local-resource grants, PDF focus replacement, comments-prompt consent, late save/refresh, reconnect ownership, independent editors/disk conflicts, watched-to-editable browser tabs, reset retention and startup typing. The opt-in **Source & context → Recover unsaved text…** action inspects/exports retained copies, prepares verified fresh namespaces for explicit recovery/reset, and retries current-text persistence without adopting another branch. Focused ownership regressions now cover completion/import/Pi-editor loads, committed preview DOM and focus actions, Quarto request sources, metadata associations, failed writes, unload beacons and initialization/reset races. Metadata POSTs retain only one in-flight and one latest snapshot per key, time out conservatively, preserve intervening autosave timers, and advance same-writer server ordering only after the persistent-state write succeeds. The opt-in comments-prompt action also protects empty or unknown-baseline file edits and rechecks the captured disk baseline/revision after confirmation. Comments and scratchpad editing/persistence wait for the first successful per-document read; failed, malformed or timed-out reads stay protected and reopening the panel retries. Previously loaded page-local notes remain editable during refresh. Additional native regressions cover corrupt/unscoped copies, stale consent, full exports, server-only recovery, reconnect, failed refresh, pending/failed metadata reads and preserved saved notes. See [the foundation contract](shared/STUDIO_BUFFER_FOUNDATION.md) for contracts and the separate switching opt-in. This remains unreleased, not default activation or the complete `0.10.0` architecture.
+
+The current second-flag prototype has two fixed entries, **Prompt** and **Document**, not a general tab manager. A fresh/v1 file-backed editor remains the Prompt; older v2 roles require an explicit choice. Opening a separate document preserves the Prompt; Files also offers explicit **Use as Prompt…**. Document view returns to Prompt without sending, including under held Enter; a distinct activation sends its visible text. Both buffers retain independent text, baselines, metadata and reading positions through switching/reload. A Follow-on Prompt catches up with a queued response on return without reusing the old response's scroll offset; paused reading remains preserved. An identified connected direct Run permits selecting kept buffers, without relaxing local save/open/recovery or unresolved Pi-draft locks. Stop remains global; Return through the Prompt tab or Run shortcut does not submit or cancel. Completed responses wait for Prompt's Follow setting, and history notifications do not reconstruct Document's live preview. Local mutating operations still gate switching; stale file reads and background responses cannot replace another buffer or steal its view. Recovery retains the pair, and Reset confirms both. Local selection and whole-document actions now append explicit Document editor snapshots to Prompt; neither copies rendered previews or performs companion handoff. Parked drafts and close/add navigation are not implemented. Review this interaction before expanding the UI; see [the current prototype scope](BUFFER-DESIGN.md#current-prototype--before-general-tabs).
 
 Studio's primary loop is prompt construction and response reading. Documents remain useful editable sources, not implicit submission targets or a replacement for Neovim. The design is **one clearly active Prompt, other prompt drafts that can be parked, and independently editable documents**.
 
-The first architectural release should add:
+The full architectural release—not the first opt-in alpha—should add:
 
 - a `StudioBufferStore` with stable buffer identity, explicit prompt/document roles, and separate selected-buffer and active-prompt identities;
 - a prominent **Prompt** view with response/Working alongside it, plus editable document tabs with their own previews;
 - recovery schema v2 with non-destructive migration from the single-document schema;
 - buffer-specific text/baseline, dirty state, cursor, selection, scroll, preview context, annotations, and resource directory, preserving existing disk-revision checks;
-- **Add selection to prompt** and **Copy document to prompt**, including an acknowledged handoff from editable companion views to the full workspace's explicit prompt destination;
+- **Add selection to Prompt** and **Add document to Prompt**, including an acknowledged handoff from editable companion views to the full workspace's explicit prompt destination;
 - editable context snapshots of the visible text, including unsaved edits and source labels/ranges where known; no automatic submission, replacement, file write, or later live update of the copied text;
 - a lightweight parked-prompt picker, with one active submission draft at a time and no per-draft conversation forks;
 - tabs at ordinary widths, a compact searchable picker at narrow widths, and MRU/next/previous/close/reopen navigation;

package/WORKFLOW.md

@@ -19,7 +19,7 @@ Studio uses a **single workspace**:
 
 ## 1) Insert annotated reply header (optional prep)
 
-Adds/updates an `annotated-reply` compatible scaffold in the editor:
+Toggles an `annotated-reply` compatible scaffold in the editor. Studio's generated opener and metadata wording remain unchanged:
 
 ```md
 annotated reply: below
@@ -35,6 +35,36 @@ annotated reply: below
 
 Studio does **not** auto-send this scaffold; it is an explicit editor transform.
 
+### Header recognition contract (local compatibility work; not yet released)
+
+The parser accepts these **whole first lines**, case-insensitively:
+
+- `annotations below`
+- `annotations below:`
+- `annotated reply: below`
+- `annotated reply below:`
+
+Each opener works with the known minimal Neovim block, the full Studio bullet metadata above, or the older plain metadata layout (`original source`, `user annotation syntax`, optional `precedence`, without bullet prefixes or a blank line after the opener). Bullet layouts require a blank line after the opener. Metadata order and the known syntax/precedence wording must match; arbitrary edited instructions are not silently consumed. The accepted syntax examples are `[an: note]` / `[an: your note]`, optionally backticked; the backticked form may carry Neovim's parenthetical below. Full legacy headers remain recognisable when **Strip annotations…** has removed their unbackticked hint.
+
+The minimal Neovim block is:
+
+```md
+annotations below
+
+- user annotation syntax: `[an: note]` (user comments on the accompanying selections)
+
+---
+
+```
+
+All layouts require a blank line before the exact `---` divider and a blank line after it. LF and CRLF are accepted without normalising or trimming the remaining body. Source/precedence lines and a footer are not required for the minimal form, whose entire remaining text is body. A backticked syntax example is not an annotation.
+
+Detection, toolbar summaries, toggling and header removal share the same parser. Prose mentions, opener prefixes, quoted/indented/fenced examples, and arbitrary later dividers do not count. An exact opener with an unknown or malformed following block prompts a warning and leaves the editor unchanged rather than adding a duplicate header.
+
+Full source-bearing legacy wrappers retain their optional footer convention: only the exact terminal `\n\n--- end annotations ---\n\n` suffix (or its all-CRLF equivalent), outside a Markdown backtick/tilde fence, is removed. Quoted/indented/prose marker text and extra trailing whitespace are preserved. Minimal headers never remove a footer. Inserting Studio's usual wrapper preserves any pre-existing body marker; its generated footer is omitted if it would fall inside an unclosed fence, so toggling still preserves the body.
+
+This compatibility change does not switch the installed Studio version or change Neovim's emitted wording. Deployment/release is a separate decision.
+
 ## 2) Run editor text (plain send)
 
 Sends current editor text to the model. If `Annotations: Hidden`, `[an: ...]` markers are stripped before send.
@@ -111,7 +141,7 @@ Rules of thumb:
 
 1. `/studio --last` opens with editor loaded and no required mode selection.
 2. **Run editor text** respects annotation mode (`On` send as-is, `Off` strip `[an: ...]`) and returns response to right pane.
-3. **Insert annotation header** updates the scaffold source metadata without duplicating headers.
+3. **Annotation header** recognises all supported openers without duplication; toggling preserves the body and refuses malformed leading header blocks.
 4. **Critique editor text** runs on current editor text and returns structured package when model complies.
 5. Structured critique helpers (`Load critique (notes)` / `Load critique (full)`) enable only when critique structure is present.
 6. Loading response/critique back into editor never loses draft unexpectedly.