pi-smart-compact 9.7.1 → 10.0.1

package/CHANGELOG.md

@@ -1,5 +1,726 @@
 # Changelog
 
+## [Unreleased]
+
+## [10.0.1] - 2026-09-28
+
+### Removed
+
+- Name-based startup scanning of unrelated extensions and instructions to
+  disable or uninstall them. An unrelated command name no longer produces a
+  guessed extension-conflict warning. Actual compaction lifecycle and
+  continuity-state checks remain in place.
+- Unrelated extension lists and compatibility-policing guidance from current
+  product documentation and release notes. Required and optional dependency
+  instructions, legacy session-data support and license attribution remain.
+
+## [10.0.0] - 2026-09-28
+
+**Pi Continuity**: the stable release of the context-hygiene and session-continuity
+rework. This release includes the work recorded in the unpublished
+`9.8.0-canary.*` candidates below; those candidates were never npm releases.
+
+### Upgrade from 9.x
+
+- Requires Pi **0.87.1+** and Node.js **22.19+**. The npm package remains
+  `pi-smart-compact`; commands, settings namespace and stored paths are not
+  renamed. See the [upgrade guide](./docs/guide.md#upgrade-from-9x).
+- A bare `/smart-compact` opens Home in the TUI. Use **Compact now** for the
+  interactive flow; print/RPC/SDK use still runs compaction directly. Apply a
+  reviewed summary with **A**, not Enter.
+- Agent tools load on demand through `smart_tools` by default. **Always
+  available** remains an explicit alternative; tool visibility is not permission
+  to bypass compaction or memory confirmation controls.
+- Mnemopi, its optional Bun runtime and image rendering are separate optional
+  components, not automatic downloads. Readiness shows the correct install
+  command. The default local graph does not require these components.
+- Claude subscription requests require the separate adapter with final-payload
+  normalization; the published adapter `0.2.2` alone does not cover nested calls
+  (upstream PR #10).
+
+### Added
+
+- Recoverable context hygiene: queued manual and automatic cleanup, large-output
+  offload, checkpoint/rewind, archived-output search and paged reads. Automatic
+  cleanup uses pressure, catalog-price break-even or a cold-cache boundary;
+  shell calls, failures, instructions and recent work retain their protections.
+- Session anchors, read-only recall across earlier sessions and branch navigation
+  with required carryover. Handoff opens a fresh session from recorded state
+  without a model call; lineage retrieval reaches parent-session evidence.
+- A five-choice Home screen, task-oriented settings, effective-state/readiness
+  diagnostics, storage reporting and a host prompt-cache ledger.
+- Exclusive project-memory backends: the local graph, an optional local Mnemopi
+  engine or an existing Hindsight server. Explicit saves need confirmation;
+  stable backend-bound refs support resolution without silent store fallback.
+- Opt-in background preparation, provider-native compaction, image snapshots
+  and an explicitly loaded RTK companion. EESV text remains the default;
+  experimental provider/image results are not proof of semantic fidelity or
+  billed savings.
+- Read-only session replay estimates and paired continuation/memory evaluation
+  tooling, with offline defaults and explicit approval for live provider use.
+- GitHub-release-driven npm Trusted Publishing. `publish.yml` uses OIDC instead
+  of a stored npm token, preserves the full `prepublishOnly` release gate and
+  maps stable releases to `latest`, prereleases to `next`.
+
+### Changed
+
+- Documentation now separates quick start, usage/reference and historical
+  evidence; long guides have clearer navigation and troubleshooting. Corrected
+  local-memory defaults, automatic-cleanup tool requirements, settings refresh
+  behavior and optional Hindsight authentication. Shipped pages link to
+  repository-only material without depending on excluded local files.
+- Pi Continuity has a unified return-path mark, outlined vector banner and
+  reproducible SVG/PNG exports. The catalog image URL is unchanged; the README
+  uses a font-independent PNG banner. Asset ownership and export instructions
+  are documented in `docs/identity.md` and `assets/README.md`.
+- Contributor and issue templates reflect current tools, privacy boundaries
+  and package contents. Repository paths and runtime identifiers are unchanged.
+- Dated reports and pilot data moved to `docs/reports/` and, like the review
+  findings, are no longer packed into the npm package (about 150 KB less per
+  install; `docs/README.md` still indexes them).
+- `replay-eval` gained `--since=DAYS` and `--progress`, and a session file that
+  changes while it is being read (a live session) is now skipped and counted
+  instead of failing the whole run.
+
+### Fixed
+
+- Context edits, queued navigation and staged compactions revalidate branch,
+  session, model, configuration and cancellation state before applying. Rewind
+  and pivot change context, never files or external side effects.
+- Archived-output integrity, lineage authorization and target-bound memory refs
+  reject tampered or mismatched evidence instead of silently reconstructing it.
+- Provider calls use the requesting Pi session's model runtime, preserving
+  registered provider overrides. Native replay requires matching provider state
+  and the exact host summary wrapper.
+
+### Release decision
+
+The release owner explicitly approved **10.0.0 stable** instead of another
+canary. The available canary report was `HOLD`, with no completed candidate
+cohort; this is a version-specific release-owner exception, not a `PROMOTE`
+result. Deterministic checks and offline pilots do not establish live-model
+quality, billed savings or a completed production observation window.
+
+## [9.8.0-canary.8] - 2026-09-28
+
+Integration candidate for the maintainer's daily Pi (real-session data
+collection on the cold-cache trim and the host cache ledger); not published.
+Offline receipts, CI and the release audit are not a live evaluation or a
+canary promotion.
+
+### Changed
+
+- The Mnemopi engine (`@oh-my-pi/pi-mnemopi`), its `bun` runtime and the
+  image-snapshot renderer (`@resvg/resvg-js`) are no longer installed with
+  the extension. They are optional peer dependencies for features that are
+  off by default; a plain install used to download about 260 MB for them
+  (macOS arm64). When a selected feature lacks its component, Readiness,
+  the effective-state report and the failure notices show the exact
+  `npm install … --prefix <Pi install root> --legacy-peer-deps` command
+  (see README "Optional components"). The release audit now installs the
+  components the user's way into a Pi-style npm root and proves they survive
+  a Pi update.
+
+### Added
+
+- Automatic cleanup held for a cold prompt cache now waits while Pi's cache
+  warming keeps the entry alive, and stops that warming once a refresh no
+  longer pays by Pi's $0.05 rule, counted net of the cache write the held
+  trim avoids; Home notes the stop. The host cache ledger and
+  `replay-eval` count a `cache_warm` refresh as keeping the prefix alive.
+- At session start a notice names loaded extensions known to compact or edit
+  the same history (pi-openai-toolkit, context-fold, pi-fold,
+  pi-context-prune, pi-dcp, pi-toolkit's `context` tool), matched by exact
+  command/tool names and whole package path segments, and advises keeping
+  only one loaded. Nothing is blocked; unknown extensions are not detected.
+- `/smart-compact handoff [-- note]` opens a new Pi session seeded with one
+  anchor message assembled from recorded state: the note, the latest branch
+  anchor, the continuity ledger (last Continuity compaction, else the saved
+  branch state), `pinPaths`, a memory recall through the selected store, and
+  pointers back to the parent session. No model call writes it; it is
+  scrubbed and capped at 16,000 characters, cutting recall first. Nothing
+  opens when there is no anchor, ledger or note.
+- Home → History & recovery → **Hand off to a new session**: write a note,
+  review the seed (size, sources, full text), then confirm; the selection
+  starts on Go back. `/smart-compact handoff dry-run [-- note]` shows the seed
+  and opens nothing (TUI preview, a message in other UI modes).
+- `smart_context` `status`/`search`/`read` take `scope: "lineage"`: they also
+  reach archived output of the sessions this one was handed off or forked
+  from, read-only, following `parentSession` up to 3 levels (files up to
+  64 MiB; cycles and missing files end the walk). Parent sources carry
+  `session` and `depth`; each parent's own archive records authorize and
+  verify its outputs. The default `session` scope is unchanged. The handoff
+  seed now points at it instead of `/resume`.
+- Automatic cleanup (`contextHygieneEnabled`) no longer waits only for
+  context pressure. A ready batch commits at a turn boundary under pressure,
+  or when the model's catalog prices say it pays back its prompt-cache rewrite
+  within 24 requests (`N* = ((w - r) × T) / (r × X)`); otherwise it is held
+  and sent with the first request after the cache expired, then committed
+  when that turn completes. Trim entries record the cause (`pressure`,
+  `break-even`, `cold`, `manual`, `agent`); `smart_context` `status` reports a
+  held batch as `deferredTrim`. The rule uses catalog price ratios and token
+  estimates, not measured cache behavior.
+- `bun run replay-eval --sessions=<dir|file>` replays recorded sessions read-only
+  and estimates prompt tokens and catalog-priced cost per request under the
+  `none`, `pressure` and `timed-<N>` trim policies (`--break-even`,
+  `--rebuild-min`, `--limit`, `--json`), next to the recorded usage. Replay
+  estimates only; see `docs/evaluation.md`.
+- The compaction result returned to Pi carries the provider-reported usage of
+  the applied run (EESV stage calls or the provider-native compaction
+  request), priced at each route model's catalog rates, so Pi's session
+  totals and cost include the extension's own work. Cached or estimated runs
+  contribute nothing rather than a guess.
+- A foreign compaction (another extension's, or Pi's built-in one) no longer
+  passes silently: a prepared Continuity summary it displaces is recorded as
+  `discarded` (`native-apply:foreign`) in the metrics log and a once-per-
+  session notice names the winner; Pi's built-in compaction is reported only
+  while automatic compaction is on.
+- `maxContextTokens` (default `0`, off) makes automatic trigger percentages
+  (`minContextPercent`, the background preparation window and the automatic
+  admission gate) count against `min(model window, maxContextTokens)`. Model
+  requests, summary sizing and hard headroom checks keep the real window.
+  Settings › Compaction has a row for it, and Home warns when a model window
+  above 400k tokens is uncapped while automatic compaction is on.
+- A session-local host prompt-cache ledger records the provider-reported
+  input, cache-read and cache-write tokens of Pi's own requests and classifies
+  cache rebuilds (uncached ≥ 16,384 tokens and ≥ half the prompt) as following
+  a committed Continuity edit (trim/rewind, pivot, compaction), idle expiry of
+  the cache lifetime, or foreign. Home › Readiness & details shows the tallies;
+  the third foreign rebuild in a session shows one notice. `smart_context` and
+  navigation gain an `onContextEdit` hook that fires only once an edit is on
+  the branch.
+- Trimming also archives old, successful shell (`bash`) output of 4,096+
+  characters. The shell call itself stays in context; errored results and
+  turns that mix shell with writes or unknown tools stay whole. Rewind still
+  keeps shell exchanges.
+- Old `smart_context` `read` pages are trimmable like read-only output; the
+  marker points at the original source ID so the agent re-reads the source.
+  Other `smart_context` results (`status`, `search`, `plan`, `trim`, `rewind`)
+  are never trimmed.
+- Trim and rewind records store a SHA-256 and length per archived output
+  (`archives: [{ id, sha256, chars }]`). `smart_context` `read`/`search`
+  refuse text that no longer matches its record, `status` sources show
+  `hashed`, and a rewind withholds mismatched outputs from recovery and says
+  how many. Records from earlier versions have no hash and read as before.
+
+### Changed
+
+- Home's `Clean up tool output` row shows `held for a cold cache` with the
+  estimated savings and pay-back when automatic cleanup is holding a batch;
+  selecting it applies the batch at the next completed turn. Settings and
+  profile texts for automatic cleanup now name all three commit causes; the
+  `handoff` "nothing to hand off" notice says how to record something.
+- Trim markers are now a deterministic digest of at most 6 lines and 400
+  characters: the retrieval line, the call's subject (read path, first shell
+  command line, or search pattern), the first non-empty output line, and up
+  to three error/failure/warning lines, each whitespace-normalized and cut to
+  100 characters.
+- Trimming archives superseded output first: read-only results whose path a
+  later call writes, edits or deletes (including literal `bash` targets),
+  then results whose path a later plain `read` without `offset`/`limit`
+  shows again in full, then the rest, each in session order, within the same
+  32-output cap. Their markers add `(superseded: edited later)` or
+  `(superseded: read again in full later)` to the
+  subject line. Paths match after `path.normalize` only. Eligibility is
+  unchanged; `smart_context` `plan` reports `superseded`.
+
+### Fixed
+
+- Sessions with automatic compaction on `background` and `contextHygieneEnabled`
+  off trimmed at break-even or on a cold cache since the timing rule landed;
+  the background strategy again trims only under pressure, as before.
+  Break-even and cold-cache cleanup need `contextHygieneEnabled`. This also
+  restored the offline evaluator's `no-compaction` baseline under
+  `--background-prep` (it had started trimming, and Pi's idle cache warming
+  stopped at every rebuilt boundary).
+- A held automatic trim is applied through Pi's `context_with_system` event,
+  not `context`: a changed `context` result makes Pi collapse
+  mid-conversation system messages into one head, so the cold request and
+  the committed transcript would differ. Verified against the checkout's
+  `ExtensionRunner.emitContext`.
+- The offline task evaluator (`release:audit`) no longer reaches the network
+  when `rg` is missing: offline arms set `PI_OFFLINE=1` and fail before the
+  first round without ripgrep; CI installs it. The offline guard now names the
+  first blocked request and its caller.
+- A trim that a cold request already carried was dropped at the end of an
+  aborted or failed turn, at a boundary busy with preparation or compaction,
+  and after a queued manual change: Pi records the interrupted response with a
+  fresh timestamp, so the next request looked warm, went out untrimmed and
+  rewrote the prefix the interrupted request had just cached. The carried trim
+  now stays in every request until a boundary commits it, a newer context
+  rewrite invalidates it, or the session changes. Turning automatic cleanup off
+  forgets a held or carried trim.
+- The cold-cache check took the prompt-cache lifetime from the last response
+  only; a fully cached or interrupted response after a 1-hour write made a
+  1-hour prefix look 5 minutes old, so the trim could be applied while the
+  entry was still warm. The lifetime now comes from the last response that
+  wrote cache, as the host cache ledger keeps it.
+- Automatic trims need a reachable `smart_context`, as artifact offload does:
+  digests point the model at that tool, so with agent tools off nothing is
+  trimmed automatically. `Readiness & details` says so on the
+  `Context hygiene` line. Manual cleanup from Home is unaffected.
+- A queued context change (manual cleanup, agent checkpoint, rewind or trim)
+  that met an aborted or failed turn vanished silently; it now leaves the usual
+  "Context operation not applied" note.
+- A committed rewind was counted as a `trim` in the host prompt-cache ledger;
+  it is now `rewind`.
+- Review reports under `docs/findings` stay out of the npm package.
+- Settings opened from Home now apply limits, pinned paths, backup
+  directory, Hindsight and model rows to the running session at once, as the
+  standalone `/smart-compact settings` screen does; they used to be saved
+  only.
+- Home → Settings → How it runs → **This branch** opens the branch-only
+  settings again; Enter on it did nothing.
+- The Home header reads `Automatic: prepare in background` for the
+  `background` strategy instead of `when idle`.
+- When the readiness check fails, Home's `Status & help` row shows
+  `unavailable` with the first line of the error instead of `checking`
+  forever.
+- Mode budget rows (Fast, Balanced, Thorough) update their "N changed" count
+  after you edit a budget, and a failed budget or Hindsight reset reports the
+  error instead of failing silently.
+- `/smart-compact dashboard` outside the TUI says it needs TUI mode and
+  points to `/smart-compact metrics` and the HTML dashboard path; it used to
+  do nothing.
+- `/smart-compact storage` and `/smart-compact metrics` without a UI print
+  their report: to stdout in print mode, to stderr in JSON/SDK mode (RPC
+  keeps the notice). Both were silent in JSON mode and `metrics` in print
+  mode.
+- Metrics p95 latency uses the nearest rank; with 20 runs it was the slowest
+  run.
+- The metrics dashboard pages to the terminal height (at most 24 lines), so
+  the position line and key hints stay visible in short terminals.
+- `/smart-compact storage` no longer marks the scan incomplete (and every
+  owner `unknown`) when a session file holds a line longer than 1 MiB, such
+  as a large tool result.
+- An unreadable Hindsight receipt ledger is no longer read as "no pending
+  saves": resolve refuses to delete and saves are refused, both naming the
+  ledger file, which is left untouched for repair.
+- Mnemopi readiness reports `ready` when the package and a supported Bun are
+  found (worker and database still unverified); the package check no longer
+  fails for an installed package.
+- A settled or background auto-trigger whose host never reports back releases
+  the session after the hook budget plus `autoTriggerTimeoutMs` and reports
+  it once, so later triggers can run.
+- A staged candidate confirmed for another session is discarded and recorded
+  instead of lingering and later surfacing as missing local records.
+- Anthropic native compaction reads the top-level usage when the response has
+  an empty `iterations` list; it used to record zero tokens.
+- Stored native compaction state larger than 1,024 items or 4 MiB is rejected
+  instead of being replayed.
+- Native compaction metrics classify tool runs like EESV runs and record the
+  run's latency and provider-cache hit rate instead of zeros; native dry runs
+  and declined native runs now appear in metrics.
+- A failed apply (native or EESV) clears only its own staged candidate, and
+  the once-per-route native skip notice no longer repeats for every session
+  after 500 sessions.
+- A resolved or superseded project-memory fact could come back as active in a
+  project with more than 2,000 facts: pruning removed the resolved markers
+  first, so a later compaction that still saw the old text re-added it. Active
+  facts and resolved markers now have separate 2,000-entry limits, each
+  trimmed oldest first.
+- Resolved saved project memories count toward the resolved-marker limit
+  instead of piling up until an explicit forget; active saved memories are
+  still never trimmed.
+- Deleting project-memory facts no longer scans every graph edge: edge
+  targets are indexed, including in existing memory databases.
+- The native continuity handoff no longer stops working for a directory after
+  a crash while staging: a lock whose process is gone, or which is older than
+  one minute, is taken over. Replacing a handoff keeps the previous one until
+  the new one is written, and a handoff recorded for another scope is left in
+  place instead of being deleted.
+- Native compaction now counts against the run's output budget and records
+  its real prompt and output tokens in the run budget (it used to spend
+  nothing); an exhausted call or output budget fails the native attempt
+  without sending a request.
+- Native compaction's after-compaction estimate counts the system prompt and
+  tool schemas as fixed context, like the smart summary planner, instead of
+  shrinking them with the compacted messages; the estimate was optimistic.
+- Native compaction now prepares the same scrubbed conversation backup as the
+  smart summary (`backupEnabled`), written after Pi confirms the compaction;
+  it used to write none.
+- Hindsight: a resolve marks only the receipts it checked as deleted, so a
+  save that lands during the delete stays open; a completed save or delete
+  whose local receipt cannot be updated (e.g. ledger locked) is reported
+  (`unknown`, or a warning on delete) instead of failing the tool, and the
+  next `smart_recall` reconciles it.
+- Hindsight status checks use the operation id the server acknowledged when
+  it differs from the one sent; they used to stay `unknown` forever.
+- Hindsight: `smart_recall` marks a receipt `failed` (`not_found`) once the
+  server has reported its operation missing for 24 hours, so a full receipt
+  ledger drains; the ledger-full message names the ledger file and no longer
+  claims `smart_recall` alone clears it.
+- Hindsight recall evidence turns line breaks and other control characters
+  from the server into spaces, so remote text cannot forge `Ref:` or
+  provenance lines.
+
+## [9.8.0-canary.7] - 2026-09-27
+
+Local-only integration candidate; not published or installed in daily Pi.
+Offline receipts, a real-terminal walkthrough and loopback proofs are not a
+live evaluation or a production canary promotion.
+
+### Added
+
+- Session navigation, owned by Pi Continuity: named anchors with summaries,
+  read-only search of anchors from earlier sessions, and returning to an
+  anchor on a new branch with a required carryover. Home → History & recovery
+  → Session navigation, `/smart-compact context`, and the `smart_navigation`
+  agent tool (`view`, `recall`, `anchor`, `pivot`). Anchors recorded earlier by
+  pi-toolkit's `context` tool stay readable. The Anthropic anchor prompt-cache
+  marker and the footer anchor status are included.
+- On-demand agent tools: with `toolLoading: "lazy"` (default) the agent sees
+  only the `smart_tools` loader (about 400 bytes of tool JSON in a stock Pi
+  request) and loads the `navigation`, `history`, `memory` or `compaction`
+  group when needed; loaded groups reset at session start, branch change and
+  compaction. `eager` shows every permitted tool; `off` shows none while Home
+  and the navigation panel keep working. The context-management guide is read
+  only on request through `smart_tools` and is never injected into the system
+  prompt; Pi does not list it as a skill.
+- Settings → Agent tools & navigation: `toolLoading`,
+  `contextNavigationEnabled`, `contextRecallEnabled`, `contextPivotEnabled`,
+  `contextAnchorCacheEnabled`, `contextAnchorStatusEnabled` and
+  `contextGuidanceEnabled`. Existing settings are preserved; turning
+  navigation off keeps recorded anchors and the other switches.
+
+### Changed
+
+- Requests the extension makes itself (EESV stages and provider-native
+  compaction) go through the requesting session's public model runtime
+  instead of pi-ai's standalone completers, so request-time auth and provider
+  overrides registered by other extensions apply. Explicit caller API keys and
+  headers are no longer sent, so stored OAuth is never bypassed; stage auth is
+  an availability preflight only.
+- Removed the pi-toolkit cooperation contracts: the
+  `pi-toolkit:adapt-provider-request` channel, the `pi-toolkit:context-pivot`
+  pause, the `piToolkit.context.thinningEnabled` fixture and the task-eval
+  `--toolkit` option (every arm now runs with `toolLoading: "eager"`).
+  pi-toolkit's auto-context must not be loaded together with this extension.
+  Claude subscription routes need the separate `pi-claude-oauth-adapter`; the
+  published `0.2.2` normalizes the request body only for Pi's own turns, so a
+  build that normalizes the final payload inside its provider is required for
+  parity on nested requests. Such a patch was prepared and verified against
+  `0.2.2` in isolation; it is not part of this package.
+- Tool guidance for `smart_recall`, `smart_save_memory` and `smart_compact`
+  moved from Pi's system-prompt tool metadata into the tool descriptions, so
+  loading a group on demand does not rewrite the system prompt.
+- The session, context-compat and native-host pilots run without a pi-toolkit
+  checkout; the native-host pilot loads the standalone adapter from
+  `PSC_CLAUDE_OAUTH_EXTENSION` and classifies its quota preflight as a
+  non-model request.
+- LICENSE credits the pi-toolkit auto-context origin (MIT, Ersin Tarhan, with
+  the pi-provider-kimi-code and pi-better-messages-cache notices).
+
+### Fixed
+
+- A pivot queued by the agent was silently dropped when a threshold
+  compaction started in the same idle window; compaction now pauses while a
+  pivot is queued or applying.
+- Returning to a human-made anchor kept the carryover but dropped the anchor
+  and its summary; anchors are now labelled natively so the anchor text stays
+  in context.
+- Artifact offload never ran in the default on-demand mode because
+  `smart_context` was not yet active; offload now runs while the agent can
+  reach `smart_context` (active or loadable) and stops with agent tools `off`
+  or `smart_context` hidden with `/tools`.
+- The "Context OK" notice no longer refers to pi-toolkit.
+
+## [9.8.0-canary.6] - 2026-09-27
+
+Local-only corrected evaluation candidate; not published or installed in daily
+Pi. Canary.5 was rejected before paid calls: a synthetic raw `KERN_PROCARGS2`
+probe retrieved another host process environment despite native sandbox rules.
+Its frozen archives remain unchanged.
+
+### Fixed
+
+- Replace native macOS tool isolation with fresh, networkless Linux containers.
+  Mount only the synthetic fixture and owned HOME; keep the image read-only,
+  drop capabilities, limit resources, and isolate host/sibling process tables.
+  Resolve the prebuilt runtime image to an immutable ID before execution.
+- Create each container before starting it, then remove the complete container
+  on exit, cancellation or timeout, including detached children. File reads
+  remain byte-preserving and bounded; startup probes fail before provider use.
+- Preserve evaluator environment/transport restoration even when sandbox
+  teardown fails. A local Docker daemon and separately built runtime image
+  are evaluation prerequisites, not extension runtime dependencies.
+
+## [9.8.0-canary.5] - 2026-09-27
+
+Local-only evaluation-hardening candidate, retaining the canary.4 UI and
+identity changes. Not published or installed in daily Pi. A bounded synthetic
+live pilot is not production canary promotion evidence.
+
+### Fixed
+
+- Load only the selected custom provider/model and frozen credential in task
+  evaluation; API keys no longer fail an OAuth-only expiry check. Credential
+  commands, ambient templates, refresh and mutation are refused.
+- Cap main responses explicitly; refuse output reservations that do not fit
+  instead of silently shrinking a later response. Count cumulative Anthropic
+  stream usage once, retain missing values as unknown, and snapshot per-arm
+  request classes, actual usage and latency. JSON mode now emits its report.
+- Observe live host tool results and the context actually sent at each probe,
+  rather than offline-only counters or pre-compaction branch history. An
+  archive lookup question alone no longer counts as delivery of its answer.
+- Sandbox live model tools, fixture reads and oracle processes with a clean
+  child environment and native macOS filesystem/network restrictions. Startup
+  denial checks fail closed; unsupported platforms cannot enter live mode.
+
+## [9.8.0-canary.4] - 2026-09-27
+
+Local-only UI, identity and settings candidate; not published or installed in
+daily Pi. Live task comparison and production canary promotion are separate
+checks; offline validation alone does not satisfy the promotion gates.
+
+### Changed
+
+- Make Mode the only user-facing compaction selector. Advanced Mode budgets
+  retain existing Fast/aggressive, Balanced/balanced and Thorough/light values;
+  editing or resetting Mode does not rewrite legacy profiles or budget overrides.
+- Run the adversarial gate on every CI pull request and push to main, matching
+  the local release checks. Latest-Pi compatibility remains scheduled/manual.
+
+- Adopt **Pi Continuity** as the product identity for context hygiene and session
+  continuity. The npm package, repository, commands, tools, settings namespace
+  and stored data identifiers remain unchanged; this is not a package migration.
+- Replace the compaction-only artwork with a generated PNG continuity mark and
+  self-contained SVG banner, retaining the established asset URLs.
+- Restructure the README as an entry point, with separate user, configuration,
+  architecture and evaluation references. Mark historical experiments as dated
+  evidence and distinguish current unpublished behavior from the npm release.
+
+- Simplified Home to five task-oriented choices, with effective context and
+  automatic/agent status above the list. Settings, recovery and diagnostics
+  remain accessible without competing with the primary compact action.
+- Replaced technical setting labels with readable choices while preserving
+  defaults, persisted values, branch overrides and experimental confirmation.
+  Compact-picker cancellation now returns to Home.
+- Compact and review screens show the decision first, with technical details
+  under `D`. Capacity refusals explain the next step; provider fallback and
+  verification warnings remain visible.
+
+### Fixed
+
+- Normalize legacy profile-only `aggressive` settings to Fast on load, while
+  preserving explicit modes, custom budgets and the stored settings file.
+- Result/help scrolling and narrow-terminal controls: summaries can be read to
+  the end, selected truncated settings are disclosed in full, and compact
+  actions remain visible when planner details need paging.
+- Review no longer describes unresolved verification gaps as already patched.
+  Approval still requires `A`; `Enter` does not apply a candidate.
+
+## [9.8.0-canary.3] - 2026-09-25
+
+Local-only reliability candidate; not published or installed in daily Pi.
+Existing frozen candidates and user settings remain unchanged. Offline proof
+does not establish live provider reliability or satisfy canary promotion gates.
+
+### Fixed
+
+- Deterministic fallback keeps each constraint’s full 300-character extraction
+  bound and omits redundant category labels. The previous 200-character
+  decorated preview could contradict its own source, causing repeated
+  verification failures regardless of model. The verifier remains unchanged
+  and still rejects genuinely contradictory output.
+- Run-wide cancellation is no longer treated as a recoverable generation
+  failure: it stops subsequent synthesis, verification, staging and apply.
+  Timed-out previews cannot report success; host cancellation is recorded
+  once as neutral `cancelled`, separately from `timeout`. Manual deadline
+  notices no longer promise an automatic native fallback. Latency defaults
+  and configured budgets are unchanged.
+- Automatic behavior preset explicitly selects the existing `settled` trigger,
+  which works with Pi auto-compaction off. Native-hook readiness states its
+  host dependency and unknown host setting; threshold labels distinguish a
+  replacement gate from actual scheduling. Existing disabled/native-hook
+  configurations are not silently changed.
+
+### Changed
+
+- Ship only runtime JavaScript entries (`index`, `rtk`, `mnemopi-worker`) and
+  declarations. Evaluation/report commands run source scripts from a checkout;
+  installed `dist/*-eval.js` and `dist/telemetry-report.js` CLI paths are removed.
+  Runtime context hygiene, continuity, recovery and memory features remain.
+- Replace source/build-string and wording-only assertions with pipeline
+  cancellation regressions, installed-package checks and real Pi lifecycle
+  proof.
+
+## [9.8.0-canary.2] - 2026-09-25
+
+Local-only candidate; not published. Prepared for the fresh canary package
+paired with the partner pi-toolkit build. The deterministic release gate
+passed locally (four-project typecheck, full test suite, adversarial gate,
+benchmarks, build, installed-package audit with package-owned Bun, and Pi
+0.87.1/latest compatibility), but offline validation never replaces the
+required live production canary evidence.
+
+### Added
+
+- `/smart-compact` Home: a bare TUI invocation opens one keyboard screen with
+  Compact now (tokens/percent or the blocking reason), Queue local cleanup,
+  Setup & readiness, How it runs, Output, Models, Memory, History, Metrics and
+  Advanced settings. Behavior presets (Manual, Agent can compact, Local cleanup
+  only, Automatic cleanup + compaction) are derived from the exact persisted
+  flags, apply as one atomic settings patch, keep branch overrides visibly
+  separate, and leave the unchanged built-in default labeled as the default.
+  Output presets mark Visual hybrid as explicit opt-in and Native as
+  experimental with a second confirming Enter, showing why a preset cannot
+  apply for the current chat model.
+- `/smart-compact trim` and the Home cleanup row queue the same zero-LLM trim
+  through the one `smart_context` controller: no model call, no forced turn, the
+  first next provider request is sent untrimmed, and the edit applies at the
+  next natural completed-turn boundary. A pending Toolkit pivot or a newer
+  boundary change cancels the queued request with a visible notice.
+- `/smart-compact storage`: a strictly read-only inventory of saved tool output
+  (totals, per-session in-use / not-referenced-in-scan / unknown status, scan
+  coverage, self-managing retention neighbors). There is no `--clean` or
+  artifact garbage collection: sessions can live outside Pi's sessions root and
+  a running session can write new references at any time, so
+  unreferenced-in-scan is never safe-to-delete.
+- Model capacity feasibility: model rows and readiness use local estimates of
+  planned stage requests from a retention-aware chunk/extraction plan, with
+  requested output and 4,096 tokens of SDK headroom reserved — not the whole
+  conversation against the stage model’s window. Ineligible rows are disabled; sizes that only exist
+  after generation (explorer feedback, assembly, repair) cannot be pre-known;
+  every actual dispatch re-estimates and revalidates the built request before
+  the provider is contacted. Snapshots refresh after mode/model/privacy
+  changes; the UI never refreshes authentication to build them.
+
+### Changed
+
+- The selected memory backend is exclusive. With Hindsight selected, confirmed
+  saves and recall use only the user's existing configured server and bank —
+  Smart Compact never installs or starts a server — with no local copy or
+  fallback (`hindsightLocalFallback` is removed and reported as a stale ignored
+  key); failures report themselves instead of quietly using another store.
+  Mnemopi and local behave symmetrically, inactive stores are preserved
+  untouched, and `scope: "session"` is unsupported on remote backends (skipped,
+  nothing else is read). Structured continuity state, backups and artifact
+  spill remain session mechanisms independent of the backend choice.
+- Mnemopi runtime resolution is package-owned first: the pinned `bun` optional
+  dependency (1.4.2), then its platform `@oven/*` package, then a supported
+  Bun (>=1.3.14) on PATH. Normal installs ship the optional runtime and engine,
+  so no global Bun is required; nothing installs itself at use time and missing
+  pieces fail closed before any memory request is submitted.
+- RTK companion eligibility is exactly bare `git status`, `cargo test` and
+  `bun test`: `bun test` joined after an RTK 0.50.0 paired native/filtered
+  runner check showed exit-code and failure/load-error parity plus full recall
+  of the filtered output, with the command executed exactly once. `git diff`,
+  `tsc` and vitest 5 stay untouched (measured lossy or growing), `npm test` and
+  `node --test` have no rule, and commands with arguments or shell composition
+  remain passthrough.
+- Native compaction and replay are documented honestly about cache effects:
+  the request prefix legitimately diverges across a native boundary, while
+  checkpoint/rewind trims keep the checkpoint-stable prefix and archive removed
+  middle evidence for retrieval.
+
+### Verified
+
+- The release audit now runs the installed, packed Mnemopi worker under stock
+  Node with a PATH that offers no Bun — proving the package-owned runtime — and
+  adds missing-engine and missing-Bun fail-closed negatives that create no
+  store, make no model/network request, and keep the unselected local engine
+  from starting.
+- Long-session storage durability is pinned with real `SessionManager`
+  fixtures: artifacts aged past 20 days (timestamps aged deterministically, not
+  a wall-clock soak), retrieval through the public `smart_context` consumer
+  after actual reload and fork, fail-closed missing/tampered bytes, and
+  per-origin caps without losing earlier evidence.
+
+### Unchanged
+
+- The frozen 9.8.0-canary.0/.1 archives and the daily installation are
+  untouched; no publish, tag or commit accompanies this candidate. Exact
+  paired-archive hashes are recorded by the release owner at final packaging.
+
+## [9.8.0-canary.1] - 2026-09-25
+
+Local-only polish candidate; not published. Offline validation does not replace
+the required live production canary evidence.
+
+### Added
+
+- Stable, backend- and target-bound refs in confirmed memory saves/recall. Resolve requires the ref instead of exact preview text, names the actual store for consent, and does not retarget when settings change. Hindsight refs additionally bind project and document; unrelated remote documents have no actionable ref.
+- Explicit derived-only reset versus full local graph deletion, with saved/derived/legacy counts, confirmation, and clear exclusions for other backends, compaction state and backups.
+- One read-only effective-state view for preflight (`S`), metrics and dashboard: routes, credential presence versus live verification, backend prerequisites, effective branch policy, pressure gates and runtime preparation.
+- Packaged `task-eval` CLI: four paired stock-Pi continuation/memory arms, repeated-compaction probes, independent executable oracles, usage/cache and policy comparisons. Offline scripted transport is the default; the separate opt-in live path has explicit budgets and limitations, not an implied quality result.
+
+### Fixed
+
+- Promotion requires host-confirmed applied runs; cancellations cannot fill the 20-run minimum or manufacture failures. Unknown release channels are not stable evidence. Both cohorts need quality/damage coverage and canary data confidence must reach 85; native runs carry explicit cohort/route metadata.
+- Copied cross-project Hindsight refs, missing targets and target-tail substitution are rejected before deletion. Unknown or missing retain status blocks deletion until terminal evidence, preventing delayed retain completion from recreating a deleted fact. Bounded recall refresh includes unknown receipts; lock failures identify the retained lock without removing it automatically.
+- Current privacy settings re-scrub stored local facts before resolve confirmation without changing ref identity. Truncated recall previews and backend changes no longer make confirmed memories unresolvable.
+- Completed but unused preparation is recorded once with discard reason, timing and cost; graceful session shutdown drains late work and metric writes. Used/discarded reports keep cache categories, reported/estimated usage, subscription billing labels and repair provenance distinct.
+- Empty model pickers report missing credentials instead of cancellation; narrow rows preserve stored values, changed-setting markers refresh after edits/resets, and review accepts uppercase/Kitty Apply, Cancel and Quit keys.
+- Evaluator `--help` exits without running tasks or creating reports.
+
+### Unchanged
+
+- Hygiene savings floors, cooldowns, pressure thresholds and preparation TTLs are not tuned from synthetic results. Daily installation, automatic-memory policy and live-provider approval remain untouched.
+
+## [9.8.0-canary.0] - 2026-09-25
+
+Local-only candidate; not published. Isolated validation is not the required
+20-run production canary evidence.
+
+### Added
+
+- `compactionEngines`: an ordered engine list (default `["eesv"]`). Engines are tried in order; the first success applies. Unavailable engines are skipped with a reason, failures are recorded, and if none succeeds the conversation is unchanged and every outcome is reported. Settings offer four presets.
+- Native engine: the current model's own provider compaction, for Anthropic Messages (API key and Claude subscription), OpenAI Codex subscription and OpenAI Responses API key. Works on stock Pi 0.87.1+ as an extension: one nested Pi request is sent once as the provider's compaction request (no retries). The state is stored in the compaction entry's `details.native` and replayed to the same provider and model through `before_provider_request`; it is opaque and not EESV-verified. If replay cannot be applied (it depends on Pi's request format), the model reads the text summary, which for OpenAI routes is only the retained user messages. Cuts keep whole turns; a result that is not smaller is rejected. Opt-in; the default stays `["eesv"]`. Live on stock Pi: Codex subscription (gpt-5.6-luna) validated (compaction accepted, replay after reload accepted and byte-exact; recall of incidental details lossy; re-compaction offline only). Claude subscription is experimental: Anthropic may bill the compaction request as extra usage (real-session attempts were rejected with "400 You're out of extra usage"; cause not isolated). Anthropic and OpenAI API-key routes: offline tests only. For subscription users, "Native, fall back to smart summary" is the safe preset. Known limitation: in sessions smaller than Pi's `compaction.keepRecentTokens`, Pi refuses to apply after the engine has run; Smart Compact reports it and the conversation is unchanged.
+- Provider-request adaptation: nested Smart Compact requests to Anthropic Messages (native compaction, native replay and EESV calls) are offered to adapters on the `pi-toolkit:adapt-provider-request` v1 channel, because stock Pi does not run `before_provider_request` for extension-made requests. Claude subscription requests from extensions (native and EESV) require pi-toolkit's Claude OAuth adapter with this contract. A native "extra usage" rejection on a Claude subscription adds a hint to use an API key or put the smart summary engine first.
+- Optional Hindsight memory backend (`memoryBackend: "hindsight"`): confirmed `smart_save_memory` saves also go to a configured Hindsight bank, and `smart_recall` adds a strict project-scoped remote section. The confirmation shows the exact server, bank and document; the key is read from a named environment variable. Accepted, completed, failed and unknown outcomes are reported as such, with a configurable local copy. No automatic uploads. See `docs/hindsight-memory.md`.
+- Optional Mnemopi backend (`memoryBackend: "mnemopi"`): host-confirmed save/resolve and bounded full-text recall through the real local SQLite engine. A separate Bun worker preserves stock Node Pi support; private per-project databases, stable duplicate identity, checked provenance and cross-process locking prevent accidental cross-project or duplicate writes. Embeddings, model calls, shared default banks and automatic uploads remain off. Settings expose an optional absolute/`~/` data directory.
+- Independent `prepareContextPercent` for background preparation, strictly below the existing `minContextPercent` apply gate. Auto/null preserves adaptive lead; the 5,000-token minimum, idle apply lifecycle and all target/yield checks stay in force. TUI validation refuses conflicting edits in both directions.
+- Issue history: `/smart-compact metrics` starts with the last 20 problems (age, repeat count).
+- Settings TUI: per-row reset with `r`, changed-setting marks, inactive dependent rows with a reason, and a warning line for ignored `settings.json` values.
+
+- Repeatable offline full-AgentSession pilot with real tools, Toolkit cooperation, automatic hygiene, evidence retrieval, context-only rewind, correlated compaction and persistent reopen. Model transport is scripted, so results do not claim autonomous quality or provider-billing savings.
+- Opt-in `contextHygieneEnabled`, independent of automatic compaction, and `smart_context plan` for non-mutating batch previews. Automatic trimming batches at least 16,384 saved characters and respects an eight-assistant-turn branch-persisted cooldown; no per-turn prompts.
+- Separate, explicitly loaded RTK companion (`pi-smart-compact/rtk`), with CLI delegation, conservative command eligibility, cancellation/session guards and no execution retries. A real RTK 0.50 pilot restricts initial rewriting to bare `git status`/`cargo test`; lossy diffs and counterproductive typecheck filtering are left unchanged.
+- Offline native-compaction compatibility probes against real Pi adapters. Pi 0.87.1's adapters do not parse or replay signed Anthropic or opaque OpenAI compaction data themselves, which is why the native engine sends its own compaction request and replays the state from `before_provider_request`.
+- Opt-in `autoTriggerStrategy: "background"`: prepare EESV on a completed-turn snapshot before the pressure gate, then reuse it at native/idle compaction boundaries without a manual tool call. Single-task speculation, retry cooldown, TTL, cancellation, projected-content checks, and response-headroom/yield validation bound reuse; new tail messages remain verbatim.
+- Regression coverage for non-blocking preparation, late completion, model/config/branch invalidation, and correlated application without a second LLM call.
+- `smart_context`: one branch-local checkpoint, deferred context-only rewind with an agent-authored report, bounded trimming of old read-only output, and paged recovery from the original session log. Errors, side-effecting/unknown tools, recent turns, and tool pairs are preserved. Native boundary metadata survives reload without a second transcript store.
+- Background mode tries local trimming before speculative EESV when the context tool is active, without repeatedly invalidating work already in flight.
+- Opt-in `artifactOffloadEnabled`: persist large successful read-only text outputs before model input, returning a bounded preview and branch-owned reference. Private, integrity-checked, deduplicated files with per-origin quotas; failures keep original output and never claim incomplete recovery.
+- `smart_context search` and line-range reads across existing archived outputs, visual excerpts and new artifacts. Literal matching, bounded scans/pages, source labels, explicit missing-file errors and re-scrubbing before retrieval; no additional model calls or database.
+- Experimental `visualArchiveEnabled` (default `false`): locally rendered bitmap excerpts beside the verified EESV text summary. Optional Node-compatible resvg plus a licensed bundled font, bounded payload/source selection, branch/model/privacy validation, native-session reload/re-rendering, and text fallback. This preserves extra evidence at an additional image-token cost; the initial Sonnet 5 synthetic pilot found no token advantage over equivalent text excerpts.
+
+### Fixed
+
+- Native compaction rejects incomplete Codex responses and state missing the required Anthropic signature or OpenAI encrypted content instead of staging unusable context.
+- Native replay requires Pi's exact summary wrapper; overlapping text in a new user instruction is never replaced. Re-compaction stops before sending a request if the prior native state cannot be replayed.
+- Anthropic on-demand compaction removes incompatible `context_management`, including fields supplied by a request adapter. This does not establish the cause of the experimental Claude subscription billing rejection.
+- A queued `smart_context` operation cancelled by a Toolkit pivot now produces a visible, model-delivered cancellation message instead of silently disappearing; context remains unchanged.
+- Rejected settings edits no longer report warnings for hypothetical states that were never written, or hide the subsequent real invalid-file warning.
+- Optional-backend recall labels its local graph section explicitly. Missing Mnemopi dependencies fail before a request is submitted rather than reporting an uncertain write.
+- Notices from a run that finishes after a session switch no longer appear in the new session.
+- Background-run problems are no longer written to stderr inside the TUI; they are shown at the next event.
+
+- Artifact byte deduplication no longer overwrites distinct source labels. Paged discovery preserves each tool/source/content combination and authorized entry-ID reads while retaining existing content-hash IDs, branch revocation and re-scrubbing.
+- Archived native-output discovery uses the shared path extractor, including `filePath`, `filename`, `target_file`, `file_uri` and `absolute_path` aliases.
+- Bitmap admission compares equivalent text using the active reader's provider/model calibration, not the summarizer's calibration; checked before and after rendering.
+- Staged summaries now revalidate reader identity/limits, current context growth, native reserve/headroom and yield before apply. New compact instructions, model changes and navigation invalidate old candidates; late native-hook work cannot reapply an invalidated plan.
+- Early artifact offload leaves file/symbol read deliveries intact, preventing pi-lens from granting read-before-edit coverage for omitted middle content. Other eligible outputs still offload; historical trimming/retrieval is unchanged.
+- Cooperate with the updated Toolkit's explicit thinning-off mode and versioned pivot notifications. Pending navigation pauses preparation/compaction/context mutations; matching completion releases it on success/cancel/failure without stale-operation leakage.
+- Pre-compaction dedup now requires identical content, not only matching arguments, and preserves changed observations, unknown tools, media and instruction reads. Status-looking user text no longer authorizes deletion.
+- Shared instruction/skill-source protection across recoverable trim, rewind, artifacts, visual selection and pre-compaction pruning. Recovery tool outputs are not recursively trimmed; missing trim boundaries fail closed.
+- Multi-page rendering now uses a fresh child cancellation signal per page; resvg's native single-use AbortSignal binding previously rejected the second page with `InvalidArg` on Node and Bun. Added a regression test and a bounded, opt-in synthetic live pilot with recorded observations.
+- Compaction and preflight now use Pi's native context projection, honoring `context_edit` replacements and omissions. Raw-log recovery restores only truncated, unedited entries rather than overwriting intentionally changed siblings.
+- Extraction-cache reuse checks a message-content prefix hash as well as entry IDs. Staged summaries reject changed projected context even when their original branch IDs still exist.
+
+### Changed
+
+- Tools are exposed lazily, decided only at session start and after compaction: `smart_recall`/`smart_save_memory` once the project has memory (or with Hindsight/Mnemopi), `smart_context` when hygiene/offload/background is on or Toolkit's `context` tool is absent. Tool descriptions were shortened; a new session with Toolkit and no project memory now starts with about 267 tokens of Smart Compact tool text instead of about 1.56k. `/tools` choices are respected.
+- Problems are visible without debug mode: each one is shown once per session as a one-line "Smart Compact: what happened. Effect. What to do." notice. Provider errors include their scrubbed first line (at most 160 characters). `DEBUG=smart-compact` now only adds stack traces.
+- Nothing is added to Pi's footer while healthy. Background preparation no longer shows "preparing/ready", the RTK companion's "inactive" status became a one-time notice, and run progress moved out of the footer. `showStatus` now only shows a note when compaction is manual-only or disabled.
+- Without a UI (print, RPC, SDK), warnings and errors go to stderr, and `/smart-compact` with no arguments runs with the configured defaults instead of reporting "Cancelled".
+- Settings TUI reorganized by task into seven categories (Compaction, Models & reasoning, Memory, Context hygiene, Privacy & safety, This branch only, Advanced › Limits) with plain labels and descriptions. Setting keys and stored values are unchanged.
+- Product focus: context hygiene and session continuity; existing package name and compaction APIs remain stable.
+- Bitmap rendering crops blank width without shrinking glyphs. New archives require a validated model cost rule and at least 25% estimated benefit over equivalent text; unsupported/unprofitable cases retain verified text. No new live model calls or billed-savings claim.
+- Minimum Pi version is **0.87.1**. Host packages remain external peers; development dependencies pin the minimum version for reproducible checks. Update older Pi installations before using this release.
+- Existing `native-hook` default and manual approval behavior are unchanged. Visual evidence is opt-in; filesystem rollback is not included.
+
 ## [9.7.1] - 2026-09-23
 
 ### Fixed