pi-smart-compact 9.7.1 → 10.0.1

package/docs/README.md

@@ -0,0 +1,69 @@
+# Pi Continuity documentation
+
+[Project overview](../README.md) · [User guide](./guide.md) · [Configuration](./configuration.md)
+
+Pi Continuity is the product name; `pi-smart-compact` remains the package,
+command family and repository. [Identity and naming](./identity.md).
+
+These guides follow the **current source checkout**, including unreleased work.
+Compare your installed version with the [changelog](../CHANGELOG.md). Dated
+reports describe their own revisions, not necessarily today's behavior.
+
+## Start with your task
+
+| I want to… | Read |
+| --- | --- |
+| Install and choose how the extension runs | [Get started](../README.md#get-started) |
+| Clean up output, compact or recover evidence | [User guide](./guide.md) |
+| Use anchors or move work to a fresh session | [Session navigation and handoff](./guide.md#session-navigation) |
+| Understand a setting, trigger, model route or budget | [Configuration reference](./configuration.md) |
+| Choose where project memory lives | [Memory stores](./guide.md#memory-store-memorybackend) |
+| Connect an existing Hindsight server | [Hindsight setup and privacy](./hindsight-memory.md) |
+| Diagnose unexpected behavior | [Troubleshooting](./guide.md#troubleshooting) · [Support](../SUPPORT.md) |
+| Report sensitive information privately | [Security policy](../SECURITY.md) |
+
+## Understand or contribute
+
+| Document | Scope |
+| --- | --- |
+| [Architecture](../ARCHITECTURE.md) | Ownership, preservation rules, apply boundaries and module responsibilities. |
+| [Evaluation](./evaluation.md) | Available checks and experiments; what quality, cost and timing evidence can establish. |
+| [Contributing](https://github.com/alpertarhan/pi-smart-compact/blob/main/CONTRIBUTING.md) | Development setup, repository map and pull-request expectations. |
+| [Release checklist](./RELEASE.md) | Package validation, compatibility and publication gates. |
+| [Identity and assets](./identity.md) | Product naming, logo sources, palette and reproducible image exports. |
+| [Changelog](../CHANGELOG.md) | Versioned changes and unpublished work. |
+
+## Keep these concepts separate
+
+| Concept | Purpose | Not a substitute for… |
+| --- | --- | --- |
+| **Context hygiene** | Reduce active tool-output noise while keeping eligible evidence retrievable. Local cleanup needs no summary-model call. | A new conversation summary. |
+| **Session continuity** | Carry constraints, decisions, failures and next steps through research, compaction and reload. | Filesystem rollback or a complete copy of the original history. |
+| **Project memory** | Recall scoped facts through one selected backend; explicit saves require confirmation. The local graph can also index derived compaction state. | Backups, output archives or automatic transcript upload. |
+
+The [storage guide](./guide.md#storage-and-privacy) explains where each kind of
+state lives and how long it is retained.
+
+## Historical evidence
+
+Reports are preserved as dated evidence. Their measurements, revision limits and
+warnings remain part of the record; they are not current setup instructions.
+
+- [Pilot and research reports on GitHub](https://github.com/alpertarhan/pi-smart-compact/tree/main/docs/reports):
+  context hygiene, AgentSession lifecycle, visual evidence, Hindsight/native
+  compaction research and the earlier provider baseline. The
+  [evaluation guide](./evaluation.md#pilots-and-dated-reports) explains each scope.
+- [Review findings on GitHub](https://github.com/alpertarhan/pi-smart-compact/blob/main/docs/findings/README.md):
+  the index of external-model and agent-harness audits, organized by reviewer
+  and date. Findings are advisory, not a release gate or product guarantee.
+- [v7 → v8 migration](./MIGRATING_TO_V8.md): instructions for that historical
+  transition, **not** the current installation baseline.
+
+`docs/reports/` and `docs/findings/` are repository-only and excluded from npm.
+Links to those archives deliberately open GitHub, so this index also works from
+an installed package. User guides and brand assets ship with the package;
+developer source, tests and evaluation scripts do not.
+
+A green scripted pilot does not establish live-model fidelity, billed savings
+or production readiness. Use the [evaluation limits](./evaluation.md) and
+[release checklist](./RELEASE.md) before making those claims.

package/docs/RELEASE.md

@@ -1,21 +1,47 @@
 # Release checklist
 
-Use this checklist before publishing `pi-smart-compact`.
-
-> **Stop condition:** validation, packing, and isolated installation are safe.
-> `npm publish`, Git tags, GitHub releases, and deployment require separate
-> explicit approval. The automated checks never perform them.
+Use this checklist before publishing Pi Continuity as the npm package
+`pi-smart-compact`. The package name, command, tool names and configuration
+key do not change with the documentation brand.
+
+> **Approval boundary:** validation, packing and isolated installation do not
+> publish anything. Creating a published GitHub release is the explicit
+> approval that starts npm publication through Trusted Publishing. Use a draft
+> release for preparation; ordinary commits, tags and pull-request CI do not
+> publish packages.
+
+Evidence classes and their limits are defined in
+[evaluation](./evaluation.md#offline-and-live-evidence). Keep the unpublished
+checkout version and the version currently on npm distinct in every note.
+Toolchain prerequisites (Bun pin, Node with npm, ripgrep) are listed at the top
+of [evaluation](./evaluation.md); `release:audit` also needs network access for
+package installation.
 
 ## 1. Prepare the candidate
 
-- [ ] Choose a SemVer version. Use a prerelease such as `8.0.0-rc.4` until the
-      stable/canary gates pass.
-- [ ] Update `package.json`; run `bun run sync-version` for `src/constants.ts`.
+- [ ] Use a distinct prerelease until the stable/canary gates pass, unless the
+      release owner explicitly approves a version-specific stable exception.
+      Record any exception and missing evidence in the release notes; it is
+      not a `PROMOTE` result. Stamp `package.json` and run
+      `bun run sync-version` before packing.
 - [ ] Move shipped notes from `[Unreleased]` into the dated version in
-      `CHANGELOG.md`.
-- [ ] Update README, architecture, and migration notes for behavior/config
-      changes.
-- [ ] Confirm Pi and TypeBox remain wildcard peer dependencies (`"*"`).
+      `CHANGELOG.md`; never word a candidate entry as if the final release
+      check or canary promotion already passed.
+- [ ] Update the guide, configuration, evaluation, architecture and migration
+      notes for behavior/config changes. Dated reports (`docs/reports/`) stay
+      historical and are not packed; add a new report or addendum instead of
+      rewriting them.
+- [ ] On a major version change, update the supported-versions row in
+      `SECURITY.md`; `release:audit` requires it to read ``Latest `<major>.x` ``.
+- [ ] For Claude subscription routes, pair the fresh candidate with the exact
+      `pi-claude-oauth-adapter` build used in the proofs (published `0.2.2`
+      plus the final-payload patch, [upstream PR #10](https://github.com/minzique/pi-claude-oauth-adapter/pull/10),
+      until it is released) and record
+      the paired archive paths and hashes at final packaging — do not
+      reconstruct them from memory.
+- [ ] Confirm Pi remains a host peer (`">=0.87.1"`) and TypeBox a wildcard peer (`"*"`); neither is bundled.
+- [ ] Confirm the visual renderer remains optional/external and the font plus its license ship in `assets/`, together with the on-demand context guide `assets/skills/context-management/SKILL.md`. Verify default Node loading without the optional addon and a real PNG render where supported.
+- [ ] Confirm Mnemopi stays an optional external engine with TypeBox external in its worker, and that `bun`, `@oh-my-pi/pi-mnemopi` and `@resvg/resvg-js` remain optional peers pinned to `OPTIONAL_COMPONENTS` (never `optionalDependencies`). Verify a plain install pulls none of them in, the failure names the install command for the install root, the installed Node-host worker runs on the user-installed `bun` component under a Pi-style npm root with no Bun on `PATH`, and the fail-closed missing-engine and missing-Bun failures submit no memory request and create no store.
 - [ ] Confirm no secrets, local JSONL, SQLite data, backups, or generated
       credentials are tracked or packed.
 
@@ -26,22 +52,40 @@ bun install --frozen-lockfile
 bun run release:check
 ```
 
-`release:check` runs the expanded CI chain: source and scripts typechecking,
-all tests, the adversarial gate, build, and release audit. The audit verifies the
-packed manifest/version/peers, supported SECURITY major, package contents,
-isolated and frozen installs, extension/tool registration, Node SQLite, and
-packaged CLIs.
+`release:check` runs the full local chain: source, scripts, test and bench
+typechecking; all tests; the adversarial `gate`; the hot-path `bench`; build;
+`release:audit`; and `compat:pi latest`. The audit verifies the packed
+manifest/version/peers, supported SECURITY major, package contents
+(runtime-only `dist`: exactly `index.js`, `rtk.js`, and `mnemopi-worker.js` plus
+declarations), isolated and frozen installs, extension/tool registration, Node
+SQLite, and the optional Mnemopi worker through real Node-host tools. It also
+runs the installed worker under a Bun-free `PATH` on the user-installed
+pinned `bun` component (installed with the command Readiness shows into a
+Pi-style npm root, then kept across a Pi update) and checks the
+install-command, missing-engine and missing-Bun negatives (no store, no
+model/network request). Evaluation and report CLIs are source-checkout tools,
+not packed: the audit runs `scripts/provider-eval.ts`,
+`scripts/telemetry-report.ts`, and all four offline continuation/memory arms of
+`scripts/task-eval.ts` under its isolated HOME; scripted transport is not
+live quality evidence. The test suite covers storage durability with real
+`SessionManager` artifacts aged past 20 days by timestamps — deterministic
+aging, not a wall-clock soak — through actual reload and fork.
+
+Run the full chain on the exact candidate. A green result from before any
+later change, including UI or documentation edits, does not count. Pull-request
+CI includes the adversarial gate, but latest-Pi compatibility runs only on a
+schedule or manual dispatch, so a green CI badge does not replace this step.
 
 Then validate the host boundary in an isolated workspace:
 
 ```bash
-bun run compat:pi 0.84.0
+bun run compat:pi 0.87.1
 bun run compat:pi latest
 bun audit
 ```
 
-The compatibility runner temporarily pins only its copied workspace; the source
-manifest must remain wildcard-only.
+The compatibility runner temporarily pins only its copied workspace; source
+peer ranges and minimum-version development pins must remain unchanged.
 
 ## 3. Inspect artifacts
 
@@ -49,18 +93,32 @@ manifest must remain wildcard-only.
 npm pack --dry-run
 bun run provider-eval --min-samples=5
 bun run telemetry-report --min-canary-runs=20
+bun run task-eval --out=/tmp/psc-task-eval-new
 ```
 
 Check that:
 
-- [ ] packed files are limited to `dist`, `docs`, README, LICENSE, CHANGELOG,
-      SECURITY, SUPPORT, ARCHITECTURE, and package metadata;
-- [ ] `dist/index.js`, declarations, and all three bundled CLIs are present;
-- [ ] the extension registers `smart_compact`, `smart_recall`, and
-      `smart_save_memory` from the packed install;
+- [ ] packed files are limited to `dist`, `docs` (without `docs/reports/` and
+      `docs/findings/`), `assets`, README, LICENSE, CHANGELOG, SECURITY,
+      SUPPORT, ARCHITECTURE, and package metadata; `release:audit` requires
+      `ARCHITECTURE.md`, `docs/RELEASE.md` and `docs/MIGRATING_TO_V8.md` and
+      rejects reports and findings;
+- [ ] `dist` holds only `index.js`, `rtk.js`, `mnemopi-worker.js`, and
+      declarations — no evaluation/report CLI bundles;
+- [ ] the extension registers `smart_compact`, `smart_context`,
+      `smart_recall`, and `smart_save_memory` from the packed install;
 - [ ] no provider route was selected automatically;
 - [ ] Data Confidence is honest (legacy evidence may keep it below 85).
 
+The task evaluator defaults to real stock Pi sessions with offline scripted
+transport and temporary memory stores. Live mode needs a fresh explicit
+request/input/output budget and selected-provider credentials; it is not part
+of `release:check`. Input estimates and output reservations are not invoices.
+The SDK fetch guard is not a subprocess network/filesystem sandbox. Codex is
+rejected unless explicitly selected as unbounded output; that exception never
+satisfies a hard output-token budget. No provider-quality or savings claim
+follows from a passing offline report.
+
 ## 4. Canary the RC
 
 After explicit approval to publish an RC, use the npm `next` tag rather than
@@ -75,51 +133,110 @@ After explicit approval to publish an RC, use the npm `next` tag rather than
 ```
 
 Keep all stage model routes null unless a separate routing decision is approved.
-Collect at least 20 non-dry, host-confirmed **applied** schema-v2 canary runs,
-≥70% verifier-quality coverage, and ≥70% run-correlated damage-observation
-coverage in both stable and canary cohorts. Inspect the report's total/applied
-counts: dry runs and staged-but-unapplied runs are not promotion evidence.
-Missing observations are missing evidence, never clean runs. A deterministic
-green release check never implies `PROMOTE`. Promotion requires:
+Collect at least 20 non-dry, host-confirmed **applied** schema-v2 canary runs
+of the candidate version, a stable baseline of at least 20 applied runs,
+≥70% verifier-quality coverage and ≥70% run-correlated damage-observation
+coverage **in both stable and canary cohorts**, and canary data confidence ≥85.
+Inspect the report's total/attempted/applied counts: dry runs, staged-but-
+unapplied runs, voluntary user cancellations, and discarded speculative
+preparations are not promotion evidence (cancellations are neutral — real
+timeouts and provider failures still count). Every metrics entry must carry an
+explicit `releaseChannel`; entries without one are excluded from both cohorts
+and surfaced in the report, never silently pooled as stable. Missing
+observations are missing evidence, never clean runs. A deterministic green
+release check never implies `PROMOTE`. Promotion requires:
 
 - [ ] `telemetry-report` says `PROMOTE`;
+- [ ] canary data confidence is ≥85 (report `HOLD` at 82 is a hold, not a pass);
 - [ ] dashboard Data Confidence is ≥85;
 - [ ] canary success is ≥95% and absolute verifier quality is ≥85;
-- [ ] failure rate did not rise by ≥5pp to at least 10%;
-- [ ] verifier quality did not fall by 5 points;
-- [ ] p95 duration and average tokens did not rise by 50%;
-- [ ] fallback and damage rates did not rise by 10pp;
+- [ ] both cohorts have ≥70% quality and damage-observation coverage;
+- [ ] canary failure rate is at most 5% and not 5pp or more above stable;
+- [ ] verifier quality did not fall by 5 points or more;
+- [ ] p95 duration and average tokens did not rise by 50% or more;
+- [ ] fallback and damage rates did not rise by 10pp or more;
 - [ ] no unresolved security, data-loss, cross-session, or cancellation issue.
 
-A `ROLLBACK` result blocks promotion. `HOLD` means collect evidence or fix data
-coverage; it is not a pass.
-
-## 5. Publish — explicit approval required
+The report evaluates rollback triggers once the canary has at least three
+attempted runs; exact rules are in
+[evaluation](./evaluation.md#decision-rules).
 
-Only after the user/release owner explicitly approves:
+The preparation-policy block (prepared/used/discarded, discard reasons,
+time-to-ready, reuse rate, discarded spend) is measurement only: thresholds,
+TTLs, and cooldowns stay manual policy decisions. Route reports keep the
+input/cache-read/cache-write/output split, mark estimated usage, and label
+subscription (OAuth) routes — never price subscription usage at API rates or
+strip cached tokens from quota.
 
-```bash
-# RC
-npm publish --tag next
+A `ROLLBACK` result blocks promotion. `HOLD` means collect evidence or fix data
+coverage; it is not a pass. Promotion authority remains manual. A release-owner
+exception must name its version and evidence limits; it does not turn missing
+evidence into a passing gate.
 
-# Stable, after canary approval and a stable SemVer bump
-npm publish
-```
+## 5. Publish — explicit approval required
 
-`prepublishOnly` reruns `release:check`; it does not bypass any gate.
+### One-time npm Trusted Publisher setup
+
+In the npm package settings for `pi-smart-compact`, add a **GitHub Actions**
+trusted publisher with these exact values:
+
+| Field | Value |
+| --- | --- |
+| Organization or user | `alpertarhan` |
+| Repository | `pi-smart-compact` |
+| Workflow filename | `publish.yml` (not `.github/workflows/publish.yml`) |
+| Environment | Leave empty; the workflow does not use an environment |
+| Publish permission | Allow direct `npm publish`, not only `npm stage publish` |
+
+The current npm default can permit staging only. Direct publication must be
+enabled to avoid a manual approval for every package. npm does not verify these
+fields when saving; the first successful workflow publication proves the link.
+See [npm Trusted Publishing](https://docs.npmjs.com/trusted-publishers/).
+
+No `NPM_TOKEN` or `NODE_AUTH_TOKEN` secret is needed.
+[`publish.yml`](https://github.com/alpertarhan/pi-smart-compact/blob/main/.github/workflows/publish.yml)
+uses a GitHub-hosted runner, `id-token: write`, Node 26.10.0, npm 11.19.1 and
+the Bun version pinned in `package.json`. npm obtains short-lived OIDC
+credentials and automatically attaches provenance for this public repository.
+Keep these versions and the workflow filename aligned when changing tooling.
+
+### Release an approved version
+
+1. Merge the version, generated `VERSION`, changelog and release documentation
+   through a PR into `main`, with required CI passing. Complete the checks above.
+2. Create a GitHub release at that exact `main` commit with tag `v<version>`,
+   matching `package.json`. Include upgrade notes and the actual validation
+   evidence; document any explicitly approved canary exception.
+3. For a SemVer prerelease, mark the GitHub release **pre-release**. For a stable
+   version, leave that flag off. Publish the release, not just its tag.
+4. Follow **Actions → Publish to npm**. The workflow rejects tags that do not
+   match the package version, mismatched prerelease flags and commits outside
+   `main`. Prereleases publish to npm `next`; stable versions publish to `latest`.
+
+The workflow checks minimum-Pi compatibility and dependency advisories, then
+calls `npm publish`. Its existing `prepublishOnly` hook runs the full
+`release:check`, including the packed install audit and latest-Pi compatibility,
+before uploading. It never uses `--ignore-scripts` to bypass these gates.
+
+If the first run fails authentication, check the exact owner/repository/workflow
+fields, the empty environment and direct-publish permission on npm. After fixing
+the configuration, rerun the failed Actions job; do not publish manually to
+mask a broken OIDC setup. Once a version is published, it is immutable: a new
+package change needs a new version, not a republish or a moved release tag.
 
 ## 6. After publishing
 
-1. Verify npm package contents and integrity.
-2. Create the matching Git tag and GitHub release with migration/compatibility
-   notes.
-3. Install through Pi in a clean profile:
+1. Confirm **Publish to npm** completed successfully. A published GitHub release
+   alone does not prove the package reached npm.
+2. Check the registry version, dist-tag, integrity and provenance:
 
    ```bash
-   pi install npm:pi-smart-compact@next   # RC
-   # or npm:pi-smart-compact for stable
+   VERSION=$(node -p 'require("./package.json").version')
+   npm view "pi-smart-compact@$VERSION" version dist.integrity dist.attestations --json
+   npm view pi-smart-compact dist-tags --json
    ```
 
-4. Re-run tool registration, one manual compaction, Smart Recall, and the local
-   dashboard.
-5. Keep canary monitoring active through the agreed observation window.
+3. Install the exact version through Pi in a clean profile, then re-run tool
+   registration, one manual compaction, Smart Recall and the local dashboard.
+4. Keep canary monitoring active through the agreed observation window;
+   successful publication is not production-quality evidence.