ds4-context-engine 0.2.0-beta.1 → 0.2.0-rc.1

package/README.md

@@ -16,7 +16,7 @@ bounded active context with provenance
 Pi provider
 ```
 
-> **Project status:** M0–M19 are implemented. Prerelease `0.2.0-beta.1` adds quality measurement, structural/hybrid retrieval, cross-session project memory, learned-ranking shadow evaluation, and the runtime adapter kit. Stable `0.1.2` remains available; both lines target Pi `0.84.3`. M20 local KV reuse is not included in this prerelease.
+> **Project status:** M0–M20 and release-candidate hardening are available in prerelease `0.2.0-rc.1`: quality measurement, structural/hybrid retrieval, cross-session project memory, learned-ranking shadow evaluation, the runtime adapter kit, opt-in local KV reuse, schema-10 upgrades, long-session gates, and frozen 0.2 contracts. Stable `0.1.2` remains available; all lines target Pi `0.84.3`.
 
 ## Why DS4
 
@@ -39,6 +39,7 @@ It provides:
 - opt-in metadata-only context-quality metrics and deterministic replay comparisons;
 - checksummed metadata-only learned ranking with shadow mode, canonical classified feedback and static fallback;
 - a versioned runtime adapter contract, reusable conformance kit and non-Pi callback/JSONL reference adapter;
+- opt-in exact-prefix local KV reuse for capable local runtime adapters, with volatile handles and full-replay fallback;
 - an inspectable Context Manifest explaining included and excluded material;
 - fail-open recovery to Pi's native context path for operational failures.
 
@@ -85,10 +86,10 @@ Install the latest stable public npm package with:
 pi install npm:ds4-context-engine
 ```
 
-Install the opt-in 0.2 prerelease from the `beta` dist-tag with:
+Install the opt-in 0.2 release candidate from the `rc` dist-tag with:
 
 ```bash
-pi install npm:ds4-context-engine@beta
+pi install npm:ds4-context-engine@rc
 ```
 
 All prerelease packages (`ds4-context-engine`, `ds4-context-core`, and `ds4-context-reference-adapter`) use the same exact version.
@@ -339,7 +340,7 @@ The following example shows the main configuration groups. Omitted values use th
 }
 ```
 
-Invalid or unknown values are ignored with a warning. Model overrides merge deterministically from `*` to `provider/*` to an exact `provider/model` profile.
+Invalid or unknown values are ignored with a warning. Model overrides merge deterministically from `*` to `provider/*` to an exact `provider/model` profile. The 0.2 release line freezes this additive surface as `ds4-context-config-v1`; existing keys, validation, and defaults are pinned by the compatibility golden.
 
 ## Privacy and provider storage
 
@@ -359,7 +360,9 @@ Native continuation is also disabled by default. Enabling it requires both expli
 
 Eligible OpenAI Responses requests then set `store: true`. Review the provider's retention policy before enabling this option. DS4 keeps response handles only in volatile memory, verifies exact managed prefixes before reuse and retries once with a full managed replay when recognized continuation state is stale.
 
-See [`docs/PRIVACY.md`](docs/PRIVACY.md) and [`docs/NATIVE_CONTINUATION.md`](docs/NATIVE_CONTINUATION.md).
+Local KV reuse is separately disabled by default through `localKvReuse.enabled`. It also requires a local runtime adapter with a versioned `local-kv-reuse` capability and a volatile runtime port. Pi exposes no such handles and remains unsupported even if configuration is enabled.
+
+See [`docs/PRIVACY.md`](docs/PRIVACY.md), [`docs/NATIVE_CONTINUATION.md`](docs/NATIVE_CONTINUATION.md), and [`docs/LOCAL_KV_REUSE.md`](docs/LOCAL_KV_REUSE.md).
 
 ## Storage and recovery
 
@@ -394,12 +397,14 @@ npm test
 npm run check
 npm run quality:compare
 npm run pack:check
+# Post-publication, with an exact version rather than a dist-tag:
+npm run registry:check -- 0.2.0-rc.1
 npm pack --dry-run
 npm pack --dry-run --workspace ds4-context-core
 npm pack --dry-run --workspace ds4-context-reference-adapter
 ```
 
-The test suite covers configuration, migrations, canonical JSONL projection, planning, atomic tool groups, retrieval, compaction, project knowledge, artifacts, memory, privacy, model awareness, continuation, runtime-adapter conformance, the portable-core dependency boundary and Pi extension lifecycle behavior. The package check builds all three tarballs, installs them in a clean temporary consumer, reruns compiled reference-adapter conformance and starts the packaged Pi extension with isolated RPC state.
+The test suite covers configuration, migrations, canonical JSONL projection, planning, atomic tool groups, retrieval, compaction, project knowledge, artifacts, memory, privacy, model awareness, continuation, local-KV eligibility/replay, runtime-adapter conformance, the portable-core dependency boundary and Pi extension lifecycle behavior. The package check builds all three tarballs, installs them in a clean temporary consumer, reruns compiled reference-adapter conformance and starts the packaged Pi extension with isolated RPC state.
 
 ### Portable core
 
@@ -436,17 +441,20 @@ scripts             package and release-readiness checks
 - [Native continuation](docs/NATIVE_CONTINUATION.md)
 - [Portable core](docs/PORTABLE_CORE.md)
 - [Runtime adapter kit](docs/RUNTIME_ADAPTER_KIT.md)
+- [Local KV reuse](docs/LOCAL_KV_REUSE.md)
 - [Storage](docs/STORAGE.md)
 - [Roadmap 0.2.0](docs/ROADMAP_0.2.0.md)
 - [Release process](docs/RELEASING.md)
+- [0.2.0 release readiness](docs/RELEASE_READINESS_0.2.0.md)
+- [0.2.0-rc.1 release notes](docs/releases/0.2.0-rc.1.md)
 - [Architecture decisions](docs/ADR/README.md)
 - [Original development plan](DS4_Context_Engine_Extension_Piano_Sviluppo.md)
 
 ## Roadmap
 
-The original M0–M13 roadmap is complete. `ds4-context-core` contains the compiled runtime-neutral implementation. M14 context-quality metrics, M15 rich symbol indexing, M16 hybrid semantic retrieval, M17 cross-session project memory, M18 learned-ranking shadow evaluation and M19's runtime adapter/conformance kit plus non-Pi reference adapter are implemented on `main`. Learned active ranking remains promotion-gated and static ranking stays authoritative on every failure.
+The original M0–M13 roadmap is complete. `ds4-context-core` contains the compiled runtime-neutral implementation. M14 context-quality metrics, M15 rich symbol indexing, M16 hybrid semantic retrieval, M17 cross-session project memory, M18 learned-ranking shadow evaluation, M19's runtime adapter/conformance kit, and M20 opt-in local KV eligibility/replay are implemented on `main`. Learned active ranking remains promotion-gated, Pi reports local KV as unsupported, and static ranking/native completion stay authoritative on every failure.
 
-The remaining [0.2.0 roadmap](docs/ROADMAP_0.2.0.md) covers optional local KV reuse. Sensitive or transport-specific behavior remains opt-in, and the 0.1 lexical planner stays available as the deterministic fallback.
+The [0.2.0 roadmap](docs/ROADMAP_0.2.0.md) is in release-candidate hardening. The [readiness record](docs/RELEASE_READINESS_0.2.0.md) maps every release gate to tests and operator commands. Sensitive or transport-specific behavior remains opt-in, and the 0.1 lexical planner stays available as the deterministic fallback.
 
 ## Contributing
 

package/docs/ARCHITECTURE.md

@@ -66,6 +66,15 @@ optional OpenAI Responses provider wrapper
   -> retry a rejected stale handle once through the complete managed replay before exposing stream events
   -> record metadata-only mode/item counts/retry/invalidation diagnostics; never record the provider handle
 
+optional local runtime KV port (non-Pi adapters)
+  -> require opt-in configuration plus a negotiated versioned local-KV capability
+  -> apply current privacy policy before runtime-specific prefix/options extraction
+  -> hash exact prefix bytes, provider/model/revision, options, privacy policy, runtime revision and capability version
+  -> let the runtime port map only the fingerprint to its volatile native handle
+  -> replay the complete sanitized payload after miss, stale rejection, unavailable state or runtime restart
+  -> report aggregate hits/misses/saved-prefill/replay latency separately from context occupancy
+  -> never place handles, prefixes or payloads in canonical history, manifests, SQLite or diagnostics
+
 assistant message_end
   -> attach uncached input plus cache read/write usage to the pending manifest
   -> append one exact provider/model calibration sample
@@ -162,7 +171,7 @@ ds4-context-engine                        ds4-context-reference-adapter
 
 `ds4-context-core` is compiled ESM and has no dependency on Pi. Its workspace contains:
 
-- `packages/core/src/adapter`: versioned runtime contract, canonical tool-group validation, isolated capability negotiation and framework-neutral conformance runner;
+- `packages/core/src/adapter`: versioned runtime contract, canonical tool-group validation, isolated capability negotiation, exact local-KV eligibility/replay orchestration and framework-neutral conformance runner;
 - `packages/core/src/core`: portable canonical messages, model profiles, robust calibration, adaptive category limits, budgets and token-estimation policy;
 - `packages/core/src/continuation`: hashed-prefix continuation decisions without provider transport or response APIs;
 - `packages/core/src/config`: runtime-neutral configuration model and filesystem loader;
@@ -178,7 +187,7 @@ ds4-context-engine                        ds4-context-reference-adapter
 - `packages/core/src/persistence`: rebuildable session/project/vector/memory/pin/quality SQLite state, repositories, FTS5, event replay and transactional migrations;
 - `packages/core/src/manifest` and `packages/core/src/shared`: runtime-neutral projections, provenance, hashing, stable serialization and logging.
 
-The `packages/reference-adapter` workspace is the non-Pi reference adapter: it reads bounded append-only canonical JSONL, injects completion through a host callback, enforces privacy at that callback boundary, rebuilds disposable snapshots and explicitly disables unsupported native features.
+The `packages/reference-adapter` workspace is the non-Pi reference adapter: it reads bounded append-only canonical JSONL, injects completion through a host callback, enforces privacy at that callback boundary, rebuilds disposable snapshots and explicitly disables unsupported native features. A local host can inject a handle-free `LocalKvRuntimePort`; the port alone retains native handles and transport while core receives only exact prefix bytes transiently for hashing.
 
 The root `ds4-context-engine` package is the Pi adapter:
 

package/docs/CONTEXT_MANIFEST.md

@@ -27,7 +27,7 @@ A Context Manifest explains the context visible at DS4's Pi `context` hook witho
 - finalized uncached input, cache-read, cache-write, total provider input, and cache shares when available;
 - optional native-continuation eligibility, storage-consent state, request mode, full/sent/omitted input-item counts, state age, generic fallback/invalidation reason, and managed-replay retry outcome.
 
-The manifest does **not** contain system instructions, message text, classified spans, pin content, memory claims, project snippets, artifact content/excerpts, learned-ranking feature vectors/labels/candidate IDs/model weights, tool arguments/results, image data, provider payloads, provider response/conversation IDs, API keys, or headers.
+The manifest does **not** contain system instructions, message text, classified spans, pin content, memory claims, project snippets, artifact content/excerpts, learned-ranking feature vectors/labels/candidate IDs/model weights, local-KV prefixes/fingerprints/handles, tool arguments/results, image data, provider payloads, provider response/conversation IDs, API keys, or headers. Local-KV hit/miss/prefill counters are volatile adapter diagnostics and are not copied into Pi manifests.
 
 ## Provenance mapping
 

package/docs/LOCAL_KV_REUSE.md

@@ -0,0 +1,131 @@
+# Local KV Reuse
+
+M20 adds an opt-in, runtime-neutral prefix/KV capability for local inference runtimes. KV state is only an inference optimization: it is not canonical history, memory, retrieval evidence, a Context Manifest source, or a SQLite projection.
+
+Pi does not expose native KV handles and continues to report `local-kv-reuse` as unsupported. The callback/JSONL reference adapter can exercise the capability when its host injects a `LocalKvRuntimePort`; no provider SDK is added to core or the reference package.
+
+## Boundary
+
+```text
+runtime/provider payload
+        ↓
+adapter privacy enforcement
+        ↓
+runtime-specific prefix/options extraction from sanitized payload
+        ↓
+core exact-byte eligibility hashes
+        ↓
+volatile LocalKvRuntimePort lookup + transport
+        ├─ exact hit ───────────────> cached completion
+        └─ miss/rejected/unavailable -> full prompt replay
+```
+
+Core receives the provider-ready prefix only long enough to hash it. It returns metadata-only component hashes and an aggregate fingerprint. A runtime port maps that fingerprint to its own volatile handle. Handles never cross the port, and core has no API for serializing them.
+
+## Configuration and negotiation
+
+Configuration is disabled by default:
+
+```json
+{
+  "localKvReuse": {
+    "enabled": false
+  }
+}
+```
+
+Enabling configuration is not sufficient. The active adapter must independently declare a versioned `local-kv-reuse` capability and provide a `LocalKvRuntimePort`. Unsupported or malformed declarations disable KV reuse without affecting history, privacy, completion fallback, or any other capability.
+
+A reference-adapter host opts in explicitly:
+
+```ts
+const adapter = new JsonlReferenceRuntimeAdapter({
+  // canonical history, model, privacy, and ordinary transport options...
+  localKv: {
+    enabled: true,
+    port: myVolatileKvPort,
+    runtimeRevision: "llama-runtime-build-42",
+    modelRevision: "model-checksum-abc",
+    prepare(sanitizedPayload) {
+      return {
+        promptPrefix: exactProviderPrefixBytes(sanitizedPayload),
+        systemOptions: providerSystemOptions(sanitizedPayload),
+        toolOptions: providerToolOptions(sanitizedPayload),
+        prefixTokenCount: 32_000,
+        contextTokenCount: 40_000,
+      };
+    },
+  },
+});
+```
+
+`prepare()` receives only the payload returned by adapter privacy enforcement. Remote destinations never enter the local KV path.
+
+## Exact eligibility
+
+`deriveLocalKvEligibility()` uses `local-kv-eligibility-v1`. An eligible fingerprint includes independent SHA-256 components for:
+
+- exact provider, model, and model revision;
+- exact UTF-8/string or byte prefix;
+- deterministic JSON-compatible system and tool options;
+- privacy policy version and local destination;
+- runtime identity, runtime revision, and capability implementation version.
+
+Identifiers are not case-folded or wildcard-matched. Prefix bytes are not normalized. Option object keys are ordered only for deterministic serialization; values, array order, missing/undefined values, and system/tool separation remain significant. Functions, symbols, binary option values, accessors, custom prototypes, circular values, and non-finite numbers are ineligible instead of being hashed ambiguously.
+
+Any changed prefix byte, provider/model identity, model revision, system/tool option, privacy version, runtime revision, or capability version produces a different fingerprint. Disabled configuration, unsupported capability, remote destination, invalid identifiers, an empty prefix, or invalid token counts bypass reuse.
+
+## Runtime port and replay
+
+`LocalKvRuntimePort` deliberately exposes no handle:
+
+```ts
+interface LocalKvRuntimePort {
+  tryReuse(request): Promise<
+    | { status: "hit"; output: unknown; savedPrefillTokens: number; prefillLatencyMs: number }
+    | { status: "miss" | "rejected" | "unavailable" }
+  >;
+  fullReplay(request): Promise<{
+    output: unknown;
+    prefillTokens: number;
+    prefillLatencyMs: number;
+  }>;
+  shutdown?(): Promise<void>;
+}
+```
+
+The runtime owns fingerprint-to-handle lookup, eviction, provider transport, and restart cleanup. `LocalKvReuseController` treats thrown lookup errors and malformed lookup results as unavailable state. Every miss, stale rejection, unavailable state, or runtime restart invokes `fullReplay()` with the complete sanitized payload. If the injected KV full replay itself fails, the reference adapter retries the complete sanitized payload through its ordinary native transport.
+
+Adapter shutdown calls optional port shutdown once. Cache deletion, eviction, process loss, or runtime restart therefore changes performance only; it cannot change canonical state or prevent a full replay.
+
+## Aggregate diagnostics
+
+`local-kv-diagnostics-v1` contains numbers only:
+
+- requests, eligible requests, and bypasses;
+- hits, misses, rejected states, unavailable states, full replays, and transport failures;
+- reusable prefix tokens and total context-occupancy tokens;
+- saved prefill tokens;
+- full-replay prefill tokens and runtime-reported prefill latency.
+
+Diagnostics never include provider/model text, prefix or payload content, component fingerprints, runtime handles, credentials, provider response IDs, or output. Per-request completion metadata keeps `contextTokens`, `savedPrefillTokens`, `replayPrefillTokens`, and `prefillLatencyMs` as separate fields; context occupancy is never reported as cache savings.
+
+## Privacy and persistence
+
+Privacy enforcement precedes runtime-specific prefix extraction and hash verification. A remote payload is sanitized and sent through ordinary transport without consulting local KV state. An enabled local policy is still applied before extraction. A prior cached state cannot authorize material excluded by the current privacy policy because the policy version and sanitized exact prefix are both part of eligibility.
+
+No KV handle or prefix copy is written to:
+
+- Pi JSONL or reference canonical JSONL;
+- Context Manifests;
+- DS4 SQLite;
+- learned-ranking artifacts;
+- adapter/core diagnostics.
+
+There is no M20 database migration. Deleting all volatile runtime KV state requires no DS4 rebuild and no canonical-history migration.
+
+## Verification and benchmark
+
+Unit and reference-adapter integration tests cover exact hits, changed bytes/options, provider/model/privacy/runtime invalidation, cache loss, stale rejection, unavailable state, full replay, privacy-before-verification, aggregate-only diagnostics, idempotent shutdown, and unchanged canonical JSONL.
+
+`tests/benchmarks/local-kv.bench.ts` hashes a synthetic 32,000-token reusable prefix inside a 40,000-token context and separately benchmarks a warm completion whose simulated runtime metadata reports 32,000 saved prefill tokens and `0.5 ms` prefill latency. On the development host, exact eligibility averaged about `0.259 ms` and the warm controller path about `0.248 ms`. These are observational, not portable provider-latency guarantees; the benchmark intentionally reports runtime prefill latency/token savings separately from context occupancy.

package/docs/PORTABLE_CORE.md

@@ -1,6 +1,6 @@
 # Portable Core
 
-M13 extracts the runtime-neutral implementation into the independently buildable `ds4-context-core` workspace package. The root `ds4-context-engine` package remains the Pi integration. M19 adds the versioned adapter contract/conformance kit in core and a separately compiled non-Pi callback/JSONL reference adapter.
+M13 extracts the runtime-neutral implementation into the independently buildable `ds4-context-core` workspace package. The root `ds4-context-engine` package remains the Pi integration. M19 adds the versioned adapter contract/conformance kit in core and a separately compiled non-Pi callback/JSONL reference adapter. M20 adds exact local-KV eligibility and replay orchestration while leaving every provider SDK and native cache handle in the runtime adapter.
 
 ## Dependency rule
 
@@ -30,6 +30,7 @@ The core may use Node.js standard-library facilities such as `node:sqlite`, file
 - memory/pin materialization;
 - privacy classification and provider policy;
 - native-continuation eligibility and hash state;
+- exact local-KV eligibility, handle-free runtime port types, replay policy and aggregate diagnostics;
 - rebuildable SQLite repositories;
 - stable serialization, hashing and logging.
 
@@ -54,10 +55,11 @@ Every `runtime-adapter-v1` implementation must:
 6. append memory/pin mutations to canonical runtime history before materializing derived state;
 7. invoke model completion at the adapter boundary for generated summaries;
 8. enforce privacy immediately before provider transport;
-9. discard or rebuild SQLite and artifact projections safely;
-10. fall back to native runtime behavior when operational integration fails.
+9. retain optional local-KV handles inside its volatile runtime port and replay the full sanitized payload after every non-hit;
+10. discard or rebuild SQLite and artifact projections safely;
+11. fall back to native runtime behavior when operational integration fails.
 
-The executable contract, independent capability negotiation and framework-neutral seven-case conformance runner are exported from `ds4-context-core/adapter/runtime-adapter` and `ds4-context-core/adapter/conformance`. See [Runtime Adapter Kit](RUNTIME_ADAPTER_KIT.md).
+The executable contract, independent capability negotiation and framework-neutral seven-case conformance runner are exported from `ds4-context-core/adapter/runtime-adapter` and `ds4-context-core/adapter/conformance`. M20's handle-free port and controller are exported from `ds4-context-core/adapter/local-kv`. See [Runtime Adapter Kit](RUNTIME_ADAPTER_KIT.md) and [Local KV Reuse](LOCAL_KV_REUSE.md).
 
 ## Build and exports
 
@@ -91,6 +93,7 @@ Extraction does not change DS4 state semantics:
 - Pi JSONL remains canonical for Pi sessions;
 - SQLite remains disposable and rebuildable;
 - compaction remains non-destructive and strictly validated;
-- provider continuation handles remain volatile;
+- provider continuation and local-KV handles remain volatile and adapter-owned;
+- local-KV cache loss changes only prefill performance and transparently replays the full sanitized payload;
 - planner, retrieval, compaction and persistence failures still fail open at the adapter boundary;
 - enabled privacy enforcement still fails closed before remote transport.

package/docs/RELEASE_READINESS_0.2.0.md

@@ -0,0 +1,122 @@
+# DS4 0.2.0 Release Readiness
+
+This document is the release-candidate hardening record for the 0.2 line. A prerelease is not a stable-release declaration. A candidate is complete only after the commands below pass on a clean commit and the exact published registry artifacts pass the post-publication check.
+
+## Frozen compatibility surface
+
+The 0.2 release line freezes three additive contracts:
+
+| Surface | Frozen value | Change rule |
+|---|---|---|
+| configuration | `ds4-context-config-v1` | Existing keys, types, validation, and defaults do not change within 0.2. Additive work requires an explicit review and remains opt-in. An incompatible shape requires a new schema version. |
+| SQLite projection | schema `15` | Migrations 1–15 are immutable. New projection changes append a migration; existing SQL/checksums are never rewritten. |
+| runtime adapter | `runtime-adapter-v1` / `runtime-history-v1` | The four capability IDs and v1 behavior are fixed. Incompatible adapter or history changes require a new contract version. |
+
+`tests/golden/compatibility-0.2.0.json` pins the full default configuration, every migration name/checksum, adapter/history/conformance versions, capability IDs, and local-KV contract versions. The golden test is intentionally strict: an intentional post-0.2 contract must create a new fixture rather than silently updating this one.
+
+## Upgrade and rebuild
+
+A 0.1.0/0.1.2 derived database ends at schema 10. Opening it with 0.2 applies migrations 11–15 in order. `tests/integration/upgrade-rebuild.test.ts` creates an exact schema-v10 database with recorded historical checksums and legacy session/project rows, upgrades it, and verifies that the original projections are unchanged while new tables remain empty.
+
+A 0.1 configuration remains valid. Newly introduced behavior retains safe defaults:
+
+- semantic retrieval: disabled;
+- cross-session memory: disabled;
+- context-quality recording: disabled;
+- learned ranking: `off`;
+- local KV reuse: disabled;
+- embedding defaults: local only.
+
+Complete deletion of `context.db`, `context.db-wal`, and `context.db-shm` loses only projections. Rebuild coverage is distributed by canonical source:
+
+- session entries and FTS: `tests/integration/session-indexer.test.ts`;
+- memory/pins and supersession: `tests/integration/memory-extension.test.ts`;
+- cross-session project mutations, corruption, and source exclusion: `tests/integration/cross-session-memory.test.ts`;
+- project files/snippets and semantic vectors: `tests/integration/project-knowledge.test.ts` and `tests/unit/semantic-index.test.ts`;
+- artifact references/objects: `tests/integration/artifact-extension.test.ts`;
+- quality aggregates from the versioned local corpus: `tests/integration/context-quality-repository.test.ts`;
+- reference-adapter snapshots: `tests/integration/reference-adapter.test.ts`.
+
+Pi JSONL, reference-adapter JSONL, and live project files are never deleted by a rebuild.
+
+## Long-session and provider-switch hardening
+
+`tests/integration/long-session.test.ts` replays a 1,201-message session through 24 managed planning cycles. It verifies:
+
+- the current request remains byte-for-byte present;
+- selected input remains below the model hard input limit without planner fallback;
+- the canonical JSONL file is unchanged;
+- the session index contains one row per canonical entry with no duplicate growth;
+- bounded quality retention remains at its configured maximum;
+- disabled manifests, embeddings, memory, and pins create no rows.
+
+`tests/integration/model-awareness-extension.test.ts` switches 32k, 128k, and 200k local/remote profiles. It verifies exact-model calibration isolation, cold/reused profile state, adaptive budgets, override precedence, and privacy re-enforcement on every destination change. Native continuation and local-KV tests independently reject stale state after model/runtime changes and fall back to full replay.
+
+## Release-gate matrix
+
+| Gate | Evidence |
+|---|---|
+| 0.1 regressions and package boundaries | `npm run check`, `npm run pack:check` |
+| disposable projection rebuild | rebuild tests listed above |
+| lexical-only operation | default config plus retrieval/runtime fallback tests |
+| local and explicitly remote embedding privacy | `tests/integration/privacy-extension.test.ts`, `tests/unit/semantic-index.test.ts` |
+| cross-session branch/supersession/corruption/isolation | `tests/integration/cross-session-memory.test.ts` |
+| learned-ranking promotion or shadow-only fallback | `tests/unit/learned-ranker.test.ts`, `tests/unit/ranking-adapter.test.ts` |
+| Pi/reference adapter conformance | `tests/unit/pi-runtime-contract.test.ts`, `tests/integration/reference-adapter.test.ts` |
+| feature-disabled p95 ≤ 110% of 0.1 | `npm run latency:check -- <0.1.2-core-root>` |
+| long-session integrity and bounded growth | `tests/integration/long-session.test.ts` |
+| matching package versions and clean consumers | `npm run pack:check`; after publish, `npm run registry:check -- <exact-version>` |
+| minimum Node and current LTS | CI matrix: Node `22.19.0` and `24.x` |
+| migration/privacy/limitations/rollback docs | this document, `STORAGE.md`, `PRIVACY.md`, adapter/KV documentation |
+
+The latency check loads exact `ds4-context-core@0.1.2` and the local 0.2 build in one process, runs the same deterministic feature-disabled 401-message fixture, alternates samples to reduce host drift, and rejects a p95 ratio above `1.10`. The comparison contains only timings and package versions.
+
+## Candidate validation
+
+Install the exact stable baseline into an isolated temporary project, then run:
+
+```bash
+BASELINE_DIR="$(mktemp -d)"
+printf '{"private":true}' > "$BASELINE_DIR/package.json"
+npm install --prefix "$BASELINE_DIR" --ignore-scripts --no-audit --no-fund \
+  --package-lock=false ds4-context-core@0.1.2
+
+npm ci
+npm run check
+npm run pack:check
+npm run latency:check -- "$BASELINE_DIR/node_modules/ds4-context-core"
+git diff --check
+git status --short
+rm -rf "$BASELINE_DIR"
+```
+
+After publishing all three packages in dependency order, verify registry bytes rather than local tarballs:
+
+```bash
+npm run registry:check -- 0.2.0-rc.1
+```
+
+The registry check accepts an exact version, never a mutable dist-tag. It installs all three public packages plus the supported Pi SDK into a fresh project, validates matching exact core dependencies, imports core and local-KV exports, runs compiled reference conformance, runs the packaged quality corpus, and starts the published Pi extension through isolated offline RPC state.
+
+## Rollback
+
+SQLite is forward-only. A 0.1 binary correctly refuses to open schema 15; do not edit `schema_migrations`, checksums, or `PRAGMA user_version` to force a downgrade.
+
+To roll back the Pi adapter:
+
+1. stop every Pi process using the shared database;
+2. retain all Pi session JSONL and project files;
+3. remove or archive only `context.db`, `context.db-wal`, `context.db-shm`, disposable artifacts/embeddings, and the learned-ranking model;
+4. install the desired 0.1 package and let it create a fresh derived database, or point it at a new `storage.databasePath`;
+5. leave reference-adapter canonical JSONL untouched if that adapter was used.
+
+New 0.2 configuration keys are ignored by 0.1 with warnings, but removing them reduces operator ambiguity. Disabling semantic retrieval, cross-session memory, ranking, quality, continuation, and local KV before rollback is optional because none of those states is canonical.
+
+## Known limitations
+
+- Active learned ranking still requires explicit promotion metadata; otherwise static ordering remains authoritative.
+- Pi exposes no local KV handles and reports that capability as unsupported.
+- The reference adapter is an inspectable callback/JSONL implementation, not a production streaming runtime.
+- Remote embeddings require exact provider/model consent and enabled privacy filtering.
+- Runtime quality samples are unlabeled for evidence recall; the versioned replay corpus supplies deterministic labels.
+- Derived manifest/calibration history grows with completed provider turns when persistence is enabled; it contains metadata only and can be discarded with the database.

package/docs/RELEASING.md

@@ -16,7 +16,7 @@ Both adapters have an exact dependency on the matching core version, so core mus
 - verify that both adapters depend exactly on that version of `ds4-context-core`;
 - do not include session data, `.pi` state, databases, credentials, or provider payloads.
 
-The automated package check enforces matching versions, exact core dependencies, runtime-SDK isolation, bounded tarball inventories, clean consumer installation, core ESM exports, compiled reference-adapter conformance, and packaged Pi extension startup through isolated RPC state.
+The automated package check enforces matching versions, exact core dependencies, runtime-SDK isolation, bounded tarball inventories, clean consumer installation, core ESM exports, compiled reference-adapter conformance, and packaged Pi extension startup through isolated RPC state. For the 0.2 line, also confirm the frozen `ds4-context-config-v1`, SQLite schema 15 migration checksums, and `runtime-adapter-v1` compatibility golden.
 
 ## Validate
 
@@ -28,6 +28,19 @@ git diff --check
 git status --short
 ```
 
+For a 0.2 release candidate, compare feature-disabled planning against exact stable `ds4-context-core@0.1.2` on the same host:
+
+```bash
+BASELINE_DIR="$(mktemp -d)"
+printf '{"private":true}' > "$BASELINE_DIR/package.json"
+npm install --prefix "$BASELINE_DIR" --ignore-scripts --no-audit --no-fund \
+  --package-lock=false ds4-context-core@0.1.2
+npm run latency:check -- "$BASELINE_DIR/node_modules/ds4-context-core"
+rm -rf "$BASELINE_DIR"
+```
+
+The check rejects a candidate p95 above 110% of the exact 0.1.2 baseline. See [`RELEASE_READINESS_0.2.0.md`](RELEASE_READINESS_0.2.0.md) for the complete gate matrix and rollback procedure.
+
 CI runs the same checks on the minimum supported Node.js version and the current Node.js LTS line. `npm run pack:check` uses a temporary directory and removes it when complete. Set `DS4_KEEP_PACK_TMP=1` only when diagnosing a failed package check.
 
 Review all public tarballs before publishing:
@@ -69,7 +82,13 @@ If core succeeds but an adapter publication fails, fix that adapter release and
 
 After all registry packages are available:
 
-1. install them in a fresh temporary project and rerun the package smoke check if registry propagation was delayed;
+1. verify the exact registry artifacts (never a mutable dist-tag):
+
+   ```bash
+   npm run registry:check -- "$VERSION"
+   ```
+
+   This installs all three packages in a fresh temporary project, checks exact adapter/core dependencies, imports public core/KV exports, runs compiled reference conformance and the packaged quality corpus, and starts the published Pi extension through isolated offline RPC state.
 2. create and push the signed or annotated `v$VERSION` tag;
 3. create the GitHub Release from that tag;
 4. update installation documentation if registry names or requirements changed.

package/docs/ROADMAP_0.2.0.md

@@ -1,6 +1,6 @@
 # DS4 0.2.0 Roadmap
 
-Status: **planned**. No release date is committed.
+Status: **release candidate published**. `0.2.0-rc.1` is the latest prerelease; no stable release date is committed.
 
 Version 0.2.0 focuses on evidence quality, safe project-wide reuse and runtime portability. It extends the released 0.1.0 architecture without changing its canonical-state or failure guarantees.
 
@@ -163,7 +163,7 @@ If the gate is not met, 0.2.0 ships shadow mode but keeps static ranking active.
 
 ## M19 — Runtime Adapter Kit
 
-Status: **implemented on `main` for `0.2.0-beta.2`**. Core ships `runtime-adapter-v1` plus a framework-neutral conformance runner; Pi exposes contract capability diagnostics, and the separately packaged callback/JSONL reference adapter passes all seven conformance cases.
+Status: **implemented on `main` and released in `0.2.0-beta.1`**. Core ships `runtime-adapter-v1` plus a framework-neutral conformance runner; Pi exposes contract capability diagnostics, and the separately packaged callback/JSONL reference adapter passes all seven conformance cases.
 
 ### Deliverables
 
@@ -182,6 +182,8 @@ Status: **implemented on `main` for `0.2.0-beta.2`**. Core ships `runtime-adapte
 
 ## M20 — Local KV Capability
 
+Status: **implemented on `main` and released in `0.2.0-beta.2` behind `localKvReuse.enabled`**. Core derives exact metadata-only eligibility fingerprints and guarantees full replay on every non-hit; injected runtime ports retain all volatile handles and transport. Pi remains explicitly unsupported.
+
 ### Deliverables
 
 - An optional adapter capability for local inference runtimes that expose reusable prefix/KV state.
@@ -231,14 +233,16 @@ SQLite schema changes use forward migrations plus complete rebuild tests from ca
 
 - M18 learned ranker in shadow mode.
 - Privacy, rebuild, corruption and performance hardening for M14–M18.
+- M19 adapter contract, conformance kit and reference adapter.
 
 ### `0.2.0-beta.2`
 
-- M19 adapter contract, conformance kit and reference adapter.
 - M20 local KV capability for runtimes that support it.
 
 ### `0.2.0-rc.1`
 
+Status: **implemented and released in `0.2.0-rc.1`**. See [`RELEASE_READINESS_0.2.0.md`](RELEASE_READINESS_0.2.0.md) and the [release notes](releases/0.2.0-rc.1.md).
+
 - Upgrade/rebuild testing from 0.1.0 state.
 - Long-session dogfooding and provider-switch tests.
 - Registry package smoke tests, documentation and release notes.
@@ -246,7 +250,7 @@ SQLite schema changes use forward migrations plus complete rebuild tests from ca
 
 ## Release gates
 
-Version 0.2.0 is ready only when:
+Version 0.2.0 is ready only when (the live evidence matrix is maintained in [`RELEASE_READINESS_0.2.0.md`](RELEASE_READINESS_0.2.0.md)):
 
 1. all 0.1 tests and package-boundary checks still pass;
 2. new projections rebuild from canonical sources after complete SQLite deletion;

package/docs/RUNTIME_ADAPTER_KIT.md

@@ -41,13 +41,15 @@ Applying a DS4 context plan must not write provider-facing synthetic messages ba
 
 Completion output remains runtime-owned and is not automatically canonical. An adapter must append a final runtime-native message through its ordinary canonical history API if the host chooses to persist it.
 
+M20-capable adapters may additionally expose `localKvPort` and aggregate `localKvDiagnostics()`. Both are optional because most runtimes, including Pi, expose no native KV state. The port interface carries eligibility hashes and complete sanitized replay payloads but never returns or accepts a cache handle.
+
 ## Tool atomicity
 
 `buildCanonicalToolAtomicGroups()` connects every canonical `toolCall` block to all matching `toolResult` blocks. Calls sharing one assistant message form one component, so parallel tool batches cannot be split. `validateRuntimeHistorySnapshot()` rejects missing, overlapping, unknown, or mismatched groups. An incomplete live call is represented by a group with `complete: false`; it cannot be treated as a complete selectable exchange.
 
 ## Capability negotiation
 
-The v1 registry is fixed and additive contracts must use a later contract version:
+The v1 registry is frozen for the 0.2 release line. Incompatible contract/history changes require a later version; capability additions require explicit additive-version review:
 
 - `compaction`;
 - `provider-continuation`;
@@ -62,7 +64,7 @@ A supported capability requires a non-empty implementation version. Unsupported
 - a duplicate, missing, or unversioned declaration is malformed and disabled;
 - no failure in one capability disables canonical history, privacy, completion fallback, or another valid capability.
 
-M19 only negotiates `local-kv-reuse`; M20 defines its eligibility and handle lifecycle. KV handles are never part of canonical history or these diagnostics.
+M19 introduced negotiation for `local-kv-reuse`; M20 adds `local-kv-eligibility-v1`, `LocalKvReuseController`, and the handle-free `LocalKvRuntimePort`. KV handles are never part of canonical history, core state, manifests, or diagnostics. See [Local KV Reuse](LOCAL_KV_REUSE.md).
 
 Pi advertises versioned compaction, OpenAI Responses continuation, and embedding-port support. It explicitly reports local KV reuse as unavailable. `/context adapter` shows the complete negotiation; `/context status` shows aggregate enabled/disabled counts.
 
@@ -97,7 +99,7 @@ The seven checks cover:
 6. safe transport failure with native fallback still available;
 7. idempotent shutdown and closed-state behavior.
 
-Reports contain case IDs, booleans, and fixed failure codes only. The private marker and credential probes never appear in a report. Package smoke tests install core, Pi, and reference tarballs in a clean consumer and rerun reference conformance from compiled exports.
+Reports contain case IDs, booleans, and fixed failure codes only. The private marker and credential probes never appear in a report. Package smoke tests install core, Pi, and reference tarballs in a clean consumer and rerun reference conformance from compiled exports. The 0.2 compatibility golden pins `runtime-adapter-v1`, `runtime-history-v1`, the conformance and local-KV versions, and all capability IDs; post-publication `npm run registry:check -- <exact-version>` reruns the boundary against registry bytes.
 
 ## Reference-adapter compatibility spike
 
@@ -113,7 +115,7 @@ The callback JSONL target was selected because it covers the full contract while
 
 Its `ds4-runtime-session-v1` file is canonical and append-only. The header binds runtime ID, session ID, and canonical project root. Message records contain DS4 canonical messages. `createReferenceHistory()` refuses overwrite, `appendReferenceHistoryMessage()` checks provenance before append, and `rebuildDerivedState()` discards only the in-memory snapshot. File size and message count are bounded. Files are mode `0600` where supported.
 
-The reference adapter provides a callback completion transport, enabled fail-closed privacy by default, optional `EmbeddingPort`, and explicit unsupported declarations for native compaction, provider continuation, and local KV state.
+The reference adapter provides a callback completion transport, enabled fail-closed privacy by default, and optional `EmbeddingPort`. Native compaction and provider continuation remain explicitly unsupported. Local KV reuse is unsupported by default but becomes a versioned supported capability when the host injects a `LocalKvRuntimePort`; configuration remains disabled until explicitly enabled.
 
 ## Packaging rules
 
@@ -138,6 +140,6 @@ npm pack --dry-run
 
 ## Limitations and rollback
 
-The reference adapter is intentionally synchronous-history/callback-completion infrastructure: it does not implement streaming, native compaction, branch editing, provider continuation, or KV reuse. Unsupported features remain disabled and visible rather than emulated.
+The reference adapter is intentionally synchronous-history/callback-completion infrastructure: it does not implement streaming, native compaction, branch editing, provider continuation, or a built-in provider-specific KV cache. Its optional KV path requires a host-owned port that retains handles and transport. Unsupported features remain disabled and visible rather than emulated.
 
-Removing the reference package does not affect Pi or core. A runtime can roll back its adapter by stopping it and returning to native history/context behavior; no canonical migration or SQLite downgrade is required. Deleting adapter caches is safe. Never delete the runtime-owned JSONL file as part of DS4 rollback.
+Removing the reference package does not affect Pi or core. A runtime can roll back its adapter by stopping it and returning to native history/context behavior; no canonical migration or SQLite downgrade is required. Deleting adapter caches is safe and produces a transparent full replay. Never delete the runtime-owned JSONL file as part of DS4 rollback.

package/docs/STORAGE.md

@@ -12,6 +12,8 @@ The extension never edits or rewrites Pi JSONL or project source files. Manual m
 
 The M19 non-Pi reference adapter owns a separate `ds4-runtime-session-v1` JSONL source selected by its host runtime. Its header binds runtime/session identity and the exact canonical project root; following records contain provenance-checked canonical messages. DS4 snapshots and capability diagnostics are disposable. `createReferenceHistory()` refuses overwrite, append uses a dedicated provenance-checked operation, files are mode `0600` where supported, and rebuild never edits this runtime-owned canonical file. Reference JSONL is not imported into Pi or `context.db`.
 
+M20 local KV state is entirely runtime-owned and volatile. Core returns only an in-memory eligibility fingerprint to the runtime port; it has no cache-handle field or serialization API. Prefixes, fingerprints, handles and provider outputs are absent from Pi/reference JSONL, Context Manifests, ranking artifacts and every SQLite table. Aggregate hit/miss/prefill counters live only on the adapter controller, and a restart safely resets them with the runtime cache. M20 adds no database migration.
+
 ## Session index
 
 Each parsed entry stores:
@@ -123,6 +125,14 @@ Schema-v2 `CompactionEntry.details.ds4ContextEngine` records the active/segment
 
 `SummaryRepository.saveGraph()` inserts a complete node batch transactionally, rejects missing/cross-session children, enforces increasing graph levels, and refuses ID collisions that would change immutable content or provenance. `summary_sources` keeps foreign keys to indexed raw entries; deleting a session cascades through the entire derived graph.
 
+## 0.2 schema freeze, upgrade, and rollback
+
+The 0.2 projection contract is frozen at schema 15. Migrations 1–10 are the exact 0.1 history; 11–15 add quality samples, structural symbols, derived embeddings, cross-process leases, and cross-session project-memory checkpoints. `tests/golden/compatibility-0.2.0.json` pins every migration name and SHA-256 checksum so an existing migration cannot be silently rewritten.
+
+Opening a schema-10 database applies only forward migrations and preserves legacy rows. No migration edits Pi JSONL, reference-adapter JSONL, project files, or runtime KV state. Complete database deletion remains the recovery path because all tables are projections; versioned local quality inputs rebuild quality aggregates separately.
+
+Rollback is also projection-based. A 0.1 binary refuses schema 15 by design. Stop all processes sharing the database, retain canonical JSONL/project files, then remove or archive `context.db`, its WAL/SHM files, and other disposable local artifacts before allowing 0.1 to create a fresh database or use another `storage.databasePath`. Never alter `schema_migrations`, stored checksums, or `PRAGMA user_version` to force a downgrade. See [`RELEASE_READINESS_0.2.0.md`](RELEASE_READINESS_0.2.0.md).
+
 ## Transactions
 
 A full rebuild does not blindly delete unchanged entries. It upserts all observed entries, marks them in a temporary seen-set, and removes only stale rows. This preserves foreign-key provenance for unchanged source entries. FTS rows and checkpoint state update in the same transaction.

package/docs/releases/0.2.0-rc.1.md

@@ -0,0 +1,43 @@
+# DS4 Context Engine 0.2.0-rc.1
+
+Status: **released under the npm `rc` dist-tag**.
+
+This release candidate freezes and hardens the 0.2 feature set delivered across the alpha and beta builds. It adds no new default-on provider behavior.
+
+## Included since 0.1.2
+
+- deterministic metadata-only context-quality metrics and comparison corpus;
+- structural project chunks and richer symbol relations;
+- opt-in local or explicitly consented remote hybrid semantic retrieval;
+- opt-in cross-session project memory replay from canonical Pi JSONL;
+- learned ranking with `off`, shadow, promotion-gated active, and static fallback modes;
+- `runtime-adapter-v1`, a reusable conformance kit, and the callback/JSONL reference adapter;
+- opt-in exact local prefix/KV reuse for adapters with a host-owned volatile runtime port;
+- shared-SQLite WAL/write coordination and renewable fenced project-index leases.
+
+## RC hardening
+
+- exact schema-v10 (0.1) to schema-v15 upgrade coverage;
+- full 0.2 configuration/database/adapter compatibility golden;
+- 1,201-message repeated-planning coverage for hard limits, canonical integrity, and bounded derived retention;
+- local/remote provider-switch, calibration, privacy, continuation, and cache invalidation coverage;
+- same-host feature-disabled p95 comparison against exact `ds4-context-core@0.1.2`;
+- exact-version registry-consumer verification for all three published packages;
+- documented release gates, limitations, and forward-only database rollback.
+
+## Compatibility
+
+- Node.js `>=22.19.0`;
+- Pi `0.84.3`;
+- package versions and adapter-to-core dependencies must match exactly;
+- configuration contract `ds4-context-config-v1`;
+- SQLite projection schema `15`;
+- runtime adapter contract `runtime-adapter-v1`.
+
+Existing 0.1 configuration remains valid. Semantic retrieval, cross-session memory, quality sampling, learned ranking, and local KV reuse remain disabled by default. Pi JSONL and live project files remain canonical; SQLite, embeddings, ranking models, artifacts, and runtime KV state remain local/disposable.
+
+## Upgrade and rollback
+
+Opening a 0.1 database applies forward migrations 11–15. No canonical history is rewritten. A 0.1 binary cannot open the newer derived schema; rollback requires stopping all users of the shared database, retaining canonical JSONL/project files, and letting 0.1 create a fresh database or use a different `storage.databasePath`. Never alter migration checksums or delete reference-adapter canonical JSONL.
+
+See [`../RELEASE_READINESS_0.2.0.md`](../RELEASE_READINESS_0.2.0.md) for the gate matrix and exact validation commands.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ds4-context-engine",
-  "version": "0.2.0-beta.1",
+  "version": "0.2.0-rc.1",
   "description": "Non-destructive, provider-independent context management for Pi.",
   "type": "module",
   "license": "MIT",
@@ -46,7 +46,9 @@
     "test:watch": "npm run build:core && npm run build:adapters && vitest",
     "check": "npm run build:core && npm run build:adapters && tsc --noEmit && vitest run",
     "quality:compare": "node scripts/compare-context-quality.mjs",
+    "latency:check": "npm run build:core && node scripts/compare-disabled-planning-latency.mjs",
     "pack:check": "node scripts/verify-packages.mjs",
+    "registry:check": "node scripts/verify-registry-packages.mjs",
     "prepare": "npm run build:core && npm run build:adapters"
   },
   "pi": {
@@ -55,7 +57,7 @@
     ]
   },
   "dependencies": {
-    "ds4-context-core": "0.2.0-beta.1"
+    "ds4-context-core": "0.2.0-rc.1"
   },
   "peerDependencies": {
     "@earendil-works/pi-ai": "0.84.3",

package/src/pi-adapter/version.ts

@@ -1,4 +1,4 @@
-export const EXTENSION_VERSION = "0.2.0-beta.1";
+export const EXTENSION_VERSION = "0.2.0-rc.1";
 export const SUPPORTED_PI_VERSION = "0.84.3";
 export const OBSERVER_PLANNER_VERSION = "observer-model-aware-v1";
 export const PLANNER_VERSION = "managed-learned-ranking-v1";