ds4-context-engine 0.2.0-beta.2 → 0.2.0-rc.1

package/README.md

@@ -16,7 +16,7 @@ bounded active context with provenance
 Pi provider
 ```
 
-> **Project status:** M0–M20 are implemented on `main`. Prerelease `0.2.0-beta.2` includes M14–M20: quality measurement, structural/hybrid retrieval, cross-session project memory, learned-ranking shadow evaluation, the runtime adapter kit, and opt-in local KV reuse. Stable `0.1.2` remains available; all lines target Pi `0.84.3`.
+> **Project status:** M0–M20 and release-candidate hardening are available in prerelease `0.2.0-rc.1`: quality measurement, structural/hybrid retrieval, cross-session project memory, learned-ranking shadow evaluation, the runtime adapter kit, opt-in local KV reuse, schema-10 upgrades, long-session gates, and frozen 0.2 contracts. Stable `0.1.2` remains available; all lines target Pi `0.84.3`.
 
 ## Why DS4
 
@@ -86,10 +86,10 @@ Install the latest stable public npm package with:
 pi install npm:ds4-context-engine
 ```
 
-Install the opt-in 0.2 prerelease from the `beta` dist-tag with:
+Install the opt-in 0.2 release candidate from the `rc` dist-tag with:
 
 ```bash
-pi install npm:ds4-context-engine@beta
+pi install npm:ds4-context-engine@rc
 ```
 
 All prerelease packages (`ds4-context-engine`, `ds4-context-core`, and `ds4-context-reference-adapter`) use the same exact version.
@@ -340,7 +340,7 @@ The following example shows the main configuration groups. Omitted values use th
 }
 ```
 
-Invalid or unknown values are ignored with a warning. Model overrides merge deterministically from `*` to `provider/*` to an exact `provider/model` profile.
+Invalid or unknown values are ignored with a warning. Model overrides merge deterministically from `*` to `provider/*` to an exact `provider/model` profile. The 0.2 release line freezes this additive surface as `ds4-context-config-v1`; existing keys, validation, and defaults are pinned by the compatibility golden.
 
 ## Privacy and provider storage
 
@@ -397,6 +397,8 @@ npm test
 npm run check
 npm run quality:compare
 npm run pack:check
+# Post-publication, with an exact version rather than a dist-tag:
+npm run registry:check -- 0.2.0-rc.1
 npm pack --dry-run
 npm pack --dry-run --workspace ds4-context-core
 npm pack --dry-run --workspace ds4-context-reference-adapter
@@ -443,6 +445,8 @@ scripts             package and release-readiness checks
 - [Storage](docs/STORAGE.md)
 - [Roadmap 0.2.0](docs/ROADMAP_0.2.0.md)
 - [Release process](docs/RELEASING.md)
+- [0.2.0 release readiness](docs/RELEASE_READINESS_0.2.0.md)
+- [0.2.0-rc.1 release notes](docs/releases/0.2.0-rc.1.md)
 - [Architecture decisions](docs/ADR/README.md)
 - [Original development plan](DS4_Context_Engine_Extension_Piano_Sviluppo.md)
 
@@ -450,7 +454,7 @@ scripts             package and release-readiness checks
 
 The original M0–M13 roadmap is complete. `ds4-context-core` contains the compiled runtime-neutral implementation. M14 context-quality metrics, M15 rich symbol indexing, M16 hybrid semantic retrieval, M17 cross-session project memory, M18 learned-ranking shadow evaluation, M19's runtime adapter/conformance kit, and M20 opt-in local KV eligibility/replay are implemented on `main`. Learned active ranking remains promotion-gated, Pi reports local KV as unsupported, and static ranking/native completion stay authoritative on every failure.
 
-The [0.2.0 roadmap](docs/ROADMAP_0.2.0.md) now proceeds to release-candidate hardening. Sensitive or transport-specific behavior remains opt-in, and the 0.1 lexical planner stays available as the deterministic fallback.
+The [0.2.0 roadmap](docs/ROADMAP_0.2.0.md) is in release-candidate hardening. The [readiness record](docs/RELEASE_READINESS_0.2.0.md) maps every release gate to tests and operator commands. Sensitive or transport-specific behavior remains opt-in, and the 0.1 lexical planner stays available as the deterministic fallback.
 
 ## Contributing
 

package/docs/RELEASE_READINESS_0.2.0.md

@@ -0,0 +1,122 @@
+# DS4 0.2.0 Release Readiness
+
+This document is the release-candidate hardening record for the 0.2 line. A prerelease is not a stable-release declaration. A candidate is complete only after the commands below pass on a clean commit and the exact published registry artifacts pass the post-publication check.
+
+## Frozen compatibility surface
+
+The 0.2 release line freezes three additive contracts:
+
+| Surface | Frozen value | Change rule |
+|---|---|---|
+| configuration | `ds4-context-config-v1` | Existing keys, types, validation, and defaults do not change within 0.2. Additive work requires an explicit review and remains opt-in. An incompatible shape requires a new schema version. |
+| SQLite projection | schema `15` | Migrations 1–15 are immutable. New projection changes append a migration; existing SQL/checksums are never rewritten. |
+| runtime adapter | `runtime-adapter-v1` / `runtime-history-v1` | The four capability IDs and v1 behavior are fixed. Incompatible adapter or history changes require a new contract version. |
+
+`tests/golden/compatibility-0.2.0.json` pins the full default configuration, every migration name/checksum, adapter/history/conformance versions, capability IDs, and local-KV contract versions. The golden test is intentionally strict: an intentional post-0.2 contract must create a new fixture rather than silently updating this one.
+
+## Upgrade and rebuild
+
+A 0.1.0/0.1.2 derived database ends at schema 10. Opening it with 0.2 applies migrations 11–15 in order. `tests/integration/upgrade-rebuild.test.ts` creates an exact schema-v10 database with recorded historical checksums and legacy session/project rows, upgrades it, and verifies that the original projections are unchanged while new tables remain empty.
+
+A 0.1 configuration remains valid. Newly introduced behavior retains safe defaults:
+
+- semantic retrieval: disabled;
+- cross-session memory: disabled;
+- context-quality recording: disabled;
+- learned ranking: `off`;
+- local KV reuse: disabled;
+- embedding defaults: local only.
+
+Complete deletion of `context.db`, `context.db-wal`, and `context.db-shm` loses only projections. Rebuild coverage is distributed by canonical source:
+
+- session entries and FTS: `tests/integration/session-indexer.test.ts`;
+- memory/pins and supersession: `tests/integration/memory-extension.test.ts`;
+- cross-session project mutations, corruption, and source exclusion: `tests/integration/cross-session-memory.test.ts`;
+- project files/snippets and semantic vectors: `tests/integration/project-knowledge.test.ts` and `tests/unit/semantic-index.test.ts`;
+- artifact references/objects: `tests/integration/artifact-extension.test.ts`;
+- quality aggregates from the versioned local corpus: `tests/integration/context-quality-repository.test.ts`;
+- reference-adapter snapshots: `tests/integration/reference-adapter.test.ts`.
+
+Pi JSONL, reference-adapter JSONL, and live project files are never deleted by a rebuild.
+
+## Long-session and provider-switch hardening
+
+`tests/integration/long-session.test.ts` replays a 1,201-message session through 24 managed planning cycles. It verifies:
+
+- the current request remains byte-for-byte present;
+- selected input remains below the model hard input limit without planner fallback;
+- the canonical JSONL file is unchanged;
+- the session index contains one row per canonical entry with no duplicate growth;
+- bounded quality retention remains at its configured maximum;
+- disabled manifests, embeddings, memory, and pins create no rows.
+
+`tests/integration/model-awareness-extension.test.ts` switches 32k, 128k, and 200k local/remote profiles. It verifies exact-model calibration isolation, cold/reused profile state, adaptive budgets, override precedence, and privacy re-enforcement on every destination change. Native continuation and local-KV tests independently reject stale state after model/runtime changes and fall back to full replay.
+
+## Release-gate matrix
+
+| Gate | Evidence |
+|---|---|
+| 0.1 regressions and package boundaries | `npm run check`, `npm run pack:check` |
+| disposable projection rebuild | rebuild tests listed above |
+| lexical-only operation | default config plus retrieval/runtime fallback tests |
+| local and explicitly remote embedding privacy | `tests/integration/privacy-extension.test.ts`, `tests/unit/semantic-index.test.ts` |
+| cross-session branch/supersession/corruption/isolation | `tests/integration/cross-session-memory.test.ts` |
+| learned-ranking promotion or shadow-only fallback | `tests/unit/learned-ranker.test.ts`, `tests/unit/ranking-adapter.test.ts` |
+| Pi/reference adapter conformance | `tests/unit/pi-runtime-contract.test.ts`, `tests/integration/reference-adapter.test.ts` |
+| feature-disabled p95 ≤ 110% of 0.1 | `npm run latency:check -- <0.1.2-core-root>` |
+| long-session integrity and bounded growth | `tests/integration/long-session.test.ts` |
+| matching package versions and clean consumers | `npm run pack:check`; after publish, `npm run registry:check -- <exact-version>` |
+| minimum Node and current LTS | CI matrix: Node `22.19.0` and `24.x` |
+| migration/privacy/limitations/rollback docs | this document, `STORAGE.md`, `PRIVACY.md`, adapter/KV documentation |
+
+The latency check loads exact `ds4-context-core@0.1.2` and the local 0.2 build in one process, runs the same deterministic feature-disabled 401-message fixture, alternates samples to reduce host drift, and rejects a p95 ratio above `1.10`. The comparison contains only timings and package versions.
+
+## Candidate validation
+
+Install the exact stable baseline into an isolated temporary project, then run:
+
+```bash
+BASELINE_DIR="$(mktemp -d)"
+printf '{"private":true}' > "$BASELINE_DIR/package.json"
+npm install --prefix "$BASELINE_DIR" --ignore-scripts --no-audit --no-fund \
+  --package-lock=false ds4-context-core@0.1.2
+
+npm ci
+npm run check
+npm run pack:check
+npm run latency:check -- "$BASELINE_DIR/node_modules/ds4-context-core"
+git diff --check
+git status --short
+rm -rf "$BASELINE_DIR"
+```
+
+After publishing all three packages in dependency order, verify registry bytes rather than local tarballs:
+
+```bash
+npm run registry:check -- 0.2.0-rc.1
+```
+
+The registry check accepts an exact version, never a mutable dist-tag. It installs all three public packages plus the supported Pi SDK into a fresh project, validates matching exact core dependencies, imports core and local-KV exports, runs compiled reference conformance, runs the packaged quality corpus, and starts the published Pi extension through isolated offline RPC state.
+
+## Rollback
+
+SQLite is forward-only. A 0.1 binary correctly refuses to open schema 15; do not edit `schema_migrations`, checksums, or `PRAGMA user_version` to force a downgrade.
+
+To roll back the Pi adapter:
+
+1. stop every Pi process using the shared database;
+2. retain all Pi session JSONL and project files;
+3. remove or archive only `context.db`, `context.db-wal`, `context.db-shm`, disposable artifacts/embeddings, and the learned-ranking model;
+4. install the desired 0.1 package and let it create a fresh derived database, or point it at a new `storage.databasePath`;
+5. leave reference-adapter canonical JSONL untouched if that adapter was used.
+
+New 0.2 configuration keys are ignored by 0.1 with warnings, but removing them reduces operator ambiguity. Disabling semantic retrieval, cross-session memory, ranking, quality, continuation, and local KV before rollback is optional because none of those states is canonical.
+
+## Known limitations
+
+- Active learned ranking still requires explicit promotion metadata; otherwise static ordering remains authoritative.
+- Pi exposes no local KV handles and reports that capability as unsupported.
+- The reference adapter is an inspectable callback/JSONL implementation, not a production streaming runtime.
+- Remote embeddings require exact provider/model consent and enabled privacy filtering.
+- Runtime quality samples are unlabeled for evidence recall; the versioned replay corpus supplies deterministic labels.
+- Derived manifest/calibration history grows with completed provider turns when persistence is enabled; it contains metadata only and can be discarded with the database.

package/docs/RELEASING.md

@@ -16,7 +16,7 @@ Both adapters have an exact dependency on the matching core version, so core mus
 - verify that both adapters depend exactly on that version of `ds4-context-core`;
 - do not include session data, `.pi` state, databases, credentials, or provider payloads.
 
-The automated package check enforces matching versions, exact core dependencies, runtime-SDK isolation, bounded tarball inventories, clean consumer installation, core ESM exports, compiled reference-adapter conformance, and packaged Pi extension startup through isolated RPC state.
+The automated package check enforces matching versions, exact core dependencies, runtime-SDK isolation, bounded tarball inventories, clean consumer installation, core ESM exports, compiled reference-adapter conformance, and packaged Pi extension startup through isolated RPC state. For the 0.2 line, also confirm the frozen `ds4-context-config-v1`, SQLite schema 15 migration checksums, and `runtime-adapter-v1` compatibility golden.
 
 ## Validate
 
@@ -28,6 +28,19 @@ git diff --check
 git status --short
 ```
 
+For a 0.2 release candidate, compare feature-disabled planning against exact stable `ds4-context-core@0.1.2` on the same host:
+
+```bash
+BASELINE_DIR="$(mktemp -d)"
+printf '{"private":true}' > "$BASELINE_DIR/package.json"
+npm install --prefix "$BASELINE_DIR" --ignore-scripts --no-audit --no-fund \
+  --package-lock=false ds4-context-core@0.1.2
+npm run latency:check -- "$BASELINE_DIR/node_modules/ds4-context-core"
+rm -rf "$BASELINE_DIR"
+```
+
+The check rejects a candidate p95 above 110% of the exact 0.1.2 baseline. See [`RELEASE_READINESS_0.2.0.md`](RELEASE_READINESS_0.2.0.md) for the complete gate matrix and rollback procedure.
+
 CI runs the same checks on the minimum supported Node.js version and the current Node.js LTS line. `npm run pack:check` uses a temporary directory and removes it when complete. Set `DS4_KEEP_PACK_TMP=1` only when diagnosing a failed package check.
 
 Review all public tarballs before publishing:
@@ -69,7 +82,13 @@ If core succeeds but an adapter publication fails, fix that adapter release and
 
 After all registry packages are available:
 
-1. install them in a fresh temporary project and rerun the package smoke check if registry propagation was delayed;
+1. verify the exact registry artifacts (never a mutable dist-tag):
+
+   ```bash
+   npm run registry:check -- "$VERSION"
+   ```
+
+   This installs all three packages in a fresh temporary project, checks exact adapter/core dependencies, imports public core/KV exports, runs compiled reference conformance and the packaged quality corpus, and starts the published Pi extension through isolated offline RPC state.
 2. create and push the signed or annotated `v$VERSION` tag;
 3. create the GitHub Release from that tag;
 4. update installation documentation if registry names or requirements changed.

package/docs/ROADMAP_0.2.0.md

@@ -1,6 +1,6 @@
 # DS4 0.2.0 Roadmap
 
-Status: **planned**. No release date is committed.
+Status: **release candidate published**. `0.2.0-rc.1` is the latest prerelease; no stable release date is committed.
 
 Version 0.2.0 focuses on evidence quality, safe project-wide reuse and runtime portability. It extends the released 0.1.0 architecture without changing its canonical-state or failure guarantees.
 
@@ -241,6 +241,8 @@ SQLite schema changes use forward migrations plus complete rebuild tests from ca
 
 ### `0.2.0-rc.1`
 
+Status: **implemented and released in `0.2.0-rc.1`**. See [`RELEASE_READINESS_0.2.0.md`](RELEASE_READINESS_0.2.0.md) and the [release notes](releases/0.2.0-rc.1.md).
+
 - Upgrade/rebuild testing from 0.1.0 state.
 - Long-session dogfooding and provider-switch tests.
 - Registry package smoke tests, documentation and release notes.
@@ -248,7 +250,7 @@ SQLite schema changes use forward migrations plus complete rebuild tests from ca
 
 ## Release gates
 
-Version 0.2.0 is ready only when:
+Version 0.2.0 is ready only when (the live evidence matrix is maintained in [`RELEASE_READINESS_0.2.0.md`](RELEASE_READINESS_0.2.0.md)):
 
 1. all 0.1 tests and package-boundary checks still pass;
 2. new projections rebuild from canonical sources after complete SQLite deletion;

package/docs/RUNTIME_ADAPTER_KIT.md

@@ -49,7 +49,7 @@ M20-capable adapters may additionally expose `localKvPort` and aggregate `localK
 
 ## Capability negotiation
 
-The v1 registry is fixed and additive contracts must use a later contract version:
+The v1 registry is frozen for the 0.2 release line. Incompatible contract/history changes require a later version; capability additions require explicit additive-version review:
 
 - `compaction`;
 - `provider-continuation`;
@@ -99,7 +99,7 @@ The seven checks cover:
 6. safe transport failure with native fallback still available;
 7. idempotent shutdown and closed-state behavior.
 
-Reports contain case IDs, booleans, and fixed failure codes only. The private marker and credential probes never appear in a report. Package smoke tests install core, Pi, and reference tarballs in a clean consumer and rerun reference conformance from compiled exports.
+Reports contain case IDs, booleans, and fixed failure codes only. The private marker and credential probes never appear in a report. Package smoke tests install core, Pi, and reference tarballs in a clean consumer and rerun reference conformance from compiled exports. The 0.2 compatibility golden pins `runtime-adapter-v1`, `runtime-history-v1`, the conformance and local-KV versions, and all capability IDs; post-publication `npm run registry:check -- <exact-version>` reruns the boundary against registry bytes.
 
 ## Reference-adapter compatibility spike
 

package/docs/STORAGE.md

@@ -125,6 +125,14 @@ Schema-v2 `CompactionEntry.details.ds4ContextEngine` records the active/segment
 
 `SummaryRepository.saveGraph()` inserts a complete node batch transactionally, rejects missing/cross-session children, enforces increasing graph levels, and refuses ID collisions that would change immutable content or provenance. `summary_sources` keeps foreign keys to indexed raw entries; deleting a session cascades through the entire derived graph.
 
+## 0.2 schema freeze, upgrade, and rollback
+
+The 0.2 projection contract is frozen at schema 15. Migrations 1–10 are the exact 0.1 history; 11–15 add quality samples, structural symbols, derived embeddings, cross-process leases, and cross-session project-memory checkpoints. `tests/golden/compatibility-0.2.0.json` pins every migration name and SHA-256 checksum so an existing migration cannot be silently rewritten.
+
+Opening a schema-10 database applies only forward migrations and preserves legacy rows. No migration edits Pi JSONL, reference-adapter JSONL, project files, or runtime KV state. Complete database deletion remains the recovery path because all tables are projections; versioned local quality inputs rebuild quality aggregates separately.
+
+Rollback is also projection-based. A 0.1 binary refuses schema 15 by design. Stop all processes sharing the database, retain canonical JSONL/project files, then remove or archive `context.db`, its WAL/SHM files, and other disposable local artifacts before allowing 0.1 to create a fresh database or use another `storage.databasePath`. Never alter `schema_migrations`, stored checksums, or `PRAGMA user_version` to force a downgrade. See [`RELEASE_READINESS_0.2.0.md`](RELEASE_READINESS_0.2.0.md).
+
 ## Transactions
 
 A full rebuild does not blindly delete unchanged entries. It upserts all observed entries, marks them in a temporary seen-set, and removes only stale rows. This preserves foreign-key provenance for unchanged source entries. FTS rows and checkpoint state update in the same transaction.

package/docs/releases/0.2.0-rc.1.md

@@ -0,0 +1,43 @@
+# DS4 Context Engine 0.2.0-rc.1
+
+Status: **released under the npm `rc` dist-tag**.
+
+This release candidate freezes and hardens the 0.2 feature set delivered across the alpha and beta builds. It adds no new default-on provider behavior.
+
+## Included since 0.1.2
+
+- deterministic metadata-only context-quality metrics and comparison corpus;
+- structural project chunks and richer symbol relations;
+- opt-in local or explicitly consented remote hybrid semantic retrieval;
+- opt-in cross-session project memory replay from canonical Pi JSONL;
+- learned ranking with `off`, shadow, promotion-gated active, and static fallback modes;
+- `runtime-adapter-v1`, a reusable conformance kit, and the callback/JSONL reference adapter;
+- opt-in exact local prefix/KV reuse for adapters with a host-owned volatile runtime port;
+- shared-SQLite WAL/write coordination and renewable fenced project-index leases.
+
+## RC hardening
+
+- exact schema-v10 (0.1) to schema-v15 upgrade coverage;
+- full 0.2 configuration/database/adapter compatibility golden;
+- 1,201-message repeated-planning coverage for hard limits, canonical integrity, and bounded derived retention;
+- local/remote provider-switch, calibration, privacy, continuation, and cache invalidation coverage;
+- same-host feature-disabled p95 comparison against exact `ds4-context-core@0.1.2`;
+- exact-version registry-consumer verification for all three published packages;
+- documented release gates, limitations, and forward-only database rollback.
+
+## Compatibility
+
+- Node.js `>=22.19.0`;
+- Pi `0.84.3`;
+- package versions and adapter-to-core dependencies must match exactly;
+- configuration contract `ds4-context-config-v1`;
+- SQLite projection schema `15`;
+- runtime adapter contract `runtime-adapter-v1`.
+
+Existing 0.1 configuration remains valid. Semantic retrieval, cross-session memory, quality sampling, learned ranking, and local KV reuse remain disabled by default. Pi JSONL and live project files remain canonical; SQLite, embeddings, ranking models, artifacts, and runtime KV state remain local/disposable.
+
+## Upgrade and rollback
+
+Opening a 0.1 database applies forward migrations 11–15. No canonical history is rewritten. A 0.1 binary cannot open the newer derived schema; rollback requires stopping all users of the shared database, retaining canonical JSONL/project files, and letting 0.1 create a fresh database or use a different `storage.databasePath`. Never alter migration checksums or delete reference-adapter canonical JSONL.
+
+See [`../RELEASE_READINESS_0.2.0.md`](../RELEASE_READINESS_0.2.0.md) for the gate matrix and exact validation commands.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ds4-context-engine",
-  "version": "0.2.0-beta.2",
+  "version": "0.2.0-rc.1",
   "description": "Non-destructive, provider-independent context management for Pi.",
   "type": "module",
   "license": "MIT",
@@ -46,7 +46,9 @@
     "test:watch": "npm run build:core && npm run build:adapters && vitest",
     "check": "npm run build:core && npm run build:adapters && tsc --noEmit && vitest run",
     "quality:compare": "node scripts/compare-context-quality.mjs",
+    "latency:check": "npm run build:core && node scripts/compare-disabled-planning-latency.mjs",
     "pack:check": "node scripts/verify-packages.mjs",
+    "registry:check": "node scripts/verify-registry-packages.mjs",
     "prepare": "npm run build:core && npm run build:adapters"
   },
   "pi": {
@@ -55,7 +57,7 @@
     ]
   },
   "dependencies": {
-    "ds4-context-core": "0.2.0-beta.2"
+    "ds4-context-core": "0.2.0-rc.1"
   },
   "peerDependencies": {
     "@earendil-works/pi-ai": "0.84.3",

package/src/pi-adapter/version.ts

@@ -1,4 +1,4 @@
-export const EXTENSION_VERSION = "0.2.0-beta.2";
+export const EXTENSION_VERSION = "0.2.0-rc.1";
 export const SUPPORTED_PI_VERSION = "0.84.3";
 export const OBSERVER_PLANNER_VERSION = "observer-model-aware-v1";
 export const PLANNER_VERSION = "managed-learned-ranking-v1";