ds4-context-engine 0.3.0-beta.1 → 0.3.0-beta.3

package/README.md

@@ -16,7 +16,7 @@ bounded active context with provenance
 Pi provider
 ```
 
-> **Project status:** Stable `0.2.0` includes M0–M20 and frozen 0.2 contracts. Candidate prerelease `0.3.0-beta.1` adds bounded Context Manifest storage, cooperative database leases, metadata-only storage diagnostics, and explicit offline inspect/compact/recover maintenance on top of alpha.5's overflow-safe hierarchical compaction. Canonical records, SQLite schema 15, and runtime contracts remain unchanged. The beta will use npm's explicit `beta` tag while `latest` remains `0.2.0`, and the maintenance line targets Pi `0.84.3`.
+> **Project status:** Stable `0.2.0` includes M0–M20 and frozen 0.2 contracts. Published prerelease `0.3.0-beta.2` carries forward beta.1's bounded Context Manifest storage, cooperative database leases, metadata-only storage diagnostics, and explicit offline inspect/compact/recover maintenance after successful live maintenance validation. Canonical records, SQLite schema 15, and runtime contracts remain unchanged. npm `beta` points to `0.3.0-beta.2`, `alpha` remains `0.3.0-alpha.5`, `latest` remains `0.2.0`, and the maintenance line targets Pi `0.84.3`.
 
 ## Why DS4
 
@@ -92,7 +92,7 @@ The stable `0.2.0` packages (`ds4-context-engine`, `ds4-context-core`, and `ds4-
 To dogfood the beta without replacing a global stable installation, pin the exact version in a disposable project:
 
 ```bash
-pi install -l npm:ds4-context-engine@0.3.0-beta.1
+pi install -l npm:ds4-context-engine@0.3.0-beta.2
 ```
 
 Follow the [0.3 beta dogfooding runbook](docs/DOGFOODING_0.3.0_BETA.md); use synthetic data and a dedicated session directory.
@@ -425,7 +425,7 @@ npm run schema:context-persistence
 npm run latency:check -- /path/to/exact/ds4-context-core@0.1.2
 npm run pack:check
 # Post-publication, with an exact version rather than a dist-tag:
-npm run registry:check -- 0.3.0-beta.1
+npm run registry:check -- 0.3.0-beta.2
 npm pack --dry-run
 npm pack --dry-run --workspace ds4-context-core
 npm pack --dry-run --workspace ds4-context-reference-adapter
@@ -479,6 +479,7 @@ scripts             package and release-readiness checks
 - [0.2.0 release readiness](docs/RELEASE_READINESS_0.2.0.md)
 - [0.2.0 release notes](docs/releases/0.2.0.md)
 - [0.2.0-rc.1 release notes](docs/releases/0.2.0-rc.1.md)
+- [0.3.0-beta.2 prerelease notes](docs/releases/0.3.0-beta.2.md)
 - [0.3.0-beta.1 prerelease notes](docs/releases/0.3.0-beta.1.md)
 - [0.3.0-alpha.5 prerelease notes](docs/releases/0.3.0-alpha.5.md)
 - [0.3.0-alpha.4 prerelease notes](docs/releases/0.3.0-alpha.4.md)
@@ -492,7 +493,7 @@ scripts             package and release-readiness checks
 
 The original M0–M13 roadmap is complete. `ds4-context-core` contains the compiled runtime-neutral implementation. M14 context-quality metrics, M15 rich symbol indexing, M16 hybrid semantic retrieval, M17 cross-session project memory, M18 learned-ranking shadow evaluation, M19's runtime adapter/conformance kit, and M20 opt-in local KV eligibility/replay are implemented on `main`. Learned active ranking remains promotion-gated, Pi reports local KV as unsupported, and static ranking/native completion stay authoritative on every failure.
 
-The [0.2.0 roadmap](docs/ROADMAP_0.2.0.md) is complete. Candidate `0.3.0-beta.1` builds on the [context persistence tool](docs/CONTEXT_PERSISTENCE_TOOL.md) and privacy-safe [compaction](docs/COMPACTION.md) hardening with bounded persisted manifests, per-profile calibration retention, cooperative client leases, storage diagnostics, and recoverable offline maintenance. Confirmation, exact targeting, strict summary grounding, Pi fallback, and stable canonical/configuration/SQLite/runtime contracts remain unchanged. The [0.2 readiness record](docs/RELEASE_READINESS_0.2.0.md) remains the compatibility baseline. Sensitive or transport-specific behavior remains opt-in, and the 0.1 lexical planner stays available as the deterministic fallback.
+The [0.2.0 roadmap](docs/ROADMAP_0.2.0.md) is complete. Published prerelease `0.3.0-beta.2` carries forward the [context persistence tool](docs/CONTEXT_PERSISTENCE_TOOL.md), privacy-safe [compaction](docs/COMPACTION.md) hardening, bounded persisted manifests, per-profile calibration retention, cooperative client leases, storage diagnostics, and recoverable offline maintenance, with successful live maintenance evidence from beta.1. Confirmation, exact targeting, strict summary grounding, Pi fallback, and stable canonical/configuration/SQLite/runtime contracts remain unchanged. The [0.2 readiness record](docs/RELEASE_READINESS_0.2.0.md) remains the compatibility baseline. Sensitive or transport-specific behavior remains opt-in, and the 0.1 lexical planner stays available as the deterministic fallback.
 
 ## Contributing
 

package/docs/DOGFOODING_0.3.0_BETA.md

@@ -1,6 +1,6 @@
 # Dogfooding DS4 0.3.0 Beta
 
-This runbook validates the published `ds4-context-engine@0.3.0-beta.1` package through sustained real Pi use. It complements automated tests and release smoke checks; it does not replace them.
+This runbook validates the published `ds4-context-engine@0.3.0-beta.2` package through sustained real Pi use. It complements automated tests and release smoke checks; it does not replace them.
 
 The primary target is the model-callable `context_persistence` surface. Pi JSONL must remain canonical and append-only, SQLite must remain rebuildable, and no model-callable write may occur without a fresh positive local UI decision.
 
@@ -29,7 +29,7 @@ mkdir -p /tmp/ds4-beta-dogfood
 cd /tmp/ds4-beta-dogfood
 git init
 mkdir -p sessions evidence
-pi install -l npm:ds4-context-engine@0.3.0-beta.1
+pi install -l npm:ds4-context-engine@0.3.0-beta.2
 pi list
 pi --version
 ```

package/docs/RELEASING.md

@@ -100,7 +100,7 @@ After all registry packages are available:
    npm run registry:check -- "$VERSION"
    ```
 
-   This installs all three packages in a fresh temporary project, checks exact adapter/core dependencies, imports public core/KV exports, runs compiled reference conformance and the packaged quality corpus, and starts the published Pi extension through isolated offline RPC state.
+   This installs all three packages in a fresh temporary project, checks exact adapter/core dependencies, imports public core/KV exports, runs compiled reference conformance and the packaged quality corpus, probes the installed `ds4-context-storage` CLI shim, and starts the published Pi extension through isolated offline RPC state.
 2. create and push the signed or annotated `v$VERSION` tag;
 3. create the GitHub Release from that tag;
 4. update installation documentation if registry names or requirements changed.

package/docs/STORAGE_MAINTENANCE.md

@@ -14,7 +14,15 @@ ds4-context-storage compact --database <exact-path>
 ds4-context-storage recover --database <exact-path>
 ```
 
-`inspect` is metadata-only and read-only. `compact` and `recover` require an interactive local TTY; there is no V1 `--yes` option. The database path is mandatory and is normalized before use.
+`inspect` is metadata-only and read-only. `compact` and `recover` require both interactive local TTY input and output; there is no V1 `--yes` option. The database path is mandatory and is normalized before use. Piping either mutating command directly through `tee` makes stdout non-TTY and therefore fails safely at `stage=confirmation` before maintenance starts. If a local transcript is required, allocate a pseudo-TTY instead of bypassing confirmation, for example:
+
+```bash
+script --quiet --flush --return \
+  --command 'ds4-context-storage compact --database "$HOME/.pi/agent/ds4-context/context.db"' \
+  ./ds4-storage-maintenance.typescript
+```
+
+The transcript contains the exact local database path and typed confirmation, so keep it private and remove it according to local evidence-retention policy.
 
 Typical default path:
 

package/docs/releases/0.3.0-beta.1.md

@@ -1,6 +1,6 @@
 # DS4 Context Engine 0.3.0-beta.1
 
-Status: release candidate; publication and exact registry verification pending.
+Status: published prerelease on 2026-09-03; exact registry verification passed.
 
 This coordinated beta contains SQLite growth caused by persisted Context Manifests and adds explicit, recoverable offline maintenance for already-large derived databases. It also hardens compaction provider retries while preserving strict grounding, privacy filtering, canonical Pi JSONL semantics, atomic tool exchanges, one final Pi compaction entry, and fail-open delegation to Pi.
 
@@ -67,7 +67,16 @@ Large-database validation used a temporary private copy created from a read-only
 - the final candidate passed quick check and foreign-key validation while the standalone backup remained present;
 - temporary copy, backup, and stage files were removed afterward, with no maintenance artifact created next to the live database or in the repository.
 
-Publication, exact registry verification, annotated tag, and GitHub prerelease are performed only after the committed candidate passes validation-only CI.
+Validation-only CI run [`33752225991`](https://github.com/Alucard24/ds4-context-engine/actions/runs/33752225991) passed on Node.js `22.19.0` and `24.x` for committed candidate `4ef936f5a7e5c6afb63fd42d0b745d9201a7b87f` before publication.
+
+## Registry evidence
+
+- `npm run registry:check -- 0.3.0-beta.1`: passed against all three exact published versions, including exact adapter/core dependencies, public core exports, compiled reference conformance, packaged quality corpus, packaged Pi extension startup, and the installed `ds4-context-storage` CLI shim/usage probe.
+- npm `beta` resolves to `0.3.0-beta.1` for all three packages.
+- npm `latest` remains `0.2.0`, `alpha` remains `0.3.0-alpha.5`, and `rc` remains `0.2.0-rc.1` for all three packages.
+- Published SHA-1 values are `fc5d6aaf2304291dc147ecc639281fb947390a51` (core), `7bfe0cf61a0f1213db058d5f89c1b7feb25db54a` (reference adapter), and `6abad04e2a2c30566c042f1e0829930c6546ed54` (Pi adapter).
+
+Exact registry verification has passed; the annotated tag and GitHub prerelease remain pending.
 
 ## Documentation
 

package/docs/releases/0.3.0-beta.2.md

@@ -0,0 +1,102 @@
+# DS4 Context Engine 0.3.0-beta.2
+
+Status: published on 2026-09-03 under the npm `beta` dist-tag.
+
+This coordinated beta carries forward beta.1 without changing canonical records, SQLite schema, runtime contracts, retention limits, privacy policy, compaction validation, or fallback semantics. It incorporates the operational evidence from a successful maintenance of the live derived SQLite projection and documents how to retain an interactive confirmation when recording a maintenance transcript.
+
+## Changed since beta.1
+
+- Recorded successful offline maintenance of a live schema-15 database after all Pi holders closed and the exact local `COMPACT` confirmation was entered.
+- Clarified that `compact` and `recover` require both stdin and stdout to be local TTYs. Directly piping a mutating command through `tee` fails safely before maintenance; transcript capture must allocate a pseudo-TTY and must not bypass confirmation.
+- Updated the beta dogfooding target and coordinated package versions to `0.3.0-beta.2`.
+
+There are no functional storage-format or provider-path changes relative to beta.1.
+
+## Live maintenance evidence
+
+The authorized beta.1 maintenance run completed backup, working-copy transformation, candidate validation, recoverable swap, and post-swap inspection:
+
+- database size: 2.42 GiB before, 377 MiB immediately after compaction;
+- Context Manifests: 1,630 before, 128 after;
+- manifest payload: 1.02 GiB before, 13.5 MiB after;
+- 1,502 manifests pruned, 123 retained manifests rewritten as bounded excluded rollups, and no irreducible oversize skipped;
+- schema 15, `quick_check=ok`, and zero foreign-key violations;
+- no working, candidate, retired, lock, or maintenance-state artifact remained;
+- after Pi reopened, the database returned to WAL mode with 128 manifests, no pending prune, and no maintenance recommendation;
+- the fixed standalone backup remains retained with private file permissions during the observation period.
+
+The first transcript-capturing launcher redirected stdout through `tee`, so the CLI rejected it with `interactive-confirmation-required` at `stage=confirmation`. The confirmation precedes `compactStorage()`, and no backup or staging artifact was created by that failed attempt. The successful retry used a pseudo-TTY and preserved the exact local confirmation requirement.
+
+## Safety and compatibility
+
+Pi JSONL remains canonical and append-only. SQLite remains disposable and rebuildable. Physical maintenance remains explicit, offline, local-TTY-only, and unavailable to model-callable tools. Every `context_persistence` write still requires a fresh positive local UI decision; destructive Pin and Memory operations still require an exact target and current revision; canonical mutations still append through Pi before SQLite reconciliation.
+
+Strict exact-value grounding, bounded repair, privacy filtering, atomic tool exchanges, overflow-safe hierarchical compaction, transport-only retries, and Pi fallback behavior are unchanged. Rejected exact values, provider payloads, SQL, bound values, and raw SQLite messages remain absent from diagnostics.
+
+Compatibility remains unchanged:
+
+- SQLite schema: `15`;
+- configuration: `ds4-context-config-v1`;
+- runtime adapter: `runtime-adapter-v1`;
+- persistence tool: `ds4-context-persistence-tool-v1`;
+- persistence result: `ds4-context-persistence-result-v1`;
+- Pi: `0.84.3`;
+- Node.js: `>=22.19.0`.
+
+Historical excluded-inventory rendering remains unsupported after downgrade once bounded `excluded-rollup` rows have been written, because earlier readers may present sampled excluded details as complete.
+
+## Package/version policy
+
+The coordinated version is `0.3.0-beta.2` for:
+
+```text
+ds4-context-core
+ds4-context-reference-adapter
+ds4-context-engine
+```
+
+Both adapters depend exactly on `ds4-context-core@0.3.0-beta.2`. Publication used the explicit npm `beta` dist-tag for all three packages. npm `latest` remains stable `0.2.0`, and GitHub Actions remains validation-only with OIDC and package-write permissions denied.
+
+## Candidate validation
+
+Local candidate verification on Node.js `26.5.1`:
+
+- `npm run check`: 69 files and 341 tests passed.
+- Three consecutive focused multi-process SQLite concurrency runs passed.
+- `npm run quality:compare`: candidate quality `0.9875` versus baseline `0.808156`.
+- `npm run schema:context-persistence`: 1,266 bytes and 317 estimated tokens; below the 1,500 absolute and 320 relative limits.
+- Three feature-disabled latency comparisons against exact `ds4-context-core@0.1.2` passed with ratios `1.075625`, `0.899241`, and `1.008488`, each at or below `1.10`, using 200 samples and 50 calls per sample.
+- `npm run pack:check`: verified core (223 files), reference adapter (7 files), and Pi adapter (68 files) in a clean consumer.
+- `npm pack --dry-run --json` for all three packages passed with the same inventories and no forbidden local/session/storage files.
+- `git diff --check` passed; the only pre-existing untracked path is `.serena/`, which is excluded from commits and package inventories.
+- Version, exact core dependencies, package-lock entries, extension constant, and reference-adapter constant are synchronized to `0.3.0-beta.2`.
+
+Validation-only CI run [`33762010502`](https://github.com/Alucard24/ds4-context-engine/actions/runs/33762010502) passed on Node.js `22.19.0` and `24.x` for candidate commit `e1dab370ea290da601c817aa358e9c384d4b068c`.
+
+## Registry evidence
+
+Exact post-publication verification passed for all three `0.3.0-beta.2` packages. The clean registry consumer verified exact adapter/core dependencies, public core exports, compiled reference-adapter conformance, packaged quality corpus, packaged Pi extension startup, and the installed `ds4-context-storage` CLI declaration, shim, and usage probe.
+
+Published package shasums:
+
+```text
+ds4-context-core:              d9cc4d10756b984aa8a9078fcad05675b09b327e
+ds4-context-reference-adapter: 6a9a829cf072ed613a5eec0756f967da81f481b4
+ds4-context-engine:            965f65d8d350f712a0bcb0a416d1eebef46f82cd
+```
+
+For all three packages, npm `beta` resolves to `0.3.0-beta.2`, `alpha` remains `0.3.0-alpha.5`, `rc` remains `0.2.0-rc.1`, and `latest` remains `0.2.0`.
+
+npm emitted a publish-time auto-correction warning for the root `bin` entry. The exact registry smoke independently confirmed that the published manifest still declares `ds4-context-storage`, a fresh install creates its shim, and the installed command returns the expected bounded usage response. The warning did not remove the CLI from the published artifact.
+
+Annotated tag `v0.3.0-beta.2` targets verified registry-evidence commit `8323fb74fca286c512ea04efab54aa6065c8d315`. The GitHub prerelease is published at <https://github.com/Alucard24/ds4-context-engine/releases/tag/v0.3.0-beta.2>.
+
+## Documentation
+
+- [`../DOGFOODING_0.3.0_BETA.md`](../DOGFOODING_0.3.0_BETA.md)
+- [`../STORAGE_MAINTENANCE.md`](../STORAGE_MAINTENANCE.md)
+- [`../STORAGE.md`](../STORAGE.md)
+- [`../CONTEXT_MANIFEST.md`](../CONTEXT_MANIFEST.md)
+- [`../COMPACTION.md`](../COMPACTION.md)
+- [`../RELEASING.md`](../RELEASING.md)
+- [`0.3.0-beta.1.md`](0.3.0-beta.1.md)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ds4-context-engine",
-  "version": "0.3.0-beta.1",
+  "version": "0.3.0-beta.3",
   "description": "Non-destructive, provider-independent context management for Pi.",
   "type": "module",
   "license": "MIT",
@@ -62,7 +62,7 @@
     ]
   },
   "dependencies": {
-    "ds4-context-core": "0.3.0-beta.1"
+    "ds4-context-core": "0.3.0-beta.3"
   },
   "peerDependencies": {
     "@earendil-works/pi-ai": "0.84.3",

package/src/pi-adapter/version.ts

@@ -1,4 +1,4 @@
-export const EXTENSION_VERSION = "0.3.0-beta.1";
+export const EXTENSION_VERSION = "0.3.0-beta.3";
 export const SUPPORTED_PI_VERSION = "0.84.3";
 export const OBSERVER_PLANNER_VERSION = "observer-model-aware-v1";
 export const PLANNER_VERSION = "managed-learned-ranking-v1";