ds4-context-engine 0.3.0-alpha.5 → 0.3.0-beta.2

package/docs/releases/0.3.0-beta.1.md

@@ -0,0 +1,90 @@
+# DS4 Context Engine 0.3.0-beta.1
+
+Status: published prerelease on 2026-09-03; exact registry verification passed.
+
+This coordinated beta contains SQLite growth caused by persisted Context Manifests and adds explicit, recoverable offline maintenance for already-large derived databases. It also hardens compaction provider retries while preserving strict grounding, privacy filtering, canonical Pi JSONL semantics, atomic tool exchanges, one final Pi compaction entry, and fail-open delegation to Pi.
+
+## Added
+
+- Retains at most 128 Context Manifests globally and at most 200 calibration samples per exact provider/model/estimator profile.
+- Bounds each online manifest prune to 32 rows and 8 MiB, while allowing one individually oversized oldest row to be removed so convergence cannot stall.
+- Persists complete manifests up to 256 KiB; larger manifests retain complete included provenance and a deterministic excluded-only rollup with complete counts, aggregates, and digests plus at most 256 sampled excluded details.
+- Skips manifest persistence above the 1 MiB hard bound without altering the active in-memory manifest or provider request.
+- Exposes persisted inventory completeness through `getStored()` while legacy complete rows remain readable through `get()`.
+- Adds metadata-only `/context storage` diagnostics and storage high-water integration in `/context health`.
+- Adds cooperative database client leases and a two-phase maintenance lock; active or ambiguous clients block physical maintenance.
+- Adds the `ds4-context-storage inspect|compact|recover --database <exact-path>` CLI. Mutating commands require a local TTY and have no non-interactive bypass.
+- Adds standalone backup, working-copy transformation, validated `VACUUM INTO` candidate, protected-table full-row digests, recoverable swap state, and deterministic crash recovery.
+
+## Changed
+
+- Provider usage is authoritative in scalar Context Manifest columns; `message_end` no longer rewrites `manifest_json`.
+- Only compaction failures categorized as `transport` are retried, with at most three attempts, abort-aware 200 ms and 500 ms delays, fresh routing session IDs, and cumulative usage accounting.
+- Runtime startup during maintenance degrades safely to Pi fallback with categorical diagnostics rather than exposing raw SQLite messages.
+- SQLite lock timeout and rollback diagnostics expose bounded categories, operation names, attempt counts, and numeric SQLite codes rather than raw database errors.
+- Package verification rejects database, WAL, SHM, backup, staging, retired-swap, maintenance-state, client-lease, JSONL, `.pi`, and `.serena` paths.
+
+## Safety and compatibility
+
+Pi JSONL remains canonical and append-only. Physical maintenance never edits session history or project files and is unavailable to model-callable tools. Every `context_persistence` write still requires a fresh positive local UI decision, exact destructive targeting remains revision-bound, and canonical Pin/Memory mutations still append through Pi before SQLite reconciliation.
+
+An individually oversized compaction group, unsupported exact value, non-transport provider failure, abort, or bounded-operation failure still delegates to Pi default compaction. Rejected values and provider payloads are not emitted in diagnostics.
+
+SQLite schema remains 15. Migrations 1–15, `ds4-context-config-v1`, `runtime-adapter-v1`, `ds4-context-persistence-tool-v1`, and `ds4-context-persistence-result-v1` are unchanged. Earlier schema-15 readers can parse additive rollup metadata but may present sampled excluded details as complete, so historical excluded-inventory rendering is unsupported after downgrade once rollups have been written.
+
+## Package/version policy
+
+The coordinated version is `0.3.0-beta.1` for:
+
+```text
+ds4-context-core
+ds4-context-reference-adapter
+ds4-context-engine
+```
+
+Both adapters depend exactly on `ds4-context-core@0.3.0-beta.1`. Publication uses the explicit npm `beta` dist-tag for all three packages while `latest` must remain stable `0.2.0`. GitHub Actions remains validation-only with OIDC and package-write permissions denied.
+
+## Candidate validation evidence
+
+Local candidate verification on Node.js `26.5.1`:
+
+- `npm run check`: 69 files and 341 tests passed.
+- Three consecutive focused multi-process SQLite concurrency runs passed.
+- `npm run quality:compare`: candidate quality `0.9875` versus baseline `0.808156`.
+- `npm run schema:context-persistence`: 1,266 bytes and 317 estimated tokens; below the 1,500 absolute and 320 relative limits.
+- `npm run latency:check -- <exact ds4-context-core@0.1.2>`: passed after clean install with ratio `0.93982`, at or below `1.10`, using 200 samples and 50 calls per sample.
+- `npm run pack:check`: verified core (223 files), reference adapter (7 files), and Pi adapter (67 files) in a clean consumer.
+- `npm pack --dry-run --json` for all three packages passed with the same inventories and no forbidden local/session/storage files.
+- `git diff --check`: passed.
+- Version, exact core dependencies, package-lock entries, extension constant, and reference-adapter constant are synchronized to `0.3.0-beta.1`.
+- Protected compatibility golden, Pi JSONL fixture, migration definitions, canonical persistence contracts, and CI publication permissions are unchanged.
+
+Large-database validation used a temporary private copy created from a read-only source connection; the live database was not compacted, vacuumed, rebuilt, replaced, or maintained:
+
+- before: schema 15, `quick_check=ok`, 3,691 manifests, 2,101,666,663 manifest bytes, 2,489,868,288 database bytes;
+- after: 128 manifests, 15,626,529 manifest bytes, 414 calibration samples across three profiles, 389,746,688 database bytes;
+- all 128 retained oversized manifests were rewritten as explicit `excluded-rollup` projections with no irreducible oversize;
+- the final candidate passed quick check and foreign-key validation while the standalone backup remained present;
+- temporary copy, backup, and stage files were removed afterward, with no maintenance artifact created next to the live database or in the repository.
+
+Validation-only CI run [`33752225991`](https://github.com/Alucard24/ds4-context-engine/actions/runs/33752225991) passed on Node.js `22.19.0` and `24.x` for committed candidate `4ef936f5a7e5c6afb63fd42d0b745d9201a7b87f` before publication.
+
+## Registry evidence
+
+- `npm run registry:check -- 0.3.0-beta.1`: passed against all three exact published versions, including exact adapter/core dependencies, public core exports, compiled reference conformance, packaged quality corpus, packaged Pi extension startup, and the installed `ds4-context-storage` CLI shim/usage probe.
+- npm `beta` resolves to `0.3.0-beta.1` for all three packages.
+- npm `latest` remains `0.2.0`, `alpha` remains `0.3.0-alpha.5`, and `rc` remains `0.2.0-rc.1` for all three packages.
+- Published SHA-1 values are `fc5d6aaf2304291dc147ecc639281fb947390a51` (core), `7bfe0cf61a0f1213db058d5f89c1b7feb25db54a` (reference adapter), and `6abad04e2a2c30566c042f1e0829930c6546ed54` (Pi adapter).
+
+Exact registry verification has passed; the annotated tag and GitHub prerelease remain pending.
+
+## Documentation
+
+- [`../CONTEXT_MANIFEST.md`](../CONTEXT_MANIFEST.md)
+- [`../STORAGE.md`](../STORAGE.md)
+- [`../STORAGE_MAINTENANCE.md`](../STORAGE_MAINTENANCE.md)
+- [`../COMPACTION.md`](../COMPACTION.md)
+- [`../ARCHITECTURE.md`](../ARCHITECTURE.md)
+- [`../DOGFOODING_0.3.0_BETA.md`](../DOGFOODING_0.3.0_BETA.md)
+- [`../RELEASING.md`](../RELEASING.md)
+- [`0.3.0-alpha.5.md`](0.3.0-alpha.5.md)

package/docs/releases/0.3.0-beta.2.md

@@ -0,0 +1,84 @@
+# DS4 Context Engine 0.3.0-beta.2
+
+Status: release candidate; publication pending.
+
+This coordinated beta carries forward beta.1 without changing canonical records, SQLite schema, runtime contracts, retention limits, privacy policy, compaction validation, or fallback semantics. It incorporates the operational evidence from a successful maintenance of the live derived SQLite projection and documents how to retain an interactive confirmation when recording a maintenance transcript.
+
+## Changed since beta.1
+
+- Recorded successful offline maintenance of a live schema-15 database after all Pi holders closed and the exact local `COMPACT` confirmation was entered.
+- Clarified that `compact` and `recover` require both stdin and stdout to be local TTYs. Directly piping a mutating command through `tee` fails safely before maintenance; transcript capture must allocate a pseudo-TTY and must not bypass confirmation.
+- Updated the beta dogfooding target and coordinated package versions to `0.3.0-beta.2`.
+
+There are no functional storage-format or provider-path changes relative to beta.1.
+
+## Live maintenance evidence
+
+The authorized beta.1 maintenance run completed backup, working-copy transformation, candidate validation, recoverable swap, and post-swap inspection:
+
+- database size: 2.42 GiB before, 377 MiB immediately after compaction;
+- Context Manifests: 1,630 before, 128 after;
+- manifest payload: 1.02 GiB before, 13.5 MiB after;
+- 1,502 manifests pruned, 123 retained manifests rewritten as bounded excluded rollups, and no irreducible oversize skipped;
+- schema 15, `quick_check=ok`, and zero foreign-key violations;
+- no working, candidate, retired, lock, or maintenance-state artifact remained;
+- after Pi reopened, the database returned to WAL mode with 128 manifests, no pending prune, and no maintenance recommendation;
+- the fixed standalone backup remains retained with private file permissions during the observation period.
+
+The first transcript-capturing launcher redirected stdout through `tee`, so the CLI rejected it with `interactive-confirmation-required` at `stage=confirmation`. The confirmation precedes `compactStorage()`, and no backup or staging artifact was created by that failed attempt. The successful retry used a pseudo-TTY and preserved the exact local confirmation requirement.
+
+## Safety and compatibility
+
+Pi JSONL remains canonical and append-only. SQLite remains disposable and rebuildable. Physical maintenance remains explicit, offline, local-TTY-only, and unavailable to model-callable tools. Every `context_persistence` write still requires a fresh positive local UI decision; destructive Pin and Memory operations still require an exact target and current revision; canonical mutations still append through Pi before SQLite reconciliation.
+
+Strict exact-value grounding, bounded repair, privacy filtering, atomic tool exchanges, overflow-safe hierarchical compaction, transport-only retries, and Pi fallback behavior are unchanged. Rejected exact values, provider payloads, SQL, bound values, and raw SQLite messages remain absent from diagnostics.
+
+Compatibility remains unchanged:
+
+- SQLite schema: `15`;
+- configuration: `ds4-context-config-v1`;
+- runtime adapter: `runtime-adapter-v1`;
+- persistence tool: `ds4-context-persistence-tool-v1`;
+- persistence result: `ds4-context-persistence-result-v1`;
+- Pi: `0.84.3`;
+- Node.js: `>=22.19.0`.
+
+Historical excluded-inventory rendering remains unsupported after downgrade once bounded `excluded-rollup` rows have been written, because earlier readers may present sampled excluded details as complete.
+
+## Package/version policy
+
+The coordinated version is `0.3.0-beta.2` for:
+
+```text
+ds4-context-core
+ds4-context-reference-adapter
+ds4-context-engine
+```
+
+Both adapters depend exactly on `ds4-context-core@0.3.0-beta.2`. Publication must use the explicit npm `beta` dist-tag for all three packages. npm `latest` must remain stable `0.2.0`, and GitHub Actions remains validation-only with OIDC and package-write permissions denied.
+
+## Candidate validation
+
+Local candidate verification on Node.js `26.5.1`:
+
+- `npm run check`: 69 files and 341 tests passed.
+- Three consecutive focused multi-process SQLite concurrency runs passed.
+- `npm run quality:compare`: candidate quality `0.9875` versus baseline `0.808156`.
+- `npm run schema:context-persistence`: 1,266 bytes and 317 estimated tokens; below the 1,500 absolute and 320 relative limits.
+- Three feature-disabled latency comparisons against exact `ds4-context-core@0.1.2` passed with ratios `1.075625`, `0.899241`, and `1.008488`, each at or below `1.10`, using 200 samples and 50 calls per sample.
+- `npm run pack:check`: verified core (223 files), reference adapter (7 files), and Pi adapter (68 files) in a clean consumer.
+- `npm pack --dry-run --json` for all three packages passed with the same inventories and no forbidden local/session/storage files.
+- `git diff --check` passed; the only pre-existing untracked path is `.serena/`, which is excluded from commits and package inventories.
+- Version, exact core dependencies, package-lock entries, extension constant, and reference-adapter constant are synchronized to `0.3.0-beta.2`.
+
+Validation-only CI on the committed candidate remains required before publication. Exact post-publication registry verification, the annotated tag, and the GitHub prerelease remain pending until the candidate and published artifacts pass their respective gates.
+
+## Documentation
+
+- [`../DOGFOODING_0.3.0_BETA.md`](../DOGFOODING_0.3.0_BETA.md)
+- [`../STORAGE_MAINTENANCE.md`](../STORAGE_MAINTENANCE.md)
+- [`../STORAGE.md`](../STORAGE.md)
+- [`../CONTEXT_MANIFEST.md`](../CONTEXT_MANIFEST.md)
+- [`../COMPACTION.md`](../COMPACTION.md)
+- [`../RELEASING.md`](../RELEASING.md)
+- [`0.3.0-beta.1.md`](0.3.0-beta.1.md)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ds4-context-engine",
-  "version": "0.3.0-alpha.5",
+  "version": "0.3.0-beta.2",
   "description": "Non-destructive, provider-independent context management for Pi.",
   "type": "module",
   "license": "MIT",
@@ -31,12 +31,16 @@
     "docs",
     "quality",
     "scripts/compare-context-quality.mjs",
+    "scripts/ds4-context-storage.mjs",
     "README.md",
     "LICENSE"
   ],
   "exports": {
     ".": "./src/extension/index.ts"
   },
+  "bin": {
+    "ds4-context-storage": "./scripts/ds4-context-storage.mjs"
+  },
   "scripts": {
     "build:core": "npm run build --workspace ds4-context-core",
     "build:reference-adapter": "npm run build --workspace ds4-context-reference-adapter",
@@ -58,7 +62,7 @@
     ]
   },
   "dependencies": {
-    "ds4-context-core": "0.3.0-alpha.5"
+    "ds4-context-core": "0.3.0-beta.2"
   },
   "peerDependencies": {
     "@earendil-works/pi-ai": "0.84.3",

package/scripts/ds4-context-storage.mjs

@@ -0,0 +1,148 @@
+#!/usr/bin/env node
+import { createInterface } from "node:readline/promises";
+import { stdin, stdout } from "node:process";
+import { DatabaseProtocolError } from "ds4-context-core/persistence/database-client-lease";
+import {
+  StorageMaintenanceError,
+  compactStorage,
+  inspectStorage,
+  recoverStorage,
+  storageMaintenancePaths,
+} from "ds4-context-core/persistence/storage-maintenance";
+
+function usage() {
+  return [
+    "Usage:",
+    "  ds4-context-storage inspect --database <exact-path>",
+    "  ds4-context-storage compact --database <exact-path>",
+    "  ds4-context-storage recover --database <exact-path>",
+    "",
+    "compact and recover require an interactive local TTY confirmation.",
+  ].join("\n");
+}
+
+function parseArguments(argv) {
+  const command = argv[0];
+  if (command !== "inspect" && command !== "compact" && command !== "recover") {
+    throw new Error("usage");
+  }
+  let database;
+  for (let index = 1; index < argv.length; index++) {
+    const argument = argv[index];
+    if (argument !== "--database" || database !== undefined || !argv[index + 1]) {
+      throw new Error("usage");
+    }
+    database = argv[++index];
+  }
+  if (!database) throw new Error("usage");
+  return { command, database };
+}
+
+function formatBytes(value) {
+  const units = ["B", "KiB", "MiB", "GiB", "TiB"];
+  let scaled = value;
+  let unit = 0;
+  while (scaled >= 1024 && unit < units.length - 1) {
+    scaled /= 1024;
+    unit++;
+  }
+  const digits = unit === 0 ? 0 : scaled >= 100 ? 0 : scaled >= 10 ? 1 : 2;
+  return `${scaled.toFixed(digits)} ${units[unit]}`;
+}
+
+function printInspection(inspection) {
+  const storage = inspection.diagnostics;
+  console.log([
+    "DS4 Context Storage Inspection",
+    "",
+    `Database:                    ${inspection.databasePath}`,
+    `Schema / journal:            ${inspection.schemaVersion} / ${storage.journalMode ?? "n/a"}`,
+    `quick_check / foreign keys:  ${inspection.quickCheck} / ${inspection.foreignKeyViolations}`,
+    `Database / WAL:              ${formatBytes(storage.databaseBytes ?? 0)} / ${formatBytes(storage.walBytes ?? 0)}`,
+    `Allocated / reusable:        ${formatBytes(storage.allocatedBytes ?? 0)} / ${formatBytes(storage.reusableBytes ?? 0)}`,
+    `Manifests / target:          ${storage.manifests.rows} / ${storage.manifests.retainedLimit}`,
+    `Manifest payload:            ${formatBytes(storage.manifests.serializedBytes)}`,
+    `Manifests to prune:          ${inspection.manifestsToPrune}`,
+    `Calibration to prune:        ${inspection.calibrationToPrune}`,
+    `Available / required space:  ${formatBytes(inspection.availableBytes)} / ${formatBytes(inspection.requiredBytes)}`,
+    `Backup:                      ${inspection.paths.backup}`,
+    `Maintenance recommended:     ${storage.maintenance.recommended ? "yes" : "no"}`,
+    ...storage.maintenance.reasons.map((reason) => `Reason:                      ${reason}`),
+  ].join("\n"));
+}
+
+async function confirm(action, database) {
+  if (!stdin.isTTY || !stdout.isTTY) {
+    throw new StorageMaintenanceError("interactive-confirmation-required", "confirmation");
+  }
+  console.log([
+    "",
+    `Exact database path: ${database}`,
+    "All Pi instances must be closed before this operation.",
+    "The source database will not be overwritten in place.",
+  ].join("\n"));
+  const expected = action.toUpperCase();
+  const prompt = createInterface({ input: stdin, output: stdout });
+  try {
+    const answer = await prompt.question(`Type ${expected} to continue: `);
+    if (answer !== expected) throw new StorageMaintenanceError("confirmation-declined", "confirmation");
+  } finally {
+    prompt.close();
+  }
+}
+
+async function main() {
+  let parsed;
+  try {
+    parsed = parseArguments(process.argv.slice(2));
+  } catch {
+    console.error(usage());
+    process.exitCode = 2;
+    return;
+  }
+
+  if (parsed.command === "inspect") {
+    printInspection(inspectStorage(parsed.database));
+    return;
+  }
+
+  if (parsed.command === "compact") {
+    const inspection = inspectStorage(parsed.database);
+    printInspection(inspection);
+    await confirm("compact", inspection.databasePath);
+    const result = await compactStorage(inspection.databasePath);
+    console.log([
+      "",
+      "DS4 Context Storage Compaction Completed",
+      `Database:              ${result.databasePath}`,
+      `Backup retained:       ${result.backupPath}`,
+      `Before / after:        ${formatBytes(result.beforeBytes)} / ${formatBytes(result.afterBytes)}`,
+      `Manifests pruned:      ${result.maintenance.prunedManifests}`,
+      `Manifests rolled up:   ${result.maintenance.rolledUpManifests}`,
+      `Oversize skipped:      ${result.maintenance.skippedOversizeManifests}`,
+      `Calibration pruned:    ${result.maintenance.prunedCalibrationSamples}`,
+      "Run /context health and /context storage after reopening Pi.",
+    ].join("\n"));
+    return;
+  }
+
+  const paths = storageMaintenancePaths(parsed.database);
+  await confirm("recover", paths.source);
+  const result = recoverStorage(paths.source);
+  console.log([
+    "DS4 Context Storage Recovery Completed",
+    `Database: ${result.databasePath}`,
+    `Action:   ${result.action}`,
+  ].join("\n"));
+}
+
+try {
+  await main();
+} catch (error) {
+  if (error instanceof StorageMaintenanceError || error instanceof DatabaseProtocolError) {
+    console.error(error.message);
+  } else {
+    console.error("Storage maintenance failed (category=unexpected-local-failure)");
+  }
+  process.exitCode = 1;
+}

package/src/extension/commands.ts

@@ -3,6 +3,7 @@ import {
   isPrivacyClassification,
   type PrivacyClassification,
 } from "ds4-context-core/privacy/privacy-policy";
+import type { StorageDiagnostics } from "ds4-context-core/persistence/storage-diagnostics";
 import type {
   Ds4ContextRuntime,
   RuntimeDiagnostics,
@@ -33,6 +34,7 @@ const SUBCOMMANDS = [
   "artifacts",
   "compaction",
   "compact-preview",
+  "storage",
   "health",
   "rebuild-index",
 ] as const;
@@ -50,6 +52,19 @@ function count(value: number | undefined): string {
   return value === undefined ? "n/a" : NUMBER_FORMAT.format(value);
 }
 
+function bytes(value: number | undefined): string {
+  if (value === undefined) return "n/a";
+  const units = ["B", "KiB", "MiB", "GiB", "TiB"];
+  let scaled = value;
+  let unit = 0;
+  while (scaled >= 1024 && unit < units.length - 1) {
+    scaled /= 1024;
+    unit++;
+  }
+  const digits = unit === 0 ? 0 : scaled >= 100 ? 0 : scaled >= 10 ? 1 : 2;
+  return `${scaled.toFixed(digits)} ${units[unit]}`;
+}
+
 function present(ctx: ExtensionCommandContext, message: string, level: "info" | "warning" | "error" = "info"): void {
   if (ctx.hasUI) ctx.ui.notify(message, level);
   else console.log(message);
@@ -251,6 +266,7 @@ function formatManifest(diagnostics: RuntimeDiagnostics): string {
   const manifest = diagnostics.lastManifest;
   if (!manifest) return "No Context Manifest has been built for this session yet.";
 
+  const inventory = manifest.persistedInventory;
   const kinds = new Map<string, { items: number; tokens: number }>();
   for (const item of manifest.included) {
     const aggregate = kinds.get(item.kind) ?? { items: 0, tokens: 0 };
@@ -273,7 +289,11 @@ function formatManifest(diagnostics: RuntimeDiagnostics): string {
     `Actual input:       ${count(manifest.actualInputTokens)}`,
     `Usage input/cache:  ${count(manifest.providerUsage?.inputTokens)} / ${count(manifest.providerUsage?.cacheReadTokens)} read / ${count(manifest.providerUsage?.cacheWriteTokens)} write`,
     `Target / hard:      ${count(manifest.targetInputTokens)} / ${count(manifest.hardInputLimit)}`,
-    `Included / excluded:${count(manifest.included.length)} / ${count(manifest.excluded.length)}`,
+    `Included / excluded:${count(manifest.included.length)} / ${count(inventory?.excluded.total ?? manifest.excluded.length)}`,
+    `Persisted inventory: ${inventory?.completeness ?? "complete"}`,
+    ...(inventory?.completeness === "excluded-rollup" ? [
+      `Excluded details:  ${count(inventory.excluded.retained)} / ${count(inventory.excluded.total)} retained in persisted projection`,
+    ] : []),
     `Planning mode:      ${manifest.planning?.mode ?? "observer"}`,
     `Original messages:  ${count(manifest.planning?.originalMessageCount ?? manifest.composition.messageCount)}`,
     `Selected messages:  ${count(manifest.composition.messageCount)}`,
@@ -299,9 +319,13 @@ function formatManifestItems(diagnostics: RuntimeDiagnostics, type: "included" |
   const manifest = diagnostics.lastManifest;
   if (!manifest) return "No Context Manifest has been built for this session yet.";
   const items = manifest[type];
+  const inventory = manifest.persistedInventory;
   return [
     `DS4 Context ${type === "included" ? "Included" : "Excluded"} Items`,
     "",
+    ...(type === "excluded" && inventory?.completeness === "excluded-rollup"
+      ? [`Persisted projection: ${count(inventory.excluded.retained)} / ${count(inventory.excluded.total)} excluded details retained; this is not the complete historical inventory.`, ""]
+      : []),
     ...(items.length === 0
       ? ["none"]
       : items.map((item, index) => {
@@ -717,6 +741,43 @@ function formatAdapter(diagnostics: RuntimeDiagnostics): string {
   ].join("\n");
 }
 
+function formatStorage(storage: StorageDiagnostics, databasePath?: string): string {
+  if (storage.status === "unavailable") {
+    return [
+      "DS4 Storage",
+      "",
+      "Status:                     unavailable",
+      `Database:                   ${databasePath ?? "unavailable"}`,
+      "Pi fallback remains active; no storage mutation was attempted.",
+    ].join("\n");
+  }
+  return [
+    "DS4 Storage",
+    "",
+    `Status:                     ${storage.status}`,
+    `Database:                   ${databasePath ?? "unavailable"}`,
+    `Schema / journal:           ${storage.schemaVersion ?? "n/a"} / ${storage.journalMode ?? "n/a"}`,
+    `Database / WAL / SHM:       ${bytes(storage.databaseBytes)} / ${bytes(storage.walBytes)} / ${bytes(storage.shmBytes)}`,
+    `Allocated / reusable:       ${bytes(storage.allocatedBytes)} / ${bytes(storage.reusableBytes)}`,
+    `Pages total / reusable:     ${count(storage.pageCount)} / ${count(storage.freePages)}`,
+    `Manifests:                  ${count(storage.manifests.rows)} / target ${count(storage.manifests.retainedLimit)}`,
+    `Manifest payload:           ${bytes(storage.manifests.serializedBytes)}`,
+    `Rolled-up manifests:        ${count(storage.manifests.rolledUpRows)}`,
+    `Calibration samples/profiles: ${count(storage.calibration.rows)} / ${count(storage.calibration.profiles)} (limit ${count(storage.calibration.retainedPerProfile)}/profile)`,
+    `Sessions:                   ${count(storage.sessions)}`,
+    `Artifact objects / refs:    ${count(storage.artifacts.objects)} / ${count(storage.artifacts.references)}`,
+    `Artifact bytes:             ${bytes(storage.artifacts.bytes)}`,
+    ...(storage.activeProject ? [
+      `Active project files:       ${count(storage.activeProject.files)}`,
+      `Project snippets/stale:     ${count(storage.activeProject.snippets)} / ${count(storage.activeProject.staleSnippets)}`,
+      `Project indexed tokens:     ${count(storage.activeProject.indexedTokens)}`,
+    ] : []),
+    `Retention converged:        ${storage.retention.converged ? "yes" : "no"}`,
+    `Offline maintenance:       ${storage.maintenance.recommended ? "recommended" : "not required"}`,
+    ...storage.maintenance.reasons.map((reason) => `Reason:                     ${reason}`),
+  ].join("\n");
+}
+
 function formatCompaction(diagnostics: RuntimeDiagnostics, preview: boolean): string {
   const compaction = diagnostics.compaction;
   return [
@@ -738,6 +799,7 @@ function formatCompaction(diagnostics: RuntimeDiagnostics, preview: boolean): st
     `Whole-source prompt:     ${count(compaction.sourcePromptTokens)}`,
     `Generated segments:      ${count(compaction.segmentCount)}`,
     `Aggregate calls:         ${count(compaction.aggregateCalls)}`,
+    `Transport retries:       ${count(compaction.transportRetries)}`,
     `Validation:              ${compaction.validationStatus ?? "n/a"}`,
     `First kept entry:        ${compaction.firstKeptEntryId ?? "n/a"}`,
     `Tokens before:           ${count(compaction.tokensBefore)}`,
@@ -1007,6 +1069,17 @@ export function registerContextCommand(pi: ExtensionAPI, runtime: Ds4ContextRunt
           return;
         }
 
+        if (subcommand === "storage") {
+          const diagnostics = runtime.diagnostics(ctx);
+          const storage = runtime.storageDiagnostics();
+          present(
+            ctx,
+            formatStorage(storage, diagnostics.databasePath),
+            storage.status === "ok" ? "info" : "warning",
+          );
+          return;
+        }
+
         if (subcommand === "rebuild-index") {
           await ctx.waitForIdle();
           const result = runtime.rebuildIndex(ctx);
@@ -1037,6 +1110,7 @@ export function registerContextCommand(pi: ExtensionAPI, runtime: Ds4ContextRunt
           }
           runtime.verifyArtifactHealth(ctx);
           const diagnostics = runtime.diagnostics(ctx);
+          const storage = runtime.storageDiagnostics();
           const staleProjectSnippets = diagnostics.project.stats?.staleSnippets ?? 0;
           const artifactIntegrityIssues = diagnostics.artifacts.stats.missing + diagnostics.artifacts.stats.corrupt;
           const healthy = health.ok
@@ -1050,7 +1124,8 @@ export function registerContextCommand(pi: ExtensionAPI, runtime: Ds4ContextRunt
             && diagnostics.quality.lastError === undefined
             && diagnostics.ranking.warnings.length === 0
             && artifactIntegrityIssues === 0
-            && diagnostics.artifacts.warnings.length === 0;
+            && diagnostics.artifacts.warnings.length === 0
+            && storage.status === "ok";
           present(
             ctx,
             [
@@ -1062,6 +1137,8 @@ export function registerContextCommand(pi: ExtensionAPI, runtime: Ds4ContextRunt
               `Foreign keys:        ${health.foreignKeys ? "enabled" : "disabled"}`,
               `Schema version:      ${health.schemaVersion}`,
               `Applied migrations:  ${health.appliedMigrations}`,
+              `Storage status:      ${storage.status}`,
+              `Maintenance recommended: ${storage.maintenance.recommended ? "yes" : "no"}`,
               `Project stale snippets: ${count(staleProjectSnippets)}`,
               `Memory/pin warnings: ${count(diagnostics.memory.warnings.length)}`,
               `Privacy enforcement:  ${diagnostics.privacy.enforcement}`,