ds4-context-engine 0.3.2 → 0.3.3

package/docs/COMPACTION.md

@@ -13,7 +13,7 @@ DS4 intercepts Pi's `session_before_compact` event but preserves Pi's cut-point
 7. The highest input classification is wrapped around each generated node, then all nodes are persisted atomically as one `prepared` graph batch. Usage is summed across every segment and aggregate request; Pi receives only the final root text and still appends exactly one canonical `CompactionEntry` with `fromHook: true`.
 8. `session_compact` commits all nodes and associates the active root with the Pi entry; failure marks the complete prepared batch `failed`.
 
-Fan-out and fan-in are bounded to 32 segment requests, 64 aggregate requests, and 16 aggregate passes. Transport replay is independently bounded to three attempts per segment or aggregate call and does not retry input, usage, rate, authentication, validation, or output-limit failures. A base prompt, individual message, atomic tool exchange, pair of child summaries, or total operation that cannot fit within those limits fails closed. Any mapping, budget, model, output-limit, validation, abort, or storage error returns `undefined` from the hook, allowing Pi's default compaction to run.
+Fan-out and fan-in are bounded to 32 segment requests, 64 aggregate requests, and 16 aggregate passes. Transport replay mirrors Pi's assistant retry policy: `compaction.transport.maxAttempts` (default 3) total attempts and `compaction.transport.baseDelayMs` (default 2000 ms, capped at 60 s) backoff, doubling per attempt, abort-aware. Replay never applies to input, usage, rate, authentication, validation, or output-limit failures. A base prompt, individual message, atomic tool exchange, pair of child summaries, or total operation that cannot fit within those limits fails closed. Any mapping, budget, model, output-limit, validation, abort, or storage error returns `undefined` from the hook, allowing Pi's default compaction to run.
 
 ## Required summary contract
 
@@ -78,6 +78,26 @@ Semantics:
 - `compaction.summary.thinking` defaults to `off` and applies only to summary requests: `off` keeps the pre-existing request shape (no thinking fields), while other levels map per API (`thinkingEnabled`/`effort` for `anthropic-messages`, `samplingParams.reasoning_effort` for OpenAI-compatible APIs) and are ignored for unsupported providers;
 - `context.maxSummaryTokens` remains a session-level limit and does not rise for the dedicated model; the minimum with the model's `maxTokens` still applies.
 
+## Transport retry policy
+
+Summary requests are replayed only for transport-classified failures (thrown transport errors or `stopReason: "error"` responses whose message matches network/timeout patterns). The replay policy defaults to Pi's assistant retry policy and can be tuned per deployment:
+
+```json
+{
+  "compaction": {
+    "transport": {
+      "maxAttempts": 3,
+      "baseDelayMs": 2000
+    }
+  }
+}
+```
+
+- `compaction.transport.maxAttempts`: total attempts per segment or aggregate call, integer 1–10, default 3. With 1, no transport failure is retried.
+- `compaction.transport.baseDelayMs`: base backoff before the first replay, integer 0–60000, default 2000. The delay doubles per attempt (2000, 4000, 8000, …) and is capped at 60 s.
+- Replays use a fresh routing session per attempt; diagnostics expose only stage, failed/next attempt, max attempts, and delay.
+- Aborts (including during backoff) never trigger replay; non-transport failures are never retried; usage is summed across replayed responses.
+
 ## Proactive trigger
 
 After a settled turn, DS4 computes:

package/docs/releases/0.3.2.md

@@ -65,9 +65,9 @@ Exact post-publication verification passed for all three `0.3.2` packages on 202
 Published package shasums:
 
 ```text
-ds4-context-core:              (recorded after publication)
-ds4-context-reference-adapter: (recorded after publication)
-ds4-context-engine:            (recorded after publication)
+ds4-context-core:              ff2501ffc1c12713591d97989e708733b7b8735e
+ds4-context-reference-adapter: f7d6e29393e248b89417d44b4c19e8c80cb684c2
+ds4-context-engine:            fd77ea3f0cc2885eff34b2702951d6dd2d4c3c14
 ```
 
 For all three packages, npm `latest` now resolves to `0.3.2`, `beta` remains `0.3.0-beta.3`, `alpha` remains `0.3.0-alpha.5`, and `rc` remains `0.2.0-rc.1`.

package/docs/releases/0.3.3.md

@@ -0,0 +1,60 @@
+# DS4 Context Engine 0.3.3
+
+Status: candidate, not yet published.
+
+This release adds a configurable transport retry policy for compaction summary requests on top of the 0.3.2 stable candidate. It carries forward 0.3.2 without changing canonical records, SQLite schema, runtime contracts, retention limits, privacy policy, compaction validation, or fallback semantics.
+
+## Added since 0.3.2
+
+- **Configurable compaction transport retry policy, mirroring Pi's assistant retry settings**:
+  - `compaction.transport.maxAttempts` (default `3`) and `compaction.transport.baseDelayMs` (default `2000` ms), with exponential backoff doubling per attempt, capped at 60 s per wait, and abort-aware: replay stops promptly when the request is cancelled;
+  - policy defaults mirror Pi (`retry.maxRetries` 3, `retry.baseDelayMs` 2000, exponential backoff);
+  - validated at config load: `maxAttempts` in `1..10`, `baseDelayMs` in `0..60000` ms; invalid values are rejected before use;
+  - `summary-generator` exposes `effectiveTransportPolicy` and `transportRetryDelayMs`, uses the configured `maxAttempts` in retry loops and diagnostics;
+  - `compaction-coordinator` threads `config.compaction.transport` through the summary request path;
+  - machine-readable catalog fields in `config-catalog.ts`, so the new keys are visible to `/context config view`;
+  - tests: unit coverage for policy defaults, backoff sequence, abort awareness, attempt counting, usage aggregation, and config validation; integration retry tests run with `baseDelayMs: 1` plus a new configurable-`maxAttempts` case; the frozen 0.2-line compatibility golden was updated to include the new transport defaults;
+  - `docs/COMPACTION.md`: new retry-policy section and revised bounds paragraph.
+- Coordinated package versions moved to `0.3.3`.
+
+There are no functional storage-format or provider-path changes relative to 0.3.2.
+
+## Safety and compatibility
+
+Unchanged from 0.3.2: Pi JSONL remains canonical and append-only; SQLite remains disposable and rebuildable; physical maintenance remains explicit, offline, local-TTY-only, and unavailable to model-callable tools; every `context_persistence` write still requires a fresh positive local UI decision.
+
+Compatibility remains unchanged:
+
+- SQLite schema: `15`;
+- configuration: `ds4-context-config-v1` (new keys added: `compaction.transport.maxAttempts`, `compaction.transport.baseDelayMs`);
+- runtime adapter: `runtime-adapter-v1`;
+- persistence tool: `ds4-context-persistence-tool-v1`;
+- persistence result: `ds4-context-persistence-result-v1`;
+- Pi: `0.84.3`;
+- Node.js: `>=22.19.0`.
+
+## Package/version policy
+
+The coordinated version is `0.3.3` for:
+
+```text
+ds4-context-core
+ds4-context-reference-adapter
+ds4-context-engine
+```
+
+Both adapters depend exactly on `ds4-context-core@0.3.3`. Publication uses npm's default `latest` tag for all three packages. GitHub Actions remains validation-only with OIDC and package-write permissions denied.
+
+## Candidate validation
+
+Local candidate verification on Node.js `26.5.1`:
+
+- `npm run check`: 72 files and 376 tests passed (including the new transport-policy unit coverage, the configurable-`maxAttempts` integration case, and the updated golden freeze).
+- `npm run quality:compare`: candidate quality versus the frozen baseline passed.
+- `npm run schema:context-persistence`: within the 1,500 / 320 absolute and relative limits.
+- `npm run latency:check` against the frozen baseline passed (`regressionRatio` `1.040189`, maximum `1.1`).
+- `npm run pack:check` and `npm pack --dry-run --json` for all three packages passed with no forbidden local/session/storage paths.
+- `git diff --check` passed; the only pre-existing untracked path is `.serena/`, which is excluded from commits and package inventories.
+- Version, exact core dependencies, package-lock entries, extension constant, and reference-adapter constant are synchronized to `0.3.3`.
+
+Validation-only CI is recorded below with the release commit. Exact registry verification and the annotated tag are recorded after execution.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ds4-context-engine",
-  "version": "0.3.2",
+  "version": "0.3.3",
   "description": "Non-destructive, provider-independent context management for Pi.",
   "type": "module",
   "license": "MIT",
@@ -62,7 +62,7 @@
     ]
   },
   "dependencies": {
-    "ds4-context-core": "0.3.2"
+    "ds4-context-core": "0.3.3"
   },
   "peerDependencies": {
     "@earendil-works/pi-ai": "0.84.3",

package/src/pi-adapter/compaction-coordinator.ts

@@ -1005,6 +1005,7 @@ export class CompactionCoordinator {
       ...input,
       validate: this.dependencies.config.compaction.validate,
       maxSummaryTokens: this.dependencies.config.context.maxSummaryTokens,
+      transport: this.dependencies.config.compaction.transport,
       now: this.dependencies.now,
       onTransportRetry: (diagnostic) => {
         this.state.transportRetries = (this.state.transportRetries ?? 0) + 1;

package/src/pi-adapter/summary-generator.ts

@@ -13,8 +13,39 @@ import {
   type SummaryValidationResult,
 } from "ds4-context-core/compaction/summary-contract";
 
-export const COMPACTION_TRANSPORT_MAX_ATTEMPTS = 3;
-const COMPACTION_TRANSPORT_RETRY_DELAYS_MS = [200, 500] as const;
+export const DEFAULT_COMPACTION_TRANSPORT_MAX_ATTEMPTS = 3;
+export const DEFAULT_COMPACTION_TRANSPORT_BASE_DELAY_MS = 2000;
+export const COMPACTION_TRANSPORT_MAX_DELAY_MS = 60_000;
+
+/**
+ * Transport retry policy for compaction summary requests. Defaults mirror Pi's
+ * assistant retry settings (`retry.maxRetries` 3, `retry.baseDelayMs` 2000,
+ * exponential backoff, abort-aware).
+ */
+export interface CompactionTransportPolicy {
+  /** Total attempts for transport-classified failures. Default: 3. */
+  maxAttempts?: number;
+  /** Base backoff delay in ms, doubled per attempt. Default: 2000. */
+  baseDelayMs?: number;
+}
+
+export function effectiveTransportPolicy(
+  policy: CompactionTransportPolicy | undefined,
+): { maxAttempts: number; baseDelayMs: number } {
+  const maxAttempts = Math.min(
+    10,
+    Math.max(1, policy?.maxAttempts ?? DEFAULT_COMPACTION_TRANSPORT_MAX_ATTEMPTS),
+  );
+  const baseDelayMs = Math.min(
+    COMPACTION_TRANSPORT_MAX_DELAY_MS,
+    Math.max(0, policy?.baseDelayMs ?? DEFAULT_COMPACTION_TRANSPORT_BASE_DELAY_MS),
+  );
+  return { maxAttempts, baseDelayMs };
+}
+
+export function transportRetryDelayMs(baseDelayMs: number, failedAttempt: number): number {
+  return Math.min(COMPACTION_TRANSPORT_MAX_DELAY_MS, baseDelayMs * 2 ** (failedAttempt - 1));
+}
 
 export interface CompactionTransportRetryDiagnostic {
   stage: "segment" | "aggregate";
@@ -38,6 +69,8 @@ export interface GenerateValidatedSummaryInput {
   model?: Model<Api>;
   /** Reasoning level for the summary request; `off` (default) keeps the pre-existing request shape. */
   thinking?: CompactionThinkingLevel;
+  /** Transport retry policy; defaults mirror Pi's assistant retry settings. */
+  transport?: CompactionTransportPolicy;
   now: () => number;
   onTransportRetry?: (diagnostic: CompactionTransportRetryDiagnostic) => void;
 }
@@ -162,6 +195,7 @@ export async function generateValidatedSummary(
   const model = input.model ?? input.ctx.model;
   if (!model) throw new Error("Compaction summary generation requires an active model");
   const maxTokens = Math.max(1, Math.min(input.maxSummaryTokens, model.maxTokens ?? input.maxSummaryTokens));
+  const { maxAttempts, baseDelayMs } = effectiveTransportPolicy(input.transport);
   const retryUsages: Usage[] = [];
   let response: Awaited<ReturnType<typeof input.ctx.modelRegistry.complete>>;
   let attempt = 0;
@@ -189,18 +223,18 @@ export async function generateValidatedSummary(
     } catch (error) {
       if (input.event.signal.aborted) throw abortedError();
       const category = providerFailureCategory(error);
-      if (category !== "transport" || attempt >= COMPACTION_TRANSPORT_MAX_ATTEMPTS) {
+      if (category !== "transport" || attempt >= maxAttempts) {
         throw new Error(
           `Compaction ${input.stage} request failed (${transportFailureSuffix(category, attempt)})`,
         );
       }
-      const delayMs = COMPACTION_TRANSPORT_RETRY_DELAYS_MS[attempt - 1] ?? 500;
+      const delayMs = transportRetryDelayMs(baseDelayMs, attempt);
       await waitForTransportRetry(input.event.signal, delayMs);
       input.onTransportRetry?.({
         stage: input.stage,
         failedAttempt: attempt,
         nextAttempt: attempt + 1,
-        maxAttempts: COMPACTION_TRANSPORT_MAX_ATTEMPTS,
+        maxAttempts,
         delayMs,
       });
       continue;
@@ -209,15 +243,15 @@ export async function generateValidatedSummary(
     const stopReason = responseStopReason(response);
     if (stopReason === "error") {
       const category = providerFailureCategory(responseErrorMessage(response));
-      if (category === "transport" && attempt < COMPACTION_TRANSPORT_MAX_ATTEMPTS) {
+      if (category === "transport" && attempt < maxAttempts) {
         retryUsages.push(response.usage);
-        const delayMs = COMPACTION_TRANSPORT_RETRY_DELAYS_MS[attempt - 1] ?? 500;
+        const delayMs = transportRetryDelayMs(baseDelayMs, attempt);
         await waitForTransportRetry(input.event.signal, delayMs);
         input.onTransportRetry?.({
           stage: input.stage,
           failedAttempt: attempt,
           nextAttempt: attempt + 1,
-          maxAttempts: COMPACTION_TRANSPORT_MAX_ATTEMPTS,
+          maxAttempts,
           delayMs,
         });
         continue;

package/src/pi-adapter/version.ts

@@ -1,4 +1,4 @@
-export const EXTENSION_VERSION = "0.3.2";
+export const EXTENSION_VERSION = "0.3.3";
 export const SUPPORTED_PI_VERSION = "0.84.3";
 export const OBSERVER_PLANNER_VERSION = "observer-model-aware-v1";
 export const PLANNER_VERSION = "managed-learned-ranking-v1";