ds4-context-engine 0.3.0-alpha.4 → 0.3.0-alpha.5

package/README.md

@@ -16,7 +16,7 @@ bounded active context with provenance
 Pi provider
 ```
 
-> **Project status:** Stable `0.2.0` includes M0–M20 and frozen 0.2 contracts. Published prerelease `0.3.0-alpha.3` adds privacy-safe exact-value repair diagnostics and tighter summary prompting while preserving strict compaction validation, Pi fallback, canonical records, SQLite schema 15, and runtime contracts. npm `alpha` points to `0.3.0-alpha.3`, `latest` remains `0.2.0`, and the maintenance line targets Pi `0.84.3`.
+> **Project status:** Stable `0.2.0` includes M0–M20 and frozen 0.2 contracts. Published prerelease `0.3.0-alpha.4` improves `context_persistence` action guidance and privacy-safe validation categories, and moves routine successful compaction lifecycle events to `debug`. Confirmation, canonical records, strict compaction validation, Pi fallback, SQLite schema 15, and runtime contracts remain unchanged. npm `alpha` points to `0.3.0-alpha.4`, `latest` remains `0.2.0`, and the maintenance line targets Pi `0.84.3`.
 
 ## Why DS4
 
@@ -29,7 +29,7 @@ It provides:
 - exact and FTS5 historical retrieval with source provenance;
 - opt-in hybrid semantic retrieval with a deterministic local embedding and lexical fallback;
 - trust-gated structural project indexing, Git-aware invalidation and bounded source snippets;
-- hierarchical, validated, non-destructive compaction summaries;
+- hierarchical, validated, non-destructive compaction summaries with overflow-safe multi-request fan-out/fan-in;
 - persistent pins and append-only durable memory stored canonically in Pi JSONL;
 - a bounded, metadata-only `context_persistence` tool with local confirmation for every model-callable write;
 - opt-in checkpointed project-memory replay across exact trusted Pi project sessions;
@@ -92,7 +92,7 @@ The stable `0.2.0` packages (`ds4-context-engine`, `ds4-context-core`, and `ds4-
 To dogfood the published alpha without replacing a global stable installation, pin it in a disposable project:
 
 ```bash
-pi install -l npm:ds4-context-engine@0.3.0-alpha.3
+pi install -l npm:ds4-context-engine@0.3.0-alpha.4
 ```
 
 Follow the [0.3 alpha dogfooding runbook](docs/DOGFOODING_0.3.0_ALPHA.md); use synthetic data and a dedicated session directory.
@@ -416,7 +416,7 @@ npm run schema:context-persistence
 npm run latency:check -- /path/to/exact/ds4-context-core@0.1.2
 npm run pack:check
 # Post-publication, with an exact version rather than a dist-tag:
-npm run registry:check -- 0.3.0-alpha.3
+npm run registry:check -- 0.3.0-alpha.4
 npm pack --dry-run
 npm pack --dry-run --workspace ds4-context-core
 npm pack --dry-run --workspace ds4-context-reference-adapter
@@ -468,6 +468,7 @@ scripts             package and release-readiness checks
 - [0.2.0 release readiness](docs/RELEASE_READINESS_0.2.0.md)
 - [0.2.0 release notes](docs/releases/0.2.0.md)
 - [0.2.0-rc.1 release notes](docs/releases/0.2.0-rc.1.md)
+- [0.3.0-alpha.4 prerelease notes](docs/releases/0.3.0-alpha.4.md)
 - [0.3.0-alpha.3 prerelease notes](docs/releases/0.3.0-alpha.3.md)
 - [0.3.0-alpha.2 prerelease notes](docs/releases/0.3.0-alpha.2.md)
 - [0.3.0-alpha.1 prerelease notes](docs/releases/0.3.0-alpha.1.md)
@@ -478,7 +479,7 @@ scripts             package and release-readiness checks
 
 The original M0–M13 roadmap is complete. `ds4-context-core` contains the compiled runtime-neutral implementation. M14 context-quality metrics, M15 rich symbol indexing, M16 hybrid semantic retrieval, M17 cross-session project memory, M18 learned-ranking shadow evaluation, M19's runtime adapter/conformance kit, and M20 opt-in local KV eligibility/replay are implemented on `main`. Learned active ranking remains promotion-gated, Pi reports local KV as unsupported, and static ranking/native completion stay authoritative on every failure.
 
-The [0.2.0 roadmap](docs/ROADMAP_0.2.0.md) is complete. Prerelease `0.3.0-alpha.3` retains the confirmed [context persistence tool](docs/CONTEXT_PERSISTENCE_TOOL.md) hardening from alpha.2 and adds privacy-safe [compaction](docs/COMPACTION.md) repair diagnostics without weakening exact-value grounding or Pi fallback. Stable canonical/configuration/SQLite/runtime contracts remain unchanged. The [0.2 readiness record](docs/RELEASE_READINESS_0.2.0.md) remains the compatibility baseline. Sensitive or transport-specific behavior remains opt-in, and the 0.1 lexical planner stays available as the deterministic fallback.
+The [0.2.0 roadmap](docs/ROADMAP_0.2.0.md) is complete. Prerelease `0.3.0-alpha.4` builds on the prior [context persistence tool](docs/CONTEXT_PERSISTENCE_TOOL.md) and privacy-safe [compaction](docs/COMPACTION.md) hardening with clearer action-specific failures and quieter routine lifecycle logs. Confirmation, exact targeting, strict summary grounding, Pi fallback, and stable canonical/configuration/SQLite/runtime contracts remain unchanged. The [0.2 readiness record](docs/RELEASE_READINESS_0.2.0.md) remains the compatibility baseline. Sensitive or transport-specific behavior remains opt-in, and the 0.1 lexical planner stays available as the deterministic fallback.
 
 ## Contributing
 

package/docs/ARCHITECTURE.md

@@ -101,12 +101,13 @@ agent_settled
 session_before_compact
   -> map Pi preparation messages to exact canonical entry IDs
   -> privacy-sanitize discarded text/instructions/file lists for the active provider
-  -> serialize only the newly discarded segment and preserve Pi's retained boundary
-  -> generate and validate an immutable segment node
+  -> preflight the complete sanitized request against the calibrated model input budget
+  -> when needed, split at contiguous message boundaries while keeping matching tool calls/results atomic
+  -> generate and validate each immutable segment against only its own sanitized evidence
   -> resolve the prior root from the active Pi branch only
-  -> generate and validate a bounded aggregate node when a prior root exists
+  -> recursively generate and validate bounded ordered aggregate layers until one root remains
   -> wrap generated nodes with their highest inherited classification for provider-switch safety
-  -> atomically persist prepared nodes/edges and return only the active root to Pi
+  -> atomically persist the complete prepared graph and return only one active root to Pi
 
 session_compact / session_compact_failed
   -> commit or fail prepared summary lifecycle

package/docs/COMPACTION.md

@@ -7,13 +7,13 @@ DS4 intercepts Pi's `session_before_compact` event but preserves Pi's cut-point
 1. Pi determines `messagesToSummarize`, optional split-turn prefix, and `firstKeptEntryId`.
 2. DS4 maps every source message by exact fingerprint to a canonical branch entry ID.
 3. Pi's serializer converts the newly discarded span to bounded conversation text; enabled privacy policy sanitizes conversation, custom instructions, and file paths for the active provider.
-4. The active model generates and validates an immutable segment summary against sanitized evidence with cache retention disabled and a fresh routing session ID.
-5. If the active branch has a previous DS4 root, a second bounded call aggregates only that root's content with the new segment content; DS4-generated IDs, hashes, kinds, and graph levels remain outside model-visible evidence. A Pi-native predecessor is imported as an explicitly unverified branch node.
-6. The highest input classification is wrapped around each generated node, then all nodes are persisted atomically as `prepared`; Pi receives only the active segment or aggregate text.
-7. Pi appends its normal `CompactionEntry` with `fromHook: true`.
-8. `session_compact` commits all nodes and associates the active root with the Pi entry; failure marks the prepared batch `failed`.
+4. DS4 estimates the complete sanitized summary request against the calibrated active-model input budget. If the whole request does not fit, it partitions the source into ordered contiguous segments. Individual messages are indivisible, and every tool call remains in the same atomic group as all matching results; an exceptionally large turn may split only between such groups.
+5. The active model generates and validates each immutable segment summary only against that segment's sanitized evidence, with cache retention disabled and a fresh routing session ID per call.
+6. DS4 recursively aggregates ordered child summaries in bounded fan-in calls until one root remains. The ordered roots include the previous branch summary, when present, followed by every new segment. DS4-generated IDs, hashes, kinds, and graph levels remain outside model-visible evidence. A Pi-native predecessor is imported as an explicitly unverified branch node.
+7. The highest input classification is wrapped around each generated node, then all nodes are persisted atomically as one `prepared` graph batch. Usage is summed across every segment and aggregate request; Pi receives only the final root text and still appends exactly one canonical `CompactionEntry` with `fromHook: true`.
+8. `session_compact` commits all nodes and associates the active root with the Pi entry; failure marks the complete prepared batch `failed`.
 
-Any mapping, model, output-limit, validation, or storage error returns `undefined` from the hook, allowing Pi's default compaction to run.
+Fan-out and fan-in are bounded to 32 segment requests, 64 aggregate requests, and 16 aggregate passes. A base prompt, individual message, atomic tool exchange, pair of child summaries, or total operation that cannot fit within those limits fails closed. Any mapping, budget, model, output-limit, validation, abort, or storage error returns `undefined` from the hook, allowing Pi's default compaction to run.
 
 ## Required summary contract
 
@@ -32,7 +32,7 @@ Any mapping, model, output-limit, validation, or storage error returns `undefine
 ## Critical Exact Values
 ```
 
-Every section must occur once, in order, and contain content or `- None`. DS4 replaces each unique `Files Read` and `Files Modified` section with one exact path per bullet from Pi's sanitized file-operation inventory before validation; missing or duplicate sections still fail. Backticked exact values must occur in the serialized segment source, ordered child-summary content, or those known file-operation paths. A bounded unsupported exact-value bullet is removed as a whole and recorded as a validation warning; unsupported prose, more than eight affected bullets, or removal above 25% still fails closed to Pi. The prompt explicitly asks the model to omit a bullet whose complete backticked span cannot be copied verbatim. Segment and aggregate outputs are validated independently; either unrepaired failure prevents the whole graph batch from being installed.
+Every section must occur once, in order, and contain content or `- None`. DS4 replaces each unique `Files Read` and `Files Modified` section with one exact path per bullet from Pi's sanitized file-operation inventory before validation; missing or duplicate sections still fail. Backticked exact values must occur in the serialized segment source, ordered child-summary content, or those known file-operation paths. A bounded unsupported exact-value bullet is removed as a whole and recorded as a validation warning; unsupported prose, more than eight affected bullets, or removal above 25% still fails closed to Pi. The prompt explicitly asks the model to omit a bullet whose complete backticked span cannot be copied verbatim. Every segment is validated independently against only its own sanitized source and deterministic file inventory. Every aggregate is validated independently against only the sanitized content of its ordered children and cumulative deterministic file inventory. Any unrepaired failure prevents the whole graph batch from being installed.
 
 An unrepaired exact-value failure reports only the stage, issue code, categorical repair status, unsupported-span count, and affected-bullet count. Repair statuses distinguish an unsupported location, more than eight bullets, removal above 25%, and an unexpected invalid second validation. The disputed text is intentionally absent from logs, UI notifications, and diagnostics because it may contain sensitive source material.
 
@@ -76,4 +76,4 @@ It requests compaction at most once per session leaf. Pi's native threshold and
 /context summaries
 ```
 
-The preview reports thresholds and eligibility; Pi remains authoritative for the exact cut point. Routine `summary_graph_prepared` and `summary_graph_committed` lifecycle events are emitted only at `debug`; fallback, failure, persistence, and reconciliation problems remain actionable warnings. The proactive-threshold TUI notification remains a user-visible `info` notice because it explains why an automatic compaction started.
+The preview reports thresholds and eligibility; Pi remains authoritative for the exact cut point. Runtime diagnostics additionally report the calibrated input budget, estimated whole-source prompt size, generated segment count, and aggregate-call count without source content. Routine `summary_graph_prepared` and `summary_graph_committed` lifecycle events are emitted only at `debug`; fallback, failure, persistence, and reconciliation problems remain actionable warnings. Oversized-group and bounded-operation errors expose only numeric budgets/counts, never rejected source text. Provider failures are reduced to metadata-only categories such as `input-limit`, `usage-limit`, `rate-limit`, `authentication`, or `transport`; raw provider error details are not logged or shown. The proactive-threshold TUI notification remains a user-visible `info` notice because it explains why an automatic compaction started.

package/docs/DOGFOODING_0.3.0_ALPHA.md

@@ -1,6 +1,6 @@
 # Dogfooding DS4 0.3.0 Alpha
 
-This runbook validates the published `ds4-context-engine@0.3.0-alpha.3` package through sustained real Pi use. It complements automated tests and release smoke checks; it does not replace them.
+This runbook validates the published `ds4-context-engine@0.3.0-alpha.4` package through sustained real Pi use. It complements automated tests and release smoke checks; it does not replace them.
 
 The primary target is the model-callable `context_persistence` surface. Pi JSONL must remain canonical and append-only, SQLite must remain rebuildable, and no model-callable write may occur without a fresh positive local UI decision.
 
@@ -10,7 +10,8 @@ The primary target is the model-callable `context_persistence` surface. Pi JSONL
 - Use a disposable trusted project and a dedicated session directory.
 - Use synthetic, non-secret Pin/Memory content. Local TUI dialogs and JSON event streams may display current tool arguments.
 - Published alpha.1 had a retry limitation: copying `[omitted-by-ds4-egress-policy]` from sanitized history could present a confirmation for the literal marker. Alpha.2 and later reserve that output-only marker and must reject it as `egress-placeholder` before runtime access, confirmation, canonical append, or derived-policy update.
-- Alpha.3 keeps exact-value summary validation strict. An unrepaired failure may expose only its stage, issue code, repair category, and counts; disputed exact text must not appear in diagnostics.
+- Alpha.3 and later keep exact-value summary validation strict. An unrepaired failure may expose only its stage, issue code, repair category, and counts; disputed exact text must not appear in diagnostics.
+- Alpha.4 distinguishes unsupported and missing action parameters without echoing rejected fields or values. Routine successful compaction lifecycle events require `diagnostics.logLevel=debug`; actionable fallback warnings remain visible.
 - Do not use `--no-session` except for the explicit fail-closed test. Without a persistent Pi JSONL destination, both reads and writes return `runtime-unavailable`.
 - Do not retry `committed_projection_pending` or `indeterminate`. Inspect state with a read, `/context health`, or `/context rebuild-index` first.
 - Use `/context` only for local inspection and recovery. Mutations under test must go through `context_persistence` so the confirmation and provider-egress boundaries are exercised.
@@ -27,7 +28,7 @@ mkdir -p /tmp/ds4-alpha-dogfood
 cd /tmp/ds4-alpha-dogfood
 git init
 mkdir -p sessions evidence
-pi install -l npm:ds4-context-engine@0.3.0-alpha.3
+pi install -l npm:ds4-context-engine@0.3.0-alpha.4
 pi list
 pi --version
 ```
@@ -47,19 +48,19 @@ Record the Pi version, DS4 version, provider/model, mode, session name, expected
 Use unique non-sensitive values so duplicates from earlier runs cannot hide a failure. Example run label:
 
 ```text
-alpha2-run-01
+alpha4-run-01
 ```
 
 Example Pin:
 
 ```text
-For alpha2-run-01 verification, use Node.js 22 in this disposable project.
+For alpha4-run-01 verification, use Node.js 22 in this disposable project.
 ```
 
 Example Memory:
 
 ```text
-For alpha2-run-01, the synthetic release channel is amber.
+For alpha4-run-01, the synthetic release channel is amber.
 ```
 
 Never use real credentials, customer data, private paths, or production policy in dogfooding prompts.
@@ -77,7 +78,7 @@ Run these scenarios in order:
 
 1. Inspect `/context health`, `/context pins`, `/context memory`, and `/context privacy`.
 2. Send an ordinary suggestion without asking to persist it, for example: `The synthetic release channel amber seems useful.` No `context_persistence` call or confirmation dialog should appear.
-3. Explicitly request the synthetic Memory: `Remember for this session that the synthetic release channel for alpha2-run-01 is amber.` Verify that the dialog identifies the action and canonical persistence class. Accept it. Expect one committed Memory mutation.
+3. Explicitly request the synthetic Memory: `Remember for this session that the synthetic release channel for alpha4-run-01 is amber.` Verify that the dialog identifies the action and canonical persistence class. Accept it. Expect one committed Memory mutation.
 4. Ask the model to use `context_persistence` to list active Memory. Verify bounded metadata and no complete claim, key, reason, path, or raw error in the result.
 5. Explicitly request the synthetic Pin, but reject or close the confirmation dialog. Verify with `/context pins` that it was not created.
 6. Ask the model to retry using the sanitized value remaining in history. If it copies `[omitted-by-ds4-egress-policy]`, expect `rejected / egress-placeholder` before any new confirmation, runtime mutation, or append. If it asks for fresh text instead, record that safe routing result and run the exact-marker case from the JSON procedure.
@@ -116,7 +117,7 @@ Enter one JSON object per line on stdin. First request a read:
 Wait for the turn to end before sending the next prompt. For a write:
 
 ```json
-{"id":"write-1","type":"prompt","message":"Persist a session Pin for alpha2-run-01 stating that this disposable project uses Node.js 22 for verification."}
+{"id":"write-1","type":"prompt","message":"Persist a session Pin for alpha4-run-01 stating that this disposable project uses Node.js 22 for verification."}
 ```
 
 Pi should emit a request shaped like:
@@ -168,7 +169,7 @@ The read should complete. Then request a write:
 
 ```bash
 pi -p --approve --session-dir "$DS4_DOGFOOD_SESSIONS" \
-  "Use context_persistence to add a session Memory saying that alpha2-run-01 uses the synthetic channel amber."
+  "Use context_persistence to add a session Memory saying that alpha4-run-01 uses the synthetic channel amber."
 ```
 
 Expected behavior:
@@ -198,7 +199,7 @@ Write case:
 
 ```bash
 pi --mode json --approve --session-dir "$DS4_DOGFOOD_SESSIONS" \
-  "Use context_persistence to add a session Pin for alpha2-run-01 stating that this disposable project uses Node.js 22." \
+  "Use context_persistence to add a session Pin for alpha4-run-01 stating that this disposable project uses Node.js 22." \
   2>evidence/json-write.stderr.log | tee evidence/json-write.jsonl
 ```
 

package/docs/releases/0.3.0-alpha.4.md

@@ -1,6 +1,6 @@
 # DS4 Context Engine 0.3.0-alpha.4
 
-Status: validated release candidate; not yet published.
+Status: published prerelease on 2026-09-02; tag `v0.3.0-alpha.4`.
 
 This coordinated prerelease improves model-callable `context_persistence` guidance and categorical validation diagnostics, and removes routine successful compaction lifecycle events from normal `info` output. It preserves strict persistence confirmation, canonical Pi JSONL semantics, privacy-safe egress, compaction validation, and Pi fallback behavior.
 
@@ -29,7 +29,7 @@ ds4-context-reference-adapter
 ds4-context-engine
 ```
 
-Both adapters depend exactly on `ds4-context-core@0.3.0-alpha.4`. Publication must use the explicit npm `alpha` dist-tag so `latest` remains `0.2.0`. GitHub Actions remains validation-only with OIDC and package-write permissions denied.
+Both adapters depend exactly on `ds4-context-core@0.3.0-alpha.4`. The packages were published manually under the explicit npm `alpha` dist-tag while `latest` remains `0.2.0`. GitHub Actions remains validation-only with OIDC and package-write permissions denied.
 
 ## Validation evidence
 
@@ -48,7 +48,9 @@ Local candidate verification on Node.js `26.5.1`:
 - `git diff --check`: passed.
 - Protected CI, compatibility golden, Pi fixture, migration, canonical Pin/Memory, and persistence result-contract files: unchanged.
 
-Clean committed-candidate replay and exact registry verification are required before tag or GitHub prerelease creation.
+- `npm run registry:check -- 0.3.0-alpha.4`: passed against all three exact published versions; `alpha` resolves to `0.3.0-alpha.4` and `latest` remains `0.2.0` for every package.
+
+Exact registry verification passed before the annotated tag and GitHub prerelease were created.
 
 ## Documentation
 

package/docs/releases/0.3.0-alpha.5.md

@@ -0,0 +1,58 @@
+# DS4 Context Engine 0.3.0-alpha.5
+
+Status: validated release candidate; not yet published.
+
+This coordinated prerelease adds overflow-safe hierarchical compaction for discarded source that cannot fit in one active-model summary request. It preserves strict summary validation, privacy sanitization, canonical Pi JSONL semantics, one final Pi compaction entry, and fail-closed fallback to Pi default compaction.
+
+## Added
+
+- Preflights the complete sanitized compaction prompt against the calibrated active-model input budget.
+- Partitions oversized discarded source at contiguous message boundaries while keeping matching tool calls and results in the same atomic group.
+- Generates and validates each segment only against that segment's sanitized source and deterministic file inventory.
+- Recursively aggregates ordered child summaries when one fan-in request cannot fit, preserving cumulative provenance and graph topology.
+- Bounds work to 32 segment requests, 64 aggregate requests, and 16 aggregate passes.
+- Exposes metadata-only diagnostics for input budget, whole-source prompt estimate, segment count, and aggregate-call count.
+- Categorizes provider failures as `input-limit`, `usage-limit`, `rate-limit`, `authentication`, `transport`, `aborted`, or `provider-error` without surfacing provider payloads.
+
+## Safety and compatibility
+
+An individual message, atomic tool exchange, aggregate pair, or total operation that cannot fit within the bounded limits fails closed to Pi default compaction. Generated segment and aggregate summaries retain strict structural and exact-value validation; bounded unsupported-exact-value bullet repair remains unchanged. Rejected exact values and raw provider error details are never emitted in diagnostics.
+
+Pi receives one final canonical compaction entry. The complete prepared summary-graph batch is committed only after Pi installs that entry; failed attempts remain non-destructive. Pi JSONL stays canonical and append-only, while SQLite remains a disposable projection. Schema 15, migrations 1–15, `ds4-context-config-v1`, `runtime-adapter-v1`, `ds4-context-persistence-tool-v1`, and `ds4-context-persistence-result-v1` are unchanged.
+
+## Package/version policy
+
+The coordinated version is `0.3.0-alpha.5` for:
+
+```text
+ds4-context-core
+ds4-context-reference-adapter
+ds4-context-engine
+```
+
+Both adapters depend exactly on `ds4-context-core@0.3.0-alpha.5`. Publication must use the explicit npm `alpha` dist-tag so `latest` remains `0.2.0`. GitHub Actions remains validation-only with OIDC and package-write permissions denied.
+
+## Validation evidence
+
+Local candidate verification on Node.js `26.5.1`:
+
+- `npm run check`: 65 files and 297 tests passed.
+- Focused compaction coverage includes three-segment fan-out, recursive fan-in, tool-exchange atomicity, exact contiguous source slicing, oversized indivisible-source fallback, and privacy-safe provider input-limit categorization.
+- `npm run quality:compare`: candidate quality `0.9875` versus baseline `0.808156`.
+- `npm run schema:context-persistence`: 1,266 bytes and 317 estimated tokens; below the 1,500 absolute and 320 relative limits.
+- `npm run latency:check -- <exact ds4-context-core@0.1.2>`: isolated clean-checkout retry passed with ratio `1.041564`, at or below `1.10`; an immediately preceding noisy sample measured `1.157479`, while the pre-commit local sample measured `0.939056`.
+- `npm run pack:check`: verified core (207 files), reference adapter (7 files), and Pi adapter (62 files) in a clean consumer.
+- `npm pack --dry-run --json` for all three packages passed with the same bounded inventories and no forbidden local/session files.
+- The committed candidate was replayed from detached clean checkout `8b609c0`; `npm ci`, the 65-file/297-test suite, quality, schema, package verification, tarball review, latency retry, and `git diff --check` passed.
+- `git diff --check`: passed.
+- Protected CI, compatibility golden, Pi fixture, migration, canonical Pin/Memory, persistence confirmation, and persistence result-contract files are unchanged.
+
+Exact registry verification is required before tag or GitHub prerelease creation.
+
+## Documentation
+
+- [`../COMPACTION.md`](../COMPACTION.md)
+- [`../ARCHITECTURE.md`](../ARCHITECTURE.md)
+- [`../PRIVACY.md`](../PRIVACY.md)
+- [`../RELEASING.md`](../RELEASING.md)
+- [`0.3.0-alpha.4.md`](0.3.0-alpha.4.md)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ds4-context-engine",
-  "version": "0.3.0-alpha.4",
+  "version": "0.3.0-alpha.5",
   "description": "Non-destructive, provider-independent context management for Pi.",
   "type": "module",
   "license": "MIT",
@@ -58,7 +58,7 @@
     ]
   },
   "dependencies": {
-    "ds4-context-core": "0.3.0-alpha.4"
+    "ds4-context-core": "0.3.0-alpha.5"
   },
   "peerDependencies": {
     "@earendil-works/pi-ai": "0.84.3",

package/src/extension/commands.ts

@@ -734,6 +734,10 @@ function formatCompaction(diagnostics: RuntimeDiagnostics, preview: boolean): st
     `Last trigger:            ${compaction.trigger ?? "n/a"}`,
     `Summary ID:              ${compaction.summaryId ?? "n/a"}`,
     `Source entries:          ${count(compaction.sourceEntries)}`,
+    `Input budget:            ${count(compaction.inputBudgetTokens)}`,
+    `Whole-source prompt:     ${count(compaction.sourcePromptTokens)}`,
+    `Generated segments:      ${count(compaction.segmentCount)}`,
+    `Aggregate calls:         ${count(compaction.aggregateCalls)}`,
     `Validation:              ${compaction.validationStatus ?? "n/a"}`,
     `First kept entry:        ${compaction.firstKeptEntryId ?? "n/a"}`,
     `Tokens before:           ${count(compaction.tokensBefore)}`,

package/src/pi-adapter/compaction-adapter.ts

@@ -19,14 +19,22 @@ interface SourceCandidate {
   used: boolean;
 }
 
-export interface PreparedCompactionSource {
-  messages: SessionBeforeCompactEvent["preparation"]["messagesToSummarize"];
+type CompactionMessage = SessionBeforeCompactEvent["preparation"]["messagesToSummarize"][number];
+
+export interface PreparedCompactionSourceSlice {
+  messages: CompactionMessage[];
   conversationText: string;
   sourceText: string;
   sourceEntryIds: string[];
   sourceHash: string;
   segmentReadFiles: string[];
   segmentModifiedFiles: string[];
+  isSplitTurn: boolean;
+}
+
+export interface PreparedCompactionSource extends PreparedCompactionSourceSlice {
+  messageEntryIds: string[];
+  turnPrefixStartIndex: number;
   readFiles: string[];
   modifiedFiles: string[];
   previousSummary?: string;
@@ -53,7 +61,7 @@ function mapSourceEntryIds(messages: readonly unknown[], entries: readonly Sessi
     const match = available.find((candidate) => !candidate.used && candidate.fingerprint === fingerprint(message));
     if (!match) throw new Error("Compaction source message has no exact canonical Pi session entry");
     match.used = true;
-    if (!ids.includes(match.entryId)) ids.push(match.entryId);
+    ids.push(match.entryId);
   }
 
   return ids;
@@ -69,8 +77,93 @@ function previousFiles(entries: readonly SessionEntry[]): { read: string[]; modi
   return { read: [], modified: [] };
 }
 
+function unique(values: Iterable<string>): string[] {
+  return [...new Set(values)];
+}
+
 function sortedUnique(values: Iterable<string>): string[] {
-  return [...new Set(values)].sort((left, right) => left.localeCompare(right));
+  return unique(values).sort((left, right) => left.localeCompare(right));
+}
+
+function segmentFiles(messages: readonly CompactionMessage[]): { readFiles: string[]; modifiedFiles: string[] } {
+  const read = new Set<string>();
+  const modified = new Set<string>();
+  for (const message of messages) {
+    if (message.role !== "assistant" || !Array.isArray(message.content)) continue;
+    for (const block of message.content) {
+      if (block.type !== "toolCall" || !block.arguments || typeof block.arguments !== "object") continue;
+      const path = "path" in block.arguments && typeof block.arguments.path === "string"
+        ? block.arguments.path
+        : undefined;
+      if (!path) continue;
+      if (block.name === "read") read.add(path);
+      if (block.name === "write" || block.name === "edit") modified.add(path);
+    }
+  }
+  return {
+    readFiles: sortedUnique([...read].filter((path) => !modified.has(path))),
+    modifiedFiles: sortedUnique(modified),
+  };
+}
+
+function createSourceSlice(input: {
+  messages: CompactionMessage[];
+  sourceEntryIds: string[];
+  segmentReadFiles: string[];
+  segmentModifiedFiles: string[];
+  isSplitTurn: boolean;
+}): PreparedCompactionSourceSlice {
+  const conversationText = serializeConversation(convertToLlm(input.messages));
+  const canonicalSourceText = stableStringify(input.messages);
+  const sourceEntryIds = unique(input.sourceEntryIds);
+  const sourceText = [
+    canonicalSourceText,
+    conversationText,
+    input.segmentReadFiles.join("\n"),
+    input.segmentModifiedFiles.join("\n"),
+  ].filter(Boolean).join("\n\n");
+  return {
+    messages: input.messages,
+    conversationText,
+    sourceText,
+    sourceEntryIds,
+    sourceHash: computeSummarySourceHash({
+      conversationText: canonicalSourceText,
+      sourceEntryIds,
+      readFiles: input.segmentReadFiles,
+      modifiedFiles: input.segmentModifiedFiles,
+    }),
+    segmentReadFiles: input.segmentReadFiles,
+    segmentModifiedFiles: input.segmentModifiedFiles,
+    isSplitTurn: input.isSplitTurn,
+  };
+}
+
+export function sliceCompactionSource(
+  source: PreparedCompactionSource,
+  messageIndices: readonly number[],
+): PreparedCompactionSourceSlice {
+  const indices = [...new Set(messageIndices)].sort((left, right) => left - right);
+  if (indices.length === 0) throw new Error("Compaction segment contains no source messages");
+  for (let offset = 0; offset < indices.length; offset++) {
+    const index = indices[offset];
+    if (index === undefined || !source.messages[index] || !source.messageEntryIds[index]) {
+      throw new Error("Compaction segment references an unavailable source message");
+    }
+    if (offset > 0 && index !== (indices[offset - 1] ?? -1) + 1) {
+      throw new Error("Compaction segment source messages must be contiguous");
+    }
+  }
+  const messages = indices.map((index) => source.messages[index] as CompactionMessage);
+  const files = segmentFiles(messages);
+  return createSourceSlice({
+    messages,
+    sourceEntryIds: indices.map((index) => source.messageEntryIds[index] as string),
+    segmentReadFiles: files.readFiles,
+    segmentModifiedFiles: files.modifiedFiles,
+    isSplitTurn: source.isSplitTurn
+      && indices.some((index) => index >= source.turnPrefixStartIndex),
+  });
 }
 
 export function findActiveBranchSummary(
@@ -109,7 +202,8 @@ export function prepareCompactionSource(event: SessionBeforeCompactEvent): Prepa
   ];
   if (messages.length === 0) throw new Error("Pi compaction preparation contains no source messages");
 
-  const sourceEntryIds = mapSourceEntryIds(messages, event.branchEntries);
+  const messageEntryIds = mapSourceEntryIds(messages, event.branchEntries);
+  const sourceEntryIds = unique(messageEntryIds);
   const priorFiles = previousFiles(event.branchEntries);
   const segmentReadFiles = sortedUnique(event.preparation.fileOps.read);
   const segmentModifiedFiles = sortedUnique([
@@ -118,30 +212,20 @@ export function prepareCompactionSource(event: SessionBeforeCompactEvent): Prepa
   ]);
   const readFiles = sortedUnique([...priorFiles.read, ...segmentReadFiles]);
   const modifiedFiles = sortedUnique([...priorFiles.modified, ...segmentModifiedFiles]);
-  const conversationText = serializeConversation(convertToLlm(messages));
-  const canonicalSourceText = stableStringify(messages);
   const previousSummary = event.preparation.previousSummary;
   const previousNode = findActiveBranchSummary(event.branchEntries, previousSummary);
-  const sourceText = [
-    canonicalSourceText,
-    conversationText,
-    segmentReadFiles.join("\n"),
-    segmentModifiedFiles.join("\n"),
-  ].filter(Boolean).join("\n\n");
-
-  return {
+  const prepared = createSourceSlice({
     messages,
-    conversationText,
-    sourceText,
     sourceEntryIds,
-    sourceHash: computeSummarySourceHash({
-      conversationText: canonicalSourceText,
-      sourceEntryIds,
-      readFiles: segmentReadFiles,
-      modifiedFiles: segmentModifiedFiles,
-    }),
     segmentReadFiles,
     segmentModifiedFiles,
+    isSplitTurn: event.preparation.isSplitTurn,
+  });
+
+  return {
+    ...prepared,
+    messageEntryIds,
+    turnPrefixStartIndex: event.preparation.messagesToSummarize.length,
     readFiles,
     modifiedFiles,
     ...(previousSummary ? { previousSummary } : {}),

package/src/pi-adapter/compaction-coordinator.ts

@@ -14,7 +14,14 @@ import {
   highestClassification,
   type PrivacyClassification,
 } from "ds4-context-core/privacy/privacy-policy";
-import { prepareCompactionSource } from "./compaction-adapter.ts";
+import {
+  prepareCompactionSource,
+  sliceCompactionSource,
+  type PreparedCompactionSource,
+  type PreparedCompactionSourceSlice,
+} from "./compaction-adapter.ts";
+import { buildCompactionAtomicGroups } from "ds4-context-core/compaction/segmentation";
+import { estimateMessageTokens } from "ds4-context-core/core/token-estimator";
 import { adaptiveRecentTailLimit } from "ds4-context-core/planner/context-planner";
 import type { Logger } from "ds4-context-core/shared/logging";
 import {
@@ -42,6 +49,10 @@ import {
   type SummaryValidationStatus,
 } from "ds4-context-core/compaction/summary-contract";
 
+const MAX_SEGMENT_REQUESTS = 32;
+const MAX_AGGREGATE_REQUESTS = 64;
+const MAX_AGGREGATE_LEVELS = 16;
+
 export type CompactionPhase =
   | "idle"
   | "requested"
@@ -66,6 +77,10 @@ export interface CompactionDiagnostics {
   requestedAt?: number;
   completedAt?: number;
   lastError?: string;
+  inputBudgetTokens?: number;
+  sourcePromptTokens?: number;
+  segmentCount?: number;
+  aggregateCalls?: number;
   contextTokens?: number;
   softLimitTokens?: number;
   proactiveThresholdTokens?: number;
@@ -108,6 +123,26 @@ export interface SessionCompactFailedLike {
   fromExtension: boolean;
 }
 
+interface SegmentGenerationPlan {
+  source: PreparedCompactionSourceSlice;
+  prompt: string;
+  validationSource: string;
+  readFiles: string[];
+  modifiedFiles: string[];
+  classification: PrivacyClassification;
+  promptTokens: number;
+}
+
+interface AggregateGenerationPlan {
+  children: EmbeddedSummaryNode[];
+  prompt: string;
+  validationSource: string;
+  readFiles: string[];
+  modifiedFiles: string[];
+  classification: PrivacyClassification;
+  promptTokens: number;
+}
+
 interface CompactionCoordinatorDependencies {
   config: Ds4ContextConfig;
   database?: ContextDatabase;
@@ -176,7 +211,8 @@ export class CompactionCoordinator {
     ctx: ExtensionContext,
   ): Promise<{ compaction?: CompactionResult } | undefined> {
     const config = this.dependencies.config;
-    if (!config.compaction.enabled || !config.enabled || !ctx.model || config.context.maxSummaryTokens <= 0) {
+    const model = ctx.model;
+    if (!config.compaction.enabled || !config.enabled || !model || config.context.maxSummaryTokens <= 0) {
       return undefined;
     }
 
@@ -193,7 +229,24 @@ export class CompactionCoordinator {
     try {
       this.dependencies.syncSessionIndex(ctx);
       const source = prepareCompactionSource(event);
-      this.state.sourceEntries = source.sourceEntryIds.length;
+      const inputBudgetTokens = this.inputBudgetTokens(model);
+      if (inputBudgetTokens <= 0) throw new Error("Active model has no safe compaction input budget");
+      const wholePlan = this.buildSegmentPlan(source, event, model.provider);
+      this.state = {
+        ...this.state,
+        sourceEntries: source.sourceEntryIds.length,
+        inputBudgetTokens,
+        sourcePromptTokens: wholePlan.promptTokens,
+      };
+      const segmentPlans = this.partitionSegmentPlans(
+        source,
+        wholePlan,
+        event,
+        model.provider,
+        inputBudgetTokens,
+      );
+      this.state.segmentCount = segmentPlans.length;
+
       const usedIds = new Set(this.graphRecords.keys());
       if (source.previousNode) usedIds.add(source.previousNode.id);
       const nextId = (): string => {
@@ -208,129 +261,70 @@ export class CompactionCoordinator {
         reason: trigger,
         isSplitTurn: event.preparation.isSplitTurn,
         messageCount: source.messages.length,
-        provider: ctx.model.provider,
-        model: ctx.model.id,
-      };
-      const classifyContent = (text: string): { value: string; classification: PrivacyClassification } => {
-        const classified = this.dependencies.classifyContent?.(text, ctx.model!.provider);
-        if (classified) return classified;
-        return {
-          value: this.dependencies.sanitizeContent?.(text, ctx.model!.provider) ?? text,
-          classification: "normal",
-        };
-      };
-      const sourcePrivacy = classifyContent(source.conversationText);
-      const validationPrivacy = classifyContent(source.sourceText);
-      const instructionPrivacy = event.customInstructions
-        ? classifyContent(event.customInstructions)
-        : undefined;
-      const readFilePrivacy = source.segmentReadFiles.map(classifyContent);
-      const modifiedFilePrivacy = source.segmentModifiedFiles.map(classifyContent);
-      const segmentClassification = [
-        sourcePrivacy.classification,
-        validationPrivacy.classification,
-        ...(instructionPrivacy ? [instructionPrivacy.classification] : []),
-        ...readFilePrivacy.map((item) => item.classification),
-        ...modifiedFilePrivacy.map((item) => item.classification),
-      ].reduce(highestClassification, "normal" as PrivacyClassification);
-      const segmentGenerated = await this.generateSummary({
-        stage: "segment",
-        prompt: buildSummaryPrompt({
-          conversationText: sourcePrivacy.value,
-          ...(instructionPrivacy ? { customInstructions: instructionPrivacy.value } : {}),
-          readFiles: readFilePrivacy.map((item) => item.value),
-          modifiedFiles: modifiedFilePrivacy.map((item) => item.value),
-          isSplitTurn: event.preparation.isSplitTurn,
-        }),
-        validationSource: validationPrivacy.value,
-        readFiles: readFilePrivacy.map((item) => item.value),
-        modifiedFiles: modifiedFilePrivacy.map((item) => item.value),
-        event,
-        ctx,
-      });
-      const segmentId = nextId();
-      const segmentNode: EmbeddedSummaryNode = {
-        id: segmentId,
-        kind: "segment",
-        content: classifiedSummary(segmentGenerated.content, segmentClassification),
-        sourceHash: source.sourceHash,
-        sourceEntryIds: [...source.sourceEntryIds],
-        childSummaryIds: [],
-        graphLevel: 0,
-        createdAt: this.dependencies.now(),
-        validationStatus: segmentGenerated.validation.status,
-        validationIssueCodes: unique(segmentGenerated.validation.issues.map((issue) => issue.code)),
-        provider: ctx.model.provider,
-        model: ctx.model.id,
+        provider: model.provider,
+        model: model.id,
       };
 
-      const embeddedNodes: EmbeddedSummaryNode[] = [];
-      let previousNode = source.previousNode;
-      if (!previousNode && source.previousSummary) {
-        previousNode = importUntrackedPreviousSummary({
+      const usages: GeneratedSummary["usage"][] = [];
+      const segmentNodes: EmbeddedSummaryNode[] = [];
+      for (const plan of segmentPlans) {
+        const generated = await this.generateSummary({
+          stage: "segment",
+          prompt: plan.prompt,
+          validationSource: plan.validationSource,
+          readFiles: plan.readFiles,
+          modifiedFiles: plan.modifiedFiles,
+          event,
+          ctx,
+        });
+        usages.push(generated.usage);
+        segmentNodes.push({
           id: nextId(),
-          content: source.previousSummary,
+          kind: "segment",
+          content: classifiedSummary(generated.content, plan.classification),
+          sourceHash: plan.source.sourceHash,
+          sourceEntryIds: [...plan.source.sourceEntryIds],
+          childSummaryIds: [],
+          graphLevel: 0,
           createdAt: this.dependencies.now(),
-          provider: ctx.model.provider,
-          model: ctx.model.id,
+          validationStatus: generated.validation.status,
+          validationIssueCodes: unique(generated.validation.issues.map((issue) => issue.code)),
+          provider: model.provider,
+          model: model.id,
         });
-        embeddedNodes.push(previousNode);
       }
+      const segmentId = segmentNodes[0]?.id;
+      if (!segmentId) throw new Error("Compaction fan-out produced no segment summary node");
 
-      const usages = [segmentGenerated.usage];
-      let activeNode = segmentNode;
-      if (previousNode) {
-        const aggregateChildren = [previousNode, segmentNode];
-        const classifiedChildren = aggregateChildren.map((child) => ({
-          child,
-          privacy: classifyContent(child.content),
-        }));
-        const promptChildren = classifiedChildren.map(({ child, privacy }) => ({
-          ...child,
-          content: privacy.value,
-        }));
-        const aggregateReadFiles = source.readFiles.map(classifyContent);
-        const aggregateModifiedFiles = source.modifiedFiles.map(classifyContent);
-        const aggregateInstruction = event.customInstructions
-          ? classifyContent(event.customInstructions)
-          : undefined;
-        const aggregateClassification = [
-          ...classifiedChildren.map((item) => item.privacy.classification),
-          ...(aggregateInstruction ? [aggregateInstruction.classification] : []),
-          ...aggregateReadFiles.map((item) => item.classification),
-          ...aggregateModifiedFiles.map((item) => item.classification),
-        ].reduce(highestClassification, "normal" as PrivacyClassification);
-        const aggregateGenerated = await this.generateSummary({
-          stage: "aggregate",
-          prompt: buildAggregateSummaryPrompt({
-            children: promptChildren,
-            ...(aggregateInstruction ? { customInstructions: aggregateInstruction.value } : {}),
-            readFiles: aggregateReadFiles.map((item) => item.value),
-            modifiedFiles: aggregateModifiedFiles.map((item) => item.value),
-          }),
-          validationSource: promptChildren.map((child) => child.content).join("\n\n"),
-          readFiles: aggregateReadFiles.map((item) => item.value),
-          modifiedFiles: aggregateModifiedFiles.map((item) => item.value),
-          event,
-          ctx,
-        });
-        usages.push(aggregateGenerated.usage);
-        activeNode = {
+      const createdNodes: EmbeddedSummaryNode[] = [];
+      let previousNode = source.previousNode;
+      if (!previousNode && source.previousSummary) {
+        previousNode = importUntrackedPreviousSummary({
           id: nextId(),
-          kind: "aggregate",
-          content: classifiedSummary(aggregateGenerated.content, aggregateClassification),
-          sourceHash: computeAggregateSourceHash(aggregateChildren),
-          sourceEntryIds: unique(aggregateChildren.flatMap((child) => child.sourceEntryIds)),
-          childSummaryIds: aggregateChildren.map((child) => child.id),
-          graphLevel: Math.max(...aggregateChildren.map((child) => child.graphLevel)) + 1,
+          content: source.previousSummary,
           createdAt: this.dependencies.now(),
-          validationStatus: aggregateGenerated.validation.status,
-          validationIssueCodes: unique(aggregateGenerated.validation.issues.map((issue) => issue.code)),
-          provider: ctx.model.provider,
-          model: ctx.model.id,
-        };
-        embeddedNodes.push(segmentNode);
+          provider: model.provider,
+          model: model.id,
+        });
+        createdNodes.push(previousNode);
       }
+      createdNodes.push(...segmentNodes);
+      const roots = [...(previousNode ? [previousNode] : []), ...segmentNodes];
+      const aggregated = await this.aggregateSummaryNodes({
+        roots,
+        event,
+        ctx,
+        provider: model.provider,
+        model: model.id,
+        readFiles: source.readFiles,
+        modifiedFiles: source.modifiedFiles,
+        inputBudgetTokens,
+        nextId,
+        createdNodes,
+        usages,
+      });
+      const activeNode = aggregated.activeNode;
+      const embeddedNodes = createdNodes.filter((node) => node.id !== activeNode.id);
 
       const records = embeddedNodes.map((node) => createSummaryRecord({
         sessionId: this.dependencies.sessionId,
@@ -359,6 +353,10 @@ export class CompactionCoordinator {
         firstKeptEntryId: event.preparation.firstKeptEntryId,
         tokensBefore: event.preparation.tokensBefore,
         requestedAt,
+        inputBudgetTokens,
+        sourcePromptTokens: wholePlan.promptTokens,
+        segmentCount: segmentPlans.length,
+        aggregateCalls: aggregated.aggregateCalls,
       };
       this.dependencies.logger.debug("compaction.summary_graph_prepared", {
         activeSummaryId: activeNode.id,
@@ -367,6 +365,10 @@ export class CompactionCoordinator {
         createdNodes: records.length,
         sourceEntries: activeNode.sourceEntryIds.length,
         validationStatus: activeNode.validationStatus,
+        inputBudgetTokens,
+        sourcePromptTokens: wholePlan.promptTokens,
+        segmentCount: segmentPlans.length,
+        aggregateCalls: aggregated.aggregateCalls,
         trigger,
       });
       const activeRecord = records.at(-1);
@@ -420,13 +422,19 @@ export class CompactionCoordinator {
       : undefined;
     if (!details || !effectiveEntry) {
       this.failPendingNodes();
-      const customError = this.state.lastError;
+      const attempt = this.state;
+      const customError = attempt.lastError;
       this.state = {
         phase: "pi-default",
         trigger: event.reason,
         firstKeptEntryId: event.compactionEntry.firstKeptEntryId,
         tokensBefore: event.compactionEntry.tokensBefore,
         completedAt: this.dependencies.now(),
+        ...(attempt.sourceEntries !== undefined ? { sourceEntries: attempt.sourceEntries } : {}),
+        ...(attempt.inputBudgetTokens !== undefined ? { inputBudgetTokens: attempt.inputBudgetTokens } : {}),
+        ...(attempt.sourcePromptTokens !== undefined ? { sourcePromptTokens: attempt.sourcePromptTokens } : {}),
+        ...(attempt.segmentCount !== undefined ? { segmentCount: attempt.segmentCount } : {}),
+        ...(attempt.aggregateCalls !== undefined ? { aggregateCalls: attempt.aggregateCalls } : {}),
         ...(customError ? { lastError: `Custom fallback: ${customError}` } : {}),
       };
       return;
@@ -459,6 +467,14 @@ export class CompactionCoordinator {
       tokensBefore: effectiveEntry.tokensBefore,
       requestedAt: metadata.generatedAt,
       completedAt: this.dependencies.now(),
+      ...(this.state.inputBudgetTokens !== undefined
+        ? { inputBudgetTokens: this.state.inputBudgetTokens }
+        : {}),
+      ...(this.state.sourcePromptTokens !== undefined
+        ? { sourcePromptTokens: this.state.sourcePromptTokens }
+        : {}),
+      ...(this.state.segmentCount !== undefined ? { segmentCount: this.state.segmentCount } : {}),
+      ...(this.state.aggregateCalls !== undefined ? { aggregateCalls: this.state.aggregateCalls } : {}),
     };
     this.dependencies.logger.debug("compaction.summary_graph_committed", {
       activeSummaryId: metadata.summaryId,
@@ -640,6 +656,286 @@ export class CompactionCoordinator {
     };
   }
 
+  private inputBudgetTokens(model: ModelDescriptor): number {
+    const config = this.dependencies.config;
+    const resolved = this.dependencies.resolveModelBudget?.(model)
+      ?? {
+        budget: calculateContextBudget(createModelProfile(model), config.context),
+        recentTailTokens: adaptiveRecentTailLimit(
+          model.contextWindow,
+          config.context.recentTailTokens,
+        ),
+      };
+    return Math.floor(resolved.budget.activeInputBudget);
+  }
+
+  private classify(text: string, provider: string): {
+    value: string;
+    classification: PrivacyClassification;
+  } {
+    const classified = this.dependencies.classifyContent?.(text, provider);
+    if (classified) return classified;
+    return {
+      value: this.dependencies.sanitizeContent?.(text, provider) ?? text,
+      classification: "normal",
+    };
+  }
+
+  private promptTokens(prompt: string): number {
+    return estimateMessageTokens({
+      role: "user",
+      content: [{ type: "text", text: prompt }],
+    });
+  }
+
+  private buildSegmentPlan(
+    source: PreparedCompactionSourceSlice,
+    event: SessionBeforeCompactEvent,
+    provider: string,
+  ): SegmentGenerationPlan {
+    const sourcePrivacy = this.classify(source.conversationText, provider);
+    const validationPrivacy = this.classify(source.sourceText, provider);
+    const instructionPrivacy = event.customInstructions
+      ? this.classify(event.customInstructions, provider)
+      : undefined;
+    const readFilePrivacy = source.segmentReadFiles.map((file) => this.classify(file, provider));
+    const modifiedFilePrivacy = source.segmentModifiedFiles.map((file) => this.classify(file, provider));
+    const classification = [
+      sourcePrivacy.classification,
+      validationPrivacy.classification,
+      ...(instructionPrivacy ? [instructionPrivacy.classification] : []),
+      ...readFilePrivacy.map((item) => item.classification),
+      ...modifiedFilePrivacy.map((item) => item.classification),
+    ].reduce(highestClassification, "normal" as PrivacyClassification);
+    const readFiles = readFilePrivacy.map((item) => item.value);
+    const modifiedFiles = modifiedFilePrivacy.map((item) => item.value);
+    const prompt = buildSummaryPrompt({
+      conversationText: sourcePrivacy.value,
+      ...(instructionPrivacy ? { customInstructions: instructionPrivacy.value } : {}),
+      readFiles,
+      modifiedFiles,
+      isSplitTurn: source.isSplitTurn,
+    });
+    return {
+      source,
+      prompt,
+      validationSource: validationPrivacy.value,
+      readFiles,
+      modifiedFiles,
+      classification,
+      promptTokens: this.promptTokens(prompt),
+    };
+  }
+
+  private partitionSegmentPlans(
+    source: PreparedCompactionSource,
+    wholePlan: SegmentGenerationPlan,
+    event: SessionBeforeCompactEvent,
+    provider: string,
+    inputBudgetTokens: number,
+  ): SegmentGenerationPlan[] {
+    if (wholePlan.promptTokens <= inputBudgetTokens) return [wholePlan];
+
+    const groups = buildCompactionAtomicGroups(source.messages);
+    const plans: SegmentGenerationPlan[] = [];
+    let currentIndices: number[] = [];
+    let currentPlan: SegmentGenerationPlan | undefined;
+
+    for (const group of groups) {
+      const candidateIndices = [...currentIndices, ...group.messageIndices];
+      const candidatePlan = this.buildSegmentPlan(
+        sliceCompactionSource(source, candidateIndices),
+        event,
+        provider,
+      );
+      if (candidatePlan.promptTokens <= inputBudgetTokens) {
+        currentIndices = candidateIndices;
+        currentPlan = candidatePlan;
+        continue;
+      }
+
+      if (currentPlan) {
+        plans.push(currentPlan);
+        if (plans.length >= MAX_SEGMENT_REQUESTS) {
+          throw new Error(`Compaction fan-out exceeded the bounded segment request limit (${MAX_SEGMENT_REQUESTS})`);
+        }
+      }
+      const atomicPlan = this.buildSegmentPlan(
+        sliceCompactionSource(source, group.messageIndices),
+        event,
+        provider,
+      );
+      if (atomicPlan.promptTokens > inputBudgetTokens) {
+        throw new Error(
+          `Compaction source contains an indivisible atomic group above the model input budget (promptTokens=${atomicPlan.promptTokens}; inputBudgetTokens=${inputBudgetTokens})`,
+        );
+      }
+      currentIndices = [...group.messageIndices];
+      currentPlan = atomicPlan;
+    }
+
+    if (currentPlan) plans.push(currentPlan);
+    if (plans.length === 0) throw new Error("Compaction fan-out produced no source segments");
+    if (plans.length > MAX_SEGMENT_REQUESTS) {
+      throw new Error(`Compaction fan-out exceeded the bounded segment request limit (${MAX_SEGMENT_REQUESTS})`);
+    }
+    return plans;
+  }
+
+  private buildAggregatePlan(
+    children: readonly EmbeddedSummaryNode[],
+    event: SessionBeforeCompactEvent,
+    provider: string,
+    readFiles: readonly string[],
+    modifiedFiles: readonly string[],
+  ): AggregateGenerationPlan {
+    const classifiedChildren = children.map((child) => ({
+      child,
+      privacy: this.classify(child.content, provider),
+    }));
+    const promptChildren = classifiedChildren.map(({ child, privacy }) => ({
+      ...child,
+      content: privacy.value,
+    }));
+    const aggregateReadFiles = readFiles.map((file) => this.classify(file, provider));
+    const aggregateModifiedFiles = modifiedFiles.map((file) => this.classify(file, provider));
+    const aggregateInstruction = event.customInstructions
+      ? this.classify(event.customInstructions, provider)
+      : undefined;
+    const classification = [
+      ...classifiedChildren.map((item) => item.privacy.classification),
+      ...(aggregateInstruction ? [aggregateInstruction.classification] : []),
+      ...aggregateReadFiles.map((item) => item.classification),
+      ...aggregateModifiedFiles.map((item) => item.classification),
+    ].reduce(highestClassification, "normal" as PrivacyClassification);
+    const sanitizedReadFiles = aggregateReadFiles.map((item) => item.value);
+    const sanitizedModifiedFiles = aggregateModifiedFiles.map((item) => item.value);
+    const prompt = buildAggregateSummaryPrompt({
+      children: promptChildren,
+      ...(aggregateInstruction ? { customInstructions: aggregateInstruction.value } : {}),
+      readFiles: sanitizedReadFiles,
+      modifiedFiles: sanitizedModifiedFiles,
+    });
+    return {
+      children: [...children],
+      prompt,
+      validationSource: promptChildren.map((child) => child.content).join("\n\n"),
+      readFiles: sanitizedReadFiles,
+      modifiedFiles: sanitizedModifiedFiles,
+      classification,
+      promptTokens: this.promptTokens(prompt),
+    };
+  }
+
+  private async aggregateSummaryNodes(input: {
+    roots: EmbeddedSummaryNode[];
+    event: SessionBeforeCompactEvent;
+    ctx: ExtensionContext;
+    provider: string;
+    model: string;
+    readFiles: readonly string[];
+    modifiedFiles: readonly string[];
+    inputBudgetTokens: number;
+    nextId: () => string;
+    createdNodes: EmbeddedSummaryNode[];
+    usages: GeneratedSummary["usage"][];
+  }): Promise<{ activeNode: EmbeddedSummaryNode; aggregateCalls: number }> {
+    let layer = [...input.roots];
+    let aggregateCalls = 0;
+    let graphPasses = 0;
+
+    while (layer.length > 1) {
+      graphPasses++;
+      if (graphPasses > MAX_AGGREGATE_LEVELS) {
+        throw new Error(`Compaction aggregation exceeded the bounded graph depth (${MAX_AGGREGATE_LEVELS})`);
+      }
+      const batches: EmbeddedSummaryNode[][] = [];
+      let current: EmbeddedSummaryNode[] = [];
+      for (const node of layer) {
+        if (current.length === 0) {
+          current = [node];
+          continue;
+        }
+        const candidate = [...current, node];
+        const candidatePlan = this.buildAggregatePlan(
+          candidate,
+          input.event,
+          input.provider,
+          input.readFiles,
+          input.modifiedFiles,
+        );
+        if (candidatePlan.promptTokens <= input.inputBudgetTokens) {
+          current = candidate;
+          continue;
+        }
+        if (current.length === 1) {
+          throw new Error(
+            `Compaction child summaries cannot be aggregated within the model input budget (inputBudgetTokens=${input.inputBudgetTokens})`,
+          );
+        }
+        batches.push(current);
+        current = [node];
+      }
+      if (current.length > 0) batches.push(current);
+
+      const nextLayer: EmbeddedSummaryNode[] = [];
+      for (const batch of batches) {
+        if (batch.length === 1) {
+          nextLayer.push(batch[0] as EmbeddedSummaryNode);
+          continue;
+        }
+        aggregateCalls++;
+        if (aggregateCalls > MAX_AGGREGATE_REQUESTS) {
+          throw new Error(`Compaction fan-in exceeded the bounded aggregate request limit (${MAX_AGGREGATE_REQUESTS})`);
+        }
+        const plan = this.buildAggregatePlan(
+          batch,
+          input.event,
+          input.provider,
+          input.readFiles,
+          input.modifiedFiles,
+        );
+        if (plan.promptTokens > input.inputBudgetTokens) {
+          throw new Error("Compaction aggregate prompt exceeded its preflight input budget");
+        }
+        const generated = await this.generateSummary({
+          stage: "aggregate",
+          prompt: plan.prompt,
+          validationSource: plan.validationSource,
+          readFiles: plan.readFiles,
+          modifiedFiles: plan.modifiedFiles,
+          event: input.event,
+          ctx: input.ctx,
+        });
+        input.usages.push(generated.usage);
+        const aggregateNode: EmbeddedSummaryNode = {
+          id: input.nextId(),
+          kind: "aggregate",
+          content: classifiedSummary(generated.content, plan.classification),
+          sourceHash: computeAggregateSourceHash(batch),
+          sourceEntryIds: unique(batch.flatMap((child) => child.sourceEntryIds)),
+          childSummaryIds: batch.map((child) => child.id),
+          graphLevel: Math.max(...batch.map((child) => child.graphLevel)) + 1,
+          createdAt: this.dependencies.now(),
+          validationStatus: generated.validation.status,
+          validationIssueCodes: unique(generated.validation.issues.map((issue) => issue.code)),
+          provider: input.provider,
+          model: input.model,
+        };
+        input.createdNodes.push(aggregateNode);
+        nextLayer.push(aggregateNode);
+      }
+      if (nextLayer.length >= layer.length) {
+        throw new Error("Compaction aggregation could not reduce the summary graph within the input budget");
+      }
+      layer = nextLayer;
+    }
+
+    const activeNode = layer[0];
+    if (!activeNode) throw new Error("Compaction aggregation produced no active summary node");
+    return { activeNode, aggregateCalls };
+  }
+
   private generateSummary(input: {
     stage: "segment" | "aggregate";
     prompt: string;

package/src/pi-adapter/summary-generator.ts

@@ -50,33 +50,65 @@ function responseStopReason(response: unknown): string | undefined {
   return typeof response.stopReason === "string" ? response.stopReason : undefined;
 }
 
+function responseErrorMessage(response: unknown): string | undefined {
+  if (!response || typeof response !== "object" || !("errorMessage" in response)) return undefined;
+  return typeof response.errorMessage === "string" ? response.errorMessage : undefined;
+}
+
+function providerFailureCategory(value: unknown): string {
+  const message = value instanceof Error
+    ? value.message
+    : typeof value === "string"
+      ? value
+      : "";
+  if (/abort|cancel/iu.test(message)) return "aborted";
+  if (/usage|quota|credit|billing/iu.test(message)) return "usage-limit";
+  if (/rate|too many requests|429/iu.test(message)) return "rate-limit";
+  if (/(?:context|prompt|input).{0,48}(?:exceed|limit|maximum|too (?:long|large)|tokens?)|tokens?.{0,48}(?:exceed|limit|maximum|too many)|maximum (?:context|input|prompt|length)/iu.test(message)) {
+    return "input-limit";
+  }
+  if (/auth|credential|api.?key|permission|forbidden|401|403/iu.test(message)) return "authentication";
+  if (/timeout|network|connection|socket|dns/iu.test(message)) return "transport";
+  return "provider-error";
+}
+
 export async function generateValidatedSummary(
   input: GenerateValidatedSummaryInput,
 ): Promise<GeneratedSummary> {
   const model = input.ctx.model;
   if (!model) throw new Error("Compaction summary generation requires an active model");
   const maxTokens = Math.max(1, Math.min(input.maxSummaryTokens, model.maxTokens ?? input.maxSummaryTokens));
-  const response = await input.ctx.modelRegistry.complete(
-    model,
-    {
-      messages: [{
-        role: "user",
-        content: [{ type: "text", text: input.prompt }],
-        timestamp: input.now(),
-      }],
-    },
-    {
-      maxTokens,
-      signal: input.event.signal,
-      cacheRetention: "none",
-      sessionId: randomUUID(),
-    },
-  );
+  let response: Awaited<ReturnType<typeof input.ctx.modelRegistry.complete>>;
+  try {
+    response = await input.ctx.modelRegistry.complete(
+      model,
+      {
+        messages: [{
+          role: "user",
+          content: [{ type: "text", text: input.prompt }],
+          timestamp: input.now(),
+        }],
+      },
+      {
+        maxTokens,
+        signal: input.event.signal,
+        cacheRetention: "none",
+        sessionId: randomUUID(),
+      },
+    );
+  } catch (error) {
+    throw new Error(
+      `Compaction ${input.stage} request failed (category=${providerFailureCategory(error)})`,
+    );
+  }
   if (input.event.signal.aborted) throw new Error("Compaction summary generation aborted");
   const stopReason = responseStopReason(response);
   if (stopReason === "length") throw new Error("Compaction summary hit the model output limit");
   if (stopReason === "error" || stopReason === "aborted") {
-    throw new Error(`Compaction summary stopped with ${stopReason}`);
+    const category = stopReason === "aborted"
+      ? "aborted"
+      : providerFailureCategory(responseErrorMessage(response));
+    throw new Error(`Compaction ${input.stage} summary stopped with ${stopReason} (category=${category})`);
   }
   if (response.content.some((block) => block.type === "toolCall")) {
     throw new Error("Compaction summarizer attempted to call a tool");

package/src/pi-adapter/version.ts

@@ -1,4 +1,4 @@
-export const EXTENSION_VERSION = "0.3.0-alpha.4";
+export const EXTENSION_VERSION = "0.3.0-alpha.5";
 export const SUPPORTED_PI_VERSION = "0.84.3";
 export const OBSERVER_PLANNER_VERSION = "observer-model-aware-v1";
 export const PLANNER_VERSION = "managed-learned-ranking-v1";