ds4-context-engine 0.3.5 → 0.3.7

package/README.md

@@ -264,6 +264,7 @@ The following example shows the main configuration groups. Omitted values use th
     "minimumOutputReserve": 8192,
     "preferredOutputReserve": 32768,
     "recentTailTokens": 64000,
+    "rescueImmediatePredecessor": true,
     "maxPinnedTokens": 16000,
     "maxMemoryTokens": 8000,
     "maxRetrievedHistoryTokens": 16000,

package/docs/ADR/062-cache-aware-context-planning.md

@@ -0,0 +1,93 @@
+# ADR-062 — Cache-aware context planning
+
+Status: accepted and implemented in the coordinated 0.3.7 release (opt-in, default off; see the [release record](../releases/0.3.7.md)).
+
+## Context
+
+DS4 reduces provider input by selecting a bounded managed context. This is
+economically sound when every input token costs the same or when the provider
+offers no prompt cache. Providers with a large context window and a large
+cache-miss/cache-hit price ratio (for example DeepSeek V4 Flash, roughly 31x
+off-peak) invert the trade-off: a small but frequently re-planned prompt can
+cost more than a larger stable one, because every re-plan invalidates the
+provider prefix cache.
+
+The 0.3.6 planner applies an automatic 64k recent-tail ceiling to all models
+above 256k until the conversational tail exceeds the cap. When the cap is
+exceeded, the oldest turn leaves and the shared prefix with the previous
+request collapses: everything after the first divergence becomes a cache miss.
+Whether this is more expensive than a wider tail depends on the workload
+(requests per turn, turns per epoch, share of cached reads observed).
+
+The runtime already records cache read/write shares per exact `provider/model`
+(volatile calibration plus persisted manifests) and Pi exposes per-million
+cost rates on the active model. The portable core never hardcodes prices.
+
+## Decision
+
+- Add an optional economic cost profile to the portable `ModelDescriptor`
+  (`input`, `output`, `cacheRead`, `cacheWrite` per million), populated from
+  Pi model metadata in `snapshotModel`. Absent fields degrade cache-aware
+  planning to the previous behavior.
+- Add `context.cacheAware` (default `off`), a policy object with:
+  - `mode`: `off` preserves the previous planner exactly; `auto` may extend
+    the recent tail when pricing and observed cache shares justify it;
+  - `minimumCacheSampleCount`, `minimumCacheReadShare`,
+    `minimumMissHitRatio`: hard gates before any extension is eligible;
+  - `minimumImprovementRatio`: relative cost improvement required before an
+    alternative plan is adopted (hysteresis);
+  - `maxTailBudgetShare`: the extended tail may never exceed this fraction of
+    the active input budget;
+  - `expectedRequestsPerTurn`, `expectedTurnsPerEpoch`, `stickinessEpochs`:
+    deterministic cost model horizon plus hysteresis before dropping an
+    adopted extended tail.
+- In `auto` the runtime computes `context.cacheAware` decision and compares
+  two plan candidates (nominal tail vs extended tail) with a deterministic
+  epoch cost model:
+  - stable plans (original conversation fits their tail cap) pay one cold
+    transition per epoch and warm requests afterwards;
+  - sliding plans (conversation exceeds their cap) pay a cold request every
+    turn of the epoch.
+  The extended plan is adopted only when it is cheaper over the whole epoch,
+  after the minimum improvement margin.
+- The planner accepts an optional `cacheAwareTailTokens` override that bypasses
+  the automatic context-window ceiling; it remains bounded by the active/hard
+  input budgets and atomic groups, so no hard limit, privacy, pin, current
+  request or atomicity guarantee is weakened.
+- Manifest `planning.cacheAware` (optional, numbers only, never content)
+  records the decision, tail, miss/hit ratio, observed cache share, sample
+  count, estimated reusable prefix tokens, estimated request cost and the
+  winning candidate. `/context tokens` and `/context explain` surface the same
+  metadata. `context.excluded_oversized_turn` and previous guarantees are
+  unchanged.
+- Add a deterministic synthetic prefix-cache simulator covering the report
+  scenarios (append-only under the cap, sliding past 64k, one response per
+  prompt, five tool cycles, wide vs sliding tails, model switch, pricing
+  comparison). No provider, no credentials, no CI network access.
+
+## Rejected alternatives
+
+- Hardcoding provider prices: rejected, the core stays provider-independent.
+- Raising the default tail ceiling globally: rejected, it changes behavior
+  for all models (including ones without cache discounts).
+- Always extending the tail for eligible models: rejected, the epoch model
+  shows the nominal plan can still be cheaper on workloads where the tail
+  slides infrequently; the improvement margin keeps the switch conservative.
+
+## Compatibility and validation
+
+Configuration is additive; `context.cacheAware.mode` defaults to `off`, so
+`0.3.6` behavior (and manifest absence of the cacheAware block) is preserved.
+Automatic behavior is opt-in and requires observed samples plus positive
+pricing. The epoch cost is an estimate used only for the comparison; the
+actual billing remains whatever the provider reports. The workaround override
+(`modelAwareness.overrides` with a large `recentTailTokens`, zeroed retrieval
+and disabled compaction) remains available and is unaffected; it is more
+aggressive than `mode: "auto"` because it also removes retrieval/project
+supplements and compaction, which `auto` deliberately preserves for quality.
+
+Coverage: cache-policy unit tests (prefix, costs, decision gates), config
+validation, planner override tests, the synthetic prefix-cache simulator, and
+runtime integration tests for off/auto/no-discount/under-budget paths. A
+real-provider DeepSeek A/B benchmark remains out of scope and voluntary
+(protocol: [CACHE_AWARE_BENCHMARK.md](../CACHE_AWARE_BENCHMARK.md)).

package/docs/ADR/README.md

@@ -65,5 +65,6 @@ The initial decisions from the development plan are accepted:
 | [059](059-optional-anchored-editing.md) | Opt in to anchored edit expansion inside Pi's native mutation queue | Accepted |
 | [060](060-optional-portable-agent-tools.md) | Opt in to edit reports, adaptive reads/results and session-owned local jobs | Accepted |
 | [061](061-compaction-latency.md) | Bound compaction update calls, input budgets, concurrent segments and phase timings | Accepted |
+| [062](062-cache-aware-context-planning.md) | Opt-in cache-aware tail planning using model pricing and observed cache shares | Accepted |
 
 Each decision will receive a dedicated record when implementation pressure introduces alternatives or consequences not already covered by the development plan.

package/docs/CACHE_AWARE_BENCHMARK.md

@@ -0,0 +1,66 @@
+# Benchmark A/B DeepSeek — cache-aware planning
+
+Protocollo volontario, fuori CI, nessuna credenziale nel repo. Serve a decidere se
+`context.cacheAware.mode` può passare da `auto` (opt-in) a **default** in una
+release futura. Gate dichiarato in [ADR-062](ADR/062-cache-aware-context-planning.md).
+
+## Obiettivo
+
+Misurare, sulla **stessa** conversazione reale, il costo provider per turno con:
+
+- **Sessione A (baseline)**: comportamento 0.3.6 — `context.cacheAware` assente
+  (default `off`), tail automatica 64k, retrieval 16k, project 20k.
+- **Sessione B (auto)**: `context.cacheAware.mode: "auto"` con i default di
+  policy (gates 0.5 cache-share, ratio 20, margine 10%).
+- **Sessione C (workaround estremo, opzionale)**: `modelAwareness.overrides` con
+  `recentTailTokens: 500000`, `maxRetrievedHistoryTokens: 0`,
+  `maxProjectTokens: 0`, `compaction.enabled: false` — cache-first puro.
+
+Le tre sessioni devono avere la **stessa sequenza di turni** (stesso carico di
+lavoro: tool call, retrieval, project touches) e lo **stesso modello/provided
+ID DeepSeek**.
+
+## Cosa registrare per ogni turno
+
+Da `/context tokens` e `/context explain`:
+
+- `recentTailTokens` scelti (nominal vs extended) e `cacheAware` decision/reasons;
+- manifest `planning.cacheAware` (missHitRatio, cacheReadShare, sampleCount,
+  reusablePrefixTokens, estimatedCost, candidate);
+- `ProviderCacheMetrics` reali dopo ogni richiesta: `inputTokens`,
+  `cacheReadTokens`, `cacheWriteTokens`, share read/write per provider/model.
+
+Dal provider (costo effettivo):
+
+- input non-cached, cache-read (hit), cache-write, output per richiesta;
+- costo totale della sessione (per-turno e totale).
+
+## Metriche
+
+1. **Costo per turno** (media e totale sessione): A vs B vs C.
+2. **Cache-read share osservata** per config: conferma che la tail estesa
+   mantiene il prefisso stabile (share alta) vs sliding (share ~0 dopo 64k).
+3. **Qualità** (non solo $$$): recall retrieval, `Planner exclusions`,
+   current request/atomicità intatte, completamenti corretti nelle fasi tool.
+   Con retrieval azzerato (C) documentare i regressi di qualità.
+4. **Eventi di transizione**: quante volte il prefisso si è invalidato (turni
+   con cacheRead ≈ 0), confronto sliding vs stable.
+
+## Criterio di promozione a default
+
+Promuovere `mode: "auto"` a default solo se, su ≥ 3 sessioni reali:
+
+- costo medio per turno di B < A (margine ≥ il `minimumImprovementRatio`
+  configurato, di default 10%);
+- nessun regresso di qualità misurabile (recall retrieval uguale o migliore;
+  current request/atomicità preservate);
+- nessuna oscillazione plan (decisione che alterna nominal/esteso senza
+  motivo: verificare `stickinessEpochs` e diagnostica).
+
+## Esecuzione sicura
+
+- Nessuna credenziale nel repo; il benchmark usa la sessione Pi normale con il
+  provider DeepSeek già autenticato.
+- Non modificare la config attiva in modo permanente: copie di configurazione
+  per sessione, o valori temporanei poi ripristinati.
+- Fuori CI: niente rete/credenziali nel test suite.

package/docs/CONTEXT_MANIFEST.md

@@ -20,6 +20,7 @@ A Context Manifest explains the context visible at DS4's Pi `context` hook witho
 - artifact IDs, SHA-256, bytes, MIME, classification, exact source entry/tool IDs, error state, and before/after token estimates;
 - provider destination and allow-set names, selected classification counts, blocked/excluded/redacted counts, final provider-check count, and enforcement stage;
 - planner mode/version, original and selected counts, group counts, internal budgets, duration, and fallback reason;
+- optional cache-aware planning decision: eligibility, tail extension, requested tail tokens, miss/hit price ratio, observed cache-read share, sample count, estimated reusable prefix tokens, estimated request cost in dollars, and winning candidate (`nominal`/`cache-aware`); numbers only, never content;
 - learned-ranking mode/status, feature/model versions, candidate count, aggregate disagreement/rank shift, duration, and generic static-fallback reason;
 - planner and policy versions;
 - deterministic SHA-256 over system prompt, active tools, and messages;
@@ -54,6 +55,8 @@ A manifest at or below 256 KiB is stored unchanged. Above that preferred bound,
 
 Each manifest transaction prunes at most 32 excess rows and 8 MiB of serialized payload; one individually oversized oldest row may exceed the byte limit to guarantee progress. Calibration pruning is independently limited to 32 rows per related profile write. This incrementally repairs an existing oversized database without adding a long startup write or extending SQLite lock duration with an unbounded purge. Deleted pages become reusable by SQLite; the database file may remain at its previous high-water size until explicit [offline maintenance](STORAGE_MAINTENANCE.md). No retention action edits canonical Pi JSONL or project files.
 
+The `save()` result carries the derived `inventory` of the persisted projection, and the runtime exposes it through `RuntimeDiagnostics.persistedInventory`; `/context manifest` and `/context excluded` therefore report the truthful persisted completeness (`complete` or `excluded-rollup` with retained/total counts) instead of defaulting to `complete` when the in-memory manifest has no inventory attached.
+
 ## Reproducibility
 
 Object keys are normalized before hashing, so equivalent tool schemas with different key insertion order produce the same prompt hash. The estimator version is stored explicitly as `chars-v1`; planner/policy versions describe selection behavior. Golden tests protect manifest shape, model-profile resolution, token accounting, and hash stability.

package/docs/CONTEXT_PLANNER.md

@@ -12,7 +12,7 @@ The managed planner is synchronous, deterministic, provider-independent, and doe
 6. Merge groups linked by assistant tool calls and every matching tool result.
 7. Select the current request, labelled pin groups, and applicable allowed persistent pins as mandatory.
 8. Enforce `maxPinnedTokens`, then fit relevant allowed durable memory under `maxMemoryTokens`.
-9. Walk older turns newest-first, stopping at the first group that would break the contiguous recent tail, target, or hard limit.
+9. Walk older turns newest-first, stopping at the first group that would break the contiguous recent tail, target, or hard limit. When the immediate-predecessor turn itself exceeds the recent-tail cap but still fits the target and hard budgets, it is kept verbatim (`context.rescueImmediatePredecessor`, default true) and the tail then closes.
 10. Privacy-filter and fit source-labelled historical retrieval groups under `maxRetrievedHistoryTokens`.
 11. Privacy-filter and fit hash-current project snippets under `maxProjectTokens`.
 12. Fit active allowed Pi compaction/branch summaries in the remaining summary and input budgets.
@@ -41,6 +41,8 @@ Artifact condensation occurs before atomic grouping. It preserves `toolCallId`,
 
 The retrieval engine produces independent synthetic user-role evidence groups. They are never mandatory: recent turns have priority 100, durable memory 90, retrieved history 85, project snippets 80, and active summaries 75. Each group is selected or excluded whole, carries its original Pi entry ID, and is represented as `retrieval` in the Context Manifest. If planner validation falls back, every synthetic evidence message is discarded and Pi receives its original `AgentMessage[]` unchanged.
 
+Retrieval deduplicates against the entries the managed plan actually commits, not against Pi's native context: before retrieving, the runtime plans the context with mandatory supplements only, maps the committed messages back to Pi session entry IDs, and passes those IDs as the retrieval exclusion set. An entry that Pi still exposes but the planner excludes (for example a turn larger than the recent-tail cap) therefore remains retrievable and reappears as bounded `retrieval` evidence instead of being silently lost. The Context Manifest planning block records `rescuedImmediatePredecessor` and `oversizedTurnExclusions`; an oversized exclusion also emits a `context.excluded_oversized_turn` warning, and `/context explain` surfaces both counters.
+
 Evidence text is a JSON-quoted historical excerpt with an explicit data-only boundary. It is inserted immediately before the latest real user request, so the current task remains the final message and provider conversation order stays deterministic.
 
 ## Project snippets
@@ -84,6 +86,42 @@ M14 can queue the finalized manifest after planning when `quality.enabled` is tr
 
 M18 can evaluate bounded metadata-only features after privacy exclusion and before supplemental candidates enter category fitting. `shadow` keeps every static score/order authoritative and records aggregate disagreement only. `active` is accepted only for a compatible checksummed model carrying an eligible held-out promotion report. Privacy exclusions, mandatory pins/current turns, atomic groups and hard budgets cannot be overridden. See [`LEARNED_RANKING.md`](LEARNED_RANKING.md).
 
+## Cache-aware tail planning (0.3.7, opt-in)
+
+With `context.cacheAware.mode = "auto"` the runtime may extend the recent tail
+beyond the automatic context-window ceiling when the model pricing and the
+observed cache shares justify it economically. The policy never hardcodes
+prices: it reads the per-million rates exposed on the active Pi model and the
+cache read/write shares already recorded for the exact `provider/model`.
+
+Eligibility gates (defaults shown):
+
+- `minimumCacheSampleCount: 3` observed calibration samples;
+- `minimumCacheReadShare: 0.5` observed share;
+- `minimumMissHitRatio: 20` cache-miss / cache-hit price ratio.
+
+When eligible, the runtime compares two plan candidates with a deterministic
+epoch cost model:
+
+- the nominal plan (existing tail, sliding below 64k) pays a cold request on
+  every turn of the epoch because its prefix is invalidated by the slide;
+- the extended plan (up to `maxTailBudgetShare` of the active input budget) is
+  stable when the conversation fits, so it pays one cold transition per epoch
+  and warm requests afterwards.
+
+The extended plan is adopted only when it wins over the whole epoch
+(`expectedRequestsPerTurn`, `expectedTurnsPerEpoch`) after the
+`minimumImprovementRatio` margin. Once adopted it is kept (deliberate epoch)
+until it loses `stickinessEpochs` consecutive comparisons, preventing
+oscillation. The override remains bounded by the active and hard input
+budgets and by atomic groups, so current request, pins, privacy and atomicity
+guarantees are unchanged.
+
+`context.cacheAware.mode` defaults to `off`, preserving the 0.3.6 behavior
+exactly. Without prices, samples or a cache discount, the decision degrades to
+the nominal plan automatically. See [ADR-062](ADR/062-cache-aware-context-planning.md)
+for the model and rejected alternatives.
+
 ## Current limits
 
 The planner does not call a model inside the `context` hook. Model calibration uses only finalized provider usage and deterministic local statistics. Historical/project retrieval can opt into derived semantic candidates; learned supplemental reranking remains off by default and active mode is promotion-gated. Project symbol extraction is heuristic, artifact search is literal, and memory/pin creation is manual-first. Automatic memory extraction remains disabled; M10 supplies policy enforcement but not an automatic classifier or confirmation workflow. Provider-payload coverage targets Pi 0.84.3's supported serializers, and DS4 must load after any extension allowed to replace payloads when strict final ordering is required.

package/docs/releases/0.3.5.md

@@ -1,6 +1,6 @@
 # DS4 Context Engine 0.3.5
 
-Status: local release gates passed; commit, push and coordinated publication authorized. Validation-only CI, publication and registry verification are pending.
+Status: published as stable on npm under `latest`; annotated tag `v0.3.5` points to validated release commit `8b1d5f4`.
 
 ## Added since 0.3.4
 
@@ -65,6 +65,22 @@ Candidate validation on Node.js `26.5.1`, from a sanitized release source with a
 
 Pre-existing local `allowScripts` additions and `.serena/` are excluded from release commits and public packages.
 
+The final clean committed worktree repeated fresh `npm ci`, all **508 tests**, clean-consumer package verification and tarball inspection successfully. Validation-only CI run [`33970061602`](https://github.com/Alucard24/ds4-context-engine/actions/runs/33970061602) on `8b1d5f4` passed both Node `22.19.0` and `24.x`, including the full suite and package checks.
+
 ## Registry verification and release
 
-Pending successful local gates, validation-only CI, manual publication and exact registry verification. No release tag has been created yet.
+Published manually as `alucard_24`, in dependency order, from reviewed tarballs built in the clean committed worktree:
+
+- `ds4-context-core@0.3.5`: shasum `71105c666a8c88f1abfe5ba3efefe63fc32bba74`;
+- `ds4-context-reference-adapter@0.3.5`: shasum `8eca29472ea4721606d947818eba0aa53a6c6f63`;
+- `ds4-context-engine@0.3.5`: shasum `304b61c21e28f89e82edbee39880210c2c9dc868`.
+
+`npm run registry:check -- 0.3.5` passed: fresh exact-version installation, matching adapter/core dependencies, public core exports, compiled reference conformance, packaged quality corpus, storage CLI usage probe and isolated offline Pi extension startup. Registry SHA-1 and SHA-512 integrity values match the local tarballs for all three packages.
+
+The immediate first post-publication install returned `ETARGET` for the engine. Once exact-version registry metadata was available, verification was repeated with npm `prefer-online` and passed; no package was republished or tag moved.
+
+Verified dist-tags for all three: `latest=0.3.5`, `beta=0.3.0-beta.3`, `alpha=0.3.0-alpha.5`, `rc=0.2.0-rc.1`.
+
+Annotated tag `v0.3.5` targets `8b1d5f4`, the tested/published source. This post-publication evidence update is documentation-only.
+
+GitHub Release: https://github.com/Alucard24/ds4-context-engine/releases/tag/v0.3.5

package/docs/releases/0.3.6.md

@@ -0,0 +1,53 @@
+# DS4 Context Engine 0.3.6
+
+Status: published release on 2026-09-05; tag `v0.3.6`.
+
+This coordinated release fixes a managed-context selection defect: a single oversized atomic conversation turn could cause the planner to discard turns that are still essential to the current request, and its own retrieval second pass could not recover them.
+
+## Fixed
+
+- **Immediate-predecessor rescue**: when the turn immediately preceding the current request exceeds the recent-tail cap but still fits the active target and hard budgets, the planner keeps it verbatim instead of closing the entire recent tail at that group. The tail closes after the rescue, so older turns keep their prior bounded behavior. Controlled by `context.rescueImmediatePredecessor` (default `true`).
+- **Retrieval second chance**: retrieval now deduplicates against the entries the managed plan actually commits, not against Pi's native context. Before the retrieval pass, the runtime plans the context with mandatory supplements only, maps the committed messages back to Pi session entry IDs, and passes those IDs as the retrieval exclusion set. Entries the planner excludes (for example an oversized turn) can therefore reappear as bounded, source-labelled `retrieval` evidence instead of being silently lost.
+- **Diagnostics**: a `context.excluded_oversized_turn` warning reports how many turn groups at or above the recent-tail cap were excluded, whether the immediate predecessor was rescued, and the active limits. The Context Manifest planning block records optional `rescuedImmediatePredecessor` and `oversizedTurnExclusions` counters, and `/context explain` surfaces both.
+
+No validation, privacy, provenance, cancellation, fallback, canonical-history, or bounded-retention behavior was weakened. The runtime retry policy, compaction pipeline, and storage formats are unchanged.
+
+## Compatibility and persistence
+
+The context manifest adds only optional metadata fields. `ds4-context-config-v1`, `runtime-adapter-v1`, `ds4-context-persistence-tool-v1`, `ds4-context-persistence-result-v1`, SQLite schema 15, and migration checksums are unchanged.
+
+## Package/version policy
+
+The coordinated version is `0.3.6` for:
+
+```text
+ds4-context-core
+ds4-context-reference-adapter
+ds4-context-engine
+```
+
+Both adapters depend exactly on `ds4-context-core@0.3.6`. The packages were published manually under npm `latest`. GitHub Actions remains validation-only with OIDC and package-write permissions denied.
+
+## Validation evidence
+
+Local candidate verification on Node.js `26.5.1`:
+
+- `npm run check`: 81 files and 518 tests passed (including planner rescue cases, second-chance retrieval integration, and entry-id mapping).
+- `npm run quality:compare`: candidate quality `0.9875` versus baseline `0.808156`.
+- `npm run schema:context-persistence`: 1,266 bytes and 317 estimated tokens; below the 1,500 absolute and 320 relative limits.
+- `npm run latency:check -- <exact ds4-context-core@0.1.2>`: passed with ratio `1.082064`, at or below `1.10`.
+- `npm run pack:check`: verified core (235 files), reference adapter (7 files), and Pi adapter (87 files) in a clean consumer.
+- `npm pack --dry-run --json` for all three packages: passed with the same bounded inventories.
+- `git diff --check`: passed.
+- Protected CI, compatibility golden, Pi fixture, migration, canonical Pin/Memory, and persistence-tool contract files: unchanged except the intentional `compatibility-0.2.0.json` default addition.
+
+- `npm run registry:check -- 0.3.6`: passed against all three exact published versions; `latest` resolves to `0.3.6` for every package.
+
+Exact registry verification passed before the annotated tag and GitHub release were created.
+
+## Documentation
+
+- [`../CONTEXT_PLANNER.md`](../CONTEXT_PLANNER.md)
+- [`../COMPACTION.md`](../COMPACTION.md)
+- [`../RELEASING.md`](../RELEASING.md)
+- [`0.3.5.md`](0.3.5.md)

package/docs/releases/0.3.7.md

@@ -0,0 +1,83 @@
+# Release 0.3.7 — Cache-aware context planning (opt-in)
+
+**Version analyzed:** DS4 Context Engine `0.3.7`
+**Commit:** `a03009a4b14d6ab1842fa800076c4b300bd70d8f`
+**Coordinated packages:** `ds4-context-core` 0.3.7, `ds4-context-reference-adapter` 0.3.7, `ds4-context-engine` 0.3.7
+
+## Summary
+
+Adds an opt-in cache-aware planning policy driven by model pricing and
+observed cache shares, plus a deterministic synthetic prefix-cache simulator.
+The default behavior is unchanged: `context.cacheAware.mode` defaults to `off`,
+the manifest does not include a `cacheAware` block, and the planner uses the
+same tail caps as 0.3.6.
+
+## New configuration
+
+`context.cacheAware` (object, default off):
+
+| Field | Default | Meaning |
+|---|---|---|
+| `mode` | `off` | `off` preserves 0.3.6 behavior; `auto` may extend the recent tail. |
+| `minimumCacheSampleCount` | `3` | Observed samples required before acting. |
+| `minimumCacheReadShare` | `0.5` | Observed cache-read share required before acting. |
+| `minimumMissHitRatio` | `20` | Minimum cache-miss / cache-hit price ratio. |
+| `minimumImprovementRatio` | `0.1` | Relative improvement required to switch plan. |
+| `maxTailBudgetShare` | `0.5` | Max fraction of the active input budget for the extended tail. |
+| `expectedRequestsPerTurn` | `4` | Provider requests per user turn in the cost model. |
+| `expectedTurnsPerEpoch` | `4` | User turns per planning epoch in the cost model. |
+| `stickinessEpochs` | `2` | Consecutive epoch losses before dropping an adopted extended tail. |
+
+## Changes
+
+- `ModelDescriptor` gains an optional `cost` profile (`input`, `output`,
+  `cacheRead`, `cacheWrite` per million), populated from Pi model metadata in
+  `snapshotModel`. No prices are hardcoded in the core.
+- `planManagedContext` accepts an optional `cacheAwareTailTokens` override that
+  bypasses the automatic context-window ceiling while remaining bounded by the
+  active/hard input budgets and atomic groups.
+- Runtime: when `mode: "auto"` and the eligibility gates pass, the runtime
+  compares the nominal plan with an extended-tail plan using a deterministic
+  epoch cost model (stable plans pay one cold transition per epoch; sliding
+  plans pay a cold request per turn) and adopts the extended plan only when it
+  wins after the minimum improvement margin.
+- Manifest: optional `planning.cacheAware` (numbers only, never content) with
+  eligibility, tail tokens, miss/hit ratio, observed share, sample count,
+  estimated reusable prefix tokens, estimated request cost and winning
+  candidate; surfaced in `/context tokens` and `/context explain`.
+- Core: `packages/core/src/planner/cache-policy.ts` with pure, deterministic
+  functions for common-prefix estimation, request cost and the tail decision.
+- Tests: `cache-policy` unit tests, config validation, planner override tests,
+  the synthetic prefix-cache simulator (`cache-prefix-simulator`) and runtime
+  integration tests for off/auto/no-discount/under-budget paths.
+
+## Compatibility
+
+- Additive configuration; absent fields use the documented defaults.
+- `context.cacheAware.mode: "off"` reproduces the 0.3.6 behavior exactly.
+- No new required database schema; manifests persist the optional block as-is.
+- The existing guarantees (current request, atomicity, privacy, pins, hard
+  limits, fail-open, naive compaction path) are unchanged.
+- Model metadata without cache pricing or observed samples degrades the
+  decision to the nominal plan.
+
+## Validation
+
+- `npm run check` (excluding the known machine-load-dependent
+  `long-session` timeout flake): 83 files, 550 tests passed.
+- Planner unit tests: 25 passed.
+- Cache-policy unit tests: 16 passed.
+- Prefix-cache simulator: 7 passed.
+- Cache-aware runtime integration: 4 passed.
+- Config catalog/loader validation: passed.
+
+## Known limits
+
+- The epoch cost model is an estimate for plan comparison; actual billing is
+  whatever the provider reports.
+- A real-provider DeepSeek A/B benchmark remains voluntary and out of CI
+  (protocol: [CACHE_AWARE_BENCHMARK.md](../CACHE_AWARE_BENCHMARK.md)).
+- The workaround override (large `recentTailTokens`, zeroed retrieval and
+disabled compaction) remains available and is unaffected; it is more aggressive
+than `mode: "auto"`, which deliberately preserves retrieval/project and
+compaction for quality.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ds4-context-engine",
-  "version": "0.3.5",
+  "version": "0.3.7",
   "description": "Non-destructive, provider-independent context management for Pi.",
   "type": "module",
   "license": "MIT",
@@ -62,7 +62,7 @@
     ]
   },
   "dependencies": {
-    "ds4-context-core": "0.3.5"
+    "ds4-context-core": "0.3.7"
   },
   "peerDependencies": {
     "@earendil-works/pi-ai": "0.84.3",

package/src/extension/commands.ts

@@ -296,6 +296,14 @@ function formatTokens(diagnostics: RuntimeDiagnostics): string {
     `Actual provider input:    ${count(manifest?.actualInputTokens)}`,
     `  Uncached input:         ${count(manifest?.providerUsage?.inputTokens)}`,
     `  Cache read / write:     ${count(manifest?.providerUsage?.cacheReadTokens)} / ${count(manifest?.providerUsage?.cacheWriteTokens)}`,
+    ...(manifest?.planning?.cacheAware
+      ? [
+          `Cache-aware plan:         ${manifest.planning.cacheAware.candidate ?? "n/a"}${manifest.planning.cacheAware.tailExtended ? " (tail extended)" : ""}`,
+          `  Reusable prefix:       ${count(manifest.planning.cacheAware.reusablePrefixTokens)} estimated`,
+          `  Est. request cost:     ${manifest.planning.cacheAware.estimatedCost === undefined ? "n/a" : `$${manifest.planning.cacheAware.estimatedCost.toFixed(6)}`}`,
+          `  Miss/hit price ratio:  ${manifest.planning.cacheAware.missHitRatio === undefined ? "n/a" : manifest.planning.cacheAware.missHitRatio.toFixed(2)}`,
+        ]
+      : []),
     `Pi reported context:      ${count(manifest?.piReportedContextTokens ?? observation?.reportedTokens)}`,
     `Model context window:     ${count(budget?.contextWindow ?? manifest?.contextWindow)}`,
     `Output reserve:           ${count(budget?.outputReserve ?? manifest?.outputReserve)}`,
@@ -316,7 +324,7 @@ function formatManifest(diagnostics: RuntimeDiagnostics): string {
   const manifest = diagnostics.lastManifest;
   if (!manifest) return "No Context Manifest has been built for this session yet.";
 
-  const inventory = manifest.persistedInventory;
+  const inventory = manifest.persistedInventory ?? diagnostics.persistedInventory;
   const kinds = new Map<string, { items: number; tokens: number }>();
   for (const item of manifest.included) {
     const aggregate = kinds.get(item.kind) ?? { items: 0, tokens: 0 };
@@ -369,7 +377,7 @@ function formatManifestItems(diagnostics: RuntimeDiagnostics, type: "included" |
   const manifest = diagnostics.lastManifest;
   if (!manifest) return "No Context Manifest has been built for this session yet.";
   const items = manifest[type];
-  const inventory = manifest.persistedInventory;
+  const inventory = manifest.persistedInventory ?? diagnostics.persistedInventory;
   return [
     `DS4 Context ${type === "included" ? "Included" : "Excluded"} Items`,
     "",
@@ -403,6 +411,29 @@ function formatExplain(diagnostics: RuntimeDiagnostics): string {
     `Message target:       ${count(planning.messageTargetTokens)}`,
     `Message hard limit:   ${count(planning.messageHardLimitTokens)}`,
     `Recent-tail limit:    ${count(planning.recentTailTokenLimit)}`,
+    ...(planning.rescuedImmediatePredecessor
+      ? ["Rescued predecessor:  yes (immediate previous turn kept beyond the recent-tail cap within the input budget)"]
+      : []),
+    ...(planning.oversizedTurnExclusions
+      ? [`Oversized turn excl:  ${count(planning.oversizedTurnExclusions)} (turn group(s) at/above the recent-tail cap; recovered by retrieval only if it fits)`]
+      : []),
+    ...(planning.cacheAware
+      ? [
+          `Cache-aware plan:    ${planning.cacheAware.candidate ?? "n/a"}${planning.cacheAware.tailExtended ? " (tail extended)" : ""}`,
+          ...(planning.cacheAware.missHitRatio !== undefined
+            ? [`  Miss/hit ratio:     ${planning.cacheAware.missHitRatio.toFixed(2)}`]
+            : []),
+          ...(planning.cacheAware.cacheReadShare !== undefined
+            ? [`  Cache-read share:   ${(planning.cacheAware.cacheReadShare * 100).toFixed(1)}% (${count(planning.cacheAware.sampleCount)} samples)`]
+            : []),
+          ...(planning.cacheAware.reusablePrefixTokens !== undefined
+            ? [`  Reusable prefix:    ${count(planning.cacheAware.reusablePrefixTokens)} tokens`]
+            : []),
+          ...(planning.cacheAware.estimatedCost !== undefined
+            ? [`  Est. request cost:  $${planning.cacheAware.estimatedCost.toFixed(6)}`]
+            : []),
+        ]
+      : []),
     `Selected groups:      ${count(planning.selectedGroupCount)}`,
     `Excluded groups:      ${count(planning.excludedGroupCount)}`,
     `Duration:             ${planning.durationMs === undefined ? "n/a" : `${planning.durationMs.toFixed(1)} ms`}`,

package/src/extension/runtime.ts

@@ -72,7 +72,10 @@ import type {
   PrivacyManifest,
   ProviderUsageManifest,
 } from "ds4-context-core/manifest/context-manifest";
-import { HARD_PERSISTED_MANIFEST_BYTES } from "ds4-context-core/manifest/context-manifest-storage";
+import {
+  HARD_PERSISTED_MANIFEST_BYTES,
+  type PersistedManifestInventory,
+} from "ds4-context-core/manifest/context-manifest-storage";
 import {
   unavailableStorageDiagnostics,
   type StorageDiagnostics,
@@ -97,7 +100,14 @@ import {
   type PinScope,
   type ProjectMemorySource,
 } from "ds4-context-core/memory/memory-types";
-import { planManagedContext, type SupplementalContextMessage } from "ds4-context-core/planner/context-planner";
+import { planManagedContext, type ManagedContextPlan, type SupplementalContextMessage } from "ds4-context-core/planner/context-planner";
+import {
+  decideCacheAwareTail,
+  estimateReusablePrefixTokens,
+  estimateRequestCost,
+  type CacheAwarePlanDecision,
+  type CachePricing,
+} from "ds4-context-core/planner/cache-policy";
 import {
   disabledContextQualityDiagnostics,
   evaluateManifestQuality,
@@ -151,6 +161,8 @@ import {
   buildPiObserverManifest,
   findExactPiMessageSourceIds,
   findPiPinnedMessageIndices,
+  findPiSourceEntryIds,
+  fingerprint,
 } from "../pi-adapter/context-observer.ts";
 import { projectSessionFileMutations } from "../pi-adapter/memory-adapter.ts";
 import { ProjectMemorySynchronizer } from "../pi-adapter/project-memory-sync.ts";
@@ -283,6 +295,10 @@ function numericUsage(value: unknown): number {
     : 0;
 }
 
+function roundedCost(value: number): number {
+  return Math.round(value * 1_000_000) / 1_000_000;
+}
+
 function providerUsageManifest(input: {
   inputTokens: number;
   cacheReadTokens: number;
@@ -373,6 +389,8 @@ export interface RuntimeDiagnostics {
   indexed?: SessionIndexStats;
   observation?: ContextObservation;
   lastManifest?: ContextManifest;
+  /** Inventory of the persisted projection of the last successfully stored manifest. */
+  persistedInventory?: PersistedManifestInventory;
   retrieval: RetrievalDiagnostics;
   project: ProjectKnowledgeDiagnostics;
   memory: MemoryDiagnostics;
@@ -410,6 +428,7 @@ export class Ds4ContextRuntime {
   private databasePath?: string;
   private observation?: ContextObservation;
   private lastManifest?: ContextManifest;
+  private lastPersistedInventory?: PersistedManifestInventory;
   private pendingManifestId?: string;
   private pendingManifestPersisted = false;
   private retrievalEngine?: HistoricalRetrievalEngine;
@@ -432,6 +451,18 @@ export class Ds4ContextRuntime {
   private lastContextProfileKey?: string;
   private readonly knownModelProfiles = new Set<string>();
   private readonly volatileCalibration = new Map<string, TokenCalibrationSample[]>();
+  /** Fingerprints of the messages sent in the last managed plan (empty when unknown). */
+  private lastPlanMessageHashes: string[] = [];
+  /** Per-message token estimates of the last managed plan, aligned with hashes. */
+  private lastPlanMessageTokens: number[] = [];
+  /** Cache-aware decision of the last plan; used for /context diagnostics. */
+  private lastCacheAwareDecision?: CacheAwarePlanDecision;
+  /** Provider/model key of the last plan; a switch invalidates the cached prefix. */
+  private lastPlanModelKey?: string;
+  /** Consecutive epochs in which the extended tail lost the comparison (hysteresis). */
+  private cacheAwareExtendedLossStreak = 0;
+  /** Extended-tail adoption state: while set, the extended tail is kept even if a single epoch comparison loses. */
+  private cacheAwareStickExtended = false;
   private lastMemoryMutationSignature?: string;
   private artifactManager?: ArtifactManager;
   private lastArtifacts: ArtifactDiagnostics = disabledArtifactDiagnostics();
@@ -480,6 +511,7 @@ export class Ds4ContextRuntime {
     this.pendingQuality.length = 0;
     this.lastIndexResult = undefined;
     this.lastManifest = undefined;
+    this.lastPersistedInventory = undefined;
     this.pendingManifestId = undefined;
     this.pendingManifestPersisted = false;
     this.retrievalEngine = undefined;
@@ -584,6 +616,7 @@ export class Ds4ContextRuntime {
         this.syncSessionIndex(ctx);
         this.lastManifest = this.database.manifests.getLatest(this.session.sessionId);
         if (this.lastManifest) {
+          this.lastPersistedInventory = this.lastManifest.persistedInventory;
           this.lastContextProfileKey = modelProfileKey(
             this.lastManifest.provider,
             this.lastManifest.model,
@@ -762,6 +795,79 @@ export class Ds4ContextRuntime {
     };
   }
 
+  /**
+   * Optional per-million pricing proxied from Pi model metadata; undefined
+   * values degrade cache-aware planning to the previous behavior.
+   */
+  private static cachePricing(cost: ModelDescriptor["cost"]): CachePricing | undefined {
+    if (cost === undefined) return undefined;
+    return {
+      inputPerMillion: cost.input,
+      cacheReadPerMillion: cost.cacheRead,
+      cacheWritePerMillion: cost.cacheWrite,
+      outputPerMillion: cost.output,
+    };
+  }
+
+  /**
+   * Estimated cost of keeping a plan for an epoch of `turnsPerEpoch` user
+   * turns with `requestsPerTurn` provider requests each, in dollars.
+   *
+   * Model (documented, deterministic):
+   * - STABLE plan (original conversation fits the tail cap): the first
+   *   request of the epoch is cold (it reuses only the prefix shared with the
+   *   previously sent plan); every following request is warm.
+   * - SLIDING plan (the conversation exceeds the tail cap): the prefix is
+   *   invalidated by every new turn, so each turn of the epoch pays one cold
+   *   request plus the remaining warm requests.
+   *
+   * Returns undefined when pricing is incomplete or the plan is not managed.
+   * Costs are metadata estimates only; no provider content is exposed.
+   */
+  private cacheAwareEpochCost(
+    plan: ManagedContextPlan<ContextEvent["messages"][number]>,
+    fixedTokens: number,
+    cost: NonNullable<ModelDescriptor["cost"]>,
+    requestsPerTurn: number,
+    turnsPerEpoch: number,
+    modelKey: string,
+  ): number | undefined {
+    const pricing = Ds4ContextRuntime.cachePricing(cost);
+    if (!pricing || plan.mode !== "managed") return undefined;
+    const hashes: string[] = [];
+    const tokens: number[] = [];
+    let total = fixedTokens;
+    for (const message of plan.messages) {
+      hashes.push(fingerprint(message));
+      const estimate = estimateMessagesTokens([message]);
+      tokens.push(estimate);
+      total += estimate;
+    }
+    const previousHashes = modelKey !== this.lastPlanModelKey ? [] : this.lastPlanMessageHashes;
+    const reusable = previousHashes.length > 0
+      ? estimateReusablePrefixTokens(previousHashes, hashes, tokens)
+      : 0;
+    const coldCost = estimateRequestCost({
+      totalInputTokens: total,
+      reusablePrefixTokens: reusable,
+    }, pricing);
+    const warmCost = estimateRequestCost({
+      totalInputTokens: total,
+      reusablePrefixTokens: total,
+    }, pricing);
+    if (coldCost.total === undefined || warmCost.total === undefined) return undefined;
+    const sliding = plan.planning.originalMessageTokens > plan.planning.recentTailTokenLimit;
+    if (sliding) {
+      // Each turn of the epoch invalidates the prefix: one cold request plus
+      // the remaining warm requests per turn.
+      const perTurn = coldCost.total + Math.max(0, requestsPerTurn - 1) * warmCost.total;
+      return roundedCost(perTurn * turnsPerEpoch);
+    }
+    // Stable plan: one cold transition for the epoch, then all warm.
+    const epochCost = coldCost.total + Math.max(0, requestsPerTurn * turnsPerEpoch - 1) * warmCost.total;
+    return roundedCost(epochCost);
+  }
+
   private resolveModelPolicy(model: ModelDescriptor): {
     awareness: ResolvedModelAwareness;
     budget: ContextBudget;
@@ -973,10 +1079,119 @@ export class Ds4ContextRuntime {
           memoryTokens: 0,
         };
         if (this.memoryManager) this.lastMemory = this.memoryManager.diagnostics();
+        const fixedTokens = baseline.composition.systemTokens + baseline.composition.toolTokens;
+        const pinnedMessageIndices = findPiPinnedMessageIndices(effectiveEvent, ctx);
+        const retrievalEnabled = this.config.retrieval.exact
+          || this.config.retrieval.fts
+          || this.config.retrieval.semantic;
+        const dedupSupplementalMessages = [
+          ...memorySelection.pins.map((evidence) => ({
+            id: `pin:${evidence.item.id}`,
+            message: evidence.message,
+            kind: "pin" as const,
+            sourceIds: [evidence.item.id],
+            score: 950,
+            reason: evidence.reason,
+          })),
+          ...memorySelection.memories.map((evidence) => ({
+            id: `memory:${evidence.item.id}`,
+            message: evidence.message,
+            kind: "memory" as const,
+            sourceIds: [evidence.item.id],
+            score: 90 + Math.min(0.999999, Math.max(0, evidence.score) / 1_000),
+            reason: evidence.reason,
+          })),
+        ] satisfies Array<SupplementalContextMessage<ContextEvent["messages"][number]>>;
+        const nominalDedupPlan = planManagedContext({
+          messages: effectiveEvent.messages,
+          fixedTokens,
+          budget,
+          config: effectiveContextConfig,
+          pinnedMessageIndices,
+          supplementalMessages: dedupSupplementalMessages,
+        });
+        /**
+         * Cache-aware candidate selection (opt-in, default off): compares the
+         * estimated input cost of the nominal plan with an extended-tail plan
+         * and switches only when the improvement beats the configured
+         * hysteresis threshold. Costs are metadata estimates from model
+         * pricing and message fingerprints; no provider content is exposed.
+         */
+        let cacheAwareTailTokens: number | undefined;
+        let cacheAwareDecision: CacheAwarePlanDecision | undefined;
+        if (effectiveContextConfig.cacheAware?.mode === "auto" && model?.cost && budget) {
+          const decision = decideCacheAwareTail({
+            config: effectiveContextConfig.cacheAware,
+            pricing: Ds4ContextRuntime.cachePricing(model.cost),
+            observedCacheReadShare: activeModel?.awareness.calibration.cache.sampleCount > 0
+              ? activeModel.awareness.calibration.cache.cacheReadShare
+              : undefined,
+            sampleCount: activeModel?.awareness.calibration.cache.sampleCount ?? 0,
+            nominalRecentTailTokens: activeModel?.awareness.limits.recentTailTokens
+              ?? effectiveContextConfig.recentTailTokens,
+            activeInputBudget: budget.activeInputBudget,
+          });
+          cacheAwareDecision = decision;
+          if (decision.eligible && decision.tailExtended) {
+            const extendedDedupPlan = planManagedContext({
+              messages: effectiveEvent.messages,
+              fixedTokens,
+              budget,
+              config: effectiveContextConfig,
+              pinnedMessageIndices,
+              supplementalMessages: dedupSupplementalMessages,
+              cacheAwareTailTokens: decision.recentTailTokens,
+            });
+            const modelKey = modelProfileKey(model.provider, model.id);
+            const nominalEpoch = this.cacheAwareEpochCost(nominalDedupPlan, fixedTokens, model.cost, effectiveContextConfig.cacheAware.expectedRequestsPerTurn, effectiveContextConfig.cacheAware.expectedTurnsPerEpoch, modelKey);
+            const extendedEpoch = this.cacheAwareEpochCost(extendedDedupPlan, fixedTokens, model.cost, effectiveContextConfig.cacheAware.expectedRequestsPerTurn, effectiveContextConfig.cacheAware.expectedTurnsPerEpoch, modelKey);
+            this.logger.debug("context.cache_aware_candidate", {
+              eligible: decision.eligible,
+              tailExtended: decision.tailExtended,
+              recentTailTokens: decision.recentTailTokens,
+              nominalEpoch,
+              extendedEpoch,
+            });
+            const extendedWon = extendedEpoch !== undefined
+              && nominalEpoch !== undefined
+              && extendedEpoch < nominalEpoch * (1 - effectiveContextConfig.cacheAware.minimumImprovementRatio);
+            // Hysteresis (deliberate epochs): once adopted, the extended tail is
+            // kept while it loses at most a single epoch comparison; it is
+            // dropped only after two consecutive losses, preventing
+            // oscillation between plans. The streak resets on a win or model switch.
+            if (extendedWon) {
+              this.cacheAwareExtendedLossStreak = 0;
+              this.cacheAwareStickExtended = true;
+            } else if (this.cacheAwareStickExtended) {
+              this.cacheAwareExtendedLossStreak += 1;
+              if (this.cacheAwareExtendedLossStreak >= effectiveContextConfig.cacheAware.stickinessEpochs) {
+                this.cacheAwareStickExtended = false;
+                this.cacheAwareExtendedLossStreak = 0;
+              }
+            }
+            if (this.cacheAwareStickExtended) {
+              cacheAwareTailTokens = decision.recentTailTokens;
+            }
+          }
+        }
+        const retrievalActiveContextEntryIds = retrievalEnabled
+          ? this.plannedContextEntryIds(ctx, cacheAwareTailTokens !== undefined
+            ? planManagedContext({
+                messages: effectiveEvent.messages,
+                fixedTokens,
+                budget,
+                config: effectiveContextConfig,
+                pinnedMessageIndices,
+                supplementalMessages: dedupSupplementalMessages,
+                cacheAwareTailTokens,
+              })
+            : nominalDedupPlan)
+          : undefined;
         const retrieval = this.retrieveHistory(
           effectiveEvent,
           ctx,
           effectiveContextConfig.maxRetrievedHistoryTokens,
+          retrievalActiveContextEntryIds,
         );
         const project = this.retrieveProjectKnowledge(
           requestText,
@@ -1110,11 +1325,12 @@ export class Ds4ContextRuntime {
         });
         const plan = planManagedContext({
           messages: effectiveEvent.messages,
-          fixedTokens: baseline.composition.systemTokens + baseline.composition.toolTokens,
+          fixedTokens,
           budget,
           config: effectiveContextConfig,
-          pinnedMessageIndices: findPiPinnedMessageIndices(effectiveEvent, ctx),
+          pinnedMessageIndices,
           supplementalMessages: rankedSupplementalMessages,
+          ...(cacheAwareTailTokens !== undefined ? { cacheAwareTailTokens } : {}),
           ...(ranking.diagnostics.status === "active"
             ? { supplementalSelectionOrder: ranking.ranked.map((candidate) => candidate.id) }
             : {}),
@@ -1175,6 +1391,75 @@ export class Ds4ContextRuntime {
           selected: plannerSelectedProject,
         };
         plan.planning.durationMs = Math.max(0, this.now() - planningStartedAt);
+        /**
+         * Cache-aware diagnostics are metadata-only (tokens, ratios, cost).
+         * The reusable prefix is estimated against the previous managed plan;
+         * if the strategy changed since the last plan, the estimate is
+         * conservative (empty hashes) rather than optimistic. The block is
+         * present only when the cache-aware policy is enabled (mode auto);
+         * mode off leaves the manifest unchanged from 0.3.6.
+         */
+        if (plan.mode === "managed" && effectiveContextConfig.cacheAware?.mode === "auto") {
+          const currentModelKey = model ? modelProfileKey(model.provider, model.id) : undefined;
+          const previousHashes = currentModelKey !== undefined && this.lastPlanModelKey === currentModelKey
+            ? this.lastPlanMessageHashes
+            : [];
+          const currentHashes: string[] = [];
+          const currentTokens: number[] = [];
+          for (const message of plan.messages) {
+            currentHashes.push(fingerprint(message));
+            currentTokens.push(estimateMessagesTokens([message]));
+          }
+          const reusablePrefixTokens = estimateReusablePrefixTokens(
+            previousHashes,
+            currentHashes,
+            currentTokens,
+          );
+          const estimatedCost = model?.cost
+            ? estimateRequestCost({
+                totalInputTokens: plan.planning.fixedTokens
+                  + plan.selected.reduce((total, item) => total + item.tokens, 0),
+                reusablePrefixTokens,
+              }, Ds4ContextRuntime.cachePricing(model.cost) ?? {}).total
+            : undefined;
+          const candidate: "nominal" | "cache-aware" = cacheAwareTailTokens !== undefined
+            ? "cache-aware"
+            : "nominal";
+          plan.planning.cacheAware = {
+            eligible: cacheAwareDecision?.eligible ?? false,
+            tailExtended: cacheAwareTailTokens !== undefined,
+            ...(cacheAwareDecision?.recentTailTokens !== undefined
+              ? { recentTailTokens: cacheAwareDecision.recentTailTokens }
+              : {}),
+            ...(cacheAwareDecision?.missHitRatio !== undefined
+              ? { missHitRatio: cacheAwareDecision.missHitRatio }
+              : {}),
+            ...(cacheAwareDecision?.cacheReadShare !== undefined
+              ? { cacheReadShare: cacheAwareDecision.cacheReadShare }
+              : {}),
+            ...(cacheAwareDecision?.sampleCount !== undefined
+              ? { sampleCount: cacheAwareDecision.sampleCount }
+              : {}),
+            ...(reusablePrefixTokens > 0 ? { reusablePrefixTokens } : {}),
+            ...(estimatedCost !== undefined ? { estimatedCost } : {}),
+            candidate,
+          };
+          this.lastPlanMessageHashes = currentHashes;
+          this.lastPlanMessageTokens = currentTokens;
+          this.lastCacheAwareDecision = cacheAwareDecision;
+          this.lastPlanModelKey = currentModelKey;
+        }
+        const oversizedTurnExclusions = plan.planning.oversizedTurnExclusions ?? 0;
+        if (plan.mode === "managed" && oversizedTurnExclusions > 0) {
+          this.logger.warn("context.excluded_oversized_turn", {
+            oversizedTurnCount: oversizedTurnExclusions,
+            rescuedImmediatePredecessor: plan.planning.rescuedImmediatePredecessor ?? false,
+            recentTailTokenLimit: plan.planning.recentTailTokenLimit,
+            messageTargetTokens: plan.planning.messageTargetTokens,
+            selectedGroupCount: plan.planning.selectedGroupCount,
+            excludedGroupCount: plan.planning.excludedGroupCount,
+          });
+        }
         const plannedEvent: ContextEvent = { type: "context", messages: plan.messages };
         const selectedArtifactReferences = plan.mode === "managed"
           ? plan.selected.flatMap((metadata) => {
@@ -1362,6 +1647,9 @@ export class Ds4ContextRuntime {
     }
     try {
       const result = this.database.manifests.save(manifest);
+      if (result.status === "stored") {
+        this.lastPersistedInventory = result.inventory;
+      }
       if (result.status === "skipped-oversize") {
         this.logger.warn("context.manifest_persistence_skipped", {
           category: "oversize",
@@ -2758,10 +3046,36 @@ export class Ds4ContextRuntime {
     }
   }
 
+  private plannedContextEntryIds(
+    ctx: ExtensionContext,
+    plan: ManagedContextPlan<ContextEvent["messages"][number]>,
+  ): Set<string> {
+    if (plan.mode === "fallback") {
+      return new Set(ctx.sessionManager.buildContextEntries().map((entry) => entry.id));
+    }
+    const syntheticIndices = new Set(
+      [...plan.selected, ...plan.excluded]
+        .filter((metadata) =>
+          metadata.kind === "memory"
+          || (metadata.kind === "pin" && metadata.sourceId !== undefined)
+        )
+        .map((metadata) => metadata.originalIndex),
+    );
+    const selectedIndices = new Set(plan.selected.map((metadata) => metadata.originalIndex));
+    const sources = findPiSourceEntryIds(plan.originalMessages, ctx, syntheticIndices);
+    const entryIds = new Set<string>();
+    for (const index of selectedIndices) {
+      const sourceId = sources[index];
+      if (sourceId) entryIds.add(sourceId);
+    }
+    return entryIds;
+  }
+
   private retrieveHistory(
     event: ContextEvent,
     ctx: ExtensionContext,
     maxTokens = this.config.context.maxRetrievedHistoryTokens,
+    activeContextEntryIds?: ReadonlySet<string>,
   ): RetrievalDiagnostics {
     const requestText = currentRequestText(event.messages);
     if (!this.retrievalEngine || !this.session?.sessionFile || !requestText) {
@@ -2775,7 +3089,8 @@ export class Ds4ContextRuntime {
         sessionId: this.session.sessionId,
         requestText,
         activeBranchEntryIds: new Set(ctx.sessionManager.getBranch().map((entry) => entry.id)),
-        activeContextEntryIds: new Set(ctx.sessionManager.buildContextEntries().map((entry) => entry.id)),
+        activeContextEntryIds: activeContextEntryIds
+          ?? new Set(ctx.sessionManager.buildContextEntries().map((entry) => entry.id)),
         exact: this.config.retrieval.exact,
         fts: this.config.retrieval.fts,
         semantic: this.config.retrieval.semantic,
@@ -2965,6 +3280,7 @@ export class Ds4ContextRuntime {
       ...(indexed ? { indexed } : {}),
       ...(this.observation ? { observation: this.observation } : {}),
       ...(this.lastManifest ? { lastManifest: this.lastManifest } : {}),
+      ...(this.lastPersistedInventory ? { persistedInventory: this.lastPersistedInventory } : {}),
       retrieval: this.lastRetrieval,
       project: this.lastProject,
       memory: this.lastMemory,

package/src/pi-adapter/context-observer.ts

@@ -87,7 +87,7 @@ function sourceKind(entry: SessionEntry, role?: string): ContextManifestItemKind
   return "history";
 }
 
-function fingerprint(message: unknown): string {
+export function fingerprint(message: unknown): string {
   return sha256(stableStringify(message));
 }
 
@@ -244,6 +244,22 @@ export function findExactPiMessageSourceIds(
   });
 }
 
+/**
+ * Maps managed-plan messages back to Pi session entry ids. Synthetic evidence
+ * indices are skipped so role/order fallbacks stay aligned with real messages.
+ */
+export function findPiSourceEntryIds(
+  messages: readonly unknown[],
+  ctx: Pick<ExtensionContext, "sessionManager">,
+  syntheticIndices: ReadonlySet<number> = new Set(),
+): Array<string | undefined> {
+  return mapMessageSources(
+    messages,
+    sourceCandidates(ctx.sessionManager.buildContextEntries()),
+    syntheticIndices,
+  ).map((source) => source.sourceId);
+}
+
 export function findPiPinnedMessageIndices(event: ContextEvent, ctx: ExtensionContext): number[] {
   const candidates = sourceCandidates(ctx.sessionManager.buildContextEntries());
   const sources = mapMessageSources(event.messages, candidates);

package/src/pi-adapter/session-reader.ts

@@ -35,5 +35,15 @@ export function snapshotModel(ctx: Pick<ExtensionContext, "model">): ModelDescri
     maxTokens: ctx.model.maxTokens,
     reasoning: ctx.model.reasoning,
     input: ctx.model.input,
+    ...(ctx.model.cost && typeof ctx.model.cost.input === "number"
+      ? {
+          cost: {
+            input: ctx.model.cost.input,
+            output: ctx.model.cost.output,
+            cacheRead: ctx.model.cost.cacheRead,
+            cacheWrite: ctx.model.cost.cacheWrite,
+          },
+        }
+      : {}),
   };
 }

package/src/pi-adapter/version.ts

@@ -1,4 +1,4 @@
-export const EXTENSION_VERSION = "0.3.5";
+export const EXTENSION_VERSION = "0.3.7";
 export const SUPPORTED_PI_VERSION = "0.84.3";
 export const OBSERVER_PLANNER_VERSION = "observer-model-aware-v1";
 export const PLANNER_VERSION = "managed-learned-ranking-v1";