ds4-context-engine 0.3.6 → 0.3.7

package/docs/ADR/062-cache-aware-context-planning.md

@@ -0,0 +1,93 @@
+# ADR-062 — Cache-aware context planning
+
+Status: accepted and implemented in the coordinated 0.3.7 release (opt-in, default off; see the [release record](../releases/0.3.7.md)).
+
+## Context
+
+DS4 reduces provider input by selecting a bounded managed context. This is
+economically sound when every input token costs the same or when the provider
+offers no prompt cache. Providers with a large context window and a large
+cache-miss/cache-hit price ratio (for example DeepSeek V4 Flash, roughly 31x
+off-peak) invert the trade-off: a small but frequently re-planned prompt can
+cost more than a larger stable one, because every re-plan invalidates the
+provider prefix cache.
+
+The 0.3.6 planner applies an automatic 64k recent-tail ceiling to all models
+above 256k until the conversational tail exceeds the cap. When the cap is
+exceeded, the oldest turn leaves and the shared prefix with the previous
+request collapses: everything after the first divergence becomes a cache miss.
+Whether this is more expensive than a wider tail depends on the workload
+(requests per turn, turns per epoch, share of cached reads observed).
+
+The runtime already records cache read/write shares per exact `provider/model`
+(volatile calibration plus persisted manifests) and Pi exposes per-million
+cost rates on the active model. The portable core never hardcodes prices.
+
+## Decision
+
+- Add an optional economic cost profile to the portable `ModelDescriptor`
+  (`input`, `output`, `cacheRead`, `cacheWrite` per million), populated from
+  Pi model metadata in `snapshotModel`. Absent fields degrade cache-aware
+  planning to the previous behavior.
+- Add `context.cacheAware` (default `off`), a policy object with:
+  - `mode`: `off` preserves the previous planner exactly; `auto` may extend
+    the recent tail when pricing and observed cache shares justify it;
+  - `minimumCacheSampleCount`, `minimumCacheReadShare`,
+    `minimumMissHitRatio`: hard gates before any extension is eligible;
+  - `minimumImprovementRatio`: relative cost improvement required before an
+    alternative plan is adopted (hysteresis);
+  - `maxTailBudgetShare`: the extended tail may never exceed this fraction of
+    the active input budget;
+  - `expectedRequestsPerTurn`, `expectedTurnsPerEpoch`, `stickinessEpochs`:
+    deterministic cost model horizon plus hysteresis before dropping an
+    adopted extended tail.
+- In `auto` the runtime computes `context.cacheAware` decision and compares
+  two plan candidates (nominal tail vs extended tail) with a deterministic
+  epoch cost model:
+  - stable plans (original conversation fits their tail cap) pay one cold
+    transition per epoch and warm requests afterwards;
+  - sliding plans (conversation exceeds their cap) pay a cold request every
+    turn of the epoch.
+  The extended plan is adopted only when it is cheaper over the whole epoch,
+  after the minimum improvement margin.
+- The planner accepts an optional `cacheAwareTailTokens` override that bypasses
+  the automatic context-window ceiling; it remains bounded by the active/hard
+  input budgets and atomic groups, so no hard limit, privacy, pin, current
+  request or atomicity guarantee is weakened.
+- Manifest `planning.cacheAware` (optional, numbers only, never content)
+  records the decision, tail, miss/hit ratio, observed cache share, sample
+  count, estimated reusable prefix tokens, estimated request cost and the
+  winning candidate. `/context tokens` and `/context explain` surface the same
+  metadata. `context.excluded_oversized_turn` and previous guarantees are
+  unchanged.
+- Add a deterministic synthetic prefix-cache simulator covering the report
+  scenarios (append-only under the cap, sliding past 64k, one response per
+  prompt, five tool cycles, wide vs sliding tails, model switch, pricing
+  comparison). No provider, no credentials, no CI network access.
+
+## Rejected alternatives
+
+- Hardcoding provider prices: rejected, the core stays provider-independent.
+- Raising the default tail ceiling globally: rejected, it changes behavior
+  for all models (including ones without cache discounts).
+- Always extending the tail for eligible models: rejected, the epoch model
+  shows the nominal plan can still be cheaper on workloads where the tail
+  slides infrequently; the improvement margin keeps the switch conservative.
+
+## Compatibility and validation
+
+Configuration is additive; `context.cacheAware.mode` defaults to `off`, so
+`0.3.6` behavior (and manifest absence of the cacheAware block) is preserved.
+Automatic behavior is opt-in and requires observed samples plus positive
+pricing. The epoch cost is an estimate used only for the comparison; the
+actual billing remains whatever the provider reports. The workaround override
+(`modelAwareness.overrides` with a large `recentTailTokens`, zeroed retrieval
+and disabled compaction) remains available and is unaffected; it is more
+aggressive than `mode: "auto"` because it also removes retrieval/project
+supplements and compaction, which `auto` deliberately preserves for quality.
+
+Coverage: cache-policy unit tests (prefix, costs, decision gates), config
+validation, planner override tests, the synthetic prefix-cache simulator, and
+runtime integration tests for off/auto/no-discount/under-budget paths. A
+real-provider DeepSeek A/B benchmark remains out of scope and voluntary
+(protocol: [CACHE_AWARE_BENCHMARK.md](../CACHE_AWARE_BENCHMARK.md)).

package/docs/ADR/README.md

@@ -65,5 +65,6 @@ The initial decisions from the development plan are accepted:
 | [059](059-optional-anchored-editing.md) | Opt in to anchored edit expansion inside Pi's native mutation queue | Accepted |
 | [060](060-optional-portable-agent-tools.md) | Opt in to edit reports, adaptive reads/results and session-owned local jobs | Accepted |
 | [061](061-compaction-latency.md) | Bound compaction update calls, input budgets, concurrent segments and phase timings | Accepted |
+| [062](062-cache-aware-context-planning.md) | Opt-in cache-aware tail planning using model pricing and observed cache shares | Accepted |
 
 Each decision will receive a dedicated record when implementation pressure introduces alternatives or consequences not already covered by the development plan.

package/docs/CACHE_AWARE_BENCHMARK.md

@@ -0,0 +1,66 @@
+# Benchmark A/B DeepSeek — cache-aware planning
+
+Protocollo volontario, fuori CI, nessuna credenziale nel repo. Serve a decidere se
+`context.cacheAware.mode` può passare da `auto` (opt-in) a **default** in una
+release futura. Gate dichiarato in [ADR-062](ADR/062-cache-aware-context-planning.md).
+
+## Obiettivo
+
+Misurare, sulla **stessa** conversazione reale, il costo provider per turno con:
+
+- **Sessione A (baseline)**: comportamento 0.3.6 — `context.cacheAware` assente
+  (default `off`), tail automatica 64k, retrieval 16k, project 20k.
+- **Sessione B (auto)**: `context.cacheAware.mode: "auto"` con i default di
+  policy (gates 0.5 cache-share, ratio 20, margine 10%).
+- **Sessione C (workaround estremo, opzionale)**: `modelAwareness.overrides` con
+  `recentTailTokens: 500000`, `maxRetrievedHistoryTokens: 0`,
+  `maxProjectTokens: 0`, `compaction.enabled: false` — cache-first puro.
+
+Le tre sessioni devono avere la **stessa sequenza di turni** (stesso carico di
+lavoro: tool call, retrieval, project touches) e lo **stesso modello/provided
+ID DeepSeek**.
+
+## Cosa registrare per ogni turno
+
+Da `/context tokens` e `/context explain`:
+
+- `recentTailTokens` scelti (nominal vs extended) e `cacheAware` decision/reasons;
+- manifest `planning.cacheAware` (missHitRatio, cacheReadShare, sampleCount,
+  reusablePrefixTokens, estimatedCost, candidate);
+- `ProviderCacheMetrics` reali dopo ogni richiesta: `inputTokens`,
+  `cacheReadTokens`, `cacheWriteTokens`, share read/write per provider/model.
+
+Dal provider (costo effettivo):
+
+- input non-cached, cache-read (hit), cache-write, output per richiesta;
+- costo totale della sessione (per-turno e totale).
+
+## Metriche
+
+1. **Costo per turno** (media e totale sessione): A vs B vs C.
+2. **Cache-read share osservata** per config: conferma che la tail estesa
+   mantiene il prefisso stabile (share alta) vs sliding (share ~0 dopo 64k).
+3. **Qualità** (non solo $$$): recall retrieval, `Planner exclusions`,
+   current request/atomicità intatte, completamenti corretti nelle fasi tool.
+   Con retrieval azzerato (C) documentare i regressi di qualità.
+4. **Eventi di transizione**: quante volte il prefisso si è invalidato (turni
+   con cacheRead ≈ 0), confronto sliding vs stable.
+
+## Criterio di promozione a default
+
+Promuovere `mode: "auto"` a default solo se, su ≥ 3 sessioni reali:
+
+- costo medio per turno di B < A (margine ≥ il `minimumImprovementRatio`
+  configurato, di default 10%);
+- nessun regresso di qualità misurabile (recall retrieval uguale o migliore;
+  current request/atomicità preservate);
+- nessuna oscillazione plan (decisione che alterna nominal/esteso senza
+  motivo: verificare `stickinessEpochs` e diagnostica).
+
+## Esecuzione sicura
+
+- Nessuna credenziale nel repo; il benchmark usa la sessione Pi normale con il
+  provider DeepSeek già autenticato.
+- Non modificare la config attiva in modo permanente: copie di configurazione
+  per sessione, o valori temporanei poi ripristinati.
+- Fuori CI: niente rete/credenziali nel test suite.

package/docs/CONTEXT_MANIFEST.md

@@ -20,6 +20,7 @@ A Context Manifest explains the context visible at DS4's Pi `context` hook witho
 - artifact IDs, SHA-256, bytes, MIME, classification, exact source entry/tool IDs, error state, and before/after token estimates;
 - provider destination and allow-set names, selected classification counts, blocked/excluded/redacted counts, final provider-check count, and enforcement stage;
 - planner mode/version, original and selected counts, group counts, internal budgets, duration, and fallback reason;
+- optional cache-aware planning decision: eligibility, tail extension, requested tail tokens, miss/hit price ratio, observed cache-read share, sample count, estimated reusable prefix tokens, estimated request cost in dollars, and winning candidate (`nominal`/`cache-aware`); numbers only, never content;
 - learned-ranking mode/status, feature/model versions, candidate count, aggregate disagreement/rank shift, duration, and generic static-fallback reason;
 - planner and policy versions;
 - deterministic SHA-256 over system prompt, active tools, and messages;
@@ -54,6 +55,8 @@ A manifest at or below 256 KiB is stored unchanged. Above that preferred bound,
 
 Each manifest transaction prunes at most 32 excess rows and 8 MiB of serialized payload; one individually oversized oldest row may exceed the byte limit to guarantee progress. Calibration pruning is independently limited to 32 rows per related profile write. This incrementally repairs an existing oversized database without adding a long startup write or extending SQLite lock duration with an unbounded purge. Deleted pages become reusable by SQLite; the database file may remain at its previous high-water size until explicit [offline maintenance](STORAGE_MAINTENANCE.md). No retention action edits canonical Pi JSONL or project files.
 
+The `save()` result carries the derived `inventory` of the persisted projection, and the runtime exposes it through `RuntimeDiagnostics.persistedInventory`; `/context manifest` and `/context excluded` therefore report the truthful persisted completeness (`complete` or `excluded-rollup` with retained/total counts) instead of defaulting to `complete` when the in-memory manifest has no inventory attached.
+
 ## Reproducibility
 
 Object keys are normalized before hashing, so equivalent tool schemas with different key insertion order produce the same prompt hash. The estimator version is stored explicitly as `chars-v1`; planner/policy versions describe selection behavior. Golden tests protect manifest shape, model-profile resolution, token accounting, and hash stability.

package/docs/CONTEXT_PLANNER.md

@@ -86,6 +86,42 @@ M14 can queue the finalized manifest after planning when `quality.enabled` is tr
 
 M18 can evaluate bounded metadata-only features after privacy exclusion and before supplemental candidates enter category fitting. `shadow` keeps every static score/order authoritative and records aggregate disagreement only. `active` is accepted only for a compatible checksummed model carrying an eligible held-out promotion report. Privacy exclusions, mandatory pins/current turns, atomic groups and hard budgets cannot be overridden. See [`LEARNED_RANKING.md`](LEARNED_RANKING.md).
 
+## Cache-aware tail planning (0.3.7, opt-in)
+
+With `context.cacheAware.mode = "auto"` the runtime may extend the recent tail
+beyond the automatic context-window ceiling when the model pricing and the
+observed cache shares justify it economically. The policy never hardcodes
+prices: it reads the per-million rates exposed on the active Pi model and the
+cache read/write shares already recorded for the exact `provider/model`.
+
+Eligibility gates (defaults shown):
+
+- `minimumCacheSampleCount: 3` observed calibration samples;
+- `minimumCacheReadShare: 0.5` observed share;
+- `minimumMissHitRatio: 20` cache-miss / cache-hit price ratio.
+
+When eligible, the runtime compares two plan candidates with a deterministic
+epoch cost model:
+
+- the nominal plan (existing tail, sliding below 64k) pays a cold request on
+  every turn of the epoch because its prefix is invalidated by the slide;
+- the extended plan (up to `maxTailBudgetShare` of the active input budget) is
+  stable when the conversation fits, so it pays one cold transition per epoch
+  and warm requests afterwards.
+
+The extended plan is adopted only when it wins over the whole epoch
+(`expectedRequestsPerTurn`, `expectedTurnsPerEpoch`) after the
+`minimumImprovementRatio` margin. Once adopted it is kept (deliberate epoch)
+until it loses `stickinessEpochs` consecutive comparisons, preventing
+oscillation. The override remains bounded by the active and hard input
+budgets and by atomic groups, so current request, pins, privacy and atomicity
+guarantees are unchanged.
+
+`context.cacheAware.mode` defaults to `off`, preserving the 0.3.6 behavior
+exactly. Without prices, samples or a cache discount, the decision degrades to
+the nominal plan automatically. See [ADR-062](ADR/062-cache-aware-context-planning.md)
+for the model and rejected alternatives.
+
 ## Current limits
 
 The planner does not call a model inside the `context` hook. Model calibration uses only finalized provider usage and deterministic local statistics. Historical/project retrieval can opt into derived semantic candidates; learned supplemental reranking remains off by default and active mode is promotion-gated. Project symbol extraction is heuristic, artifact search is literal, and memory/pin creation is manual-first. Automatic memory extraction remains disabled; M10 supplies policy enforcement but not an automatic classifier or confirmation workflow. Provider-payload coverage targets Pi 0.84.3's supported serializers, and DS4 must load after any extension allowed to replace payloads when strict final ordering is required.

package/docs/releases/0.3.7.md

@@ -0,0 +1,83 @@
+# Release 0.3.7 — Cache-aware context planning (opt-in)
+
+**Version analyzed:** DS4 Context Engine `0.3.7`
+**Commit:** `a03009a4b14d6ab1842fa800076c4b300bd70d8f`
+**Coordinated packages:** `ds4-context-core` 0.3.7, `ds4-context-reference-adapter` 0.3.7, `ds4-context-engine` 0.3.7
+
+## Summary
+
+Adds an opt-in cache-aware planning policy driven by model pricing and
+observed cache shares, plus a deterministic synthetic prefix-cache simulator.
+The default behavior is unchanged: `context.cacheAware.mode` defaults to `off`,
+the manifest does not include a `cacheAware` block, and the planner uses the
+same tail caps as 0.3.6.
+
+## New configuration
+
+`context.cacheAware` (object, default off):
+
+| Field | Default | Meaning |
+|---|---|---|
+| `mode` | `off` | `off` preserves 0.3.6 behavior; `auto` may extend the recent tail. |
+| `minimumCacheSampleCount` | `3` | Observed samples required before acting. |
+| `minimumCacheReadShare` | `0.5` | Observed cache-read share required before acting. |
+| `minimumMissHitRatio` | `20` | Minimum cache-miss / cache-hit price ratio. |
+| `minimumImprovementRatio` | `0.1` | Relative improvement required to switch plan. |
+| `maxTailBudgetShare` | `0.5` | Max fraction of the active input budget for the extended tail. |
+| `expectedRequestsPerTurn` | `4` | Provider requests per user turn in the cost model. |
+| `expectedTurnsPerEpoch` | `4` | User turns per planning epoch in the cost model. |
+| `stickinessEpochs` | `2` | Consecutive epoch losses before dropping an adopted extended tail. |
+
+## Changes
+
+- `ModelDescriptor` gains an optional `cost` profile (`input`, `output`,
+  `cacheRead`, `cacheWrite` per million), populated from Pi model metadata in
+  `snapshotModel`. No prices are hardcoded in the core.
+- `planManagedContext` accepts an optional `cacheAwareTailTokens` override that
+  bypasses the automatic context-window ceiling while remaining bounded by the
+  active/hard input budgets and atomic groups.
+- Runtime: when `mode: "auto"` and the eligibility gates pass, the runtime
+  compares the nominal plan with an extended-tail plan using a deterministic
+  epoch cost model (stable plans pay one cold transition per epoch; sliding
+  plans pay a cold request per turn) and adopts the extended plan only when it
+  wins after the minimum improvement margin.
+- Manifest: optional `planning.cacheAware` (numbers only, never content) with
+  eligibility, tail tokens, miss/hit ratio, observed share, sample count,
+  estimated reusable prefix tokens, estimated request cost and winning
+  candidate; surfaced in `/context tokens` and `/context explain`.
+- Core: `packages/core/src/planner/cache-policy.ts` with pure, deterministic
+  functions for common-prefix estimation, request cost and the tail decision.
+- Tests: `cache-policy` unit tests, config validation, planner override tests,
+  the synthetic prefix-cache simulator (`cache-prefix-simulator`) and runtime
+  integration tests for off/auto/no-discount/under-budget paths.
+
+## Compatibility
+
+- Additive configuration; absent fields use the documented defaults.
+- `context.cacheAware.mode: "off"` reproduces the 0.3.6 behavior exactly.
+- No new required database schema; manifests persist the optional block as-is.
+- The existing guarantees (current request, atomicity, privacy, pins, hard
+  limits, fail-open, naive compaction path) are unchanged.
+- Model metadata without cache pricing or observed samples degrades the
+  decision to the nominal plan.
+
+## Validation
+
+- `npm run check` (excluding the known machine-load-dependent
+  `long-session` timeout flake): 83 files, 550 tests passed.
+- Planner unit tests: 25 passed.
+- Cache-policy unit tests: 16 passed.
+- Prefix-cache simulator: 7 passed.
+- Cache-aware runtime integration: 4 passed.
+- Config catalog/loader validation: passed.
+
+## Known limits
+
+- The epoch cost model is an estimate for plan comparison; actual billing is
+  whatever the provider reports.
+- A real-provider DeepSeek A/B benchmark remains voluntary and out of CI
+  (protocol: [CACHE_AWARE_BENCHMARK.md](../CACHE_AWARE_BENCHMARK.md)).
+- The workaround override (large `recentTailTokens`, zeroed retrieval and
+disabled compaction) remains available and is unaffected; it is more aggressive
+than `mode: "auto"`, which deliberately preserves retrieval/project and
+compaction for quality.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ds4-context-engine",
-  "version": "0.3.6",
+  "version": "0.3.7",
   "description": "Non-destructive, provider-independent context management for Pi.",
   "type": "module",
   "license": "MIT",
@@ -62,7 +62,7 @@
     ]
   },
   "dependencies": {
-    "ds4-context-core": "0.3.6"
+    "ds4-context-core": "0.3.7"
   },
   "peerDependencies": {
     "@earendil-works/pi-ai": "0.84.3",

package/src/extension/commands.ts

@@ -296,6 +296,14 @@ function formatTokens(diagnostics: RuntimeDiagnostics): string {
     `Actual provider input:    ${count(manifest?.actualInputTokens)}`,
     `  Uncached input:         ${count(manifest?.providerUsage?.inputTokens)}`,
     `  Cache read / write:     ${count(manifest?.providerUsage?.cacheReadTokens)} / ${count(manifest?.providerUsage?.cacheWriteTokens)}`,
+    ...(manifest?.planning?.cacheAware
+      ? [
+          `Cache-aware plan:         ${manifest.planning.cacheAware.candidate ?? "n/a"}${manifest.planning.cacheAware.tailExtended ? " (tail extended)" : ""}`,
+          `  Reusable prefix:       ${count(manifest.planning.cacheAware.reusablePrefixTokens)} estimated`,
+          `  Est. request cost:     ${manifest.planning.cacheAware.estimatedCost === undefined ? "n/a" : `$${manifest.planning.cacheAware.estimatedCost.toFixed(6)}`}`,
+          `  Miss/hit price ratio:  ${manifest.planning.cacheAware.missHitRatio === undefined ? "n/a" : manifest.planning.cacheAware.missHitRatio.toFixed(2)}`,
+        ]
+      : []),
     `Pi reported context:      ${count(manifest?.piReportedContextTokens ?? observation?.reportedTokens)}`,
     `Model context window:     ${count(budget?.contextWindow ?? manifest?.contextWindow)}`,
     `Output reserve:           ${count(budget?.outputReserve ?? manifest?.outputReserve)}`,
@@ -316,7 +324,7 @@ function formatManifest(diagnostics: RuntimeDiagnostics): string {
   const manifest = diagnostics.lastManifest;
   if (!manifest) return "No Context Manifest has been built for this session yet.";
 
-  const inventory = manifest.persistedInventory;
+  const inventory = manifest.persistedInventory ?? diagnostics.persistedInventory;
   const kinds = new Map<string, { items: number; tokens: number }>();
   for (const item of manifest.included) {
     const aggregate = kinds.get(item.kind) ?? { items: 0, tokens: 0 };
@@ -369,7 +377,7 @@ function formatManifestItems(diagnostics: RuntimeDiagnostics, type: "included" |
   const manifest = diagnostics.lastManifest;
   if (!manifest) return "No Context Manifest has been built for this session yet.";
   const items = manifest[type];
-  const inventory = manifest.persistedInventory;
+  const inventory = manifest.persistedInventory ?? diagnostics.persistedInventory;
   return [
     `DS4 Context ${type === "included" ? "Included" : "Excluded"} Items`,
     "",
@@ -409,6 +417,23 @@ function formatExplain(diagnostics: RuntimeDiagnostics): string {
     ...(planning.oversizedTurnExclusions
       ? [`Oversized turn excl:  ${count(planning.oversizedTurnExclusions)} (turn group(s) at/above the recent-tail cap; recovered by retrieval only if it fits)`]
       : []),
+    ...(planning.cacheAware
+      ? [
+          `Cache-aware plan:    ${planning.cacheAware.candidate ?? "n/a"}${planning.cacheAware.tailExtended ? " (tail extended)" : ""}`,
+          ...(planning.cacheAware.missHitRatio !== undefined
+            ? [`  Miss/hit ratio:     ${planning.cacheAware.missHitRatio.toFixed(2)}`]
+            : []),
+          ...(planning.cacheAware.cacheReadShare !== undefined
+            ? [`  Cache-read share:   ${(planning.cacheAware.cacheReadShare * 100).toFixed(1)}% (${count(planning.cacheAware.sampleCount)} samples)`]
+            : []),
+          ...(planning.cacheAware.reusablePrefixTokens !== undefined
+            ? [`  Reusable prefix:    ${count(planning.cacheAware.reusablePrefixTokens)} tokens`]
+            : []),
+          ...(planning.cacheAware.estimatedCost !== undefined
+            ? [`  Est. request cost:  $${planning.cacheAware.estimatedCost.toFixed(6)}`]
+            : []),
+        ]
+      : []),
     `Selected groups:      ${count(planning.selectedGroupCount)}`,
     `Excluded groups:      ${count(planning.excludedGroupCount)}`,
     `Duration:             ${planning.durationMs === undefined ? "n/a" : `${planning.durationMs.toFixed(1)} ms`}`,

package/src/extension/runtime.ts

@@ -72,7 +72,10 @@ import type {
   PrivacyManifest,
   ProviderUsageManifest,
 } from "ds4-context-core/manifest/context-manifest";
-import { HARD_PERSISTED_MANIFEST_BYTES } from "ds4-context-core/manifest/context-manifest-storage";
+import {
+  HARD_PERSISTED_MANIFEST_BYTES,
+  type PersistedManifestInventory,
+} from "ds4-context-core/manifest/context-manifest-storage";
 import {
   unavailableStorageDiagnostics,
   type StorageDiagnostics,
@@ -98,6 +101,13 @@ import {
   type ProjectMemorySource,
 } from "ds4-context-core/memory/memory-types";
 import { planManagedContext, type ManagedContextPlan, type SupplementalContextMessage } from "ds4-context-core/planner/context-planner";
+import {
+  decideCacheAwareTail,
+  estimateReusablePrefixTokens,
+  estimateRequestCost,
+  type CacheAwarePlanDecision,
+  type CachePricing,
+} from "ds4-context-core/planner/cache-policy";
 import {
   disabledContextQualityDiagnostics,
   evaluateManifestQuality,
@@ -152,6 +162,7 @@ import {
   findExactPiMessageSourceIds,
   findPiPinnedMessageIndices,
   findPiSourceEntryIds,
+  fingerprint,
 } from "../pi-adapter/context-observer.ts";
 import { projectSessionFileMutations } from "../pi-adapter/memory-adapter.ts";
 import { ProjectMemorySynchronizer } from "../pi-adapter/project-memory-sync.ts";
@@ -284,6 +295,10 @@ function numericUsage(value: unknown): number {
     : 0;
 }
 
+function roundedCost(value: number): number {
+  return Math.round(value * 1_000_000) / 1_000_000;
+}
+
 function providerUsageManifest(input: {
   inputTokens: number;
   cacheReadTokens: number;
@@ -374,6 +389,8 @@ export interface RuntimeDiagnostics {
   indexed?: SessionIndexStats;
   observation?: ContextObservation;
   lastManifest?: ContextManifest;
+  /** Inventory of the persisted projection of the last successfully stored manifest. */
+  persistedInventory?: PersistedManifestInventory;
   retrieval: RetrievalDiagnostics;
   project: ProjectKnowledgeDiagnostics;
   memory: MemoryDiagnostics;
@@ -411,6 +428,7 @@ export class Ds4ContextRuntime {
   private databasePath?: string;
   private observation?: ContextObservation;
   private lastManifest?: ContextManifest;
+  private lastPersistedInventory?: PersistedManifestInventory;
   private pendingManifestId?: string;
   private pendingManifestPersisted = false;
   private retrievalEngine?: HistoricalRetrievalEngine;
@@ -433,6 +451,18 @@ export class Ds4ContextRuntime {
   private lastContextProfileKey?: string;
   private readonly knownModelProfiles = new Set<string>();
   private readonly volatileCalibration = new Map<string, TokenCalibrationSample[]>();
+  /** Fingerprints of the messages sent in the last managed plan (empty when unknown). */
+  private lastPlanMessageHashes: string[] = [];
+  /** Per-message token estimates of the last managed plan, aligned with hashes. */
+  private lastPlanMessageTokens: number[] = [];
+  /** Cache-aware decision of the last plan; used for /context diagnostics. */
+  private lastCacheAwareDecision?: CacheAwarePlanDecision;
+  /** Provider/model key of the last plan; a switch invalidates the cached prefix. */
+  private lastPlanModelKey?: string;
+  /** Consecutive epochs in which the extended tail lost the comparison (hysteresis). */
+  private cacheAwareExtendedLossStreak = 0;
+  /** Extended-tail adoption state: while set, the extended tail is kept even if a single epoch comparison loses. */
+  private cacheAwareStickExtended = false;
   private lastMemoryMutationSignature?: string;
   private artifactManager?: ArtifactManager;
   private lastArtifacts: ArtifactDiagnostics = disabledArtifactDiagnostics();
@@ -481,6 +511,7 @@ export class Ds4ContextRuntime {
     this.pendingQuality.length = 0;
     this.lastIndexResult = undefined;
     this.lastManifest = undefined;
+    this.lastPersistedInventory = undefined;
     this.pendingManifestId = undefined;
     this.pendingManifestPersisted = false;
     this.retrievalEngine = undefined;
@@ -585,6 +616,7 @@ export class Ds4ContextRuntime {
         this.syncSessionIndex(ctx);
         this.lastManifest = this.database.manifests.getLatest(this.session.sessionId);
         if (this.lastManifest) {
+          this.lastPersistedInventory = this.lastManifest.persistedInventory;
           this.lastContextProfileKey = modelProfileKey(
             this.lastManifest.provider,
             this.lastManifest.model,
@@ -763,6 +795,79 @@ export class Ds4ContextRuntime {
     };
   }
 
+  /**
+   * Optional per-million pricing proxied from Pi model metadata; undefined
+   * values degrade cache-aware planning to the previous behavior.
+   */
+  private static cachePricing(cost: ModelDescriptor["cost"]): CachePricing | undefined {
+    if (cost === undefined) return undefined;
+    return {
+      inputPerMillion: cost.input,
+      cacheReadPerMillion: cost.cacheRead,
+      cacheWritePerMillion: cost.cacheWrite,
+      outputPerMillion: cost.output,
+    };
+  }
+
+  /**
+   * Estimated cost of keeping a plan for an epoch of `turnsPerEpoch` user
+   * turns with `requestsPerTurn` provider requests each, in dollars.
+   *
+   * Model (documented, deterministic):
+   * - STABLE plan (original conversation fits the tail cap): the first
+   *   request of the epoch is cold (it reuses only the prefix shared with the
+   *   previously sent plan); every following request is warm.
+   * - SLIDING plan (the conversation exceeds the tail cap): the prefix is
+   *   invalidated by every new turn, so each turn of the epoch pays one cold
+   *   request plus the remaining warm requests.
+   *
+   * Returns undefined when pricing is incomplete or the plan is not managed.
+   * Costs are metadata estimates only; no provider content is exposed.
+   */
+  private cacheAwareEpochCost(
+    plan: ManagedContextPlan<ContextEvent["messages"][number]>,
+    fixedTokens: number,
+    cost: NonNullable<ModelDescriptor["cost"]>,
+    requestsPerTurn: number,
+    turnsPerEpoch: number,
+    modelKey: string,
+  ): number | undefined {
+    const pricing = Ds4ContextRuntime.cachePricing(cost);
+    if (!pricing || plan.mode !== "managed") return undefined;
+    const hashes: string[] = [];
+    const tokens: number[] = [];
+    let total = fixedTokens;
+    for (const message of plan.messages) {
+      hashes.push(fingerprint(message));
+      const estimate = estimateMessagesTokens([message]);
+      tokens.push(estimate);
+      total += estimate;
+    }
+    const previousHashes = modelKey !== this.lastPlanModelKey ? [] : this.lastPlanMessageHashes;
+    const reusable = previousHashes.length > 0
+      ? estimateReusablePrefixTokens(previousHashes, hashes, tokens)
+      : 0;
+    const coldCost = estimateRequestCost({
+      totalInputTokens: total,
+      reusablePrefixTokens: reusable,
+    }, pricing);
+    const warmCost = estimateRequestCost({
+      totalInputTokens: total,
+      reusablePrefixTokens: total,
+    }, pricing);
+    if (coldCost.total === undefined || warmCost.total === undefined) return undefined;
+    const sliding = plan.planning.originalMessageTokens > plan.planning.recentTailTokenLimit;
+    if (sliding) {
+      // Each turn of the epoch invalidates the prefix: one cold request plus
+      // the remaining warm requests per turn.
+      const perTurn = coldCost.total + Math.max(0, requestsPerTurn - 1) * warmCost.total;
+      return roundedCost(perTurn * turnsPerEpoch);
+    }
+    // Stable plan: one cold transition for the epoch, then all warm.
+    const epochCost = coldCost.total + Math.max(0, requestsPerTurn * turnsPerEpoch - 1) * warmCost.total;
+    return roundedCost(epochCost);
+  }
+
   private resolveModelPolicy(model: ModelDescriptor): {
     awareness: ResolvedModelAwareness;
     budget: ContextBudget;
@@ -979,32 +1084,108 @@ export class Ds4ContextRuntime {
         const retrievalEnabled = this.config.retrieval.exact
           || this.config.retrieval.fts
           || this.config.retrieval.semantic;
-        const retrievalActiveContextEntryIds = retrievalEnabled
-          ? this.plannedContextEntryIds(ctx, planManagedContext({
+        const dedupSupplementalMessages = [
+          ...memorySelection.pins.map((evidence) => ({
+            id: `pin:${evidence.item.id}`,
+            message: evidence.message,
+            kind: "pin" as const,
+            sourceIds: [evidence.item.id],
+            score: 950,
+            reason: evidence.reason,
+          })),
+          ...memorySelection.memories.map((evidence) => ({
+            id: `memory:${evidence.item.id}`,
+            message: evidence.message,
+            kind: "memory" as const,
+            sourceIds: [evidence.item.id],
+            score: 90 + Math.min(0.999999, Math.max(0, evidence.score) / 1_000),
+            reason: evidence.reason,
+          })),
+        ] satisfies Array<SupplementalContextMessage<ContextEvent["messages"][number]>>;
+        const nominalDedupPlan = planManagedContext({
+          messages: effectiveEvent.messages,
+          fixedTokens,
+          budget,
+          config: effectiveContextConfig,
+          pinnedMessageIndices,
+          supplementalMessages: dedupSupplementalMessages,
+        });
+        /**
+         * Cache-aware candidate selection (opt-in, default off): compares the
+         * estimated input cost of the nominal plan with an extended-tail plan
+         * and switches only when the improvement beats the configured
+         * hysteresis threshold. Costs are metadata estimates from model
+         * pricing and message fingerprints; no provider content is exposed.
+         */
+        let cacheAwareTailTokens: number | undefined;
+        let cacheAwareDecision: CacheAwarePlanDecision | undefined;
+        if (effectiveContextConfig.cacheAware?.mode === "auto" && model?.cost && budget) {
+          const decision = decideCacheAwareTail({
+            config: effectiveContextConfig.cacheAware,
+            pricing: Ds4ContextRuntime.cachePricing(model.cost),
+            observedCacheReadShare: activeModel?.awareness.calibration.cache.sampleCount > 0
+              ? activeModel.awareness.calibration.cache.cacheReadShare
+              : undefined,
+            sampleCount: activeModel?.awareness.calibration.cache.sampleCount ?? 0,
+            nominalRecentTailTokens: activeModel?.awareness.limits.recentTailTokens
+              ?? effectiveContextConfig.recentTailTokens,
+            activeInputBudget: budget.activeInputBudget,
+          });
+          cacheAwareDecision = decision;
+          if (decision.eligible && decision.tailExtended) {
+            const extendedDedupPlan = planManagedContext({
               messages: effectiveEvent.messages,
               fixedTokens,
               budget,
               config: effectiveContextConfig,
               pinnedMessageIndices,
-              supplementalMessages: [
-                ...memorySelection.pins.map((evidence) => ({
-                  id: `pin:${evidence.item.id}`,
-                  message: evidence.message,
-                  kind: "pin" as const,
-                  sourceIds: [evidence.item.id],
-                  score: 950,
-                  reason: evidence.reason,
-                })),
-                ...memorySelection.memories.map((evidence) => ({
-                  id: `memory:${evidence.item.id}`,
-                  message: evidence.message,
-                  kind: "memory" as const,
-                  sourceIds: [evidence.item.id],
-                  score: 90 + Math.min(0.999999, Math.max(0, evidence.score) / 1_000),
-                  reason: evidence.reason,
-                })),
-              ],
-            }))
+              supplementalMessages: dedupSupplementalMessages,
+              cacheAwareTailTokens: decision.recentTailTokens,
+            });
+            const modelKey = modelProfileKey(model.provider, model.id);
+            const nominalEpoch = this.cacheAwareEpochCost(nominalDedupPlan, fixedTokens, model.cost, effectiveContextConfig.cacheAware.expectedRequestsPerTurn, effectiveContextConfig.cacheAware.expectedTurnsPerEpoch, modelKey);
+            const extendedEpoch = this.cacheAwareEpochCost(extendedDedupPlan, fixedTokens, model.cost, effectiveContextConfig.cacheAware.expectedRequestsPerTurn, effectiveContextConfig.cacheAware.expectedTurnsPerEpoch, modelKey);
+            this.logger.debug("context.cache_aware_candidate", {
+              eligible: decision.eligible,
+              tailExtended: decision.tailExtended,
+              recentTailTokens: decision.recentTailTokens,
+              nominalEpoch,
+              extendedEpoch,
+            });
+            const extendedWon = extendedEpoch !== undefined
+              && nominalEpoch !== undefined
+              && extendedEpoch < nominalEpoch * (1 - effectiveContextConfig.cacheAware.minimumImprovementRatio);
+            // Hysteresis (deliberate epochs): once adopted, the extended tail is
+            // kept while it loses at most a single epoch comparison; it is
+            // dropped only after two consecutive losses, preventing
+            // oscillation between plans. The streak resets on a win or model switch.
+            if (extendedWon) {
+              this.cacheAwareExtendedLossStreak = 0;
+              this.cacheAwareStickExtended = true;
+            } else if (this.cacheAwareStickExtended) {
+              this.cacheAwareExtendedLossStreak += 1;
+              if (this.cacheAwareExtendedLossStreak >= effectiveContextConfig.cacheAware.stickinessEpochs) {
+                this.cacheAwareStickExtended = false;
+                this.cacheAwareExtendedLossStreak = 0;
+              }
+            }
+            if (this.cacheAwareStickExtended) {
+              cacheAwareTailTokens = decision.recentTailTokens;
+            }
+          }
+        }
+        const retrievalActiveContextEntryIds = retrievalEnabled
+          ? this.plannedContextEntryIds(ctx, cacheAwareTailTokens !== undefined
+            ? planManagedContext({
+                messages: effectiveEvent.messages,
+                fixedTokens,
+                budget,
+                config: effectiveContextConfig,
+                pinnedMessageIndices,
+                supplementalMessages: dedupSupplementalMessages,
+                cacheAwareTailTokens,
+              })
+            : nominalDedupPlan)
           : undefined;
         const retrieval = this.retrieveHistory(
           effectiveEvent,
@@ -1149,6 +1330,7 @@ export class Ds4ContextRuntime {
           config: effectiveContextConfig,
           pinnedMessageIndices,
           supplementalMessages: rankedSupplementalMessages,
+          ...(cacheAwareTailTokens !== undefined ? { cacheAwareTailTokens } : {}),
           ...(ranking.diagnostics.status === "active"
             ? { supplementalSelectionOrder: ranking.ranked.map((candidate) => candidate.id) }
             : {}),
@@ -1209,6 +1391,64 @@ export class Ds4ContextRuntime {
           selected: plannerSelectedProject,
         };
         plan.planning.durationMs = Math.max(0, this.now() - planningStartedAt);
+        /**
+         * Cache-aware diagnostics are metadata-only (tokens, ratios, cost).
+         * The reusable prefix is estimated against the previous managed plan;
+         * if the strategy changed since the last plan, the estimate is
+         * conservative (empty hashes) rather than optimistic. The block is
+         * present only when the cache-aware policy is enabled (mode auto);
+         * mode off leaves the manifest unchanged from 0.3.6.
+         */
+        if (plan.mode === "managed" && effectiveContextConfig.cacheAware?.mode === "auto") {
+          const currentModelKey = model ? modelProfileKey(model.provider, model.id) : undefined;
+          const previousHashes = currentModelKey !== undefined && this.lastPlanModelKey === currentModelKey
+            ? this.lastPlanMessageHashes
+            : [];
+          const currentHashes: string[] = [];
+          const currentTokens: number[] = [];
+          for (const message of plan.messages) {
+            currentHashes.push(fingerprint(message));
+            currentTokens.push(estimateMessagesTokens([message]));
+          }
+          const reusablePrefixTokens = estimateReusablePrefixTokens(
+            previousHashes,
+            currentHashes,
+            currentTokens,
+          );
+          const estimatedCost = model?.cost
+            ? estimateRequestCost({
+                totalInputTokens: plan.planning.fixedTokens
+                  + plan.selected.reduce((total, item) => total + item.tokens, 0),
+                reusablePrefixTokens,
+              }, Ds4ContextRuntime.cachePricing(model.cost) ?? {}).total
+            : undefined;
+          const candidate: "nominal" | "cache-aware" = cacheAwareTailTokens !== undefined
+            ? "cache-aware"
+            : "nominal";
+          plan.planning.cacheAware = {
+            eligible: cacheAwareDecision?.eligible ?? false,
+            tailExtended: cacheAwareTailTokens !== undefined,
+            ...(cacheAwareDecision?.recentTailTokens !== undefined
+              ? { recentTailTokens: cacheAwareDecision.recentTailTokens }
+              : {}),
+            ...(cacheAwareDecision?.missHitRatio !== undefined
+              ? { missHitRatio: cacheAwareDecision.missHitRatio }
+              : {}),
+            ...(cacheAwareDecision?.cacheReadShare !== undefined
+              ? { cacheReadShare: cacheAwareDecision.cacheReadShare }
+              : {}),
+            ...(cacheAwareDecision?.sampleCount !== undefined
+              ? { sampleCount: cacheAwareDecision.sampleCount }
+              : {}),
+            ...(reusablePrefixTokens > 0 ? { reusablePrefixTokens } : {}),
+            ...(estimatedCost !== undefined ? { estimatedCost } : {}),
+            candidate,
+          };
+          this.lastPlanMessageHashes = currentHashes;
+          this.lastPlanMessageTokens = currentTokens;
+          this.lastCacheAwareDecision = cacheAwareDecision;
+          this.lastPlanModelKey = currentModelKey;
+        }
         const oversizedTurnExclusions = plan.planning.oversizedTurnExclusions ?? 0;
         if (plan.mode === "managed" && oversizedTurnExclusions > 0) {
           this.logger.warn("context.excluded_oversized_turn", {
@@ -1407,6 +1647,9 @@ export class Ds4ContextRuntime {
     }
     try {
       const result = this.database.manifests.save(manifest);
+      if (result.status === "stored") {
+        this.lastPersistedInventory = result.inventory;
+      }
       if (result.status === "skipped-oversize") {
         this.logger.warn("context.manifest_persistence_skipped", {
           category: "oversize",
@@ -3037,6 +3280,7 @@ export class Ds4ContextRuntime {
       ...(indexed ? { indexed } : {}),
       ...(this.observation ? { observation: this.observation } : {}),
       ...(this.lastManifest ? { lastManifest: this.lastManifest } : {}),
+      ...(this.lastPersistedInventory ? { persistedInventory: this.lastPersistedInventory } : {}),
       retrieval: this.lastRetrieval,
       project: this.lastProject,
       memory: this.lastMemory,

package/src/pi-adapter/context-observer.ts

@@ -87,7 +87,7 @@ function sourceKind(entry: SessionEntry, role?: string): ContextManifestItemKind
   return "history";
 }
 
-function fingerprint(message: unknown): string {
+export function fingerprint(message: unknown): string {
   return sha256(stableStringify(message));
 }
 

package/src/pi-adapter/session-reader.ts

@@ -35,5 +35,15 @@ export function snapshotModel(ctx: Pick<ExtensionContext, "model">): ModelDescri
     maxTokens: ctx.model.maxTokens,
     reasoning: ctx.model.reasoning,
     input: ctx.model.input,
+    ...(ctx.model.cost && typeof ctx.model.cost.input === "number"
+      ? {
+          cost: {
+            input: ctx.model.cost.input,
+            output: ctx.model.cost.output,
+            cacheRead: ctx.model.cost.cacheRead,
+            cacheWrite: ctx.model.cost.cacheWrite,
+          },
+        }
+      : {}),
   };
 }

package/src/pi-adapter/version.ts

@@ -1,4 +1,4 @@
-export const EXTENSION_VERSION = "0.3.6";
+export const EXTENSION_VERSION = "0.3.7";
 export const SUPPORTED_PI_VERSION = "0.84.3";
 export const OBSERVER_PLANNER_VERSION = "observer-model-aware-v1";
 export const PLANNER_VERSION = "managed-learned-ranking-v1";