@nhtio/adk 0.1.0-master-d0e68851 → 0.1.0-master-298eb7ff

package/CHANGELOG.md

@@ -15,6 +15,169 @@ you *when* you got it, not *what changed*: a `^` range will float across battery
 breaking changes, so pin an exact version if you need stability and read the entry before
 upgrading.
 
+## 2026-07-07
+
+### Security
+
+- **Prototype pollution in the `data.*` media steps (GHSA-xwg2-cvvj-3w4v).** `data.set` and
+  `data.delete` walk a caller-supplied dot/bracket path (`a.b[2].c`) with plain bracket access
+  (`container[seg]`); a path segment of `__proto__`, `prototype`, or `constructor` resolved
+  through the real prototype chain instead of stopping at an own property, so `data.set` with
+  `path: "__proto__.polluted"` reached `Object.prototype` and poisoned every plain object in
+  the process for the remainder of its lifetime — reachable from LLM tool-call arguments via
+  the forged `media_query`/`data_set` tool surface. `parsePath` now rejects any segment in that
+  denylist before `walkToParent` ever runs, closing both verbs (they share the same parser).
+  `data.merge` used a different code path (`{ ...target }` spread) that is not independently
+  exploitable — spreading onto a fresh object literal makes `__proto__` an inert own property,
+  not a prototype reassignment — but a new `assertSafeObjectKeys` guard now rejects the same
+  three keys anywhere in a merge fragment's tree before either the shallow or deep merge
+  strategy runs, so the verb can't be used to smuggle a poisoned key back out through a
+  round-tripped document.
+- **Defense-in-depth: reject the same three keys in vector-adapter metadata.** An audit for
+  the same vulnerability class found five adapters (`pinecone`, `s3vectors`, `qdrant`, `redis`,
+  `cloudflare`) that spread caller-supplied `record.metadata` onto a fresh object literal before
+  upsert. None of these were independently exploitable for the reason above — the spread target
+  is always a new `{}`, never a walked reference — but a `__proto__`-keyed metadata object would
+  otherwise round-trip verbatim through storage and back out to a caller. A new
+  `sanitizeMetadata` helper (`src/batteries/vector/helpers.ts`) strips `__proto__`, `prototype`,
+  and `constructor` keys before each adapter's upsert path builds its stored record.
+- Audited the rest of the codebase for the same reachable-prototype-chain pattern — the
+  `data_structure` and `structured_data` tool batteries, the `apply_patch` media step, the
+  data-format `MediaEngine`, and `Registry`'s `dset`-backed path storage — and found no further
+  instances; everything else either only reads, or only ever writes to a freshly constructed
+  object.
+
+### Added
+
+- **`Tokenizable` accepts a dynamic evaluator, resolved at prompt-assembly time.** Alongside a plain
+  string, the constructor now takes a {@link TokenizableEvaluator} — a `(ctx?) => string` — so wrapped
+  content can compute itself coherent with the live `DispatchContext` it ships in (e.g. an instruction
+  that adapts to whether a tool survived the subtractive-context pass). Resolutions are cached
+  per-context in a `WeakMap` so repeated measures of the same dispatch (subtractive pass + overflow
+  guard) don't re-invoke the evaluator. A `render(ctx)` method is the explicit, context-aware read;
+  the standard string-coercion protocol (`toString`/`valueOf`/`toJSON`) resolves with no context, hitting
+  the evaluator's own `undefined`-branch fallback. An evaluator that throws or returns a non-string
+  raises the new `E_TOKENIZABLE_EVALUATOR_INVALID` — loud, no silent coercion. New `'gemma'` encoding
+  identifier for {@link Tokenizable.estimateTokens} (Gemma 2/3/4, backed by the same
+  `@lenml/tokenizer-gemini` SentencePiece vocabulary as `'gemini'` — deliberate reuse, distinct name).
+  Files: `src/lib/classes/tokenizable.ts`, `src/lib/exceptions/runtime.ts`.
+- **Token-estimation failures degrade instead of silently returning `Infinity`.** A real-tokenizer
+  failure (e.g. a special-token literal, an encoder bug) inside a `TurnRunner` run or `DispatchRunner`
+  dispatch now emits a `warning` and falls back to a char-based guesstimate, rather than the previous
+  silent `Number.POSITIVE_INFINITY` — which, combined with the overflow guards, could spuriously trip
+  an `E_*_CONTEXT_OVERFLOW` on ordinary text. Outside any runner execution, the failure still re-throws
+  (a genuine bug in non-runner code must surface). The ambient channel a runner publishes for the
+  duration of its run is `src/lib/utils/estimation_context.ts` (new) — a LIFO stack of warn-emit sinks
+  so a dispatch nested inside a turn routes to its own (richer) emitter.
+- **WebGPU memory observability for the on-device LLM batteries.** `probeGpuBudget()`
+  (`src/batteries/llm/chat_common/gpu_budget.ts`, new) reads the WebGPU adapter's buffer-size limits
+  and adapter info, non-invasively — observability only, allocates nothing. A new opt-in
+  `instrumentGpuBuffers()` wraps `GPUDevice.prototype.createBuffer` to track live/peak GPU buffer bytes
+  against that budget, for an application that wants a live "you're at X of Y GiB" gauge. Paired with a
+  new typed `E_LLM_GPU_OUT_OF_MEMORY` (`chat_common/exceptions.ts`, new) — `isGpuOutOfMemoryError()`
+  matches ORT-web's several GPU-exhaustion and WASM-linear-memory-exhaustion error signatures and both
+  the `transformers_js` and `litert_lm` batteries translate a raw provider throw into this one typed,
+  catchable error, surfaced via a non-fatal `ctx.nack(...)` rather than a throw. A new `gpuBudget` field
+  on the battery lifecycle report carries the probed snapshot. Consistent with the ADK's
+  surface-don't-impose stance: the batteries never auto-cap the caller's context window.
+- **Shared tool-call parser layer expanded and hardened.** The `gemma` parser is rewritten as a
+  string-aware balanced-brace scanner — correctly handles nested argument objects and the curly smart
+  quotes (`“…”`, `‘…’`) small models emit in place of ASCII quotes, instead of the previous lazy-regex
+  approach. Two new parser families: `'bare_pythonic'` and `'loose_keyed'` (`toolCallParser` /
+  `ToolCallParserName`). New observer seams on every LLM battery's options: `onRawGeneration` (the raw
+  model text for a completed generation, after envelope-stripping but before persistence — reasoning /
+  tool-call parser bring-up, live abstention debugging, fixture capture) and `onPromptAssembled` (the
+  fully-assembled request about to ship, the mirror tap on the way in). Both are purely observational,
+  default-absent, and consumed by all five LLM batteries (`chat_common/tool_parsers.ts`,
+  `chat_common/lifecycle.ts`, `chat_common/types.ts`).
+- **`litert_lm` battery: engine hosted in a disposable Web Worker.** New standalone worker build
+  configs — `litert-lm-worker.vite.config.mts` and `webllm-worker.vite.config.mts` — with matching
+  `build:litert-lm-worker` / `build:webllm-worker` package scripts, compiling the LiteRT-LM (IIFE,
+  classic-worker-compatible — LiteRT's Emscripten glue calls `importScripts()`, illegal in a module
+  worker) and WebLLM (ES module worker) engine handlers as separate bundles co-located with their wasm
+  assets, so a long-lived session can recover from a browser-level WebGPU device loss by terminating
+  and respawning the worker rather than reusing a dead `GPUAdapter`. The adapter's existing
+  `createEngine` injection seam (`LiteRtLmAdapterOptions.createEngine`) is what a host wires a
+  worker-backed engine through; the battery itself stays runtime-agnostic.
+- **`DispatchRunner` / `TurnRunner` emit a `WarningEvent`** observability payload — non-fatal
+  conditions (starting with the token-estimation degrade above) surfaced through the same observability
+  bus as `LogEvent` / `GenerationStatsEvent`, carrying `dispatchId`/`iteration`, a `source`, and a
+  `kind`. Executor-thrown and nacked errors also now preserve a meaningful `Error`-shaped `cause` even
+  when the thrown/nacked value is not itself a strict `Error` (a raw string or cross-realm error no
+  longer collapses to a cause-less generic wrapper) — `toErrorCause` in `src/lib/dispatch_runner.ts`.
+- **Documentation: the "Punching Above Its Weights" showcase family.** The flagship agent showcase
+  (`docs/showcase/punching-above-its-weights.md`) demonstrates building a real tool-using agent under
+  hostile conditions — Gemma-4 E2B via `LiteRtLmAdapter` in a browser tab, a 4GB GPU ceiling, a
+  live-draggable context window — technique by technique (planner book-end, subtractive pass over the
+  shipped context battery, gate cascade, artifact handles, GPU survival), each with real code embeds
+  and field-note receipts, closing on the blind-judged 5-cell evaluation matrix. Its companion
+  **"The Agent, In Full"** (`docs/showcase/punching-above-its-weights-source.md`) exposes the complete
+  31-file agent source in a read-only in-page Monaco viewer, with an LLM-consumable full-source
+  mirror emitted through the docs pipeline for coding agents to port from. Method-side pages:
+  **Token Thrift** (`docs/the-loop/token-thrift.md`, the context-discipline lever), **Behavioral
+  Rails** (`docs/the-loop/behavioral-rails.md`, gates/own-voice nudges/the planner contract), **Read
+  the Wire** (`docs/the-loop/read-the-wire.md`, evidence-directed agent debugging), and **Runtime
+  Loading** (`docs/assembly/runtime-loading.md`, the `@nhtio/adk/shims` consumption guide). The
+  underlying evaluation/research harness (corpus runs, floor calibration, adversarial threads, the
+  LiteRT worker Step-0 probe) is committed under `research/`.
+
+- **New context battery domain: `thrift` and `compact`, two strategies for what goes into one
+  dispatch's window.** `src/batteries/context/thrift` is the subtractive strategy already backing
+  the Token Thrift work above — `subtractToFit`, `stripPriorTurnThoughts`, and the calibrated
+  `selectRelevantTurns`/`scaledRelevanceFloor` relevance-based turn selection (floor constants
+  `RELEVANCE_FLOOR_MIN`/`MAX`/`CURVE` calibrated against a triple-oracle, 94-turn stress corpus) are
+  now a standalone, importable battery rather than flagship-agent-only code.
+  `src/batteries/context/compact` is new: a faithful extraction of the flagship agent's own
+  Claude-Code-style auto-compaction (`assembleCompactedTurns`, `summariseTurns`,
+  `COMPACTION_SYSTEM_PROMPT`) — keep the newest turns verbatim, fold everything older into a
+  rolling summary once it crosses a token threshold. Both batteries are built entirely on injected
+  resolvers rather than bundled capabilities: `EstimateTokensFn` (no default tokenizer) and, for
+  `compact`, `SummarizeFn` (no default model transport) — with **zero imports from `@nhtio/adk`
+  core** at the structural-contract layer (`WorkingMessage`, `WorkingMemory`, `WorkingRetrievable`,
+  and friends are locally-declared, duck-typed shapes a real core object satisfies structurally
+  without either side importing the other). This decoupling is practical against real models
+  because of the token-estimator registry added above ({@link registerTokenEstimator}) — a caller
+  can register a custom encoding's estimator without editing `Tokenizable`'s internal switch, so
+  `thrift`/`compact` work against any encoding a project uses, built-in or not. The two batteries
+  are composable: `thrift`'s `isSummaryMessage` predicate (default id `'__compact-summary'`,
+  matching `compact`'s `DEFAULT_SUMMARY_MESSAGE_ID`) protects `compact`'s rolling summary message
+  from being shed like an ordinary old turn when both run in the same pipeline. Evaluated
+  head-to-head against a naive-recency baseline across five model/window cells on a shared 94-turn
+  corpus: thrift is the lightest arm nearly everywhere and never collapses, while compact tops the
+  two cells where real context pressure meets a paid summarizer budget (kimi-k2.5 @ 128k, 1.48 vs.
+  1.13 3-judge; gemma-31b @ 128k, 2-judge pool pending its third judge) — documented in full,
+  including the naive baseline's 0.08 collapse on the kimi cell and per-cell dispatch/summarizer-
+  overhead tables, in the new `docs/batteries/context/` pages.
+
+- **`@nhtio/adk/shims` — an async-resolver seam for binding a runtime-loaded ADK bundle without
+  importing core into the consumer's module graph.** {@link createAdkShim} wraps a consumer-supplied
+  {@link AdkResolverFn} (all environment knowledge — `fetch` + dynamic `import()`, a Worker handshake,
+  a host-injected global — lives in that one function; the shim ships no loading policy of its own)
+  and returns `{ resolve, get, resolved, proxy }`: single-flight `resolve()`, a synchronous `get()` for
+  already-resolved reads, a live `resolved` boolean, and a `proxy` that replaces the hand-rolled
+  `export let Foo: typeof Module.Foo` holder pattern with one destructurable object. Memoization is
+  GC-safe — the resolved bundle is held via `WeakRef` (never strongly retained by the shim itself),
+  falling back to a plain strong reference only where `WeakRef` is unavailable. A module-scope ambient
+  variant (`registerAdkResolver` + `adk`) covers the "many files, one shared binding" case. Three typed
+  exceptions cover the failure modes: `E_SHIM_NOT_RESOLVED` (a sync read before anything resolved),
+  `E_SHIM_RESOLUTION_FAILED` (the resolver rejected or threw, cause preserved), and
+  `E_SHIM_RESOLVER_ALREADY_RESOLVED` (re-registering the ambient resolver after it already resolved
+  once — a split-brain guard). `src/shims/index.ts` is a **leaf module** — proven at dist level
+  (`shims.mjs`, 17.7KB) to import only the exceptions chunk plus `@nhtio/validation` and `fast-printf`,
+  zero core graph — and deliberately not re-exported from the root `@nhtio/adk` barrel, since doing so
+  would drag the very module graph this subpath exists to let you avoid back into the import. The docs
+  site itself now dogfoods this exact seam: `docs/.vitepress/theme/components/quickstart_demo_runtime.ts`
+  replaced its own four-times-hand-rolled memoizing loader (the one that exists because importing ADK
+  source into the VitePress module graph overflows the JS call stack on iOS WebKit) with
+  `createAdkShim(resolver)`, the resolver supplying only the docs app's URL-resolving policy.
+
+### Changed
+
+- **`@sqlite.org/sqlite-wasm` and `kysely` are now docs-site devDependencies** — used by the docs
+  site's in-browser SQLite demo tooling, never shipped in the published package. **`katex`** is now a
+  runtime `dependency` (previously absent) — it backs the math tools battery
+  (`src/batteries/tools/math/index.ts`).
+
 ## 2026-06-26
 
 ### Added
@@ -82,6 +245,10 @@ upgrading.
   (or the existing `--base-url` / `--api-key` flags), replacing internal-specific env-var names. Dev tool only;
   not part of the published runtime surface.
 
+### Fixed
+
+- Fixed the documentation release pipeline.
+
 ## 2026-06-25
 
 ### Added

package/batteries/context/compact/contracts.cjs

@@ -0,0 +1,2 @@
+Object.defineProperty(exports, Symbol.toStringTag, { value: "Module" });
+require("../../../chunk-Cek0wNdY.js");

package/batteries/context/compact/contracts.d.ts

@@ -0,0 +1,120 @@
+/**
+ * Structural contracts for the Compact (summarize-over-threshold) context-management battery — the
+ * seams it invokes via injected functions, and the one cost-observability shape it hands back.
+ *
+ * @module @nhtio/adk/batteries/context/compact/contracts
+ *
+ * @remarks
+ * This module has **zero imports** — not even a type-only import from `@nhtio/adk` core. Same
+ * discipline as {@link @nhtio/adk/batteries/context/thrift/contracts}: every capability this battery
+ * cannot perform itself is an INJECTED function, and every shape it reads or writes is a local
+ * structural declaration.
+ *
+ * Compact needs exactly TWO capabilities it cannot bundle itself:
+ *
+ * 1. **Summarization** ({@link SummarizeFn}) — an actual model call. Unlike Token Thrift, which is a
+ *    zero-model-call algorithm end-to-end, Compact's entire thesis is PAYING for a model call to
+ *    compress old turns into a running summary — so this seam is unavoidable, and the battery never
+ *    pretends otherwise.
+ * 2. **Token estimation** ({@link EstimateTokensFn}, re-exported from
+ *    {@link @nhtio/adk/batteries/context/thrift/contracts} — same signature, same rationale: no
+ *    bundled tokenizer, ever).
+ */
+export type { EstimateTokensFn } from "../thrift/contracts";
+/**
+ * THE model-call seam this battery cannot perform itself: given a system prompt and a body of text to
+ * summarize, return the summary text.
+ *
+ * @remarks
+ * The battery never names `DispatchRunner`, an adapter, or an executor — it has no idea how the
+ * caller talks to a model, and it must not. The canonical ADK recipe for building one (the recipe the
+ * production flagship agent this battery was extracted from actually uses, and the one the docs page
+ * for this battery walks through in full) is:
+ *
+ * ```ts
+ * const summarize: SummarizeFn = async ({ system, text }) => {
+ *   let out = ''
+ *   const noop = async (): Promise<void> => undefined
+ *   const noopArr = async (): Promise<never[]> => []
+ *   await DispatchRunner.dispatch({
+ *     raw: {
+ *       systemPrompt: new Tokenizable(system),
+ *       standingInstructions: [],
+ *       messages: [new Message({ id: `__compact-${Date.now()}`, role: 'user', content: text, createdAt: new Date().toISOString(), updatedAt: new Date().toISOString() })],
+ *       // Every other RawTurnContext capability is a noop — this dispatch never reads or writes any
+ *       // OTHER persisted state, it only wants the model's completion text.
+ *       fetchMemories: noopArr, fetchRetrievables: noopArr, fetchMessages: noopArr,
+ *       fetchThoughts: noopArr, fetchToolCalls: noopArr, fetchTools: noopArr,
+ *       refreshStandingInstructions: noopArr,
+ *       storeStandingInstruction: noop, mutateStandingInstruction: noop, deleteStandingInstruction: noop,
+ *       storeMemory: noop, mutateMemory: noop, deleteMemory: noop,
+ *       storeRetrievable: noop, mutateRetrievable: noop, deleteRetrievable: noop,
+ *       storeMessage: async (_c, v) => { out += v.content?.toString?.() ?? '' },
+ *       mutateMessage: noop, deleteMessage: noop,
+ *       storeThought: noop, mutateThought: noop, deleteThought: noop,
+ *       storeToolCall: noop, mutateToolCall: noop, deleteToolCall: noop,
+ *       storeMediaBytes: noop, storeRetrievableBytes: noop,
+ *     } as never,
+ *     executor: await adapter.executor({ maxTokens: 1200, toolCallParser: 'none', stream: false, autoAck: true, enableThinking: false }),
+ *   })
+ *   return out.trim()
+ * }
+ * ```
+ *
+ * A caller with a different transport (a direct HTTP call to an LLM API, a queue-backed worker, a
+ * mocked test double) supplies whatever function satisfies this signature — the battery only ever
+ * awaits the returned string.
+ *
+ * @param req - `system`: the compaction instructions (see {@link @nhtio/adk/batteries/context/compact/summarizer!COMPACTION_SYSTEM_PROMPT}
+ *   for the shipped default); `text`: the body of older conversation (and, on a rolling re-summarize,
+ *   the prior summary folded in — see {@link @nhtio/adk/batteries/context/compact/summarizer!summariseTurns}) to compress.
+ * @returns The summary text, awaited. A rejected promise propagates to the caller of
+ *   `summariseTurns`/`assembleCompactedTurns` uncaught — this battery does not swallow summarizer
+ *   failures; a caller wanting graceful degradation (e.g. "fall back to the prior summary") implements
+ *   that inside their own `SummarizeFn`, exactly as the production flagship agent's own summarizer did.
+ */
+export type SummarizeFn = (req: {
+    system: string;
+    text: string;
+}) => Promise<string>;
+/**
+ * One summarization call's estimated token cost — reported via {@link OnCostFn} so a caller can total
+ * up the real overhead this battery pays (the cost profile a `thrift`-only pipeline never incurs).
+ *
+ * @remarks
+ * `inTok`/`outTok` are ESTIMATES, not metered usage: when the injected {@link SummarizeFn}'s transport
+ * doesn't report real token usage back (many don't — a bare `Promise<string>` return carries no usage
+ * envelope), the battery has no other way to know what the call cost. It measures what it CAN measure
+ * — the exact text it sent (`system + text`) and the exact text it got back (the summary) — via the
+ * same injected {@link EstimateTokensFn} the caller already supplies for its own token accounting, so
+ * these numbers are at least self-consistent with whatever budget arithmetic the caller runs
+ * elsewhere. A caller whose `SummarizeFn` DOES have access to real usage counts can ignore this
+ * estimate and total its own numbers from inside the function instead.
+ */
+export interface CompactionCostEvent {
+    /** How many summarization calls this event represents — always `1` per {@link OnCostFn} firing (one
+     *  call, one event); a caller totals `calls` across events to get the run's summarizer call count. */
+    calls: number;
+    /** Estimated input token cost: `estimateTokens(system + text, encoding)` for the exact request sent
+     *  to {@link SummarizeFn} this call (system prompt plus the history/prior-summary text). */
+    inTok: number;
+    /** Estimated output token cost: `estimateTokens(summary, encoding)` for the text {@link SummarizeFn}
+     *  returned. */
+    outTok: number;
+}
+/**
+ * Optional per-summarization cost callback — fired once per {@link SummarizeFn} call that actually
+ * runs, replacing the production flagship agent's `globalThis.__agentCompactionCost` ring buffer with
+ * an injectable, testable seam that leaves no global state behind.
+ *
+ * @remarks
+ * Never invoked when the threshold hasn't been crossed (no summarization call made — see
+ * {@link @nhtio/adk/batteries/context/compact/summarizer!assembleCompactedTurns}'s below-threshold
+ * pass-through path). A caller who wants a running total across a whole session's worth of dispatches
+ * accumulates across `onCost` firings themselves; this battery holds no cross-call state of its own
+ * beyond what a single `assembleCompactedTurns` invocation's caller threads through (the rolling prior
+ * summary is passed in and returned explicitly, never stashed on `globalThis`).
+ *
+ * @param event - See {@link CompactionCostEvent}.
+ */
+export type OnCostFn = (event: CompactionCostEvent) => void;

package/batteries/context/compact/index.d.ts

@@ -0,0 +1,102 @@
+/**
+ * Compact — summarize-over-threshold context management. Instead of shedding old turns outright, pay
+ * for a model call to fold them into a running summary, trading a real API cost for retaining
+ * compressed signal from everything folded away.
+ *
+ * @module @nhtio/adk/batteries/context/compact
+ *
+ * @remarks
+ * ## The thesis
+ *
+ * Where {@link @nhtio/adk/batteries/context/thrift} is a zero-model-call subtractive algorithm — shed
+ * the lowest-signal content until the dispatch fits, never pay for compression — Compact takes the
+ * opposite bet: keep the most recent `keepVerbatim` turns in full, and once everything OLDER than that
+ * grows past a token threshold, pay for ONE summarization call to compress it into a running summary
+ * message that stands in for all of it. On the next threshold crossing, the prior summary is folded
+ * BACK into the next summarization request (a rolling summary), so detail degrades gracefully across
+ * many compactions rather than being recomputed from scratch or lost outright.
+ *
+ * This is a direct extraction of the flagship reference agent's production "compact" baseline — the
+ * faithful Claude Code auto-compaction schema, lifted verbatim (see
+ * {@link @nhtio/adk/batteries/context/compact/summarizer!COMPACTION_SYSTEM_PROMPT}) — evaluated
+ * HEAD-TO-HEAD against Token Thrift and a naive-recency baseline on a shared 94-turn stress corpus
+ * across five models.
+ *
+ * ## The evaluation (honest, both ways)
+ *
+ * - **The one win**: on a SINGLE reasoning-model cell, Compact beat Thrift 1.48 vs. 1.13 (3-judge
+ *   committee scoring) — a reasoning model made better use of a paid compression step than of
+ *   subtraction alone. Compact also led at the 128k-window control condition, 1.40 vs. Thrift's 1.27.
+ * - **The price**: ~80-89 summarizer calls and roughly 380k-550k EXTRA tokens over a 94-turn run,
+ *   compared to Thrift's zero. This is the fundamental trade-off, not an implementation detail —
+ *   Compact is a paid strategy by construction.
+ * - **Where it degenerates**: on a 1M-token window, the older-region threshold never triggered across
+ *   the whole run (0 summarizer calls) — with that much headroom, Compact never fires and behaves
+ *   identically to naive unbounded accumulation. It only earns its cost at real context pressure.
+ * - **Under a TIGHT budget, an honest caveat**: this extraction's message-shedding (whatever a caller
+ *   layers on top to fit the FINAL dispatch, e.g. Thrift's own subtractive pass running downstream)
+ *   is UNFAITHFUL to what real compaction products do under pressure — it drops verbatim recent turns
+ *   outright rather than triggering an earlier/tighter re-summarization. Real Claude-Code-style
+ *   compaction re-summarizes more aggressively as the wall approaches; a caller who needs that
+ *   fidelity under tight budgets should tune `summariseAtTokens` down rather than rely on a downstream
+ *   shed to save them.
+ *
+ * See {@link @nhtio/adk/batteries/context/thrift} for the full Thrift-side account of this same
+ * evaluation, including where Thrift wins (cost, everywhere; quality, almost everywhere).
+ *
+ * ## Choosing between the two
+ *
+ * Reach for `compact` when: the target model is a reasoning model, the pipeline can afford a
+ * summarizer budget (extra latency + tokens per compaction), and the window is small enough that the
+ * threshold will actually fire. Reach for `thrift` (the always-safe default) when: cost matters, the
+ * window is large enough that compaction would degenerate anyway, or the model doesn't specifically
+ * benefit from compressed-prose continuity over raw subtraction. The two are COMPOSABLE, not
+ * mutually exclusive — Thrift ships an `isSummaryMessage` predicate (default `id === '__compact-summary'`)
+ * that recognizes and protects Compact's synthetic summary message from being shed like an ordinary
+ * old turn, so a pipeline can run `compact` upstream (turn-level assembly) and `thrift` downstream
+ * (final budget fit) together. See
+ * {@link @nhtio/adk/batteries/context/compact/summarizer!DEFAULT_SUMMARY_MESSAGE_ID} for that
+ * contract's exact terms.
+ *
+ * ## Zero-coupling, one unavoidable seam
+ *
+ * Every structural type here is a local, duck-typed declaration (zero core imports — see
+ * {@link @nhtio/adk/batteries/context/compact/contracts}) except for the turn/message shapes, which
+ * are reused directly from {@link @nhtio/adk/batteries/context/thrift/relevance} via a relative,
+ * battery-internal import (compact operates on the SAME upstream turn shape thrift's relevance
+ * selection produces — no need to redeclare it). The one capability this battery truly cannot bundle
+ * is the model call itself: {@link @nhtio/adk/batteries/context/compact/contracts!SummarizeFn} is an
+ * injected function, never a bundled transport — see its TSDoc for the canonical ADK recipe (a
+ * `DispatchRunner.dispatch` call with an all-noop `RawTurnContext`).
+ *
+ * ## Usage sketch
+ *
+ * ```ts
+ * import { assembleCompactedTurns, type SummarizeFn } from '@nhtio/adk/batteries/context/compact'
+ * import { groupHistoryIntoTurns } from '@nhtio/adk/batteries/context/thrift'
+ * import { Tokenizable } from '@nhtio/adk'
+ *
+ * // A stub SummarizeFn — see the contract's TSDoc for the real DispatchRunner recipe.
+ * const summarize: SummarizeFn = async ({ system, text }) => {
+ *   const completion = await myLlmCall({ system, prompt: text })
+ *   return completion.trim()
+ * }
+ *
+ * const turns = groupHistoryIntoTurns(myMessages, myToolCalls)
+ * let priorState: { summary: string | null; coveredOlder: number } | undefined
+ *
+ * const result = await assembleCompactedTurns(turns, {
+ *   summarize,
+ *   estimateTokens: (value, encoding, ctx) => Tokenizable.estimateTokens(value, encoding, ctx),
+ *   priorState,
+ *   onCost: (event) => trackCompactionCost(event), // replaces globalThis.__agentCompactionCost
+ * })
+ *
+ * priorState = { summary: result.summary, coveredOlder: result.coveredOlder } // persist for next turn
+ * // result.turns: [syntheticSummaryTurn?, ...recentVerbatimTurns] — feed downstream (e.g. into
+ * // thrift's subtractToFit) exactly like any other grouped-turn history.
+ * ```
+ */
+export * from "./contracts";
+export * from "./summarizer";
+export { E_CONTEXT_RESOLVER_MISSING } from "../exceptions";

package/batteries/context/compact/summarizer.cjs

@@ -0,0 +1,201 @@
+Object.defineProperty(exports, Symbol.toStringTag, { value: "Module" });
+require("../../../chunk-Cek0wNdY.js");
+const require_batteries_context_exceptions = require("../exceptions.cjs");
+//#region src/batteries/context/compact/summarizer.ts
+/**
+* The Compact summarizer and history assembler — the whole Compact thesis, in code.
+*
+* @module @nhtio/adk/batteries/context/compact/summarizer
+*
+* @remarks
+* This is a direct extraction of the flagship reference agent's `#summariseTurns` /
+* `#assembleCompactedTurns` head-to-head baseline (the "compact" arm evaluated against Token Thrift —
+* see the battery barrel's TSDoc for the honest, both-ways evaluation results), retargeted at the
+* local structural contracts in {@link @nhtio/adk/batteries/context/compact/contracts} and the shared
+* {@link @nhtio/adk/batteries/context/thrift/relevance!HistoryTurn} shape so it couples to nothing in
+* `@nhtio/adk` core — the one true dependency this battery cannot avoid (an actual model call to
+* summarize) is an injected {@link SummarizeFn}, never a bundled transport.
+*/
+/**
+* How many of the MOST RECENT turns stay verbatim, never folded into the running summary —
+* coreference ("it", "that file") needs the immediately preceding turns present no matter what.
+*
+* @remarks
+* The flagship reference agent's own calibrated value (`COMPACT_KEEP_VERBATIM`), lifted unchanged.
+*/
+var DEFAULT_KEEP_VERBATIM = 2;
+/**
+* The token threshold, measured over the OLDER (non-verbatim) region's combined text, past which a
+* (re)summarization call fires.
+*
+* @remarks
+* The flagship reference agent's own calibrated value (`COMPACT_SUMMARISE_AT_TOKENS`), lifted
+* unchanged — modeled on Claude Code auto-compacting as it approaches the context limit (there, a
+* fraction of an 8k window).
+*/
+var DEFAULT_SUMMARISE_AT_TOKENS = 2500;
+/**
+* The default id assigned to the synthetic summary message {@link assembleCompactedTurns} emits.
+*
+* @remarks
+* This exact string is the cross-battery contract with Token Thrift: `thrift`'s
+* {@link @nhtio/adk/batteries/context/thrift/contracts!IsSummaryMessageFn} default predicate is
+* `id === '__compact-summary'` — a caller running `compact` upstream of `thrift` in the same pipeline
+* gets, for free, thrift's protection of this message from being shed like an ordinary old turn. If a
+* caller overrides `summaryMessageId` here, they must also override thrift's `isSummaryMessage` option
+* to match, or thrift will treat the running summary as just another shed-able message.
+*/
+var DEFAULT_SUMMARY_MESSAGE_ID = "__compact-summary";
+/**
+* The FAITHFUL Claude Code compaction schema — the 9 sections extracted VERBATIM from the flagship
+* reference agent's own real auto-compactions (~4700-token structured summaries observed across 28
+* compactions in that agent's development session). Using the real prompt (not an invented one) is
+* what makes this battery's evaluation honest: the compact strategy loses exactly the detail Claude
+* Code's own compaction loses, no more and no less.
+*
+* @remarks
+* The 9 sections: (1) Primary Request and Intent, (2) Key Technical Concepts, (3) Files and Code
+* Sections, (4) Errors and Fixes, (5) Problem Solving, (6) All User Messages, (7) Pending Tasks,
+* (8) Current Work, (9) Next Step. Callers may supply their own prompt via the `systemPrompt` option
+* on {@link assembleCompactedTurns} (and {@link summariseTurns} directly) — this default is a
+* calibrated starting point, not a hard requirement.
+*/
+var COMPACTION_SYSTEM_PROMPT = "Your task is to create a detailed summary of the conversation so far, paying close attention to the user's explicit requests and your previous actions. This summary will REPLACE the older conversation history, so it must capture every fact a later turn might need. Structure it under these sections:\n1. Primary Request and Intent — what the user is trying to accomplish, verbatim where possible.\n2. Key Technical Concepts — technologies, APIs, and terms discussed.\n3. Files and Code Sections — specific files/functions/values examined or referenced.\n4. Errors and Fixes — problems hit and how they were resolved.\n5. Problem Solving — decisions made and why.\n6. All User Messages — a list of every non-tool user message, to preserve intent.\n7. Pending Tasks — what remains to do.\n8. Current Work — what was happening most recently.\n9. Next Step — the immediate next action.\nBe precise and factual. Preserve exact names, paths, and values. Do not summarise away specifics.";
+/** The token encoding this battery measures cost under, by default (matches Token Thrift's own
+*  default — see {@link @nhtio/adk/batteries/context/thrift/subtractive_pass!DEFAULT_ENCODING}). */
+var DEFAULT_ENCODING = "cl100k_base";
+/** The character-length cap the flagship reference agent applied to the summarizer's input text
+*  (prior summary + older-turn text combined), to keep the summarizer dispatch itself bounded. Lifted
+*  unchanged from `#summariseTurns`'s `.slice(0, 12000)`. */
+var HISTORY_TEXT_CHAR_CAP = 12e3;
+/**
+* Run ONE summarization call: fold `priorSummary` (if any) and `historyText` into the request text
+* exactly as the flagship reference agent's `#summariseTurns` did, dispatch it through the injected
+* {@link SummarizeFn}, and report the estimated token cost via `options.onCost`.
+*
+* @remarks
+* **Faithfulness to the original**: this is a line-for-line port of `#summariseTurns`'s request
+* assembly and prompt shape, with exactly one deliberate behavioral change: the original SWALLOWED a
+* failed summarizer call and degraded to `priorSummary ?? ''` (never letting a summarizer error
+* propagate, since it ran inline in a chat turn that had to keep going); this battery instead lets a
+* rejected {@link SummarizeFn} promise propagate UNCAUGHT (see {@link SummarizeFn}'s own TSDoc) — a
+* battery has no chat-turn context to know whether "degrade silently" is the right failure mode for
+* a given caller, so it surfaces the error and lets the caller decide (their own `SummarizeFn` can
+* implement the original's degrade-on-failure behavior internally if desired, by catching there
+* instead). The original's GPU-OOM special-case rethrow is likewise the caller's concern now — it
+* lived inside the original's own adapter-specific `SummarizeFn` equivalent, not in the algorithm.
+*
+* @param historyText - The older-turn text to summarize (already assembled by the caller, e.g. from
+*   {@link assembleCompactedTurns}'s older-turns join).
+* @param priorSummary - The prior rolling summary, if one exists, folded into the request text ahead
+*   of `historyText` under a `PREVIOUS SUMMARY (extend/merge, do not lose facts)` header — `null` on
+*   the first summarization of a run.
+* @param options - See {@link SummariseTurnsOptions}.
+* @returns The new summary text, trimmed. Falls back to `priorSummary ?? ''` only when
+*   {@link SummarizeFn} resolves to an empty/whitespace-only string (never on rejection — see above).
+* @throws {@link @nhtio/adk/batteries/context/exceptions!E_CONTEXT_RESOLVER_MISSING} When
+*   `options.summarize` or `options.estimateTokens` is not a function.
+*/
+async function summariseTurns(historyText, priorSummary, options) {
+	if (typeof options?.summarize !== "function") throw new require_batteries_context_exceptions.E_CONTEXT_RESOLVER_MISSING(["summariseTurns", "summarize"]);
+	if (typeof options?.estimateTokens !== "function") throw new require_batteries_context_exceptions.E_CONTEXT_RESOLVER_MISSING(["summariseTurns", "estimateTokens"]);
+	const encoding = options.encoding ?? DEFAULT_ENCODING;
+	const system = options.systemPrompt ?? "Your task is to create a detailed summary of the conversation so far, paying close attention to the user's explicit requests and your previous actions. This summary will REPLACE the older conversation history, so it must capture every fact a later turn might need. Structure it under these sections:\n1. Primary Request and Intent — what the user is trying to accomplish, verbatim where possible.\n2. Key Technical Concepts — technologies, APIs, and terms discussed.\n3. Files and Code Sections — specific files/functions/values examined or referenced.\n4. Errors and Fixes — problems hit and how they were resolved.\n5. Problem Solving — decisions made and why.\n6. All User Messages — a list of every non-tool user message, to preserve intent.\n7. Pending Tasks — what remains to do.\n8. Current Work — what was happening most recently.\n9. Next Step — the immediate next action.\nBe precise and factual. Preserve exact names, paths, and values. Do not summarise away specifics.";
+	const text = ((priorSummary ? `PREVIOUS SUMMARY (extend/merge, do not lose facts):\n${priorSummary}\n\n---\n\n` : "") + `CONVERSATION TO SUMMARISE:\n${historyText}`).slice(0, HISTORY_TEXT_CHAR_CAP);
+	const summary = (await options.summarize({
+		system,
+		text
+	})).trim();
+	options.onCost?.({
+		calls: 1,
+		inTok: options.estimateTokens(`${system}\n\n${text}`, encoding),
+		outTok: options.estimateTokens(summary, encoding)
+	});
+	return summary || (priorSummary ?? "");
+}
+/**
+* The Compact assembly step: keep the newest `keepVerbatim` turns in full; fold everything OLDER into
+* a running structured summary, (re)generated via {@link summariseTurns} when the older region grows
+* past `summariseAtTokens` AND new turns have aged into it since the last summarization. Returns
+* `[syntheticSummaryTurn, ...recentVerbatimTurns]`.
+*
+* @remarks
+* **Faithfulness to the original** (`#assembleCompactedTurns`): the threshold/keep-verbatim/rolling
+* logic is a line-for-line port — same re-summarize condition (`older.length > coveredOlder &&
+* (summary === null || olderTokens > summariseAtTokens)`), same synthetic turn shape (`qa` = the
+* summary text, `createdAt` = the epoch-zero sentinel that sorts before every real turn, one message
+* with `id: summaryMessageId`, content prefixed `[Earlier conversation, compacted summary]`, no tool
+* calls), same "nothing to compact yet" early return when there's no older region at all. The ONE
+* deviation: state that lived on private class fields (`#compactSummary`, `#compactCoveredOlder`) in
+* the original is now explicit input/output (`options.priorState` in, `{ summary, coveredOlder }` out)
+* — a pure battery function cannot own mutable instance state, so the caller threads it through. This
+* is a mechanical decoupling change, not a behavioral one: a caller who persists and re-supplies
+* `priorState` exactly as returned reproduces the original's stateful behavior turn-for-turn.
+*
+* Unlike Token Thrift, this function CANNOT resurface the exact older detail a later turn needs — it
+* only has the summary's blurred prose. See the module barrel for the honest, both-ways evaluation of
+* that fidelity trade-off against thrift's subtractive approach.
+*
+* @param turns - The full grouped history, chronological (e.g. from
+*   {@link @nhtio/adk/batteries/context/thrift/relevance!groupHistoryIntoTurns}).
+* @param options - See {@link AssembleCompactedTurnsOptions}.
+* @returns See {@link AssembleCompactedTurnsResult}.
+* @throws {@link @nhtio/adk/batteries/context/exceptions!E_CONTEXT_RESOLVER_MISSING} When
+*   `options.summarize` or `options.estimateTokens` is not a function.
+*/
+async function assembleCompactedTurns(turns, options) {
+	if (typeof options?.summarize !== "function") throw new require_batteries_context_exceptions.E_CONTEXT_RESOLVER_MISSING(["assembleCompactedTurns", "summarize"]);
+	if (typeof options?.estimateTokens !== "function") throw new require_batteries_context_exceptions.E_CONTEXT_RESOLVER_MISSING(["assembleCompactedTurns", "estimateTokens"]);
+	const encoding = options.encoding ?? DEFAULT_ENCODING;
+	const keepVerbatim = options.keepVerbatim ?? 2;
+	const summariseAtTokens = options.summariseAtTokens ?? 2500;
+	const summaryMessageId = options.summaryMessageId ?? "__compact-summary";
+	let summary = options.priorState?.summary ?? null;
+	let coveredOlder = options.priorState?.coveredOlder ?? 0;
+	const recentStart = Math.max(0, turns.length - keepVerbatim);
+	const older = turns.slice(0, recentStart);
+	const recent = turns.slice(recentStart);
+	if (older.length === 0) return {
+		turns: [...recent],
+		summary,
+		coveredOlder
+	};
+	const olderText = older.map((t) => t.qa).join("\n\n");
+	const olderTokens = options.estimateTokens(olderText, encoding);
+	if (older.length > coveredOlder && (summary === null || olderTokens > summariseAtTokens)) {
+		summary = await summariseTurns(olderText, summary, {
+			summarize: options.summarize,
+			estimateTokens: options.estimateTokens,
+			encoding,
+			systemPrompt: options.systemPrompt,
+			onCost: options.onCost
+		});
+		coveredOlder = older.length;
+	}
+	const summaryText = summary ?? olderText;
+	const sentinelCreatedAt = (/* @__PURE__ */ new Date(0)).toISOString();
+	return {
+		turns: [{
+			qa: summaryText,
+			createdAt: sentinelCreatedAt,
+			messages: [{
+				id: summaryMessageId,
+				role: "user",
+				content: `[Earlier conversation, compacted summary]\n${summaryText}`,
+				createdAt: sentinelCreatedAt
+			}],
+			toolCalls: []
+		}, ...recent],
+		summary,
+		coveredOlder
+	};
+}
+//#endregion
+exports.COMPACTION_SYSTEM_PROMPT = COMPACTION_SYSTEM_PROMPT;
+exports.DEFAULT_KEEP_VERBATIM = DEFAULT_KEEP_VERBATIM;
+exports.DEFAULT_SUMMARISE_AT_TOKENS = DEFAULT_SUMMARISE_AT_TOKENS;
+exports.DEFAULT_SUMMARY_MESSAGE_ID = DEFAULT_SUMMARY_MESSAGE_ID;
+exports.assembleCompactedTurns = assembleCompactedTurns;
+exports.summariseTurns = summariseTurns;
+
+//# sourceMappingURL=summarizer.cjs.map