mcp-context-optimizer 0.1.4 → 0.1.6

package/README.md

@@ -225,12 +225,12 @@ impossible-budget             C_optimizer                79        58      27%
 required facts fully retained in every scenario: YES
 budget compliance: 9/9 scenarios within budget
 attribution integrity (all source tags resolve): YES
-avg reduction: C=37%  B=21%
-avg retrieval precision(C)=72%  recall(C)=100%
+avg reduction: C=42%  B=21%
+avg retrieval precision(C)=71%  recall(C)=100%
 ```
 
 Candidate C retains 100% of required facts in all 9 scenarios, stays within budget, and
-achieves 37% average reduction (vs 21% for simple dedupe+truncation). On tight budgets
+achieves 42% average reduction (vs 21% for simple dedupe+truncation). On tight budgets
 it preserves mandatory constraints while Baseline B loses them.
 
 ## Testing
@@ -248,3 +248,127 @@ similar-but-non-identical errors, required-fact preservation, source-reference
 integrity, stable ranking, empty/malformed input, impossible budgets, MCP tool
 validation, and local-only operation (no network).
 
+
+## Custom tokenizers
+
+By default, token counts use the `chars/4` heuristic (deterministic, zero dependencies).
+For production use with a specific model, plug in a real tokenizer:
+
+```ts
+import { ContextOptimizer, type TokenEstimator } from 'mcp-context-optimizer';
+import { encoding_for_model } from 'tiktoken'; // npm install tiktoken (optional dep)
+
+const enc = encoding_for_model('gpt-4');
+const tokenizer: TokenEstimator = {
+  name: 'tiktoken/gpt-4',
+  estimate: (text: string) => enc.encode(text).length,
+};
+
+const optimizer = new ContextOptimizer({
+  tokenBudget: 12_000,
+  tokenEstimator: tokenizer,
+});
+```
+
+The `estimate` function must be **deterministic** — the same string must always
+return the same number. This keeps `buildContext` output byte-identical for
+identical inputs, which is required for caching and reproducibility.
+
+See `src/tokens.ts` for the `TokenEstimator` interface and the built-in
+`defaultTokenEstimator` / `createTokenEstimator` helpers.
+
+## Versioning & SemVer
+
+`mcp-context-optimizer` follows [SemVer](https://semver.org/).
+
+| Component | Stability | Breaking change policy |
+| --- | --- | --- |
+| Public TypeScript API (`src/index.ts` exports) | Stable | Minor version bump |
+| Rendered context text format | Stable | Minor version bump |
+| `OptimizationReport` JSON shape | Stable | Minor version bump |
+| `PIPELINE_VERSION` constant | Internal | Patch may change if output format changes |
+| MCP tool names / arguments | Stable | Minor version bump |
+| Scoring weight defaults | Internal | Patch may change (re-optimize after upgrade) |
+
+`PIPELINE_VERSION` is incremented whenever the pipeline changes in a way that
+affects output (e.g. new compaction stage, new scoring signal). It is part of
+the config hash so that cached results are invalidated automatically.
+
+Current version: `0.1.x` — pre-1.0. Breaking changes may land in minor releases
+until `1.0.0`. After `1.0.0`, breaking changes require a major version bump.
+
+## Comparison with alternatives
+
+| Approach | Deterministic | Local-only | Zero deps | Structured signals | Budget control | Provenance |
+| --- | --- | --- | --- | --- | --- | --- |
+| **mcp-context-optimizer** | ✅ Yes | ✅ Yes | ✅ Yes | ✅ Paths/symbols/stacks/errors | ✅ Hard budget + mandatory | ✅ Every fragment tagged |
+| LLM summarization (external API) | ❌ No | ❌ No | ❌ No | ⚠️ Partial | ⚠️ Soft / best-effort | ❌ None |
+| Simple truncation (head/tail) | ✅ Yes | ✅ Yes | ✅ Yes | ❌ None | ✅ Yes | ❌ None |
+| Semantic chunking (embeddings) | ⚠️ Model-dependent | ❌ Usually remote | ❌ Heavy deps | ❌ None | ⚠️ Soft | ❌ None |
+| RAG / vector retrieval | ⚠️ Approximate | ❌ Usually remote | ❌ Heavy deps | ❌ None | ✅ Yes | ⚠️ Source docs only |
+
+Key differences:
+
+- **No model calls.** Most "context optimizers" are just wrappers around another
+  LLM that summarizes your context — expensive, slow, non-deterministic, and
+  sends your code to a third party. This package does everything locally with
+  pure functions.
+- **Structured, not blind.** It extracts file paths, symbols, stack frames, test
+  failures, and explicit requirements before scoring, so a 200-line stack trace
+  is reduced to its relevant frames rather than chopped at an arbitrary line.
+- **Mandatory facts are never silently dropped.** If required content exceeds
+  the budget, the conflict is reported (`BUDGET_CONFLICT`) instead of silently
+  losing evidence.
+- **Full audit trail.** Every retained fragment carries a `[source: ...]` tag,
+  and the machine-readable report lists every merge, omission, and score
+  component.
+
+## MCP client integration examples
+
+### Claude Code
+
+```bash
+claude mcp add mcp-context-optimizer -- node /absolute/path/to/dist/mcp/cli.js
+```
+
+Then in Claude Code, the agent can call `build_context`, `search_context`,
+`get_decisions`, `get_relevant_errors`, `get_file_history`, and
+`ingest_records` directly.
+
+### Cursor / Claude Desktop (`mcp.json`)
+
+```json
+{
+  "mcpServers": {
+    "mcp-context-optimizer": {
+      "command": "node",
+      "args": ["/absolute/path/to/mcp-context-optimizer/dist/mcp/cli.js"]
+    }
+  }
+}
+```
+
+### Continue.dev
+
+Add to `~/.continue/config.json`:
+
+```json
+{
+  "mcpServers": {
+    "mcp-context-optimizer": {
+      "command": "node",
+      "args": ["/absolute/path/to/mcp-context-optimizer/dist/mcp/cli.js"]
+    }
+  }
+}
+```
+
+**Important:** MCP tool availability does not automatically intercept model
+requests. The client/agent must explicitly call these tools (e.g. call
+`build_context` before answering). Automatic context replacement requires a
+native integration inside the coding agent.
+
+The MCP server stores data **in memory per process lifetime** — restarting the
+server starts from an empty store. Re-ingest records or implement your own
+`ContextStore` (see `src/store.ts`) for persistence.
+

package/dist/compact.d.ts

@@ -0,0 +1,70 @@
+/**
+ * Text compaction stage (Stage 3b).
+ *
+ * Reduces fragment text size without losing information needed for scoring,
+ * provenance, or task-solving. Applied AFTER extraction (so structured
+ * signals like stack frames and symbols are already captured) and BEFORE
+ * dedup (so compacted text improves similarity matching).
+ *
+ * All functions are pure and deterministic.
+ */
+import type { Fragment } from './types.js';
+export interface CompactOptions {
+    /** Maximum stack frames to keep per error fragment. */
+    maxStackFrames: number;
+    /** Maximum lines to keep from tool_call / test_result output. */
+    maxToolOutputLines: number;
+    /** Whether to strip blank lines and trailing whitespace from code files. */
+    compactCode: boolean;
+    /** Whether to truncate stack traces. */
+    compactErrors: boolean;
+    /** Whether to collapse verbose tool output. */
+    compactToolOutput: boolean;
+    /** Whether to trim boilerplate from conversational messages. */
+    compactMessages: boolean;
+}
+export declare const DEFAULT_COMPACT_OPTIONS: CompactOptions;
+export interface CompactResult {
+    fragments: Fragment[];
+    warnings: Array<{
+        code: 'COMPACT_TRUNCATED';
+        message: string;
+    }>;
+    /** Per-fragment chars saved by compaction. */
+    savings: Map<string, number>;
+}
+/**
+ * Compact code text: remove blank lines and trailing whitespace.
+ * Safe for source files — only removes formatting whitespace that does
+ * not change semantics (empty lines, trailing spaces/tabs).
+ */
+export declare function compactCode(text: string): string;
+/**
+ * Compact a stack trace: keep the first line (error message) and the top
+ * N application frames. Node-internal and library frames are filtered
+ * so only relevant frames are retained.
+ */
+export declare function compactStackTrace(text: string, maxFrames?: number): string;
+/**
+ * Compact tool output: collapse trailing whitespace and truncate long
+ * outputs while keeping the beginning and end (summary) for context.
+ */
+export declare function compactToolOutput(text: string, maxLines?: number): string;
+/**
+ * Compact general conversational text: collapse runs of blank lines to
+ * a single blank line and trim trailing whitespace per line.
+ */
+export declare function compactGeneralText(text: string): string;
+/**
+ * Apply kind-specific compaction to a single fragment.
+ * Returns a new fragment with compacted text (or the same fragment if
+ * no reduction occurred). All structured signals (extracted paths,
+ * symbols, stack frames) are preserved unchanged.
+ */
+export declare function compactFragment(fragment: Fragment, opts: CompactOptions): Fragment;
+/**
+ * Stage 3b: apply text compaction to all fragments.
+ * Returns new fragments (does not mutate input).
+ */
+export declare function compactStage(fragments: Fragment[], opts?: CompactOptions): CompactResult;
+//# sourceMappingURL=compact.d.ts.map

package/dist/compact.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"compact.d.ts","sourceRoot":"","sources":["../src/compact.ts"],"names":[],"mappings":"AAAA;;;;;;;;;GASG;AAEH,OAAO,KAAK,EAAE,QAAQ,EAAE,MAAM,YAAY,CAAC;AAM3C,MAAM,WAAW,cAAc;IAC7B,uDAAuD;IACvD,cAAc,EAAE,MAAM,CAAC;IACvB,iEAAiE;IACjE,kBAAkB,EAAE,MAAM,CAAC;IAC3B,4EAA4E;IAC5E,WAAW,EAAE,OAAO,CAAC;IACrB,wCAAwC;IACxC,aAAa,EAAE,OAAO,CAAC;IACvB,+CAA+C;IAC/C,iBAAiB,EAAE,OAAO,CAAC;IAC3B,gEAAgE;IAChE,eAAe,EAAE,OAAO,CAAC;CAC1B;AAED,eAAO,MAAM,uBAAuB,EAAE,cAOrC,CAAC;AAEF,MAAM,WAAW,aAAa;IAC5B,SAAS,EAAE,QAAQ,EAAE,CAAC;IACtB,QAAQ,EAAE,KAAK,CAAC;QAAE,IAAI,EAAE,mBAAmB,CAAC;QAAC,OAAO,EAAE,MAAM,CAAA;KAAE,CAAC,CAAC;IAChE,8CAA8C;IAC9C,OAAO,EAAE,GAAG,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;CAC9B;AAMD;;;;GAIG;AACH,wBAAgB,WAAW,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CAMhD;AAED;;;;GAIG;AACH,wBAAgB,iBAAiB,CAAC,IAAI,EAAE,MAAM,EAAE,SAAS,GAAE,MAAU,GAAG,MAAM,CAwB7E;AAED;;;GAGG;AACH,wBAAgB,iBAAiB,CAAC,IAAI,EAAE,MAAM,EAAE,QAAQ,GAAE,MAAW,GAAG,MAAM,CAe7E;AAED;;;GAGG;AACH,wBAAgB,kBAAkB,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM,CAKvD;AAMD;;;;;GAKG;AACH,wBAAgB,eAAe,CAAC,QAAQ,EAAE,QAAQ,EAAE,IAAI,EAAE,cAAc,GAAG,QAAQ,CA8ClF;AAMD;;;GAGG;AACH,wBAAgB,YAAY,CAC1B,SAAS,EAAE,QAAQ,EAAE,EACrB,IAAI,GAAE,cAAwC,GAC7C,aAAa,CAuBf"}

package/dist/compact.js

@@ -0,0 +1,174 @@
+/**
+ * Text compaction stage (Stage 3b).
+ *
+ * Reduces fragment text size without losing information needed for scoring,
+ * provenance, or task-solving. Applied AFTER extraction (so structured
+ * signals like stack frames and symbols are already captured) and BEFORE
+ * dedup (so compacted text improves similarity matching).
+ *
+ * All functions are pure and deterministic.
+ */
+export const DEFAULT_COMPACT_OPTIONS = {
+    maxStackFrames: 5,
+    maxToolOutputLines: 15,
+    compactCode: true,
+    compactErrors: true,
+    compactToolOutput: true,
+    compactMessages: true,
+};
+// ---------------------------------------------------------------------------
+// Individual compaction functions
+// ---------------------------------------------------------------------------
+/**
+ * Compact code text: remove blank lines and trailing whitespace.
+ * Safe for source files — only removes formatting whitespace that does
+ * not change semantics (empty lines, trailing spaces/tabs).
+ */
+export function compactCode(text) {
+    const lines = text.split('\n');
+    const result = lines
+        .map((l) => l.replace(/[ \t]+$/g, ''))
+        .filter((l) => l.trim().length > 0);
+    return result.join('\n');
+}
+/**
+ * Compact a stack trace: keep the first line (error message) and the top
+ * N application frames. Node-internal and library frames are filtered
+ * so only relevant frames are retained.
+ */
+export function compactStackTrace(text, maxFrames = 5) {
+    const lines = text.split('\n');
+    if (lines.length <= maxFrames + 2)
+        return text;
+    const errorLine = lines[0] ?? '';
+    const rest = lines.slice(1);
+    const kept = [];
+    let skipped = 0;
+    for (const line of rest) {
+        if (kept.length >= maxFrames) {
+            skipped++;
+            continue;
+        }
+        if (/node:internal|node_modules|webpack:\/\//.test(line)) {
+            skipped++;
+            continue;
+        }
+        kept.push(line);
+    }
+    if (skipped === 0)
+        return text;
+    const suffix = `... (${skipped} frame(s) omitted)`;
+    return [errorLine, ...kept, suffix].join('\n');
+}
+/**
+ * Compact tool output: collapse trailing whitespace and truncate long
+ * outputs while keeping the beginning and end (summary) for context.
+ */
+export function compactToolOutput(text, maxLines = 15) {
+    const lines = text.split('\n');
+    if (lines.length <= maxLines) {
+        return lines.map((l) => l.replace(/[ \t]+$/g, '')).join('\n');
+    }
+    const keepHead = Math.max(3, maxLines - 5);
+    const head = lines.slice(0, keepHead);
+    const tail = lines.slice(-5);
+    const omitted = lines.length - head.length - tail.length;
+    return [
+        ...head.map((l) => l.replace(/[ \t]+$/g, '')),
+        `... (${omitted} line(s) omitted)`,
+        ...tail.map((l) => l.replace(/[ \t]+$/g, '')),
+    ].join('\n');
+}
+/**
+ * Compact general conversational text: collapse runs of blank lines to
+ * a single blank line and trim trailing whitespace per line.
+ */
+export function compactGeneralText(text) {
+    return text
+        .replace(/[ \t]+$/gm, '')
+        .replace(/\n{3,}/g, '\n\n')
+        .trim();
+}
+// ---------------------------------------------------------------------------
+// Fragment-level compaction dispatcher
+// ---------------------------------------------------------------------------
+/**
+ * Apply kind-specific compaction to a single fragment.
+ * Returns a new fragment with compacted text (or the same fragment if
+ * no reduction occurred). All structured signals (extracted paths,
+ * symbols, stack frames) are preserved unchanged.
+ */
+export function compactFragment(fragment, opts) {
+    const original = fragment.text;
+    let compacted;
+    switch (fragment.kind) {
+        case 'file':
+            compacted = opts.compactCode ? compactCode(original) : original;
+            break;
+        case 'error':
+            compacted = opts.compactErrors
+                ? compactStackTrace(original, opts.maxStackFrames)
+                : original;
+            break;
+        case 'tool_call':
+            compacted = opts.compactToolOutput
+                ? compactToolOutput(original, opts.maxToolOutputLines)
+                : original;
+            break;
+        case 'test_result':
+            compacted = opts.compactToolOutput
+                ? compactToolOutput(original, opts.maxToolOutputLines)
+                : compactGeneralText(original);
+            break;
+        case 'message':
+        case 'note':
+            compacted = opts.compactMessages ? compactGeneralText(original) : original;
+            break;
+        case 'diff':
+            compacted = compactGeneralText(original);
+            break;
+        case 'decision':
+            compacted = original;
+            break;
+    }
+    if (compacted === original)
+        return fragment;
+    return {
+        ...fragment,
+        text: compacted,
+        meta: { ...fragment.meta, compacted: true },
+        provenance: {
+            ...fragment.provenance,
+            originalLength: original.length,
+        },
+    };
+}
+// ---------------------------------------------------------------------------
+// Stage entry point
+// ---------------------------------------------------------------------------
+/**
+ * Stage 3b: apply text compaction to all fragments.
+ * Returns new fragments (does not mutate input).
+ */
+export function compactStage(fragments, opts = DEFAULT_COMPACT_OPTIONS) {
+    const warnings = [];
+    const savings = new Map();
+    const result = [];
+    for (const f of fragments) {
+        const compacted = compactFragment(f, opts);
+        result.push(compacted);
+        const saved = f.text.length - compacted.text.length;
+        if (saved > 0) {
+            savings.set(f.id, saved);
+        }
+    }
+    const totalSaved = [...savings.values()].reduce((a, b) => a + b, 0);
+    if (totalSaved > 0) {
+        warnings.push({
+            code: 'COMPACT_TRUNCATED',
+            message: `Text compaction saved ~${Math.ceil(totalSaved / 4)} est. tokens across ${savings.size} fragment(s).`,
+        });
+    }
+    return { fragments: result, warnings, savings };
+}
+//# sourceMappingURL=compact.js.map

package/dist/compact.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"compact.js","sourceRoot":"","sources":["../src/compact.ts"],"names":[],"mappings":"AAAA;;;;;;;;;GASG;AAuBH,MAAM,CAAC,MAAM,uBAAuB,GAAmB;IACrD,cAAc,EAAE,CAAC;IACjB,kBAAkB,EAAE,EAAE;IACtB,WAAW,EAAE,IAAI;IACjB,aAAa,EAAE,IAAI;IACnB,iBAAiB,EAAE,IAAI;IACvB,eAAe,EAAE,IAAI;CACtB,CAAC;AASF,8EAA8E;AAC9E,kCAAkC;AAClC,8EAA8E;AAE9E;;;;GAIG;AACH,MAAM,UAAU,WAAW,CAAC,IAAY;IACtC,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAC/B,MAAM,MAAM,GAAG,KAAK;SACjB,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC;SACrC,MAAM,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC;IACtC,OAAO,MAAM,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AAC3B,CAAC;AAED;;;;GAIG;AACH,MAAM,UAAU,iBAAiB,CAAC,IAAY,EAAE,YAAoB,CAAC;IACnE,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAC/B,IAAI,KAAK,CAAC,MAAM,IAAI,SAAS,GAAG,CAAC;QAAE,OAAO,IAAI,CAAC;IAE/C,MAAM,SAAS,GAAG,KAAK,CAAC,CAAC,CAAC,IAAI,EAAE,CAAC;IACjC,MAAM,IAAI,GAAG,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;IAE5B,MAAM,IAAI,GAAa,EAAE,CAAC;IAC1B,IAAI,OAAO,GAAG,CAAC,CAAC;IAChB,KAAK,MAAM,IAAI,IAAI,IAAI,EAAE,CAAC;QACxB,IAAI,IAAI,CAAC,MAAM,IAAI,SAAS,EAAE,CAAC;YAC7B,OAAO,EAAE,CAAC;YACV,SAAS;QACX,CAAC;QACD,IAAI,yCAAyC,CAAC,IAAI,CAAC,IAAI,CAAC,EAAE,CAAC;YACzD,OAAO,EAAE,CAAC;YACV,SAAS;QACX,CAAC;QACD,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAClB,CAAC;IAED,IAAI,OAAO,KAAK,CAAC;QAAE,OAAO,IAAI,CAAC;IAC/B,MAAM,MAAM,GAAG,QAAQ,OAAO,oBAAoB,CAAC;IACnD,OAAO,CAAC,SAAS,EAAE,GAAG,IAAI,EAAE,MAAM,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AACjD,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,iBAAiB,CAAC,IAAY,EAAE,WAAmB,EAAE;IACnE,MAAM,KAAK,GAAG,IAAI,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC;IAC/B,IAAI,KAAK,CAAC,MAAM,IAAI,QAAQ,EAAE,CAAC;QAC7B,OAAO,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;IAChE,CAAC;IAED,MAAM,QAAQ,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,QAAQ,GAAG,CAAC,CAAC,CAAC;IAC3C,MAAM,IAAI,GAAG,KAAK,CAAC,KAAK,CAAC,CAAC,EAAE,QAAQ,CAAC,CAAC;IACtC,MAAM,IAAI,GAAG,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;IAC7B,MAAM,OAAO,GAAG,KAAK,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,GAAG,IAAI,CAAC,MAAM,CAAC;IACzD,OAAO;QACL,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC;QAC7C,QAAQ,OAAO,mBAAmB;QAClC,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC;KAC9C,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC;AACf,CAAC;AAED;;;GAGG;AACH,MAAM,UAAU,kBAAkB,CAAC,IAAY;IAC7C,OAAO,IAAI;SACR,OAAO,CAAC,WAAW,EAAE,EAAE,CAAC;SACxB,OAAO,CAAC,SAAS,EAAE,MAAM,CAAC;SAC1B,IAAI,EAAE,CAAC;AACZ,CAAC;AAED,8EAA8E;AAC9E,uCAAuC;AACvC,8EAA8E;AAE9E;;;;;GAKG;AACH,MAAM,UAAU,eAAe,CAAC,QAAkB,EAAE,IAAoB;IACtE,MAAM,QAAQ,GAAG,QAAQ,CAAC,IAAI,CAAC;IAC/B,IAAI,SAAiB,CAAC;IAEtB,QAAQ,QAAQ,CAAC,IAAI,EAAE,CAAC;QACtB,KAAK,MAAM;YACT,SAAS,GAAG,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,WAAW,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC;YAChE,MAAM;QACR,KAAK,OAAO;YACV,SAAS,GAAG,IAAI,CAAC,aAAa;gBAC5B,CAAC,CAAC,iBAAiB,CAAC,QAAQ,EAAE,IAAI,CAAC,cAAc,CAAC;gBAClD,CAAC,CAAC,QAAQ,CAAC;YACb,MAAM;QACR,KAAK,WAAW;YACd,SAAS,GAAG,IAAI,CAAC,iBAAiB;gBAChC,CAAC,CAAC,iBAAiB,CAAC,QAAQ,EAAE,IAAI,CAAC,kBAAkB,CAAC;gBACtD,CAAC,CAAC,QAAQ,CAAC;YACb,MAAM;QACR,KAAK,aAAa;YAChB,SAAS,GAAG,IAAI,CAAC,iBAAiB;gBAChC,CAAC,CAAC,iBAAiB,CAAC,QAAQ,EAAE,IAAI,CAAC,kBAAkB,CAAC;gBACtD,CAAC,CAAC,kBAAkB,CAAC,QAAQ,CAAC,CAAC;YACjC,MAAM;QACR,KAAK,SAAS,CAAC;QACf,KAAK,MAAM;YACT,SAAS,GAAG,IAAI,CAAC,eAAe,CAAC,CAAC,CAAC,kBAAkB,CAAC,QAAQ,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC;YAC3E,MAAM;QACR,KAAK,MAAM;YACT,SAAS,GAAG,kBAAkB,CAAC,QAAQ,CAAC,CAAC;YACzC,MAAM;QACR,KAAK,UAAU;YACb,SAAS,GAAG,QAAQ,CAAC;YACrB,MAAM;IACV,CAAC;IAED,IAAI,SAAS,KAAK,QAAQ;QAAE,OAAO,QAAQ,CAAC;IAE5C,OAAO;QACL,GAAG,QAAQ;QACX,IAAI,EAAE,SAAS;QACf,IAAI,EAAE,EAAE,GAAG,QAAQ,CAAC,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE;QAC3C,UAAU,EAAE;YACV,GAAG,QAAQ,CAAC,UAAU;YACtB,cAAc,EAAE,QAAQ,CAAC,MAAM;SAChC;KACF,CAAC;AACJ,CAAC;AAED,8EAA8E;AAC9E,oBAAoB;AACpB,8EAA8E;AAE9E;;;GAGG;AACH,MAAM,UAAU,YAAY,CAC1B,SAAqB,EACrB,OAAuB,uBAAuB;IAE9C,MAAM,QAAQ,GAA8B,EAAE,CAAC;IAC/C,MAAM,OAAO,GAAG,IAAI,GAAG,EAAkB,CAAC;IAC1C,MAAM,MAAM,GAAe,EAAE,CAAC;IAE9B,KAAK,MAAM,CAAC,IAAI,SAAS,EAAE,CAAC;QAC1B,MAAM,SAAS,GAAG,eAAe,CAAC,CAAC,EAAE,IAAI,CAAC,CAAC;QAC3C,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC;QACvB,MAAM,KAAK,GAAG,CAAC,CAAC,IAAI,CAAC,MAAM,GAAG,SAAS,CAAC,IAAI,CAAC,MAAM,CAAC;QACpD,IAAI,KAAK,GAAG,CAAC,EAAE,CAAC;YACd,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,KAAK,CAAC,CAAC;QAC3B,CAAC;IACH,CAAC;IAED,MAAM,UAAU,GAAG,CAAC,GAAG,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC,CAAC;IACpE,IAAI,UAAU,GAAG,CAAC,EAAE,CAAC;QACnB,QAAQ,CAAC,IAAI,CAAC;YACZ,IAAI,EAAE,mBAAmB;YACzB,OAAO,EAAE,0BAA0B,IAAI,CAAC,IAAI,CAAC,UAAU,GAAG,CAAC,CAAC,uBAAuB,OAAO,CAAC,IAAI,eAAe;SAC/G,CAAC,CAAC;IACL,CAAC;IAED,OAAO,EAAE,SAAS,EAAE,MAAM,EAAE,QAAQ,EAAE,OAAO,EAAE,CAAC;AAClD,CAAC"}

package/dist/index.d.ts

@@ -7,8 +7,9 @@
 export { ContextOptimizer, PIPELINE_VERSION } from './optimizer.js';
 export { defaultTokenEstimator, createTokenEstimator, sha256, shortHash, } from './tokens.js';
 export { DEFAULT_WEIGHTS, DECISION_RECENCY_FLOOR, tokenize, scoreStage } from './score.js';
-export { renderContext, sourceTag } from './render.js';
-export { selectStage, fragmentCost } from './select.js';
+export { renderContext, sourceTag, type RenderOptions } from './render.js';
+export { selectStage, fragmentCost, truncateText } from './select.js';
+export { compactStage, compactFragment, compactCode, compactStackTrace, compactToolOutput, compactGeneralText, DEFAULT_COMPACT_OPTIONS, type CompactOptions, type CompactResult } from './compact.js';
 export { dedupeStage, containmentSimilarity } from './dedupe.js';
 export { extractStage, extractSignals, extractPaths, extractSymbols, extractStackFrames, extractRequirements, } from './extract.js';
 export { normalizeStage, collectRecords, recordToFragment, normalizeText, collapseRepeatedLines, stableStringify, validateRecord, } from './normalize.js';

package/dist/index.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,MAAM,gBAAgB,CAAC;AACpE,OAAO,EACL,qBAAqB,EACrB,oBAAoB,EACpB,MAAM,EACN,SAAS,GACV,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,eAAe,EAAE,sBAAsB,EAAE,QAAQ,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAC3F,OAAO,EAAE,aAAa,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AACvD,OAAO,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AACxD,OAAO,EAAE,WAAW,EAAE,qBAAqB,EAAE,MAAM,aAAa,CAAC;AACjE,OAAO,EACL,YAAY,EACZ,cAAc,EACd,YAAY,EACZ,cAAc,EACd,kBAAkB,EAClB,mBAAmB,GACpB,MAAM,cAAc,CAAC;AACtB,OAAO,EACL,cAAc,EACd,cAAc,EACd,gBAAgB,EAChB,aAAa,EACb,qBAAqB,EACrB,eAAe,EACf,cAAc,GACf,MAAM,gBAAgB,CAAC;AACxB,OAAO,EAAE,oBAAoB,EAAE,YAAY,EAAE,eAAe,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC/F,YAAY,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAE/C,YAAY,EACV,iBAAiB,EACjB,cAAc,EACd,aAAa,EACb,cAAc,EACd,cAAc,EACd,UAAU,EACV,WAAW,EACX,cAAc,EACd,aAAa,EACb,oBAAoB,EACpB,gBAAgB,EAChB,eAAe,EACf,UAAU,EACV,SAAS,EACT,QAAQ,EACR,YAAY,EACZ,WAAW,EACX,aAAa,EACb,WAAW,EACX,WAAW,EACX,UAAU,EACV,eAAe,EACf,cAAc,EACd,gBAAgB,EAChB,gBAAgB,EAChB,kBAAkB,EAClB,WAAW,EACX,UAAU,EACV,UAAU,EACV,UAAU,EACV,cAAc,EACd,cAAc,EACd,cAAc,EACd,gBAAgB,EAChB,eAAe,EACf,SAAS,EACT,WAAW,EACX,gBAAgB,EAChB,cAAc,EACd,cAAc,EACd,OAAO,EACP,WAAW,GACZ,MAAM,YAAY,CAAC"}
+{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,MAAM,gBAAgB,CAAC;AACpE,OAAO,EACL,qBAAqB,EACrB,oBAAoB,EACpB,MAAM,EACN,SAAS,GACV,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,eAAe,EAAE,sBAAsB,EAAE,QAAQ,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAC3F,OAAO,EAAE,aAAa,EAAE,SAAS,EAAE,KAAK,aAAa,EAAE,MAAM,aAAa,CAAC;AAC3E,OAAO,EAAE,WAAW,EAAE,YAAY,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AACtE,OAAO,EAAE,YAAY,EAAE,eAAe,EAAE,WAAW,EAAE,iBAAiB,EAAE,iBAAiB,EAAE,kBAAkB,EAAE,uBAAuB,EAAE,KAAK,cAAc,EAAE,KAAK,aAAa,EAAE,MAAM,cAAc,CAAC;AACtM,OAAO,EAAE,WAAW,EAAE,qBAAqB,EAAE,MAAM,aAAa,CAAC;AACjE,OAAO,EACL,YAAY,EACZ,cAAc,EACd,YAAY,EACZ,cAAc,EACd,kBAAkB,EAClB,mBAAmB,GACpB,MAAM,cAAc,CAAC;AACtB,OAAO,EACL,cAAc,EACd,cAAc,EACd,gBAAgB,EAChB,aAAa,EACb,qBAAqB,EACrB,eAAe,EACf,cAAc,GACf,MAAM,gBAAgB,CAAC;AACxB,OAAO,EAAE,oBAAoB,EAAE,YAAY,EAAE,eAAe,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC/F,YAAY,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAE/C,YAAY,EACV,iBAAiB,EACjB,cAAc,EACd,aAAa,EACb,cAAc,EACd,cAAc,EACd,UAAU,EACV,WAAW,EACX,cAAc,EACd,aAAa,EACb,oBAAoB,EACpB,gBAAgB,EAChB,eAAe,EACf,UAAU,EACV,SAAS,EACT,QAAQ,EACR,YAAY,EACZ,WAAW,EACX,aAAa,EACb,WAAW,EACX,WAAW,EACX,UAAU,EACV,eAAe,EACf,cAAc,EACd,gBAAgB,EAChB,gBAAgB,EAChB,kBAAkB,EAClB,WAAW,EACX,UAAU,EACV,UAAU,EACV,UAAU,EACV,cAAc,EACd,cAAc,EACd,cAAc,EACd,gBAAgB,EAChB,eAAe,EACf,SAAS,EACT,WAAW,EACX,gBAAgB,EAChB,cAAc,EACd,cAAc,EACd,OAAO,EACP,WAAW,GACZ,MAAM,YAAY,CAAC"}

package/dist/index.js

@@ -8,7 +8,8 @@ export { ContextOptimizer, PIPELINE_VERSION } from './optimizer.js';
 export { defaultTokenEstimator, createTokenEstimator, sha256, shortHash, } from './tokens.js';
 export { DEFAULT_WEIGHTS, DECISION_RECENCY_FLOOR, tokenize, scoreStage } from './score.js';
 export { renderContext, sourceTag } from './render.js';
-export { selectStage, fragmentCost } from './select.js';
+export { selectStage, fragmentCost, truncateText } from './select.js';
+export { compactStage, compactFragment, compactCode, compactStackTrace, compactToolOutput, compactGeneralText, DEFAULT_COMPACT_OPTIONS } from './compact.js';
 export { dedupeStage, containmentSimilarity } from './dedupe.js';
 export { extractStage, extractSignals, extractPaths, extractSymbols, extractStackFrames, extractRequirements, } from './extract.js';
 export { normalizeStage, collectRecords, recordToFragment, normalizeText, collapseRepeatedLines, stableStringify, validateRecord, } from './normalize.js';

package/dist/index.js.map

@@ -1 +1 @@
-{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,MAAM,gBAAgB,CAAC;AACpE,OAAO,EACL,qBAAqB,EACrB,oBAAoB,EACpB,MAAM,EACN,SAAS,GACV,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,eAAe,EAAE,sBAAsB,EAAE,QAAQ,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAC3F,OAAO,EAAE,aAAa,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AACvD,OAAO,EAAE,WAAW,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AACxD,OAAO,EAAE,WAAW,EAAE,qBAAqB,EAAE,MAAM,aAAa,CAAC;AACjE,OAAO,EACL,YAAY,EACZ,cAAc,EACd,YAAY,EACZ,cAAc,EACd,kBAAkB,EAClB,mBAAmB,GACpB,MAAM,cAAc,CAAC;AACtB,OAAO,EACL,cAAc,EACd,cAAc,EACd,gBAAgB,EAChB,aAAa,EACb,qBAAqB,EACrB,eAAe,EACf,cAAc,GACf,MAAM,gBAAgB,CAAC;AACxB,OAAO,EAAE,oBAAoB,EAAE,YAAY,EAAE,eAAe,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC"}
+{"version":3,"file":"index.js","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AAAA;;;;;GAKG;AAEH,OAAO,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,MAAM,gBAAgB,CAAC;AACpE,OAAO,EACL,qBAAqB,EACrB,oBAAoB,EACpB,MAAM,EACN,SAAS,GACV,MAAM,aAAa,CAAC;AACrB,OAAO,EAAE,eAAe,EAAE,sBAAsB,EAAE,QAAQ,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAC3F,OAAO,EAAE,aAAa,EAAE,SAAS,EAAsB,MAAM,aAAa,CAAC;AAC3E,OAAO,EAAE,WAAW,EAAE,YAAY,EAAE,YAAY,EAAE,MAAM,aAAa,CAAC;AACtE,OAAO,EAAE,YAAY,EAAE,eAAe,EAAE,WAAW,EAAE,iBAAiB,EAAE,iBAAiB,EAAE,kBAAkB,EAAE,uBAAuB,EAA2C,MAAM,cAAc,CAAC;AACtM,OAAO,EAAE,WAAW,EAAE,qBAAqB,EAAE,MAAM,aAAa,CAAC;AACjE,OAAO,EACL,YAAY,EACZ,cAAc,EACd,YAAY,EACZ,cAAc,EACd,kBAAkB,EAClB,mBAAmB,GACpB,MAAM,cAAc,CAAC;AACtB,OAAO,EACL,cAAc,EACd,cAAc,EACd,gBAAgB,EAChB,aAAa,EACb,qBAAqB,EACrB,eAAe,EACf,cAAc,GACf,MAAM,gBAAgB,CAAC;AACxB,OAAO,EAAE,oBAAoB,EAAE,YAAY,EAAE,eAAe,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC"}

package/dist/optimizer.d.ts

@@ -1,5 +1,7 @@
+import { type CompactOptions } from './compact.js';
 import type { BuildContextInput, Fragment, InputRecord, OptimizerOptions, OptimizedContext, ScoringWeights, TokenEstimator } from './types.js';
-export declare const PIPELINE_VERSION = "1";
+/** Bumped when the pipeline changes in a way that affects output. */
+export declare const PIPELINE_VERSION = "1.1";
 interface ResolvedOptions {
     tokenBudget: number;
     estimator: TokenEstimator;
@@ -7,11 +9,14 @@ interface ResolvedOptions {
     recencyHalfLifeDays: number;
     nearDuplicateThreshold: number;
     maxFragments: number;
+    enableCompaction: boolean;
+    compactOptions: CompactOptions;
+    compactSourceTags: boolean;
 }
 /**
  * Deterministic context optimizer.
  *
- * Pipeline: normalize -> extract -> dedupe -> score -> select -> render.
+ * Pipeline: normalize -> extract -> compact -> dedupe -> score -> select -> render.
  * Identical inputs and configuration always produce identical outputs
  * (no wall-clock time, no randomness, stable tie-breaking).
  */

package/dist/optimizer.d.ts.map

@@ -1 +1 @@
-{"version":3,"file":"optimizer.d.ts","sourceRoot":"","sources":["../src/optimizer.ts"],"names":[],"mappings":"AAOA,OAAO,KAAK,EACV,iBAAiB,EACjB,QAAQ,EACR,WAAW,EACX,gBAAgB,EAChB,gBAAgB,EAEhB,cAAc,EACd,cAAc,EAEf,MAAM,YAAY,CAAC;AAEpB,eAAO,MAAM,gBAAgB,MAAM,CAAC;AAEpC,UAAU,eAAe;IACvB,WAAW,EAAE,MAAM,CAAC;IACpB,SAAS,EAAE,cAAc,CAAC;IAC1B,OAAO,EAAE,cAAc,CAAC;IACxB,mBAAmB,EAAE,MAAM,CAAC;IAC5B,sBAAsB,EAAE,MAAM,CAAC;IAC/B,YAAY,EAAE,MAAM,CAAC;CACtB;AAED;;;;;;GAMG;AACH,qBAAa,gBAAgB;IAC3B,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAkB;IACvC,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAS;gBAExB,OAAO,GAAE,gBAAqB;IAsB1C,+DAA+D;IAC/D,IAAI,MAAM,IAAI,QAAQ,CAAC,eAAe,CAAC,CAEtC;IAED;;;;OAIG;IACG,YAAY,CAAC,KAAK,EAAE,iBAAiB,GAAG,OAAO,CAAC,gBAAgB,CAAC;IAIvE;;;;;;OAMG;IACH,OAAO,CAAC,aAAa;IA8BrB;;;OAGG;IACH,MAAM,CAAC,KAAK,EAAE;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,WAAW,EAAE,CAAC;QAAC,KAAK,CAAC,EAAE,MAAM,CAAA;KAAE,GAAG,KAAK,CAAC;QAC/E,QAAQ,EAAE,QAAQ,CAAC;QACnB,KAAK,EAAE,WAAW,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC;KACvC,CAAC;IAoCF,mDAAmD;IACnD,gBAAgB,CAAC,KAAK,EAAE,iBAAiB,GAAG,gBAAgB;CAiH7D"}
+{"version":3,"file":"optimizer.d.ts","sourceRoot":"","sources":["../src/optimizer.ts"],"names":[],"mappings":"AAAA,OAAO,EAAyC,KAAK,cAAc,EAAE,MAAM,cAAc,CAAC;AAQ1F,OAAO,KAAK,EACV,iBAAiB,EACjB,QAAQ,EACR,WAAW,EACX,gBAAgB,EAChB,gBAAgB,EAEhB,cAAc,EACd,cAAc,EAEf,MAAM,YAAY,CAAC;AAEpB,qEAAqE;AACrE,eAAO,MAAM,gBAAgB,QAAQ,CAAC;AAEtC,UAAU,eAAe;IACvB,WAAW,EAAE,MAAM,CAAC;IACpB,SAAS,EAAE,cAAc,CAAC;IAC1B,OAAO,EAAE,cAAc,CAAC;IACxB,mBAAmB,EAAE,MAAM,CAAC;IAC5B,sBAAsB,EAAE,MAAM,CAAC;IAC/B,YAAY,EAAE,MAAM,CAAC;IACrB,gBAAgB,EAAE,OAAO,CAAC;IAC1B,cAAc,EAAE,cAAc,CAAC;IAC/B,iBAAiB,EAAE,OAAO,CAAC;CAC5B;AAED;;;;;;GAMG;AACH,qBAAa,gBAAgB;IAC3B,OAAO,CAAC,QAAQ,CAAC,IAAI,CAAkB;IACvC,OAAO,CAAC,QAAQ,CAAC,UAAU,CAAS;gBAExB,OAAO,GAAE,gBAAqB;IAmC1C,+DAA+D;IAC/D,IAAI,MAAM,IAAI,QAAQ,CAAC,eAAe,CAAC,CAEtC;IAED;;;;OAIG;IACG,YAAY,CAAC,KAAK,EAAE,iBAAiB,GAAG,OAAO,CAAC,gBAAgB,CAAC;IAIvE;;;;;;OAMG;IACH,OAAO,CAAC,aAAa;IA8BrB;;;OAGG;IACH,MAAM,CAAC,KAAK,EAAE;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,OAAO,CAAC,EAAE,WAAW,EAAE,CAAC;QAAC,KAAK,CAAC,EAAE,MAAM,CAAA;KAAE,GAAG,KAAK,CAAC;QAC/E,QAAQ,EAAE,QAAQ,CAAC;QACnB,KAAK,EAAE,WAAW,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC,CAAC;KACvC,CAAC;IAwCF,mDAAmD;IACnD,gBAAgB,CAAC,KAAK,EAAE,iBAAiB,GAAG,gBAAgB;CA6H7D"}

package/dist/optimizer.js

@@ -1,3 +1,4 @@
+import { compactStage, DEFAULT_COMPACT_OPTIONS } from './compact.js';
 import { dedupeStage } from './dedupe.js';
 import { extractStage } from './extract.js';
 import { collectRecords, normalizeStage } from './normalize.js';
@@ -5,11 +6,12 @@ import { renderContext } from './render.js';
 import { buildScoringContext, DEFAULT_WEIGHTS, scoreStage } from './score.js';
 import { selectStage } from './select.js';
 import { defaultTokenEstimator, sha256 } from './tokens.js';
-export const PIPELINE_VERSION = '1';
+/** Bumped when the pipeline changes in a way that affects output. */
+export const PIPELINE_VERSION = '1.1';
 /**
  * Deterministic context optimizer.
  *
- * Pipeline: normalize -> extract -> dedupe -> score -> select -> render.
+ * Pipeline: normalize -> extract -> compact -> dedupe -> score -> select -> render.
  * Identical inputs and configuration always produce identical outputs
  * (no wall-clock time, no randomness, stable tie-breaking).
  */
@@ -22,8 +24,18 @@ export class ContextOptimizer {
             estimator: options.tokenEstimator ?? defaultTokenEstimator,
             weights: { ...DEFAULT_WEIGHTS, ...options.weights },
             recencyHalfLifeDays: options.recencyHalfLifeDays ?? 7,
-            nearDuplicateThreshold: options.nearDuplicateThreshold ?? 0.92,
+            nearDuplicateThreshold: options.nearDuplicateThreshold ?? 0.85,
             maxFragments: options.maxFragments ?? 5000,
+            enableCompaction: options.enableCompaction ?? true,
+            compactOptions: {
+                maxStackFrames: options.maxStackFrames ?? DEFAULT_COMPACT_OPTIONS.maxStackFrames,
+                maxToolOutputLines: options.maxToolOutputLines ?? DEFAULT_COMPACT_OPTIONS.maxToolOutputLines,
+                compactCode: options.compactCode ?? DEFAULT_COMPACT_OPTIONS.compactCode,
+                compactErrors: options.compactErrors ?? DEFAULT_COMPACT_OPTIONS.compactErrors,
+                compactToolOutput: options.compactToolOutput ?? DEFAULT_COMPACT_OPTIONS.compactToolOutput,
+                compactMessages: options.compactMessages ?? DEFAULT_COMPACT_OPTIONS.compactMessages,
+            },
+            compactSourceTags: options.compactSourceTags ?? true,
         };
         this.configHash = sha256(JSON.stringify({
             v: PIPELINE_VERSION,
@@ -33,6 +45,9 @@ export class ContextOptimizer {
             recencyHalfLifeDays: this.opts.recencyHalfLifeDays,
             nearDuplicateThreshold: this.opts.nearDuplicateThreshold,
             maxFragments: this.opts.maxFragments,
+            enableCompaction: this.opts.enableCompaction,
+            compactOptions: this.opts.compactOptions,
+            compactSourceTags: this.opts.compactSourceTags,
         })).slice(0, 16);
     }
     /** Resolved configuration (read-only view for diagnostics). */
@@ -86,6 +101,10 @@ export class ContextOptimizer {
         const limit = input.limit ?? 10;
         const normalized = normalizeStage(input.records ?? []);
         let fragments = extractStage(normalized.fragments);
+        if (this.opts.enableCompaction) {
+            const comp = compactStage(fragments, this.opts.compactOptions);
+            fragments = comp.fragments;
+        }
         if (fragments.length > this.opts.maxFragments) {
             fragments = fragments.slice(0, this.opts.maxFragments);
         }
@@ -117,8 +136,14 @@ export class ContextOptimizer {
         const rawRecords = collectRecords(input);
         const normalized = normalizeStage(rawRecords);
         warnings.push(...normalized.warnings);
-        // Extraction (before dedup so dedupe can rely on signals if needed).
+        // Stage 2: extraction.
         let fragments = extractStage(normalized.fragments);
+        // Stage 3: text compaction (after extraction so signals are preserved).
+        if (this.opts.enableCompaction) {
+            const comp = compactStage(fragments, this.opts.compactOptions);
+            fragments = comp.fragments;
+            warnings.push(...comp.warnings);
+        }
         // Deterministic fragment cap.
         if (fragments.length > this.opts.maxFragments) {
             fragments = fragments.slice(0, this.opts.maxFragments);
@@ -163,7 +188,9 @@ export class ContextOptimizer {
         });
         warnings.push(...selection.warnings);
         // Stage 6: render + report.
-        const context = renderContext(input.task, selection.selected);
+        const context = renderContext(input.task, selection.selected, {
+            compactSourceTags: this.opts.compactSourceTags,
+        });
         const estimatedOutput = this.opts.estimator.estimate(context);
         const estimatedInput = this.opts.estimator.estimate(normalized.records.map((r) => JSON.stringify(r)).join('\n'));
         const reductionRatio = estimatedInput > 0 ? (estimatedInput - estimatedOutput) / estimatedInput : 0;
@@ -172,6 +199,9 @@ export class ContextOptimizer {
                 recordsIn: normalized.records.length,
                 fragmentsAfterExtraction: normalized.fragments.length,
                 fragmentsAfterDedup: deduped.fragments.length,
+                fragmentsAfterCompaction: this.opts.enableCompaction
+                    ? fragments.length
+                    : normalized.fragments.length,
                 fragmentsSelected: selection.selected.length,
                 fragmentsOmitted: selection.omitted.length,
             },

package/dist/optimizer.js.map

@@ -1 +1 @@
-{"version":3,"file":"optimizer.js","sourceRoot":"","sources":["../src/optimizer.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAC1C,OAAO,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,EAAE,cAAc,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;AAChE,OAAO,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AAC5C,OAAO,EAAE,mBAAmB,EAAE,eAAe,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAC9E,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAC1C,OAAO,EAAE,qBAAqB,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAa5D,MAAM,CAAC,MAAM,gBAAgB,GAAG,GAAG,CAAC;AAWpC;;;;;;GAMG;AACH,MAAM,OAAO,gBAAgB;IACV,IAAI,CAAkB;IACtB,UAAU,CAAS;IAEpC,YAAY,UAA4B,EAAE;QACxC,IAAI,CAAC,IAAI,GAAG;YACV,WAAW,EAAE,OAAO,CAAC,WAAW,IAAI,MAAM;YAC1C,SAAS,EAAE,OAAO,CAAC,cAAc,IAAI,qBAAqB;YAC1D,OAAO,EAAE,EAAE,GAAG,eAAe,EAAE,GAAG,OAAO,CAAC,OAAO,EAAE;YACnD,mBAAmB,EAAE,OAAO,CAAC,mBAAmB,IAAI,CAAC;YACrD,sBAAsB,EAAE,OAAO,CAAC,sBAAsB,IAAI,IAAI;YAC9D,YAAY,EAAE,OAAO,CAAC,YAAY,IAAI,IAAI;SAC3C,CAAC;QACF,IAAI,CAAC,UAAU,GAAG,MAAM,CACtB,IAAI,CAAC,SAAS,CAAC;YACb,CAAC,EAAE,gBAAgB;YACnB,WAAW,EAAE,IAAI,CAAC,IAAI,CAAC,WAAW;YAClC,SAAS,EAAE,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI;YACnC,OAAO,EAAE,IAAI,CAAC,IAAI,CAAC,OAAO;YAC1B,mBAAmB,EAAE,IAAI,CAAC,IAAI,CAAC,mBAAmB;YAClD,sBAAsB,EAAE,IAAI,CAAC,IAAI,CAAC,sBAAsB;YACxD,YAAY,EAAE,IAAI,CAAC,IAAI,CAAC,YAAY;SACrC,CAAC,CACH,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;IACjB,CAAC;IAED,+DAA+D;IAC/D,IAAI,MAAM;QACR,OAAO,IAAI,CAAC,IAAI,CAAC;IACnB,CAAC;IAED;;;;OAIG;IACH,KAAK,CAAC,YAAY,CAAC,KAAwB;QACzC,OAAO,IAAI,CAAC,gBAAgB,CAAC,KAAK,CAAC,CAAC;IACtC,CAAC;IAED;;;;;;OAMG;IACK,aAAa,CAAC,SAAqB,EAAE,IAAY,EAAE,KAAa;QACtE,MAAM,GAAG,GAAG,mBAAmB,CAC7B,SAAS,EACT,IAAI,EACJ,KAAK,IAAI,SAAS,EAClB,SAAS,EACT,IAAI,CAAC,IAAI,CAAC,mBAAmB,EAC7B,IAAI,CAAC,IAAI,CAAC,OAAO,CAClB,CAAC;QACF,MAAM,EAAE,KAAK,EAAE,OAAO,EAAE,GAAG,GAAG,CAAC,WAAW,CAAC;QAC3C,OAAO,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;YACzB,IAAI,SAAS,GAAG,KAAK,CAAC;YACtB,IAAI,CAAC,CAAC,IAAI,KAAK,UAAU,IAAI,CAAC,CAAC,IAAI,EAAE,cAAc,KAAK,UAAU;gBAAE,SAAS,GAAG,IAAI,CAAC;YACrF,IAAI,CAAC,CAAC,SAAS,CAAC,YAAY,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,CAAC,KAAK,KAAK,SAAS,EAAE,CAAC;gBACjE,qEAAqE;gBACrE,+DAA+D;gBAC/D,MAAM,kBAAkB,GAAG,CAAC,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;gBAC5E,IAAI,kBAAkB;oBAAE,SAAS,GAAG,IAAI,CAAC;YAC3C,CAAC;YACD,IAAI,CAAC,CAAC,SAAS,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,CAAC,SAAS,CAAC,OAAO,EAAE,MAAM,KAAK,SAAS,EAAE,CAAC;gBAC/E,MAAM,WAAW,GACf,CAAC,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;oBAChD,CAAC,CAAC,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;oBACpD,CAAC,CAAC,CAAC,IAAI,EAAE,IAAI,IAAI,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC;gBAC/D,IAAI,WAAW;oBAAE,SAAS,GAAG,IAAI,CAAC;YACpC,CAAC;YACD,OAAO,SAAS,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;QACnD,CAAC,CAAC,CAAC;IACL,CAAC;IAED;;;OAGG;IACH,MAAM,CAAC,KAAiE;QAItE,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,IAAI,EAAE,CAAC;QAChC,MAAM,UAAU,GAAG,cAAc,CAAC,KAAK,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC;QACvD,IAAI,SAAS,GAAG,YAAY,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;QACnD,IAAI,SAAS,CAAC,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,YAAY,EAAE,CAAC;YAC9C,SAAS,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,IAAI,CAAC,YAAY,CAAC,CAAC;QACzD,CAAC;QACD,MAAM,OAAO,GAAG,WAAW,CAAC,SAAS,EAAE;YACrC,sBAAsB,EAAE,IAAI,CAAC,IAAI,CAAC,sBAAsB;SACzD,CAAC,CAAC;QACH,IAAI,YAAgC,CAAC;QACrC,KAAK,MAAM,CAAC,IAAI,UAAU,CAAC,OAAO,EAAE,CAAC;YACnC,IAAI,CAAC,CAAC,CAAC,SAAS;gBAAE,SAAS;YAC3B,MAAM,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;YAClC,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,YAAY,KAAK,SAAS,IAAI,CAAC,GAAG,YAAY,CAAC;gBAAE,YAAY,GAAG,CAAC,CAAC;QAC7F,CAAC;QACD,MAAM,GAAG,GAAG,mBAAmB,CAC7B,OAAO,CAAC,SAAS,EACjB,KAAK,CAAC,KAAK,EACX,SAAS,EACT,YAAY,EACZ,IAAI,CAAC,IAAI,CAAC,mBAAmB,EAC7B,IAAI,CAAC,IAAI,CAAC,OAAO,CAClB,CAAC;QACF,MAAM,MAAM,GAAG,UAAU,CAAC,OAAO,CAAC,SAAS,EAAE,GAAG,CAAC,CAAC;QAClD,OAAO,CAAC,GAAG,MAAM,CAAC;aACf,IAAI,CACH,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CACP,CAAC,CAAC,CAAC,KAAK,EAAE,KAAK,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,KAAK,EAAE,KAAK,IAAI,CAAC,CAAC;YAC7C,CAAC,CAAC,WAAW,GAAG,CAAC,CAAC,WAAW;YAC7B,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CACzB;aACA,KAAK,CAAC,CAAC,EAAE,KAAK,CAAC;aACf,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC,EAAE,QAAQ,EAAE,KAAK,EAAE,QAAQ,CAAC,KAAuC,EAAE,CAAC,CAAC,CAAC;IAChG,CAAC;IAED,mDAAmD;IACnD,gBAAgB,CAAC,KAAwB;QACvC,MAAM,QAAQ,GAAc,EAAE,CAAC;QAC/B,MAAM,MAAM,GAAG,KAAK,CAAC,WAAW,IAAI,IAAI,CAAC,IAAI,CAAC,WAAW,CAAC;QAE1D,iCAAiC;QACjC,MAAM,UAAU,GAAG,cAAc,CAAC,KAAK,CAAC,CAAC;QACzC,MAAM,UAAU,GAAG,cAAc,CAAC,UAAU,CAAC,CAAC;QAC9C,QAAQ,CAAC,IAAI,CAAC,GAAG,UAAU,CAAC,QAAQ,CAAC,CAAC;QAEtC,qEAAqE;QACrE,IAAI,SAAS,GAAG,YAAY,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;QAEnD,8BAA8B;QAC9B,IAAI,SAAS,CAAC,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,YAAY,EAAE,CAAC;YAC9C,SAAS,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,IAAI,CAAC,YAAY,CAAC,CAAC;YACvD,QAAQ,CAAC,IAAI,CAAC;gBACZ,IAAI,EAAE,sBAAsB;gBAC5B,OAAO,EAAE,4BAA4B,IAAI,CAAC,IAAI,CAAC,YAAY,iDAAiD;aAC7G,CAAC,CAAC;QACL,CAAC;QAED,0BAA0B;QAC1B,MAAM,OAAO,GAAG,WAAW,CAAC,SAAS,EAAE;YACrC,sBAAsB,EAAE,IAAI,CAAC,IAAI,CAAC,sBAAsB;SACzD,CAAC,CAAC;QACH,QAAQ,CAAC,IAAI,CAAC,GAAG,OAAO,CAAC,QAAQ,CAAC,CAAC;QAEnC,8DAA8D;QAC9D,IAAI,YAAgC,CAAC;QACrC,KAAK,MAAM,CAAC,IAAI,UAAU,CAAC,OAAO,EAAE,CAAC;YACnC,IAAI,CAAC,CAAC,CAAC,SAAS;gBAAE,SAAS;YAC3B,MAAM,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;YAClC,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,YAAY,KAAK,SAAS,IAAI,CAAC,GAAG,YAAY,CAAC;gBAAE,YAAY,GAAG,CAAC,CAAC;QAC7F,CAAC;QAED,yEAAyE;QACzE,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,IAAI,EAAE,CAAC;QAChC,MAAM,aAAa,GAAG,IAAI,CAAC,aAAa,CAAC,OAAO,CAAC,SAAS,EAAE,KAAK,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;QAE/E,8BAA8B;QAC9B,MAAM,UAAU,GAAG,mBAAmB,CACpC,aAAa,EACb,KAAK,CAAC,IAAI,EACV,KAAK,IAAI,SAAS,EAClB,YAAY,EACZ,IAAI,CAAC,IAAI,CAAC,mBAAmB,EAC7B,IAAI,CAAC,IAAI,CAAC,OAAO,CAClB,CAAC;QACF,MAAM,MAAM,GAAG,UAAU,CAAC,aAAa,EAAE,UAAU,CAAC,CAAC;QAErD,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,EAAE,CAAC;YAChD,QAAQ,CAAC,IAAI,CAAC;gBACZ,IAAI,EAAE,qBAAqB;gBAC3B,OAAO,EAAE,yBAAyB,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI,sGAAsG;aACjK,CAAC,CAAC;QACL,CAAC;QAED,mCAAmC;QACnC,MAAM,UAAU,GAAG,aAAa,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,EAAE,CAAC;QACpD,MAAM,UAAU,GAAG,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC;QAC5D,MAAM,SAAS,GAAG,WAAW,CAAC,MAAM,EAAE;YACpC,UAAU;YACV,MAAM;YACN,SAAS,EAAE,IAAI,CAAC,IAAI,CAAC,SAAS;SAC/B,CAAC,CAAC;QACH,QAAQ,CAAC,IAAI,CAAC,GAAG,SAAS,CAAC,QAAQ,CAAC,CAAC;QAErC,4BAA4B;QAC5B,MAAM,OAAO,GAAG,aAAa,CAAC,KAAK,CAAC,IAAI,EAAE,SAAS,CAAC,QAAQ,CAAC,CAAC;QAC9D,MAAM,eAAe,GAAG,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;QAC9D,MAAM,cAAc,GAAG,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,CACjD,UAAU,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAC5D,CAAC;QACF,MAAM,cAAc,GAClB,cAAc,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,cAAc,GAAG,eAAe,CAAC,GAAG,cAAc,CAAC,CAAC,CAAC,CAAC,CAAC;QAE/E,MAAM,MAAM,GAAuB;YACjC,MAAM,EAAE;gBACN,SAAS,EAAE,UAAU,CAAC,OAAO,CAAC,MAAM;gBACpC,wBAAwB,EAAE,UAAU,CAAC,SAAS,CAAC,MAAM;gBACrD,mBAAmB,EAAE,OAAO,CAAC,SAAS,CAAC,MAAM;gBAC7C,iBAAiB,EAAE,SAAS,CAAC,QAAQ,CAAC,MAAM;gBAC5C,gBAAgB,EAAE,SAAS,CAAC,OAAO,CAAC,MAAM;aAC3C;YACD,MAAM,EAAE;gBACN,SAAS,EAAE,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI;gBACnC,cAAc;gBACd,eAAe;gBACf,cAAc;aACf;YACD,MAAM,EAAE,SAAS,CAAC,MAAM;YACxB,MAAM,EAAE,OAAO,CAAC,MAAM;YACtB,SAAS,EAAE,SAAS,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;gBACvC,EAAE,EAAE,CAAC,CAAC,EAAE;gBACR,MAAM,EAAE,CAAC,CAAC,MAAM;gBAChB,GAAG,CAAC,CAAC,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;aACrD,CAAC,CAAC;YACH,QAAQ;YACR,UAAU,EAAE,IAAI,CAAC,UAAU;SAC5B,CAAC;QAEF,OAAO;YACL,OAAO;YACP,OAAO,EAAE,SAAS,CAAC,QAAQ;YAC3B,OAAO,EAAE,SAAS,CAAC,OAAO;YAC1B,MAAM;YACN,KAAK,EAAE;gBACL,oBAAoB,EAAE,cAAc;gBACpC,qBAAqB,EAAE,eAAe;gBACtC,cAAc;aACf;SACF,CAAC;IACJ,CAAC;CACF"}
+{"version":3,"file":"optimizer.js","sourceRoot":"","sources":["../src/optimizer.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,uBAAuB,EAAuB,MAAM,cAAc,CAAC;AAC1F,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAC1C,OAAO,EAAE,YAAY,EAAE,MAAM,cAAc,CAAC;AAC5C,OAAO,EAAE,cAAc,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;AAChE,OAAO,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AAC5C,OAAO,EAAE,mBAAmB,EAAE,eAAe,EAAE,UAAU,EAAE,MAAM,YAAY,CAAC;AAC9E,OAAO,EAAE,WAAW,EAAE,MAAM,aAAa,CAAC;AAC1C,OAAO,EAAE,qBAAqB,EAAE,MAAM,EAAE,MAAM,aAAa,CAAC;AAa5D,qEAAqE;AACrE,MAAM,CAAC,MAAM,gBAAgB,GAAG,KAAK,CAAC;AActC;;;;;;GAMG;AACH,MAAM,OAAO,gBAAgB;IACV,IAAI,CAAkB;IACtB,UAAU,CAAS;IAEpC,YAAY,UAA4B,EAAE;QACxC,IAAI,CAAC,IAAI,GAAG;YACV,WAAW,EAAE,OAAO,CAAC,WAAW,IAAI,MAAM;YAC1C,SAAS,EAAE,OAAO,CAAC,cAAc,IAAI,qBAAqB;YAC1D,OAAO,EAAE,EAAE,GAAG,eAAe,EAAE,GAAG,OAAO,CAAC,OAAO,EAAE;YACnD,mBAAmB,EAAE,OAAO,CAAC,mBAAmB,IAAI,CAAC;YACrD,sBAAsB,EAAE,OAAO,CAAC,sBAAsB,IAAI,IAAI;YAC9D,YAAY,EAAE,OAAO,CAAC,YAAY,IAAI,IAAI;YAC1C,gBAAgB,EAAE,OAAO,CAAC,gBAAgB,IAAI,IAAI;YAClD,cAAc,EAAE;gBACd,cAAc,EAAE,OAAO,CAAC,cAAc,IAAI,uBAAuB,CAAC,cAAc;gBAChF,kBAAkB,EAAE,OAAO,CAAC,kBAAkB,IAAI,uBAAuB,CAAC,kBAAkB;gBAC5F,WAAW,EAAE,OAAO,CAAC,WAAW,IAAI,uBAAuB,CAAC,WAAW;gBACvE,aAAa,EAAE,OAAO,CAAC,aAAa,IAAI,uBAAuB,CAAC,aAAa;gBAC7E,iBAAiB,EAAE,OAAO,CAAC,iBAAiB,IAAI,uBAAuB,CAAC,iBAAiB;gBACzF,eAAe,EAAE,OAAO,CAAC,eAAe,IAAI,uBAAuB,CAAC,eAAe;aACpF;YACD,iBAAiB,EAAE,OAAO,CAAC,iBAAiB,IAAI,IAAI;SACrD,CAAC;QACF,IAAI,CAAC,UAAU,GAAG,MAAM,CACtB,IAAI,CAAC,SAAS,CAAC;YACb,CAAC,EAAE,gBAAgB;YACnB,WAAW,EAAE,IAAI,CAAC,IAAI,CAAC,WAAW;YAClC,SAAS,EAAE,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI;YACnC,OAAO,EAAE,IAAI,CAAC,IAAI,CAAC,OAAO;YAC1B,mBAAmB,EAAE,IAAI,CAAC,IAAI,CAAC,mBAAmB;YAClD,sBAAsB,EAAE,IAAI,CAAC,IAAI,CAAC,sBAAsB;YACxD,YAAY,EAAE,IAAI,CAAC,IAAI,CAAC,YAAY;YACpC,gBAAgB,EAAE,IAAI,CAAC,IAAI,CAAC,gBAAgB;YAC5C,cAAc,EAAE,IAAI,CAAC,IAAI,CAAC,cAAc;YACxC,iBAAiB,EAAE,IAAI,CAAC,IAAI,CAAC,iBAAiB;SAC/C,CAAC,CACH,CAAC,KAAK,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;IACjB,CAAC;IAED,+DAA+D;IAC/D,IAAI,MAAM;QACR,OAAO,IAAI,CAAC,IAAI,CAAC;IACnB,CAAC;IAED;;;;OAIG;IACH,KAAK,CAAC,YAAY,CAAC,KAAwB;QACzC,OAAO,IAAI,CAAC,gBAAgB,CAAC,KAAK,CAAC,CAAC;IACtC,CAAC;IAED;;;;;;OAMG;IACK,aAAa,CAAC,SAAqB,EAAE,IAAY,EAAE,KAAa;QACtE,MAAM,GAAG,GAAG,mBAAmB,CAC7B,SAAS,EACT,IAAI,EACJ,KAAK,IAAI,SAAS,EAClB,SAAS,EACT,IAAI,CAAC,IAAI,CAAC,mBAAmB,EAC7B,IAAI,CAAC,IAAI,CAAC,OAAO,CAClB,CAAC;QACF,MAAM,EAAE,KAAK,EAAE,OAAO,EAAE,GAAG,GAAG,CAAC,WAAW,CAAC;QAC3C,OAAO,SAAS,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE;YACzB,IAAI,SAAS,GAAG,KAAK,CAAC;YACtB,IAAI,CAAC,CAAC,IAAI,KAAK,UAAU,IAAI,CAAC,CAAC,IAAI,EAAE,cAAc,KAAK,UAAU;gBAAE,SAAS,GAAG,IAAI,CAAC;YACrF,IAAI,CAAC,CAAC,SAAS,CAAC,YAAY,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,CAAC,KAAK,KAAK,SAAS,EAAE,CAAC;gBACjE,qEAAqE;gBACrE,+DAA+D;gBAC/D,MAAM,kBAAkB,GAAG,CAAC,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC;gBAC5E,IAAI,kBAAkB;oBAAE,SAAS,GAAG,IAAI,CAAC;YAC3C,CAAC;YACD,IAAI,CAAC,CAAC,SAAS,CAAC,MAAM,CAAC,MAAM,GAAG,CAAC,IAAI,CAAC,CAAC,SAAS,CAAC,OAAO,EAAE,MAAM,KAAK,SAAS,EAAE,CAAC;gBAC/E,MAAM,WAAW,GACf,CAAC,CAAC,SAAS,CAAC,KAAK,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;oBAChD,CAAC,CAAC,SAAS,CAAC,OAAO,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,CAAC;oBACpD,CAAC,CAAC,CAAC,IAAI,EAAE,IAAI,IAAI,KAAK,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,GAAG,CAAC,CAAC,CAAC,CAAC;gBAC/D,IAAI,WAAW;oBAAE,SAAS,GAAG,IAAI,CAAC;YACpC,CAAC;YACD,OAAO,SAAS,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,EAAE,SAAS,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC;QACnD,CAAC,CAAC,CAAC;IACL,CAAC;IAED;;;OAGG;IACH,MAAM,CAAC,KAAiE;QAItE,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,IAAI,EAAE,CAAC;QAChC,MAAM,UAAU,GAAG,cAAc,CAAC,KAAK,CAAC,OAAO,IAAI,EAAE,CAAC,CAAC;QACvD,IAAI,SAAS,GAAG,YAAY,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;QACnD,IAAI,IAAI,CAAC,IAAI,CAAC,gBAAgB,EAAE,CAAC;YAC/B,MAAM,IAAI,GAAG,YAAY,CAAC,SAAS,EAAE,IAAI,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC;YAC/D,SAAS,GAAG,IAAI,CAAC,SAAS,CAAC;QAC7B,CAAC;QACD,IAAI,SAAS,CAAC,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,YAAY,EAAE,CAAC;YAC9C,SAAS,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,IAAI,CAAC,YAAY,CAAC,CAAC;QACzD,CAAC;QACD,MAAM,OAAO,GAAG,WAAW,CAAC,SAAS,EAAE;YACrC,sBAAsB,EAAE,IAAI,CAAC,IAAI,CAAC,sBAAsB;SACzD,CAAC,CAAC;QACH,IAAI,YAAgC,CAAC;QACrC,KAAK,MAAM,CAAC,IAAI,UAAU,CAAC,OAAO,EAAE,CAAC;YACnC,IAAI,CAAC,CAAC,CAAC,SAAS;gBAAE,SAAS;YAC3B,MAAM,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;YAClC,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,YAAY,KAAK,SAAS,IAAI,CAAC,GAAG,YAAY,CAAC;gBAAE,YAAY,GAAG,CAAC,CAAC;QAC7F,CAAC;QACD,MAAM,GAAG,GAAG,mBAAmB,CAC7B,OAAO,CAAC,SAAS,EACjB,KAAK,CAAC,KAAK,EACX,SAAS,EACT,YAAY,EACZ,IAAI,CAAC,IAAI,CAAC,mBAAmB,EAC7B,IAAI,CAAC,IAAI,CAAC,OAAO,CAClB,CAAC;QACF,MAAM,MAAM,GAAG,UAAU,CAAC,OAAO,CAAC,SAAS,EAAE,GAAG,CAAC,CAAC;QAClD,OAAO,CAAC,GAAG,MAAM,CAAC;aACf,IAAI,CACH,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CACP,CAAC,CAAC,CAAC,KAAK,EAAE,KAAK,IAAI,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,KAAK,EAAE,KAAK,IAAI,CAAC,CAAC;YAC7C,CAAC,CAAC,WAAW,GAAG,CAAC,CAAC,WAAW;YAC7B,CAAC,CAAC,CAAC,EAAE,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CACzB;aACA,KAAK,CAAC,CAAC,EAAE,KAAK,CAAC;aACf,GAAG,CAAC,CAAC,QAAQ,EAAE,EAAE,CAAC,CAAC,EAAE,QAAQ,EAAE,KAAK,EAAE,QAAQ,CAAC,KAAuC,EAAE,CAAC,CAAC,CAAC;IAChG,CAAC;IAED,mDAAmD;IACnD,gBAAgB,CAAC,KAAwB;QACvC,MAAM,QAAQ,GAAc,EAAE,CAAC;QAC/B,MAAM,MAAM,GAAG,KAAK,CAAC,WAAW,IAAI,IAAI,CAAC,IAAI,CAAC,WAAW,CAAC;QAE1D,iCAAiC;QACjC,MAAM,UAAU,GAAG,cAAc,CAAC,KAAK,CAAC,CAAC;QACzC,MAAM,UAAU,GAAG,cAAc,CAAC,UAAU,CAAC,CAAC;QAC9C,QAAQ,CAAC,IAAI,CAAC,GAAG,UAAU,CAAC,QAAQ,CAAC,CAAC;QAEtC,uBAAuB;QACvB,IAAI,SAAS,GAAG,YAAY,CAAC,UAAU,CAAC,SAAS,CAAC,CAAC;QAEnD,wEAAwE;QACxE,IAAI,IAAI,CAAC,IAAI,CAAC,gBAAgB,EAAE,CAAC;YAC/B,MAAM,IAAI,GAAG,YAAY,CAAC,SAAS,EAAE,IAAI,CAAC,IAAI,CAAC,cAAc,CAAC,CAAC;YAC/D,SAAS,GAAG,IAAI,CAAC,SAAS,CAAC;YAC3B,QAAQ,CAAC,IAAI,CAAC,GAAG,IAAI,CAAC,QAAQ,CAAC,CAAC;QAClC,CAAC;QAED,8BAA8B;QAC9B,IAAI,SAAS,CAAC,MAAM,GAAG,IAAI,CAAC,IAAI,CAAC,YAAY,EAAE,CAAC;YAC9C,SAAS,GAAG,SAAS,CAAC,KAAK,CAAC,CAAC,EAAE,IAAI,CAAC,IAAI,CAAC,YAAY,CAAC,CAAC;YACvD,QAAQ,CAAC,IAAI,CAAC;gBACZ,IAAI,EAAE,sBAAsB;gBAC5B,OAAO,EAAE,4BAA4B,IAAI,CAAC,IAAI,CAAC,YAAY,iDAAiD;aAC7G,CAAC,CAAC;QACL,CAAC;QAED,0BAA0B;QAC1B,MAAM,OAAO,GAAG,WAAW,CAAC,SAAS,EAAE;YACrC,sBAAsB,EAAE,IAAI,CAAC,IAAI,CAAC,sBAAsB;SACzD,CAAC,CAAC;QACH,QAAQ,CAAC,IAAI,CAAC,GAAG,OAAO,CAAC,QAAQ,CAAC,CAAC;QAEnC,8DAA8D;QAC9D,IAAI,YAAgC,CAAC;QACrC,KAAK,MAAM,CAAC,IAAI,UAAU,CAAC,OAAO,EAAE,CAAC;YACnC,IAAI,CAAC,CAAC,CAAC,SAAS;gBAAE,SAAS;YAC3B,MAAM,CAAC,GAAG,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC,SAAS,CAAC,CAAC;YAClC,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,YAAY,KAAK,SAAS,IAAI,CAAC,GAAG,YAAY,CAAC;gBAAE,YAAY,GAAG,CAAC,CAAC;QAC7F,CAAC;QAED,yEAAyE;QACzE,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,IAAI,EAAE,CAAC;QAChC,MAAM,aAAa,GAAG,IAAI,CAAC,aAAa,CAAC,OAAO,CAAC,SAAS,EAAE,KAAK,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;QAE/E,8BAA8B;QAC9B,MAAM,UAAU,GAAG,mBAAmB,CACpC,aAAa,EACb,KAAK,CAAC,IAAI,EACV,KAAK,IAAI,SAAS,EAClB,YAAY,EACZ,IAAI,CAAC,IAAI,CAAC,mBAAmB,EAC7B,IAAI,CAAC,IAAI,CAAC,OAAO,CAClB,CAAC;QACF,MAAM,MAAM,GAAG,UAAU,CAAC,aAAa,EAAE,UAAU,CAAC,CAAC;QAErD,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI,CAAC,QAAQ,CAAC,OAAO,CAAC,EAAE,CAAC;YAChD,QAAQ,CAAC,IAAI,CAAC;gBACZ,IAAI,EAAE,qBAAqB;gBAC3B,OAAO,EAAE,yBAAyB,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI,sGAAsG;aACjK,CAAC,CAAC;QACL,CAAC;QAED,mCAAmC;QACnC,MAAM,UAAU,GAAG,aAAa,KAAK,CAAC,IAAI,CAAC,IAAI,EAAE,EAAE,CAAC;QACpD,MAAM,UAAU,GAAG,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC;QAC5D,MAAM,SAAS,GAAG,WAAW,CAAC,MAAM,EAAE;YACpC,UAAU;YACV,MAAM;YACN,SAAS,EAAE,IAAI,CAAC,IAAI,CAAC,SAAS;SAC/B,CAAC,CAAC;QACH,QAAQ,CAAC,IAAI,CAAC,GAAG,SAAS,CAAC,QAAQ,CAAC,CAAC;QAErC,4BAA4B;QAC5B,MAAM,OAAO,GAAG,aAAa,CAAC,KAAK,CAAC,IAAI,EAAE,SAAS,CAAC,QAAQ,EAAE;YAC5D,iBAAiB,EAAE,IAAI,CAAC,IAAI,CAAC,iBAAiB;SAC/C,CAAC,CAAC;QACH,MAAM,eAAe,GAAG,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,OAAO,CAAC,CAAC;QAC9D,MAAM,cAAc,GAAG,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,CACjD,UAAU,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAC5D,CAAC;QACF,MAAM,cAAc,GAClB,cAAc,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,cAAc,GAAG,eAAe,CAAC,GAAG,cAAc,CAAC,CAAC,CAAC,CAAC,CAAC;QAE/E,MAAM,MAAM,GAAuB;YACjC,MAAM,EAAE;gBACN,SAAS,EAAE,UAAU,CAAC,OAAO,CAAC,MAAM;gBACpC,wBAAwB,EAAE,UAAU,CAAC,SAAS,CAAC,MAAM;gBACrD,mBAAmB,EAAE,OAAO,CAAC,SAAS,CAAC,MAAM;gBAC7C,wBAAwB,EAAE,IAAI,CAAC,IAAI,CAAC,gBAAgB;oBAClD,CAAC,CAAC,SAAS,CAAC,MAAM;oBAClB,CAAC,CAAC,UAAU,CAAC,SAAS,CAAC,MAAM;gBAC/B,iBAAiB,EAAE,SAAS,CAAC,QAAQ,CAAC,MAAM;gBAC5C,gBAAgB,EAAE,SAAS,CAAC,OAAO,CAAC,MAAM;aAC3C;YACD,MAAM,EAAE;gBACN,SAAS,EAAE,IAAI,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI;gBACnC,cAAc;gBACd,eAAe;gBACf,cAAc;aACf;YACD,MAAM,EAAE,SAAS,CAAC,MAAM;YACxB,MAAM,EAAE,OAAO,CAAC,MAAM;YACtB,SAAS,EAAE,SAAS,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC;gBACvC,EAAE,EAAE,CAAC,CAAC,EAAE;gBACR,MAAM,EAAE,CAAC,CAAC,MAAM;gBAChB,GAAG,CAAC,CAAC,CAAC,KAAK,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,CAAC,CAAC,KAAK,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;aACrD,CAAC,CAAC;YACH,QAAQ;YACR,UAAU,EAAE,IAAI,CAAC,UAAU;SAC5B,CAAC;QAEF,OAAO;YACL,OAAO;YACP,OAAO,EAAE,SAAS,CAAC,QAAQ;YAC3B,OAAO,EAAE,SAAS,CAAC,OAAO;YAC1B,MAAM;YACN,KAAK,EAAE;gBACL,oBAAoB,EAAE,cAAc;gBACpC,qBAAqB,EAAE,eAAe;gBACtC,cAAc;aACf;SACF,CAAC;IACJ,CAAC;CACF"}

package/dist/render.d.ts

@@ -1,10 +1,29 @@
 import type { Fragment, SelectedFragment } from './types.js';
-/** Compact source tag kept with every rendered fragment. */
-export declare function sourceTag(f: Fragment): string;
+/** Default options for source tags. */
+export interface SourceTagOptions {
+    /** When true, emit a compact tag with just the fragment id. */
+    compact: boolean;
+}
+/**
+ * Compact source tag kept with every rendered fragment.
+ *
+ * Full format:  `[source: fr_xxx | r0 | src/path.ts | +N]`
+ * Compact format: `[source: fr_xxx]`  (just the id; provenance in report)
+ *
+ * The full format includes the record id, file path (when available), and
+ * merged count so the tag is self-describing inline. The compact format
+ * minimises per-fragment overhead when budget is tight.
+ */
+export declare function sourceTag(f: Fragment, opts?: Partial<SourceTagOptions>): string;
+/** Rendering options. */
+export interface RenderOptions {
+    /** Use compact source tags (shorter provenance markers). */
+    compactSourceTags?: boolean;
+}
 /**
  * Stage 6: render the selected fragments into the compact context text.
  * Sections appear in a fixed order; fragments inside sections follow input
  * order. The task is rendered as the header and is always present.
  */
-export declare function renderContext(task: string, selected: SelectedFragment[]): string;
+export declare function renderContext(task: string, selected: SelectedFragment[], opts?: RenderOptions): string;
 //# sourceMappingURL=render.d.ts.map