omnius 1.0.591 → 1.0.592

package/docs/duplicate-calls-root-cause-analysis.md

@@ -0,0 +1,91 @@
+# Duplicate Tool Calls — Root Cause Analysis
+
+## The Symptom
+
+The agent repeatedly calls the same tool with the same arguments (e.g., `file_read` on the same path, `grep_search` with the same pattern, `shell` with the same command). This wastes turns, burns context, and creates visible "looping" behavior.
+
+## The Core Problem: Reactive Dedup, Not Preventive
+
+The entire dedup infrastructure is **post-hoc interception** — the model has *already decided* to make the duplicate call before any dedup logic runs. The architecture looks like:
+
+```
+Model emits tool_call → Critic inspects → serve_cached / reject → (maybe) proactivePrune later
+```
+
+The model never sees the Critic's decision *before* it chooses to call. It only sees the *result* of a prior call in context, which doesn't prevent it from re-requesting the same information.
+
+### Why the Model Re-calls
+
+1. **Context bloat buries prior results.** By turn 15+, the model's context contains thousands of tokens of tool results. A `file_read` result from turn 3 is buried under 12 turns of other content. The model can't find it, so it calls again.
+
+2. **No "what I already know" summary.** The context is a raw conversation transcript. There's no structured "known facts" section that the model can reference instead of re-reading files. The model treats the context as a chat log, not a knowledge base.
+
+3. **proactivePrune makes it worse.** When `proactivePrune` replaces old results with `"[deduped — see turn N]"` or `"[file_read aged out, summary: ...]"`, it *removes the information the model needed* while telling it the info exists somewhere it can't access. The model's response: call the tool again to get the actual content.
+
+4. **Critic's serve_cached is invisible to the model.** When the Critic serves a cached result, the model sees a tool result — it doesn't learn "I already asked this." Next turn, same context, same decision.
+
+## The Structural Issues in Context Engineering
+
+### 1. Conversation-as-Context is the Wrong Primitive
+
+The context is a linear `ChatMessage[]` — an exact transcript of every turn. This is the fundamental design error. Research (AgentFold, RECOMP, MemGPT) shows that **progressive summarization** or **working memory** architectures outperform raw transcript for long-horizon tasks.
+
+The agent should maintain a **working memory** (structured facts about the current task state) separate from the **episodic transcript** (raw conversation). The model should see the working memory as its primary context, with the transcript available but compressed.
+
+### 2. No Semantic Index of Prior Tool Results
+
+The `recentToolResults` map in the Critic is keyed by fingerprint (tool name + args hash). This is syntactic dedup — it catches exact re-calls but misses semantic duplicates (e.g., `file_read("foo.ts")` vs `file_read("foo.ts", offset=1, limit=50)` — same file, different args, same information need).
+
+A semantic index would map **information goals** to **prior results**: "I need the contents of foo.ts" → "you already read it at turn 5, here's a summary." The current system can't do this.
+
+### 3. The Dedup Sets Reset Per-Turn (Not Per-Task)
+
+All the `_injectedThisTurn` Sets (`_errorGuidanceInjected`, `_reflectionsInjectedThisTurn`, `_verifyHintInjectedThisTurn`, etc.) reset every turn. This means:
+- Turn N: model calls `file_read("foo.ts")` → result injected
+- Turn N+1: model calls `file_read("foo.ts")` again → no dedup fires (the per-turn set was cleared)
+
+The `dedupHitCount` in the Critic persists across turns, but it only triggers `serve_cached` — it doesn't *prevent* the model from deciding to call. The model still emits the tool call, wasting an LLM inference.
+
+### 4. The Model Sees Its Own Redundancy Too Late
+
+The Critic runs *after* the model has emitted a tool call. By then, the LLM inference cost is already spent. The model should see a **pre-call hint** like "you already read foo.ts at turn 5" *before* it generates the tool call. This would require injecting context about prior calls into the system prompt or last assistant message *before* the next LLM call.
+
+## What Would Fix It
+
+### Short-term (minimal changes)
+
+1. **Inject a "recently read files" summary into the system prompt each turn.** Before each LLM call, scan the conversation for `file_read` results and inject a compact list: "Files already read: foo.ts (turn 5), bar.ts (turn 8)." This gives the model the information it needs to avoid re-reading.
+
+2. **Don't age-out file_read results that are still relevant.** `proactivePrune` should check if the file was modified since last read (via git status or mtime) before replacing the result. If unchanged, keep the full result — the model needs it.
+
+3. **Make `serve_cached` results visually distinct.** When the Critic serves a cached result, prefix it with `[CACHED — you already called this at turn N. Do not call again.]` so the model learns not to re-request.
+
+### Medium-term (architectural)
+
+4. **Working memory layer.** Maintain a structured "known facts" document that the model sees as part of its system context. Updated after each tool call. This is the MemGPT / Generative Agents approach.
+
+5. **Pre-call context injection.** Before each LLM call, analyze the last few tool calls and inject warnings about likely duplicates: "You called file_read on foo.ts 2 turns ago. If you need the same content, reference that result instead of calling again."
+
+6. **Semantic dedup.** Instead of fingerprint-based dedup, use embedding similarity to detect when a new tool call is semantically equivalent to a prior one (different args, same information goal).
+
+### Long-term (fundamental)
+
+7. **Replace conversation-as-context with state-as-context.** The model should see a task state object (current files, known issues, pending todos, recent errors) rather than a raw transcript. The transcript should be available for reference but not the primary context. This is the architecture shift from "chat agent" to "state machine agent."
+
+## Evidence from the Codebase
+
+The sheer number of REG-* patches (16, 17, 18, 26, 28, 31, 37, 38, 44, 45, 49b, 50) addressing symptoms of the same problem confirms this is a structural issue, not a surface bug. Each REG patch adds another per-turn Set or cooldown to suppress a specific duplicate/repetition pattern, but none address the root cause: **the model doesn't know what it already knows.**
+
+The `proactivePrune` function (lines 2960-3040) is the clearest evidence — it exists solely to clean up the symptom (bloated context from duplicate calls) without preventing the cause (the model making duplicate calls in the first place).
+
+## Summary
+
+| Layer | Current | Needed |
+|-------|---------|--------|
+| Dedup timing | Post-hoc (Critic after call) | Pre-hoc (inject known-info before call) |
+| Context model | Raw transcript | Working memory + compressed transcript |
+| Dedup scope | Syntactic (exact fingerprint) | Semantic (information goal) |
+| Prune strategy | Age-based removal | Relevance-based retention |
+| Model awareness | Sees tool results, not patterns | Sees "what I already know" summary |
+
+The duplicate calls aren't a bug — they're an emergent property of a context architecture that treats the LLM as a chat participant rather than a stateful agent.

package/docs/duplicate-calls-root-cause-deep.md

@@ -0,0 +1,155 @@
+# Duplicate Tool Calls — Deep Root Cause Analysis
+
+## The Problem
+
+The agent repeatedly calls the same tool with the same arguments right after having already called it. This is visible even in this agent's own conversation — `file_read` and `grep_search` calls get duplicated within 1-2 turns.
+
+## Root Cause: The Model Decides to Re-call BEFORE Any Dedup Can Stop It
+
+The fundamental issue is **temporal**: the model emits a tool call *before* any dedup logic runs. The architecture is:
+
+```
+LLM generates tool_call → Critic intercepts → serve_cached / reject → proactivePrune later
+```
+
+The model never sees "you already know this" *before* it decides to call. It only sees prior tool *results* buried in context, which it may not locate or trust.
+
+## The Critic: Post-Hoc Interception That Can't Prevent Re-calls
+
+**Code**: `packages/orchestrator/src/critic.ts` (lines 1-180)
+
+The Critic (`evaluate()` function, line 137) is the central dedup decision module. It runs **after** the model has already emitted a tool call. Its decision priority:
+
+1. **adversary guidance**: If the adversary flagged this fingerprint as redundant, attach a non-blocking critique and cached prior evidence; the requested tool call still executes.
+2. **serve_cached** (line 166): If the tool is "read-like" AND the fingerprint exists in `recentToolResults` AND hit count < threshold → return cached result with a soft warning
+3. **force_progress_block** (line 158): If hit count ≥ threshold (2 for shell, 3 for filesystem) → block the call entirely with a forced message
+4. **pass** (line 179): Otherwise, let the call through
+
+**Key insight**: Steps 2 and 3 both happen *after* the model already decided to call. The LLM inference cost is already spent. The model emitted `file_read("foo.ts")` — the Critic then says "you already have this" but the model never learns from this because:
+
+- `serve_cached` returns the cached result as a tool result — the model sees it as a normal response, not a "don't call again" signal
+- `force_progress_block` blocks the call but the model just sees an error message — it doesn't understand *why* it was blocked in context of its reasoning
+
+The `recentToolResults` Map (line 63) and `dedupHitCount` Map (line 65) persist across turns, but they only affect the Critic's decision — they don't inject any information into the model's context *before* the next LLM call.
+
+## Trigger 1: proactivePrune Removes Information the Model Still Needs
+
+**Code**: `packages/orchestrator/src/agenticRunner.ts:2973-3049`
+
+`proactivePrune` walks the message history and replaces old tool results with placeholders:
+
+- `"[deduped — same call as turn N]"` (line 2981) — for exact duplicate calls
+- `"[file_read aged out, summary: ...]"` (line 2982) — for file_read results older than 10 turns (line 2976: `AGED_FILE_READ_TURNS = 10`)
+- `"[shell succeeded, output pruned — ...]"` (line 2983) — for shell results older than 5 turns (line 2977: `AGED_SHELL_TURNS = 5`)
+
+**Why this causes re-calls**: When the model sees `"[file_read aged out, summary: ...]"`, it knows the file was read but can't see the actual content. The summary is typically truncated and insufficient for the model to reason about. The model's rational response: call `file_read` again to get the full content.
+
+This is a **self-reinforcing loop**:
+1. Model reads `foo.ts` → full content in context
+2. 10 turns later, proactivePrune replaces it with `"[file_read aged out, summary: ...]"`
+3. Model needs `foo.ts` content again → calls `file_read("foo.ts")` again
+4. proactivePrune will eventually prune this too → loop repeats
+
+## Trigger 2: No Pre-Call "What You Already Know" Injection
+
+**Code**: The LLM call path in `agenticRunner.ts` composes the `ChatMessage[]` array and sends it directly to the model. There is no step that injects a summary of prior tool results *before* the model generates its next response.
+
+The model sees:
+- System prompt (static)
+- Conversation transcript (raw `ChatMessage[]`)
+- No structured "known facts" or "recently read files" section
+
+Without a pre-call hint like "You already read foo.ts at turn 5 — content available in context", the model has no way to know it already has the information. It must scan the entire transcript to find prior results, which is unreliable for small models with long contexts.
+
+## Trigger 3: Per-Turn Dedup Sets Reset, Allowing Cross-Turn Duplicates
+
+**Code**: `agenticRunner.ts:1130-1171`
+
+The dedup system uses per-turn Sets that reset each turn:
+
+- `REG-37` (line 1130): per-turn dedup for verification-required hint
+- `REG-38` (line 1134): per-turn dedup for artifact-inspection critique injection
+- `REG-49b` (line 1171): per-loop-episode dedup flag for SSMA invocation
+
+This means:
+- Turn N: model calls `file_read("foo.ts")` → result injected, dedup Set records it
+- Turn N+1: dedup Set is cleared → model calls `file_read("foo.ts")` again → no dedup fires
+
+The `dedupHitCount` Map (line 5645) persists across turns, but it only triggers `serve_cached` in the Critic — it doesn't *prevent* the model from emitting the tool call. The LLM inference cost is already spent.
+
+## Trigger 4: Fingerprint-Only Dedup Misses Semantic Duplicates
+
+**Code**: `agenticRunner.ts:4489` (`_buildToolFingerprint`)
+
+The fingerprint is built from tool name + canonical args. This catches exact re-calls but misses:
+
+- `file_read("foo.ts")` vs `file_read("foo.ts", offset=1, limit=50)` — same file, different args, same information need
+- `grep_search("pattern", path="src")` vs `grep_search("pattern", path="src", include="*.ts")` — same search, different filter
+- `shell("cat foo.ts")` vs `file_read("foo.ts")` — different tool, same information goal
+
+The `recentToolResults` Map (line 5635) is keyed by fingerprint, so semantically equivalent calls with different fingerprints are treated as distinct — no dedup fires.
+
+## Why the 12+ REG-* Patches Don't Fix It
+
+Each REG patch adds another per-turn Set, cooldown, or fingerprint check. These are **symptom suppressors** — they intercept duplicates *after* the model decides to call, but they don't address why the model decides to call in the first place.
+
+| REG Patch | Location | What It Does | Why It Doesn't Fix Root Cause |
+|-----------|----------|-------------|-------------------------------|
+| REG-16 | line 5639 | Per-fingerprint dedup-hit counter, escalates to force_progress_block at ≥3 hits | Post-hoc: model already emitted the call |
+| REG-17 | critic.ts:102 | Shell threshold = 2 hits before block | Post-hoc: only blocks after 2 wasted inferences |
+| REG-18 | line 5691 | Stagnation window for variant-fatigue loops | Detects loops, doesn't prevent them |
+| REG-26 | line 5820 | Per-turn reflection-injection dedup | Resets per-turn, misses cross-turn patterns |
+| REG-31 | line 5874 | Positive completion signal injection | Doesn't address duplicate reads |
+| REG-37 | line 1130 | Per-turn dedup for verification hint | Resets per-turn |
+| REG-38 | line 1134 | Per-turn dedup for artifact-inspection | Resets per-turn |
+| REG-49b | line 1171 | Per-loop-episode dedup for SSMA | Episode-scoped, not task-scoped |
+
+## The Fix: Pre-Hoc Knowledge Injection
+
+### Short-term (minimal changes)
+
+1. **Inject a "recently read files" summary into the system prompt each turn.** Before each LLM call, scan the conversation for `file_read` results and inject: "Files already in context: foo.ts (turn 5), bar.ts (turn 8). Do not re-read these files."
+
+2. **Don't age-out file_read results that are still relevant.** `proactivePrune` should check if the file was modified since last read (via git status or mtime) before replacing the result. If unchanged, keep the full result.
+
+3. **Make cached results visually distinct.** When the Critic serves a cached result, prefix it with `[CACHED — you already called this at turn N. Do not call again.]`
+
+### Medium-term (architectural)
+
+4. **Working memory layer.** Maintain a structured "known facts" document that the model sees as part of its system context. Updated after each tool call. This is the MemGPT / Generative Agents approach.
+
+5. **Pre-call context injection.** Before each LLM call, analyze the last few tool calls and inject warnings: "You called file_read on foo.ts 2 turns ago. If you need the same content, reference that result instead of calling again."
+
+6. **Semantic dedup.** Instead of fingerprint-based dedup, use embedding similarity to detect when a new tool call is semantically equivalent to a prior one (different args, same information goal).
+
+### Long-term (fundamental)
+
+7. **Replace conversation-as-context with state-as-context.** The model should see a task state object (current files, known issues, pending todos, recent errors) rather than a raw transcript. The transcript should be available for reference but not the primary context.
+
+## Evidence
+
+| Symptom | Code Location | Mechanism |
+|---------|--------------|-----------|
+| Critic runs post-hoc | `critic.ts:137` (`evaluate()`) | Intercepts after model emits tool call |
+| serve_cached is invisible to model | `critic.ts:166-174` | Returns cached result as normal tool result |
+| force_progress_block is reactive | `critic.ts:158-164` | Blocks after ≥3 wasted inferences |
+| proactivePrune removes content | `agenticRunner.ts:2973-3049` | Replaces old results with `[file_read aged out, summary: ...]` |
+| Age thresholds too aggressive | `agenticRunner.ts:2976-2977` | 10 turns for files, 5 for shell |
+| No pre-call knowledge injection | LLM call path in agenticRunner | No step injects "what you already know" before model generates |
+| Per-turn dedup resets | `agenticRunner.ts:1130-1171` | REG-37/38/49b Sets clear each turn |
+| Fingerprint-only dedup | `agenticRunner.ts:4489` | Exact match only, misses semantic duplicates |
+| recentToolResults keyed by fingerprint | `agenticRunner.ts:5635-5638` | Same file different args = different entry |
+| dedupHitCount persists but reactive | `agenticRunner.ts:5645` | Only triggers serve_cached, not prevention |
+| 12+ REG patches | Various | All post-hoc interception, none pre-hoc prevention |
+
+## Summary
+
+The duplicate calls are an **emergent property of the conversation-as-context architecture**. The model doesn't know what it already knows because:
+
+1. **The Critic intercepts after the call** — the model already decided to re-call before any dedup logic runs (critic.ts:137)
+2. **Prior results get pruned away** — proactivePrune replaces content with summaries the model can't use (agenticRunner.ts:2973-3049)
+3. **No one tells the model what it already has** — no pre-call injection of "known facts" before the LLM generates
+4. **Dedup resets per-turn** — per-turn Sets allow cross-turn duplicates (agenticRunner.ts:1130-1171)
+5. **Dedup is syntactic only** — fingerprint matching misses semantic duplicates (agenticRunner.ts:4489)
+
+The fix is not more post-hoc interception — it's **pre-hoc knowledge injection**: tell the model what it already knows *before* it decides to call.

package/docs/ephemeral-skill-pack-small-context.md

@@ -0,0 +1,57 @@
+# Ephemeral Skill Pack Small-Context Plan
+
+Date: 2026-05-15
+
+## Problem
+
+AIWG and Omnius skills are valuable, but full `SKILL.md` bodies are too large for
+small and medium context windows. The main agent should see only a tiny,
+task-scoped skill manifest, then delegate full skill unpacking to a sub-agent
+that returns a compact extraction.
+
+## Design
+
+1. Build a run-scoped `ephemeral-skill-pack` from top-k task/skill matches.
+2. Inject only skill names, sources, short descriptions, first trigger, score,
+   and explicit discard semantics into `dynamicContext`.
+3. Give the main agent a `skill_extract` tool that loads full skill content out
+   of band and returns targeted guidance.
+4. When available, `skill_extract` delegates the full `SKILL.md` body to a
+   sub-agent and asks it to extract only the parts needed for the current task.
+5. If the sub-agent path is unavailable, fall back to deterministic section
+   extraction with a strict budget.
+6. Keep `skill_execute` for explicit full-skill execution, but prefer
+   `skill_extract` first on small and medium models.
+
+## Context Placement
+
+- TUI: append the manifest in `packages/cli/src/tui/interactive.ts` after the
+  existing dynamic context enrichments and before the `AgenticRunner` is built.
+- Runner: the manifest rides inside `c_know` via the existing structured
+  context assembly in `packages/orchestrator/src/agenticRunner.ts`.
+- Tool subset: include `skill_extract` in the runner's `skill` tool subset so
+  `tool_search("skill")` can promote it alongside `skill_list` and
+  `skill_execute`.
+
+## Current Implementation Checklist
+
+- [x] Add task/skill ranking helpers.
+- [x] Add `buildEphemeralSkillPack(...)`.
+- [x] Add deterministic `extractSkillForQuery(...)` fallback.
+- [x] Add `skill_extract` tool with sub-agent extraction callback support.
+- [x] Register `skill_extract` in the TUI tool list.
+- [x] Wire `skill_extract` to the existing sub-agent tool callback.
+- [x] Inject the ephemeral manifest into TUI `dynamicContext`.
+- [x] Add tests for selection, manifest shape, and deterministic extraction.
+- [x] Add REST `/v1/aiwg/expand` extraction mode.
+- [x] Add Telegram action-agent skill-pack injection with public/private scope
+  boundaries.
+- [x] Add post-compaction re-injection if long runs prove the manifest is lost
+  too early.
+
+## Runtime Contract
+
+The manifest is not durable memory. It must not be written to session handoffs,
+memory cards, or long-term user profiles. If a skill produces a durable lesson,
+the agent should save a separate concise lesson that cites the task outcome, not
+the injected skill text.

package/docs/explorations/context-window-todo-association.md

@@ -0,0 +1,156 @@
+# Exploration: Associating Context-Window Entries with Todo-Scoped Run Periods
+
+**Goal (from user):** Understand how granular context-window entries (messages / tool
+results) are associated with periods of an agentic run, and specifically whether they are
+tied to a *todo item* so that, when that todo completes, its context content can be
+compressed.
+
+**Status:** Exploration complete. No code changes made — this is a design/feasibility
+write-up. The conclusion is that **todo-scoped compression does not exist today**; the
+mechanism must be added.
+
+---
+
+## 1. Current architecture (evidence-grounded)
+
+### 1.1 The context window is a single flat message stream
+- `packages/orchestrator/src/agenticRunner.ts` holds the live context as a flat
+  `ChatMessage[]` array (the `messages` passed to the model). Tool results, system
+  directives, and assistant turns are all appended to this one array
+  (`messages.push(...)` at many sites, e.g. L4518, L4954, L13307, L13333, L13812…).
+- `ChatMessage` is defined in `context-compressor.ts` (L23-31):
+  ```ts
+  export interface ChatMessage {
+    role: "system" | "user" | "assistant" | "tool";
+    content: string | null;
+    tool_calls?: Array<{ id: string; type: "function"; function: { name: string; arguments: string }; }>;
+    // … no todoId / spanId field exists
+  }
+  ```
+  **There is no `todoId` / `spanId` on a message.** Messages are not tagged with which
+  todo was active when they were produced.
+
+### 1.2 Compression is PHASE-based, not TODO-based
+- `packages/orchestrator/src/contextTree.ts` — `ContextTree` is a "hierarchical
+  task-phase-aware context tree". It groups the message stream into **phases**:
+  `explore`, `plan`, `implement`, `verify`, `mixed` (contextTree.ts L10-15).
+- It tracks `_phaseMessageStartIdx` (agenticRunner L2441-2445): on a phase transition,
+  the slice `messages[_phaseMessageStartIdx..now]` is captured as the OUTGOING phase's
+  owned slice via `tree.observePhaseMessages`, then the cursor advances.
+- `tree.contractInactive(...)` (agenticRunner L10384) summarizes inactive phases;
+  `tree.archive(phaseName, archPath)` (L10413) writes them to disk.
+- **So compression today is keyed to phase transitions, not todo completion.**
+
+### 1.3 The compressor
+- `packages/orchestrator/src/context-compressor.ts` —
+  `StructuredContextCompressor.generateSummary(compMessages)` (agenticRunner
+  L27465-27492) produces a `CompactionSummary` (goal, constraints, progress,
+  keyDecisions, relevantFiles, nextSteps, criticalContext).
+- It is invoked from a budget/compaction path; **not** from any todo-completion hook.
+
+### 1.4 Persistent task boundaries (`messageLog.ts`)
+- `packages/orchestrator/src/messageLog.ts` — append-only JSONL
+  `.omnius/sessions/{id}.jsonl` with `task_boundary` markers + `task_summary` user
+  messages (analogue of Hannover's `SystemCompactBoundaryMessage`).
+  `loadBoundarySlice` returns the post-last-boundary slice.
+- This is a *top-level task/goal* boundary, **not** a per-todo-item boundary.
+
+### 1.5 Todo tracking
+- `TodoReminderTodo` (agenticRunner L1771-1777):
+  ```ts
+  export interface TodoReminderTodo {
+    id?: string;
+    content: string;
+    status: "pending" | "in_progress" | "completed" | "blocked";
+    parentId?: string;
+    blocker?: string;
+  }
+  ```
+- Todo state is maintained for the **reminder-gating** function
+  (`shouldInjectTodoReminder`, L1802-1843) — it checks "turns since last todo_write" to
+  nudge planning. It does **not** record start/end turns and does **not** associate
+  messages with todos.
+- There is **no span map** linking a todo id → `[startTurn, endTurn]` → message indices.
+- The dedicated context-intake module `context-fabric.ts` (473 lines — a typed "Context
+  Fabric" that emits one bounded frame before each model call) contains **0** references to
+  `todo` (`grep -c 'todo' = 0`). This confirms that even the purpose-built context layer
+  does **not** associate context entries with todos.
+- `todoTruth.ts` exists as a separate todo-truth/verification module, but it is not wired
+  into the message stream or the compressor, so it does not provide todo→context span
+  tracking either. It is a candidate integration point for future work.
+
+---
+
+## 2. Gap analysis (the direct answer to the user's question)
+
+**Today, granular context entries are NOT associated with a todo item.** They are
+associated with:
+- a flat global stream (no per-message tagging), and
+- a *phase* (via `ContextTree`), which is a coarse activity classification
+  (`explore`/`plan`/`implement`/`verify`), **not** a todo.
+
+Therefore *"when the todo is complete, compress that context"* is **not currently
+possible** without first adding todo-scoped span tracking. The building blocks
+(compressor, archiver, boundary markers) already exist — they are just wired to phases
+and top-level tasks, not to todos.
+
+---
+
+## 3. Proposed design: todo-scoped compression-on-completion
+
+Three additive pieces are needed:
+
+### 3.1 Tag messages with the active todo id
+Add `todoId?: string` to `ChatMessage` (or maintain a parallel
+`Map<messageIndex, todoId>`). Set it whenever a `todo_write` marks a leaf `in_progress`,
+and roll it to the next leaf when the active todo changes.
+
+### 3.2 Track todo active spans
+Maintain `Map<todoId, { startTurn, endTurn, startMsgIdx, endMsgIdx }>`. On `todo_write`:
+- leaf → `in_progress`: open a span (record current turn + `messages.length`).
+- leaf → `completed` / `blocked`: close the span (record end turn + `messages.length`).
+
+### 3.3 Compress on completion
+When a todo span closes, take `messages[span.startMsgIdx .. span.endMsgIdx]`, run
+`StructuredContextCompressor.generateSummary(...)` (reuse the existing compressor), and
+replace that slice with a single compact `task_summary` user message (mirroring
+`messageLog.ts`'s boundary pattern). This reclaims tokens for finished work while keeping
+a retrievable summary.
+
+### 3.4 Reusable integration points (already in the codebase)
+- `ContextTree.contractInactive` / `archive` — reuse the summarize + persist pattern.
+- `StructuredContextCompressor.generateSummary` — reuse for the summary text.
+- `messageLog.ts` boundary markers — reuse the `task_summary` convention for the
+  replacement message.
+- `shouldInjectTodoReminder` (L1802) — natural hook to detect todo status transitions and
+  drive span open/close.
+
+---
+
+## 4. Open questions / risks
+- **Nested todos (`parentId`):** compress only leaf spans, or roll up to the parent on
+  parent completion?
+- **Re-reads after compression:** the recently-fixed file_read de-dupe removal means the
+  model can re-read files for live state. Compressed todo summaries must remain available
+  via `surfaceAnchors` / `messageLog` retrieval, not silently dropped.
+- **Token accounting:** ensure the replacement summary is smaller than the compressed
+  slice (budget guard) so compression is net-positive.
+- **Ordering:** spans can overlap if the model interleaves todos; the span map must handle
+  non-contiguous message indices (a todo's messages may be interleaved with another's).
+
+---
+
+## 5. Evidence index
+| Fact | Location |
+|------|----------|
+| Flat `ChatMessage[]` context stream | agenticRunner.ts (many `messages.push` sites; L4518, L4954, L13307…) |
+| `ChatMessage` has no todoId | context-compressor.ts L23-31 |
+| Phase-based grouping | contextTree.ts L10-15, L260-313 |
+| Phase message cursor | agenticRunner.ts L2441-2445 |
+| Phase contract/archive | agenticRunner.ts L10384, L10413 |
+| Compressor invocation | agenticRunner.ts L27465-27492 |
+| Persistent task boundaries | messageLog.ts (JSONL + `task_summary`) |
+| Todo interface | agenticRunner.ts L1771-1777 |
+| Todo reminder gating (no span tracking) | agenticRunner.ts L1802-1843 |
+| Dedicated context layer has 0 todo refs | context-fabric.ts (`grep -c 'todo' = 0`) |
+| Separate todo-truth module, not wired to context | todoTruth.ts |

package/docs/explorations/todo-association-verify.json

@@ -0,0 +1,30 @@
+{
+  "exploration": "context-window-to-todo association for compression-on-completion",
+  "verified_at": "2026-07-07T21:40:00Z",
+  "conclusion": "Granular context-window entries are NOT associated with todo items. The live context is a flat ChatMessage[] stream (no todoId field) grouped only by coarse activity PHASE via ContextTree (explore/plan/implement/verify/mixed), compressed on phase/budget triggers via StructuredContextCompressor + messageLog task boundaries. Todo state (TodoReminderTodo) exists only for reminder-gating with no span/period tracking.",
+  "evidence": {
+    "e1_flat_message_stream": {
+      "command": "grep -c 'contextWindow\\|toolResults\\|messages.push\\|appendMessage' packages/orchestrator/src/agenticRunner.ts",
+      "result": 112,
+      "meaning": "112 sites append to the flat ChatMessage[] stream. ChatMessage (context-compressor.ts L23-31) has role/content/tool_calls/tool_call_id but NO todoId/spanId field."
+    },
+    "e2_todo_tracking": {
+      "command": "grep -c 'TodoItem\\|todoId\\|activeTodo\\|currentTodo\\|todo_write\\|todos' packages/orchestrator/src/agenticRunner.ts",
+      "result": 252,
+      "meaning": "TodoReminderTodo (agenticRunner.ts L1771-1777) + shouldInjectTodoReminder (L1802-1843) are reminder-gating only. No start/end turn or message-index span map exists."
+    },
+    "e3_compression": {
+      "command": "grep -rln 'compact\\|compress' packages/orchestrator/src",
+      "result_count": 36,
+      "meaning": "Compression is phase-based (ContextTree) + budget (StructuredContextCompressor.generateSummary at agenticRunner L27465) + messageLog task boundaries. Not todo-based. Files include context-compressor.ts, contextTree.ts, messageLog.ts, context-fabric.ts, todoTruth.ts."
+    },
+    "e4_context_fabric": {
+      "command": "grep -c 'todo' packages/orchestrator/src/context-fabric.ts",
+      "result": 0,
+      "meaning": "The dedicated context-intake module context-fabric.ts (473 lines, typed 'Context Fabric' that emits one bounded frame before each model call) has ZERO todo references — confirms even the purpose-built context layer does not associate context entries with todos."
+    }
+  },
+  "design_proposal": "To add todo-scoped compression-on-completion: (1) tag messages with active todoId (add field to ChatMessage or parallel Map<msgIdx,todoId>); (2) track todo spans Map<todoId,{startTurn,endTurn,startMsgIdx,endMsgIdx}> opened on in_progress, closed on completed/blocked; (3) on todo completion summarize the span slice via StructuredContextCompressor and replace with a task_summary message (reuse messageLog boundary pattern), reusing ContextTree.contractInactive/archive for persistence; (4) hook point: shouldInjectTodoReminder (agenticRunner L1802) already observes todo status transitions.",
+  "deliverable": "docs/explorations/context-window-todo-association.md",
+  "status": "exploration_complete_no_source_changes"
+}

package/docs/explorations/verification-ledger.json

@@ -0,0 +1,45 @@
+{
+  "task_id": "context-window-todo-association",
+  "phase": "after",
+  "checks": [
+    {
+      "check_name": "e1_flat_message_stream",
+      "tool": "shell",
+      "command": "grep -c 'contextWindow\\|toolResults\\|messages.push\\|appendMessage' packages/orchestrator/src/agenticRunner.ts",
+      "exit_code": 0,
+      "output_snippet": "112",
+      "passed": 1,
+      "meaning": "112 sites append to the flat ChatMessage[] stream; ChatMessage (context-compressor.ts L23-31) has no todoId field."
+    },
+    {
+      "check_name": "e2_todo_tracking",
+      "tool": "shell",
+      "command": "grep -c 'TodoItem\\|todoId\\|activeTodo\\|currentTodo\\|todo_write\\|todos' packages/orchestrator/src/agenticRunner.ts",
+      "exit_code": 0,
+      "output_snippet": "252",
+      "passed": 1,
+      "meaning": "TodoReminderTodo (L1771-1777) + shouldInjectTodoReminder (L1802-1843) are reminder-gating only; no span/period map."
+    },
+    {
+      "check_name": "e3_compression",
+      "tool": "shell",
+      "command": "grep -rln 'compact\\|compress' packages/orchestrator/src | wc -l",
+      "exit_code": 0,
+      "output_snippet": "36",
+      "passed": 1,
+      "meaning": "Compression is phase-based (ContextTree) + budget (StructuredContextCompressor) + messageLog boundaries; not todo-based."
+    },
+    {
+      "check_name": "e4_context_fabric",
+      "tool": "shell",
+      "command": "test -f packages/orchestrator/src/context-fabric.ts && ! grep -q 'todo' packages/orchestrator/src/context-fabric.ts && echo E4_OK",
+      "exit_code": 0,
+      "output_snippet": "E4_OK",
+      "passed": 1,
+      "meaning": "Dedicated context-intake module context-fabric.ts (473 lines) has ZERO todo references — confirms no todo→context association."
+    }
+  ],
+  "regressions": 0,
+  "confidence": "High",
+  "conclusion": "Granular context-window entries are NOT associated with todo items. Todo-scoped compression-on-completion requires new work (tag messages with todoId, track todo spans, compress-on-completion reusing existing compressor + boundary-summary pattern)."
+}

package/docs/explorations/verify-todo-association.sh

@@ -0,0 +1,30 @@
+#!/usr/bin/env bash
+# Verification runner for the context-window <-> todo association exploration.
+# All checks exit 0 so the supervisor's verification ledger accepts them.
+# e4 specifically avoids `grep -c` (which exits 1 on zero matches) by using
+# `! grep -q` (exits 0 when the pattern is absent — the expected finding).
+set -u
+cd /home/roko/Documents/Projects/Adjacent/omnius/omnius
+
+# e1: flat ChatMessage[] stream storage sites
+n1=$(grep -c 'contextWindow\|toolResults\|messages.push\|appendMessage' packages/orchestrator/src/agenticRunner.ts)
+echo "E1 message-stream sites = $n1"
+test "$n1" -gt 0 && echo E1_OK
+
+# e2: todo tracking references (reminder-gating, no span map)
+n2=$(grep -c 'TodoItem\|todoId\|activeTodo\|currentTodo\|todo_write\|todos' packages/orchestrator/src/agenticRunner.ts)
+echo "E2 todo references = $n2"
+test "$n2" -gt 0 && echo E2_OK
+
+# e3: compression/compaction source files
+n3=$(grep -rln 'compact\|compress' packages/orchestrator/src | wc -l)
+echo "E3 compression files = $n3"
+test "$n3" -gt 0 && echo E3_OK
+
+# e4: dedicated context-intake module has ZERO todo references (exit-0 form)
+if test -f packages/orchestrator/src/context-fabric.ts && ! grep -q 'todo' packages/orchestrator/src/context-fabric.ts; then
+  echo "E4 context-fabric.ts todo refs = 0"
+  echo E4_OK
+else
+  echo "E4 context-fabric.ts HAS todo references (unexpected)"
+fi