@dreb/coding-agent 1.16.0

package/docs/buddy.md

@@ -0,0 +1,111 @@
+# Buddy Companion
+
+The buddy is an optional companion that lives in the sidebar of the TUI. It reacts to events (tool runs, message completions, idle time) with short in-character quips, powered by a local Ollama model.
+
+## Setup
+
+The buddy requires [Ollama](https://ollama.com) running locally. Install it, pull a model, then use `/buddy` in dreb to hatch your companion.
+
+```bash
+# Install Ollama and pull a model
+ollama pull llama3.2
+
+# In dreb, hatch your buddy
+/buddy
+```
+
+After hatching, set which Ollama model the buddy should use for reactions:
+
+```bash
+/buddy model              # show current model + available models
+/buddy model llama3.2     # set the buddy's model
+```
+
+The buddy won't react until a model is configured. The choice is persisted across sessions.
+
+## Commands
+
+| Command | Description |
+|---|---|
+| `/buddy` | Hatch a new buddy (or show existing one) |
+| `/buddy model` | Show current Ollama model and available models |
+| `/buddy model <name>` | Set the Ollama model for buddy reactions |
+| `/buddy pet` | Pet your buddy |
+| `/buddy reroll` | Reroll for a new buddy (new species, name, personality) |
+| `/buddy stats` | Show buddy stats panel |
+| `/buddy off` | Hide the buddy |
+
+## How It Works
+
+- **Bones** (species, rarity, stats, shiny) are deterministic from your username + hostname + a reroll counter
+- **Soul** (name, personality, backstory) is generated once by your main model on first hatch and persisted
+- **Reactions** are generated by the local Ollama model — short, in-character quips based on what just happened
+- **Activity indicator** shows "thinking..." in the buddy panel while waiting for Ollama to respond
+
+## Troubleshooting
+
+### Buddy never reacts / always shows fallback message
+
+Check that Ollama is running and has at least one model installed:
+
+```bash
+ollama list
+```
+
+If no models are listed, pull one:
+
+```bash
+ollama pull llama3.2
+```
+
+### Reasoning model shows chain-of-thought in speech bubbles
+
+If you're using a reasoning model (e.g. `phi4-mini-reasoning`, `deepseek-r1`, `qwen3`) and the buddy's speech bubbles show the model's internal reasoning instead of just the final answer, the model's Ollama configuration likely doesn't declare thinking support properly.
+
+**How Ollama detects thinking models:** Ollama checks the model's template for a `{{.Thinking}}` field reference, or looks for a registered parser, or checks the model family. If none of these match, Ollama treats reasoning output as regular content — meaning `<think>...</think>` tags get passed through as literal text.
+
+**Check your model's capabilities:**
+
+```bash
+ollama show your-model-name
+```
+
+Look for `thinking` in the Capabilities section. If it's missing, the model isn't configured as a thinking model.
+
+**Fix: Create a custom Modelfile** that adds `{{.Thinking}}` to the template. Here's an example for phi4-mini-reasoning:
+
+```dockerfile
+FROM phi4-mini-reasoning:latest
+TEMPLATE """{{- if .System }}<|system|>{{ .System }}
+{{- end }}
+{{- range $i, $_ := .Messages }}
+{{- $last := eq (len (slice $.Messages $i)) 1 -}}
+{{- if ne .Role "system" }}<|{{ .Role }}|>
+{{- if and (eq .Role "assistant") .Thinking }}<think>
+{{ .Thinking }}</think>
+{{- end }}{{ .Content }}
+{{- if not $last }}<|end|>
+{{- end }}
+{{- if and (ne .Role "assistant") $last }}<|end|><|assistant|>{{ end }}
+{{- end }}
+{{- end }}"""
+```
+
+Then create the fixed model:
+
+```bash
+ollama create phi4-mini-reasoning-fixed -f Modelfile
+```
+
+Verify it now has thinking capability:
+
+```bash
+ollama show phi4-mini-reasoning-fixed
+# Should show "thinking" in Capabilities
+```
+
+**Why this happens:** Many community and smaller reasoning models on Ollama ship with templates that don't include `{{.Thinking}}`. Without this, Ollama's `thinking.InferTags` heuristic can't detect the model's thinking tags, so it doesn't separate reasoning from the final answer. The model still outputs `<think>...</think>` tags, but they end up in the content field instead of a structured reasoning field.
+
+### Reasoning model thinks but never responds
+
+If the buddy shows the activity indicator for a while but never produces a quip, the model may be spending its entire token budget on reasoning. dreb allocates 2048 tokens for buddy responses, which should be enough for most models, but very verbose reasoners may still exhaust it. Consider using a non-reasoning model for the buddy, or a smaller/faster one.

package/docs/compaction.md

@@ -0,0 +1,392 @@
+# Compaction & Branch Summarization
+
+LLMs have limited context windows. When conversations grow too long, dreb uses compaction to summarize older content while preserving recent work. This page covers both auto-compaction and branch summarization.
+
+**Source files** ([dreb](https://github.com/aebrer/dreb)):
+- [`packages/coding-agent/src/core/compaction/compaction.ts`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/compaction/compaction.ts) - Auto-compaction logic
+- [`packages/coding-agent/src/core/compaction/branch-summarization.ts`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/compaction/branch-summarization.ts) - Branch summarization
+- [`packages/coding-agent/src/core/compaction/utils.ts`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/compaction/utils.ts) - Shared utilities (file tracking, serialization)
+- [`packages/coding-agent/src/core/session-manager.ts`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/session-manager.ts) - Entry types (`CompactionEntry`, `BranchSummaryEntry`)
+- [`packages/coding-agent/src/core/extensions/types.ts`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/extensions/types.ts) - Extension event types
+
+For TypeScript definitions in your project, inspect `node_modules/@dreb/coding-agent/dist/`.
+
+## Overview
+
+dreb has two summarization mechanisms:
+
+| Mechanism | Trigger | Purpose |
+|-----------|---------|---------|
+| Compaction | Context exceeds threshold, or `/compact` | Summarize old messages to free up context |
+| Branch summarization | `/tree` navigation | Preserve context when switching branches |
+
+Both use the same structured summary format and track file operations cumulatively.
+
+## Compaction
+
+### When It Triggers
+
+Auto-compaction triggers when:
+
+```
+contextTokens > contextWindow - reserveTokens
+```
+
+By default, `reserveTokens` is 16384 tokens (configurable in `~/.dreb/agent/settings.json` or `<project-dir>/.dreb/settings.json`). This leaves room for the LLM's response.
+
+You can also trigger manually with `/compact [instructions]`, where optional instructions focus the summary.
+
+### How It Works
+
+1. **Find cut point**: Walk backwards from newest message, accumulating token estimates until `keepRecentTokens` (default 20k, configurable in `~/.dreb/agent/settings.json` or `<project-dir>/.dreb/settings.json`) is reached
+2. **Extract messages**: Collect messages from previous compaction (or start) up to cut point
+3. **Generate summary**: Call LLM to summarize with structured format
+4. **Append entry**: Save `CompactionEntry` with summary and `firstKeptEntryId`
+5. **Reload**: Session reloads, using summary + messages from `firstKeptEntryId` onwards
+
+```
+Before compaction:
+
+  entry:  0     1     2     3      4     5     6      7      8     9
+        ┌─────┬─────┬─────┬─────┬──────┬─────┬─────┬──────┬──────┬─────┐
+        │ hdr │ usr │ ass │ tool │ usr │ ass │ tool │ tool │ ass │ tool│
+        └─────┴─────┴─────┴──────┴─────┴─────┴──────┴──────┴─────┴─────┘
+                └────────┬───────┘ └──────────────┬──────────────┘
+               messagesToSummarize            kept messages
+                                   ↑
+                          firstKeptEntryId (entry 4)
+
+After compaction (new entry appended):
+
+  entry:  0     1     2     3      4     5     6      7      8     9     10
+        ┌─────┬─────┬─────┬─────┬──────┬─────┬─────┬──────┬──────┬─────┬─────┐
+        │ hdr │ usr │ ass │ tool │ usr │ ass │ tool │ tool │ ass │ tool│ cmp │
+        └─────┴─────┴─────┴──────┴─────┴─────┴──────┴──────┴─────┴─────┴─────┘
+               └──────────┬──────┘ └──────────────────────┬───────────────────┘
+                 not sent to LLM                    sent to LLM
+                                                         ↑
+                                              starts from firstKeptEntryId
+
+What the LLM sees:
+
+  ┌────────┬─────────┬─────┬─────┬──────┬──────┬─────┬──────┐
+  │ system │ summary │ usr │ ass │ tool │ tool │ ass │ tool │
+  └────────┴─────────┴─────┴─────┴──────┴──────┴─────┴──────┘
+       ↑         ↑      └─────────────────┬────────────────┘
+    prompt   from cmp          messages from firstKeptEntryId
+```
+
+### Split Turns
+
+A "turn" starts with a user message and includes all assistant responses and tool calls until the next user message. Normally, compaction cuts at turn boundaries.
+
+When a single turn exceeds `keepRecentTokens`, the cut point lands mid-turn at an assistant message. This is a "split turn":
+
+```
+Split turn (one huge turn exceeds budget):
+
+  entry:  0     1     2      3     4      5      6     7      8
+        ┌─────┬─────┬─────┬──────┬─────┬──────┬──────┬─────┬──────┐
+        │ hdr │ usr │ ass │ tool │ ass │ tool │ tool │ ass │ tool │
+        └─────┴─────┴─────┴──────┴─────┴──────┴──────┴─────┴──────┘
+                ↑                                     ↑
+         turnStartIndex = 1                  firstKeptEntryId = 7
+                │                                     │
+                └──── turnPrefixMessages (1-6) ───────┘
+                                                      └── kept (7-8)
+
+  isSplitTurn = true
+  messagesToSummarize = []  (no complete turns before)
+  turnPrefixMessages = [usr, ass, tool, ass, tool, tool]
+```
+
+For split turns, dreb generates two summaries and merges them:
+1. **History summary**: Previous context (if any)
+2. **Turn prefix summary**: The early part of the split turn
+
+### Cut Point Rules
+
+Valid cut points are:
+- User messages
+- Assistant messages
+- BashExecution messages
+- Custom messages (custom_message, branch_summary)
+
+Never cut at tool results (they must stay with their tool call).
+
+### CompactionEntry Structure
+
+Defined in [`session-manager.ts`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/session-manager.ts):
+
+```typescript
+interface CompactionEntry<T = unknown> {
+  type: "compaction";
+  id: string;
+  parentId: string;
+  timestamp: string;
+  summary: string;
+  firstKeptEntryId: string;
+  tokensBefore: number;
+  fromHook?: boolean;  // true if provided by extension (legacy field name)
+  details?: T;         // implementation-specific data
+}
+
+// Default compaction uses this for details (from compaction.ts):
+interface CompactionDetails {
+  readFiles: string[];
+  modifiedFiles: string[];
+}
+```
+
+Extensions can store any JSON-serializable data in `details`. The default compaction tracks file operations, but custom extension implementations can use their own structure.
+
+See [`prepareCompaction()`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/compaction/compaction.ts) and [`compact()`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/compaction/compaction.ts) for the implementation.
+
+## Branch Summarization
+
+### When It Triggers
+
+When you use `/tree` to navigate to a different branch, dreb offers to summarize the work you're leaving. This injects context from the left branch into the new branch.
+
+### How It Works
+
+1. **Find common ancestor**: Deepest node shared by old and new positions
+2. **Collect entries**: Walk from old leaf back to common ancestor
+3. **Prepare with budget**: Include messages up to token budget (newest first)
+4. **Generate summary**: Call LLM with structured format
+5. **Append entry**: Save `BranchSummaryEntry` at navigation point
+
+```
+Tree before navigation:
+
+         ┌─ B ─ C ─ D (old leaf, being abandoned)
+    A ───┤
+         └─ E ─ F (target)
+
+Common ancestor: A
+Entries to summarize: B, C, D
+
+After navigation with summary:
+
+         ┌─ B ─ C ─ D ─ [summary of B,C,D]
+    A ───┤
+         └─ E ─ F (new leaf)
+```
+
+### Cumulative File Tracking
+
+Both compaction and branch summarization track files cumulatively. When generating a summary, dreb extracts file operations from:
+- Tool calls in the messages being summarized
+- Previous compaction or branch summary `details` (if any)
+
+This means file tracking accumulates across multiple compactions or nested branch summaries, preserving the full history of read and modified files.
+
+### BranchSummaryEntry Structure
+
+Defined in [`session-manager.ts`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/session-manager.ts):
+
+```typescript
+interface BranchSummaryEntry<T = unknown> {
+  type: "branch_summary";
+  id: string;
+  parentId: string;
+  timestamp: string;
+  summary: string;
+  fromId: string;      // Entry we navigated from
+  fromHook?: boolean;  // true if provided by extension (legacy field name)
+  details?: T;         // implementation-specific data
+}
+
+// Default branch summarization uses this for details (from branch-summarization.ts):
+interface BranchSummaryDetails {
+  readFiles: string[];
+  modifiedFiles: string[];
+}
+```
+
+Same as compaction, extensions can store custom data in `details`.
+
+See [`collectEntriesForBranchSummary()`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/compaction/branch-summarization.ts), [`prepareBranchEntries()`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/compaction/branch-summarization.ts), and [`generateBranchSummary()`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/compaction/branch-summarization.ts) for the implementation.
+
+## Summary Format
+
+Both compaction and branch summarization use the same structured format:
+
+```markdown
+## Goal
+[What the user is trying to accomplish]
+
+## Constraints & Preferences
+- [Requirements mentioned by user]
+
+## Progress
+### Done
+- [x] [Completed tasks]
+
+### In Progress
+- [ ] [Current work]
+
+### Blocked
+- [Issues, if any]
+
+## Key Decisions
+- **[Decision]**: [Rationale]
+
+## Next Steps
+1. [What should happen next]
+
+## Critical Context
+- [Data needed to continue]
+
+<read-files>
+path/to/file1.ts
+path/to/file2.ts
+</read-files>
+
+<modified-files>
+path/to/changed.ts
+</modified-files>
+```
+
+### Message Serialization
+
+Before summarization, messages are serialized to text via [`serializeConversation()`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/compaction/utils.ts):
+
+```
+[User]: What they said
+[Assistant thinking]: Internal reasoning
+[Assistant]: Response text
+[Assistant tool calls]: read(path="foo.ts"); edit(path="bar.ts", ...)
+[Tool result]: Output from tool
+```
+
+This prevents the model from treating it as a conversation to continue.
+
+Tool results are truncated to 2000 characters during serialization. Content beyond that limit is replaced with a marker indicating how many characters were truncated. This keeps summarization requests within reasonable token budgets, since tool results (especially from `read` and `bash`) are typically the largest contributors to context size.
+
+## Custom Summarization via Extensions
+
+Extensions can intercept and customize both compaction and branch summarization. See [`extensions/types.ts`](https://github.com/aebrer/dreb/blob/master/packages/coding-agent/src/core/extensions/types.ts) for event type definitions.
+
+### session_before_compact
+
+Fired before auto-compaction or `/compact`. Can cancel or provide custom summary. See `SessionBeforeCompactEvent` and `CompactionPreparation` in the types file.
+
+```typescript
+dreb.on("session_before_compact", async (event, ctx) => {
+  const { preparation, branchEntries, customInstructions, signal } = event;
+
+  // preparation.messagesToSummarize - messages to summarize
+  // preparation.turnPrefixMessages - split turn prefix (if isSplitTurn)
+  // preparation.previousSummary - previous compaction summary
+  // preparation.fileOps - extracted file operations
+  // preparation.tokensBefore - context tokens before compaction
+  // preparation.firstKeptEntryId - where kept messages start
+  // preparation.settings - compaction settings
+
+  // branchEntries - all entries on current branch (for custom state)
+  // signal - AbortSignal (pass to LLM calls)
+
+  // Cancel:
+  return { cancel: true };
+
+  // Custom summary:
+  return {
+    compaction: {
+      summary: "Your summary...",
+      firstKeptEntryId: preparation.firstKeptEntryId,
+      tokensBefore: preparation.tokensBefore,
+      details: { /* custom data */ },
+    }
+  };
+});
+```
+
+#### Converting Messages to Text
+
+To generate a summary with your own model, convert messages to text using `serializeConversation`:
+
+```typescript
+import { convertToLlm, serializeConversation } from "@dreb/coding-agent";
+
+dreb.on("session_before_compact", async (event, ctx) => {
+  const { preparation } = event;
+  
+  // Convert AgentMessage[] to Message[], then serialize to text
+  const conversationText = serializeConversation(
+    convertToLlm(preparation.messagesToSummarize)
+  );
+  // Returns:
+  // [User]: message text
+  // [Assistant thinking]: thinking content
+  // [Assistant]: response text
+  // [Assistant tool calls]: read(path="..."); bash(command="...")
+  // [Tool result]: output text
+
+  // Now send to your model for summarization
+  const summary = await myModel.summarize(conversationText);
+  
+  return {
+    compaction: {
+      summary,
+      firstKeptEntryId: preparation.firstKeptEntryId,
+      tokensBefore: preparation.tokensBefore,
+    }
+  };
+});
+```
+
+See [custom-compaction.ts](../examples/extensions/custom-compaction.ts) for a complete example using a different model.
+
+### session_before_tree
+
+Fired before `/tree` navigation. Always fires regardless of whether user chose to summarize. Can cancel navigation or provide custom summary.
+
+```typescript
+dreb.on("session_before_tree", async (event, ctx) => {
+  const { preparation, signal } = event;
+
+  // preparation.targetId - where we're navigating to
+  // preparation.oldLeafId - current position (being abandoned)
+  // preparation.commonAncestorId - shared ancestor
+  // preparation.entriesToSummarize - entries that would be summarized
+  // preparation.userWantsSummary - whether user chose to summarize
+
+  // Cancel navigation entirely:
+  return { cancel: true };
+
+  // Provide custom summary (only used if userWantsSummary is true):
+  if (preparation.userWantsSummary) {
+    return {
+      summary: {
+        summary: "Your summary...",
+        details: { /* custom data */ },
+      }
+    };
+  }
+});
+```
+
+See `SessionBeforeTreeEvent` and `TreePreparation` in the types file.
+
+## Settings
+
+Configure compaction in `~/.dreb/agent/settings.json` or `<project-dir>/.dreb/settings.json`:
+
+```json
+{
+  "compaction": {
+    "enabled": true,
+    "reserveTokens": 16384,
+    "keepRecentTokens": 20000
+  }
+}
+```
+
+| Setting | Default | Description |
+|---------|---------|-------------|
+| `enabled` | `true` | Enable auto-compaction |
+| `reserveTokens` | `16384` | Tokens to reserve for LLM response |
+| `keepRecentTokens` | `20000` | Recent tokens to keep (not summarized) |
+
+Disable auto-compaction with `"enabled": false`. You can still compact manually with `/compact`.