@mariozechner/pi-coding-agent 0.16.0 → 0.18.0

package/docs/truncation.md

@@ -0,0 +1,235 @@
+# Tool Output Truncation
+
+## Limits
+
+- **Line limit**: 2000 lines
+- **Byte limit**: 30KB
+- **Grep line limit**: 500 chars per match line
+
+Whichever limit is hit first wins. **Never return partial lines** (except bash edge case).
+
+---
+
+## read
+
+Head truncation (first N lines). Has offset/limit params for continuation.
+
+### Scenarios
+
+**First line > 30KB:**
+```
+LLM sees:
+[Line 1 is 50KB, exceeds 30KB limit. Use bash to read: head -c 30000 path/to/file]
+
+Details:
+{ truncation: { truncated: true, truncatedBy: "bytes", outputLines: 0, ... } }
+```
+
+**Hit line limit (2000 lines, < 30KB):**
+```
+LLM sees:
+[lines 1-2000 content]
+
+[Showing lines 1-2000 of 5000. Use offset=2001 to continue]
+
+Details:
+{ truncation: { truncated: true, truncatedBy: "lines", outputLines: 2000, totalLines: 5000 } }
+```
+
+**Hit byte limit (< 2000 lines, 30KB):**
+```
+LLM sees:
+[lines 1-500 content]
+
+[Showing lines 1-500 of 5000 (30KB limit). Use offset=501 to continue]
+
+Details:
+{ truncation: { truncated: true, truncatedBy: "bytes", outputLines: 500, totalLines: 5000 } }
+```
+
+**With offset, hit line limit (e.g., offset=1000):**
+```
+LLM sees:
+[lines 1000-2999 content]
+
+[Showing lines 1000-2999 of 5000. Use offset=3000 to continue]
+
+Details:
+{ truncation: { truncatedBy: "lines", ... } }
+```
+
+**With offset, hit byte limit (e.g., offset=1000, 30KB after 500 lines):**
+```
+LLM sees:
+[lines 1000-1499 content]
+
+[Showing lines 1000-1499 of 5000 (30KB limit). Use offset=1500 to continue]
+
+Details:
+{ truncation: { truncatedBy: "bytes", outputLines: 500, ... } }
+```
+
+**With offset, first line at offset > 30KB (e.g., offset=1000, line 1000 is 50KB):**
+```
+LLM sees:
+[Line 1000 is 50KB, exceeds 30KB limit. Use bash: sed -n '1000p' file | head -c 30000]
+
+Details:
+{ truncation: { truncated: true, truncatedBy: "bytes", outputLines: 0 } }
+```
+
+---
+
+## bash
+
+Tail truncation (last N lines). Writes full output to temp file if truncated.
+
+### Scenarios
+
+**Hit line limit (2000 lines):**
+```
+LLM sees:
+[lines 48001-50000 content]
+
+[Showing lines 48001-50000 of 50000. Full output: /tmp/pi-bash-xxx.log]
+
+Details:
+{ truncation: { truncated: true, truncatedBy: "lines", outputLines: 2000, totalLines: 50000 }, fullOutputPath: "/tmp/..." }
+```
+
+**Hit byte limit (< 2000 lines, 30KB):**
+```
+LLM sees:
+[lines 49501-50000 content]
+
+[Showing lines 49501-50000 of 50000 (30KB limit). Full output: /tmp/pi-bash-xxx.log]
+
+Details:
+{ truncation: { truncatedBy: "bytes", ... }, fullOutputPath: "/tmp/..." }
+```
+
+**Last line alone > 30KB (edge case, partial OK here):**
+```
+LLM sees:
+[last 30KB of final line]
+
+[Showing last 30KB of line 50000 (line is 100KB). Full output: /tmp/pi-bash-xxx.log]
+
+Details:
+{ truncation: { truncatedBy: "bytes", lastLinePartial: true }, fullOutputPath: "/tmp/..." }
+```
+
+---
+
+## grep
+
+Head truncation. Primary limit: 100 matches. Each match line truncated to 500 chars.
+
+### Scenarios
+
+**Hit match limit (100 matches):**
+```
+LLM sees:
+file.ts:10: matching content here...
+file.ts:25: another match...
+...
+
+[100 matches limit reached. Use limit=200 for more, or refine pattern]
+
+Details:
+{ matchLimitReached: 100 }
+```
+
+**Hit byte limit (< 100 matches, 30KB):**
+```
+LLM sees:
+[matches that fit in 30KB]
+
+[30KB limit reached (50 of 100+ matches shown)]
+
+Details:
+{ truncation: { truncatedBy: "bytes", ... } }
+```
+
+**Match lines truncated (any line > 500 chars):**
+```
+LLM sees:
+file.ts:10: very long matching content that exceeds 500 chars gets cut off here... [truncated]
+file.ts:25: normal match
+
+[Some lines truncated to 500 chars. Use read tool to see full lines]
+
+Details:
+{ linesTruncated: true }
+```
+
+---
+
+## find
+
+Head truncation. Primary limit: 1000 results. File paths only (never > 30KB each).
+
+### Scenarios
+
+**Hit result limit (1000 results):**
+```
+LLM sees:
+src/file1.ts
+src/file2.ts
+[998 more paths]
+
+[1000 results limit reached. Use limit=2000 for more, or refine pattern]
+
+Details:
+{ resultLimitReached: 1000 }
+```
+
+**Hit byte limit (unlikely, < 1000 results, 30KB):**
+```
+LLM sees:
+[paths that fit]
+
+[30KB limit reached]
+
+Details:
+{ truncation: { truncatedBy: "bytes", ... } }
+```
+
+---
+
+## ls
+
+Head truncation. Primary limit: 500 entries. Entry names only (never > 30KB each).
+
+### Scenarios
+
+**Hit entry limit (500 entries):**
+```
+LLM sees:
+.gitignore
+README.md
+src/
+[497 more entries]
+
+[500 entries limit reached. Use limit=1000 for more]
+
+Details:
+{ entryLimitReached: 500 }
+```
+
+**Hit byte limit (unlikely):**
+```
+LLM sees:
+[entries that fit]
+
+[30KB limit reached]
+
+Details:
+{ truncation: { truncatedBy: "bytes", ... } }
+```
+
+---
+
+## TUI Display
+
+`tool-execution.ts` reads `details.truncation` and related fields to display truncation notices in warning color. The LLM text content and TUI display show the same information.

package/docs/undercompaction.md

@@ -0,0 +1,313 @@
+# Under-Compaction Analysis
+
+## Problem Statement
+
+Auto-compaction triggers too late, causing context window overflows that result in failed LLM calls with `stopReason == "length"`.
+
+## Architecture Overview
+
+### Event Flow
+
+```
+User prompt
+    │
+    ▼
+agent.prompt()
+    │
+    ▼
+agentLoop() in packages/ai/src/agent/agent-loop.ts
+    │
+    ├─► streamAssistantResponse()
+    │       │
+    │       ▼
+    │   LLM provider (Anthropic, OpenAI, etc.)
+    │       │
+    │       ▼
+    │   Events: message_start → message_update* → message_end
+    │       │
+    │       ▼
+    │   AssistantMessage with usage stats (input, output, cacheRead, cacheWrite)
+    │
+    ├─► If assistant has tool calls:
+    │       │
+    │       ▼
+    │   executeToolCalls()
+    │       │
+    │       ├─► tool_execution_start (toolCallId, toolName, args)
+    │       │
+    │       ├─► tool.execute() runs (read, bash, write, edit, etc.)
+    │       │
+    │       ├─► tool_execution_end (toolCallId, toolName, result, isError)
+    │       │
+    │       └─► message_start + message_end for ToolResultMessage
+    │
+    └─► Loop continues until no more tool calls
+            │
+            ▼
+        agent_end
+```
+
+### Token Usage Reporting
+
+Token usage is ONLY available in `AssistantMessage.usage` after the LLM responds:
+
+```typescript
+// From packages/ai/src/types.ts
+export interface Usage {
+    input: number;      // Tokens in the request
+    output: number;     // Tokens generated
+    cacheRead: number;  // Cached tokens read
+    cacheWrite: number; // Cached tokens written
+    cost: Cost;
+}
+```
+
+The `input` field represents the total context size sent to the LLM, which includes:
+- System prompt
+- All conversation messages
+- All tool results from previous calls
+
+### Current Compaction Check
+
+Both TUI (`tui-renderer.ts`) and RPC (`main.ts`) modes check compaction identically:
+
+```typescript
+// In agent.subscribe() callback:
+if (event.type === "message_end") {
+    // ...
+    if (event.message.role === "assistant") {
+        await checkAutoCompaction();
+    }
+}
+
+async function checkAutoCompaction() {
+    // Get last non-aborted assistant message
+    const messages = agent.state.messages;
+    let lastAssistant = findLastNonAbortedAssistant(messages);
+    if (!lastAssistant) return;
+
+    const contextTokens = calculateContextTokens(lastAssistant.usage);
+    const contextWindow = agent.state.model.contextWindow;
+
+    if (!shouldCompact(contextTokens, contextWindow, settings)) return;
+
+    // Trigger compaction...
+}
+```
+
+**The check happens on `message_end` for assistant messages only.**
+
+## The Under-Compaction Problem
+
+### Failure Scenario
+
+```
+Context window: 200,000 tokens
+Reserve tokens: 16,384 (default)
+Threshold: 200,000 - 16,384 = 183,616
+
+Turn N:
+  1. Assistant message received, usage shows 180,000 tokens
+  2. shouldCompact(180000, 200000, settings) → 180000 > 183616 → FALSE
+  3. Tool executes: `cat large-file.txt` → outputs 100KB (~25,000 tokens)
+  4. Context now effectively 205,000 tokens, but we don't know this
+  5. Next LLM call fails: context exceeds 200,000 window
+```
+
+The problem occurs when:
+1. Context is below threshold (so compaction doesn't trigger)
+2. A tool adds enough content to push it over the window limit
+3. We only discover this when the next LLM call fails
+
+### Root Cause
+
+1. **Token counts are retrospective**: We only learn the context size AFTER the LLM processes it
+2. **Tool results are blind spots**: When a tool executes and returns a large result, we don't know how many tokens it adds until the next LLM call
+3. **No estimation before submission**: We submit the context and hope it fits
+
+## Current Tool Output Limits
+
+| Tool | Our Limit | Worst Case |
+|------|-----------|------------|
+| bash | 10MB per stream | 20MB (~5M tokens) |
+| read | 2000 lines × 2000 chars | 4MB (~1M tokens) |
+| write | Byte count only | Minimal |
+| edit | Diff output | Variable |
+
+## How Other Tools Handle This
+
+### SST/OpenCode
+
+**Tool Output Limits (during execution):**
+
+| Tool | Limit | Details |
+|------|-------|---------|
+| bash | 30KB chars | `MAX_OUTPUT_LENGTH = 30_000`, truncates with notice |
+| read | 2000 lines × 2000 chars/line | No total cap, theoretically 4MB |
+| grep | 100 matches, 2000 chars/line | Truncates with notice |
+| ls | 100 files | Truncates with notice |
+| glob | 100 results | Truncates with notice |
+| webfetch | 5MB | `MAX_RESPONSE_SIZE` |
+
+**Overflow Detection:**
+- `isOverflow()` runs BEFORE each turn (not during)
+- Uses last LLM-reported token count: `tokens.input + tokens.cache.read + tokens.output`
+- Triggers if `count > context - maxOutput`
+- Does NOT detect overflow from tool results in current turn
+
+**Recovery - Pruning:**
+- `prune()` runs AFTER each turn completes
+- Walks backwards through completed tool results
+- Keeps last 40k tokens of tool outputs (`PRUNE_PROTECT`)
+- Removes content from older tool results (marks `time.compacted`)
+- Only prunes if savings > 20k tokens (`PRUNE_MINIMUM`)
+- Token estimation: `chars / 4`
+
+**Recovery - Compaction:**
+- Triggered when `isOverflow()` returns true before a turn
+- LLM generates summary of conversation
+- Replaces old messages with summary
+
+**Gap:** No mid-turn protection. A single read returning 4MB would overflow. The 30KB bash limit is their primary practical protection.
+
+### OpenAI/Codex
+
+**Tool Output Limits (during execution):**
+
+| Tool | Limit | Details |
+|------|-------|---------|
+| shell/exec | 10k tokens or 10k bytes | Per-model `TruncationPolicy`, user-configurable |
+| read_file | 2000 lines, 500 chars/line | `MAX_LINE_LENGTH = 500`, ~1MB max |
+| grep_files | 100 matches | Default limit |
+| list_dir | Configurable | BFS with depth limits |
+
+**Truncation Policy:**
+- Per-model family setting: `TruncationPolicy::Bytes(10_000)` or `TruncationPolicy::Tokens(10_000)`
+- User can override via `tool_output_token_limit` config
+- Applied to ALL tool outputs uniformly via `truncate_function_output_items_with_policy()`
+- Preserves beginning and end, removes middle with `"…N tokens truncated…"` marker
+
+**Overflow Detection:**
+- After each successful turn: `if total_usage_tokens >= auto_compact_token_limit { compact() }`
+- Per-model thresholds (e.g., 180k for 200k context window)
+- `ContextWindowExceeded` error caught and handled
+
+**Recovery - Compaction:**
+- If tokens exceed threshold after turn, triggers `run_inline_auto_compact_task()`
+- During compaction, if `ContextWindowExceeded`: removes oldest history item and retries
+- Loop: `history.remove_first_item()` until it fits
+- Notifies user: "Trimmed N older conversation item(s)"
+
+**Recovery - Turn Error:**
+- On `ContextWindowExceeded` during normal turn: marks tokens as full, returns error to user
+- Does NOT auto-retry the failed turn
+- User must manually continue
+
+**Gap:** Still no mid-turn protection, but aggressive 10k token truncation on all tool outputs prevents most issues in practice.
+
+### Comparison
+
+| Feature | pi-coding-agent | OpenCode | Codex |
+|---------|-----------------|----------|-------|
+| Bash limit | 10MB | 30KB | ~40KB (10k tokens) |
+| Read limit | 2000×2000 (4MB) | 2000×2000 (4MB) | 2000×500 (1MB) |
+| Truncation policy | None | Per-tool | Per-model, uniform |
+| Token estimation | None | chars/4 | chars/4 |
+| Pre-turn check | No | Yes (last tokens) | Yes (threshold) |
+| Mid-turn check | No | No | No |
+| Post-turn pruning | No | Yes (removes old tool output) | No |
+| Overflow recovery | No | Compaction | Trim oldest + compact |
+
+**Key insight:** None of these tools protect against mid-turn overflow. Their practical protection is aggressive static limits on tool output, especially bash. OpenCode's 30KB bash limit vs our 10MB is the critical difference.
+
+## Recommended Solution
+
+### Phase 1: Static Limits (immediate)
+
+Add hard limits to tool outputs matching industry practice:
+
+```typescript
+// packages/coding-agent/src/tools/limits.ts
+export const MAX_TOOL_OUTPUT_CHARS = 30_000; // ~7.5k tokens, matches OpenCode bash
+export const MAX_TOOL_OUTPUT_NOTICE = "\n\n...(truncated, output exceeded limit)...";
+```
+
+Apply to all tools:
+- bash: 10MB → 30KB
+- read: Add 100KB total output cap
+- edit: Cap diff output
+
+### Phase 2: Post-Tool Estimation
+
+After `tool_execution_end`, estimate and flag:
+
+```typescript
+let needsCompactionAfterTurn = false;
+
+agent.subscribe(async (event) => {
+    if (event.type === "tool_execution_end") {
+        const resultChars = extractTextLength(event.result);
+        const estimatedTokens = Math.ceil(resultChars / 4);
+        
+        const lastUsage = getLastAssistantUsage(agent.state.messages);
+        if (lastUsage) {
+            const current = calculateContextTokens(lastUsage);
+            const projected = current + estimatedTokens;
+            const threshold = agent.state.model.contextWindow - settings.reserveTokens;
+            if (projected > threshold) {
+                needsCompactionAfterTurn = true;
+            }
+        }
+    }
+    
+    if (event.type === "turn_end" && needsCompactionAfterTurn) {
+        needsCompactionAfterTurn = false;
+        await triggerCompaction();
+    }
+});
+```
+
+### Phase 3: Overflow Recovery (like Codex)
+
+Handle `stopReason === "length"` gracefully:
+
+```typescript
+if (event.type === "message_end" && event.message.role === "assistant") {
+    if (event.message.stopReason === "length") {
+        // Context overflow occurred
+        await triggerCompaction();
+        // Optionally: retry the turn
+    }
+}
+```
+
+During compaction, if it also overflows, trim oldest messages:
+
+```typescript
+async function compactWithRetry() {
+    while (true) {
+        try {
+            await compact();
+            break;
+        } catch (e) {
+            if (isContextOverflow(e) && messages.length > 1) {
+                messages.shift(); // Remove oldest
+                continue;
+            }
+            throw e;
+        }
+    }
+}
+```
+
+## Summary
+
+The under-compaction problem occurs because:
+1. We only check context size after assistant messages
+2. Tool results can add arbitrary amounts of content
+3. We discover overflows only when the next LLM call fails
+
+The fix requires:
+1. Aggressive static limits on tool output (immediate safety net)
+2. Token estimation after tool execution (proactive detection)
+3. Graceful handling of overflow errors (fallback recovery)

package/package.json

@@ -1,6 +1,6 @@
 {
 	"name": "@mariozechner/pi-coding-agent",
-	"version": "0.16.0",
+	"version": "0.18.0",
 	"description": "Coding agent CLI with read, bash, edit, write tools and session management",
 	"type": "module",
 	"piConfig": {
@@ -12,8 +12,19 @@
 	},
 	"main": "./dist/index.js",
 	"types": "./dist/index.d.ts",
+	"exports": {
+		".": {
+			"types": "./dist/index.d.ts",
+			"import": "./dist/index.js"
+		},
+		"./hooks": {
+			"types": "./dist/core/hooks/index.d.ts",
+			"import": "./dist/core/hooks/index.js"
+		}
+	},
 	"files": [
 		"dist",
+		"docs",
 		"CHANGELOG.md"
 	],
 	"scripts": {
@@ -21,19 +32,20 @@
 		"build": "tsgo -p tsconfig.build.json && chmod +x dist/cli.js && npm run copy-assets",
 		"build:binary": "npm run build && bun build --compile ./dist/cli.js --outfile dist/pi && npm run copy-binary-assets",
 		"copy-assets": "mkdir -p dist/modes/interactive/theme && cp src/modes/interactive/theme/*.json dist/modes/interactive/theme/",
-		"copy-binary-assets": "cp package.json dist/ && cp README.md dist/ && cp CHANGELOG.md dist/ && mkdir -p dist/theme && cp src/modes/interactive/theme/*.json dist/theme/",
+		"copy-binary-assets": "cp package.json dist/ && cp README.md dist/ && cp CHANGELOG.md dist/ && mkdir -p dist/theme && cp src/modes/interactive/theme/*.json dist/theme/ && cp -r docs dist/",
 		"dev": "tsgo -p tsconfig.build.json --watch --preserveWatchOutput",
 		"check": "tsgo --noEmit",
 		"test": "vitest --run",
 		"prepublishOnly": "npm run clean && npm run build"
 	},
 	"dependencies": {
-		"@mariozechner/pi-agent-core": "^0.16.0",
-		"@mariozechner/pi-ai": "^0.16.0",
-		"@mariozechner/pi-tui": "^0.16.0",
+		"@mariozechner/pi-agent-core": "^0.18.0",
+		"@mariozechner/pi-ai": "^0.18.0",
+		"@mariozechner/pi-tui": "^0.18.0",
 		"chalk": "^5.5.0",
 		"diff": "^8.0.2",
-		"glob": "^11.0.3"
+		"glob": "^11.0.3",
+		"jiti": "^2.6.1"
 	},
 	"devDependencies": {
 		"@types/diff": "^7.0.2",