pi-harness-runtime 1.1.166 → 1.1.167

package/AGENTS.md

@@ -0,0 +1,272 @@
+# Agent Instructions
+
+## Terminology: Two Task Tracking Systems (CRITICAL)
+
+This project has **TWO separate task trackers** with different purposes:
+
+| Term | Tool | Purpose |
+|------|------|---------|
+| **bd** / **beads** / **issues** | `bd create`, `bd list`, `bd close` | Issue tracker synced with GitHub issues |
+| **todo** / **todos** / **task list** | `todo` tool (JSON-RPC) | Local task list for in-session tracking |
+
+**When in doubt, ASK**: If a user mentions "tasks", "todos", or "track this" without specifying, ask:
+
+> **"Did you mean bd (issue tracker) or todo (local task list)?"**
+
+### When to use each:
+- **bd (beads)**: Long-lived issues, bugs, feature requests, GitHub-synced items
+- **todo (local task list)**: Short-lived session tasks, step-by-step workflows, immediate action items
+
+---
+
+## Pre-Commit/Build Checklist
+
+**BEFORE any `git commit`, `git push`, or `bench build`:**
+
+1. Run `bd ready` to check for pending tasks
+2. Check if any todo-list items are marked as done → **mark them complete** with `bd close <id>`
+3. Ensure all pending tasks are tracked in bd before committing
+4. If todos exist, acknowledge them in response before proceeding
+
+```bash
+# Check pending todos before commit
+bd ready
+bd list --status pending
+```
+
+**Why**: Agent often builds/commits without checking todos. This causes incomplete tasks to be forgotten.
+
+## Versioning Convention
+
+**Only patch-level updates (`0.10.x`)** — no minor/major bumps until the app is fully functional.
+
+When releasing:
+
+```bash
+bun scripts/release-all.ts --release-as patch   # default, use this
+bun scripts/release-all.ts --release-as minor   # only if explicitly requested
+bun scripts/release-all.ts --release-as major   # only if explicitly requested
+```
+
+## Release Tag Convention (CRITICAL)
+
+**ALWAYS release a tag after ANY code change.** Use patch version bump (0.10.x → 0.10.x+1).
+
+This repo uses tag-based GitHub Actions builds — no CI on push. Every code change should be tagged for release.
+
+```bash
+# After git push, immediately tag and push:
+git tag v0.10.32 && git push origin v0.10.32
+
+# Or use the release script:
+bun scripts/release-all.ts --release-as patch
+```
+
+**Rule**: Every `git push` should be followed by a tag push. No code change should go unreleased.
+
+## npm Publish Authentication
+
+- **Normal release path: do NOT run local `npm publish`.** Push the code and release tag; GitHub Actions will sync/publish to npm.
+- **GitHub Actions OIDC release path should not require `npm login`.**
+- If a **local/manual npm publish fallback** is needed and `npm whoami` fails, ask the user to run:
+
+  ```bash
+  npm login
+  ```
+
+  before retrying the publish.
+- Do not assume a previous local npm login is still valid.
+
+## Context-Mode Tool Priority
+
+Use tools in this order (highest to lowest priority):
+
+1. **`ctx_batch_execute`** — Multiple commands with queries in one call
+2. **`ctx_execute`** — Single command or data derivation
+3. **`ctx_execute_file`** — Read/analyze file before editing
+4. **`ctx_search`** — Query indexed knowledge base
+
+**File operations**: Always use `ctx_execute_file` to read/analyze files before editing.
+
+## File Copy Rule (Mimic/Clone Between Folders)
+
+**When asked to mimic, clone, or copy code/files from one folder to another:**
+
+1. **USE `cp` or `sudo cp`** instead of writing files from scratch
+2. **Do NOT write files from scratch** — copying is faster, more accurate, and preserves exact code
+3. **Use `sudo cp` when permissions require it** (different owners, protected directories)
+
+```bash
+# ✅ CORRECT - direct copy
+cp /path/to/source/file.ts /path/to/dest/file.ts
+
+# ✅ CORRECT - sudo for permission issues
+sudo cp /path/to/source/file.ts /path/to/dest/file.ts
+
+# ❌ WRONG - writing from scratch (loses fidelity, wastes time)
+write /path/to/dest/file.ts
+<content>
+```
+
+**Why**: Copying preserves exact code including comments, formatting, and subtle details. Writing from scratch introduces drift and incompleteness.
+
+## NPM Package Build Issues (Important!)
+
+### Bug: Missing `dist/` Directories in Packages
+
+**Problem**: When `packages/*/dist/` is gitignored and a package has `.js` imports but no compiled output, the published npm package will crash pi on load with:
+
+```
+Cannot find module '../packages/notification/dist/notification-center.js'
+```
+
+**Root Cause**:
+
+- The main build script (`bun run build`) only compiles packages with `tsconfig.json`
+- If TypeScript is missing from `devDependencies`, `node_modules/.bin/tsc` doesn't exist
+- The build loop silently ignores errors: `|| true`
+- Packages get published WITHOUT their `dist/` directories
+- Runtime imports from `dist/*.js` fail
+
+**Solution**:
+
+1. Ensure `typescript` is in `devDependencies` (so `tsc` is available)
+2. Run `bun run build` locally BEFORE pushing
+3. Verify `packages/*/dist/` contains compiled `.js` files
+4. Commit the `dist/` output or ensure build succeeds in CI
+
+```bash
+# Check if dist exists before publishing
+ls packages/notification/dist/
+
+# If missing, build and check again
+bun run build
+ls packages/notification/dist/
+```
+
+**Prevention**: Add `typescript` to `devDependencies`:
+
+```bash
+bun add -d typescript
+```
+
+## Icon Usage
+
+**This is a Node.js CLI daemon — do NOT add React as a dependency for icons.**
+
+### Default: ASCII (no extra deps, keeps bundle small)
+
+ASCII is the default choice for regular cases — it keeps the bundle small and works everywhere.
+
+```typescript
+// Stage indicators
+[>>>] [==>] [###] [---] [OK!]
+
+// Status icons
+> write   ✓ review   ✗ blocked   ~ retry
+
+// Compact flags (preferred for status lines)
+[P] [W] [R] [F]  // plan/write/review/finish
+```
+
+**Never use emoji** — they render inconsistently across terminals and fonts, and CJK terminals display them at double width which breaks layout alignment.
+
+### Nice/beautiful icons: Lucide (SVG, no React)
+
+Use `lucide` only when you need polished SVG icons for display in a UI (not terminal output).
+
+```typescript
+import { FileCode, CheckCircle, XCircle, AlertTriangle } from 'lucide';
+
+// Render to SVG string (no React, no DOM)
+const svg = FileCode({ size: 16, strokeWidth: 2 });
+// svg is an HTML string like '<svg width="16" height="16" ...>...</svg>'
+```
+
+Or use `lucide-static` for pre-built SVG strings — no runtime import:
+
+```typescript
+// No React, just static SVG strings
+const icon = await readFile('./node_modules/lucide-static/icons/file-code.svg', 'utf8');
+```
+
+**Never use `react-icons`** — it pulls in React as a dependency.
+
+## Todo List Rule (CRITICAL)
+
+**When you identify 2+ tasks, issues, or jobs — add ALL of them to the task list immediately.**
+
+Do NOT just list them in text. Use the `todo` tool to create tracked tasks:
+
+```bash
+# Creating multiple tasks at once (use todo tool, NOT bd!)
+todo create "Fix initialRemotionData not defined"
+todo create "Fix dictionary.title undefined"
+todo create "Replace Film icon with themed icon"
+```
+
+**Rule**: If you write something like:
+
+- "There are 3 issues: ..."
+- "I found several problems: ..."
+- "Let me fix these: 1) X 2) Y 3) Z"
+- "Here are the tasks to do: ..."
+
+**Then you MUST create Todo items for each one.** Single-issue fixes don't need tracking.
+
+**Why**: Agents often identify issues verbally but forget to track them. This rule ensures nothing slips through.
+
+**IMPORTANT**: The `todo` tool and `bd` commands are DIFFERENT systems. Use `todo` for local task tracking.
+
+## Remote Server Commands (SSH)
+
+**ALWAYS use detached SSH pattern to prevent long hangs.**
+
+### Anti-Pattern (blocks SSH, hangs ~2000s)
+
+```bash
+# WRONG - SSH blocks waiting for background process
+ssh user@host "kill old; sleep 2; uvicorn ... &"
+# SSH never returns because it waits for uvicorn
+```
+
+### Correct Pattern (fast exit, process stays on server)
+
+```bash
+# CORRECT - SSH exits fast, process continues on server
+ssh -o BatchMode=yes user@host \
+  "kill -9 917094 970065 2>/dev/null; sleep 1; \
+   cd /home/frappe/amos-saas/backend/services/api && \
+   nohup .venv/bin/uvicorn amos_api.main:app --host 0.0.0.0 --port 8100 \
+   > /tmp/uvicorn.log 2>&1 & \
+   echo 'UVICORN_STARTED'"
+```
+
+**Key changes:**
+
+- `nohup ... &` — detaches from SSH session
+- `> /tmp/uvicorn.log 2>&1` — captures output to file
+- `echo 'UVICORN_STARTED'` — SSH exits fast with confirmation
+- `-o BatchMode=yes` — non-interactive, fails fast on auth issues
+
+**Why this matters**: A bare `&` in SSH keeps the shell open. `nohup` + redirect + `echo` ensures SSH exits immediately while the process runs on the server.
+
+**For long-running commands**, add a timeout guard:
+
+```bash
+timeout 60s ssh -o BatchMode=yes user@host "long-command; echo DONE"
+```
+
+### Status files
+
+The `.harness-status` file is consumed by the pi host, which adds its own UI. Keep daemon output plain ASCII:
+
+```
+# ✅ Good — plain ASCII with compact flags
+[P] planning  |  MiniMax: 5h: 89% left
+[W] coding    |  MiniMax: 5h: 89% left
+
+# ❌ Bad — emoji in machine-readable files (CJK alignment issues)
+🧭 planning  |  MiniMax: 5h: 89% left
+✍️  coding   |  MiniMax: 5h: 89% left
+```

package/CLAUDE.md

@@ -0,0 +1,77 @@
+# Project Instructions for AI Agents
+
+This file provides instructions and context for AI coding agents working on this project.
+
+<!-- BEGIN BEADS INTEGRATION v:1 profile:minimal hash:ca08a54f -->
+## Beads Issue Tracker
+
+This project uses **bd (beads)** for issue tracking. Run `bd prime` to see full workflow context and commands.
+
+### Quick Reference
+
+```bash
+bd ready              # Find available work
+bd show <id>          # View issue details
+bd update <id> --claim  # Claim work
+bd close <id>         # Complete work
+```
+
+### Rules
+
+- Use `bd` for ALL task tracking — do NOT use TodoWrite, TaskCreate, or markdown TODO lists
+- Run `bd prime` for detailed command reference and session close protocol
+- Use `bd remember` for persistent knowledge — do NOT use MEMORY.md files
+
+## Session Completion
+
+**When ending a work session**, you MUST complete ALL steps below. Work is NOT complete until `git push` succeeds.
+
+**MANDATORY WORKFLOW:**
+
+1. **File issues for remaining work** - Create issues for anything that needs follow-up
+2. **Run quality gates** (if code changed) - Tests, linters, builds
+3. **Update issue status** - Close finished work, update in-progress items
+4. **PUSH TO REMOTE** - This is MANDATORY:
+   ```bash
+   git pull --rebase
+   bd dolt push
+   git push
+   git status  # MUST show "up to date with origin"
+   ```
+5. **Clean up** - Clear stashes, prune remote branches
+6. **Verify** - All changes committed AND pushed
+7. **Hand off** - Provide context for next session
+
+**CRITICAL RULES:**
+- Work is NOT complete until `git push` succeeds
+- NEVER stop before pushing - that leaves work stranded locally
+- NEVER say "ready to push when you are" - YOU must push
+- If push fails, resolve and retry until it succeeds
+<!-- END BEADS INTEGRATION -->
+
+
+## Build & Test
+
+_Add your build and test commands here_
+
+```bash
+# Example:
+# npm install
+# npm test
+```
+
+## Architecture Overview
+
+_Add a brief overview of your project architecture_
+
+## Conventions & Patterns
+
+### Git Workflow
+
+After running `git status`, **ALWAYS** run `git add --all` before asking about committing. Never assume changes are staged.
+
+```bash
+git status  # inspect changes
+git add --all  # stage everything
+git commit    # then proceed
+```

package/SECURITY.md

@@ -0,0 +1,21 @@
+# Security Policy
+
+## Supported Versions
+
+Use this section to tell people about which versions of your project are
+currently being supported with security updates.
+
+| Version | Supported          |
+| ------- | ------------------ |
+| 5.1.x   | :white_check_mark: |
+| 5.0.x   | :x:                |
+| 4.0.x   | :white_check_mark: |
+| < 4.0   | :x:                |
+
+## Reporting a Vulnerability
+
+Use this section to tell people how to report a vulnerability.
+
+Tell them where to go, how often they can expect to get an update on a
+reported vulnerability, what to expect if the vulnerability is accepted or
+declined, etc.

package/TODO-todo-bd-sync.md

@@ -0,0 +1,124 @@
+# TODO: Complete todo-bd-sync Implementation
+
+## Status: Phase 1 & 2 Complete
+
+### What was implemented
+
+1. **Package structure** (`packages/todo-bd-sync/`)
+   - ✅ `src/types.ts` - Type definitions for TodoTask, BdIssue, IdMappingRegistry
+   - ✅ `src/detector.ts` - Dependency detection (rpiv-todo, bd CLI)
+   - ✅ `src/sync.ts` - Two-way sync logic (todo ↔ bd)
+   - ✅ `src/index.ts` - Main API with TodoBdSync class
+   - ✅ `src/extension.ts` - pi-coding-agent extension entry
+   - ✅ `src/todo-reminder.ts` - Auto-reminder for continuing todos
+   - ✅ `package.json` - Package manifest
+
+2. **Integration into main extension**
+   - ✅ Added import and lazy initialization to `index.ts`
+
+3. **Documentation**
+   - ✅ `docs/todo-bd-sync.md` - Full documentation with auto-reminder feature
+   - ✅ `README.md` - Added section and commands
+
+---
+
+## New: Auto-Reminder Feature (Phase 2)
+
+### Problem Fixed
+
+- **Bug**: LLM completes tasks but stays silent about remaining todos
+- **User had to manually ask**: "continue with todos" or "wrap up"
+
+### Solution
+
+The `todo-reminder.ts` module injects reminders when todos remain:
+
+**Prerequisite**: Tasks must be created with `bd create` (not via `/todo` command).
+
+```typescript
+// After agent completes a task:
+// Checks bd for remaining todos
+// Injects message:
+"Remaining todos detected:
+[bd-a1b2] Review PR (pending)
+
+Please continue with the next pending task or ask the user which to prioritize."
+```
+
+### How It Works
+
+1. Hooks into `agent_end` and `tool_execution_end`
+2. Calls `bd ready --json` to check remaining todos
+3. If pending tasks exist, sends reminder via `pi.sendUserMessage()`
+4. 5-second cooldown to prevent spam
+
+---
+
+## Remaining Tasks
+
+### P1: Test Basic Integration
+
+```bash
+# Restart pi and test
+pi
+
+# Check if todo-bd-sync is enabled
+/bd-todo-status
+```
+
+### P2: Test Auto-Reminder
+
+1. Create a multi-step task via bd:
+
+   ```bash
+   bd create "Build API" -p 2
+   bd create "Add tests" --parent <parent-id>
+   bd create "Deploy" --parent <parent-id>
+   ```
+
+2. Ask LLM to work on it:
+
+   ```
+   Build the API and manage tasks with bd
+   ```
+
+3. After completing a task, verify reminder is injected:
+   - Should see message about remaining todos
+   - LLM should offer to continue
+
+### P3: Tune Reminder Behavior
+
+- Adjust cooldown (currently 5 seconds)
+- Customize reminder message template
+- Test with various task counts
+
+---
+
+## Files Summary
+
+| File | Purpose |
+|------|---------|
+| `src/types.ts` | Type definitions |
+| `src/detector.ts` | Check dependencies |
+| `src/sync.ts` | bd CLI integration |
+| `src/todo-reminder.ts` | **NEW: Auto-reminder injection** |
+| `src/index.ts` | Main API |
+| `src/extension.ts` | Extension entry |
+
+---
+
+## Testing Plan
+
+1. **Basic sync**: Create bd issue → verify sync
+2. **Auto-reminder**: Multi-step task → verify reminder appears
+3. **Integration**: Full workflow test
+
+---
+
+## Notes
+
+- The implementation uses lazy imports to avoid breaking if dependencies are missing
+- bd CLI must be installed for full functionality
+- rpiv-todo is optional (fallback mode available)
+- Auto-reminder does NOT modify the overlay (it works fine)
+- Only injects context to prompt LLM to continue

package/auto-resume-fix-prompt.md

@@ -0,0 +1,189 @@
+# Auto-Resume Fix After Context Overflow Compaction
+
+## Problem Statement
+
+After context overflow compaction with `willRetry=false`, the agent does NOT automatically resume. The session pauses waiting for user input, even though context-mode has a resume snapshot that should be delivered.
+
+## Root Cause Analysis
+
+### The Compaction Flow
+
+1. **Context overflow occurs** -> `_checkCompaction()` is called
+2. **For overflow with `stopReason === "stop"`** (the common case):
+   - `willRetry = assistantMessage.stopReason !== "stop"` -> **FALSE**
+   - Calls `_runAutoCompaction("overflow", false)`
+3. **Inside `_runAutoCompaction`** (line 1685):
+
+   ```javascript
+   return this.agent.hasQueuedMessages();  // <-  Returns FALSE for overflow case!
+   ```
+
+4. **Result**: Session pauses waiting for user input
+
+### Why Context-Mode Resume Wasn't Delivered
+
+Context-mode builds resume snapshot in `session_before_compact` and stores it in DB:
+
+```javascript
+// From extension.js
+pi.on("session_before_compact", () => {
+    const snapshot = buildResumeSnapshot(allEvents, { compactCount });
+    db.upsertResume(_sessionId, snapshot, allEvents.length);
+});
+```
+
+But resume was only injected in `before_agent_start` via the `context` hook:
+
+```javascript
+pi.on("before_agent_start", async (event, ctx) => {
+    const resume = db.getResume(_sessionId);
+    if (resume && !resume.consumed && resume.snapshot) {
+        parts.push(resume.snapshot);
+        db.markResumeConsumed(_sessionId);
+    }
+    // ...
+});
+```
+
+**Problem**: `before_agent_start` only fires when a **new prompt is submitted**. After overflow compaction with `willRetry=false`, no prompt is submitted -> resume is never delivered.
+
+## Solution Implemented: Post-Compaction Message Queue
+
+Added a new extension API `queuePostCompactionMessage()` that allows extensions to queue messages that should be delivered after compaction completes, even when `willRetry=false`.
+
+### Flow After Fix
+
+1. `session_before_compact` -> Build resume snapshot in DB
+2. `session_compact` -> Queue resume via `ctx.queuePostCompactionMessage()`
+3. After compaction completes -> `drainPostCompactionMessages()` delivers queued messages
+4. Agent continues and delivers resume automatically
+
+## Files Changed
+
+### 1. `pi-coding-agent/dist/core/extensions/types.d.ts`
+
+Added new API to `ExtensionContext` and `ExtensionContextActions`:
+
+```typescript
+// ExtensionContext
+/**
+ * Queue a message to be delivered after compaction completes.
+ * When willRetry=false (e.g. overflow without context recovery), the agent
+ * will continue and deliver these queued messages before pausing for user input.
+ * Useful for resuming context-mode's snapshot after overflow compaction.
+ */
+queuePostCompactionMessage(message: { role: string; content: string }): void;
+
+// ExtensionContextActions
+queuePostCompactionMessage: (message: { role: string; content: string }) => void;
+```
+
+### 2. `pi-coding-agent/dist/core/extensions/runner.js`
+
+Added new methods to `ExtensionRunner`:
+
+```javascript
+// Instance variable
+_postCompactionMessages = [];
+
+// Method to queue messages (called from extension context)
+queuePostCompactionMessageFn = (message) => {
+    this._postCompactionMessages.push(message);
+};
+
+// Method to get and clear messages (called by agent-session after compaction)
+drainPostCompactionMessages() {
+    if (this.staleMessage) {
+        return [];  // Don't drain if stale
+    }
+    const messages = this._postCompactionMessages;
+    this._postCompactionMessages = [];
+    return messages;
+}
+
+// Check if runner is stale (without throwing)
+isStale() {
+    return Boolean(this.staleMessage);
+}
+```
+
+### 3. `pi-coding-agent/dist/core/agent-session.js`
+
+Modified `_runAutoCompaction()` to check for and deliver post-compaction messages:
+
+```javascript
+// After compaction completes (line 1682)
+this._emit({ type: "compaction_end", reason, result, aborted: false, willRetry });
+
+if (willRetry) {
+    // ... existing retry logic
+    return true;
+}
+
+// Check for post-compaction messages from extensions
+if (this._extensionRunner && !this._extensionRunner.isStale()) {
+    const postCompactionMessages = this._extensionRunner.drainPostCompactionMessages();
+    for (const msg of postCompactionMessages) {
+        await this.agent.followUp(msg.content);
+    }
+    if (postCompactionMessages.length > 0) {
+        return true; // Continue to deliver the queued messages
+    }
+}
+
+// Fall back to existing behavior
+return this.agent.hasQueuedMessages();
+```
+
+### 4. `context-mode/build/adapters/pi/extension.js`
+
+Modified `session_compact` handler to queue resume:
+
+```javascript
+pi.on("session_compact", (event, ctx) => {
+    try {
+        if (!_sessionId)
+            return;
+        db.incrementCompactCount(_sessionId);
+
+        // Queue resume snapshot for post-compaction delivery when willRetry=false
+        if (!event.willRetry) {
+            const resume = db.getResume(_sessionId);
+            if (resume && !resume.consumed && resume.snapshot) {
+                ctx.queuePostCompactionMessage?.({
+                    role: "user",
+                    content: resume.snapshot,
+                });
+                db.markResumeConsumed(_sessionId);
+            }
+        }
+    }
+    catch {
+        // best effort
+    }
+});
+```
+
+## Testing
+
+To verify the fix works:
+
+1. **Trigger context overflow** with a long conversation
+2. **Observe compaction happens automatically**
+3. **Agent continues WITHOUT user input**
+4. **Resume snapshot is delivered** and agent continues working
+
+Check with:
+
+```bash
+pi
+# Run a long conversation that triggers overflow
+# Verify agent continues automatically after compaction
+/ctx-stats
+```
+
+## Backward Compatibility
+
+- The new `queuePostCompactionMessage` API is optional (uses `?.` when calling)
+- Extensions that don't use it continue to work unchanged
+- Existing behavior for `willRetry=true` cases is preserved