@ebrindley/build 0.0.0-stage → 1.42.1-rc28

package/docs/TROUBLESHOOTING.md

@@ -0,0 +1,2209 @@
+# Troubleshooting Guide
+
+> **Comprehensive troubleshooting guide for Build AI orchestration CLI**
+>
+> Common errors, solutions, and debugging techniques based on real-world issues
+
+---
+
+## Table of Contents
+
+- [Quick Reference](#quick-reference)
+- [Disk Filled by Provider Temp Files](#disk-filled-by-provider-temp-files)
+- [Recovering Competition Results](#recovering-competition-results)
+- [Flywheel Operator Recovery](#flywheel-operator-recovery)
+- [Common Errors & Solutions](#common-errors--solutions)
+- [Error Codes & Meanings](#error-codes--meanings)
+- [Provider-Specific Issues](#provider-specific-issues)
+- [Installation & Setup Problems](#installation--setup-problems)
+- [Debugging Tips](#debugging-tips)
+- [Advanced Troubleshooting](#advanced-troubleshooting)
+
+---
+
+## Quick Reference
+
+| Issue Type | Quick Fix | Full Guide |
+|------------|-----------|------------|
+| **Disk pressure** | `build cleanup provider-temp --use-default-roots` | [Disk Filled by Provider Temp Files](#disk-filled-by-provider-temp-files) |
+| **LaunchAgent stale** | `build cleanup temp-agent --status` | [LaunchAgent Stale After Repo Move](#launchagent-stale-after-repo-move) |
+| **Stale CLI Build** | `./scripts/compile-dist.sh` | [Stale CLI Build](#stale-cli-build) |
+| **Missing Dependencies** | `npm ci` | [Missing Runtime Dependencies](#missing-runtime-dependencies) |
+| **Authentication** | Check API keys | [Provider-Specific Issues](#provider-specific-issues) |
+| **Grok auth/readiness** | `grok login --device-auth` (or `grok login` / `grok`); optional `XAI_API_KEY` for CI/headless | [Grok Issues](#grok-issues) |
+| **Installation** | Verify CLI tools | [Installation & Setup Problems](#installation--setup-problems) |
+| **Execution** | Use `--verbose` flag | [Debugging Tips](#debugging-tips) |
+| **Codex Timeout** | Check bot comments | [Codex Issues](#codex-issues) |
+| **Parallel Failures** | Reduce concurrency | [Parallel Execution Problems](#parallel-execution-problems) |
+| **Worktree cleaned up** | Use `build run competition apply` | [Recovering Competition Results](#recovering-competition-results) |
+| **Autoloop stuck or unclear** | `build loop explain <taskId>` | [Autoloop Diagnosis](#autoloop-diagnosis) |
+
+---
+
+## Disk Filled by Provider Temp Files
+
+**Symptoms**: Disk full warnings on a development machine, or `df -h` shows `/` or `/private` near capacity. Common on always-on machines that run long Claude Code sessions.
+
+**Cause**: provider CLIs write temp files during execution, and a run that is interrupted or killed leaves them behind. Two directories accumulate the most:
+
+- `/private/tmp/claude-<UID>/` — Claude Code writes sandbox isolation files here during sessions
+- `/private/tmp/claude-mcp-npm-cache` — npm package cache used by MCP server processes; redirected here intentionally to keep it out of `~/.npm`
+
+> macOS may clear `/private/tmp` on its own schedule, but this is not reliable on always-on machines and should not be relied upon when disk pressure is active.
+
+### Quick fix (one-time dry-run)
+
+```bash
+# Preview what would be deleted (no changes made)
+build cleanup provider-temp --use-default-roots
+
+# Run the actual cleanup (files older than 24h by default)
+build cleanup provider-temp --use-default-roots --execute
+```
+
+### Automated cleanup (recommended for long-running machines)
+
+Install a macOS LaunchAgent that runs cleanup daily in the background by default:
+
+```bash
+build cleanup temp-agent --install-launchagent --use-default-roots
+```
+
+Logs are written to `~/.build/logs/temp-cleanup.log`. To check status:
+
+```bash
+build cleanup temp-agent --status
+```
+
+### Configuration
+
+Override the defaults in `.build-project/config/build.config.jsonc`:
+
+```js
+cleanup: {
+  providerTemp: {
+    roots: ['/private/tmp/claude-<UID>', '/private/tmp/claude-mcp-npm-cache'],
+    maxAgeHours: 24,  // default: 24
+  }
+}
+```
+
+When `cleanup.providerTemp.roots` is set in config, `build cleanup temp-agent --install-launchagent` uses those roots without requiring `--use-default-roots`.
+
+### LaunchAgent stale after repo move
+
+If you move or rebuild the Build repo, the LaunchAgent wrapper embeds the old binary path and silently skips cleanup. Check for staleness:
+
+```bash
+build cleanup temp-agent --status
+# Shows "MISSING" next to Binary or Node if the embedded paths are gone
+```
+
+Fix:
+
+```bash
+build cleanup temp-agent --install-launchagent --use-default-roots
+# Reinstalls the wrapper with current binary and Node paths
+```
+
+---
+
+## Recovering Competition Results
+
+If the worktree was cleaned up before you could apply competition results (e.g. the output showed "Worktree cleaned up. Recover via: ..."):
+
+Before applying, check out a delivery branch. Non-dry-run
+`build run competition apply` is intentionally blocked on `run/*`, `validation/*`, and
+generated `build/*` artifact branches. Use `fix/*` for product code, `docs/*`
+for documentation-only patches, or `test/*` for test-only patches.
+
+### Method 1: Use `build run competition apply` (Recommended)
+
+```bash
+# Apply winner's changes from most recent competition
+build run competition apply
+
+# Apply specific competition
+build run competition apply <task-id>
+
+# Preview without applying
+build run competition apply <task-id> --dry-run
+```
+
+The command recovers the diff from telemetry (state/variants) when the worktree directory no longer exists.
+
+### Method 2: Manual recovery from telemetry
+
+If `build run competition apply` is unavailable or you need to inspect the raw diff:
+
+1. Find the variant state file: `ls ~/.build/data/projects/<project-id>/telemetry/state/variants/ | grep <task-id>`
+2. Extract the diff: `cat ~/.build/data/projects/<project-id>/telemetry/state/variants/<filename>.json | jq -r '.worktreeDiff' > changes.patch`
+3. Apply the patch: `git apply changes.patch`
+
+---
+
+## Autoloop Diagnosis
+
+Use these commands when `build run --autoloop` stops unexpectedly, appears
+stuck, or reports an unclear completion status:
+
+```bash
+build run autoloop history --last 10
+build run autoloop stats
+build loop explain <taskId>
+```
+
+`build run autoloop history` lists recent CLI and Flywheel autoloop runs.
+`build run autoloop stats` summarizes aggregate outcomes. `build loop explain`
+shows a run's termination reason, completion summary, blocking issues, gate
+state, semantic validation state, artifact paths, and worktree path when
+available.
+
+If durable run artifacts exist but `build run autoloop history` is incomplete,
+rebuild the append-only ledger:
+
+```bash
+build run autoloop backfill
+```
+
+Timebox units differ by surface: `--autoloop-timebox` is whole minutes, while
+`BUILD_AUTOLOOP_TIMEBOX_MS` is milliseconds. `BUILD_AUTOLOOP_TIMEBOX` remains
+supported as a legacy millisecond alias. If an env value was intended as
+minutes, convert it before retrying. When both environment variables are set,
+the explicit `_MS` variable takes precedence.
+
+Autoloop always runs in an isolated worktree. Use `--preserve-worktrees` before
+retrying when you need to inspect failed or partial edits. `--autoloop
+--in-place` is refused before any provider runs.
+
+Only one CLI autoloop can hold the lock per checkout at a time (a linked
+worktree has its own lock). If a new run reports an active lock after a crash,
+inspect recent history and check that no owning process is still running. A
+holder process that is dead on the same host is reclaimed automatically, and
+any other stale lock is taken over once it is older than the 90-minute stale
+window. The lock itself is the sidecar `.build-project/autoloop/.run.lock.lock`;
+`.build-project/autoloop/.run.lock` is the holder record used for the automatic
+reclaim. If manual removal is needed, remove the `.run.lock.lock` sidecar only
+when the process is gone, and do not delete the holder record alone.
+
+---
+
+## Flywheel Operator Recovery
+
+Use this sequence when operating an independent Flywheel workflow or recovering
+from an incomplete run:
+
+```bash
+# Before provider execution
+build run flywheel catalog
+build run flywheel validate <workflow>
+build run flywheel plan <workflow>
+build run flywheel run <workflow>              # preview only
+build run flywheel run <workflow> --execute
+
+# After the run prints a workflowId
+build run flywheel status <workflow-id>
+build run flywheel inspect <workflow-id>
+
+# Review before integrating
+git diff <baseSha>..build/workflow/<workflow-id>   # build/sprint/<workflow-id> for a sprint
+build run flywheel merge-result <workflow-id> --dry-run
+build run flywheel merge-result <workflow-id>
+
+# Cleanup is preview-only by default
+build run flywheel cleanup <workflow-id>
+build run flywheel cleanup <workflow-id> --execute
+```
+
+If a competition stage needs manual repair, fix the run branch and then re-run its
+persisted quality gates with `build run flywheel reverify <workflow-id> <stage-id> --dry-run`.
+After reviewing the commands, use `--yes` for non-interactive reverification.
+`reverify` applies to competition stages only.
+
+If a review gate stage shows `halted-awaiting-user` in `build run flywheel status <id>`,
+the authorized repair loop could not clear a reviewer finding. Read the finding
+(and its `evidence`, when the reviewer supplied one), then dispose it:
+
+```bash
+build run flywheel gate <workflow-id> <stage-id> --reject F1 --reason "why it is a false positive" --dry-run
+build run flywheel gate <workflow-id> <stage-id> --reject F1 --reason "..." --yes
+```
+
+Use `--accept` instead when the defect is real but will be handled outside this
+run. The gate re-runs with the disposed IDs excluded; if other stages never ran,
+resume the workflow (`build run sprint --resume <id>` / `build run flywheel run --resume <id>`)
+before `merge-result`. Do not reach for `merge-result --skip-verify`: it also
+disables post-merge verification and the tree-fingerprint proof.
+`flywheel gate` handles top-level gate stages; a gate nested under an orchestrator
+seeded plan is reported as unsupported.
+
+---
+
+## Common Errors & Solutions
+
+### CLI Task Description Quoting Issues
+
+**Error**: `Error: Too many arguments provided` when running `build run competition` or `build run`
+
+**Symptoms**: Command fails immediately with "too many arguments" error message.
+
+**Cause**: Task descriptions containing spaces must be quoted. Without quotes, the shell splits the description into multiple arguments.
+
+**Resolution**:
+```bash
+#  Wrong (unquoted task description):
+build run competition Add error handling to auth.ts --variant claude:local
+
+#  Correct (quoted task description):
+build run competition "Add error handling to auth.ts" --variant claude:local
+```
+
+**Improved Error Messages (v0.1.0+):**
+The CLI now provides helpful guidance when this error occurs:
+```
+ Error: Too many arguments provided
+
+ Tip: Task description must be quoted if it contains spaces:
+   Correct: build run competition "Task description here" --variant ...
+   Wrong:    build run competition Task description here --variant ...
+```
+
+**Prevention:**
+- Always quote task descriptions that contain spaces
+- Use single quotes if description contains double quotes: `'Task with "quoted" text'`
+- Use backslash escaping for complex cases: `"Task with \"nested\" quotes"`
+- For long or multiline tasks, prefer `--task-file`:
+  - `build run --task-file task.txt`
+  - `build run competition --task-file task.txt --variant ...`
+
+### Stale CLI Build
+
+**Error**: `Failed to load CLI: Dynamic require of "fs" is not supported` or `Cannot find module './index.js'`
+
+**Symptoms**:
+- CLI commands fail immediately on startup
+- Error message mentions "Dynamic require" or missing modules
+- Occurs after `git pull`, `git checkout`, or editing source files
+
+**Cause**: The compiled CLI in `dist/` is missing or outdated. This happens because:
+- `dist/` is gitignored (not tracked in version control)
+- Source files (`src/`) changed after your last build
+- The `prepare` hook only runs during `npm install`, not after git operations
+
+**Resolution**:
+
+1. **Quick fix** - Rebuild and rewire the local CLI:
+   ```bash
+   ./scripts/compile-dist.sh
+   ```
+
+2. **Compilation-only alternative** - Rebuild `dist/` without relinking:
+   ```bash
+   npm run build
+   ```
+
+3. **Verify the build**:
+   ```bash
+   # Check that dist/ was created and is recent
+   ls -la dist/entry.js
+   ```
+
+**When to rebuild**:
+-  After `git pull` or `git fetch && git merge`
+-  After `git checkout` to a different branch
+-  After editing any files in `src/` directory
+-  After `git cherry-pick`, `git rebase`, or similar operations
+-  NOT needed after `npm install` (automatic via `prepare` hook)
+
+**Improved error message (v0.1.0+)**:
+
+The CLI now detects stale builds and provides clear instructions:
+
+```
+ CLI build is missing or outdated
+
+ How to fix:
+  1) Rebuild and rewire the CLI:
+     ./scripts/compile-dist.sh
+
+  2) Or compile dist/ only:
+     npm run build
+
+ Common causes:
+  • git pull/checkout changed source files
+  • Local edits to src/ directory
+  • Deleted dist/ directory
+
+For more help: docs/TROUBLESHOOTING.md#stale-cli-build
+```
+
+**Prevention**:
+- Remember to run `./scripts/compile-dist.sh` after pulling source changes
+- Add to your workflow: `git pull && ./scripts/compile-dist.sh`
+- Consider setting up a git post-merge hook for automatic rebuilds
+
+**Technical details**:
+- The CLI uses ESM modules with dynamic imports for performance
+- When `dist/` is stale, the import chain fails with cryptic Node.js module errors
+- The build process (via tsup) bundles TypeScript source into optimized JavaScript
+
+### Missing Runtime Dependencies
+
+**Error**: `Missing runtime dependencies for Build CLI in /path/to/install: <package>, ...`
+
+**Symptoms**:
+- `build run competition` or `build run` fails immediately with a missing dependencies message
+
+**Cause**: Node.js runtime dependencies declared in `package.json` are not installed. This typically happens after:
+- Cloning the repo without running `npm ci`
+- Switching branches where `package.json` changed
+- Deleted or corrupt `node_modules/`
+
+**Resolution**:
+```bash
+npm ci
+npm run build
+```
+
+**Bypass** (for test/mock environments):
+
+| Variable | Effect |
+|----------|--------|
+| `BUILD_SKIP_RUNTIME_DEP_CHECKS=1` | Skip only the runtime dependency preflight |
+| `BUILD_SKIP_TOOLING_CHECKS=1` | Skip all tooling preflight checks (includes dependency check) |
+| `BUILD_E2E_TEST=1` | Skip preflight (end-to-end test harness) |
+| `BUILD_MOCK_PROVIDERS=1` (with test runner or `BUILD_ALLOW_MOCK_PROVIDERS=1`) | Skip preflight when mock providers are actually permitted; outside tests, both flags are required or startup scrubs the mock flag |
+
+These env vars are intended for CI, test harnesses, and environments where dependencies are lazily or externally provided. Do not use them to mask a broken local install.
+
+### Quick-Mode Apply Conflicts (`build --quick`)
+
+**Error**: `Failed to apply worktree diff to repo` or `Cannot apply patch - conflicts detected`
+
+**Symptoms**:
+- Quick-mode execution (`build --quick`) completes successfully in the isolated worktree
+- Quality gates pass (if enabled)
+- Apply step fails with conflict errors
+- Worktree is preserved for inspection
+- Patch file saved to `.build-project/tmp/do-apply-failures/`
+
+**Cause**: The changes made in the isolated worktree conflict with the current state of your repository. This happens when:
+- Repository HEAD moved during quick-mode execution (someone pushed changes)
+- Working tree has uncommitted changes that conflict with worktree changes
+- File was modified/deleted in repo since worktree was created
+- Merge conflict between worktree patch and current repo state
+
+**Resolution**:
+
+The error output provides four recovery options:
+
+**Option 1: Inspect Changes (Recommended first step)**
+```bash
+# Navigate to the preserved worktree
+cd <preserved-worktree-path-from-output>
+
+# View all changes made
+git diff <baseline-sha>
+
+# View specific file changes
+git diff <baseline-sha> -- path/to/file.ts
+```
+
+**Option 2: Manual Apply with 3-Way Merge**
+```bash
+# Attempt manual apply with conflict resolution
+git apply --3way .build-project/tmp/do-apply-failures/apply-failed-<timestamp>.patch
+
+# If conflicts occur, you'll see markers in files:
+# <<<<<<< ours (current repo state)
+# =======
+# >>>>>>> theirs (worktree changes)
+
+# Fix conflicts manually, then:
+git add <conflicted-files>
+git status  # Verify resolution
+```
+
+**Option 3: View Patch Content**
+```bash
+# Inspect what the patch would do
+cat .build-project/tmp/do-apply-failures/apply-failed-<timestamp>.patch
+
+# Or use less for large patches
+less .build-project/tmp/do-apply-failures/apply-failed-<timestamp>.patch
+```
+
+**Option 4: Cherry-Pick from Worktree**
+```bash
+# Navigate to worktree
+cd <preserved-worktree-path-from-output>
+
+# Find the commit(s) made by the provider
+git log --oneline -n 5
+
+# Cherry-pick specific commit(s) into main repo
+cd <back-to-main-repo>
+git cherry-pick <commit-sha-from-worktree>
+```
+
+**After Recovery:**
+```bash
+# Clean up the preserved worktree
+build cleanup stale --only worktrees --worktree-age 0 --execute
+```
+
+**Prevention:**
+
+1. **Keep repo clean** - Commit or stash changes before running `build --quick`:
+   ```bash
+   git status  # Check for uncommitted changes
+   git stash   # Or commit them
+   build --quick "task"
+   ```
+
+2. **Use `--allow-dirty` carefully** - Only when you understand the risk:
+   ```bash
+   build --quick "task" --allow-dirty
+   ```
+
+3. **Check for upstream changes** - Pull latest before long-running tasks:
+   ```bash
+   git pull
+   build --quick "task"
+   ```
+
+4. **Use `--preserve-worktrees` for debugging** - Inspect without auto-cleanup:
+   ```bash
+   build run "task" --preserve-worktrees --verbose
+   ```
+
+**Related:**
+- [build](CLI_REFERENCE.md#build) command reference
+- Worktree isolation behavior and its limits; isolation is not a comprehensive safety guarantee
+
+### Authentication Failures
+
+> **Info: Authentication at Runtime**
+>
+> Provider execution defers authentication to the underlying CLI. Auth errors surface at
+> execution time with actionable guidance. Routing and dry-run may perform advisory
+> pre-verification, but are not a universal gate. Use `build provider list` for readiness verification.
+
+#### Claude Authentication Issues
+
+**Error**: `Authentication failed` or CLI reports not authenticated
+
+**Solutions**:
+```bash
+# 1. Re-authenticate via Claude CLI
+claude auth logout
+claude auth login
+
+# 2. Verify CLI installation
+which claude
+claude --version
+```
+
+**Common Causes**:
+- Claude CLI session expired
+- Claude CLI not installed or not in PATH
+- Wrong config directory
+
+#### Gemini Authentication Issues
+
+**Error**: `GEMINI_API_KEY not found` or `Authentication failed`
+
+**Solutions**:
+```bash
+# 1. Set API key
+export GEMINI_API_KEY='your-api-key-here'
+
+# 2. Verify key format (should start with AIza...)
+echo $GEMINI_API_KEY | head -c 10
+
+# 3. Test with simple request
+build run "test" --engine gemini --model gemini-2.5-flash
+```
+
+**Common Causes**:
+- Missing `GEMINI_API_KEY` environment variable
+- Invalid API key format
+- Quota exceeded (check Google AI Studio)
+
+#### Grok Authentication Issues
+
+**Error**: `XAI_API_KEY not found`, Grok readiness is `unknown`, or browser auth is required
+
+**Solutions**:
+```bash
+# 1. Install Grok through Build's guarded setup path
+BUILD_ALLOW_CURL_BASH=1 build provider setup --providers grok
+
+# 2. Primary for local use: cached OAuth / device login via the Grok CLI
+grok login --device-auth
+# or: grok login
+# or first interactive launch: grok
+
+# 3. Optional for CI/headless automation (official key)
+export XAI_API_KEY='your-key-here'
+# Deprecated Build compatibility alias (still accepted; not preferred):
+# export GROK_CODE_XAI_API_KEY='your-key-here'
+
+# 4. Inspect passive readiness (key presence remains unverified)
+build provider list
+build doctor --provider grok
+```
+
+**Common Causes**:
+- Grok CLI is not installed or not on `PATH`
+- Cached CLI OAuth / device login has not been completed (`~/.grok` auth missing)
+- CI/headless runs have neither a usable CLI auth cache nor `XAI_API_KEY` (or the deprecated `GROK_CODE_XAI_API_KEY` alias)
+- Sandboxed agent sessions cannot read Grok's CLI auth state
+- `BUILD_GROK_ALLOW_BROWSER_AUTH=1` was not set when deliberately allowing browser auth probes (opt-in only; not safe by default in headless/CI)
+
+#### Cursor Authentication Issues
+
+**Error**: `ConnectError: [invalid_argument]` or `Cursor CLI not found`
+
+**Solutions**:
+```bash
+# 1. Install Cursor Agent
+curl https://cursor.com/install -fsS | bash
+
+# 2. Verify installation
+cursor-agent --version
+```
+
+**Common Causes**:
+- Cursor CLI not installed
+- Invalid API key
+- Plan mode conflicts (try disabling)
+
+#### Codex Authentication Issues
+
+**Error**: `Invalid token` or `Command not found: codex`
+
+**Solutions**:
+```bash
+# 1. Install Codex CLI
+npm install -g @openai/codex
+
+# 2. Verify installation
+codex --version
+
+# 3. Authenticate
+codex login
+
+# 4. Verify via Build
+build doctor --provider codex
+```
+
+**Common Causes**:
+- Codex CLI not installed
+- Invalid or expired token
+- Wrong environment variable name
+
+### Installation Issues
+
+#### Missing CLIs
+
+**Error**: `'claude' is not available on this system` or similar
+
+**Solutions**:
+```bash
+# 1. Install all required CLIs
+npm install -g @anthropic-ai/claude-code
+npm install -g @openai/codex
+curl https://cursor.com/install -fsS | bash
+
+# 2. Verify installations
+claude --version
+codex --version
+cursor-agent --version
+
+# 3. Add to PATH if needed
+export PATH="$PATH:$(npm config get prefix)/bin"
+```
+
+#### PATH Problems
+
+**Error**: `Command not found` despite installation
+
+**Solutions**:
+```bash
+# 1. Find where npm installs global packages
+npm config get prefix
+
+# 2. Add to PATH
+export PATH="$PATH:$(npm config get prefix)/bin"
+
+# 3. Make permanent (add to ~/.bashrc or ~/.zshrc)
+echo 'export PATH="$PATH:$(npm config get prefix)/bin"' >> ~/.bashrc
+source ~/.bashrc
+
+# 4. Verify PATH
+which claude
+which codex
+which cursor-agent
+```
+
+#### Node.js Version Issues
+
+**Error**: `Unsupported Node.js version` or `Module not found`
+
+**Solutions**:
+```bash
+# 1. Check Node.js version
+node --version
+# Requires: Node.js 24.x or 26.x
+
+# 2. Update Node.js (using nvm)
+nvm install 24
+nvm use 24
+
+# 3. Clear npm cache
+npm cache clean --force
+
+# 4. Reinstall dependencies
+rm -rf node_modules package-lock.json
+npm install
+```
+
+### Execution Failures
+
+#### Timeout Errors
+
+**Error**: `Timeout waiting for task to complete`
+
+**Solutions**:
+```bash
+# 1. Increase timeout
+build run "task" --timeout 60
+
+# 2. Use faster model
+build run "task" --model gemini-2.5-flash
+
+# 3. Check provider status
+build provider
+
+# 4. Use verbose mode for details
+build run "task" --verbose
+```
+
+**Common Causes**:
+- Task too complex for timeout limit
+- Provider rate limiting
+- Network connectivity issues
+- Memory pressure (check system resources)
+
+#### Memory Issues
+
+**Error**: `Memory limit exceeded` or `Out of memory`
+
+The periodic MemoryGuardian monitors the Node process where it is started:
+heap occupancy relative to V8's limit, plus that process's RSS when explicitly
+capped. It can terminate owned child work to reduce incoming load. Competition
+parents use their process-limiter worker PIDs when no provider PIDs are tracked
+directly; both critical and emergency handling use that same ownership rule.
+Workers evaluate individual child RSS through the lifecycle manager separately,
+even when their periodic guardian is not started. Without an explicit RSS cap,
+that child evaluation only warns. Host pressure and validation pacing are
+separate controls; the guardian percentage is not a host RAM utilization reading.
+
+**Solutions**:
+```bash
+# 1. Check memory usage
+build run "task" --verbose
+# Look for: "Memory: XX% heap"
+
+# 2. Reduce parallel execution
+build run competition "task" --variant claude --max-concurrent 2
+
+# 3. Configure guardian behavior
+#    - Log-only (observe without killing)
+export BUILD_MEMORY_KILL=0
+#    - Aggressive kill mode (stress testing)
+export BUILD_MEMORY_KILL=1
+#    - Explicit per-process RSS cap (e.g., 2GB; not a process-tree total)
+export BUILD_RSS_LIMIT_MB=2048
+
+# 4. Disable periodic monitoring and heap pacing (child RSS evaluation remains active)
+export BUILD_MEMORY_GUARD=0
+
+# 5. Increase Node.js memory limit
+node --max-old-space-size=4096 build.js run "task"
+```
+
+**Common Causes**:
+- Large codebase processing
+- Too many parallel executions
+- Memory leaks in provider code
+- Insufficient system RAM
+
+#### Crashes
+
+**Error**: `Process exited with code 1` or `Segmentation fault`
+
+**Solutions**:
+```bash
+# 1. Check system resources
+free -h
+df -h
+
+# 2. Run with verbose logging
+build run "task" --verbose
+
+# 3. Check logs (resolved telemetry dir; `build doctor --verbose` prints it)
+TELEMETRY_DIR=~/.build/data/telemetry
+ls -la "$TELEMETRY_DIR"/logs/
+
+# 4. Test with simple task
+build run "Create hello.js" --engine claude
+```
+
+**Common Causes**:
+- System resource exhaustion
+- Provider-specific bugs
+- Corrupted configuration
+- Node.js version incompatibility
+
+#### Run ends when the terminal closes (SIGHUP)
+
+**Symptom**: a run started with `nohup build ... &` stops when the launching
+terminal or ssh session ends. The log shows `Received SIGHUP, running cleanup
+handlers` and the exit status is 129.
+
+**Cause**: Build installs its own SIGHUP handler to clean up its child
+processes. In one observed macOS probe (Node 24), `nohup node ...` was killed by
+SIGHUP while `nohup sleep` survived, so `nohup` did not protect a Node process.
+This was observed once; it is not established for every platform or Node version.
+
+**Fix**: run work that must outlive the terminal inside `tmux` or `screen`, and
+reattach to check on it. On Linux, `setsid` also detaches the run from the
+terminal. For tool-managed shells, prefer `--background` as described in
+[CLI_REFERENCE.md](./CLI_REFERENCE.md).
+
+#### Project lint or format check fails on `.build-project/` files
+
+**Symptom**: after `build init` or a run, the repository's own whole-tree check
+(for example `biome check .`) reports format errors in JSON files under
+`.build-project/state`, `.build-project/cache` or `.build-project/artifacts`.
+
+**Cause**: these are Build runtime files and Git ignores them, either through
+a `.build-project/.gitignore` or, in a repository that already tracks its own
+`.build-project/.gitignore` (which Build leaves as it is), through `/.build-project/...`
+lines in `.git/info/exclude`. A formatter or linter that does not read Git's
+ignore rules (for example Biome without `vcs.useIgnoreFile`) scans the files
+anyway.
+
+**Fix**: exclude `.build-project/` in that tool's own configuration. For Biome 2, add
+`"!**/.build-project"` to `files.includes` (checked against Biome 2.3.8; Biome 1 used
+`files.ignore` instead). For tools with an ignore file (Prettier, ESLint, and
+similar), add `.build-project/` to it.
+
+### Git/GitHub Integration Errors
+
+#### Git Conflicts
+
+**Error**: `The following untracked working tree files would be overwritten by merge`
+
+**Solutions**:
+```bash
+# 1. Inspect the working directory before changing it
+git status
+
+# 2. Preserve tracked and untracked changes
+git stash push --include-untracked -m "pre-merge recovery"
+
+# 3. Pull latest changes
+git pull origin main
+
+# 4. Apply stash
+git stash pop
+
+# 5. Retry task
+build run "task"
+```
+
+**Common Causes**:
+- Parallel executions creating overlapping files
+- Uncommitted changes in working directory
+- Git lock file conflicts
+- Branch synchronization issues
+
+#### GitHub API Errors
+
+**Error**: `GitHub API rate limit exceeded` or `403 Forbidden`
+
+**Solutions**:
+```bash
+# 1. Check GitHub token
+echo $GITHUB_TOKEN
+
+# 2. Wait for rate limit reset (usually 1 hour)
+gh api rate_limit
+
+# 3. Use personal access token
+export GITHUB_TOKEN='your-token-here'
+```
+
+**Common Causes**:
+- GitHub API rate limiting
+- Invalid or expired token
+- Insufficient token permissions
+- Network connectivity issues
+
+### Parallel Execution Problems
+
+#### Concurrency Issues
+
+**Error**: `Too many concurrent executions` or `Resource exhausted`
+
+**Solutions**:
+```bash
+# 1. Reduce concurrency
+build run competition "task" --variant claude --max-concurrent 2
+
+# 2. Use sequential execution
+build run "task"
+
+# 3. Check system resources
+htop
+free -h
+
+# 4. Cap per-provider concurrency
+build run competition "task" --variant claude --provider-limit claude=2
+```
+
+**Common Causes**:
+- System resource limits
+- Provider rate limiting
+- Memory pressure
+- Network bandwidth constraints
+
+#### Worktree Conflicts
+
+**Error**: `Git worktree already exists` or `Branch already exists`
+
+**Solutions**:
+```bash
+# 1. Clean up worktrees
+git worktree prune
+
+# 2. Remove specific worktree
+git worktree remove <path>
+
+# 3. Check for existing branches
+git branch -a
+
+# 4. Let Build choose a unique managed branch name for a new run
+build run "task"
+```
+
+**Common Causes**:
+- Previous execution didn't clean up properly
+- Manual git operations interfering
+- Concurrent executions using same branch names
+- Git lock file issues
+
+---
+
+## Error Codes & Meanings
+
+### Exit Codes
+
+| Code | Meaning | Common Causes | Solutions |
+|------|---------|---------------|-----------|
+| `0` | Success | All tasks completed successfully | - |
+| `1` | General failure | Task failed, quality gate failed | Check logs, use `--verbose` |
+| `2` | Invalid arguments | Wrong CLI syntax | Check command format |
+
+### Provider-Specific Error Messages
+
+#### Claude Errors
+
+| Error Message | Cause | Solution |
+|---------------|-------|----------|
+| `Authentication failed` | Invalid API key | Re-authenticate with `claude auth login` |
+| `Rate limit exceeded` | Too many requests | Wait 60 seconds or upgrade plan |
+| `Model not available` | Invalid model name | Use a model reported by `build provider show claude` or omit the model to use the configured default |
+| `Context length exceeded` | Task too large | Split into smaller tasks |
+
+#### Gemini Errors
+
+| Error Message | Cause | Solution |
+|---------------|-------|----------|
+| `GEMINI_API_KEY not found` | Missing API key | Set `export GEMINI_API_KEY='key'` |
+| `Quota exceeded` | API quota limit | Check Google AI Studio quotas |
+| `Invalid model` | Wrong model name | Use `gemini-2.5-flash` or `gemini-2.5-pro` |
+| `Permission denied` | API key permissions | Regenerate key with correct permissions |
+
+#### Grok Errors
+
+| Error Message | Cause | Solution |
+|---------------|-------|----------|
+| `Command not found: grok` | Grok CLI not installed or not on `PATH` | Run `BUILD_ALLOW_CURL_BASH=1 build provider setup --providers grok` or install with xAI's installer |
+| `Authentication unknown` | Passive readiness cannot verify a configured key or cached CLI auth | Primary: `grok login --device-auth` (or `grok login` / `grok`). Optional for CI/headless: set `XAI_API_KEY`. Key presence remains unverified until requested execution or `build doctor --probe --provider grok` succeeds. |
+| `XAI_API_KEY not found` / headless requires key | Headless path has no key and no usable CLI auth cache | Prefer CLI OAuth for local use; keys are optional only when a cache exists. For CI/headless without cache set `XAI_API_KEY` (deprecated alias `GROK_CODE_XAI_API_KEY` still accepted). Do not rely on browser auth unless you opt in |
+| Requested Grok execution reports an auth failure | The Grok CLI could not authenticate the requested run | Re-run `grok login --device-auth` or `grok login`; for CI set optional `XAI_API_KEY`. Build does not run a separate auth preflight. |
+| `Browser auth required` | The Grok CLI wants to start an interactive auth flow | Complete cached login in a terminal (`grok login --device-auth` / `grok`). Set `BUILD_GROK_ALLOW_BROWSER_AUTH=1` only when browser auth is acceptable (opt-in; not automatically safe in headless/CI) |
+| `unsafe for Grok headless execution` | `BUILD_GROK_PERMISSION_MODE` is `acceptEdits` or `auto` | Unset the variable (omit `--permission-mode`) or use locked native read-only posture (`dontAsk` + allow rules + tool/MCP/subagent locks; CLI >= 0.2.93) / `bypassPermissions` only when policy permits |
+| `reasoningEffort` rejected | The selected Grok model does not declare native effort support, or `auto` discovery could not establish it | Use `grok-4.6`, or leave effort unset. With `auto`, confirm `grok models` exposes an effort-capable default. |
+| `max_turns exceeded` / early Cancelled | Explicit `--max-turns` too low, or transport death under a bad permission mode | Leave `BUILD_GROK_MAX_TURNS` unset for normal runs; do not use `acceptEdits`/`auto` headless. Auth/edit probes may set a low turn cap deliberately |
+
+#### Cursor Errors
+
+| Error Message | Cause | Solution |
+|---------------|-------|----------|
+| `ConnectError: [invalid_argument]` | Plan mode conflict | Disable plan mode or use different model |
+| `Cursor CLI not found` | CLI not installed | Install with `curl https://cursor.com/install -fsS | bash` |
+| `Workspace not found` | Wrong directory | Run from project root |
+
+#### Codex Errors
+
+| Error Message | Cause | Solution |
+|---------------|-------|----------|
+| `Command not found: codex` | CLI not installed | Install with `npm install -g @openai/codex` |
+| `Authentication required` | Not logged in | Run `codex login`, then `build doctor --provider codex` |
+| `Codex execution timed out` | Local CLI exceeded the execution deadline | Inspect the retained worktree/branch, then retry with a suitable executor timeout |
+| `Codex CLI terminated without an exit code` | Local process or transport ended unexpectedly | Inspect stderr and retained git changes before retrying |
+
+### File Compliance Failure Messages
+
+| Error Message | Cause | Solution |
+|---------------|-------|----------|
+| `FILE: specification missing` | No FILE: in task | Add `FILE: path/to/file` to task description |
+| `File not found: <path>` | Invalid file path | Check file exists and path is correct |
+| `Compliance check failed` | Wrong file edited | Use exact FILE: specification |
+| `Multiple files not supported` | Too many files | Use single file or split into multiple tasks |
+
+### Timeout Error Patterns
+
+| Pattern | Cause | Solution |
+|---------|-------|----------|
+| `Timeout after 30 minutes` | Provider execution timeout | Inspect retained local changes, then retry or simplify the task |
+| `Timeout after 10 minutes` | Provider execution timeout | Use faster model or simpler task |
+| `Timeout after 5 minutes` | Network connectivity | Check internet connection |
+| `Timeout after 1 minute` | Provider rate limiting | Wait and retry |
+
+---
+
+## Provider-Specific Issues
+
+### Grok Issues
+
+Grok is a supported validated provider, but Build does
+not manage Grok credentials. **Primary for local use:** cached OAuth / device
+login via the Grok CLI (`grok login --device-auth`, `grok login`, or an initial
+`grok`). Keys are **optional** and only needed when there is no usable CLI auth
+cache (typical CI/headless): official `XAI_API_KEY`.
+`GROK_CODE_XAI_API_KEY` is a deprecated Build compatibility alias (still
+accepted; not preferred for new setup). Provider-scoped keys are accepted via
+Build's provider policy when present; they are not mandatory for local OAuth.
+
+#### Authentication at execution
+
+Build does not infer Grok authentication from keys or local auth files before a
+run. It starts the requested `grok --prompt-file` invocation and reports the
+CLI's actual authentication or transport error. This supports CLI-owned auth
+state without coupling execution to a particular cache-file layout.
+
+Passive `build doctor --provider grok` never spawns `grok` and may report auth
+as unknown; that cold state is not a blocker. Use
+`build doctor --probe --provider grok` only when you intentionally want an
+active readiness probe. `BUILD_GROK_ALLOW_BROWSER_AUTH=1` remains an explicit
+opt-in when a requested execution or probe may open a browser.
+
+Headless defaults (no operator action required): `--max-turns` is **not** set unless you opt in with `BUILD_GROK_MAX_TURNS`; output defaults to `streaming-json` when provider streaming is on; `--permission-mode` is omitted for normal edit runs. Do not set `BUILD_GROK_PERMISSION_MODE=acceptEdits` or `auto` for headless automation.
+
+#### First-Task Check with Grok
+
+This check uses `build run` for direct single-provider execution with an
+explicit provider. Use `build run competition` when you want judge
+scores/rationale or multi-variant comparison. Claude and Codex are primary
+validated providers; Grok is a supported validated alternative.
+
+```bash
+build provider setup --providers grok
+grok login --device-auth
+# or: grok login  /  grok
+build provider list
+
+git checkout -b run/my-task
+# Direct single-provider execution
+build run "Create a small marker file"
+# Delivery caveat: changes may stay in a worktree or patch under .build-project/
+
+# Graded evaluation (AI judge default — omit --judge ai)
+build run competition "Create a small marker file" --variant grok
+git checkout -b fix/my-task
+build run competition apply <competition-id>
+git status
+```
+
+If `build provider list` reports Grok auth as unknown, that is expected when
+Build can see the binary but cannot verify a usable CLI auth cache without an
+active probe. Complete cached CLI login for local use, or set optional
+`XAI_API_KEY` for non-interactive/CI runs. Set
+`BUILD_GROK_ALLOW_BROWSER_AUTH=1` only when you deliberately accept active Grok
+CLI auth probes that may open a browser (opt-in; not automatically safe in
+headless/CI).
+
+If `build run` succeeds but your repository root is unchanged, inspect the
+printed worktree and/or patch paths (default isolation). If you used
+`build run competition` and stopped after the `run/*` execution branch, note the
+competition ID, create a `fix/*`, `docs/*`, or `test/*` branch, and run
+`build run competition apply <competition-id>`.
+
+For validation-only checks from `main`, use:
+
+```bash
+build run competition --validation-run "Validate first-task provider readiness" \
+  --variant grok
+```
+
+That creates a disposable `validation/<date>-<slug>` branch. Do not use that
+branch as the product apply target; move real fixes to a focused delivery
+branch first.
+
+### Claude Issues
+
+> **Info: Authentication at Runtime**
+>
+> Provider execution defers authentication to the Claude CLI. Auth errors surface at
+> execution time with actionable guidance. Use `build provider list` for readiness verification.
+> The manual verification commands below are useful for deeper troubleshooting.
+
+#### Authentication Problems
+
+**Symptoms**:
+- `Authentication failed` errors
+- CLI reports not authenticated
+- Tasks fail immediately
+
+**Diagnosis**:
+```bash
+# Verify CLI installation
+which claude
+claude --version
+```
+
+**Solutions**:
+```bash
+# 1. Re-authenticate via Claude CLI
+claude auth logout
+claude auth login
+
+# 2. Test with simple task
+build run "Create hello.js" --engine claude
+```
+
+#### Rate Limits
+
+**Symptoms**:
+- `Rate limit exceeded` errors
+- Tasks fail after working initially
+- 429 HTTP status codes
+
+**Solutions**:
+```bash
+# 1. Wait for rate limit reset (usually 60 seconds)
+sleep 60
+
+# 2. Upgrade Anthropic plan for higher limits
+# Visit: https://console.anthropic.com/
+
+# 4. Use different provider temporarily
+build run "task" --engine gemini
+```
+
+
+
+### Gemini Issues
+
+> **Info: Authentication at Runtime**
+>
+> Provider execution defers authentication to the Gemini CLI. Auth errors surface at
+> execution time with actionable guidance. Use `build provider list` for readiness verification.
+> The manual verification commands below are useful for deeper troubleshooting.
+
+#### Auth Flow Problems
+
+**Symptoms**:
+- `GEMINI_API_KEY not found` errors
+- Authentication failures
+- Model not available errors
+
+**Diagnosis**:
+```bash
+# Check API key format
+echo $GEMINI_API_KEY | head -c 10
+# Should start with "AIza"
+
+# Test API key
+curl -H "Authorization: Bearer $GEMINI_API_KEY" \
+  "https://generativelanguage.googleapis.com/v1/models"
+```
+
+**Solutions**:
+```bash
+# 1. Get API key from Google AI Studio
+# https://aistudio.google.com/
+
+# 2. Set environment variable
+export GEMINI_API_KEY='AIza...'
+
+# 3. Test with simple request
+build run "test" --engine gemini --model gemini-2.5-flash
+
+# 4. Check quotas in Google AI Studio
+```
+
+#### Model Availability
+
+**Symptoms**:
+- `Model not available` errors
+- `Invalid model` messages
+- Tasks fail with model-specific errors
+
+**Solutions**:
+```bash
+# 1. Use supported models only
+build run "task" --model gemini-2.5-flash
+build run "task" --model gemini-2.5-pro
+
+# 2. Check model availability
+build provider --json
+
+# 3. Use auto-detection
+build run "task" --engine gemini
+```
+
+#### Legacy command syntax errors
+
+**Symptoms**:
+- `bash: gemini: command not found: code`
+- Shell prints `no matches found: --allowed-tools=[]`
+- Command succeeds manually but fails when invoked by Build
+
+**Diagnosis**:
+```bash
+# Compare against Build's invocation
+build --verbose run "Say hello" --engine gemini
+
+# Reproduce directly
+gemini --output-format json --model gemini-2.5-flash "Say hello"
+```
+
+**Solutions**:
+```bash
+# 1. Drop removed flags and the legacy subcommand (v0.8+)
+gemini --output-format json --model gemini-2.5-flash "prompt"
+
+# 2. Quote brackets if you must pass arrays (advanced usage only)
+gemini --output-format json --model gemini-2.5-flash --allowed-tools '[]' "prompt"
+
+# 3. Prefer Build defaults unless a flag is officially documented
+```
+
+#### Gemini CLI stderr handling
+
+**Symptoms**:
+- Build reports `Loaded cached credentials.` as an error
+- CLI exits with code 1 when run from Build but succeeds directly
+- Retry loop runs 15 times without additional diagnostics
+
+**Diagnosis**:
+```bash
+# Run with verbose logging to capture the exact command
+BUILD_DEBUG=1 build run "test" --engine gemini --verbose
+
+# Check exit code from manual invocation
+gemini --output-format json --model gemini-2.5-flash "test"
+echo $?  # Should be 0 on success
+```
+
+**Solutions**:
+```bash
+# 1. Trust the exit code, not stderr noise
+# (Build filters the message internally; treat exitCode===0 as success)
+
+# 2. Failures that look like retries: inspect verbose process I/O
+# (there is no live --max-retries CLI flag on build run)
+BUILD_DEBUG=1 build run "task" --engine gemini --verbose
+
+# 3. Confirm the provider CLI succeeds outside Build
+gemini --output-format json --model gemini-2.5-flash "test"
+echo $?
+
+# 4. See docs/GEMINI_BEST_PRACTICES.md § CLI Invocation Requirements for context
+```
+
+### Cursor Issues
+
+#### Agent Selection Problems
+
+**Symptoms**:
+- Wrong agent selected for task
+- Poor task execution quality
+- Inconsistent results
+
+**Diagnosis**:
+```bash
+# Agent selection is provider-native; there is no live --explain-agents flag.
+# Check run output for agent labels when present, then routing status:
+build routing status
+build routing status --provider cursor
+
+# After competitions, inspect recorded outcomes:
+build run competition results <task-id>
+```
+
+**Solutions**:
+```bash
+# 1. Pin provider/model when automatic selection is a poor fit
+build run "task" --engine cursor --model auto
+
+# 2. Check agent/provider performance telemetry
+build telemetry leaderboard
+```
+
+#### Workspace Issues
+
+**Symptoms**:
+- `Workspace not found` errors
+- File not found errors
+- Context not loaded properly
+
+**Solutions**:
+```bash
+# 1. Run from project root
+cd /path/to/your/project
+build run "task"
+
+# 2. Check workspace structure
+ls -la
+ls -la .build-project/
+
+# 3. Initialize if needed
+build init
+```
+
+### Codex Issues
+
+#### Local Result Recovery
+
+Codex runs locally in the selected workspace or isolated worktree. There is no
+Codex cloud task to poll or apply.
+
+```bash
+# Confirm CLI availability and authentication
+build doctor --provider codex
+
+# Inspect durable local delivery after a timeout or interrupted process
+git status
+git diff
+git log --oneline --max-count=10
+```
+
+For competition runs, inspect the retained variant branch/worktree reported by
+Build. Codex executors respect `--executor-timeout-minutes`; `--timeout`
+controls the overall competition deadline. Do not use `codex apply` recovery
+instructions from historical run records.
+
+---
+
+## Installation & Setup Problems
+
+### Node.js Version Issues
+
+#### Unsupported Version
+
+**Error**: `Unsupported Node.js version` or `Module not found`
+
+**Diagnosis**:
+```bash
+# Check current version
+node --version
+# Requires: Node.js 24.x or 26.x
+
+# Check npm version
+npm --version
+```
+
+**Solutions**:
+```bash
+# 1. Activate a supported Node with your installed version manager
+nvm install 24
+nvm use 24
+
+# 2. Verify installation
+node --version
+npm --version
+
+# 3. Repair the installed Build package
+npx --yes @ebrindley/build-install
+```
+
+For a custom installation prefix, pass the same `-- --install-dir /absolute/prefix`
+to the bootstrap command. These steps repair Build itself. Do not delete the
+current project's `node_modules` or lockfile to repair a global Build install.
+
+#### Version Conflicts
+
+**Error**: `Module version mismatch` or `Incompatible versions`
+
+**Solutions**:
+```bash
+# 1. Use Node.js 24 (the version Build is built and tested against)
+nvm install 24
+nvm use 24
+
+# 2. Reinstall Build with the selected runtime
+npx --yes @ebrindley/build-install
+
+# 3. Verify the installation
+build --version
+build doctor
+```
+
+If you develop Build from source, run `npm ci` and `npm run build` only inside
+your Build source checkout. Keep its committed lockfile; unrelated consumer
+project dependencies do not need to change.
+
+#### Replicate the Node 24 Fix on Another Computer
+
+Pulling the repository propagates the runtime guards, installer, launcher generator,
+workflow checks, and documentation. It does not change that computer's shell `PATH`,
+installed Node executable, native modules, or generated `build` launcher.
+
+On every development computer:
+
+```bash
+git pull
+nvm install 24
+nvm use 24
+node --version
+node -p 'process.versions.modules'
+npm ci
+./scripts/compile-dist.sh
+build doctor --json
+build provider list --json
+```
+
+`./scripts/compile-dist.sh` installs a launcher pinned to the verified Node 24 or 26
+executable into `~/.local/bin` (override with `BUILD_BIN_DIR`) and puts that
+executable first in child `PATH`. An existing launcher the script did not
+generate is left in place, so a hand-maintained wrapper is never overwritten.
+A Homebrew Node is recorded by its stable `opt` path, so `brew upgrade` keeps the
+launcher working. If that Node is otherwise moved or reinstalled, rerun the build.
+Package installations should be refreshed with `bash install.sh --force` for the
+same reason.
+
+For a temporary override, point the launcher at another Node 24 or 26 executable:
+
+```bash
+BUILD_NODE_BIN=/path/to/node build doctor --json
+```
+
+The launcher validates that override on every invocation and rejects executables
+that are not Node 24 or 26. Set it in the service or editor environment as well when those
+processes do not inherit the interactive shell's `PATH`.
+
+Claude readiness is the native Claude CLI (`build provider list` /
+`build doctor`), not a Build Claude Code plugin. Pre-1.42 installs that
+still have a leftover plugin registration can remove it with
+`build provider disconnect claude --scope user|project|local`. That cleanup does
+not sign out, revoke credentials, or disable Claude. Project and local scopes
+must be cleaned from each affected repository.
+
+For Homebrew without `nvm`, put `/opt/homebrew/opt/node@24/bin` before other Node
+locations in the shell startup file, reopen the shell, then run the rebuild and
+verification commands above. Shell profile edits and generated launchers are
+machine-local and must be repeated on each computer.
+
+### npm Install Failures
+
+#### Network Issues
+
+**Error**: `npm ERR! network timeout` or `npm ERR! fetch failed`
+
+**Solutions**:
+```bash
+# 1. Check internet connection
+ping google.com
+
+# 2. Clear npm cache
+npm cache clean --force
+
+# 3. Use different registry
+npm config set registry https://registry.npmjs.org/
+
+# 4. Increase timeout
+npm config set timeout 300000
+
+# 5. Retry installation
+npm install
+```
+
+#### Permission Issues
+
+**Error**: `npm ERR! EACCES: permission denied` or `npm ERR! EPERM`
+
+**Solutions**:
+```bash
+# 1. Fix npm permissions (recommended)
+mkdir ~/.npm-global
+npm config set prefix '~/.npm-global'
+echo 'export PATH=~/.npm-global/bin:$PATH' >> ~/.bashrc
+source ~/.bashrc
+
+# 2. Or use sudo (not recommended)
+sudo npm install -g @anthropic-ai/claude-code
+
+# 3. Or use nvm with Node 24 (best practice)
+curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.39.0/install.sh | bash
+source ~/.bashrc
+nvm install 24
+nvm use 24
+```
+
+#### Dependency Conflicts
+
+**Error**: `npm ERR! peer dep missing` or `npm ERR! conflict`
+
+**Solutions**:
+```bash
+# 1. Check for conflicts
+npm ls
+
+# 2. Install missing peer dependencies
+npm install --save-dev <missing-package>
+
+# 3. Use --force to override conflicts
+npm install --force
+
+# 4. Or use --legacy-peer-deps
+npm install --legacy-peer-deps
+```
+
+### CLI Installation Verification
+
+#### Claude CLI
+
+**Verification**:
+```bash
+# Check installation
+which claude
+claude --version
+
+# Test execution
+claude run "Create hello.js"
+```
+
+**Common Issues**:
+- Not in PATH
+- Wrong version installed
+- Authentication not configured
+
+#### Codex CLI
+
+**Verification**:
+```bash
+# Check installation
+which codex
+codex --version
+
+# Test authentication
+codex login
+
+# Verify via Build
+build doctor --provider codex
+```
+
+**Common Issues**:
+- Not in PATH
+- Wrong package name
+- Authentication not configured
+
+#### Cursor CLI
+
+**Verification**:
+```bash
+# Check installation
+which cursor-agent
+cursor-agent --version
+```
+
+**Common Issues**:
+- Not in PATH
+- Wrong package name
+- Authentication not configured
+
+### Configuration File Errors
+
+#### Invalid JSONC
+
+**Error**: `JSONC parse error`, `SyntaxError: Unexpected token`, or `Invalid JSON`
+
+**Diagnosis**:
+```bash
+# JSONC is JSON with comments and trailing commas.
+# Validate config with Build's JSONC-aware parser.
+build config validate
+build config diagnose
+```
+
+**Solutions**:
+```bash
+# Repair fixable validation issues first
+build config validate --fix
+```
+
+If the file remains unparseable, follow [Configuration Corruption](#configuration-corruption).
+That procedure preserves the original project configuration before regenerating it.
+
+#### Missing Configuration
+
+**Error**: `Configuration not found` or `Config file missing`
+
+**Solutions**:
+```bash
+# 1. Initialize configuration
+build init
+
+# 2. Check if .build-project directory exists
+ls -la .build-project/
+
+# 3. Create manually if needed
+mkdir -p .build-project/config
+touch .build-project/config/build.config.jsonc
+```
+
+#### Permission Issues
+
+**Error**: `EACCES: permission denied` or `Cannot write to config`
+
+**Solutions**:
+```bash
+# 1. Fix directory permissions
+chmod 755 .build-project/
+chmod 644 .build-project/config/build.config.jsonc
+
+# 2. Check ownership
+ls -la .build-project/
+
+# 3. Fix ownership if needed
+sudo chown -R $USER:$USER .build-project/
+```
+
+---
+
+## Debugging Tips
+
+### Using --verbose Flag
+
+**Enable detailed logging**:
+```bash
+# Basic verbose mode
+build run "task" --verbose
+
+# Verbose with specific provider
+build run "task" --engine claude --verbose
+
+# Verbose with a competition
+build run competition "task" --variant claude --verbose
+```
+
+**What verbose mode shows**:
+- Provider selection reasoning
+- API request/response details
+- File operations and git commands
+- Error stack traces
+- Performance metrics
+
+### Checking Telemetry Logs
+
+**View execution logs**:
+```bash
+# Print the resolved telemetry directory (live DBs and the main JSONL stream)
+build doctor --verbose
+TELEMETRY_DIR=~/.build/data/telemetry   # default; use the path doctor printed
+
+# List log files (durable JSONL, organized by month)
+ls -la "$TELEMETRY_DIR"/logs/
+
+# Follow today's events
+tail -f "$TELEMETRY_DIR"/logs/<YYYY-MM>/<YYYY-MM-DD>.jsonl
+
+# Search for errors
+grep -i error "$TELEMETRY_DIR"/logs/*/*.jsonl
+
+# View a specific day's events
+cat "$TELEMETRY_DIR"/logs/<YYYY-MM>/<YYYY-MM-DD>.jsonl | jq .
+```
+
+**Log file locations**:
+- `{resolved}/logs/<YYYY-MM>/<YYYY-MM-DD>.jsonl` — durable execution event logs (not `*.log`)
+- `{resolved}/db/` — SQLite databases (`variants.db`, `validations.db`)
+- `{resolved}/invocations.db` — cost/token invocations
+- `~/.build/data/projects/<project-id>/telemetry/state/` — per-run state, outside the checkout
+- Checkout `.build-project/telemetry/logs/` — flywheel JSONL from older versions; flywheel now writes under `{resolved}/logs/`
+- Checkout `.build-project/telemetry/db/` — leftover/legacy default; the resolver ignores it
+
+### Reviewing PR Comments
+
+**For ordinary PR review or historical cloud-record recovery**:
+```bash
+# List recent PRs
+gh pr list --limit 10
+
+# View specific PR
+gh pr view <PR_NUMBER>
+
+# Check for bot comments
+gh pr view <PR_NUMBER> --json comments \
+  --jq '.comments[] | select(.author.login | contains("bot"))'
+
+# View PR diff
+gh pr diff <PR_NUMBER>
+```
+
+**Common bot comment patterns** (including historical cloud records):
+- `chatgpt-codex-connector` - Codex bot (historical cloud comments)
+- `cursor-agent` - Cursor bot (historical cloud comments)
+- Look for `Summary:` sections
+- Extract historical task IDs from comments when recovering old records
+
+### Using Evaluation Scripts
+
+**Check if evaluation scripts exist**:
+```bash
+# Look for evaluation scripts
+find . -name "*evaluate*" -type f
+find . -name "*evaluation*" -type f
+
+# Check scripts directory
+ls -la scripts/
+
+# Run evaluation if available
+./scripts/evaluate-agent.sh <TASK_ID> <PROVIDER>
+```
+
+**Manual evaluation process**:
+```bash
+# 1. Find agent's implementation commit
+git log --author="Cursor Agent" --grep="feat:\|Implement\|Refactor" --oneline
+
+# 2. Show actual changes
+git show <commit_hash> --stat
+
+# 3. Check if merged to main
+git branch --contains <commit_hash> | grep main
+
+# 4. For Codex and other local providers, inspect the variant branch
+git log <variant_branch> --oneline --max-count=10
+git diff main...<variant_branch>
+```
+
+### System Resource Monitoring
+
+**Check system resources**:
+```bash
+# Memory usage
+free -h
+htop
+
+# Disk space
+df -h
+
+# CPU usage
+top
+htop
+
+# Network connectivity
+ping google.com
+curl -I https://api.anthropic.com
+```
+
+**Common resource issues**:
+- Low memory (causes crashes)
+- Full disk (prevents file operations)
+- High CPU usage (causes timeouts)
+- Network issues (causes API failures)
+
+### Provider-Specific Debugging
+
+#### Claude Debugging
+
+```bash
+# Test with simple request
+claude run "Create hello.js"
+
+# Check logs
+claude logs
+
+# Verify API key (only for direct HTTP debugging; Build uses Claude CLI auth by default)
+curl -H "Authorization: Bearer $ANTHROPIC_API_KEY" \
+  "https://api.anthropic.com/v1/messages" \
+  -d '{"model":"claude-3-sonnet-20240229","max_tokens":10,"messages":[{"role":"user","content":"test"}]}'
+```
+
+#### Gemini Debugging
+
+```bash
+# Test API key
+curl -H "Authorization: Bearer $GEMINI_API_KEY" \
+  "https://generativelanguage.googleapis.com/v1/models"
+
+# Check quotas
+curl -H "Authorization: Bearer $GEMINI_API_KEY" \
+  "https://generativelanguage.googleapis.com/v1/models/gemini-2.5-flash"
+
+# Test with simple request
+build run "test" --engine gemini --model gemini-2.5-flash
+```
+
+#### Cursor Debugging
+
+```bash
+# Verify Build can find Cursor Agent
+build doctor --provider cursor
+
+# If you installed Cursor Agent somewhere non-standard, override the binary:
+export CURSOR_CLI_COMMAND=/path/to/cursor-agent
+```
+
+#### Grok Debugging
+
+```bash
+# Verify Build can find Grok and evaluate passive readiness
+build doctor --provider grok
+
+# Local primary path: confirm CLI login / auth cache exists
+test -f "${HOME}/.grok/auth.json" && echo "Grok CLI auth cache present"
+
+# Optional CI/headless path: confirm the official key is present without printing it
+test -n "$XAI_API_KEY" && echo "XAI_API_KEY is set"
+# Deprecated compatibility alias (still accepted):
+# test -n "$GROK_CODE_XAI_API_KEY" && echo "GROK_CODE_XAI_API_KEY is set"
+```
+
+#### Codex Debugging
+
+```bash
+# Verify Build can find Codex CLI + your login state
+build doctor --provider codex
+
+# Inspect local delivery
+git status
+git diff
+git log --oneline --max-count=10
+```
+
+---
+
+## Advanced Troubleshooting
+
+### Memory Guardian Issues
+
+**Symptoms**:
+- `Memory: XX% heap` warnings
+- Process kills due to memory pressure
+- Performance degradation
+
+**Diagnosis**:
+```bash
+# Check memory usage
+build run "task" --verbose
+# Look for: "Memory: XX% heap"
+
+# Check memory guardian logs (resolved monthly JSONL; `build doctor --verbose`)
+TELEMETRY_DIR=~/.build/data/telemetry
+grep -i "memory" "$TELEMETRY_DIR"/logs/*/*.jsonl
+```
+
+**Solutions**:
+```bash
+# 1. Reduce parallel execution
+build run competition "task" --variant claude --max-concurrent 2
+
+# 2. Configure guardian behavior
+export BUILD_MEMORY_KILL=0        # Log-only
+export BUILD_MEMORY_KILL=1        # Aggressive kills
+export BUILD_RSS_LIMIT_MB=2048    # RSS cap (MB)
+export BUILD_RSS_LIMIT_PCT=75     # RSS cap (% of total RAM; ignored if BUILD_RSS_LIMIT_MB is set)
+
+# 3. Disable memory guardian (development only)
+export BUILD_MEMORY_GUARD=0
+
+# 4. Increase Node.js memory limit
+node --max-old-space-size=4096 build.js run "task"
+
+# 5. Use faster model
+build run "task" --model gemini-2.5-flash
+```
+
+#### Memory Guardian Stress Harness
+
+Use the stress harness to reproduce memory pressure locally or during scheduled health checks:
+
+```bash
+# Developer workstation
+npm run test:stress
+
+# CI / cron job (uses gentler thresholds)
+npm run ci:stress
+```
+
+The script streams heap usage as it allocates memory and exits automatically when the guardian intervenes or the target threshold is reached. Capture the console output when reporting memory regressions.
+
+### Recursion Detection Issues
+
+**Symptoms**:
+- `Recursion depth exceeded` errors
+- Infinite loops in parallel execution
+- Process hangs
+
+**Diagnosis**:
+```bash
+# Check recursion depth
+echo $BUILD_MAX_RECURSION_DEPTH
+
+# Look for recursion warnings (resolved monthly JSONL; `build doctor --verbose`)
+TELEMETRY_DIR=~/.build/data/telemetry
+grep -i "recursion" "$TELEMETRY_DIR"/logs/*/*.jsonl
+```
+
+**Solutions**:
+```bash
+# 1. Increase recursion limit (careful!)
+export BUILD_MAX_RECURSION_DEPTH=3
+
+# 2. Use sequential execution
+build run "task"
+
+# 3. Check for recursive calls
+grep -r "build run" .build-project/
+```
+
+### Git Lock Issues
+
+**Symptoms**:
+- `fatal: Unable to create '.../index.lock'` errors
+- Git operations fail
+- Parallel execution conflicts
+
+**Diagnosis**:
+```bash
+# Resolve the lock for this checkout, including linked worktrees
+git_dir="$(git rev-parse --path-format=absolute --git-dir)"
+lock_path="$git_dir/index.lock"
+ls -l "$lock_path"
+
+# Check for active Git operations
+ps -axo pid=,command= | grep '[g]it'
+```
+
+Do not move the lock while any Git process is active. If no Git process is active and the
+lock is older than the failed operation, quarantine only that exact lock, then verify the
+worktree:
+
+```bash
+mv "$lock_path" "${lock_path}.stale.$(date +%Y%m%dT%H%M%S)"
+git status
+```
+
+Build automatically removes only a stale `index.lock`, and it refuses recovery when any
+Git process is live. Moving the lock aside preserves it for inspection and avoids changing
+tracked or untracked work.
+
+### Provider Rate Limiting
+
+**Symptoms**:
+- `Rate limit exceeded` errors
+- 429 HTTP status codes
+- Tasks fail after working initially
+
+**Solutions**:
+```bash
+# 1. Wait for rate limit reset
+sleep 60
+
+# 2. Use different provider
+build run "task" --engine gemini
+
+# 3. Reduce concurrency
+build run "task"
+
+# 4. Check provider quotas
+# Claude: https://console.anthropic.com/
+# Gemini: https://aistudio.google.com/
+# Cursor: https://cursor.sh/settings
+```
+
+### Network Connectivity Issues
+
+**Symptoms**:
+- `Network timeout` errors
+- `Connection refused` errors
+- API requests fail
+
+**Diagnosis**:
+```bash
+# Test basic connectivity
+ping google.com
+
+# Test provider APIs
+curl -I https://api.anthropic.com
+curl -I https://generativelanguage.googleapis.com
+curl -I https://api.cursor.sh
+
+# Check DNS resolution
+nslookup api.anthropic.com
+```
+
+**Solutions**:
+```bash
+# 1. Check internet connection
+ping google.com
+
+# 2. Check DNS settings
+cat /etc/resolv.conf
+
+# 3. Use different DNS
+echo "nameserver 8.8.8.8" | sudo tee /etc/resolv.conf
+
+# 4. Check firewall settings
+sudo ufw status
+
+# 5. Use proxy if needed
+export HTTP_PROXY=http://proxy:port
+export HTTPS_PROXY=http://proxy:port
+```
+
+### Configuration Corruption
+
+**Symptoms**:
+- `Invalid configuration` errors
+- Unexpected behavior
+- Settings not applied
+
+**Diagnosis**:
+```bash
+build config validate
+build config diagnose
+```
+
+First try the parser-aware repair:
+
+```bash
+build config validate --fix
+build config validate
+```
+
+If the project configuration is still unparseable, stop active Build runs and copy that
+exact file to a location outside the repository before resetting it:
+
+```bash
+cp .build-project/config/build.config.jsonc /path/outside/repo/build.config.jsonc.saved
+build config reset --level project --backup
+build init --force
+build config validate
+```
+
+`build config reset --backup` prints the name of a `build-config-backup-<timestamp>.json`
+file created in the current directory. Move that exact printed file outside the repository
+if you need to retain it. Restore individual custom settings only after validation passes.
+
+### Telemetry Database Not Initialized
+
+**Symptoms**:
+- `build run competition results` fails with "No evaluation found" or "Database not initialized"
+- AI Judge evaluation commands fail
+- SQLite error messages when querying results
+
+**Cause**:
+The database may be missing, corrupt, or missing its schema. This can occur after an
+interrupted write, incomplete restore, or removal of ignored runtime state.
+
+**Diagnosis**:
+```bash
+build doctor --json
+build telemetry checkpoint --verify --json
+```
+
+`build telemetry checkpoint --repair` reports corruption but does not rebuild the database
+automatically.
+
+If the database is missing, initialize it and verify the result:
+
+```bash
+build init --db
+build telemetry checkpoint --verify --json
+```
+
+If the database is corrupt and a Build backup exists, inspect the restore before applying
+it:
+
+```bash
+build backup list
+build backup restore --latest --dry-run
+```
+
+After reviewing the dry-run, stop active Build, provider, and agent processes, apply the
+restore, and verify:
+
+```bash
+build backup restore --latest --force --offline
+build telemetry checkpoint --verify --json
+```
+
+If no usable backup exists, stop all Build, provider, and agent activity before moving
+database files. Attempt a database backup first:
+
+```bash
+build backup
+```
+
+The backup command uses the external `sqlite3` executable. If it is unavailable or the
+corruption prevents a backup, copy the database and any SQLite sidecars together from
+the **resolved** telemetry directory (`build doctor --verbose` prints it; default
+`~/.build/data/telemetry/`) to a location outside the repository. Do not copy
+checkout `.build-project/telemetry/db/` — that is leftover, not the live store.
+
+```bash
+build doctor --verbose
+TELEMETRY_DIR=~/.build/data/telemetry   # use the path doctor printed
+mkdir -p /path/outside/repo/build-db-saved
+cp "$TELEMETRY_DIR"/db/variants.db* /path/outside/repo/build-db-saved/
+```
+
+Only after activity has stopped and the snapshot or copy has completed, quarantine the
+database and any sidecars rather than deleting them. Do not delete the whole
+`.build-project/telemetry/` tree.
+
+```bash
+TELEMETRY_DIR=~/.build/data/telemetry
+mkdir -p "$TELEMETRY_DIR"/db/quarantine
+mv "$TELEMETRY_DIR"/db/variants.db "$TELEMETRY_DIR"/db/quarantine/
+test ! -e "$TELEMETRY_DIR"/db/variants.db-wal || mv "$TELEMETRY_DIR"/db/variants.db-wal "$TELEMETRY_DIR"/db/quarantine/
+test ! -e "$TELEMETRY_DIR"/db/variants.db-shm || mv "$TELEMETRY_DIR"/db/variants.db-shm "$TELEMETRY_DIR"/db/quarantine/
+build init --db
+build telemetry checkpoint --verify --json
+```
+
+Keep the quarantined files until you have confirmed that the rebuilt database is usable
+and decided whether historical telemetry needs recovery.
+
+**Common causes**:
+- Force-cleaning ignored files with Git, which can remove the `.build-project/` directory
+- Insufficient disk space during database creation
+- File permission issues on `.build-project/` directory
+- Corrupted SQLite database file (rare)
+
+---
+
+## Implementation Artifacts (.build-project/implementations/)
+
+### "Implementation files missing" Warning
+
+**Status**:  **This is normal and healthy**
+
+Implementation artifacts are **ephemeral files** created only in variant worktrees during execution (including historical cloud-era runs). The main working directory should NOT contain `.build-project/implementations/*.md` files.
+
+**Why you're seeing this**:
+-  Empty `.build-project/implementations/` directory - **Normal**
+-  Missing `.build-project/implementations/` directory - **Normal**
+-  Old stale files - **Run `build cleanup prune`**
+
+**What to do**:
+```bash
+# Check artifact status
+build doctor --verbose
+
+# Clean up governed runtime artifacts
+build cleanup prune
+build cleanup prune --execute
+```
+
+**See also**: Placeholder implementation files are ephemeral worktree artifacts and should not persist in the main working tree.
+
+### CLI Load Failures ("Dynamic require" Errors)
+
+**Error**: `Dynamic require of "fs" is not supported`
+
+**Cause**: ESM/CommonJS incompatibility in bundled CLI
+
+**Status**:  **Fixed** (as of 2025-11-15)
+
+For an installed npm package, use the Build bootstrap repair above. In a Build
+source checkout, rebuild and inspect its ESM output:
+```bash
+# Rebuild the CLI
+npm run build
+
+# Verify ESM compliance
+npm run check:esm
+
+# If error persists, report issue with:
+node --version  # Node.js version
+npm list better-sqlite3  # Module versions
+```
+
+**Check which installation is being launched**
+
+```bash
+# 1) Inspect every build binary on PATH + global installs
+which -a build
+npm ls -g @ebrindley/build
+
+# 2) Verify the selected installation
+build --version
+build doctor
+```
+
+Choose the intended installation's `bin` directory on PATH. Do not uninstall
+other installations as a generic diagnostic step. Run work through the CLI:
+`build run competition "Task description"`.
+
+---
+
+## Getting Help
+
+### Self-Service Resources
+
+1. **Check this guide first** - Most common issues are covered
+2. **Use `--verbose` flag** - Get detailed error information
+3. **Check telemetry logs** - Review execution history
+4. **Test with simple tasks** - Isolate the problem
+
+### Community Support
+
+- **Documentation**: [Bundled documentation index](README.md)
+- **Examples**: [Quick Start](getting-started/QUICK_START.md)
+- **GitHub Issues**: [Report bugs and request features](https://github.com/ebrindley/build/issues)
+  once the repository is public. Until then, this channel requires repository
+  access; no public email intake is provided.
+
+### Provider Support
+
+- **Claude**: [Anthropic Support](https://support.anthropic.com/)
+- **Gemini**: [Google AI Studio Support](https://aistudio.google.com/)
+- **Cursor**: [Cursor Support](https://cursor.sh/support)
+- **Codex**: see the Codex CLI's own support channels
+
+### Reporting Issues
+
+When reporting issues, include:
+
+1. **Error message** (exact text)
+2. **Command used** (with all flags)
+3. **Verbose output** (use `--verbose`)
+4. **System information** (OS, Node.js version)
+5. **Provider status** (authentication, quotas)
+6. **Log files** (relevant entries from the resolved monthly JSONL; `build doctor --verbose`)
+
+Verbose output and log entries can include prompts, file contents, paths, and
+credentials. Paste only the relevant lines, and remove anything private before
+posting.