codeharness 0.16.0 → 0.17.0

package/README.md

@@ -1,66 +1,118 @@
 # codeharness
 
-## Quick Start
+Makes autonomous coding agents produce software that actually works — not software that passes tests.
 
-```bash
-# Install
-npm install -g codeharness
+codeharness is an **npm CLI** + **Claude Code plugin** that packages verification-driven development as an installable tool: black-box verification via Docker, agent-first observability via VictoriaMetrics, and mechanical enforcement via hooks that make skipping verification architecturally impossible.
 
-# Initialize the project
-codeharness init
+## What it does
 
-# Check project status
-codeharness status
-```
+1. **Verifies features work** — not just that tests pass. Black-box verification runs the built CLI inside a Docker container with no source code access. If the feature doesn't work from a user's perspective, verification fails.
+2. **Fixes what it finds** — verification failures with code bugs automatically return to development with specific findings. The dev agent gets told exactly what's broken and why.
+3. **Runs sprints autonomously** — reads your sprint plan, picks the highest-priority story, implements it, reviews it, verifies it, and moves to the next one. Cross-epic prioritization, retry management, and session handoff built in.
+4. **Makes agents see runtime** — ephemeral VictoriaMetrics stack (logs, metrics, traces) that agents query programmatically during development. No guessing at what the code does at runtime.
 
 ## Installation
 
+Two components — install both:
+
 ```bash
+# CLI (npm package)
 npm install -g codeharness
-```
 
-## Usage
+# Claude Code plugin (slash commands, hooks, skills)
+claude plugin install github:iVintik/codeharness
+```
 
-After installation, initialize codeharness in your project directory:
+## Quick Start
 
 ```bash
+# Initialize in your project
 codeharness init
+
+# Start autonomous sprint execution (inside Claude Code)
+/harness-run
 ```
 
-This sets up the harness with stack detection, observability, and documentation scaffolding.
+## How it works
+
+### As a CLI (`codeharness`)
+
+The CLI handles all mechanical work — stack detection, Docker management, verification, coverage, retry state.
+
+| Command | Purpose |
+|---------|---------|
+| `codeharness init` | Detect stack, install dependencies, start observability, scaffold docs |
+| `codeharness run` | Execute the autonomous coding loop (Ralph) |
+| `codeharness verify --story <key>` | Run verification pipeline for a story |
+| `codeharness status` | Show harness health, sprint progress, Docker stack |
+| `codeharness coverage` | Run tests with coverage and evaluate against targets |
+| `codeharness onboard epic` | Scan codebase for gaps, generate onboarding stories |
+| `codeharness retry --status` | Show retry counts and flagged stories |
+| `codeharness retry --reset` | Clear retry state for re-verification |
+| `codeharness verify-env build` | Build Docker image for black-box verification |
+| `codeharness stack start` | Start the shared observability stack |
+| `codeharness teardown` | Remove harness from project |
 
-## CLI Reference
+All commands support `--json` for machine-readable output.
+
+### As a Claude Code plugin (`/harness-*`)
+
+The plugin provides slash commands that orchestrate the CLI within Claude Code sessions:
+
+| Command | Purpose |
+|---------|---------|
+| `/harness-run` | Autonomous sprint execution — picks stories by priority, runs create → dev → review → verify loop |
+| `/harness-init` | Interactive project initialization |
+| `/harness-status` | Quick overview of sprint progress and harness health |
+| `/harness-onboard` | Scan project and generate onboarding plan |
+| `/harness-verify` | Verify a story with real-world evidence |
+
+### BMAD Method integration
+
+codeharness integrates with [BMAD Method](https://github.com/bmadcode/BMAD-METHOD) for structured sprint planning:
+
+| Phase | Commands |
+|-------|----------|
+| Analysis | `/create-brief`, `/brainstorm-project`, `/market-research` |
+| Planning | `/create-prd`, `/create-ux` |
+| Solutioning | `/create-architecture`, `/create-epics-stories` |
+| Implementation | `/sprint-planning`, `/create-story`, then `/harness-run` |
+
+## Verification architecture
 
 ```
-Usage: codeharness [options] [command]
-
-Makes autonomous coding agents produce software that actually works
-
-Options:
-  -V, --version            output the version number
-  --json                   Output in machine-readable JSON format
-  -h, --help               display help for command
-
-Commands:
-  init [options]           Initialize the harness in a project
-  bridge [options]         Bridge BMAD epics/stories into beads task store
-  run [options]            Execute the autonomous coding loop
-  verify [options]         Run verification pipeline on completed work
-  status [options]         Show current harness status and health
-  onboard [options]        Onboard an existing codebase into the harness
-  teardown [options]       Remove harness from a project
-  state                    Manage harness state
-  sync [options]           Synchronize beads issue statuses with story files and
-                           sprint-status.yaml
-  coverage [options]       Run tests with coverage and evaluate against targets
-  doc-health [options]     Scan documentation for freshness and quality issues
-  stack                    Manage the shared observability stack
-  query                    Query observability data (logs, metrics, traces)
-                           scoped to current project
-  retro-import [options]   Import retrospective action items as beads issues
-  github-import [options]  Import GitHub issues labeled for sprint planning into
-                           beads
-  verify-env               Manage verification environment (Docker image + clean
-                           workspace)
-  help [command]           display help for command
+┌─────────────────────────────────────────┐
+│  Claude Code Session                     │
+│  /harness-run picks next story           │
+│  → create-story → dev → review → verify  │
+└────────────────────┬────────────────────┘
+                     │ verify
+                     ▼
+┌─────────────────────────────────────────┐
+│  Docker Container (no source code)       │
+│  - codeharness CLI installed from tarball│
+│  - claude CLI for nested verification    │
+│  - curl/jq for observability queries     │
+│  Exercises CLI as a real user would      │
+└────────────────────┬────────────────────┘
+                     │ queries
+                     ▼
+┌─────────────────────────────────────────┐
+│  Observability Stack (VictoriaMetrics)   │
+│  - VictoriaLogs  :9428 (LogQL)          │
+│  - VictoriaMetrics :8428 (PromQL)       │
+│  - OTEL Collector :4318                  │
+└─────────────────────────────────────────┘
 ```
+
+When verification finds code bugs → story returns to dev with findings → dev fixes → re-verify. This loop runs up to 10 times per story. Infrastructure failures (timeouts, Docker errors) retry 3 times then skip.
+
+## Requirements
+
+- Node.js >= 18
+- Docker (for observability and verification)
+- Claude Code (for plugin features)
+
+## License
+
+MIT

package/dist/index.js

@@ -464,6 +464,10 @@ var WEB_OTLP_PACKAGES = [
 var AGENT_OTLP_PACKAGES_NODE = ["@traceloop/node-server-sdk"];
 var AGENT_OTLP_PACKAGES_PYTHON = ["traceloop-sdk"];
 var NODE_REQUIRE_FLAG = "--require @opentelemetry/auto-instrumentations-node/register";
+function truncateError(message, maxLength = 200) {
+  if (message.length <= maxLength) return message;
+  return message.slice(0, maxLength) + "... (truncated)";
+}
 function installNodeOtlp(projectDir) {
   try {
     execFileSync2("npm", ["install", ...NODE_OTLP_PACKAGES], { cwd: projectDir, stdio: "pipe", timeout: 3e5 });
@@ -480,7 +484,7 @@ function installNodeOtlp(projectDir) {
       packages_installed: false,
       start_script_patched: false,
       env_vars_configured: false,
-      error: `Failed to install Node.js OTLP packages: ${message}`
+      error: `Failed to install Node.js OTLP packages: ${truncateError(message)}`
     };
   }
 }
@@ -489,8 +493,14 @@ function patchNodeStartScript(projectDir) {
   if (!existsSync3(pkgPath)) {
     return false;
   }
-  const raw = readFileSync3(pkgPath, "utf-8");
-  const pkg = JSON.parse(raw);
+  let raw;
+  let pkg;
+  try {
+    raw = readFileSync3(pkgPath, "utf-8");
+    pkg = JSON.parse(raw);
+  } catch {
+    return false;
+  }
   const scripts = pkg["scripts"];
   if (!scripts) {
     return false;
@@ -554,6 +564,14 @@ function configureWeb(projectDir, stack) {
     } catch {
     }
   }
+  let endpoint = "http://localhost:4318";
+  try {
+    const currentState = readState(projectDir);
+    if (currentState.otlp?.endpoint) {
+      endpoint = currentState.otlp.endpoint;
+    }
+  } catch {
+  }
   const snippet = `// OpenTelemetry Web SDK initialization \u2014 generated by codeharness
 import { WebTracerProvider } from '@opentelemetry/sdk-trace-web';
 import { BatchSpanProcessor } from '@opentelemetry/sdk-trace-base';
@@ -563,7 +581,7 @@ import { XMLHttpRequestInstrumentation } from '@opentelemetry/instrumentation-xm
 import { registerInstrumentations } from '@opentelemetry/instrumentation';
 
 const exporter = new OTLPTraceExporter({
-  url: 'http://localhost:4318/v1/traces',
+  url: '${endpoint}/v1/traces',
 });
 
 const provider = new WebTracerProvider();
@@ -675,12 +693,12 @@ function instrumentProject(projectDir, stack, opts) {
       error: "Unsupported stack for OTLP instrumentation"
     };
   }
+  configureOtlpEnvVars(projectDir, stack, { appType });
+  result.env_vars_configured = true;
+  if (!isJson) {
+    ok("OTLP: environment variables configured");
+  }
   if (result.status === "configured") {
-    configureOtlpEnvVars(projectDir, stack, { appType });
-    result.env_vars_configured = true;
-    if (!isJson) {
-      ok("OTLP: environment variables configured");
-    }
     if (appType === "cli") {
       configureCli(projectDir);
       if (!isJson) {
@@ -874,6 +892,11 @@ ${patchContent}
 ${markers.end}`;
   const startIdx = content.indexOf(markers.start);
   const endIdx = content.indexOf(markers.end);
+  if (startIdx !== -1 !== (endIdx !== -1)) {
+    throw new Error(
+      `Corrupted patch markers for '${patchName}': only ${startIdx !== -1 ? "start" : "end"} marker found in ${filePath}`
+    );
+  }
   if (startIdx !== -1 && endIdx !== -1) {
     if (endIdx < startIdx) {
       throw new Error(
@@ -1060,8 +1083,8 @@ var BmadError = class extends Error {
 };
 var PATCH_TARGETS = {
   "story-verification": "bmm/workflows/4-implementation/create-story/template.md",
-  "dev-enforcement": "bmm/workflows/4-implementation/dev-story/checklist.md",
-  "review-enforcement": "bmm/workflows/4-implementation/code-review/checklist.md",
+  "dev-enforcement": "bmm/workflows/4-implementation/dev-story/instructions.xml",
+  "review-enforcement": "bmm/workflows/4-implementation/code-review/instructions.xml",
   "retro-enforcement": "bmm/workflows/4-implementation/retrospective/instructions.md",
   "sprint-beads": "bmm/workflows/4-implementation/sprint-planning/checklist.md",
   "sprint-retro": "bmm/workflows/4-implementation/sprint-planning/instructions.md"
@@ -1110,9 +1133,9 @@ function installBmad(dir) {
       patches_applied: []
     };
   }
-  const cmdStr = "npx bmad-method init";
+  const cmdStr = "npx bmad-method install";
   try {
-    execFileSync4("npx", ["bmad-method", "init"], {
+    execFileSync4("npx", ["bmad-method", "install"], {
       stdio: "pipe",
       timeout: 6e4,
       cwd: root
@@ -1121,6 +1144,9 @@ function installBmad(dir) {
     const message = err instanceof Error ? err.message : String(err);
     throw new BmadError(cmdStr, message);
   }
+  if (!isBmadInstalled(root)) {
+    throw new BmadError(cmdStr, "_bmad/ directory was not created after successful npx bmad-method install");
+  }
   const version = detectBmadVersion(root);
   return {
     status: "installed",
@@ -1177,6 +1203,21 @@ function applyAllPatches(dir) {
   }
   return results;
 }
+function detectBmalph(dir) {
+  const root = dir ?? process.cwd();
+  const files = [];
+  const ralphRcPath = join5(root, ".ralph", ".ralphrc");
+  if (existsSync5(ralphRcPath)) {
+    files.push(".ralph/.ralphrc");
+  }
+  const dotRalphDir = join5(root, ".ralph");
+  if (existsSync5(dotRalphDir)) {
+    if (files.length === 0) {
+      files.push(".ralph/");
+    }
+  }
+  return { detected: files.length > 0, files };
+}
 function generateStoryKey(epicNumber, storyNumber, title) {
   const slug = title.toLowerCase().replace(/[^a-z0-9\s-]/g, "").replace(/\s+/g, "-").replace(/-+/g, "-").replace(/^-|-$/g, "");
   return `${epicNumber}-${storyNumber}-${slug}`;
@@ -1402,7 +1443,7 @@ function getInstallCommand(stack) {
 }
 
 // src/commands/init.ts
-var HARNESS_VERSION = true ? "0.16.0" : "0.0.0-dev";
+var HARNESS_VERSION = true ? "0.17.0" : "0.0.0-dev";
 function getProjectName(projectDir) {
   try {
     const pkgPath = join6(projectDir, "package.json");
@@ -1498,7 +1539,7 @@ function generateDocsIndexContent() {
 }
 var DO_NOT_EDIT_HEADER = "<!-- DO NOT EDIT MANUALLY -->\n";
 function registerInitCommand(program) {
-  program.command("init").description("Initialize the harness in a project").option("--no-frontend", "Disable frontend enforcement").option("--no-database", "Disable database enforcement").option("--no-api", "Disable API enforcement").option("--otel-endpoint <url>", "Remote OTLP endpoint (skips local Docker stack)").option("--logs-url <url>", "Remote VictoriaLogs URL").option("--metrics-url <url>", "Remote VictoriaMetrics URL").option("--traces-url <url>", "Remote Jaeger/VictoriaTraces URL").action(async (options, cmd) => {
+  program.command("init").description("Initialize the harness in a project").option("--no-frontend", "Disable frontend enforcement").option("--no-database", "Disable database enforcement").option("--no-api", "Disable API enforcement").option("--no-observability", "Skip OTLP package installation").option("--otel-endpoint <url>", "Remote OTLP endpoint (skips local Docker stack)").option("--logs-url <url>", "Remote VictoriaLogs URL").option("--metrics-url <url>", "Remote VictoriaMetrics URL").option("--traces-url <url>", "Remote Jaeger/VictoriaTraces URL").action(async (options, cmd) => {
     const globalOpts = cmd.optsWithGlobals();
     const isJson = globalOpts.json === true;
     const projectDir = process.cwd();
@@ -1527,6 +1568,47 @@ function registerInitCommand(program) {
           result.documentation.agents_md = "exists";
           result.documentation.docs_scaffold = "exists";
           result.documentation.readme = "exists";
+          const depResults = [];
+          for (const spec of DEPENDENCY_REGISTRY) {
+            const check = checkInstalled(spec);
+            const depResult = {
+              name: spec.name,
+              displayName: spec.displayName,
+              status: check.installed ? "already-installed" : "failed",
+              version: check.version
+            };
+            depResults.push(depResult);
+            if (!isJson) {
+              if (check.installed) {
+                const versionStr = check.version ? ` (v${check.version})` : "";
+                ok(`${spec.displayName}: already installed${versionStr}`);
+              } else {
+                fail(`${spec.displayName}: not found`);
+              }
+            }
+          }
+          result.dependencies = depResults;
+          if (isBmadInstalled(projectDir)) {
+            try {
+              const patchResults = applyAllPatches(projectDir);
+              const patchNames = patchResults.filter((r) => r.applied).map((r) => r.patchName);
+              const version = detectBmadVersion(projectDir);
+              const bmalpHDetection = detectBmalph(projectDir);
+              result.bmad = {
+                status: "already-installed",
+                version,
+                patches_applied: patchNames,
+                bmalph_detected: bmalpHDetection.detected
+              };
+              if (!isJson) {
+                info("BMAD: already installed, patches verified");
+                if (bmalpHDetection.detected) {
+                  warn("bmalph detected \u2014 superseded files noted for cleanup");
+                }
+              }
+            } catch {
+            }
+          }
           if (isJson) {
             jsonOutput(result);
           } else {
@@ -1661,7 +1743,8 @@ function registerInitCommand(program) {
         result.bmad = {
           status: "already-installed",
           version,
-          patches_applied: patchNames
+          patches_applied: patchNames,
+          bmalph_detected: false
         };
         if (!isJson) {
           info("BMAD: existing installation detected, patches applied");
@@ -1673,18 +1756,27 @@ function registerInitCommand(program) {
         result.bmad = {
           status: installResult.status,
           version: installResult.version,
-          patches_applied: patchNames
+          patches_applied: patchNames,
+          bmalph_detected: false
         };
         if (!isJson) {
           ok(`BMAD: installed (v${installResult.version ?? "unknown"}), harness patches applied`);
         }
       }
+      const bmalpHDetection = detectBmalph(projectDir);
+      if (bmalpHDetection.detected && result.bmad) {
+        result.bmad.bmalph_detected = true;
+        if (!isJson) {
+          warn("bmalph detected \u2014 superseded files noted for cleanup");
+        }
+      }
     } catch (err) {
       if (err instanceof BmadError) {
         result.bmad = {
           status: "failed",
           version: null,
           patches_applied: [],
+          bmalph_detected: false,
           error: err.message
         };
         if (!isJson) {
@@ -1757,7 +1849,20 @@ function registerInitCommand(program) {
         ok("Documentation: README.md created");
       }
     }
-    const otlpResult = instrumentProject(projectDir, stack, { json: isJson, appType });
+    let otlpResult;
+    if (!options.observability) {
+      otlpResult = {
+        status: "skipped",
+        packages_installed: false,
+        start_script_patched: false,
+        env_vars_configured: false
+      };
+      if (!isJson) {
+        info("OTLP: skipped (--no-observability)");
+      }
+    } else {
+      otlpResult = instrumentProject(projectDir, stack, { json: isJson, appType });
+    }
     result.otlp = otlpResult;
     try {
       const updatedState = readState(projectDir);
@@ -7675,7 +7780,7 @@ function handleStatus(dir, isJson, filterStory) {
 }
 
 // src/index.ts
-var VERSION = true ? "0.16.0" : "0.0.0-dev";
+var VERSION = true ? "0.17.0" : "0.0.0-dev";
 function createProgram() {
   const program = new Command();
   program.name("codeharness").description("Makes autonomous coding agents produce software that actually works").version(VERSION).option("--json", "Output in machine-readable JSON format");

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "codeharness",
-  "version": "0.16.0",
+  "version": "0.17.0",
   "type": "module",
   "description": "CLI for codeharness — makes autonomous coding agents produce software that actually works",
   "bin": {

package/ralph/drivers/claude-code.sh

@@ -48,6 +48,9 @@ driver_valid_tools() {
         "Bash(showboat *)"
         "Bash(codeharness *)"
         "NotebookEdit"
+        "Skill"
+        "Agent"
+        "ToolSearch"
     )
 }
 

package/ralph/ralph.sh

@@ -7,6 +7,9 @@
 
 set -e
 
+# DEBUG: catch unexpected exits from set -e
+trap 'echo "[$(date "+%Y-%m-%d %H:%M:%S")] [FATAL] ralph.sh died at line $LINENO (exit code: $?)" >> "${LOG_DIR:-ralph/logs}/ralph_crash.log" 2>/dev/null' ERR
+
 SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 source "$SCRIPT_DIR/lib/date_utils.sh"
 source "$SCRIPT_DIR/lib/timeout_utils.sh"
@@ -581,6 +584,11 @@ execute_iteration() {
     local deadline=$(( $(date +%s) + timeout_seconds ))
     echo "$deadline" > "ralph/.iteration_deadline"
 
+    # DEBUG: log the command being run
+    log_status "DEBUG" "Command: ${CLAUDE_CMD_ARGS[*]}"
+    log_status "DEBUG" "Output file: $output_file"
+    log_status "DEBUG" "LIVE_OUTPUT=$LIVE_OUTPUT, timeout=${timeout_seconds}s"
+
     log_status "INFO" "Starting $(driver_display_name) (timeout: ${ITERATION_TIMEOUT_MINUTES}m)..."
 
     # Execute with timeout
@@ -606,6 +614,8 @@ execute_iteration() {
         local claude_pid=$!
         local progress_counter=0
 
+        log_status "DEBUG" "Background PID: $claude_pid"
+
         while kill -0 $claude_pid 2>/dev/null; do
             progress_counter=$((progress_counter + 1))
             if [[ -f "$output_file" && -s "$output_file" ]]; then
@@ -614,8 +624,23 @@ execute_iteration() {
             sleep 10
         done
 
+        # Protect wait from set -e — capture exit code without crashing
+        set +e
         wait $claude_pid
         exit_code=$?
+        set -e
+        log_status "DEBUG" "Claude exited with code: $exit_code, output size: $(wc -c < "$output_file" 2>/dev/null || echo 0) bytes"
+
+        # If output is empty and exit code is non-zero, log diagnostic info
+        if [[ ! -s "$output_file" && $exit_code -ne 0 ]]; then
+            log_status "ERROR" "Claude produced no output and exited with code $exit_code"
+            log_status "DEBUG" "Checking if claude binary is responsive..."
+            if claude --version > /dev/null 2>&1; then
+                log_status "DEBUG" "claude binary OK: $(claude --version 2>&1)"
+            else
+                log_status "ERROR" "claude binary not responding"
+            fi
+        fi
     fi
 
     if [[ $exit_code -eq 0 ]]; then