@matt82198/aesop 0.4.0 → 0.4.1

package/docs/README.md

@@ -77,7 +77,7 @@ The `/buildsystem` skill runs **one complete iteration of the autonomous deliver
 This is the repeatable loop that runs your delivery cycle indefinitely, with each wave learning from the prior audit. You can run `/buildsystem` once per wave (typically 30 min–2 hours depending on backlog size). See [HOW-THE-LOOP-WORKS.md](HOW-THE-LOOP-WORKS.md) for a concrete walkthrough.
 
 ### Team State & Multi-Instance Design
-**Current Status (0.1.0)**: Single-instance proven. A team uses Aesop by designating one operator who runs the wave loop. State is durably checkpointed in git (STATE.md, BUILDLOG.md, tracker.json exports).
+**Current Status (0.4.0)**: Single-instance proven, with swappable worker + orchestrator model backends (Claude, Codex, OpenAI-compatible). See [MICROKERNEL.md](MICROKERNEL.md) for the two-seat architecture + a 60-second model-swap quickstart. State is durably checkpointed in git (STATE.md, BUILDLOG.md, tracker.json exports).
 
 **In Design**: Multi-instance coordination via the state_store substrate. The event-sourced SQLite layer is production-ready but currently opt-in. A future release will enable multiple Aesop instances (e.g., per-team subgroups or geographic regions) to coordinate around a single source of truth—a Postgres-backed event log, with git as a diffable export. See [TEAM-STATE.md](TEAM-STATE.md) (design in progress) for the vision and current architecture decisions.
 
@@ -122,7 +122,7 @@ Once you've completed the adopter journey, use these for operational reference:
 - **[av-resilience.md](av-resilience.md)** — Antivirus and behavioral-engine resilience patterns for reliable agent execution
 
 ### Lessons & Case Studies
-- **[autonomous-swe.md](autonomous-swe.md)** — The 0.1.0-rc.1 milestone told honestly: what "autonomous SWE" means here (a fleet running the wave loop under a human who owns the outward gates), the committed evidence behind each claim (held-out benchmark, verified audit, proven kill-switch, reproducible package), and the limits the project owns
+- **[autonomous-swe.md](autonomous-swe.md)** — What "autonomous SWE" means here (a fleet running the wave loop under a human who owns the outward gates): the committed evidence behind each claim (held-out benchmark, verified audit, proven kill-switch, reproducible package), the 0.1.0-rc.1 baseline, 0.4.0 evolution (multi-model support), and the limits the project owns
 - **[case-study-portfolio.md](case-study-portfolio.md)** — How Aesop built its own portfolio site; full audit trail and cost breakdown
 - **[SCRIPTS-POLICY.md](SCRIPTS-POLICY.md)** — Local-only execution, shared script library (`~/scripts`), task-local vs. reusable heuristics
 
@@ -136,7 +136,7 @@ Once you've completed the adopter journey, use these for operational reference:
 **I want to understand the cost model**
 → [DISPATCH-MODEL.md](DISPATCH-MODEL.md) or [HOW-THE-LOOP-WORKS.md](HOW-THE-LOOP-WORKS.md#why-its-fast--cheap)
 
-**I want to know what's actually proven vs. claimed (the 0.1.0 milestone)**
+**I want to know what's actually proven vs. claimed**
 → [autonomous-swe.md](autonomous-swe.md)
 
 **I need to understand how state survives a crash**

package/docs/THE-AESOP-HYPOTHESIS.md

@@ -8,6 +8,19 @@
 
 **Agent behavior is source code.** Everything a fleet does — every decision, every checkpoint, every recovery path — lives in durable, human-diffable files: git history, plain-text STATE.md, append-only BUILDLOG.md, Python scripts, shell hooks. No vector embeddings, no distributed consensus, no magic. When a machine fails, you re-read from disk. When a human operator needs to audit a decision, they grep the git log or read the state file. When you need to reason about cost, you look at the dispatch rules in code.
 
+## Ancestors: Naming the Ideas
+
+Aesop does not invent crash-recoverable orchestration. It **composes and measures** four proven ideas from distributed systems research:
+
+1. **Temporal** (Czezatke & Stengel, 2019) — durable execution via plain-text event logs, essential for crash recovery without external state.
+2. **Crash-only software** (Candea & Fox, 2003) — design every component as stateless; recovery is the normal startup path.
+3. **Erlang/OTP supervision trees** (Armstrong et al., 1996) — organize fault tolerance as a hierarchy of restarts, each with a bounded retry policy.
+4. **Kubernetes controller reconciliation** (Burns et al., 2015) — durable desired state + controllers that converge reality to it via append-only logs.
+
+Aesop ports these patterns to agent orchestration: the wave loop is a reconciliation controller; STATE.md + BUILDLOG.md are the durable event log; workers are supervised processes with retry caps; crashes trigger re-reads from disk. The contribution is not any single idea, but the composition, measurement (benchmarks with committed artifacts), and the discipline of naming ancestors instead of claiming novelty.
+
+---
+
 This hypothesis rests on five pillars:
 
 1. **Durable plain-text state** — git + POSIX text as the state layer, not Postgres or vector DBs.
@@ -60,7 +73,9 @@ Today's dispatch is flat: one Opus/Fable orchestrator on the main thread, 5–8
 
 **The benchmark proves Haiku is good enough.** The held-out judgment benchmark (v3 = 28 additional tasks, building on v2 = 11 prior) tested Haiku, Sonnet, and Opus across 39 combined judgment tasks: bug-in-diff (with concurrency races and resource leaks), finding-inflation, acceptance-criteria coverage, severity calibration, root-cause-from-trace, refactor-equivalence, security issue spotting. All three models converged on identical answers for all 28 v3 tasks. Combined score: **Haiku 39/39** vs **Opus 38/39** (Opus erred on one severity call; Haiku did not). At ~1/3 the per-token cost.
 
-**Honest limits on the benchmark:** Curated (N=39), not sampled from real fleet transcripts. No frontier-reaching task found where Opus beats Haiku. The benchmark maps a floor ("Haiku is sufficient for these judgment shapes"), not the absolute frontier. Cost is token-price ratio, not wall-clock latency. These are not hidden; they are load-bearing caveats.
+**Ceiling rule: the benchmark demonstrates sufficiency, not equivalence.** A pre-declared ceiling rule (when ≥2 tiers score ≥92%, the instrument failed to discriminate) trips on this result: both Haiku and Sonnet achieved 39/39 (100%), confirming the benchmark found a *convergence zone* where both models ace the task set, not a *separating frontier* where one outperforms the other. This is the intended result — the honest interpretation is that **Haiku is sufficient for the judgment shapes measured here**, not that it is tier-equivalent to Sonnet or Opus everywhere. See [`bench/results/2026-07-26-judgment-v3-ceiling-addendum.md`](../bench/results/2026-07-26-judgment-v3-ceiling-addendum.md) for the full forensic analysis.
+
+**Honest limits on the benchmark:** Curated (N=39), not sampled from real fleet transcripts. No frontier-reaching task found where Opus beats Haiku in this set. The benchmark maps a floor ("Haiku is sufficient for scoped judgment and extraction tasks with context at the seam"), not the absolute frontier. **What it does NOT test**: open-ended synthesis (designing novel systems from scratch), frontier reasoning (problems requiring 100+ steps of chaining or novel proof techniques), long-horizon planning (multi-phase dependency graphs). Cost is token-price ratio, not wall-clock latency. For detailed equivalence analysis, see [`bench/EQUIVALENCE-MARGIN.md`](../bench/EQUIVALENCE-MARGIN.md). These caveats are not hidden; they are load-bearing.
 
 **Cite:** [`docs/DISPATCH-MODEL.md`](./DISPATCH-MODEL.md) — cost model and patterns; [`bench/results/2026-07-17-judgment-v3-haiku-sonnet-opus.md`](../bench/results/2026-07-17-judgment-v3-haiku-sonnet-opus.md) — benchmark run and interpretation.
 
@@ -103,8 +118,8 @@ The point: **observability means you see the real bottleneck**, and you fix it,
 
 These are not claims about what Aesop *could* do. They are receipts:
 
-- **1,088 commits, 251 merged PRs, 30 waves** (verified by anyone who clones; `tools/self_stats.py`).
-- **143,403 lines of code** across 546 files, delivered end-to-end: from feature intake to merge.
+- **1,181 commits, 387 merged PRs, 30 waves** (verified by anyone who clones; `tools/self_stats.py`).
+- **173,035 lines of code** across 642 files tracked, delivered end-to-end: from feature intake to merge.
 - **Benchmark results** committed in `bench/results/` — 39 judgment tasks, all models scored by deterministic Python scoring (no LLM in the grading loop).
 - **Kill-switch proof** — `tools/halt.py` is wired into the live dispatch path and was exercised on a real wave.
 - **Cost ceiling** — implemented in `tools/cost_ceiling.py`, enforced per-wave.
@@ -119,9 +134,10 @@ This is not a universal solution. The system has explicit boundaries:
 
 1. **Single-box by design.** Aesop runs on one machine. Multi-instance coordination is on the roadmap, not shipped. If you need 100-machine scale today, this is not the tool.
 2. **Small-N benchmarks.** 39 judgment tasks is directional evidence, not statistical proof. Frontier reasoning (where Opus depth might matter 3×) is not tested here.
-3. **Lab-measured multi-writer throughput.** 800 events/sec is measured in a stress test, not production. Team scale beyond one machine requires additional work (leases, event-sourcing, distributed consensus).
-4. **No third-party verification yet.** The artifacts are committed so a skeptic can reproduce — that is transparency, not independent replication.
-5. **Release candidate.** APIs, config, and dashboard contracts may still shift. Pin the exact version if you need stability.
+3. **Scored judgment vs. open-ended generation — a measured boundary.** An all-Haiku audit (wave-24) reported four P0 issues, verification found zero real (2 hallucinated, 2 severity-inflated). This is not a Haiku failure — it is a *cheap-model failure mode* when the model chooses what to report in open-ended generation. The benchmark proves Haiku holds on *scored, single-shot judgment* (when the task structure and rubric are explicit). The architecture mitigates this boundary by pairing cheap generation (workers) with independent verification (multi-tier review gate) — the separation of concerns is precisely designed for this measured failure mode. Selection (deciding what to report) is where cheap models diverge from expensive ones; scoring (evaluating a bounded task with a rubric) is where they converge.
+4. **Lab-measured multi-writer throughput.** 800 events/sec is measured in a stress test, not production. Team scale beyond one machine requires additional work (leases, event-sourcing, distributed consensus).
+5. **No third-party verification yet.** The artifacts are committed so a skeptic can reproduce — that is transparency, not independent replication.
+6. **Release candidate.** APIs, config, and dashboard contracts may still shift. Pin the exact version if you need stability.
 
 ---
 
@@ -135,6 +151,6 @@ Aesop bets on:
 - **Small is faster than smart.** Flat fan-out (5–8 Haiku agents) beats hierarchical dispatch (4.3× cost), even at scale, because the simpler system has fewer failure modes.
 - **Cost as a first-class constraint.** The whole system is designed around $0.01–0.02 per wave. Expensive paths are rejected before they ship.
 
-The evidence is in the receipts: 1,088 commits, 251 PRs, 30 waves, zero hallucinated audits (via adversarial verification), and a benchmark that proves Haiku is good enough.
+The evidence is in the receipts: 1,181 commits, 387 PRs, 30 waves, zero hallucinated audits (via adversarial verification), and a benchmark that proves Haiku is good enough for scoped judgment work.
 
 **Read more:** [`docs/autonomous-swe.md`](./autonomous-swe.md) — honest account of what shipped, what didn't, and where the gaps are.

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@matt82198/aesop",
-  "version": "0.4.0",
+  "version": "0.4.1",
   "description": "Multi-agent orchestration for AI coding agents, designed to survive failure — a plain-file brain, git as the only durable layer, cheap subagent fleets, and guardrails enforced in code.",
   "bin": {
     "aesop": "bin/cli.js"

package/tools/doctor.js

@@ -9,6 +9,7 @@
 
 const fs = require('fs');
 const path = require('path');
+const os = require('os');
 const { spawnSync } = require('child_process');
 const net = require('net');
 
@@ -34,17 +35,19 @@ function colorFail() {
 function checkNodeVersion() {
   const version = parseInt(process.versions.node.split('.')[0], 10);
   const passed = version >= 18;
-  const hint = passed ? '' : `Found Node.js v${process.versions.node}, need >=18`;
+  const hint = passed ? `v${process.versions.node}` : `Found Node.js v${process.versions.node}, need >=18`;
   return { passed, hint };
 }
 
-// Check Python available (python3 or python)
+// Check Python available and version >= 3.10
 function checkPython() {
   // Try python3 first
-  const result3 = spawnSync('python3', ['--version'], { stdio: 'ignore', timeout: 5000 });
+  let pythonBin = null;
+  let result3 = spawnSync('python3', ['--version'], { encoding: 'utf8', timeout: 5000 });
+
   if (result3.error && result3.error.code === 'ENOENT') {
     // python3 not found, try python fallback
-    const result = spawnSync('python', ['--version'], { stdio: 'ignore', timeout: 5000 });
+    const result = spawnSync('python', ['--version'], { encoding: 'utf8', timeout: 5000 });
     if (result.error && result.error.code === 'ENOENT') {
       // Neither python3 nor python found
       return { passed: false, hint: 'python3 or python not found on PATH' };
@@ -53,13 +56,35 @@ function checkPython() {
       // python exists but returned non-zero exit code
       return { passed: false, hint: 'python found but returned non-zero exit code' };
     }
-    return { passed: true, hint: '' };
-  }
-  if (result3.status !== 0) {
+    pythonBin = 'python';
+  } else if (result3.status !== 0) {
     // python3 exists but returned non-zero exit code
     return { passed: false, hint: 'python3 found but returned non-zero exit code' };
+  } else {
+    pythonBin = 'python3';
+  }
+
+  // Extract version from python output
+  let versionStr = '';
+  try {
+    const versionOutput = result3.stdout || result3.stderr || '';
+    const match = versionOutput.match(/Python\s+(\d+)\.(\d+)/i);
+    if (match) {
+      const major = parseInt(match[1], 10);
+      const minor = parseInt(match[2], 10);
+      versionStr = `${major}.${minor}`;
+      if (major > 3 || (major === 3 && minor >= 10)) {
+        return { passed: true, hint: `v${versionStr}` };
+      } else {
+        return { passed: false, hint: `Found Python v${versionStr}, need >=3.10` };
+      }
+    }
+  } catch (e) {
+    // Version extraction failed but python runs
+    return { passed: true, hint: 'Available (version check skipped)' };
   }
-  return { passed: true, hint: '' };
+
+  return { passed: true, hint: 'Available' };
 }
 
 // Check git repo (.git directory exists)
@@ -78,13 +103,98 @@ function checkConfig() {
       return { passed: false, hint: 'aesop.config.json not found' };
     }
     const content = fs.readFileSync(configPath, 'utf8');
-    JSON.parse(content);
+    const config = JSON.parse(content);
+
+    // Validate repos array exists and is an array
+    if (!('repos' in config)) {
+      return { passed: false, hint: 'Missing required field: repos' };
+    }
+    if (!Array.isArray(config.repos)) {
+      return { passed: false, hint: 'repos field must be an array' };
+    }
+
+    // Validate aesop_root exists on disk (if present in config)
+    if (config.aesop_root && !fs.existsSync(config.aesop_root)) {
+      return { passed: false, hint: `aesop_root does not exist: ${config.aesop_root}` };
+    }
+
+    // Validate repo paths exist (if specified)
+    for (const repo of config.repos) {
+      if (repo.path && !fs.existsSync(repo.path)) {
+        return { passed: false, hint: `Repo path does not exist: ${repo.path}` };
+      }
+    }
+
     return { passed: true, hint: '' };
   } catch (e) {
-    return { passed: false, hint: `Config parse error: ${e.message}` };
+    return { passed: false, hint: `Config error: ${e.message}` };
   }
 }
 
+// Check for placeholder repo URLs and warn
+function checkRepoURLs() {
+  const configPath = path.join(CURRENT_DIR, 'aesop.config.json');
+  try {
+    if (!fs.existsSync(configPath)) {
+      // Skip this check if config doesn't exist (main checkConfig will fail)
+      return { passed: true, hint: '' };
+    }
+    const content = fs.readFileSync(configPath, 'utf8');
+    const config = JSON.parse(content);
+
+    // Check for placeholder URLs
+    const placeholderPattern = /https:\/\/github\.com\/user\//i;
+    if (config.repos && Array.isArray(config.repos)) {
+      for (const repo of config.repos) {
+        if (repo.url && placeholderPattern.test(repo.url)) {
+          return {
+            passed: false,
+            hint: `Placeholder URL found: ${repo.url}. Replace with actual repo URL or remove the repo entry.`
+          };
+        }
+      }
+    }
+
+    return { passed: true, hint: '' };
+  } catch (e) {
+    return { passed: true, hint: '' };  // Skip check on config parse error (main checkConfig handles it)
+  }
+}
+
+// Check if ~/.claude/skills/power/SKILL.md and ~/.claude/skills/buildsystem/SKILL.md exist (WARN only)
+function checkSkillsFiles() {
+  const homeDir = os.homedir();
+  const powerSkill = path.join(homeDir, '.claude', 'skills', 'power', 'SKILL.md');
+  const buildsystemSkill = path.join(homeDir, '.claude', 'skills', 'buildsystem', 'SKILL.md');
+
+  const powerExists = fs.existsSync(powerSkill);
+  const buildsystemExists = fs.existsSync(buildsystemSkill);
+
+  // Skills are required but we WARN, not FAIL, to allow doctor to proceed on partial setup
+  if (!powerExists && !buildsystemExists) {
+    return {
+      passed: true,  // WARN, not fail
+      hint: `⚠ Skills not found at ~/.claude/skills/. Required before running orchestrator. Copy: cp -r ~/.claude/skills/ here`
+    };
+  }
+
+  if (!powerExists) {
+    return {
+      passed: true,  // WARN, not fail
+      hint: `⚠ power/SKILL.md not found at ${powerSkill}. Required before running orchestrator.`
+    };
+  }
+
+  if (!buildsystemExists) {
+    return {
+      passed: true,  // WARN, not fail
+      hint: `⚠ buildsystem/SKILL.md not found at ${buildsystemSkill}. Required before running orchestrator.`
+    };
+  }
+
+  return { passed: true, hint: 'Both skills present' };
+}
+
 // Check required directories exist
 function checkDirectories() {
   const requiredDirs = ['daemons', 'dash', 'monitor', 'tools', 'ui'];
@@ -174,9 +284,11 @@ function formatRow(label, status, hint) {
 
     const syncChecks = [
       { label: 'Node.js version ≥18', fn: checkNodeVersion },
-      { label: 'Python (python3 or python)', fn: checkPython },
+      { label: 'Python version ≥3.10', fn: checkPython },
       { label: 'Git repository', fn: checkGitRepo },
-      { label: 'aesop.config.json (valid JSON)', fn: checkConfig },
+      { label: 'aesop.config.json (structure & required fields)', fn: checkConfig },
+      { label: 'Repository URLs (no placeholders)', fn: checkRepoURLs },
+      { label: 'Skills files (power & buildsystem)', fn: checkSkillsFiles },
       { label: 'Required directories (daemons, dash, monitor, tools, ui)', fn: checkDirectories },
       { label: 'Git pre-push hook installed', fn: checkPrePushHook }
     ];

package/tools/reproduce.js

@@ -44,9 +44,10 @@ function colorSkip() {
 
 // Detect context: repo checkout vs installed package
 function detectContext() {
-  // Repo checkout has .git/config and package.json with npm scripts
+  // Repo checkout has .git/config, package.json with npm scripts, AND package-lock.json
   const hasGitConfig = fs.existsSync(path.join(PACKAGE_ROOT, '.git', 'config'));
   const hasPackageJson = fs.existsSync(path.join(PACKAGE_ROOT, 'package.json'));
+  const hasPackageLock = fs.existsSync(path.join(PACKAGE_ROOT, 'package-lock.json'));
   const hasTestScripts = hasPackageJson && (() => {
     try {
       const pkg = JSON.parse(fs.readFileSync(path.join(PACKAGE_ROOT, 'package.json'), 'utf8'));
@@ -57,7 +58,8 @@ function detectContext() {
   })();
 
   // Installed package is in node_modules with limited structure
-  const isInstalled = !hasGitConfig && !hasTestScripts;
+  // OR repo-mode detected but package-lock.json is missing (fresh scaffold — degrade gracefully)
+  const isInstalled = (!hasGitConfig && !hasTestScripts) || (hasGitConfig && !hasPackageLock);
 
   return isInstalled ? 'installed' : 'repo';
 }
@@ -368,7 +370,14 @@ function runInstalledMode() {
 (async function main() {
   try {
     const context = detectContext();
+    const hasGitConfig = fs.existsSync(path.join(PACKAGE_ROOT, '.git', 'config'));
+    const hasPackageLock = fs.existsSync(path.join(PACKAGE_ROOT, 'package-lock.json'));
+    const isDegradedFromRepo = hasGitConfig && !hasPackageLock;
+
     console.log(`${COLORS.DIM}Context: ${context}${COLORS.RESET}`);
+    if (isDegradedFromRepo) {
+      console.log(`${COLORS.YELLOW}Scaffolded project detected — running installed-mode verification; full reproduce is for the aesop repo itself${COLORS.RESET}`);
+    }
 
     let results;
     if (context === 'repo') {

package/tools/self_stats.py

@@ -69,19 +69,67 @@ class GitStats:
 
     @property
     def merged_prs(self) -> int:
-        """Count merge commits with 'Merge pull request #' in message."""
+        """Count merged PRs using gh API (preferred) or git fallback.
+
+        Prefers `gh` API: gh api -X GET search/issues -f q="repo:matt82198/aesop is:pr is:merged" --jq .total_count
+        Falls back to git: counts distinct PR numbers from both merge-commit and squash-merge patterns.
+        """
         if self._merged_prs is not None:
             return self._merged_prs
 
+        # Try gh API first (preferred, authoritative)
         try:
-            # Get all commit messages
-            output = self._run_git("log", "--format=%B", check=False)
+            result = subprocess.run(
+                [
+                    "gh",
+                    "api",
+                    "-X",
+                    "GET",
+                    "search/issues",
+                    "-f",
+                    "q=repo:matt82198/aesop is:pr is:merged",
+                    "--jq",
+                    ".total_count"
+                ],
+                cwd=str(self.repo_root),
+                capture_output=True,
+                text=True,
+                encoding="utf-8",
+                errors="replace",
+                check=False,
+                timeout=10
+            )
+            if result.returncode == 0 and result.stdout:
+                try:
+                    count = int(result.stdout.strip())
+                    self._merged_prs = count
+                    return count
+                except (ValueError, TypeError):
+                    # gh returned non-numeric output, fall through to git
+                    pass
+        except (FileNotFoundError, subprocess.TimeoutExpired, Exception):
+            # gh not found, timeout, or other error — fall through to git
+            pass
+
+        # Git fallback: count distinct PR numbers from commit subjects
+        try:
+            output = self._run_git("log", "--format=%s", check=False)
             if not output:
                 self._merged_prs = 0
                 return 0
 
-            # Count lines matching "Merge pull request #"
-            count = output.count("Merge pull request #")
+            # Collect distinct PR numbers from both patterns:
+            # 1. "Merge pull request #N" (merge-commit style)
+            # 2. "(#N)" at end of subject (squash-merge style)
+            pr_numbers = set()
+
+            for match in re.finditer(r"^Merge pull request #(\d+)", output, re.MULTILINE):
+                pr_numbers.add(int(match.group(1)))
+
+            for match in re.finditer(r"\(#(\d+)\)\s*$", output, re.MULTILINE):
+                pr_numbers.add(int(match.group(1)))
+
+            count = len(pr_numbers)
             self._merged_prs = count
             return count
         except Exception:
@@ -211,7 +259,11 @@ class GitStats:
 
     @property
     def distinct_coauthors(self) -> int:
-        """Count of distinct authors including co-authors."""
+        """Count of distinct authors including co-authors.
+
+        Filters out fixture identities (Test User <test@example.com>) that leaked
+        into commits due to test config pollution (fix/git-identity-guard).
+        """
         if self._distinct_coauthors is not None:
             return self._distinct_coauthors
 
@@ -230,6 +282,9 @@ class GitStats:
                 for match in re.finditer(r"Co-Authored-By:\s*(.+?)(?:\n|$)", commit_msg):
                     coauthor = match.group(1).strip()
                     if coauthor:
+                        # Exclude fixture identities (test pollution)
+                        if coauthor == "Test User <test@example.com>":
+                            continue
                         authors.add(coauthor)
 
             count = len(authors)