@matt82198/aesop 0.2.0 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +28 -0
- package/README.md +50 -260
- package/bin/cli.js +7 -3
- package/docs/HOOK-INSTALL.md +15 -56
- package/docs/INSTALL.md +4 -3
- package/docs/PORTING.md +166 -0
- package/docs/TEAM-STATE.md +16 -184
- package/docs/reproduce.md +33 -3
- package/driver/CLAUDE.md +50 -48
- package/driver/codex_driver.py +59 -12
- package/driver/openai_transport.py +33 -0
- package/driver/wave_bridge.py +9 -1
- package/driver/wave_loop.py +437 -70
- package/driver/wave_scheduler.py +890 -0
- package/hooks/pre-push-policy.sh +22 -5
- package/monitor/collect-signals.mjs +69 -0
- package/package.json +4 -4
- package/state_store/__init__.py +2 -1
- package/state_store/projections.py +63 -0
- package/state_store/read_api.py +156 -0
- package/state_store/write_api.py +462 -0
- package/tools/cost_ceiling.py +106 -15
- package/tools/cost_projection.py +559 -0
- package/tools/crossos_drift.py +394 -0
- package/tools/eod_sweep.py +137 -33
- package/tools/mutation_test.py +130 -8
- package/tools/proposals.mjs +47 -2
- package/tools/reproduce.js +405 -0
- package/tools/secret_scan.py +73 -18
- package/tools/stall_check.py +247 -16
- package/tools/stateapi_lint.py +325 -0
- package/tools/test_battery.py +173 -0
- package/tools/verify_cost_panel.py +345 -0
- package/tools/wave_preflight.py +296 -29
- package/tools/wave_templates.py +170 -45
- package/ui/collectors.py +81 -0
- package/ui/handler.py +230 -85
- package/ui/sse.py +3 -3
- package/ui/wave_telemetry.py +24 -18
- package/ui/web/dist/assets/{index-DqZLgwNg.css → index-CNQxaiOW.css} +1 -1
- package/ui/web/dist/assets/index-CP68RIh3.js +9 -0
- package/ui/web/dist/index.html +2 -2
- package/docs/QUICKSTART.md +0 -80
- package/ui/web/dist/assets/index-BGbgw2Nh.js +0 -9
package/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,26 @@ All notable changes to Aesop are documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [0.3.1] - 2026-07-22 - Waves 28-31
|
|
9
|
+
|
|
10
|
+
### Added
|
|
11
|
+
- **Non-Claude core proof (release gate 1)**: a Codex-backed worker (gpt-4o-mini via AgentDriver) built, verified, and shipped a full wave increment end to end (PR #325); JSON-schema-capable model gate with `allow_unverified_models` escape.
|
|
12
|
+
- **Wave scheduler + gate-1 kit** (wave-28): dispatch prioritization plus the proof harness used for the multi-model gate.
|
|
13
|
+
- **WriteAPI seam + OCC** (wave-29): mutation-gate write layer with optimistic concurrency control for state updates.
|
|
14
|
+
- **Cost window unification + analytics panel** (wave-30): unified cost-tracking window with dashboard burn-rate and per-model spend views.
|
|
15
|
+
- **Parallel local test battery** (`tools/test_battery.py`): runs the 4-harness union (python/node/shell/ui) concurrently with per-harness logs and enforced timeouts; ~5.4 min vs ~10 min serial, now the standard local gate.
|
|
16
|
+
- **Frontier external-benchmark slice** (wave-31): spend-capped external grading slice for the held-out benchmark.
|
|
17
|
+
- **Stall recovery + preflight backlog flags** (wave-31): recovery heuristics and preflight validation that rejects malformed backlog items before dispatch.
|
|
18
|
+
|
|
19
|
+
### Fixed
|
|
20
|
+
- **Windows CI green and release-blocking**: `eod_sweep` repo-list split on `:` ate drive letters (silent vacuous SAFE) — now `os.pathsep` with fail-closed missing-repo findings; `stall_check` path containment resolves 8.3 short paths on both sides; per-harness timeout now kills the process tree.
|
|
21
|
+
- **Refinement-loop convergence (release gate 2)**: 4 full audit rounds, ~30 verified defects fixed, ~10 findings refuted on adversarial verification, final round clean.
|
|
22
|
+
- **Test-fixture git-identity isolation**: the pre-push hook self-test wrote `git config user.name` unscoped on every shell-suite run — now scoped inside its fixture; added a writes-only identity-hygiene scanner, self-guarding fixtures, and a README canary in the regression lens.
|
|
23
|
+
|
|
24
|
+
### Security
|
|
25
|
+
- `secret_scan`: narrow reviewed exemption (`ALLOWED-REDACTION-SOURCE`) for the redaction-regex source fixture — single pattern, single file, always reported, never silent.
|
|
26
|
+
- Pre-push gate runs **main's** scanner so a branch cannot weaken its own gate.
|
|
27
|
+
|
|
8
28
|
## [0.2.0] - 2026-07-21
|
|
9
29
|
|
|
10
30
|
Minor release shipping multi-model orchestration portability and adaptive verification safety.
|
|
@@ -17,6 +37,14 @@ Minor release shipping multi-model orchestration portability and adaptive verifi
|
|
|
17
37
|
- **Cost-ceiling enforcement** (wave-32): Per-wave spend ceiling enforced at dispatch time; blocks work if budget exceeded.
|
|
18
38
|
- **Transcript-sampled benchmark Phase 1** (wave-32): Infrastructure to extract coding tasks from real Claude Code session transcripts, enabling dynamic benchmark growth beyond hand-written examples.
|
|
19
39
|
- **Backend config & role resolution** (wave-32): `backend_config.py` for per-deployment model mappings (worker/setup/verify roles) without orchestrator changes.
|
|
40
|
+
- **Accuracy harness Phase 1** (wave-27): Offline + live benchmark infrastructure; live run measured gpt-4o-mini 32/32 composite on 2026-07-22 (`bench/results/accuracy-live-2026-07-22.json`).
|
|
41
|
+
- **Cost projection tool** (wave-27): Burn-rate tracking with 70/90-percentile alerting for proactive spend monitoring.
|
|
42
|
+
- **StateAPI read facade** (wave-27): ReadAPI layer delegates to existing parsers (tracker snapshot, orchestrator status, heartbeat via `tools/common`, ledger via `tools/fleet_ledger`); read-only, no logic fork.
|
|
43
|
+
- **Cross-repo portability Phase 1** (wave-27): Per-repo aesop.config.json and initialization enables multi-repo orchestration with unified state backend.
|
|
44
|
+
- **aesop reproduce subcommand** (wave-27): Re-run completed wave items end-to-end from transcript (debugging, verification, metrics).
|
|
45
|
+
- **PORTING guide** (wave-27): Comprehensive docs/PORTING.md for adopters integrating Aesop into existing codebases.
|
|
46
|
+
- **Windows CI job** (wave-27): Non-required but validated GitHub Actions workflow for Windows build parity.
|
|
47
|
+
- **Transcript-sampled benchmark N=150** (wave-27): Expanded dynamic benchmark sourced from real Claude Code transcripts; N=150 curated task extracts.
|
|
20
48
|
|
|
21
49
|
### Changed
|
|
22
50
|
- **Verification policy transparent** (wave-32): Verification tiers now driven by `AgentDriver.probe_capabilities()` — config-free, capability-driven safety.
|
package/README.md
CHANGED
|
@@ -12,159 +12,67 @@
|
|
|
12
12
|
<a href="LICENSE"><img src="https://img.shields.io/badge/license-PolyForm%20Strict%201.0.0-orange.svg" alt="License: PolyForm Strict 1.0.0 (source-available)"></a>
|
|
13
13
|
</p>
|
|
14
14
|
|
|
15
|
-
|
|
15
|
+
## What It Does
|
|
16
16
|
|
|
17
|
-
|
|
17
|
+
**Aesop** is a source-available orchestration harness that runs multi-agent workflows on any repository. It ranks backlog, dispatches parallel Haiku workers (scoped to disjoint file ownership), verifies merges with adversarial safety, and feeds learnings into the next wave. Durable state (SQLite + git-rendered checkpoints) survives machine wipes and enables team coordination. This repo's own 223 PRs across 880 commits were delivered by Aesop's own `/buildsystem` wave loop—a supervised loop under a human operator who sets goals and owns outward gates (npm publish, releases, history rewrites). The stack is portable: swappable backends (Claude Code reference, OpenAI-compatible Ollama/OpenRouter, Codex bridge), with auto-tuned verification safety calibrated per backend.
|
|
18
18
|
|
|
19
|
-
|
|
20
|
-
|
|
21
|
-
**Cost-optimized multi-agent dispatch** (Haiku-first subagents, lean orchestrator) · **Durable state** (SQLite events + git exports, survives wipes) · **Observable machinery** (every agent run logged, every cost tracked) · **Live dashboard** (real-time fleet health at http://localhost:8770) · **Security gates** (secret-scan blocks pushes, CI validates each merge).
|
|
22
|
-
|
|
23
|
-
## Milestone: it shipped itself (0.1.0)
|
|
24
|
-
|
|
25
|
-
Aesop reached its first release candidate by running its own wave loop—**audit → parallel build → verify → merge-train**—across the backlog that produced it. This is the version where the load-bearing claims stopped being illustrations and became measurements. The word "autonomous" has a precise, deliberately narrow meaning here.
|
|
26
|
-
|
|
27
|
-
**What "autonomous" means here (and what it doesn't).** The *fleet* autonomously runs the wave loop: it ranks work, dispatches parallel Haiku workers on file-disjoint domains, verifies merges, and feeds the next wave from a closing audit. A *human* sets the goals and owns every outward gate—npm publish, tagged releases, and history rewrites all stay human-approved. It is a supervised loop, not an unsupervised agent, and not AGI. The true model-dispatch core runs inside the Claude Code harness, which lives outside this repo; what ships here is the harness around it—orchestration, guardrails, dashboard, and tooling.
|
|
28
|
-
|
|
29
|
-
The differentiator is not "an AI wrote code." It's that the credibility and safety claims come with receipts:
|
|
30
|
-
|
|
31
|
-
| Claim | Evidence (committed, checkable) | Honest caveat |
|
|
32
|
-
| --- | --- | --- |
|
|
33
|
-
| **Haiku is good enough for fleet judgment** | Across 39 held-out judgment tasks, Haiku scored **39/39** vs Opus **38/39** at ~1/3 the per-token cost — measured by a plain-Python scorer with no model in the grading loop. See [`bench/results/`](./bench/results/). | Curated set, **not** sampled from real fleet transcripts (N=39). The benchmark found *no* task where Opus beats Haiku — so it proves sufficiency for these shapes, not parity at the reasoning frontier. |
|
|
34
|
-
| **Audits don't hallucinate findings** | A full release audit was run with adversarial verification of every finding: **0 hallucinated issues**, closing the prior all-Haiku severity-inflation risk. | An internal audit, not a third-party one. |
|
|
35
|
-
| **Kill-switch actually stops the fleet** | The fleet-wide halt is wired into the live dispatch path and proven end-to-end — one signal **aborted a real wave with zero workers spawned**. See [`tools/halt.py`](./tools/halt.py). | Operator-triggered; it is a manual brake, not an autonomous safety monitor. |
|
|
36
|
-
| **A cost ceiling brakes runaway spend** | Dispatch halts when a per-wave budget is exceeded. See [`tools/cost_ceiling.py`](./tools/cost_ceiling.py). | A brake on a *configured* ceiling, **not yet tied to live token spend**. |
|
|
37
|
-
| **The package installs and reproduces** | A **~409 kB** npm tarball (measured via `npm pack`) builds and validates from a fresh clone in CI. See [docs/reproduce.md](./docs/reproduce.md). | UI browser-proofs (Playwright) and real-model runs need local setup / API keys; they don't run in the offline reproduce job. |
|
|
38
|
-
|
|
39
|
-
**Nobody outside this project has reproduced these results yet.** The evidence is committed so a skeptical reader can check it — that transparency is the point, not a substitute for independent replication. For the full, unhedged account of what is and isn't proven, read [docs/autonomous-swe.md](./docs/autonomous-swe.md) and the ["Honest limits" section of the release notes](./RELEASE-NOTES.md#honest-limits).
|
|
40
|
-
|
|
41
|
-
## Core Concepts and Operations
|
|
42
|
-
|
|
43
|
-
### What /power Does
|
|
44
|
-
|
|
45
|
-
`/power` initializes Aesop into a fresh repository. It:
|
|
46
|
-
- Loads your orchestrator brain (cardinal rules, domain map, team memory, system state)
|
|
47
|
-
- Verifies the system is healthy (filesystem, git, API keys, watchdog)
|
|
48
|
-
- Outputs a health brief and next-step recommendations
|
|
49
|
-
|
|
50
|
-
Run `/power` at the start of each session when using Claude Code. It's idempotent—safe to run multiple times. Setup once:
|
|
19
|
+
## Feature Demo
|
|
51
20
|
|
|
21
|
+
**One-turn wave** — Run a complete build cycle (tests, build, docs, review, merge, audit) end-to-end:
|
|
52
22
|
```bash
|
|
53
|
-
|
|
23
|
+
python driver/wave_loop.py --manifest wave.json --one-turn
|
|
54
24
|
```
|
|
25
|
+
Ranks backlog, dispatches parallel Haiku workers, runs tests, audits the output. Produces a JSON report of all agent runs and their verdicts.
|
|
55
26
|
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
`/buildsystem` runs **one complete wave cycle** of the autonomous delivery loop. It:
|
|
61
|
-
1. **Ranks** the backlog (priority, dependencies, team affinity)
|
|
62
|
-
2. **Dispatches** parallel Haiku agents on file-disjoint domains (tests, build, docs, UI, review, etc.)
|
|
63
|
-
3. **Verifies** each merge (CI, security scans, audit spot-checks)
|
|
64
|
-
4. **Checkpoints** (STATE.md + BUILDLOG.md committed to git, survive wipes)
|
|
65
|
-
5. **Audits** fleet health and feeds next backlog (monitor signals, signal collectors, findings)
|
|
66
|
-
|
|
67
|
-
This is **not** a one-off fix or a single pass—it's the repeatable loop that runs your delivery cycle indefinitely, with each wave learning from the prior audit. You can run `/buildsystem` once per wave (typically 30 min–2 hours per wave depending on backlog size and scope).
|
|
68
|
-
|
|
69
|
-
See [docs/HOW-THE-LOOP-WORKS.md](./docs/HOW-THE-LOOP-WORKS.md) for a concrete walkthrough of one complete cycle.
|
|
70
|
-
|
|
71
|
-
### Team State & Multi-Instance Vision
|
|
72
|
-
|
|
73
|
-
**Current Status (0.1.0)**: Single-instance proven. A team uses Aesop by designating one operator who runs the wave loop. State is durably checkpointed in git (STATE.md, BUILDLOG.md, tracker.json exports).
|
|
74
|
-
|
|
75
|
-
**In Design**: Multi-instance coordination via the state_store substrate. The event-sourced SQLite layer (`state_store/`) is production-ready but currently opt-in. A future release will enable multiple Aesop instances (e.g., per-team subgroups, geographic regions, or specialized fleets) to coordinate around a single source of truth—a Postgres-backed event log, with git as a diffable export. See [docs/TEAM-STATE.md](./docs/TEAM-STATE.md) (design in progress) for the vision and current architecture decisions.
|
|
76
|
-
|
|
77
|
-
## Why Aesop?
|
|
78
|
-
|
|
79
|
-
Multi-agent AI fleets hit two walls: all-Opus orchestrators cost $10k+ per wave, and machine crashes lose state—losing work and context on restart. Aesop solves both:
|
|
80
|
-
|
|
81
|
-
- **Haiku-first dispatch** cuts costs to ~1/3 of Opus (Haiku subagents, Opus orchestrator).
|
|
82
|
-
- **Durable state** (SQLite event log + git-rendered exports) survives machine wipes with zero data loss and enables team/multi-instance coordination.
|
|
83
|
-
- **Portable orchestration** works on any repository; no custom agents per repo needed.
|
|
84
|
-
|
|
85
|
-
```
|
|
86
|
-
ranked backlog
|
|
87
|
-
↓
|
|
88
|
-
parallel haiku fleet (worktree-isolated)
|
|
89
|
-
[test] [build] [docs] [ui] [review] ...
|
|
90
|
-
↓
|
|
91
|
-
integration-branch merge train
|
|
92
|
-
↓
|
|
93
|
-
checkpoint (STATE.md + BUILDLOG.md)
|
|
94
|
-
↓
|
|
95
|
-
audit + fleet-ops monitoring
|
|
96
|
-
↓
|
|
97
|
-
(feeds next wave's backlog)
|
|
27
|
+
**Multi-model drivers** — Choose your backend (Claude Code, Ollama, OpenRouter) with one config line:
|
|
28
|
+
```json
|
|
29
|
+
{ "backend": "openai-compatible", "model": "mistral-small", "base_url": "http://localhost:1234/v1" }
|
|
98
30
|
```
|
|
31
|
+
Verification tiers auto-adapt to backend capability—weaker models get stronger safety checking without code changes.
|
|
99
32
|
|
|
100
|
-
|
|
101
|
-
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
- **Durable state** — STATE.md + BUILDLOG.md checkpoints survive machine wipes; re-sync on resume, zero data loss.
|
|
106
|
-
- **Observable & auditable** — Every agent run logged, every cost tracked, every security event triaged.
|
|
107
|
-
- **Self-healing watchdog** — Runs every 150s: backs up work, scans for secrets, detects drift, restores on reboot.
|
|
108
|
-
- **Live web dashboard** — Real-time fleet health, security alerts, work-item kanban at `http://localhost:8770`.
|
|
109
|
-
- **Secret-scan gates** — Pre-push hook blocks leaks; audit trail logged. Pair with GitHub branch protection for enforcement.
|
|
110
|
-
- **Read-only MCP Fleet Server** — Expose fleet status, active agents, work items, and cost metrics to Claude Code (fleet_status, fleet_agents, fleet_tracker, fleet_cost tools). See [mcp/CLAUDE.md](./mcp/CLAUDE.md) for setup.
|
|
111
|
-
- **Multi-model portability** — AgentDriver abstraction decouples the wave loop from Claude Code; drivers for Claude Code, OpenAI-compatible backends (Ollama, OpenRouter), and Codex. Honest verification tiers: weaker backends get more checking. See [driver/README.md](./driver/README.md).
|
|
112
|
-
- **Self-diagnosing npm publish** — OIDC token generation and publish reliability verified on each release; workflow surfaces diagnostics inline.
|
|
113
|
-
- **Verification & quality tooling** — mutation_test.py for test-quality assessment, defect_escape.py for first-try-green telemetry, held-out benchmark for robustness, and adversarial review to break and harden the orchestration loop.
|
|
114
|
-
|
|
115
|
-
## Get Started (3 steps, 5 min)
|
|
116
|
-
|
|
117
|
-
**Note:** Aesop's first stable release is `0.1.0`, published to npm under the `latest` tag — a plain `npx @matt82198/aesop` or `npm install @matt82198/aesop` pulls it. Earlier prereleases remain available under the `@rc` and `@beta` tags.
|
|
33
|
+
**Wave templates** — Bootstrap a new fleet with a preset architecture:
|
|
34
|
+
```bash
|
|
35
|
+
python tools/wave_templates.py saas --project-name my-api --base-dir /workspace
|
|
36
|
+
```
|
|
37
|
+
Generates a manifest for typical 3-tier (API, frontend, ops) or data pipelines.
|
|
118
38
|
|
|
119
|
-
|
|
39
|
+
**Live dashboard** — Real-time view of fleet health, security alerts, work-item kanban, cost analytics:
|
|
40
|
+
```bash
|
|
41
|
+
npx @matt82198/aesop dash
|
|
42
|
+
```
|
|
43
|
+
Opens http://localhost:8770. Four views: Overview (agents, events), Work (kanban), Activity (reasoning tail), Cost (spend/tokens).
|
|
120
44
|
|
|
45
|
+
**Health score** — Readiness assessment: env, git, Python, Node, ports, config, hooks:
|
|
121
46
|
```bash
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
cd my-fleet
|
|
47
|
+
python tools/health_score.py
|
|
48
|
+
```
|
|
49
|
+
Outputs a scorecard of system readiness; --json for parsing.
|
|
126
50
|
|
|
127
|
-
|
|
128
|
-
bash
|
|
51
|
+
**Self-healing daemon** — Runs every 150s: backs up work, scans secrets, detects drift:
|
|
52
|
+
```bash
|
|
53
|
+
npx @matt82198/aesop watch
|
|
54
|
+
```
|
|
55
|
+
Pre-push hook blocks leaks. Heartbeat signals liveness; monitor auto-detects stalls and restarts the fleet.
|
|
129
56
|
|
|
130
|
-
|
|
131
|
-
|
|
57
|
+
**Hardened gate stack** — Fail-closed secret-scan, adversarial review (default-on):
|
|
58
|
+
```bash
|
|
59
|
+
python tools/secret_scan.py --staged # Blocks push if leak detected
|
|
132
60
|
```
|
|
61
|
+
Exits with failure on file-read errors (not silently passing). CI validates every merge.
|
|
133
62
|
|
|
134
|
-
|
|
63
|
+
## Proof Numbers
|
|
135
64
|
|
|
136
|
-
|
|
65
|
+
Aesop builds itself. These numbers are live from git, verified by anyone who clones.
|
|
137
66
|
|
|
138
|
-
|
|
67
|
+
## Get Started
|
|
139
68
|
|
|
140
69
|
```bash
|
|
141
|
-
|
|
142
|
-
cd ~/aesop
|
|
143
|
-
cp aesop.config.example.json aesop.config.json
|
|
144
|
-
# Edit paths and repos
|
|
145
|
-
|
|
146
|
-
export AESOP_ROOT=$HOME/aesop
|
|
147
|
-
bash $AESOP_ROOT/daemons/run-watchdog.sh --once
|
|
148
|
-
python ui/serve.py
|
|
70
|
+
npx @matt82198/aesop my-fleet --name "api" --repos "/path/to/repo"
|
|
149
71
|
```
|
|
72
|
+
→ Copy `skills/` into `~/.claude/skills` to enable the `/power` and `/buildsystem` commands.
|
|
73
|
+
→ See [docs/INSTALL.md](./docs/INSTALL.md) for setup and first `/power` → `/buildsystem` cycle.
|
|
74
|
+
→ See [docs/DEMO.md](./docs/DEMO.md) for a complete walkthrough of one wave.
|
|
150
75
|
|
|
151
|
-
## How It Works
|
|
152
|
-
|
|
153
|
-
```
|
|
154
|
-
daemons/run-watchdog.sh Every 150s: backs up work, scans secrets, detects drift
|
|
155
|
-
↓
|
|
156
|
-
orchestrator (via Claude Code) Reads backlog, dispatches Haiku subagents in parallel
|
|
157
|
-
↓
|
|
158
|
-
parallel Haiku fleet Tiny, scoped domains (tests, build, review, docs, etc.)
|
|
159
|
-
↓
|
|
160
|
-
watchdog backs up & gates Heartbeat, secret-scan, push to backup branch; merging is orchestrator/human-driven
|
|
161
|
-
↓
|
|
162
|
-
monitor/collect-signals.mjs Audits orchestration health, feeds next wave's backlog
|
|
163
|
-
↓
|
|
164
|
-
STATE.md + BUILDLOG.md Git-committed, survives machine wipes
|
|
165
|
-
```
|
|
166
|
-
|
|
167
|
-
See [docs/DISPATCH-MODEL.md](./docs/DISPATCH-MODEL.md) for cost analysis and parallel patterns.
|
|
168
76
|
|
|
169
77
|
<!-- STATS:START -->
|
|
170
78
|
|
|
@@ -191,137 +99,19 @@ Aesop is built entirely by its own `/buildsystem` wave cycle—running parallel
|
|
|
191
99
|
|
|
192
100
|
|
|
193
101
|
|
|
194
|
-
##
|
|
195
|
-
|
|
196
|
-
Aesop pairs well with the open-source catalog of ~130 community-authored specialized agent definitions (TDD orchestrators, security reviewers, performance engineers, etc.). For optimal results, install agents from the upstream source and pair them with Aesop's cost-optimized Haiku dispatch. This is optional; Aesop works standalone with general-purpose Claude Code agents.
|
|
197
|
-
|
|
198
|
-
## Use with Claude Code
|
|
199
|
-
|
|
200
|
-
If you're using **Claude Code**, invoke `/power` at the start of each session. It loads your orchestrator brain (cardinal rules, domain map, team memory, system state) and outputs a health brief. Setup once:
|
|
201
|
-
|
|
202
|
-
```bash
|
|
203
|
-
# Copy the /power skill and optional /healthcheck skill
|
|
204
|
-
cp -r skills/power/ ~/.claude/skills/power/
|
|
205
|
-
cp -r skills/healthcheck/ ~/.claude/skills/healthcheck/
|
|
206
|
-
```
|
|
207
|
-
|
|
208
|
-
Then in Claude Code, type `/power` or `/buildsystem` to start a wave cycle. Use `/healthcheck` to audit fleet machinery health before running waves. See [skills/power/SKILL.md](./skills/power/SKILL.md) and [skills/healthcheck/SKILL.md](./skills/healthcheck/SKILL.md) for details.
|
|
209
|
-
|
|
210
|
-
## Core Principles
|
|
211
|
-
|
|
212
|
-
1. **Haiku-first dispatch** — Subagents always cheap; orchestrator stays lean on main thread.
|
|
213
|
-
2. **Durable state** — STATE.md + BUILDLOG.md survive wipes; re-sync on resume.
|
|
214
|
-
3. **Observable** — Every agent run logged, every cost tracked, every security event triaged.
|
|
215
|
-
4. **TDD-first** — Fail tests before implementation; one Haiku per scoped domain.
|
|
216
|
-
5. **Never wait** — Dispatch work in parallel; connect with heartbeats, not polling.
|
|
217
|
-
6. **Push discipline** — feature/* branches only; secret-scan gates every push.
|
|
218
|
-
|
|
219
|
-
Read [docs/CARDINAL-RULES.md](./docs/CARDINAL-RULES.md) for the full text.
|
|
220
|
-
|
|
221
|
-
## Requirements
|
|
222
|
-
|
|
223
|
-
- Claude Code CLI (v0.1+)
|
|
224
|
-
- Git (v2.40+)
|
|
225
|
-
- Bash (v4+) or Git Bash on Windows
|
|
226
|
-
- Node.js (v18+) for dashboard and monitor
|
|
227
|
-
- Python (v3.10+) for log rotation and secret-scan
|
|
228
|
-
- jq (optional) for TUI dashboard
|
|
229
|
-
|
|
230
|
-
## Scaling Cheaply
|
|
231
|
-
|
|
232
|
-
The **dispatch model** fans work across parallel Haiku subagents (each 1/3 the cost of Opus). The orchestrator stays lean on the main thread, coordinating via durable STATE.md. Result: ~25% the cost of an all-Opus fleet.
|
|
233
|
-
|
|
234
|
-
**Action tiers**: AUTO (immediate, logged) for read-only checks and appends; PROPOSE (staged in `monitor/PROPOSALS.md`) for changes requiring approval. See [docs/GOVERNANCE.md](./docs/GOVERNANCE.md).
|
|
235
|
-
|
|
236
|
-
## Security
|
|
237
|
-
|
|
238
|
-
The pre-push hook (`hooks/pre-push-policy.sh`) enforces branch discipline and secret scanning locally. It is bypassable (use `--no-verify` to skip), so **pair it with GitHub branch protection** for real enforcement:
|
|
239
|
-
|
|
240
|
-
```
|
|
241
|
-
Settings > Branches > main
|
|
242
|
-
✓ Require pull request reviews
|
|
243
|
-
✓ Require status checks to pass
|
|
244
|
-
✓ Dismiss stale PR approvals
|
|
245
|
-
✓ Restrict pushes to (Admins only)
|
|
246
|
-
```
|
|
247
|
-
|
|
248
|
-
**Host-header validation** in the dashboard UI handler prevents HTTP header injection attacks; all requests are validated against the configured origin. Private brain (`~/.claude`) is never committed to this repo. Keep `aesop.config.json` git-ignored. Implement `tools/secret_scan.py` with your security rules. See [docs/HOOK-INSTALL.md](./docs/HOOK-INSTALL.md) for setup.
|
|
249
|
-
|
|
250
|
-
## Dashboard (Wave-14 Rewrite)
|
|
251
|
-
|
|
252
|
-
The dashboard is a **React 18 + Vite + TypeScript** single-page app with four hash-routed views:
|
|
253
|
-
|
|
254
|
-
### Viewing the Dashboard
|
|
255
|
-
```bash
|
|
256
|
-
python ui/serve.py
|
|
257
|
-
```
|
|
258
|
-
Opens `http://localhost:8770` — live fleet health, security alerts, work-item kanban, cost analytics.
|
|
259
|
-
|
|
260
|
-
### Architecture
|
|
261
|
-
- **Backend**: Python stdlib HTTP server (`ui/handler.py`) serves the built React app + JSON/SSE APIs (`/api/state`, `/api/cost`, `/events`).
|
|
262
|
-
- **Frontend**: `ui/web/` (React app) is built to `dist/` (committed to git) and served as static files by the Python server.
|
|
263
|
-
- **CSRF protection**: Token injected into `dist/index.html` via sentinel substitution; mutations gated by `/submit` and `/api/tracker` endpoints.
|
|
264
|
-
|
|
265
|
-
### Development
|
|
266
|
-
```bash
|
|
267
|
-
cd ui/web
|
|
268
|
-
npm install
|
|
269
|
-
npm run dev # Vite dev server with API proxy to http://localhost:8770
|
|
270
|
-
npm run build # Build to dist/ (commit the dist/)
|
|
271
|
-
```
|
|
272
|
-
|
|
273
|
-
The dev server proxies `/data`, `/api`, `/events`, `/agent`, `/submit` to the Python backend on :8770, so the frontend can develop against live APIs.
|
|
274
|
-
|
|
275
|
-
### Views
|
|
276
|
-
- **Overview**: Fleet agents, security alerts, recent events.
|
|
277
|
-
- **Work** (`#/work`): Tracker kanban (4 lanes: proposed/ranked/in-progress/done), audit backlog progress.
|
|
278
|
-
- **Activity** (`#/activity`): Agent timeline, main-thread message tail (live reasoning).
|
|
279
|
-
- **Cost** (`#/cost`): Per-model spend/tokens, per-day bar chart, verdict scorecard (success/failure rates).
|
|
280
|
-
|
|
281
|
-
## Extending Aesop
|
|
282
|
-
|
|
283
|
-
**Custom signal collectors**: Edit `monitor/collect-signals.mjs` to add domain-specific health checks.
|
|
284
|
-
|
|
285
|
-
**Custom watchdog hooks**: Edit `daemons/backup-fleet.sh` to run linters, integrate with your CI, or customize secret-scan logic.
|
|
286
|
-
|
|
287
|
-
**Dashboard components**: Add React components in `ui/web/src/components/` or new views in `ui/web/src/views/`. Rebuild and commit `dist/`.
|
|
288
|
-
|
|
289
|
-
## Troubleshooting
|
|
290
|
-
|
|
291
|
-
| Issue | Check |
|
|
292
|
-
|-------|-------|
|
|
293
|
-
| Watchdog doesn't start | `state/FLEET-BACKUP.log` for errors; verify `AESOP_ROOT` is set |
|
|
294
|
-
| Dashboard shows "unavailable" | Install Node.js v18+; check `dash-extra.mjs` is in sync |
|
|
295
|
-
| Secret-scan blocks push | Add suppression to `tools/secret_scan.py`; no auto-bypass (by design) |
|
|
296
|
-
| Monitor doesn't start | Verify Node.js on PATH; check `monitor/BRIEF.md` for logs |
|
|
297
|
-
|
|
298
|
-
## Documentation
|
|
299
|
-
|
|
300
|
-
**Adopter journey** (start here):
|
|
301
|
-
- [docs/INSTALL.md](./docs/INSTALL.md) — Install Aesop and verify setup
|
|
302
|
-
- [docs/CONFIGURE.md](./docs/CONFIGURE.md) — Configure repos, ports, and brain root
|
|
303
|
-
- [docs/FIRST-WAVE.md](./docs/FIRST-WAVE.md) — Run your first `/power` → `/buildsystem` cycle
|
|
304
|
-
- [docs/CONCEPTS.md](./docs/CONCEPTS.md) — Key concepts (dispatch, state, security, governance) with links to deep dives
|
|
305
|
-
- [docs/ARCHITECTURE.md](./docs/ARCHITECTURE.md) — System architecture diagram and components
|
|
102
|
+
## Why Haiku-First Works
|
|
306
103
|
|
|
307
|
-
**
|
|
308
|
-
- [docs/HOW-THE-LOOP-WORKS.md](./docs/HOW-THE-LOOP-WORKS.md) — Concrete walkthrough of one wave cycle
|
|
309
|
-
- [docs/DISPATCH-MODEL.md](./docs/DISPATCH-MODEL.md) — Cost analysis, dispatch patterns, scaling
|
|
310
|
-
- [docs/CHECKPOINTING.md](./docs/CHECKPOINTING.md) — STATE.md + BUILDLOG.md lifecycle, recovery on wipe
|
|
311
|
-
- [docs/CARDINAL-RULES.md](./docs/CARDINAL-RULES.md) — 10 foundational principles
|
|
312
|
-
- [docs/GOVERNANCE.md](./docs/GOVERNANCE.md) — Single-writer files, heartbeat protocol, AUTO/PROPOSE tiers
|
|
313
|
-
- [docs/RELIABILITY.md](./docs/RELIABILITY.md) — Reliability guarantees, pride bar, inputs-always-outputs
|
|
314
|
-
- [docs/HOOK-INSTALL.md](./docs/HOOK-INSTALL.md) — Secret-scan and branch protection setup
|
|
104
|
+
The benchmark proves it: across 39 judgment tasks (code review, severity calibration, root-cause analysis, refactor equivalence, security spots), Haiku scored **39/39** vs Opus **38/39** at ~1/3 the per-token cost. See [`bench/results/2026-07-17-judgment-v3-haiku-sonnet-opus.md`](./bench/results/2026-07-17-judgment-v3-haiku-sonnet-opus.md). Honest caveat: curated set (N=39), not real-transcript sampled; the benchmark found no task where Opus beats Haiku, proving sufficiency for this workload, not parity at the reasoning frontier.
|
|
315
105
|
|
|
316
|
-
|
|
317
|
-
- [docs/FORENSICS.md](./docs/FORENSICS.md) — Debug agent failures (git-bisectable)
|
|
318
|
-
- [docs/RESTORE.md](./docs/RESTORE.md) — Reconstitute Aesop on a new machine
|
|
319
|
-
- [docs/PUBLISHING.md](./docs/PUBLISHING.md) — Release Aesop to npm
|
|
320
|
-
- [docs/autonomous-swe.md](./docs/autonomous-swe.md) — The 0.1.0-rc.1 milestone told honestly: what "autonomous SWE" means here, the evidence behind each claim, and the limits the project owns
|
|
321
|
-
- [docs/case-study-portfolio.md](./docs/case-study-portfolio.md) — How Aesop built its own portfolio site; full audit trail and cost breakdown
|
|
322
|
-
- [docs/TEAM-STATE.md](./docs/TEAM-STATE.md) — Multi-instance and team coordination via state_store (design in progress)
|
|
106
|
+
## Learn More
|
|
323
107
|
|
|
324
|
-
|
|
108
|
+
- **[docs/INSTALL.md](./docs/INSTALL.md)** — Setup and first wave
|
|
109
|
+
- **[docs/PORTING.md](./docs/PORTING.md)** — Adopter's guide: port Aesop to your repo (prerequisites, scaffold, 10 failure modes)
|
|
110
|
+
- **[docs/HOW-THE-LOOP-WORKS.md](./docs/HOW-THE-LOOP-WORKS.md)** — Concrete walkthrough of a wave cycle
|
|
111
|
+
- **[docs/DISPATCH-MODEL.md](./docs/DISPATCH-MODEL.md)** — Cost analysis and scaling
|
|
112
|
+
- **[docs/CARDINAL-RULES.md](./docs/CARDINAL-RULES.md)** — 10 foundational principles
|
|
113
|
+
- **[docs/autonomous-swe.md](./docs/autonomous-swe.md)** — What "autonomous" means (and doesn't), evidence for all claims, honest limits
|
|
114
|
+
- **[RELEASE-NOTES.md](./RELEASE-NOTES.md)** — Version 0.3.0: non-Claude core proof (Codex wave), refinement-loop convergence, Windows-green CI
|
|
325
115
|
|
|
326
116
|
## Contributing
|
|
327
117
|
|
package/bin/cli.js
CHANGED
|
@@ -11,8 +11,8 @@ const helpFlag = args.includes('--help') || args.includes('-h');
|
|
|
11
11
|
const forceFlag = args.includes('--force');
|
|
12
12
|
const yesFlag = args.includes('--yes');
|
|
13
13
|
|
|
14
|
-
// Check for runtime subcommands (doctor, watch, dash, status, fleet, health-score)
|
|
15
|
-
const runtimeCommands = ['doctor', 'watch', 'dash', 'status', 'fleet', 'health-score'];
|
|
14
|
+
// Check for runtime subcommands (doctor, watch, dash, status, fleet, health-score, reproduce)
|
|
15
|
+
const runtimeCommands = ['doctor', 'watch', 'dash', 'status', 'fleet', 'health-score', 'reproduce'];
|
|
16
16
|
const isRuntimeCommand = runtimeCommands.includes(args[0]);
|
|
17
17
|
|
|
18
18
|
if (isRuntimeCommand) {
|
|
@@ -22,7 +22,8 @@ if (isRuntimeCommand) {
|
|
|
22
22
|
'dash': '../tools/dash.js',
|
|
23
23
|
'status': '../tools/status.js',
|
|
24
24
|
'fleet': '../tools/fleet.js',
|
|
25
|
-
'health-score': '../tools/health-score.js'
|
|
25
|
+
'health-score': '../tools/health-score.js',
|
|
26
|
+
'reproduce': '../tools/reproduce.js'
|
|
26
27
|
};
|
|
27
28
|
// Load and run the appropriate runtime module
|
|
28
29
|
// These modules run async code that sets process.exitCode and will cause Node to exit
|
|
@@ -154,6 +155,7 @@ Usage:
|
|
|
154
155
|
npx @matt82198/aesop dash
|
|
155
156
|
npx @matt82198/aesop status
|
|
156
157
|
npx @matt82198/aesop fleet
|
|
158
|
+
npx @matt82198/aesop reproduce
|
|
157
159
|
|
|
158
160
|
Commands:
|
|
159
161
|
doctor Preflight readiness check (Node.js, Python, git, config, dirs, hook, port)
|
|
@@ -161,6 +163,7 @@ Commands:
|
|
|
161
163
|
dash Launch the web dashboard (spawns python3 ui/serve.py or python fallback)
|
|
162
164
|
status One-shot fleet status snapshot (heartbeats, dashboard port, git branch)
|
|
163
165
|
fleet One-shot fleet snapshot (agents, heartbeats, tracker lanes, orchestrator status)
|
|
166
|
+
reproduce Offline verification suite (repo: full test suite; installed: self-checks)
|
|
164
167
|
wizard Interactive onboarding (prompts for project name, repos, port)
|
|
165
168
|
|
|
166
169
|
Arguments:
|
|
@@ -181,6 +184,7 @@ Examples:
|
|
|
181
184
|
npx @matt82198/aesop dash # Launch web dashboard (default localhost:8770)
|
|
182
185
|
npx @matt82198/aesop status # Show fleet status (heartbeats, port, git)
|
|
183
186
|
npx @matt82198/aesop fleet # Show fleet snapshot (JSON agents, heartbeats, tracker, orchestrator)
|
|
187
|
+
npx @matt82198/aesop reproduce # Run offline verification suite (full repo tests or installed checks)
|
|
184
188
|
npx @matt82198/aesop # Creates ./aesop-fleet/ with template
|
|
185
189
|
npx @matt82198/aesop my-fleet # Creates ./my-fleet/ with template
|
|
186
190
|
npx @matt82198/aesop wizard # Interactive onboarding (60-second setup)
|
package/docs/HOOK-INSTALL.md
CHANGED
|
@@ -1,82 +1,41 @@
|
|
|
1
|
-
# Git Pre-Push Hook
|
|
1
|
+
# Git Pre-Push Hook Setup
|
|
2
2
|
|
|
3
|
-
**
|
|
4
|
-
|
|
5
|
-
## Security Model: Local Convenience Defense Only
|
|
6
|
-
|
|
7
|
-
**IMPORTANT**: The pre-push hook is a **local-machine convenience defense** — it is **NOT cryptographic protection**. Any developer can bypass it with `git push --no-verify` or by editing/deleting `.git/hooks/pre-push`. The audit log (`SECURITY-AUDIT.log`) is stored locally and can also be edited by a user with file system access.
|
|
8
|
-
|
|
9
|
-
**Real enforcement requires server-side branch protection.** See GitHub Configuration below.
|
|
3
|
+
**TL;DR**: The hook is **auto-installed during scaffold**. Real enforcement requires server-side GitHub branch protection.
|
|
10
4
|
|
|
11
5
|
## What the Hook Does
|
|
12
6
|
|
|
13
|
-
`hooks/pre-push-policy.sh` enforces
|
|
7
|
+
`hooks/pre-push-policy.sh` enforces:
|
|
8
|
+
1. **Branch Policy** — No direct pushes to `main`/`master` (feature branches only)
|
|
9
|
+
2. **Secret Scan** — `tools/secret_scan.py --staged` blocks credentials
|
|
14
10
|
|
|
15
|
-
|
|
16
|
-
2. **Secret Scan**: Runs `tools/secret_scan.py --staged` to detect credentials before they reach the remote.
|
|
11
|
+
## Security Model
|
|
17
12
|
|
|
18
|
-
|
|
13
|
+
**IMPORTANT**: Local hook is a **convenience defense only**—NOT cryptographic protection. Bypass with `git push --no-verify`.
|
|
19
14
|
|
|
20
|
-
|
|
15
|
+
**Real enforcement**: Pair with GitHub branch protection (server-side). See below.
|
|
21
16
|
|
|
22
|
-
|
|
17
|
+
## Auto-Installation
|
|
23
18
|
|
|
24
|
-
-
|
|
25
|
-
- **On Windows**: Copies the hook directly (symlinks don't work reliably on all NTFS setups)
|
|
26
|
-
- **Idempotent**: Re-running scaffold doesn't clobber a user-customized hook
|
|
27
|
-
- **Preserve Existing**: If you have a different pre-push hook, scaffold warns and preserves it
|
|
28
|
-
- **Force Replace**: Use `npx @matt82198/aesop [target-dir] --force` to replace any existing hook
|
|
29
|
-
|
|
30
|
-
**Example:**
|
|
19
|
+
Scaffold auto-installs the hook (symlink on Unix, copy on Windows):
|
|
31
20
|
|
|
32
21
|
```bash
|
|
33
|
-
# Initial scaffold (creates and installs hook automatically)
|
|
34
|
-
npx @matt82198/aesop my-fleet
|
|
35
|
-
|
|
36
|
-
# Later: re-scaffold the same directory (preserves customizations)
|
|
37
22
|
npx @matt82198/aesop my-fleet
|
|
38
|
-
|
|
39
|
-
# Force replace (even if hook was customized)
|
|
40
|
-
npx @matt82198/aesop my-fleet --force
|
|
41
23
|
```
|
|
42
24
|
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
If you're installing into an existing repo (not scaffolded), or you need to manually add the hook:
|
|
46
|
-
|
|
47
|
-
### Option 1: Symlink (Linux / macOS / Git Bash on Windows)
|
|
48
|
-
|
|
49
|
-
The cleanest method — hook stays in sync with repo updates:
|
|
50
|
-
|
|
25
|
+
Manual install:
|
|
51
26
|
```bash
|
|
27
|
+
# Option 1: Symlink (Unix/macOS/Git Bash)
|
|
52
28
|
ln -s ../../hooks/pre-push-policy.sh .git/hooks/pre-push
|
|
53
|
-
chmod +x .git/hooks/pre-push
|
|
54
|
-
```
|
|
55
|
-
|
|
56
|
-
### Option 2: Copy (Windows, or to break sync)
|
|
57
|
-
|
|
58
|
-
Copy the hook directly into `.git/hooks/`:
|
|
59
|
-
|
|
60
|
-
```powershell
|
|
61
|
-
Copy-Item hooks\pre-push-policy.sh .git\hooks\pre-push
|
|
62
|
-
```
|
|
63
29
|
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
```bash
|
|
67
|
-
git config core.filemode false
|
|
30
|
+
# Option 2: Copy (Windows)
|
|
31
|
+
cp hooks/pre-push-policy.sh .git/hooks/pre-push
|
|
68
32
|
```
|
|
69
33
|
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
Before committing to org-wide deployment, verify the hook works:
|
|
73
|
-
|
|
34
|
+
Test:
|
|
74
35
|
```bash
|
|
75
36
|
bash hooks/pre-push-policy.sh --test
|
|
76
37
|
```
|
|
77
38
|
|
|
78
|
-
Expected output: **PASS** for all three checks (branch policy, feature branch allowance, audit log format).
|
|
79
|
-
|
|
80
39
|
## GitHub Configuration (Server-Side Enforcement)
|
|
81
40
|
|
|
82
41
|
To pair this local hook with real enforcement, enable branch protection on GitHub:
|
package/docs/INSTALL.md
CHANGED
|
@@ -285,9 +285,10 @@ To bypass during testing: `git push --no-verify` (not recommended for production
|
|
|
285
285
|
|
|
286
286
|
## Next Steps
|
|
287
287
|
|
|
288
|
-
1. **Read [
|
|
289
|
-
2. **
|
|
290
|
-
3. **
|
|
288
|
+
1. **Read [PORTING.md](PORTING.md)** — Step-by-step guide for adopting Aesop on a foreign repo (10 common failure modes)
|
|
289
|
+
2. **Read [CONFIGURE.md](CONFIGURE.md)** — Customize repos, ports, and brain root
|
|
290
|
+
3. **Run [FIRST-WAVE.md](FIRST-WAVE.md)** — Test a full `/power` → `/buildsystem` cycle
|
|
291
|
+
4. **Understand [CONCEPTS.md](CONCEPTS.md)** — Learn the dispatch model and state model
|
|
291
292
|
4. **Explore the dashboard** — `python3 ui/serve.py` then open http://localhost:8770
|
|
292
293
|
|
|
293
294
|
For troubleshooting, see the [Aesop README](../README.md#troubleshooting) or [GOVERNANCE.md](GOVERNANCE.md) for operational policies.
|