@matt82198/aesop 0.2.0 → 0.3.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. package/CHANGELOG.md +28 -0
  2. package/README.md +50 -260
  3. package/bin/cli.js +7 -3
  4. package/docs/HOOK-INSTALL.md +15 -56
  5. package/docs/INSTALL.md +4 -3
  6. package/docs/PORTING.md +166 -0
  7. package/docs/TEAM-STATE.md +16 -184
  8. package/docs/reproduce.md +33 -3
  9. package/driver/CLAUDE.md +50 -48
  10. package/driver/codex_driver.py +59 -12
  11. package/driver/openai_transport.py +33 -0
  12. package/driver/wave_bridge.py +9 -1
  13. package/driver/wave_loop.py +437 -70
  14. package/driver/wave_scheduler.py +890 -0
  15. package/hooks/pre-push-policy.sh +22 -5
  16. package/monitor/collect-signals.mjs +69 -0
  17. package/package.json +4 -4
  18. package/state_store/__init__.py +2 -1
  19. package/state_store/projections.py +63 -0
  20. package/state_store/read_api.py +156 -0
  21. package/state_store/write_api.py +462 -0
  22. package/tools/cost_ceiling.py +106 -15
  23. package/tools/cost_projection.py +559 -0
  24. package/tools/crossos_drift.py +394 -0
  25. package/tools/eod_sweep.py +137 -33
  26. package/tools/mutation_test.py +130 -8
  27. package/tools/proposals.mjs +47 -2
  28. package/tools/reproduce.js +405 -0
  29. package/tools/secret_scan.py +73 -18
  30. package/tools/stall_check.py +247 -16
  31. package/tools/stateapi_lint.py +325 -0
  32. package/tools/test_battery.py +173 -0
  33. package/tools/verify_cost_panel.py +345 -0
  34. package/tools/wave_preflight.py +296 -29
  35. package/tools/wave_templates.py +170 -45
  36. package/ui/collectors.py +81 -0
  37. package/ui/handler.py +230 -85
  38. package/ui/sse.py +3 -3
  39. package/ui/wave_telemetry.py +24 -18
  40. package/ui/web/dist/assets/{index-DqZLgwNg.css → index-CNQxaiOW.css} +1 -1
  41. package/ui/web/dist/assets/index-CP68RIh3.js +9 -0
  42. package/ui/web/dist/index.html +2 -2
  43. package/docs/QUICKSTART.md +0 -80
  44. package/ui/web/dist/assets/index-BGbgw2Nh.js +0 -9
@@ -0,0 +1,166 @@
1
+ # Porting Aesop to Your Repository
2
+
3
+ Guide for adopters porting the orchestration harness to a foreign repo. Step-by-step with prerequisites, scaffold, config, and the 10 likeliest failure modes from real deployments.
4
+
5
+ ---
6
+
7
+ ## Prerequisites
8
+
9
+ Ensure you have:
10
+ - **Node.js** ≥18 (from `package.json`, `.nvmrc`: v18.20.5)
11
+ - **Python** ≥3.10 (for daemons, health checks, secret-scan)
12
+ - **Git** ≥2.40 (worktree + pre-push hooks)
13
+ - **Bash** v4+ (or Git Bash on Windows)
14
+ - **Optional**: Playwright (for UI verification testing); jq (for dashboard JSON)
15
+
16
+ Check versions:
17
+ ```bash
18
+ node --version && python3 --version && git --version && bash --version
19
+ ```
20
+
21
+ ### OS Notes
22
+ - **Windows**: Use Git Bash for all shell commands; paths use `/c/Users/...` POSIX style
23
+ - **Linux/macOS**: Standard Bash; ensure `/usr/bin/bash` or equivalent exists
24
+
25
+ ---
26
+
27
+ ## 1. Scaffold & Install
28
+
29
+ ### Step 1: Create your harness
30
+ ```bash
31
+ npx @matt82198/aesop my-fleet \
32
+ --name "my-project" \
33
+ --repos "/path/to/repo1,/path/to/repo2"
34
+ ```
35
+
36
+ Creates `my-fleet/` with daemons, skills, config, and UI.
37
+
38
+ ### Step 2: Copy skills to Claude Code home
39
+ ```bash
40
+ cp -r my-fleet/skills/power ~/.claude/skills/power
41
+ cp -r my-fleet/skills/buildsystem ~/.claude/skills/buildsystem
42
+ ```
43
+
44
+ ### Step 3: Configure & test
45
+ Edit `my-fleet/aesop.config.json`:
46
+ ```json
47
+ {
48
+ "backend": "claude",
49
+ "aesop_root": "/path/to/my-fleet",
50
+ "brain_root": "~/.claude",
51
+ "repos": [
52
+ { "path": "/path/to/repo1", "name": "my-project" }
53
+ ],
54
+ "dashboardPort": 8770
55
+ }
56
+ ```
57
+
58
+ Test dashboard backend (Ollama example):
59
+ ```json
60
+ {
61
+ "backend": "openai-compatible",
62
+ "base_url": "http://localhost:11434/v1",
63
+ "model": "mistral",
64
+ "is_local": true
65
+ }
66
+ ```
67
+
68
+ ### Step 4: Install pre-push hook
69
+ ```bash
70
+ mkdir -p my-fleet/.git/hooks
71
+ cp my-fleet/hooks/pre-push-policy.sh my-fleet/.git/hooks/pre-push
72
+ chmod +x my-fleet/.git/hooks/pre-push
73
+ ```
74
+
75
+ ### Step 5: Verify with health check
76
+ ```bash
77
+ cd my-fleet
78
+ python tools/health_score.py
79
+ ```
80
+ Expected output: All checks ✓; port 8770 available; git hooks installed.
81
+
82
+ ---
83
+
84
+ ## 2. First `/power`, First Wave
85
+
86
+ Run the orchestrator once to prime the brain:
87
+ ```bash
88
+ cd my-fleet
89
+ /power
90
+ ```
91
+ Expected output: State created (STATE.md, BUILDLOG.md, .watchdog-heartbeat).
92
+
93
+ Run one complete wave:
94
+ ```bash
95
+ /buildsystem --one-turn
96
+ ```
97
+ Expected verdicts: All items "PASS" or "REVIEW"; zero code defects. Health score should report ≥85/100.
98
+
99
+ ---
100
+
101
+ ## 3. The 10 Likeliest Failure Modes
102
+
103
+ Each: symptom → cause → fix.
104
+
105
+ ### 1. Secret-scan blocks legit push
106
+ - **Symptom**: `git push` fails: "secret detected" but code has no real secret
107
+ - **Cause**: Test fixtures with dummy secrets (credit card, API key patterns) are parsed as real
108
+ - **Fix**: Assemble secrets at runtime: `"key" + "_" + "123"` not `"key_123"`; use `---` delimiters in comments. Secret-scan only scans staged files, not .git.
109
+
110
+ ### 2. Worktree isolation violated
111
+ - **Symptom**: Editing feature branch X; changes also appear in primary tree
112
+ - **Cause**: Agents not using isolated worktrees; sharing same `.git/index`
113
+ - **Fix**: Each agent task must use `git worktree add ../wt-item-slug -b feature/item-slug origin/main`; verify with `git worktree list`
114
+
115
+ ### 3. Heartbeat stale/missing
116
+ - **Symptom**: Watchdog won't start; error: "unreadable state dir" or "stale heartbeat"
117
+ - **Cause**: `state/.watchdog-heartbeat` missing/corrupted or directory unreadable (permissions)
118
+ - **Fix**: `rm -f state/.watchdog-heartbeat && bash daemons/run-watchdog.sh --once` (re-creates it)
119
+
120
+ ### 4. Port 8770 conflict
121
+ - **Symptom**: `python ui/serve.py` fails: "Address already in use"
122
+ - **Cause**: Old dashboard process still bound, or another service using port 8770
123
+ - **Fix**: `lsof -i :8770 | grep -v PID | awk '{print $2}' | xargs kill -9` (Mac/Linux); Windows: `netstat -ano | grep 8770`, kill the PID. Or change `dashboardPort` in config.
124
+
125
+ ### 5. Git identity placeholder
126
+ - **Symptom**: Pre-push hook fails: "user identity invalid"; commits fail
127
+ - **Cause**: Git config has template defaults (e.g., "John Doe") instead of your name
128
+ - **Fix**: `git config user.name "Your Name" && git config user.email "you@example.com"`
129
+
130
+ ### 6. CRLF line endings corrupt scripts
131
+ - **Symptom**: Bash script fails: "command not found" at line 50 (but line 50 exists)
132
+ - **Cause**: Editor/Windows converted LF to CRLF; bash reads `\r` as part of command
133
+ - **Fix**: `git config core.autocrlf false` locally; convert file: `dos2unix daemons/run-watchdog.sh`
134
+
135
+ ### 7. Test count drift in CI
136
+ - **Symptom**: Health-check fails: "expected 127 tests, got 128"
137
+ - **Cause**: Added new test file but didn't update baseline in tools/health_score.py or CI config
138
+ - **Fix**: Run `python tools/health_score.py --json` to get true count; update baseline in CI gate
139
+
140
+ ### 8. UTF-8 encoding on Windows
141
+ - **Symptom**: Pre-publish gate fails with encoding error in secret_scan.py
142
+ - **Cause**: Python opens files in system encoding (cp1252) instead of UTF-8
143
+ - **Fix**: Aesop tools now force UTF-8 internally; ensure `PYTHONIOENCODING=utf-8` if running external Python scripts
144
+
145
+ ### 9. Cost ceiling never triggers
146
+ - **Symptom**: Cost ceilings configured; spending uncapped; halt never fires
147
+ - **Cause**: Claude Code driver returns `get_tokens_spent()=None`; coerces to 0; ledger fallback skipped
148
+ - **Fix**: For Claude Code, cost tracking integrates live API; for non-Claude backends, ledger fallback logs at end of wave
149
+
150
+ ### 10. Hook TTY behavior blocks CI pushes
151
+ - **Symptom**: `git push` fails in CI/cron: "cannot read stdin"; interactive prompt hangs
152
+ - **Cause**: Hook tries to detect TTY when stdin is `/dev/null` (non-interactive context)
153
+ - **Fix**: Hook now handles empty stdin (rc=2 for delete-only, skip scan + log); use `hooks/pre-push-policy.sh --test` in CI (no-op validation)
154
+
155
+ ---
156
+
157
+ ## Next Steps
158
+
159
+ 1. **Read [INSTALL.md](./INSTALL.md)** — Full setup and environment variables
160
+ 2. **Run [health_score.py](../tools/health_score.py)** — Continuous readiness checks
161
+ 3. **Explore the [README](../README.md)** — Demo walkthrough and proof numbers
162
+ 4. **For troubleshooting**, check [GOVERNANCE.md](./GOVERNANCE.md) or open an issue on GitHub
163
+
164
+ ---
165
+
166
+ **License**: PolyForm Strict 1.0.0 (source-available, noncommercial). See [LICENSE](../LICENSE) for details.
@@ -1,203 +1,35 @@
1
- # Team-Shared State: Architecture & Migration
1
+ # Team-Shared State: Current & Future Architecture
2
2
 
3
- **Status**: Design document. Prototype complete (`state_store/` SQLite module shipped). Migration path defined. NOT YET INTEGRATED into orchestrator reader/writer paths.
3
+ **Status (0.1.0)**: Single-instance with durable git checkpointing. Event-sourced SQLite module (`state_store/`) is production-ready but not yet integrated into orchestrator reader/writer paths.
4
4
 
5
5
  ---
6
6
 
7
- ## Problem Statement
7
+ ## Current State Model (Git-as-Checkpoint)
8
8
 
9
9
  Aesop currently stores orchestration state in **two git-tracked files**:
10
- - `STATE.md` — intent, decisions, phase, NEXT STEPS (single-writer orchestrator)
11
- - `BUILDLOG.md` — append-only progress snapshots (agents append one line per work unit)
12
10
 
13
- This works **for single-user/single-orchestrator** workflows but does not scale to teams:
11
+ - **STATE.md** Orchestrator intent, decisions, phase, and NEXT STEPS (single-writer)
12
+ - **BUILDLOG.md** — Append-only progress snapshots (agents append one line per work unit)
14
13
 
15
- 1. **Hot-file merge conflicts** Multiple agents appending to the same files requires rebasing and re-coordination; a team of 3+ users editing these simultaneously causes conflicts.
16
- 2. **No transactions/concurrency** — Git has no concept of atomic multi-file updates or per-agent locks; coordination is manual and error-prone.
17
- 3. **No real-time status** — Readers must `git fetch` to see the latest state; no subscription/push model for live updates.
18
- 4. **Single-writer bottleneck** — The orchestrator is the only writer to STATE.md; other workers cannot safely report their own state.
14
+ **Why it works**: Single-instance is proven. State survives machine wipes, resync on session restart, human-diffable for review and forensics.
19
15
 
20
- ---
21
-
22
- ## Current Truth: Git-as-State (Single-Instance)
23
-
24
- ### Data model
25
-
26
- State lives as **human-readable JSON** in two git-tracked files:
27
-
28
- ```
29
- orchestrator (Fable/Opus) ←→ STATE.md (single-writer, locked decisions + phase + NEXT STEPS)
30
- ←→ BUILDLOG.md (append-only, agents append one line per work unit)
31
-
32
- ↓ (read-only consumers)
33
-
34
- dashboards, monitoring, forensics
35
- ```
36
-
37
- **Invariants**:
38
- - Only the orchestrator writes to STATE.md (enforced by discipline, not by code)
39
- - BUILDLOG.md is append-only; orchestrator never rewrites earlier entries
40
- - All state is human-diffable (intentional — enables review, forensics, inheritance)
41
- - Durability via git: push after each update, recover from `git log` on session restart
42
- - Git is the **source of truth** for current state
43
-
44
- ### Limitations for team scale
45
-
46
- - **No concurrent writers** — Two orchestrators or agents trying to write simultaneously causes merge conflicts
47
- - **No transactions** — A phase update + next-steps change is two separate commits, not atomic
48
- - **High latency** — State changes require `git push`; observability is on a push-frequency, not real-time
49
- - **No locking** — State can be read/modified while orchestrator is deciding; no optimistic concurrency
50
- - **No history/events** — Only final state (current file contents) is stored; intermediate states are lost
16
+ **Limitation**: Does not scale to teams—no concurrent writers, no transactions, high latency (requires `git push`).
51
17
 
52
18
  ---
53
19
 
54
- ## Target: SQLite WAL as Live Substrate, Git as Export
55
-
56
- ### Proposed data model
20
+ ## Future: SQLite Event Log + Git Export
57
21
 
58
- ```
59
- orchestrator/agents write events ─→ SQLite WAL event log (LIVE substrate, fast, transactional)
60
-
61
- state_store/ EventStore
62
- (append() atomic across processes/threads)
63
-
64
- [snapshots for O(n) tail-replay]
65
-
66
- projections/StateAPI (project tracker state)
67
-
68
- export_tracker() ─────────→ STATE.md / BUILDLOG.md (git-tracked EXPORT, human-diffable, immutable)
69
-
70
- [dashboards, monitoring, forensics read git export]
71
- ```
72
-
73
- ### Key design principles
74
-
75
- 1. **Event sourcing**: State is a fold of immutable events, not mutable files. Each write is a single `append(stream, event_type, payload, actor)` call — atomic, ordered, versionable.
76
-
77
- 2. **Append-only**: No deletes or mutations. History is preserved forever. Replayability + auditing for free.
78
-
79
- 3. **Per-stream versioning**: Each "stream" (e.g., `"tracker"`, `"orchestrator_status"`) has its own 1-based, gapless version. Enables deterministic snapshots and tail-replay (see `state_store/store.py` line 81–92 and `state_store/projections.py` line 117–127).
80
-
81
- 4. **Concurrency-safe writes**: SQLite WAL mode + `BEGIN IMMEDIATE` transactions guarantee that two writers on the same file never collide or duplicate a version (see `state_store/store.py` line 30–94). Tested in `tests/test_state_store.py::EventStoreTest::test_concurrent_appends_have_no_dupes_or_gaps`.
82
-
83
- 5. **Snapshots for O(n) tail-replay**: Instead of replaying all events on every read, save materialized state snapshots at key versions; future replays fold only newer events (see `state_store/store.py` line 130–196 and `state_store/projections.py` line 84–114).
84
-
85
- 6. **Git as export, not source**: The SQLite store is the system-of-record. `export_tracker(api, out_path)` renders the current projection back to human-readable JSON for git durability and review (see `state_store/export.py`). Git is now **read-only for state** (except for manual edits, which become new events on next `ingest_tracker_json`).
86
-
87
- 7. **Projection-based readers**: Callers do not query the raw event log; instead, they call `api.project(view)` to get the current state as a dict. New projections (e.g., `project_orchestrator_status`) can be registered in `state_store/api.py` without changing the event store (see line 12–35).
88
-
89
- ---
22
+ **Vision**: Event-sourced SQLite WAL becomes the system-of-record. Git exports are rendered from current projections for durability and review.
90
23
 
91
- ## Architecture: The Three Layers
24
+ **In the repo now**:
25
+ - `state_store/store.py` — EventStore with atomic `append()`, concurrency-safe writes
26
+ - `state_store/projections.py` — Fold events into tracker state
27
+ - `state_store/api.py` — Facade for backend swaps (SQLite today, Postgres later)
28
+ - `tests/test_state_store.py` — Concurrent-write proofs, projection tests
92
29
 
93
- ### 1. Event Store (SQLite WAL)
30
+ **Next steps**: Wire the orchestrator and agent writers to use `StateAPI` instead of git direct writes. This unblocks team-scale coordination without breaking single-instance durability (git exports remain).
94
31
 
95
- **File**: `state_store/store.py`
96
-
97
- **Class**: `EventStore(db_path)`
98
-
99
- **Public API**:
100
- - `append(stream: str, event_type: str, payload: dict, actor: str = "system") -> int` — Append one event, return its per-stream version. Atomic: multiple writers see no dupes or gaps (tests line 90–108).
101
- - `read(stream: str) -> list` — All events in a stream, ascending by version. Returns list of dicts with `{"id", "ts", "actor", "stream", "type", "payload", "version"}`.
102
- - `read_all() -> list` — All events across all streams, ascending by global id.
103
- - `save_snapshot(stream: str, event_version: int, projection: dict) -> None` — Persist a materialized projection at a specific version (line 130–153).
104
- - `read_snapshot(stream: str) -> tuple | None` — Read the most recent snapshot; returns `(event_version, projection_dict, checksum)` or None (line 155–197).
105
-
106
- **Concurrency model**:
107
- - Multiple writers: `BEGIN IMMEDIATE` locks the write lock up front, so read-max-version-then-insert is atomic (line 80).
108
- - Multiple readers: SQLite WAL mode allows readers to run concurrently with writers on separate .db-wal shadow files (no blocking).
109
- - Process/thread-safe: each `append()` call opens its own connection; `busy_timeout=5000` (5 seconds) waits if the DB is locked (line 32–94).
110
- - Tested with two threads, each with its own EventStore instance, appending 100 events concurrently with no dupes (tests line 90–108).
111
-
112
- ### 2. Projections (Fold Events into State)
113
-
114
- **File**: `state_store/projections.py`
115
-
116
- **Functions**:
117
- - `_fold_events(events, order=None, items=None) -> (order, items)` — Fold events into an accumulated state tuple. Mutates order + items in place. Helper used by both full and snapshot-aware projections (line 25–71).
118
-
119
- - `project_tracker(events: list) -> dict` — Original full-replay projection: fold all events → `{"version": 1, "items": [items in first-seen order]}`. Defines tracker event schema:
120
- - `item_created` (payload = full item dict): establishes item, kept in first-seen order
121
- - `item_updated` (payload = `{"id": ..., ...partial fields}`): merges partial fields onto existing item
122
- - `item_archived` (payload = `{"id": ...}`): sets `status` to `"archived"`, optionally merges `completed_at`
123
- - Unknown event types: ignored (line 74–81).
124
-
125
- - `project_tracker_with_snapshot(store, stream: str, events: list) -> dict` — Snapshot-aware projection: load latest snapshot; fold only tail events (after snapshot version) for O(n) tail-replay. Falls back to full replay if snapshot is missing or corrupt (line 84–114).
126
-
127
- - `save_snapshot(store, stream: str, event_version: int, projection: dict) -> None` — Wrapper over `store.save_snapshot()` (line 117–126).
128
-
129
- **Event schema for tracker**:
130
- ```json
131
- {
132
- "type": "item_created",
133
- "payload": {"id": "abc", "title": "...", "lane": "proposed", "status": "todo", ...}
134
- }
135
- {
136
- "type": "item_updated",
137
- "payload": {"id": "abc", "lane": "in-progress"}
138
- }
139
- {
140
- "type": "item_archived",
141
- "payload": {"id": "abc", "completed_at": "2026-07-17T...Z"}
142
- }
143
- ```
144
-
145
- Tolerance: unknown ids on update/archive are ignored (no error); unknown event types are skipped (line 62–70).
146
-
147
- ### 3. Facade API (Single Seam for Backend Swaps)
148
-
149
- **File**: `state_store/api.py`
150
-
151
- **Class**: `StateAPI(db_path)`
152
-
153
- **Public API**:
154
- - `append(stream: str, event_type: str, payload: dict, actor: str = "system") -> int` — Append event, return version.
155
- - `get(stream: str) -> list` — Read all events in a stream.
156
- - `project(view: str) -> dict` — Fold the named stream through its projector into current state. Raises `ValueError` if projector not registered (line 15–35).
157
-
158
- **Backend swaps**: The backend (SQLite now, Postgres later) is abstracted here. Callers **never** touch `EventStore` directly; they use `StateAPI` only. On cutover to Postgres, only `api.py` changes; call sites stay the same.
159
-
160
- ---
161
-
162
- ## New State Streams (Beyond Tracker)
163
-
164
- The tracker stream is the first proof of concept. The event log supports **arbitrary streams**; new streams are registered by adding a projector to `_PROJECTORS` in `state_store/api.py` (line 12).
165
-
166
- ### Proposed: `orchestrator_status` stream
167
-
168
- **Purpose**: Record orchestrator phase, NEXT STEPS, decisions, and audit results in real-time without git writes.
169
-
170
- **Events**:
171
- ```json
172
- {
173
- "type": "phase_changed",
174
- "payload": {"phase": "wave-rc.3", "reason": "stable release shipped"}
175
- }
176
- {
177
- "type": "next_steps_updated",
178
- "payload": {"steps": ["reconcile state_store SI", "add orchestrator_status stream", ...]}
179
- }
180
- {
181
- "type": "decision_locked",
182
- "payload": {"key": "model_dispatch_core", "value": "out-of-repo (structural)", "justification": "..."}
183
- }
184
- {
185
- "type": "audit_findings_recorded",
186
- "payload": {"level": "wave-rc.2", "findings_count": 3, "summary": "..."}
187
- }
188
- ```
189
-
190
- **Projection** (pseudo-code):
191
- ```python
192
- def project_orchestrator_status(events):
193
- status = {
194
- "phase": None,
195
- "decisions": {},
196
- "next_steps": [],
197
- "latest_audit": None,
198
- }
199
- for ev in events:
200
- if ev["type"] == "phase_changed":
32
+ For implementation details, see `state_store/` source code and inline documentation.
201
33
  status["phase"] = ev["payload"]["phase"]
202
34
  elif ev["type"] == "decision_locked":
203
35
  key = ev["payload"]["key"]
package/docs/reproduce.md CHANGED
@@ -1,6 +1,21 @@
1
1
  # Reproducing Aesop
2
2
 
3
- This document explains how to reproduce the project's claims from a clean checkout.
3
+ This document explains how to reproduce the project's claims from a clean checkout or installed package.
4
+
5
+ ## Quick Start: One-Command Verification
6
+
7
+ Run the verification suite with automatic context detection:
8
+
9
+ ```bash
10
+ aesop reproduce
11
+ ```
12
+
13
+ This command automatically detects your context (repository checkout or installed package) and runs the appropriate test suite:
14
+
15
+ - **Repository checkout**: Runs the full test suite (Node syntax, shell syntax, Node tests, Python tests, benchmark, browser proofs)
16
+ - **Installed package**: Runs shipped self-checks (preflight checks, health score, secret-scan selftest, packaging assertions)
17
+
18
+ Output includes per-step timing and a clear pass/fail summary. Exit code 0 = all checks passed.
4
19
 
5
20
  ## Overview
6
21
 
@@ -55,20 +70,35 @@ Both can be reproduced entirely offline, without API keys or external dependenci
55
70
  python tools/bench_runner.py --runner mock
56
71
  ```
57
72
 
58
- ## What the `.github/workflows/reproduce.yml` Job Does
73
+ ## CI Equivalence: aesop reproduce vs. GitHub Actions
59
74
 
60
- The `reproduce` workflow in GitHub Actions automates the above steps:
75
+ The `aesop reproduce` command mirrors `.github/workflows/reproduce.yml` for local execution.
76
+
77
+ ### GitHub Actions Workflow (`reproduce.yml`)
78
+
79
+ The `reproduce` workflow in GitHub Actions automates testing:
61
80
 
62
81
  1. **Fresh checkout**: Uses a clean clone (no state reuse from other CI jobs)
63
82
  2. **Syntax checks**: Validates all Node.js (.mjs) and shell scripts (.sh)
64
83
  3. **Node.js tests**: Runs the full test suite with `npm run test:node`
65
84
  4. **Python tests**: Runs the full test suite with `python -m unittest discover`
66
85
  5. **Benchmark scorer**: Proves the offline mock benchmark reproduces (no external API calls)
86
+ 6. **Browser proofs**: Validates UI components with Playwright
67
87
 
68
88
  The workflow is triggered:
69
89
  - **Manually**: Via GitHub Actions "Run workflow" button (workflow_dispatch)
70
90
  - **Weekly**: Every Sunday at 2:00 AM UTC (schedule)
71
91
 
92
+ ### Local Reproduction
93
+
94
+ To run the same tests locally from a clean clone:
95
+
96
+ ```bash
97
+ aesop reproduce
98
+ ```
99
+
100
+ This produces the same verification steps and provides immediate feedback before pushing.
101
+
72
102
  ## What Is and Isn't Reproduced
73
103
 
74
104
  ### Reproduced (offline, no API keys needed)
package/driver/CLAUDE.md CHANGED
@@ -1,11 +1,8 @@
1
1
  # driver/ — AgentDriver backend-portability seam
2
2
 
3
- **What**: The one-file domain contract that lets aesop's wave loop run on
4
- backends other than Claude Code (Codex, open models). The wave loop dispatches
5
- through the `AgentDriver` interface and nothing else; each backend is a subclass.
6
-
7
- Grounded in a multi-model portability design spike.
8
- **Phase 1 (interface + reference adapter, shipped)** + **Phase 2 (Codex OpenAI Chat Completions, shipped)** + **Phase 3 (wave-manifest bridge, shipped)**.
3
+ **What**: the domain contract letting aesop's wave loop run on non-Claude backends
4
+ (Codex, open models); the loop dispatches only through the `AgentDriver` interface.
5
+ **Phases 1-3 shipped** (interface + reference adapter; Codex Chat Completions; wave bridge).
9
6
 
10
7
  ## Files
11
8
 
@@ -24,6 +21,8 @@ Grounded in a multi-model portability design spike.
24
21
  - **openai_compatible_driver.py** — OpenAI-compatible backend (Ollama, OpenRouter, etc.).
25
22
  - **verification_policy.py** — Maps verification tier -> orchestrator tuning (validate_all_json,
26
23
  spot_check_frac, repair_cap, require_adversarial_review).
24
+ - **wave_loop.py** — the wave ENGINE: preflight ownership guard, parallel build,
25
+ bounded repair, adversarial review, per-repo batched git ship, recovery journal.
27
26
  - **wave_bridge.py** — Phase 3: bridges AgentDriver backends to wave manifest items.
28
27
  build_manifest_item() enriches with verificationTier + model; dispatch_item() routes
29
28
  by capability and decides green ONLY from test exit code (not model's say-so).
@@ -33,8 +32,7 @@ Grounded in a multi-model portability design spike.
33
32
  - **../tests/test_codex_driver_e2e.py** — Phase 2 end-to-end offline tests
34
33
  (FakeTransport, red-to-green verification, retry logic, ownership enforcement)
35
34
  + gated live test (AESOP_CODEX_LIVE env var).
36
- - **../tests/test_wave_bridge.py** — Phase 3 end-to-end offline tests proving
37
- non-Claude backend drives a real RED stub to verified-GREEN offline (no API key,
35
+ - **../tests/test_wave_bridge.py** — Phase 3 offline e2e (honest green: exit 0 only).
38
36
  no network). Tests: manifest building, routing, fail-safe, ownership enforcement,
39
37
  headline test (red stub + FakeTransport fix + test pass -> ok=True).
40
38
 
@@ -85,47 +83,18 @@ Optional (non-abstract): `get_tokens_spent()`.
85
83
 
86
84
  ## Phase 2 Codex Implementation Details
87
85
 
88
- The Codex driver proves a non-Claude backend can take a real coding task
89
- end-to-end through the AgentDriver and produce orchestrator-verified results,
90
- entirely offline (no API key, no network in CI).
91
-
92
- **dispatch_worker**: Orchestrator-managed worker (Tier 2):
93
- - Injects owned-file contents into the prompt (worker has no filesystem access).
94
- - Calls OpenAI Chat Completions API via injectable transport (default=urllib).
95
- - Requests strict JSON schema output (full-file replacements).
96
- - Validates ALL JSON with bounded in-turn retry (<=2 attempts).
97
- - Enforces ownership: rejects out-of-scope paths wholesale.
98
- - Pre-dispatch max_owned_bytes guard: fails safe on oversized files (no
99
- truncation).
100
- - Writes full-file replacements to disk; never applies diffs.
101
- - Records usage.total_tokens for cost tracking.
102
- - CRITICAL: Green is NOT decided by the model's done:true; it is decided by
103
- the orchestrator running run_command and getting exit 0 (center verification).
104
-
105
- **Transport seam**: The injectable transport callable keeps CI offline. Tests
106
- pass FakeTransport with canned responses; production code reads OPENAI_API_KEY
107
- from environment and uses default_openai_transport (stdlib urllib, hard timeout).
108
-
109
- **Verification policy**: verification_policy(caps) maps tier 2 -> {
110
- validate_all_json: True, spot_check_frac: 0.50, repair_cap: 2,
111
- require_adversarial_review: True }. Feeds the wave's integration verifier.
86
+ Codex driver (Tier 2): injects file contents into prompt, calls OpenAI Chat Completions via injectable transport, validates JSON with bounded retry, enforces ownership. CRITICAL: Green = exit 0 only. Verification policy: tier 2 -> {validate_all_json:True, spot_check_frac:0.50, repair_cap:2, require_adversarial_review:True}. **P1 Security**: Default model map uses gpt-4o-mini (worker, supports json_schema); init-time guard rejects models lacking json_schema support unless `allow_unverified_models=True` (P1 gate: prevent gpt-3.5-turbo silent failures).
112
87
 
113
88
  ## Phase 3 Bridge Implementation Details
114
89
 
115
- The wave bridge connects AgentDriver backends to wave-flat-dispatch manifest
116
- items, making verification tier driven by backend capability, not config.
117
-
118
- **build_manifest_item(driver, item) -> dict**:
119
- - Takes a backlog item {slug, ownsFiles, prompt, testCmd, workDir, ...} + driver.
120
- - Returns manifest-item dict enriched with: model (from driver.resolve_model('worker')),
121
- verificationTier (from driver.probe_capabilities().recommended_verification_tier),
122
- AND all four resolved policy knobs (repairCap, requireAdversarialReview, spotCheckFrac,
123
- validateAllJson from verification_policy(caps)).
124
- - Preserves all input fields; adds model, tier, and all four policy fields.
125
- - The policy is RESOLVED ONCE in Python and carried as literal manifest fields, so
126
- the template cannot recompute/drift. Tier-1/Claude path with no verificationTier
127
- maintains byte-identical behavior (tier-1 defaults: repairCap=1, requireAdversarialReview=false,
128
- spotCheckFrac=0.10, validateAllJson=false).
90
+ Connects AgentDriver backends to wave-flat-dispatch manifest items; verification
91
+ tier is driven by backend capability, not config.
92
+
93
+ **build_manifest_item(driver, item) -> dict**: enriches a backlog item with model
94
+ (driver.resolve_model), verificationTier (probe), and the four policy knobs from
95
+ verification_policy(caps) RESOLVED ONCE, carried as literal manifest fields so the
96
+ template cannot recompute/drift; tier-1/Claude path stays byte-identical (repairCap=1,
97
+ requireAdversarialReview=false, spotCheckFrac=0.10, validateAllJson=false).
129
98
 
130
99
  **dispatch_item(driver, item) -> dict**:
131
100
  - Routes by worker_filesystem_access: True→{route:'harness'}; False→orchestrator-managed
@@ -138,11 +107,44 @@ items, making verification tier driven by backend capability, not config.
138
107
  stub, applies a fix, runs the test, and returns ok=True ONLY because the test passed
139
108
  (exit 0). All offline, no API key, no network.
140
109
 
110
+ ## Wave Scheduler (WS3a Pilot) + GATE-1 Handoff Kit
111
+
112
+ **wave_scheduler.py** — single-cycle backlog orchestration: intake up to N file-disjoint todo items from tracker.json (empty/missing ownsFiles REJECTED; paths normalized posix+casefold-on-Windows before overlap checks; required fields pre-validated) -> manifest via build_manifest_item (model + verificationTier from driver.probe) -> HALT + cost-ceiling gates (fail-CLOSED: module import failure or check exception = abort with honest Report, phase=gate_unavailable) -> run_wave (recovery journal + git ship) -> STOP before merge; Report JSON with per-item observability (GATE-1). After ship, selected items atomically marked in_progress in tracker (temp+os.replace; dry-run never mutates) so a second run cannot double-dispatch.
113
+
114
+ **CLI** (GATE-1): `python driver/wave_scheduler.py --tracker <path> --max-items N --dry-run|--execute --driver claude|codex` (default: claude). For codex+execute, requires OPENAI_API_KEY env var; dry-run works without it.
115
+
116
+ **Tests** (35+): disjoint/normalization/rejection, gate fail-closed, dry-run, GATE-1 per-item/driver/ceiling/codex tests; module-tmpdir hygiene; all TestCase.
117
+
118
+ **Invariants**: stdlib-only, ASCII, Windows+Linux safe (list-form subprocess); manifest items carry resolved policy knobs from verification_policy (no recompute drift); merge stays manual in the pilot.
119
+
120
+ ### REPORT-CONTRACT (GATE-1 Orchestrator Handoff)
121
+
122
+ The scheduler emits a Report JSON consumed by the orchestrator to decide merge eligibility:
123
+ ```
124
+ {
125
+ "phase": "dispatch"|"intake"|"halt"|"ceiling"|"gate_unavailable"|"manifest",
126
+ "wave_id": "<uuid>",
127
+ "items_selected": ["id1", "id2"], // IDs selected for this wave
128
+ "items_shipped": [
129
+ { "slug": "feat/x", "backend": "claude-code|codex", "tier": 1|2|3|4, "verified": true|false (false = NOT PROVEN, not necessarily failed; tier null = no build record), "testExit": 0|1|null },
130
+ ...
131
+ ],
132
+ "merged": false, "tracker_update_attempted": true|absent, "tracker_unmapped_slugs": [..]|absent (LOUD -> success false), // pilot always false (manual merge)
133
+ "success": true|false,
134
+ "timestamp": "<ISO8601>",
135
+ "branch": "<branch_name>", // optional; set when ship succeeds
136
+ "sha": "<git_sha>", // optional; set when ship succeeds
137
+ "halt_reason": "...", // optional; set if halted
138
+ "ceiling_reason": "...", // optional; ceiling exceeded before dispatch (abort-before-dispatch only)
139
+ "error": "..." // optional; unexpected error
140
+ }
141
+ ```
142
+ Ceiling semantics: checked BEFORE run_wave dispatch (phase=ceiling, ceiling_reason populated). Mid-wave ceiling trips are run_wave's responsibility. items_shipped[] carries per-item observability: slug, backend (driver name), tier (verificationTier), verified (from test exit 0), testExit (test command exit code or null if not run).
143
+
141
144
  ## Status
142
145
 
143
146
  - **Phase 1**: shipped. Interface + Claude reference adapter + contract tests.
144
147
  - **Phase 2**: shipped. Codex OpenAI Chat Completions implementation. Offline tests GREEN.
145
148
  - **Phase 3**: shipped. Wave bridge: driver → manifest, orchestrator-side dispatch.
146
149
  Proves non-Claude backends drive items end-to-end with honest green (test exit 0 only).
147
- - **Next**: Refactor wave-flat-dispatch onto the driver (Phase 1 handoff).
148
- - **Future**: Open-model adapter (Tier-4 backend).
150
+ - **Wave Scheduler (WS3a) + GATE-1**: shipped. Single-cycle orchestration: intake → manifest → dispatch report (manual merge). Per-item observability, driver injection (--driver claude|codex), ceiling semantics documented.