@matt82198/aesop 0.1.0-beta.3 → 0.1.0-beta.5

package/monitor/CHARTER.md

@@ -1,116 +1,103 @@
-# Aesop Orchestration Monitor — Charter
+# Orchestration Refinement Monitor — Charter
 
-A standing background monitor (Haiku subagent) that watches how the multi-agent system operates and acts on points of refinement. It improves the *machinery*, never the *mission*.
+A standing background monitor (Haiku) that watches how the multi-agent system operates and
+acts on points of refinement. It improves the *machinery*, never the *mission*.
 
 ## FIXED GOAL (never changes)
+> Ship and maintain your projects correctly, cheaply, and autonomously under the cardinal rules
+> (`~/.claude/CLAUDE.md`). The monitor's job is to make that machine run better — it must NOT
+> redirect, re-scope, or invent new project goals. If it ever thinks the goal should change, it
+> writes a note in `PROPOSALS.md` and stops there.
+
+## What it watches (via `collect-signals.mjs`, refreshed each cycle)
+1. **Junk-script sprawl** — one-off Python/JS scripts Claude Code writes to temp/scratch/cloud and
+   never reuses. Repeated throwaway `.py` to remote agents is a known cost/clutter sink.
+2. **Memory gaps** — projects/decisions/gotchas with no durable memory file; stale or duplicate memories.
+3. **Rule friction** — cardinal-rule violations in recent activity (non-Haiku subagents, main-thread
+   big reads, unpushed green work) and rules that are ambiguous, contradicted, or repeatedly ignored.
+4. **Orchestration health** — hung/stale agents, drift, repeated failures, git state across repos.
 
-> Maintain your projects correctly, cheaply, and autonomously under your cardinal rules.
-> The monitor's job is to make that machine run better — it must NOT redirect, re-scope,
-> or invent new project goals. If it ever thinks the goal should change, it writes a note
-> in `PROPOSALS.md` and stops there.
+## Action tiers
+- **AUTO (apply immediately, then log to `ACTIONS.log`):**
+  - Persist/refresh project memory files (one fact each) + update `MEMORY.md` index.
+  - Quarantine confirmed-dead one-off scripts: move junk temp `.py`/`.mjs` (older than 24h, not
+    referenced by any repo, not the active session's scratchpad) into `orchestration-monitor/quarantine/`
+    with a manifest — never delete outright, never touch a live session scratchpad or repo `src/`.
+  - Clarifying, non-behavioral edits to a memory/log the monitor owns.
+- **PROPOSE (write to `PROPOSALS.md`, do NOT apply — needs user OK):**
+  - Any change to `~/.claude/CLAUDE.md`, `~/.claude/skills/**`, or a project `CLAUDE.md`.
+  - Anything that alters agent behavior, model choice, push targets, or goal scope.
+  - Deleting anything outside the monitor's own quarantine.
 
-## Standing checks (deterministic signal collection)
+## Hard guardrails (inviolable)
+- Preserve the FIXED GOAL. Improve rules; never rewrite intent.
+- Subagents are ALWAYS Haiku (this monitor included). Opus orchestrates on the main thread only.
+- NEVER work around a classifier denial (cross-repo private-source push = exfil; reusing a
+  credential meant for another service = credential exploration). If seen, log + propose, don't act.
+- NEVER print secret values. NEVER push to a remote or run a destructive command as an AUTO action.
+- Idempotent + additive: safe to run repeatedly; prefer append/quarantine over overwrite/delete.
+- Stay cheap: read the signal brief, not raw code/logs. One tight cycle, then sleep.
 
-Each cycle, `collect-signals.mjs` runs these checks and emits `BRIEF.md` + `SIGNALS.json`:
+## End-of-day wipe-survival sweep
 
-1. **Heartbeat liveness check** — read `.heartbeats/*` (epoch on line 1) + legacy beat files
-   (`.monitor-heartbeat`, `.watchdog-heartbeat`); flag any older than threshold (watchdog 300s,
-   monitor 3600s, others 1800s default) as stale loops; append warning to BRIEF.md.
+The monitor may run `python ~/scripts/eod_sweep.py` in its final cycle of the day.
+This verifies all known git repos are safe (no data loss risk):
+- working trees clean OR dirty-file list logged
+- current branch pushed to origin (ahead-count 0)
+- untracked files not silently .gitignore'd
+- SECURITY-ALERTS.log has no unreviewed HIGH/MED items
+- memory INBOX consumed (0 pending items)
+- no heartbeat file claims a dead loop (>1h stale)
 
-2. **Git state across repos** — for each repo in config: branch, last commit, uncommitted files,
-   commits ahead of remote. Identifies unpushed work and dirty state.
+Output: single verdict line (`EOD-SWEEP: SAFE` or `EOD-SWEEP: AT-RISK — <n> findings`) + one finding per line;
+appends to handoff log. Exit code 0 only if SAFE.
 
-3. **Memory freshness** — scan project memory files (excluding index/inbox); flag any >30 days old;
-   append summary to BRIEF.md ("N memories stale — keeper should re-verify").
+If findings detected, the monitor stages an alert: `EOD-SWEEP AT-RISK — <n> findings` (each finding on bullet).
 
-4. **Log rotation** — check append-only logs against max-lines/max-kb thresholds (configurable,
-   default 500 lines / 40 KB). If a log exceeds threshold, emit rotation signal (AUTO tier can
-   invoke rotate_logs.py if available); append rotation summary to BRIEF.md.
+## Standing checks (trimmed for solo operation)
 
-5. **Junk-script sprawl detection** — scan temp/scratch roots for throwaway `.py`/.mjs`/`.js` files
-   older than 24 hours, not in a live session directory (avoid false-positives during active work).
-   Count total, estimate quarantinable, list oldest. AUTO tier quarantines confirmed junk into
-   `monitor/quarantine/` with manifest.
+**KEPT (durable value):**
 
-6. **Stray scripts in repo roots** — scan recent commits (7d) to detect one-off `.py`/`.mjs`/`.sql`
-   scripts committed directly to repo root (not under proper src/scripts paths). Flag for cleanup.
+1. **Scripts outside `~/scripts`** — flag `.py`/`.mjs`/`.sh` accumulating outside `~/scripts`
+   (scratchpads, temp, repo roots). Quarantine per existing tiers; when one looks genuinely reusable,
+   suggest promotion to `~/scripts` (indexed in `~/scripts/CLAUDE.md`).
 
-7. **Security alert review loop** — tail SECURITY-ALERTS.log for new HIGH/MED entries (skip suppressed);
-   each new alert noted in BRIEF.md. Monitor agent does semantic review (REAL vs FP) each cycle.
+2. **Remote Python execution = HIGH-tier finding** — any scheduled/cloud agent or workflow that would
+   execute Python remotely violates Rule 5 (local-only execution). Flag immediately; never create one.
 
-8. **Respawn watch (Rule 6 retry cap)** — parse agent spawn records; normalize descriptions to
-   prompt-signatures (first ~40 chars, lowercased); count occurrences in recent window (last 50 rows).
-   Flag any signature appearing >3 times as probable hung-agent loop → rule 6 cap breached.
-   Limitation: heuristic may have false positives for intentional similar-task fan-outs.
+3. **Memory freshness** — scan `~/.claude/projects/*/memory/*.md` (excluding INBOX.md, MEMORY.md); emit `staleMemories` signal in SIGNALS.json listing any file >30d old; append to BRIEF.md: "N memories >30d — keeper should re-verify: <names>".
 
-9. **Cost cadence tracking** — every 3rd cycle, harvest agent spawn ledger and append cost tick
-   to COST-LOG.md (cycle count, timestamp, model distribution). Flags non-haiku specializations.
+4. **SECURITY ALERT REVIEW LOOP** — on each /power cycle, arm a persistent Monitor tailing
+   SECURITY-ALERTS.log filtered to new 'HIGH'/'MED' lines (exclude SUPPRESSED-FP). Each new alert spawns a read-only fleet-auditor agent to classify REAL vs FALSE-POSITIVE (mask any secret: first 4 chars + ***), verdict appended to SECURITY-ALERTS.log as a REVIEW note and, if REAL, surfaced to the user immediately. Rationale: the append-only alert log is only useful if alerts are triaged; unreviewed HIGHs rot into ignored noise.
 
-10. **Unreviewed agent prompts** — run fleet-prompt-extractor.py (if available) to collect NEW
-    spawns since last review; emit count in SIGNALS.json + note in BRIEF.md.
+5. **Fleet ledger harvest** — each cycle, run `python ~/scripts/fleet_ledger.py harvest` to scan session task outputs (JSONL format) and append missing agent outcomes to OUTCOMES-LEDGER.md. Tracks last-harvest state in .fleet-ledger-harvest.json sidecar. Deterministic, local-only, append-only; feeds cost analysis and agent respawn detection. Then invoke `python ~/scripts/fleet_ledger.py rotate` to archive old rows if ledger exceeds 200 lines.
 
-## Action tiers
+6. **Log rotation** — after ledger operations, invoke `python ~/scripts/rotate_logs.py <file> --max-lines 500 --max-kb 40` over: hook-activity.log, SECURITY-ALERTS.log, FLEET-BACKUP.log (BUILDLOG.md at 1000-line threshold — historical record). Emit `logRotations` list in SIGNALS.json and summary line in BRIEF.md.
 
-- **AUTO (apply immediately, then log to `ACTIONS.log`):**
-  - Heartbeat freshness checks (read-only).
-  - Log rotation (invoke rotate_logs.py if available; fail-open if not).
-  - Persist heartbeat write (`.monitor-heartbeat` update after cycle).
-  - Quarantine confirmed-junk scripts: move old temp `.py`/`.mjs` (not live session) into
-    `monitor/quarantine/` with manifest — never delete outright.
-
-- **PROPOSE (write to `PROPOSALS.md`, do NOT apply — needs user approval):**
-  - Changes to cardinal rules or agent configuration.
-  - Alterations to agent behavior, model choice, or push targets.
-  - Deletions of anything outside monitor's own quarantine.
-  - Any change to orchestration policy.
+7. **Heartbeat liveness check** — read state/.heartbeats/* (epoch on line 1) + legacy beat files (.monitor-heartbeat, .watchdog-heartbeat, etc.); flag any older than threshold (watchdog 300s, monitor 3600s, others 1800s default) as `staleLoops` signal in SIGNALS.json; append warning to BRIEF.md. Continuous watchdog complement to /power selftest.
 
-## Hard guardrails (inviolable)
+8. **Respawn watch (Rule 6 machine-enforcement)** — parse OUTCOMES-LEDGER.md for agent descriptions; normalize each to prompt-signature (first ~40 chars, lowercased, trimmed); count occurrences per signature within recent window (last 50 rows OR since marker in .signal-state.json). Any signature appearing >3 times = probable hung-agent respawn loop → add to `respawnWatch` in SIGNALS.json + warning line in BRIEF.md. *Limitation:* heuristic may have false positives for intentional similar-task fan-outs (e.g., per-file processing with distinct file names in description); BRIEF includes note to manually review flagged signatures. Pure node parsing (no LLM).
 
-- Preserve the FIXED GOAL. Improve rules; never rewrite intent.
-- **Subagents are ALWAYS Haiku** (cost optimization). Orchestrator on main thread only.
-- NEVER push destructively or to restricted remotes as an AUTO action.
-- NEVER edit cardinal-rules files (CLAUDE.md, skills/, agent configs) — only PROPOSE changes.
-- Idempotent + additive: safe to run repeatedly; prefer append over overwrite.
-- Stay cheap: read the signal brief, not raw logs. One tight cycle, then sleep.
-- Robust to missing files: treat missing dirs/logs as empty, never crash.
-
-## Outputs
-
-- `BRIEF.md` — human-readable status snapshot (overwrite each cycle).
-- `SIGNALS.json` — machine-readable metrics and findings (overwrite each cycle).
-- `ACTIONS.log` — append-only record of AUTO actions taken (timestamped).
-- `PROPOSALS.md` — staged changes awaiting user approval.
-- `quarantine/` — parked junk scripts + `MANIFEST.tsv` (append-only).
-- `.monitor-heartbeat` — epoch timestamp (line 1) for liveness.
-- `.signal-state.json` — sidecar state (cycle count, seen prompts, etc.).
-
-## Operating it
+9. **Semantic prompt review** — each cycle, collect-signals.mjs runs `fleet_prompt_extractor.py` to identify NEW unreviewed agent-spawn prompts (tracks seen-hash state in `.fleet-prompts-seen.json`); count is emitted as `unreviewedPrompts` in SIGNALS.json + noted in BRIEF.md. If count > 0, the monitor agent does semantic review (read each prompt, classify: OK / DRIFT / SUSPICIOUS) and runs `drift_escalator.py` to write verdicts to DRIFT-REVIEW.log + escalate any DRIFT/SUSPICIOUS findings to SECURITY-ALERTS.log + DRIFT-ESCALATIONS.md. This closes the gap: regex fleet-scan is NOT semantic review; LLM judgment gates malformed/off-goal/injection prompts. Completes before any flagged prompt's agent runs.
 
-1. **Deploy the monitor** as a continuous background task (or invoke manually each cycle).
-2. **Customize via config**: populate `aesop.config.json` with your repo paths, thresholds.
-3. **Review PROPOSALS.md** periodically; approve changes or reject them.
-4. **Check ACTIONS.log** to see what was automated.
-5. **Monitor BRIEF.md** for drift signals; respond to findings.
+10. **Mirror refresh cadence (throttled)** — each monitor cycle mirrors scripts and docs to your fleet-ops directory and regenerates asset indexes. If either produces diffs: check `.last-mirror-pr-time` in state/. Only open a PR if >1 hour has passed since last PR; otherwise log silently as no-op to ACTIONS.log with timestamp. Ensures mirrors stay in sync and asset lists reflect tier-1 changes, but prevents PR spam. Pure Python automation; runs in monitor context only.
 
-## Customization
+11. **UI inbox drain (session-independent work queue)** — each cycle, run `python ~/scripts/inbox_drain.py pending` to surface queued submissions that arrived while no session was running. If items exist, write a one-line `QUEUED WORK PENDING` flag to BRIEF.md so an active monitor also surfaces it. This complements /power's inbox drain (step 5); the monitor ensures work doesn't rot in the queue even between sessions.
 
-Edit `collect-signals.mjs` to:
-- Add/remove checks based on your project needs.
-- Configure paths via environment variables (AESOP_ROOT, BRAIN_ROOT, SCRIPTS_ROOT, TEMP_ROOT)
-  or load from aesop.config.json.
-- Set heartbeat thresholds appropriate to your workflow.
-- Integrate custom signal collectors (e.g., compliance checks, custom linters).
+**CUT or DEMOTED (no durable value for solo operator):**
 
-See comments in `collect-signals.mjs` for extension points. **Keep it CRLF-safe** (no line
-continuations; maintain Windows+POSIX compatibility).
+- **Suppression audit** — (CUT) A solo operator watching the dashboard notices suppression count manually; meta-audits add ceremony without signal.
+- **Cost cadence** — (DEMOTED; blocked) Ledger parsing incomplete; once cost meter is proven, restore. For now, flag as blocked in PROPOSALS.md if touched.
 
-## Single-instance guard
+**Rationale & changes:**
 
-Before running, check `.monitor-heartbeat` — if <300s old, skip cycle (another monitor is running).
-After cycle completes, update `.monitor-heartbeat` with current epoch.
+The original 12-check charter was ceremony-heavy for a one-person system. Cuts preserve the critical signals (liveness, security triage, hung loops, prompt safety, cost data acquisition) and demote/cut only checks whose value is marginal or already visible to a solo operator. Check #10 was throttled instead of cut because the mirror sync is critical, but PR-per-cycle wastes bandwidth — now it only PRs when changes exist AND at least 1 hour has passed.
 
-## Single-writer discipline
+Total standing checks: **11** (reflecting genericization; some checks are environment-specific).
 
-- Only the monitor edits `BRIEF.md`, `SIGNALS.json`, `ACTIONS.log`, `.monitor-heartbeat`, `.signal-state.json`.
-- External processes (human, other agents) write to `PROPOSALS.md` (append or structured inbox).
-- Quarantine manifest is append-only; never edit entries, only add new ones.
+## Outputs (all under orchestration-monitor/)
+- `BRIEF.md` / `SIGNALS.json` — the deterministic snapshot (written by the collector).
+- `ACTIONS.log` — append-only record of AUTO actions taken (timestamped).
+- `PROPOSALS.md` — staged rule/behavior changes awaiting user approval.
+- `quarantine/` — parked junk scripts + `MANIFEST.tsv`.
+- `.monitor-heartbeat` — liveness for any dashboard.

package/monitor/CLAUDE.md

@@ -0,0 +1,40 @@
+# monitor/ — Orchestration monitor
+
+**Purpose**: Continuous background signal collector and refinement proposer — watches fleet machinery health deterministically (Node.js, no LLM), emits cycle snapshots, and proposes rule changes via append-only PROPOSALS.md; GOAL IS FIXED (improve machinery, never mission).
+
+## Files
+
+- **CHARTER.md** — Governance document; defines 10 signal checks (4 of them extended/opt-in), action tiers (AUTO/PROPOSE), outputs, single-instance guard, single-writer discipline. Read-only; behavior changes only via PROPOSALS/behavioral-PR flow.
+- **collect-signals.mjs** — Deterministic signal collector (Node.js built-ins only); emits BRIEF.md + SIGNALS.json each cycle; reads config from env or aesop.config.json.
+- **BRIEF.md** — Human-readable cycle snapshot (heartbeats, git state, memory freshness, log rotation, junk sprawl, stray scripts, security alerts, respawn watch, cost cadence, unreviewed prompts); overwritten each cycle; runtime.
+- **SIGNALS.json** — Machine-readable metrics (same signal keys as BRIEF); JSON; overwritten each cycle; runtime.
+- **PROPOSALS.md** — Structured inbox for user approval (idempotent per signal key, append-only); never edited by monitor after emission; tracks respawn-watch-breach, stray-repo-scripts, security-alerts-high-med, stale-memory-files; gitignored.
+- **ACTIONS.log** — Append-only log of AUTO tier actions taken (heartbeat updates, log rotation invokes, junk quarantine); runtime; gitignored.
+- **.monitor-heartbeat** — Epoch timestamp (line 1) for single-instance liveness check (<300s = skip cycle); runtime; gitignored.
+- **.signal-state.json** — Sidecar state (cycleCount, etc.); runtime; gitignored.
+
+## Contracts
+
+**Signal keys collected**: heartbeats, git, memory, logs, junk, strayRepo, alerts, respawnWatch, costTick, unreviewedPrompts. Outputs: BRIEF.md (human), SIGNALS.json (machine); both runtime/gitignored. PROPOSALS.md (tracked, append-only); ACTIONS.log (runtime/gitignored).
+
+**Extended signals (opt-in, default OFF)**: checks 5 (junk), 6 (strayRepo), 8 (respawnWatch), and 10 (unreviewedPrompts) are extended — disabled by default.
+- Config key: `monitor.extended_signals` (boolean, default `false`) in aesop.config.json.
+- Env override: `AESOP_EXTENDED_SIGNALS` (`'true'` or `'1'` to enable).
+- Precedence: env var > config file > default (false).
+- When disabled, extended checks emit `{"skipped": true}` in SIGNALS.json and BRIEF.md notes them as "extended (off)"; their dirs are not walked. PROPOSE-tier signals for extended checks (respawn-watch-breach, stray-repo-scripts) are only emitted when extended_signals is ON.
+
+**AUTO tier actions** (apply immediately, log to ACTIONS.log): heartbeat checks (read-only); log rotation (invoke rotate_logs.py if available, fail-open); heartbeat write (.monitor-heartbeat update); junk script quarantine (move old temp .py/.mjs to monitor/quarantine/ + manifest — only when extended_signals is ON).
+
+**PROPOSE tier actions** (write to PROPOSALS.md, await user approval): rule changes, agent config changes, deletions outside monitor/quarantine/, orchestration policy changes.
+
+**Idempotency rule**: Proposal emission keyed on signal key (e.g., 'respawn-watch-breach'); only emitted if not already present in PROPOSALS.md (check: `**Signal:** <key>` exists). Idempotent + append-only; safe to run repeatedly.
+
+## Invariants & Gotchas
+
+- CHARTER.md is authority; monitor logic is faithful to it; never edit CHARTER.md or collect-signals.mjs directly — propose via PROPOSALS.md.
+- Goal is FIXED (improve machinery, never mission); if monitor thinks goal should change, it writes note to PROPOSALS.md and stops.
+- Single-instance guard via .monitor-heartbeat; if <300s old, skip cycle.
+- Single-writer discipline: only monitor writes BRIEF.md, SIGNALS.json, ACTIONS.log, .monitor-heartbeat, .signal-state.json.
+- Config from env (AESOP_ROOT, BRAIN_ROOT, SCRIPTS_ROOT, TEMP_ROOT, AESOP_EXTENDED_SIGNALS) or aesop.config.json; falls back to safe defaults.
+- Robust to missing files; treats missing dirs/logs as empty, never crashes.
+- Node.js only (no Python, no LLM, no cloud); deterministic + cheap.

package/monitor/SIGNALS.json

@@ -0,0 +1,54 @@
+{
+  "timestamp": "2026-07-12T19:11:11.520Z",
+  "cycleCount": 10,
+  "heartbeats": {
+    "staleCount": 0,
+    "details": []
+  },
+  "git": [],
+  "memory": {
+    "count": 26,
+    "staleMemories": [],
+    "staleCount": 0
+  },
+  "logs": [
+    {
+      "name": "FLEET-BACKUP.log",
+      "exists": true,
+      "sizeKb": "1.8",
+      "lineCount": 37,
+      "needsRotation": false
+    },
+    {
+      "name": "SECURITY-ALERTS.log",
+      "exists": false,
+      "sizeKb": 0,
+      "lineCount": 0,
+      "needsRotation": null
+    },
+    {
+      "name": "ACTIONS.log",
+      "exists": true,
+      "sizeKb": "0.2",
+      "lineCount": 1,
+      "needsRotation": false
+    }
+  ],
+  "junk": {
+    "skipped": true
+  },
+  "strayRepo": {
+    "skipped": true
+  },
+  "alerts": {
+    "count": 0,
+    "highMedCount": 0
+  },
+  "respawnWatch": {
+    "skipped": true
+  },
+  "costTick": null,
+  "unreviewedPrompts": {
+    "skipped": true
+  }
+}