@cohortapp/agent-sdk 2.3.0

package/docs/runbooks/recovery-and-failover.md

@@ -0,0 +1,260 @@
+# Recovery and Failover Runbook
+
+How a Maestro agent recovers from reboots, power loss, memory pressure, rate
+limits, and budget exhaustion — and the short list of cases that still need a
+human. **The design goal (WS4) is self-healing: almost every failure ends in
+"the agent is running again" with no human in the loop.**
+
+Two invariants drive everything below:
+
+1. **Never brick.** Every automated terminal action ends in the agent running
+   again. The **only** thing that permanently stops an agent is a human writing
+   `.emergency-stop`. The memory watchdog does **not** write it on OOM.
+2. **Never drop work.** Under pressure, work is **QUEUED** (in memory,
+   re-derivable from durable `state/queues/*.yaml`) or **DEFERRED** (left on
+   disk). A governor/rate deferral never burns an item's retry budget.
+
+---
+
+## 0. Architecture in one paragraph
+
+Scheduled work flows through the **cadence bus** (`state/cadence-bus/`): launchd
+plists drop tiny JSON ticks onto `inbox/`; the single persistent daemon
+(`scripts/daemon/maestro-daemon.mjs` → `agent-daemon.mjs` + `cadence-consumer.mjs`)
+drains them, handling each inline or escalating to a managed `claude --print`
+sub-session. Inbound messages flow through the reactive daemon → `dispatcher.mjs`,
+which spawns sessions under a concurrency cap. **There is no separate
+"orchestrator" process** (older runbooks referenced one — it does not exist).
+Recovery is built into these components, not bolted on.
+
+---
+
+## 1. Reboot / power-outage recovery (automatic)
+
+This is the common case and it is **fully automatic** when the box is set up
+correctly. The recovery chain on boot:
+
+```
+power restored / reboot
+  └─ pmset autorestart=1            → Mac mini powers back on
+       └─ auto-login                → user session starts
+            └─ RunAtLoad daemon plist → maestro-daemon.mjs starts
+                 ├─ cadence-consumer drains state/cadence-bus/ (events survived on disk)
+                 ├─ agent-daemon sweepBacklog re-scans state/queues/*.yaml (durable)
+                 ├─ scanStaleLocks() + sweepStaleItemClaims() release orphaned claims
+                 ├─ resetActiveSessions({reconcile:true}) re-dispatches mid-flight work
+                 │     from state/sessions/resume-pending/ via `claude --print --resume`
+                 └─ missed cadence ticks FAST-FORWARD (one catch-up, not the burst)
+```
+
+**Missed ticks are not replayed.** The bus's at-most-once gate
+(`state/cadence-bus/schedule.json`) means an outage spanning N intervals enqueues
+**exactly one** catch-up tick per cadence on the next fire — never a thundering
+herd of N. (Grace window absorbs launchd double-fires; stale fast-forward absorbs
+outages.)
+
+**In-flight sessions resume.** Each spawned session writes
+`state/sessions/resume-pending/<sessionId>.json` and deletes it on a clean close.
+A marker still present after a crash means "mid-flight"; on startup the dispatcher
+re-dispatches `claude --print --resume <claudeSessionId>` within a 6h freshness
+window. After **3 strikes** the queue item is set `status: blocked` (so the sweep
+stops re-dispatching it) and an operator can review it — the agent keeps running
+regardless.
+
+### One-time host setup (so recovery is automatic)
+
+```bash
+# Reboot automatically after power loss
+sudo pmset -a autorestart 1
+
+# Auto-login: System Settings → Users & Groups → Automatic Login → <agent user>
+
+# Daemon plist must have RunAtLoad + KeepAlive (generate-plists.sh does this)
+launchctl list | grep ai.maestro
+```
+
+`maestro doctor` checks `pmset autorestart`, the watchdog heartbeat, the daemon
+plist, and the recovery markers — run it after any host change.
+
+### Verifying after a reboot
+
+```bash
+maestro doctor                       # heartbeat, plists, governor, budget, pmset
+npm run cadence:status               # bus depth + consumer heartbeat
+ls state/sessions/resume-pending/    # should drain to empty as work resumes
+tail -f logs/daemon/$(date +%F)-sessions.jsonl
+```
+
+---
+
+## 2. Memory pressure (self-healing, no brick)
+
+The memory watchdog (`scripts/watchdog/memory-watchdog.sh`, every 30s via launchd)
+degrades gracefully through escalating tiers. **It never writes `.emergency-stop`.**
+
+| Tier | Trigger | Action |
+|---|---|---|
+| **SOFT** | sys 70–80% / claude RSS ≥60% | Write `state/throttle.json` with `ceiling = liveCount-2` (auto-expires in 5 min). The resource governor reads it and admits fewer sessions. **No kills.** |
+| **HARD** | sys 80–90% / claude RSS ≥75% | SIGTERM the youngest/biggest subagents, sparing the oldest "main" session (`MIN_AGE_SECONDS=60`). Lowers the throttle ceiling so freed slots aren't instantly refilled. |
+| **CRITICAL** | sys ≥90%, sustained 2 runs | Clean daemon restart via `launchctl kickstart -k` — the daemon re-reads queues + cooldowns + sweeps claims. **A restart, not a halt.** |
+| **escalation** | >3 failed CRITICAL restarts / 15 min | `force-reboot.sh --graceful` — drains, SIGTERMs claude, reboots. `pmset autorestart` brings the box back (§1). |
+
+`state/throttle.json` carries `{ceiling,reason,set_at,expires_at}` with epoch-ms
+timestamps. Because the governor honours `expires_at`, a dead watchdog can't pin
+the throttle — it lifts on its own.
+
+```bash
+scripts/watchdog/memory-watchdog.sh --check    # live tiers + throttle state
+cat state/throttle.json 2>/dev/null || echo "no active throttle"
+tail -20 logs/watchdog/$(date +%F)-watchdog.jsonl
+```
+
+---
+
+## 3. The resource governor (never exceed the host, never drop work)
+
+`lib/resource-governor.mjs` gates **every** spawn (dispatcher + cadence consumer)
+with one decision:
+
+- **ADMIT** — launch now.
+- **QUEUE** — keep it in the in-memory queue (re-derivable from `state/queues/`);
+  it spawns when a slot frees.
+- **DEFER** — leave it on disk untouched; the 10-min backlog sweep / next cadence
+  tick retries it. **DEFER never calls `failTick` / never burns retry budget.**
+
+Dynamic ceiling (replaces the old hardcoded cap):
+`dynamicMax = clamp(floor(totalmem*0.60 / 750MB), 1, min(10, cpus))`, then
+`effectiveMax = min(dynamicMax, throttle.ceiling)`. Thresholds are env-tunable
+(`GOV_*` / `config/recovery.yaml`). Inbox `critical`/`high` work always at least
+QUEUEs and can preempt a running backlog session — **a user message is never
+dropped**.
+
+---
+
+## 4. Rate limits (shared 429 breaker)
+
+`lib/rate-guard.mjs` is one shared circuit breaker per provider
+(`state/rate-limits/<provider>.json`). When any spawn source sees a `429` /
+`rate limit` / `overloaded` in a sub-session's stderr it opens the breaker for a
+decorrelated-jitter backoff (`min(15min, rand(1s, lastBackoff*3))`). While open,
+**all** sources gate their spawns (backlog DEFERs, inbox QUEUEs, cadence requeues
+without burning retry budget). A clean session closes the breaker.
+
+```bash
+cat state/rate-limits/anthropic.json 2>/dev/null   # openUntil / consecutive429
+maestro doctor | grep -i breaker
+```
+
+No action needed — it self-clears. If it's *stuck* open, the provider is likely
+genuinely rate-limiting; wait, or lower concurrency in `config/recovery.yaml`.
+
+---
+
+## 5. Daily budget cap (essential-only, not dark)
+
+`lib/budget-guard.mjs` sums today's cost ledger (`state/cost-tracking/<date>.jsonl`)
+against `DAILY_SPEND_CAP_USD` (default $25). One escalating notice fires at
+50 / 75 / 90 %. At **100 %** the governor goes **essential-only**: backlog +
+cadence-escalation work is DEFERRED, **but inbox DMs are still answered** — the
+agent degrades, it doesn't go dark. Resets at UTC midnight.
+
+```bash
+maestro doctor | grep -i budget
+node scripts/cost/track-claude-usage.mjs summarise --days 1
+```
+
+Raise the cap in `config/recovery.yaml` (`budget.daily_spend_cap_usd`) or via the
+`DAILY_SPEND_CAP_USD` env var.
+
+---
+
+## 6. The human kill switch (`.emergency-stop`)
+
+`.emergency-stop` at the repo root is the **only** thing that permanently stops an
+agent, and it is **human-only**. When present:
+
+- the daemon refuses to start (exits clean so launchd doesn't treadmill),
+- the cadence bus suppresses enqueues (still audited to `queue.jsonl`),
+- the consumer skips all processing.
+
+The watchdog, the governor, the rate breaker, and `force-reboot.sh` **never** write
+it. To stop an agent deliberately:
+
+```bash
+npm run emergency-stop          # writes .emergency-stop + unloads launchd jobs
+```
+
+To resume (the **only** way to clear it):
+
+```bash
+scripts/resume-operations.sh    # health check → rm .emergency-stop → reload plists
+```
+
+`maestro doctor` FAILS while `.emergency-stop` is present, pointing at
+`resume-operations.sh`.
+
+---
+
+## 7. Manual cadence-bus recovery (rarely needed)
+
+The consumer recovers stale claims automatically every 5 min. For an impatient
+operator:
+
+```bash
+npm run cadence:status     # inbox/claimed/dlq depth + heartbeat age
+
+# Force a stale-claim sweep (returns claims >30min to inbox/, or DLQs over budget)
+node -e "import('./lib/cadence-bus.mjs').then(m => console.log(m.recoverStaleClaims(process.cwd())))"
+
+# Inspect the dead-letter queue (events past their retry budget)
+ls state/cadence-bus/dlq/
+node scripts/cadence/enqueue-cadence-tick.mjs <cadence> --source=manual --metadata=note=dlq-replay
+```
+
+Last-resort bus reset (loses pending ticks; they re-arrive next interval, and the
+at-most-once gate means just one catch-up each):
+
+```bash
+ts=$(date -u +%Y-%m-%dT%H-%M-%SZ)
+mkdir -p .maestro/archive/cadence-bus/$ts
+mv state/cadence-bus/{inbox,claimed,dlq} .maestro/archive/cadence-bus/$ts/
+mkdir -p state/cadence-bus/{inbox,claimed,processed,failed,dlq}
+launchctl kickstart -k gui/$(id -u)/ai.maestro.<agent>-daemon
+```
+
+---
+
+## 8. Cases that still need a human
+
+Almost everything self-heals. The genuine human-only cases:
+
+1. **Revoked macOS permissions (TCC).** After some OS updates, Accessibility /
+   Screen Recording / Automation grants reset. Desktop control fails until
+   re-granted in **System Settings → Privacy & Security**. (Headless API/daemon
+   work is unaffected.)
+2. **App re-authentication.** Slack / WhatsApp / Gmail device link or OAuth
+   expiry needs a manual sign-in (e.g. re-scan the WhatsApp QR, re-auth Gmail).
+3. **A human-set `.emergency-stop`** (§6) — cleared only by
+   `resume-operations.sh`.
+4. **Invalid `ANTHROPIC_API_KEY`** in `.env` (401s on every spawn). `maestro
+   doctor` flags it; fix the key or set `MAESTRO_PREFER_SUBSCRIPTION_AUTH=1`.
+5. **Disk full / corrupt state YAML** — `maestro doctor` surfaces it; restore the
+   file from git history (`git checkout HEAD~1 -- <file>`).
+
+---
+
+## 9. Post-incident review
+
+After a significant incident (agent down >1 hr, wrong comms sent, data loss),
+write a short review to `knowledge/decisions/incidents/{date}-{name}.md` and
+commit: timeline, root cause, impact (workflows missed / comms affected / data
+loss), recovery actions, preventive measures. Update this runbook with anything
+learned.
+
+---
+
+Related:
+- `docs/runbooks/perpetual-operations.md` — normal operations + monitoring
+- `docs/runbooks/mac-mini-bootstrap.md` — initial host setup
+- `config/recovery.yaml` — all governance/recovery knobs
+- `CLAUDE.md` → "Cadence Bus" — the bus data flow

package/framework-features.json

@@ -0,0 +1,267 @@
+{
+  "$schema": "https://cohortapp.com/schemas/agent-sdk/framework-features.v1.json",
+  "schema_version": "1",
+  "_doc": "Versioned registry of framework features. Each entry pairs a feature name with its current version, the maestro version that introduced (or last revved) it, and a per-feature init descriptor. `maestro upgrade` reads this file, compares against the agent's .maestro/features.json, and runs `init.command` automatically for features marked `auto: true`. Features with `auto: false` land on the agent's pending-init queue and are surfaced by the SessionStart banner + completed via `maestro init`.",
+  "features": {
+    "cadence-bus": {
+      "version": "1",
+      "since": "1.8.0",
+      "title": "Cadence bus (persistent main session + lightweight launchd enqueue)",
+      "init": {
+        "auto": true,
+        "command": "node scripts/setup/init-cadence-bus.mjs",
+        "description": "Create state/cadence-bus/{inbox,claimed,processed,failed,dlq} + logs/cadence-bus/. Verify daemon plist uses cadence-bus arch."
+      }
+    },
+    "known-agents-registry": {
+      "version": "1",
+      "since": "1.9.0",
+      "title": "Cross-agent registry (peer agent recognition)",
+      "init": {
+        "auto": true,
+        "command": "node scripts/setup/init-known-agents.mjs",
+        "description": "Scaffold config/known-agents.json with the Adaptic agent roster so classifier.mjs can detect peer @mentions and avoid responding to messages routed at another agent."
+      }
+    },
+    "memory-executive-scaffold": {
+      "version": "1",
+      "since": "1.9.0",
+      "title": "Executive memory directory structure",
+      "init": {
+        "auto": true,
+        "command": "node scripts/setup/init-memory-executive.mjs",
+        "description": "Create memory/executive/ and memory/workstreams/ subdirectories with seed files. Referenced by agents/ceo-briefing/ and skill prompts."
+      }
+    },
+    "inputs-overnight-scaffold": {
+      "version": "1",
+      "since": "1.9.0",
+      "title": "Overnight inputs directory",
+      "init": {
+        "auto": true,
+        "command": "mkdir -p inputs/overnight && [ ! -f inputs/overnight/.gitkeep ] && touch inputs/overnight/.gitkeep || true",
+        "description": "Create inputs/overnight/ referenced by morning-brief skill."
+      }
+    },
+    "emergency-stop-paths": {
+      "version": "2",
+      "since": "1.9.0",
+      "title": "Emergency-stop / resume plist path fix",
+      "init": {
+        "auto": true,
+        "command": "true",
+        "description": "Cosmetic version bump — emergency-stop.sh and resume-operations.sh now look in ~/Library/LaunchAgents/ instead of $REPO/schedules/. No agent-side init needed; just receive the updated scripts."
+      }
+    },
+    "rag-foundation": {
+      "version": "2",
+      "since": "1.12.20",
+      "title": "RAG foundation (FTS5 + OpenAI embeddings + hybrid retrieval, hourly reindex)",
+      "init": {
+        "auto": false,
+        "command": "node scripts/setup/init-rag.mjs",
+        "description": "Populate config/repo-registry.yaml, install ai.adaptic.<firstname>-rag-reindex.plist (hourly FTS5 refresh), and optionally run an initial build (--build). Requires interactive --yes for repo selection. Indexer at scripts/rag/build-index.mjs walks memory/, knowledge/, state/queues, config/, agents/, workflows/, outputs/, policies/. Hybrid BM25 + cosine-similarity retrieval via lib/rag/search.mjs. Voice sessions pre-load top-3 RAG hits via lib/voice/context-loader.mjs."
+      }
+    },
+    "decision-capture": {
+      "version": "1",
+      "since": "1.9.0",
+      "title": "Decision capture automation",
+      "init": {
+        "auto": true,
+        "command": "node scripts/setup/init-decision-capture.mjs",
+        "description": "Wire knowledge/decisions/ decision-log automation. Post-action hook detects decision-shaped tool calls and surfaces capture prompts. Schema-validated via knowledge/decisions/decision-schema.yaml."
+      }
+    },
+    "cost-tracking": {
+      "version": "1",
+      "since": "1.9.0",
+      "title": "Cost tracking + quota awareness",
+      "init": {
+        "auto": true,
+        "command": "node scripts/setup/init-cost-tracking.mjs",
+        "description": "Set up state/cost-tracking/, scripts/cost/track-claude-usage.mjs, and daily state/dashboards/cost-summary.yaml. Surfaces burn rate in doctor."
+      }
+    },
+    "session-router": {
+      "version": "1",
+      "since": "1.9.0",
+      "title": "Session router (LRU reuse of long-running Claude sessions)",
+      "init": {
+        "auto": false,
+        "command": "node scripts/setup/init-session-router.mjs",
+        "description": "Enable session-router for backlog / inbox processing. Opt-in: requires SESSION_ROUTER_ENABLED=1 in .env. Init wizard helps configure registry path + LRU caps."
+      }
+    },
+    "backup-replication": {
+      "version": "1",
+      "since": "1.9.0",
+      "title": "Off-machine state backup",
+      "init": {
+        "auto": false,
+        "command": "node scripts/setup/init-backup.mjs",
+        "description": "Configure off-machine backup of state/, knowledge/, outputs/, and rotated logs to GCS or S3. Requires bucket name + credentials. Doctor flags if last successful backup is >24h old once configured."
+      }
+    },
+    "slack-socket-mode": {
+      "version": "1",
+      "since": "1.10.0",
+      "title": "Slack Socket Mode (realtime DM + @mention ingestion)",
+      "init": {
+        "auto": false,
+        "command": "node scripts/setup/init-slack-socket-mode.mjs",
+        "description": "Replace the 60s slack-poller cycle with a persistent WebSocket to Slack. Three steps in the Slack admin UI: (1) open the agent's Slack app at api.slack.com/apps; (2) enable Socket Mode and generate an app-level token (xapp-…) with the connections:write scope; (3) enable Event Subscriptions and subscribe to message.channels, message.groups, message.im, message.mpim, and app_mention. The init wizard prompts for the token, appends it to .env, regenerates the launchd plist, and launchctl-loads it. Doctor verifies the token is present, the plist is loaded, and the listener recently connected."
+      }
+    },
+    "model-router": {
+      "version": "1",
+      "since": "1.12.0",
+      "title": "Model router (cost-optimised LLM routing: Anthropic / Moonshot / Qwen)",
+      "init": {
+        "auto": true,
+        "command": "node scripts/setup/init-model-router.mjs",
+        "description": "Place scaffold/config/model-routing.yaml.example at config/model-routing.yaml.disabled so the operator can opt in. Patches .env.example with MOONSHOT_API_KEY and OPENROUTER_API_KEY placeholders. Routes daemon spawns to the cheapest backend that satisfies each request's capability needs while pinning sensitive roles (CEO pre-pass, audit, decision writers) to Anthropic. Inert until the operator renames .disabled → .yaml; the historical Claude-CLI-on-Max-subscription default is preserved verbatim."
+      }
+    },
+    "voice-realtime": {
+      "version": "2",
+      "since": "1.12.20",
+      "title": "OpenAI Realtime voice (gpt-realtime-2) + Twilio SIP + Railway cloud-relay + outbound Media Streams",
+      "init": {
+        "auto": false,
+        "command": "node scripts/setup/init-voice-realtime.mjs",
+        "description": "Full voice stack scaffold: config/voice.yaml, .env.example placeholders (OpenAI/Twilio/Cloud-relay), logs/voice/* dirs, AND ai.adaptic.<firstname>-{voice-webhook,voice-relay-poller}.plist installed in ~/Library/LaunchAgents (auto-booted via launchctl bootstrap). Voice sessions pre-load identity + top queues + recent decisions + today's calendar + team org chart + RAG hits via lib/voice/context-loader.mjs. Inbound goes through Railway-relay (accepts immediately, 750ms poll cadence for WS attach). Outbound goes through Twilio Media Streams via lib/voice/outbound.mjs. Requires interactive Twilio + Railway setup (docs/guides/voice-mode.md)."
+      }
+    },
+    "channel-bus": {
+      "version": "1",
+      "since": "1.12.0",
+      "title": "Unified channel bus (Slack / WhatsApp / Gmail / Voice via one abstraction)",
+      "init": {
+        "auto": true,
+        "command": "node scripts/setup/init-channel-bus.mjs",
+        "description": "Introduce lib/channels/*: a Channel interface, conversation-resolver, and adapter stubs that consolidate Slack/Gmail/WhatsApp/SMS/Voice behind one abstraction. Existing pollers/senders continue to work; the bus is additive. Enables future channel plugins (Discord, Telegram, iMessage) without touching dispatcher.mjs. Inspired by the openclaw plugin SDK."
+      }
+    },
+    "whatsapp-baileys": {
+      "version": "1",
+      "since": "1.12.0",
+      "title": "WhatsApp via Baileys (multi-modal: media, voice notes, group threads)",
+      "init": {
+        "auto": false,
+        "command": "node scripts/setup/init-whatsapp-baileys.mjs",
+        "description": "Optional WhatsApp upgrade path: replace the Twilio webhook adapter with a Baileys-based persistent session. Adds support for media (images, documents, voice notes), quoted-reply threading, group chats, and reactions. Auth state is persisted under state/whatsapp/auth/ via QR-code onboarding. Voice notes are transcribed via Whisper. Twilio adapter remains available for production-critical or compliance-driven deployments."
+      }
+    },
+    "archetype-library": {
+      "version": "1",
+      "since": "1.13.0",
+      "title": "Function × altitude archetype library (data-driven operating model)",
+      "init": {
+        "auto": true,
+        "command": "node scripts/setup/init-archetype.mjs",
+        "description": "Normalise config/agent.json to a { function, altitude } archetype pair (migrating a legacy single-axis `archetype` value to { function: <same>, altitude: c-suite }), then resolve + validate the operating profile via the framework's archetypes/ library (lib/archetype.mjs). Fails closed if the resolved profile is invalid. The resolved profile (towers, decision rights, autonomy bands, KPI tree, backlog streams, comms profile) is the data spine every downstream generator reads — charter, backlog, cadences, capabilities, comms."
+      }
+    },
+    "self-learning": {
+      "version": "1",
+      "since": "1.16.0",
+      "title": "Self-learning loop (autonomous skill authoring + verbatim session recall)",
+      "init": {
+        "auto": true,
+        "command": "node scripts/setup/init-learning.mjs",
+        "description": "Provision the WS3 self-learning loop: config/learning.yaml (from scaffold), state/learning/counters.json (nudge counters), and the state/learning/sessions.db FTS5 transcript store. After each finished session the Stop-hook reflect worker (scripts/collective/hook-runner.mjs) runs Pass A (collective memory cards) then Pass B (lib/learning/reflect.mjs, sonnet) which may autonomously create or patch a flat Claude Code skill under plugins/agent-skills/ — secret-scanned (lib/security/redact) before write and audited to logs/audit/skills.jsonl. A guarded skill-curator cadence ages skills active→stale→archived (lib/learning/curator.mjs) and warms the transcript index. The session-search skill (scripts/learning/session-search.mjs) recalls past sessions verbatim. Tuning lives in config/learning.yaml; the loop is fail-open + silent on non-agent machines. Requires `maestro global-setup` to wire the global hook."
+      }
+    },
+    "org-cohort": {
+      "version": "1",
+      "since": "1.18.0",
+      "title": "Org / Cohort server enrollment (shared knowledge plane + directory)",
+      "init": {
+        "auto": false,
+        "command": "true",
+        "description": "Enroll this agent in the org's Cohort knowledge plane. Ships config/org.yaml from scaffold (DISABLED by default — org.cohort.enabled:false, empty base/token), so the agent runs fully standalone until configured. lib/org/* (client/protocol/registry/board/leases/handoff/knowledge/keys/verify/policy/cost-sync/approvals) consume it; the Cohort server itself is services/cohort/. enabled flips true only once base + a bearer token are set. Easiest path: `maestro setup --only org` (captures URL + admin-issued token or pairing code, rewrites config/org.yaml). recall()/remember() are fail-open at runtime — an unreachable or disabled server never blocks the agent."
+      }
+    },
+    "org-credential-broker": {
+      "version": "1",
+      "since": "1.18.0",
+      "title": "Org credential broker (org-managed secret layering)",
+      "init": {
+        "auto": false,
+        "command": "true",
+        "description": "Optional: layer org-managed secrets above the local store. lib/secrets/broker.mjs reads config/secrets.yaml broker.{base_url,token_env,identity}; `maestro secrets sync` pulls org secrets into the local provider (fail-closed per secret). Inert until the operator un-comments the broker block in config/secrets.yaml and points it at the org credential broker endpoint (typically the Cohort /secrets binding). No agent-side init beyond receiving the updated config + lib modules on upgrade."
+      }
+    },
+    "secrets-lifecycle": {
+      "version": "1",
+      "since": "1.18.0",
+      "title": "Secret lifecycle (sealed local store + list/rotate/sync)",
+      "init": {
+        "auto": true,
+        "command": "mkdir -p state/secrets logs/audit && true",
+        "description": "Ship config/secrets.yaml from scaffold (provider: file by default — an AES-256-GCM-sealed store under state/secrets/.secrets.json keyed by a first-use master key). Surfaces `maestro secrets <list|sync|rotate>`: list prints NAMES only, rotate reads the new value from stdin (never argv), sync pulls from the org broker when configured. Every mutation is audited names-only to logs/audit/secrets.jsonl; values are never logged. lib/secrets/{broker,providers} are the runtime modules."
+      }
+    },
+    "diagnostics-spine": {
+      "version": "1",
+      "since": "1.18.0",
+      "title": "Diagnostics / observability spine (traces, events, counters)",
+      "init": {
+        "auto": true,
+        "command": "mkdir -p logs/diagnostics logs/diagnostics/counters logs/traces logs/events && true",
+        "description": "Provision the observability spine: lib/diagnostics/{trace,events,counters,backup-freshness,otel}. Structured traces land under logs/traces/, lifecycle events under logs/events/, daily counters under logs/diagnostics/counters/ (rolled by UTC date). counters feed the alerting evaluator and doctor's cost-telemetry tripwire; backup-freshness flags a stale DR backup; otel is the optional OpenTelemetry bridge (inert unless an OTLP endpoint is set). Fail-open: a write error never blocks the agent."
+      }
+    },
+    "alerting": {
+      "version": "1",
+      "since": "1.18.0",
+      "title": "Behavioral alerting (thresholds + webhook sink)",
+      "init": {
+        "auto": true,
+        "command": "true",
+        "description": "Ship config/alerts.yaml from scaffold. The monitoring-alerts cadence evaluates diagnostics counters + today's cost ledger + peer presence + daemon/backup health against the configured thresholds (guardFailSpike, refusalSpike, costPerSessionP99USD, dailySpendWarnUSD, absentAgentSec, daemonHealthStaleSec) and POSTs firing alerts to alerts.webhook (Slack/PagerDuty/Alertmanager-compatible JSON). Omit the webhook to log-only. lib/diagnostics/alerts.mjs is the pure evaluator; unset thresholds fall back to built-in defaults."
+      }
+    },
+    "hook-bus": {
+      "version": "1",
+      "since": "1.18.0",
+      "title": "Hook bus (in-process lifecycle event fan-out)",
+      "init": {
+        "auto": true,
+        "command": "mkdir -p logs/audit/hooks && true",
+        "description": "lib/hooks/bus.mjs — a small, fail-open in-process pub/sub that decouples lifecycle producers (send-gate, scheduler, diagnostics, org sync) from subscribers (counters, audit, alerting). Handlers are isolated: one throwing subscriber never derails the emit or the caller. Audited to logs/audit/hooks/. No config; ships with the lib upgrade."
+      }
+    },
+    "dynamic-scheduling": {
+      "version": "1",
+      "since": "1.18.0",
+      "title": "Dynamic scheduling (runtime jobs + jitter)",
+      "init": {
+        "auto": true,
+        "command": "mkdir -p state/scheduling/jobs && true",
+        "description": "lib/scheduling/{dynamic-jobs,jitter} — register/cancel one-off and recurring jobs at runtime (persisted under state/scheduling/jobs/) without rewriting launchd plists, and spread enqueue times with decorrelated jitter so fleet ticks don't thunder. Complements the cadence bus (which still owns the fixed launchd cadences). Fail-open: a corrupt job record is skipped, not fatal."
+      }
+    },
+    "send-gate": {
+      "version": "1",
+      "since": "1.18.0",
+      "title": "Outbound send-gate (pre-send policy choke point)",
+      "init": {
+        "auto": true,
+        "command": "true",
+        "description": "lib/comms/send-gate.mjs — the single programmatic choke point every outbound message passes through before a channel adapter sends it (banned-phrase / communication-style enforcement, identity-token leak guard, dedupe, rate awareness). Complements the pre-send-audit hook (which gates MCP-driven sends) by covering library-initiated sends. Fail-closed on a hard policy violation, fail-open on evaluator errors so a transient fault never silences the agent."
+      }
+    },
+    "knowledge-plane": {
+      "version": "1",
+      "since": "1.18.0",
+      "title": "Org knowledge plane (recall / remember over the fleet)",
+      "init": {
+        "auto": false,
+        "command": "true",
+        "description": "lib/org/knowledge.mjs — recall()/remember() over the org's shared knowledge plane (backed by the Cohort server). Once the agent is enrolled (org-cohort feature: config/org.yaml enabled with base + token), the daemon's prompt-builder can consult recall() so replies are informed by what the fleet already knows, and finished work can remember() durable facts. Server-side ACL governs visibility. Inert + fail-open until enrollment; no standalone init."
+      }
+    }
+  }
+}

package/ingest/README.md

@@ -0,0 +1,87 @@
+# Ingestion Pipeline
+
+This directory contains configuration and processing artefacts for the Maestro agent's information ingestion pipeline. The pipeline is responsible for incorporating new documents, repository changes, and external information into the knowledge base.
+
+## Pipeline Flow
+
+```
+Source Identification
+        |
+        v
+Content Extraction
+        |
+        v
+Relevance Assessment
+        |
+        v
+Entity Extraction
+        |
+        v
+Memory Update
+        |
+        v
+Synthesis Generation
+```
+
+### 1. Source Identification
+
+New information sources are identified through:
+
+- Changes detected in registered source repositories (see `knowledge/sources/source-registry.yaml`)
+- Manual addition of new documents, emails, or meeting notes
+- Alerts from monitored external sources (regulatory updates, market data, news)
+
+Each source is validated against the source registry. New sources are evaluated for authority and relevance before being added to the registry.
+
+### 2. Content Extraction
+
+Raw content is extracted from the source in a structured format:
+
+- **Code repositories:** Diffs, new files, commit messages, PR descriptions
+- **Documents:** Full text extraction with metadata (author, date, version)
+- **Emails/communications:** Sender, recipients, subject, body, attachments
+- **Regulatory filings:** Structured fields, requirements, deadlines, conditions
+
+### 3. Relevance Assessment
+
+Extracted content is evaluated for relevance to active strategic domains:
+
+- Scored against active initiatives and open loops in executive memory
+- Classified by domain (regulatory, M&A, technology, fundraising, etc.)
+- Prioritised by urgency and strategic importance
+- Low-relevance items are logged but not promoted to active memory
+
+### 4. Entity Extraction
+
+Named entities are identified and reconciled against the entity map (`knowledge/entities/entity-map.yaml`):
+
+- People, companies, regulators, products, and targets
+- New entities are flagged for addition to the entity map
+- Relationships between entities are captured and updated
+
+### 5. Memory Update
+
+Relevant information is used to update executive memory (`knowledge/memory/executive-memory.yaml`):
+
+- Initiative status changes
+- New open loops or loop closures
+- Relationship updates (new contacts, last-contact dates)
+- Regulatory dependency changes
+- Pipeline progression (fundraising, hiring, rollup targets)
+
+### 6. Synthesis Generation
+
+Updated information is incorporated into the next scheduled synthesis cycle:
+
+- Critical updates trigger immediate digest generation
+- Routine updates are batched into the next daily digest
+- Domain-significant changes are flagged for the next weekly synthesis
+
+## Adding New Sources
+
+To add a new information source to the pipeline:
+
+1. Add an entry to `knowledge/sources/source-registry.yaml` with path, type, domains, and freshness check method.
+2. Run an initial ingestion pass on the source.
+3. Update the entity map with any new entities discovered.
+4. Verify that executive memory reflects any relevant state captured from the new source.