@cohortapp/agent-sdk 2.3.0

package/docs/guides/setup-wizard.md

@@ -0,0 +1,178 @@
+# Setup Wizard Guide (`maestro setup`)
+
+How to configure a Maestro agent with `maestro setup` — the deterministic,
+resumable, self-verifying wizard that brings a freshly-scaffolded repo online.
+
+## 1. What `maestro setup` is (and why it replaced `/init-maestro`)
+
+`maestro create <dir>` is a pure file copy — it never asks a question. Everything
+that turns that skeleton into a working agent (identity, company context, model
+choice, channels, the operating model, and verification) happens in one place:
+
+```bash
+cd jacob-ai
+maestro setup
+```
+
+The previous onboarding was `/init-maestro`, a ~1,200-line Markdown prompt that
+handed the entire job to an LLM in a single Claude Code session. That worked, but
+it was non-deterministic, hard to resume if it stalled halfway, and impossible to
+run in CI. `maestro setup` inverts the split: **Node code drives the mechanical
+orchestration, and the LLM is a bounded function** invoked only to write
+company-specific prose. The result is repeatable, checkpointed, and self-checking.
+
+`claude "/init-maestro"` still exists as a manual, LLM-driven fallback — but
+`maestro setup` is the supported path for every new agent.
+
+## 2. The section model
+
+The wizard is a registry of **sections**. `lib/setup/runner.mjs` globs
+`lib/setup/sections/*.mjs`, sorts them by their `order`, and runs each in turn.
+Every section implements the same small contract:
+
+```js
+{
+  id, label, order,
+  detect(ctx),   // → { status: 'unconfigured' | 'partial' | 'complete', summary?, missing? }
+  prompt(ctx),   // the ONLY method that touches the TTY (honors --headless)
+  apply(ctx),    // writes the source of truth + runs generators; idempotent
+  verify(ctx),   // → { ok, checks: [{ name, ok, remedy? }] }
+}
+```
+
+Because `detect`, `apply`, and `verify` never block on input, the same sections
+power `--headless` runs, `maestro doctor`, and CI. New sections register simply by
+dropping a file into `lib/setup/sections/` — the messaging, learning, and recovery
+sections (below) are added by their respective workstreams without editing the
+core runner.
+
+### Sections, in run order
+
+| Section | Order | What it does |
+|---------|-------|-------------|
+| `identity` | 10 | Agent name and title, the **function × altitude** archetype, and the principal. Writes `config/agent.json`. |
+| `company` | 20 | Company-context interview — name, website, industry, one-line + detailed overview, stage/size, footprint, regulation, top priorities, and key people. Writes `config/company.json`. |
+| `model` | 30 | Model/provider choice: a Claude Code subscription (keychain OAuth, no API key) or a pasted Anthropic API key. Writes the model lines of `.env`. |
+| `comms` | 45 | Wires each messaging channel — Slack, Gmail, SMS, WhatsApp, Telegram, voice — into `.env`/gate files and verifies inbound. |
+| `tools` | 50 | Selects which channels and MCP servers the agent should run. |
+| `operating-model` | 60 | Deterministically generates the operating charter, a seeded WBS backlog (≥5 open items), 40–60 sub-agents + skills + workflows + MCP servers + event-routing, archetype cadences, the autonomy policy, the communication profile, and the launchd plists. |
+| `enrich` | 70 | Fills the company-specific prose slots via bounded LLM calls (see §4). |
+| `learning` | 80 | Enables the self-learning loop and seeds its counters/DB. |
+| `recovery` | 85 | Installs the memory watchdog + resource governor and verifies recovery behaviour. |
+| `verify` | 90 | Runs the live capability probe and prints the capability table (see §5). |
+
+`maestro setup --help` lists the core sections (`identity company model comms
+tools operating-model enrich verify`); `learning` and `recovery` are contributed
+by the self-learning and recovery workstreams and appear once their files are
+present.
+
+## 3. Resume, checkpoints, and the source of truth
+
+The wizard writes a checkpoint to `state/setup/progress.json` after each section —
+per-section status, the verify result, and an `answersHash`. When you re-run
+`maestro setup`, it **re-`detect`s every section from disk first**: the on-disk
+state is authoritative and the checkpoint is only a hint. It then continues at the
+first section that isn't `complete`. So if a run stalls (you Ctrl-C'd, a token was
+wrong, the machine slept), just run `maestro setup` again — it picks up where the
+files actually left off, not where the checkpoint last claimed.
+
+**One source of truth.** `config/agent.json` is the sole identity SoT.
+`config/agent.ts` is a generated thin wrapper that re-exports it (with a few typed
+computed values) — never hand-edit the `.ts`. Editing `config/agent.json` and
+re-running the relevant section (or `node scripts/setup/generate-agent-env.mjs`)
+is the supported way to change identity after setup. This kills the old
+split-brain where `agent.ts` and the JSON snapshot could drift.
+
+## 4. Enrichment (the bounded LLM step) and `--no-enrich`
+
+The deterministic generators leave **prose slots** that genuinely need a model —
+the agent persona, its responsibilities and operating principles, the company
+description, and the company-specific lines in the charter/backlog. The `enrich`
+section fills them by spawning **one `claude --print` per enrichment unit** under a
+strict **field allowlist**: the LLM may only return values for the enumerated
+fields, as strict JSON. Output is validated (right keys, no banned phrases per
+`policies/communication-style.md`, no residual `{{tokens}}`). If a unit fails
+validation, the deterministic skeleton stays in place, the field is marked
+`pending`, and setup continues — enrichment is best-effort and never blocks.
+
+Pass `--no-enrich` to skip this entirely and keep the skeletons (useful in CI or
+when you want to write the prose by hand). You can always fill them later by
+re-running `maestro setup` (or `maestro setup enrich`) without the flag.
+
+## 5. The verify probe and capability table
+
+The final `verify` section runs a live check and prints an `N/M` capability table.
+It aggregates every other section's `verify()` plus a Claude API probe and the
+completeness gate. Per enabled channel/MCP it runs the right liveness check
+(`auth.test` for Slack, an IMAP login for Gmail, a Twilio lookup, a relay
+`/health`, etc.), and prints a remedy line under any failure:
+
+```
+Capability summary — 7/9 checks passing
+  ✓ Claude API reachable
+  ✓ identity complete
+  ✓ Slack auth.test
+  ✗ Gmail IMAP login        [comms]
+      ↳ set GMAIL_APP_PASSWORD in .env (see docs/guides/email-setup.md)
+  ...
+```
+
+Setup exits **non-zero only on hard failures** — no identity, an unresolvable
+archetype, or the Claude API being unreachable. Optional channels and best-effort
+enrichment are reported but never fail the run. The same completeness gate backs
+`maestro doctor`, so `doctor` and `setup --status` agree on what "done" means.
+
+## 6. Flags
+
+```bash
+maestro setup [section] [flags]
+```
+
+| Flag | Effect |
+|------|--------|
+| *(positional)* `section` | Run just one section by id, e.g. `maestro setup comms`. |
+| `--resume` | Continue from the checkpoint (the default when one exists). Re-detects from disk first. |
+| `--from <id>` | Start at section `<id>` and continue through the rest. |
+| `--only <id,id,…>` | Run only these sections — forced, even if they're already complete. |
+| `--status` | Print each section's `detect()` status and exit (writes nothing). |
+| `--headless` | Never prompt; take answers from `--answers` / env / defaults. For CI and scripted runs. |
+| `--answers <file.json>` | A JSON file of pre-supplied answers, consumed under `--headless`. |
+| `--no-enrich` | Skip LLM enrichment; leave the deterministic skeletons. |
+| `--dry-run` | Print what would run and write nothing. |
+
+### Examples
+
+```bash
+# Full interactive setup of a fresh repo
+maestro setup
+
+# See what's done without changing anything
+maestro setup --status
+
+# Re-run just the channels section (e.g. after adding a Telegram token)
+maestro setup comms
+
+# Re-do identity + company from a known point, forced
+maestro setup --only identity,company
+
+# Non-interactive, scripted bring-up (CI)
+maestro setup --headless --answers ./setup-answers.json --no-enrich
+
+# Preview the plan
+maestro setup --dry-run
+```
+
+## 7. Migrating an existing agent
+
+`maestro upgrade` auto-migrates a legacy standalone `config/agent.ts` (one that
+defined the config inline rather than re-exporting JSON) into the unified model:
+it derives `config/agent.json` and replaces `config/agent.ts` with the generated
+wrapper. After upgrading, `maestro setup --status` shows where the repo stands;
+run `maestro setup` to fill anything the migration left `pending`.
+
+## 8. Related documents
+
+- [Agent Persona Setup](agent-persona-setup.md) — the per-file persona reference (charter, contacts, priorities, control towers) the `identity`/`operating-model` sections populate
+- [Telegram Setup](telegram-setup.md) — the channel the `comms` section can wire
+- [Channel bus](channel-bus.md) — the unified messaging layer behind the `comms`/`tools` sections
+- [Recovery and Failover](../runbooks/recovery-and-failover.md) — what the `recovery` section installs

package/docs/guides/slack-setup.md

@@ -0,0 +1,350 @@
+# Slack Setup Guide
+
+How to enable Slack integration for a Maestro agent: dedicated app creation, token configuration, message sending (with typing indicators), file uploads, emoji reactions, real-time events via Railway relay, and Slack CDP for huddle automation.
+
+> **Hard requirement: every agent gets its own dedicated Slack app.** Do NOT share a Slack app across agents. Each app has its own bot user, signing secret, OAuth tokens, and scope set. Sharing breaks identity (messages appear as the wrong agent), confuses signature verification on the webhook relay, and prevents per-agent revocation. The init-maestro wizard creates a new app for each agent via the Slack manifest API.
+
+**Prerequisites**: Complete the [Mac Mini Bootstrap](../runbooks/mac-mini-bootstrap.md). The agent needs a Slack workspace account with admin rights to create apps.
+
+---
+
+## Architecture Overview
+
+```
+┌──────────────────────────────────────────────────────────────────────┐
+│  INBOUND                                                              │
+│                                                                       │
+│  Slack workspace ──▶ slack-poller.mjs ──▶ state/inbox/slack/*.yaml   │
+│  (via xoxp- token)   (every 60s)          (inbox processor routes)   │
+│                                                                       │
+│  OR (real-time):                                                      │
+│  Slack Events API ──▶ slack-events-server.mjs ──▶ inbox / daemon     │
+│  (webhook)             (Socket Mode or HTTP)                          │
+├──────────────────────────────────────────────────────────────────────┤
+│  OUTBOUND                                                             │
+│                                                                       │
+│  ┌───────────────┐  ┌─────────────────┐  ┌──────────────────┐       │
+│  │ slack-send.sh  │  │ slack-upload-   │  │ slack-react.mjs  │       │
+│  │ (messages)     │  │ v2.py (files)   │  │ (emoji reactions)│       │
+│  └──────┬────────┘  └────────┬────────┘  └──────────────────┘       │
+│         │                    │                                        │
+│  ┌──────▼────────────────────▼──────────────────────────────────┐   │
+│  │  Pre-send pipeline:                                           │   │
+│  │  1. validate-outbound.py (factual checks)                     │   │
+│  │  2. slack-responded.sh   (atomic response dedup)              │   │
+│  │  3. pre_draft_lookup.py  (context enrichment)                 │   │
+│  │  4. slack-typing.mjs     (typing indicator)                   │   │
+│  │  5. markdown → mrkdwn    (format conversion)                  │   │
+│  └──────────────────────────────────────────────────────────────┘   │
+│         │                                                            │
+│         ▼                                                            │
+│    Slack Web API (chat.postMessage) via xoxp- User Token            │
+└──────────────────────────────────────────────────────────────────────┘
+```
+
+**Critical design decision**: All Slack messages are sent via the **User OAuth Token (xoxp-)**, not the Bot Token (xoxb-). Messages sent with the user token appear as the agent's Slack user account (e.g. "Jordan Lee"), not as a bot app. The `block-mcp-slack-send.sh` hook enforces this — MCP Slack sends are blocked because they add a "Sent using @Claude" label.
+
+---
+
+## 1. Slack App Creation
+
+### 1.1 Create a Slack App
+
+1. Go to https://api.slack.com/apps → **Create New App**
+2. Choose "From scratch"
+3. Name it (e.g. "Northwind Agent") and select your workspace
+4. Note the **App ID** from Basic Information
+
+### 1.2 Configure OAuth Scopes
+
+Under **OAuth & Permissions** → **Scopes**, add:
+
+**Bot Token Scopes** (xoxb-):
+
+| Scope | Purpose |
+|---|---|
+| `chat:write` | Send messages |
+| `channels:read` | List channels |
+| `groups:read` | List private channels |
+| `im:read` | List DM conversations |
+| `users:read` | Look up user info |
+| `files:write` | Upload files |
+| `reactions:write` | Add emoji reactions |
+
+**User Token Scopes** (xoxp-):
+
+| Scope | Purpose |
+|---|---|
+| `channels:history` | Read channel messages |
+| `groups:history` | Read private channel messages |
+| `im:history` | Read DM messages |
+| `search:read` | Search messages |
+| `chat:write` | Send as user (primary send method) |
+| `files:write` | Upload as user |
+| `reactions:write` | React as user |
+
+### 1.3 Install the App
+
+1. Click **Install to Workspace**
+2. Authorize the requested permissions
+3. Copy both tokens:
+   - **Bot User OAuth Token** (starts with `xoxb-`)
+   - **User OAuth Token** (starts with `xoxp-`)
+
+### 1.4 Configure Environment Variables
+
+Add to `.env`:
+
+```bash
+# User token (xoxp-) — used for sending messages as the agent's user account
+SLACK_USER_TOKEN=xoxp-...
+
+# Bot token (xoxb-) — used for API calls that don't need user identity
+SLACK_BOT_TOKEN=xoxb-...
+
+# Optional: signing secret for Events API webhook verification
+SLACK_SIGNING_SECRET=...
+```
+
+---
+
+## 2. Sending Messages (`slack-send.sh`)
+
+The primary script for all Slack message sending.
+
+### 2.1 Usage
+
+```bash
+# Send to a channel
+./scripts/slack-send.sh "C1234567890" "Hello from the agent"
+
+# Reply in a thread
+./scripts/slack-send.sh "C1234567890" "Thread reply" --thread_ts "1234567890.123456"
+
+# With response dedup (prevents double-replying to the same message)
+./scripts/slack-send.sh "C1234567890" "Reply" --responding_to "1234567890.123456"
+
+# Skip typing indicator
+./scripts/slack-send.sh "C1234567890" "Quick message" --no-typing
+```
+
+### 2.2 Pre-Send Pipeline
+
+Every Slack send goes through this pipeline (in order):
+
+1. **Response dedup** (`slack-responded.sh`): If `--responding_to` is set, acquires an atomic mkdir-based lock to prevent concurrent sessions from replying to the same message
+2. **Typing indicator** (`slack-typing.mjs`): Shows "Agent is typing..." for a duration proportional to message length (1.5s–4s)
+3. **Pre-draft context lookup** (`pre_draft_lookup.py`): Looks up recipient in entity index (advisory, never blocks)
+4. **Factual validation** (`validate-outbound.py`): Checks for relationship claims, AI disclosure, scheduling errors — **blocks if critical issues found**
+5. **Markdown → mrkdwn conversion**: Converts standard Markdown (`**bold**`, `[link](url)`, `## heading`) to Slack's mrkdwn format (`*bold*`, `<url|link>`, `*heading*`)
+6. **Send via `chat.postMessage`**: Uses User OAuth Token
+
+### 2.3 Typing Indicators (`slack-typing.mjs`)
+
+Creates a natural feel by showing typing dots before messages arrive:
+
+- Duration scales with message length: <50 chars → 1.5s, 50-200 → 2.5s, 200-500 → 3s, >500 → 4s
+- Connects via WebSocket RTM API using the user token
+- Requires `rtm:stream` scope (degrades gracefully if missing — message sends without dots)
+- Disable globally: `SLACK_TYPING_ENABLED=0` in `.env`
+
+### 2.4 Response Deduplication (`slack-responded.sh`)
+
+Prevents the agent from sending multiple replies to the same message:
+
+- Uses atomic `mkdir` for race-safe locking (POSIX-guaranteed atomic)
+- Lock path: `state/locks/slack-responded/{channel}/{message_ts}/`
+- After send succeeds, records a preview of the sent message for audit
+- Lock TTL handled by `outbound-dedup-cleanup.sh`
+
+---
+
+## 3. File Uploads (`slack-upload-v2.py`)
+
+Uploads files to Slack channels or threads:
+
+```bash
+python3 scripts/slack-upload-v2.py --channel "C1234567890" --file /path/to/document.pdf --title "Q2 Board Pack"
+
+# Upload to a thread
+python3 scripts/slack-upload-v2.py --channel "C1234567890" --file report.pdf --thread_ts "1234567890.123456"
+```
+
+Uses Slack's v2 file upload API (`files.uploadV2`) which supports larger files and better threading.
+
+---
+
+## 4. Emoji Reactions (`slack-react.mjs`)
+
+Adds emoji reactions to messages:
+
+```bash
+node scripts/slack-react.mjs --channel "C1234567890" --timestamp "1234567890.123456" --emoji "white_check_mark"
+```
+
+Used by the agent to acknowledge messages (e.g. ✅ after processing a request).
+
+---
+
+## 5. Event Polling (`slack-poller.mjs`)
+
+### 5.1 How It Works
+
+The Slack poller runs as part of the main poller loop (every 60 seconds):
+
+1. Reads recent messages from monitored channels and DMs using `conversations.history`
+2. Filters for new messages since last poll (tracks last-seen timestamps)
+3. Writes new messages to `state/inbox/slack/` as YAML files
+4. Inbox processor classifies and routes each message
+
+### 5.2 Monitored Channels
+
+The poller monitors channels configured in the agent's operational setup. Typically:
+- All DM conversations
+- Channels where the agent is mentioned
+- Priority channels (CEO DM, team channels)
+
+### 5.3 Priority Detection
+
+CEO DMs and messages containing urgent keywords trigger immediate processing rather than waiting for the next inbox processor cycle.
+
+---
+
+## 6. Real-Time Events Server (`slack-events-server.mjs`)
+
+### 6.1 Overview
+
+For faster event detection than polling, the events server receives Slack events in real-time:
+
+```bash
+node scripts/slack-events-server.mjs
+```
+
+### 6.2 Control Script
+
+```bash
+# Start the events server
+./scripts/slack-events-ctl.sh start
+
+# Stop it
+./scripts/slack-events-ctl.sh stop
+
+# Check status
+./scripts/slack-events-ctl.sh status
+```
+
+### 6.3 Configuration
+
+Requires the Slack app to have **Event Subscriptions** enabled:
+
+1. Go to Slack App settings → Event Subscriptions → Enable Events
+2. Request URL: `https://your-tunnel-url/slack/events`
+3. Subscribe to events: `message.channels`, `message.groups`, `message.im`, `app_mention`
+4. The events server verifies requests using `SLACK_SIGNING_SECRET`
+
+---
+
+## 7. MCP Slack Send Block
+
+The `block-mcp-slack-send.sh` hook prevents sending via MCP Slack:
+
+```
+BLOCKED: Per CEO directive si-010 — do NOT use MCP Slack for sending.
+Use scripts/slack-send.sh with SLACK_USER_TOKEN instead.
+```
+
+**Why**: MCP Slack sends add a "Sent using @Claude" label to messages, breaking the agent's identity. All sends must go through `slack-send.sh` using the User OAuth Token so messages appear as the agent's Slack user account.
+
+This hook is registered in `.claude/settings.json` as a `PreToolUse` hook matching MCP Slack send tools.
+
+---
+
+## 8. Slack CDP (Chrome DevTools Protocol)
+
+For advanced Slack UI automation (huddle participation, keyboard shortcuts), the Slack desktop app is launched with CDP enabled:
+
+```bash
+./scripts/huddle/launch-slack.sh
+# or manually:
+/Applications/Slack.app/Contents/MacOS/Slack --remote-debugging-port=9222
+```
+
+See [Voice & SMS Setup](voice-sms-setup.md) for the full huddle voice infrastructure that builds on Slack CDP.
+
+---
+
+## 9. Testing
+
+| # | Test | How to Verify |
+|---|---|---|
+| 1 | Token validation | `curl -s -H "Authorization: Bearer $SLACK_USER_TOKEN" https://slack.com/api/auth.test \| jq .` |
+| 2 | Simple send | `./scripts/slack-send.sh "C_TEST_CHANNEL" "Test message"` |
+| 3 | Thread reply | Send with `--thread_ts` and verify it threads in Slack |
+| 4 | Typing indicator | Watch Slack while sending — "Agent is typing..." should appear |
+| 5 | Response dedup | Send same `--responding_to` twice; second should `DEDUP_SKIP` |
+| 6 | File upload | `python3 scripts/slack-upload-v2.py --channel C_TEST --file test.txt` |
+| 7 | Emoji reaction | `node scripts/slack-react.mjs --channel C_TEST --timestamp TS --emoji thumbsup` |
+| 8 | MCP block | Attempt MCP Slack send — should be blocked by hook |
+| 9 | Poller | Send a DM to the agent; check `state/inbox/slack/` |
+| 10 | Markdown conversion | Send message with `**bold**` and `[link](url)` — verify Slack renders correctly |
+
+---
+
+## 10. Troubleshooting
+
+### "invalid_auth" or "token_revoked" errors
+
+1. Regenerate tokens: Slack App → OAuth & Permissions → Reinstall to Workspace
+2. Verify token type: user sends need `xoxp-` token, not `xoxb-`
+3. Update `.env` with new tokens
+
+### Messages appearing as bot (not user)
+
+1. Verify `SLACK_USER_TOKEN` starts with `xoxp-` (not `xoxb-`)
+2. Check `slack-send.sh` is using `SLACK_USER_TOKEN` (line 12)
+3. Verify `block-mcp-slack-send.sh` hook is active in `.claude/settings.json`
+
+### Typing indicator not showing
+
+1. Check `rtm:stream` scope on user token (not available on all Slack plans)
+2. Verify `SLACK_TYPING_ENABLED` is not set to `0`
+3. Check `slack-typing.mjs` can connect: `node scripts/slack-typing.mjs C_TEST 2000`
+4. Degrades gracefully — messages still send without typing dots
+
+### Poller missing messages
+
+1. Verify channel access: agent's Slack account must be a member of monitored channels
+2. Check `conversations.history` scope is granted
+3. Check last-poll timestamp in poller state isn't in the future (clock sync issue)
+
+### Response dedup false positives
+
+1. Check lock directory: `ls state/locks/slack-responded/CHANNEL/`
+2. Clean stale locks: `./scripts/outbound-dedup-cleanup.sh`
+3. Locks should auto-expire — check TTL in `outbound-dedup.sh`
+
+---
+
+## Key Files
+
+| File | Purpose |
+|---|---|
+| `scripts/slack-send.sh` | Primary message sender (user token, typing, dedup, markdown) |
+| `scripts/slack-typing.mjs` | WebSocket RTM typing indicator |
+| `scripts/slack-react.mjs` | Add emoji reactions to messages |
+| `scripts/slack-upload-v2.py` | File upload (v2 API) |
+| `scripts/slack-responded.sh` | Atomic response deduplication |
+| `scripts/slack-events-server.mjs` | Real-time Slack events receiver |
+| `scripts/slack-events-ctl.sh` | Events server control (start/stop/status) |
+| `scripts/poll-slack-events.sh` | Legacy event polling script |
+| `scripts/hooks/block-mcp-slack-send.sh` | Blocks MCP Slack sends (enforces user token) |
+| `scripts/poller/slack-poller.mjs` | Slack channel/DM polling |
+
+---
+
+## Related Documents
+
+- [Voice & SMS Setup](voice-sms-setup.md) — Slack huddle voice via CDP
+- [Outbound Governance Setup](outbound-governance-setup.md) — Dedup and validation pipeline
+- [Poller & Daemon Setup](poller-daemon-setup.md) — How Slack polling integrates with the event loop
+- [Communications Policy](../governance/communications-policy.md) — Voice modes and approval rules for Slack