@adaptic/maestro 1.1.8 → 1.5.0

package/.claude/commands/init-maestro.md

@@ -297,7 +297,7 @@ Do NOT modify these sections (keep them exactly as they are, except for agent na
    - product-leader: product roadmap, user research, feature delivery, product-market fit, design system
    - operations-leader: process automation, operational efficiency, fund operations, organisational design, vendor management
 
-### Sub-agent 4: Update package.json and scripts
+### Sub-agent 4: Update package.json, scripts, and identity-baked content
 
 **Instruction to sub-agent:**
 
@@ -307,9 +307,57 @@ Do NOT modify these sections (keep them exactly as they are, except for agent na
    - Variable names like `SOPHIE_AI_DIR` -> `{UPPER_FIRSTNAME}_AI_DIR`
    - Path references like `/Users/sophie/sophie-ai` -> `/Users/{lowercase-firstname}/{repoSlug}`
    - LaunchD labels like `ai.adaptic.sophie-` -> `ai.adaptic.{lowercase-firstname}-`
+   - Pronouns: if the new agent's gender differs from the scaffolding template, update he/she/him/her/his/hers/himself/herself across system prompts, comments, and documentation. Be surgical — do NOT change pronouns inside generic regex patterns or third-party detection logic.
 
 3. **LaunchD plists** in `scripts/local-triggers/plists/` -- Update labels and paths in all `.plist` files.
 
+4. **Identity-baked content rewrites (CRITICAL — full overwrites, not grep-replace).** These files contain the agent's outbound identity (name, title, email, phone, signature) and MUST be fully rewritten with the new agent's values. Do not rely on grep-replace alone — read each file, then OVERWRITE it with content that uses these exact values:
+
+   - `firstName + lastName` (e.g., "Lucas Ferreira")
+   - `title` (e.g., "VP, Regulatory & Licensing")
+   - `email` (e.g., "lucas@adaptic.ai")
+   - `phone` (e.g., "+61 478 964 324" — use the spaced pretty form for human-facing display, the E.164 form for code)
+   - `companyName` (e.g., "Adaptic.ai")
+   - `companyAddress` (use the company's primary office address — typically Adaptic's DIFC office unless explicitly different)
+
+   **Files to rewrite:**
+
+   a. **`scripts/email-signature.html`** — The HTML signature appended to all outbound emails by `send-email.sh` and the Python send scripts. Must contain: name (bold, 14px), title (grey, 13px), Adaptic logo (`https://adaptic.ai/logo.png`), email, phone, company address line, full confidentiality disclaimer footer. Pattern matches the template in `~/maestro/scripts/email-signature.html` — use placeholders {{AGENT_NAME}}, {{AGENT_TITLE}}, {{AGENT_EMAIL}}, {{AGENT_PHONE}}, {{COMPANY_ADDRESS}} and substitute them.
+
+   b. **`scripts/email-signature-mehran.html`** — Principal's signature block (used by `send-email-as-mehran.py` or equivalent send-as-principal scripts). Update with the principal's values: `principal.fullName`, `principal.title`, `principal.email`. If the principal doesn't have a phone in config/agent.ts, omit the phone line.
+
+   c. **`scripts/send-email.sh`** — Hardcoded `From:` header and inline signature fallback. Update both. The From header should be in the form `"Lucas Ferreira" <lucas@adaptic.ai>`.
+
+   d. **`scripts/send-email-threaded.py`** — `USER`, `From` header construction, inline signature, argparse description. All must reflect the new agent.
+
+   e. **`scripts/send-email-with-attachment.py`** — Same as above.
+
+   f. **`scripts/pdf-generation/build-document.mjs`** — Default `author` value (used in PDF metadata) and the help text describing the default. Set to the new agent's full name.
+
+   g. **`scripts/pdf-generation/templates/memo.latex`** — Footer line "Prepared by ... Chief of Staff" — replace with "Prepared by {fullName}, {title}".
+
+   h. **`scripts/daemon/responder.mjs` `FALLBACK_PREAMBLE`** — System prompt that introduces the agent to Claude. Identity intro line must reference the new agent. Preserve the operational rules.
+
+   i. **`scripts/daemon/prompt-builder.mjs` `FALLBACK_PREAMBLE`** — Same treatment.
+
+   j. **`scripts/daemon/classifier.mjs` `SYSTEM_PROMPT`** — Identity intro line. Preserve everything else.
+
+   k. **`scripts/huddle/huddle-server.mjs` `HUDDLE_SYSTEM_PROMPT`** — Voice agent identity line.
+
+   l. **`scripts/spawn-session.sh`** — Sub-session bootstrap prompt that names the agent.
+
+   m. **`scripts/continuous-monitor.sh`** — Channel monitor agent prompt.
+
+   n. **`scripts/llm_email_dedup.py`, `scripts/comms-monitor.sh`, `scripts/archive-email.sh`, `scripts/poller/gmail-poller.mjs`, `scripts/poller/imap-client.mjs`** — Hardcoded `LUCAS_EMAIL`/`USER`/`gmail_user` constants. Set to the new agent's email.
+
+   o. **`scripts/{firstname}-inbox-poller.py`** — Rename file from `sophie-inbox-poller.py` (or current scaffolding name) to `{firstname}-inbox-poller.py`. Update internal `LUCAS_EMAIL` constant. Update any plist references to the new filename.
+
+   p. **`scripts/rag-indexer.py`, `scripts/user-context-search.py`** — Author docstring at the top. Set to the new agent's full name.
+
+   q. **`scripts/validate-outbound.py`** — Three test/regex references to "Sophie Nguyen" or `lookup_entity("Sophie Nguyen")`. Replace with the new agent's full name. Leave the generic third-party pronoun regex (around line 1007) UNCHANGED — it's a detector, not an identity reference.
+
+   **Verification step**: After rewrites, run `grep -rn "sophie@adaptic\|Sophie Nguyen\|Chief of Staff" scripts/ docs/business-synthesis/executive-operating-model.md 2>&1` and confirm only intentional peer references remain (Sophie Nguyen as a real Chief of Staff peer, NOT as the agent's own outbound identity). Report any remaining matches you couldn't safely auto-resolve so the main agent can decide.
+
 ### Sub-agent 5: Update agent definitions
 
 **Instruction to sub-agent:**
@@ -436,6 +484,123 @@ If yes, run:
 sudo ./scripts/setup/configure-macos.sh
 ```
 
+### Step 4: External SSD configuration (REQUIRED if /Volumes/{name}-SSD is mounted)
+
+The maestro daemon and its launchd-spawned trigger jobs should write all runtime data — Claude Code per-cwd temp dirs, daemon logs, state, outputs, memory, knowledge — to an external SSD when one is available. This keeps the internal disk free for macOS and avoids wear on the system disk.
+
+**Two macOS hurdles need to be cleared before the SSD redirect actually works for launchd-spawned processes:**
+
+#### 4a. Enable file ownership on the volume
+
+By default, external volumes have Owners disabled, which makes file permissions advisory rather than enforced. The daemon's wrapper writes per-agent log files, and that requires real owners.
+
+Detect the SSD and enable owners:
+
+```bash
+SSD_VOLUME=""
+for v in /Volumes/*-SSD /Volumes/*SSD* /Volumes/maestro-data; do
+  if [ -d "$v" ] && [ "$v" != "/Volumes/Macintosh HD" ]; then SSD_VOLUME="$v"; break; fi
+done
+
+if [ -n "$SSD_VOLUME" ]; then
+  # Tell the user we found an SSD and need sudo to enable owners
+  echo "Found external SSD at $SSD_VOLUME — enabling file ownership."
+  echo "Please run this in your terminal (it needs sudo):"
+  echo "  sudo diskutil enableOwnership \"$SSD_VOLUME\""
+  echo "Reply 'done' when complete."
+fi
+```
+
+Wait for the user to confirm. Then verify:
+
+```bash
+diskutil info "$SSD_VOLUME" | grep "Owners" | grep -q "Enabled" && echo "OK" || echo "FAIL"
+```
+
+#### 4b. Grant Full Disk Access to bash and node (TCC)
+
+Even with owners enabled, **macOS TCC blocks launchd-spawned processes from writing to /Volumes/ unless the binary has Full Disk Access**. This is the single most common cause of "Operation not permitted" errors when you run a daemon under launchd that tries to write to an external volume.
+
+You cannot grant Full Disk Access programmatically without disabling SIP (which is unsafe). The user must do this via System Settings UI:
+
+```
+1. Open System Settings → Privacy & Security → Full Disk Access
+2. Click the + button
+3. Press Cmd+Shift+G to "Go to Folder", then enter:
+     /bin/bash
+   Press Enter, select 'bash', click Open
+4. Click + again, then:
+     /usr/bin/node     OR     ~/.nvm/versions/node/v24.11.1/bin/node
+   (whichever node binary the wrapper uses)
+5. Make sure both toggles are ON
+
+The toggles take effect immediately. No restart needed.
+```
+
+After the user confirms, test that launchd can now write to the SSD:
+
+```bash
+cat > /tmp/ssd-tcc-test.plist <<EOF
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>Label</key><string>ai.adaptic.ssd-tcc-test</string>
+    <key>ProgramArguments</key><array>
+        <string>/bin/bash</string><string>-c</string>
+        <string>touch "$SSD_VOLUME/.tcc-test" && echo "ok=$?" > /tmp/ssd-tcc-result.log || echo "fail=$?" > /tmp/ssd-tcc-result.log</string>
+    </array>
+    <key>RunAtLoad</key><true/>
+</dict>
+</plist>
+EOF
+launchctl load /tmp/ssd-tcc-test.plist
+sleep 2
+cat /tmp/ssd-tcc-result.log
+launchctl unload /tmp/ssd-tcc-test.plist
+rm -f "$SSD_VOLUME/.tcc-test"
+```
+
+If you see `ok=0`, TCC is configured correctly and the daemon can use the SSD. If you see `fail=1`, TCC is still blocking — repeat the System Settings step and make sure the toggles are ON.
+
+#### 4c. Set up SSD layout and symlinks
+
+```bash
+AGENT_NAME="$(grep firstName config/agent.ts | head -1 | sed 's/.*[\x27\"]\([a-zA-Z]*\)[\x27\"].*/\1/' | tr A-Z a-z)"
+SSD_AGENT_ROOT="$SSD_VOLUME/maestro/$AGENT_NAME"
+mkdir -p "$SSD_AGENT_ROOT"/{state,outputs,memory,knowledge,claude-tmp,logs,tmp}
+
+# Symlink runtime data dirs from the agent repo to the SSD.
+# IMPORTANT: do NOT symlink logs/ — launchd's StandardErrorPath cannot follow
+# symlinks to external volumes. The daemon's wrapper writes its own log file
+# directly to the SSD via shell redirection (see launchd-wrapper.sh).
+for d in state outputs memory knowledge; do
+  if [ -d "$d" ] && [ ! -L "$d" ]; then
+    rsync -a "$d/" "$SSD_AGENT_ROOT/$d/"
+    rm -rf "$d"
+    ln -sfn "$SSD_AGENT_ROOT/$d" "$d"
+  fi
+done
+
+# Create internal-disk logs/ as a real directory (NOT a symlink)
+mkdir -p logs/{daemon,polling,workflows,sessions,audit,security,evolution,huddle,infra,monitor,phone,sms,whatsapp,email,launchd,cloudflared}
+```
+
+The wrapper scripts (`scripts/daemon/launchd-wrapper.sh` and `launchd-wrapper-generic.sh`) handle the runtime side: they detect the SSD, set `CLAUDE_CODE_TMPDIR`, and redirect daemon stdout/stderr to a log file on the SSD. They gracefully fall back to internal-disk paths if the SSD isn't writable (e.g. if TCC isn't granted yet).
+
+#### 4d. Verify
+
+```bash
+# Daemon should now be writing to the SSD log file
+launchctl unload ~/Library/LaunchAgents/ai.adaptic.${AGENT_NAME}-daemon.plist
+launchctl load   ~/Library/LaunchAgents/ai.adaptic.${AGENT_NAME}-daemon.plist
+sleep 4
+ls -la "$SSD_AGENT_ROOT/logs/daemon/" | tail -5
+ls -la "$SSD_AGENT_ROOT/state/inbox/" | tail -5
+```
+
+You should see the daemon log file growing and inbox directories populating. If you don't, repeat steps 4a–4b (most often it's TCC).
+
 ## Phase 4: Autonomous Service Configuration
 
 This phase sets up all third-party integrations **autonomously**. Use Playwright MCP for web-based setup (Slack API portal, Twilio Console, Google Account, ElevenLabs, Deepgram) and Bash for local scripts. Only ask the user for input when genuinely required (existing credentials, 2FA codes, payment authorisation).
@@ -537,14 +702,145 @@ source .env && python3 -c "import imaplib,os; m=imaplib.IMAP4_SSL('imap.gmail.co
 4. Note and report the sandbox join keyword to the user
 5. Write `WHATSAPP_MODE=sandbox`, `WHATSAPP_PORT=3002` to `.env`
 
-### Step 4: Cloudflare Tunnels — per `docs/guides/voice-sms-setup.md` § 7
+### Step 4: Webhook Relay (Railway) — per `docs/guides/webhook-relay-setup.md`
+
+**This is the canonical pattern. Do NOT use Cloudflare Tunnels for new agents** — they were a transitional approach. Each agent gets its own Railway-deployed webhook relay. The local Mac mini polls the relay every 5 seconds and never needs an inbound tunnel.
+
+The relay handles:
+- `POST /slack/events` — Slack Events API (HMAC verified via SLACK_SIGNING_SECRET)
+- `POST /sms` — Twilio SMS inbound (HMAC verified via TWILIO_AUTH_TOKEN)
+- `POST /whatsapp` — Twilio WhatsApp inbound
+- `POST /whatsapp/status` — Twilio WhatsApp delivery status
+- `GET /events`, `/sms/messages`, `/whatsapp/messages` — drained by Mac mini poller
+- `GET /health` — service status
+
+**Source code** is already in the repo at `services/webhook-relay/` (copied from the maestro framework). It's a ~250-line Node 20 HTTP server, no dependencies, deployable straight to Railway.
+
+**Prerequisites:**
+- Railway CLI installed: `brew install railway 2>/dev/null || true`
+- User must run `railway login` once (interactive — opens browser)
+- User must have admin rights in the company's Railway workspace (e.g., "Adaptic")
+
+**Deploy steps (run from the agent's repo root):**
+
+```bash
+# 1. Create the project in the company's Railway workspace
+cd services/webhook-relay
+railway init --name {firstname-lower}-webhook-relay --workspace {Company}
+
+# 2. Add the service and deploy
+railway up --service {firstname-lower}-webhook-relay --detach
+
+# 3. Generate a public domain
+railway domain --service {firstname-lower}-webhook-relay
+# Captures: https://{firstname-lower}-webhook-relay-production.up.railway.app
+
+# 4. Set env vars (must include the agent's own SLACK_SIGNING_SECRET and TWILIO_AUTH_TOKEN)
+source ../../.env
+railway variables --service {firstname-lower}-webhook-relay \
+  --set "SLACK_SIGNING_SECRET=$SLACK_SIGNING_SECRET" \
+  --set "TWILIO_AUTH_TOKEN=$TWILIO_AUTH_TOKEN" \
+  --set "PUBLIC_HOSTNAME={firstname-lower}-webhook-relay-production.up.railway.app" \
+  --set "BUFFER_TTL_MS=600000" \
+  --set "MAX_BUFFER_SIZE=1000"
+
+# 5. Trigger redeploy so the running container picks up the new env vars
+railway up --service {firstname-lower}-webhook-relay --detach
+
+# 6. Wait until /health returns slack_signature: true and twilio_signature: true
+for i in 1 2 3 4 5 6 7 8 9 10 11 12; do
+  RESP=$(curl -sf -m 5 https://{firstname-lower}-webhook-relay-production.up.railway.app/health)
+  if echo "$RESP" | grep -q '"slack_signature":true' && echo "$RESP" | grep -q '"twilio_signature":true'; then
+    echo "Relay live with signature verification"
+    break
+  fi
+  sleep 10
+done
+```
+
+**Configure external services to point at the relay:**
+
+```bash
+# Twilio SMS webhook (uses Twilio API directly, no UI)
+RELAY_URL="https://{firstname-lower}-webhook-relay-production.up.railway.app"
+curl -s -u "$TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN" -X POST \
+  "https://api.twilio.com/2010-04-01/Accounts/$TWILIO_ACCOUNT_SID/IncomingPhoneNumbers/$TWILIO_PHONE_SID.json" \
+  --data-urlencode "SmsUrl=$RELAY_URL/sms" --data-urlencode "SmsMethod=POST"
+```
+
+For **Slack Events Subscription**: use Playwright to update via the App Manifest editor (more reliable than the events page). Navigate to `https://app.slack.com/app-settings/{TEAM_ID}/{APP_ID}/app-manifest`, read the JSON via the CodeMirror instance, add this block to `settings`, and click Save Changes:
+
+```json
+"event_subscriptions": {
+  "request_url": "https://{firstname-lower}-webhook-relay-production.up.railway.app/slack/events",
+  "bot_events": [
+    "app_mention",
+    "message.channels",
+    "message.groups",
+    "message.im",
+    "message.mpim"
+  ]
+}
+```
+
+After save, navigate to the Event Subscriptions page and check for the yellow "Click here to verify" button — click it. Then **reinstall the app** at `https://api.slack.com/apps/{APP_ID}/install-on-team` so the new event scopes activate.
+
+For **Twilio WhatsApp sandbox**: this requires a per-agent Twilio sub-account (see Phase 4 Step 3.5). Cannot share with other agents because the sandbox webhook is account-wide.
+
+**Update local poll script:**
+
+```bash
+# Edit scripts/poll-slack-events.sh and scripts/comms-monitor.sh
+# Set EVENTS_URL to https://{firstname-lower}-webhook-relay-production.up.railway.app/events
+```
+
+**Install the launchd job that polls the relay every 5 seconds:**
+
+```bash
+cat > scripts/local-triggers/plists/ai.adaptic.{firstname-lower}-poll-relay.plist <<EOF
+<?xml version="1.0" encoding="UTF-8"?>
+<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
+<plist version="1.0">
+<dict>
+    <key>Label</key><string>ai.adaptic.{firstname-lower}-poll-relay</string>
+    <key>ProgramArguments</key><array>
+        <string>/bin/bash</string>
+        <string>{REPO_ROOT}/scripts/poll-slack-events.sh</string>
+    </array>
+    <key>WorkingDirectory</key><string>{REPO_ROOT}</string>
+    <key>StartInterval</key><integer>5</integer>
+    <key>RunAtLoad</key><true/>
+    <key>StandardOutPath</key><string>{REPO_ROOT}/logs/polling/poll-relay-stdout.log</string>
+    <key>StandardErrorPath</key><string>{REPO_ROOT}/logs/polling/poll-relay-stderr.log</string>
+</dict>
+</plist>
+EOF
+cp scripts/local-triggers/plists/ai.adaptic.{firstname-lower}-poll-relay.plist ~/Library/LaunchAgents/
+launchctl load ~/Library/LaunchAgents/ai.adaptic.{firstname-lower}-poll-relay.plist
+```
+
+**Add the relay URL block to `.env`:**
+
+```bash
+cat >> .env <<EOF
+
+# ─── RAILWAY WEBHOOK RELAY ──────────────────────────────────────────────────
+WEBHOOK_RELAY_URL=https://{firstname-lower}-webhook-relay-production.up.railway.app
+WEBHOOK_RELAY_SLACK_EVENTS=https://{firstname-lower}-webhook-relay-production.up.railway.app/slack/events
+WEBHOOK_RELAY_SMS_INBOUND=https://{firstname-lower}-webhook-relay-production.up.railway.app/sms
+WEBHOOK_RELAY_WHATSAPP_INBOUND=https://{firstname-lower}-webhook-relay-production.up.railway.app/whatsapp
+WEBHOOK_RELAY_POLL_EVENTS=https://{firstname-lower}-webhook-relay-production.up.railway.app/events
+WEBHOOK_RELAY_POLL_SMS=https://{firstname-lower}-webhook-relay-production.up.railway.app/sms/messages
+WEBHOOK_RELAY_POLL_WHATSAPP=https://{firstname-lower}-webhook-relay-production.up.railway.app/whatsapp/messages
+EOF
+```
+
+**End-to-end test:**
 
-1. `brew install cloudflared 2>/dev/null || true`
-2. Determine needed ports: SMS=3001, WhatsApp=3002, Slack Events=3100
-3. Ask: quick tunnel (temporary) or named tunnel (persistent)?
-4. For quick: start tunnels in background, capture URLs
-5. For named: `cloudflared tunnel login` (may require browser auth), `cloudflared tunnel create {agent-name}`, write config
-6. Go back and update Twilio webhook URLs with the tunnel URLs (Steps 3 SMS/WhatsApp)
+1. Have Lucas (or any user) send a Slack message to a channel where the bot is a member, OR @-mention the bot in a public channel
+2. Within ~5 seconds, the local Mac mini should fetch the buffered event and write a YAML file to `state/inbox/slack/`
+3. The inbox processor picks it up and routes it
+4. Verify `railway logs --service {firstname-lower}-webhook-relay` shows `[slack] buffered ...`
 
 ### Step 5: Voice / Huddle — per `docs/guides/voice-sms-setup.md` § 5
 

package/README.md

@@ -373,6 +373,14 @@ Security policies are defined in `policies/` and `docs/governance/`.
 - [RAG & Context Setup](docs/guides/rag-context-setup.md) -- SQLite FTS5 search, pre-draft context, entity indexing
 - [PDF Generation Setup](docs/guides/pdf-generation-setup.md) -- Pandoc + XeLaTeX branded document generation
 - [Media Generation Setup](docs/guides/media-generation-setup.md) -- Gemini/Veo image and video generation
+- [Claude-Mem Setup](docs/guides/claude-mem-setup.md) -- Persistent session memory with semantic recall
+- [Claude-Pace Setup](docs/guides/claude-pace-setup.md) -- Real-time rate limit tracking and burn rate awareness
+- [ccxray Diagnostics](docs/guides/ccxray-diagnostics.md) -- Token/cost analysis and session debugging
+- [Claudraband Sessions](docs/guides/claudraband-sessions.md) -- Persistent sessions, daemon mode, HTTP API
+- [ClawTeam Swarm](docs/guides/clawteam-swarm.md) -- Multi-agent coding swarm orchestration via git worktrees
+- [Agents Observe](docs/guides/agents-observe-setup.md) -- Real-time multi-agent observability dashboard
+- [Code-Review-Graph](docs/guides/code-review-graph-setup.md) -- Tree-sitter structural knowledge graph for codebases
+- [Self-Optimization Pattern](docs/guides/self-optimization-pattern.md) -- AutoAgent-inspired benchmark-driven self-improvement
 
 ### Architecture & Governance
 
@@ -381,6 +389,26 @@ Security policies are defined in `policies/` and `docs/governance/`.
 - [Action Approval Model](docs/governance/action-approval-model.md) -- Communication governance and approval levels
 - [Communications Policy](docs/governance/communications-policy.md) -- Voice modes, autonomy model, escalation rules
 
+### Dev Tooling
+
+Approved third-party tools for agent development and observability. Install with:
+
+```bash
+./scripts/setup/install-dev-tools.sh --all
+```
+
+| Tool | Purpose | Install |
+|------|---------|---------|
+| **claude-pace** | Rate limit status line tracker | `--tool claude-pace` |
+| **agents-observe** | Multi-agent observability dashboard | `--tool agents-observe` |
+| **ccxray** | Token/cost observability proxy | `--tool ccxray` |
+| **ClawTeam** | Git worktree swarm orchestrator | `--tool clawteam` |
+| **code-review-graph** | Tree-sitter codebase knowledge graph | `--tool code-review-graph` |
+
+### Cross-Agent Message Routing
+
+When multiple agents monitor the same Slack channels, the daemon classifier uses `config/known-agents.json` to prevent cross-agent message interception. If a message @-mentions a specific agent, only that agent's daemon will respond. Update this file when agents are added or removed.
+
 ### Runbooks
 
 - [Mac Mini Bootstrap](docs/runbooks/mac-mini-bootstrap.md) -- Hardware setup and initial configuration