@adaptic/maestro 1.1.6 → 1.1.8

package/docs/guides/whatsapp-setup.md

@@ -0,0 +1,282 @@
+# WhatsApp Setup Guide
+
+How to enable WhatsApp messaging for a Maestro agent: Twilio WhatsApp sandbox/production configuration, inbound webhook handler, outbound messaging (free-form and templates), media support, and Cloudflare tunnel exposure.
+
+**Prerequisites**: Complete the [Voice & SMS Setup](voice-sms-setup.md) sections 1.1–1.3 (Twilio account and credentials). WhatsApp uses the same Twilio account as SMS.
+
+---
+
+## Architecture Overview
+
+```
+┌─────────────────────────────────────────────────────────────────┐
+│  INBOUND                                                         │
+│                                                                   │
+│  WhatsApp user ──▶ Twilio Cloud ──▶ Cloudflare Tunnel           │
+│                     (webhook)        (port 3002)                 │
+│                                         │                        │
+│                                         ▼                        │
+│                                  whatsapp-handler.mjs            │
+│                                         │                        │
+│                                         ▼                        │
+│                             state/inbox/whatsapp/*.yaml          │
+│                             (inbox processor routes)             │
+├─────────────────────────────────────────────────────────────────┤
+│  OUTBOUND                                                        │
+│                                                                   │
+│  send-whatsapp.sh ──▶ Twilio Messages API ──▶ WhatsApp user     │
+│  (with dedup +         (whatsapp: prefix)                        │
+│   validation)                                                    │
+├─────────────────────────────────────────────────────────────────┤
+│  CONFIGURATION                                                   │
+│                                                                   │
+│  configure-whatsapp-sandbox.sh                                   │
+│  (prints manual steps for Twilio Console webhook setup)          │
+└─────────────────────────────────────────────────────────────────┘
+```
+
+---
+
+## 1. WhatsApp Modes
+
+Twilio offers two WhatsApp integration modes:
+
+| Mode | Use Case | Sender Number | Limitations |
+|---|---|---|---|
+| **Sandbox** | Testing and development | +1 415 523 8886 (shared Twilio number) | Recipients must opt-in by sending "join \<keyword\>" |
+| **Production** | Live operations | Agent's own Twilio number | Requires approved WhatsApp Business Account (WABA) |
+
+**Start with sandbox mode** — it's free and requires no approval. Switch to production when ready for live operations.
+
+---
+
+## 2. Sandbox Setup
+
+### 2.1 Activate the Sandbox
+
+1. Go to Twilio Console → Messaging → Try it Out → Send a WhatsApp Message
+2. Follow the on-screen instructions to activate the sandbox
+3. Note the **join keyword** (e.g. "join example-word")
+
+### 2.2 Add Recipients to Sandbox
+
+Each person who wants to message the agent must opt-in:
+
+1. Save +1 415 523 8886 as a WhatsApp contact
+2. Send the message: `join <keyword>` (the keyword from step 2.1)
+3. They'll receive a confirmation reply
+
+### 2.3 Configure Environment Variables
+
+Add to `.env`:
+
+```bash
+# WhatsApp mode
+WHATSAPP_MODE=sandbox
+
+# Twilio sandbox number (don't change)
+WHATSAPP_SANDBOX_NUMBER=+14155238886
+
+# Local port for webhook handler
+WHATSAPP_PORT=3002
+```
+
+---
+
+## 3. Production Setup
+
+### 3.1 Apply for WhatsApp Business Account
+
+1. Go to Twilio Console → Messaging → Senders → WhatsApp Senders
+2. Submit a request to connect your Twilio number as a WhatsApp Business sender
+3. This requires Meta/WhatsApp approval (can take days to weeks)
+
+### 3.2 Configure for Production
+
+Once approved, update `.env`:
+
+```bash
+WHATSAPP_MODE=production
+```
+
+In production mode, `send-whatsapp.sh` uses the agent's Twilio number instead of the sandbox number. Business-initiated messages (outside the 24-hour conversation window) require approved message templates.
+
+---
+
+## 4. Inbound — Webhook Handler
+
+### 4.1 Start the Handler
+
+```bash
+node scripts/whatsapp-handler.mjs
+```
+
+The handler listens on port 3002 (configurable via `WHATSAPP_PORT`) and provides:
+
+| Endpoint | Method | Purpose |
+|---|---|---|
+| `/whatsapp` | POST | Twilio inbound message webhook |
+| `/whatsapp/status` | POST | Delivery status callbacks |
+| `/health` | GET | Health check |
+
+### 4.2 How It Works
+
+1. Twilio receives a WhatsApp message to your number (or sandbox)
+2. Twilio POSTs to your webhook URL at `/whatsapp`
+3. Handler parses the message: strips `whatsapp:` prefix from From/To, extracts body, media
+4. Looks up sender in `config/caller-id-map.yaml` for identity and access level
+5. Writes a YAML inbox item to `state/inbox/whatsapp/`
+6. CEO messages create priority trigger files for immediate processing
+7. Returns empty TwiML (no auto-reply)
+
+### 4.3 Expose via Cloudflare Tunnel
+
+```bash
+# Start a tunnel to the WhatsApp handler
+cloudflared tunnel --url http://localhost:3002
+```
+
+Copy the generated URL (e.g. `https://random-words.trycloudflare.com`).
+
+### 4.4 Configure Twilio Webhook
+
+Use the configuration helper:
+
+```bash
+# Show current sandbox config
+./scripts/configure-whatsapp-sandbox.sh --status
+
+# Generate instructions for manual Twilio Console setup
+./scripts/configure-whatsapp-sandbox.sh --url https://your-tunnel-url.trycloudflare.com
+```
+
+The script prints manual steps since Twilio's WhatsApp sandbox webhooks must be configured in the Console:
+
+1. Go to Twilio Console → Messaging → Try it Out → WhatsApp → Sandbox Settings
+2. Set "When a message comes in": `https://your-tunnel-url/whatsapp` (HTTP POST)
+3. Set "Status callback URL": `https://your-tunnel-url/whatsapp/status` (HTTP POST)
+4. Save
+
+---
+
+## 5. Outbound — Sending Messages
+
+### 5.1 Free-Form Messages
+
+```bash
+# Sandbox mode (default)
+./scripts/send-whatsapp.sh --to "+971501234567" --body "Hello from the agent"
+
+# Production mode
+./scripts/send-whatsapp.sh --to "+971501234567" --body "Hello" --production
+```
+
+### 5.2 Template Messages
+
+For business-initiated messages outside the 24-hour window (production mode only):
+
+```bash
+./scripts/send-whatsapp.sh --to "+971501234567" --template "intro_greeting" --vars "AgentName,Company"
+```
+
+### 5.3 Media Messages
+
+```bash
+./scripts/send-whatsapp.sh --to "+971501234567" --body "Here's the report" --media "https://example.com/report.pdf"
+```
+
+### 5.4 Pre-Send Pipeline
+
+Outbound WhatsApp messages go through:
+
+1. **Content-hash dedup** (`outbound-dedup.sh`) — prevents identical concurrent sends
+2. **Factual validation** (`validate-outbound.py`) — blocks if critical issues found
+3. **Audit logging** — every send logged to `logs/whatsapp/` and `logs/audit/`
+
+---
+
+## 6. Process Management
+
+### 6.1 launchd Plist
+
+A template plist exists at `scripts/poller-launchd/ai.adaptic.sophie-whatsapp-handler.plist`. Copy and customize for your agent:
+
+```bash
+# Replace agent-specific values and install
+cp scripts/poller-launchd/ai.adaptic.sophie-whatsapp-handler.plist \
+   ~/Library/LaunchAgents/com.adaptic.AGENT.whatsapp-handler.plist
+
+# Edit the plist to update paths and labels
+# Then load:
+launchctl load ~/Library/LaunchAgents/com.adaptic.AGENT.whatsapp-handler.plist
+```
+
+### 6.2 Cloudflare Tunnel Persistence
+
+For production, configure a named tunnel (see [Voice & SMS Setup](voice-sms-setup.md) section 7.2) with an ingress rule for port 3002.
+
+---
+
+## 7. Testing
+
+| # | Test | How to Verify |
+|---|---|---|
+| 1 | Handler running | `curl http://localhost:3002/health` |
+| 2 | Tunnel accessible | `curl https://your-tunnel-url/health` |
+| 3 | Sandbox opt-in | Send "join \<keyword\>" from your phone to +1 415 523 8886 |
+| 4 | Inbound message | Send WhatsApp to sandbox; check `state/inbox/whatsapp/` |
+| 5 | Outbound message | `./scripts/send-whatsapp.sh --to "+YOUR_NUMBER" --body "Test"` |
+| 6 | Caller ID lookup | Inbound from known number should show correct name in YAML |
+| 7 | CEO priority | Message from principal should create priority trigger file |
+| 8 | Dedup working | Send identical outbound twice; second should `DEDUP_SKIP` |
+| 9 | Media outbound | Send with `--media` flag; verify media arrives |
+| 10 | Audit logging | Check `logs/whatsapp/` and `logs/audit/` for entries |
+
+---
+
+## 8. Troubleshooting
+
+### Sandbox: "Recipient has not opted in"
+
+Error code 63007 from Twilio means the recipient hasn't joined the sandbox:
+1. Have them send "join \<keyword\>" to +1 415 523 8886
+2. Verify the keyword matches what's shown in Twilio Console
+3. They must send from the exact WhatsApp number you're trying to reach
+
+### Messages not arriving at handler
+
+1. Check tunnel: `curl https://your-tunnel-url/health`
+2. Check Twilio webhook config matches your tunnel URL
+3. Check handler is running: `curl http://localhost:3002/health`
+4. Check handler logs: `tail -f logs/whatsapp-handler-stderr.log`
+
+### "Template not found" in production mode
+
+1. Templates must be approved by WhatsApp/Meta before use
+2. Verify template name matches exactly (case-sensitive)
+3. Verify template variables match the approved format
+
+### Media not sending
+
+1. Media URL must be publicly accessible (Twilio fetches it)
+2. Supported types: images (JPEG, PNG), documents (PDF), audio (OGG)
+3. Check Twilio's [media size limits](https://www.twilio.com/docs/whatsapp/guidance-whatsapp-media-messages)
+
+---
+
+## Key Files
+
+| File | Purpose |
+|---|---|
+| `scripts/send-whatsapp.sh` | Outbound WhatsApp sender (sandbox + production) |
+| `scripts/whatsapp-handler.mjs` | Inbound WhatsApp webhook handler |
+| `scripts/configure-whatsapp-sandbox.sh` | Sandbox webhook configuration helper |
+| `config/caller-id-map.yaml` | WhatsApp number → identity mapping |
+
+---
+
+## Related Documents
+
+- [Voice & SMS Setup](voice-sms-setup.md) — Twilio account setup, SMS, and voice calls
+- [Outbound Governance Setup](outbound-governance-setup.md) — Dedup and validation pipeline
+- [Poller & Daemon Setup](poller-daemon-setup.md) — How WhatsApp events integrate with the event loop

package/docs/runbooks/mac-mini-bootstrap.md

@@ -393,6 +393,17 @@ echo "=============================="
 - [ ] Log rotation configured
 - [ ] Health check runs cleanly
 - [ ] First morning brief generated successfully
+- [ ] (Optional) Voice/SMS configured -- see `docs/guides/voice-sms-setup.md`
+
+---
+
+## Optional: Voice & SMS Setup
+
+If this agent needs phone call or SMS capabilities (Slack huddle voice participation, inbound/outbound SMS via Twilio), follow the dedicated guide:
+
+**[Voice & SMS Setup Guide](../guides/voice-sms-setup.md)**
+
+This covers Twilio phone number purchase, inbound SMS webhook, outbound SMS, Slack huddle voice (Deepgram STT + ElevenLabs TTS), Cloudflare tunnel exposure, caller ID mapping, and launchd process management for the voice/SMS services.
 
 ---
 
@@ -400,5 +411,15 @@ Related documents:
 
 - `docs/runbooks/perpetual-operations.md` -- How the system runs continuously
 - `docs/runbooks/recovery-and-failover.md` -- Recovery procedures
+- `docs/guides/agent-persona-setup.md` -- Agent identity and configuration
+- `docs/guides/email-setup.md` -- Gmail IMAP and SMTP setup
+- `docs/guides/slack-setup.md` -- Slack app, tokens, and events
+- `docs/guides/whatsapp-setup.md` -- WhatsApp Business integration
+- `docs/guides/voice-sms-setup.md` -- Voice calls and SMS setup
+- `docs/guides/poller-daemon-setup.md` -- Event loop, daemon, triggers, watchdog
+- `docs/guides/outbound-governance-setup.md` -- Hooks, dedup, information barriers
+- `docs/guides/rag-context-setup.md` -- Search index and context retrieval
+- `docs/guides/pdf-generation-setup.md` -- Branded PDF documents
+- `docs/guides/media-generation-setup.md` -- AI-generated visual assets
 - `docs/workflows/executive-cadence.md` -- Full operating rhythm
 - `config/environment.yaml` -- Environment configuration reference

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@adaptic/maestro",
-  "version": "1.1.6",
+  "version": "1.1.8",
   "description": "Maestro — Autonomous AI agent operating system. Deploy AI employees on dedicated Mac minis.",
   "type": "module",
   "bin": {

package/scaffold/config/caller-id-map.yaml

@@ -0,0 +1,46 @@
+# Caller ID to user mapping for voice/SMS authorization
+#
+# Used by: sms-handler.mjs, parse-voice-transcript.mjs, user-context-search.py
+# Maps phone numbers, Slack IDs, and emails to user identities with access levels.
+#
+# Access levels:
+#   ceo        — Full access to all paths (memory, knowledge, state, outputs, docs, logs, all interactions)
+#   leadership — Company knowledge + docs + research/briefs + own interaction logs
+#   partner    — Public knowledge + research + own interaction logs
+#   default    — Unknown callers — public company knowledge only (knowledge/sources/)
+#
+# Hot-reloaded every 60 seconds by sms-handler.mjs — no restart needed after edits.
+#
+# Setup: See docs/guides/voice-sms-setup.md for full configuration instructions.
+
+users:
+  # ── Principal (the person this agent reports to) ──────────────────────────
+  # Replace with your principal's details. This entry should have access_level: ceo.
+  principal:
+    name: "Principal Full Name"
+    phone: ["+1XXXXXXXXXX"]
+    whatsapp: ["+1XXXXXXXXXX"]
+    slack_id: "UXXXXXXXXXX"
+    email: "principal@company.com"
+    access_level: ceo
+
+  # ── Leadership / Team ─────────────────────────────────────────────────────
+  # Add team members who should be recognized by the SMS handler and voice parser.
+  # Each entry needs at minimum: name, slack_id, email, and access_level.
+  # Phone numbers are only needed if the person will contact the agent via SMS/voice.
+
+  # example-leader:
+  #   name: "Example Leader"
+  #   phone: []
+  #   slack_id: "UXXXXXXXXXX"
+  #   email: "leader@company.com"
+  #   access_level: leadership
+
+  # ── Partners / External ───────────────────────────────────────────────────
+
+  # example-partner:
+  #   name: "External Partner"
+  #   phone: []
+  #   slack_id: ""
+  #   email: "partner@external.com"
+  #   access_level: partner

package/scripts/media-generation/README.md

@@ -1,5 +1,7 @@
 # Media Generation Pipeline
 
+> **Full setup guide**: See [docs/guides/media-generation-setup.md](../../docs/guides/media-generation-setup.md) for complete API setup, prompt spec authoring, brand alignment, testing, and troubleshooting.
+
 Generates branded illustrations, diagrams, and video assets using Google Gemini and Veo APIs.
 Ported from `~/adapticai/app/scripts/media-generation/` and `~/ai-born/`.
 

package/scripts/pdf-generation/README.md

@@ -1,5 +1,7 @@
 # PDF Generation Pipeline
 
+> **Full setup guide**: See [docs/guides/pdf-generation-setup.md](../../docs/guides/pdf-generation-setup.md) for complete installation, template customisation, brand integration, testing, and troubleshooting.
+
 Generates professional PDF documents from Markdown using Pandoc + XeLaTeX.
 Ported from the `~/ai-born` book generation pipeline, adapted for corporate documents.
 

package/scripts/poller/slack-poller.mjs

@@ -643,8 +643,11 @@ export async function pollSlack() {
         const data = JSON.parse(raw);
 
         // Dedup: skip if we already have this item from API polling
-        const eventTs = data.ts || data.event_ts || "";
-        const eventRef = data.raw_ref || `slack:${data.channel || ""}:${eventTs}`;
+        // Events-server JSON uses `slack_event` key; also check `event` for compatibility
+        const evtPre = data.slack_event || data.event || {};
+        const eventTs = data.ts || data.event_ts || data.received_at || evtPre.ts || "";
+        const evtChannel = data.channel_id || data.channel || evtPre.channel || "";
+        const eventRef = data.raw_ref || `slack:${evtChannel}:${eventTs}`;
         const alreadyHave = items.some((i) => i.raw_ref === eventRef);
         if (alreadyHave) {
           // Mark as processed since the API poll already got it
@@ -653,20 +656,22 @@ export async function pollSlack() {
         }
 
         // Convert events-server JSON to daemon item format
-        const sender = data.sender || resolveName(data.user || data.user_id || "unknown");
+        // Events-server JSON uses `slack_event` key; also check `event` for compatibility
+        const evt = data.slack_event || data.event || {};
+        const sender = data.sender || resolveName(data.user || data.user_id || evt.user || "unknown");
 
         // Skip Sophie's own messages — the events server should filter these,
         // but if any slip through (e.g. message_changed edge cases), catch here
-        const userId = data.user || data.user_id || "";
+        const userId = data.user || data.user_id || evt.user || "";
         if (userId === SOPHIE_USER_ID || sender === "sophie-nguyen") {
           try { renameSync(join(SLACK_INBOX_DIR, file), join(SLACK_INBOX_DIR, file + ".processed")); } catch {}
           continue;
         }
 
         const privilege = data.sender_privilege || resolvePrivilege(userId);
-        const content = data.content || data.text || data.event?.text || "";
-        const channelId = data.channel_id || data.channel || data.event?.channel || "";
-        const threadTs = data.thread_id || data.thread_ts || data.event?.thread_ts || "";
+        const content = data.content || data.text || evt.text || "";
+        const channelId = data.channel_id || data.channel || evt.channel || "";
+        const threadTs = data.thread_id || data.thread_ts || evt.thread_ts || "";
 
         // Fetch thread context if this is a thread reply
         let threadContext = data.thread_context || null;
@@ -674,6 +679,15 @@ export async function pollSlack() {
           threadContext = await fetchThreadContext(channelId, threadTs, eventTs);
         }
 
+        // Extract and download attachments from events-server items
+        const evtFiles = evt.files || data.files || [];
+        const evtAttachments = extractSlackAttachments(evtFiles);
+        for (const att of evtAttachments) {
+          const fileObj = evtFiles.find((f) => f.id === att.id);
+          const localPath = await downloadSlackAttachment(fileObj, SLACK_TOKEN);
+          if (localPath) att.local_path = localPath;
+        }
+
         items.push({
           id: data.id || eventTs.replace(".", "-") || file.replace(".json", ""),
           service: "slack",
@@ -687,6 +701,7 @@ export async function pollSlack() {
           thread_id: threadTs,
           thread_context: threadContext,
           is_reply: data.is_reply || (!!threadTs && threadTs !== eventTs),
+          attachments: evtAttachments.length > 0 ? evtAttachments : undefined,
           priority_signals: data.priority_signals || {
             from_ceo: privilege === "ceo",
             tagged_urgent: /\b(urgent|emergency|asap|blocker|critical)\b/i.test(content),

package/scripts/poller/trigger.mjs

@@ -28,7 +28,7 @@ export function triggerSophie(item) {
     const dir = join(SOPHIE_AI_DIR, "state", "inbox", "internal");
     mkdirSync(dir, { recursive: true });
     const taskFile = join(dir, `priority-${Date.now()}.yaml`);
-    const content = `type: priority_trigger
+    let content = `type: priority_trigger
 reason: "${reason}"
 source_item: "${item.raw_ref}"
 timestamp: "${new Date().toISOString()}"
@@ -37,6 +37,17 @@ channel: "${item.channel}"
 content: |
   ${(item.content || "").replace(/\n/g, "\n  ")}
 `;
+    // Include attachment metadata so downstream sessions can view files
+    if (item.attachments && item.attachments.length > 0) {
+      content += `attachments:\n`;
+      for (const att of item.attachments) {
+        content += `  - name: "${att.name || "unnamed"}"\n`;
+        content += `    mimetype: "${att.mimetype || "unknown"}"\n`;
+        content += `    size: ${att.size || 0}\n`;
+        if (att.local_path) content += `    local_path: "${att.local_path}"\n`;
+        if (att.url_private) content += `    url_private: "${att.url_private}"\n`;
+      }
+    }
     writeFileSync(taskFile, content);
     console.log(`[trigger] Priority task written to ${taskFile}`);
     return true;

package/scripts/setup/boot-claude-session.sh

@@ -21,14 +21,10 @@ SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
 MAESTRO_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
 
 # Detect agent directory — prefer sophie-ai if it exists
-AGENT_DIR="${1:-}"
-if [ -z "$AGENT_DIR" ]; then
-  if [ -d "$HOME/sophie-ai" ]; then
-    AGENT_DIR="$HOME/sophie-ai"
-  else
-    echo "ERROR: No agent directory specified and ~/sophie-ai not found" >&2
-    exit 1
-  fi
+AGENT_DIR="${1:?Agent directory required as first argument (e.g. ~/sophie-ai)}"
+if [ ! -d "$AGENT_DIR" ]; then
+  echo "ERROR: Agent directory does not exist: $AGENT_DIR" >&2
+  exit 1
 fi
 
 AGENT_NAME=$(basename "$AGENT_DIR")

package/scripts/setup/configure-macos.sh

@@ -29,6 +29,10 @@ AGENT_DIR="$(cd "$SCRIPT_DIR/../.." && pwd)"
 AGENT_NAME=$(basename "$AGENT_DIR")
 CURRENT_USER=$(whoami)
 
+# The directory from which this script was invoked — used as the boot-time
+# Claude Code working directory (e.g. ~/sophie-ai, ~/wundr, etc.)
+CALLER_DIR="${PWD}"
+
 # Colours
 RED='\033[0;31m'
 GREEN='\033[0;32m'
@@ -341,9 +345,8 @@ configure_app_launches() {
     local real_home
     real_home=$(eval echo "~$real_user")
 
-    # Detect the agent working directory (prefer sophie-ai)
-    local boot_agent_dir="$HOME/sophie-ai"
-    [ -d "$boot_agent_dir" ] || boot_agent_dir="$AGENT_DIR"
+    # Use the directory from which configure-macos.sh was invoked
+    local boot_agent_dir="$CALLER_DIR"
 
     cat > "$PLIST_PATH" << PLIST_EOF
 <?xml version="1.0" encoding="UTF-8"?>
@@ -618,7 +621,8 @@ main() {
   echo "╔══════════════════════════════════════════════════════════════╗"
   echo "║        Maestro — Mac Mini Configuration                     ║"
   echo "╠══════════════════════════════════════════════════════════════╣"
-  echo "║  Agent directory: $AGENT_DIR"
+  echo "║  Maestro:         $AGENT_DIR"
+  echo "║  Boot target:     $CALLER_DIR"
   echo "║  Current user:    $CURRENT_USER"
   echo "╚══════════════════════════════════════════════════════════════╝"