@adaptic/maestro 1.1.7 → 1.4.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.claude/commands/init-maestro.md +502 -260
- package/README.md +47 -2
- package/bin/maestro.mjs +1 -1
- package/docs/guides/agents-observe-setup.md +64 -0
- package/docs/guides/ccxray-diagnostics.md +65 -0
- package/docs/guides/claude-mem-setup.md +79 -0
- package/docs/guides/claude-pace-setup.md +56 -0
- package/docs/guides/claudraband-sessions.md +98 -0
- package/docs/guides/clawteam-swarm.md +116 -0
- package/docs/guides/code-review-graph-setup.md +86 -0
- package/docs/guides/email-setup.md +399 -0
- package/docs/guides/media-generation-setup.md +349 -0
- package/docs/guides/outbound-governance-setup.md +438 -0
- package/docs/guides/pdf-generation-setup.md +315 -0
- package/docs/guides/poller-daemon-setup.md +550 -0
- package/docs/guides/rag-context-setup.md +459 -0
- package/docs/guides/self-optimization-pattern.md +82 -0
- package/docs/guides/slack-setup.md +350 -0
- package/docs/guides/twilio-subaccounts-setup.md +223 -0
- package/docs/guides/voice-sms-setup.md +698 -0
- package/docs/guides/webhook-relay-setup.md +349 -0
- package/docs/guides/whatsapp-setup.md +282 -0
- package/docs/runbooks/mac-mini-bootstrap.md +21 -0
- package/package.json +2 -1
- package/plugins/maestro-skills/plugin.json +16 -0
- package/plugins/maestro-skills/skills/agents-observe.md +110 -0
- package/plugins/maestro-skills/skills/ccxray-diagnostics.md +91 -0
- package/plugins/maestro-skills/skills/claude-pace.md +61 -0
- package/plugins/maestro-skills/skills/code-review-graph.md +99 -0
- package/scaffold/CLAUDE.md +64 -0
- package/scaffold/config/agent.ts.example +2 -1
- package/scaffold/config/caller-id-map.yaml +46 -0
- package/scaffold/config/known-agents.json +35 -0
- package/scripts/daemon/classifier.mjs +264 -50
- package/scripts/daemon/dispatcher.mjs +109 -5
- package/scripts/daemon/launchd-wrapper-generic.sh +96 -0
- package/scripts/daemon/launchd-wrapper-slack-events.sh +37 -0
- package/scripts/daemon/launchd-wrapper.sh +91 -0
- package/scripts/daemon/lib/session-router.mjs +274 -0
- package/scripts/daemon/lib/session-router.test.mjs +295 -0
- package/scripts/daemon/prompt-builder.mjs +51 -11
- package/scripts/daemon/responder.mjs +234 -19
- package/scripts/daemon/session-lock.mjs +194 -0
- package/scripts/daemon/sophie-daemon.mjs +16 -2
- package/scripts/email-signature.html +20 -4
- package/scripts/local-triggers/generate-plists.sh +62 -10
- package/scripts/media-generation/README.md +2 -0
- package/scripts/pdf-generation/README.md +2 -0
- package/scripts/poller/imap-client.mjs +4 -2
- package/scripts/poller/slack-poller.mjs +126 -59
- package/scripts/poller/trigger.mjs +12 -1
- package/scripts/setup/init-agent.sh +91 -1
- package/scripts/setup/install-dev-tools.sh +150 -0
- package/scripts/spawn-session.sh +21 -6
- package/workflows/continuous/backlog-executor.yaml +141 -0
- package/workflows/daily/evening-wrap.yaml +41 -1
- package/workflows/daily/morning-brief.yaml +17 -0
- package/workflows/event-driven/agent-failure-investigation.yaml +137 -0
- package/workflows/event-driven/pr-review.yaml +104 -0
- package/workflows/weekly/engineering-health.yaml +154 -0
|
@@ -0,0 +1,349 @@
|
|
|
1
|
+
# Webhook Relay Setup Guide (Railway)
|
|
2
|
+
|
|
3
|
+
How to deploy each agent's per-instance webhook relay to Railway. The relay is the canonical pattern for receiving inbound webhooks from Slack and Twilio without requiring a stable inbound connection from the Mac mini.
|
|
4
|
+
|
|
5
|
+
**Prerequisites**: Complete the [Mac Mini Bootstrap](../runbooks/mac-mini-bootstrap.md). The agent needs:
|
|
6
|
+
- A Slack app already created (see `slack-setup.md`) so we have `SLACK_SIGNING_SECRET`
|
|
7
|
+
- A Twilio account (or sub-account) and number purchased so we have `TWILIO_AUTH_TOKEN`
|
|
8
|
+
- Admin rights in the company's Railway workspace (e.g., "Adaptic")
|
|
9
|
+
|
|
10
|
+
---
|
|
11
|
+
|
|
12
|
+
## Why a relay?
|
|
13
|
+
|
|
14
|
+
Maestro agents run on a Mac mini that doesn't have a stable public IP. External services need a permanent HTTPS URL for webhooks. Cloudflare Quick Tunnels work but the URL changes on restart, and named tunnels require Cloudflare DNS configuration.
|
|
15
|
+
|
|
16
|
+
Railway gives every agent a permanent `*.up.railway.app` domain with HTTPS and zero infrastructure. The pattern: Slack/Twilio send to Railway → Railway buffers in memory → Mac mini polls Railway every 5 seconds → events land in `state/inbox/`.
|
|
17
|
+
|
|
18
|
+
```
|
|
19
|
+
┌─────────────┐ ┌──────────────────────┐
|
|
20
|
+
│ Slack │──webhook──────▶│ {agent}-webhook-relay │
|
|
21
|
+
│ Twilio │ │ (Railway, in {Org}) │
|
|
22
|
+
└─────────────┘ │ in-memory buffers │
|
|
23
|
+
└──────┬─────────────────┘
|
|
24
|
+
│ /events, /sms/messages, /whatsapp/messages
|
|
25
|
+
▼
|
|
26
|
+
┌──────────────────┐
|
|
27
|
+
│ Mac mini │
|
|
28
|
+
│ poll-relay.sh │
|
|
29
|
+
│ every 5 sec │
|
|
30
|
+
└────────┬─────────┘
|
|
31
|
+
▼
|
|
32
|
+
state/inbox/{slack,sms,whatsapp}/
|
|
33
|
+
```
|
|
34
|
+
|
|
35
|
+
---
|
|
36
|
+
|
|
37
|
+
## 1. Source code
|
|
38
|
+
|
|
39
|
+
The relay source is at `services/webhook-relay/` in every agent's repo (copied from the maestro framework at create time). It's a single ~250-line Node 20 HTTP server, no dependencies, deployable straight to Railway.
|
|
40
|
+
|
|
41
|
+
```
|
|
42
|
+
services/webhook-relay/
|
|
43
|
+
├── server.mjs # The HTTP server with all endpoints
|
|
44
|
+
├── package.json # Node 20+, type: module, no deps
|
|
45
|
+
├── railway.json # NIXPACKS builder, /health check, restart-on-failure
|
|
46
|
+
├── .gitignore
|
|
47
|
+
└── README.md
|
|
48
|
+
```
|
|
49
|
+
|
|
50
|
+
### Endpoints
|
|
51
|
+
|
|
52
|
+
#### Webhook receivers (called by external services)
|
|
53
|
+
|
|
54
|
+
| Path | Method | Caller | Notes |
|
|
55
|
+
|---|---|---|---|
|
|
56
|
+
| `/slack/events` | POST | Slack Events API | HMAC verified via `SLACK_SIGNING_SECRET`. Echoes `url_verification` challenges. |
|
|
57
|
+
| `/sms` | POST | Twilio SMS | HMAC verified via `TWILIO_AUTH_TOKEN`. Returns empty TwiML. |
|
|
58
|
+
| `/whatsapp` | POST | Twilio WhatsApp | Same. |
|
|
59
|
+
| `/whatsapp/status` | POST | Twilio WhatsApp status callback | Same. |
|
|
60
|
+
|
|
61
|
+
#### Drain endpoints (called by the local Mac mini)
|
|
62
|
+
|
|
63
|
+
| Path | Method | Returns |
|
|
64
|
+
|---|---|---|
|
|
65
|
+
| `/events` | GET | All buffered Slack events, drains buffer |
|
|
66
|
+
| `/sms/messages` | GET | All buffered SMS events, drains buffer |
|
|
67
|
+
| `/whatsapp/messages` | GET | All buffered WhatsApp events |
|
|
68
|
+
| `/whatsapp/statuses` | GET | All buffered WhatsApp delivery status events |
|
|
69
|
+
| `/health` | GET | Service status, buffer sizes, signature flags |
|
|
70
|
+
|
|
71
|
+
---
|
|
72
|
+
|
|
73
|
+
## 2. Install Railway CLI
|
|
74
|
+
|
|
75
|
+
```bash
|
|
76
|
+
brew install railway 2>/dev/null || true
|
|
77
|
+
railway --version # should be 4.x
|
|
78
|
+
```
|
|
79
|
+
|
|
80
|
+
Then log in (interactive — opens browser):
|
|
81
|
+
|
|
82
|
+
```bash
|
|
83
|
+
railway login
|
|
84
|
+
```
|
|
85
|
+
|
|
86
|
+
Confirm with `railway whoami`. If you don't see the company workspace in `railway list`, ask an existing admin to grant you access.
|
|
87
|
+
|
|
88
|
+
---
|
|
89
|
+
|
|
90
|
+
## 3. Create the project
|
|
91
|
+
|
|
92
|
+
Run from the agent's repo:
|
|
93
|
+
|
|
94
|
+
```bash
|
|
95
|
+
cd services/webhook-relay
|
|
96
|
+
railway init --name {firstname-lower}-webhook-relay --workspace {Company}
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
For Lucas, this is `lucas-webhook-relay` in the `Adaptic` workspace.
|
|
100
|
+
|
|
101
|
+
The CLI will print a project URL — note it for reference.
|
|
102
|
+
|
|
103
|
+
---
|
|
104
|
+
|
|
105
|
+
## 4. Deploy
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
railway up --service {firstname-lower}-webhook-relay --detach
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
This packages the local directory, uploads it to Railway, and triggers a Nixpacks build. Build typically takes 30–60 seconds. Watch the build logs in the URL the CLI prints.
|
|
112
|
+
|
|
113
|
+
---
|
|
114
|
+
|
|
115
|
+
## 5. Generate a public domain
|
|
116
|
+
|
|
117
|
+
```bash
|
|
118
|
+
railway domain --service {firstname-lower}-webhook-relay
|
|
119
|
+
```
|
|
120
|
+
|
|
121
|
+
Prints something like:
|
|
122
|
+
```
|
|
123
|
+
🚀 https://lucas-webhook-relay-production.up.railway.app
|
|
124
|
+
```
|
|
125
|
+
|
|
126
|
+
Save this URL — you'll use it everywhere.
|
|
127
|
+
|
|
128
|
+
---
|
|
129
|
+
|
|
130
|
+
## 6. Set environment variables
|
|
131
|
+
|
|
132
|
+
```bash
|
|
133
|
+
source ../../.env
|
|
134
|
+
railway variables --service {firstname-lower}-webhook-relay \
|
|
135
|
+
--set "SLACK_SIGNING_SECRET=$SLACK_SIGNING_SECRET" \
|
|
136
|
+
--set "TWILIO_AUTH_TOKEN=$TWILIO_AUTH_TOKEN" \
|
|
137
|
+
--set "PUBLIC_HOSTNAME={firstname-lower}-webhook-relay-production.up.railway.app" \
|
|
138
|
+
--set "BUFFER_TTL_MS=600000" \
|
|
139
|
+
--set "MAX_BUFFER_SIZE=1000"
|
|
140
|
+
```
|
|
141
|
+
|
|
142
|
+
| Var | Required | Source | Purpose |
|
|
143
|
+
|---|---|---|---|
|
|
144
|
+
| `SLACK_SIGNING_SECRET` | yes | Slack app → Basic Information | HMAC verify Slack webhooks |
|
|
145
|
+
| `TWILIO_AUTH_TOKEN` | yes | Twilio Console → Account | HMAC verify Twilio webhooks |
|
|
146
|
+
| `PUBLIC_HOSTNAME` | yes | Railway domain (without https://) | Used in Twilio HMAC base string |
|
|
147
|
+
| `BUFFER_TTL_MS` | no | default 600000 (10 min) | Drop events older than this |
|
|
148
|
+
| `MAX_BUFFER_SIZE` | no | default 1000 | Max events per channel |
|
|
149
|
+
| `POLL_AUTH_TOKEN` | no | (unset) | If set, GET endpoints require Bearer token |
|
|
150
|
+
|
|
151
|
+
The `PORT` variable is provided automatically by Railway — don't set it.
|
|
152
|
+
|
|
153
|
+
After setting variables, **trigger a redeploy** so the running container picks them up. The variables exist in the project but the running container was built with the previous values:
|
|
154
|
+
|
|
155
|
+
```bash
|
|
156
|
+
railway up --service {firstname-lower}-webhook-relay --detach
|
|
157
|
+
```
|
|
158
|
+
|
|
159
|
+
---
|
|
160
|
+
|
|
161
|
+
## 7. Verify deployment
|
|
162
|
+
|
|
163
|
+
```bash
|
|
164
|
+
for i in 1 2 3 4 5 6 7 8 9 10 11 12; do
|
|
165
|
+
RESP=$(curl -sf -m 5 https://{firstname-lower}-webhook-relay-production.up.railway.app/health)
|
|
166
|
+
if echo "$RESP" | grep -q '"slack_signature":true' && echo "$RESP" | grep -q '"twilio_signature":true'; then
|
|
167
|
+
echo "✅ Relay live with signature verification"
|
|
168
|
+
break
|
|
169
|
+
fi
|
|
170
|
+
sleep 10
|
|
171
|
+
done
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
Expected output of `/health`:
|
|
175
|
+
|
|
176
|
+
```json
|
|
177
|
+
{
|
|
178
|
+
"ok": true,
|
|
179
|
+
"service": "{firstname-lower}-webhook-relay",
|
|
180
|
+
"uptime_sec": 12.3,
|
|
181
|
+
"buffers": {
|
|
182
|
+
"slack": { "events": 0, "seen": 0 },
|
|
183
|
+
"sms": { "events": 0, "seen": 0 },
|
|
184
|
+
"whatsapp": { "events": 0, "seen": 0 },
|
|
185
|
+
"whatsapp_status": { "events": 0, "seen": 0 }
|
|
186
|
+
},
|
|
187
|
+
"config": {
|
|
188
|
+
"slack_signature": true,
|
|
189
|
+
"twilio_signature": true,
|
|
190
|
+
"poll_auth": false,
|
|
191
|
+
"buffer_ttl_ms": 600000,
|
|
192
|
+
"max_buffer_size": 1000
|
|
193
|
+
}
|
|
194
|
+
}
|
|
195
|
+
```
|
|
196
|
+
|
|
197
|
+
If `slack_signature` or `twilio_signature` is `false`, the env vars didn't reach the running container — re-run `railway up`.
|
|
198
|
+
|
|
199
|
+
---
|
|
200
|
+
|
|
201
|
+
## 8. Configure external services to point at the relay
|
|
202
|
+
|
|
203
|
+
### Slack Events Subscription
|
|
204
|
+
|
|
205
|
+
Update via the App Manifest editor (more reliable than the events page):
|
|
206
|
+
|
|
207
|
+
1. Navigate to `https://app.slack.com/app-settings/{TEAM_ID}/{APP_ID}/app-manifest`
|
|
208
|
+
2. Read the JSON, add this block to `settings`:
|
|
209
|
+
|
|
210
|
+
```json
|
|
211
|
+
"event_subscriptions": {
|
|
212
|
+
"request_url": "https://{firstname-lower}-webhook-relay-production.up.railway.app/slack/events",
|
|
213
|
+
"bot_events": [
|
|
214
|
+
"app_mention",
|
|
215
|
+
"message.channels",
|
|
216
|
+
"message.groups",
|
|
217
|
+
"message.im",
|
|
218
|
+
"message.mpim"
|
|
219
|
+
]
|
|
220
|
+
}
|
|
221
|
+
```
|
|
222
|
+
|
|
223
|
+
3. Click **Save Changes**
|
|
224
|
+
4. Navigate to `https://api.slack.com/apps/{APP_ID}/event-subscriptions` and click **Click here to verify** if the yellow banner appears
|
|
225
|
+
5. **Reinstall the app** at `https://api.slack.com/apps/{APP_ID}/install-on-team` so the new event scopes activate
|
|
226
|
+
|
|
227
|
+
### Twilio SMS webhook
|
|
228
|
+
|
|
229
|
+
```bash
|
|
230
|
+
RELAY_URL="https://{firstname-lower}-webhook-relay-production.up.railway.app"
|
|
231
|
+
curl -s -u "$TWILIO_ACCOUNT_SID:$TWILIO_AUTH_TOKEN" -X POST \
|
|
232
|
+
"https://api.twilio.com/2010-04-01/Accounts/$TWILIO_ACCOUNT_SID/IncomingPhoneNumbers/$TWILIO_PHONE_SID.json" \
|
|
233
|
+
--data-urlencode "SmsUrl=$RELAY_URL/sms" --data-urlencode "SmsMethod=POST"
|
|
234
|
+
```
|
|
235
|
+
|
|
236
|
+
### Twilio WhatsApp sandbox
|
|
237
|
+
|
|
238
|
+
This requires a per-agent Twilio sub-account (the WhatsApp sandbox webhook is account-wide). See `docs/guides/twilio-subaccounts-setup.md`.
|
|
239
|
+
|
|
240
|
+
Once you have the sub-account credentials, configure the sandbox via the Twilio Console UI (no API support):
|
|
241
|
+
|
|
242
|
+
1. Navigate to `https://console.twilio.com/us1/develop/sms/try-it-out/whatsapp-learn?frameUrl=%2Fconsole%2Fsms%2Fwhatsapp%2Flearn`
|
|
243
|
+
2. Click the **Sandbox settings** tab
|
|
244
|
+
3. Set the inbound webhook URL to `https://{firstname-lower}-webhook-relay-production.up.railway.app/whatsapp` (HTTP POST)
|
|
245
|
+
4. Set the status callback URL to `https://{firstname-lower}-webhook-relay-production.up.railway.app/whatsapp/status` (HTTP POST)
|
|
246
|
+
5. Save
|
|
247
|
+
|
|
248
|
+
---
|
|
249
|
+
|
|
250
|
+
## 9. Local poll job
|
|
251
|
+
|
|
252
|
+
The Mac mini runs a launchd job that polls the relay every 5 seconds. Install it:
|
|
253
|
+
|
|
254
|
+
```bash
|
|
255
|
+
# 1. Update scripts/poll-slack-events.sh — set EVENTS_URL to your relay's /events endpoint
|
|
256
|
+
# 2. Update scripts/comms-monitor.sh — set RAILWAY_URL similarly
|
|
257
|
+
# 3. Install the launchd plist:
|
|
258
|
+
|
|
259
|
+
cat > scripts/local-triggers/plists/ai.adaptic.{firstname-lower}-poll-relay.plist <<EOF
|
|
260
|
+
<?xml version="1.0" encoding="UTF-8"?>
|
|
261
|
+
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
|
|
262
|
+
<plist version="1.0">
|
|
263
|
+
<dict>
|
|
264
|
+
<key>Label</key><string>ai.adaptic.{firstname-lower}-poll-relay</string>
|
|
265
|
+
<key>ProgramArguments</key><array>
|
|
266
|
+
<string>/bin/bash</string>
|
|
267
|
+
<string>{REPO_ROOT}/scripts/poll-slack-events.sh</string>
|
|
268
|
+
</array>
|
|
269
|
+
<key>WorkingDirectory</key><string>{REPO_ROOT}</string>
|
|
270
|
+
<key>StartInterval</key><integer>5</integer>
|
|
271
|
+
<key>RunAtLoad</key><true/>
|
|
272
|
+
<key>StandardOutPath</key><string>{REPO_ROOT}/logs/polling/poll-relay-stdout.log</string>
|
|
273
|
+
<key>StandardErrorPath</key><string>{REPO_ROOT}/logs/polling/poll-relay-stderr.log</string>
|
|
274
|
+
</dict>
|
|
275
|
+
</plist>
|
|
276
|
+
EOF
|
|
277
|
+
|
|
278
|
+
cp scripts/local-triggers/plists/ai.adaptic.{firstname-lower}-poll-relay.plist ~/Library/LaunchAgents/
|
|
279
|
+
launchctl load ~/Library/LaunchAgents/ai.adaptic.{firstname-lower}-poll-relay.plist
|
|
280
|
+
launchctl list | grep poll-relay # verify loaded
|
|
281
|
+
```
|
|
282
|
+
|
|
283
|
+
---
|
|
284
|
+
|
|
285
|
+
## 10. End-to-end test
|
|
286
|
+
|
|
287
|
+
1. Have a teammate (or yourself, from a separate Slack account) send a DM to the agent's bot in Slack, OR @-mention the bot in a public channel where the bot is a member
|
|
288
|
+
2. Within ~5 seconds:
|
|
289
|
+
- `railway logs --service {firstname-lower}-webhook-relay` should show `[slack] buffered message...`
|
|
290
|
+
- `state/inbox/slack/` should contain a new YAML file
|
|
291
|
+
- `logs/polling/poll-relay-stdout.log` should show the inbox write
|
|
292
|
+
3. The inbox processor (running as launchd `ai.adaptic.{firstname-lower}-inbox-processor`) picks up the YAML and routes it through the daemon
|
|
293
|
+
|
|
294
|
+
---
|
|
295
|
+
|
|
296
|
+
## Troubleshooting
|
|
297
|
+
|
|
298
|
+
### `railway up` returns "Service not found"
|
|
299
|
+
|
|
300
|
+
Run `railway add --service {firstname-lower}-webhook-relay` first to create the service container. The flag is required even when the project exists.
|
|
301
|
+
|
|
302
|
+
### `railway init` returns "Unauthorized"
|
|
303
|
+
|
|
304
|
+
Your Railway account doesn't have admin rights in the workspace. Ask an existing admin (or your principal) to grant you Admin role in the workspace's Members settings.
|
|
305
|
+
|
|
306
|
+
### `slack_signature: false` in /health after deploy
|
|
307
|
+
|
|
308
|
+
The env vars exist in the project but the running container was built before they were set. Re-run `railway up --service {firstname-lower}-webhook-relay --detach` to redeploy.
|
|
309
|
+
|
|
310
|
+
### Slack URL verification fails with "URL didn't respond"
|
|
311
|
+
|
|
312
|
+
The relay is reachable but signature verification fails. Causes:
|
|
313
|
+
|
|
314
|
+
- `SLACK_SIGNING_SECRET` mismatch (check Slack app → Basic Information vs. Railway env var)
|
|
315
|
+
- The relay is responding correctly to `url_verification` (returns the challenge), but Slack's caching layer hasn't flushed. Re-save the manifest, or click "Click here to verify" on the event subscriptions page.
|
|
316
|
+
|
|
317
|
+
### Twilio webhook returns 401 "invalid signature"
|
|
318
|
+
|
|
319
|
+
`PUBLIC_HOSTNAME` env var is wrong. It must match the actual host Twilio sends to (typically `{firstname-lower}-webhook-relay-production.up.railway.app`, no scheme, no trailing slash). Twilio includes the full URL in its signature base string.
|
|
320
|
+
|
|
321
|
+
### Buffer fills up but Mac mini doesn't drain
|
|
322
|
+
|
|
323
|
+
Check the launchd job:
|
|
324
|
+
|
|
325
|
+
```bash
|
|
326
|
+
launchctl list | grep poll-relay # should show the label with a non-error exit code
|
|
327
|
+
tail logs/polling/poll-relay-stderr.log
|
|
328
|
+
```
|
|
329
|
+
|
|
330
|
+
Also check `EVENTS_URL` in `scripts/poll-slack-events.sh` matches the deployed Railway domain.
|
|
331
|
+
|
|
332
|
+
### The relay restarts and loses buffered events
|
|
333
|
+
|
|
334
|
+
In-memory buffers are TTL'd at 10 minutes. If the relay restarts (Railway redeploy, instance recycle, OOM), buffered events are lost. The local Slack poller (`scripts/poller/slack-poller.mjs`) runs every 60 seconds as a fallback and will catch any missed channel/DM messages — at most a 60s gap. For SMS/WhatsApp this is acceptable because Twilio retries on failure.
|
|
335
|
+
|
|
336
|
+
If you need persistence, add a Postgres or Redis service in the same Railway project and persist buffers there. Not currently necessary at our message volume.
|
|
337
|
+
|
|
338
|
+
---
|
|
339
|
+
|
|
340
|
+
## Why one service per agent?
|
|
341
|
+
|
|
342
|
+
Each agent has its own Slack signing secret and Twilio account/sub-account. A shared relay would have to multiplex by header or path, which adds complexity and a single point of failure. Per-agent deployment is simpler, isolated, and gives independent scaling/restarts.
|
|
343
|
+
|
|
344
|
+
## Related guides
|
|
345
|
+
|
|
346
|
+
- [Slack Setup](slack-setup.md) — Slack app creation and OAuth
|
|
347
|
+
- [Voice & SMS Setup](voice-sms-setup.md) — Twilio account, SMS handlers
|
|
348
|
+
- [WhatsApp Setup](whatsapp-setup.md) — WhatsApp sandbox / production
|
|
349
|
+
- [Twilio Subaccounts Setup](twilio-subaccounts-setup.md) — per-agent isolation for WhatsApp
|
|
@@ -0,0 +1,282 @@
|
|
|
1
|
+
# WhatsApp Setup Guide
|
|
2
|
+
|
|
3
|
+
How to enable WhatsApp messaging for a Maestro agent: Twilio WhatsApp sandbox/production configuration, inbound webhook handler, outbound messaging (free-form and templates), media support, and Cloudflare tunnel exposure.
|
|
4
|
+
|
|
5
|
+
**Prerequisites**: Complete the [Voice & SMS Setup](voice-sms-setup.md) sections 1.1–1.3 (Twilio account and credentials). WhatsApp uses the same Twilio account as SMS.
|
|
6
|
+
|
|
7
|
+
---
|
|
8
|
+
|
|
9
|
+
## Architecture Overview
|
|
10
|
+
|
|
11
|
+
```
|
|
12
|
+
┌─────────────────────────────────────────────────────────────────┐
|
|
13
|
+
│ INBOUND │
|
|
14
|
+
│ │
|
|
15
|
+
│ WhatsApp user ──▶ Twilio Cloud ──▶ Cloudflare Tunnel │
|
|
16
|
+
│ (webhook) (port 3002) │
|
|
17
|
+
│ │ │
|
|
18
|
+
│ ▼ │
|
|
19
|
+
│ whatsapp-handler.mjs │
|
|
20
|
+
│ │ │
|
|
21
|
+
│ ▼ │
|
|
22
|
+
│ state/inbox/whatsapp/*.yaml │
|
|
23
|
+
│ (inbox processor routes) │
|
|
24
|
+
├─────────────────────────────────────────────────────────────────┤
|
|
25
|
+
│ OUTBOUND │
|
|
26
|
+
│ │
|
|
27
|
+
│ send-whatsapp.sh ──▶ Twilio Messages API ──▶ WhatsApp user │
|
|
28
|
+
│ (with dedup + (whatsapp: prefix) │
|
|
29
|
+
│ validation) │
|
|
30
|
+
├─────────────────────────────────────────────────────────────────┤
|
|
31
|
+
│ CONFIGURATION │
|
|
32
|
+
│ │
|
|
33
|
+
│ configure-whatsapp-sandbox.sh │
|
|
34
|
+
│ (prints manual steps for Twilio Console webhook setup) │
|
|
35
|
+
└─────────────────────────────────────────────────────────────────┘
|
|
36
|
+
```
|
|
37
|
+
|
|
38
|
+
---
|
|
39
|
+
|
|
40
|
+
## 1. WhatsApp Modes
|
|
41
|
+
|
|
42
|
+
Twilio offers two WhatsApp integration modes:
|
|
43
|
+
|
|
44
|
+
| Mode | Use Case | Sender Number | Limitations |
|
|
45
|
+
|---|---|---|---|
|
|
46
|
+
| **Sandbox** | Testing and development | +1 415 523 8886 (shared Twilio number) | Recipients must opt-in by sending "join \<keyword\>" |
|
|
47
|
+
| **Production** | Live operations | Agent's own Twilio number | Requires approved WhatsApp Business Account (WABA) |
|
|
48
|
+
|
|
49
|
+
**Start with sandbox mode** — it's free and requires no approval. Switch to production when ready for live operations.
|
|
50
|
+
|
|
51
|
+
---
|
|
52
|
+
|
|
53
|
+
## 2. Sandbox Setup
|
|
54
|
+
|
|
55
|
+
### 2.1 Activate the Sandbox
|
|
56
|
+
|
|
57
|
+
1. Go to Twilio Console → Messaging → Try it Out → Send a WhatsApp Message
|
|
58
|
+
2. Follow the on-screen instructions to activate the sandbox
|
|
59
|
+
3. Note the **join keyword** (e.g. "join example-word")
|
|
60
|
+
|
|
61
|
+
### 2.2 Add Recipients to Sandbox
|
|
62
|
+
|
|
63
|
+
Each person who wants to message the agent must opt-in:
|
|
64
|
+
|
|
65
|
+
1. Save +1 415 523 8886 as a WhatsApp contact
|
|
66
|
+
2. Send the message: `join <keyword>` (the keyword from step 2.1)
|
|
67
|
+
3. They'll receive a confirmation reply
|
|
68
|
+
|
|
69
|
+
### 2.3 Configure Environment Variables
|
|
70
|
+
|
|
71
|
+
Add to `.env`:
|
|
72
|
+
|
|
73
|
+
```bash
|
|
74
|
+
# WhatsApp mode
|
|
75
|
+
WHATSAPP_MODE=sandbox
|
|
76
|
+
|
|
77
|
+
# Twilio sandbox number (don't change)
|
|
78
|
+
WHATSAPP_SANDBOX_NUMBER=+14155238886
|
|
79
|
+
|
|
80
|
+
# Local port for webhook handler
|
|
81
|
+
WHATSAPP_PORT=3002
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
---
|
|
85
|
+
|
|
86
|
+
## 3. Production Setup
|
|
87
|
+
|
|
88
|
+
### 3.1 Apply for WhatsApp Business Account
|
|
89
|
+
|
|
90
|
+
1. Go to Twilio Console → Messaging → Senders → WhatsApp Senders
|
|
91
|
+
2. Submit a request to connect your Twilio number as a WhatsApp Business sender
|
|
92
|
+
3. This requires Meta/WhatsApp approval (can take days to weeks)
|
|
93
|
+
|
|
94
|
+
### 3.2 Configure for Production
|
|
95
|
+
|
|
96
|
+
Once approved, update `.env`:
|
|
97
|
+
|
|
98
|
+
```bash
|
|
99
|
+
WHATSAPP_MODE=production
|
|
100
|
+
```
|
|
101
|
+
|
|
102
|
+
In production mode, `send-whatsapp.sh` uses the agent's Twilio number instead of the sandbox number. Business-initiated messages (outside the 24-hour conversation window) require approved message templates.
|
|
103
|
+
|
|
104
|
+
---
|
|
105
|
+
|
|
106
|
+
## 4. Inbound — Webhook Handler
|
|
107
|
+
|
|
108
|
+
### 4.1 Start the Handler
|
|
109
|
+
|
|
110
|
+
```bash
|
|
111
|
+
node scripts/whatsapp-handler.mjs
|
|
112
|
+
```
|
|
113
|
+
|
|
114
|
+
The handler listens on port 3002 (configurable via `WHATSAPP_PORT`) and provides:
|
|
115
|
+
|
|
116
|
+
| Endpoint | Method | Purpose |
|
|
117
|
+
|---|---|---|
|
|
118
|
+
| `/whatsapp` | POST | Twilio inbound message webhook |
|
|
119
|
+
| `/whatsapp/status` | POST | Delivery status callbacks |
|
|
120
|
+
| `/health` | GET | Health check |
|
|
121
|
+
|
|
122
|
+
### 4.2 How It Works
|
|
123
|
+
|
|
124
|
+
1. Twilio receives a WhatsApp message to your number (or sandbox)
|
|
125
|
+
2. Twilio POSTs to your webhook URL at `/whatsapp`
|
|
126
|
+
3. Handler parses the message: strips `whatsapp:` prefix from From/To, extracts body, media
|
|
127
|
+
4. Looks up sender in `config/caller-id-map.yaml` for identity and access level
|
|
128
|
+
5. Writes a YAML inbox item to `state/inbox/whatsapp/`
|
|
129
|
+
6. CEO messages create priority trigger files for immediate processing
|
|
130
|
+
7. Returns empty TwiML (no auto-reply)
|
|
131
|
+
|
|
132
|
+
### 4.3 Expose via Cloudflare Tunnel
|
|
133
|
+
|
|
134
|
+
```bash
|
|
135
|
+
# Start a tunnel to the WhatsApp handler
|
|
136
|
+
cloudflared tunnel --url http://localhost:3002
|
|
137
|
+
```
|
|
138
|
+
|
|
139
|
+
Copy the generated URL (e.g. `https://random-words.trycloudflare.com`).
|
|
140
|
+
|
|
141
|
+
### 4.4 Configure Twilio Webhook
|
|
142
|
+
|
|
143
|
+
Use the configuration helper:
|
|
144
|
+
|
|
145
|
+
```bash
|
|
146
|
+
# Show current sandbox config
|
|
147
|
+
./scripts/configure-whatsapp-sandbox.sh --status
|
|
148
|
+
|
|
149
|
+
# Generate instructions for manual Twilio Console setup
|
|
150
|
+
./scripts/configure-whatsapp-sandbox.sh --url https://your-tunnel-url.trycloudflare.com
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
The script prints manual steps since Twilio's WhatsApp sandbox webhooks must be configured in the Console:
|
|
154
|
+
|
|
155
|
+
1. Go to Twilio Console → Messaging → Try it Out → WhatsApp → Sandbox Settings
|
|
156
|
+
2. Set "When a message comes in": `https://your-tunnel-url/whatsapp` (HTTP POST)
|
|
157
|
+
3. Set "Status callback URL": `https://your-tunnel-url/whatsapp/status` (HTTP POST)
|
|
158
|
+
4. Save
|
|
159
|
+
|
|
160
|
+
---
|
|
161
|
+
|
|
162
|
+
## 5. Outbound — Sending Messages
|
|
163
|
+
|
|
164
|
+
### 5.1 Free-Form Messages
|
|
165
|
+
|
|
166
|
+
```bash
|
|
167
|
+
# Sandbox mode (default)
|
|
168
|
+
./scripts/send-whatsapp.sh --to "+971501234567" --body "Hello from the agent"
|
|
169
|
+
|
|
170
|
+
# Production mode
|
|
171
|
+
./scripts/send-whatsapp.sh --to "+971501234567" --body "Hello" --production
|
|
172
|
+
```
|
|
173
|
+
|
|
174
|
+
### 5.2 Template Messages
|
|
175
|
+
|
|
176
|
+
For business-initiated messages outside the 24-hour window (production mode only):
|
|
177
|
+
|
|
178
|
+
```bash
|
|
179
|
+
./scripts/send-whatsapp.sh --to "+971501234567" --template "intro_greeting" --vars "AgentName,Company"
|
|
180
|
+
```
|
|
181
|
+
|
|
182
|
+
### 5.3 Media Messages
|
|
183
|
+
|
|
184
|
+
```bash
|
|
185
|
+
./scripts/send-whatsapp.sh --to "+971501234567" --body "Here's the report" --media "https://example.com/report.pdf"
|
|
186
|
+
```
|
|
187
|
+
|
|
188
|
+
### 5.4 Pre-Send Pipeline
|
|
189
|
+
|
|
190
|
+
Outbound WhatsApp messages go through:
|
|
191
|
+
|
|
192
|
+
1. **Content-hash dedup** (`outbound-dedup.sh`) — prevents identical concurrent sends
|
|
193
|
+
2. **Factual validation** (`validate-outbound.py`) — blocks if critical issues found
|
|
194
|
+
3. **Audit logging** — every send logged to `logs/whatsapp/` and `logs/audit/`
|
|
195
|
+
|
|
196
|
+
---
|
|
197
|
+
|
|
198
|
+
## 6. Process Management
|
|
199
|
+
|
|
200
|
+
### 6.1 launchd Plist
|
|
201
|
+
|
|
202
|
+
A template plist exists at `scripts/poller-launchd/ai.adaptic.sophie-whatsapp-handler.plist`. Copy and customize for your agent:
|
|
203
|
+
|
|
204
|
+
```bash
|
|
205
|
+
# Replace agent-specific values and install
|
|
206
|
+
cp scripts/poller-launchd/ai.adaptic.sophie-whatsapp-handler.plist \
|
|
207
|
+
~/Library/LaunchAgents/com.adaptic.AGENT.whatsapp-handler.plist
|
|
208
|
+
|
|
209
|
+
# Edit the plist to update paths and labels
|
|
210
|
+
# Then load:
|
|
211
|
+
launchctl load ~/Library/LaunchAgents/com.adaptic.AGENT.whatsapp-handler.plist
|
|
212
|
+
```
|
|
213
|
+
|
|
214
|
+
### 6.2 Cloudflare Tunnel Persistence
|
|
215
|
+
|
|
216
|
+
For production, configure a named tunnel (see [Voice & SMS Setup](voice-sms-setup.md) section 7.2) with an ingress rule for port 3002.
|
|
217
|
+
|
|
218
|
+
---
|
|
219
|
+
|
|
220
|
+
## 7. Testing
|
|
221
|
+
|
|
222
|
+
| # | Test | How to Verify |
|
|
223
|
+
|---|---|---|
|
|
224
|
+
| 1 | Handler running | `curl http://localhost:3002/health` |
|
|
225
|
+
| 2 | Tunnel accessible | `curl https://your-tunnel-url/health` |
|
|
226
|
+
| 3 | Sandbox opt-in | Send "join \<keyword\>" from your phone to +1 415 523 8886 |
|
|
227
|
+
| 4 | Inbound message | Send WhatsApp to sandbox; check `state/inbox/whatsapp/` |
|
|
228
|
+
| 5 | Outbound message | `./scripts/send-whatsapp.sh --to "+YOUR_NUMBER" --body "Test"` |
|
|
229
|
+
| 6 | Caller ID lookup | Inbound from known number should show correct name in YAML |
|
|
230
|
+
| 7 | CEO priority | Message from principal should create priority trigger file |
|
|
231
|
+
| 8 | Dedup working | Send identical outbound twice; second should `DEDUP_SKIP` |
|
|
232
|
+
| 9 | Media outbound | Send with `--media` flag; verify media arrives |
|
|
233
|
+
| 10 | Audit logging | Check `logs/whatsapp/` and `logs/audit/` for entries |
|
|
234
|
+
|
|
235
|
+
---
|
|
236
|
+
|
|
237
|
+
## 8. Troubleshooting
|
|
238
|
+
|
|
239
|
+
### Sandbox: "Recipient has not opted in"
|
|
240
|
+
|
|
241
|
+
Error code 63007 from Twilio means the recipient hasn't joined the sandbox:
|
|
242
|
+
1. Have them send "join \<keyword\>" to +1 415 523 8886
|
|
243
|
+
2. Verify the keyword matches what's shown in Twilio Console
|
|
244
|
+
3. They must send from the exact WhatsApp number you're trying to reach
|
|
245
|
+
|
|
246
|
+
### Messages not arriving at handler
|
|
247
|
+
|
|
248
|
+
1. Check tunnel: `curl https://your-tunnel-url/health`
|
|
249
|
+
2. Check Twilio webhook config matches your tunnel URL
|
|
250
|
+
3. Check handler is running: `curl http://localhost:3002/health`
|
|
251
|
+
4. Check handler logs: `tail -f logs/whatsapp-handler-stderr.log`
|
|
252
|
+
|
|
253
|
+
### "Template not found" in production mode
|
|
254
|
+
|
|
255
|
+
1. Templates must be approved by WhatsApp/Meta before use
|
|
256
|
+
2. Verify template name matches exactly (case-sensitive)
|
|
257
|
+
3. Verify template variables match the approved format
|
|
258
|
+
|
|
259
|
+
### Media not sending
|
|
260
|
+
|
|
261
|
+
1. Media URL must be publicly accessible (Twilio fetches it)
|
|
262
|
+
2. Supported types: images (JPEG, PNG), documents (PDF), audio (OGG)
|
|
263
|
+
3. Check Twilio's [media size limits](https://www.twilio.com/docs/whatsapp/guidance-whatsapp-media-messages)
|
|
264
|
+
|
|
265
|
+
---
|
|
266
|
+
|
|
267
|
+
## Key Files
|
|
268
|
+
|
|
269
|
+
| File | Purpose |
|
|
270
|
+
|---|---|
|
|
271
|
+
| `scripts/send-whatsapp.sh` | Outbound WhatsApp sender (sandbox + production) |
|
|
272
|
+
| `scripts/whatsapp-handler.mjs` | Inbound WhatsApp webhook handler |
|
|
273
|
+
| `scripts/configure-whatsapp-sandbox.sh` | Sandbox webhook configuration helper |
|
|
274
|
+
| `config/caller-id-map.yaml` | WhatsApp number → identity mapping |
|
|
275
|
+
|
|
276
|
+
---
|
|
277
|
+
|
|
278
|
+
## Related Documents
|
|
279
|
+
|
|
280
|
+
- [Voice & SMS Setup](voice-sms-setup.md) — Twilio account setup, SMS, and voice calls
|
|
281
|
+
- [Outbound Governance Setup](outbound-governance-setup.md) — Dedup and validation pipeline
|
|
282
|
+
- [Poller & Daemon Setup](poller-daemon-setup.md) — How WhatsApp events integrate with the event loop
|
|
@@ -393,6 +393,17 @@ echo "=============================="
|
|
|
393
393
|
- [ ] Log rotation configured
|
|
394
394
|
- [ ] Health check runs cleanly
|
|
395
395
|
- [ ] First morning brief generated successfully
|
|
396
|
+
- [ ] (Optional) Voice/SMS configured -- see `docs/guides/voice-sms-setup.md`
|
|
397
|
+
|
|
398
|
+
---
|
|
399
|
+
|
|
400
|
+
## Optional: Voice & SMS Setup
|
|
401
|
+
|
|
402
|
+
If this agent needs phone call or SMS capabilities (Slack huddle voice participation, inbound/outbound SMS via Twilio), follow the dedicated guide:
|
|
403
|
+
|
|
404
|
+
**[Voice & SMS Setup Guide](../guides/voice-sms-setup.md)**
|
|
405
|
+
|
|
406
|
+
This covers Twilio phone number purchase, inbound SMS webhook, outbound SMS, Slack huddle voice (Deepgram STT + ElevenLabs TTS), Cloudflare tunnel exposure, caller ID mapping, and launchd process management for the voice/SMS services.
|
|
396
407
|
|
|
397
408
|
---
|
|
398
409
|
|
|
@@ -400,5 +411,15 @@ Related documents:
|
|
|
400
411
|
|
|
401
412
|
- `docs/runbooks/perpetual-operations.md` -- How the system runs continuously
|
|
402
413
|
- `docs/runbooks/recovery-and-failover.md` -- Recovery procedures
|
|
414
|
+
- `docs/guides/agent-persona-setup.md` -- Agent identity and configuration
|
|
415
|
+
- `docs/guides/email-setup.md` -- Gmail IMAP and SMTP setup
|
|
416
|
+
- `docs/guides/slack-setup.md` -- Slack app, tokens, and events
|
|
417
|
+
- `docs/guides/whatsapp-setup.md` -- WhatsApp Business integration
|
|
418
|
+
- `docs/guides/voice-sms-setup.md` -- Voice calls and SMS setup
|
|
419
|
+
- `docs/guides/poller-daemon-setup.md` -- Event loop, daemon, triggers, watchdog
|
|
420
|
+
- `docs/guides/outbound-governance-setup.md` -- Hooks, dedup, information barriers
|
|
421
|
+
- `docs/guides/rag-context-setup.md` -- Search index and context retrieval
|
|
422
|
+
- `docs/guides/pdf-generation-setup.md` -- Branded PDF documents
|
|
423
|
+
- `docs/guides/media-generation-setup.md` -- AI-generated visual assets
|
|
403
424
|
- `docs/workflows/executive-cadence.md` -- Full operating rhythm
|
|
404
425
|
- `config/environment.yaml` -- Environment configuration reference
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@adaptic/maestro",
|
|
3
|
-
"version": "1.1
|
|
3
|
+
"version": "1.4.1",
|
|
4
4
|
"description": "Maestro — Autonomous AI agent operating system. Deploy AI employees on dedicated Mac minis.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"bin": {
|
|
@@ -11,6 +11,7 @@
|
|
|
11
11
|
"./tools": "./lib/tool-definitions.js",
|
|
12
12
|
"./executor": "./lib/action-executor.js",
|
|
13
13
|
"./singleton": "./lib/singleton.js",
|
|
14
|
+
"./tts": "./lib/tts.mjs",
|
|
14
15
|
"./package.json": "./package.json"
|
|
15
16
|
},
|
|
16
17
|
"files": [
|