@intent-systems/nexus 2026.1.5-3

package/docs/gmail-pubsub.md

@@ -0,0 +1,191 @@
+---
+summary: "Gmail Pub/Sub push wired into Nexus webhooks via gogcli"
+read_when:
+  - Wiring Gmail inbox triggers to Nexus
+  - Setting up Pub/Sub push for agent wake
+---
+
+# Gmail Pub/Sub -> Nexus
+
+Goal: Gmail watch -> Pub/Sub push -> `gog gmail watch serve` -> Nexus webhook.
+
+## Prereqs
+
+- `gcloud` installed and logged in.
+- `gog` (gogcli) installed and authorized for the Gmail account.
+- Nexus hooks enabled (see [`docs/webhook.md`](https://docs.nexus.bot/webhook)).
+- `tailscale` logged in if you want a public HTTPS endpoint via Funnel.
+
+Example hook config (enable Gmail preset mapping):
+
+```json5
+{
+  hooks: {
+    enabled: true,
+    token: "NEXUS_HOOK_TOKEN",
+    path: "/hooks",
+    presets: ["gmail"]
+  }
+}
+```
+
+To customize payload handling, add `hooks.mappings` or a JS/TS transform module
+under `hooks.transformsDir` (see [`docs/webhook.md`](https://docs.nexus.bot/webhook)).
+
+## Wizard (recommended)
+
+Use the Nexus helper to wire everything together (installs deps on macOS via brew):
+
+```bash
+nexus hooks gmail setup \
+  --account nexus@gmail.com
+```
+
+Defaults:
+- Uses Tailscale Funnel for the public push endpoint.
+- Writes `hooks.gmail` config for `nexus hooks gmail run`.
+- Enables the Gmail hook preset (`hooks.presets: ["gmail"]`).
+
+Path note: when `tailscale.mode` is enabled, Nexus automatically sets
+`hooks.gmail.serve.path` to `/` and keeps the public path at
+`hooks.gmail.tailscale.path` (default `/gmail-pubsub`) because Tailscale
+strips the set-path prefix before proxying.
+
+Want a custom endpoint? Use `--push-endpoint <url>` or `--tailscale off`.
+
+Platform note: on macOS the wizard installs `gcloud`, `gogcli`, and `tailscale`
+via Homebrew; on Linux install them manually first.
+
+Gateway auto-start (recommended):
+- When `hooks.enabled=true` and `hooks.gmail.account` is set, the Gateway starts
+  `gog gmail watch serve` on boot and auto-renews the watch.
+- Set `NEXUS_SKIP_GMAIL_WATCHER=1` to opt out (useful if you run the daemon yourself).
+- Do not run the manual daemon at the same time, or you will hit
+  `listen tcp 127.0.0.1:8788: bind: address already in use`.
+
+Manual daemon (starts `gog gmail watch serve` + auto-renew):
+
+```bash
+nexus hooks gmail run
+```
+
+## One-time setup
+
+1) Select the GCP project **that owns the OAuth client** used by `gog`.
+
+```bash
+gcloud auth login
+gcloud config set project <project-id>
+```
+
+Note: Gmail watch requires the Pub/Sub topic to live in the same project as the OAuth client.
+
+2) Enable APIs:
+
+```bash
+gcloud services enable gmail.googleapis.com pubsub.googleapis.com
+```
+
+3) Create a topic:
+
+```bash
+gcloud pubsub topics create gog-gmail-watch
+```
+
+4) Allow Gmail push to publish:
+
+```bash
+gcloud pubsub topics add-iam-policy-binding gog-gmail-watch \
+  --member=serviceAccount:gmail-api-push@system.gserviceaccount.com \
+  --role=roles/pubsub.publisher
+```
+
+## Start the watch
+
+```bash
+gog gmail watch start \
+  --account nexus@gmail.com \
+  --label INBOX \
+  --topic projects/<project-id>/topics/gog-gmail-watch
+```
+
+Save the `history_id` from the output (for debugging).
+
+## Run the push handler
+
+Local example (shared token auth):
+
+```bash
+gog gmail watch serve \
+  --account nexus@gmail.com \
+  --bind 127.0.0.1 \
+  --port 8788 \
+  --path /gmail-pubsub \
+  --token <shared> \
+  --hook-url http://127.0.0.1:18789/hooks/gmail \
+  --hook-token NEXUS_HOOK_TOKEN \
+  --include-body \
+  --max-bytes 20000
+```
+
+Notes:
+- `--token` protects the push endpoint (`x-gog-token` or `?token=`).
+- `--hook-url` points to Nexus `/hooks/gmail` (mapped; isolated run + summary to main).
+- `--include-body` and `--max-bytes` control the body snippet sent to Nexus.
+
+Recommended: `nexus hooks gmail run` wraps the same flow and auto-renews the watch.
+
+## Expose the handler (dev)
+
+For local testing, tunnel the handler and use the public URL in the push subscription:
+
+```bash
+cloudflared tunnel --url http://127.0.0.1:8788 --no-autoupdate
+```
+
+Use the generated URL as the push endpoint:
+
+```bash
+gcloud pubsub subscriptions create gog-gmail-watch-push \
+  --topic gog-gmail-watch \
+  --push-endpoint "https://<public-url>/gmail-pubsub?token=<shared>"
+```
+
+Production: use a stable HTTPS endpoint and configure Pub/Sub OIDC JWT, then run:
+
+```bash
+gog gmail watch serve --verify-oidc --oidc-email <svc@...>
+```
+
+## Test
+
+Send a message to the watched inbox:
+
+```bash
+gog gmail send \
+  --account nexus@gmail.com \
+  --to nexus@gmail.com \
+  --subject "watch test" \
+  --body "ping"
+```
+
+Check watch state and history:
+
+```bash
+gog gmail watch status --account nexus@gmail.com
+gog gmail history --account nexus@gmail.com --since <historyId>
+```
+
+## Troubleshooting
+
+- `Invalid topicName`: project mismatch (topic not in the OAuth client project).
+- `User not authorized`: missing `roles/pubsub.publisher` on the topic.
+- Empty messages: Gmail push only provides `historyId`; fetch via `gog gmail history`.
+
+## Cleanup
+
+```bash
+gog gmail watch stop --account nexus@gmail.com
+gcloud pubsub subscriptions delete gog-gmail-watch-push
+gcloud pubsub topics delete gog-gmail-watch
+```

package/docs/grammy.md

@@ -0,0 +1,27 @@
+---
+summary: "Telegram Bot API integration via grammY with setup notes"
+read_when:
+  - Working on Telegram or grammY pathways
+---
+# grammY Integration (Telegram Bot API)
+
+Updated: 2025-12-07
+
+# Why grammY
+- TS-first Bot API client with built-in long-poll + webhook helpers, middleware, error handling, rate limiter.
+- Cleaner media helpers than hand-rolling fetch + FormData; supports all Bot API methods.
+- Extensible: proxy support via custom fetch, session middleware (optional), type-safe context.
+
+# What we shipped
+- **Single client path:** fetch-based implementation removed; grammY is now the sole Telegram client (send + gateway) with the grammY throttler enabled by default.
+- **Gateway:** `monitorTelegramProvider` builds a grammY `Bot`, wires mention/allowlist gating, media download via `getFile`/`download`, and delivers replies with `sendMessage/sendPhoto/sendVideo/sendAudio/sendDocument`. Supports long-poll or webhook via `webhookCallback`.
+- **Proxy:** optional `telegram.proxy` uses `undici.ProxyAgent` through grammY’s `client.baseFetch`.
+- **Webhook support:** `webhook-set.ts` wraps `setWebhook/deleteWebhook`; `webhook.ts` hosts the callback with health + graceful shutdown. Gateway enables webhook mode when `telegram.webhookUrl` is set (otherwise it long-polls).
+- **Sessions:** direct chats collapse into the agent main session (`agent:<agentId>:<mainKey>`); groups use `agent:<agentId>:telegram:group:<chatId>`; replies route back to the same provider.
+- **Config knobs:** `telegram.botToken`, `telegram.dmPolicy`, `telegram.groups` (allowlist + mention defaults), `telegram.allowFrom`, `telegram.groupAllowFrom`, `telegram.groupPolicy`, `telegram.mediaMaxMb`, `telegram.proxy`, `telegram.webhookSecret`, `telegram.webhookUrl`.
+- **Tests:** grammy mocks cover DM + group mention gating and outbound send; more media/webhook fixtures still welcome.
+
+Open questions
+- Optional grammY plugins (throttler) if we hit Bot API 429s.
+- Add more structured media tests (stickers, voice notes).
+- Make webhook listen port configurable (currently fixed to 8787 unless wired through the gateway).

package/docs/group-messages.md

@@ -0,0 +1,73 @@
+---
+summary: "Behavior and config for WhatsApp group message handling (mentionPatterns are shared across surfaces)"
+read_when:
+  - Changing group message rules or mentions
+---
+# Group messages (web provider)
+
+Goal: let Nexus sit in WhatsApp groups, wake up only when pinged, and keep that thread separate from the personal DM session.
+
+Note: `routing.groupChat.mentionPatterns` is now used by Telegram/Discord/Slack/iMessage as well; this doc focuses on WhatsApp-specific behavior.
+
+## What’s implemented (2025-12-03)
+- Activation modes: `mention` (default) or `always`. `mention` requires a ping (real WhatsApp @-mentions via `mentionedJids`, regex patterns, or the bot’s E.164 anywhere in the text). `always` wakes the agent on every message but it should reply only when it can add meaningful value; otherwise it returns the silent token `NO_REPLY`. Defaults can be set in config (`whatsapp.groups`) and overridden per group via `/activation`. When `whatsapp.groups` is set, it also acts as a group allowlist (include `"*"` to allow all).
+- Group policy: `whatsapp.groupPolicy` controls whether group messages are accepted (`open|disabled|allowlist`). `allowlist` uses `whatsapp.groupAllowFrom` (fallback: explicit `whatsapp.allowFrom`).
+- Per-group sessions: session keys look like `whatsapp:group:<jid>` so commands such as `/verbose on` or `/think:high` are scoped to that group; personal DM state is untouched. Heartbeats are skipped for group threads.
+- Context injection: last N (default 50) group messages are prefixed under `[Chat messages since your last reply - for context]`, with the triggering line under `[Current message - respond to this]`.
+- Sender surfacing: every group batch now ends with `[from: Sender Name (+E164)]` so Pi knows who is speaking.
+- Ephemeral/view-once: we unwrap those before extracting text/mentions, so pings inside them still trigger.
+- Group system prompt: on the first turn of a group session (and whenever `/activation` changes the mode) we inject a short blurb into the system prompt like `You are replying inside the WhatsApp group "<subject>". Group members: Alice (+44...), Bob (+43...), … Activation: trigger-only … Address the specific sender noted in the message context.` If metadata isn’t available we still tell the agent it’s a group chat.
+
+## Config for Nexus UK (+447700900123)
+Add a `groupChat` block to `~/nexus/state/nexus.json` so display-name pings work even when WhatsApp strips the visual `@` in the text body:
+
+```json5
+{
+  "whatsapp": {
+    "groups": {
+      "*": { "requireMention": true }
+    }
+  },
+  "routing": {
+    "groupChat": {
+      "historyLimit": 50,
+      "mentionPatterns": [
+        "@?nexus",
+        "@?nexus\\s*uk",
+        "@?nexus",
+        "\\+?447700900123"
+      ]
+    }
+  }
+}
+```
+
+Notes:
+- The regexes are case-insensitive; they cover `@nexus`, `@nexus uk`, `nexus`, and the raw number with or without `+`/spaces.
+- WhatsApp still sends canonical mentions via `mentionedJids` when someone taps the contact, so the number fallback is rarely needed but is a good safety net.
+
+### Activation command (owner-only)
+
+Use the group chat command:
+- `/activation mention`
+- `/activation always`
+
+Only the owner number (from `whatsapp.allowFrom`, or the bot’s own E.164 when unset) can change this. `/status` in the group shows the current activation mode.
+
+## How to use
+1) Add Nexus UK (`+447700900123`) to the group.
+2) Say `@nexus …` (or `@nexus uk`, `@nexus`, or include the number). Anyone in the group can trigger it.
+3) The agent prompt will include recent group context plus the trailing `[from: …]` marker so it can address the right person.
+4) Session-level directives (`/verbose on`, `/think:high`, `/new` or `/reset`, `/compact`) apply only to that group’s session; your personal DM session remains independent.
+
+## Testing / verification
+- Automated: `pnpm test -- src/web/auto-reply.test.ts --runInBand` (covers mention gating, history injection, sender suffix).
+- Manual smoke:
+  - Send an `@nexus` ping in the group and confirm a reply that references the sender name.
+  - Send a second ping and verify the history block is included then cleared on the next turn.
+- Check gateway logs (run with `--verbose`) to see `inbound web message` entries showing `from: <groupJid>` and the `[from: …]` suffix.
+
+## Known considerations
+- Heartbeats are intentionally skipped for groups to avoid noisy broadcasts.
+- Echo suppression uses the combined batch string; if you send identical text twice without mentions, only the first will get a response.
+- Session store entries will appear as `agent:<agentId>:whatsapp:group:<jid>` in the session store (`~/nexus/state/agents/<agentId>/sessions/sessions.json` by default); a missing entry just means the group hasn’t triggered a run yet.

package/docs/groups.md

@@ -0,0 +1,130 @@
+---
+summary: "Group chat behavior across surfaces (WhatsApp/Telegram/Discord/Slack/Signal/iMessage)"
+read_when:
+  - Changing group chat behavior or mention gating
+---
+# Groups
+
+Nexus treats group chats consistently across surfaces: WhatsApp, Telegram, Discord, Slack, Signal, iMessage.
+
+## Session keys
+- Group sessions use `agent:<agentId>:<provider>:group:<id>` session keys (rooms/channels use `agent:<agentId>:<provider>:channel:<id>`).
+- Direct chats use the main session (or per-sender if configured).
+- Heartbeats are skipped for group sessions.
+
+## Display labels
+- UI labels use `displayName` when available, formatted as `<provider>:<token>`.
+- `#room` is reserved for rooms/channels; group chats use `g-<slug>` (lowercase, spaces -> `-`, keep `#@+._-`).
+
+## Group policy
+Control how group/room messages are handled per provider:
+
+```json5
+{
+  whatsapp: {
+    groupPolicy: "disabled", // "open" | "disabled" | "allowlist"
+    groupAllowFrom: ["+15551234567"]
+  },
+  telegram: {
+    groupPolicy: "disabled",
+    groupAllowFrom: ["123456789", "@username"]
+  },
+  signal: {
+    groupPolicy: "disabled",
+    groupAllowFrom: ["+15551234567"]
+  },
+  imessage: {
+    groupPolicy: "disabled",
+    groupAllowFrom: ["chat_id:123"]
+  },
+  discord: {
+    groupPolicy: "allowlist",
+    guilds: {
+      "GUILD_ID": { channels: { help: { allow: true } } }
+    }
+  },
+  slack: {
+    groupPolicy: "allowlist",
+    channels: { "#general": { allow: true } }
+  }
+}
+```
+
+| Policy | Behavior |
+|--------|----------|
+| `"open"` | Default. Groups bypass allowlists; mention-gating still applies. |
+| `"disabled"` | Block all group messages entirely. |
+| `"allowlist"` | Only allow groups/rooms that match the configured allowlist. |
+
+Notes:
+- `groupPolicy` is separate from mention-gating (which requires @mentions).
+- WhatsApp/Telegram/Signal/iMessage: use `groupAllowFrom` (fallback: explicit `allowFrom`).
+- Discord: allowlist uses `discord.guilds.<id>.channels`.
+- Slack: allowlist uses `slack.channels`.
+- Group DMs are controlled separately (`discord.dm.*`, `slack.dm.*`).
+- Telegram allowlist can match user IDs (`"123456789"`, `"telegram:123456789"`, `"tg:123456789"`) or usernames (`"@alice"` or `"alice"`); prefixes are case-insensitive.
+
+## Mention gating (default)
+Group messages require a mention unless overridden per group. Defaults live per subsystem under `*.groups."*"`.
+
+```json5
+{
+  whatsapp: {
+    groups: {
+      "*": { requireMention: true },
+      "123@g.us": { requireMention: false }
+    }
+  },
+  telegram: {
+    groups: {
+      "*": { requireMention: true },
+      "123456789": { requireMention: false }
+    }
+  },
+  imessage: {
+    groups: {
+      "*": { requireMention: true },
+      "123": { requireMention: false }
+    }
+  },
+  routing: {
+    groupChat: {
+      mentionPatterns: ["@nexus", "nexus", "\\+15555550123"],
+      historyLimit: 50
+    }
+  }
+}
+```
+
+Notes:
+- `mentionPatterns` are case-insensitive regexes.
+- Surfaces that provide explicit mentions still pass; patterns are a fallback.
+- Mention gating is only enforced when mention detection is possible (native mentions or `mentionPatterns` are configured).
+- Discord defaults live in `discord.guilds."*"` (overridable per guild/channel).
+
+## Group allowlists
+When `whatsapp.groups`, `telegram.groups`, or `imessage.groups` is configured, the keys act as a group allowlist. Use `"*"` to allow all groups while still setting default mention behavior.
+
+## Activation (owner-only)
+Group owners can toggle per-group activation:
+- `/activation mention`
+- `/activation always`
+
+Owner is determined by `whatsapp.allowFrom` (or the bot’s self E.164 when unset). Other surfaces currently ignore `/activation`.
+
+## Context fields
+Group inbound payloads set:
+- `ChatType=group`
+- `GroupSubject` (if known)
+- `GroupMembers` (if known)
+- `WasMentioned` (mention gating result)
+
+The agent system prompt includes a group intro on the first turn of a new group session.
+
+## iMessage specifics
+- Prefer `chat_id:<id>` when routing or allowlisting.
+- List chats: `imsg chats --limit 20`.
+- Group replies always go back to the same `chat_id`.
+
+## WhatsApp specifics
+See [`docs/group-messages.md`](https://docs.nexus.bot/group-messages) for WhatsApp-only behavior (history injection, mention handling details).

package/docs/health.md

@@ -0,0 +1,28 @@
+---
+summary: "Health check steps for Baileys/WhatsApp connectivity"
+read_when:
+  - Diagnosing web provider health
+---
+# Health Checks (CLI)
+
+Short guide to verify the WhatsApp Web / Baileys stack without guessing.
+
+## Quick checks
+- `nexus status` — local summary: whether creds exist, auth age, session store path + recent sessions.
+- `nexus status --deep` — also probes the running Gateway (WhatsApp connect + Telegram + Discord APIs).
+- `nexus health --json` — asks the running Gateway for a full health snapshot (WS-only; no direct Baileys socket).
+- Send `/status` in WhatsApp/WebChat to get a status reply without invoking the agent.
+- Logs: tail `~/nexus/state/logs/nexus-*.log` and filter for `web-heartbeat`, `web-reconnect`, `web-auto-reply`, `web-inbound`.
+
+## Deep diagnostics
+- Creds on disk: `ls -l ~/nexus/state/credentials/whatsapp/<accountId>/creds.json` (mtime should be recent).
+- Session store: `ls -l ~/nexus/state/agents/<agentId>/sessions/sessions.json` (path can be overridden in config). Count and recent recipients are surfaced via `status`.
+- Relink flow: `nexus logout && nexus login --verbose` when status codes 409–515 or `loggedOut` appear in logs. (Note: the QR login flow auto-restarts once for status 515 after pairing.)
+
+## When something fails
+- `logged out` or status 409–515 → relink with `nexus logout` then `nexus login`.
+- Gateway unreachable → start it: `nexus gateway --port 18789` (use `--force` if the port is busy).
+- No inbound messages → confirm linked phone is online and the sender is allowed (`whatsapp.allowFrom`); for group chats, ensure allowlist + mention rules match (`whatsapp.groups`, `routing.groupChat.mentionPatterns`).
+
+## Dedicated "health" command
+`nexus health --json` asks the running Gateway for its health snapshot (no direct Baileys socket from the CLI). It reports linked creds, auth age, Baileys connect result/status code, session-store summary, and a probe duration. It exits non-zero if the Gateway is unreachable or the probe fails/timeouts. Use `--timeout <ms>` to override the 10s default.

package/docs/heartbeat.md

@@ -0,0 +1,73 @@
+---
+summary: "Plan for heartbeat polling messages and notification rules"
+read_when:
+  - Adjusting heartbeat cadence or messaging
+---
+# Heartbeat (Gateway)
+
+Heartbeat runs periodic agent turns in the **main session** so the model can
+surface anything that needs attention without spamming the user.
+
+## Prompt contract
+- Heartbeat body defaults to `HEARTBEAT` (configurable via `agent.heartbeat.prompt`).
+- If nothing needs attention, the model should reply `HEARTBEAT_OK`.
+- During heartbeat runs, Nexus treats `HEARTBEAT_OK` as an ack when it appears at
+  the **start or end** of the reply. Nexus strips the token and discards the
+  reply if the remaining content is **≤ `ackMaxChars`** (default: 30).
+- If `HEARTBEAT_OK` is in the **middle** of a reply, it is not treated specially.
+- For alerts, do **not** include `HEARTBEAT_OK`; return only the alert text.
+
+### Stray `HEARTBEAT_OK` outside heartbeats
+If the model accidentally includes `HEARTBEAT_OK` at the start or end of a
+normal (non-heartbeat) reply, Nexus strips the token and logs a verbose
+message. If the reply is only `HEARTBEAT_OK`, it is dropped.
+
+### Outbound normalization (all providers)
+For **all providers** (WhatsApp/Web, Telegram, Slack, Discord, Signal, iMessage),
+Nexus applies the same filtering to tool summaries, streaming block replies,
+and final replies:
+- drop payloads that are only `HEARTBEAT_OK` with no media
+- strip `HEARTBEAT_OK` at the edges when mixed with other text
+
+## Config
+
+```json5
+{
+  agent: {
+    heartbeat: {
+      every: "30m",           // duration string: ms|s|m|h (0m disables)
+      model: "anthropic/claude-opus-4-5",
+      target: "last",          // last | whatsapp | telegram | none
+      to: "+15551234567",      // optional override for whatsapp/telegram
+      prompt: "HEARTBEAT",     // optional override
+      ackMaxChars: 30          // max chars allowed after HEARTBEAT_OK
+    }
+  }
+}
+```
+
+### Fields
+- `every`: heartbeat interval (duration string; default unit minutes). Omit or set
+  to `0m` to disable.
+- `model`: optional model override for heartbeat runs (`provider/model`).
+- `target`: where heartbeat output is delivered.
+  - `last` (default): send to the last used external provider.
+  - `whatsapp` / `telegram`: force the provider (optionally set `to`).
+  - `none`: do not deliver externally; output stays in the session (WebChat-visible).
+- `to`: optional recipient override (E.164 for WhatsApp, chat id for Telegram).
+- `prompt`: optional override for the heartbeat body (default: `HEARTBEAT`).
+- `ackMaxChars`: max chars allowed after `HEARTBEAT_OK` before delivery (default: 30).
+
+## Behavior
+- Runs in the main session (`main`, or `global` when scope is global).
+- Uses the main lane queue; if requests are in flight, the wake is retried.
+- Empty output or `HEARTBEAT_OK` is treated as “ok” and does **not** keep the
+  session alive (`updatedAt` is restored).
+- If `target` resolves to no external destination (no last route or `none`), the
+  heartbeat still runs but no outbound message is sent.
+
+## Wake hook
+- The gateway exposes a heartbeat wake hook so cron/jobs/webhooks can request an
+  immediate run (`requestHeartbeatNow`).
+- `wake` endpoints should enqueue system events and optionally trigger a wake; the
+  heartbeat runner picks those up on the next tick or immediately.