@intent-systems/nexus 2026.1.5-3

package/docs/usage-suggestions-pipeline.md

@@ -0,0 +1,126 @@
+# Nexus Usage Event Log + Suggestions (Local Spec)
+
+This document captures the local ingestion, analysis, and suggestion pipeline
+for Nexus CLI usage. The goal is to make the system concrete and testable
+locally before any cloud aggregation.
+
+## Goals
+
+- Persist ordered usage events for every Nexus CLI invocation.
+- Track tool and skill usage from agent event streams.
+- Provide deterministic, local suggestions based on real sequences.
+- Keep the system simple to backfill and audit.
+
+## Non-Goals (for now)
+
+- Cloud aggregation or anonymization.
+- ML-driven recommendation models.
+- Cross-device sync of usage data.
+
+## Data Sources
+
+### CLI Events (Nexus Core)
+
+Captured by the CLI entrypoint and command hooks:
+- `cli_session_start`
+- `command_started`
+- `command_finished`
+- `command_failed`
+- `cli_session_end`
+
+### Agent Tool/Skill Events
+
+Agent tool lifecycle events are persisted via the existing `emitAgentEvent`
+stream and a log listener.
+
+## Storage (Local)
+
+Events are written as JSONL in the Nexus state directory:
+
+```
+~/nexus/state/events/events-YYYY-MM-DD.jsonl
+```
+
+Configuration:
+- `NEXUS_EVENT_LOG=0|false|off` disables event logging
+- `NEXUS_EVENT_LOG_DIR=/custom/path` overrides the event log directory
+
+## Event Schema (JSONL)
+
+Each line is a single JSON object. Minimal shape:
+
+```json
+{
+  "id": "uuid",
+  "ts": 1712345678901,
+  "seq": 12,
+  "session_id": "uuid",
+  "source": "nexus_cli",
+  "event_type": "command_finished",
+  "command_path": "status",
+  "argv": ["nexus", "status", "--json"],
+  "cwd": "/Users/tyler/nexus",
+  "status": "ok",
+  "data": { "version": "2026.1.5-3" },
+  "schema_version": 1
+}
+```
+
+Notes:
+- `seq` is monotonic per session.
+- `session_id` groups all events for a single CLI run.
+- `argv` is sanitized (sensitive flags are redacted).
+
+## Analysis Pipeline (Local)
+
+We use an n-gram style transition table built from ordered events.
+
+### Key Extraction
+
+- CLI usage key: `source:cli:<command_path>`
+- Tool usage key: `source:tool:<tool_name>` from agent tool stream
+
+### Transition Table
+
+For each session sequence:
+```
+key[i] -> key[i+1] (count)
+```
+
+### Ranking
+
+Scores are proportional to transition frequency:
+```
+score = count / total_transitions_from_last_key
+```
+
+### Fallbacks
+
+If no transitions exist for the last key, fall back to global top keys
+by frequency.
+
+## Suggestion Scope
+
+- **global**: use all local sessions (default)
+- **session**: restrict to a specific session
+
+## Surfaces (Current)
+
+- `nexus suggestions` CLI command with JSON output
+
+Potential future surfaces:
+- Inline suggestion after command completion
+- Suggestions in `nexus status`
+
+## Comms Import (Optional Consumer)
+
+The Comms adapter can import `~/nexus/state/events/*.jsonl` into
+`comms.events` for analysis or cross-source correlation.
+
+## Next Steps
+
+- Add `invocation_kind` (direct vs wrapper vs cargo) if needed.
+- Add richer tool/skill metadata in `data` for better ranking.
+- Add optional gating on `status` (only successful commands).
+
+See `docs/usage-cloud-aggregation-spec.md` for the cloud schema and upload plan.

package/docs/voicewake.md

@@ -0,0 +1,61 @@
+---
+summary: "Global voice wake words (Gateway-owned) and how they sync across nodes"
+read_when:
+  - Changing voice wake words behavior or defaults
+  - Adding new node platforms that need wake word sync
+---
+# Voice Wake (Global Wake Words)
+
+Nexus treats **wake words as a single global list** owned by the **Gateway**.
+
+- There are **no per-node custom wake words**.
+- **Any node/app UI may edit** the list; changes are persisted by the Gateway and broadcast to everyone.
+- Each device still keeps its own **Voice Wake enabled/disabled** toggle (local UX + permissions differ).
+
+## Storage (Gateway host)
+
+Wake words are stored on the gateway machine at:
+
+- `~/nexus/state/settings/voicewake.json`
+
+Shape:
+
+```json
+{ "triggers": ["nexus", "claude", "computer"], "updatedAtMs": 1730000000000 }
+```
+
+## Protocol
+
+### Methods
+
+- `voicewake.get` → `{ triggers: string[] }`
+- `voicewake.set` with params `{ triggers: string[] }` → `{ triggers: string[] }`
+
+Notes:
+- Triggers are normalized (trimmed, empties dropped). Empty lists fall back to defaults.
+- Limits are enforced for safety (count/length caps).
+
+### Events
+
+- `voicewake.changed` payload `{ triggers: string[] }`
+
+Who receives it:
+- All WebSocket clients (macOS app, WebChat, etc.)
+- All connected bridge nodes (iOS/Android), and also on node connect as an initial “current state” push.
+
+## Client behavior
+
+### macOS app
+
+- Uses the global list to gate `VoiceWakeRuntime` triggers.
+- Editing “Trigger words” in Voice Wake settings calls `voicewake.set` and then relies on the broadcast to keep other clients in sync.
+
+### iOS node
+
+- Uses the global list for `VoiceWakeManager` trigger detection.
+- Editing Wake Words in Settings calls `voicewake.set` (over the bridge) and also keeps local wake-word detection responsive.
+
+### Android node
+
+- Exposes a Wake Words editor in Settings.
+- Calls `voicewake.set` over the bridge so edits sync everywhere.

package/docs/web.md

@@ -0,0 +1,115 @@
+---
+summary: "Gateway web surfaces: Control UI, bind modes, and security"
+read_when:
+  - You want to access the Gateway over Tailscale
+  - You want the browser Control UI and config editing
+---
+# Web (Gateway)
+
+The Gateway serves a small **browser Control UI** (Vite + Lit) from the same port as the Gateway WebSocket:
+
+- default: `http://<host>:18789/`
+- optional prefix: set `gateway.controlUi.basePath` (e.g. `/nexus`)
+
+The UI talks directly to the Gateway WS and supports:
+- Chat (`chat.history`, `chat.send`, `chat.abort`)
+- Chat tool cards (agent tool events)
+- Connections (provider status, WhatsApp QR, Telegram config)
+- Instances (`system-presence`)
+- Sessions (`sessions.list`, `sessions.patch`)
+- Cron (`cron.*`)
+- Skills (`skills.status`, `skills.update`, `skills.install`)
+- Nodes (`node.list`, `node.describe`, `node.invoke`)
+- Config (`config.get`, `config.set`) for `~/nexus/state/nexus.json`
+- Debug (status/health/models snapshots + manual calls)
+
+## Webhooks
+
+When `hooks.enabled=true`, the Gateway also exposes a small webhook endpoint on the same HTTP server.
+See [`docs/configuration.md`](https://docs.nexus.bot/configuration) → `hooks` for auth + payloads.
+
+## Config (default-on)
+
+The Control UI is **enabled by default** when assets are present (`dist/control-ui`).
+You can control it via config:
+
+```json5
+{
+  gateway: {
+    controlUi: { enabled: true, basePath: "/nexus" } // basePath optional
+  }
+}
+```
+
+## Tailscale access
+
+### Integrated Serve (recommended)
+
+Keep the Gateway on loopback and let Tailscale Serve proxy it:
+
+```json5
+{
+  gateway: {
+    bind: "loopback",
+    tailscale: { mode: "serve" }
+  }
+}
+```
+
+Then start the gateway:
+
+```bash
+nexus gateway
+```
+
+Open:
+- `https://<magicdns>/` (or your configured `gateway.controlUi.basePath`)
+
+### Tailnet bind + token (legacy)
+
+```json5
+{
+  gateway: {
+    bind: "tailnet",
+    controlUi: { enabled: true }
+  }
+}
+```
+
+Then start the gateway (token required for non-loopback binds):
+
+```bash
+export NEXUS_GATEWAY_TOKEN="…your token…"
+nexus gateway
+```
+
+Open:
+- `http://<tailscale-ip>:18789/` (or your configured `gateway.controlUi.basePath`)
+
+### Public internet (Funnel)
+
+```json5
+{
+  gateway: {
+    bind: "loopback",
+    tailscale: { mode: "funnel" },
+    auth: { mode: "password" } // or NEXUS_GATEWAY_PASSWORD
+  }
+}
+```
+
+## Security notes
+
+- Binding the Gateway to a non-loopback address **requires** auth (`NEXUS_GATEWAY_TOKEN` or `gateway.auth`).
+- The UI sends `connect.params.auth.token` or `connect.params.auth.password`.
+- Use `gateway.auth.allowTailscale: false` to require explicit credentials even in Serve mode.
+- `gateway.tailscale.mode: "funnel"` requires `gateway.auth.mode: "password"` (shared password).
+
+## Building the UI
+
+The Gateway serves static files from `dist/control-ui`. Build them with:
+
+```bash
+bun run ui:install
+bun run ui:build
+```

package/docs/webchat.md

@@ -0,0 +1,34 @@
+---
+summary: "Loopback WebChat static host and Gateway WS usage for chat UI"
+read_when:
+  - Debugging or configuring WebChat access
+---
+# WebChat (SwiftUI + Gateway WS)
+
+Updated: 2025-12-17
+
+## What it is
+- A native SwiftUI chat UI (macOS app / iOS) that talks directly to the Gateway WebSocket.
+- No embedded browser/WKWebView and no bundled static WebChat HTTP server.
+- Data plane is entirely Gateway WS: methods `chat.history`, `chat.send`, `chat.abort`; events `chat`, `agent`, `presence`, `tick`, `health`.
+
+## How it connects
+- The UI performs Gateway WS `connect`, then calls `chat.history` for bootstrap and `chat.send` for sends; it listens to `chat/agent/presence/tick/health` events.
+- History comes from the Gateway via `chat.history` (no local session file watching).
+- If Gateway WS is unavailable, the UI surfaces the error and blocks send.
+
+## Remote use
+- In remote mode, the macOS app forwards the Gateway WebSocket control port via SSH and uses that for the SwiftUI chat UI.
+- You generally should not need to manage tunnels manually; see `RemoteTunnelManager` in the app.
+
+## Config
+- WebChat does not have a separate HTTP port/config anymore.
+- Gateway WS is configured via the app’s gateway endpoint settings (`GatewayEndpointStore`) or `nexus gateway --port` when running locally.
+
+## Failure handling
+- UI errors when the Gateway handshake fails or the WS drops.
+- No fallback transport; the Gateway WS is required.
+
+## Dev notes
+- macOS glue: `apps/macos/Sources/Nexus/WebChatSwiftUI.swift` + `apps/macos/Sources/Nexus/WebChatManager.swift`.
+- Remote tunnel helper: `apps/macos/Sources/Nexus/RemotePortTunnel.swift`.

package/docs/webhook.md

@@ -0,0 +1,132 @@
+---
+summary: "Webhook ingress for wake and isolated agent runs"
+read_when:
+  - Adding or changing webhook endpoints
+  - Wiring external systems into Nexus
+---
+
+# Webhooks
+
+Gateway can expose a small HTTP webhook endpoint for external triggers.
+
+## Enable
+
+```json5
+{
+  hooks: {
+    enabled: true,
+    token: "shared-secret",
+    path: "/hooks"
+  }
+}
+```
+
+Notes:
+- `hooks.token` is required when `hooks.enabled=true`.
+- `hooks.path` defaults to `/hooks`.
+
+## Auth
+
+Every request must include the hook token:
+- `Authorization: Bearer <token>`
+- or `x-nexus-token: <token>`
+- or `?token=<token>`
+
+## Endpoints
+
+### `POST /hooks/wake`
+
+Payload:
+```json
+{ "text": "System line", "mode": "now" }
+```
+
+- `text` required (string)
+- `mode` optional: `now` | `next-heartbeat` (default `now`)
+
+Effect:
+- Enqueues a system event for the **main** session
+- If `mode=now`, triggers an immediate heartbeat
+
+### `POST /hooks/agent`
+
+Payload:
+```json
+{
+  "message": "Run this",
+  "name": "Email",
+  "sessionKey": "hook:email:msg-123",
+  "wakeMode": "now",
+  "deliver": false,
+  "provider": "last",
+  "to": "+15551234567",
+  "thinking": "low",
+  "timeoutSeconds": 120
+}
+```
+
+- `message` required (string)
+- `name` optional (used in the summary prefix)
+- `sessionKey` optional (default random `hook:<uuid>`)
+- `wakeMode` optional: `now` | `next-heartbeat` (default `now`)
+- `deliver` optional (default `false`)
+- `provider` optional: `last` | `whatsapp` | `telegram`
+- `to` optional (provider-specific target)
+- `thinking` optional (override)
+- `timeoutSeconds` optional
+
+Effect:
+- Runs an **isolated** agent turn (own session key)
+- Always posts a summary into the **main** session
+- If `wakeMode=now`, triggers an immediate heartbeat
+
+### `POST /hooks/<name>` (mapped)
+
+Custom hook names are resolved via `hooks.mappings` (see configuration). A mapping can
+turn arbitrary payloads into `wake` or `agent` actions, with optional templates or
+code transforms.
+
+Mapping options (summary):
+- `hooks.presets: ["gmail"]` enables the built-in Gmail mapping.
+- `hooks.mappings` lets you define `match`, `action`, and templates in config.
+- `hooks.transformsDir` + `transform.module` loads a JS/TS module for custom logic.
+- Use `match.source` to keep a generic ingest endpoint (payload-driven routing).
+- TS transforms require a TS loader (e.g. `bun`) or precompiled `.js` at runtime.
+- `nexus hooks gmail setup` writes `hooks.gmail` config for `nexus hooks gmail run`.
+
+## Responses
+
+- `200` for `/hooks/wake`
+- `202` for `/hooks/agent` (async run started)
+- `401` on auth failure
+- `400` on invalid payload
+- `413` on oversized payloads
+
+## Examples
+
+```bash
+curl -X POST http://127.0.0.1:18789/hooks/wake \
+  -H 'Authorization: Bearer SECRET' \
+  -H 'Content-Type: application/json' \
+  -d '{"text":"New email received","mode":"now"}'
+```
+
+```bash
+curl -X POST http://127.0.0.1:18789/hooks/agent \
+  -H 'x-nexus-token: SECRET' \
+  -H 'Content-Type: application/json' \
+  -d '{"message":"Summarize inbox","name":"Email","wakeMode":"next-heartbeat"}'
+```
+
+```bash
+curl -X POST http://127.0.0.1:18789/hooks/gmail \
+  -H 'Authorization: Bearer SECRET' \
+  -H 'Content-Type: application/json' \
+  -d '{"source":"gmail","messages":[{"from":"Ada","subject":"Hello","snippet":"Hi"}]}'
+```
+
+## Security
+
+- Keep hook endpoints behind loopback, tailnet, or trusted reverse proxy.
+- Use a dedicated hook token; do not reuse gateway auth tokens.
+- Avoid including sensitive raw payloads in webhook logs.

package/docs/whatsapp.md

@@ -0,0 +1,170 @@
+---
+summary: "WhatsApp (web provider) integration: login, inbox, replies, media, and ops"
+read_when:
+  - Working on WhatsApp/web provider behavior or inbox routing
+---
+# WhatsApp (web provider)
+
+Updated: 2025-12-23
+
+Status: WhatsApp Web via Baileys only. Gateway owns the session(s).
+
+## Goals
+- Multiple WhatsApp accounts (multi-account) in one Gateway process.
+- Deterministic routing: replies return to WhatsApp, no model routing.
+- Model sees enough context to understand quoted replies.
+
+## Architecture (who owns what)
+- **Gateway** owns the Baileys socket and inbox loop.
+- **CLI / macOS app** talk to the gateway; no direct Baileys use.
+- **Active listener** is required for outbound sends; otherwise send fails fast.
+
+## Getting a phone number
+
+WhatsApp requires a real mobile number for verification. VoIP and virtual numbers are usually blocked.
+
+**Recommended approaches:**
+- **Local eSIM** from your country's mobile carrier (most reliable)
+  - Austria: [hot.at](https://www.hot.at)
+  - UK: [giffgaff](https://www.giffgaff.com) — free SIM, no contract
+- **Prepaid SIM** — cheap, just needs to receive one SMS for verification
+
+**Avoid:** TextNow, Google Voice, most "free SMS" services — WhatsApp blocks these aggressively.
+
+**Tip:** The number only needs to receive one verification SMS. After that, WhatsApp Web sessions persist via `creds.json`.
+
+**WhatsApp Business:** You can use WhatsApp Business on the same phone with a different number. This is a great option if you want to keep your personal WhatsApp separate — just install WhatsApp Business and register it with Nexus's dedicated number.
+
+## Login + credentials
+- Login command: `nexus login` (QR via Linked Devices).
+- Multi-account login: `nexus login --account <id>` (`<id>` = `accountId`).
+- Default account (when `--account` is omitted): `default` if present, otherwise the first configured account id (sorted).
+- Credentials stored in `~/nexus/state/credentials/whatsapp/<accountId>/creds.json`.
+- Backup copy at `creds.json.bak` (restored on corruption).
+- Legacy compatibility: older installs stored Baileys files directly in `~/nexus/state/credentials/`.
+- Logout: `nexus logout` (or `--account <id>`) deletes WhatsApp auth state (but keeps shared `oauth.json`).
+- Logged-out socket => error instructs re-link.
+
+## Inbound flow (DM + group)
+- WhatsApp events come from `messages.upsert` (Baileys).
+- Inbox listeners are detached on shutdown to avoid accumulating event handlers in tests/restarts.
+- Status/broadcast chats are ignored.
+- Direct chats use E.164; groups use group JID.
+- **DM policy**: `whatsapp.dmPolicy` controls direct chat access (default: `pairing`).
+  - Pairing: unknown senders get a pairing code (approve via `nexus pairing approve --provider whatsapp <code>`).
+  - Open: requires `whatsapp.allowFrom` to include `"*"`.
+  - Self messages are always allowed; “self-chat mode” still requires `whatsapp.allowFrom` to include your own number.
+- **Group policy**: `whatsapp.groupPolicy` controls group handling (`open|disabled|allowlist`).
+  - `allowlist` uses `whatsapp.groupAllowFrom` (fallback: explicit `whatsapp.allowFrom`).
+- **Self-chat mode**: avoids auto read receipts and ignores mention JIDs.
+- Read receipts sent for non-self-chat DMs.
+
+## WhatsApp FAQ: sending messages + pairing
+
+**Will Nexus message random contacts when I link WhatsApp?**  
+No. Default DM policy is **pairing**, so unknown senders only get a pairing code and their message is **not processed**. Nexus only replies to chats it receives, or to sends you explicitly trigger (agent/CLI).
+
+**How does pairing work on WhatsApp?**  
+Pairing is a DM gate for unknown senders:
+- First DM from a new sender returns a short code (message is not processed).
+- Approve with: `nexus pairing approve --channel whatsapp <code>` (list with `nexus pairing list --channel whatsapp`).
+- Codes expire after 1 hour; pending requests are capped at 3 per provider.
+
+**Why do you ask for my phone number in the wizard?**  
+The wizard uses it to set your **allowlist/owner** so your own DMs are permitted. It’s not used for auto-sending. If you run on your personal WhatsApp number, use that same number and enable `whatsapp.selfChatMode`.
+
+## Message normalization (what the model sees)
+- `Body` is the current message body with envelope.
+- Quoted reply context is **always appended**:
+  ```
+  [Replying to +1555 id:ABC123]
+  <quoted text or <media:...>>
+  [/Replying]
+  ```
+- Reply metadata also set:
+  - `ReplyToId` = stanzaId
+  - `ReplyToBody` = quoted body or media placeholder
+  - `ReplyToSender` = E.164 when known
+- Media-only inbound messages use placeholders:
+  - `<media:image|video|audio|document|sticker>`
+
+## Groups
+- Groups map to `agent:<agentId>:whatsapp:group:<jid>` sessions.
+- Group policy: `whatsapp.groupPolicy = open|disabled|allowlist` (default `open`).
+- Activation modes:
+  - `mention` (default): requires @mention or regex match.
+  - `always`: always triggers.
+- `/activation mention|always` is owner-only.
+- Owner = `whatsapp.allowFrom` (or self E.164 if unset).
+- **History injection**:
+  - Recent messages (default 50) inserted under:
+    `[Chat messages since your last reply - for context]`
+  - Current message under:
+    `[Current message - respond to this]`
+  - Sender suffix appended: `[from: Name (+E164)]`
+- Group metadata cached 5 min (subject + participants).
+
+## Reply delivery (threading)
+- WhatsApp Web sends standard messages (no quoted reply threading in the current gateway).
+- Reply tags are ignored on this provider.
+
+## Outbound send (text + media)
+- Uses active web listener; error if gateway not running.
+- Text chunking: 4k max per message.
+- Media:
+  - Image/video/audio/document supported.
+  - Audio sent as PTT; `audio/ogg` => `audio/ogg; codecs=opus`.
+  - Caption only on first media item.
+  - Media fetch supports HTTP(S) and local paths.
+  - Animated GIFs: WhatsApp expects MP4 with `gifPlayback: true` for inline looping.
+    - CLI: `nexus send --media <mp4> --gif-playback`
+    - Gateway: `send` params include `gifPlayback: true`
+
+## Media limits + optimization
+- Default cap: 5 MB (per media item).
+- Override: `agent.mediaMaxMb`.
+- Images are auto-optimized to JPEG under cap (resize + quality sweep).
+- Oversize media => error; media reply falls back to text warning.
+
+## Heartbeats
+- **Gateway heartbeat** logs connection health (`web.heartbeatSeconds`, default 60s).
+- **Agent heartbeat** is global (`agent.heartbeat.*`) and runs in the main session.
+  - Uses `HEARTBEAT` prompt + `HEARTBEAT_OK` skip behavior.
+  - Delivery defaults to the last used provider (or configured target).
+
+## Reconnect behavior
+- Backoff policy: `web.reconnect`:
+  - `initialMs`, `maxMs`, `factor`, `jitter`, `maxAttempts`.
+- If maxAttempts reached, web monitoring stops (degraded).
+- Logged-out => stop and require re-link.
+
+## Config quick map
+- `whatsapp.dmPolicy` (DM policy: pairing/allowlist/open/disabled).
+- `whatsapp.allowFrom` (DM allowlist).
+- `whatsapp.accounts.<accountId>.*` (per-account settings + optional `authDir`).
+- `whatsapp.groupAllowFrom` (group sender allowlist).
+- `whatsapp.groupPolicy` (group policy).
+- `whatsapp.groups` (group allowlist + mention gating defaults; use `"*"` to allow all)
+- `routing.groupChat.mentionPatterns`
+- `routing.groupChat.historyLimit`
+- `messages.messagePrefix` (inbound prefix)
+- `messages.responsePrefix` (outbound prefix)
+- `agent.mediaMaxMb`
+- `agent.heartbeat.every`
+- `agent.heartbeat.model` (optional override)
+- `agent.heartbeat.target`
+- `agent.heartbeat.to`
+- `session.*` (scope, idle, store, mainKey)
+- `web.enabled` (disable provider startup when false)
+- `web.heartbeatSeconds`
+- `web.reconnect.*`
+
+## Logs + troubleshooting
+- Subsystems: `whatsapp/inbound`, `whatsapp/outbound`, `web-heartbeat`, `web-reconnect`.
+- Log file: `~/nexus/state/logs/nexus-YYYY-MM-DD.log` (configurable).
+- Troubleshooting guide: [`docs/troubleshooting.md`](https://docs.nexus.bot/troubleshooting).
+
+## Tests
+- `src/web/auto-reply.test.ts` (mention gating, history injection, reply flow)
+- `src/web/monitor-inbox.test.ts` (inbound parsing + reply context)
+- `src/web/outbound.test.ts` (send mapping + media)

package/docs/windows.md

@@ -0,0 +1,11 @@
+---
+summary: "Windows app status + contribution call"
+read_when:
+  - Looking for Windows companion app status
+  - Planning platform coverage or contributions
+---
+# Windows App
+
+Nexus core is fully supported on Windows. The core is written in TypeScript, so it runs anywhere Node runs.
+
+We do not have a Windows companion app yet. It is planned, and we would love contributions to make it happen.