synurex 2026.2.19-1

package/docs/nodes/images.md

@@ -0,0 +1,72 @@
+---
+summary: "Image and media handling rules for send, gateway, and agent replies"
+read_when:
+  - Modifying media pipeline or attachments
+title: "Image and Media Support"
+---
+
+# Image & Media Support — 2025-12-05
+
+The WhatsApp channel runs via **Baileys Web**. This document captures the current media handling rules for send, gateway, and agent replies.
+
+## Goals
+
+- Send media with optional captions via `Synurex message send --media`.
+- Allow auto-replies from the web inbox to include media alongside text.
+- Keep per-type limits sane and predictable.
+
+## CLI Surface
+
+- `Synurex message send --media <path-or-url> [--message <caption>]`
+  - `--media` optional; caption can be empty for media-only sends.
+  - `--dry-run` prints the resolved payload; `--json` emits `{ channel, to, messageId, mediaUrl, caption }`.
+
+## WhatsApp Web channel behavior
+
+- Input: local file path **or** HTTP(S) URL.
+- Flow: load into a Buffer, detect media kind, and build the correct payload:
+  - **Images:** resize & recompress to JPEG (max side 2048px) targeting `agents.defaults.mediaMaxMb` (default 5 MB), capped at 6 MB.
+  - **Audio/Voice/Video:** pass-through up to 16 MB; audio is sent as a voice note (`ptt: true`).
+  - **Documents:** anything else, up to 100 MB, with filename preserved when available.
+- WhatsApp GIF-style playback: send an MP4 with `gifPlayback: true` (CLI: `--gif-playback`) so mobile clients loop inline.
+- MIME detection prefers magic bytes, then headers, then file extension.
+- Caption comes from `--message` or `reply.text`; empty caption is allowed.
+- Logging: non-verbose shows `↩️`/`✅`; verbose includes size and source path/URL.
+
+## Auto-Reply Pipeline
+
+- `getReplyFromConfig` returns `{ text?, mediaUrl?, mediaUrls? }`.
+- When media is present, the web sender resolves local paths or URLs using the same pipeline as `Synurex message send`.
+- Multiple media entries are sent sequentially if provided.
+
+## Inbound Media to Commands (Pi)
+
+- When inbound web messages include media, Synurex downloads to a temp file and exposes templating variables:
+  - `{{MediaUrl}}` pseudo-URL for the inbound media.
+  - `{{MediaPath}}` local temp path written before running the command.
+- When a per-session Docker sandbox is enabled, inbound media is copied into the sandbox workspace and `MediaPath`/`MediaUrl` are rewritten to a relative path like `media/inbound/<filename>`.
+- Media understanding (if configured via `tools.media.*` or shared `tools.media.models`) runs before templating and can insert `[Image]`, `[Audio]`, and `[Video]` blocks into `Body`.
+  - Audio sets `{{Transcript}}` and uses the transcript for command parsing so slash commands still work.
+  - Video and image descriptions preserve any caption text for command parsing.
+- By default only the first matching image/audio/video attachment is processed; set `tools.media.<cap>.attachments` to process multiple attachments.
+
+## Limits & Errors
+
+**Outbound send caps (WhatsApp web send)**
+
+- Images: ~6 MB cap after recompression.
+- Audio/voice/video: 16 MB cap; documents: 100 MB cap.
+- Oversize or unreadable media → clear error in logs and the reply is skipped.
+
+**Media understanding caps (transcription/description)**
+
+- Image default: 10 MB (`tools.media.image.maxBytes`).
+- Audio default: 20 MB (`tools.media.audio.maxBytes`).
+- Video default: 50 MB (`tools.media.video.maxBytes`).
+- Oversize media skips understanding, but replies still go through with the original body.
+
+## Notes for Tests
+
+- Cover send + reply flows for image/audio/document cases.
+- Validate recompression for images (size bound) and voice-note flag for audio.
+- Ensure multi-media replies fan out as sequential sends.

package/docs/nodes/index.md

@@ -0,0 +1,342 @@
+---
+summary: "Nodes: pairing, capabilities, permissions, and CLI helpers for canvas/camera/screen/system"
+read_when:
+  - Pairing iOS/Android nodes to a gateway
+  - Using node canvas/camera for agent context
+  - Adding new node commands or CLI helpers
+title: "Nodes"
+---
+
+# Nodes
+
+A **node** is a companion device (macOS/iOS/Android/headless) that connects to the Gateway **WebSocket** (same port as operators) with `role: "node"` and exposes a command surface (e.g. `canvas.*`, `camera.*`, `system.*`) via `node.invoke`. Protocol details: [Gateway protocol](/gateway/protocol).
+
+Legacy transport: [Bridge protocol](/gateway/bridge-protocol) (TCP JSONL; deprecated/removed for current nodes).
+
+macOS can also run in **node mode**: the menubar app connects to the Gateway’s WS server and exposes its local canvas/camera commands as a node (so `Synurex nodes …` works against this Mac).
+
+Notes:
+
+- Nodes are **peripherals**, not gateways. They don’t run the gateway service.
+- Telegram/WhatsApp/etc. messages land on the **gateway**, not on nodes.
+- Troubleshooting runbook: [/nodes/troubleshooting](/nodes/troubleshooting)
+
+## Pairing + status
+
+**WS nodes use device pairing.** Nodes present a device identity during `connect`; the Gateway
+creates a device pairing request for `role: node`. Approve via the devices CLI (or UI).
+
+Quick CLI:
+
+```bash
+Synurex devices list
+Synurex devices approve <requestId>
+Synurex devices reject <requestId>
+Synurex nodes status
+Synurex nodes describe --node <idOrNameOrIp>
+```
+
+Notes:
+
+- `nodes status` marks a node as **paired** when its device pairing role includes `node`.
+- `node.pair.*` (CLI: `Synurex nodes pending/approve/reject`) is a separate gateway-owned
+  node pairing store; it does **not** gate the WS `connect` handshake.
+
+## Remote node host (system.run)
+
+Use a **node host** when your Gateway runs on one machine and you want commands
+to execute on another. The model still talks to the **gateway**; the gateway
+forwards `exec` calls to the **node host** when `host=node` is selected.
+
+### What runs where
+
+- **Gateway host**: receives messages, runs the model, routes tool calls.
+- **Node host**: executes `system.run`/`system.which` on the node machine.
+- **Approvals**: enforced on the node host via `~/.synurex/exec-approvals.json`.
+
+### Start a node host (foreground)
+
+On the node machine:
+
+```bash
+Synurex node run --host <gateway-host> --port 18789 --display-name "Build Node"
+```
+
+### Remote gateway via SSH tunnel (loopback bind)
+
+If the Gateway binds to loopback (`gateway.bind=loopback`, default in local mode),
+remote node hosts cannot connect directly. Create an SSH tunnel and point the
+node host at the local end of the tunnel.
+
+Example (node host -> gateway host):
+
+```bash
+# Terminal A (keep running): forward local 18790 -> gateway 127.0.0.1:18789
+ssh -N -L 18790:127.0.0.1:18789 user@gateway-host
+
+# Terminal B: export the gateway token and connect through the tunnel
+export SYNUREX_GATEWAY_TOKEN="<gateway-token>"
+Synurex node run --host 127.0.0.1 --port 18790 --display-name "Build Node"
+```
+
+Notes:
+
+- The token is `gateway.auth.token` from the gateway config (`~/.synurex/synurex.json` on the gateway host).
+- `Synurex node run` reads `SYNUREX_GATEWAY_TOKEN` for auth.
+
+### Start a node host (service)
+
+```bash
+Synurex node install --host <gateway-host> --port 18789 --display-name "Build Node"
+Synurex node restart
+```
+
+### Pair + name
+
+On the gateway host:
+
+```bash
+Synurex nodes pending
+Synurex nodes approve <requestId>
+Synurex nodes list
+```
+
+Naming options:
+
+- `--display-name` on `Synurex node run` / `Synurex node install` (persists in `~/.synurex/node.json` on the node).
+- `Synurex nodes rename --node <id|name|ip> --name "Build Node"` (gateway override).
+
+### Allowlist the commands
+
+Exec approvals are **per node host**. Add allowlist entries from the gateway:
+
+```bash
+Synurex approvals allowlist add --node <id|name|ip> "/usr/bin/uname"
+Synurex approvals allowlist add --node <id|name|ip> "/usr/bin/sw_vers"
+```
+
+Approvals live on the node host at `~/.synurex/exec-approvals.json`.
+
+### Point exec at the node
+
+Configure defaults (gateway config):
+
+```bash
+Synurex config set tools.exec.host node
+Synurex config set tools.exec.security allowlist
+Synurex config set tools.exec.node "<id-or-name>"
+```
+
+Or per session:
+
+```
+/exec host=node security=allowlist node=<id-or-name>
+```
+
+Once set, any `exec` call with `host=node` runs on the node host (subject to the
+node allowlist/approvals).
+
+Related:
+
+- [Node host CLI](/cli/node)
+- [Exec tool](/tools/exec)
+- [Exec approvals](/tools/exec-approvals)
+
+## Invoking commands
+
+Low-level (raw RPC):
+
+```bash
+Synurex nodes invoke --node <idOrNameOrIp> --command canvas.eval --params '{"javaScript":"location.href"}'
+```
+
+Higher-level helpers exist for the common “give the agent a MEDIA attachment” workflows.
+
+## Screenshots (canvas snapshots)
+
+If the node is showing the Canvas (WebView), `canvas.snapshot` returns `{ format, base64 }`.
+
+CLI helper (writes to a temp file and prints `MEDIA:<path>`):
+
+```bash
+Synurex nodes canvas snapshot --node <idOrNameOrIp> --format png
+Synurex nodes canvas snapshot --node <idOrNameOrIp> --format jpg --max-width 1200 --quality 0.9
+```
+
+### Canvas controls
+
+```bash
+Synurex nodes canvas present --node <idOrNameOrIp> --target https://example.com
+Synurex nodes canvas hide --node <idOrNameOrIp>
+Synurex nodes canvas navigate https://example.com --node <idOrNameOrIp>
+Synurex nodes canvas eval --node <idOrNameOrIp> --js "document.title"
+```
+
+Notes:
+
+- `canvas present` accepts URLs or local file paths (`--target`), plus optional `--x/--y/--width/--height` for positioning.
+- `canvas eval` accepts inline JS (`--js`) or a positional arg.
+
+### A2UI (Canvas)
+
+```bash
+Synurex nodes canvas a2ui push --node <idOrNameOrIp> --text "Hello"
+Synurex nodes canvas a2ui push --node <idOrNameOrIp> --jsonl ./payload.jsonl
+Synurex nodes canvas a2ui reset --node <idOrNameOrIp>
+```
+
+Notes:
+
+- Only A2UI v0.8 JSONL is supported (v0.9/createSurface is rejected).
+
+## Photos + videos (node camera)
+
+Photos (`jpg`):
+
+```bash
+Synurex nodes camera list --node <idOrNameOrIp>
+Synurex nodes camera snap --node <idOrNameOrIp>            # default: both facings (2 MEDIA lines)
+Synurex nodes camera snap --node <idOrNameOrIp> --facing front
+```
+
+Video clips (`mp4`):
+
+```bash
+Synurex nodes camera clip --node <idOrNameOrIp> --duration 10s
+Synurex nodes camera clip --node <idOrNameOrIp> --duration 3000 --no-audio
+```
+
+Notes:
+
+- The node must be **foregrounded** for `canvas.*` and `camera.*` (background calls return `NODE_BACKGROUND_UNAVAILABLE`).
+- Clip duration is clamped (currently `<= 60s`) to avoid oversized base64 payloads.
+- Android will prompt for `CAMERA`/`RECORD_AUDIO` permissions when possible; denied permissions fail with `*_PERMISSION_REQUIRED`.
+
+## Screen recordings (nodes)
+
+Nodes expose `screen.record` (mp4). Example:
+
+```bash
+Synurex nodes screen record --node <idOrNameOrIp> --duration 10s --fps 10
+Synurex nodes screen record --node <idOrNameOrIp> --duration 10s --fps 10 --no-audio
+```
+
+Notes:
+
+- `screen.record` requires the node app to be foregrounded.
+- Android will show the system screen-capture prompt before recording.
+- Screen recordings are clamped to `<= 60s`.
+- `--no-audio` disables microphone capture (supported on iOS/Android; macOS uses system capture audio).
+- Use `--screen <index>` to select a display when multiple screens are available.
+
+## Location (nodes)
+
+Nodes expose `location.get` when Location is enabled in settings.
+
+CLI helper:
+
+```bash
+Synurex nodes location get --node <idOrNameOrIp>
+Synurex nodes location get --node <idOrNameOrIp> --accuracy precise --max-age 15000 --location-timeout 10000
+```
+
+Notes:
+
+- Location is **off by default**.
+- “Always” requires system permission; background fetch is best-effort.
+- The response includes lat/lon, accuracy (meters), and timestamp.
+
+## SMS (Android nodes)
+
+Android nodes can expose `sms.send` when the user grants **SMS** permission and the device supports telephony.
+
+Low-level invoke:
+
+```bash
+Synurex nodes invoke --node <idOrNameOrIp> --command sms.send --params '{"to":"+15555550123","message":"Hello from Synurex"}'
+```
+
+Notes:
+
+- The permission prompt must be accepted on the Android device before the capability is advertised.
+- Wi-Fi-only devices without telephony will not advertise `sms.send`.
+
+## System commands (node host / mac node)
+
+The macOS node exposes `system.run`, `system.notify`, and `system.execApprovals.get/set`.
+The headless node host exposes `system.run`, `system.which`, and `system.execApprovals.get/set`.
+
+Examples:
+
+```bash
+Synurex nodes run --node <idOrNameOrIp> -- echo "Hello from mac node"
+Synurex nodes notify --node <idOrNameOrIp> --title "Ping" --body "Gateway ready"
+```
+
+Notes:
+
+- `system.run` returns stdout/stderr/exit code in the payload.
+- `system.notify` respects notification permission state on the macOS app.
+- `system.run` supports `--cwd`, `--env KEY=VAL`, `--command-timeout`, and `--needs-screen-recording`.
+- `system.notify` supports `--priority <passive|active|timeSensitive>` and `--delivery <system|overlay|auto>`.
+- macOS nodes drop `PATH` overrides; headless node hosts only accept `PATH` when it prepends the node host PATH.
+- On macOS node mode, `system.run` is gated by exec approvals in the macOS app (Settings → Exec approvals).
+  Ask/allowlist/full behave the same as the headless node host; denied prompts return `SYSTEM_RUN_DENIED`.
+- On headless node host, `system.run` is gated by exec approvals (`~/.synurex/exec-approvals.json`).
+
+## Exec node binding
+
+When multiple nodes are available, you can bind exec to a specific node.
+This sets the default node for `exec host=node` (and can be overridden per agent).
+
+Global default:
+
+```bash
+Synurex config set tools.exec.node "node-id-or-name"
+```
+
+Per-agent override:
+
+```bash
+Synurex config get agents.list
+Synurex config set agents.list[0].tools.exec.node "node-id-or-name"
+```
+
+Unset to allow any node:
+
+```bash
+Synurex config unset tools.exec.node
+Synurex config unset agents.list[0].tools.exec.node
+```
+
+## Permissions map
+
+Nodes may include a `permissions` map in `node.list` / `node.describe`, keyed by permission name (e.g. `screenRecording`, `accessibility`) with boolean values (`true` = granted).
+
+## Headless node host (cross-platform)
+
+Synurex can run a **headless node host** (no UI) that connects to the Gateway
+WebSocket and exposes `system.run` / `system.which`. This is useful on Linux/Windows
+or for running a minimal node alongside a server.
+
+Start it:
+
+```bash
+Synurex node run --host <gateway-host> --port 18789
+```
+
+Notes:
+
+- Pairing is still required (the Gateway will show a node approval prompt).
+- The node host stores its node id, token, display name, and gateway connection info in `~/.synurex/node.json`.
+- Exec approvals are enforced locally via `~/.synurex/exec-approvals.json`
+  (see [Exec approvals](/tools/exec-approvals)).
+- On macOS, the headless node host prefers the companion app exec host when reachable and falls
+  back to local execution if the app is unavailable. Set `SYNUREX_NODE_EXEC_HOST=app` to require
+  the app, or `SYNUREX_NODE_EXEC_FALLBACK=0` to disable fallback.
+- Add `--tls` / `--tls-fingerprint` when the Gateway WS uses TLS.
+
+## Mac node mode
+
+- The macOS menubar app connects to the Gateway WS server as a node (so `Synurex nodes …` works against this Mac).
+- In remote mode, the app opens an SSH tunnel for the Gateway port and connects to `localhost`.

package/docs/nodes/location-command.md

@@ -0,0 +1,113 @@
+---
+summary: "Location command for nodes (location.get), permission modes, and background behavior"
+read_when:
+  - Adding location node support or permissions UI
+  - Designing background location + push flows
+title: "Location Command"
+---
+
+# Location command (nodes)
+
+## TL;DR
+
+- `location.get` is a node command (via `node.invoke`).
+- Off by default.
+- Settings use a selector: Off / While Using / Always.
+- Separate toggle: Precise Location.
+
+## Why a selector (not just a switch)
+
+OS permissions are multi-level. We can expose a selector in-app, but the OS still decides the actual grant.
+
+- iOS/macOS: user can choose **While Using** or **Always** in system prompts/Settings. App can request upgrade, but OS may require Settings.
+- Android: background location is a separate permission; on Android 10+ it often requires a Settings flow.
+- Precise location is a separate grant (iOS 14+ “Precise”, Android “fine” vs “coarse”).
+
+Selector in UI drives our requested mode; actual grant lives in OS settings.
+
+## Settings model
+
+Per node device:
+
+- `location.enabledMode`: `off | whileUsing | always`
+- `location.preciseEnabled`: bool
+
+UI behavior:
+
+- Selecting `whileUsing` requests foreground permission.
+- Selecting `always` first ensures `whileUsing`, then requests background (or sends user to Settings if required).
+- If OS denies requested level, revert to the highest granted level and show status.
+
+## Permissions mapping (node.permissions)
+
+Optional. macOS node reports `location` via the permissions map; iOS/Android may omit it.
+
+## Command: `location.get`
+
+Called via `node.invoke`.
+
+Params (suggested):
+
+```json
+{
+  "timeoutMs": 10000,
+  "maxAgeMs": 15000,
+  "desiredAccuracy": "coarse|balanced|precise"
+}
+```
+
+Response payload:
+
+```json
+{
+  "lat": 48.20849,
+  "lon": 16.37208,
+  "accuracyMeters": 12.5,
+  "altitudeMeters": 182.0,
+  "speedMps": 0.0,
+  "headingDeg": 270.0,
+  "timestamp": "2026-01-03T12:34:56.000Z",
+  "isPrecise": true,
+  "source": "gps|wifi|cell|unknown"
+}
+```
+
+Errors (stable codes):
+
+- `LOCATION_DISABLED`: selector is off.
+- `LOCATION_PERMISSION_REQUIRED`: permission missing for requested mode.
+- `LOCATION_BACKGROUND_UNAVAILABLE`: app is backgrounded but only While Using allowed.
+- `LOCATION_TIMEOUT`: no fix in time.
+- `LOCATION_UNAVAILABLE`: system failure / no providers.
+
+## Background behavior (future)
+
+Goal: model can request location even when node is backgrounded, but only when:
+
+- User selected **Always**.
+- OS grants background location.
+- App is allowed to run in background for location (iOS background mode / Android foreground service or special allowance).
+
+Push-triggered flow (future):
+
+1. Gateway sends a push to the node (silent push or FCM data).
+2. Node wakes briefly and requests location from the device.
+3. Node forwards payload to Gateway.
+
+Notes:
+
+- iOS: Always permission + background location mode required. Silent push may be throttled; expect intermittent failures.
+- Android: background location may require a foreground service; otherwise, expect denial.
+
+## Model/tooling integration
+
+- Tool surface: `nodes` tool adds `location_get` action (node required).
+- CLI: `Synurex nodes location get --node <id>`.
+- Agent guidelines: only call when user enabled location and understands the scope.
+
+## UX copy (suggested)
+
+- Off: “Location sharing is disabled.”
+- While Using: “Only when Synurex is open.”
+- Always: “Allow background location. Requires system permission.”
+- Precise: “Use precise GPS location. Toggle off to share approximate location.”