silentlake 2026.3.24

package/docs/gateway/tailscale.md

@@ -0,0 +1,132 @@
+---
+summary: "Integrated Tailscale Serve/Funnel for the Gateway dashboard"
+read_when:
+  - Exposing the Gateway Control UI outside localhost
+  - Automating tailnet or public dashboard access
+title: "Tailscale"
+---
+
+# Tailscale (Gateway dashboard)
+
+OpenClaw can auto-configure Tailscale **Serve** (tailnet) or **Funnel** (public) for the
+Gateway dashboard and WebSocket port. This keeps the Gateway bound to loopback while
+Tailscale provides HTTPS, routing, and (for Serve) identity headers.
+
+## Modes
+
+- `serve`: Tailnet-only Serve via `tailscale serve`. The gateway stays on `127.0.0.1`.
+- `funnel`: Public HTTPS via `tailscale funnel`. OpenClaw requires a shared password.
+- `off`: Default (no Tailscale automation).
+
+## Auth
+
+Set `gateway.auth.mode` to control the handshake:
+
+- `token` (default when `OPENCLAW_GATEWAY_TOKEN` is set)
+- `password` (shared secret via `OPENCLAW_GATEWAY_PASSWORD` or config)
+
+When `tailscale.mode = "serve"` and `gateway.auth.allowTailscale` is `true`,
+Control UI/WebSocket auth can use Tailscale identity headers
+(`tailscale-user-login`) without supplying a token/password. OpenClaw verifies
+the identity by resolving the `x-forwarded-for` address via the local Tailscale
+daemon (`tailscale whois`) and matching it to the header before accepting it.
+OpenClaw only treats a request as Serve when it arrives from loopback with
+Tailscale’s `x-forwarded-for`, `x-forwarded-proto`, and `x-forwarded-host`
+headers.
+HTTP API endpoints (for example `/v1/*`, `/tools/invoke`, and `/api/channels/*`)
+still require token/password auth.
+This tokenless flow assumes the gateway host is trusted. If untrusted local code
+may run on the same host, disable `gateway.auth.allowTailscale` and require
+token/password auth instead.
+To require explicit credentials, set `gateway.auth.allowTailscale: false` or
+force `gateway.auth.mode: "password"`.
+
+## Config examples
+
+### Tailnet-only (Serve)
+
+```json5
+{
+  gateway: {
+    bind: "loopback",
+    tailscale: { mode: "serve" },
+  },
+}
+```
+
+Open: `https://<magicdns>/` (or your configured `gateway.controlUi.basePath`)
+
+### Tailnet-only (bind to Tailnet IP)
+
+Use this when you want the Gateway to listen directly on the Tailnet IP (no Serve/Funnel).
+
+```json5
+{
+  gateway: {
+    bind: "tailnet",
+    auth: { mode: "token", token: "your-token" },
+  },
+}
+```
+
+Connect from another Tailnet device:
+
+- Control UI: `http://<tailscale-ip>:18789/`
+- WebSocket: `ws://<tailscale-ip>:18789`
+
+Note: loopback (`http://127.0.0.1:18789`) will **not** work in this mode.
+
+### Public internet (Funnel + shared password)
+
+```json5
+{
+  gateway: {
+    bind: "loopback",
+    tailscale: { mode: "funnel" },
+    auth: { mode: "password", password: "replace-me" },
+  },
+}
+```
+
+Prefer `OPENCLAW_GATEWAY_PASSWORD` over committing a password to disk.
+
+## CLI examples
+
+```bash
+openclaw gateway --tailscale serve
+openclaw gateway --tailscale funnel --auth password
+```
+
+## Notes
+
+- Tailscale Serve/Funnel requires the `tailscale` CLI to be installed and logged in.
+- `tailscale.mode: "funnel"` refuses to start unless auth mode is `password` to avoid public exposure.
+- Set `gateway.tailscale.resetOnExit` if you want OpenClaw to undo `tailscale serve`
+  or `tailscale funnel` configuration on shutdown.
+- `gateway.bind: "tailnet"` is a direct Tailnet bind (no HTTPS, no Serve/Funnel).
+- `gateway.bind: "auto"` prefers loopback; use `tailnet` if you want Tailnet-only.
+- Serve/Funnel only expose the **Gateway control UI + WS**. Nodes connect over
+  the same Gateway WS endpoint, so Serve can work for node access.
+
+## Browser control (remote Gateway + local browser)
+
+If you run the Gateway on one machine but want to drive a browser on another machine,
+run a **node host** on the browser machine and keep both on the same tailnet.
+The Gateway will proxy browser actions to the node; no separate control server or Serve URL needed.
+
+Avoid Funnel for browser control; treat node pairing like operator access.
+
+## Tailscale prerequisites + limits
+
+- Serve requires HTTPS enabled for your tailnet; the CLI prompts if it is missing.
+- Serve injects Tailscale identity headers; Funnel does not.
+- Funnel requires Tailscale v1.38.3+, MagicDNS, HTTPS enabled, and a funnel node attribute.
+- Funnel only supports ports `443`, `8443`, and `10000` over TLS.
+- Funnel on macOS requires the open-source Tailscale app variant.
+
+## Learn more
+
+- Tailscale Serve overview: [https://tailscale.com/kb/1312/serve](https://tailscale.com/kb/1312/serve)
+- `tailscale serve` command: [https://tailscale.com/kb/1242/tailscale-serve](https://tailscale.com/kb/1242/tailscale-serve)
+- Tailscale Funnel overview: [https://tailscale.com/kb/1223/tailscale-funnel](https://tailscale.com/kb/1223/tailscale-funnel)
+- `tailscale funnel` command: [https://tailscale.com/kb/1311/tailscale-funnel](https://tailscale.com/kb/1311/tailscale-funnel)

package/docs/gateway/tools-invoke-http-api.md

@@ -0,0 +1,110 @@
+---
+summary: "Invoke a single tool directly via the Gateway HTTP endpoint"
+read_when:
+  - Calling tools without running a full agent turn
+  - Building automations that need tool policy enforcement
+title: "Tools Invoke API"
+---
+
+# Tools Invoke (HTTP)
+
+OpenClaw’s Gateway exposes a simple HTTP endpoint for invoking a single tool directly. It is always enabled, but gated by Gateway auth and tool policy.
+
+- `POST /tools/invoke`
+- Same port as the Gateway (WS + HTTP multiplex): `http://<gateway-host>:<port>/tools/invoke`
+
+Default max payload size is 2 MB.
+
+## Authentication
+
+Uses the Gateway auth configuration. Send a bearer token:
+
+- `Authorization: Bearer <token>`
+
+Notes:
+
+- When `gateway.auth.mode="token"`, use `gateway.auth.token` (or `OPENCLAW_GATEWAY_TOKEN`).
+- When `gateway.auth.mode="password"`, use `gateway.auth.password` (or `OPENCLAW_GATEWAY_PASSWORD`).
+- If `gateway.auth.rateLimit` is configured and too many auth failures occur, the endpoint returns `429` with `Retry-After`.
+
+## Request body
+
+```json
+{
+  "tool": "sessions_list",
+  "action": "json",
+  "args": {},
+  "sessionKey": "main",
+  "dryRun": false
+}
+```
+
+Fields:
+
+- `tool` (string, required): tool name to invoke.
+- `action` (string, optional): mapped into args if the tool schema supports `action` and the args payload omitted it.
+- `args` (object, optional): tool-specific arguments.
+- `sessionKey` (string, optional): target session key. If omitted or `"main"`, the Gateway uses the configured main session key (honors `session.mainKey` and default agent, or `global` in global scope).
+- `dryRun` (boolean, optional): reserved for future use; currently ignored.
+
+## Policy + routing behavior
+
+Tool availability is filtered through the same policy chain used by Gateway agents:
+
+- `tools.profile` / `tools.byProvider.profile`
+- `tools.allow` / `tools.byProvider.allow`
+- `agents.<id>.tools.allow` / `agents.<id>.tools.byProvider.allow`
+- group policies (if the session key maps to a group or channel)
+- subagent policy (when invoking with a subagent session key)
+
+If a tool is not allowed by policy, the endpoint returns **404**.
+
+Gateway HTTP also applies a hard deny list by default (even if session policy allows the tool):
+
+- `sessions_spawn`
+- `sessions_send`
+- `gateway`
+- `whatsapp_login`
+
+You can customize this deny list via `gateway.tools`:
+
+```json5
+{
+  gateway: {
+    tools: {
+      // Additional tools to block over HTTP /tools/invoke
+      deny: ["browser"],
+      // Remove tools from the default deny list
+      allow: ["gateway"],
+    },
+  },
+}
+```
+
+To help group policies resolve context, you can optionally set:
+
+- `x-openclaw-message-channel: <channel>` (example: `slack`, `telegram`)
+- `x-openclaw-account-id: <accountId>` (when multiple accounts exist)
+
+## Responses
+
+- `200` → `{ ok: true, result }`
+- `400` → `{ ok: false, error: { type, message } }` (invalid request or tool input error)
+- `401` → unauthorized
+- `429` → auth rate-limited (`Retry-After` set)
+- `404` → tool not available (not found or not allowlisted)
+- `405` → method not allowed
+- `500` → `{ ok: false, error: { type, message } }` (unexpected tool execution error; sanitized message)
+
+## Example
+
+```bash
+curl -sS http://127.0.0.1:18789/tools/invoke \
+  -H 'Authorization: Bearer YOUR_TOKEN' \
+  -H 'Content-Type: application/json' \
+  -d '{
+    "tool": "sessions_list",
+    "action": "json",
+    "args": {}
+  }'
+```

package/docs/gateway/troubleshooting.md

@@ -0,0 +1,378 @@
+---
+summary: "Deep troubleshooting runbook for gateway, channels, automation, nodes, and browser"
+read_when:
+  - The troubleshooting hub pointed you here for deeper diagnosis
+  - You need stable symptom based runbook sections with exact commands
+title: "Troubleshooting"
+---
+
+# Gateway troubleshooting
+
+This page is the deep runbook.
+Start at [/help/troubleshooting](/help/troubleshooting) if you want the fast triage flow first.
+
+## Command ladder
+
+Run these first, in this order:
+
+```bash
+openclaw status
+openclaw gateway status
+openclaw logs --follow
+openclaw doctor
+openclaw channels status --probe
+```
+
+Expected healthy signals:
+
+- `openclaw gateway status` shows `Runtime: running` and `RPC probe: ok`.
+- `openclaw doctor` reports no blocking config/service issues.
+- `openclaw channels status --probe` shows connected/ready channels.
+
+## Anthropic 429 extra usage required for long context
+
+Use this when logs/errors include:
+`HTTP 429: rate_limit_error: Extra usage is required for long context requests`.
+
+```bash
+openclaw logs --follow
+openclaw models status
+openclaw config get agents.defaults.models
+```
+
+Look for:
+
+- Selected Anthropic Opus/Sonnet model has `params.context1m: true`.
+- Current Anthropic credential is not eligible for long-context usage.
+- Requests fail only on long sessions/model runs that need the 1M beta path.
+
+Fix options:
+
+1. Disable `context1m` for that model to fall back to the normal context window.
+2. Use an Anthropic API key with billing, or enable Anthropic Extra Usage on the subscription account.
+3. Configure fallback models so runs continue when Anthropic long-context requests are rejected.
+
+Related:
+
+- [/providers/anthropic](/providers/anthropic)
+- [/reference/token-use](/reference/token-use)
+- [/help/faq#why-am-i-seeing-http-429-ratelimiterror-from-anthropic](/help/faq#why-am-i-seeing-http-429-ratelimiterror-from-anthropic)
+
+## No replies
+
+If channels are up but nothing answers, check routing and policy before reconnecting anything.
+
+```bash
+openclaw status
+openclaw channels status --probe
+openclaw pairing list --channel <channel> [--account <id>]
+openclaw config get channels
+openclaw logs --follow
+```
+
+Look for:
+
+- Pairing pending for DM senders.
+- Group mention gating (`requireMention`, `mentionPatterns`).
+- Channel/group allowlist mismatches.
+
+Common signatures:
+
+- `drop guild message (mention required` → group message ignored until mention.
+- `pairing request` → sender needs approval.
+- `blocked` / `allowlist` → sender/channel was filtered by policy.
+
+Related:
+
+- [/channels/troubleshooting](/channels/troubleshooting)
+- [/channels/pairing](/channels/pairing)
+- [/channels/groups](/channels/groups)
+
+## Dashboard control ui connectivity
+
+When dashboard/control UI will not connect, validate URL, auth mode, and secure context assumptions.
+
+```bash
+openclaw gateway status
+openclaw status
+openclaw logs --follow
+openclaw doctor
+openclaw gateway status --json
+```
+
+Look for:
+
+- Correct probe URL and dashboard URL.
+- Auth mode/token mismatch between client and gateway.
+- HTTP usage where device identity is required.
+
+Common signatures:
+
+- `device identity required` → non-secure context or missing device auth.
+- `device nonce required` / `device nonce mismatch` → client is not completing the
+  challenge-based device auth flow (`connect.challenge` + `device.nonce`).
+- `device signature invalid` / `device signature expired` → client signed the wrong
+  payload (or stale timestamp) for the current handshake.
+- `AUTH_TOKEN_MISMATCH` with `canRetryWithDeviceToken=true` → client can do one trusted retry with cached device token.
+- repeated `unauthorized` after that retry → shared token/device token drift; refresh token config and re-approve/rotate device token if needed.
+- `gateway connect failed:` → wrong host/port/url target.
+
+### Auth detail codes quick map
+
+Use `error.details.code` from the failed `connect` response to pick the next action:
+
+| Detail code                  | Meaning                                                  | Recommended action                                                                                                                                                   |
+| ---------------------------- | -------------------------------------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
+| `AUTH_TOKEN_MISSING`         | Client did not send a required shared token.             | Paste/set token in the client and retry. For dashboard paths: `openclaw config get gateway.auth.token` then paste into Control UI settings.                          |
+| `AUTH_TOKEN_MISMATCH`        | Shared token did not match gateway auth token.           | If `canRetryWithDeviceToken=true`, allow one trusted retry. If still failing, run the [token drift recovery checklist](/cli/devices#token-drift-recovery-checklist). |
+| `AUTH_DEVICE_TOKEN_MISMATCH` | Cached per-device token is stale or revoked.             | Rotate/re-approve device token using [devices CLI](/cli/devices), then reconnect.                                                                                    |
+| `PAIRING_REQUIRED`           | Device identity is known but not approved for this role. | Approve pending request: `openclaw devices list` then `openclaw devices approve <requestId>`.                                                                        |
+
+Device auth v2 migration check:
+
+```bash
+openclaw --version
+openclaw doctor
+openclaw gateway status
+```
+
+If logs show nonce/signature errors, update the connecting client and verify it:
+
+1. waits for `connect.challenge`
+2. signs the challenge-bound payload
+3. sends `connect.params.device.nonce` with the same challenge nonce
+
+Related:
+
+- [/web/control-ui](/web/control-ui)
+- [/gateway/authentication](/gateway/authentication)
+- [/gateway/remote](/gateway/remote)
+- [/cli/devices](/cli/devices)
+
+## Gateway service not running
+
+Use this when service is installed but process does not stay up.
+
+```bash
+openclaw gateway status
+openclaw status
+openclaw logs --follow
+openclaw doctor
+openclaw gateway status --deep
+```
+
+Look for:
+
+- `Runtime: stopped` with exit hints.
+- Service config mismatch (`Config (cli)` vs `Config (service)`).
+- Port/listener conflicts.
+
+Common signatures:
+
+- `Gateway start blocked: set gateway.mode=local` → local gateway mode is not enabled. Fix: set `gateway.mode="local"` in your config (or run `openclaw configure`). If you are running OpenClaw via Podman using the dedicated `openclaw` user, the config lives at `~openclaw/.openclaw/openclaw.json`.
+- `refusing to bind gateway ... without auth` → non-loopback bind without token/password.
+- `another gateway instance is already listening` / `EADDRINUSE` → port conflict.
+
+Related:
+
+- [/gateway/background-process](/gateway/background-process)
+- [/gateway/configuration](/gateway/configuration)
+- [/gateway/doctor](/gateway/doctor)
+
+## Channel connected messages not flowing
+
+If channel state is connected but message flow is dead, focus on policy, permissions, and channel specific delivery rules.
+
+```bash
+openclaw channels status --probe
+openclaw pairing list --channel <channel> [--account <id>]
+openclaw status --deep
+openclaw logs --follow
+openclaw config get channels
+```
+
+Look for:
+
+- DM policy (`pairing`, `allowlist`, `open`, `disabled`).
+- Group allowlist and mention requirements.
+- Missing channel API permissions/scopes.
+
+Common signatures:
+
+- `mention required` → message ignored by group mention policy.
+- `pairing` / pending approval traces → sender is not approved.
+- `missing_scope`, `not_in_channel`, `Forbidden`, `401/403` → channel auth/permissions issue.
+
+Related:
+
+- [/channels/troubleshooting](/channels/troubleshooting)
+- [/channels/whatsapp](/channels/whatsapp)
+- [/channels/telegram](/channels/telegram)
+- [/channels/discord](/channels/discord)
+
+## Cron and heartbeat delivery
+
+If cron or heartbeat did not run or did not deliver, verify scheduler state first, then delivery target.
+
+```bash
+openclaw cron status
+openclaw cron list
+openclaw cron runs --id <jobId> --limit 20
+openclaw system heartbeat last
+openclaw logs --follow
+```
+
+Look for:
+
+- Cron enabled and next wake present.
+- Job run history status (`ok`, `skipped`, `error`).
+- Heartbeat skip reasons (`quiet-hours`, `requests-in-flight`, `alerts-disabled`).
+
+Common signatures:
+
+- `cron: scheduler disabled; jobs will not run automatically` → cron disabled.
+- `cron: timer tick failed` → scheduler tick failed; check file/log/runtime errors.
+- `heartbeat skipped` with `reason=quiet-hours` → outside active hours window.
+- `heartbeat: unknown accountId` → invalid account id for heartbeat delivery target.
+- `heartbeat skipped` with `reason=dm-blocked` → heartbeat target resolved to a DM-style destination while `agents.defaults.heartbeat.directPolicy` (or per-agent override) is set to `block`.
+
+Related:
+
+- [/automation/troubleshooting](/automation/troubleshooting)
+- [/automation/cron-jobs](/automation/cron-jobs)
+- [/gateway/heartbeat](/gateway/heartbeat)
+
+## Node paired tool fails
+
+If a node is paired but tools fail, isolate foreground, permission, and approval state.
+
+```bash
+openclaw nodes status
+openclaw nodes describe --node <idOrNameOrIp>
+openclaw approvals get --node <idOrNameOrIp>
+openclaw logs --follow
+openclaw status
+```
+
+Look for:
+
+- Node online with expected capabilities.
+- OS permission grants for camera/mic/location/screen.
+- Exec approvals and allowlist state.
+
+Common signatures:
+
+- `NODE_BACKGROUND_UNAVAILABLE` → node app must be in foreground.
+- `*_PERMISSION_REQUIRED` / `LOCATION_PERMISSION_REQUIRED` → missing OS permission.
+- `SYSTEM_RUN_DENIED: approval required` → exec approval pending.
+- `SYSTEM_RUN_DENIED: allowlist miss` → command blocked by allowlist.
+
+Related:
+
+- [/nodes/troubleshooting](/nodes/troubleshooting)
+- [/nodes/index](/nodes/index)
+- [/tools/exec-approvals](/tools/exec-approvals)
+
+## Browser tool fails
+
+Use this when browser tool actions fail even though the gateway itself is healthy.
+
+```bash
+openclaw browser status
+openclaw browser start --browser-profile openclaw
+openclaw browser profiles
+openclaw logs --follow
+openclaw doctor
+```
+
+Look for:
+
+- Valid browser executable path.
+- CDP profile reachability.
+- Local Chrome availability for `existing-session` / `user` profiles.
+
+Common signatures:
+
+- `Failed to start Chrome CDP on port` → browser process failed to launch.
+- `browser.executablePath not found` → configured path is invalid.
+- `No Chrome tabs found for profile="user"` → the Chrome MCP attach profile has no open local Chrome tabs.
+- `Browser attachOnly is enabled ... not reachable` → attach-only profile has no reachable target.
+
+Related:
+
+- [/tools/browser-linux-troubleshooting](/tools/browser-linux-troubleshooting)
+- [/tools/browser](/tools/browser)
+
+## If you upgraded and something suddenly broke
+
+Most post-upgrade breakage is config drift or stricter defaults now being enforced.
+
+### 1) Auth and URL override behavior changed
+
+```bash
+openclaw gateway status
+openclaw config get gateway.mode
+openclaw config get gateway.remote.url
+openclaw config get gateway.auth.mode
+```
+
+What to check:
+
+- If `gateway.mode=remote`, CLI calls may be targeting remote while your local service is fine.
+- Explicit `--url` calls do not fall back to stored credentials.
+
+Common signatures:
+
+- `gateway connect failed:` → wrong URL target.
+- `unauthorized` → endpoint reachable but wrong auth.
+
+### 2) Bind and auth guardrails are stricter
+
+```bash
+openclaw config get gateway.bind
+openclaw config get gateway.auth.token
+openclaw gateway status
+openclaw logs --follow
+```
+
+What to check:
+
+- Non-loopback binds (`lan`, `tailnet`, `custom`) need auth configured.
+- Old keys like `gateway.token` do not replace `gateway.auth.token`.
+
+Common signatures:
+
+- `refusing to bind gateway ... without auth` → bind+auth mismatch.
+- `RPC probe: failed` while runtime is running → gateway alive but inaccessible with current auth/url.
+
+### 3) Pairing and device identity state changed
+
+```bash
+openclaw devices list
+openclaw pairing list --channel <channel> [--account <id>]
+openclaw logs --follow
+openclaw doctor
+```
+
+What to check:
+
+- Pending device approvals for dashboard/nodes.
+- Pending DM pairing approvals after policy or identity changes.
+
+Common signatures:
+
+- `device identity required` → device auth not satisfied.
+- `pairing required` → sender/device must be approved.
+
+If the service config and runtime still disagree after checks, reinstall service metadata from the same profile/state directory:
+
+```bash
+openclaw gateway install --force
+openclaw gateway restart
+```
+
+Related:
+
+- [/gateway/pairing](/gateway/pairing)
+- [/gateway/authentication](/gateway/authentication)
+- [/gateway/background-process](/gateway/background-process)