oomi-ai 0.2.13 → 0.2.15

package/README.md

@@ -1,148 +1,224 @@
 # oomi-ai
 
-CLI to install Oomi agent instructions and skills into OpenClaw, plus sync personas to the Oomi backend registry.
+OpenClaw channel plugin and bridge tooling for Oomi managed chat and voice.
 
-## Install
-```
-pnpm add -g oomi-ai
+This package is for two audiences:
+- OpenClaw operators who need to connect a machine to Oomi and keep chat or voice healthy
+- Developers evaluating the plugin on npm and deciding whether it matches their OpenClaw + Oomi setup
+
+## What This Package Ships
+
+The npm package contains two Oomi integration surfaces:
+
+1. OpenClaw channel extension
+- File: `openclaw.extension.js`
+- Purpose: stable managed text transport through the Oomi backend channel API
+- This is the preferred integration surface for normal chat
+
+2. Local bridge + CLI
+- Files: `bin/oomi-ai.js`, `bin/sessionBridgeState.js`
+- Purpose: pair a device, manage the OpenClaw bridge worker, and support managed gateway traffic needed by device-backed chat and voice
+- This is the part that deals with broker sockets, local gateway sessions, challenge auth, and bridge health
+
+In practical terms:
+- If you only need a clean managed chat channel, the extension is the main reason to install this package
+- If you need Oomi device-backed chat or voice on an OpenClaw machine, you also need the bridge tooling in this package
+
+## When To Install It
+
+Install `oomi-ai` if all of the following are true:
+- you use OpenClaw
+- you want Oomi as a managed channel inside OpenClaw
+- you want device-backed Oomi chat, Oomi voice, or both
+
+Do not install it just to use the Oomi web app by itself.
+
+## Install And Upgrade
+
+Global install:
+
+```bash
+pnpm add -g oomi-ai@latest
 ```
 
 Fallback:
+
+```bash
+npm install -g oomi-ai@latest
 ```
-npm install -g oomi-ai
+
+Install the OpenClaw plugin:
+
+```bash
+openclaw plugins install oomi-ai@latest
 ```
 
-## Usage
+Upgrade an existing machine:
 
-Install as an OpenClaw channel extension (preferred architecture):
+```bash
+pnpm add -g oomi-ai@latest
+openclaw plugins install oomi-ai@latest
 ```
+
+## Operator Quick Start
+
+The packaged operator instructions live in [agent_instructions.md](./agent_instructions.md).
+That is the primary reference for:
+- pairing a device
+- installing the plugin
+- configuring `channels.oomi.accounts.default`
+- running or supervising the bridge
+- checking whether the system is healthy
+- troubleshooting chat and voice failures
+
+Fast-path install flow:
+
+```bash
+oomi openclaw pair --app-url https://www.oomi.ai --no-start
 openclaw plugins install oomi-ai@latest
+oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai
+```
+
+Then apply the printed `channels.oomi.accounts.default` config and restart OpenClaw.
+
+## Configuration
+
+OpenClaw channel config lives under:
+
+```json
+{
+  "channels": {
+    "oomi": {
+      "defaultAccountId": "default",
+      "accounts": {
+        "default": {
+          "backendUrl": "https://api.oomi.ai",
+          "deviceToken": "...",
+          "defaultSessionKey": "agent:main:webchat:channel:oomi",
+          "requestTimeoutMs": 15000
+        }
+      }
+    }
+  }
+}
 ```
 
-This package now ships an OpenClaw channel plugin (`openclaw.plugin.json`) with channel id `oomi`.
-Channel account config fields (`channels.oomi.accounts.<accountId>`):
+Required fields:
 - `backendUrl`
 - `deviceToken`
-- `defaultSessionKey` (optional, default `agent:main:webchat:channel:oomi`)
-- `requestTimeoutMs` (optional)
 
-Print plugin install/config guidance from local pair state:
-```
-oomi openclaw plugin
-```
+Optional fields:
+- `defaultSessionKey`
+- `requestTimeoutMs`
 
-Install agent instructions only:
-```
-oomi init
-```
+## Runtime Model
 
-Install agent instructions + Oomi skill:
-```
-oomi openclaw install
-```
+There are two runtime contracts worth understanding.
 
-Pair and provision device token from Oomi web backend:
-```
-oomi openclaw pair --app-url https://your-oomi-app.vercel.app --device-id my-openclaw-mac --no-start
-```
+### Managed Text Chat
 
-`--app-url` must be reachable from the OpenClaw host. If OpenClaw runs on a different machine/network, do not use `localhost` unless tunneled.
+Managed text chat uses the OpenClaw channel extension and the Oomi backend channel API.
+This path is the more stable contract and should be preferred when evaluating the plugin for normal chat.
 
-This prints:
-- `Auth invite URL: https://.../connect/<single-use-token>`
-- A copy-ready block for the user:
-  - `Oomi Connect Ready`
-  - `Auth Link: ...`
+### Device-Backed Chat And Voice
 
-If you need a fresh auth link later (without re-pairing), run:
-```
-oomi openclaw invite --app-url https://your-oomi-app.vercel.app
-```
+Device-backed chat and voice use the local bridge.
+That bridge:
+- keeps a broker socket open to Oomi
+- opens local gateway sessions on demand
+- enforces `connect`-first request ordering
+- preserves or synthesizes `idempotencyKey` for `chat.send`
+- keeps voice-session faults from poisoning normal provider health where possible
 
-Bridge lifecycle handler (singleton, one bridge per host/device):
-```
-oomi openclaw bridge ensure --detach   # start if needed; no-op if already running
-oomi openclaw bridge ps                # list bridge pids
-oomi openclaw bridge stop              # stop all bridge workers
-oomi openclaw bridge restart --detach  # clean restart as background worker
-tail -f ~/.openclaw/logs/oomi-bridge-live.log  # detached bridge logs
-```
-`oomi openclaw bridge --detach` is equivalent to `oomi openclaw bridge start --detach`.
+This is the part of the package most likely to matter when debugging voice turn failures.
 
-macOS launchd supervision (recommended for durability):
-```
-oomi openclaw bridge service install      # install + start service
-oomi openclaw bridge service status
-oomi openclaw bridge service restart
-oomi openclaw bridge service stop
-oomi openclaw bridge service uninstall
-```
-Optional: `oomi openclaw bridge service install --no-start` to install without starting.
-
-Agent-intent mapping (recommended):
-- If user says `Connect yourself to Oomi. Use app URL https://www.oomi.ai.`
-- Run:
-  - `curl -fsSL https://www.oomi.ai/install.sh | bash`
-  - or `pnpm add -g oomi-ai@latest` (`npm install -g oomi-ai@latest` fallback)
-  - `oomi openclaw pair --app-url https://www.oomi.ai --no-start`
-  - `openclaw plugins install oomi-ai@latest`
-  - `oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai`
-  - Apply `channels.oomi.accounts.default` config and restart OpenClaw.
-
-Important distinction:
-- `pairCode` is one-time and used internally by the pair/bootstrap flow.
-- Invite auth links are the required user flow.
-- Managed chat connect now uses OpenClaw challenge auth (`connect.challenge` nonce + signed device payload) in the local bridge path.
-
-Sync personas from the repo into the backend registry:
-```
-oomi personas sync --backend-url http://localhost:3001
-```
+## Bridge Health States
 
-Create a new persona (and sync it):
-```
-oomi personas create chef --name "Oomi Chef" --summary "Cooking ideas and nutrition guidance"
-```
+The bridge status file is written locally and should roughly be interpreted as:
+- `starting`: process booting or waiting for managed subscription
+- `connected`: broker connected and managed subscription confirmed
+- `reconnecting`: broker or gateway transport dropped and reconnect is scheduled
+- `degraded`: bridge is still alive but hit a runtime fault that needs attention
+- `error`: startup or auth-level failure that prevents useful operation
+- `stopped`: bridge is not running or was intentionally stopped
 
-Optional flags:
-```
-oomi init --workspace /path/to/openclaw/workspace
-oomi init --agents-file /path/to/AGENTS.md
-oomi openclaw install --skills-dir /path/to/openclaw/skills
-oomi openclaw pair --app-url https://your-oomi-app.vercel.app --no-start
-oomi openclaw pair --app-url https://your-oomi-app.vercel.app --json
-oomi personas sync --root /path/to/oomi
-oomi personas create creator --status active --chat-session agent:main:webchat:channel:oomi-creator
-```
+For voice support, a `voice_session_*` failure should be treated as narrower than a full provider outage.
 
-Defaults:
-- Agent instructions are written to `$OPENCLAW_WORKSPACE/AGENTS.md` (if set), otherwise `~/.openclaw/workspace/AGENTS.md`.
-- Skills are installed to `~/.openclaw/skills` and `~/clawd/skills` if present.
-
-Restart OpenClaw after running `oomi init` or `oomi openclaw install`.
-
-## Update Notifications
-- `oomi` checks npm for a newer `oomi-ai` version on normal commands (cached, best-effort).
-- When an update is available it prints:
-  - `pnpm add -g oomi-ai@latest`
-  - fallback: `npm install -g oomi-ai@latest`
-- Optional env controls:
-  - `OOMI_SKIP_UPDATE_CHECK=1` disables checks
-  - `OOMI_UPDATE_CHECK_INTERVAL_MS=<ms>` changes check interval
-  - `OOMI_UPDATE_CHECK_TIMEOUT_MS=<ms>` changes network timeout
-  - `OOMI_BRIDGE_GATEWAY_CONNECT_TIMEOUT_MS=<ms>` changes local gateway socket connect timeout
-  - `OOMI_BRIDGE_CONNECT_CHALLENGE_TIMEOUT_MS=<ms>` changes wait timeout for gateway `connect.challenge` nonce
-  - `OOMI_BRIDGE_GATEWAY_REQUEST_TIMEOUT_MS=<ms>` changes timeout for forwarded gateway `connect`/`chat.send` request responses
-
-Bridge alert helper (reads `~/.openclaw/oomi-bridge-status.json` counters):
-```
-node <repo-root>/scripts/openclaw/bridge-alert-check.mjs
+## Troubleshooting
+
+### `invalid handshake: first request must be connect`
+
+Meaning:
+- a gateway request was forwarded before the session had accepted `connect`
+
+What to check:
+- update to the latest `oomi-ai`
+- restart the bridge worker
+- confirm only one active bridge worker exists for the device
+
+### `duplicate plugin id detected`
+
+Meaning:
+- OpenClaw can see more than one `oomi-ai` plugin source
+
+What to check:
+- ensure there is only one active install under OpenClaw plugin discovery paths
+- remove stale local extension copies before reinstalling
+
+### Bridge keeps flipping between `reconnecting`, `degraded`, or `stopped`
+
+What to check:
+- `oomi openclaw bridge ps`
+- `oomi openclaw bridge service status`
+- `tail -f ~/.openclaw/logs/oomi-bridge-live.log`
+- `tail -f ~/.openclaw/logs/gateway.log`
+
+If the process is alive but runtime faults are being caught, expect `degraded` rather than an immediate hard stop.
+
+### Voice STT works but the agent does not answer
+
+This usually means one of these:
+- the managed gateway/device side is not actually ready
+- the bridge or agent run failed after delivery
+- the OpenClaw run stopped with an upstream provider `network_error`
+
+In that situation, inspect:
+- `~/.openclaw/logs/gateway.log`
+- `~/.openclaw/logs/gateway.err.log`
+- the relevant session JSONL in `~/.openclaw/agents/main/sessions/`
+
+## Developer Notes
+
+If you are inspecting this package on npm, the main architectural points are:
+- the extension path is the stable managed text contract
+- the local bridge exists because Oomi also needs device-backed and voice-capable flows
+- the bridge has been hardened for:
+  - strict `connect`-first forwarding
+  - method-specific request shaping
+  - `idempotencyKey` handling
+  - bridge status that does not report `connected` before managed subscription is ready
+  - runtime fault isolation so local session failures are less likely to crash the whole provider
+
+If you are developing the plugin, test the packaged surface with:
+
+```bash
+cd packages/oomi-ai
+node --test test/*.test.mjs
+npm pack --dry-run
 ```
 
-## Package Audit + Publish (pnpm)
+## Release Process
+
+Before publishing:
+
+```bash
+cd packages/oomi-ai
+node --test test/*.test.mjs
+npm pack --dry-run
 ```
-cd <repo-root>/packages/oomi-ai
-pnpm check
-pnpm publish --dry-run --no-git-checks --access public
-pnpm publish --access public
+
+Then publish the bumped version:
+
+```bash
+npm publish --access public
 ```

package/agent_instructions.md

@@ -1,46 +1,186 @@
-# Oomi Avatar Control
+# Oomi OpenClaw Operator Guide
 
-Before using avatar commands, call `get_avatar_capabilities` and prefer canonical values.
-Inline tags are stripped from user-visible text.
+Use these instructions when you need to connect, repair, or verify an OpenClaw machine that should talk to Oomi.
 
-Use inline tags like:
-- [anim:Waving], [anim:Walking], [anim:Idle], [anim:Sitting Idle]
-- [face:happy], [face:sad], [face:surprised], [face:focused], [face:gentle], [face:thinking]
-- [gesture:nod], [gesture:think], [gesture:shrug], [gesture:wave], [gesture:bow]
-- [look:camera], [look:left], [look:right], [look:up], [look:down]
+## Primary Goal
 
-Aliases allowed (if needed): wave -> Waving, walk -> Walking, idle -> Idle, sit/sitting -> Sitting Idle
+Get OpenClaw into a state where:
+- Oomi managed text chat works
+- Oomi device-backed voice can start cleanly
+- the bridge is supervised or restartable
+- the user never has to paste raw pair identifiers or gateway secrets manually when managed connect is available
 
-## Oomi Connect Flow
+## Standard Connect Flow
 
-If the user asks to connect OpenClaw to Oomi voice/web:
+If the user asks to connect OpenClaw to Oomi chat or voice:
 
-1. Refresh bootstrap/update first:
+1. Refresh bootstrap first:
    - `curl -fsSL https://www.oomi.ai/install.sh | bash`
-2. Ensure `oomi-ai` is installed (`pnpm add -g oomi-ai@latest`).
-   - Fallback: `npm install -g oomi-ai@latest`
-3. Pair device token without starting legacy bridge:
+2. Ensure `oomi-ai` is installed:
+   - `pnpm add -g oomi-ai@latest`
+   - fallback: `npm install -g oomi-ai@latest`
+3. Pair the device without starting a legacy one-off flow:
    - `oomi openclaw pair --app-url https://www.oomi.ai --no-start`
-4. Install OpenClaw plugin:
+4. Install the OpenClaw plugin:
    - `openclaw plugins install oomi-ai@latest`
-5. Configure channel account (`channels.oomi.accounts.default`) from:
+5. Print the config block and secrets:
    - `oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai`
-6. Restart OpenClaw.
-7. Ensure exactly one bridge worker is running (singleton handler):
+6. Apply the printed `channels.oomi.accounts.default` config.
+7. Restart OpenClaw.
+8. Start or repair the bridge:
    - `oomi openclaw bridge ensure --detach`
-   - If stale/disconnected: `oomi openclaw bridge restart --detach`
-   - On macOS, prefer supervised mode: `oomi openclaw bridge service install`
-8. If user provides an app URL (for example "Connect yourself to Oomi. Use app URL <URL>."):
-   - Use that URL directly in the pair command.
-   - Example: `oomi openclaw pair --app-url <URL> --no-start`
-9. Return this exact result format to the user:
-   - `Oomi Connect Ready`
-   - `Auth Link: ...`
-   - No extra narrative text before or after those lines.
-
-Primary UX requirement:
-- Never ask the user to enter raw pairing identifiers.
-- The user should open the auth link, complete sign-in, and land directly in connected chat.
-- Return `/connect/<invite-token>` auth links only.
-
-Do not ask users to paste gateway IP/token/password when managed connect is available.
+   - if stale: `oomi openclaw bridge restart --detach`
+   - on macOS, prefer supervision: `oomi openclaw bridge service install`
+
+If the user provides a specific app URL, use it directly in the pair flow:
+- `oomi openclaw pair --app-url <URL> --no-start`
+
+## Required User-Facing Result
+
+When pairing completes successfully, return this exact format:
+- `Oomi Connect Ready`
+- `Auth Link: ...`
+
+Do not add extra narrative before or after those lines.
+Do not ask the user to enter raw pair codes, gateway passwords, or nonce values manually.
+
+## Runtime Checks
+
+Use these commands to inspect the installed machine:
+
+```bash
+oomi openclaw bridge ps
+oomi openclaw bridge service status
+oomi openclaw status
+tail -f ~/.openclaw/logs/oomi-bridge-live.log
+tail -f ~/.openclaw/logs/gateway.log
+tail -f ~/.openclaw/logs/gateway.err.log
+```
+
+Useful local files:
+- `~/.openclaw/oomi-bridge-status.json`
+- `~/.openclaw/logs/oomi-bridge-live.log`
+- `~/.openclaw/logs/gateway.log`
+- `~/.openclaw/logs/gateway.err.log`
+- `~/.openclaw/agents/main/sessions/*.jsonl`
+
+## Healthy State
+
+Treat the machine as healthy when all of the following are true:
+- OpenClaw loads the `oomi-ai` plugin without duplicate-id conflicts
+- `channels.oomi.accounts.default` is populated with a valid `backendUrl` and `deviceToken`
+- the bridge shows `connected` after managed subscription is confirmed
+- text chat reaches the Oomi assistant
+- voice STT can produce `asr.final`
+- assistant replies can come back without the bridge dropping into `stopped`
+
+Bridge status meanings:
+- `starting`: bridge booting or waiting for managed subscription
+- `connected`: ready for managed chat and voice traffic
+- `reconnecting`: transport dropped and retry is scheduled
+- `degraded`: bridge caught a runtime fault but is still alive
+- `error`: startup/auth failure blocked useful operation
+- `stopped`: not running or intentionally stopped
+
+## Troubleshooting
+
+### Duplicate plugin id warning
+
+Symptom:
+- OpenClaw reports `duplicate plugin id detected`
+
+Action:
+- ensure only one active `oomi-ai` plugin install is discoverable
+- remove stale extension copies before reinstalling
+
+### `invalid handshake: first request must be connect`
+
+Meaning:
+- a gateway request was sent before `connect` had been accepted
+
+Action:
+- update `oomi-ai`
+- restart the bridge
+- confirm only one bridge worker is running
+
+### Device is linked but voice start still fails
+
+Meaning:
+- linked ownership is not enough; the device side still needs to be live
+
+Action:
+- confirm the device websocket is actually online
+- confirm the bridge is `connected`
+- restart the bridge if it is stuck in `reconnecting` or `degraded`
+
+### STT works but the assistant does not reply
+
+Meaning:
+- the voice turn likely reached Oomi, but the managed gateway or OpenClaw run failed later
+
+Action:
+- inspect `gateway.log`, `gateway.err.log`, and the session JSONL
+- check for `network_error`, auth failures, or repeated bridge restarts
+
+### Bridge keeps restarting with `reason: stopped`
+
+Action:
+- confirm the newest `oomi-ai` is installed
+- inspect `~/.openclaw/logs/oomi-bridge-live.log` for runtime exceptions
+- use supervised mode on macOS: `oomi openclaw bridge service install`
+- if the process is alive but faulted, expect `degraded` rather than an immediate hard stop on newer bridge builds
+
+## Voice Notes
+
+Voice depends on the same Oomi plugin and bridge layer as managed chat.
+That means:
+- if plugin install or bridge health is wrong, voice replies will also fail
+- STT can succeed even when assistant reply delivery is broken later in the run
+- a `voice_session_*` failure should be investigated, but it should not automatically be treated as proof that all normal Oomi chat is down
+
+### Hidden Speech Payload
+
+For managed voice turns, keep visible assistant chat text natural and user-facing.
+Do not put spoken-style tags like `[happy]`, `[sad]`, or `[excited]` into visible chat text.
+
+When the runtime supports it, voice turns may include a hidden speech sidecar on the assistant message:
+
+```json
+{
+  "metadata": {
+    "spoken": {
+      "text": "Speech-optimized text for TTS only.",
+      "instructions": "Speak with upbeat, warm excitement and slightly rising intonation.",
+      "style": {
+        "emotion": "excited",
+        "energy": "medium_high"
+      }
+    }
+  }
+}
+```
+
+Rules:
+- visible `content` remains the source of truth for Oomi chat rendering
+- for managed voice replies, include `metadata.spoken` when delivery benefits from cleaner phrasing or explicit speaking guidance
+- `metadata.spoken.text` is for backend TTS only
+- `metadata.spoken.instructions` should be natural-language guidance, not raw bracket tags
+- `metadata.spoken.style` is optional metadata for debugging/future mapping
+- if no hidden speech sidecar exists, Oomi falls back to speaking the visible assistant text
+
+## Avatar Commands
+
+Before using avatar commands, call `get_avatar_capabilities` and prefer canonical values.
+Inline tags are stripped from user-visible text.
+
+Use inline tags like:
+- `[anim:Waving]`, `[anim:Walking]`, `[anim:Idle]`, `[anim:Sitting Idle]`
+- `[face:happy]`, `[face:sad]`, `[face:surprised]`, `[face:focused]`, `[face:gentle]`, `[face:thinking]`
+- `[gesture:nod]`, `[gesture:think]`, `[gesture:shrug]`, `[gesture:wave]`, `[gesture:bow]`
+- `[look:camera]`, `[look:left]`, `[look:right]`, `[look:up]`, `[look:down]`
+
+Aliases allowed if needed:
+- `wave -> Waving`
+- `walk -> Walking`
+- `idle -> Idle`
+- `sit` or `sitting -> Sitting Idle`