oomi-ai 0.2.17 → 0.2.19

package/agent_instructions.md

@@ -1,193 +1,223 @@
-# Oomi OpenClaw Operator Guide
-
-Use these instructions when you need to connect, repair, or verify an OpenClaw machine that should talk to Oomi.
-
-## Primary Goal
-
-Get OpenClaw into a state where:
-- Oomi managed text chat works
-- Oomi device-backed voice can start cleanly
-- the bridge is supervised or restartable
-- the user never has to paste raw pair identifiers or gateway secrets manually when managed connect is available
-
-## Standard Connect Flow
-
-If the user asks to connect OpenClaw to Oomi chat or voice:
-
-1. Refresh bootstrap first:
-   - `curl -fsSL https://www.oomi.ai/install.sh | bash`
-2. Ensure `oomi-ai` is installed:
-   - `pnpm add -g oomi-ai@latest`
-   - fallback: `npm install -g oomi-ai@latest`
-3. Pair the device without starting a legacy one-off flow:
-   - `oomi openclaw pair --app-url https://www.oomi.ai --no-start`
-4. Install the OpenClaw plugin:
-   - `openclaw plugins install oomi-ai@latest`
-5. Print the config block and secrets:
-   - `oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai`
-6. Apply the printed `channels.oomi.accounts.default` config.
-7. Restart OpenClaw.
-8. Start or repair the bridge:
-   - `oomi openclaw bridge ensure --detach`
-   - if stale: `oomi openclaw bridge restart --detach`
-   - on macOS, prefer supervision: `oomi openclaw bridge service install`
-
-If the user provides a specific app URL, use it directly in the pair flow:
-- `oomi openclaw pair --app-url <URL> --no-start`
-
-## Required User-Facing Result
-
-When pairing completes successfully, return this exact format:
-- `Oomi Connect Ready`
-- `Auth Link: ...`
-
-Do not add extra narrative before or after those lines.
-Do not ask the user to enter raw pair codes, gateway passwords, or nonce values manually.
-
-## Runtime Checks
-
-Use these commands to inspect the installed machine:
-
-```bash
-oomi openclaw bridge ps
-oomi openclaw bridge service status
-oomi openclaw status
-tail -f ~/.openclaw/logs/oomi-bridge-live.log
-tail -f ~/.openclaw/logs/gateway.log
-tail -f ~/.openclaw/logs/gateway.err.log
-```
-
-Useful local files:
-- `~/.openclaw/oomi-bridge-status.json`
-- `~/.openclaw/logs/oomi-bridge-live.log`
-- `~/.openclaw/logs/gateway.log`
-- `~/.openclaw/logs/gateway.err.log`
-- `~/.openclaw/agents/main/sessions/*.jsonl`
-
-## Healthy State
-
-Treat the machine as healthy when all of the following are true:
-- OpenClaw loads the `oomi-ai` plugin without duplicate-id conflicts
-- `channels.oomi.accounts.default` is populated with a valid `backendUrl` and `deviceToken`
-- the bridge shows `connected` after managed subscription is confirmed
-- text chat reaches the Oomi assistant
-- voice STT can produce `asr.final`
-- assistant replies can come back without the bridge dropping into `stopped`
-
-Bridge status meanings:
-- `starting`: bridge booting or waiting for managed subscription
-- `connected`: ready for managed chat and voice traffic
-- `reconnecting`: transport dropped and retry is scheduled
-- `degraded`: bridge caught a runtime fault but is still alive
-- `error`: startup/auth failure blocked useful operation
-- `stopped`: not running or intentionally stopped
-
-## Troubleshooting
-
-### Duplicate plugin id warning
-
-Symptom:
-- OpenClaw reports `duplicate plugin id detected`
-
-Action:
-- ensure only one active `oomi-ai` plugin install is discoverable
-- remove stale extension copies before reinstalling
-
-### `invalid handshake: first request must be connect`
-
-Meaning:
-- a gateway request was sent before `connect` had been accepted
-
-Action:
-- update `oomi-ai`
-- restart the bridge
-- confirm only one bridge worker is running
-
-### Device is linked but voice start still fails
-
-Meaning:
-- linked ownership is not enough; the device side still needs to be live
-
-Action:
-- confirm the device websocket is actually online
-- confirm the bridge is `connected`
-- restart the bridge if it is stuck in `reconnecting` or `degraded`
-
-### STT works but the assistant does not reply
-
-Meaning:
-- the voice turn likely reached Oomi, but the managed gateway or OpenClaw run failed later
-
-Action:
-- inspect `gateway.log`, `gateway.err.log`, and the session JSONL
-- check for `network_error`, auth failures, or repeated bridge restarts
-
-### Bridge keeps restarting with `reason: stopped`
-
-Action:
-- confirm the newest `oomi-ai` is installed
-- inspect `~/.openclaw/logs/oomi-bridge-live.log` for runtime exceptions
-- use supervised mode on macOS: `oomi openclaw bridge service install`
-- if the process is alive but faulted, expect `degraded` rather than an immediate hard stop on newer bridge builds
-
-## Voice Notes
-
-Voice depends on the same Oomi plugin and bridge layer as managed chat.
-That means:
-- if plugin install or bridge health is wrong, voice replies will also fail
-- STT can succeed even when assistant reply delivery is broken later in the run
-- a `voice_session_*` failure should be investigated, but it should not automatically be treated as proof that all normal Oomi chat is down
-
-### Hidden Speech Payload
+# Oomi OpenClaw Operator Guide
+
+Use these instructions when you need to connect, repair, or verify an OpenClaw machine that should talk to Oomi.
+
+## Primary Goal
+
+Get OpenClaw into a state where:
+- Oomi managed text chat works
+- Oomi device-backed voice can start cleanly
+- the bridge is supervised or restartable
+- the user never has to paste raw pair identifiers or gateway secrets manually when managed connect is available
+
+## Standard Connect Flow
+
+If the user asks to connect OpenClaw to Oomi chat or voice:
+
+1. Refresh bootstrap first:
+   - `curl -fsSL https://www.oomi.ai/install.sh | bash`
+2. Ensure `oomi-ai` is installed:
+   - `pnpm add -g oomi-ai@latest`
+   - fallback: `npm install -g oomi-ai@latest`
+3. Pair the device without starting a legacy one-off flow:
+   - `oomi openclaw pair --app-url https://www.oomi.ai --no-start`
+4. Install the OpenClaw plugin:
+   - `openclaw plugins install oomi-ai@latest`
+5. Print the config block and secrets:
+   - `oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai`
+6. Apply the printed `channels.oomi.accounts.default` config.
+7. Restart OpenClaw.
+8. Start or repair the bridge:
+   - `oomi openclaw bridge ensure --detach`
+   - if stale: `oomi openclaw bridge restart --detach`
+   - on macOS, prefer supervision: `oomi openclaw bridge service install`
+
+If the user provides a specific app URL, use it directly in the pair flow:
+- `oomi openclaw pair --app-url <URL> --no-start`
+
+## Required User-Facing Result
+
+When pairing completes successfully, return this exact format:
+- `Oomi Connect Ready`
+- `Auth Link: ...`
+
+Do not add extra narrative before or after those lines.
+Do not ask the user to enter raw pair codes, gateway passwords, or nonce values manually.
+
+## Runtime Checks
+
+Use these commands to inspect the installed machine:
+
+```bash
+oomi openclaw bridge ps
+oomi openclaw bridge service status
+oomi openclaw status
+tail -f ~/.openclaw/logs/oomi-bridge-live.log
+tail -f ~/.openclaw/logs/gateway.log
+tail -f ~/.openclaw/logs/gateway.err.log
+```
+
+Useful local files:
+- `~/.openclaw/oomi-bridge-status.json`
+- `~/.openclaw/logs/oomi-bridge-live.log`
+- `~/.openclaw/logs/gateway.log`
+- `~/.openclaw/logs/gateway.err.log`
+- `~/.openclaw/agents/main/sessions/*.jsonl`
+
+## Healthy State
+
+Treat the machine as healthy when all of the following are true:
+- OpenClaw loads the `oomi-ai` plugin without duplicate-id conflicts
+- `channels.oomi.accounts.default` is populated with a valid `backendUrl` and `deviceToken`
+- the bridge shows `connected` after managed subscription is confirmed
+- text chat reaches the Oomi assistant
+- voice STT can produce `asr.final`
+- assistant replies can come back without the bridge dropping into `stopped`
+
+Bridge status meanings:
+- `starting`: bridge booting or waiting for managed subscription
+- `connected`: ready for managed chat and voice traffic
+- `reconnecting`: transport dropped and retry is scheduled
+- `degraded`: bridge caught a runtime fault but is still alive
+- `error`: startup/auth failure blocked useful operation
+- `stopped`: not running or intentionally stopped
+
+## Troubleshooting
+
+### Duplicate plugin id warning
+
+Symptom:
+- OpenClaw reports `duplicate plugin id detected`
+
+Action:
+- ensure only one active `oomi-ai` plugin install is discoverable
+- remove stale extension copies before reinstalling
+
+### `invalid handshake: first request must be connect`
+
+Meaning:
+- a gateway request was sent before `connect` had been accepted
+
+Action:
+- update `oomi-ai`
+- restart the bridge
+- confirm only one bridge worker is running
+
+### Device is linked but voice start still fails
+
+Meaning:
+- linked ownership is not enough; the device side still needs to be live
+
+Action:
+- confirm the device websocket is actually online
+- confirm the bridge is `connected`
+- restart the bridge if it is stuck in `reconnecting` or `degraded`
+
+### STT works but the assistant does not reply
+
+Meaning:
+- the voice turn likely reached Oomi, but the managed gateway or OpenClaw run failed later
+
+Action:
+- inspect `gateway.log`, `gateway.err.log`, and the session JSONL
+- check for `network_error`, auth failures, or repeated bridge restarts
+
+### Bridge keeps restarting with `reason: stopped`
+
+Action:
+- confirm the newest `oomi-ai` is installed
+- inspect `~/.openclaw/logs/oomi-bridge-live.log` for runtime exceptions
+- use supervised mode on macOS: `oomi openclaw bridge service install`
+- if the process is alive but faulted, expect `degraded` rather than an immediate hard stop on newer bridge builds
+
+## Voice Notes
+
+Voice depends on the same Oomi plugin and bridge layer as managed chat.
+That means:
+- if plugin install or bridge health is wrong, voice replies will also fail
+- STT can succeed even when assistant reply delivery is broken later in the run
+- a `voice_session_*` failure should be investigated, but it should not automatically be treated as proof that all normal Oomi chat is down
+
+### Hidden Speech Payload
+
+For managed voice turns, keep visible assistant chat text natural and user-facing.
+Do not put spoken-style tags like `[happy]`, `[sad]`, or `[excited]` into visible chat text.
+
+When the runtime supports it, voice turns may include a hidden speech sidecar on the assistant message:
+
+```json
+{
+  "metadata": {
+    "spoken": {
+      "text": "Speech-optimized text for TTS only.",
+      "instructions": "Speak with upbeat, warm excitement and slightly rising intonation.",
+      "style": {
+        "emotion": "excited",
+        "energy": "medium_high"
+      }
+    }
+  }
+}
+```
+
+Rules:
+- visible `content` remains the source of truth for Oomi chat rendering
+- for managed voice replies, include `metadata.spoken` when delivery benefits from cleaner phrasing or explicit speaking guidance
+- `metadata.spoken.text` is for backend TTS only
+- `metadata.spoken.language` should be one of the supported Qwen language values such as `English`
+- `metadata.spoken.segments` can carry bounded per-segment prosody for pace, pitch, volume, and pause timing
+- `metadata.spoken.instructions` should be natural-language guidance, not raw bracket tags
+- `metadata.spoken.style` is optional metadata for debugging/future mapping
+- if no hidden speech sidecar exists, Oomi falls back to speaking the visible assistant text
+
+Current plugin behavior:
+- if you provide `metadata.spoken`, the plugin preserves it unchanged
+- if you do not provide `metadata.spoken`, the plugin now synthesizes a bounded hidden fallback from visible assistant text for backend TTS
+- visible chat text is still never rewritten by the plugin
+
+## Avatar Commands
+
+Before using avatar commands, call `get_avatar_capabilities` and prefer canonical values.
+Inline tags are stripped from user-visible text.
+
+Use inline tags like:
+- `[anim:Waving]`, `[anim:Walking]`, `[anim:Idle]`, `[anim:Sitting Idle]`
+- `[face:happy]`, `[face:sad]`, `[face:surprised]`, `[face:focused]`, `[face:gentle]`, `[face:thinking]`
+- `[gesture:nod]`, `[gesture:think]`, `[gesture:shrug]`, `[gesture:wave]`, `[gesture:bow]`
+- `[look:camera]`, `[look:left]`, `[look:right]`, `[look:up]`, `[look:down]`
 
-For managed voice turns, keep visible assistant chat text natural and user-facing.
-Do not put spoken-style tags like `[happy]`, `[sad]`, or `[excited]` into visible chat text.
+Aliases allowed if needed:
+- `wave -> Waving`
+- `walk -> Walking`
+- `idle -> Idle`
+- `sit` or `sitting -> Sitting Idle`
 
-When the runtime supports it, voice turns may include a hidden speech sidecar on the assistant message:
+## Persona App Generation
 
-```json
-{
-  "metadata": {
-    "spoken": {
-      "text": "Speech-optimized text for TTS only.",
-      "instructions": "Speak with upbeat, warm excitement and slightly rising intonation.",
-      "style": {
-        "emotion": "excited",
-        "energy": "medium_high"
-      }
-    }
-  }
-}
-```
+When a user asks you in chat to create a new persona for Oomi:
 
-Rules:
-- visible `content` remains the source of truth for Oomi chat rendering
-- for managed voice replies, include `metadata.spoken` when delivery benefits from cleaner phrasing or explicit speaking guidance
-- `metadata.spoken.text` is for backend TTS only
-- `metadata.spoken.language` should be one of the supported Qwen language values such as `English`
-- `metadata.spoken.segments` can carry bounded per-segment prosody for pace, pitch, volume, and pause timing
-- `metadata.spoken.instructions` should be natural-language guidance, not raw bracket tags
-- `metadata.spoken.style` is optional metadata for debugging/future mapping
-- if no hidden speech sidecar exists, Oomi falls back to speaking the visible assistant text
+1. Prefer `oomi personas create-managed --name "<name>" --description "<description>"`.
+2. That is the primary end-to-end command for agent-driven persona creation.
+3. It creates the managed persona record in Oomi and relies on the running bridge to consume the queued `persona_job` automatically.
+4. Do not manually scaffold first unless you are recovering a failed persona job or working outside the queued Oomi flow.
 
-Current plugin behavior:
-- if you provide `metadata.spoken`, the plugin preserves it unchanged
-- if you do not provide `metadata.spoken`, the plugin now synthesizes a bounded hidden fallback from visible assistant text for backend TTS
-- visible chat text is still never rewritten by the plugin
+When generating a managed persona app for Oomi:
 
-## Avatar Commands
+1. Do not build the app shell from scratch.
+2. Always run `oomi personas scaffold <slug> --name "<name>" --description "<description>" --out <path>` first.
+3. Only customize persona-specific files inside `src/persona/` and `persona/` unless Oomi explicitly instructs otherwise.
+4. Preserve the scaffolded WebSpatial/Vite shell, `public/oomi.health.json`, `oomi.runtime.json`, and `public/manifest.webmanifest`.
+5. After customization, start the app and register the runtime with Oomi using the current runtime contract.
 
-Before using avatar commands, call `get_avatar_capabilities` and prefer canonical values.
-Inline tags are stripped from user-visible text.
+When executing a structured persona job from Oomi:
 
-Use inline tags like:
-- `[anim:Waving]`, `[anim:Walking]`, `[anim:Idle]`, `[anim:Sitting Idle]`
-- `[face:happy]`, `[face:sad]`, `[face:surprised]`, `[face:focused]`, `[face:gentle]`, `[face:thinking]`
-- `[gesture:nod]`, `[gesture:think]`, `[gesture:shrug]`, `[gesture:wave]`, `[gesture:bow]`
-- `[look:camera]`, `[look:left]`, `[look:right]`, `[look:up]`, `[look:down]`
+1. Prefer `oomi persona-jobs execute --message-file <job.json>` when the backend has already produced a machine-readable job payload.
+2. That command is allowed to scaffold the app, install dependencies, start the local runtime, wait for the health document, register the runtime, and report job success or failure.
+3. Use the lower-level commands only for recovery or partial reruns:
+   - `oomi personas runtime-register <slug> --local-port 4789`
+   - `oomi personas heartbeat <slug> --local-port 4789`
+   - `oomi persona-jobs start <jobId>`
+   - `oomi persona-jobs succeed <jobId> --workspace-path <path> --local-port 4789`
+   - `oomi persona-jobs fail <jobId> --code <code> --message "<text>"`
 
-Aliases allowed if needed:
-- `wave -> Waving`
-- `walk -> Walking`
-- `idle -> Idle`
-- `sit` or `sitting -> Sitting Idle`
+When the Oomi bridge is running on the machine, queued persona jobs from Oomi are now polled and executed automatically through the filtered control-message lane. You should still use the explicit commands above for manual retries, recovery, or direct operator workflows.