oomi-ai 0.2.13 → 0.2.15

package/skills/oomi/SKILL.md

@@ -1,102 +1,169 @@
 ---
 name: oomi
-description: Interact with the Oomi 3D Avatar application to retrieve health data, set goals, and control the avatar persona.
+description: Support Oomi OpenClaw installs, bridge health, managed chat and voice setup, and avatar control.
 ---
 
 # Oomi Skill
 
-This skill allows you to interact with the running Oomi application (localhost). You can fetch user activity data, set new goals, and sync context.
+Use this skill when you need to:
+- connect an OpenClaw machine to Oomi
+- repair the Oomi plugin or bridge on a machine
+- inspect managed chat or voice health
+- control the Oomi avatar with inline tags
 
-## Prerequisites
+## Primary Operator Workflow
 
-- The Oomi Next.js app must be running locally (`npm run dev`) at `http://localhost:3000`.
+If the user wants OpenClaw connected to Oomi:
 
-## Configuration
+1. Ensure `oomi-ai` is installed or updated:
+```bash
+pnpm add -g oomi-ai@latest
+```
+Fallback:
+```bash
+npm install -g oomi-ai@latest
+```
+
+2. Pair the device:
+```bash
+oomi openclaw pair --app-url https://www.oomi.ai --no-start
+```
+
+3. Install the plugin:
+```bash
+openclaw plugins install oomi-ai@latest
+```
+
+4. Print config guidance:
+```bash
+oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai
+```
+
+5. Apply the `channels.oomi.accounts.default` config and restart OpenClaw.
+
+6. Start or repair the bridge:
+```bash
+oomi openclaw bridge ensure --detach
+```
+If stale:
+```bash
+oomi openclaw bridge restart --detach
+```
+On macOS, prefer supervised mode:
+```bash
+oomi openclaw bridge service install
+```
 
-Before using the skill, run the setup script to configure the API URL:
-```python
-python3 skills/oomi/setup.py
+## Health Checks
+
+Use these when chat or voice is failing:
+
+```bash
+oomi openclaw bridge ps
+oomi openclaw bridge service status
+oomi openclaw status
+tail -f ~/.openclaw/logs/oomi-bridge-live.log
+tail -f ~/.openclaw/logs/gateway.log
+tail -f ~/.openclaw/logs/gateway.err.log
 ```
-Default URL is `http://localhost:3000/api/skill`.
 
-## Tools
+Interpret bridge states like this:
+- `starting`: booting or waiting for managed subscription
+- `connected`: ready for managed traffic
+- `reconnecting`: retry scheduled after transport failure
+- `degraded`: bridge caught a runtime fault but is still alive
+- `error`: startup or auth failure blocked operation
+- `stopped`: not running or intentionally shut down
+
+## Common Failures
+
+### Duplicate plugin id
+- Cause: multiple discoverable `oomi-ai` installs
+- Action: remove stale plugin copies and reinstall once
+
+### `invalid handshake: first request must be connect`
+- Cause: gateway request ordering broke
+- Action: update `oomi-ai`, restart the bridge, confirm only one bridge worker exists
+
+### STT works but the assistant does not reply
+- Cause: the voice turn reached Oomi, but the managed gateway or OpenClaw run failed later
+- Action: inspect `gateway.log`, `gateway.err.log`, and the session JSONL for that run
+
+## Local Oomi API Tools
+
+These scripts interact with the local Oomi application when it is running.
 
 ### `get_data`
-Fetches the user's latest health and activity data from the Oomi app.
+Fetch the latest user activity data.
 
-**Usage:**
-```python
+```bash
 python3 skills/oomi/scripts/get_data.py
 ```
 
-**Returns:**
-JSON string containing:
-- `steps`: Daily step count
-- `sleep`: Sleep duration in hours
-- `energy`: Calculated energy level (0-100)
-- `mood`: Current user mood (if tracked)
-
 ### `set_goal`
-Sets a new activity or behavior goal for the user in the Oomi app.
+Set a new goal in the local Oomi app.
 
-**Usage:**
-```python
+```bash
 python3 skills/oomi/scripts/send_goal.py --type "steps" --value 10000 --message "Let's hit 10k today!"
 ```
 
-**Arguments:**
-- `--type`: Type of goal (e.g., "steps", "sleep", "focus")
-- `--value`: Target value (number)
-- `--message`: Motivational message to display to the user
-
 ### `sync`
-Performs a full context sync, updating Oomi with the Agent's current understanding of the user's state.
+Sync local context.
 
-**Usage:**
-```python
+```bash
 python3 skills/oomi/scripts/sync.py
 ```
 
 ### `get_avatar_capabilities`
-Returns the current avatar command schema (supported animations, expressions, gestures, and aliases).
+Read the avatar command schema before emitting inline avatar tags.
 
-**Usage:**
-```python
+```bash
 python3 skills/oomi/scripts/get_avatar_capabilities.py
 ```
 
-**Returns:**
-JSON containing:
-- `commands.anim.values` (supported animation names)
-- `commands.anim.aliases` (accepted shorthand -> animation name)
-- `commands.face.values` (supported expressions)
-- `commands.gesture.values` (supported gestures)
-- `commands.look.values` (supported look targets)
-
 ### `install_agent_instructions`
-Installs Oomi avatar command instructions into an OpenClaw `AGENTS.md` file.
+Install packaged Oomi operator instructions into an OpenClaw `AGENTS.md` file.
 
-**Usage:**
-```python
+```bash
 python3 skills/oomi/scripts/install_agent_instructions.py
 ```
 
-**Options:**
-- `--agents-file` Path to the `AGENTS.md` file (defaults to `OPENCLAW_WORKSPACE/AGENTS.md` or repo `AGENTS.md`).
-- `--instructions-file` Override the instructions markdown file.
-
-## Persona Control (Inline)
+## Hidden Speech Payload
+
+Managed voice can carry a hidden TTS-only speech sidecar alongside the normal assistant message.
+
+Use this shape when a voice turn needs more natural delivery without changing visible chat text:
+
+```json
+{
+  "metadata": {
+    "spoken": {
+      "text": "Speech-optimized text for TTS only.",
+      "instructions": "Speak with upbeat, warm excitement and slightly rising intonation.",
+      "style": {
+        "emotion": "excited",
+        "energy": "medium_high"
+      }
+    }
+  }
+}
+```
 
-In addition to these scripts, you can control the avatar's visualization directly in your text responses using the following tags. These tags are invisible to the user.
+Rules:
+- keep visible assistant `content` clean and user-facing
+- do not place raw intonation tags in visible chat
+- for managed voice replies, include `metadata.spoken` when delivery benefits from cleaner phrasing or explicit speaking guidance
+- `metadata.spoken.text` is backend TTS input only
+- `metadata.spoken.instructions` should use natural-language speaking guidance
+- if the speech sidecar is absent, Oomi speaks the visible assistant text
 
-- **Animations (canonical)**: `[anim:Waving]`, `[anim:Walking]`, `[anim:Idle]`, `[anim:Sitting Idle]`
-  - **Aliases**: `wave -> Waving`, `walk -> Walking`, `idle -> Idle`, `sit/sitting -> Sitting Idle`
-- **Expressions**: `[face:happy]`, `[face:sad]`, `[face:surprised]`, `[face:focused]`, `[face:gentle]`, `[face:thinking]`, `[face:curious]`, `[face:confused]`
-- **Gestures**: `[gesture:nod]`, `[gesture:think]`, `[gesture:shrug]`, `[gesture:wave]`, `[gesture:bow]`
-- **Gaze**: `[look:camera]`, `[look:left]`, `[look:right]`, `[look:up]`, `[look:down]`
+## Avatar Control
 
-**Example:**
-"I see you didn't sleep well last night. [face:worried] [gesture:think] Maybe we should take it easy today?"
+Before emitting avatar commands, call `get_avatar_capabilities` and prefer canonical values.
+Use aliases only when explicitly needed.
 
-**Recommended instruction for agents:**
-Before emitting avatar commands, call `get_avatar_capabilities` and prefer canonical values. Use aliases only if explicitly needed.
+Supported inline tags include:
+- animations: `[anim:Waving]`, `[anim:Walking]`, `[anim:Idle]`, `[anim:Sitting Idle]`
+- expressions: `[face:happy]`, `[face:sad]`, `[face:surprised]`, `[face:focused]`, `[face:gentle]`, `[face:thinking]`
+- gestures: `[gesture:nod]`, `[gesture:think]`, `[gesture:shrug]`, `[gesture:wave]`, `[gesture:bow]`
+- gaze: `[look:camera]`, `[look:left]`, `[look:right]`, `[look:up]`, `[look:down]`

package/skills/oomi/agent_instructions.md

@@ -44,3 +44,33 @@ Primary UX requirement:
 - Return `/connect/<invite-token>` auth links only.
 
 Do not ask users to paste gateway IP/token/password when managed connect is available.
+
+## Hidden Speech Payload
+
+For managed voice turns, keep visible assistant chat text natural and user-facing.
+Do not put spoken-style tags like `[happy]`, `[sad]`, or `[excited]` into visible chat text.
+
+When the runtime supports it, voice turns may include a hidden speech sidecar on the assistant message:
+
+```json
+{
+  "metadata": {
+    "spoken": {
+      "text": "Speech-optimized text for TTS only.",
+      "instructions": "Speak with upbeat, warm excitement and slightly rising intonation.",
+      "style": {
+        "emotion": "excited",
+        "energy": "medium_high"
+      }
+    }
+  }
+}
+```
+
+Rules:
+- visible `content` remains the source of truth for Oomi chat rendering
+- for managed voice replies, include `metadata.spoken` when delivery benefits from cleaner phrasing or explicit speaking guidance
+- `metadata.spoken.text` is for backend TTS only
+- `metadata.spoken.instructions` should be natural-language guidance, not raw bracket tags
+- `metadata.spoken.style` is optional metadata for debugging or future mapping
+- if no hidden speech sidecar exists, Oomi falls back to speaking the visible assistant text