oomi-ai 0.3.8 → 0.4.0

package/skills/oomi/agent_instructions.md

@@ -1,49 +1,84 @@
-# Oomi Avatar Control
+# Oomi Local Agent Operator Guide
 
-Prefer canonical avatar values.
-Inline tags are stripped from user-visible text.
+Connect the agent the user is speaking to. Identify your runtime and profile first. Never choose OpenClaw merely because it is installed on the same machine as Hermes.
 
-Use inline tags like:
-- [anim:Waving], [anim:Walking], [anim:Idle], [anim:Sitting Idle]
-- [face:happy], [face:sad], [face:surprised], [face:focused], [face:gentle], [face:thinking]
-- [gesture:nod], [gesture:think], [gesture:shrug], [gesture:wave], [gesture:bow]
-- [look:camera], [look:left], [look:right], [look:up], [look:down]
-
-Aliases allowed (if needed): wave -> Waving, walk -> Walking, idle -> Idle, sit/sitting -> Sitting Idle
-
-## Oomi Connect Flow
-
-If the user asks to connect OpenClaw to Oomi voice or chat:
-
-1. For a first-time install or missing local pairing state, run bootstrap exactly once using the requested app URL:
-   - `curl -fsSL https://www.oomi.ai/install.sh | bash`
-   - Bootstrap installs or updates `oomi-ai`, performs the one initial pair, installs the plugin, and starts the saved bridge detached.
-   - Do not pair again after bootstrap.
-2. Configure channel account (`channels.oomi.accounts.default`) from:
-   - `oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai`
-3. Restart OpenClaw.
-4. Ensure exactly one bridge worker is running (singleton handler):
-   - `oomi openclaw refresh --skip-version-check`
-   - `oomi openclaw bridge ensure --detach`
-   - If stale/disconnected: `oomi openclaw bridge restart --detach`
-   - On macOS, prefer supervised mode: `oomi openclaw bridge service install`
-5. If the Auth Link printed by bootstrap is missing or expired, create a replacement without re-pairing:
-   - `oomi openclaw invite --app-url https://www.oomi.ai`
-6. If the user provides an app URL, substitute it in both the bootstrap URL and the `invite --app-url` command.
-7. If the machine is already paired, do not rerun bootstrap. Use `invite` for link recovery and `bridge ensure` or `bridge restart` for runtime recovery.
-8. Return this exact result format to the user:
-   - `Oomi Connect Ready`
-   - `Auth Link: ...`
-   - No extra narrative text before or after those lines.
-
-Primary UX requirement:
-- Never ask the user to enter raw pairing identifiers.
-- The user should open the auth link, complete sign-in, and land directly in connected chat.
-- Return `/connect/<invite-token>` auth links only.
-
-Do not ask users to paste gateway IP/token/password when managed connect is available.
-
-## Hidden Speech Payload
+## Standard Connect Flow
+
+Install or update the local package:
+
+```bash
+npm install -g oomi-ai@latest
+```
+
+Run ONE command for your runtime:
+
+```bash
+# OpenClaw (reuses existing default OpenClaw pairing)
+oomi connect --runtime openclaw --app-url https://www.oomi.ai
+
+# Hermes (separate connection, no OpenClaw required)
+oomi connect --runtime hermes --name hermes --app-url https://www.oomi.ai
+
+# Another local agent, through its own adapter
+oomi connect --runtime command --name my-agent --command '["/absolute/path/to/adapter"]' --app-url https://www.oomi.ai
+```
+
+Use the user's app URL when provided. Hermes supports `--profile <profile>` and `--resume <exact-session-id>`. Without `--resume`, Oomi starts a persistent conversation with the same configured agent. Do not use `latest` or claim that an existing chat session was transferred unless its exact ID was selected. Hermes uses `chat --quiet --query-file -`, reads the final reply from stdout and the exact `session_id` from stderr, and resumes that ID on subsequent turns.
+
+The command verifies a real local reply, reuses saved credentials or pairs once, starts the bridge, waits for Oomi's subscription, then creates a single-use Auth Link. Failed verification means setup is incomplete: repair the selected runtime, never substitute a different one.
+
+The installer is an alternative to npm plus connect, and requires an explicit runtime:
+
+```bash
+curl -fsSL https://www.oomi.ai/install.sh | bash -s -- --runtime hermes --name hermes
+```
+
+## Required User-Facing Result
+
+Return the verified runtime/profile and the printed Auth Link. The human opens the link, signs in, and approves ownership. Only after a message receives a reply in Oomi can you claim end-to-end delivery works. Never expose raw tokens, pairing codes, or gateway credentials.
+
+## Repair and Reconnect
+
+Use the SAME connection name from setup. Omit `--name` for default OpenClaw.
+
+```bash
+oomi status --name hermes --json
+oomi verify --name hermes
+oomi bridge restart --name hermes --detach
+oomi invite --name hermes --app-url https://www.oomi.ai
+```
+
+Do not pair again to refresh an expired Auth Link. Re-running `connect` reuses the saved pairing and verifies the runtime again. A different command, profile, or working directory requires a new connection name, so one agent cannot silently replace another. Named connections store private state and logs under `~/.oomi/connections/<name>`; default OpenClaw keeps `~/.openclaw`. A detached worker requires the computer to remain awake and does not start automatically after reboot. Configure the host's process supervisor when persistence across reboot is needed.
+
+`connected` describes the broker transport. Inspect `runtimeReady`, `verifiedAt`, and `lastReplyAt` separately. A process ID, Auth Link, or broker connection alone does not prove the agent is answering.
+
+## Other Runtime Adapter Contract
+
+`--command` is a JSON argv array, not a shell command. Write an adapter against the selected agent's documented API or noninteractive CLI. The executable reads one JSON object from stdin:
+
+```json
+{"sessionKey":"...","message":"...","idempotencyKey":"...","history":[],"runtimeSessionId":"..."}
+```
+
+It writes `{"text":"final reply","sessionId":"optional-runtime-session"}` to stdout and exits zero on success. Errors must exit nonzero; diagnostics go to stderr. Resume `runtimeSessionId`, or use `history` for stateless runtimes. Never bypass the agent's existing approval rules. The adapter supports text (including voice transcripts); attachments are rejected explicitly. A failed or interrupted action is not silently repeated. Check the agent before sending a new message after a failure.
+
+## OpenClaw Channel Extension
+
+Managed web text and voice connect through the local gateway. The optional OpenClaw `oomi-ai` channel plugin handles agent-initiated channel output. If you need that path, install the plugin, enable its entry in OpenClaw, apply `oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai` locally, and restart OpenClaw. Never share that secret config with the user. Disabling the plugin does not explain a gateway protocol rejection.
+
+OpenClaw gateway protocols 3 and 4 are negotiated independently of the broker. For an existing install affected by a protocol mismatch, update `oomi-ai`, then run `oomi connect --runtime openclaw --app-url https://www.oomi.ai`.
+
+## Voice Notes
+
+Voice depends on the same Oomi plugin and bridge layer as managed chat.
+
+That means:
+
+- if plugin install or bridge health is wrong, voice replies will also fail
+- STT can succeed even when assistant reply delivery is broken later in the run
+- a `voice_session_*` failure should be investigated, but it should not automatically be treated as proof that all normal Oomi chat is down
+
+### Hidden Speech Payload
 
 For managed voice turns, keep visible assistant chat text natural and user-facing.
 Do not put spoken-style tags like `[happy]`, `[sad]`, or `[excited]` into visible chat text.
@@ -66,38 +101,65 @@ When the runtime supports it, voice turns may include a hidden speech sidecar on
 ```
 
 Rules:
+
 - visible `content` remains the source of truth for Oomi chat rendering
-- for managed voice replies, include `metadata.spoken` when delivery benefits from cleaner phrasing or explicit speaking guidance
+- for managed voice replies, include `metadata.spoken` whenever backend TTS should speak the turn
 - `metadata.spoken.text` is for backend TTS only
 - `metadata.spoken.language` should use the backend-supported language value for the active provider; use `English` only when no more specific locale is supplied
 - `metadata.spoken.segments` can carry bounded per-segment prosody for pace, pitch, volume, and pause timing
 - `metadata.spoken.instructions` should be natural-language guidance, not raw bracket tags
-- `metadata.spoken.style` is optional metadata for debugging or future mapping
-- if no hidden speech sidecar exists, Oomi falls back to speaking the visible assistant text
-- if you omit `metadata.spoken`, the plugin now synthesizes a bounded hidden fallback from visible assistant text
-- visible chat text should be clean before sending; the plugin may strip private runtime wrappers and avatar command tags as a safety guardrail
+- visible chat text should be clean before sending; the package may strip private runtime wrappers and avatar command tags as a safety guardrail
+
+## Avatar Commands
+
+Prefer canonical avatar values.
+Inline tags are stripped from user-visible text.
+
+Use inline tags like:
+
+- `[anim:Waving]`, `[anim:Walking]`, `[anim:Idle]`, `[anim:Sitting Idle]`
+- `[face:happy]`, `[face:sad]`, `[face:surprised]`, `[face:focused]`, `[face:gentle]`, `[face:thinking]`
+- `[gesture:nod]`, `[gesture:think]`, `[gesture:shrug]`, `[gesture:wave]`, `[gesture:bow]`
+- `[look:camera]`, `[look:left]`, `[look:right]`, `[look:up]`, `[look:down]`
+
+Aliases allowed if needed:
+
+- `wave -> Waving`
+- `walk -> Walking`
+- `idle -> Idle`
+- `sit` or `sitting -> Sitting Idle`
 
 ## Persona App API Tools
 
 Oomi persona apps are component-composed client surfaces backed by approved templates, components, data bindings, actions, and permissions.
 
-Persona UI creation and rendering happen inside Oomi-managed systems. Use the Oomi API tools instead:
+The OpenClaw agent should inspect and update persona app state through the Oomi API tools in this package.
 
-```bash
-oomi persona-apps list --json
-oomi persona-apps show fitness-today --json
-oomi persona-apps apply-action fitness-today --action fitness.complete_workout --payload-json '{"goalId":"easy-run","minutes":20}' --json
-```
+When a user asks about an existing persona app:
 
-Rules:
-- answer from returned `appState.bindings`, not memory
-- apply updates only through approved `persona-apps apply-action` actions
-- if the persona app is missing, ask the user to add it from the Oomi client first so permissions and hydration can run
-- do not execute persona UI jobs or start local persona app runtimes
+1. Run `oomi persona-apps list --json` to see the persona apps linked to this user's account.
+2. Run `oomi persona-apps show <slug> --json` before answering detailed questions about that persona's state.
+3. Answer from the returned `appState.bindings`, not from memory or guesses.
+4. To update state, use a validated action: `oomi persona-apps apply-action <slug> --action <action-id> --payload-json '<json>' --json`.
+5. If `persona-apps list` returns no app for the user's request, tell the user they can add that persona from the Oomi client so the client can collect the right permissions and data sources.
+
+For the Fitness v0 slice:
+
+- Use `fitness.complete_goal` with payload `{"goalId":"easy-run"}` to mark a known goal complete.
+- Use `fitness.complete_workout` with payload like `{"goalId":"easy-run","minutes":20}` when the user says they finished a workout.
+
+Strict rules:
+
+- Do not create local persona UI projects.
+- Do not run local persona app runtimes.
+- Do not execute persona UI jobs.
+- Do not install app framework dependencies for persona UI work.
+- Do not present local URLs as persona app URLs.
+- If a persona app needs to be created, tell the user to add it from the Oomi client first so permissions and hydration run correctly.
 
 ## Permissioned Context Tools
 
-Use Oomi context tools when the user asks about personal data that should come from mobile permissions instead of memory.
+Use Oomi context tools when the user asks about personal data that should come from mobile permissions instead of model memory.
 
 For health, fitness, sleep, workout, cardio, or movement questions:
 
@@ -106,11 +168,29 @@ oomi context health --json
 ```
 
 Rules:
-- answer only from `healthContext.summary` and `healthContext.derived`
-- respect `healthContext.agentGuidance.canAnswerFromContext`
-- if `healthContext.agentGuidance.recommendedCommand` is present, run it before answering current/latest health questions
-- if `healthContext.status` is `stale` or `needs_sync`, run `oomi context health sync --wait --json`; answer only after fresh context is uploaded, or tell the user the phone sync is still pending if it times out
-- if `canAnswerFromContext` is false and no sync command is recommended, tell the user the relevant `healthContext.repair.label` or `healthContext.repair.reason`
-- never present stale HealthKit values as current values
-- do not infer unavailable health fields
-- do not request HealthKit directly from the phone; the Oomi mobile app owns permission prompts and syncs approved data to the backend
+
+- Answer only from `healthContext.summary` and `healthContext.derived`.
+- Respect `healthContext.agentGuidance.canAnswerFromContext`.
+- If `healthContext.agentGuidance.recommendedCommand` is present, run it before answering current/latest health questions.
+- If `healthContext.status` is `stale` or `needs_sync`, run `oomi context health sync --wait --json`; answer only after fresh context is uploaded, or tell the user the phone sync is still pending if it times out.
+- If `canAnswerFromContext` is false and no sync command is recommended, tell the user the relevant `healthContext.repair.label` or `healthContext.repair.reason`.
+- Never present stale HealthKit values as current values.
+- Do not infer unavailable health fields.
+- Do not request HealthKit directly from the phone; the Oomi mobile app owns permission prompts and syncs approved data to the backend.
+
+## Phone Notifications
+
+To reach the user on their phone when they are not in the conversation, send a push notification:
+
+```bash
+oomi notify "Your 7-day plan is ready. Want to look now?" --title "Fitness" --json
+```
+
+Rules:
+
+- Notify only for things worth interrupting the user for: a reminder they asked for, a finished task, or something time-sensitive. Say everything else in chat.
+- Keep the message short and specific. The title defaults to your agent name.
+- Tapping the notification opens the Oomi app to the conversation, where the message also appears.
+- Messages you send the user on the Oomi channel already notify their phone. Do not also run `oomi notify` for the same message.
+- If `notification.status` is `skipped`, tell the user `notification.guidance` instead of retrying.
+- If the request is rate limited, do not retry until `retryAfterSeconds` has passed.