oomi-ai 0.2.49 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +227 -463
- package/agent_instructions.md +244 -234
- package/bin/oomi-ai.js +4028 -5797
- package/bin/sessionBridgeState.js +78 -78
- package/lib/openclawPaths.js +70 -71
- package/lib/openclawProfile.js +216 -216
- package/lib/personaApiClient.js +133 -303
- package/lib/spokenMetadata.js +137 -137
- package/openclaw.extension.js +341 -341
- package/openclaw.plugin.json +17 -17
- package/package.json +59 -59
- package/persona-app/README.md +27 -0
- package/persona-app/registry/v1.json +63 -0
- package/persona-app/schema/persona-app.v1.schema.json +90 -0
- package/skills/oomi/SKILL.md +165 -182
- package/skills/oomi/agent_instructions.md +99 -80
- package/lib/channelPluginClient.js +0 -119
- package/lib/openclawDevGateway.js +0 -384
- package/lib/personaJobExecutor.js +0 -139
- package/lib/personaJobPoller.js +0 -112
- package/lib/personaPortAllocator.js +0 -36
- package/lib/personaRuntimeManager.js +0 -496
- package/lib/personaRuntimeProcess.js +0 -924
- package/lib/personaRuntimeRegistry.js +0 -67
- package/lib/personaRuntimeSupervisor.js +0 -330
- package/lib/scaffold.js +0 -108
- package/lib/template.js +0 -45
- package/skills/oomi/config.json +0 -3
- package/skills/oomi/scripts/get_avatar_capabilities.py +0 -40
- package/skills/oomi/scripts/get_data.py +0 -49
- package/skills/oomi/scripts/install_agent_instructions.py +0 -78
- package/skills/oomi/scripts/send_goal.py +0 -53
- package/skills/oomi/scripts/sync.py +0 -46
- package/skills/oomi/setup.py +0 -41
- package/templates/persona-app/.env.example +0 -8
- package/templates/persona-app/README.md +0 -47
- package/templates/persona-app/eslint.config.js +0 -28
- package/templates/persona-app/index.html +0 -18
- package/templates/persona-app/oomi.runtime.json +0 -13
- package/templates/persona-app/package.json +0 -44
- package/templates/persona-app/persona/brief.md +0 -14
- package/templates/persona-app/persona.json +0 -14
- package/templates/persona-app/public/manifest.webmanifest +0 -8
- package/templates/persona-app/public/oomi.health.json +0 -6
- package/templates/persona-app/src/App.css +0 -379
- package/templates/persona-app/src/App.tsx +0 -17
- package/templates/persona-app/src/index.css +0 -53
- package/templates/persona-app/src/main.tsx +0 -23
- package/templates/persona-app/src/pages/HomePage.tsx +0 -127
- package/templates/persona-app/src/pages/ScenePage.tsx +0 -158
- package/templates/persona-app/src/persona/config.ts +0 -6
- package/templates/persona-app/src/persona/notes.ts +0 -9
- package/templates/persona-app/src/spatial.ts +0 -82
- package/templates/persona-app/src/vite-env.d.ts +0 -3
- package/templates/persona-app/template.json +0 -13
- package/templates/persona-app/tsconfig.app.json +0 -23
- package/templates/persona-app/tsconfig.json +0 -7
- package/templates/persona-app/tsconfig.node.json +0 -21
- package/templates/persona-app/vendor/webspatial/FORK.md +0 -6
- package/templates/persona-app/vendor/webspatial/core-sdk/LICENSE +0 -21
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/iife/index.d.ts +0 -906
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/iife/index.global.js +0 -75
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/iife/index.global.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/index.d.ts +0 -906
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/index.js +0 -3131
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/index.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/core-sdk/package.json +0 -45
- package/templates/persona-app/vendor/webspatial/react-sdk/LICENSE +0 -21
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/default/index.d.ts +0 -365
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/default/index.js +0 -4167
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/default/index.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.d.ts +0 -82
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.js +0 -66
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.web.d.ts +0 -2
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.web.js +0 -18
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.web.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.d.ts +0 -5
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.js +0 -66
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.web.d.ts +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.web.js +0 -18
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.web.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/web/index.d.ts +0 -365
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/web/index.js +0 -4207
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/web/index.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/package.json +0 -94
- package/templates/persona-app/vite.config.ts +0 -31
package/README.md
CHANGED
|
@@ -1,486 +1,250 @@
|
|
|
1
1
|
# oomi-ai
|
|
2
2
|
|
|
3
|
-
Managed Oomi
|
|
3
|
+
Managed Oomi channel, bridge, voice, and persona app API tooling for OpenClaw.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
`oomi-ai` connects an OpenClaw machine to Oomi. Persona UI creation and rendering happen inside Oomi-managed systems. The OpenClaw machine uses API actions only.
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
- local bridge and pairing tooling for device-backed chat and voice
|
|
9
|
-
- XR-first WebSpatial persona scaffolding for generated Oomi work surfaces
|
|
10
|
-
- vendored AndroidXR WebSpatial fork sync and scaffold repair for managed persona runtimes
|
|
11
|
-
- managed persona launch, register, stop, delete, and queued persona-job execution flows
|
|
12
|
-
- repo-local runtime and TTS validation loops for package developers
|
|
7
|
+
The current model is:
|
|
13
8
|
|
|
14
|
-
|
|
15
|
-
-
|
|
16
|
-
-
|
|
17
|
-
|
|
18
|
-
## Current Package Focus
|
|
19
|
-
|
|
20
|
-
The current package is centered on three operational contracts:
|
|
21
|
-
|
|
22
|
-
1. managed Oomi text transport inside OpenClaw
|
|
23
|
-
2. device-backed chat and voice through the local bridge
|
|
24
|
-
3. XR-first persona runtime generation and repair for Oomi work surfaces
|
|
9
|
+
- OpenClaw talks to Oomi through managed channel and bridge APIs.
|
|
10
|
+
- Oomi clients render persona apps from approved templates, components, data bindings, actions, and permissions.
|
|
11
|
+
- The agent may inspect and update persona app state through approved Oomi API actions.
|
|
12
|
+
- The user adds persona apps from the Oomi client so platform permissions and data hydration happen in the right place.
|
|
25
13
|
|
|
26
14
|
## What This Package Ships
|
|
27
15
|
|
|
28
|
-
The npm package contains three Oomi integration surfaces:
|
|
29
|
-
|
|
30
16
|
1. OpenClaw channel extension
|
|
17
|
+
|
|
31
18
|
- File: `openclaw.extension.js`
|
|
32
|
-
- Purpose:
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
19
|
+
- Purpose: managed text transport through the Oomi backend channel API.
|
|
20
|
+
|
|
21
|
+
2. Local bridge and CLI
|
|
22
|
+
|
|
36
23
|
- Files: `bin/oomi-ai.js`, `bin/sessionBridgeState.js`
|
|
37
|
-
- Purpose: pair a device, manage the OpenClaw bridge worker, and support managed gateway traffic
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
- Files: `
|
|
42
|
-
- Purpose:
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
-
|
|
47
|
-
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
Global install:
|
|
62
|
-
|
|
63
|
-
```bash
|
|
64
|
-
pnpm add -g oomi-ai@latest
|
|
65
|
-
```
|
|
66
|
-
|
|
67
|
-
Fallback:
|
|
68
|
-
|
|
69
|
-
```bash
|
|
70
|
-
npm install -g oomi-ai@latest
|
|
71
|
-
```
|
|
72
|
-
|
|
73
|
-
Install the OpenClaw plugin:
|
|
74
|
-
|
|
75
|
-
```bash
|
|
76
|
-
openclaw plugins install oomi-ai@latest
|
|
77
|
-
```
|
|
78
|
-
|
|
79
|
-
Upgrade an existing machine:
|
|
80
|
-
|
|
81
|
-
```bash
|
|
82
|
-
pnpm add -g oomi-ai@latest
|
|
83
|
-
openclaw plugins install oomi-ai@latest
|
|
84
|
-
```
|
|
85
|
-
|
|
86
|
-
## Operator Quick Start
|
|
87
|
-
|
|
88
|
-
The packaged operator instructions live in [agent_instructions.md](./agent_instructions.md).
|
|
89
|
-
That is the primary reference for:
|
|
90
|
-
- pairing a device
|
|
91
|
-
- installing the plugin
|
|
92
|
-
- configuring `channels.oomi.accounts.default`
|
|
93
|
-
- running or supervising the bridge
|
|
94
|
-
- checking whether the system is healthy
|
|
95
|
-
- troubleshooting chat and voice failures
|
|
96
|
-
- generating or repairing XR-first managed persona apps
|
|
97
|
-
|
|
98
|
-
Fast-path install flow:
|
|
99
|
-
|
|
100
|
-
```bash
|
|
101
|
-
oomi openclaw pair --app-url https://www.oomi.ai --no-start
|
|
102
|
-
openclaw plugins install oomi-ai@latest
|
|
103
|
-
oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai
|
|
104
|
-
```
|
|
105
|
-
|
|
106
|
-
Then apply the printed `channels.oomi.accounts.default` config and restart OpenClaw.
|
|
107
|
-
|
|
108
|
-
## Configuration
|
|
109
|
-
|
|
110
|
-
OpenClaw channel config lives under:
|
|
111
|
-
|
|
112
|
-
```json
|
|
113
|
-
{
|
|
114
|
-
"channels": {
|
|
115
|
-
"oomi": {
|
|
116
|
-
"defaultAccountId": "default",
|
|
117
|
-
"accounts": {
|
|
118
|
-
"default": {
|
|
119
|
-
"backendUrl": "https://api.oomi.ai",
|
|
120
|
-
"deviceToken": "...",
|
|
121
|
-
"defaultSessionKey": "agent:main:webchat:channel:oomi",
|
|
122
|
-
"requestTimeoutMs": 15000
|
|
123
|
-
}
|
|
124
|
-
}
|
|
125
|
-
}
|
|
126
|
-
}
|
|
127
|
-
}
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
Required fields:
|
|
131
|
-
- `backendUrl`
|
|
132
|
-
- `deviceToken`
|
|
133
|
-
|
|
134
|
-
Optional fields:
|
|
135
|
-
- `defaultSessionKey`
|
|
136
|
-
- `requestTimeoutMs`
|
|
137
|
-
|
|
138
|
-
## Runtime Model
|
|
139
|
-
|
|
140
|
-
There are two runtime contracts worth understanding.
|
|
141
|
-
|
|
142
|
-
### Managed Text Chat
|
|
143
|
-
|
|
144
|
-
Managed text chat uses the OpenClaw channel extension and the Oomi backend channel API.
|
|
145
|
-
This path is the more stable contract and should be preferred when evaluating the plugin for normal chat.
|
|
146
|
-
|
|
147
|
-
### Device-Backed Chat And Voice
|
|
148
|
-
|
|
149
|
-
Device-backed chat and voice use the local bridge.
|
|
150
|
-
That bridge:
|
|
151
|
-
- keeps a broker socket open to Oomi
|
|
152
|
-
- opens local gateway sessions on demand
|
|
153
|
-
- enforces `connect`-first request ordering
|
|
154
|
-
- preserves or synthesizes `idempotencyKey` for `chat.send`
|
|
155
|
-
- keeps voice-session faults from poisoning normal provider health where possible
|
|
156
|
-
|
|
157
|
-
This is the part of the package most likely to matter when debugging voice turn failures.
|
|
158
|
-
|
|
159
|
-
For managed cloned-voice replies, the canonical contract is:
|
|
160
|
-
- visible assistant `content` stays user-facing
|
|
161
|
-
- hidden `metadata.spoken` carries the backend TTS payload
|
|
162
|
-
- the shared helper in `lib/spokenMetadata.js` is used by both the extension and the local bridge to preserve or normalize that sidecar before it reaches the backend
|
|
163
|
-
|
|
164
|
-
The backend cloned-voice path is intentionally strict. If `metadata.spoken` does not reach Oomi, backend TTS fails instead of speaking a flat fallback voice.
|
|
165
|
-
|
|
166
|
-
## Bridge Logging
|
|
167
|
-
|
|
168
|
-
The bridge is intentionally quiet by default in production so normal deploys do not spam logs with frame-level transport noise.
|
|
169
|
-
|
|
170
|
-
To enable verbose bridge tracing temporarily, set:
|
|
171
|
-
|
|
172
|
-
```bash
|
|
173
|
-
OOMI_BRIDGE_DEBUG=1
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
With that flag enabled, the bridge will emit low-level session, frame, and spoken-metadata debug logs again.
|
|
177
|
-
|
|
178
|
-
## Local TTS Validation
|
|
179
|
-
|
|
180
|
-
If you are developing this package inside the Oomi repo, you can now validate the managed voice path locally before publishing.
|
|
181
|
-
|
|
182
|
-
This local gate does three things:
|
|
183
|
-
- replays an assistant `chat.final` frame through the same spoken-metadata normalization path used by the OpenClaw extension and the bridge
|
|
184
|
-
- feeds that normalized frame into the Rails backend replay harness
|
|
185
|
-
- optionally calls the real Qwen cloned-voice provider and confirms that audio deltas come back
|
|
186
|
-
|
|
187
|
-
Important:
|
|
188
|
-
- this is a repo developer workflow, not a generic npm-only operator command
|
|
189
|
-
- it expects the Oomi repo checkout, the Rails backend, and local provider env vars
|
|
190
|
-
- the real-provider replay can auto-enroll a disposable default sample voice profile from `assets/voice/source/nemu-enrollment-sample.mp3`
|
|
191
|
-
|
|
192
|
-
Assistant-final contract only:
|
|
193
|
-
|
|
194
|
-
```bash
|
|
195
|
-
oomi openclaw debug assistant-final --text "Hey Justin! How is the testing going?" --json
|
|
196
|
-
```
|
|
197
|
-
|
|
198
|
-
Full local backend replay:
|
|
199
|
-
|
|
200
|
-
```bash
|
|
201
|
-
oomi openclaw debug tts-pipeline --text "When your voice reaches me, it gets turned into text, I read it and think about it, then I speak back through the managed chat session." --json
|
|
202
|
-
```
|
|
203
|
-
|
|
204
|
-
Real Qwen provider replay:
|
|
205
|
-
|
|
206
|
-
```bash
|
|
207
|
-
oomi openclaw debug tts-pipeline --text "When your voice reaches me, it gets turned into text, I read it and think about it, then I speak back through the managed chat session." --live-provider --env-file .env.local --provider-timeout-ms 20000 --json
|
|
208
|
-
```
|
|
209
|
-
|
|
210
|
-
What a good result looks like:
|
|
211
|
-
- `backend.success = true`
|
|
212
|
-
- `managed.assistantSpeechFinal.present = true`
|
|
213
|
-
- `qwen.errorCode = null`
|
|
214
|
-
- `qwen.audioDeltaCount > 0` when `--live-provider` is used
|
|
215
|
-
|
|
216
|
-
This is the preferred pre-publish gate for managed voice regressions, because it is much faster than publishing to npm and testing through a live OpenClaw machine first.
|
|
217
|
-
|
|
218
|
-
## Local OpenClaw Dev Harness
|
|
219
|
-
|
|
220
|
-
For plugin/runtime work, the preferred pre-publish loop is:
|
|
221
|
-
|
|
222
|
-
1. run the repo-local CLI directly from source
|
|
223
|
-
2. run the same flow inside the Dockerized OpenClaw dev harness using a local packed tarball
|
|
224
|
-
3. only then update a real OpenClaw machine
|
|
225
|
-
|
|
226
|
-
Fast source smoke from the repo checkout:
|
|
227
|
-
|
|
228
|
-
```bash
|
|
229
|
-
node packages/oomi-ai/bin/oomi-ai.js openclaw debug persona-runtime --name "Chef Dev" --json
|
|
230
|
-
```
|
|
231
|
-
|
|
232
|
-
Containerized real-runtime smoke:
|
|
233
|
-
|
|
234
|
-
```bash
|
|
235
|
-
docker compose -f docker/openclaw-dev/compose.yml build openclaw-dev
|
|
236
|
-
docker compose -f docker/openclaw-dev/compose.yml up -d openclaw-dev
|
|
237
|
-
docker compose -f docker/openclaw-dev/compose.yml exec -T openclaw-dev openclaw gateway health --url ws://127.0.0.1:18789 --token dev-gateway-token --json
|
|
238
|
-
docker compose -f docker/openclaw-dev/compose.yml exec -T openclaw-dev oomi-local openclaw debug persona-runtime --name "Chef Dev" --json
|
|
24
|
+
- Purpose: pair a device, manage the OpenClaw bridge worker, and support managed gateway traffic for device-backed chat and voice.
|
|
25
|
+
|
|
26
|
+
3. Persona app API tools
|
|
27
|
+
|
|
28
|
+
- Files: `persona-app/*`, `lib/personaApiClient.js`, `bin/oomi-ai.js`
|
|
29
|
+
- Purpose: inspect account-linked component-composed persona apps and apply approved backend actions.
|
|
30
|
+
|
|
31
|
+
4. Agent/operator instructions
|
|
32
|
+
|
|
33
|
+
- Files: `agent_instructions.md`, `skills/oomi/*`
|
|
34
|
+
- Purpose: tell OpenClaw agents how to connect, repair, and use Oomi without local UI code generation.
|
|
35
|
+
|
|
36
|
+
## Boundary
|
|
37
|
+
|
|
38
|
+
Persona UI is an Oomi-managed client capability, not an OpenClaw-machine responsibility.
|
|
39
|
+
|
|
40
|
+
This package exposes connection, bridge, voice, and persona app state APIs. It does not include a local persona app workspace, local persona runtime manager, queued UI job runner, or app framework bundle.
|
|
41
|
+
|
|
42
|
+
## Install And Upgrade
|
|
43
|
+
|
|
44
|
+
Global install:
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
pnpm add -g oomi-ai@latest
|
|
239
48
|
```
|
|
240
49
|
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
`oomi-local` is a deterministic container wrapper that executes the installed packed `oomi-ai` artifact directly with Node. In the Docker harness, it is only the package wrapper. The agent itself is the real OpenClaw runtime running in the foreground.
|
|
244
|
-
|
|
245
|
-
Shared profile contract smoke:
|
|
246
|
-
|
|
247
|
-
```bash
|
|
248
|
-
node packages/oomi-ai/bin/oomi-ai.js openclaw profile init --profile-id oomi-dev-local --label "Oomi Local Dev" --backend-url http://127.0.0.1:3001 --device-token dev-device-token --json
|
|
249
|
-
node packages/oomi-ai/bin/oomi-ai.js openclaw profile apply --profile ~/.openclaw/oomi-openclaw-profile.json --openclaw-home ~/.openclaw --json
|
|
250
|
-
```
|
|
251
|
-
|
|
252
|
-
What the harness does:
|
|
253
|
-
|
|
254
|
-
- bootstraps an isolated OpenClaw home rooted at `HOME/.openclaw`
|
|
255
|
-
- runs `openclaw onboard --non-interactive ...`
|
|
256
|
-
- writes and applies `HOME/.openclaw/oomi-dev-profile.json` using the same shared profile contract the future onboarding UI and hosted-agent bootstrap should use
|
|
257
|
-
- enables the Oomi channel account through that applied profile and relies on local OpenClaw plugin auto-discovery for the installed `oomi-ai` plugin
|
|
258
|
-
- writes device identity material used by the `oomi-ai` bridge tooling
|
|
259
|
-
- packs the local `packages/oomi-ai` checkout into a `.tgz`
|
|
260
|
-
- installs that tarball globally in the container
|
|
261
|
-
- installs the same tarball as a real OpenClaw plugin
|
|
262
|
-
- defaults model auth to `oomi-managed` so onboarding/bootstrap does not require end-user provider keys
|
|
263
|
-
- runs `openclaw gateway` as the foreground container process
|
|
264
|
-
|
|
265
|
-
Useful env overrides for local integration:
|
|
266
|
-
|
|
267
|
-
- `OOMI_DEV_BACKEND_URL`
|
|
268
|
-
- `OOMI_DEV_DEVICE_TOKEN`
|
|
269
|
-
- `OOMI_DEV_MODEL_AUTH_MODE`
|
|
270
|
-
- `OPENCLAW_GATEWAY_TOKEN`
|
|
271
|
-
- `OPENCLAW_GATEWAY_PASSWORD`
|
|
272
|
-
|
|
273
|
-
Recommended local modes:
|
|
274
|
-
|
|
275
|
-
- onboarding/runtime checks without provider keys
|
|
276
|
-
- `OOMI_DEV_MODEL_AUTH_MODE=oomi-managed`
|
|
277
|
-
- internal real-response smoke before publish
|
|
278
|
-
- `OPENROUTER_API_KEY=...`
|
|
279
|
-
- optional explicit override: `OOMI_DEV_MODEL_AUTH_MODE=provider-env`
|
|
280
|
-
|
|
281
|
-
The default container config is intentionally safe for onboarding and runtime testing. It does not require a published npm version, and it does not require end-user provider keys.
|
|
282
|
-
|
|
283
|
-
To make the Dockerized OpenClaw runtime actually answer managed chat locally today, add this to the repo `.env.local`:
|
|
50
|
+
Fallback:
|
|
284
51
|
|
|
285
52
|
```bash
|
|
286
|
-
|
|
287
|
-
OPENROUTER_API_KEY=<your-openrouter-key>
|
|
53
|
+
npm install -g oomi-ai@latest
|
|
288
54
|
```
|
|
289
55
|
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
|
|
293
|
-
|
|
294
|
-
Use the scaffold flow when OpenClaw needs to build a managed persona app that will live inside Oomi:
|
|
295
|
-
|
|
296
|
-
```bash
|
|
297
|
-
oomi personas scaffold market-analyst --name "Market Analyst" --description "Private app for reviewing my broker positions and risk." --out ~/.openclaw/workspace/personas/market-analyst
|
|
298
|
-
```
|
|
299
|
-
|
|
300
|
-
Use:
|
|
301
|
-
- `oomi personas create <id>` only for repo-local manifest work, not for the normal managed Oomi persona flow
|
|
302
|
-
- `oomi personas create-managed --name "Cooking Persona" --description "Private cooking workspace"` for the end-to-end Oomi-managed persona flow
|
|
303
|
-
- `oomi personas scaffold <slug>` for an XR-first WebSpatial Oomi app shell with runtime metadata and health documents
|
|
304
|
-
- `oomi persona-jobs execute --message-file <job.json>` when OpenClaw receives a structured persona orchestration job from Oomi
|
|
305
|
-
|
|
306
|
-
Additional persona runtime commands:
|
|
307
|
-
|
|
308
|
-
```bash
|
|
309
|
-
oomi personas launch-managed market-analyst --name "Market Analyst" --description "Private app for reviewing my broker positions and risk."
|
|
310
|
-
oomi personas status market-analyst
|
|
311
|
-
oomi personas stop market-analyst
|
|
312
|
-
oomi personas delete market-analyst
|
|
313
|
-
oomi personas runtime-register market-analyst --local-port 4789
|
|
314
|
-
oomi personas heartbeat market-analyst --local-port 4789
|
|
315
|
-
oomi persona-jobs start pj_123
|
|
316
|
-
oomi persona-jobs succeed pj_123 --workspace-path ~/.openclaw/workspace/personas/market-analyst --local-port 4789
|
|
317
|
-
oomi persona-jobs fail pj_123 --code JOB_FAILED --message "Scaffold generation failed."
|
|
56
|
+
Install or update the OpenClaw plugin:
|
|
57
|
+
|
|
58
|
+
```bash
|
|
59
|
+
openclaw plugins install oomi-ai@latest
|
|
318
60
|
```
|
|
319
61
|
|
|
320
|
-
|
|
321
|
-
|
|
322
|
-
|
|
323
|
-
|
|
324
|
-
|
|
325
|
-
|
|
326
|
-
|
|
327
|
-
|
|
328
|
-
|
|
329
|
-
|
|
330
|
-
|
|
331
|
-
oomi
|
|
332
|
-
```
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
337
|
-
|
|
338
|
-
|
|
339
|
-
|
|
340
|
-
|
|
341
|
-
|
|
342
|
-
|
|
343
|
-
|
|
344
|
-
-
|
|
345
|
-
-
|
|
346
|
-
|
|
347
|
-
|
|
348
|
-
|
|
349
|
-
|
|
350
|
-
|
|
351
|
-
|
|
62
|
+
After upgrading, refresh the running bridge so stale gateway connections do not keep using the previous package:
|
|
63
|
+
|
|
64
|
+
```bash
|
|
65
|
+
oomi openclaw refresh --skip-version-check
|
|
66
|
+
oomi openclaw bridge ps
|
|
67
|
+
```
|
|
68
|
+
|
|
69
|
+
If the bridge still looks stale, restart it explicitly:
|
|
70
|
+
|
|
71
|
+
```bash
|
|
72
|
+
oomi openclaw bridge restart --detach
|
|
73
|
+
oomi openclaw bridge ps
|
|
74
|
+
```
|
|
75
|
+
|
|
76
|
+
On macOS supervised installs:
|
|
77
|
+
|
|
78
|
+
```bash
|
|
79
|
+
oomi openclaw bridge service restart
|
|
80
|
+
oomi openclaw bridge service status
|
|
81
|
+
```
|
|
82
|
+
|
|
83
|
+
Healthy update expectation:
|
|
84
|
+
|
|
85
|
+
- exactly one bridge worker is active
|
|
86
|
+
- the worker reports the newly installed package version
|
|
87
|
+
- bridge status reaches `connected`
|
|
88
|
+
|
|
89
|
+
## Standard Connect Flow
|
|
90
|
+
|
|
91
|
+
Pair the machine:
|
|
92
|
+
|
|
93
|
+
```bash
|
|
94
|
+
oomi openclaw pair --app-url https://www.oomi.ai --no-start
|
|
95
|
+
```
|
|
96
|
+
|
|
97
|
+
Print the OpenClaw config:
|
|
98
|
+
|
|
99
|
+
```bash
|
|
100
|
+
oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai
|
|
101
|
+
```
|
|
102
|
+
|
|
103
|
+
Apply the printed `channels.oomi.accounts.default` config and restart OpenClaw.
|
|
104
|
+
|
|
105
|
+
Start or repair the bridge:
|
|
106
|
+
|
|
107
|
+
```bash
|
|
108
|
+
oomi openclaw bridge ensure --detach
|
|
109
|
+
```
|
|
110
|
+
|
|
111
|
+
## Configuration
|
|
112
|
+
|
|
113
|
+
OpenClaw channel config lives under:
|
|
114
|
+
|
|
115
|
+
```json
|
|
116
|
+
{
|
|
117
|
+
"channels": {
|
|
118
|
+
"oomi": {
|
|
119
|
+
"defaultAccountId": "default",
|
|
120
|
+
"accounts": {
|
|
121
|
+
"default": {
|
|
122
|
+
"backendUrl": "https://api.oomi.ai",
|
|
123
|
+
"deviceToken": "...",
|
|
124
|
+
"defaultSessionKey": "agent:main:webchat:channel:oomi",
|
|
125
|
+
"requestTimeoutMs": 15000
|
|
126
|
+
}
|
|
127
|
+
}
|
|
128
|
+
}
|
|
129
|
+
}
|
|
130
|
+
}
|
|
131
|
+
```
|
|
132
|
+
|
|
133
|
+
Required fields:
|
|
134
|
+
|
|
135
|
+
- `backendUrl`
|
|
136
|
+
- `deviceToken`
|
|
137
|
+
|
|
138
|
+
Optional fields:
|
|
139
|
+
|
|
140
|
+
- `defaultSessionKey`
|
|
141
|
+
- `requestTimeoutMs`
|
|
142
|
+
|
|
143
|
+
## Persona App API Tools
|
|
144
|
+
|
|
145
|
+
Component-composed persona apps are the default Oomi mini-app path. Use these commands when the user asks about or updates an existing Oomi persona app such as Fitness Today:
|
|
146
|
+
|
|
147
|
+
```bash
|
|
148
|
+
oomi persona-apps list --json
|
|
149
|
+
oomi persona-apps show fitness-today --json
|
|
150
|
+
oomi persona-apps apply-action fitness-today --action fitness.complete_workout --payload-json '{"goalId":"easy-run","minutes":20}' --json
|
|
151
|
+
```
|
|
152
|
+
|
|
153
|
+
Rules for agents:
|
|
154
|
+
|
|
155
|
+
- Answer from returned `appState.bindings`, not from memory.
|
|
156
|
+
- Apply updates only through approved `persona-apps apply-action` actions.
|
|
157
|
+
- If the persona app does not exist, tell the user to add it from the Oomi client first.
|
|
158
|
+
- Do not create local persona UI projects.
|
|
159
|
+
|
|
160
|
+
For the Fitness v0 slice:
|
|
161
|
+
|
|
162
|
+
```bash
|
|
163
|
+
oomi persona-apps apply-action fitness-today --action fitness.complete_goal --payload-json '{"goalId":"easy-run"}' --json
|
|
164
|
+
oomi persona-apps apply-action fitness-today --action fitness.complete_workout --payload-json '{"goalId":"easy-run","minutes":20}' --json
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
## Voice Contract
|
|
168
|
+
|
|
169
|
+
Managed voice uses the same Oomi plugin and bridge layer as managed chat.
|
|
170
|
+
|
|
171
|
+
For assistant replies, visible chat text stays user-facing. Optional hidden TTS metadata may live at `metadata.spoken`:
|
|
172
|
+
|
|
173
|
+
```json
|
|
174
|
+
{
|
|
175
|
+
"metadata": {
|
|
176
|
+
"spoken": {
|
|
177
|
+
"text": "Speech-optimized text for TTS only.",
|
|
178
|
+
"language": "English",
|
|
179
|
+
"instructions": "Speak with warm, upbeat conversational energy and natural pacing."
|
|
180
|
+
}
|
|
181
|
+
}
|
|
182
|
+
}
|
|
183
|
+
```
|
|
184
|
+
|
|
185
|
+
Rules:
|
|
186
|
+
|
|
187
|
+
- visible `content` is the source of truth for chat rendering
|
|
188
|
+
- `metadata.spoken.text` is backend TTS input only
|
|
189
|
+
- visible chat text should not contain raw speaking tags
|
|
190
|
+
- the package preserves valid `metadata.spoken`
|
|
191
|
+
- if omitted, the package may synthesize a bounded fallback from visible assistant text
|
|
192
|
+
|
|
193
|
+
## Health Checks
|
|
194
|
+
|
|
195
|
+
Use these when chat or voice is failing:
|
|
196
|
+
|
|
197
|
+
```bash
|
|
198
|
+
oomi openclaw bridge ps
|
|
199
|
+
oomi openclaw bridge service status
|
|
200
|
+
oomi openclaw status
|
|
201
|
+
tail -f ~/.openclaw/logs/oomi-bridge-live.log
|
|
202
|
+
tail -f ~/.openclaw/logs/gateway.log
|
|
203
|
+
tail -f ~/.openclaw/logs/gateway.err.log
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
Bridge status meanings:
|
|
207
|
+
|
|
208
|
+
- `starting`: bridge booting or waiting for managed subscription
|
|
209
|
+
- `connected`: ready for managed chat and voice traffic
|
|
210
|
+
- `reconnecting`: transport dropped and retry is scheduled
|
|
211
|
+
- `degraded`: bridge caught a runtime fault but is still alive
|
|
212
|
+
- `error`: startup or auth failure blocked useful operation
|
|
213
|
+
- `stopped`: not running or intentionally stopped
|
|
214
|
+
|
|
215
|
+
## Troubleshooting
|
|
216
|
+
|
|
217
|
+
Duplicate plugin id warning:
|
|
218
|
+
|
|
219
|
+
- Cause: multiple discoverable `oomi-ai` installs.
|
|
220
|
+
- Action: remove stale extension copies and reinstall once.
|
|
221
|
+
|
|
222
|
+
`invalid handshake: first request must be connect`:
|
|
223
|
+
|
|
224
|
+
- Cause: a gateway request was sent before `connect` had been accepted.
|
|
225
|
+
- Action: update `oomi-ai`, restart the bridge, and confirm only one bridge worker is running.
|
|
226
|
+
|
|
227
|
+
STT works but the assistant does not reply:
|
|
228
|
+
|
|
229
|
+
- Cause: the voice turn likely reached Oomi, but the managed gateway or OpenClaw run failed later.
|
|
230
|
+
- Action: inspect `gateway.log`, `gateway.err.log`, and the session JSONL for auth, network, or bridge restart errors.
|
|
231
|
+
|
|
232
|
+
Bridge keeps using an old package after update:
|
|
233
|
+
|
|
234
|
+
- Action: run `oomi openclaw refresh --skip-version-check`.
|
|
235
|
+
- If still stale: run `oomi openclaw bridge restart --detach`.
|
|
236
|
+
- On macOS service installs: run `oomi openclaw bridge service restart`.
|
|
237
|
+
- Confirm with `oomi openclaw bridge ps`.
|
|
238
|
+
|
|
239
|
+
## Developer Verification
|
|
240
|
+
|
|
241
|
+
From `packages/oomi-ai`:
|
|
352
242
|
|
|
353
243
|
```bash
|
|
354
|
-
|
|
244
|
+
npm run check
|
|
245
|
+
npm test
|
|
246
|
+
npm pack --dry-run
|
|
247
|
+
npm publish --dry-run --no-git-checks --access public
|
|
355
248
|
```
|
|
356
249
|
|
|
357
|
-
|
|
358
|
-
|
|
359
|
-
## Bridge Health States
|
|
360
|
-
|
|
361
|
-
The bridge status file is written locally and should roughly be interpreted as:
|
|
362
|
-
- `starting`: process booting or waiting for managed subscription
|
|
363
|
-
- `connected`: broker connected and managed subscription confirmed
|
|
364
|
-
- `reconnecting`: broker or gateway transport dropped and reconnect is scheduled
|
|
365
|
-
- `degraded`: bridge is still alive but hit a runtime fault that needs attention
|
|
366
|
-
- `error`: startup or auth-level failure that prevents useful operation
|
|
367
|
-
- `stopped`: bridge is not running or was intentionally stopped
|
|
368
|
-
|
|
369
|
-
For voice support, a `voice_session_*` failure should be treated as narrower than a full provider outage.
|
|
370
|
-
|
|
371
|
-
## Troubleshooting
|
|
372
|
-
|
|
373
|
-
### `invalid handshake: first request must be connect`
|
|
374
|
-
|
|
375
|
-
Meaning:
|
|
376
|
-
- a gateway request was forwarded before the session had accepted `connect`
|
|
377
|
-
|
|
378
|
-
What to check:
|
|
379
|
-
- update to the latest `oomi-ai`
|
|
380
|
-
- restart the bridge worker
|
|
381
|
-
- confirm only one active bridge worker exists for the device
|
|
382
|
-
|
|
383
|
-
### `duplicate plugin id detected`
|
|
384
|
-
|
|
385
|
-
Meaning:
|
|
386
|
-
- OpenClaw can see more than one `oomi-ai` plugin source
|
|
387
|
-
|
|
388
|
-
What to check:
|
|
389
|
-
- ensure there is only one active install under OpenClaw plugin discovery paths
|
|
390
|
-
- remove stale local extension copies before reinstalling
|
|
391
|
-
|
|
392
|
-
### Bridge keeps flipping between `reconnecting`, `degraded`, or `stopped`
|
|
393
|
-
|
|
394
|
-
What to check:
|
|
395
|
-
- `oomi openclaw bridge ps`
|
|
396
|
-
- `oomi openclaw bridge service status`
|
|
397
|
-
- `tail -f ~/.openclaw/logs/oomi-bridge-live.log`
|
|
398
|
-
- `tail -f ~/.openclaw/logs/gateway.log`
|
|
399
|
-
|
|
400
|
-
If the process is alive but runtime faults are being caught, expect `degraded` rather than an immediate hard stop.
|
|
401
|
-
|
|
402
|
-
### Voice STT works but the agent does not answer
|
|
403
|
-
|
|
404
|
-
This usually means one of these:
|
|
405
|
-
- the managed gateway/device side is not actually ready
|
|
406
|
-
- the bridge or agent run failed after delivery
|
|
407
|
-
- the OpenClaw run stopped with an upstream provider `network_error`
|
|
408
|
-
|
|
409
|
-
In that situation, inspect:
|
|
410
|
-
- `~/.openclaw/logs/gateway.log`
|
|
411
|
-
- `~/.openclaw/logs/gateway.err.log`
|
|
412
|
-
- the relevant session JSONL in `~/.openclaw/agents/main/sessions/`
|
|
413
|
-
|
|
414
|
-
### Voice text works but cloned TTS fails with `MISSING_SPOKEN_METADATA`
|
|
415
|
-
|
|
416
|
-
Meaning:
|
|
417
|
-
- the assistant text arrived
|
|
418
|
-
- the backend voice relay never received valid hidden `metadata.spoken`
|
|
419
|
-
|
|
420
|
-
What to check:
|
|
421
|
-
- run the local replay gate before publishing:
|
|
422
|
-
- `oomi openclaw debug assistant-final --text "..."`
|
|
423
|
-
- `oomi openclaw debug tts-pipeline --text "..."`
|
|
424
|
-
- if the package local replay succeeds but the live machine fails, verify the OpenClaw machine is actually running the updated bridge binary
|
|
425
|
-
- if the local replay fails, fix the assistant-final contract first instead of debugging the browser or backend deployment
|
|
426
|
-
|
|
427
|
-
## Developer Notes
|
|
428
|
-
|
|
429
|
-
If you are inspecting this package on npm, the main architectural points are:
|
|
430
|
-
- the extension path is the stable managed text contract
|
|
431
|
-
- the local bridge exists because Oomi also needs device-backed and voice-capable flows
|
|
432
|
-
- the persona scaffold exists because Oomi specialist personas can surface as scoped XR/web work surfaces, not just chat tabs
|
|
433
|
-
- the bridge has been hardened for:
|
|
434
|
-
- strict `connect`-first forwarding
|
|
435
|
-
- method-specific request shaping
|
|
436
|
-
- `idempotencyKey` handling
|
|
437
|
-
- bridge status that does not report `connected` before managed subscription is ready
|
|
438
|
-
- runtime fault isolation so local session failures are less likely to crash the whole provider
|
|
439
|
-
- one shared hidden managed-voice speech metadata helper used by both the extension and the local bridge
|
|
440
|
-
|
|
441
|
-
For generated WebSpatial persona apps, the intended scaffold contract is:
|
|
442
|
-
- XR mode defaults directly into the mounted scene route
|
|
443
|
-
- `configurePersonaScene()` runs from that scene component
|
|
444
|
-
- runtime diagnostics are logged on scene boot
|
|
445
|
-
- multiple meaningful UI surfaces use `enable-xr` and `xrStyle()`
|
|
446
|
-
- `html.is-spatial` keeps the host shell transparent
|
|
447
|
-
|
|
448
|
-
If you are developing the plugin, test the packaged surface with:
|
|
449
|
-
|
|
450
|
-
```bash
|
|
451
|
-
cd packages/oomi-ai
|
|
452
|
-
node --test test/*.test.mjs
|
|
453
|
-
npm pack --dry-run
|
|
454
|
-
```
|
|
455
|
-
|
|
456
|
-
For managed voice changes, do not stop at the package tests. Run the local replay gate from the repo root as well, especially before publishing:
|
|
457
|
-
|
|
458
|
-
```bash
|
|
459
|
-
oomi openclaw debug tts-pipeline --text "Local managed voice validation text." --json
|
|
460
|
-
oomi openclaw debug tts-pipeline --text "Local managed voice validation text." --live-provider --env-file .env.local --provider-timeout-ms 20000 --json
|
|
461
|
-
```
|
|
462
|
-
|
|
463
|
-
## Release Process
|
|
464
|
-
|
|
465
|
-
Before publishing:
|
|
466
|
-
|
|
467
|
-
```bash
|
|
468
|
-
cd packages/oomi-ai
|
|
469
|
-
node --test test/*.test.mjs
|
|
470
|
-
npm pack --dry-run
|
|
471
|
-
```
|
|
472
|
-
|
|
473
|
-
For voice-related changes, also run the repo-backed local replay gate before publish:
|
|
474
|
-
|
|
475
|
-
```bash
|
|
476
|
-
oomi openclaw debug tts-pipeline --text "Local managed voice validation text." --json
|
|
477
|
-
oomi openclaw debug tts-pipeline --text "Local managed voice validation text." --live-provider --env-file .env.local --provider-timeout-ms 20000 --json
|
|
478
|
-
```
|
|
479
|
-
|
|
480
|
-
Then publish the bumped version:
|
|
481
|
-
|
|
482
|
-
```bash
|
|
483
|
-
pnpm check
|
|
484
|
-
pnpm publish --dry-run --no-git-checks --access public
|
|
485
|
-
pnpm publish --access public
|
|
486
|
-
```
|
|
250
|
+
Expected package contents are restricted by the `files` allowlist in `package.json`.
|