oomi-ai 0.2.50 → 0.3.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +224 -507
- package/agent_instructions.md +264 -253
- package/bin/oomi-ai.js +4036 -5791
- package/bin/sessionBridgeState.js +78 -78
- package/lib/openclawPaths.js +70 -71
- package/lib/openclawProfile.js +216 -216
- package/lib/personaApiClient.js +142 -303
- package/lib/spokenMetadata.js +137 -137
- package/openclaw.extension.js +341 -341
- package/openclaw.plugin.json +17 -17
- package/package.json +59 -59
- package/persona-app/README.md +33 -0
- package/persona-app/registry/v1.json +85 -0
- package/persona-app/schema/persona-app.v1.schema.json +90 -0
- package/skills/oomi/SKILL.md +182 -182
- package/skills/oomi/agent_instructions.md +117 -80
- package/lib/channelPluginClient.js +0 -119
- package/lib/openclawDevGateway.js +0 -384
- package/lib/personaJobExecutor.js +0 -139
- package/lib/personaJobPoller.js +0 -112
- package/lib/personaPortAllocator.js +0 -36
- package/lib/personaRuntimeManager.js +0 -496
- package/lib/personaRuntimeProcess.js +0 -924
- package/lib/personaRuntimeRegistry.js +0 -67
- package/lib/personaRuntimeSupervisor.js +0 -330
- package/lib/scaffold.js +0 -108
- package/lib/template.js +0 -45
- package/skills/oomi/config.json +0 -3
- package/skills/oomi/scripts/get_avatar_capabilities.py +0 -40
- package/skills/oomi/scripts/get_data.py +0 -49
- package/skills/oomi/scripts/install_agent_instructions.py +0 -78
- package/skills/oomi/scripts/send_goal.py +0 -53
- package/skills/oomi/scripts/sync.py +0 -46
- package/skills/oomi/setup.py +0 -41
- package/templates/persona-app/.env.example +0 -8
- package/templates/persona-app/README.md +0 -58
- package/templates/persona-app/eslint.config.js +0 -28
- package/templates/persona-app/index.html +0 -18
- package/templates/persona-app/oomi.runtime.json +0 -13
- package/templates/persona-app/package.json +0 -44
- package/templates/persona-app/persona/brief.md +0 -14
- package/templates/persona-app/persona.json +0 -14
- package/templates/persona-app/public/manifest.webmanifest +0 -8
- package/templates/persona-app/public/oomi.health.json +0 -6
- package/templates/persona-app/src/App.css +0 -379
- package/templates/persona-app/src/App.tsx +0 -17
- package/templates/persona-app/src/index.css +0 -53
- package/templates/persona-app/src/main.tsx +0 -23
- package/templates/persona-app/src/pages/HomePage.tsx +0 -127
- package/templates/persona-app/src/pages/ScenePage.tsx +0 -158
- package/templates/persona-app/src/persona/config.ts +0 -6
- package/templates/persona-app/src/persona/notes.ts +0 -10
- package/templates/persona-app/src/spatial.ts +0 -82
- package/templates/persona-app/src/vite-env.d.ts +0 -3
- package/templates/persona-app/template.json +0 -13
- package/templates/persona-app/tsconfig.app.json +0 -23
- package/templates/persona-app/tsconfig.json +0 -7
- package/templates/persona-app/tsconfig.node.json +0 -21
- package/templates/persona-app/vendor/webspatial/FORK.md +0 -6
- package/templates/persona-app/vendor/webspatial/core-sdk/LICENSE +0 -21
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/iife/index.d.ts +0 -906
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/iife/index.global.js +0 -75
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/iife/index.global.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/index.d.ts +0 -906
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/index.js +0 -3131
- package/templates/persona-app/vendor/webspatial/core-sdk/dist/index.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/core-sdk/package.json +0 -45
- package/templates/persona-app/vendor/webspatial/react-sdk/LICENSE +0 -21
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/default/index.d.ts +0 -365
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/default/index.js +0 -4167
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/default/index.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.d.ts +0 -82
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.js +0 -66
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.web.d.ts +0 -2
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.web.js +0 -18
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-dev-runtime.web.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.d.ts +0 -5
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.js +0 -66
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.web.d.ts +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.web.js +0 -18
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/jsx/jsx-runtime.web.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/web/index.d.ts +0 -365
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/web/index.js +0 -4207
- package/templates/persona-app/vendor/webspatial/react-sdk/dist/web/index.js.map +0 -1
- package/templates/persona-app/vendor/webspatial/react-sdk/package.json +0 -94
- package/templates/persona-app/vite.config.ts +0 -31
package/README.md
CHANGED
|
@@ -1,554 +1,271 @@
|
|
|
1
1
|
# oomi-ai
|
|
2
2
|
|
|
3
|
-
Managed Oomi
|
|
3
|
+
Managed Oomi channel, bridge, voice, and persona app API tooling for OpenClaw.
|
|
4
4
|
|
|
5
|
-
|
|
5
|
+
`oomi-ai` connects an OpenClaw machine to Oomi. Persona UI creation and rendering happen inside Oomi-managed systems. The OpenClaw machine uses API actions only.
|
|
6
6
|
|
|
7
|
-
|
|
8
|
-
- local bridge and pairing tooling for device-backed chat and voice
|
|
9
|
-
- XR-first WebSpatial persona scaffolding for generated Oomi work surfaces
|
|
10
|
-
- vendored AndroidXR WebSpatial fork sync and scaffold repair for managed persona runtimes
|
|
11
|
-
- managed persona launch, register, stop, delete, and queued persona-job execution flows
|
|
12
|
-
- repo-local runtime and TTS validation loops for package developers
|
|
7
|
+
The current model is:
|
|
13
8
|
|
|
14
|
-
|
|
15
|
-
-
|
|
16
|
-
-
|
|
17
|
-
|
|
18
|
-
## Current Package Focus
|
|
19
|
-
|
|
20
|
-
The current package is centered on three operational contracts:
|
|
21
|
-
|
|
22
|
-
1. managed Oomi text transport inside OpenClaw
|
|
23
|
-
2. device-backed chat and voice through the local bridge
|
|
24
|
-
3. XR-first persona runtime generation and repair for Oomi work surfaces
|
|
9
|
+
- OpenClaw talks to Oomi through managed channel and bridge APIs.
|
|
10
|
+
- Oomi clients render persona apps from approved templates, components, data bindings, actions, and permissions.
|
|
11
|
+
- The agent may inspect and update persona app state through approved Oomi API actions.
|
|
12
|
+
- The user adds persona apps from the Oomi client so platform permissions and data hydration happen in the right place.
|
|
25
13
|
|
|
26
14
|
## What This Package Ships
|
|
27
15
|
|
|
28
|
-
The npm package contains three Oomi integration surfaces:
|
|
29
|
-
|
|
30
16
|
1. OpenClaw channel extension
|
|
17
|
+
|
|
31
18
|
- File: `openclaw.extension.js`
|
|
32
|
-
- Purpose:
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
19
|
+
- Purpose: managed text transport through the Oomi backend channel API.
|
|
20
|
+
|
|
21
|
+
2. Local bridge and CLI
|
|
22
|
+
|
|
36
23
|
- Files: `bin/oomi-ai.js`, `bin/sessionBridgeState.js`
|
|
37
|
-
- Purpose: pair a device, manage the OpenClaw bridge worker, and support managed gateway traffic
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
- Files: `
|
|
42
|
-
- Purpose:
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
-
|
|
47
|
-
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
|
|
54
|
-
|
|
55
|
-
|
|
56
|
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
openclaw
|
|
84
|
-
|
|
85
|
-
|
|
86
|
-
|
|
87
|
-
|
|
88
|
-
|
|
89
|
-
|
|
90
|
-
|
|
91
|
-
|
|
92
|
-
|
|
93
|
-
|
|
94
|
-
|
|
95
|
-
|
|
96
|
-
|
|
97
|
-
|
|
98
|
-
|
|
99
|
-
|
|
100
|
-
```bash
|
|
101
|
-
oomi openclaw pair --app-url https://www.oomi.ai --no-start
|
|
102
|
-
openclaw plugins install oomi-ai@latest
|
|
103
|
-
oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai
|
|
104
|
-
```
|
|
105
|
-
|
|
106
|
-
Then apply the printed `channels.oomi.accounts.default` config and restart OpenClaw.
|
|
107
|
-
|
|
108
|
-
## Configuration
|
|
109
|
-
|
|
110
|
-
OpenClaw channel config lives under:
|
|
111
|
-
|
|
112
|
-
```json
|
|
113
|
-
{
|
|
114
|
-
"channels": {
|
|
115
|
-
"oomi": {
|
|
116
|
-
"defaultAccountId": "default",
|
|
117
|
-
"accounts": {
|
|
118
|
-
"default": {
|
|
119
|
-
"backendUrl": "https://api.oomi.ai",
|
|
120
|
-
"deviceToken": "...",
|
|
121
|
-
"defaultSessionKey": "agent:main:webchat:channel:oomi",
|
|
122
|
-
"requestTimeoutMs": 15000
|
|
123
|
-
}
|
|
124
|
-
}
|
|
125
|
-
}
|
|
126
|
-
}
|
|
127
|
-
}
|
|
128
|
-
```
|
|
129
|
-
|
|
130
|
-
Required fields:
|
|
131
|
-
- `backendUrl`
|
|
132
|
-
- `deviceToken`
|
|
133
|
-
|
|
134
|
-
Optional fields:
|
|
135
|
-
- `defaultSessionKey`
|
|
136
|
-
- `requestTimeoutMs`
|
|
137
|
-
|
|
138
|
-
## Runtime Model
|
|
139
|
-
|
|
140
|
-
There are two runtime contracts worth understanding.
|
|
141
|
-
|
|
142
|
-
### Managed Text Chat
|
|
143
|
-
|
|
144
|
-
Managed text chat uses the OpenClaw channel extension and the Oomi backend channel API.
|
|
145
|
-
This path is the more stable contract and should be preferred when evaluating the plugin for normal chat.
|
|
146
|
-
|
|
147
|
-
### Device-Backed Chat And Voice
|
|
148
|
-
|
|
149
|
-
Device-backed chat and voice use the local bridge.
|
|
150
|
-
That bridge:
|
|
151
|
-
- keeps a broker socket open to Oomi
|
|
152
|
-
- opens local gateway sessions on demand
|
|
153
|
-
- enforces `connect`-first request ordering
|
|
154
|
-
- preserves or synthesizes `idempotencyKey` for `chat.send`
|
|
155
|
-
- keeps voice-session faults from poisoning normal provider health where possible
|
|
156
|
-
|
|
157
|
-
This is the part of the package most likely to matter when debugging voice turn failures.
|
|
158
|
-
|
|
159
|
-
For managed cloned-voice replies, the canonical contract is:
|
|
160
|
-
- visible assistant `content` stays user-facing
|
|
161
|
-
- hidden `metadata.spoken` carries the backend TTS payload
|
|
162
|
-
- the shared helper in `lib/spokenMetadata.js` is used by both the extension and the local bridge to preserve or normalize that sidecar before it reaches the backend
|
|
163
|
-
|
|
164
|
-
The backend cloned-voice path is intentionally strict. If `metadata.spoken` does not reach Oomi, backend TTS fails instead of speaking a flat fallback voice.
|
|
165
|
-
|
|
166
|
-
## Bridge Logging
|
|
167
|
-
|
|
168
|
-
The bridge is intentionally quiet by default in production so normal deploys do not spam logs with frame-level transport noise.
|
|
169
|
-
|
|
170
|
-
To enable verbose bridge tracing temporarily, set:
|
|
171
|
-
|
|
172
|
-
```bash
|
|
173
|
-
OOMI_BRIDGE_DEBUG=1
|
|
174
|
-
```
|
|
175
|
-
|
|
176
|
-
With that flag enabled, the bridge will emit low-level session, frame, and spoken-metadata debug logs again.
|
|
177
|
-
|
|
178
|
-
## Local TTS Validation
|
|
179
|
-
|
|
180
|
-
If you are developing this package inside the Oomi repo, you can now validate the managed voice path locally before publishing.
|
|
181
|
-
|
|
182
|
-
This local gate does three things:
|
|
183
|
-
- replays an assistant `chat.final` frame through the same spoken-metadata normalization path used by the OpenClaw extension and the bridge
|
|
184
|
-
- feeds that normalized frame into the Rails backend replay harness
|
|
185
|
-
- optionally calls the real Qwen cloned-voice provider and confirms that audio deltas come back
|
|
186
|
-
|
|
187
|
-
Important:
|
|
188
|
-
- this is a repo developer workflow, not a generic npm-only operator command
|
|
189
|
-
- it expects the Oomi repo checkout, the Rails backend, and local provider env vars
|
|
190
|
-
- the real-provider replay can auto-enroll a disposable default sample voice profile from `assets/voice/source/nemu-enrollment-sample.mp3`
|
|
191
|
-
|
|
192
|
-
Assistant-final contract only:
|
|
193
|
-
|
|
194
|
-
```bash
|
|
195
|
-
oomi openclaw debug assistant-final --text "Hey Justin! How is the testing going?" --json
|
|
196
|
-
```
|
|
197
|
-
|
|
198
|
-
Full local backend replay:
|
|
199
|
-
|
|
200
|
-
```bash
|
|
201
|
-
oomi openclaw debug tts-pipeline --text "When your voice reaches me, it gets turned into text, I read it and think about it, then I speak back through the managed chat session." --json
|
|
202
|
-
```
|
|
203
|
-
|
|
204
|
-
Real Qwen provider replay:
|
|
205
|
-
|
|
206
|
-
```bash
|
|
207
|
-
oomi openclaw debug tts-pipeline --text "When your voice reaches me, it gets turned into text, I read it and think about it, then I speak back through the managed chat session." --live-provider --env-file .env.local --provider-timeout-ms 20000 --json
|
|
208
|
-
```
|
|
209
|
-
|
|
210
|
-
What a good result looks like:
|
|
211
|
-
- `backend.success = true`
|
|
212
|
-
- `managed.assistantSpeechFinal.present = true`
|
|
213
|
-
- `qwen.errorCode = null`
|
|
214
|
-
- `qwen.audioDeltaCount > 0` when `--live-provider` is used
|
|
215
|
-
|
|
216
|
-
This is the preferred pre-publish gate for managed voice regressions, because it is much faster than publishing to npm and testing through a live OpenClaw machine first.
|
|
217
|
-
|
|
218
|
-
## Local OpenClaw Dev Harness
|
|
219
|
-
|
|
220
|
-
For plugin/runtime work, the preferred pre-publish loop is:
|
|
221
|
-
|
|
222
|
-
1. run the repo-local CLI directly from source
|
|
223
|
-
2. run the same flow inside the Dockerized OpenClaw dev harness using a local packed tarball
|
|
224
|
-
3. only then update a real OpenClaw machine
|
|
225
|
-
|
|
226
|
-
Fast source smoke from the repo checkout:
|
|
227
|
-
|
|
228
|
-
```bash
|
|
229
|
-
node packages/oomi-ai/bin/oomi-ai.js openclaw debug persona-runtime --name "Chef Dev" --json
|
|
230
|
-
```
|
|
231
|
-
|
|
232
|
-
Containerized real-runtime smoke:
|
|
233
|
-
|
|
234
|
-
```bash
|
|
235
|
-
docker compose -f docker/openclaw-dev/compose.yml build openclaw-dev
|
|
236
|
-
docker compose -f docker/openclaw-dev/compose.yml up -d openclaw-dev
|
|
237
|
-
docker compose -f docker/openclaw-dev/compose.yml exec -T openclaw-dev openclaw gateway health --url ws://127.0.0.1:18789 --token dev-gateway-token --json
|
|
238
|
-
docker compose -f docker/openclaw-dev/compose.yml exec -T openclaw-dev oomi-local openclaw debug persona-runtime --name "Chef Dev" --json
|
|
24
|
+
- Purpose: pair a device, manage the OpenClaw bridge worker, and support managed gateway traffic for device-backed chat and voice.
|
|
25
|
+
|
|
26
|
+
3. Persona app API tools
|
|
27
|
+
|
|
28
|
+
- Files: `persona-app/*`, `lib/personaApiClient.js`, `bin/oomi-ai.js`
|
|
29
|
+
- Purpose: inspect account-linked component-composed persona apps and apply approved backend actions.
|
|
30
|
+
|
|
31
|
+
4. Permissioned context tools
|
|
32
|
+
|
|
33
|
+
- Files: `lib/personaApiClient.js`, `bin/oomi-ai.js`
|
|
34
|
+
- Purpose: read user-approved Oomi backend context, such as HealthKit summaries synced by the mobile app.
|
|
35
|
+
|
|
36
|
+
5. Agent/operator instructions
|
|
37
|
+
|
|
38
|
+
- Files: `agent_instructions.md`, `skills/oomi/*`
|
|
39
|
+
- Purpose: tell OpenClaw agents how to connect, repair, and use Oomi without local UI code generation.
|
|
40
|
+
|
|
41
|
+
## Boundary
|
|
42
|
+
|
|
43
|
+
Persona UI is an Oomi-managed client capability, not an OpenClaw-machine responsibility.
|
|
44
|
+
|
|
45
|
+
This package exposes connection, bridge, voice, and persona app state APIs. It does not include a local persona app workspace, local persona runtime manager, queued UI job runner, or app framework bundle.
|
|
46
|
+
|
|
47
|
+
## Install And Upgrade
|
|
48
|
+
|
|
49
|
+
Global install:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
pnpm add -g oomi-ai@latest
|
|
53
|
+
```
|
|
54
|
+
|
|
55
|
+
Fallback:
|
|
56
|
+
|
|
57
|
+
```bash
|
|
58
|
+
npm install -g oomi-ai@latest
|
|
59
|
+
```
|
|
60
|
+
|
|
61
|
+
Install or update the OpenClaw plugin:
|
|
62
|
+
|
|
63
|
+
```bash
|
|
64
|
+
openclaw plugins install oomi-ai@latest
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
After upgrading, refresh the running bridge so stale gateway connections do not keep using the previous package:
|
|
68
|
+
|
|
69
|
+
```bash
|
|
70
|
+
oomi openclaw refresh --skip-version-check
|
|
71
|
+
oomi openclaw bridge ps
|
|
72
|
+
```
|
|
73
|
+
|
|
74
|
+
If the bridge still looks stale, restart it explicitly:
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
oomi openclaw bridge restart --detach
|
|
78
|
+
oomi openclaw bridge ps
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
On macOS supervised installs:
|
|
82
|
+
|
|
83
|
+
```bash
|
|
84
|
+
oomi openclaw bridge service restart
|
|
85
|
+
oomi openclaw bridge service status
|
|
239
86
|
```
|
|
240
87
|
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
246
|
-
|
|
247
|
-
|
|
248
|
-
|
|
249
|
-
|
|
250
|
-
```
|
|
251
|
-
|
|
252
|
-
What the harness does:
|
|
253
|
-
|
|
254
|
-
- bootstraps an isolated OpenClaw home rooted at `HOME/.openclaw`
|
|
255
|
-
- runs `openclaw onboard --non-interactive ...`
|
|
256
|
-
- writes and applies `HOME/.openclaw/oomi-dev-profile.json` using the same shared profile contract the future onboarding UI and hosted-agent bootstrap should use
|
|
257
|
-
- enables the Oomi channel account through that applied profile and relies on local OpenClaw plugin auto-discovery for the installed `oomi-ai` plugin
|
|
258
|
-
- writes device identity material used by the `oomi-ai` bridge tooling
|
|
259
|
-
- packs the local `packages/oomi-ai` checkout into a `.tgz`
|
|
260
|
-
- installs that tarball globally in the container
|
|
261
|
-
- installs the same tarball as a real OpenClaw plugin
|
|
262
|
-
- defaults model auth to `oomi-managed` so onboarding/bootstrap does not require end-user provider keys
|
|
263
|
-
- runs `openclaw gateway` as the foreground container process
|
|
264
|
-
|
|
265
|
-
Useful env overrides for local integration:
|
|
266
|
-
|
|
267
|
-
- `OOMI_DEV_BACKEND_URL`
|
|
268
|
-
- `OOMI_DEV_DEVICE_TOKEN`
|
|
269
|
-
- `OOMI_DEV_MODEL_AUTH_MODE`
|
|
270
|
-
- `OPENCLAW_GATEWAY_TOKEN`
|
|
271
|
-
- `OPENCLAW_GATEWAY_PASSWORD`
|
|
272
|
-
|
|
273
|
-
Recommended local modes:
|
|
274
|
-
|
|
275
|
-
- onboarding/runtime checks without provider keys
|
|
276
|
-
- `OOMI_DEV_MODEL_AUTH_MODE=oomi-managed`
|
|
277
|
-
- internal real-response smoke before publish
|
|
278
|
-
- `OPENROUTER_API_KEY=...`
|
|
279
|
-
- optional explicit override: `OOMI_DEV_MODEL_AUTH_MODE=provider-env`
|
|
280
|
-
|
|
281
|
-
The default container config is intentionally safe for onboarding and runtime testing. It does not require a published npm version, and it does not require end-user provider keys.
|
|
282
|
-
|
|
283
|
-
To make the Dockerized OpenClaw runtime actually answer managed chat locally today, add this to the repo `.env.local`:
|
|
88
|
+
Healthy update expectation:
|
|
89
|
+
|
|
90
|
+
- exactly one bridge worker is active
|
|
91
|
+
- the worker reports the newly installed package version
|
|
92
|
+
- bridge status reaches `connected`
|
|
93
|
+
|
|
94
|
+
## Standard Connect Flow
|
|
95
|
+
|
|
96
|
+
Pair the machine:
|
|
284
97
|
|
|
285
98
|
```bash
|
|
286
|
-
|
|
287
|
-
OPENROUTER_API_KEY=<your-openrouter-key>
|
|
99
|
+
oomi openclaw pair --app-url https://www.oomi.ai --no-start
|
|
288
100
|
```
|
|
289
101
|
|
|
290
|
-
|
|
291
|
-
|
|
292
|
-
## Persona Scaffolding
|
|
102
|
+
Print the OpenClaw config:
|
|
293
103
|
|
|
294
|
-
|
|
295
|
-
|
|
296
|
-
```
|
|
297
|
-
oomi personas scaffold market-analyst --name "Market Analyst" --description "Private app for reviewing my broker positions and risk." --out ~/.openclaw/workspace/personas/market-analyst
|
|
298
|
-
```
|
|
299
|
-
|
|
300
|
-
Use:
|
|
301
|
-
- `oomi personas create <id>` only for repo-local manifest work, not for the normal managed Oomi persona flow
|
|
302
|
-
- `oomi personas create-managed <slug>` to create the backend managed persona record before launch or runtime registration
|
|
303
|
-
- `oomi personas scaffold <slug>` for an XR-first WebSpatial Oomi app shell with runtime metadata and health documents
|
|
304
|
-
- `oomi persona-jobs execute --message-file <job.json>` when OpenClaw receives a structured persona orchestration job from Oomi
|
|
104
|
+
```bash
|
|
105
|
+
oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai
|
|
106
|
+
```
|
|
305
107
|
|
|
306
|
-
|
|
108
|
+
Apply the printed `channels.oomi.accounts.default` config and restart OpenClaw.
|
|
109
|
+
|
|
110
|
+
Start or repair the bridge:
|
|
307
111
|
|
|
308
112
|
```bash
|
|
309
|
-
oomi
|
|
310
|
-
oomi personas status market-analyst
|
|
311
|
-
oomi personas stop market-analyst
|
|
312
|
-
oomi personas delete market-analyst
|
|
313
|
-
# recovery only, after create-managed already exists
|
|
314
|
-
oomi personas runtime-register market-analyst --local-port 4789
|
|
315
|
-
oomi personas heartbeat market-analyst --local-port 4789
|
|
316
|
-
oomi persona-jobs start pj_123
|
|
317
|
-
oomi persona-jobs succeed pj_123 --workspace-path ~/.openclaw/workspace/personas/market-analyst --local-port 4789
|
|
318
|
-
oomi persona-jobs fail pj_123 --code JOB_FAILED --message "Scaffold generation failed."
|
|
113
|
+
oomi openclaw bridge ensure --detach
|
|
319
114
|
```
|
|
320
115
|
|
|
321
|
-
|
|
116
|
+
## Configuration
|
|
117
|
+
|
|
118
|
+
OpenClaw channel config lives under:
|
|
119
|
+
|
|
120
|
+
```json
|
|
121
|
+
{
|
|
122
|
+
"channels": {
|
|
123
|
+
"oomi": {
|
|
124
|
+
"defaultAccountId": "default",
|
|
125
|
+
"accounts": {
|
|
126
|
+
"default": {
|
|
127
|
+
"backendUrl": "https://api.oomi.ai",
|
|
128
|
+
"deviceToken": "...",
|
|
129
|
+
"defaultSessionKey": "agent:main:webchat:channel:oomi",
|
|
130
|
+
"requestTimeoutMs": 15000
|
|
131
|
+
}
|
|
132
|
+
}
|
|
133
|
+
}
|
|
134
|
+
}
|
|
135
|
+
}
|
|
136
|
+
```
|
|
322
137
|
|
|
323
|
-
|
|
324
|
-
- When you pass only `--local-port`, `oomi-ai` derives a LAN-reachable endpoint automatically and keeps the healthcheck on loopback.
|
|
325
|
-
- If you pass `--endpoint` explicitly, it should be a LAN or relay URL that the client can actually open.
|
|
326
|
-
- The scaffolded XR route should default directly into a mounted scene component that calls `configurePersonaScene()`, logs `detectSpatialEnvironment()`, and renders multiple authored `enable-xr` panels.
|
|
327
|
-
- The browser route should remain valid, but the XR entry path should not fall back to a flat marketing or home page.
|
|
328
|
-
- Do not use manual `npm run dev` as the first-time persona launch path for managed Oomi personas.
|
|
138
|
+
Required fields:
|
|
329
139
|
|
|
330
|
-
|
|
140
|
+
- `backendUrl`
|
|
141
|
+
- `deviceToken`
|
|
142
|
+
|
|
143
|
+
Optional fields:
|
|
144
|
+
|
|
145
|
+
- `defaultSessionKey`
|
|
146
|
+
- `requestTimeoutMs`
|
|
147
|
+
|
|
148
|
+
## Persona App API Tools
|
|
149
|
+
|
|
150
|
+
Component-composed persona apps are the default Oomi mini-app path. Use these commands when the user asks about or updates an existing Oomi persona app such as Fitness Today:
|
|
331
151
|
|
|
332
152
|
```bash
|
|
333
|
-
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
export DESCRIPTION="Personal travel assistant for Tokyo trip planning - itineraries, food, culture, and summer tips for July"
|
|
337
|
-
|
|
338
|
-
# 1. Scaffold the local app workspace
|
|
339
|
-
oomi personas scaffold "$SLUG" \
|
|
340
|
-
--name "$NAME" \
|
|
341
|
-
--description "$DESCRIPTION" \
|
|
342
|
-
--out "$WORKSPACE_ROOT/$SLUG" \
|
|
343
|
-
--force
|
|
344
|
-
|
|
345
|
-
# 2. Verify scaffold actually wrote files
|
|
346
|
-
test -f "$WORKSPACE_ROOT/$SLUG/persona.json"
|
|
347
|
-
test -f "$WORKSPACE_ROOT/$SLUG/oomi.runtime.json"
|
|
348
|
-
test -f "$WORKSPACE_ROOT/$SLUG/package.json"
|
|
349
|
-
|
|
350
|
-
# 3. Create the backend persona record first
|
|
351
|
-
oomi personas create-managed "$SLUG" \
|
|
352
|
-
--name "$NAME" \
|
|
353
|
-
--description "$DESCRIPTION" \
|
|
354
|
-
--json
|
|
355
|
-
|
|
356
|
-
# 4. Launch the managed runtime and register it with Oomi
|
|
357
|
-
oomi personas launch-managed "$SLUG" \
|
|
358
|
-
--workspace-root "$WORKSPACE_ROOT" \
|
|
359
|
-
--force-install \
|
|
360
|
-
--json
|
|
361
|
-
|
|
362
|
-
# 5. Verify local runtime state exists
|
|
363
|
-
test -f "$WORKSPACE_ROOT/$SLUG/.oomi/runtime.json"
|
|
364
|
-
cat "$WORKSPACE_ROOT/$SLUG/.oomi/runtime.json"
|
|
365
|
-
|
|
366
|
-
# 6. Verify Oomi-local status
|
|
367
|
-
oomi personas status "$SLUG" --json
|
|
368
|
-
|
|
369
|
-
# 7. Send a heartbeat using the actual assigned port
|
|
370
|
-
PORT="$(jq -r '.localPort' "$WORKSPACE_ROOT/$SLUG/.oomi/runtime.json")"
|
|
371
|
-
oomi personas heartbeat "$SLUG" \
|
|
372
|
-
--local-port "$PORT" \
|
|
373
|
-
--json
|
|
153
|
+
oomi persona-apps list --json
|
|
154
|
+
oomi persona-apps show fitness-today --json
|
|
155
|
+
oomi persona-apps apply-action fitness-today --action fitness.complete_workout --payload-json '{"goalId":"easy-run","minutes":20}' --json
|
|
374
156
|
```
|
|
375
157
|
|
|
376
|
-
Rules for
|
|
158
|
+
Rules for agents:
|
|
377
159
|
|
|
378
|
-
-
|
|
379
|
-
-
|
|
380
|
-
-
|
|
381
|
-
-
|
|
382
|
-
- heartbeat only after `.oomi/runtime.json` exists
|
|
383
|
-
- fail fast if any `--json` command does not return success
|
|
384
|
-
- never treat local runtime success as backend registration success
|
|
160
|
+
- Answer from returned `appState.bindings`, not from memory.
|
|
161
|
+
- Apply updates only through approved `persona-apps apply-action` actions.
|
|
162
|
+
- If the persona app does not exist, tell the user to add it from the Oomi client first.
|
|
163
|
+
- Do not create local persona UI projects.
|
|
385
164
|
|
|
386
|
-
|
|
165
|
+
For the Fitness v0 slice:
|
|
387
166
|
|
|
388
167
|
```bash
|
|
389
|
-
oomi
|
|
390
|
-
oomi
|
|
391
|
-
npm run dev &
|
|
168
|
+
oomi persona-apps apply-action fitness-today --action fitness.complete_goal --payload-json '{"goalId":"easy-run"}' --json
|
|
169
|
+
oomi persona-apps apply-action fitness-today --action fitness.complete_workout --payload-json '{"goalId":"easy-run","minutes":20}' --json
|
|
392
170
|
```
|
|
393
171
|
|
|
394
|
-
|
|
172
|
+
## Context Tools
|
|
395
173
|
|
|
396
|
-
|
|
174
|
+
Use context tools when the user asks a data-backed question that should come from Oomi-approved mobile permissions, not model memory. For health and fitness questions, read the latest backend HealthKit context first:
|
|
397
175
|
|
|
398
176
|
```bash
|
|
399
|
-
oomi
|
|
177
|
+
oomi context health --json
|
|
400
178
|
```
|
|
401
179
|
|
|
402
|
-
|
|
180
|
+
Rules for agents:
|
|
181
|
+
|
|
182
|
+
- Answer from `healthContext.summary` and `healthContext.derived`.
|
|
183
|
+
- Respect `healthContext.agentGuidance.canAnswerFromContext`.
|
|
184
|
+
- If context is `missing`, `needs_sync`, `permission_denied`, `disabled_by_user`, or `unavailable`, tell the user what repair action is needed instead of guessing.
|
|
185
|
+
- If context is `stale`, mention freshness when timing matters.
|
|
186
|
+
- Do not ask the phone for HealthKit directly; the Oomi mobile app syncs HealthKit to the backend.
|
|
187
|
+
|
|
188
|
+
## Voice Contract
|
|
189
|
+
|
|
190
|
+
Managed voice uses the same Oomi plugin and bridge layer as managed chat.
|
|
191
|
+
|
|
192
|
+
For assistant replies, visible chat text stays user-facing. Optional hidden TTS metadata may live at `metadata.spoken`:
|
|
193
|
+
|
|
194
|
+
```json
|
|
195
|
+
{
|
|
196
|
+
"metadata": {
|
|
197
|
+
"spoken": {
|
|
198
|
+
"text": "Speech-optimized text for TTS only.",
|
|
199
|
+
"language": "English",
|
|
200
|
+
"instructions": "Speak with warm, upbeat conversational energy and natural pacing."
|
|
201
|
+
}
|
|
202
|
+
}
|
|
203
|
+
}
|
|
204
|
+
```
|
|
205
|
+
|
|
206
|
+
Rules:
|
|
207
|
+
|
|
208
|
+
- visible `content` is the source of truth for chat rendering
|
|
209
|
+
- `metadata.spoken.text` is backend TTS input only
|
|
210
|
+
- visible chat text should not contain raw speaking tags
|
|
211
|
+
- the package preserves valid `metadata.spoken`
|
|
212
|
+
- if omitted, the package may synthesize a bounded fallback from visible assistant text
|
|
213
|
+
|
|
214
|
+
## Health Checks
|
|
215
|
+
|
|
216
|
+
Use these when chat or voice is failing:
|
|
217
|
+
|
|
218
|
+
```bash
|
|
219
|
+
oomi openclaw bridge ps
|
|
220
|
+
oomi openclaw bridge service status
|
|
221
|
+
oomi openclaw status
|
|
222
|
+
tail -f ~/.openclaw/logs/oomi-bridge-live.log
|
|
223
|
+
tail -f ~/.openclaw/logs/gateway.log
|
|
224
|
+
tail -f ~/.openclaw/logs/gateway.err.log
|
|
225
|
+
```
|
|
226
|
+
|
|
227
|
+
Bridge status meanings:
|
|
228
|
+
|
|
229
|
+
- `starting`: bridge booting or waiting for managed subscription
|
|
230
|
+
- `connected`: ready for managed chat and voice traffic
|
|
231
|
+
- `reconnecting`: transport dropped and retry is scheduled
|
|
232
|
+
- `degraded`: bridge caught a runtime fault but is still alive
|
|
233
|
+
- `error`: startup or auth failure blocked useful operation
|
|
234
|
+
- `stopped`: not running or intentionally stopped
|
|
235
|
+
|
|
236
|
+
## Troubleshooting
|
|
237
|
+
|
|
238
|
+
Duplicate plugin id warning:
|
|
239
|
+
|
|
240
|
+
- Cause: multiple discoverable `oomi-ai` installs.
|
|
241
|
+
- Action: remove stale extension copies and reinstall once.
|
|
242
|
+
|
|
243
|
+
`invalid handshake: first request must be connect`:
|
|
244
|
+
|
|
245
|
+
- Cause: a gateway request was sent before `connect` had been accepted.
|
|
246
|
+
- Action: update `oomi-ai`, restart the bridge, and confirm only one bridge worker is running.
|
|
247
|
+
|
|
248
|
+
STT works but the assistant does not reply:
|
|
249
|
+
|
|
250
|
+
- Cause: the voice turn likely reached Oomi, but the managed gateway or OpenClaw run failed later.
|
|
251
|
+
- Action: inspect `gateway.log`, `gateway.err.log`, and the session JSONL for auth, network, or bridge restart errors.
|
|
252
|
+
|
|
253
|
+
Bridge keeps using an old package after update:
|
|
403
254
|
|
|
404
|
-
|
|
405
|
-
|
|
406
|
-
|
|
407
|
-
|
|
408
|
-
```
|
|
409
|
-
|
|
410
|
-
This command:
|
|
255
|
+
- Action: run `oomi openclaw refresh --skip-version-check`.
|
|
256
|
+
- If still stale: run `oomi openclaw bridge restart --detach`.
|
|
257
|
+
- On macOS service installs: run `oomi openclaw bridge service restart`.
|
|
258
|
+
- Confirm with `oomi openclaw bridge ps`.
|
|
411
259
|
|
|
412
|
-
|
|
413
|
-
- scaffolds only when the workspace is missing
|
|
414
|
-
- installs dependencies only when needed or forced
|
|
415
|
-
- allocates or reuses a free local port
|
|
416
|
-
- starts or reuses the local runtime
|
|
417
|
-
- registers the runtime URL back to Oomi unless `--no-register` is set
|
|
260
|
+
## Developer Verification
|
|
418
261
|
|
|
419
|
-
|
|
262
|
+
From `packages/oomi-ai`:
|
|
420
263
|
|
|
421
264
|
```bash
|
|
422
|
-
|
|
265
|
+
npm run check
|
|
266
|
+
npm test
|
|
267
|
+
npm pack --dry-run
|
|
268
|
+
npm publish --dry-run --no-git-checks --access public
|
|
423
269
|
```
|
|
424
270
|
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
## Bridge Health States
|
|
428
|
-
|
|
429
|
-
The bridge status file is written locally and should roughly be interpreted as:
|
|
430
|
-
- `starting`: process booting or waiting for managed subscription
|
|
431
|
-
- `connected`: broker connected and managed subscription confirmed
|
|
432
|
-
- `reconnecting`: broker or gateway transport dropped and reconnect is scheduled
|
|
433
|
-
- `degraded`: bridge is still alive but hit a runtime fault that needs attention
|
|
434
|
-
- `error`: startup or auth-level failure that prevents useful operation
|
|
435
|
-
- `stopped`: bridge is not running or was intentionally stopped
|
|
436
|
-
|
|
437
|
-
For voice support, a `voice_session_*` failure should be treated as narrower than a full provider outage.
|
|
438
|
-
|
|
439
|
-
## Troubleshooting
|
|
440
|
-
|
|
441
|
-
### `invalid handshake: first request must be connect`
|
|
442
|
-
|
|
443
|
-
Meaning:
|
|
444
|
-
- a gateway request was forwarded before the session had accepted `connect`
|
|
445
|
-
|
|
446
|
-
What to check:
|
|
447
|
-
- update to the latest `oomi-ai`
|
|
448
|
-
- restart the bridge worker
|
|
449
|
-
- confirm only one active bridge worker exists for the device
|
|
450
|
-
|
|
451
|
-
### `duplicate plugin id detected`
|
|
452
|
-
|
|
453
|
-
Meaning:
|
|
454
|
-
- OpenClaw can see more than one `oomi-ai` plugin source
|
|
455
|
-
|
|
456
|
-
What to check:
|
|
457
|
-
- ensure there is only one active install under OpenClaw plugin discovery paths
|
|
458
|
-
- remove stale local extension copies before reinstalling
|
|
459
|
-
|
|
460
|
-
### Bridge keeps flipping between `reconnecting`, `degraded`, or `stopped`
|
|
461
|
-
|
|
462
|
-
What to check:
|
|
463
|
-
- `oomi openclaw bridge ps`
|
|
464
|
-
- `oomi openclaw bridge service status`
|
|
465
|
-
- `tail -f ~/.openclaw/logs/oomi-bridge-live.log`
|
|
466
|
-
- `tail -f ~/.openclaw/logs/gateway.log`
|
|
467
|
-
|
|
468
|
-
If the process is alive but runtime faults are being caught, expect `degraded` rather than an immediate hard stop.
|
|
469
|
-
|
|
470
|
-
### Voice STT works but the agent does not answer
|
|
471
|
-
|
|
472
|
-
This usually means one of these:
|
|
473
|
-
- the managed gateway/device side is not actually ready
|
|
474
|
-
- the bridge or agent run failed after delivery
|
|
475
|
-
- the OpenClaw run stopped with an upstream provider `network_error`
|
|
476
|
-
|
|
477
|
-
In that situation, inspect:
|
|
478
|
-
- `~/.openclaw/logs/gateway.log`
|
|
479
|
-
- `~/.openclaw/logs/gateway.err.log`
|
|
480
|
-
- the relevant session JSONL in `~/.openclaw/agents/main/sessions/`
|
|
481
|
-
|
|
482
|
-
### Voice text works but cloned TTS fails with `MISSING_SPOKEN_METADATA`
|
|
483
|
-
|
|
484
|
-
Meaning:
|
|
485
|
-
- the assistant text arrived
|
|
486
|
-
- the backend voice relay never received valid hidden `metadata.spoken`
|
|
487
|
-
|
|
488
|
-
What to check:
|
|
489
|
-
- run the local replay gate before publishing:
|
|
490
|
-
- `oomi openclaw debug assistant-final --text "..."`
|
|
491
|
-
- `oomi openclaw debug tts-pipeline --text "..."`
|
|
492
|
-
- if the package local replay succeeds but the live machine fails, verify the OpenClaw machine is actually running the updated bridge binary
|
|
493
|
-
- if the local replay fails, fix the assistant-final contract first instead of debugging the browser or backend deployment
|
|
494
|
-
|
|
495
|
-
## Developer Notes
|
|
496
|
-
|
|
497
|
-
If you are inspecting this package on npm, the main architectural points are:
|
|
498
|
-
- the extension path is the stable managed text contract
|
|
499
|
-
- the local bridge exists because Oomi also needs device-backed and voice-capable flows
|
|
500
|
-
- the persona scaffold exists because Oomi specialist personas can surface as scoped XR/web work surfaces, not just chat tabs
|
|
501
|
-
- the bridge has been hardened for:
|
|
502
|
-
- strict `connect`-first forwarding
|
|
503
|
-
- method-specific request shaping
|
|
504
|
-
- `idempotencyKey` handling
|
|
505
|
-
- bridge status that does not report `connected` before managed subscription is ready
|
|
506
|
-
- runtime fault isolation so local session failures are less likely to crash the whole provider
|
|
507
|
-
- one shared hidden managed-voice speech metadata helper used by both the extension and the local bridge
|
|
508
|
-
|
|
509
|
-
For generated WebSpatial persona apps, the intended scaffold contract is:
|
|
510
|
-
- XR mode defaults directly into the mounted scene route
|
|
511
|
-
- `configurePersonaScene()` runs from that scene component
|
|
512
|
-
- runtime diagnostics are logged on scene boot
|
|
513
|
-
- multiple meaningful UI surfaces use `enable-xr` and `xrStyle()`
|
|
514
|
-
- `html.is-spatial` keeps the host shell transparent
|
|
515
|
-
|
|
516
|
-
If you are developing the plugin, test the packaged surface with:
|
|
517
|
-
|
|
518
|
-
```bash
|
|
519
|
-
cd packages/oomi-ai
|
|
520
|
-
node --test test/*.test.mjs
|
|
521
|
-
npm pack --dry-run
|
|
522
|
-
```
|
|
523
|
-
|
|
524
|
-
For managed voice changes, do not stop at the package tests. Run the local replay gate from the repo root as well, especially before publishing:
|
|
525
|
-
|
|
526
|
-
```bash
|
|
527
|
-
oomi openclaw debug tts-pipeline --text "Local managed voice validation text." --json
|
|
528
|
-
oomi openclaw debug tts-pipeline --text "Local managed voice validation text." --live-provider --env-file .env.local --provider-timeout-ms 20000 --json
|
|
529
|
-
```
|
|
530
|
-
|
|
531
|
-
## Release Process
|
|
532
|
-
|
|
533
|
-
Before publishing:
|
|
534
|
-
|
|
535
|
-
```bash
|
|
536
|
-
cd packages/oomi-ai
|
|
537
|
-
node --test test/*.test.mjs
|
|
538
|
-
npm pack --dry-run
|
|
539
|
-
```
|
|
540
|
-
|
|
541
|
-
For voice-related changes, also run the repo-backed local replay gate before publish:
|
|
542
|
-
|
|
543
|
-
```bash
|
|
544
|
-
oomi openclaw debug tts-pipeline --text "Local managed voice validation text." --json
|
|
545
|
-
oomi openclaw debug tts-pipeline --text "Local managed voice validation text." --live-provider --env-file .env.local --provider-timeout-ms 20000 --json
|
|
546
|
-
```
|
|
547
|
-
|
|
548
|
-
Then publish the bumped version:
|
|
549
|
-
|
|
550
|
-
```bash
|
|
551
|
-
pnpm check
|
|
552
|
-
pnpm publish --dry-run --no-git-checks --access public
|
|
553
|
-
pnpm publish --access public
|
|
554
|
-
```
|
|
271
|
+
Expected package contents are restricted by the `files` allowlist in `package.json`.
|