oomi-ai 0.2.12 → 0.2.14
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +192 -116
- package/agent_instructions.md +145 -35
- package/bin/oomi-ai.js +351 -26
- package/bin/sessionBridgeState.js +29 -2
- package/openclaw.plugin.json +1 -1
- package/package.json +2 -2
- package/skills/oomi/SKILL.md +100 -62
package/README.md
CHANGED
|
@@ -1,148 +1,224 @@
|
|
|
1
1
|
# oomi-ai
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
OpenClaw channel plugin and bridge tooling for Oomi managed chat and voice.
|
|
4
4
|
|
|
5
|
-
|
|
6
|
-
|
|
7
|
-
|
|
5
|
+
This package is for two audiences:
|
|
6
|
+
- OpenClaw operators who need to connect a machine to Oomi and keep chat or voice healthy
|
|
7
|
+
- Developers evaluating the plugin on npm and deciding whether it matches their OpenClaw + Oomi setup
|
|
8
|
+
|
|
9
|
+
## What This Package Ships
|
|
10
|
+
|
|
11
|
+
The npm package contains two Oomi integration surfaces:
|
|
12
|
+
|
|
13
|
+
1. OpenClaw channel extension
|
|
14
|
+
- File: `openclaw.extension.js`
|
|
15
|
+
- Purpose: stable managed text transport through the Oomi backend channel API
|
|
16
|
+
- This is the preferred integration surface for normal chat
|
|
17
|
+
|
|
18
|
+
2. Local bridge + CLI
|
|
19
|
+
- Files: `bin/oomi-ai.js`, `bin/sessionBridgeState.js`
|
|
20
|
+
- Purpose: pair a device, manage the OpenClaw bridge worker, and support managed gateway traffic needed by device-backed chat and voice
|
|
21
|
+
- This is the part that deals with broker sockets, local gateway sessions, challenge auth, and bridge health
|
|
22
|
+
|
|
23
|
+
In practical terms:
|
|
24
|
+
- If you only need a clean managed chat channel, the extension is the main reason to install this package
|
|
25
|
+
- If you need Oomi device-backed chat or voice on an OpenClaw machine, you also need the bridge tooling in this package
|
|
26
|
+
|
|
27
|
+
## When To Install It
|
|
28
|
+
|
|
29
|
+
Install `oomi-ai` if all of the following are true:
|
|
30
|
+
- you use OpenClaw
|
|
31
|
+
- you want Oomi as a managed channel inside OpenClaw
|
|
32
|
+
- you want device-backed Oomi chat, Oomi voice, or both
|
|
33
|
+
|
|
34
|
+
Do not install it just to use the Oomi web app by itself.
|
|
35
|
+
|
|
36
|
+
## Install And Upgrade
|
|
37
|
+
|
|
38
|
+
Global install:
|
|
39
|
+
|
|
40
|
+
```bash
|
|
41
|
+
pnpm add -g oomi-ai@latest
|
|
8
42
|
```
|
|
9
43
|
|
|
10
44
|
Fallback:
|
|
45
|
+
|
|
46
|
+
```bash
|
|
47
|
+
npm install -g oomi-ai@latest
|
|
11
48
|
```
|
|
12
|
-
|
|
49
|
+
|
|
50
|
+
Install the OpenClaw plugin:
|
|
51
|
+
|
|
52
|
+
```bash
|
|
53
|
+
openclaw plugins install oomi-ai@latest
|
|
13
54
|
```
|
|
14
55
|
|
|
15
|
-
|
|
56
|
+
Upgrade an existing machine:
|
|
16
57
|
|
|
17
|
-
|
|
58
|
+
```bash
|
|
59
|
+
pnpm add -g oomi-ai@latest
|
|
60
|
+
openclaw plugins install oomi-ai@latest
|
|
18
61
|
```
|
|
62
|
+
|
|
63
|
+
## Operator Quick Start
|
|
64
|
+
|
|
65
|
+
The packaged operator instructions live in [agent_instructions.md](./agent_instructions.md).
|
|
66
|
+
That is the primary reference for:
|
|
67
|
+
- pairing a device
|
|
68
|
+
- installing the plugin
|
|
69
|
+
- configuring `channels.oomi.accounts.default`
|
|
70
|
+
- running or supervising the bridge
|
|
71
|
+
- checking whether the system is healthy
|
|
72
|
+
- troubleshooting chat and voice failures
|
|
73
|
+
|
|
74
|
+
Fast-path install flow:
|
|
75
|
+
|
|
76
|
+
```bash
|
|
77
|
+
oomi openclaw pair --app-url https://www.oomi.ai --no-start
|
|
19
78
|
openclaw plugins install oomi-ai@latest
|
|
79
|
+
oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai
|
|
80
|
+
```
|
|
81
|
+
|
|
82
|
+
Then apply the printed `channels.oomi.accounts.default` config and restart OpenClaw.
|
|
83
|
+
|
|
84
|
+
## Configuration
|
|
85
|
+
|
|
86
|
+
OpenClaw channel config lives under:
|
|
87
|
+
|
|
88
|
+
```json
|
|
89
|
+
{
|
|
90
|
+
"channels": {
|
|
91
|
+
"oomi": {
|
|
92
|
+
"defaultAccountId": "default",
|
|
93
|
+
"accounts": {
|
|
94
|
+
"default": {
|
|
95
|
+
"backendUrl": "https://api.oomi.ai",
|
|
96
|
+
"deviceToken": "...",
|
|
97
|
+
"defaultSessionKey": "agent:main:webchat:channel:oomi",
|
|
98
|
+
"requestTimeoutMs": 15000
|
|
99
|
+
}
|
|
100
|
+
}
|
|
101
|
+
}
|
|
102
|
+
}
|
|
103
|
+
}
|
|
20
104
|
```
|
|
21
105
|
|
|
22
|
-
|
|
23
|
-
Channel account config fields (`channels.oomi.accounts.<accountId>`):
|
|
106
|
+
Required fields:
|
|
24
107
|
- `backendUrl`
|
|
25
108
|
- `deviceToken`
|
|
26
|
-
- `defaultSessionKey` (optional, default `agent:main:webchat:channel:oomi`)
|
|
27
|
-
- `requestTimeoutMs` (optional)
|
|
28
109
|
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
```
|
|
110
|
+
Optional fields:
|
|
111
|
+
- `defaultSessionKey`
|
|
112
|
+
- `requestTimeoutMs`
|
|
33
113
|
|
|
34
|
-
|
|
35
|
-
```
|
|
36
|
-
oomi init
|
|
37
|
-
```
|
|
114
|
+
## Runtime Model
|
|
38
115
|
|
|
39
|
-
|
|
40
|
-
```
|
|
41
|
-
oomi openclaw install
|
|
42
|
-
```
|
|
116
|
+
There are two runtime contracts worth understanding.
|
|
43
117
|
|
|
44
|
-
|
|
45
|
-
```
|
|
46
|
-
oomi openclaw pair --app-url https://your-oomi-app.vercel.app --device-id my-openclaw-mac --no-start
|
|
47
|
-
```
|
|
118
|
+
### Managed Text Chat
|
|
48
119
|
|
|
49
|
-
|
|
120
|
+
Managed text chat uses the OpenClaw channel extension and the Oomi backend channel API.
|
|
121
|
+
This path is the more stable contract and should be preferred when evaluating the plugin for normal chat.
|
|
50
122
|
|
|
51
|
-
|
|
52
|
-
- `Auth invite URL: https://.../connect/<single-use-token>`
|
|
53
|
-
- A copy-ready block for the user:
|
|
54
|
-
- `Oomi Connect Ready`
|
|
55
|
-
- `Auth Link: ...`
|
|
123
|
+
### Device-Backed Chat And Voice
|
|
56
124
|
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
125
|
+
Device-backed chat and voice use the local bridge.
|
|
126
|
+
That bridge:
|
|
127
|
+
- keeps a broker socket open to Oomi
|
|
128
|
+
- opens local gateway sessions on demand
|
|
129
|
+
- enforces `connect`-first request ordering
|
|
130
|
+
- preserves or synthesizes `idempotencyKey` for `chat.send`
|
|
131
|
+
- keeps voice-session faults from poisoning normal provider health where possible
|
|
61
132
|
|
|
62
|
-
|
|
63
|
-
```
|
|
64
|
-
oomi openclaw bridge ensure --detach # start if needed; no-op if already running
|
|
65
|
-
oomi openclaw bridge ps # list bridge pids
|
|
66
|
-
oomi openclaw bridge stop # stop all bridge workers
|
|
67
|
-
oomi openclaw bridge restart --detach # clean restart as background worker
|
|
68
|
-
tail -f ~/.openclaw/logs/oomi-bridge-live.log # detached bridge logs
|
|
69
|
-
```
|
|
70
|
-
`oomi openclaw bridge --detach` is equivalent to `oomi openclaw bridge start --detach`.
|
|
133
|
+
This is the part of the package most likely to matter when debugging voice turn failures.
|
|
71
134
|
|
|
72
|
-
|
|
73
|
-
```
|
|
74
|
-
oomi openclaw bridge service install # install + start service
|
|
75
|
-
oomi openclaw bridge service status
|
|
76
|
-
oomi openclaw bridge service restart
|
|
77
|
-
oomi openclaw bridge service stop
|
|
78
|
-
oomi openclaw bridge service uninstall
|
|
79
|
-
```
|
|
80
|
-
Optional: `oomi openclaw bridge service install --no-start` to install without starting.
|
|
81
|
-
|
|
82
|
-
Agent-intent mapping (recommended):
|
|
83
|
-
- If user says `Connect yourself to Oomi. Use app URL https://www.oomi.ai.`
|
|
84
|
-
- Run:
|
|
85
|
-
- `curl -fsSL https://www.oomi.ai/install.sh | bash`
|
|
86
|
-
- or `pnpm add -g oomi-ai@latest` (`npm install -g oomi-ai@latest` fallback)
|
|
87
|
-
- `oomi openclaw pair --app-url https://www.oomi.ai --no-start`
|
|
88
|
-
- `openclaw plugins install oomi-ai@latest`
|
|
89
|
-
- `oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai`
|
|
90
|
-
- Apply `channels.oomi.accounts.default` config and restart OpenClaw.
|
|
91
|
-
|
|
92
|
-
Important distinction:
|
|
93
|
-
- `pairCode` is one-time and used internally by the pair/bootstrap flow.
|
|
94
|
-
- Invite auth links are the required user flow.
|
|
95
|
-
- Managed chat connect now uses OpenClaw challenge auth (`connect.challenge` nonce + signed device payload) in the local bridge path.
|
|
96
|
-
|
|
97
|
-
Sync personas from the repo into the backend registry:
|
|
98
|
-
```
|
|
99
|
-
oomi personas sync --backend-url http://localhost:3001
|
|
100
|
-
```
|
|
135
|
+
## Bridge Health States
|
|
101
136
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
|
|
105
|
-
|
|
137
|
+
The bridge status file is written locally and should roughly be interpreted as:
|
|
138
|
+
- `starting`: process booting or waiting for managed subscription
|
|
139
|
+
- `connected`: broker connected and managed subscription confirmed
|
|
140
|
+
- `reconnecting`: broker or gateway transport dropped and reconnect is scheduled
|
|
141
|
+
- `degraded`: bridge is still alive but hit a runtime fault that needs attention
|
|
142
|
+
- `error`: startup or auth-level failure that prevents useful operation
|
|
143
|
+
- `stopped`: bridge is not running or was intentionally stopped
|
|
106
144
|
|
|
107
|
-
|
|
108
|
-
```
|
|
109
|
-
oomi init --workspace /path/to/openclaw/workspace
|
|
110
|
-
oomi init --agents-file /path/to/AGENTS.md
|
|
111
|
-
oomi openclaw install --skills-dir /path/to/openclaw/skills
|
|
112
|
-
oomi openclaw pair --app-url https://your-oomi-app.vercel.app --no-start
|
|
113
|
-
oomi openclaw pair --app-url https://your-oomi-app.vercel.app --json
|
|
114
|
-
oomi personas sync --root /path/to/oomi
|
|
115
|
-
oomi personas create creator --status active --chat-session agent:main:webchat:channel:oomi-creator
|
|
116
|
-
```
|
|
145
|
+
For voice support, a `voice_session_*` failure should be treated as narrower than a full provider outage.
|
|
117
146
|
|
|
118
|
-
|
|
119
|
-
|
|
120
|
-
|
|
121
|
-
|
|
122
|
-
|
|
123
|
-
|
|
124
|
-
|
|
125
|
-
|
|
126
|
-
-
|
|
127
|
-
|
|
128
|
-
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
139
|
-
|
|
147
|
+
## Troubleshooting
|
|
148
|
+
|
|
149
|
+
### `invalid handshake: first request must be connect`
|
|
150
|
+
|
|
151
|
+
Meaning:
|
|
152
|
+
- a gateway request was forwarded before the session had accepted `connect`
|
|
153
|
+
|
|
154
|
+
What to check:
|
|
155
|
+
- update to the latest `oomi-ai`
|
|
156
|
+
- restart the bridge worker
|
|
157
|
+
- confirm only one active bridge worker exists for the device
|
|
158
|
+
|
|
159
|
+
### `duplicate plugin id detected`
|
|
160
|
+
|
|
161
|
+
Meaning:
|
|
162
|
+
- OpenClaw can see more than one `oomi-ai` plugin source
|
|
163
|
+
|
|
164
|
+
What to check:
|
|
165
|
+
- ensure there is only one active install under OpenClaw plugin discovery paths
|
|
166
|
+
- remove stale local extension copies before reinstalling
|
|
167
|
+
|
|
168
|
+
### Bridge keeps flipping between `reconnecting`, `degraded`, or `stopped`
|
|
169
|
+
|
|
170
|
+
What to check:
|
|
171
|
+
- `oomi openclaw bridge ps`
|
|
172
|
+
- `oomi openclaw bridge service status`
|
|
173
|
+
- `tail -f ~/.openclaw/logs/oomi-bridge-live.log`
|
|
174
|
+
- `tail -f ~/.openclaw/logs/gateway.log`
|
|
175
|
+
|
|
176
|
+
If the process is alive but runtime faults are being caught, expect `degraded` rather than an immediate hard stop.
|
|
177
|
+
|
|
178
|
+
### Voice STT works but the agent does not answer
|
|
179
|
+
|
|
180
|
+
This usually means one of these:
|
|
181
|
+
- the managed gateway/device side is not actually ready
|
|
182
|
+
- the bridge or agent run failed after delivery
|
|
183
|
+
- the OpenClaw run stopped with an upstream provider `network_error`
|
|
184
|
+
|
|
185
|
+
In that situation, inspect:
|
|
186
|
+
- `~/.openclaw/logs/gateway.log`
|
|
187
|
+
- `~/.openclaw/logs/gateway.err.log`
|
|
188
|
+
- the relevant session JSONL in `~/.openclaw/agents/main/sessions/`
|
|
189
|
+
|
|
190
|
+
## Developer Notes
|
|
191
|
+
|
|
192
|
+
If you are inspecting this package on npm, the main architectural points are:
|
|
193
|
+
- the extension path is the stable managed text contract
|
|
194
|
+
- the local bridge exists because Oomi also needs device-backed and voice-capable flows
|
|
195
|
+
- the bridge has been hardened for:
|
|
196
|
+
- strict `connect`-first forwarding
|
|
197
|
+
- method-specific request shaping
|
|
198
|
+
- `idempotencyKey` handling
|
|
199
|
+
- bridge status that does not report `connected` before managed subscription is ready
|
|
200
|
+
- runtime fault isolation so local session failures are less likely to crash the whole provider
|
|
201
|
+
|
|
202
|
+
If you are developing the plugin, test the packaged surface with:
|
|
203
|
+
|
|
204
|
+
```bash
|
|
205
|
+
cd packages/oomi-ai
|
|
206
|
+
node --test test/*.test.mjs
|
|
207
|
+
npm pack --dry-run
|
|
140
208
|
```
|
|
141
209
|
|
|
142
|
-
##
|
|
210
|
+
## Release Process
|
|
211
|
+
|
|
212
|
+
Before publishing:
|
|
213
|
+
|
|
214
|
+
```bash
|
|
215
|
+
cd packages/oomi-ai
|
|
216
|
+
node --test test/*.test.mjs
|
|
217
|
+
npm pack --dry-run
|
|
143
218
|
```
|
|
144
|
-
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
219
|
+
|
|
220
|
+
Then publish the bumped version:
|
|
221
|
+
|
|
222
|
+
```bash
|
|
223
|
+
npm publish --access public
|
|
148
224
|
```
|
package/agent_instructions.md
CHANGED
|
@@ -1,46 +1,156 @@
|
|
|
1
|
-
# Oomi
|
|
1
|
+
# Oomi OpenClaw Operator Guide
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
Inline tags are stripped from user-visible text.
|
|
3
|
+
Use these instructions when you need to connect, repair, or verify an OpenClaw machine that should talk to Oomi.
|
|
5
4
|
|
|
6
|
-
|
|
7
|
-
- [anim:Waving], [anim:Walking], [anim:Idle], [anim:Sitting Idle]
|
|
8
|
-
- [face:happy], [face:sad], [face:surprised], [face:focused], [face:gentle], [face:thinking]
|
|
9
|
-
- [gesture:nod], [gesture:think], [gesture:shrug], [gesture:wave], [gesture:bow]
|
|
10
|
-
- [look:camera], [look:left], [look:right], [look:up], [look:down]
|
|
5
|
+
## Primary Goal
|
|
11
6
|
|
|
12
|
-
|
|
7
|
+
Get OpenClaw into a state where:
|
|
8
|
+
- Oomi managed text chat works
|
|
9
|
+
- Oomi device-backed voice can start cleanly
|
|
10
|
+
- the bridge is supervised or restartable
|
|
11
|
+
- the user never has to paste raw pair identifiers or gateway secrets manually when managed connect is available
|
|
13
12
|
|
|
14
|
-
##
|
|
13
|
+
## Standard Connect Flow
|
|
15
14
|
|
|
16
|
-
If the user asks to connect OpenClaw to Oomi voice
|
|
15
|
+
If the user asks to connect OpenClaw to Oomi chat or voice:
|
|
17
16
|
|
|
18
|
-
1. Refresh bootstrap
|
|
17
|
+
1. Refresh bootstrap first:
|
|
19
18
|
- `curl -fsSL https://www.oomi.ai/install.sh | bash`
|
|
20
|
-
2. Ensure `oomi-ai` is installed
|
|
21
|
-
-
|
|
22
|
-
|
|
19
|
+
2. Ensure `oomi-ai` is installed:
|
|
20
|
+
- `pnpm add -g oomi-ai@latest`
|
|
21
|
+
- fallback: `npm install -g oomi-ai@latest`
|
|
22
|
+
3. Pair the device without starting a legacy one-off flow:
|
|
23
23
|
- `oomi openclaw pair --app-url https://www.oomi.ai --no-start`
|
|
24
|
-
4. Install OpenClaw plugin:
|
|
24
|
+
4. Install the OpenClaw plugin:
|
|
25
25
|
- `openclaw plugins install oomi-ai@latest`
|
|
26
|
-
5.
|
|
26
|
+
5. Print the config block and secrets:
|
|
27
27
|
- `oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai`
|
|
28
|
-
6.
|
|
29
|
-
7.
|
|
28
|
+
6. Apply the printed `channels.oomi.accounts.default` config.
|
|
29
|
+
7. Restart OpenClaw.
|
|
30
|
+
8. Start or repair the bridge:
|
|
30
31
|
- `oomi openclaw bridge ensure --detach`
|
|
31
|
-
-
|
|
32
|
-
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
32
|
+
- if stale: `oomi openclaw bridge restart --detach`
|
|
33
|
+
- on macOS, prefer supervision: `oomi openclaw bridge service install`
|
|
34
|
+
|
|
35
|
+
If the user provides a specific app URL, use it directly in the pair flow:
|
|
36
|
+
- `oomi openclaw pair --app-url <URL> --no-start`
|
|
37
|
+
|
|
38
|
+
## Required User-Facing Result
|
|
39
|
+
|
|
40
|
+
When pairing completes successfully, return this exact format:
|
|
41
|
+
- `Oomi Connect Ready`
|
|
42
|
+
- `Auth Link: ...`
|
|
43
|
+
|
|
44
|
+
Do not add extra narrative before or after those lines.
|
|
45
|
+
Do not ask the user to enter raw pair codes, gateway passwords, or nonce values manually.
|
|
46
|
+
|
|
47
|
+
## Runtime Checks
|
|
48
|
+
|
|
49
|
+
Use these commands to inspect the installed machine:
|
|
50
|
+
|
|
51
|
+
```bash
|
|
52
|
+
oomi openclaw bridge ps
|
|
53
|
+
oomi openclaw bridge service status
|
|
54
|
+
oomi openclaw status
|
|
55
|
+
tail -f ~/.openclaw/logs/oomi-bridge-live.log
|
|
56
|
+
tail -f ~/.openclaw/logs/gateway.log
|
|
57
|
+
tail -f ~/.openclaw/logs/gateway.err.log
|
|
58
|
+
```
|
|
59
|
+
|
|
60
|
+
Useful local files:
|
|
61
|
+
- `~/.openclaw/oomi-bridge-status.json`
|
|
62
|
+
- `~/.openclaw/logs/oomi-bridge-live.log`
|
|
63
|
+
- `~/.openclaw/logs/gateway.log`
|
|
64
|
+
- `~/.openclaw/logs/gateway.err.log`
|
|
65
|
+
- `~/.openclaw/agents/main/sessions/*.jsonl`
|
|
66
|
+
|
|
67
|
+
## Healthy State
|
|
68
|
+
|
|
69
|
+
Treat the machine as healthy when all of the following are true:
|
|
70
|
+
- OpenClaw loads the `oomi-ai` plugin without duplicate-id conflicts
|
|
71
|
+
- `channels.oomi.accounts.default` is populated with a valid `backendUrl` and `deviceToken`
|
|
72
|
+
- the bridge shows `connected` after managed subscription is confirmed
|
|
73
|
+
- text chat reaches the Oomi assistant
|
|
74
|
+
- voice STT can produce `asr.final`
|
|
75
|
+
- assistant replies can come back without the bridge dropping into `stopped`
|
|
76
|
+
|
|
77
|
+
Bridge status meanings:
|
|
78
|
+
- `starting`: bridge booting or waiting for managed subscription
|
|
79
|
+
- `connected`: ready for managed chat and voice traffic
|
|
80
|
+
- `reconnecting`: transport dropped and retry is scheduled
|
|
81
|
+
- `degraded`: bridge caught a runtime fault but is still alive
|
|
82
|
+
- `error`: startup/auth failure blocked useful operation
|
|
83
|
+
- `stopped`: not running or intentionally stopped
|
|
84
|
+
|
|
85
|
+
## Troubleshooting
|
|
86
|
+
|
|
87
|
+
### Duplicate plugin id warning
|
|
88
|
+
|
|
89
|
+
Symptom:
|
|
90
|
+
- OpenClaw reports `duplicate plugin id detected`
|
|
91
|
+
|
|
92
|
+
Action:
|
|
93
|
+
- ensure only one active `oomi-ai` plugin install is discoverable
|
|
94
|
+
- remove stale extension copies before reinstalling
|
|
95
|
+
|
|
96
|
+
### `invalid handshake: first request must be connect`
|
|
97
|
+
|
|
98
|
+
Meaning:
|
|
99
|
+
- a gateway request was sent before `connect` had been accepted
|
|
100
|
+
|
|
101
|
+
Action:
|
|
102
|
+
- update `oomi-ai`
|
|
103
|
+
- restart the bridge
|
|
104
|
+
- confirm only one bridge worker is running
|
|
105
|
+
|
|
106
|
+
### Device is linked but voice start still fails
|
|
107
|
+
|
|
108
|
+
Meaning:
|
|
109
|
+
- linked ownership is not enough; the device side still needs to be live
|
|
110
|
+
|
|
111
|
+
Action:
|
|
112
|
+
- confirm the device websocket is actually online
|
|
113
|
+
- confirm the bridge is `connected`
|
|
114
|
+
- restart the bridge if it is stuck in `reconnecting` or `degraded`
|
|
115
|
+
|
|
116
|
+
### STT works but the assistant does not reply
|
|
117
|
+
|
|
118
|
+
Meaning:
|
|
119
|
+
- the voice turn likely reached Oomi, but the managed gateway or OpenClaw run failed later
|
|
120
|
+
|
|
121
|
+
Action:
|
|
122
|
+
- inspect `gateway.log`, `gateway.err.log`, and the session JSONL
|
|
123
|
+
- check for `network_error`, auth failures, or repeated bridge restarts
|
|
124
|
+
|
|
125
|
+
### Bridge keeps restarting with `reason: stopped`
|
|
126
|
+
|
|
127
|
+
Action:
|
|
128
|
+
- confirm the newest `oomi-ai` is installed
|
|
129
|
+
- inspect `~/.openclaw/logs/oomi-bridge-live.log` for runtime exceptions
|
|
130
|
+
- use supervised mode on macOS: `oomi openclaw bridge service install`
|
|
131
|
+
- if the process is alive but faulted, expect `degraded` rather than an immediate hard stop on newer bridge builds
|
|
132
|
+
|
|
133
|
+
## Voice Notes
|
|
134
|
+
|
|
135
|
+
Voice depends on the same Oomi plugin and bridge layer as managed chat.
|
|
136
|
+
That means:
|
|
137
|
+
- if plugin install or bridge health is wrong, voice replies will also fail
|
|
138
|
+
- STT can succeed even when assistant reply delivery is broken later in the run
|
|
139
|
+
- a `voice_session_*` failure should be investigated, but it should not automatically be treated as proof that all normal Oomi chat is down
|
|
140
|
+
|
|
141
|
+
## Avatar Commands
|
|
142
|
+
|
|
143
|
+
Before using avatar commands, call `get_avatar_capabilities` and prefer canonical values.
|
|
144
|
+
Inline tags are stripped from user-visible text.
|
|
145
|
+
|
|
146
|
+
Use inline tags like:
|
|
147
|
+
- `[anim:Waving]`, `[anim:Walking]`, `[anim:Idle]`, `[anim:Sitting Idle]`
|
|
148
|
+
- `[face:happy]`, `[face:sad]`, `[face:surprised]`, `[face:focused]`, `[face:gentle]`, `[face:thinking]`
|
|
149
|
+
- `[gesture:nod]`, `[gesture:think]`, `[gesture:shrug]`, `[gesture:wave]`, `[gesture:bow]`
|
|
150
|
+
- `[look:camera]`, `[look:left]`, `[look:right]`, `[look:up]`, `[look:down]`
|
|
151
|
+
|
|
152
|
+
Aliases allowed if needed:
|
|
153
|
+
- `wave -> Waving`
|
|
154
|
+
- `walk -> Walking`
|
|
155
|
+
- `idle -> Idle`
|
|
156
|
+
- `sit` or `sitting -> Sitting Idle`
|
package/bin/oomi-ai.js
CHANGED
|
@@ -8,7 +8,7 @@ import net from 'net';
|
|
|
8
8
|
import { lookup as dnsLookup } from 'dns/promises';
|
|
9
9
|
import { fileURLToPath } from 'url';
|
|
10
10
|
import WebSocket from 'ws';
|
|
11
|
-
import { ensureSessionBridge,
|
|
11
|
+
import { ensureSessionBridge, flushSessionQueue, flushWaitingForConnect, forwardFrameToSession } from './sessionBridgeState.js';
|
|
12
12
|
|
|
13
13
|
const MARKER_START = '<oomi-agent-instructions>';
|
|
14
14
|
const MARKER_END = '</oomi-agent-instructions>';
|
|
@@ -637,6 +637,59 @@ function updateBridgeStatus(partial) {
|
|
|
637
637
|
return next;
|
|
638
638
|
}
|
|
639
639
|
|
|
640
|
+
function resolveBridgeStatusForBrokerOpen({ actionCableMode, deviceSubscribed }) {
|
|
641
|
+
if (!actionCableMode) {
|
|
642
|
+
return 'connected';
|
|
643
|
+
}
|
|
644
|
+
return deviceSubscribed ? 'connected' : 'starting';
|
|
645
|
+
}
|
|
646
|
+
|
|
647
|
+
function classifyBridgeSessionScope(sessionId) {
|
|
648
|
+
const normalized = String(sessionId || '').trim();
|
|
649
|
+
return normalized.startsWith('voice_session_') ? 'voice' : 'default';
|
|
650
|
+
}
|
|
651
|
+
|
|
652
|
+
function resolveBridgeStatusForRuntimeFault({ currentStatus, sessionId }) {
|
|
653
|
+
if (classifyBridgeSessionScope(sessionId) === 'voice') {
|
|
654
|
+
return currentStatus === 'connected' ? 'connected' : currentStatus || 'starting';
|
|
655
|
+
}
|
|
656
|
+
if (currentStatus === 'connected' || currentStatus === 'reconnecting' || currentStatus === 'degraded') {
|
|
657
|
+
return 'degraded';
|
|
658
|
+
}
|
|
659
|
+
return 'error';
|
|
660
|
+
}
|
|
661
|
+
|
|
662
|
+
function runBridgeCallbackSafely(callback, onError) {
|
|
663
|
+
return (...args) => {
|
|
664
|
+
try {
|
|
665
|
+
return callback(...args);
|
|
666
|
+
} catch (err) {
|
|
667
|
+
onError(err instanceof Error ? err : new Error(String(err)));
|
|
668
|
+
return undefined;
|
|
669
|
+
}
|
|
670
|
+
};
|
|
671
|
+
}
|
|
672
|
+
|
|
673
|
+
function createBridgeProcessFaultHandler({ readStatus, onReport, onExit }) {
|
|
674
|
+
return ({ phase, error }) => {
|
|
675
|
+
const normalizedError = error instanceof Error ? error : new Error(String(error || 'unknown process fault'));
|
|
676
|
+
const currentStatus = String(readStatus?.()?.status || '').trim();
|
|
677
|
+
const nextStatus = resolveBridgeStatusForRuntimeFault({ currentStatus, sessionId: '' });
|
|
678
|
+
|
|
679
|
+
onReport?.({
|
|
680
|
+
phase,
|
|
681
|
+
status: nextStatus,
|
|
682
|
+
error: normalizedError,
|
|
683
|
+
currentStatus,
|
|
684
|
+
shouldExit: nextStatus === 'error',
|
|
685
|
+
});
|
|
686
|
+
|
|
687
|
+
if (nextStatus === 'error') {
|
|
688
|
+
onExit?.(1);
|
|
689
|
+
}
|
|
690
|
+
};
|
|
691
|
+
}
|
|
692
|
+
|
|
640
693
|
function normalizeBridgeMetrics(value) {
|
|
641
694
|
if (!value || typeof value !== 'object') return {};
|
|
642
695
|
const next = {};
|
|
@@ -945,7 +998,64 @@ function prepareGatewayFrameForLocalGateway(frameText, gatewayAuth, options = {}
|
|
|
945
998
|
|
|
946
999
|
try {
|
|
947
1000
|
const frame = JSON.parse(frameText);
|
|
948
|
-
if (frame?.type !== 'req'
|
|
1001
|
+
if (frame?.type !== 'req') {
|
|
1002
|
+
return { frameText, waitForChallenge: false };
|
|
1003
|
+
}
|
|
1004
|
+
const method = typeof frame.method === 'string' ? frame.method.trim() : '';
|
|
1005
|
+
if (!method) {
|
|
1006
|
+
return { frameText, waitForChallenge: false };
|
|
1007
|
+
}
|
|
1008
|
+
|
|
1009
|
+
if (method !== 'connect') {
|
|
1010
|
+
const rawParams = frame.params && typeof frame.params === 'object' ? frame.params : {};
|
|
1011
|
+
if (method === 'chat.send') {
|
|
1012
|
+
const sanitized = {};
|
|
1013
|
+
if (typeof rawParams.sessionKey === 'string' && rawParams.sessionKey.trim()) {
|
|
1014
|
+
sanitized.sessionKey = rawParams.sessionKey.trim();
|
|
1015
|
+
}
|
|
1016
|
+
if (typeof rawParams.message === 'string') {
|
|
1017
|
+
sanitized.message = rawParams.message;
|
|
1018
|
+
}
|
|
1019
|
+
if (typeof rawParams.thinking === 'boolean') {
|
|
1020
|
+
sanitized.thinking = rawParams.thinking;
|
|
1021
|
+
}
|
|
1022
|
+
if (typeof rawParams.deliver === 'string' && rawParams.deliver.trim()) {
|
|
1023
|
+
sanitized.deliver = rawParams.deliver.trim();
|
|
1024
|
+
}
|
|
1025
|
+
if (Array.isArray(rawParams.attachments)) {
|
|
1026
|
+
sanitized.attachments = rawParams.attachments;
|
|
1027
|
+
}
|
|
1028
|
+
if (Number.isFinite(rawParams.timeoutMs) && rawParams.timeoutMs > 0) {
|
|
1029
|
+
sanitized.timeoutMs = Math.floor(rawParams.timeoutMs);
|
|
1030
|
+
}
|
|
1031
|
+
|
|
1032
|
+
const idempotencyKeyCandidates = [
|
|
1033
|
+
rawParams.idempotencyKey,
|
|
1034
|
+
rawParams.requestId,
|
|
1035
|
+
rawParams.correlationId,
|
|
1036
|
+
frame.id,
|
|
1037
|
+
];
|
|
1038
|
+
const idempotencyKey = idempotencyKeyCandidates.find((value) => typeof value === 'string' && value.trim());
|
|
1039
|
+
if (typeof idempotencyKey === 'string' && idempotencyKey.trim()) {
|
|
1040
|
+
sanitized.idempotencyKey = idempotencyKey.trim();
|
|
1041
|
+
}
|
|
1042
|
+
|
|
1043
|
+
frame.params = sanitized;
|
|
1044
|
+
return { frameText: JSON.stringify(frame), waitForChallenge: false };
|
|
1045
|
+
}
|
|
1046
|
+
|
|
1047
|
+
if (method === 'chat.history') {
|
|
1048
|
+
const sanitized = {};
|
|
1049
|
+
if (typeof rawParams.sessionKey === 'string' && rawParams.sessionKey.trim()) {
|
|
1050
|
+
sanitized.sessionKey = rawParams.sessionKey.trim();
|
|
1051
|
+
}
|
|
1052
|
+
if (Number.isFinite(rawParams.limit) && rawParams.limit > 0) {
|
|
1053
|
+
sanitized.limit = Math.floor(rawParams.limit);
|
|
1054
|
+
}
|
|
1055
|
+
frame.params = sanitized;
|
|
1056
|
+
return { frameText: JSON.stringify(frame), waitForChallenge: false };
|
|
1057
|
+
}
|
|
1058
|
+
|
|
949
1059
|
return { frameText, waitForChallenge: false };
|
|
950
1060
|
}
|
|
951
1061
|
|
|
@@ -1720,6 +1830,82 @@ async function startOpenclawBridge(flags) {
|
|
|
1720
1830
|
})();
|
|
1721
1831
|
const actionCableMode = brokerPath.endsWith('/cable');
|
|
1722
1832
|
const deviceChannelIdentifier = JSON.stringify({ channel: 'DeviceChannel' });
|
|
1833
|
+
let deviceChannelSubscribed = false;
|
|
1834
|
+
|
|
1835
|
+
const reportBridgeRuntimeFault = ({ phase, sessionId = '', error }) => {
|
|
1836
|
+
const message = error instanceof Error ? error.message : String(error || 'unknown bridge callback error');
|
|
1837
|
+
const currentStatus = String(readBridgeStatus().status || '').trim();
|
|
1838
|
+
const nextStatus = resolveBridgeStatusForRuntimeFault({ currentStatus, sessionId });
|
|
1839
|
+
incrementBridgeMetric('bridge_callback_error_count');
|
|
1840
|
+
console.error(
|
|
1841
|
+
`[bridge] callback.error phase=${phase}${sessionId ? ` session=${sessionId}` : ''}: ${message}`
|
|
1842
|
+
);
|
|
1843
|
+
updateBridgeStatus({
|
|
1844
|
+
status: nextStatus,
|
|
1845
|
+
deviceId,
|
|
1846
|
+
brokerWs,
|
|
1847
|
+
brokerHttp,
|
|
1848
|
+
gatewayUrl: gateway.gatewayUrl,
|
|
1849
|
+
lastDisconnectAt: bridgeNowIso(),
|
|
1850
|
+
lastErrorCode: 'bridge_callback_error',
|
|
1851
|
+
lastErrorClass: 'internal',
|
|
1852
|
+
lastErrorMessage: `${phase}: ${message}`,
|
|
1853
|
+
hint:
|
|
1854
|
+
classifyBridgeSessionScope(sessionId) === 'voice'
|
|
1855
|
+
? 'A voice-session bridge callback failed, but provider health remains available for normal chat.'
|
|
1856
|
+
: 'The bridge caught an internal callback error and kept running.',
|
|
1857
|
+
pid: process.pid,
|
|
1858
|
+
});
|
|
1859
|
+
};
|
|
1860
|
+
|
|
1861
|
+
const reportBridgeProcessFault = ({ phase, status, error }) => {
|
|
1862
|
+
const message = error instanceof Error ? error.message : String(error || 'unknown bridge process fault');
|
|
1863
|
+
incrementBridgeMetric('bridge_process_fault_count');
|
|
1864
|
+
console.error(`[bridge] process.error phase=${phase}: ${message}`);
|
|
1865
|
+
updateBridgeStatus({
|
|
1866
|
+
status,
|
|
1867
|
+
deviceId,
|
|
1868
|
+
brokerWs,
|
|
1869
|
+
brokerHttp,
|
|
1870
|
+
gatewayUrl: gateway.gatewayUrl,
|
|
1871
|
+
lastDisconnectAt: bridgeNowIso(),
|
|
1872
|
+
lastErrorCode: 'bridge_process_fault',
|
|
1873
|
+
lastErrorClass: 'internal',
|
|
1874
|
+
lastErrorMessage: `${phase}: ${message}`,
|
|
1875
|
+
hint:
|
|
1876
|
+
status === 'error'
|
|
1877
|
+
? 'The bridge hit a runtime fault before it was fully connected and will restart.'
|
|
1878
|
+
: 'The bridge caught a process-level runtime fault and stayed alive in degraded mode.',
|
|
1879
|
+
pid: process.pid,
|
|
1880
|
+
});
|
|
1881
|
+
};
|
|
1882
|
+
|
|
1883
|
+
const handleBridgeProcessFault = createBridgeProcessFaultHandler({
|
|
1884
|
+
readStatus: readBridgeStatus,
|
|
1885
|
+
onReport: ({ phase, status, error }) => {
|
|
1886
|
+
reportBridgeProcessFault({ phase, status, error });
|
|
1887
|
+
},
|
|
1888
|
+
onExit: (code) => {
|
|
1889
|
+
reconnectState.stopped = true;
|
|
1890
|
+
if (reconnectState.timer) {
|
|
1891
|
+
clearTimeout(reconnectState.timer);
|
|
1892
|
+
reconnectState.timer = null;
|
|
1893
|
+
}
|
|
1894
|
+
releaseBridgeLock();
|
|
1895
|
+
process.exit(code);
|
|
1896
|
+
},
|
|
1897
|
+
});
|
|
1898
|
+
|
|
1899
|
+
const uncaughtExceptionHandler = (error) => {
|
|
1900
|
+
handleBridgeProcessFault({ phase: 'process.uncaughtException', error });
|
|
1901
|
+
};
|
|
1902
|
+
|
|
1903
|
+
const unhandledRejectionHandler = (reason) => {
|
|
1904
|
+
handleBridgeProcessFault({ phase: 'process.unhandledRejection', error: reason });
|
|
1905
|
+
};
|
|
1906
|
+
|
|
1907
|
+
process.on('uncaughtException', uncaughtExceptionHandler);
|
|
1908
|
+
process.on('unhandledRejection', unhandledRejectionHandler);
|
|
1723
1909
|
|
|
1724
1910
|
const sendBrokerPayload = (brokerSocket, payload) => {
|
|
1725
1911
|
if (brokerSocket.readyState !== WebSocket.OPEN) return;
|
|
@@ -2157,6 +2343,12 @@ async function startOpenclawBridge(flags) {
|
|
|
2157
2343
|
if (!(sessionBridge.pendingRequestTimers instanceof Map)) {
|
|
2158
2344
|
sessionBridge.pendingRequestTimers = new Map();
|
|
2159
2345
|
}
|
|
2346
|
+
if (sessionBridge.connectAccepted !== true) {
|
|
2347
|
+
sessionBridge.connectAccepted = false;
|
|
2348
|
+
}
|
|
2349
|
+
if (!Array.isArray(sessionBridge.waitingForConnect)) {
|
|
2350
|
+
sessionBridge.waitingForConnect = [];
|
|
2351
|
+
}
|
|
2160
2352
|
if (typeof sessionBridge.lastChatCorrelationId !== 'string') {
|
|
2161
2353
|
sessionBridge.lastChatCorrelationId = '';
|
|
2162
2354
|
}
|
|
@@ -2189,7 +2381,7 @@ async function startOpenclawBridge(flags) {
|
|
|
2189
2381
|
flushSessionQueue(sessionBridge);
|
|
2190
2382
|
});
|
|
2191
2383
|
|
|
2192
|
-
gatewaySocket.on('message', (gatewayRaw) => {
|
|
2384
|
+
gatewaySocket.on('message', runBridgeCallbackSafely((gatewayRaw) => {
|
|
2193
2385
|
const frame = typeof gatewayRaw === 'string' ? gatewayRaw : gatewayRaw.toString();
|
|
2194
2386
|
const gatewayPayload = parseJsonPayload(frame);
|
|
2195
2387
|
if (gatewayPayload?.event === 'connect.challenge') {
|
|
@@ -2230,6 +2422,86 @@ async function startOpenclawBridge(flags) {
|
|
|
2230
2422
|
correlationId: requestMeta.correlationId,
|
|
2231
2423
|
stage: responseMeta.ok ? 'gateway.accepted' : 'gateway.rejected',
|
|
2232
2424
|
});
|
|
2425
|
+
if (requestMeta.method === 'connect' && responseMeta.ok) {
|
|
2426
|
+
const releasedFrames = flushWaitingForConnect(sessionBridge);
|
|
2427
|
+
for (const released of releasedFrames) {
|
|
2428
|
+
const releasedMeta = extractGatewayRequestMeta(released.frameText);
|
|
2429
|
+
if (!releasedMeta) continue;
|
|
2430
|
+
|
|
2431
|
+
if (released.result === 'queued') {
|
|
2432
|
+
startPendingRequestTimeout(brokerSocket, sessionId, sessionBridge, releasedMeta);
|
|
2433
|
+
sendGatewayAck(brokerSocket, {
|
|
2434
|
+
sessionId,
|
|
2435
|
+
requestId: releasedMeta.requestId,
|
|
2436
|
+
method: releasedMeta.method,
|
|
2437
|
+
correlationId: releasedMeta.correlationId,
|
|
2438
|
+
stage: 'bridge.queued',
|
|
2439
|
+
});
|
|
2440
|
+
} else if (released.result === 'dropped') {
|
|
2441
|
+
const pending = sessionBridge.pendingRequests.get(releasedMeta.requestId);
|
|
2442
|
+
const lastSuccessfulHop =
|
|
2443
|
+
pending && typeof pending.lastSuccessfulHop === 'string' && pending.lastSuccessfulHop
|
|
2444
|
+
? pending.lastSuccessfulHop
|
|
2445
|
+
: 'bridge.waiting_for_connect';
|
|
2446
|
+
clearPendingRequestTimeout(sessionBridge, releasedMeta.requestId);
|
|
2447
|
+
sessionBridge.pendingRequests.delete(releasedMeta.requestId);
|
|
2448
|
+
sendGatewayErrorResponse(brokerSocket, {
|
|
2449
|
+
sessionId,
|
|
2450
|
+
requestMeta: releasedMeta,
|
|
2451
|
+
code: 'bridge_dropped',
|
|
2452
|
+
message: 'Bridge dropped deferred request because gateway socket is not open.',
|
|
2453
|
+
lastSuccessfulHop,
|
|
2454
|
+
retryable: true,
|
|
2455
|
+
});
|
|
2456
|
+
sendGatewayAck(brokerSocket, {
|
|
2457
|
+
sessionId,
|
|
2458
|
+
requestId: releasedMeta.requestId,
|
|
2459
|
+
method: releasedMeta.method,
|
|
2460
|
+
correlationId: releasedMeta.correlationId,
|
|
2461
|
+
stage: 'bridge.dropped',
|
|
2462
|
+
});
|
|
2463
|
+
} else {
|
|
2464
|
+
startPendingRequestTimeout(brokerSocket, sessionId, sessionBridge, releasedMeta);
|
|
2465
|
+
sendGatewayAck(brokerSocket, {
|
|
2466
|
+
sessionId,
|
|
2467
|
+
requestId: releasedMeta.requestId,
|
|
2468
|
+
method: releasedMeta.method,
|
|
2469
|
+
correlationId: releasedMeta.correlationId,
|
|
2470
|
+
stage: 'bridge.forwarded',
|
|
2471
|
+
});
|
|
2472
|
+
}
|
|
2473
|
+
}
|
|
2474
|
+
} else if (requestMeta.method === 'connect' && !responseMeta.ok) {
|
|
2475
|
+
const deferredFrames = Array.isArray(sessionBridge.waitingForConnect)
|
|
2476
|
+
? sessionBridge.waitingForConnect.splice(0, sessionBridge.waitingForConnect.length)
|
|
2477
|
+
: [];
|
|
2478
|
+
for (const deferredFrame of deferredFrames) {
|
|
2479
|
+
const deferredMeta = extractGatewayRequestMeta(deferredFrame);
|
|
2480
|
+
if (!deferredMeta) continue;
|
|
2481
|
+
const pending = sessionBridge.pendingRequests.get(deferredMeta.requestId);
|
|
2482
|
+
const lastSuccessfulHop =
|
|
2483
|
+
pending && typeof pending.lastSuccessfulHop === 'string' && pending.lastSuccessfulHop
|
|
2484
|
+
? pending.lastSuccessfulHop
|
|
2485
|
+
: 'bridge.waiting_for_connect';
|
|
2486
|
+
clearPendingRequestTimeout(sessionBridge, deferredMeta.requestId);
|
|
2487
|
+
sessionBridge.pendingRequests.delete(deferredMeta.requestId);
|
|
2488
|
+
sendGatewayErrorResponse(brokerSocket, {
|
|
2489
|
+
sessionId,
|
|
2490
|
+
requestMeta: deferredMeta,
|
|
2491
|
+
code: 'gateway_connect_failed',
|
|
2492
|
+
message: 'Bridge could not forward request because gateway connect did not complete.',
|
|
2493
|
+
lastSuccessfulHop,
|
|
2494
|
+
retryable: true,
|
|
2495
|
+
});
|
|
2496
|
+
sendGatewayAck(brokerSocket, {
|
|
2497
|
+
sessionId,
|
|
2498
|
+
requestId: deferredMeta.requestId,
|
|
2499
|
+
method: deferredMeta.method,
|
|
2500
|
+
correlationId: deferredMeta.correlationId,
|
|
2501
|
+
stage: 'gateway.rejected',
|
|
2502
|
+
});
|
|
2503
|
+
}
|
|
2504
|
+
}
|
|
2233
2505
|
if (!responseMeta.ok) {
|
|
2234
2506
|
incrementBridgeMetric('gateway_rejected_count');
|
|
2235
2507
|
}
|
|
@@ -2247,9 +2519,11 @@ async function startOpenclawBridge(flags) {
|
|
|
2247
2519
|
}
|
|
2248
2520
|
|
|
2249
2521
|
sendBrokerPayload(brokerSocket, { action: 'gateway_frame', type: 'gateway.frame', sessionId, frame });
|
|
2250
|
-
})
|
|
2522
|
+
}, (error) => {
|
|
2523
|
+
reportBridgeRuntimeFault({ phase: 'gateway.message', sessionId, error });
|
|
2524
|
+
}));
|
|
2251
2525
|
|
|
2252
|
-
gatewaySocket.on('close', (code, reason) => {
|
|
2526
|
+
gatewaySocket.on('close', runBridgeCallbackSafely((code, reason) => {
|
|
2253
2527
|
if (connectTimeout) {
|
|
2254
2528
|
clearTimeout(connectTimeout);
|
|
2255
2529
|
connectTimeout = null;
|
|
@@ -2299,9 +2573,11 @@ async function startOpenclawBridge(flags) {
|
|
|
2299
2573
|
code,
|
|
2300
2574
|
reason: reasonText,
|
|
2301
2575
|
});
|
|
2302
|
-
})
|
|
2576
|
+
}, (error) => {
|
|
2577
|
+
reportBridgeRuntimeFault({ phase: 'gateway.close', sessionId, error });
|
|
2578
|
+
}));
|
|
2303
2579
|
|
|
2304
|
-
gatewaySocket.on('error', (err) => {
|
|
2580
|
+
gatewaySocket.on('error', runBridgeCallbackSafely((err) => {
|
|
2305
2581
|
if (connectTimeout && gatewaySocket.readyState !== WebSocket.CONNECTING) {
|
|
2306
2582
|
clearTimeout(connectTimeout);
|
|
2307
2583
|
connectTimeout = null;
|
|
@@ -2314,7 +2590,9 @@ async function startOpenclawBridge(flags) {
|
|
|
2314
2590
|
level: 'error',
|
|
2315
2591
|
message: `Gateway socket error (${sessionId}): ${String(err)}`,
|
|
2316
2592
|
});
|
|
2317
|
-
})
|
|
2593
|
+
}, (error) => {
|
|
2594
|
+
reportBridgeRuntimeFault({ phase: 'gateway.error', sessionId, error });
|
|
2595
|
+
}));
|
|
2318
2596
|
};
|
|
2319
2597
|
|
|
2320
2598
|
const getOrCreateGatewaySession = (sessionId) => {
|
|
@@ -2333,18 +2611,22 @@ async function startOpenclawBridge(flags) {
|
|
|
2333
2611
|
console.log('[bridge] Connected to managed broker.');
|
|
2334
2612
|
reconnectState.attempt = 0;
|
|
2335
2613
|
reconnectState.lastFailure = null;
|
|
2336
|
-
if (
|
|
2337
|
-
|
|
2338
|
-
|
|
2339
|
-
|
|
2340
|
-
|
|
2341
|
-
|
|
2342
|
-
|
|
2343
|
-
|
|
2344
|
-
|
|
2345
|
-
|
|
2614
|
+
if (actionCableMode) {
|
|
2615
|
+
brokerSocket.send(
|
|
2616
|
+
JSON.stringify({
|
|
2617
|
+
command: 'subscribe',
|
|
2618
|
+
identifier: deviceChannelIdentifier,
|
|
2619
|
+
})
|
|
2620
|
+
);
|
|
2621
|
+
actionCableHeartbeat = setInterval(() => {
|
|
2622
|
+
sendBrokerPayload(brokerSocket, { action: 'heartbeat' });
|
|
2623
|
+
}, 15000);
|
|
2624
|
+
}
|
|
2346
2625
|
updateBridgeStatus({
|
|
2347
|
-
status:
|
|
2626
|
+
status: resolveBridgeStatusForBrokerOpen({
|
|
2627
|
+
actionCableMode,
|
|
2628
|
+
deviceSubscribed: deviceChannelSubscribed,
|
|
2629
|
+
}),
|
|
2348
2630
|
deviceId,
|
|
2349
2631
|
brokerWs,
|
|
2350
2632
|
brokerHttp,
|
|
@@ -2359,12 +2641,27 @@ async function startOpenclawBridge(flags) {
|
|
|
2359
2641
|
});
|
|
2360
2642
|
});
|
|
2361
2643
|
|
|
2362
|
-
brokerSocket.on('message', (rawData) => {
|
|
2644
|
+
brokerSocket.on('message', runBridgeCallbackSafely((rawData) => {
|
|
2363
2645
|
const text = typeof rawData === 'string' ? rawData : rawData.toString();
|
|
2364
2646
|
const payload = parseBrokerEnvelope(text);
|
|
2365
2647
|
if (!payload || typeof payload.type !== 'string') return;
|
|
2366
2648
|
|
|
2367
2649
|
if (payload.type === 'device.subscribed') {
|
|
2650
|
+
deviceChannelSubscribed = true;
|
|
2651
|
+
updateBridgeStatus({
|
|
2652
|
+
status: 'connected',
|
|
2653
|
+
deviceId,
|
|
2654
|
+
brokerWs,
|
|
2655
|
+
brokerHttp,
|
|
2656
|
+
gatewayUrl: gateway.gatewayUrl,
|
|
2657
|
+
lastConnectedAt: bridgeNowIso(),
|
|
2658
|
+
lastErrorCode: '',
|
|
2659
|
+
lastErrorClass: '',
|
|
2660
|
+
lastErrorMessage: '',
|
|
2661
|
+
hint: '',
|
|
2662
|
+
consecutiveFailures: 0,
|
|
2663
|
+
pid: process.pid,
|
|
2664
|
+
});
|
|
2368
2665
|
return;
|
|
2369
2666
|
}
|
|
2370
2667
|
|
|
@@ -2511,7 +2808,22 @@ async function startOpenclawBridge(flags) {
|
|
|
2511
2808
|
}
|
|
2512
2809
|
return;
|
|
2513
2810
|
}
|
|
2514
|
-
const result = forwardFrameToSession(sessionBridge, prepared.frameText
|
|
2811
|
+
const result = forwardFrameToSession(sessionBridge, prepared.frameText, {
|
|
2812
|
+
requiresConnectAccepted: Boolean(requestMeta && requestMeta.method !== 'connect'),
|
|
2813
|
+
});
|
|
2814
|
+
if (result === 'waiting_for_connect') {
|
|
2815
|
+
console.log(`[bridge] client.frame waiting for connect ${sessionId}`);
|
|
2816
|
+
if (requestMeta) {
|
|
2817
|
+
sendGatewayAck(brokerSocket, {
|
|
2818
|
+
sessionId,
|
|
2819
|
+
requestId: requestMeta.requestId,
|
|
2820
|
+
method: requestMeta.method,
|
|
2821
|
+
correlationId: requestMeta.correlationId,
|
|
2822
|
+
stage: 'bridge.waiting_for_connect',
|
|
2823
|
+
});
|
|
2824
|
+
}
|
|
2825
|
+
return;
|
|
2826
|
+
}
|
|
2515
2827
|
if (result === 'queued') {
|
|
2516
2828
|
console.log(`[bridge] client.frame queued ${sessionId}`);
|
|
2517
2829
|
if (requestMeta) {
|
|
@@ -2586,9 +2898,11 @@ async function startOpenclawBridge(flags) {
|
|
|
2586
2898
|
}
|
|
2587
2899
|
return;
|
|
2588
2900
|
}
|
|
2589
|
-
})
|
|
2901
|
+
}, (error) => {
|
|
2902
|
+
reportBridgeRuntimeFault({ phase: 'broker.message', error });
|
|
2903
|
+
}));
|
|
2590
2904
|
|
|
2591
|
-
brokerSocket.on('close', (code, reason) => {
|
|
2905
|
+
brokerSocket.on('close', runBridgeCallbackSafely((code, reason) => {
|
|
2592
2906
|
if (actionCableHeartbeat) {
|
|
2593
2907
|
clearInterval(actionCableHeartbeat);
|
|
2594
2908
|
actionCableHeartbeat = null;
|
|
@@ -2620,16 +2934,20 @@ async function startOpenclawBridge(flags) {
|
|
|
2620
2934
|
});
|
|
2621
2935
|
}
|
|
2622
2936
|
scheduleReconnect();
|
|
2623
|
-
})
|
|
2937
|
+
}, (error) => {
|
|
2938
|
+
reportBridgeRuntimeFault({ phase: 'broker.close', error });
|
|
2939
|
+
}));
|
|
2624
2940
|
|
|
2625
|
-
brokerSocket.on('error', (err) => {
|
|
2941
|
+
brokerSocket.on('error', runBridgeCallbackSafely((err) => {
|
|
2626
2942
|
incrementBridgeMetric('bridge_socket_error_count');
|
|
2627
2943
|
reconnectState.lastFailure = classifyBridgeFailure({ err });
|
|
2628
2944
|
console.error(
|
|
2629
2945
|
`[bridge] Broker socket error [${reconnectState.lastFailure.failureClass}/${reconnectState.lastFailure.errorCode}]: ${reconnectState.lastFailure.message}`
|
|
2630
2946
|
);
|
|
2631
2947
|
console.error(`[bridge] ${reconnectState.lastFailure.hint}`);
|
|
2632
|
-
})
|
|
2948
|
+
}, (error) => {
|
|
2949
|
+
reportBridgeRuntimeFault({ phase: 'broker.error', error });
|
|
2950
|
+
}));
|
|
2633
2951
|
};
|
|
2634
2952
|
|
|
2635
2953
|
const markStopped = (signal) => {
|
|
@@ -2638,6 +2956,8 @@ async function startOpenclawBridge(flags) {
|
|
|
2638
2956
|
clearTimeout(reconnectState.timer);
|
|
2639
2957
|
reconnectState.timer = null;
|
|
2640
2958
|
}
|
|
2959
|
+
process.off('uncaughtException', uncaughtExceptionHandler);
|
|
2960
|
+
process.off('unhandledRejection', unhandledRejectionHandler);
|
|
2641
2961
|
updateBridgeStatus({
|
|
2642
2962
|
status: 'stopped',
|
|
2643
2963
|
deviceId,
|
|
@@ -3219,7 +3539,12 @@ if (__isDirectExecution) {
|
|
|
3219
3539
|
export {
|
|
3220
3540
|
prepareGatewayFrameForLocalGateway,
|
|
3221
3541
|
classifyBridgeFailure,
|
|
3542
|
+
classifyBridgeSessionScope,
|
|
3543
|
+
createBridgeProcessFaultHandler,
|
|
3222
3544
|
computeReconnectDelayMs,
|
|
3545
|
+
resolveBridgeStatusForBrokerOpen,
|
|
3546
|
+
resolveBridgeStatusForRuntimeFault,
|
|
3547
|
+
runBridgeCallbackSafely,
|
|
3223
3548
|
extractGatewayRequestMeta,
|
|
3224
3549
|
extractGatewayResponseMeta,
|
|
3225
3550
|
isGatewayRunStartedFrame,
|
|
@@ -12,7 +12,12 @@ export function ensureSessionBridge({ sessions, sessionId, createSocket }) {
|
|
|
12
12
|
if (existing) return existing;
|
|
13
13
|
|
|
14
14
|
const socket = createSocket(id);
|
|
15
|
-
const next = {
|
|
15
|
+
const next = {
|
|
16
|
+
socket,
|
|
17
|
+
queue: [],
|
|
18
|
+
connectAccepted: false,
|
|
19
|
+
waitingForConnect: [],
|
|
20
|
+
};
|
|
16
21
|
sessions.set(id, next);
|
|
17
22
|
return next;
|
|
18
23
|
}
|
|
@@ -20,11 +25,19 @@ export function ensureSessionBridge({ sessions, sessionId, createSocket }) {
|
|
|
20
25
|
/**
|
|
21
26
|
* Forward a frame to the gateway socket or queue it while connecting.
|
|
22
27
|
*/
|
|
23
|
-
export function forwardFrameToSession(sessionBridge, frameText) {
|
|
28
|
+
export function forwardFrameToSession(sessionBridge, frameText, options = {}) {
|
|
24
29
|
if (!sessionBridge || !sessionBridge.socket || typeof frameText !== 'string' || !frameText) {
|
|
25
30
|
return 'dropped';
|
|
26
31
|
}
|
|
27
32
|
|
|
33
|
+
if (options.requiresConnectAccepted === true && sessionBridge.connectAccepted !== true) {
|
|
34
|
+
if (!Array.isArray(sessionBridge.waitingForConnect)) {
|
|
35
|
+
sessionBridge.waitingForConnect = [];
|
|
36
|
+
}
|
|
37
|
+
sessionBridge.waitingForConnect.push(frameText);
|
|
38
|
+
return 'waiting_for_connect';
|
|
39
|
+
}
|
|
40
|
+
|
|
28
41
|
const { socket } = sessionBridge;
|
|
29
42
|
if (socket.readyState === WS_OPEN) {
|
|
30
43
|
socket.send(frameText);
|
|
@@ -39,6 +52,20 @@ export function forwardFrameToSession(sessionBridge, frameText) {
|
|
|
39
52
|
return 'dropped';
|
|
40
53
|
}
|
|
41
54
|
|
|
55
|
+
export function flushWaitingForConnect(sessionBridge) {
|
|
56
|
+
if (!sessionBridge) return [];
|
|
57
|
+
|
|
58
|
+
sessionBridge.connectAccepted = true;
|
|
59
|
+
const pending = Array.isArray(sessionBridge.waitingForConnect)
|
|
60
|
+
? sessionBridge.waitingForConnect.splice(0, sessionBridge.waitingForConnect.length)
|
|
61
|
+
: [];
|
|
62
|
+
|
|
63
|
+
return pending.map((frameText) => ({
|
|
64
|
+
frameText,
|
|
65
|
+
result: forwardFrameToSession(sessionBridge, frameText),
|
|
66
|
+
}));
|
|
67
|
+
}
|
|
68
|
+
|
|
42
69
|
export function flushSessionQueue(sessionBridge) {
|
|
43
70
|
if (!sessionBridge || !sessionBridge.socket) return;
|
|
44
71
|
const socket = sessionBridge.socket;
|
package/openclaw.plugin.json
CHANGED
package/package.json
CHANGED
package/skills/oomi/SKILL.md
CHANGED
|
@@ -1,102 +1,140 @@
|
|
|
1
1
|
---
|
|
2
2
|
name: oomi
|
|
3
|
-
description:
|
|
3
|
+
description: Support Oomi OpenClaw installs, bridge health, managed chat and voice setup, and avatar control.
|
|
4
4
|
---
|
|
5
5
|
|
|
6
6
|
# Oomi Skill
|
|
7
7
|
|
|
8
|
-
|
|
8
|
+
Use this skill when you need to:
|
|
9
|
+
- connect an OpenClaw machine to Oomi
|
|
10
|
+
- repair the Oomi plugin or bridge on a machine
|
|
11
|
+
- inspect managed chat or voice health
|
|
12
|
+
- control the Oomi avatar with inline tags
|
|
9
13
|
|
|
10
|
-
##
|
|
14
|
+
## Primary Operator Workflow
|
|
11
15
|
|
|
12
|
-
|
|
16
|
+
If the user wants OpenClaw connected to Oomi:
|
|
13
17
|
|
|
14
|
-
|
|
18
|
+
1. Ensure `oomi-ai` is installed or updated:
|
|
19
|
+
```bash
|
|
20
|
+
pnpm add -g oomi-ai@latest
|
|
21
|
+
```
|
|
22
|
+
Fallback:
|
|
23
|
+
```bash
|
|
24
|
+
npm install -g oomi-ai@latest
|
|
25
|
+
```
|
|
26
|
+
|
|
27
|
+
2. Pair the device:
|
|
28
|
+
```bash
|
|
29
|
+
oomi openclaw pair --app-url https://www.oomi.ai --no-start
|
|
30
|
+
```
|
|
31
|
+
|
|
32
|
+
3. Install the plugin:
|
|
33
|
+
```bash
|
|
34
|
+
openclaw plugins install oomi-ai@latest
|
|
35
|
+
```
|
|
36
|
+
|
|
37
|
+
4. Print config guidance:
|
|
38
|
+
```bash
|
|
39
|
+
oomi openclaw plugin --show-secrets --backend-url https://api.oomi.ai
|
|
40
|
+
```
|
|
41
|
+
|
|
42
|
+
5. Apply the `channels.oomi.accounts.default` config and restart OpenClaw.
|
|
15
43
|
|
|
16
|
-
|
|
17
|
-
```
|
|
18
|
-
|
|
44
|
+
6. Start or repair the bridge:
|
|
45
|
+
```bash
|
|
46
|
+
oomi openclaw bridge ensure --detach
|
|
47
|
+
```
|
|
48
|
+
If stale:
|
|
49
|
+
```bash
|
|
50
|
+
oomi openclaw bridge restart --detach
|
|
51
|
+
```
|
|
52
|
+
On macOS, prefer supervised mode:
|
|
53
|
+
```bash
|
|
54
|
+
oomi openclaw bridge service install
|
|
19
55
|
```
|
|
20
|
-
Default URL is `http://localhost:3000/api/skill`.
|
|
21
56
|
|
|
22
|
-
##
|
|
57
|
+
## Health Checks
|
|
58
|
+
|
|
59
|
+
Use these when chat or voice is failing:
|
|
60
|
+
|
|
61
|
+
```bash
|
|
62
|
+
oomi openclaw bridge ps
|
|
63
|
+
oomi openclaw bridge service status
|
|
64
|
+
oomi openclaw status
|
|
65
|
+
tail -f ~/.openclaw/logs/oomi-bridge-live.log
|
|
66
|
+
tail -f ~/.openclaw/logs/gateway.log
|
|
67
|
+
tail -f ~/.openclaw/logs/gateway.err.log
|
|
68
|
+
```
|
|
69
|
+
|
|
70
|
+
Interpret bridge states like this:
|
|
71
|
+
- `starting`: booting or waiting for managed subscription
|
|
72
|
+
- `connected`: ready for managed traffic
|
|
73
|
+
- `reconnecting`: retry scheduled after transport failure
|
|
74
|
+
- `degraded`: bridge caught a runtime fault but is still alive
|
|
75
|
+
- `error`: startup or auth failure blocked operation
|
|
76
|
+
- `stopped`: not running or intentionally shut down
|
|
77
|
+
|
|
78
|
+
## Common Failures
|
|
79
|
+
|
|
80
|
+
### Duplicate plugin id
|
|
81
|
+
- Cause: multiple discoverable `oomi-ai` installs
|
|
82
|
+
- Action: remove stale plugin copies and reinstall once
|
|
83
|
+
|
|
84
|
+
### `invalid handshake: first request must be connect`
|
|
85
|
+
- Cause: gateway request ordering broke
|
|
86
|
+
- Action: update `oomi-ai`, restart the bridge, confirm only one bridge worker exists
|
|
87
|
+
|
|
88
|
+
### STT works but the assistant does not reply
|
|
89
|
+
- Cause: the voice turn reached Oomi, but the managed gateway or OpenClaw run failed later
|
|
90
|
+
- Action: inspect `gateway.log`, `gateway.err.log`, and the session JSONL for that run
|
|
91
|
+
|
|
92
|
+
## Local Oomi API Tools
|
|
93
|
+
|
|
94
|
+
These scripts interact with the local Oomi application when it is running.
|
|
23
95
|
|
|
24
96
|
### `get_data`
|
|
25
|
-
|
|
97
|
+
Fetch the latest user activity data.
|
|
26
98
|
|
|
27
|
-
|
|
28
|
-
```python
|
|
99
|
+
```bash
|
|
29
100
|
python3 skills/oomi/scripts/get_data.py
|
|
30
101
|
```
|
|
31
102
|
|
|
32
|
-
**Returns:**
|
|
33
|
-
JSON string containing:
|
|
34
|
-
- `steps`: Daily step count
|
|
35
|
-
- `sleep`: Sleep duration in hours
|
|
36
|
-
- `energy`: Calculated energy level (0-100)
|
|
37
|
-
- `mood`: Current user mood (if tracked)
|
|
38
|
-
|
|
39
103
|
### `set_goal`
|
|
40
|
-
|
|
104
|
+
Set a new goal in the local Oomi app.
|
|
41
105
|
|
|
42
|
-
|
|
43
|
-
```python
|
|
106
|
+
```bash
|
|
44
107
|
python3 skills/oomi/scripts/send_goal.py --type "steps" --value 10000 --message "Let's hit 10k today!"
|
|
45
108
|
```
|
|
46
109
|
|
|
47
|
-
**Arguments:**
|
|
48
|
-
- `--type`: Type of goal (e.g., "steps", "sleep", "focus")
|
|
49
|
-
- `--value`: Target value (number)
|
|
50
|
-
- `--message`: Motivational message to display to the user
|
|
51
|
-
|
|
52
110
|
### `sync`
|
|
53
|
-
|
|
111
|
+
Sync local context.
|
|
54
112
|
|
|
55
|
-
|
|
56
|
-
```python
|
|
113
|
+
```bash
|
|
57
114
|
python3 skills/oomi/scripts/sync.py
|
|
58
115
|
```
|
|
59
116
|
|
|
60
117
|
### `get_avatar_capabilities`
|
|
61
|
-
|
|
118
|
+
Read the avatar command schema before emitting inline avatar tags.
|
|
62
119
|
|
|
63
|
-
|
|
64
|
-
```python
|
|
120
|
+
```bash
|
|
65
121
|
python3 skills/oomi/scripts/get_avatar_capabilities.py
|
|
66
122
|
```
|
|
67
123
|
|
|
68
|
-
**Returns:**
|
|
69
|
-
JSON containing:
|
|
70
|
-
- `commands.anim.values` (supported animation names)
|
|
71
|
-
- `commands.anim.aliases` (accepted shorthand -> animation name)
|
|
72
|
-
- `commands.face.values` (supported expressions)
|
|
73
|
-
- `commands.gesture.values` (supported gestures)
|
|
74
|
-
- `commands.look.values` (supported look targets)
|
|
75
|
-
|
|
76
124
|
### `install_agent_instructions`
|
|
77
|
-
|
|
125
|
+
Install packaged Oomi operator instructions into an OpenClaw `AGENTS.md` file.
|
|
78
126
|
|
|
79
|
-
|
|
80
|
-
```python
|
|
127
|
+
```bash
|
|
81
128
|
python3 skills/oomi/scripts/install_agent_instructions.py
|
|
82
129
|
```
|
|
83
130
|
|
|
84
|
-
|
|
85
|
-
- `--agents-file` Path to the `AGENTS.md` file (defaults to `OPENCLAW_WORKSPACE/AGENTS.md` or repo `AGENTS.md`).
|
|
86
|
-
- `--instructions-file` Override the instructions markdown file.
|
|
87
|
-
|
|
88
|
-
## Persona Control (Inline)
|
|
89
|
-
|
|
90
|
-
In addition to these scripts, you can control the avatar's visualization directly in your text responses using the following tags. These tags are invisible to the user.
|
|
91
|
-
|
|
92
|
-
- **Animations (canonical)**: `[anim:Waving]`, `[anim:Walking]`, `[anim:Idle]`, `[anim:Sitting Idle]`
|
|
93
|
-
- **Aliases**: `wave -> Waving`, `walk -> Walking`, `idle -> Idle`, `sit/sitting -> Sitting Idle`
|
|
94
|
-
- **Expressions**: `[face:happy]`, `[face:sad]`, `[face:surprised]`, `[face:focused]`, `[face:gentle]`, `[face:thinking]`, `[face:curious]`, `[face:confused]`
|
|
95
|
-
- **Gestures**: `[gesture:nod]`, `[gesture:think]`, `[gesture:shrug]`, `[gesture:wave]`, `[gesture:bow]`
|
|
96
|
-
- **Gaze**: `[look:camera]`, `[look:left]`, `[look:right]`, `[look:up]`, `[look:down]`
|
|
131
|
+
## Avatar Control
|
|
97
132
|
|
|
98
|
-
|
|
99
|
-
|
|
133
|
+
Before emitting avatar commands, call `get_avatar_capabilities` and prefer canonical values.
|
|
134
|
+
Use aliases only when explicitly needed.
|
|
100
135
|
|
|
101
|
-
|
|
102
|
-
|
|
136
|
+
Supported inline tags include:
|
|
137
|
+
- animations: `[anim:Waving]`, `[anim:Walking]`, `[anim:Idle]`, `[anim:Sitting Idle]`
|
|
138
|
+
- expressions: `[face:happy]`, `[face:sad]`, `[face:surprised]`, `[face:focused]`, `[face:gentle]`, `[face:thinking]`
|
|
139
|
+
- gestures: `[gesture:nod]`, `[gesture:think]`, `[gesture:shrug]`, `[gesture:wave]`, `[gesture:bow]`
|
|
140
|
+
- gaze: `[look:camera]`, `[look:left]`, `[look:right]`, `[look:up]`, `[look:down]`
|