@elizaos/plugin-capacitor-bridge 2.0.0-beta.1 → 2.0.11-beta.7

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE ADDED
@@ -0,0 +1,21 @@
1
+ MIT License
2
+
3
+ Copyright (c) 2026 Shaw Walters and elizaOS Contributors
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in all
13
+ copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
21
+ SOFTWARE.
package/README.md ADDED
@@ -0,0 +1,87 @@
1
+ # @elizaos/plugin-capacitor-bridge
2
+
3
+ Agent-side bridge that enables stock iOS and Android Eliza builds to run local GGUF inference through the device's native Capacitor llama.cpp plugin.
4
+
5
+ ## What it does
6
+
7
+ AOSP builds run llama.cpp directly inside the agent process via `bun:ffi`. Stock Capacitor builds (App Store iOS, standard Android APK) cannot do that — llama.cpp is exposed to the WebView through a native Capacitor plugin instead. This package is the agent-side half of that path:
8
+
9
+ - **Android**: accepts a loopback WebSocket from the Capacitor WebView, delegates `TEXT_SMALL`, `TEXT_LARGE`, and `TEXT_EMBEDDING` model requests to the connected device, and lets the normal elizaOS model-handler system work unchanged.
10
+ - **iOS**: runs the elizaOS runtime inside the Bun binary bundled into the iOS app and dispatches API calls in-process over native Bun host IPC (no HTTP loopback).
11
+
12
+ Both paths install a sandboxed virtual filesystem (`installMobileFsShim`) that confines all `node:fs` operations to the app's writable workspace directory, enforcing App Store and Play Store code-execution policies.
13
+
14
+ ## Capabilities added
15
+
16
+ - `TEXT_SMALL` model handler — routes to the connected Capacitor device.
17
+ - `TEXT_LARGE` model handler — routes to the connected Capacitor device.
18
+ - `TEXT_EMBEDDING` model handler — routes to the connected Capacitor device.
19
+ - Automatic GGUF model download from `elizaos/eliza-1` on HuggingFace when no local model is found (unless `ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD=1`).
20
+ - WebSocket endpoint `/api/local-inference/device-bridge` for Capacitor WebView registration and inference RPC.
21
+
22
+ ## Installation
23
+
24
+ This package is used by the elizaOS agent bundle. It is not a standard elizaOS plugin and cannot be added to `character.plugins`. The agent bundle entry point imports and calls its bootstrap functions directly.
25
+
26
+ ```
27
+ @elizaos/plugin-capacitor-bridge
28
+ ```
29
+
30
+ ## Configuration
31
+
32
+ ### Android (WebSocket device bridge)
33
+
34
+ | Env var | Required | Description |
35
+ |---|---|---|
36
+ | `ELIZA_DEVICE_BRIDGE_ENABLED=1` | Yes | Enables the WebSocket bridge. |
37
+ | `ELIZA_DEVICE_PAIRING_TOKEN` | Yes | Shared secret — must match the token sent by the Capacitor WebView. |
38
+ | `ELIZA_DEVICE_BRIDGE_TOKEN` | Alias | Fallback for `ELIZA_DEVICE_PAIRING_TOKEN`. |
39
+
40
+ ### Model path (both platforms)
41
+
42
+ | Env var | Description |
43
+ |---|---|
44
+ | `ELIZA_LOCAL_CHAT_MODEL_PATH` | Absolute path to a GGUF for chat (TEXT_SMALL / TEXT_LARGE). |
45
+ | `ELIZA_LOCAL_EMBEDDING_MODEL_PATH` | Absolute path to an embedding GGUF. |
46
+ | `ELIZA_LOCAL_MODEL_PATH` | Fallback when neither slot-specific var is set. |
47
+ | `ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD=1` | Disables auto-download from HuggingFace. |
48
+
49
+ If no model path is set and auto-download is enabled, the bridge downloads recommended eliza-1 GGUFs from `elizaos/eliza-1` on HuggingFace into `$ELIZA_STATE_DIR/local-inference/models/`.
50
+
51
+ ### Timeouts (all optional, default 600000 ms)
52
+
53
+ - `ELIZA_DEVICE_LOAD_TIMEOUT_MS`
54
+ - `ELIZA_DEVICE_GENERATE_TIMEOUT_MS`
55
+ - `ELIZA_DEVICE_EMBED_TIMEOUT_MS`
56
+
57
+ ## Filesystem sandbox
58
+
59
+ Both platforms install a deny-by-default `node:fs` interceptor (`installMobileFsShim`) before booting the runtime:
60
+
61
+ - All paths are resolved relative to the app's writable workspace root (`MOBILE_WORKSPACE_ROOT` on iOS, `HOME` on Android).
62
+ - Path traversal outside the root throws `EACCES`.
63
+ - System directories (`/etc`, `/usr`, `/System`, `/proc`, etc.) are blocked unconditionally.
64
+ - Writes to native binary extensions (`.so`, `.dylib`, `.node`) are blocked.
65
+ - `require()` of file paths is blocked — all code must be bundled.
66
+
67
+ ## WebSocket protocol (Android)
68
+
69
+ The Capacitor WebView connects to `ws://127.0.0.1:<port>/api/local-inference/device-bridge?token=<ELIZA_DEVICE_PAIRING_TOKEN>`.
70
+
71
+ Connection flow:
72
+ 1. WebView sends `{ type: "register", payload: { deviceId, pairingToken, capabilities, loadedPath } }`.
73
+ 2. Agent sends `{ type: "load", correlationId, modelPath, ... }` → device replies `{ type: "loadResult", correlationId, ok, loadedPath }`.
74
+ 3. Agent sends `{ type: "generate", correlationId, prompt, ... }` → device replies `{ type: "generateResult", correlationId, ok, text }`.
75
+ 4. Agent sends `{ type: "embed", correlationId, input }` → device replies `{ type: "embedResult", correlationId, ok, embedding }`.
76
+ 5. Agent sends `{ type: "formatChat", correlationId, messages }` → device replies `{ type: "formatChatResult", correlationId, ok, prompt }` (invokes native Jinja chat template).
77
+ 6. Agent pings every 15 s; device replies `{ type: "pong" }`.
78
+
79
+ Note: iOS connections are rejected with close code `4003`. iOS uses native IPC, not this WebSocket path.
80
+
81
+ ## Recommended default models
82
+
83
+ | Slot | Model ID | HuggingFace path |
84
+ |---|---|---|
85
+ | TEXT_SMALL | `eliza-1-0_8b` | `elizaos/eliza-1` — `bundles/0_8b/text/eliza-1-0_8b-128k.gguf` |
86
+ | TEXT_LARGE | `eliza-1-2b` | `elizaos/eliza-1` — `bundles/2b/text/eliza-1-2b-128k.gguf` |
87
+ | TEXT_EMBEDDING | `eliza-1-embedding` | `elizaos/eliza-1` — `bundles/4b/embedding/eliza-1-embedding.gguf` |
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "@elizaos/plugin-capacitor-bridge",
3
- "version": "2.0.0-beta.1",
3
+ "version": "2.0.11-beta.7",
4
4
  "description": "Capacitor WebSocket bridge to device llama for stock mobile (non-AOSP) Eliza builds.",
5
5
  "type": "module",
6
6
  "main": "dist/index.js",
@@ -10,36 +10,67 @@
10
10
  "types": "./dist/index.d.ts",
11
11
  "import": "./dist/index.js",
12
12
  "default": "./dist/index.js"
13
+ },
14
+ "./android/bridge": {
15
+ "types": "./dist/android/bridge.d.ts",
16
+ "import": "./dist/android/bridge.js",
17
+ "default": "./dist/android/bridge.js"
18
+ },
19
+ "./ios/bridge": {
20
+ "types": "./dist/ios/bridge.d.ts",
21
+ "import": "./dist/ios/bridge.js",
22
+ "default": "./dist/ios/bridge.js"
23
+ },
24
+ "./mobile-device-bridge-bootstrap": {
25
+ "types": "./dist/mobile-device-bridge-bootstrap.d.ts",
26
+ "import": "./dist/mobile-device-bridge-bootstrap.js",
27
+ "default": "./dist/mobile-device-bridge-bootstrap.js"
28
+ },
29
+ "./shared/fs-shim": {
30
+ "types": "./dist/shared/fs-shim.d.ts",
31
+ "import": "./dist/shared/fs-shim.js",
32
+ "default": "./dist/shared/fs-shim.js"
33
+ },
34
+ "./*.css": "./dist/*.css",
35
+ "./*": {
36
+ "types": "./dist/*.d.ts",
37
+ "import": "./dist/*.js",
38
+ "default": "./dist/*.js"
13
39
  }
14
40
  },
15
41
  "files": [
16
42
  "dist"
17
43
  ],
18
44
  "scripts": {
19
- "build": "tsup src/index.ts --format esm --dts --clean",
20
- "dev": "tsup src/index.ts --format esm --dts --watch",
45
+ "build": "bun run check:android-manifest && NODE_OPTIONS='--max-old-space-size=8192' tsup",
46
+ "check:android-manifest": "node scripts/check-android-manifest.mjs",
47
+ "dev": "tsup --watch",
21
48
  "clean": "rm -rf dist .turbo node_modules",
22
49
  "lint": "bunx @biomejs/biome check --write --unsafe .",
23
50
  "lint:check": "bunx @biomejs/biome check .",
24
51
  "format": "bunx @biomejs/biome format --write .",
25
52
  "format:check": "bunx @biomejs/biome format .",
53
+ "test": "vitest run --config vitest.config.ts",
26
54
  "typecheck": "tsc --noEmit"
27
55
  },
28
56
  "dependencies": {
29
- "@elizaos/core": "2.0.0-beta.1",
57
+ "@elizaos/core": "2.0.11-beta.7",
30
58
  "ws": "^8.18.0"
31
59
  },
32
60
  "peerDependencies": {
33
- "@elizaos/core": "2.0.0-beta.1"
61
+ "@elizaos/core": "2.0.11-beta.7"
34
62
  },
35
63
  "devDependencies": {
36
64
  "@biomejs/biome": "^2.4.14",
65
+ "@types/bun": "^1.3.12",
37
66
  "@types/node": "24.12.2",
38
67
  "@types/ws": "^8.5.13",
39
68
  "tsup": "8.5.1",
40
- "typescript": "^6.0.3"
69
+ "typescript": "^6.0.3",
70
+ "vitest": "^4.0.18"
41
71
  },
42
72
  "publishConfig": {
43
73
  "access": "public"
44
- }
74
+ },
75
+ "gitHead": "cdbc876f793d96073d7eb0d09715a031ce0cd32e"
45
76
  }
package/dist/index.d.ts DELETED
@@ -1,85 +0,0 @@
1
- import { Server } from 'node:http';
2
- import { AgentRuntime } from '@elizaos/core';
3
-
4
- /**
5
- * Stock Capacitor mobile local-inference bridge.
6
- *
7
- * AOSP builds run llama.cpp inside the agent process via bun:ffi. Stock
8
- * Capacitor Android/iOS builds cannot do that: llama.cpp is exposed to the
9
- * WebView through the native Capacitor plugin. This module is the agent-side
10
- * half of that path. It accepts a loopback WebSocket from the WebView,
11
- * forwards TEXT_SMALL / TEXT_LARGE requests to the device, and lets the
12
- * normal conversation routes keep using runtime model handlers.
13
- */
14
-
15
- interface LocalInferenceLoadArgs {
16
- modelPath: string;
17
- contextSize?: number;
18
- useGpu?: boolean;
19
- maxThreads?: number;
20
- draftModelPath?: string;
21
- draftContextSize?: number;
22
- draftMin?: number;
23
- draftMax?: number;
24
- speculativeSamples?: number;
25
- mobileSpeculative?: boolean;
26
- cacheTypeK?: string;
27
- cacheTypeV?: string;
28
- disableThinking?: boolean;
29
- }
30
- interface DeviceCapabilities {
31
- platform: "ios" | "android" | "web";
32
- deviceModel: string;
33
- totalRamGb: number;
34
- cpuCores: number;
35
- gpu: {
36
- backend: "metal" | "vulkan" | "gpu-delegate";
37
- available: boolean;
38
- } | null;
39
- }
40
- interface MobileDeviceBridgeStatus {
41
- enabled: boolean;
42
- connected: boolean;
43
- devices: Array<{
44
- deviceId: string;
45
- capabilities: DeviceCapabilities;
46
- loadedPath: string | null;
47
- connectedSince: string;
48
- }>;
49
- primaryDeviceId: string | null;
50
- pendingRequests: number;
51
- modelPath: string | null;
52
- }
53
- declare class MobileDeviceBridge {
54
- private wss;
55
- private readonly devices;
56
- private readonly pendingLoads;
57
- private readonly pendingUnloads;
58
- private readonly pendingGenerates;
59
- private readonly pendingEmbeds;
60
- status(): MobileDeviceBridgeStatus;
61
- attachToHttpServer(server: Server): Promise<void>;
62
- private handleConnection;
63
- private handleDeviceMessage;
64
- private primaryDevice;
65
- private sendToPrimary;
66
- loadModel(args: LocalInferenceLoadArgs): Promise<void>;
67
- unloadModel(): Promise<void>;
68
- generate(args: {
69
- prompt: string;
70
- stopSequences?: string[];
71
- maxTokens?: number;
72
- temperature?: number;
73
- }): Promise<string>;
74
- embed(args: {
75
- input: string;
76
- }): Promise<number[]>;
77
- }
78
- declare const mobileDeviceBridge: MobileDeviceBridge;
79
- declare function getMobileDeviceBridgeStatus(): MobileDeviceBridgeStatus;
80
- declare function loadMobileDeviceBridgeModel(modelPath: string, modelId?: string): Promise<void>;
81
- declare function unloadMobileDeviceBridgeModel(): Promise<void>;
82
- declare function attachMobileDeviceBridgeToServer(server: Server): Promise<void>;
83
- declare function ensureMobileDeviceBridgeInferenceHandlers(runtime: AgentRuntime): Promise<boolean>;
84
-
85
- export { type MobileDeviceBridgeStatus, attachMobileDeviceBridgeToServer, ensureMobileDeviceBridgeInferenceHandlers, getMobileDeviceBridgeStatus, loadMobileDeviceBridgeModel, mobileDeviceBridge, unloadMobileDeviceBridgeModel };
package/dist/index.js DELETED
@@ -1,639 +0,0 @@
1
- // src/mobile-device-bridge-bootstrap.ts
2
- import { randomUUID } from "crypto";
3
- import {
4
- createWriteStream,
5
- existsSync,
6
- mkdirSync,
7
- readdirSync,
8
- readFileSync,
9
- renameSync,
10
- statSync,
11
- unlinkSync
12
- } from "fs";
13
- import path from "path";
14
- import { Readable } from "stream";
15
- import { pipeline } from "stream/promises";
16
- import {
17
- logger,
18
- ModelType,
19
- resolveStateDir
20
- } from "@elizaos/core";
21
- var DEVICE_BRIDGE_PATH = "/api/local-inference/device-bridge";
22
- var PROVIDER = "capacitor-llama";
23
- var LOCAL_INFERENCE_PRIORITY = 0;
24
- var DEFAULT_CALL_TIMEOUT_MS = 12e4;
25
- var DEFAULT_LOAD_TIMEOUT_MS = 18e4;
26
- var SERVICE_ENABLED = process.env.ELIZA_DEVICE_BRIDGE_ENABLED?.trim() === "1";
27
- var registeredRuntimes = /* @__PURE__ */ new WeakSet();
28
- var KNOWN_EMBEDDING_DIMENSIONS = {
29
- "eliza-1-lite-0_6b": 1024,
30
- "eliza-1-mobile-1_7b": 2048
31
- };
32
- var ELIZA_1_LOAD_METADATA = {
33
- "eliza-1-lite-0_6b": { contextSize: 32768 },
34
- "eliza-1-mobile-1_7b": { contextSize: 32768 },
35
- "eliza-1-desktop-9b": { contextSize: 65536 },
36
- "eliza-1-pro-27b": { contextSize: 131072 },
37
- "eliza-1-server-h200": { contextSize: 262144 }
38
- };
39
- function isWsModule(value) {
40
- return typeof value === "object" && value !== null && typeof value.WebSocketServer === "function" && typeof value.WebSocket === "function";
41
- }
42
- var MobileDeviceBridge = class {
43
- wss = null;
44
- devices = /* @__PURE__ */ new Map();
45
- pendingLoads = /* @__PURE__ */ new Map();
46
- pendingUnloads = /* @__PURE__ */ new Map();
47
- pendingGenerates = /* @__PURE__ */ new Map();
48
- pendingEmbeds = /* @__PURE__ */ new Map();
49
- status() {
50
- const devices = [...this.devices.values()].map((device) => ({
51
- deviceId: device.deviceId,
52
- capabilities: device.capabilities,
53
- loadedPath: device.loadedPath,
54
- connectedSince: new Date(device.connectedAt).toISOString()
55
- }));
56
- return {
57
- enabled: SERVICE_ENABLED,
58
- connected: devices.length > 0,
59
- devices,
60
- primaryDeviceId: devices[0]?.deviceId ?? null,
61
- pendingRequests: this.pendingLoads.size + this.pendingUnloads.size + this.pendingGenerates.size + this.pendingEmbeds.size,
62
- modelPath: resolveLocalModelPath("TEXT_LARGE")
63
- };
64
- }
65
- async attachToHttpServer(server) {
66
- if (!SERVICE_ENABLED || this.wss) return;
67
- const wsModule = await import("ws");
68
- if (!isWsModule(wsModule)) {
69
- throw new Error("ws module did not expose WebSocketServer/WebSocket");
70
- }
71
- const ws = wsModule;
72
- const wss = new ws.WebSocketServer({
73
- noServer: true,
74
- maxPayload: 1024 * 1024
75
- });
76
- this.wss = wss;
77
- wss.on("error", (err) => {
78
- logger.warn("[mobile-device-bridge] WSS error:", err.message);
79
- });
80
- server.on("upgrade", (request, socket, head) => {
81
- const url = new URL(request.url ?? "/", "http://localhost");
82
- if (url.pathname !== DEVICE_BRIDGE_PATH) return;
83
- wss.handleUpgrade(request, socket, head, (client) => {
84
- this.handleConnection(client, ws.WebSocket);
85
- });
86
- });
87
- logger.info(
88
- `[mobile-device-bridge] Listening for Capacitor device bridge at ${DEVICE_BRIDGE_PATH}`
89
- );
90
- }
91
- handleConnection(socket, WsCtor) {
92
- let registeredDeviceId = null;
93
- socket.on("message", (raw) => {
94
- let msg;
95
- try {
96
- const text = typeof raw === "string" ? raw : raw.toString("utf8");
97
- msg = JSON.parse(text);
98
- } catch {
99
- logger.warn("[mobile-device-bridge] Ignoring non-JSON frame");
100
- return;
101
- }
102
- if (!registeredDeviceId) {
103
- if (msg.type !== "register") {
104
- socket.close(4002, "must-register-first");
105
- return;
106
- }
107
- registeredDeviceId = msg.payload.deviceId;
108
- this.devices.set(registeredDeviceId, {
109
- deviceId: registeredDeviceId,
110
- socket,
111
- capabilities: msg.payload.capabilities,
112
- loadedPath: msg.payload.loadedPath,
113
- connectedAt: Date.now()
114
- });
115
- logger.info(
116
- `[mobile-device-bridge] Device connected: ${registeredDeviceId} (${msg.payload.capabilities.platform})`
117
- );
118
- return;
119
- }
120
- this.handleDeviceMessage(msg);
121
- });
122
- socket.on("close", () => {
123
- if (!registeredDeviceId) return;
124
- const current = this.devices.get(registeredDeviceId);
125
- if (current?.socket === socket) {
126
- this.devices.delete(registeredDeviceId);
127
- logger.info(
128
- `[mobile-device-bridge] Device disconnected: ${registeredDeviceId}`
129
- );
130
- }
131
- });
132
- socket.on("error", (err) => {
133
- logger.warn("[mobile-device-bridge] Socket error:", err.message);
134
- });
135
- const heartbeat = setInterval(() => {
136
- if (!registeredDeviceId || socket.readyState !== WsCtor.OPEN) return;
137
- try {
138
- socket.send(JSON.stringify({ type: "ping", at: Date.now() }));
139
- } catch {
140
- clearInterval(heartbeat);
141
- }
142
- }, 15e3);
143
- if (typeof heartbeat === "object" && "unref" in heartbeat) {
144
- heartbeat.unref();
145
- }
146
- }
147
- handleDeviceMessage(msg) {
148
- if (msg.type === "pong" || msg.type === "register") return;
149
- if (msg.type === "loadResult") {
150
- const pending = this.pendingLoads.get(msg.correlationId);
151
- if (!pending) return;
152
- clearTimeout(pending.timeout);
153
- this.pendingLoads.delete(msg.correlationId);
154
- if (msg.ok === true) {
155
- const device = this.devices.get(pending.routedDeviceId);
156
- if (device) device.loadedPath = msg.loadedPath;
157
- pending.resolve(void 0);
158
- } else {
159
- pending.reject(new Error(msg.error));
160
- }
161
- return;
162
- }
163
- if (msg.type === "unloadResult") {
164
- const pending = this.pendingUnloads.get(msg.correlationId);
165
- if (!pending) return;
166
- clearTimeout(pending.timeout);
167
- this.pendingUnloads.delete(msg.correlationId);
168
- if (msg.ok === true) {
169
- const device = this.devices.get(pending.routedDeviceId);
170
- if (device) device.loadedPath = null;
171
- pending.resolve(void 0);
172
- } else {
173
- pending.reject(new Error(msg.error));
174
- }
175
- return;
176
- }
177
- if (msg.type === "generateResult") {
178
- const pending = this.pendingGenerates.get(msg.correlationId);
179
- if (!pending) return;
180
- clearTimeout(pending.timeout);
181
- this.pendingGenerates.delete(msg.correlationId);
182
- if (msg.ok === true) {
183
- pending.resolve(msg.text);
184
- } else {
185
- pending.reject(new Error(msg.error));
186
- }
187
- return;
188
- }
189
- if (msg.type === "embedResult") {
190
- const pending = this.pendingEmbeds.get(msg.correlationId);
191
- if (!pending) return;
192
- clearTimeout(pending.timeout);
193
- this.pendingEmbeds.delete(msg.correlationId);
194
- if (msg.ok === true) {
195
- pending.resolve(msg.embedding);
196
- } else {
197
- pending.reject(new Error(msg.error));
198
- }
199
- }
200
- }
201
- primaryDevice() {
202
- return this.devices.values().next().value ?? null;
203
- }
204
- sendToPrimary(pendingMap, makeMessage, timeoutMs, timeoutMessage) {
205
- const device = this.primaryDevice();
206
- if (!device) {
207
- return Promise.reject(
208
- new Error(
209
- "DEVICE_DISCONNECTED: no Capacitor llama device bridge attached"
210
- )
211
- );
212
- }
213
- const correlationId = randomUUID();
214
- const message = makeMessage(correlationId);
215
- return new Promise((resolve, reject) => {
216
- const timeout = setTimeout(() => {
217
- pendingMap.delete(correlationId);
218
- reject(new Error(timeoutMessage));
219
- }, timeoutMs);
220
- if (typeof timeout === "object" && "unref" in timeout) {
221
- timeout.unref();
222
- }
223
- pendingMap.set(correlationId, {
224
- resolve,
225
- reject,
226
- timeout,
227
- routedDeviceId: device.deviceId
228
- });
229
- try {
230
- device.socket.send(JSON.stringify(message));
231
- } catch (err) {
232
- clearTimeout(timeout);
233
- pendingMap.delete(correlationId);
234
- reject(err instanceof Error ? err : new Error(String(err)));
235
- }
236
- });
237
- }
238
- async loadModel(args) {
239
- const device = this.primaryDevice();
240
- if (device?.loadedPath === args.modelPath) return;
241
- return this.sendToPrimary(
242
- this.pendingLoads,
243
- (correlationId) => ({
244
- type: "load",
245
- correlationId,
246
- ...args
247
- }),
248
- readTimeoutMs("ELIZA_DEVICE_LOAD_TIMEOUT_MS", DEFAULT_LOAD_TIMEOUT_MS),
249
- "DEVICE_TIMEOUT: model load exceeded deadline"
250
- );
251
- }
252
- async unloadModel() {
253
- const device = this.primaryDevice();
254
- if (!device?.loadedPath) return;
255
- return this.sendToPrimary(
256
- this.pendingUnloads,
257
- (correlationId) => ({ type: "unload", correlationId }),
258
- readTimeoutMs(
259
- "ELIZA_DEVICE_GENERATE_TIMEOUT_MS",
260
- DEFAULT_CALL_TIMEOUT_MS
261
- ),
262
- "DEVICE_TIMEOUT: unload exceeded deadline"
263
- );
264
- }
265
- generate(args) {
266
- return this.sendToPrimary(
267
- this.pendingGenerates,
268
- (correlationId) => ({
269
- type: "generate",
270
- correlationId,
271
- prompt: args.prompt,
272
- stopSequences: args.stopSequences,
273
- maxTokens: args.maxTokens,
274
- temperature: args.temperature
275
- }),
276
- readTimeoutMs(
277
- "ELIZA_DEVICE_GENERATE_TIMEOUT_MS",
278
- DEFAULT_CALL_TIMEOUT_MS
279
- ),
280
- "DEVICE_TIMEOUT: no device responded within deadline"
281
- );
282
- }
283
- embed(args) {
284
- return this.sendToPrimary(
285
- this.pendingEmbeds,
286
- (correlationId) => ({
287
- type: "embed",
288
- correlationId,
289
- input: args.input
290
- }),
291
- readTimeoutMs("ELIZA_DEVICE_EMBED_TIMEOUT_MS", DEFAULT_CALL_TIMEOUT_MS),
292
- "DEVICE_TIMEOUT: no device returned embeddings within deadline"
293
- );
294
- }
295
- };
296
- var mobileDeviceBridge = new MobileDeviceBridge();
297
- function readTimeoutMs(envKey, fallback) {
298
- const parsed = Number.parseInt(process.env[envKey]?.trim() ?? "", 10);
299
- return Number.isFinite(parsed) && parsed > 0 ? parsed : fallback;
300
- }
301
- function modelsDir() {
302
- return path.join(resolveStateDir(), "local-inference", "models");
303
- }
304
- function registryPath() {
305
- return path.join(resolveStateDir(), "local-inference", "registry.json");
306
- }
307
- function assignmentsPath() {
308
- return path.join(resolveStateDir(), "local-inference", "assignments.json");
309
- }
310
- function readJsonFile(filePath) {
311
- try {
312
- return JSON.parse(readFileSync(filePath, "utf8"));
313
- } catch {
314
- return null;
315
- }
316
- }
317
- function positiveInteger(value) {
318
- const numeric = typeof value === "number" ? value : typeof value === "string" ? Number.parseInt(value, 10) : Number.NaN;
319
- return Number.isInteger(numeric) && numeric > 0 ? numeric : null;
320
- }
321
- function resolveFromEnv(slot) {
322
- const key = slot === "TEXT_EMBEDDING" ? "ELIZA_LOCAL_EMBEDDING_MODEL_PATH" : "ELIZA_LOCAL_CHAT_MODEL_PATH";
323
- const specific = process.env[key]?.trim();
324
- if (specific && existsSync(specific)) return specific;
325
- const fallback = process.env.ELIZA_LOCAL_MODEL_PATH?.trim();
326
- if (fallback && existsSync(fallback)) return fallback;
327
- return null;
328
- }
329
- function resolveFromRegistry(slot) {
330
- const assignments = readJsonFile(
331
- assignmentsPath()
332
- )?.assignments;
333
- const assigned = assignments?.[slot];
334
- if (typeof assigned !== "string" || !assigned.trim()) return null;
335
- const models = readRegistryModels();
336
- const matched = models.find((model) => model.id === assigned);
337
- return typeof matched?.path === "string" && existsSync(matched.path) ? matched.path : null;
338
- }
339
- function readRegistryModels() {
340
- return readJsonFile(registryPath())?.models ?? [];
341
- }
342
- function resolveAssignedRegistryModel(slot) {
343
- const assignments = readJsonFile(
344
- assignmentsPath()
345
- )?.assignments;
346
- const assigned = assignments?.[slot];
347
- if (typeof assigned !== "string" || !assigned.trim()) return null;
348
- const models = readRegistryModels();
349
- const matched = models.find((model) => model.id === assigned);
350
- if (typeof matched?.path !== "string" || !existsSync(matched.path)) {
351
- return null;
352
- }
353
- return {
354
- id: assigned,
355
- path: matched.path,
356
- dimensions: matched.dimensions,
357
- embeddingDimension: matched.embeddingDimension,
358
- embeddingDimensions: matched.embeddingDimensions
359
- };
360
- }
361
- function resolveFromManifest(slot) {
362
- const manifest = readJsonFile(
363
- path.join(modelsDir(), "manifest.json")
364
- );
365
- const targetRole = slot === "TEXT_EMBEDDING" ? "embedding" : "chat";
366
- for (const entry of manifest?.models ?? []) {
367
- if (entry.role !== targetRole) continue;
368
- const fileName = entry.ggufFile ?? entry.filename;
369
- if (!fileName) continue;
370
- const absolute = path.join(modelsDir(), fileName);
371
- if (existsSync(absolute)) return absolute;
372
- }
373
- return null;
374
- }
375
- function resolveFirstGguf() {
376
- const dir = modelsDir();
377
- if (!existsSync(dir)) return null;
378
- for (const name of readdirSync(dir)) {
379
- if (!name.toLowerCase().endsWith(".gguf")) continue;
380
- const absolute = path.join(dir, name);
381
- if (existsSync(absolute)) return absolute;
382
- }
383
- return null;
384
- }
385
- function resolveLocalModelPath(slot) {
386
- return resolveFromEnv(slot) ?? resolveFromRegistry(slot) ?? resolveFromManifest(slot) ?? resolveFirstGguf();
387
- }
388
- function buildLoadArgsFromRegistryModel(model) {
389
- const args = { modelPath: model.path };
390
- const eliza1 = ELIZA_1_LOAD_METADATA[model.id];
391
- if (eliza1) args.contextSize = eliza1.contextSize;
392
- return args;
393
- }
394
- function resolveLocalLoadArgs(slot) {
395
- const envPath = resolveFromEnv(slot);
396
- if (envPath) return { modelPath: envPath };
397
- const registryModel = resolveAssignedRegistryModel(slot);
398
- if (registryModel) return buildLoadArgsFromRegistryModel(registryModel);
399
- const manifestPath = resolveFromManifest(slot);
400
- if (manifestPath) return { modelPath: manifestPath };
401
- const firstGguf = resolveFirstGguf();
402
- return firstGguf ? { modelPath: firstGguf } : null;
403
- }
404
- var RECOMMENDED_MODELS = {
405
- TEXT_SMALL: {
406
- id: "eliza-1-lite-0_6b",
407
- hfRepo: "elizaos/eliza-1-lite-0_6b",
408
- ggufFile: "text/eliza-1-lite-0_6b-32k.gguf",
409
- localFile: "eliza-1-lite-0_6b-32k.gguf"
410
- },
411
- TEXT_LARGE: {
412
- id: "eliza-1-mobile-1_7b",
413
- hfRepo: "elizaos/eliza-1-mobile-1_7b",
414
- ggufFile: "text/eliza-1-mobile-1_7b-32k.gguf",
415
- localFile: "eliza-1-mobile-1_7b-32k.gguf"
416
- },
417
- TEXT_EMBEDDING: {
418
- id: "eliza-1-lite-0_6b",
419
- hfRepo: "elizaos/eliza-1-lite-0_6b",
420
- ggufFile: "text/eliza-1-lite-0_6b-32k.gguf",
421
- localFile: "eliza-1-lite-0_6b-32k.gguf"
422
- }
423
- };
424
- var inflightDownloads = /* @__PURE__ */ new Map();
425
- function buildHfResolveUrl(model) {
426
- const encodedPath = model.ggufFile.split("/").map((segment) => encodeURIComponent(segment)).join("/");
427
- return `https://huggingface.co/${model.hfRepo}/resolve/main/${encodedPath}?download=true`;
428
- }
429
- function buildRecommendedLoadArgs(slot, modelPath) {
430
- const model = RECOMMENDED_MODELS[slot];
431
- return buildLoadArgsFromRegistryModel({ id: model.id, path: modelPath });
432
- }
433
- async function downloadRecommendedModelFor(slot) {
434
- const model = RECOMMENDED_MODELS[slot];
435
- const dir = modelsDir();
436
- mkdirSync(dir, { recursive: true });
437
- const finalPath = path.join(
438
- dir,
439
- model.localFile ?? path.basename(model.ggufFile)
440
- );
441
- if (existsSync(finalPath)) {
442
- const sz = statSync(finalPath).size;
443
- if (!model.expectedSizeBytes || sz === model.expectedSizeBytes) {
444
- return finalPath;
445
- }
446
- logger.warn(
447
- `[mobile-device-bridge] ${model.ggufFile} present but size ${sz} != expected ${model.expectedSizeBytes}; re-downloading.`
448
- );
449
- try {
450
- unlinkSync(finalPath);
451
- } catch {
452
- }
453
- }
454
- const dedupKey = model.id;
455
- const existing = inflightDownloads.get(dedupKey);
456
- if (existing) return existing;
457
- const promise = (async () => {
458
- const url = buildHfResolveUrl(model);
459
- const stagingPath = `${finalPath}.part`;
460
- try {
461
- unlinkSync(stagingPath);
462
- } catch {
463
- }
464
- logger.info(
465
- `[mobile-device-bridge] Auto-downloading recommended ${slot} model ${model.id} from ${url}`
466
- );
467
- const response = await fetch(url, { redirect: "follow" });
468
- if (!response.ok || !response.body) {
469
- throw new Error(
470
- `[mobile-device-bridge] Recommended-model download failed (${slot}): HTTP ${response.status} ${response.statusText} from ${url}`
471
- );
472
- }
473
- await pipeline(
474
- Readable.fromWeb(response.body),
475
- createWriteStream(stagingPath)
476
- );
477
- const stagedSize = statSync(stagingPath).size;
478
- if (model.expectedSizeBytes && stagedSize !== model.expectedSizeBytes) {
479
- try {
480
- unlinkSync(stagingPath);
481
- } catch {
482
- }
483
- throw new Error(
484
- `[mobile-device-bridge] Downloaded ${model.ggufFile} size ${stagedSize} != expected ${model.expectedSizeBytes}; aborting and removing partial file.`
485
- );
486
- }
487
- renameSync(stagingPath, finalPath);
488
- logger.info(
489
- `[mobile-device-bridge] Auto-download complete: ${finalPath} (${stagedSize} bytes)`
490
- );
491
- return finalPath;
492
- })();
493
- inflightDownloads.set(dedupKey, promise);
494
- try {
495
- return await promise;
496
- } finally {
497
- inflightDownloads.delete(dedupKey);
498
- }
499
- }
500
- async function resolveLoadArgsWithAutoDownload(slot) {
501
- const existing = resolveLocalLoadArgs(slot);
502
- if (existing) return existing;
503
- if (process.env.ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD?.trim() === "1") {
504
- return null;
505
- }
506
- const downloaded = await downloadRecommendedModelFor(slot);
507
- return buildRecommendedLoadArgs(slot, downloaded);
508
- }
509
- function resolveEmbeddingDimension() {
510
- const assigned = resolveAssignedRegistryModel("TEXT_EMBEDDING");
511
- return positiveInteger(process.env.ELIZA_LOCAL_EMBEDDING_DIMENSIONS) ?? positiveInteger(process.env.TEXT_EMBEDDING_DIMENSIONS) ?? positiveInteger(assigned?.dimensions) ?? positiveInteger(assigned?.embeddingDimension) ?? positiveInteger(assigned?.embeddingDimensions) ?? (assigned?.id ? KNOWN_EMBEDDING_DIMENSIONS[assigned.id] : null) ?? KNOWN_EMBEDDING_DIMENSIONS[RECOMMENDED_MODELS.TEXT_EMBEDDING.id] ?? 1024;
512
- }
513
- function makeGenerateHandler(slot) {
514
- return async (_runtime, params) => {
515
- const loadArgs = await resolveLoadArgsWithAutoDownload(slot);
516
- if (!loadArgs) {
517
- throw new Error(
518
- `[mobile-device-bridge] No local GGUF model installed under ${modelsDir()} and auto-download is disabled (ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD=1). Install a model or unset the disable flag.`
519
- );
520
- }
521
- await mobileDeviceBridge.loadModel(loadArgs);
522
- return mobileDeviceBridge.generate({
523
- prompt: params.prompt ?? "",
524
- stopSequences: params.stopSequences,
525
- maxTokens: params.maxTokens,
526
- temperature: params.temperature
527
- });
528
- };
529
- }
530
- function extractEmbeddingText(params) {
531
- if (params === null) return "";
532
- if (typeof params === "string") return params;
533
- return params.text;
534
- }
535
- function makeEmbeddingHandler() {
536
- return async (_runtime, params) => {
537
- if (params === null) {
538
- return new Array(resolveEmbeddingDimension()).fill(0);
539
- }
540
- let modelPath = resolveLocalModelPath("TEXT_EMBEDDING");
541
- let loadArgs = modelPath ? { modelPath } : null;
542
- if (!modelPath) {
543
- if (process.env.ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD?.trim() === "1") {
544
- throw new Error(
545
- `[mobile-device-bridge] No local GGUF embedding model installed under ${modelsDir()} and auto-download is disabled.`
546
- );
547
- }
548
- modelPath = await downloadRecommendedModelFor("TEXT_EMBEDDING");
549
- loadArgs = buildRecommendedLoadArgs("TEXT_EMBEDDING", modelPath);
550
- }
551
- if (!loadArgs) {
552
- throw new Error(
553
- `[mobile-device-bridge] No local GGUF embedding model resolved for ${modelsDir()}.`
554
- );
555
- }
556
- await mobileDeviceBridge.loadModel(loadArgs);
557
- return mobileDeviceBridge.embed({
558
- input: extractEmbeddingText(params)
559
- });
560
- };
561
- }
562
- function getMobileDeviceBridgeStatus() {
563
- return mobileDeviceBridge.status();
564
- }
565
- async function loadMobileDeviceBridgeModel(modelPath, modelId) {
566
- await mobileDeviceBridge.loadModel(
567
- modelId ? buildLoadArgsFromRegistryModel({ id: modelId, path: modelPath }) : { modelPath }
568
- );
569
- }
570
- async function unloadMobileDeviceBridgeModel() {
571
- await mobileDeviceBridge.unloadModel();
572
- }
573
- async function attachMobileDeviceBridgeToServer(server) {
574
- await mobileDeviceBridge.attachToHttpServer(server);
575
- }
576
- async function ensureMobileDeviceBridgeInferenceHandlers(runtime) {
577
- logger.debug("[mobile-device-bridge] Bootstrap entered");
578
- if (!SERVICE_ENABLED || process.env.ELIZA_LOCAL_LLAMA?.trim() === "1") {
579
- logger.debug("[mobile-device-bridge] Disabled or AOSP local llama active");
580
- return false;
581
- }
582
- if (registeredRuntimes.has(runtime)) {
583
- logger.debug("[mobile-device-bridge] Handlers already registered");
584
- return true;
585
- }
586
- const runtimeWithRegistration = runtime;
587
- if (typeof runtimeWithRegistration.getModel !== "function" || typeof runtimeWithRegistration.registerModel !== "function") {
588
- logger.error(
589
- "[mobile-device-bridge] Runtime is missing getModel/registerModel; cannot wire handlers."
590
- );
591
- return false;
592
- }
593
- runtimeWithRegistration.registerModel(
594
- ModelType.TEXT_SMALL,
595
- makeGenerateHandler("TEXT_SMALL"),
596
- PROVIDER,
597
- LOCAL_INFERENCE_PRIORITY
598
- );
599
- runtimeWithRegistration.registerModel(
600
- ModelType.TEXT_LARGE,
601
- makeGenerateHandler("TEXT_LARGE"),
602
- PROVIDER,
603
- LOCAL_INFERENCE_PRIORITY
604
- );
605
- if (!resolveLocalLoadArgs("TEXT_SMALL") && process.env.ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD?.trim() !== "1") {
606
- downloadRecommendedModelFor("TEXT_SMALL").catch(
607
- (err) => logger.warn(
608
- `[mobile-device-bridge] Background chat-model download failed: ${err.message}`
609
- )
610
- );
611
- }
612
- runtimeWithRegistration.registerModel(
613
- ModelType.TEXT_EMBEDDING,
614
- makeEmbeddingHandler(),
615
- PROVIDER,
616
- LOCAL_INFERENCE_PRIORITY
617
- );
618
- const embeddingModelPath = resolveLocalModelPath("TEXT_EMBEDDING");
619
- if (!embeddingModelPath && process.env.ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD?.trim() !== "1") {
620
- downloadRecommendedModelFor("TEXT_EMBEDDING").catch(
621
- (err) => logger.warn(
622
- `[mobile-device-bridge] Background embedding-model download failed: ${err.message}`
623
- )
624
- );
625
- }
626
- logger.info(
627
- `[mobile-device-bridge] Registered ${PROVIDER} handlers for TEXT_SMALL / TEXT_LARGE${embeddingModelPath ? " / TEXT_EMBEDDING" : ""} at priority ${LOCAL_INFERENCE_PRIORITY}`
628
- );
629
- registeredRuntimes.add(runtime);
630
- return true;
631
- }
632
- export {
633
- attachMobileDeviceBridgeToServer,
634
- ensureMobileDeviceBridgeInferenceHandlers,
635
- getMobileDeviceBridgeStatus,
636
- loadMobileDeviceBridgeModel,
637
- mobileDeviceBridge,
638
- unloadMobileDeviceBridgeModel
639
- };