@elizaos/plugin-capacitor-bridge 2.0.0-beta.1 → 2.0.3-beta.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +21 -0
- package/README.md +87 -0
- package/package.json +38 -7
- package/dist/index.d.ts +0 -85
- package/dist/index.js +0 -639
package/LICENSE
ADDED
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
MIT License
|
|
2
|
+
|
|
3
|
+
Copyright (c) 2026 Shaw Walters and elizaOS Contributors
|
|
4
|
+
|
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
6
|
+
of this software and associated documentation files (the "Software"), to deal
|
|
7
|
+
in the Software without restriction, including without limitation the rights
|
|
8
|
+
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
|
9
|
+
copies of the Software, and to permit persons to whom the Software is
|
|
10
|
+
furnished to do so, subject to the following conditions:
|
|
11
|
+
|
|
12
|
+
The above copyright notice and this permission notice shall be included in all
|
|
13
|
+
copies or substantial portions of the Software.
|
|
14
|
+
|
|
15
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
|
16
|
+
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
|
17
|
+
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
|
18
|
+
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
|
19
|
+
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
|
20
|
+
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
|
|
21
|
+
SOFTWARE.
|
package/README.md
ADDED
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# @elizaos/plugin-capacitor-bridge
|
|
2
|
+
|
|
3
|
+
Agent-side bridge that enables stock iOS and Android Eliza builds to run local GGUF inference through the device's native Capacitor llama.cpp plugin.
|
|
4
|
+
|
|
5
|
+
## What it does
|
|
6
|
+
|
|
7
|
+
AOSP builds run llama.cpp directly inside the agent process via `bun:ffi`. Stock Capacitor builds (App Store iOS, standard Android APK) cannot do that — llama.cpp is exposed to the WebView through a native Capacitor plugin instead. This package is the agent-side half of that path:
|
|
8
|
+
|
|
9
|
+
- **Android**: accepts a loopback WebSocket from the Capacitor WebView, delegates `TEXT_SMALL`, `TEXT_LARGE`, and `TEXT_EMBEDDING` model requests to the connected device, and lets the normal elizaOS model-handler system work unchanged.
|
|
10
|
+
- **iOS**: runs the elizaOS runtime inside the Bun binary bundled into the iOS app and dispatches API calls in-process over native Bun host IPC (no HTTP loopback).
|
|
11
|
+
|
|
12
|
+
Both paths install a sandboxed virtual filesystem (`installMobileFsShim`) that confines all `node:fs` operations to the app's writable workspace directory, enforcing App Store and Play Store code-execution policies.
|
|
13
|
+
|
|
14
|
+
## Capabilities added
|
|
15
|
+
|
|
16
|
+
- `TEXT_SMALL` model handler — routes to the connected Capacitor device.
|
|
17
|
+
- `TEXT_LARGE` model handler — routes to the connected Capacitor device.
|
|
18
|
+
- `TEXT_EMBEDDING` model handler — routes to the connected Capacitor device.
|
|
19
|
+
- Automatic GGUF model download from `elizaos/eliza-1` on HuggingFace when no local model is found (unless `ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD=1`).
|
|
20
|
+
- WebSocket endpoint `/api/local-inference/device-bridge` for Capacitor WebView registration and inference RPC.
|
|
21
|
+
|
|
22
|
+
## Installation
|
|
23
|
+
|
|
24
|
+
This package is used by the elizaOS agent bundle. It is not a standard elizaOS plugin and cannot be added to `character.plugins`. The agent bundle entry point imports and calls its bootstrap functions directly.
|
|
25
|
+
|
|
26
|
+
```
|
|
27
|
+
@elizaos/plugin-capacitor-bridge
|
|
28
|
+
```
|
|
29
|
+
|
|
30
|
+
## Configuration
|
|
31
|
+
|
|
32
|
+
### Android (WebSocket device bridge)
|
|
33
|
+
|
|
34
|
+
| Env var | Required | Description |
|
|
35
|
+
|---|---|---|
|
|
36
|
+
| `ELIZA_DEVICE_BRIDGE_ENABLED=1` | Yes | Enables the WebSocket bridge. |
|
|
37
|
+
| `ELIZA_DEVICE_PAIRING_TOKEN` | Yes | Shared secret — must match the token sent by the Capacitor WebView. |
|
|
38
|
+
| `ELIZA_DEVICE_BRIDGE_TOKEN` | Alias | Fallback for `ELIZA_DEVICE_PAIRING_TOKEN`. |
|
|
39
|
+
|
|
40
|
+
### Model path (both platforms)
|
|
41
|
+
|
|
42
|
+
| Env var | Description |
|
|
43
|
+
|---|---|
|
|
44
|
+
| `ELIZA_LOCAL_CHAT_MODEL_PATH` | Absolute path to a GGUF for chat (TEXT_SMALL / TEXT_LARGE). |
|
|
45
|
+
| `ELIZA_LOCAL_EMBEDDING_MODEL_PATH` | Absolute path to an embedding GGUF. |
|
|
46
|
+
| `ELIZA_LOCAL_MODEL_PATH` | Fallback when neither slot-specific var is set. |
|
|
47
|
+
| `ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD=1` | Disables auto-download from HuggingFace. |
|
|
48
|
+
|
|
49
|
+
If no model path is set and auto-download is enabled, the bridge downloads recommended eliza-1 GGUFs from `elizaos/eliza-1` on HuggingFace into `$ELIZA_STATE_DIR/local-inference/models/`.
|
|
50
|
+
|
|
51
|
+
### Timeouts (all optional, default 600000 ms)
|
|
52
|
+
|
|
53
|
+
- `ELIZA_DEVICE_LOAD_TIMEOUT_MS`
|
|
54
|
+
- `ELIZA_DEVICE_GENERATE_TIMEOUT_MS`
|
|
55
|
+
- `ELIZA_DEVICE_EMBED_TIMEOUT_MS`
|
|
56
|
+
|
|
57
|
+
## Filesystem sandbox
|
|
58
|
+
|
|
59
|
+
Both platforms install a deny-by-default `node:fs` interceptor (`installMobileFsShim`) before booting the runtime:
|
|
60
|
+
|
|
61
|
+
- All paths are resolved relative to the app's writable workspace root (`MOBILE_WORKSPACE_ROOT` on iOS, `HOME` on Android).
|
|
62
|
+
- Path traversal outside the root throws `EACCES`.
|
|
63
|
+
- System directories (`/etc`, `/usr`, `/System`, `/proc`, etc.) are blocked unconditionally.
|
|
64
|
+
- Writes to native binary extensions (`.so`, `.dylib`, `.node`) are blocked.
|
|
65
|
+
- `require()` of file paths is blocked — all code must be bundled.
|
|
66
|
+
|
|
67
|
+
## WebSocket protocol (Android)
|
|
68
|
+
|
|
69
|
+
The Capacitor WebView connects to `ws://127.0.0.1:<port>/api/local-inference/device-bridge?token=<ELIZA_DEVICE_PAIRING_TOKEN>`.
|
|
70
|
+
|
|
71
|
+
Connection flow:
|
|
72
|
+
1. WebView sends `{ type: "register", payload: { deviceId, pairingToken, capabilities, loadedPath } }`.
|
|
73
|
+
2. Agent sends `{ type: "load", correlationId, modelPath, ... }` → device replies `{ type: "loadResult", correlationId, ok, loadedPath }`.
|
|
74
|
+
3. Agent sends `{ type: "generate", correlationId, prompt, ... }` → device replies `{ type: "generateResult", correlationId, ok, text }`.
|
|
75
|
+
4. Agent sends `{ type: "embed", correlationId, input }` → device replies `{ type: "embedResult", correlationId, ok, embedding }`.
|
|
76
|
+
5. Agent sends `{ type: "formatChat", correlationId, messages }` → device replies `{ type: "formatChatResult", correlationId, ok, prompt }` (invokes native Jinja chat template).
|
|
77
|
+
6. Agent pings every 15 s; device replies `{ type: "pong" }`.
|
|
78
|
+
|
|
79
|
+
Note: iOS connections are rejected with close code `4003`. iOS uses native IPC, not this WebSocket path.
|
|
80
|
+
|
|
81
|
+
## Recommended default models
|
|
82
|
+
|
|
83
|
+
| Slot | Model ID | HuggingFace path |
|
|
84
|
+
|---|---|---|
|
|
85
|
+
| TEXT_SMALL | `eliza-1-0_8b` | `elizaos/eliza-1` — `bundles/0_8b/text/eliza-1-0_8b-128k.gguf` |
|
|
86
|
+
| TEXT_LARGE | `eliza-1-2b` | `elizaos/eliza-1` — `bundles/2b/text/eliza-1-2b-128k.gguf` |
|
|
87
|
+
| TEXT_EMBEDDING | `eliza-1-embedding` | `elizaos/eliza-1` — `bundles/4b/embedding/eliza-1-embedding.gguf` |
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@elizaos/plugin-capacitor-bridge",
|
|
3
|
-
"version": "2.0.
|
|
3
|
+
"version": "2.0.3-beta.2",
|
|
4
4
|
"description": "Capacitor WebSocket bridge to device llama for stock mobile (non-AOSP) Eliza builds.",
|
|
5
5
|
"type": "module",
|
|
6
6
|
"main": "dist/index.js",
|
|
@@ -10,36 +10,67 @@
|
|
|
10
10
|
"types": "./dist/index.d.ts",
|
|
11
11
|
"import": "./dist/index.js",
|
|
12
12
|
"default": "./dist/index.js"
|
|
13
|
+
},
|
|
14
|
+
"./android/bridge": {
|
|
15
|
+
"types": "./dist/android/bridge.d.ts",
|
|
16
|
+
"import": "./dist/android/bridge.js",
|
|
17
|
+
"default": "./dist/android/bridge.js"
|
|
18
|
+
},
|
|
19
|
+
"./ios/bridge": {
|
|
20
|
+
"types": "./dist/ios/bridge.d.ts",
|
|
21
|
+
"import": "./dist/ios/bridge.js",
|
|
22
|
+
"default": "./dist/ios/bridge.js"
|
|
23
|
+
},
|
|
24
|
+
"./mobile-device-bridge-bootstrap": {
|
|
25
|
+
"types": "./dist/mobile-device-bridge-bootstrap.d.ts",
|
|
26
|
+
"import": "./dist/mobile-device-bridge-bootstrap.js",
|
|
27
|
+
"default": "./dist/mobile-device-bridge-bootstrap.js"
|
|
28
|
+
},
|
|
29
|
+
"./shared/fs-shim": {
|
|
30
|
+
"types": "./dist/shared/fs-shim.d.ts",
|
|
31
|
+
"import": "./dist/shared/fs-shim.js",
|
|
32
|
+
"default": "./dist/shared/fs-shim.js"
|
|
33
|
+
},
|
|
34
|
+
"./*.css": "./dist/*.css",
|
|
35
|
+
"./*": {
|
|
36
|
+
"types": "./dist/*.d.ts",
|
|
37
|
+
"import": "./dist/*.js",
|
|
38
|
+
"default": "./dist/*.js"
|
|
13
39
|
}
|
|
14
40
|
},
|
|
15
41
|
"files": [
|
|
16
42
|
"dist"
|
|
17
43
|
],
|
|
18
44
|
"scripts": {
|
|
19
|
-
"build": "
|
|
20
|
-
"
|
|
45
|
+
"build": "bun run check:android-manifest && NODE_OPTIONS='--max-old-space-size=8192' tsup",
|
|
46
|
+
"check:android-manifest": "node scripts/check-android-manifest.mjs",
|
|
47
|
+
"dev": "tsup --watch",
|
|
21
48
|
"clean": "rm -rf dist .turbo node_modules",
|
|
22
49
|
"lint": "bunx @biomejs/biome check --write --unsafe .",
|
|
23
50
|
"lint:check": "bunx @biomejs/biome check .",
|
|
24
51
|
"format": "bunx @biomejs/biome format --write .",
|
|
25
52
|
"format:check": "bunx @biomejs/biome format .",
|
|
53
|
+
"test": "vitest run --config vitest.config.ts",
|
|
26
54
|
"typecheck": "tsc --noEmit"
|
|
27
55
|
},
|
|
28
56
|
"dependencies": {
|
|
29
|
-
"@elizaos/core": "2.0.
|
|
57
|
+
"@elizaos/core": "2.0.3-beta.2",
|
|
30
58
|
"ws": "^8.18.0"
|
|
31
59
|
},
|
|
32
60
|
"peerDependencies": {
|
|
33
|
-
"@elizaos/core": "2.0.
|
|
61
|
+
"@elizaos/core": "2.0.3-beta.2"
|
|
34
62
|
},
|
|
35
63
|
"devDependencies": {
|
|
36
64
|
"@biomejs/biome": "^2.4.14",
|
|
65
|
+
"@types/bun": "^1.3.12",
|
|
37
66
|
"@types/node": "24.12.2",
|
|
38
67
|
"@types/ws": "^8.5.13",
|
|
39
68
|
"tsup": "8.5.1",
|
|
40
|
-
"typescript": "^6.0.3"
|
|
69
|
+
"typescript": "^6.0.3",
|
|
70
|
+
"vitest": "^4.0.18"
|
|
41
71
|
},
|
|
42
72
|
"publishConfig": {
|
|
43
73
|
"access": "public"
|
|
44
|
-
}
|
|
74
|
+
},
|
|
75
|
+
"gitHead": "82fe0f44215954c2417328203f5bd6510985c1fc"
|
|
45
76
|
}
|
package/dist/index.d.ts
DELETED
|
@@ -1,85 +0,0 @@
|
|
|
1
|
-
import { Server } from 'node:http';
|
|
2
|
-
import { AgentRuntime } from '@elizaos/core';
|
|
3
|
-
|
|
4
|
-
/**
|
|
5
|
-
* Stock Capacitor mobile local-inference bridge.
|
|
6
|
-
*
|
|
7
|
-
* AOSP builds run llama.cpp inside the agent process via bun:ffi. Stock
|
|
8
|
-
* Capacitor Android/iOS builds cannot do that: llama.cpp is exposed to the
|
|
9
|
-
* WebView through the native Capacitor plugin. This module is the agent-side
|
|
10
|
-
* half of that path. It accepts a loopback WebSocket from the WebView,
|
|
11
|
-
* forwards TEXT_SMALL / TEXT_LARGE requests to the device, and lets the
|
|
12
|
-
* normal conversation routes keep using runtime model handlers.
|
|
13
|
-
*/
|
|
14
|
-
|
|
15
|
-
interface LocalInferenceLoadArgs {
|
|
16
|
-
modelPath: string;
|
|
17
|
-
contextSize?: number;
|
|
18
|
-
useGpu?: boolean;
|
|
19
|
-
maxThreads?: number;
|
|
20
|
-
draftModelPath?: string;
|
|
21
|
-
draftContextSize?: number;
|
|
22
|
-
draftMin?: number;
|
|
23
|
-
draftMax?: number;
|
|
24
|
-
speculativeSamples?: number;
|
|
25
|
-
mobileSpeculative?: boolean;
|
|
26
|
-
cacheTypeK?: string;
|
|
27
|
-
cacheTypeV?: string;
|
|
28
|
-
disableThinking?: boolean;
|
|
29
|
-
}
|
|
30
|
-
interface DeviceCapabilities {
|
|
31
|
-
platform: "ios" | "android" | "web";
|
|
32
|
-
deviceModel: string;
|
|
33
|
-
totalRamGb: number;
|
|
34
|
-
cpuCores: number;
|
|
35
|
-
gpu: {
|
|
36
|
-
backend: "metal" | "vulkan" | "gpu-delegate";
|
|
37
|
-
available: boolean;
|
|
38
|
-
} | null;
|
|
39
|
-
}
|
|
40
|
-
interface MobileDeviceBridgeStatus {
|
|
41
|
-
enabled: boolean;
|
|
42
|
-
connected: boolean;
|
|
43
|
-
devices: Array<{
|
|
44
|
-
deviceId: string;
|
|
45
|
-
capabilities: DeviceCapabilities;
|
|
46
|
-
loadedPath: string | null;
|
|
47
|
-
connectedSince: string;
|
|
48
|
-
}>;
|
|
49
|
-
primaryDeviceId: string | null;
|
|
50
|
-
pendingRequests: number;
|
|
51
|
-
modelPath: string | null;
|
|
52
|
-
}
|
|
53
|
-
declare class MobileDeviceBridge {
|
|
54
|
-
private wss;
|
|
55
|
-
private readonly devices;
|
|
56
|
-
private readonly pendingLoads;
|
|
57
|
-
private readonly pendingUnloads;
|
|
58
|
-
private readonly pendingGenerates;
|
|
59
|
-
private readonly pendingEmbeds;
|
|
60
|
-
status(): MobileDeviceBridgeStatus;
|
|
61
|
-
attachToHttpServer(server: Server): Promise<void>;
|
|
62
|
-
private handleConnection;
|
|
63
|
-
private handleDeviceMessage;
|
|
64
|
-
private primaryDevice;
|
|
65
|
-
private sendToPrimary;
|
|
66
|
-
loadModel(args: LocalInferenceLoadArgs): Promise<void>;
|
|
67
|
-
unloadModel(): Promise<void>;
|
|
68
|
-
generate(args: {
|
|
69
|
-
prompt: string;
|
|
70
|
-
stopSequences?: string[];
|
|
71
|
-
maxTokens?: number;
|
|
72
|
-
temperature?: number;
|
|
73
|
-
}): Promise<string>;
|
|
74
|
-
embed(args: {
|
|
75
|
-
input: string;
|
|
76
|
-
}): Promise<number[]>;
|
|
77
|
-
}
|
|
78
|
-
declare const mobileDeviceBridge: MobileDeviceBridge;
|
|
79
|
-
declare function getMobileDeviceBridgeStatus(): MobileDeviceBridgeStatus;
|
|
80
|
-
declare function loadMobileDeviceBridgeModel(modelPath: string, modelId?: string): Promise<void>;
|
|
81
|
-
declare function unloadMobileDeviceBridgeModel(): Promise<void>;
|
|
82
|
-
declare function attachMobileDeviceBridgeToServer(server: Server): Promise<void>;
|
|
83
|
-
declare function ensureMobileDeviceBridgeInferenceHandlers(runtime: AgentRuntime): Promise<boolean>;
|
|
84
|
-
|
|
85
|
-
export { type MobileDeviceBridgeStatus, attachMobileDeviceBridgeToServer, ensureMobileDeviceBridgeInferenceHandlers, getMobileDeviceBridgeStatus, loadMobileDeviceBridgeModel, mobileDeviceBridge, unloadMobileDeviceBridgeModel };
|
package/dist/index.js
DELETED
|
@@ -1,639 +0,0 @@
|
|
|
1
|
-
// src/mobile-device-bridge-bootstrap.ts
|
|
2
|
-
import { randomUUID } from "crypto";
|
|
3
|
-
import {
|
|
4
|
-
createWriteStream,
|
|
5
|
-
existsSync,
|
|
6
|
-
mkdirSync,
|
|
7
|
-
readdirSync,
|
|
8
|
-
readFileSync,
|
|
9
|
-
renameSync,
|
|
10
|
-
statSync,
|
|
11
|
-
unlinkSync
|
|
12
|
-
} from "fs";
|
|
13
|
-
import path from "path";
|
|
14
|
-
import { Readable } from "stream";
|
|
15
|
-
import { pipeline } from "stream/promises";
|
|
16
|
-
import {
|
|
17
|
-
logger,
|
|
18
|
-
ModelType,
|
|
19
|
-
resolveStateDir
|
|
20
|
-
} from "@elizaos/core";
|
|
21
|
-
var DEVICE_BRIDGE_PATH = "/api/local-inference/device-bridge";
|
|
22
|
-
var PROVIDER = "capacitor-llama";
|
|
23
|
-
var LOCAL_INFERENCE_PRIORITY = 0;
|
|
24
|
-
var DEFAULT_CALL_TIMEOUT_MS = 12e4;
|
|
25
|
-
var DEFAULT_LOAD_TIMEOUT_MS = 18e4;
|
|
26
|
-
var SERVICE_ENABLED = process.env.ELIZA_DEVICE_BRIDGE_ENABLED?.trim() === "1";
|
|
27
|
-
var registeredRuntimes = /* @__PURE__ */ new WeakSet();
|
|
28
|
-
var KNOWN_EMBEDDING_DIMENSIONS = {
|
|
29
|
-
"eliza-1-lite-0_6b": 1024,
|
|
30
|
-
"eliza-1-mobile-1_7b": 2048
|
|
31
|
-
};
|
|
32
|
-
var ELIZA_1_LOAD_METADATA = {
|
|
33
|
-
"eliza-1-lite-0_6b": { contextSize: 32768 },
|
|
34
|
-
"eliza-1-mobile-1_7b": { contextSize: 32768 },
|
|
35
|
-
"eliza-1-desktop-9b": { contextSize: 65536 },
|
|
36
|
-
"eliza-1-pro-27b": { contextSize: 131072 },
|
|
37
|
-
"eliza-1-server-h200": { contextSize: 262144 }
|
|
38
|
-
};
|
|
39
|
-
function isWsModule(value) {
|
|
40
|
-
return typeof value === "object" && value !== null && typeof value.WebSocketServer === "function" && typeof value.WebSocket === "function";
|
|
41
|
-
}
|
|
42
|
-
var MobileDeviceBridge = class {
|
|
43
|
-
wss = null;
|
|
44
|
-
devices = /* @__PURE__ */ new Map();
|
|
45
|
-
pendingLoads = /* @__PURE__ */ new Map();
|
|
46
|
-
pendingUnloads = /* @__PURE__ */ new Map();
|
|
47
|
-
pendingGenerates = /* @__PURE__ */ new Map();
|
|
48
|
-
pendingEmbeds = /* @__PURE__ */ new Map();
|
|
49
|
-
status() {
|
|
50
|
-
const devices = [...this.devices.values()].map((device) => ({
|
|
51
|
-
deviceId: device.deviceId,
|
|
52
|
-
capabilities: device.capabilities,
|
|
53
|
-
loadedPath: device.loadedPath,
|
|
54
|
-
connectedSince: new Date(device.connectedAt).toISOString()
|
|
55
|
-
}));
|
|
56
|
-
return {
|
|
57
|
-
enabled: SERVICE_ENABLED,
|
|
58
|
-
connected: devices.length > 0,
|
|
59
|
-
devices,
|
|
60
|
-
primaryDeviceId: devices[0]?.deviceId ?? null,
|
|
61
|
-
pendingRequests: this.pendingLoads.size + this.pendingUnloads.size + this.pendingGenerates.size + this.pendingEmbeds.size,
|
|
62
|
-
modelPath: resolveLocalModelPath("TEXT_LARGE")
|
|
63
|
-
};
|
|
64
|
-
}
|
|
65
|
-
async attachToHttpServer(server) {
|
|
66
|
-
if (!SERVICE_ENABLED || this.wss) return;
|
|
67
|
-
const wsModule = await import("ws");
|
|
68
|
-
if (!isWsModule(wsModule)) {
|
|
69
|
-
throw new Error("ws module did not expose WebSocketServer/WebSocket");
|
|
70
|
-
}
|
|
71
|
-
const ws = wsModule;
|
|
72
|
-
const wss = new ws.WebSocketServer({
|
|
73
|
-
noServer: true,
|
|
74
|
-
maxPayload: 1024 * 1024
|
|
75
|
-
});
|
|
76
|
-
this.wss = wss;
|
|
77
|
-
wss.on("error", (err) => {
|
|
78
|
-
logger.warn("[mobile-device-bridge] WSS error:", err.message);
|
|
79
|
-
});
|
|
80
|
-
server.on("upgrade", (request, socket, head) => {
|
|
81
|
-
const url = new URL(request.url ?? "/", "http://localhost");
|
|
82
|
-
if (url.pathname !== DEVICE_BRIDGE_PATH) return;
|
|
83
|
-
wss.handleUpgrade(request, socket, head, (client) => {
|
|
84
|
-
this.handleConnection(client, ws.WebSocket);
|
|
85
|
-
});
|
|
86
|
-
});
|
|
87
|
-
logger.info(
|
|
88
|
-
`[mobile-device-bridge] Listening for Capacitor device bridge at ${DEVICE_BRIDGE_PATH}`
|
|
89
|
-
);
|
|
90
|
-
}
|
|
91
|
-
handleConnection(socket, WsCtor) {
|
|
92
|
-
let registeredDeviceId = null;
|
|
93
|
-
socket.on("message", (raw) => {
|
|
94
|
-
let msg;
|
|
95
|
-
try {
|
|
96
|
-
const text = typeof raw === "string" ? raw : raw.toString("utf8");
|
|
97
|
-
msg = JSON.parse(text);
|
|
98
|
-
} catch {
|
|
99
|
-
logger.warn("[mobile-device-bridge] Ignoring non-JSON frame");
|
|
100
|
-
return;
|
|
101
|
-
}
|
|
102
|
-
if (!registeredDeviceId) {
|
|
103
|
-
if (msg.type !== "register") {
|
|
104
|
-
socket.close(4002, "must-register-first");
|
|
105
|
-
return;
|
|
106
|
-
}
|
|
107
|
-
registeredDeviceId = msg.payload.deviceId;
|
|
108
|
-
this.devices.set(registeredDeviceId, {
|
|
109
|
-
deviceId: registeredDeviceId,
|
|
110
|
-
socket,
|
|
111
|
-
capabilities: msg.payload.capabilities,
|
|
112
|
-
loadedPath: msg.payload.loadedPath,
|
|
113
|
-
connectedAt: Date.now()
|
|
114
|
-
});
|
|
115
|
-
logger.info(
|
|
116
|
-
`[mobile-device-bridge] Device connected: ${registeredDeviceId} (${msg.payload.capabilities.platform})`
|
|
117
|
-
);
|
|
118
|
-
return;
|
|
119
|
-
}
|
|
120
|
-
this.handleDeviceMessage(msg);
|
|
121
|
-
});
|
|
122
|
-
socket.on("close", () => {
|
|
123
|
-
if (!registeredDeviceId) return;
|
|
124
|
-
const current = this.devices.get(registeredDeviceId);
|
|
125
|
-
if (current?.socket === socket) {
|
|
126
|
-
this.devices.delete(registeredDeviceId);
|
|
127
|
-
logger.info(
|
|
128
|
-
`[mobile-device-bridge] Device disconnected: ${registeredDeviceId}`
|
|
129
|
-
);
|
|
130
|
-
}
|
|
131
|
-
});
|
|
132
|
-
socket.on("error", (err) => {
|
|
133
|
-
logger.warn("[mobile-device-bridge] Socket error:", err.message);
|
|
134
|
-
});
|
|
135
|
-
const heartbeat = setInterval(() => {
|
|
136
|
-
if (!registeredDeviceId || socket.readyState !== WsCtor.OPEN) return;
|
|
137
|
-
try {
|
|
138
|
-
socket.send(JSON.stringify({ type: "ping", at: Date.now() }));
|
|
139
|
-
} catch {
|
|
140
|
-
clearInterval(heartbeat);
|
|
141
|
-
}
|
|
142
|
-
}, 15e3);
|
|
143
|
-
if (typeof heartbeat === "object" && "unref" in heartbeat) {
|
|
144
|
-
heartbeat.unref();
|
|
145
|
-
}
|
|
146
|
-
}
|
|
147
|
-
handleDeviceMessage(msg) {
|
|
148
|
-
if (msg.type === "pong" || msg.type === "register") return;
|
|
149
|
-
if (msg.type === "loadResult") {
|
|
150
|
-
const pending = this.pendingLoads.get(msg.correlationId);
|
|
151
|
-
if (!pending) return;
|
|
152
|
-
clearTimeout(pending.timeout);
|
|
153
|
-
this.pendingLoads.delete(msg.correlationId);
|
|
154
|
-
if (msg.ok === true) {
|
|
155
|
-
const device = this.devices.get(pending.routedDeviceId);
|
|
156
|
-
if (device) device.loadedPath = msg.loadedPath;
|
|
157
|
-
pending.resolve(void 0);
|
|
158
|
-
} else {
|
|
159
|
-
pending.reject(new Error(msg.error));
|
|
160
|
-
}
|
|
161
|
-
return;
|
|
162
|
-
}
|
|
163
|
-
if (msg.type === "unloadResult") {
|
|
164
|
-
const pending = this.pendingUnloads.get(msg.correlationId);
|
|
165
|
-
if (!pending) return;
|
|
166
|
-
clearTimeout(pending.timeout);
|
|
167
|
-
this.pendingUnloads.delete(msg.correlationId);
|
|
168
|
-
if (msg.ok === true) {
|
|
169
|
-
const device = this.devices.get(pending.routedDeviceId);
|
|
170
|
-
if (device) device.loadedPath = null;
|
|
171
|
-
pending.resolve(void 0);
|
|
172
|
-
} else {
|
|
173
|
-
pending.reject(new Error(msg.error));
|
|
174
|
-
}
|
|
175
|
-
return;
|
|
176
|
-
}
|
|
177
|
-
if (msg.type === "generateResult") {
|
|
178
|
-
const pending = this.pendingGenerates.get(msg.correlationId);
|
|
179
|
-
if (!pending) return;
|
|
180
|
-
clearTimeout(pending.timeout);
|
|
181
|
-
this.pendingGenerates.delete(msg.correlationId);
|
|
182
|
-
if (msg.ok === true) {
|
|
183
|
-
pending.resolve(msg.text);
|
|
184
|
-
} else {
|
|
185
|
-
pending.reject(new Error(msg.error));
|
|
186
|
-
}
|
|
187
|
-
return;
|
|
188
|
-
}
|
|
189
|
-
if (msg.type === "embedResult") {
|
|
190
|
-
const pending = this.pendingEmbeds.get(msg.correlationId);
|
|
191
|
-
if (!pending) return;
|
|
192
|
-
clearTimeout(pending.timeout);
|
|
193
|
-
this.pendingEmbeds.delete(msg.correlationId);
|
|
194
|
-
if (msg.ok === true) {
|
|
195
|
-
pending.resolve(msg.embedding);
|
|
196
|
-
} else {
|
|
197
|
-
pending.reject(new Error(msg.error));
|
|
198
|
-
}
|
|
199
|
-
}
|
|
200
|
-
}
|
|
201
|
-
primaryDevice() {
|
|
202
|
-
return this.devices.values().next().value ?? null;
|
|
203
|
-
}
|
|
204
|
-
sendToPrimary(pendingMap, makeMessage, timeoutMs, timeoutMessage) {
|
|
205
|
-
const device = this.primaryDevice();
|
|
206
|
-
if (!device) {
|
|
207
|
-
return Promise.reject(
|
|
208
|
-
new Error(
|
|
209
|
-
"DEVICE_DISCONNECTED: no Capacitor llama device bridge attached"
|
|
210
|
-
)
|
|
211
|
-
);
|
|
212
|
-
}
|
|
213
|
-
const correlationId = randomUUID();
|
|
214
|
-
const message = makeMessage(correlationId);
|
|
215
|
-
return new Promise((resolve, reject) => {
|
|
216
|
-
const timeout = setTimeout(() => {
|
|
217
|
-
pendingMap.delete(correlationId);
|
|
218
|
-
reject(new Error(timeoutMessage));
|
|
219
|
-
}, timeoutMs);
|
|
220
|
-
if (typeof timeout === "object" && "unref" in timeout) {
|
|
221
|
-
timeout.unref();
|
|
222
|
-
}
|
|
223
|
-
pendingMap.set(correlationId, {
|
|
224
|
-
resolve,
|
|
225
|
-
reject,
|
|
226
|
-
timeout,
|
|
227
|
-
routedDeviceId: device.deviceId
|
|
228
|
-
});
|
|
229
|
-
try {
|
|
230
|
-
device.socket.send(JSON.stringify(message));
|
|
231
|
-
} catch (err) {
|
|
232
|
-
clearTimeout(timeout);
|
|
233
|
-
pendingMap.delete(correlationId);
|
|
234
|
-
reject(err instanceof Error ? err : new Error(String(err)));
|
|
235
|
-
}
|
|
236
|
-
});
|
|
237
|
-
}
|
|
238
|
-
async loadModel(args) {
|
|
239
|
-
const device = this.primaryDevice();
|
|
240
|
-
if (device?.loadedPath === args.modelPath) return;
|
|
241
|
-
return this.sendToPrimary(
|
|
242
|
-
this.pendingLoads,
|
|
243
|
-
(correlationId) => ({
|
|
244
|
-
type: "load",
|
|
245
|
-
correlationId,
|
|
246
|
-
...args
|
|
247
|
-
}),
|
|
248
|
-
readTimeoutMs("ELIZA_DEVICE_LOAD_TIMEOUT_MS", DEFAULT_LOAD_TIMEOUT_MS),
|
|
249
|
-
"DEVICE_TIMEOUT: model load exceeded deadline"
|
|
250
|
-
);
|
|
251
|
-
}
|
|
252
|
-
async unloadModel() {
|
|
253
|
-
const device = this.primaryDevice();
|
|
254
|
-
if (!device?.loadedPath) return;
|
|
255
|
-
return this.sendToPrimary(
|
|
256
|
-
this.pendingUnloads,
|
|
257
|
-
(correlationId) => ({ type: "unload", correlationId }),
|
|
258
|
-
readTimeoutMs(
|
|
259
|
-
"ELIZA_DEVICE_GENERATE_TIMEOUT_MS",
|
|
260
|
-
DEFAULT_CALL_TIMEOUT_MS
|
|
261
|
-
),
|
|
262
|
-
"DEVICE_TIMEOUT: unload exceeded deadline"
|
|
263
|
-
);
|
|
264
|
-
}
|
|
265
|
-
generate(args) {
|
|
266
|
-
return this.sendToPrimary(
|
|
267
|
-
this.pendingGenerates,
|
|
268
|
-
(correlationId) => ({
|
|
269
|
-
type: "generate",
|
|
270
|
-
correlationId,
|
|
271
|
-
prompt: args.prompt,
|
|
272
|
-
stopSequences: args.stopSequences,
|
|
273
|
-
maxTokens: args.maxTokens,
|
|
274
|
-
temperature: args.temperature
|
|
275
|
-
}),
|
|
276
|
-
readTimeoutMs(
|
|
277
|
-
"ELIZA_DEVICE_GENERATE_TIMEOUT_MS",
|
|
278
|
-
DEFAULT_CALL_TIMEOUT_MS
|
|
279
|
-
),
|
|
280
|
-
"DEVICE_TIMEOUT: no device responded within deadline"
|
|
281
|
-
);
|
|
282
|
-
}
|
|
283
|
-
embed(args) {
|
|
284
|
-
return this.sendToPrimary(
|
|
285
|
-
this.pendingEmbeds,
|
|
286
|
-
(correlationId) => ({
|
|
287
|
-
type: "embed",
|
|
288
|
-
correlationId,
|
|
289
|
-
input: args.input
|
|
290
|
-
}),
|
|
291
|
-
readTimeoutMs("ELIZA_DEVICE_EMBED_TIMEOUT_MS", DEFAULT_CALL_TIMEOUT_MS),
|
|
292
|
-
"DEVICE_TIMEOUT: no device returned embeddings within deadline"
|
|
293
|
-
);
|
|
294
|
-
}
|
|
295
|
-
};
|
|
296
|
-
var mobileDeviceBridge = new MobileDeviceBridge();
|
|
297
|
-
function readTimeoutMs(envKey, fallback) {
|
|
298
|
-
const parsed = Number.parseInt(process.env[envKey]?.trim() ?? "", 10);
|
|
299
|
-
return Number.isFinite(parsed) && parsed > 0 ? parsed : fallback;
|
|
300
|
-
}
|
|
301
|
-
function modelsDir() {
|
|
302
|
-
return path.join(resolveStateDir(), "local-inference", "models");
|
|
303
|
-
}
|
|
304
|
-
function registryPath() {
|
|
305
|
-
return path.join(resolveStateDir(), "local-inference", "registry.json");
|
|
306
|
-
}
|
|
307
|
-
function assignmentsPath() {
|
|
308
|
-
return path.join(resolveStateDir(), "local-inference", "assignments.json");
|
|
309
|
-
}
|
|
310
|
-
function readJsonFile(filePath) {
|
|
311
|
-
try {
|
|
312
|
-
return JSON.parse(readFileSync(filePath, "utf8"));
|
|
313
|
-
} catch {
|
|
314
|
-
return null;
|
|
315
|
-
}
|
|
316
|
-
}
|
|
317
|
-
function positiveInteger(value) {
|
|
318
|
-
const numeric = typeof value === "number" ? value : typeof value === "string" ? Number.parseInt(value, 10) : Number.NaN;
|
|
319
|
-
return Number.isInteger(numeric) && numeric > 0 ? numeric : null;
|
|
320
|
-
}
|
|
321
|
-
function resolveFromEnv(slot) {
|
|
322
|
-
const key = slot === "TEXT_EMBEDDING" ? "ELIZA_LOCAL_EMBEDDING_MODEL_PATH" : "ELIZA_LOCAL_CHAT_MODEL_PATH";
|
|
323
|
-
const specific = process.env[key]?.trim();
|
|
324
|
-
if (specific && existsSync(specific)) return specific;
|
|
325
|
-
const fallback = process.env.ELIZA_LOCAL_MODEL_PATH?.trim();
|
|
326
|
-
if (fallback && existsSync(fallback)) return fallback;
|
|
327
|
-
return null;
|
|
328
|
-
}
|
|
329
|
-
function resolveFromRegistry(slot) {
|
|
330
|
-
const assignments = readJsonFile(
|
|
331
|
-
assignmentsPath()
|
|
332
|
-
)?.assignments;
|
|
333
|
-
const assigned = assignments?.[slot];
|
|
334
|
-
if (typeof assigned !== "string" || !assigned.trim()) return null;
|
|
335
|
-
const models = readRegistryModels();
|
|
336
|
-
const matched = models.find((model) => model.id === assigned);
|
|
337
|
-
return typeof matched?.path === "string" && existsSync(matched.path) ? matched.path : null;
|
|
338
|
-
}
|
|
339
|
-
function readRegistryModels() {
|
|
340
|
-
return readJsonFile(registryPath())?.models ?? [];
|
|
341
|
-
}
|
|
342
|
-
function resolveAssignedRegistryModel(slot) {
|
|
343
|
-
const assignments = readJsonFile(
|
|
344
|
-
assignmentsPath()
|
|
345
|
-
)?.assignments;
|
|
346
|
-
const assigned = assignments?.[slot];
|
|
347
|
-
if (typeof assigned !== "string" || !assigned.trim()) return null;
|
|
348
|
-
const models = readRegistryModels();
|
|
349
|
-
const matched = models.find((model) => model.id === assigned);
|
|
350
|
-
if (typeof matched?.path !== "string" || !existsSync(matched.path)) {
|
|
351
|
-
return null;
|
|
352
|
-
}
|
|
353
|
-
return {
|
|
354
|
-
id: assigned,
|
|
355
|
-
path: matched.path,
|
|
356
|
-
dimensions: matched.dimensions,
|
|
357
|
-
embeddingDimension: matched.embeddingDimension,
|
|
358
|
-
embeddingDimensions: matched.embeddingDimensions
|
|
359
|
-
};
|
|
360
|
-
}
|
|
361
|
-
function resolveFromManifest(slot) {
|
|
362
|
-
const manifest = readJsonFile(
|
|
363
|
-
path.join(modelsDir(), "manifest.json")
|
|
364
|
-
);
|
|
365
|
-
const targetRole = slot === "TEXT_EMBEDDING" ? "embedding" : "chat";
|
|
366
|
-
for (const entry of manifest?.models ?? []) {
|
|
367
|
-
if (entry.role !== targetRole) continue;
|
|
368
|
-
const fileName = entry.ggufFile ?? entry.filename;
|
|
369
|
-
if (!fileName) continue;
|
|
370
|
-
const absolute = path.join(modelsDir(), fileName);
|
|
371
|
-
if (existsSync(absolute)) return absolute;
|
|
372
|
-
}
|
|
373
|
-
return null;
|
|
374
|
-
}
|
|
375
|
-
function resolveFirstGguf() {
|
|
376
|
-
const dir = modelsDir();
|
|
377
|
-
if (!existsSync(dir)) return null;
|
|
378
|
-
for (const name of readdirSync(dir)) {
|
|
379
|
-
if (!name.toLowerCase().endsWith(".gguf")) continue;
|
|
380
|
-
const absolute = path.join(dir, name);
|
|
381
|
-
if (existsSync(absolute)) return absolute;
|
|
382
|
-
}
|
|
383
|
-
return null;
|
|
384
|
-
}
|
|
385
|
-
function resolveLocalModelPath(slot) {
|
|
386
|
-
return resolveFromEnv(slot) ?? resolveFromRegistry(slot) ?? resolveFromManifest(slot) ?? resolveFirstGguf();
|
|
387
|
-
}
|
|
388
|
-
function buildLoadArgsFromRegistryModel(model) {
|
|
389
|
-
const args = { modelPath: model.path };
|
|
390
|
-
const eliza1 = ELIZA_1_LOAD_METADATA[model.id];
|
|
391
|
-
if (eliza1) args.contextSize = eliza1.contextSize;
|
|
392
|
-
return args;
|
|
393
|
-
}
|
|
394
|
-
function resolveLocalLoadArgs(slot) {
|
|
395
|
-
const envPath = resolveFromEnv(slot);
|
|
396
|
-
if (envPath) return { modelPath: envPath };
|
|
397
|
-
const registryModel = resolveAssignedRegistryModel(slot);
|
|
398
|
-
if (registryModel) return buildLoadArgsFromRegistryModel(registryModel);
|
|
399
|
-
const manifestPath = resolveFromManifest(slot);
|
|
400
|
-
if (manifestPath) return { modelPath: manifestPath };
|
|
401
|
-
const firstGguf = resolveFirstGguf();
|
|
402
|
-
return firstGguf ? { modelPath: firstGguf } : null;
|
|
403
|
-
}
|
|
404
|
-
var RECOMMENDED_MODELS = {
|
|
405
|
-
TEXT_SMALL: {
|
|
406
|
-
id: "eliza-1-lite-0_6b",
|
|
407
|
-
hfRepo: "elizaos/eliza-1-lite-0_6b",
|
|
408
|
-
ggufFile: "text/eliza-1-lite-0_6b-32k.gguf",
|
|
409
|
-
localFile: "eliza-1-lite-0_6b-32k.gguf"
|
|
410
|
-
},
|
|
411
|
-
TEXT_LARGE: {
|
|
412
|
-
id: "eliza-1-mobile-1_7b",
|
|
413
|
-
hfRepo: "elizaos/eliza-1-mobile-1_7b",
|
|
414
|
-
ggufFile: "text/eliza-1-mobile-1_7b-32k.gguf",
|
|
415
|
-
localFile: "eliza-1-mobile-1_7b-32k.gguf"
|
|
416
|
-
},
|
|
417
|
-
TEXT_EMBEDDING: {
|
|
418
|
-
id: "eliza-1-lite-0_6b",
|
|
419
|
-
hfRepo: "elizaos/eliza-1-lite-0_6b",
|
|
420
|
-
ggufFile: "text/eliza-1-lite-0_6b-32k.gguf",
|
|
421
|
-
localFile: "eliza-1-lite-0_6b-32k.gguf"
|
|
422
|
-
}
|
|
423
|
-
};
|
|
424
|
-
var inflightDownloads = /* @__PURE__ */ new Map();
|
|
425
|
-
function buildHfResolveUrl(model) {
|
|
426
|
-
const encodedPath = model.ggufFile.split("/").map((segment) => encodeURIComponent(segment)).join("/");
|
|
427
|
-
return `https://huggingface.co/${model.hfRepo}/resolve/main/${encodedPath}?download=true`;
|
|
428
|
-
}
|
|
429
|
-
function buildRecommendedLoadArgs(slot, modelPath) {
|
|
430
|
-
const model = RECOMMENDED_MODELS[slot];
|
|
431
|
-
return buildLoadArgsFromRegistryModel({ id: model.id, path: modelPath });
|
|
432
|
-
}
|
|
433
|
-
async function downloadRecommendedModelFor(slot) {
|
|
434
|
-
const model = RECOMMENDED_MODELS[slot];
|
|
435
|
-
const dir = modelsDir();
|
|
436
|
-
mkdirSync(dir, { recursive: true });
|
|
437
|
-
const finalPath = path.join(
|
|
438
|
-
dir,
|
|
439
|
-
model.localFile ?? path.basename(model.ggufFile)
|
|
440
|
-
);
|
|
441
|
-
if (existsSync(finalPath)) {
|
|
442
|
-
const sz = statSync(finalPath).size;
|
|
443
|
-
if (!model.expectedSizeBytes || sz === model.expectedSizeBytes) {
|
|
444
|
-
return finalPath;
|
|
445
|
-
}
|
|
446
|
-
logger.warn(
|
|
447
|
-
`[mobile-device-bridge] ${model.ggufFile} present but size ${sz} != expected ${model.expectedSizeBytes}; re-downloading.`
|
|
448
|
-
);
|
|
449
|
-
try {
|
|
450
|
-
unlinkSync(finalPath);
|
|
451
|
-
} catch {
|
|
452
|
-
}
|
|
453
|
-
}
|
|
454
|
-
const dedupKey = model.id;
|
|
455
|
-
const existing = inflightDownloads.get(dedupKey);
|
|
456
|
-
if (existing) return existing;
|
|
457
|
-
const promise = (async () => {
|
|
458
|
-
const url = buildHfResolveUrl(model);
|
|
459
|
-
const stagingPath = `${finalPath}.part`;
|
|
460
|
-
try {
|
|
461
|
-
unlinkSync(stagingPath);
|
|
462
|
-
} catch {
|
|
463
|
-
}
|
|
464
|
-
logger.info(
|
|
465
|
-
`[mobile-device-bridge] Auto-downloading recommended ${slot} model ${model.id} from ${url}`
|
|
466
|
-
);
|
|
467
|
-
const response = await fetch(url, { redirect: "follow" });
|
|
468
|
-
if (!response.ok || !response.body) {
|
|
469
|
-
throw new Error(
|
|
470
|
-
`[mobile-device-bridge] Recommended-model download failed (${slot}): HTTP ${response.status} ${response.statusText} from ${url}`
|
|
471
|
-
);
|
|
472
|
-
}
|
|
473
|
-
await pipeline(
|
|
474
|
-
Readable.fromWeb(response.body),
|
|
475
|
-
createWriteStream(stagingPath)
|
|
476
|
-
);
|
|
477
|
-
const stagedSize = statSync(stagingPath).size;
|
|
478
|
-
if (model.expectedSizeBytes && stagedSize !== model.expectedSizeBytes) {
|
|
479
|
-
try {
|
|
480
|
-
unlinkSync(stagingPath);
|
|
481
|
-
} catch {
|
|
482
|
-
}
|
|
483
|
-
throw new Error(
|
|
484
|
-
`[mobile-device-bridge] Downloaded ${model.ggufFile} size ${stagedSize} != expected ${model.expectedSizeBytes}; aborting and removing partial file.`
|
|
485
|
-
);
|
|
486
|
-
}
|
|
487
|
-
renameSync(stagingPath, finalPath);
|
|
488
|
-
logger.info(
|
|
489
|
-
`[mobile-device-bridge] Auto-download complete: ${finalPath} (${stagedSize} bytes)`
|
|
490
|
-
);
|
|
491
|
-
return finalPath;
|
|
492
|
-
})();
|
|
493
|
-
inflightDownloads.set(dedupKey, promise);
|
|
494
|
-
try {
|
|
495
|
-
return await promise;
|
|
496
|
-
} finally {
|
|
497
|
-
inflightDownloads.delete(dedupKey);
|
|
498
|
-
}
|
|
499
|
-
}
|
|
500
|
-
async function resolveLoadArgsWithAutoDownload(slot) {
|
|
501
|
-
const existing = resolveLocalLoadArgs(slot);
|
|
502
|
-
if (existing) return existing;
|
|
503
|
-
if (process.env.ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD?.trim() === "1") {
|
|
504
|
-
return null;
|
|
505
|
-
}
|
|
506
|
-
const downloaded = await downloadRecommendedModelFor(slot);
|
|
507
|
-
return buildRecommendedLoadArgs(slot, downloaded);
|
|
508
|
-
}
|
|
509
|
-
function resolveEmbeddingDimension() {
|
|
510
|
-
const assigned = resolveAssignedRegistryModel("TEXT_EMBEDDING");
|
|
511
|
-
return positiveInteger(process.env.ELIZA_LOCAL_EMBEDDING_DIMENSIONS) ?? positiveInteger(process.env.TEXT_EMBEDDING_DIMENSIONS) ?? positiveInteger(assigned?.dimensions) ?? positiveInteger(assigned?.embeddingDimension) ?? positiveInteger(assigned?.embeddingDimensions) ?? (assigned?.id ? KNOWN_EMBEDDING_DIMENSIONS[assigned.id] : null) ?? KNOWN_EMBEDDING_DIMENSIONS[RECOMMENDED_MODELS.TEXT_EMBEDDING.id] ?? 1024;
|
|
512
|
-
}
|
|
513
|
-
function makeGenerateHandler(slot) {
|
|
514
|
-
return async (_runtime, params) => {
|
|
515
|
-
const loadArgs = await resolveLoadArgsWithAutoDownload(slot);
|
|
516
|
-
if (!loadArgs) {
|
|
517
|
-
throw new Error(
|
|
518
|
-
`[mobile-device-bridge] No local GGUF model installed under ${modelsDir()} and auto-download is disabled (ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD=1). Install a model or unset the disable flag.`
|
|
519
|
-
);
|
|
520
|
-
}
|
|
521
|
-
await mobileDeviceBridge.loadModel(loadArgs);
|
|
522
|
-
return mobileDeviceBridge.generate({
|
|
523
|
-
prompt: params.prompt ?? "",
|
|
524
|
-
stopSequences: params.stopSequences,
|
|
525
|
-
maxTokens: params.maxTokens,
|
|
526
|
-
temperature: params.temperature
|
|
527
|
-
});
|
|
528
|
-
};
|
|
529
|
-
}
|
|
530
|
-
function extractEmbeddingText(params) {
|
|
531
|
-
if (params === null) return "";
|
|
532
|
-
if (typeof params === "string") return params;
|
|
533
|
-
return params.text;
|
|
534
|
-
}
|
|
535
|
-
function makeEmbeddingHandler() {
|
|
536
|
-
return async (_runtime, params) => {
|
|
537
|
-
if (params === null) {
|
|
538
|
-
return new Array(resolveEmbeddingDimension()).fill(0);
|
|
539
|
-
}
|
|
540
|
-
let modelPath = resolveLocalModelPath("TEXT_EMBEDDING");
|
|
541
|
-
let loadArgs = modelPath ? { modelPath } : null;
|
|
542
|
-
if (!modelPath) {
|
|
543
|
-
if (process.env.ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD?.trim() === "1") {
|
|
544
|
-
throw new Error(
|
|
545
|
-
`[mobile-device-bridge] No local GGUF embedding model installed under ${modelsDir()} and auto-download is disabled.`
|
|
546
|
-
);
|
|
547
|
-
}
|
|
548
|
-
modelPath = await downloadRecommendedModelFor("TEXT_EMBEDDING");
|
|
549
|
-
loadArgs = buildRecommendedLoadArgs("TEXT_EMBEDDING", modelPath);
|
|
550
|
-
}
|
|
551
|
-
if (!loadArgs) {
|
|
552
|
-
throw new Error(
|
|
553
|
-
`[mobile-device-bridge] No local GGUF embedding model resolved for ${modelsDir()}.`
|
|
554
|
-
);
|
|
555
|
-
}
|
|
556
|
-
await mobileDeviceBridge.loadModel(loadArgs);
|
|
557
|
-
return mobileDeviceBridge.embed({
|
|
558
|
-
input: extractEmbeddingText(params)
|
|
559
|
-
});
|
|
560
|
-
};
|
|
561
|
-
}
|
|
562
|
-
function getMobileDeviceBridgeStatus() {
|
|
563
|
-
return mobileDeviceBridge.status();
|
|
564
|
-
}
|
|
565
|
-
async function loadMobileDeviceBridgeModel(modelPath, modelId) {
|
|
566
|
-
await mobileDeviceBridge.loadModel(
|
|
567
|
-
modelId ? buildLoadArgsFromRegistryModel({ id: modelId, path: modelPath }) : { modelPath }
|
|
568
|
-
);
|
|
569
|
-
}
|
|
570
|
-
async function unloadMobileDeviceBridgeModel() {
|
|
571
|
-
await mobileDeviceBridge.unloadModel();
|
|
572
|
-
}
|
|
573
|
-
async function attachMobileDeviceBridgeToServer(server) {
|
|
574
|
-
await mobileDeviceBridge.attachToHttpServer(server);
|
|
575
|
-
}
|
|
576
|
-
async function ensureMobileDeviceBridgeInferenceHandlers(runtime) {
|
|
577
|
-
logger.debug("[mobile-device-bridge] Bootstrap entered");
|
|
578
|
-
if (!SERVICE_ENABLED || process.env.ELIZA_LOCAL_LLAMA?.trim() === "1") {
|
|
579
|
-
logger.debug("[mobile-device-bridge] Disabled or AOSP local llama active");
|
|
580
|
-
return false;
|
|
581
|
-
}
|
|
582
|
-
if (registeredRuntimes.has(runtime)) {
|
|
583
|
-
logger.debug("[mobile-device-bridge] Handlers already registered");
|
|
584
|
-
return true;
|
|
585
|
-
}
|
|
586
|
-
const runtimeWithRegistration = runtime;
|
|
587
|
-
if (typeof runtimeWithRegistration.getModel !== "function" || typeof runtimeWithRegistration.registerModel !== "function") {
|
|
588
|
-
logger.error(
|
|
589
|
-
"[mobile-device-bridge] Runtime is missing getModel/registerModel; cannot wire handlers."
|
|
590
|
-
);
|
|
591
|
-
return false;
|
|
592
|
-
}
|
|
593
|
-
runtimeWithRegistration.registerModel(
|
|
594
|
-
ModelType.TEXT_SMALL,
|
|
595
|
-
makeGenerateHandler("TEXT_SMALL"),
|
|
596
|
-
PROVIDER,
|
|
597
|
-
LOCAL_INFERENCE_PRIORITY
|
|
598
|
-
);
|
|
599
|
-
runtimeWithRegistration.registerModel(
|
|
600
|
-
ModelType.TEXT_LARGE,
|
|
601
|
-
makeGenerateHandler("TEXT_LARGE"),
|
|
602
|
-
PROVIDER,
|
|
603
|
-
LOCAL_INFERENCE_PRIORITY
|
|
604
|
-
);
|
|
605
|
-
if (!resolveLocalLoadArgs("TEXT_SMALL") && process.env.ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD?.trim() !== "1") {
|
|
606
|
-
downloadRecommendedModelFor("TEXT_SMALL").catch(
|
|
607
|
-
(err) => logger.warn(
|
|
608
|
-
`[mobile-device-bridge] Background chat-model download failed: ${err.message}`
|
|
609
|
-
)
|
|
610
|
-
);
|
|
611
|
-
}
|
|
612
|
-
runtimeWithRegistration.registerModel(
|
|
613
|
-
ModelType.TEXT_EMBEDDING,
|
|
614
|
-
makeEmbeddingHandler(),
|
|
615
|
-
PROVIDER,
|
|
616
|
-
LOCAL_INFERENCE_PRIORITY
|
|
617
|
-
);
|
|
618
|
-
const embeddingModelPath = resolveLocalModelPath("TEXT_EMBEDDING");
|
|
619
|
-
if (!embeddingModelPath && process.env.ELIZA_DISABLE_MODEL_AUTO_DOWNLOAD?.trim() !== "1") {
|
|
620
|
-
downloadRecommendedModelFor("TEXT_EMBEDDING").catch(
|
|
621
|
-
(err) => logger.warn(
|
|
622
|
-
`[mobile-device-bridge] Background embedding-model download failed: ${err.message}`
|
|
623
|
-
)
|
|
624
|
-
);
|
|
625
|
-
}
|
|
626
|
-
logger.info(
|
|
627
|
-
`[mobile-device-bridge] Registered ${PROVIDER} handlers for TEXT_SMALL / TEXT_LARGE${embeddingModelPath ? " / TEXT_EMBEDDING" : ""} at priority ${LOCAL_INFERENCE_PRIORITY}`
|
|
628
|
-
);
|
|
629
|
-
registeredRuntimes.add(runtime);
|
|
630
|
-
return true;
|
|
631
|
-
}
|
|
632
|
-
export {
|
|
633
|
-
attachMobileDeviceBridgeToServer,
|
|
634
|
-
ensureMobileDeviceBridgeInferenceHandlers,
|
|
635
|
-
getMobileDeviceBridgeStatus,
|
|
636
|
-
loadMobileDeviceBridgeModel,
|
|
637
|
-
mobileDeviceBridge,
|
|
638
|
-
unloadMobileDeviceBridgeModel
|
|
639
|
-
};
|