local-embed 0.2.3__tar.gz → 0.2.4__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of local-embed might be problematic. Click here for more details.

@@ -1,5 +1,5 @@
1
1
  # Runtime config — personal per-user settings. The three initial configs
2
- # (slife.json5 / local_embed.json5 / mcp-plugin.json5) are git-tracked so the
2
+ # (slife.json5 / local_embed.json5 / tools.json5) are git-tracked so the
3
3
  # one-click installers can seed them as out-of-the-box defaults. Copies in a
4
4
  # user's data dir (~/.slife/) remain personal and are NOT tracked.
5
5
 
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.3
2
2
  Name: local-embed
3
- Version: 0.2.3
3
+ Version: 0.2.4
4
4
  Summary: Standalone local embedding server — GGUF / transformer models as an OpenAI-compatible /v1/embeddings service
5
5
  Project-URL: Homepage, https://github.com/juzcn/slife
6
6
  Project-URL: Repository, https://github.com/juzcn/slife
@@ -16,7 +16,7 @@ Classifier: Programming Language :: Python :: 3.13
16
16
  Classifier: Topic :: Terminals
17
17
  Requires-Python: >=3.13
18
18
  Requires-Dist: fastmcp>=2.0.0
19
- Requires-Dist: json5>=0.15.0
19
+ Requires-Dist: json-five>=1.1.2
20
20
  Requires-Dist: starlette>=0.37.0
21
21
  Provides-Extra: gguf
22
22
  Requires-Dist: llama-cpp-python==0.3.34; extra == 'gguf'
@@ -51,15 +51,18 @@ them ([Model weights](#model-weights)).
51
51
  vector DB / RAG pipeline ──────┘ ▼
52
52
  ┌───────────────────────┐
53
53
  slife (plugin host) ─── /mcp ────────────> │ local-embed │
54
- │ ONE loaded model │
54
+ │ models load on demand │
55
55
  └───────────────────────┘
56
56
  ```
57
57
 
58
58
  ## Features
59
59
 
60
60
  - **OpenAI-compatible** `/v1/embeddings` + Models API + `/health`.
61
- - Two backends (GGUF / transformer), **many models configured, one active**.
62
- - **Lazy load** — startup is fast; the model materialises on first use.
61
+ - Two backends (GGUF / transformer), **many models as peers — each request
62
+ names the one it wants** (standard OpenAI semantics, no "active model").
63
+ - **Lazy load** — startup is fast; each model materialises on the first
64
+ request that names it (per-model `autoload: true` preloads a specific
65
+ model at startup).
63
66
  - **Real dimension** reported after load (a guessed width is never served).
64
67
  - Runs standalone, and can also be **loaded as a slife plugin** (see
65
68
  [Loading as a slife plugin](#loading-as-a-slife-plugin)).
@@ -68,18 +71,36 @@ Requires Python ≥ 3.13.
68
71
 
69
72
  ## Install
70
73
 
71
- The core package is `fastmcp` + `starlette` + `json5`; the model backends are
74
+ The core package is `fastmcp` + `starlette` + `json-five` (the JSON5 parser,
75
+ imported as `json5`); the model backends are
72
76
  optional extras.
73
77
 
78
+ One-click installers (install `uv` if needed, then `uv tool install --force
79
+ local-embed` — the backend is **not** included, see below):
80
+
81
+ ```bash
82
+ # macOS / Linux / WSL
83
+ curl -fsSL https://raw.githubusercontent.com/juzcn/slife/main/local-embed/install.sh | bash
84
+
85
+ # Windows PowerShell
86
+ powershell -ExecutionPolicy Bypass -Command "irm https://raw.githubusercontent.com/juzcn/slife/main/local-embed/install.ps1 | iex"
87
+ ```
88
+
89
+ Manual installs are equivalent:
90
+
74
91
  ```bash
75
92
  # the app + CLI (standalone tool)
76
- uv tool install local-embed
93
+ uv tool install --force local-embed
77
94
 
78
95
  # with a backend:
79
- uv tool install "local-embed[transformer]"
80
- uv tool install "local-embed[gguf]"
96
+ uv tool install --force "local-embed[transformer]"
97
+ uv tool install --force "local-embed[gguf]"
81
98
  ```
82
99
 
100
+ `--force` (used by the installers too) is **idempotent** — the first run
101
+ installs, re-running **updates** to the latest PyPI release. A plain
102
+ `uv tool install local-embed` no-ops when the tool is already installed.
103
+
83
104
  If you run local-embed inside an *existing* environment (alongside another
84
105
  tool), install the backend into **that** environment's interpreter instead of
85
106
  creating a separate one:
@@ -140,15 +161,14 @@ Installing does **not** fetch a model — get the weights first.
140
161
 
141
162
  ## Configuration: `local_embed.json5`
142
163
 
143
- Everything — host, port, models, active model, backend — comes from
164
+ Everything — host, port, models, backend — comes from
144
165
  `local_embed.json5`, written by the CLI helpers or by hand. Path resolution:
145
166
  `$LOCAL_EMBED_FILE` > slife project root (dev) > `~/.local-embed/local_embed.json5`.
146
167
 
147
168
  ```json5
148
169
  {
149
- active_model: "bge-m3", // model served by default
150
170
  models: {
151
- "bge-m3": { backend: "gguf", gguf_path: "…", device: "" },
171
+ "bge-m3": { backend: "gguf", gguf_path: "…", device: "", autoload: false },
152
172
  "bge-m3-transformer": { backend: "transformer", model: "BAAI/bge-m3", device: "" }
153
173
  },
154
174
  env: { // injected into this process before any model loads
@@ -160,8 +180,14 @@ Everything — host, port, models, active model, backend — comes from
160
180
  }
161
181
  ```
162
182
 
163
- - `models` — map of name → `{backend, gguf_path | model, device, max_tokens}`.
164
- - `active_model` — key into `models`.
183
+ - `models` — map of name → `{backend, gguf_path | model, device, max_tokens, autoload}`.
184
+ Every configured model is a peer; there is **no `active_model`** — each
185
+ request names the model it wants (standard OpenAI semantics).
186
+ - `autoload` (per model, default `false`) — a model's weights are large and
187
+ memory-hungry, so nothing loads until a request names it (**lazy**).
188
+ `autoload: true` on one model eager-loads just that model in the
189
+ background at startup (memory paid up front for a warm first embed); every
190
+ unflagged model stays lazy.
165
191
  - `env:` — injected before any backend loads; an existing shell env var wins.
166
192
  Without `HF_HUB_CACHE`, transformer repos resolve against the default cache
167
193
  and a model downloaded elsewhere is silently re-fetched.
@@ -232,8 +258,8 @@ curl http://127.0.0.1:17347/v1/embeddings \
232
258
  -d '{"model": "bge-m3", "input": ["hello world", "another text"]}'
233
259
  ```
234
260
 
235
- `input` is a string or a list of strings; `model` names any configured model
236
- (defaults to the active one). Returns the standard shape:
261
+ `input` is a string or a list of strings; `model` is **required** and names
262
+ any configured model. Returns the standard shape:
237
263
 
238
264
  ```json
239
265
  {
@@ -247,27 +273,43 @@ curl http://127.0.0.1:17347/v1/embeddings \
247
273
  }
248
274
  ```
249
275
 
250
- Empty/whitespace inputs get a zero vector of the model's dimension, keeping
251
- row alignment.
276
+ OpenAI forbids empty-string input, so an empty/whitespace string in the
277
+ batch is rejected with a strict `400` — there is no zero-vector row
278
+ alignment on the wire; filter blanks before batching.
279
+
280
+ #### Errors — the OpenAI contract
281
+
282
+ Every error uses the standard envelope
283
+ `{"error": {"message", "type", "param", "code"}}` (`param` / `code` are
284
+ `null` when not applicable):
285
+
286
+ | Status | `type` | When | `param` / `code` |
287
+ |---|---|---|---|
288
+ | `400` | `invalid_request_error` | unparseable JSON body; `input` not a string/array of strings; missing `model`; empty / whitespace-only input | `input` / `model` |
289
+ | `400` | `invalid_request_error` | input exceeds the model's context length (`max_tokens` — never silently truncated) | `input` / `context_length_exceeded` |
290
+ | `404` | `invalid_request_error` | unknown `model` ("does not exist or you do not have access to it") | — / `model_not_found` |
291
+ | `503` | `server_error` | the model's engine is **still loading** (is_loading — retry shortly) or unavailable (backend dependency missing, load failed) | — |
292
+ | `500` | `server_error` | unexpected internal failure | — |
293
+
294
+ Split long documents into pieces of at most `max_tokens` tokens client-side
295
+ (slife's semantic drainer does this automatically).
252
296
 
253
297
  ### `GET /v1/models`
254
298
 
255
299
  Every configured model, each with its real embedding dimension
256
- (`dimension` / `dimension_known`), backend, load state, and an `active` flag.
300
+ (`dimension` / `dimension_known`), backend, and load state. The listing is
301
+ standard — no `active` marker, all models are peers.
257
302
 
258
303
  ### `GET /v1/models/{id}`
259
304
 
260
305
  One model's detail (the OpenAI `retrieve` endpoint); 404 with the standard
261
306
  error envelope when the id is unknown.
262
307
 
263
- ### `POST /v1/models/{id}/activate`
264
-
265
- Switch the active model (loads it on demand). A local-embed extension, not an
266
- OpenAI endpoint.
267
-
268
308
  ### `GET /health`
269
309
 
270
- `{status, active_model, backend, model, dimension, dimension_known, loaded}`.
310
+ Liveness + engine state: `{status, models: [{name, backend, model,
311
+ dimension, dimension_known, loaded, available, max_tokens}]}` — `status`
312
+ is `ok` when any configured model's backend is usable.
271
313
 
272
314
  ### Any OpenAI client
273
315
 
@@ -300,10 +342,11 @@ config.
300
342
 
301
343
  ### `local-embed set` / `set-gguf` — write the config
302
344
 
303
- `set` (transformer) and `set-gguf` (gguf) upsert a model in the config, make it
304
- **active**, and pin port (and, for `set`, the HF cache + offline flag). Both
305
- are **idempotent** — re-running yields the same config — and leave other models
306
- untouched.
345
+ `set` (transformer) and `set-gguf` (gguf) upsert a model in the config and
346
+ pin port (and, for `set`, the HF cache + offline flag). Both are
347
+ **idempotent** — re-running yields the same config — and leave other models
348
+ untouched. There is no `active_model`: every configured model is a peer
349
+ and requests name the one they want.
307
350
 
308
351
  | | `set` (transformer) | `set-gguf` |
309
352
  |---|---|---|
@@ -373,84 +416,54 @@ claude mcp add --transport http local-embed http://127.0.0.1:17347/mcp
373
416
  > reachable from the desktop app.
374
417
 
375
418
  **Over MCP you get status, not embeddings.** The only MCP tool is the internal
376
- `__check` status probe (active model, dimensions, load state) — local-embed is
419
+ `__check` status probe (model list, dimensions, load state) — local-embed is
377
420
  a *service-provider* MCP server, not a tool provider. To embed text, point
378
421
  your consumer at the OpenAI-compatible `/v1/*` API instead. A client that only
379
422
  speaks stdio can reach it through a stdio→HTTP bridge (e.g. `mcp-remote`).
380
423
 
381
- ## Loading as a slife plugin
382
-
383
- local-embed is a standalone app, but it also conforms to the **slife plugin
384
- contract** (defined in slife's `slife/server_utils.py`), so slife can load and
385
- manage it as an external plugin — spawn the process, discover the port, and
386
- monitor status — without any slife code changes. Conformance is additive:
387
- none of it constrains standalone use, and local-embed never imports slife. The
388
- few pieces of the contract it needs are carried as minimal, self-contained
389
- copies (`local_embed.server_utils`, `local_embed.threads`,
390
- `local_embed.logging`).
391
-
392
- The MCP endpoint's single tool is the internal **`__check`**, which returns
393
- engine status (active model, model list, dimensions, load state) as JSON. It
394
- is a **service-provider** surface — slife probes `__check` programmatically and
395
- consumes embeddings over the HTTP API, never through MCP tools.
396
-
397
- How it conforms:
398
-
399
- - **Spawn target** — `local_embed/server.py` exposes `main()`, spawnable via
400
- `python -m local_embed.server` (the contract's `sys.executable -m <module>`
401
- shape).
402
- - **Port signal** — emits `{"port": N}` on stdout (then closes stdout) only
403
- after the FastMCP app is ready to serve (its lifespan completed), so the
404
- host's first `initialize` always lands on a ready server.
405
- - **Readiness = MCP `initialize` handshake** — startup is handshake-fast:
406
- backend imports are resolved lazily and the model warms up *after* the first
407
- `tools/list` (`warm_after_handshake`), never in the lifespan.
408
- - **Internal `__check` tool** — `__`-prefixed, so the host's tool registration
409
- filters it out of the LLM registry and `system_health` probes it
410
- programmatically.
411
- - **Non-MCP routes on the same port** — the OpenAI endpoints are
412
- `@mcp.custom_route` handlers on the same uvicorn app (the "one port, two
413
- protocols" pattern).
414
- - **Logging adoption** — when spawned by slife it adopts `SLIFE_LOG_DIR` /
415
- `SLIFE_AGENT_NAME` / `SLIFE_SESSION_ID` / `SLIFE_PLUGIN_NAME`, so the session
416
- log lands next to the main log; an uncaught startup exception prints one
417
- clean stderr line (the host relays it) with the full traceback in the file.
418
- Standalone falls back to `~/.local-embed/logs`.
419
- - **Config path** — honors `$LOCAL_EMBED_FILE` (slife exports it as
420
- `<dir of slife.json5>/local_embed.json5`), else the standalone default.
421
- - **Stable port** — binds the configured `17347` rather than an OS-assigned
422
- port, so slife's `base_url` is fixed; a taken port is a hard error, not a
423
- silent fallback.
424
-
425
- ### Registering with slife
426
-
427
- Register local-embed as an external plugin and point slife's embedding config
428
- at it (the unified OpenAI-compatible format slife uses for every embedding
429
- model):
424
+ ## Running as a slife embedding backend
425
+
426
+ local-embed is a **plugin** — one row in slife's central plugin spec, so
427
+ slife spawns it, watches it, and reports it like every other child plugin. It
428
+ is also runnable standalone (`local-embed` on PATH), which is what the CLI is
429
+ for; slife still consumes the MODEL SERVICE purely as an OpenAI-compatible
430
+ HTTP endpoint (never through MCP tools).
431
+
432
+ Because its config pins a port that a static embeddings `base_url` points at,
433
+ it is the one plugin with a **fixed** port: it binds that port itself and a
434
+ taken port is a hard error (no fallback — a second instance would double the
435
+ model in memory and serve nothing new). The host then reports the plugin as
436
+ failed to load, with the reason, and the running instance keeps serving.
437
+
438
+ Point slife's embedding config at the daemon's stable port:
430
439
 
431
440
  ```json5
432
- plugins: {
433
- external: [
434
- { name: "local-embed", module: "local_embed.server" }
435
- ]
436
- },
437
441
  embeddings: {
438
442
  providers: {
439
- "local-embed": {
443
+ local: {
440
444
  base_url: "http://127.0.0.1:17347/v1", // stable port from local_embed.json5
441
445
  api_key: "local",
446
+ model: "bge-m3", // id POSTed on /v1/embeddings
442
447
  }
443
448
  },
444
- active_model: "local-embed", // provider-id only, or "local-embed/<model>"
449
+ active_model: "local", // provider-id only, or "local/<model>"
445
450
  enabled: true
446
451
  }
447
452
  ```
448
453
 
449
454
  slife treats every embedding model as a remote OpenAI-compatible endpoint —
450
- local-embed is one such endpoint. The model is **determined by the plugin's
451
- active model**: slife discovers it from `GET /v1/models` (the entry flagged
452
- `active: true`) on load. When the service is unreachable, slife degrades
453
- gracefully to keyword search.
455
+ local-embed is one such endpoint. The model a request hits is the one the
456
+ caller names; when the config names no model, slife discovers it from
457
+ `GET /v1/models` (the first entry) on load. When the daemon is unreachable,
458
+ slife degrades gracefully to keyword search (`check_embeddings` in
459
+ `system_health` probes the **active** provider's HTTP endpoint and reports it
460
+ down — it never assumes that provider is local-embed).
461
+
462
+ The MCP surface it still serves is the internal **`__check`** (engine status:
463
+ model list, dimensions, load state) — a service-provider facade
464
+ for direct probing, not a slife plugin contract. It also serves plain OpenAPI
465
+ routes (`/v1/embeddings`, `/v1/models`, `/health`) on the same port via
466
+ `@mcp.custom_route` — one port, two protocols, no slife involvement.
454
467
 
455
468
  ## License
456
469
 
@@ -25,15 +25,18 @@ them ([Model weights](#model-weights)).
25
25
  vector DB / RAG pipeline ──────┘ ▼
26
26
  ┌───────────────────────┐
27
27
  slife (plugin host) ─── /mcp ────────────> │ local-embed │
28
- │ ONE loaded model │
28
+ │ models load on demand │
29
29
  └───────────────────────┘
30
30
  ```
31
31
 
32
32
  ## Features
33
33
 
34
34
  - **OpenAI-compatible** `/v1/embeddings` + Models API + `/health`.
35
- - Two backends (GGUF / transformer), **many models configured, one active**.
36
- - **Lazy load** — startup is fast; the model materialises on first use.
35
+ - Two backends (GGUF / transformer), **many models as peers — each request
36
+ names the one it wants** (standard OpenAI semantics, no "active model").
37
+ - **Lazy load** — startup is fast; each model materialises on the first
38
+ request that names it (per-model `autoload: true` preloads a specific
39
+ model at startup).
37
40
  - **Real dimension** reported after load (a guessed width is never served).
38
41
  - Runs standalone, and can also be **loaded as a slife plugin** (see
39
42
  [Loading as a slife plugin](#loading-as-a-slife-plugin)).
@@ -42,18 +45,36 @@ Requires Python ≥ 3.13.
42
45
 
43
46
  ## Install
44
47
 
45
- The core package is `fastmcp` + `starlette` + `json5`; the model backends are
48
+ The core package is `fastmcp` + `starlette` + `json-five` (the JSON5 parser,
49
+ imported as `json5`); the model backends are
46
50
  optional extras.
47
51
 
52
+ One-click installers (install `uv` if needed, then `uv tool install --force
53
+ local-embed` — the backend is **not** included, see below):
54
+
55
+ ```bash
56
+ # macOS / Linux / WSL
57
+ curl -fsSL https://raw.githubusercontent.com/juzcn/slife/main/local-embed/install.sh | bash
58
+
59
+ # Windows PowerShell
60
+ powershell -ExecutionPolicy Bypass -Command "irm https://raw.githubusercontent.com/juzcn/slife/main/local-embed/install.ps1 | iex"
61
+ ```
62
+
63
+ Manual installs are equivalent:
64
+
48
65
  ```bash
49
66
  # the app + CLI (standalone tool)
50
- uv tool install local-embed
67
+ uv tool install --force local-embed
51
68
 
52
69
  # with a backend:
53
- uv tool install "local-embed[transformer]"
54
- uv tool install "local-embed[gguf]"
70
+ uv tool install --force "local-embed[transformer]"
71
+ uv tool install --force "local-embed[gguf]"
55
72
  ```
56
73
 
74
+ `--force` (used by the installers too) is **idempotent** — the first run
75
+ installs, re-running **updates** to the latest PyPI release. A plain
76
+ `uv tool install local-embed` no-ops when the tool is already installed.
77
+
57
78
  If you run local-embed inside an *existing* environment (alongside another
58
79
  tool), install the backend into **that** environment's interpreter instead of
59
80
  creating a separate one:
@@ -114,15 +135,14 @@ Installing does **not** fetch a model — get the weights first.
114
135
 
115
136
  ## Configuration: `local_embed.json5`
116
137
 
117
- Everything — host, port, models, active model, backend — comes from
138
+ Everything — host, port, models, backend — comes from
118
139
  `local_embed.json5`, written by the CLI helpers or by hand. Path resolution:
119
140
  `$LOCAL_EMBED_FILE` > slife project root (dev) > `~/.local-embed/local_embed.json5`.
120
141
 
121
142
  ```json5
122
143
  {
123
- active_model: "bge-m3", // model served by default
124
144
  models: {
125
- "bge-m3": { backend: "gguf", gguf_path: "…", device: "" },
145
+ "bge-m3": { backend: "gguf", gguf_path: "…", device: "", autoload: false },
126
146
  "bge-m3-transformer": { backend: "transformer", model: "BAAI/bge-m3", device: "" }
127
147
  },
128
148
  env: { // injected into this process before any model loads
@@ -134,8 +154,14 @@ Everything — host, port, models, active model, backend — comes from
134
154
  }
135
155
  ```
136
156
 
137
- - `models` — map of name → `{backend, gguf_path | model, device, max_tokens}`.
138
- - `active_model` — key into `models`.
157
+ - `models` — map of name → `{backend, gguf_path | model, device, max_tokens, autoload}`.
158
+ Every configured model is a peer; there is **no `active_model`** — each
159
+ request names the model it wants (standard OpenAI semantics).
160
+ - `autoload` (per model, default `false`) — a model's weights are large and
161
+ memory-hungry, so nothing loads until a request names it (**lazy**).
162
+ `autoload: true` on one model eager-loads just that model in the
163
+ background at startup (memory paid up front for a warm first embed); every
164
+ unflagged model stays lazy.
139
165
  - `env:` — injected before any backend loads; an existing shell env var wins.
140
166
  Without `HF_HUB_CACHE`, transformer repos resolve against the default cache
141
167
  and a model downloaded elsewhere is silently re-fetched.
@@ -206,8 +232,8 @@ curl http://127.0.0.1:17347/v1/embeddings \
206
232
  -d '{"model": "bge-m3", "input": ["hello world", "another text"]}'
207
233
  ```
208
234
 
209
- `input` is a string or a list of strings; `model` names any configured model
210
- (defaults to the active one). Returns the standard shape:
235
+ `input` is a string or a list of strings; `model` is **required** and names
236
+ any configured model. Returns the standard shape:
211
237
 
212
238
  ```json
213
239
  {
@@ -221,27 +247,43 @@ curl http://127.0.0.1:17347/v1/embeddings \
221
247
  }
222
248
  ```
223
249
 
224
- Empty/whitespace inputs get a zero vector of the model's dimension, keeping
225
- row alignment.
250
+ OpenAI forbids empty-string input, so an empty/whitespace string in the
251
+ batch is rejected with a strict `400` — there is no zero-vector row
252
+ alignment on the wire; filter blanks before batching.
253
+
254
+ #### Errors — the OpenAI contract
255
+
256
+ Every error uses the standard envelope
257
+ `{"error": {"message", "type", "param", "code"}}` (`param` / `code` are
258
+ `null` when not applicable):
259
+
260
+ | Status | `type` | When | `param` / `code` |
261
+ |---|---|---|---|
262
+ | `400` | `invalid_request_error` | unparseable JSON body; `input` not a string/array of strings; missing `model`; empty / whitespace-only input | `input` / `model` |
263
+ | `400` | `invalid_request_error` | input exceeds the model's context length (`max_tokens` — never silently truncated) | `input` / `context_length_exceeded` |
264
+ | `404` | `invalid_request_error` | unknown `model` ("does not exist or you do not have access to it") | — / `model_not_found` |
265
+ | `503` | `server_error` | the model's engine is **still loading** (is_loading — retry shortly) or unavailable (backend dependency missing, load failed) | — |
266
+ | `500` | `server_error` | unexpected internal failure | — |
267
+
268
+ Split long documents into pieces of at most `max_tokens` tokens client-side
269
+ (slife's semantic drainer does this automatically).
226
270
 
227
271
  ### `GET /v1/models`
228
272
 
229
273
  Every configured model, each with its real embedding dimension
230
- (`dimension` / `dimension_known`), backend, load state, and an `active` flag.
274
+ (`dimension` / `dimension_known`), backend, and load state. The listing is
275
+ standard — no `active` marker, all models are peers.
231
276
 
232
277
  ### `GET /v1/models/{id}`
233
278
 
234
279
  One model's detail (the OpenAI `retrieve` endpoint); 404 with the standard
235
280
  error envelope when the id is unknown.
236
281
 
237
- ### `POST /v1/models/{id}/activate`
238
-
239
- Switch the active model (loads it on demand). A local-embed extension, not an
240
- OpenAI endpoint.
241
-
242
282
  ### `GET /health`
243
283
 
244
- `{status, active_model, backend, model, dimension, dimension_known, loaded}`.
284
+ Liveness + engine state: `{status, models: [{name, backend, model,
285
+ dimension, dimension_known, loaded, available, max_tokens}]}` — `status`
286
+ is `ok` when any configured model's backend is usable.
245
287
 
246
288
  ### Any OpenAI client
247
289
 
@@ -274,10 +316,11 @@ config.
274
316
 
275
317
  ### `local-embed set` / `set-gguf` — write the config
276
318
 
277
- `set` (transformer) and `set-gguf` (gguf) upsert a model in the config, make it
278
- **active**, and pin port (and, for `set`, the HF cache + offline flag). Both
279
- are **idempotent** — re-running yields the same config — and leave other models
280
- untouched.
319
+ `set` (transformer) and `set-gguf` (gguf) upsert a model in the config and
320
+ pin port (and, for `set`, the HF cache + offline flag). Both are
321
+ **idempotent** — re-running yields the same config — and leave other models
322
+ untouched. There is no `active_model`: every configured model is a peer
323
+ and requests name the one they want.
281
324
 
282
325
  | | `set` (transformer) | `set-gguf` |
283
326
  |---|---|---|
@@ -347,84 +390,54 @@ claude mcp add --transport http local-embed http://127.0.0.1:17347/mcp
347
390
  > reachable from the desktop app.
348
391
 
349
392
  **Over MCP you get status, not embeddings.** The only MCP tool is the internal
350
- `__check` status probe (active model, dimensions, load state) — local-embed is
393
+ `__check` status probe (model list, dimensions, load state) — local-embed is
351
394
  a *service-provider* MCP server, not a tool provider. To embed text, point
352
395
  your consumer at the OpenAI-compatible `/v1/*` API instead. A client that only
353
396
  speaks stdio can reach it through a stdio→HTTP bridge (e.g. `mcp-remote`).
354
397
 
355
- ## Loading as a slife plugin
356
-
357
- local-embed is a standalone app, but it also conforms to the **slife plugin
358
- contract** (defined in slife's `slife/server_utils.py`), so slife can load and
359
- manage it as an external plugin — spawn the process, discover the port, and
360
- monitor status — without any slife code changes. Conformance is additive:
361
- none of it constrains standalone use, and local-embed never imports slife. The
362
- few pieces of the contract it needs are carried as minimal, self-contained
363
- copies (`local_embed.server_utils`, `local_embed.threads`,
364
- `local_embed.logging`).
365
-
366
- The MCP endpoint's single tool is the internal **`__check`**, which returns
367
- engine status (active model, model list, dimensions, load state) as JSON. It
368
- is a **service-provider** surface — slife probes `__check` programmatically and
369
- consumes embeddings over the HTTP API, never through MCP tools.
370
-
371
- How it conforms:
372
-
373
- - **Spawn target** — `local_embed/server.py` exposes `main()`, spawnable via
374
- `python -m local_embed.server` (the contract's `sys.executable -m <module>`
375
- shape).
376
- - **Port signal** — emits `{"port": N}` on stdout (then closes stdout) only
377
- after the FastMCP app is ready to serve (its lifespan completed), so the
378
- host's first `initialize` always lands on a ready server.
379
- - **Readiness = MCP `initialize` handshake** — startup is handshake-fast:
380
- backend imports are resolved lazily and the model warms up *after* the first
381
- `tools/list` (`warm_after_handshake`), never in the lifespan.
382
- - **Internal `__check` tool** — `__`-prefixed, so the host's tool registration
383
- filters it out of the LLM registry and `system_health` probes it
384
- programmatically.
385
- - **Non-MCP routes on the same port** — the OpenAI endpoints are
386
- `@mcp.custom_route` handlers on the same uvicorn app (the "one port, two
387
- protocols" pattern).
388
- - **Logging adoption** — when spawned by slife it adopts `SLIFE_LOG_DIR` /
389
- `SLIFE_AGENT_NAME` / `SLIFE_SESSION_ID` / `SLIFE_PLUGIN_NAME`, so the session
390
- log lands next to the main log; an uncaught startup exception prints one
391
- clean stderr line (the host relays it) with the full traceback in the file.
392
- Standalone falls back to `~/.local-embed/logs`.
393
- - **Config path** — honors `$LOCAL_EMBED_FILE` (slife exports it as
394
- `<dir of slife.json5>/local_embed.json5`), else the standalone default.
395
- - **Stable port** — binds the configured `17347` rather than an OS-assigned
396
- port, so slife's `base_url` is fixed; a taken port is a hard error, not a
397
- silent fallback.
398
-
399
- ### Registering with slife
400
-
401
- Register local-embed as an external plugin and point slife's embedding config
402
- at it (the unified OpenAI-compatible format slife uses for every embedding
403
- model):
398
+ ## Running as a slife embedding backend
399
+
400
+ local-embed is a **plugin** — one row in slife's central plugin spec, so
401
+ slife spawns it, watches it, and reports it like every other child plugin. It
402
+ is also runnable standalone (`local-embed` on PATH), which is what the CLI is
403
+ for; slife still consumes the MODEL SERVICE purely as an OpenAI-compatible
404
+ HTTP endpoint (never through MCP tools).
405
+
406
+ Because its config pins a port that a static embeddings `base_url` points at,
407
+ it is the one plugin with a **fixed** port: it binds that port itself and a
408
+ taken port is a hard error (no fallback — a second instance would double the
409
+ model in memory and serve nothing new). The host then reports the plugin as
410
+ failed to load, with the reason, and the running instance keeps serving.
411
+
412
+ Point slife's embedding config at the daemon's stable port:
404
413
 
405
414
  ```json5
406
- plugins: {
407
- external: [
408
- { name: "local-embed", module: "local_embed.server" }
409
- ]
410
- },
411
415
  embeddings: {
412
416
  providers: {
413
- "local-embed": {
417
+ local: {
414
418
  base_url: "http://127.0.0.1:17347/v1", // stable port from local_embed.json5
415
419
  api_key: "local",
420
+ model: "bge-m3", // id POSTed on /v1/embeddings
416
421
  }
417
422
  },
418
- active_model: "local-embed", // provider-id only, or "local-embed/<model>"
423
+ active_model: "local", // provider-id only, or "local/<model>"
419
424
  enabled: true
420
425
  }
421
426
  ```
422
427
 
423
428
  slife treats every embedding model as a remote OpenAI-compatible endpoint —
424
- local-embed is one such endpoint. The model is **determined by the plugin's
425
- active model**: slife discovers it from `GET /v1/models` (the entry flagged
426
- `active: true`) on load. When the service is unreachable, slife degrades
427
- gracefully to keyword search.
429
+ local-embed is one such endpoint. The model a request hits is the one the
430
+ caller names; when the config names no model, slife discovers it from
431
+ `GET /v1/models` (the first entry) on load. When the daemon is unreachable,
432
+ slife degrades gracefully to keyword search (`check_embeddings` in
433
+ `system_health` probes the **active** provider's HTTP endpoint and reports it
434
+ down — it never assumes that provider is local-embed).
435
+
436
+ The MCP surface it still serves is the internal **`__check`** (engine status:
437
+ model list, dimensions, load state) — a service-provider facade
438
+ for direct probing, not a slife plugin contract. It also serves plain OpenAPI
439
+ routes (`/v1/embeddings`, `/v1/models`, `/health`) on the same port via
440
+ `@mcp.custom_route` — one port, two protocols, no slife involvement.
428
441
 
429
442
  ## License
430
443