local-embed 0.2.2__tar.gz → 0.2.4__tar.gz

{local_embed-0.2.2 → local_embed-0.2.4}/.gitignore

@@ -1,7 +1,7 @@
-# Runtime config — personal per-user settings; template is slife.template.json5.
-slife.json5
-mcp-plugin.json5
-local_embed.json5
+# Runtime config — personal per-user settings.  The three initial configs
+# (slife.json5 / local_embed.json5 / tools.json5) are git-tracked so the
+# one-click installers can seed them as out-of-the-box defaults.  Copies in a
+# user's data dir (~/.slife/) remain personal and are NOT tracked.
 
 # Playwright MCP browser traces (test artifacts)
 .playwright-mcp/

local_embed-0.2.4/PKG-INFO

@@ -0,0 +1,470 @@
+Metadata-Version: 2.3
+Name: local-embed
+Version: 0.2.4
+Summary: Standalone local embedding server — GGUF / transformer models as an OpenAI-compatible /v1/embeddings service
+Project-URL: Homepage, https://github.com/juzcn/slife
+Project-URL: Repository, https://github.com/juzcn/slife
+Project-URL: Documentation, https://github.com/juzcn/slife/blob/main/local-embed/README.md
+Author-email: juzcn <zhangjun@cueb.edu.cn>
+License: MIT
+Classifier: Development Status :: 3 - Alpha
+Classifier: Environment :: Console
+Classifier: Intended Audience :: Developers
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Topic :: Terminals
+Requires-Python: >=3.13
+Requires-Dist: fastmcp>=2.0.0
+Requires-Dist: json-five>=1.1.2
+Requires-Dist: starlette>=0.37.0
+Provides-Extra: gguf
+Requires-Dist: llama-cpp-python==0.3.34; extra == 'gguf'
+Provides-Extra: transformer
+Requires-Dist: sentence-transformers; extra == 'transformer'
+Description-Content-Type: text/markdown
+
+# local-embed
+
+**A standalone local embedding service.** It loads one local GGUF
+(llama-cpp) or HF transformer embedding model and exposes it as an
+[OpenAI-compatible](https://platform.openai.com/docs/api-reference/embeddings)
+`/v1/embeddings` URL — any OpenAI-compatible client can embed text through it.
+
+On one port it serves two surfaces: an OpenAI-compatible `/v1/*` API for
+embedding, and a Streamable HTTP MCP endpoint at `/mcp`.
+
+local-embed is three things at once: an independent app, a Streamable HTTP
+**MCP server** (see [As an MCP server](#as-an-mcp-server)), and a conforming
+**slife plugin** (see [Loading as a slife plugin](#loading-as-a-slife-plugin)).
+The product is the embedding URL; the MCP and plugin surfaces are how hosts
+manage the same standalone service.
+
+Two backends: **`gguf`** loads a local `.gguf` file (llama-cpp-python);
+**`transformer`** loads an HF repo id from the local hub cache
+(sentence-transformers). The server never downloads weights — you pre-download
+them ([Model weights](#model-weights)).
+
+```
+   any OpenAI-compatible client ──┐
+   slife (memdb / memfiles) ──────┤── POST /v1/embeddings ──┐
+   vector DB / RAG pipeline ──────┘                          ▼
+                                              ┌───────────────────────┐
+   slife (plugin host) ─── /mcp ────────────> │       local-embed     │
+                                              │  models load on demand │
+                                              └───────────────────────┘
+```
+
+## Features
+
+- **OpenAI-compatible** `/v1/embeddings` + Models API + `/health`.
+- Two backends (GGUF / transformer), **many models as peers — each request
+  names the one it wants** (standard OpenAI semantics, no "active model").
+- **Lazy load** — startup is fast; each model materialises on the first
+  request that names it (per-model `autoload: true` preloads a specific
+  model at startup).
+- **Real dimension** reported after load (a guessed width is never served).
+- Runs standalone, and can also be **loaded as a slife plugin** (see
+  [Loading as a slife plugin](#loading-as-a-slife-plugin)).
+
+Requires Python ≥ 3.13.
+
+## Install
+
+The core package is `fastmcp` + `starlette` + `json-five` (the JSON5 parser,
+imported as `json5`); the model backends are
+optional extras.
+
+One-click installers (install `uv` if needed, then `uv tool install --force
+local-embed` — the backend is **not** included, see below):
+
+```bash
+# macOS / Linux / WSL
+curl -fsSL https://raw.githubusercontent.com/juzcn/slife/main/local-embed/install.sh | bash
+
+# Windows PowerShell
+powershell -ExecutionPolicy Bypass -Command "irm https://raw.githubusercontent.com/juzcn/slife/main/local-embed/install.ps1 | iex"
+```
+
+Manual installs are equivalent:
+
+```bash
+# the app + CLI (standalone tool)
+uv tool install --force local-embed
+
+# with a backend:
+uv tool install --force "local-embed[transformer]"
+uv tool install --force "local-embed[gguf]"
+```
+
+`--force` (used by the installers too) is **idempotent** — the first run
+installs, re-running **updates** to the latest PyPI release.  A plain
+`uv tool install local-embed` no-ops when the tool is already installed.
+
+If you run local-embed inside an *existing* environment (alongside another
+tool), install the backend into **that** environment's interpreter instead of
+creating a separate one:
+
+```bash
+uv pip install --python <venv-python> llama-cpp-python==0.3.34
+```
+
+where `<venv-python>` is the interpreter that runs `local-embed`.
+
+### `gguf` backend — platform matrix
+
+`gguf` (llama-cpp-python) is platform-sensitive: PyPI ships **only the sdist**
+(no prebuilt wheels), so on Linux / WSL / macOS a plain install **compiles from
+source** (needs a C compiler + CMake ≥ 3.21). **Windows has no default C
+toolchain** (no MSVC), so it uses the upstream prebuilt CPU wheel — the one
+workaround. GPU variants pass `CMAKE_ARGS`:
+
+| Platform | Command |
+|---|---|
+| Linux / WSL / macOS, CPU | `uv pip install --python <venv-python> llama-cpp-python==0.3.34` (compiles from source) |
+| macOS arm64 (Metal) | `CMAKE_ARGS="-DGGML_METAL=on" uv pip install --python <venv-python> llama-cpp-python==0.3.34` |
+| NVIDIA CUDA (Linux) | `CMAKE_ARGS="-DGGML_CUDA=on" uv pip install --python <venv-python> llama-cpp-python==0.3.34` (needs the CUDA toolkit) |
+| **Windows, CPU** | `uv pip install --python <venv-python> --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu llama-cpp-python==0.3.34` (prebuilt wheel — no MSVC) |
+
+The Windows CPU row is the only workaround — everywhere else uses the standard
+source build from PyPI. If the `gguf` backend is missing, the server errors at
+startup with the exact command for your platform.
+
+**Both backends at once** — one `uv pip install`:
+
+```bash
+uv pip install --python <venv-python> sentence-transformers llama-cpp-python==0.3.34
+# Windows CPU: add the upstream wheel index (see the matrix above)
+```
+
+Installing does **not** fetch a model — get the weights first.
+
+## Quick start
+
+1. **Install** the app and a backend (above).
+2. **Download weights** — see [Model weights](#model-weights).
+3. **Configure a model** — the CLI helper writes `local_embed.json5`:
+
+   ```bash
+   local-embed set BAAI/bge-m3 --HF_HUB_CACHE <dir>     # transformer
+   local-embed set-gguf bge-m3 --path D:\models\bge.gguf # gguf
+   ```
+
+4. **Start the service**:
+
+   ```bash
+   local-embed          # serve on http://127.0.0.1:17347
+   ```
+
+5. **Embed text** — point any OpenAI-compatible client at
+   `http://127.0.0.1:17347/v1` (see [HTTP API](#http-api-openai-compatible)).
+
+## Configuration: `local_embed.json5`
+
+Everything — host, port, models, backend — comes from
+`local_embed.json5`, written by the CLI helpers or by hand. Path resolution:
+`$LOCAL_EMBED_FILE` > slife project root (dev) > `~/.local-embed/local_embed.json5`.
+
+```json5
+{
+  models: {
+    "bge-m3": { backend: "gguf", gguf_path: "…", device: "", autoload: false },
+    "bge-m3-transformer": { backend: "transformer", model: "BAAI/bge-m3", device: "" }
+  },
+  env: {                            // injected into this process before any model loads
+    HF_HUB_CACHE: "C:\\…\\HuggingFace\\hub",   // where transformer repos resolve
+    HF_HUB_OFFLINE: "1"                        // force offline
+  },
+  host: "127.0.0.1",                // standalone only
+  port: 17347                        // standalone only
+}
+```
+
+- `models` — map of name → `{backend, gguf_path | model, device, max_tokens, autoload}`.
+  Every configured model is a peer; there is **no `active_model`** — each
+  request names the model it wants (standard OpenAI semantics).
+- `autoload` (per model, default `false`) — a model's weights are large and
+  memory-hungry, so nothing loads until a request names it (**lazy**).
+  `autoload: true` on one model eager-loads just that model in the
+  background at startup (memory paid up front for a warm first embed); every
+  unflagged model stays lazy.
+- `env:` — injected before any backend loads; an existing shell env var wins.
+  Without `HF_HUB_CACHE`, transformer repos resolve against the default cache
+  and a model downloaded elsewhere is silently re-fetched.
+- Single-model convenience — top-level `backend`/`model`/`gguf_path`/`device`
+  — still works.
+
+Reads are read-only at runtime — the server has no config-mutating tools.
+
+## Model weights
+
+The server never downloads models; both backends fail at load time when their
+weights are missing (`gguf` needs the `gguf_path` file, `transformer` needs the
+repo in the HF cache).
+
+### GGUF
+
+`.gguf` files are, in general, **community conversions** — upstream releases
+are PyTorch weights, not GGUF — so there is **no single authoritative source**:
+conversions are scattered across Hugging Face, ModelScope, Ollama, llama.cpp
+community uploads and various project sites. Pick any source you trust; the
+transport doesn't matter, local-embed only needs the file on disk.
+
+HF is the most common host and its CLI pulls a single file:
+
+```bash
+uv tool install "huggingface-hub[cli]"      # provides `hf`
+hf download <owner>/<repo> <model>.gguf --local-dir D:\models\bge-m3
+# one-off, no permanent tool install:
+uvx --from huggingface-hub hf download <owner>/<repo> <model>.gguf --local-dir D:\models\bge-m3
+```
+
+Any source also works from a browser or `wget`/`curl` — on HF, every file is
+fetchable from `https://huggingface.co/<owner>/<repo>/resolve/main/<model>.gguf`.
+
+Prefer a high-fidelity quant such as **Q8_0** (~99 % of the original accuracy at
+roughly a third of the size). Point `gguf_path` at the file:
+
+```json5
+models: {
+  "bge-m3": { backend: "gguf", gguf_path: "D:\\models\\bge-m3\\bge-m3-q8_0.gguf" }
+}
+```
+
+### Transformer
+
+`sentence-transformers` resolves an HF repo id against the local hub cache:
+
+```bash
+hf download BAAI/bge-m3     # -> ~/.cache/huggingface/hub/models--BAAI--bge-m3
+```
+
+```json5
+models: {
+  "bge-m3": { backend: "transformer", model: "BAAI/bge-m3" }
+}
+```
+
+## HTTP API (OpenAI-compatible)
+
+The endpoint surface is a superset of the OpenAI Embeddings + Models APIs, all
+on one port.
+
+### `POST /v1/embeddings`
+
+```bash
+curl http://127.0.0.1:17347/v1/embeddings \
+  -H 'Content-Type: application/json' \
+  -d '{"model": "bge-m3", "input": ["hello world", "another text"]}'
+```
+
+`input` is a string or a list of strings; `model` is **required** and names
+any configured model.  Returns the standard shape:
+
+```json
+{
+  "object": "list",
+  "data": [
+    {"object": "embedding", "index": 0, "embedding": [0.012, ...]},
+    {"object": "embedding", "index": 1, "embedding": [...]}
+  ],
+  "model": "bge-m3",
+  "usage": {"prompt_tokens": 3, "total_tokens": 3}
+}
+```
+
+OpenAI forbids empty-string input, so an empty/whitespace string in the
+batch is rejected with a strict `400` — there is no zero-vector row
+alignment on the wire; filter blanks before batching.
+
+#### Errors — the OpenAI contract
+
+Every error uses the standard envelope
+`{"error": {"message", "type", "param", "code"}}` (`param` / `code` are
+`null` when not applicable):
+
+| Status | `type` | When | `param` / `code` |
+|---|---|---|---|
+| `400` | `invalid_request_error` | unparseable JSON body; `input` not a string/array of strings; missing `model`; empty / whitespace-only input | `input` / `model` |
+| `400` | `invalid_request_error` | input exceeds the model's context length (`max_tokens` — never silently truncated) | `input` / `context_length_exceeded` |
+| `404` | `invalid_request_error` | unknown `model` ("does not exist or you do not have access to it") | — / `model_not_found` |
+| `503` | `server_error` | the model's engine is **still loading** (is_loading — retry shortly) or unavailable (backend dependency missing, load failed) | — |
+| `500` | `server_error` | unexpected internal failure | — |
+
+Split long documents into pieces of at most `max_tokens` tokens client-side
+(slife's semantic drainer does this automatically).
+
+### `GET /v1/models`
+
+Every configured model, each with its real embedding dimension
+(`dimension` / `dimension_known`), backend, and load state.  The listing is
+standard — no `active` marker, all models are peers.
+
+### `GET /v1/models/{id}`
+
+One model's detail (the OpenAI `retrieve` endpoint); 404 with the standard
+error envelope when the id is unknown.
+
+### `GET /health`
+
+Liveness + engine state: `{status, models: [{name, backend, model,
+dimension, dimension_known, loaded, available, max_tokens}]}` — `status`
+is `ok` when any configured model's backend is usable.
+
+### Any OpenAI client
+
+Point `base_url` at `http://127.0.0.1:17347/v1` — the `openai` package, or any
+OpenAI-compatible SDK:
+
+```python
+from openai import OpenAI
+
+client = OpenAI(base_url="http://127.0.0.1:17347/v1", api_key="local")
+vecs = client.embeddings.create(model="bge-m3", input=["hello"])
+```
+
+## CLI (auxiliary tools)
+
+The `local-embed` CLI is a **helper**, not the product: it starts the service
+and writes the config file. The service is configured by `local_embed.json5`
+and consumed over the HTTP API — the CLI only makes both steps convenient.
+
+### `local-embed` — run the service
+
+```bash
+local-embed                 # binds 127.0.0.1:17347 by default
+```
+
+The CLI takes no model/endpoint flags — the config is the only source of truth.
+A port already in use is a hard error with one actionable line (no silent
+fallback to a free port): stop the other instance or change `port` in the
+config.
+
+### `local-embed set` / `set-gguf` — write the config
+
+`set` (transformer) and `set-gguf` (gguf) upsert a model in the config and
+pin port (and, for `set`, the HF cache + offline flag). Both are
+**idempotent** — re-running yields the same config — and leave other models
+untouched.  There is no `active_model`: every configured model is a peer
+and requests name the one they want.
+
+| | `set` (transformer) | `set-gguf` |
+|---|---|---|
+| `<model_name>` | required — HF repo id (key + repo loaded) | required — config key |
+| weight ref | `--HF_HUB_CACHE <dir>` — cache must contain the repo | `--path <PATH>` required — existing `.gguf` file |
+| cache fallback | env `HF_HUB_CACHE`, else error | — |
+| env written | `HF_HUB_CACHE` + `HF_HUB_OFFLINE: "1"` (offline server) | — |
+| `--port <n>` | default `17347` | default `17347` |
+
+```bash
+local-embed set BAAI/bge-m3 [--HF_HUB_CACHE <dir>] [--port 17347]
+local-embed set-gguf bge-m3 --path D:\models\bge-m3\bge-m3-q8_0.gguf [--port 17347]
+```
+
+Any error — cache unset, repo not in cache, file missing — exits non-zero and
+writes nothing. Changes apply on the next server start.
+
+### Model download — offline by default
+
+`HF_HUB_OFFLINE=1` by default, so the server **never downloads** a model (it's
+too slow). Make the model available yourself — see
+[Model weights](#model-weights): `hf download BAAI/bge-m3` for the transformer
+route (into the HF cache), or drop a GGUF file and point `gguf_path` at it.
+
+## Dimension
+
+The real output width is only known once the model is loaded (`n_embd` /
+`get_sentence_embedding_dimension`). `GET /v1/models` (or `/v1/models/{id}`)
+reports the real dimension, so a client can size its vector table correctly —
+a wrong width silently drops every mis-sized embedding. A guessed dimension is
+never served as authoritative.
+
+## As an MCP server
+
+`/mcp` is a standard **MCP Streamable HTTP** endpoint, so any MCP client that
+supports Streamable HTTP can register local-embed as an MCP server. Start it
+first:
+
+```bash
+local-embed          # serve on http://127.0.0.1:17347
+```
+
+then register the URL. The MCP client config shape is `{"type": "http",
+"url": …}` — for example, Claude Code's `.mcp.json` (or `~/.claude.json`):
+
+```json
+{
+  "mcpServers": {
+    "local-embed": {
+      "type": "http",
+      "url": "http://127.0.0.1:17347/mcp"
+    }
+  }
+}
+```
+
+Or with the Claude Code CLI:
+
+```bash
+claude mcp add --transport http local-embed http://127.0.0.1:17347/mcp
+```
+
+> **Claude Desktop** registers remote HTTP MCP servers through its **connectors
+> UI** (Settings → Connectors → Add connector → Remote), not
+> `claude_desktop_config.json` — and a remote connector expects a hosted HTTPS
+> URL, so a `127.0.0.1` server needs a tunnel or a stdio→HTTP bridge to be
+> reachable from the desktop app.
+
+**Over MCP you get status, not embeddings.** The only MCP tool is the internal
+`__check` status probe (model list, dimensions, load state) — local-embed is
+a *service-provider* MCP server, not a tool provider. To embed text, point
+your consumer at the OpenAI-compatible `/v1/*` API instead. A client that only
+speaks stdio can reach it through a stdio→HTTP bridge (e.g. `mcp-remote`).
+
+## Running as a slife embedding backend
+
+local-embed is a **plugin** — one row in slife's central plugin spec, so
+slife spawns it, watches it, and reports it like every other child plugin. It
+is also runnable standalone (`local-embed` on PATH), which is what the CLI is
+for; slife still consumes the MODEL SERVICE purely as an OpenAI-compatible
+HTTP endpoint (never through MCP tools).
+
+Because its config pins a port that a static embeddings `base_url` points at,
+it is the one plugin with a **fixed** port: it binds that port itself and a
+taken port is a hard error (no fallback — a second instance would double the
+model in memory and serve nothing new). The host then reports the plugin as
+failed to load, with the reason, and the running instance keeps serving.
+
+Point slife's embedding config at the daemon's stable port:
+
+```json5
+embeddings: {
+  providers: {
+    local: {
+      base_url: "http://127.0.0.1:17347/v1",  // stable port from local_embed.json5
+      api_key: "local",
+      model: "bge-m3",                        // id POSTed on /v1/embeddings
+    }
+  },
+  active_model: "local",      // provider-id only, or "local/<model>"
+  enabled: true
+}
+```
+
+slife treats every embedding model as a remote OpenAI-compatible endpoint —
+local-embed is one such endpoint. The model a request hits is the one the
+caller names; when the config names no model, slife discovers it from
+`GET /v1/models` (the first entry) on load. When the daemon is unreachable,
+slife degrades gracefully to keyword search (`check_embeddings` in
+`system_health` probes the **active** provider's HTTP endpoint and reports it
+down — it never assumes that provider is local-embed).
+
+The MCP surface it still serves is the internal **`__check`** (engine status:
+model list, dimensions, load state) — a service-provider facade
+for direct probing, not a slife plugin contract. It also serves plain OpenAPI
+routes (`/v1/embeddings`, `/v1/models`, `/health`) on the same port via
+`@mcp.custom_route` — one port, two protocols, no slife involvement.
+
+## License
+
+MIT — see the repository root `LICENSE`.