local-embed 0.2.0__tar.gz → 0.2.2__tar.gz

{local_embed-0.2.0 → local_embed-0.2.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: local-embed
-Version: 0.2.0
+Version: 0.2.2
 Summary: Standalone local embedding server — GGUF / transformer models as an OpenAI-compatible /v1/embeddings service
 Project-URL: Homepage, https://github.com/juzcn/slife
 Project-URL: Repository, https://github.com/juzcn/slife
@@ -16,6 +16,7 @@ Classifier: Programming Language :: Python :: 3.13
 Classifier: Topic :: Terminals
 Requires-Python: >=3.13
 Requires-Dist: fastmcp>=2.0.0
+Requires-Dist: json5>=0.15.0
 Requires-Dist: starlette>=0.37.0
 Provides-Extra: gguf
 Requires-Dist: llama-cpp-python; extra == 'gguf'
@@ -59,11 +60,22 @@ OpenAI-compatible client works.
 
 ## Install
 
-Requires [uv](https://docs.astral.sh/uv) (it manages the Python 3.13+
-runtime).  The `transformer` backend installs the same everywhere:
+Requires [uv](https://docs.astral.sh/uv).  All installs below pin
+**Python 3.13** (`requires-python` is `>=3.13`, but 3.13 is what CI tests,
+and uv would otherwise pick the newest 3.14 it finds).  Re-running any of
+them over an **existing** install requires `--reinstall` (`uv tool install`
+is a no-op on an already-installed tool otherwise), e.g. to upgrade to a new
+release or pick up the `json5` dependency fix:
 
 ```bash
-uv tool install 'local-embed[transformer]'   # sentence-transformers
+uv tool install --python 3.13 --reinstall 'local-embed[gguf,transformer]' \
+  --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
+```
+
+The `transformer` backend installs the same everywhere:
+
+```bash
+uv tool install --python 3.13 'local-embed[transformer]'   # sentence-transformers
 ```
 
 `gguf` (llama-cpp-python) is the platform-sensitive one: PyPI ships
@@ -74,10 +86,10 @@ for your platform:
 
 | Platform | Command |
 |---|---|
-| Linux / macOS, CPU | `uv tool install 'local-embed[gguf]'` (PyPI wheel) |
-| **Windows, CPU** | `uv tool install 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu` |
-| macOS arm64 (Metal) | `uv tool install 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal` |
-| CUDA (Linux or Windows) | `uv tool install 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu122` — pick `cu118` / `cu121` / `cu122` / `cu123` / `cu124` / `cu125` / `cu130` / `cu132` to match your CUDA version |
+| Linux / macOS, CPU | `uv tool install --python 3.13 'local-embed[gguf]'` (PyPI wheel) |
+| **Windows, CPU** | `uv tool install --python 3.13 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu` |
+| macOS arm64 (Metal) | `uv tool install --python 3.13 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal` |
+| CUDA (Linux or Windows) | `uv tool install --python 3.13 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu122` — pick `cu118` / `cu121` / `cu122` / `cu123` / `cu124` / `cu125` / `cu130` / `cu132` to match your CUDA version |
 
 When the index is an *extra*, uv uses its wheel only when PyPI has no
 compatible wheel for the platform (the Windows / Metal cases); on Linux
@@ -87,6 +99,17 @@ CUDA index.  Note the CUDA Linux wheels need **glibc ≥ 2.35** (built for
 If the `gguf` backend is missing, the server errors at startup with the
 exact command for your platform (the Windows CPU one on Windows).
 
+**Want both backends in one environment?** Install the two extras together
+in a single command — running `uv tool install` twice replaces the first
+environment with the second, so they would not coexist:
+
+```bash
+# both backends, Windows CPU:
+uv tool install --python 3.13 'local-embed[gguf,transformer]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
+# both backends, Linux / macOS CPU:
+uv tool install --python 3.13 'local-embed[gguf,transformer]'
+```
+
 Core deps are `fastmcp` + `starlette`; the model backends are optional
 extras.  Installing does **not** fetch a model — get the weights first.
 
@@ -182,14 +205,15 @@ source of truth.
 
 `local-embed set` (transformer) and `local-embed set-gguf` (gguf) upsert a
 model in the config, make it **active**, and pin port (and, for `set`, the
-HF cache).  Both are **idempotent** — re-running yields the same config —
-and leave other models untouched.
+HF cache + offline flag).  Both are **idempotent** — re-running yields the
+same config — and leave other models untouched.
 
 | | `set` (transformer) | `set-gguf` |
 |---|---|---|
 | `<model_name>` | required — HF repo id (key + repo loaded) | required — config key |
 | weight ref | `--HF_HUB_CACHE <dir>` — cache must contain the repo | `--path <PATH>` required — existing `.gguf` file |
 | cache fallback | env `HF_HUB_CACHE`, else error | — |
+| env written | `HF_HUB_CACHE` + `HF_HUB_OFFLINE: "1"` (offline server) | — |
 | `--port <n>` | default `8000` | default `8000` |
 
 ```bash

{local_embed-0.2.0 → local_embed-0.2.2}/README.md

@@ -34,11 +34,22 @@ OpenAI-compatible client works.
 
 ## Install
 
-Requires [uv](https://docs.astral.sh/uv) (it manages the Python 3.13+
-runtime).  The `transformer` backend installs the same everywhere:
+Requires [uv](https://docs.astral.sh/uv).  All installs below pin
+**Python 3.13** (`requires-python` is `>=3.13`, but 3.13 is what CI tests,
+and uv would otherwise pick the newest 3.14 it finds).  Re-running any of
+them over an **existing** install requires `--reinstall` (`uv tool install`
+is a no-op on an already-installed tool otherwise), e.g. to upgrade to a new
+release or pick up the `json5` dependency fix:
 
 ```bash
-uv tool install 'local-embed[transformer]'   # sentence-transformers
+uv tool install --python 3.13 --reinstall 'local-embed[gguf,transformer]' \
+  --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
+```
+
+The `transformer` backend installs the same everywhere:
+
+```bash
+uv tool install --python 3.13 'local-embed[transformer]'   # sentence-transformers
 ```
 
 `gguf` (llama-cpp-python) is the platform-sensitive one: PyPI ships
@@ -49,10 +60,10 @@ for your platform:
 
 | Platform | Command |
 |---|---|
-| Linux / macOS, CPU | `uv tool install 'local-embed[gguf]'` (PyPI wheel) |
-| **Windows, CPU** | `uv tool install 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu` |
-| macOS arm64 (Metal) | `uv tool install 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal` |
-| CUDA (Linux or Windows) | `uv tool install 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu122` — pick `cu118` / `cu121` / `cu122` / `cu123` / `cu124` / `cu125` / `cu130` / `cu132` to match your CUDA version |
+| Linux / macOS, CPU | `uv tool install --python 3.13 'local-embed[gguf]'` (PyPI wheel) |
+| **Windows, CPU** | `uv tool install --python 3.13 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu` |
+| macOS arm64 (Metal) | `uv tool install --python 3.13 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/metal` |
+| CUDA (Linux or Windows) | `uv tool install --python 3.13 'local-embed[gguf]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cu122` — pick `cu118` / `cu121` / `cu122` / `cu123` / `cu124` / `cu125` / `cu130` / `cu132` to match your CUDA version |
 
 When the index is an *extra*, uv uses its wheel only when PyPI has no
 compatible wheel for the platform (the Windows / Metal cases); on Linux
@@ -62,6 +73,17 @@ CUDA index.  Note the CUDA Linux wheels need **glibc ≥ 2.35** (built for
 If the `gguf` backend is missing, the server errors at startup with the
 exact command for your platform (the Windows CPU one on Windows).
 
+**Want both backends in one environment?** Install the two extras together
+in a single command — running `uv tool install` twice replaces the first
+environment with the second, so they would not coexist:
+
+```bash
+# both backends, Windows CPU:
+uv tool install --python 3.13 'local-embed[gguf,transformer]' --extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu
+# both backends, Linux / macOS CPU:
+uv tool install --python 3.13 'local-embed[gguf,transformer]'
+```
+
 Core deps are `fastmcp` + `starlette`; the model backends are optional
 extras.  Installing does **not** fetch a model — get the weights first.
 
@@ -157,14 +179,15 @@ source of truth.
 
 `local-embed set` (transformer) and `local-embed set-gguf` (gguf) upsert a
 model in the config, make it **active**, and pin port (and, for `set`, the
-HF cache).  Both are **idempotent** — re-running yields the same config —
-and leave other models untouched.
+HF cache + offline flag).  Both are **idempotent** — re-running yields the
+same config — and leave other models untouched.
 
 | | `set` (transformer) | `set-gguf` |
 |---|---|---|
 | `<model_name>` | required — HF repo id (key + repo loaded) | required — config key |
 | weight ref | `--HF_HUB_CACHE <dir>` — cache must contain the repo | `--path <PATH>` required — existing `.gguf` file |
 | cache fallback | env `HF_HUB_CACHE`, else error | — |
+| env written | `HF_HUB_CACHE` + `HF_HUB_OFFLINE: "1"` (offline server) | — |
 | `--port <n>` | default `8000` | default `8000` |
 
 ```bash

{local_embed-0.2.0 → local_embed-0.2.2}/local_embed/cli.py

@@ -108,26 +108,46 @@ def main(argv: "list[str] | None" = None) -> int:
 
     # Everything (host/port/backend/model/gguf_path/device) comes from
     # local_embed.json5 — the CLI deliberately takes no model/endpoint flags.
-    settings = resolve_engine_settings()
+    # A missing file is fine (defaults); a file that exists but won't parse
+    # must fail cleanly, not traceback.
+    try:
+        settings = resolve_engine_settings()
+    except ValueError as e:
+        print(f"Error: {e}", file=sys.stderr)
+        print("Fix local_embed.json5, or run `local-embed set` to rewrite it.",
+              file=sys.stderr)
+        return 2
+    except OSError as e:
+        print(f"Error: cannot read config: {e}", file=sys.stderr)
+        return 2
+
     engine = Engine(specs=settings["specs"], active=settings["active"])
 
-    # Validate the models can actually run.  `resolve_backend_runtime` (not
-    # `check_backend_runtime`, which never imports) so the answer reflects
-    # reality.  Only the ACTIVE model blocks startup — the server can serve
-    # it without the others; a non-active model with a missing backend or
-    # gguf_path is a warning (it fails at load time if requested).
+    # Validate the active model can actually run.  `resolve_backend_runtime`
+    # (not `check_backend_runtime`, which never imports) so the answer
+    # reflects reality.  Only the ACTIVE model is import-checked at startup:
+    # a non-active model's backend import cost (torch!) is only worth paying
+    # when the model is actually requested — it fails at load time then,
+    # and this CLI must not stall on it (or hang an impatient Ctrl-C during
+    # the import).  gguf_path is still checked for every model.
     active_name = engine.active_model
     for spec in settings["specs"]:
         problems: list[str] = []
         if spec.backend == "gguf" and not spec.gguf_path:
             problems.append("no gguf_path (set gguf_path in local_embed.json5)")
-        if not resolve_backend_runtime(spec.backend):
-            from local_embed.cmd_set import backend_install_hint
-
-            problems.append(
-                f"{spec.backend} backend not installed "
-                f"({backend_install_hint(spec.backend)})"
-            )
+        if spec.name == active_name:
+            try:
+                installed = resolve_backend_runtime(spec.backend)
+            except KeyboardInterrupt:
+                print("Interrupted — local-embed not started.", file=sys.stderr)
+                return 130
+            if not installed:
+                from local_embed.cmd_set import backend_install_hint
+
+                problems.append(
+                    f"{spec.backend} backend not installed "
+                    f"({backend_install_hint(spec.backend)})"
+                )
         if spec.name == active_name and problems:
             print(
                 f"Error: active model '{spec.name}' cannot start: {'; '.join(problems)}",

{local_embed-0.2.0 → local_embed-0.2.2}/local_embed/cmd_set.py

@@ -24,6 +24,7 @@ from pathlib import Path
 from local_embed.config import load_config, write_config
 
 ENV_CACHE_KEY = "HF_HUB_CACHE"
+ENV_OFFLINE_KEY = "HF_HUB_OFFLINE"
 
 # Official prebuilt-wheel indexes for llama-cpp-python (the upstream llama.cpp
 # project).  PyPI carries no Windows wheels for llama-cpp-python, so on
@@ -42,16 +43,19 @@ def backend_install_hint(backend: str, platform: "str | None" = None) -> str:
     must be added or uv falls back to a source build (MSVC + CMake).  On
     Linux/macOS the plain PyPI wheel is fine.  (CUDA / Metal need their own
     indexes — see the README install matrix.)
+
+    The command always pins ``--python 3.13``: it is the version CI tests,
+    and without it uv would silently pick the newest Python it finds.
     """
     platform = platform or sys.platform
     if backend == "gguf":
         if platform == "win32":
             return (
-                "uv tool install 'local-embed[gguf]' "
+                "uv tool install --python 3.13 'local-embed[gguf]' "
                 f"--extra-index-url {LLAMA_CPP_INDEXES['cpu']}"
             )
-        return "uv tool install 'local-embed[gguf]'"
-    return f"uv tool install 'local-embed[{backend}]'"
+        return "uv tool install --python 3.13 'local-embed[gguf]'"
+    return f"uv tool install --python 3.13 'local-embed[{backend}]'"
 
 
 def resolve_cache(raw: "str | None", environ: "dict[str, str] | None" = None) -> str:
@@ -92,11 +96,18 @@ def _upsert_model(cfg: dict, model: str, entry: dict, port: int) -> dict:
 
 
 def set_transformer_model(cfg: dict, model: str, cache: str, port: int) -> dict:
-    """Upsert ``model`` as a transformer model, pin the cache it loads from."""
+    """Upsert ``model`` as a transformer model, pin the pre-downloaded cache.
+
+    The server is intended to run **offline** — the model is pre-downloaded
+    into ``cache`` and ``set`` requires it to already be there — so
+    ``HF_HUB_OFFLINE=1`` is written alongside the cache (the server resolves
+    the repo against this cache and never hits the network).
+    """
     env = dict(cfg.get("env") or {})
     env[ENV_CACHE_KEY] = cache
+    env[ENV_OFFLINE_KEY] = "1"
     out = _upsert_model(cfg, model, {"backend": "transformer", "model": model}, port)
-    out["env"] = env  # existing env keys preserved; HF_HUB_CACHE (re)written
+    out["env"] = env  # existing env keys preserved; cache + offline (re)written
     return out
 
 

{local_embed-0.2.0 → local_embed-0.2.2}/pyproject.toml

@@ -1,6 +1,6 @@
 [project]
 name = "local-embed"
-version = "0.2.0"
+version = "0.2.2"
 description = "Standalone local embedding server — GGUF / transformer models as an OpenAI-compatible /v1/embeddings service"
 readme = "README.md"
 license = { text = "MIT" }
@@ -11,6 +11,7 @@ requires-python = ">=3.13"
 dependencies = [
     "fastmcp>=2.0.0",
     "starlette>=0.37.0",
+    "json5>=0.15.0",
 ]
 classifiers = [
     "Development Status :: 3 - Alpha",

{local_embed-0.2.0 → local_embed-0.2.2}/tests/test_set.py

@@ -10,15 +10,18 @@ import pytest
 pytestmark = pytest.mark.unit
 
 from local_embed.cli import main
+
+import local_embed.cli as cli
 from local_embed.cmd_set import (
     ENV_CACHE_KEY,
+    ENV_OFFLINE_KEY,
     backend_install_hint,
     model_in_cache,
     resolve_cache,
     set_gguf_model,
     set_transformer_model,
 )
-from local_embed.config import load_config
+from local_embed.config import load_config, write_config
 
 
 class TestResolveCache:
@@ -41,9 +44,15 @@ class TestSetTransformerModel:
             "models": {"BAAI/bge-m3": {"backend": "transformer", "model": "BAAI/bge-m3"}},
             "active_model": "BAAI/bge-m3",
             "port": 8000,
-            "env": {ENV_CACHE_KEY: "C:\\hub"},
+            "env": {ENV_CACHE_KEY: "C:\\hub", ENV_OFFLINE_KEY: "1"},
         }
 
+    def test_offline_flag_defaults_on(self):
+        # the server should never silently refetch a missing repo — pin
+        # HF_HUB_OFFLINE even when the user only supplied a cache dir
+        out = set_transformer_model({}, "BAAI/bge-m3", "C:\\hub", 8000)
+        assert out["env"][ENV_OFFLINE_KEY] == "1"
+
     def test_preserves_existing_models_and_env(self):
         cfg = {
             "models": {"old": {"backend": "gguf", "gguf_path": "/a.gguf"}},
@@ -111,18 +120,22 @@ def config_path(tmp_path, monkeypatch):
 
 class TestBackendInstallHint:
     def test_gguf_linux_macos_plain(self):
-        assert backend_install_hint("gguf", "linux") == "uv tool install 'local-embed[gguf]'"
-        assert backend_install_hint("gguf", "darwin") == "uv tool install 'local-embed[gguf]'"
+        assert backend_install_hint("gguf", "linux") == (
+            "uv tool install --python 3.13 'local-embed[gguf]'"
+        )
+        assert backend_install_hint("gguf", "darwin") == (
+            "uv tool install --python 3.13 'local-embed[gguf]'"
+        )
 
     def test_gguf_windows_adds_cpu_index(self):
         assert backend_install_hint("gguf", "win32") == (
-            "uv tool install 'local-embed[gguf]' "
+            "uv tool install --python 3.13 'local-embed[gguf]' "
             "--extra-index-url https://abetlen.github.io/llama-cpp-python/whl/cpu"
         )
 
     def test_transformer_plain(self):
         assert backend_install_hint("transformer", "win32") == (
-            "uv tool install 'local-embed[transformer]'"
+            "uv tool install --python 3.13 'local-embed[transformer]'"
         )
 
 
@@ -206,4 +219,70 @@ class TestRunSetGguf:
         assert main(["set-gguf", "bge-m3", "--path", str(gguf)]) == 0
         first = config_path.read_text(encoding="utf-8")
         assert main(["set-gguf", "bge-m3", "--path", str(gguf)]) == 0
-        assert config_path.read_text(encoding="utf-8") == first
+        assert config_path.read_text(encoding="utf-8") == first
+
+
+class TestCliCtrlC:
+    """Ctrl-C during the active backend check must exit 130, never traceback.
+
+    Regression: the startup validation calls ``resolve_backend_runtime``
+    (a REAL import — torch takes seconds), and a Ctrl-C landing mid-import
+    used to propagate as a raw KeyboardInterrupt traceback.
+    """
+
+    def _write_active_transformer(self, config_path):
+        write_config(
+            {
+                "active_model": "bge-m3-transformer",
+                "models": {
+                    "bge-m3-transformer": {
+                        "backend": "transformer",
+                        "model": "BAAI/bge-m3",
+                    }
+                },
+            },
+            config_path,
+        )
+
+    def test_interrupt_during_active_check_exits_130(self, monkeypatch, config_path, capsys):
+        self._write_active_transformer(config_path)
+
+        def interrupt(_backend):
+            raise KeyboardInterrupt()
+
+        monkeypatch.setattr(cli, "resolve_backend_runtime", interrupt)
+        code = cli.main([])
+        assert code == 130
+        assert "Interrupted" in capsys.readouterr().err
+
+    def test_non_active_backend_not_resolved(self, monkeypatch, config_path):
+        # active = gguf; the non-active transformer must NOT trigger the
+        # torch import at startup (it is only checked when actually loaded).
+        write_config(
+            {
+                "active_model": "bge-m3",
+                "models": {
+                    "bge-m3": {"backend": "gguf", "gguf_path": "/x.gguf"},
+                    "bge-m3-transformer": {
+                        "backend": "transformer",
+                        "model": "BAAI/bge-m3",
+                    },
+                },
+            },
+            config_path,
+        )
+        resolved = []
+
+        def tracking(backend):
+            resolved.append(backend)
+            return True
+
+        monkeypatch.setattr(cli, "resolve_backend_runtime", tracking)
+        # stop after the validation loop instead of serving (serve_standalone
+        # is imported lazily inside main, so patch its source module)
+        monkeypatch.setattr(
+            "local_embed.server.serve_standalone", lambda *a, **k: 0
+        )
+        code = cli.main([])
+        assert code == 0
+        assert resolved == ["gguf"]  # only the active backend was imported