abstractcore 2.35.0__tar.gz → 2.35.1__tar.gz

{abstractcore-2.35.0 → abstractcore-2.35.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: abstractcore
-Version: 2.35.0
+Version: 2.35.1
 Summary: Unified interface to all LLM providers with essential infrastructure for tool calling, streaming, and model management
 Author-email: Laurent-Philippe Albou <contact@abstractcore.ai>
 Maintainer-email: Laurent-Philippe Albou <contact@abstractcore.ai>
@@ -36,10 +36,11 @@ Requires-Dist: anthropic<1.0.0,>=0.25.0
 Requires-Dist: tiktoken<1.0.0,>=0.5.0
 Requires-Dist: requests<3.0.0,>=2.25.0
 Requires-Dist: beautifulsoup4<5.0.0,>=4.12.0
-Requires-Dist: lxml<6.0.0,>=4.9.0
+Requires-Dist: lxml<7.0.0,>=4.9.0
 Requires-Dist: ddgs<10.0.0,>=9.10.0; python_version >= "3.10"
 Requires-Dist: duckduckgo-search<4.0.0,>=3.8.0; python_version < "3.10"
 Requires-Dist: psutil<8,>=5.9
+Requires-Dist: huggingface_hub<2.0.0,>=0.20.0
 Requires-Dist: Pillow<13.0.0,>=10.0.0
 Requires-Dist: pypdf<7.0.0,>=6.0.0
 Requires-Dist: unstructured[docx,odt,pptx,rtf,xlsx]<1.0.0,>=0.10.0
@@ -49,8 +50,8 @@ Requires-Dist: uvicorn[standard]<1.0.0,>=0.23.0
 Requires-Dist: python-multipart<1.0.0,>=0.0.6
 Requires-Dist: sse-starlette<4,>=1.8
 Requires-Dist: cryptography>=41.0.0
-Requires-Dist: abstractvoice>=0.20.0
-Requires-Dist: abstractvision>=0.3.34
+Requires-Dist: abstractvoice>=0.21.1
+Requires-Dist: abstractvision>=0.3.36
 Requires-Dist: abstractmusic>=0.1.17; python_version >= "3.10"
 Requires-Dist: abstract3d>=0.3.2; python_version >= "3.10"
 Provides-Extra: apple
@@ -69,9 +70,9 @@ Requires-Dist: accelerate>=1.0.0; extra == "apple"
 Requires-Dist: sentence-transformers<6.0.0,>=5.1.0; extra == "apple"
 Requires-Dist: numpy<3.0.0,>=2.1.0; extra == "apple"
 Requires-Dist: Pillow<13.0.0,>=12.1.1; extra == "apple"
-Requires-Dist: abstractvoice[all-apple]>=0.20.0; extra == "apple"
+Requires-Dist: abstractvoice[all-apple]>=0.21.1; extra == "apple"
 Requires-Dist: omnivoice>=0.1.5; extra == "apple"
-Requires-Dist: abstractvision[all-apple]>=0.3.34; extra == "apple"
+Requires-Dist: abstractvision[all-apple]>=0.3.36; extra == "apple"
 Requires-Dist: abstractmusic[all-apple]>=0.1.17; extra == "apple"
 Requires-Dist: playwright<2.0.0,>=1.49.0; extra == "apple"
 Provides-Extra: gpu
@@ -87,9 +88,9 @@ Requires-Dist: gguf<1.0.0,>=0.10.0; extra == "gpu"
 Requires-Dist: sentence-transformers<6.0.0,>=5.1.0; extra == "gpu"
 Requires-Dist: numpy<3.0.0,>=1.20.0; python_version < "3.13" and extra == "gpu"
 Requires-Dist: numpy<3.0.0,>=2.1.0; python_version >= "3.13" and extra == "gpu"
-Requires-Dist: abstractvoice[all-gpu]>=0.20.0; extra == "gpu"
+Requires-Dist: abstractvoice[all-gpu]>=0.21.1; extra == "gpu"
 Requires-Dist: omnivoice>=0.1.5; extra == "gpu"
-Requires-Dist: abstractvision[all-gpu]>=0.3.34; extra == "gpu"
+Requires-Dist: abstractvision[all-gpu]>=0.3.36; extra == "gpu"
 Requires-Dist: abstractmusic[all-gpu]>=0.1.17; extra == "gpu"
 Requires-Dist: playwright<2.0.0,>=1.49.0; extra == "gpu"
 Provides-Extra: remote
@@ -135,11 +136,11 @@ Requires-Dist: sentence-transformers<6.0.0,>=5.1.0; extra == "embeddings"
 Requires-Dist: numpy<3.0.0,>=1.20.0; python_version < "3.13" and extra == "embeddings"
 Requires-Dist: numpy<3.0.0,>=2.1.0; python_version >= "3.13" and extra == "embeddings"
 Provides-Extra: vision-diffusers
-Requires-Dist: abstractvision[huggingface]>=0.3.34; extra == "vision-diffusers"
+Requires-Dist: abstractvision[huggingface]>=0.3.36; extra == "vision-diffusers"
 Provides-Extra: vision-sdcpp
-Requires-Dist: abstractvision[sdcpp]>=0.3.34; extra == "vision-sdcpp"
+Requires-Dist: abstractvision[sdcpp]>=0.3.36; extra == "vision-sdcpp"
 Provides-Extra: vision-local
-Requires-Dist: abstractvision[local]>=0.3.34; extra == "vision-local"
+Requires-Dist: abstractvision[local]>=0.3.36; extra == "vision-local"
 Provides-Extra: browser
 Requires-Dist: playwright<2.0.0,>=1.49.0; extra == "browser"
 Provides-Extra: pdf-pymupdf-commercial
@@ -196,7 +197,7 @@ Dynamic: license-file
 Unified LLM Interface
 > Write once, run everywhere
 
-What's new in 2.35.0: the model capability registry covers the current vendor lineups (GPT-6 and GPT-6.1, Claude 5.5 and Fable 5.1, Grok 4.3 to 4.7, Qwen 3.8, DeepSeek V4.1 Flash, GLM 5.3, MiniMax M3.1 Flash, Kimi K3 and K3.1 Pro, LFM3, Spark X2.5/X2.6, Nemotron 3.5, Bonsai 2 and more), so an image, video or voice route can name them; and a GGUF or MLX re-packaging of a registered model (`unsloth/...-GGUF:Q4_K_M`, `mlx-community/...-4bit`, bartowski's `Org_Model-GGUF`) resolves to that model's entry instead of the text-only default. Transcriptions report `language` and `detected_language`. See the [changelog](CHANGELOG.md#2350---2026-11-07).
+What's new in 2.35.1: a plain `pip install "abstractcore[gpu]"` on a Linux NVIDIA machine now runs GGUF models on the GPU (AbstractCore installs llama.cpp's CUDA build itself, once, the first time it is needed). In 2.35.0: the model capability registry covers the current vendor lineups (GPT-6 and GPT-6.1, Claude 5.5 and Fable 5.1, Grok 4.3 to 4.7, Qwen 3.8, DeepSeek V4.1 Flash, GLM 5.3, MiniMax M3.1 Flash, Kimi K3 and K3.1 Pro, LFM3, Spark X2.5/X2.6, Nemotron 3.5, Bonsai 2 and more), so an image, video or voice route can name them; and a GGUF or MLX re-packaging of a registered model (`unsloth/...-GGUF:Q4_K_M`, `mlx-community/...-4bit`, bartowski's `Org_Model-GGUF`) resolves to that model's entry instead of the text-only default. Transcriptions report `language` and `detected_language`. See the [changelog](CHANGELOG.md#2351---2026-11-10).
 
 What's new in 2.34.0: a local model that no call has used for 60 minutes is unloaded from memory and loads
 again on its next call; a model kept loaded (Keep loaded) never is, and no running call is interrupted

{abstractcore-2.35.0 → abstractcore-2.35.1}/README.md

@@ -9,7 +9,7 @@
 Unified LLM Interface
 > Write once, run everywhere
 
-What's new in 2.35.0: the model capability registry covers the current vendor lineups (GPT-6 and GPT-6.1, Claude 5.5 and Fable 5.1, Grok 4.3 to 4.7, Qwen 3.8, DeepSeek V4.1 Flash, GLM 5.3, MiniMax M3.1 Flash, Kimi K3 and K3.1 Pro, LFM3, Spark X2.5/X2.6, Nemotron 3.5, Bonsai 2 and more), so an image, video or voice route can name them; and a GGUF or MLX re-packaging of a registered model (`unsloth/...-GGUF:Q4_K_M`, `mlx-community/...-4bit`, bartowski's `Org_Model-GGUF`) resolves to that model's entry instead of the text-only default. Transcriptions report `language` and `detected_language`. See the [changelog](CHANGELOG.md#2350---2026-11-07).
+What's new in 2.35.1: a plain `pip install "abstractcore[gpu]"` on a Linux NVIDIA machine now runs GGUF models on the GPU (AbstractCore installs llama.cpp's CUDA build itself, once, the first time it is needed). In 2.35.0: the model capability registry covers the current vendor lineups (GPT-6 and GPT-6.1, Claude 5.5 and Fable 5.1, Grok 4.3 to 4.7, Qwen 3.8, DeepSeek V4.1 Flash, GLM 5.3, MiniMax M3.1 Flash, Kimi K3 and K3.1 Pro, LFM3, Spark X2.5/X2.6, Nemotron 3.5, Bonsai 2 and more), so an image, video or voice route can name them; and a GGUF or MLX re-packaging of a registered model (`unsloth/...-GGUF:Q4_K_M`, `mlx-community/...-4bit`, bartowski's `Org_Model-GGUF`) resolves to that model's entry instead of the text-only default. Transcriptions report `language` and `detected_language`. See the [changelog](CHANGELOG.md#2351---2026-11-10).
 
 What's new in 2.34.0: a local model that no call has used for 60 minutes is unloaded from memory and loads
 again on its next call; a model kept loaded (Keep loaded) never is, and no running call is interrupted

{abstractcore-2.35.0 → abstractcore-2.35.1}/abstractcore/config/route_engines.py

@@ -63,7 +63,7 @@ __all__ = [
 # AbstractCore's AbstractVoice floor (it has `abstractvoice.engine_runtime`, the
 # public runtime probe, since 0.15.0). Mirrors the abstractvoice pins in
 # pyproject.toml; the release stager raises both together.
-ABSTRACTVOICE_ENGINE_RUNTIME_FLOOR = "0.20.0"
+ABSTRACTVOICE_ENGINE_RUNTIME_FLOOR = "0.21.1"
 
 # Voice providers that run remotely: AbstractVoice needs nothing beyond its
 # core install for them, and whether they are configured is not a runtime

{abstractcore-2.35.0 → abstractcore-2.35.1}/abstractcore/download.py

@@ -12,7 +12,7 @@ from enum import Enum
 from typing import Any, AsyncIterator, Dict, List, Optional
 
 import httpx
-from .utils.install_settings import local_engines_hint
+from .utils.install_settings import light_repair_hint, local_engines_hint
 
 
 class DownloadStatus(Enum):
@@ -226,10 +226,8 @@ async def _download_huggingface(
     except ImportError:
         yield DownloadProgress(
             status=DownloadStatus.ERROR,
-            message=(
-                "huggingface_hub is not installed. "
-                f"{local_engines_hint('Downloading Hugging Face models')}"
-            ),
+            # huggingface_hub is part of the light set (2.35.1): missing means a broken install.
+            message=f"huggingface_hub is not installed: {light_repair_hint()}",
         )
         return
 

{abstractcore-2.35.0 → abstractcore-2.35.1}/abstractcore/embeddings/manager.py

@@ -250,6 +250,35 @@ def _refuse_resident_gguf_embedding(path: Any) -> None:
                 )
 
 
+def release_shared_gguf_embeddings_at_exit() -> int:
+    """Close every resident GGUF embedding encoder (llama.cpp) before the interpreter tears modules
+    down. Without this, a process that embedded with a GGUF model on Apple silicon aborted at exit
+    with ``ggml-metal-device.m: GGML_ASSERT([rsets->data count] == 0)`` (exit code 134) after every
+    call had succeeded: the llama.cpp context was freed during module teardown, after the Metal
+    device, instead of before it (found on the Mac during the 0.14.0 profile proof, 2.35.1).
+    SentenceTransformer models need nothing here. Returns the number of encoders closed."""
+    closed = 0
+    with _SHARED_EMBEDDING_LOCK:
+        entries = list(_SHARED_EMBEDDING_MODELS.values())
+    for entry in entries:
+        model = entry.model
+        close = getattr(model, "close", None)
+        if model is None or not callable(close) or not hasattr(model, "_llm"):
+            continue
+        try:
+            close()
+            closed += 1
+        except Exception:
+            pass
+        entry.model = None
+    return closed
+
+
+# Registered once at import: atexit handlers run before module teardown, in reverse order of
+# registration, so the encoders go first (see release_shared_gguf_embeddings_at_exit).
+atexit.register(release_shared_gguf_embeddings_at_exit)
+
+
 def _atexit_save_caches(ref: "weakref.ReferenceType") -> None:
     manager = ref()
     if manager is None:

{abstractcore-2.35.0 → abstractcore-2.35.1}/abstractcore/providers/huggingface_provider.py

@@ -1771,6 +1771,8 @@ class HuggingFaceProvider(BaseProvider):
             "context_reservation": dict(getattr(self, "_context_reservation", None) or {}),
             "requested_context": getattr(self, "_gguf_requested_context", None),
             "n_gpu_layers": self.n_gpu_layers,
+            # Why GGUF runs where it runs on Linux + NVIDIA (utils/llama_cuda_swap): outcome and reason.
+            "llama_cuda_swap": getattr(self, "gguf_cuda_swap", None),
             "max_output_tokens": self.max_output_tokens,
             "output_ceiling_source": getattr(self, "_output_ceiling_source", None),
             "prompt_cache_store": getattr(self, "_prompt_cache_store", None),
@@ -7054,6 +7056,21 @@ class HuggingFaceProvider(BaseProvider):
         # don't need to import PyTorch.
         supports_gpu_offload = False
         try:
+            # Linux + NVIDIA: an install from PyPI alone carries the processor build of llama.cpp; the
+            # CUDA build is swapped in once per environment before the first import (llama_cuda_swap).
+            from ..utils.llama_cuda_swap import ensure_llama_cpp_cuda_build
+
+            self.gguf_cuda_swap = ensure_llama_cpp_cuda_build()
+            if self.gguf_cuda_swap.get("outcome") == "skipped" and self.gguf_cuda_swap.get("build"):
+                # A driver or card reason (a failed swap announces itself in llama_cuda_swap).
+                from ..utils.llama_cuda_swap import notice
+
+                notice(
+                    "GGUF on the processor: llama-cpp-python is the CPU build and the CUDA build was not "
+                    f"installed ({self.gguf_cuda_swap.get('reason')}). The one-line installer does it; see "
+                    "docs/installation.md."
+                )
+            prepare_llama_cpp_import()
             import llama_cpp  # type: ignore
 
             if sys.platform == "win32":

{abstractcore-2.35.0 → abstractcore-2.35.1}/abstractcore/utils/install_settings.py

@@ -116,6 +116,11 @@ def light_repair_hint() -> str:
     return f"the AbstractCore install is incomplete; repair it with: {LIGHT_INSTALL}"
 
 
+LLAMA_CPP_LINUX_NOTE = (
+    "llama.cpp has no prebuilt wheel on PyPI; the AbstractCore installer adds the right build for this "
+    "machine (curl -LsSf https://abstractframework.ai/core/install.sh | sh), and "
+    + GPU_INSTALL + " alone builds it from source (needs a C compiler)"
+)
 LLAMA_CPP_WINDOWS_NOTE = (
     "on Windows the AbstractFramework installer adds llama.cpp's prebuilt GPU build; with plain pip, "
     "llama.cpp is not part of abstractcore[gpu] on Windows"
@@ -137,4 +142,12 @@ def llama_cpp_hint(what: str = "GGUF models (llama.cpp)") -> str:
 
     if sys.platform == "win32" and local_engines_setting() == "gpu":
         return f"{what}: {LLAMA_CPP_WINDOWS_NOTE}"
+    if sys.platform.startswith("linux") and local_engines_setting() == "gpu":
+        # PyPI has llama-cpp-python as a source build only: `abstractcore[gpu]` compiles it (a C
+        # compiler and, for CUDA, the toolkit), while the installer adds the prebuilt wheel for this
+        # machine (CPU, or CUDA swapped in by AbstractCore itself; utils/llama_cuda_swap). Found on a
+        # processor-only Linux host during the 0.14.0 profile proof (2.35.1).
+        return (
+            f"{what}: {LLAMA_CPP_LINUX_NOTE}"
+        )
     return local_engines_hint(what)

abstractcore-2.35.1/abstractcore/utils/llama_cuda_swap.py

@@ -0,0 +1,277 @@
+"""Give a plain ``pip install "abstractcore[gpu]"`` the CUDA build of llama.cpp on an NVIDIA Linux host.
+
+The requirement (operator, 2026-11-10): the ``gpu`` install must work fully on a GPU machine, GGUF models
+included. PyPI carries ``llama-cpp-python`` as a source build only, so pip resolves abetlen's prebuilt
+*processor* wheel (``llama_supports_gpu_offload()`` is False) and every GGUF model runs on the CPU
+although PyTorch sees the card. The prebuilt CUDA builds live on abetlen's own index
+(https://abetlen.github.io/llama-cpp-python/whl/<build>/), which a pip extra cannot name, and they are
+400 to 600 MB each, above PyPI's file limit, so they cannot be re-hosted there either.
+
+The one-line installers already swap the CUDA build in after the install (``scripts/install.sh``,
+the framework installer). This module does the same swap from inside the package, once per
+environment, the first time AbstractCore is about to import ``llama_cpp`` on a host where it would
+otherwise run GGUF on the processor:
+
+1. Linux x86_64 only (the CUDA pages carry ``manylinux_2_35_x86_64`` wheels alone), opt-out with
+   ``ABSTRACTCORE_LLAMA_CUDA_SWAP=0``.
+2. The installed ``llama_cpp`` is a CUDA build already (``lib/libggml-cuda.so``, the way the installers
+   leave it): nothing to do, no pip call. This is the common case on a host set up by the one-liners.
+3. PyTorch's CUDA major is read from ``torch/version.py`` on disk (torch is never imported here) and the
+   NVIDIA driver from ``nvidia-smi``: CUDA 13 wants driver 580 or newer (build ``cu130``), CUDA 12 wants
+   525 or newer (``cu125``). Without a CUDA torch, a driver, or with one too old, the processor build
+   stays and the reason is recorded.
+4. The swap is the installers' command: ``pip install --no-index --no-deps --force-reinstall
+   --find-links <index>/<build>/llama-cpp-python/ llama-cpp-python==<installed version>`` (``uv pip``
+   with ``--reinstall-package`` when the environment has no pip, as uv venvs do). Same version, same
+   files the installer would have written. One loud log line before and after.
+5. The outcome is written next to the package (``site-packages/llama_cpp/.abstractcore-cuda-swap.json``)
+   so a failed swap (offline host, read-only site-packages) is not retried at every model load; a
+   reinstall of llama-cpp-python removes the marker with the folder and the check runs again.
+
+`ensure_llama_cpp_cuda_build()` returns a dict (``outcome``, ``reason``, ``build``, ``version``) that the
+HuggingFace provider records in its runtime info, so a user whose GGUF model still runs on the processor
+reads why in the first answer. When ``llama_cpp`` was already imported in this process (the processor
+library is loaded), the swap still happens on disk and the outcome says the next process gets the GPU.
+"""
+
+from __future__ import annotations
+
+import importlib.metadata
+import importlib.util
+import json
+import logging
+import os
+import platform
+import re
+import shutil
+import subprocess
+import sys
+import time
+from pathlib import Path
+from typing import Any, Dict, List, Optional
+
+logger = logging.getLogger(__name__)
+
+INDEX = "https://abetlen.github.io/llama-cpp-python/whl"
+# CUDA major of the installed torch -> (llama.cpp build folder, minimum NVIDIA driver major).
+BUILDS: Dict[int, tuple] = {13: ("cu130", 580), 12: ("cu125", 525)}
+ENV_OPT_OUT = "ABSTRACTCORE_LLAMA_CUDA_SWAP"
+MARKER_NAME = ".abstractcore-cuda-swap.json"
+SWAP_TIMEOUT_S = 900
+
+_result: Optional[Dict[str, Any]] = None
+
+
+def _package_dir(name: str) -> Optional[Path]:
+    try:
+        spec = importlib.util.find_spec(name)
+    except Exception:
+        return None
+    if spec is None:
+        return None
+    locations = list(spec.submodule_search_locations or [])
+    if not locations and spec.origin:
+        locations = [str(Path(spec.origin).parent)]
+    return Path(locations[0]) if locations else None
+
+
+def llama_cpp_is_cuda_build() -> bool:
+    """True when the installed llama-cpp-python ships its CUDA backend. Never imports llama_cpp."""
+    pkg = _package_dir("llama_cpp")
+    if pkg is None:
+        return False
+    lib = pkg / "lib"
+    return lib.is_dir() and any(lib.glob("libggml-cuda.so*"))
+
+
+def torch_cuda_major() -> Optional[int]:
+    """The CUDA major of the installed torch wheel, read from ``torch/version.py`` (``cuda = '13.0'``);
+    None without torch or for a processor/ROCm build. torch is never imported."""
+    pkg = _package_dir("torch")
+    if pkg is None:
+        return None
+    try:
+        text = (pkg / "version.py").read_text(encoding="utf-8", errors="replace")
+    except OSError:
+        return None
+    # torch <= 2.12: `cuda = '13.0'`; torch 2.13: `cuda: Optional[str] = '13.0'` (annotated).
+    match = re.search(r"^cuda\s*(?::[^=\n]*)?=\s*['\"](\d+)\.(\d+)['\"]", text, re.MULTILINE)
+    return int(match.group(1)) if match else None
+
+
+def nvidia_driver_major() -> Optional[int]:
+    """The NVIDIA driver's major version from ``nvidia-smi``; None when no driver answers."""
+    exe = shutil.which("nvidia-smi")
+    if not exe:
+        return None
+    try:
+        out = subprocess.run([exe, "--query-gpu=driver_version", "--format=csv,noheader"],
+                             capture_output=True, text=True, timeout=10)
+    except (OSError, subprocess.SubprocessError):
+        return None
+    if out.returncode != 0:
+        return None
+    first = (out.stdout or "").strip().splitlines()
+    if not first:
+        return None
+    head = first[0].strip().split(".")[0]
+    return int(head) if head.isdigit() else None
+
+
+def installed_version() -> Optional[str]:
+    try:
+        return importlib.metadata.version("llama-cpp-python")
+    except importlib.metadata.PackageNotFoundError:
+        return None
+
+
+def marker_path() -> Optional[Path]:
+    pkg = _package_dir("llama_cpp")
+    return (pkg / MARKER_NAME) if pkg is not None else None
+
+
+def _read_marker() -> Optional[Dict[str, Any]]:
+    path = marker_path()
+    if path is None or not path.is_file():
+        return None
+    try:
+        data = json.loads(path.read_text(encoding="utf-8"))
+    except (OSError, ValueError):
+        return None
+    return data if isinstance(data, dict) else None
+
+
+def _write_marker(data: Dict[str, Any]) -> None:
+    path = marker_path()
+    if path is None:
+        return
+    try:
+        path.write_text(json.dumps(data, indent=1, sort_keys=True), encoding="utf-8")
+    except OSError as exc:
+        logger.debug("could not write %s: %s", path, exc)
+
+
+def swap_command(build: str, version: str) -> Optional[List[str]]:
+    """The installers' swap for this interpreter: pip when the environment has it, else ``uv pip``
+    (uv venvs carry no pip). None when neither is available."""
+    links = f"{INDEX}/{build}/llama-cpp-python/"
+    requirement = f"llama-cpp-python=={version}"
+    if importlib.util.find_spec("pip") is not None:
+        return [sys.executable, "-m", "pip", "install", "--no-index", "--no-deps", "--force-reinstall",
+                "--find-links", links, requirement]
+    uv = shutil.which("uv")
+    if uv:
+        return [uv, "pip", "install", "--python", sys.executable, "--no-index", "--no-deps",
+                "--reinstall-package", "llama-cpp-python", "--find-links", links, requirement]
+    return None
+
+
+def run_swap(command: List[str]) -> subprocess.CompletedProcess:
+    """Run the swap command (separate so tests replace it)."""
+    return subprocess.run(command, capture_output=True, text=True, timeout=SWAP_TIMEOUT_S)
+
+
+def notice(message: str) -> None:
+    """One line on stderr AND in the log: AbstractCore's console log level is ERROR by default, so a
+    `logger.warning` alone never reaches a user; a download of several hundred MB inside
+    `create_llm` must be announced where the user is looking."""
+    try:
+        sys.stderr.write(f"AbstractCore: {message}\n")
+        sys.stderr.flush()
+    except Exception:
+        pass
+    logger.warning("AbstractCore: %s", message)
+
+
+def _outcome(outcome: str, reason: str = "", **extra: Any) -> Dict[str, Any]:
+    data: Dict[str, Any] = {"outcome": outcome, "reason": reason}
+    data.update(extra)
+    return data
+
+
+def ensure_llama_cpp_cuda_build() -> Dict[str, Any]:
+    """Make sure llama.cpp is its CUDA build on an NVIDIA Linux host before AbstractCore imports it.
+
+    Returns the outcome (cached for the process):
+    ``cuda-build-present`` (nothing done), ``swapped`` (the CUDA build is now installed),
+    ``swapped-restart-needed`` (installed, but this process already loaded the processor build),
+    ``failed`` (the swap ran and did not deliver; ``reason`` says why; not retried until
+    llama-cpp-python is reinstalled), ``skipped`` (not applicable here; ``reason`` says why)."""
+    global _result
+    if _result is not None:
+        return dict(_result)
+    _result = _decide_and_swap()
+    return dict(_result)
+
+
+def _decide_and_swap() -> Dict[str, Any]:
+    if not sys.platform.startswith("linux"):
+        return _outcome("skipped", "not Linux")
+    if platform.machine().lower() not in {"x86_64", "amd64"}:
+        return _outcome("skipped", f"no prebuilt CUDA llama.cpp for {platform.machine()} (x86_64 only)")
+    if os.environ.get(ENV_OPT_OUT, "").strip() == "0":
+        return _outcome("skipped", f"{ENV_OPT_OUT}=0")
+    version = installed_version()
+    if version is None or _package_dir("llama_cpp") is None:
+        return _outcome("skipped", "llama-cpp-python is not installed")
+    if llama_cpp_is_cuda_build():
+        return _outcome("cuda-build-present", "", version=version)
+    marker = _read_marker()
+    if marker and marker.get("version") == version and marker.get("outcome") in {"failed", "swapped"}:
+        # "swapped" with a processor build on disk means someone reinstalled the CPU wheel over it
+        # without removing the marker (pip keeps unknown files in the folder): run again.
+        if marker.get("outcome") == "failed":
+            return dict(marker)
+    major = torch_cuda_major()
+    if major is None:
+        return _outcome("skipped", "PyTorch has no CUDA build here", version=version)
+    if major not in BUILDS:
+        return _outcome("skipped", f"no prebuilt llama.cpp build for CUDA {major}", version=version)
+    build, floor = BUILDS[major]
+    driver = nvidia_driver_major()
+    if driver is None:
+        return _outcome("skipped", "no NVIDIA driver answers (nvidia-smi)", version=version, build=build)
+    if driver < floor:
+        return _outcome("skipped", f"PyTorch uses CUDA {major}, which needs NVIDIA driver {floor} or newer "
+                        f"(this one is {driver})", version=version, build=build)
+    command = swap_command(build, version)
+    if command is None:
+        data = _outcome("failed", "neither pip nor uv is available to install the CUDA build",
+                        version=version, build=build)
+        _write_marker(data)
+        return data
+    already_loaded = "llama_cpp" in sys.modules
+    notice(
+        f"llama-cpp-python {version} here is the processor build and this machine has an NVIDIA card "
+        f"(CUDA {major} torch, driver {driver}): installing its CUDA build ({build}) from {INDEX} so GGUF "
+        f"models run on the GPU; one-time, a few hundred MB. Set {ENV_OPT_OUT}=0 to keep the processor build."
+    )
+    started = time.time()
+    try:
+        proc = run_swap(command)
+    except (OSError, subprocess.SubprocessError) as exc:
+        data = _outcome("failed", f"the swap command did not run: {exc}", version=version, build=build,
+                        command=" ".join(command))
+        _write_marker(data)
+        notice(f"llama.cpp CUDA build not installed: {data['reason']}. GGUF models stay on the processor; the "
+               "one-line installer does this step too (docs/installation.md).")
+        return data
+    tail = ((proc.stdout or "") + (proc.stderr or "")).strip().splitlines()[-6:]
+    if proc.returncode != 0 or not llama_cpp_is_cuda_build():
+        why = (f"exit {proc.returncode}" if proc.returncode != 0 else
+               "the command succeeded but the installed files are not a CUDA build")
+        data = _outcome("failed", f"{why}: {' | '.join(tail)[:600]}", version=version, build=build,
+                        command=" ".join(command), seconds=round(time.time() - started, 1))
+        _write_marker(data)
+        notice(f"llama.cpp CUDA build not installed ({data['reason']}); GGUF models stay on the processor. "
+               "The one-line installer does this step too: see docs/installation.md.")
+        return data
+    data = _outcome("swapped-restart-needed" if already_loaded else "swapped", "", version=version, build=build,
+                    command=" ".join(command), seconds=round(time.time() - started, 1))
+    _write_marker(data)
+    if already_loaded:
+        notice(f"llama.cpp CUDA build {build} installed in {data['seconds']:.0f} s; this process had already loaded "
+               "the processor build, the next one runs GGUF models on the GPU.")
+    else:
+        notice(f"llama.cpp CUDA build {build} installed in {data['seconds']:.0f} s; GGUF models run on the GPU.")
+    return data

{abstractcore-2.35.0 → abstractcore-2.35.1}/abstractcore/utils/version.py

@@ -10,4 +10,4 @@ including when the package is installed from PyPI where pyproject.toml is not av
 """
 
 # Package version - update this when releasing new versions
-__version__ = "2.35.0"
+__version__ = "2.35.1"

{abstractcore-2.35.0 → abstractcore-2.35.1}/abstractcore/utils/windows_dll.py

@@ -146,7 +146,15 @@ def prepare_llama_cpp_import() -> List[str]:
             if folder not in entries:
                 os.environ["PATH"] = folder + (os.pathsep + os.environ["PATH"] if os.environ.get("PATH") else "")
             added.append(folder)
-    elif sys.platform.startswith("linux") and llama_cpp_is_cuda_build():
+    elif sys.platform.startswith("linux"):
+        # A plain `pip install abstractcore[gpu]` leaves the processor build of llama.cpp on an NVIDIA
+        # host (PyPI has no CUDA build); swap the CUDA build in first, once per environment.
+        from .llama_cuda_swap import ensure_llama_cpp_cuda_build
+
+        ensure_llama_cpp_cuda_build()
+        if not llama_cpp_is_cuda_build():
+            _prepared = added
+            return list(added)
         for library in linux_nvidia_cuda_libraries():
             try:
                 _handles.append(ctypes.CDLL(str(library), mode=getattr(ctypes, "RTLD_GLOBAL", 0)))

{abstractcore-2.35.0 → abstractcore-2.35.1}/abstractcore.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: abstractcore
-Version: 2.35.0
+Version: 2.35.1
 Summary: Unified interface to all LLM providers with essential infrastructure for tool calling, streaming, and model management
 Author-email: Laurent-Philippe Albou <contact@abstractcore.ai>
 Maintainer-email: Laurent-Philippe Albou <contact@abstractcore.ai>
@@ -36,10 +36,11 @@ Requires-Dist: anthropic<1.0.0,>=0.25.0
 Requires-Dist: tiktoken<1.0.0,>=0.5.0
 Requires-Dist: requests<3.0.0,>=2.25.0
 Requires-Dist: beautifulsoup4<5.0.0,>=4.12.0
-Requires-Dist: lxml<6.0.0,>=4.9.0
+Requires-Dist: lxml<7.0.0,>=4.9.0
 Requires-Dist: ddgs<10.0.0,>=9.10.0; python_version >= "3.10"
 Requires-Dist: duckduckgo-search<4.0.0,>=3.8.0; python_version < "3.10"
 Requires-Dist: psutil<8,>=5.9
+Requires-Dist: huggingface_hub<2.0.0,>=0.20.0
 Requires-Dist: Pillow<13.0.0,>=10.0.0
 Requires-Dist: pypdf<7.0.0,>=6.0.0
 Requires-Dist: unstructured[docx,odt,pptx,rtf,xlsx]<1.0.0,>=0.10.0
@@ -49,8 +50,8 @@ Requires-Dist: uvicorn[standard]<1.0.0,>=0.23.0
 Requires-Dist: python-multipart<1.0.0,>=0.0.6
 Requires-Dist: sse-starlette<4,>=1.8
 Requires-Dist: cryptography>=41.0.0
-Requires-Dist: abstractvoice>=0.20.0
-Requires-Dist: abstractvision>=0.3.34
+Requires-Dist: abstractvoice>=0.21.1
+Requires-Dist: abstractvision>=0.3.36
 Requires-Dist: abstractmusic>=0.1.17; python_version >= "3.10"
 Requires-Dist: abstract3d>=0.3.2; python_version >= "3.10"
 Provides-Extra: apple
@@ -69,9 +70,9 @@ Requires-Dist: accelerate>=1.0.0; extra == "apple"
 Requires-Dist: sentence-transformers<6.0.0,>=5.1.0; extra == "apple"
 Requires-Dist: numpy<3.0.0,>=2.1.0; extra == "apple"
 Requires-Dist: Pillow<13.0.0,>=12.1.1; extra == "apple"
-Requires-Dist: abstractvoice[all-apple]>=0.20.0; extra == "apple"
+Requires-Dist: abstractvoice[all-apple]>=0.21.1; extra == "apple"
 Requires-Dist: omnivoice>=0.1.5; extra == "apple"
-Requires-Dist: abstractvision[all-apple]>=0.3.34; extra == "apple"
+Requires-Dist: abstractvision[all-apple]>=0.3.36; extra == "apple"
 Requires-Dist: abstractmusic[all-apple]>=0.1.17; extra == "apple"
 Requires-Dist: playwright<2.0.0,>=1.49.0; extra == "apple"
 Provides-Extra: gpu
@@ -87,9 +88,9 @@ Requires-Dist: gguf<1.0.0,>=0.10.0; extra == "gpu"
 Requires-Dist: sentence-transformers<6.0.0,>=5.1.0; extra == "gpu"
 Requires-Dist: numpy<3.0.0,>=1.20.0; python_version < "3.13" and extra == "gpu"
 Requires-Dist: numpy<3.0.0,>=2.1.0; python_version >= "3.13" and extra == "gpu"
-Requires-Dist: abstractvoice[all-gpu]>=0.20.0; extra == "gpu"
+Requires-Dist: abstractvoice[all-gpu]>=0.21.1; extra == "gpu"
 Requires-Dist: omnivoice>=0.1.5; extra == "gpu"
-Requires-Dist: abstractvision[all-gpu]>=0.3.34; extra == "gpu"
+Requires-Dist: abstractvision[all-gpu]>=0.3.36; extra == "gpu"
 Requires-Dist: abstractmusic[all-gpu]>=0.1.17; extra == "gpu"
 Requires-Dist: playwright<2.0.0,>=1.49.0; extra == "gpu"
 Provides-Extra: remote
@@ -135,11 +136,11 @@ Requires-Dist: sentence-transformers<6.0.0,>=5.1.0; extra == "embeddings"
 Requires-Dist: numpy<3.0.0,>=1.20.0; python_version < "3.13" and extra == "embeddings"
 Requires-Dist: numpy<3.0.0,>=2.1.0; python_version >= "3.13" and extra == "embeddings"
 Provides-Extra: vision-diffusers
-Requires-Dist: abstractvision[huggingface]>=0.3.34; extra == "vision-diffusers"
+Requires-Dist: abstractvision[huggingface]>=0.3.36; extra == "vision-diffusers"
 Provides-Extra: vision-sdcpp
-Requires-Dist: abstractvision[sdcpp]>=0.3.34; extra == "vision-sdcpp"
+Requires-Dist: abstractvision[sdcpp]>=0.3.36; extra == "vision-sdcpp"
 Provides-Extra: vision-local
-Requires-Dist: abstractvision[local]>=0.3.34; extra == "vision-local"
+Requires-Dist: abstractvision[local]>=0.3.36; extra == "vision-local"
 Provides-Extra: browser
 Requires-Dist: playwright<2.0.0,>=1.49.0; extra == "browser"
 Provides-Extra: pdf-pymupdf-commercial
@@ -196,7 +197,7 @@ Dynamic: license-file
 Unified LLM Interface
 > Write once, run everywhere
 
-What's new in 2.35.0: the model capability registry covers the current vendor lineups (GPT-6 and GPT-6.1, Claude 5.5 and Fable 5.1, Grok 4.3 to 4.7, Qwen 3.8, DeepSeek V4.1 Flash, GLM 5.3, MiniMax M3.1 Flash, Kimi K3 and K3.1 Pro, LFM3, Spark X2.5/X2.6, Nemotron 3.5, Bonsai 2 and more), so an image, video or voice route can name them; and a GGUF or MLX re-packaging of a registered model (`unsloth/...-GGUF:Q4_K_M`, `mlx-community/...-4bit`, bartowski's `Org_Model-GGUF`) resolves to that model's entry instead of the text-only default. Transcriptions report `language` and `detected_language`. See the [changelog](CHANGELOG.md#2350---2026-11-07).
+What's new in 2.35.1: a plain `pip install "abstractcore[gpu]"` on a Linux NVIDIA machine now runs GGUF models on the GPU (AbstractCore installs llama.cpp's CUDA build itself, once, the first time it is needed). In 2.35.0: the model capability registry covers the current vendor lineups (GPT-6 and GPT-6.1, Claude 5.5 and Fable 5.1, Grok 4.3 to 4.7, Qwen 3.8, DeepSeek V4.1 Flash, GLM 5.3, MiniMax M3.1 Flash, Kimi K3 and K3.1 Pro, LFM3, Spark X2.5/X2.6, Nemotron 3.5, Bonsai 2 and more), so an image, video or voice route can name them; and a GGUF or MLX re-packaging of a registered model (`unsloth/...-GGUF:Q4_K_M`, `mlx-community/...-4bit`, bartowski's `Org_Model-GGUF`) resolves to that model's entry instead of the text-only default. Transcriptions report `language` and `detected_language`. See the [changelog](CHANGELOG.md#2351---2026-11-10).
 
 What's new in 2.34.0: a local model that no call has used for 60 minutes is unloaded from memory and loads
 again on its next call; a model kept loaded (Keep loaded) never is, and no running call is interrupted

{abstractcore-2.35.0 → abstractcore-2.35.1}/abstractcore.egg-info/SOURCES.txt

@@ -279,6 +279,7 @@ abstractcore/utils/identity.py
 abstractcore/utils/install_settings.py
 abstractcore/utils/jsonish.py
 abstractcore/utils/kv_geometry.py
+abstractcore/utils/llama_cuda_swap.py
 abstractcore/utils/memory.py
 abstractcore/utils/message_preprocessor.py
 abstractcore/utils/model_cache.py