poolprobe 1.1.0__tar.gz → 1.2.0__tar.gz

{poolprobe-1.1.0 → poolprobe-1.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: poolprobe
-Version: 1.1.0
+Version: 1.2.0
 Summary: A doctor for OpenAI-compatible LLM gateways: probe the pool, classify every failure, and only then talk about pruning.
 License: MIT License
         
@@ -119,7 +119,7 @@ python -m pip install .
 poolprobe --version
 ```
 
-Expected version output: `poolprobe 1.1.0` with exit code `0`. Installation may
+Expected version output: `poolprobe 1.2.0` with exit code `0`. Installation may
 fetch the build tooling declared in [pyproject.toml](pyproject.toml); installing
 is not the offline check below. No gateway credentials or real config are needed.
 
@@ -135,6 +135,30 @@ to attribute models to providers and to plan config edits from a gateway YAML.
 For that existing adapter, install `python -m pip install '.[cliproxy]'` from the
 checkout; it may fetch PyYAML. It is not needed for the offline first run.
 
+### CLIProxyAPI config attribution
+
+The adapter reads the legacy `openai-compatibility` / `claude-api-key` lists
+and CLIProxyAPI v8 `api-keys.openai-compatibility` / `api-keys.claude` groups.
+For each family, a present v8 list replaces the legacy one, including `[]`.
+Claude key entries inherit group fields and may override models/prefixes.
+The read contract follows [CLIProxyAPI v8.0.45](https://github.com/router-for-me/CLIProxyAPI/tree/ad8bdd3f20bef44f0684b8a6096a216b2c76040d);
+it is not a claim about every installed gateway version or provider family.
+
+Configured model IDs use the normalized one-segment `prefix`: normally both
+`alias` and `prefix/alias`, or only the prefixed ID with `force-model-prefix`.
+As upstream specifies, the unprefixed ID is also retained when it equals the
+prefix. A present `routing.force-model-prefix` overrides the legacy flag,
+including `false`. The catalog remains the union of declared IDs and `/models`;
+a declaration is not proof that the running gateway publishes or can serve it.
+
+**Reading is not editing permission.** v8/mixed, prefixed, empty or ambiguous
+layouts are read-only; the existing text-edit path remains limited to recognized,
+unambiguous, unprefixed legacy declarations. Layouts with no supported section,
+or malformed supported sections, raise an error with `--adapter cliproxy`;
+`auto` falls back to the generic read-only adapter. Other v8 provider families
+are not attributed here. No live
+gateway, removal plan or config-write safety is established by the offline checks.
+
 ## Quickstart
 
 ### First run without a gateway
@@ -200,7 +224,7 @@ every argv echo — reports are meant to be pasted into issues.
 ### What a sweep looks like
 
 ```
-poolprobe 1.1.0  ·  sweep
+poolprobe 1.2.0  ·  sweep
 http://127.0.0.1:8317/v1 · generic adapter (read-only)
 
 5 models  ·  4 alive  ·  0 starved  ·  1 dead  ·  0 prunable  ·  median 5.43s  ·  run #1
@@ -262,7 +286,10 @@ a tool like this, so the guards live in one function, checked in one place:
 
 1. **dead verdict** — `EMPTY`, `ERR`, or `TIMEOUT` (`OK`/`EMPTY-LENGTH` never qualify);
 2. **kind `PERMANENT`** — the upstream does not know this model;
-3. **fail streak** — dead for at least `--streak-min` consecutive *runs* (needs `--state`);
+3. **fail streak** — dead for at least `--streak-min` consecutive *runs* (needs `--state`)
+   **and** for at least `--min-dead-for` seconds of wall time (default 1800), so a
+   burst of runs during one outage counts as one observation (`--min-interval SECONDS`
+   is the matching probe budget: models probed that recently are skipped, not re-asked);
 4. **not protected** — absent from `--protect` (a namespaced alias's base name matches too);
 5. **not a generation modality** — image/video/tts/music models legitimately fail a chat probe.
 
@@ -283,12 +310,16 @@ ran out — alive) · `ERR` (non-2xx) · `TIMEOUT`.
 |------|---------|:---------:|
 | `ALIVE` | Answered. | no |
 | `ALIVE-STARVED` | Answered `EMPTY-LENGTH` — raise `--max-tokens`. | no |
-| `PERMANENT` | Upstream does not know this model (`404`, "does not exist"). | **yes** |
+| `PERMANENT` | Explicit model-error evidence, such as structured `model_not_found`; a bare/route `404` is not enough. | **yes** |
 | `TRANSIENT` | Timeout, `429`, `5xx` — weather. Burst-confirm before believing it. | no |
 | `CREDENTIAL` | `401`/`403`, "invalid api key" — rotate the key, never the model. | no |
 | `MODALITY` | Model lives on another endpoint (`/v1/images/…`), not chat. | no |
 | `UNKNOWN` | None of the above; reported, never acted on. | no |
 
+Structured model-error codes are classified before the display note is truncated
+to 160 characters. An ambiguous `404` stays `UNKNOWN`; a permanent classification
+alone never bypasses the five prune guards or makes an adapter editable.
+
 Three renderers, one report: the terminal table (ANSI, honours `NO_COLOR` /
 `FORCE_COLOR` / `TERM=dumb` / isatty), a machine-readable `--json`, and a
 self-contained `--html` you can attach to a post. They cannot disagree about

{poolprobe-1.1.0 → poolprobe-1.2.0}/README.md

@@ -69,7 +69,7 @@ python -m pip install .
 poolprobe --version
 ```
 
-Expected version output: `poolprobe 1.1.0` with exit code `0`. Installation may
+Expected version output: `poolprobe 1.2.0` with exit code `0`. Installation may
 fetch the build tooling declared in [pyproject.toml](pyproject.toml); installing
 is not the offline check below. No gateway credentials or real config are needed.
 
@@ -85,6 +85,30 @@ to attribute models to providers and to plan config edits from a gateway YAML.
 For that existing adapter, install `python -m pip install '.[cliproxy]'` from the
 checkout; it may fetch PyYAML. It is not needed for the offline first run.
 
+### CLIProxyAPI config attribution
+
+The adapter reads the legacy `openai-compatibility` / `claude-api-key` lists
+and CLIProxyAPI v8 `api-keys.openai-compatibility` / `api-keys.claude` groups.
+For each family, a present v8 list replaces the legacy one, including `[]`.
+Claude key entries inherit group fields and may override models/prefixes.
+The read contract follows [CLIProxyAPI v8.0.45](https://github.com/router-for-me/CLIProxyAPI/tree/ad8bdd3f20bef44f0684b8a6096a216b2c76040d);
+it is not a claim about every installed gateway version or provider family.
+
+Configured model IDs use the normalized one-segment `prefix`: normally both
+`alias` and `prefix/alias`, or only the prefixed ID with `force-model-prefix`.
+As upstream specifies, the unprefixed ID is also retained when it equals the
+prefix. A present `routing.force-model-prefix` overrides the legacy flag,
+including `false`. The catalog remains the union of declared IDs and `/models`;
+a declaration is not proof that the running gateway publishes or can serve it.
+
+**Reading is not editing permission.** v8/mixed, prefixed, empty or ambiguous
+layouts are read-only; the existing text-edit path remains limited to recognized,
+unambiguous, unprefixed legacy declarations. Layouts with no supported section,
+or malformed supported sections, raise an error with `--adapter cliproxy`;
+`auto` falls back to the generic read-only adapter. Other v8 provider families
+are not attributed here. No live
+gateway, removal plan or config-write safety is established by the offline checks.
+
 ## Quickstart
 
 ### First run without a gateway
@@ -150,7 +174,7 @@ every argv echo — reports are meant to be pasted into issues.
 ### What a sweep looks like
 
 ```
-poolprobe 1.1.0  ·  sweep
+poolprobe 1.2.0  ·  sweep
 http://127.0.0.1:8317/v1 · generic adapter (read-only)
 
 5 models  ·  4 alive  ·  0 starved  ·  1 dead  ·  0 prunable  ·  median 5.43s  ·  run #1
@@ -212,7 +236,10 @@ a tool like this, so the guards live in one function, checked in one place:
 
 1. **dead verdict** — `EMPTY`, `ERR`, or `TIMEOUT` (`OK`/`EMPTY-LENGTH` never qualify);
 2. **kind `PERMANENT`** — the upstream does not know this model;
-3. **fail streak** — dead for at least `--streak-min` consecutive *runs* (needs `--state`);
+3. **fail streak** — dead for at least `--streak-min` consecutive *runs* (needs `--state`)
+   **and** for at least `--min-dead-for` seconds of wall time (default 1800), so a
+   burst of runs during one outage counts as one observation (`--min-interval SECONDS`
+   is the matching probe budget: models probed that recently are skipped, not re-asked);
 4. **not protected** — absent from `--protect` (a namespaced alias's base name matches too);
 5. **not a generation modality** — image/video/tts/music models legitimately fail a chat probe.
 
@@ -233,12 +260,16 @@ ran out — alive) · `ERR` (non-2xx) · `TIMEOUT`.
 |------|---------|:---------:|
 | `ALIVE` | Answered. | no |
 | `ALIVE-STARVED` | Answered `EMPTY-LENGTH` — raise `--max-tokens`. | no |
-| `PERMANENT` | Upstream does not know this model (`404`, "does not exist"). | **yes** |
+| `PERMANENT` | Explicit model-error evidence, such as structured `model_not_found`; a bare/route `404` is not enough. | **yes** |
 | `TRANSIENT` | Timeout, `429`, `5xx` — weather. Burst-confirm before believing it. | no |
 | `CREDENTIAL` | `401`/`403`, "invalid api key" — rotate the key, never the model. | no |
 | `MODALITY` | Model lives on another endpoint (`/v1/images/…`), not chat. | no |
 | `UNKNOWN` | None of the above; reported, never acted on. | no |
 
+Structured model-error codes are classified before the display note is truncated
+to 160 characters. An ambiguous `404` stays `UNKNOWN`; a permanent classification
+alone never bypasses the five prune guards or makes an adapter editable.
+
 Three renderers, one report: the terminal table (ANSI, honours `NO_COLOR` /
 `FORCE_COLOR` / `TERM=dumb` / isatty), a machine-readable `--json`, and a
 self-contained `--html` you can attach to a post. They cannot disagree about

{poolprobe-1.1.0 → poolprobe-1.2.0}/poolprobe/__init__.py

@@ -37,7 +37,7 @@ CLI use::
 
 from __future__ import annotations
 
-__version__ = "1.1.0"
+__version__ = "1.2.0"
 
 # Submodules first: probe, bench and adapter all resolve ``taxonomy`` through
 # the package attribute at import time, and a partially initialised package must

{poolprobe-1.1.0 → poolprobe-1.2.0}/poolprobe/adapter.py

@@ -8,7 +8,8 @@ Two adapters, one interface:
   against OpenRouter, one-api, LiteLLM, vLLM, cliproxy or a bare vendor endpoint.
 * :class:`CliproxyAdapter` — adds attribution (which upstream provider serves
   this alias, is it disabled, what is the real model name) by reading the
-  gateway's own YAML config, and can plan a removal.
+  gateway's own YAML config. Only unambiguous, unprefixed legacy layouts can
+  use the existing removal planner; v8 attribution stays read-only.
 
 The split between *reading* and *removing* is deliberate:
 
@@ -31,9 +32,16 @@ from pathlib import Path
 
 from .probe import Gateway, GatewayError, list_models
 
-#: Top-level YAML keys whose list items are *provider blocks* holding models.
-#: Kept as a tuple so a different gateway layout is a one-line extension.
+#: Supported legacy sections and their v8 ``api-keys`` family names.
 PROVIDER_SECTIONS: tuple[str, ...] = ("openai-compatibility", "claude-api-key")
+V8_FAMILIES: dict[str, str] = {
+    "openai-compatibility": "openai-compatibility",
+    "claude-api-key": "claude",
+}
+_SHARED_KEY_FIELDS = frozenset({
+    "priority", "prefix", "proxy-url", "headers", "models", "excluded-models",
+    "disable-cooling", "request-retry", "request-scoped-errors",
+})
 
 #: A list item is a model entry only when its nearest lower-indented ancestor
 #: line is one of these keys. Provider blocks also carry ``name:`` and would
@@ -72,6 +80,66 @@ def effective_alias(name: str, alias: str = "") -> str:
     return (alias or name or "").strip()
 
 
+def _v8_blocks(groups: object, family: str) -> list[dict]:
+    """Expand the two supported v8 families, without interpreting credentials."""
+    if not isinstance(groups, list):
+        raise ConfigError(f"api-keys.{family} must be a list")
+    blocks = []
+    for group in groups:
+        if not isinstance(group, dict) or not isinstance(group.get("keys"), list):
+            raise ConfigError(f"api-keys.{family} groups need a keys list")
+        keys = group["keys"]
+        if any(not isinstance(key, dict) for key in keys):
+            raise ConfigError(f"api-keys.{family} keys must be mappings")
+        if family == "openai-compatibility":
+            # Upstream keeps the group's models/prefix on the provider block;
+            # individual keys do not override them for this family.
+            blocks.append({key: value for key, value in group.items() if key != "keys"})
+            continue
+        allowed = _SHARED_KEY_FIELDS | {"name", "base-url", "keys"}
+        if set(group) - allowed:
+            raise ConfigError(f"unsupported api-keys.{family} group fields")
+        for key in keys:
+            if "base-url" in key:
+                raise ConfigError(f"api-keys.{family}: base-url belongs to the group")
+            block = {
+                name: value for name, value in group.items()
+                if name == "base-url" or name in _SHARED_KEY_FIELDS
+            }
+            block.update({
+                name: value for name, value in key.items()
+                if name not in {"auth_index", "auth-index"} and value is not None
+            })
+            blocks.append(block)
+    return blocks
+
+
+def _has_models(value: object) -> bool:
+    """An unhandled models subtree must not be eligible for text surgery."""
+    pending = [value]
+    seen = set()
+    while pending:
+        item = pending.pop()
+        if id(item) in seen:
+            continue  # YAML aliases may share or even cycle containers.
+        seen.add(id(item))
+        if isinstance(item, dict):
+            if "models" in item:
+                return True
+            pending.extend(item.values())
+        elif isinstance(item, list):
+            pending.extend(item)
+    return False
+
+
+def _prefix(value: object) -> str:
+    if not isinstance(value, str):
+        raise ConfigError("model prefix must be a string")
+    prefix = value.strip().strip("/")
+    # CLIProxyAPI normalizes an empty or multi-segment prefix away.
+    return "" if not prefix or "/" in prefix else prefix.strip()
+
+
 # --- attribution ------------------------------------------------------------
 
 
@@ -180,7 +248,7 @@ class CliproxyAdapter(Adapter):
     """
 
     name = "cliproxy"
-    removable = True
+    removable = False
 
     def __init__(self, gateway: Gateway, config_path: str | Path):
         self.gateway = gateway
@@ -192,51 +260,97 @@ class CliproxyAdapter(Adapter):
     # -- reading ---------------------------------------------------------
 
     def _parse(self) -> dict[str, Attribution]:
+        self.removable = False
         doc = _load_yaml(self.config_path)
         if not isinstance(doc, dict):
             raise ConfigError(f"{self.config_path}: expected a mapping at top level")
+        if "config-version" in doc and (
+            type(doc["config-version"]) is not int or doc["config-version"] != 8
+        ):
+            raise ConfigError("unsupported config-version (expected 8)")
+        families = doc.get("api-keys")
+        if families is not None and not isinstance(families, (dict, list)):
+            raise ConfigError("api-keys must be a client-key list or a v8 provider mapping")
+        v8 = families if isinstance(families, dict) else {}
+        force = doc.get("force-model-prefix", False)
+        routing = doc.get("routing", {})
+        if not isinstance(routing, dict):
+            raise ConfigError("routing must be a mapping")
+        if "force-model-prefix" in routing:
+            force = routing["force-model-prefix"]
+        if not isinstance(force, bool):
+            raise ConfigError("force-model-prefix must be a boolean")
 
         attr: dict[str, Attribution] = {}
         providers = 0
+        recognized = False
+        editable = (
+            "config-version" not in doc and not isinstance(families, dict) and not force
+        )
+        for key, value in doc.items():
+            if key not in PROVIDER_SECTIONS and _has_models(value):
+                editable = False
         for section in PROVIDER_SECTIONS:
-            blocks = doc.get(section)
-            if not isinstance(blocks, list):
+            family = V8_FAMILIES[section]
+            if family in v8:
+                # Presence, not truthiness: [] replaces a legacy section too.
+                blocks = _v8_blocks(v8[family], family)
+                editable = False
+            elif section in doc:
+                blocks = doc[section]
+            else:
                 continue
+            recognized = True
+            if not isinstance(blocks, list):
+                raise ConfigError(f"{section} must be a list")
             for block in blocks:
                 if not isinstance(block, dict):
-                    continue
+                    raise ConfigError(f"{section} entries must be mappings")
+                if any(_has_models(value) for key, value in block.items() if key != "models"):
+                    editable = False
                 providers += 1
                 provider = str(block.get("name") or section)
                 base_url = str(block.get("base-url") or block.get("base_url") or "")
                 disabled = bool(block.get("disabled"))
-                entries = block.get("models")
+                raw_prefix = block.get("prefix", "")
+                prefix = _prefix(raw_prefix)
+                if raw_prefix:
+                    editable = False
+                entries = block.get("models", [])
                 if not isinstance(entries, list):
-                    continue
+                    raise ConfigError(f"{section}.models must be a list")
                 for entry in entries:
                     if isinstance(entry, str):
                         name, alias = entry, ""
                     elif isinstance(entry, dict):
-                        name = str(entry.get("name") or "")
-                        alias = str(entry.get("alias") or "")
+                        name = entry.get("name") or ""
+                        alias = entry.get("alias") or ""
+                        if not isinstance(name, str) or not isinstance(alias, str):
+                            raise ConfigError("model name and alias must be strings")
                     else:
-                        continue
+                        raise ConfigError("models entries must be names or mappings")
                     key = effective_alias(name, alias)
                     if not key:
+                        editable = False
                         continue
-                    # First declaration wins: a duplicate alias later in the
-                    # file is a config bug, and silently overwriting attribution
-                    # would hide it from the report.
-                    attr.setdefault(
-                        key,
-                        Attribution(
+                    routes = [key] if not prefix or not force or prefix == key else []
+                    if prefix:
+                        routes.append(f"{prefix}/{key}")
+                    for route in routes:
+                        # Keep the first hint, but ambiguity cannot enable edits.
+                        if route in attr:
+                            editable = False
+                        attr.setdefault(route, Attribution(
                             provider=provider,
                             upstream=name.strip(),
                             base_url=base_url,
                             disabled=disabled,
                             source="config",
-                        ),
-                    )
+                        ))
+        if not recognized:
+            raise ConfigError("no supported CLIProxyAPI provider layout")
         self._providers = providers
+        self.removable = editable and bool(attr)
         return attr
 
     def attribution_map(self) -> dict[str, Attribution]:
@@ -279,21 +393,21 @@ class CliproxyAdapter(Adapter):
         # we never measured. An empty list (gateway down) is a real 0 and shows.
         if self._live is not None:
             head += f", {len(self._live)} published"
-        return head + ")"
+        return head + (")" if self.removable else ", read-only)")
 
     # -- removal ---------------------------------------------------------
 
     def plan_removal(self, aliases: Sequence[str]) -> EditPlan:
         """Cut model entries out of the config text, preserving everything else.
 
-        Removal is driven by the text scan, which locates any ``- name:/alias:``
-        item under a ``models:`` key in the file. Attribution is narrower — it
-        only reads :data:`PROVIDER_SECTIONS` — so an alias from a section this
-        adapter does not know can still be cut, and the diff shown to the
-        operator is the only authority. Aliases that appear nowhere in the text
-        (published by a running process, or already removed) come back in
-        ``not_found`` and are left alone.
+        The existing text scan is limited to unambiguous, unprefixed legacy
+        layouts. v8, mixed, prefixed and otherwise ambiguous attribution is
+        read-only: successful parsing is not proof that this scanner can edit it.
+        Aliases absent from the text come back in ``not_found`` unchanged.
         """
+        self.attribution_map()
+        if not self.removable:
+            raise ConfigError("this config layout supports attribution only (read-only)")
         wanted = [a for a in dict.fromkeys(aliases) if a]
         before = read_text(self.config_path)
         entries = {entry.alias: entry for entry in iter_model_entries(before)}

{poolprobe-1.1.0 → poolprobe-1.2.0}/poolprobe/bench.py

@@ -184,19 +184,24 @@ def _sample(
 
     result = BenchResult(model=model, provider=provider)
     started = time.monotonic()
+    error_body = None
     try:
         with urllib.request.urlopen(request, timeout=timeout) as response:
             result.status = getattr(response, "status", 200) or 200
             if stream:
                 _read_stream(response, result, started)
             else:
-                _read_json(response.read(), result)
+                error_body = _read_json(response.read(), result)
     except urllib.error.HTTPError as exc:
         result.status = exc.code
         try:
             detail = exc.read().decode("utf-8", "replace")
         except Exception:
             detail = str(exc.reason or exc)
+        try:
+            error_body = json.loads(detail)
+        except (ValueError, TypeError):
+            error_body = None
         result.error = _flatten(detail)
         result.verdict = taxonomy.ERR
     except Exception as exc:
@@ -215,12 +220,14 @@ def _sample(
         if result.tokens and result.latency_s > 0
         else 0.0
     )
-    result.kind = taxonomy.classify_death(result.verdict, result.error, result.status)
+    result.kind = taxonomy.classify_death(
+        result.verdict, result.error, result.status, body=error_body
+    )
     result.ok = result.verdict not in taxonomy.DEAD
     return result
 
 
-def _read_json(raw: bytes, result: BenchResult) -> None:
+def _read_json(raw: bytes, result: BenchResult) -> object:
     """Non-streaming path: tokens come from ``usage`` or not at all."""
     try:
         body = json.loads(raw.decode("utf-8", "replace"))
@@ -244,6 +251,7 @@ def _read_json(raw: bytes, result: BenchResult) -> None:
         # No usage block. Falling back to len(content) would report characters
         # as tokens, so the honest answer is "not measured".
         result.token_source = "none"
+    return body
 
 
 def _read_stream(response: object, result: BenchResult, started: float) -> None: