poolprobe 1.0.0__tar.gz → 1.2.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (23) hide show
  1. {poolprobe-1.0.0 → poolprobe-1.2.0}/PKG-INFO +109 -16
  2. {poolprobe-1.0.0 → poolprobe-1.2.0}/README.md +108 -15
  3. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe/__init__.py +1 -1
  4. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe/adapter.py +142 -28
  5. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe/bench.py +11 -3
  6. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe/cli.py +117 -8
  7. poolprobe-1.2.0/poolprobe/compatcheck.py +270 -0
  8. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe/probe.py +6 -1
  9. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe/state.py +27 -1
  10. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe/taxonomy.py +117 -8
  11. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe.egg-info/PKG-INFO +109 -16
  12. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe.egg-info/SOURCES.txt +1 -0
  13. {poolprobe-1.0.0 → poolprobe-1.2.0}/pyproject.toml +1 -1
  14. {poolprobe-1.0.0 → poolprobe-1.2.0}/LICENSE +0 -0
  15. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe/__main__.py +0 -0
  16. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe/chains.py +0 -0
  17. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe/report.py +0 -0
  18. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe/safewrite.py +0 -0
  19. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe.egg-info/dependency_links.txt +0 -0
  20. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe.egg-info/entry_points.txt +0 -0
  21. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe.egg-info/requires.txt +0 -0
  22. {poolprobe-1.0.0 → poolprobe-1.2.0}/poolprobe.egg-info/top_level.txt +0 -0
  23. {poolprobe-1.0.0 → poolprobe-1.2.0}/setup.cfg +0 -0
@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: poolprobe
3
- Version: 1.0.0
3
+ Version: 1.2.0
4
4
  Summary: A doctor for OpenAI-compatible LLM gateways: probe the pool, classify every failure, and only then talk about pruning.
5
5
  License: MIT License
6
6
 
@@ -48,20 +48,31 @@ Provides-Extra: cliproxy
48
48
  Requires-Dist: PyYAML>=6.0; extra == "cliproxy"
49
49
  Dynamic: license-file
50
50
 
51
- <div align="center">
51
+ ![poolprobe — Probe first. Classify before pruning.](assets/brand/readme-cover.svg)
52
52
 
53
53
  # poolprobe
54
54
 
55
- **A doctor for OpenAI-compatible LLM gateways.**
56
- Probe the pool, classify every failure, and only *then* talk about pruning.
55
+ **Probe first. Classify before pruning.**
56
+
57
+ Inspect OpenAI-compatible model pools, classify failures and produce terminal, JSON or HTML reports. Keep transient and credential failures separate from permanent-model prune candidates.
58
+
59
+ [Install](#install) · [Quickstart](#quickstart) · [Commands](#commands) · [Prune guards](#the-five-prune-guards) · [Contributing](CONTRIBUTING.md)
60
+
61
+ **Scope:** Live sweeps and benchmarks send model requests and may incur provider charges. Pruning requires all five guards; config writes require `--apply`. Offline selfcheck does not establish live gateway availability.
62
+
63
+ **Checks:** run `python -m poolprobe selfcheck` before pushing.
64
+ This is the repository's offline contract; it needs no live service or credentials.
65
+
66
+ <details>
67
+ <summary>Package and license badges</summary>
57
68
 
58
69
  [![python](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/downloads/)
59
70
  [![dependencies](https://img.shields.io/badge/dependencies-stdlib%20only-brightgreen.svg)](#install)
60
71
  [![license](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
61
72
 
62
- </div>
73
+ </details>
63
74
 
64
- ---
75
+ ## Why it exists
65
76
 
66
77
  A pool of routed models rots quietly. An upstream drops a model, a key hits an
67
78
  account tier, a reasoning model starts spending its whole token budget on
@@ -96,23 +107,98 @@ Everything in the tool obeys these. They are the reason it exists.
96
107
 
97
108
  ## Install
98
109
 
99
- ```bash
100
- pip install poolprobe # stdlib engine, generic gateway lane
101
- pip install 'poolprobe[cliproxy]' # + PyYAML, to read a cliproxy-style config
110
+ Use **Python 3.10+**. For a first run from a checkout, use a virtual environment
111
+ (the activation command below is for a POSIX shell):
112
+
113
+ ```sh
114
+ git clone https://gitlab.com/ameobius-ai/poolprobe.git
115
+ cd poolprobe
116
+ python3 -m venv .venv
117
+ . .venv/bin/activate
118
+ python -m pip install .
119
+ poolprobe --version
102
120
  ```
103
121
 
104
- No PyPI release yet? Install straight from the checkout:
122
+ Expected version output: `poolprobe 1.2.0` with exit code `0`. Installation may
123
+ fetch the build tooling declared in [pyproject.toml](pyproject.toml); installing
124
+ is not the offline check below. No gateway credentials or real config are needed.
125
+
126
+ When a release is available on PyPI, the package-install alternatives are:
105
127
 
106
128
  ```bash
107
- git clone https://gitlab.com/ameobius-ai/poolprobe
108
- cd poolprobe && pip install '.[cliproxy]'
129
+ pip install poolprobe # stdlib engine, generic gateway lane
130
+ pip install 'poolprobe[cliproxy]' # + PyYAML, to read a cliproxy-style config
109
131
  ```
110
132
 
111
133
  The engine has **zero runtime dependencies**. PyYAML is optional and only needed
112
134
  to attribute models to providers and to plan config edits from a gateway YAML.
135
+ For that existing adapter, install `python -m pip install '.[cliproxy]'` from the
136
+ checkout; it may fetch PyYAML. It is not needed for the offline first run.
137
+
138
+ ### CLIProxyAPI config attribution
139
+
140
+ The adapter reads the legacy `openai-compatibility` / `claude-api-key` lists
141
+ and CLIProxyAPI v8 `api-keys.openai-compatibility` / `api-keys.claude` groups.
142
+ For each family, a present v8 list replaces the legacy one, including `[]`.
143
+ Claude key entries inherit group fields and may override models/prefixes.
144
+ The read contract follows [CLIProxyAPI v8.0.45](https://github.com/router-for-me/CLIProxyAPI/tree/ad8bdd3f20bef44f0684b8a6096a216b2c76040d);
145
+ it is not a claim about every installed gateway version or provider family.
146
+
147
+ Configured model IDs use the normalized one-segment `prefix`: normally both
148
+ `alias` and `prefix/alias`, or only the prefixed ID with `force-model-prefix`.
149
+ As upstream specifies, the unprefixed ID is also retained when it equals the
150
+ prefix. A present `routing.force-model-prefix` overrides the legacy flag,
151
+ including `false`. The catalog remains the union of declared IDs and `/models`;
152
+ a declaration is not proof that the running gateway publishes or can serve it.
153
+
154
+ **Reading is not editing permission.** v8/mixed, prefixed, empty or ambiguous
155
+ layouts are read-only; the existing text-edit path remains limited to recognized,
156
+ unambiguous, unprefixed legacy declarations. Layouts with no supported section,
157
+ or malformed supported sections, raise an error with `--adapter cliproxy`;
158
+ `auto` falls back to the generic read-only adapter. Other v8 provider families
159
+ are not attributed here. No live
160
+ gateway, removal plan or config-write safety is established by the offline checks.
113
161
 
114
162
  ## Quickstart
115
163
 
164
+ ### First run without a gateway
165
+
166
+ With the virtual environment above active, run the installed entry point:
167
+
168
+ ```sh
169
+ poolprobe selfcheck
170
+ ```
171
+
172
+ Expected output:
173
+
174
+ ```text
175
+ poolprobe selfcheck ok
176
+ ```
177
+
178
+ A pass exits `0`; a failed selfcheck exits `1`. Keep Python assertions enabled
179
+ (do not use `python -O`). The equivalent checkout command is
180
+ `python -m poolprobe selfcheck`.
181
+
182
+ This runs built-in synthetic checks for chain rules, prune guards, argv
183
+ redaction, terminal/JSON/HTML rendering and atomic writes. It needs no API key,
184
+ gateway, user config or saved state. The atomic-write check creates and cleans
185
+ its own temporary directory; no report or config file should be kept from this
186
+ run. A pass does **not** establish live gateway/model availability or validate a
187
+ real YAML config. No `cliproxy` extra is needed for these checks.
188
+
189
+ Next safe step: `poolprobe --help` lists commands without sending requests.
190
+ See [the existing contribution checks](CONTRIBUTING.md#run-the-existing-contract)
191
+ for the module/installed-entry and optional-adapter check sequence.
192
+
193
+ ### Live gateway examples — not offline checks
194
+
195
+ The examples below are a separate operational path, not the first-run check.
196
+ `ping` contacts the gateway even though it sends no chat request; sweeps,
197
+ benchmarks and **pruning without `--apply`** can send model requests and incur
198
+ charges. `--apply` guards config changes, not network access. Review
199
+ [commands](#commands) and [all five prune guards](#the-five-prune-guards) before
200
+ using an authorized gateway. Do not run these examples as an offline test.
201
+
116
202
  ```bash
117
203
  # 1. Is the gateway even up, and what does it publish? (no chat request sent)
118
204
  poolprobe ping -u http://127.0.0.1:8317/v1 -k "$KEY"
@@ -123,7 +209,7 @@ poolprobe sweep -u http://127.0.0.1:8317/v1 -k "$KEY" --html out.html
123
209
  # 3. Give it memory: fail streaks across runs are what make `prune` safe
124
210
  poolprobe sweep -u ... -k "$KEY" --state ~/.cache/poolprobe.json
125
211
 
126
- # 4. Only list what passed all five guards — nothing on disk changes without --apply
212
+ # 4. Probe and plan candidates; only config writes require --apply
127
213
  poolprobe prune -u ... -k "$KEY" --state ~/.cache/poolprobe.json \
128
214
  --adapter cliproxy --config ~/.config/gateway.yaml
129
215
 
@@ -138,7 +224,7 @@ every argv echo — reports are meant to be pasted into issues.
138
224
  ### What a sweep looks like
139
225
 
140
226
  ```
141
- poolprobe 1.0.0 · sweep
227
+ poolprobe 1.2.0 · sweep
142
228
  http://127.0.0.1:8317/v1 · generic adapter (read-only)
143
229
 
144
230
  5 models · 4 alive · 0 starved · 1 dead · 0 prunable · median 5.43s · run #1
@@ -200,7 +286,10 @@ a tool like this, so the guards live in one function, checked in one place:
200
286
 
201
287
  1. **dead verdict** — `EMPTY`, `ERR`, or `TIMEOUT` (`OK`/`EMPTY-LENGTH` never qualify);
202
288
  2. **kind `PERMANENT`** — the upstream does not know this model;
203
- 3. **fail streak** — dead for at least `--streak-min` consecutive *runs* (needs `--state`);
289
+ 3. **fail streak** — dead for at least `--streak-min` consecutive *runs* (needs `--state`)
290
+ **and** for at least `--min-dead-for` seconds of wall time (default 1800), so a
291
+ burst of runs during one outage counts as one observation (`--min-interval SECONDS`
292
+ is the matching probe budget: models probed that recently are skipped, not re-asked);
204
293
  4. **not protected** — absent from `--protect` (a namespaced alias's base name matches too);
205
294
  5. **not a generation modality** — image/video/tts/music models legitimately fail a chat probe.
206
295
 
@@ -221,12 +310,16 @@ ran out — alive) · `ERR` (non-2xx) · `TIMEOUT`.
221
310
  |------|---------|:---------:|
222
311
  | `ALIVE` | Answered. | no |
223
312
  | `ALIVE-STARVED` | Answered `EMPTY-LENGTH` — raise `--max-tokens`. | no |
224
- | `PERMANENT` | Upstream does not know this model (`404`, "does not exist"). | **yes** |
313
+ | `PERMANENT` | Explicit model-error evidence, such as structured `model_not_found`; a bare/route `404` is not enough. | **yes** |
225
314
  | `TRANSIENT` | Timeout, `429`, `5xx` — weather. Burst-confirm before believing it. | no |
226
315
  | `CREDENTIAL` | `401`/`403`, "invalid api key" — rotate the key, never the model. | no |
227
316
  | `MODALITY` | Model lives on another endpoint (`/v1/images/…`), not chat. | no |
228
317
  | `UNKNOWN` | None of the above; reported, never acted on. | no |
229
318
 
319
+ Structured model-error codes are classified before the display note is truncated
320
+ to 160 characters. An ambiguous `404` stays `UNKNOWN`; a permanent classification
321
+ alone never bypasses the five prune guards or makes an adapter editable.
322
+
230
323
  Three renderers, one report: the terminal table (ANSI, honours `NO_COLOR` /
231
324
  `FORCE_COLOR` / `TERM=dumb` / isatty), a machine-readable `--json`, and a
232
325
  self-contained `--html` you can attach to a post. They cannot disagree about
@@ -1,17 +1,28 @@
1
- <div align="center">
1
+ ![poolprobe — Probe first. Classify before pruning.](assets/brand/readme-cover.svg)
2
2
 
3
3
  # poolprobe
4
4
 
5
- **A doctor for OpenAI-compatible LLM gateways.**
6
- Probe the pool, classify every failure, and only *then* talk about pruning.
5
+ **Probe first. Classify before pruning.**
6
+
7
+ Inspect OpenAI-compatible model pools, classify failures and produce terminal, JSON or HTML reports. Keep transient and credential failures separate from permanent-model prune candidates.
8
+
9
+ [Install](#install) · [Quickstart](#quickstart) · [Commands](#commands) · [Prune guards](#the-five-prune-guards) · [Contributing](CONTRIBUTING.md)
10
+
11
+ **Scope:** Live sweeps and benchmarks send model requests and may incur provider charges. Pruning requires all five guards; config writes require `--apply`. Offline selfcheck does not establish live gateway availability.
12
+
13
+ **Checks:** run `python -m poolprobe selfcheck` before pushing.
14
+ This is the repository's offline contract; it needs no live service or credentials.
15
+
16
+ <details>
17
+ <summary>Package and license badges</summary>
7
18
 
8
19
  [![python](https://img.shields.io/badge/python-3.10%2B-blue.svg)](https://www.python.org/downloads/)
9
20
  [![dependencies](https://img.shields.io/badge/dependencies-stdlib%20only-brightgreen.svg)](#install)
10
21
  [![license](https://img.shields.io/badge/license-MIT-blue.svg)](LICENSE)
11
22
 
12
- </div>
23
+ </details>
13
24
 
14
- ---
25
+ ## Why it exists
15
26
 
16
27
  A pool of routed models rots quietly. An upstream drops a model, a key hits an
17
28
  account tier, a reasoning model starts spending its whole token budget on
@@ -46,23 +57,98 @@ Everything in the tool obeys these. They are the reason it exists.
46
57
 
47
58
  ## Install
48
59
 
49
- ```bash
50
- pip install poolprobe # stdlib engine, generic gateway lane
51
- pip install 'poolprobe[cliproxy]' # + PyYAML, to read a cliproxy-style config
60
+ Use **Python 3.10+**. For a first run from a checkout, use a virtual environment
61
+ (the activation command below is for a POSIX shell):
62
+
63
+ ```sh
64
+ git clone https://gitlab.com/ameobius-ai/poolprobe.git
65
+ cd poolprobe
66
+ python3 -m venv .venv
67
+ . .venv/bin/activate
68
+ python -m pip install .
69
+ poolprobe --version
52
70
  ```
53
71
 
54
- No PyPI release yet? Install straight from the checkout:
72
+ Expected version output: `poolprobe 1.2.0` with exit code `0`. Installation may
73
+ fetch the build tooling declared in [pyproject.toml](pyproject.toml); installing
74
+ is not the offline check below. No gateway credentials or real config are needed.
75
+
76
+ When a release is available on PyPI, the package-install alternatives are:
55
77
 
56
78
  ```bash
57
- git clone https://gitlab.com/ameobius-ai/poolprobe
58
- cd poolprobe && pip install '.[cliproxy]'
79
+ pip install poolprobe # stdlib engine, generic gateway lane
80
+ pip install 'poolprobe[cliproxy]' # + PyYAML, to read a cliproxy-style config
59
81
  ```
60
82
 
61
83
  The engine has **zero runtime dependencies**. PyYAML is optional and only needed
62
84
  to attribute models to providers and to plan config edits from a gateway YAML.
85
+ For that existing adapter, install `python -m pip install '.[cliproxy]'` from the
86
+ checkout; it may fetch PyYAML. It is not needed for the offline first run.
87
+
88
+ ### CLIProxyAPI config attribution
89
+
90
+ The adapter reads the legacy `openai-compatibility` / `claude-api-key` lists
91
+ and CLIProxyAPI v8 `api-keys.openai-compatibility` / `api-keys.claude` groups.
92
+ For each family, a present v8 list replaces the legacy one, including `[]`.
93
+ Claude key entries inherit group fields and may override models/prefixes.
94
+ The read contract follows [CLIProxyAPI v8.0.45](https://github.com/router-for-me/CLIProxyAPI/tree/ad8bdd3f20bef44f0684b8a6096a216b2c76040d);
95
+ it is not a claim about every installed gateway version or provider family.
96
+
97
+ Configured model IDs use the normalized one-segment `prefix`: normally both
98
+ `alias` and `prefix/alias`, or only the prefixed ID with `force-model-prefix`.
99
+ As upstream specifies, the unprefixed ID is also retained when it equals the
100
+ prefix. A present `routing.force-model-prefix` overrides the legacy flag,
101
+ including `false`. The catalog remains the union of declared IDs and `/models`;
102
+ a declaration is not proof that the running gateway publishes or can serve it.
103
+
104
+ **Reading is not editing permission.** v8/mixed, prefixed, empty or ambiguous
105
+ layouts are read-only; the existing text-edit path remains limited to recognized,
106
+ unambiguous, unprefixed legacy declarations. Layouts with no supported section,
107
+ or malformed supported sections, raise an error with `--adapter cliproxy`;
108
+ `auto` falls back to the generic read-only adapter. Other v8 provider families
109
+ are not attributed here. No live
110
+ gateway, removal plan or config-write safety is established by the offline checks.
63
111
 
64
112
  ## Quickstart
65
113
 
114
+ ### First run without a gateway
115
+
116
+ With the virtual environment above active, run the installed entry point:
117
+
118
+ ```sh
119
+ poolprobe selfcheck
120
+ ```
121
+
122
+ Expected output:
123
+
124
+ ```text
125
+ poolprobe selfcheck ok
126
+ ```
127
+
128
+ A pass exits `0`; a failed selfcheck exits `1`. Keep Python assertions enabled
129
+ (do not use `python -O`). The equivalent checkout command is
130
+ `python -m poolprobe selfcheck`.
131
+
132
+ This runs built-in synthetic checks for chain rules, prune guards, argv
133
+ redaction, terminal/JSON/HTML rendering and atomic writes. It needs no API key,
134
+ gateway, user config or saved state. The atomic-write check creates and cleans
135
+ its own temporary directory; no report or config file should be kept from this
136
+ run. A pass does **not** establish live gateway/model availability or validate a
137
+ real YAML config. No `cliproxy` extra is needed for these checks.
138
+
139
+ Next safe step: `poolprobe --help` lists commands without sending requests.
140
+ See [the existing contribution checks](CONTRIBUTING.md#run-the-existing-contract)
141
+ for the module/installed-entry and optional-adapter check sequence.
142
+
143
+ ### Live gateway examples — not offline checks
144
+
145
+ The examples below are a separate operational path, not the first-run check.
146
+ `ping` contacts the gateway even though it sends no chat request; sweeps,
147
+ benchmarks and **pruning without `--apply`** can send model requests and incur
148
+ charges. `--apply` guards config changes, not network access. Review
149
+ [commands](#commands) and [all five prune guards](#the-five-prune-guards) before
150
+ using an authorized gateway. Do not run these examples as an offline test.
151
+
66
152
  ```bash
67
153
  # 1. Is the gateway even up, and what does it publish? (no chat request sent)
68
154
  poolprobe ping -u http://127.0.0.1:8317/v1 -k "$KEY"
@@ -73,7 +159,7 @@ poolprobe sweep -u http://127.0.0.1:8317/v1 -k "$KEY" --html out.html
73
159
  # 3. Give it memory: fail streaks across runs are what make `prune` safe
74
160
  poolprobe sweep -u ... -k "$KEY" --state ~/.cache/poolprobe.json
75
161
 
76
- # 4. Only list what passed all five guards — nothing on disk changes without --apply
162
+ # 4. Probe and plan candidates; only config writes require --apply
77
163
  poolprobe prune -u ... -k "$KEY" --state ~/.cache/poolprobe.json \
78
164
  --adapter cliproxy --config ~/.config/gateway.yaml
79
165
 
@@ -88,7 +174,7 @@ every argv echo — reports are meant to be pasted into issues.
88
174
  ### What a sweep looks like
89
175
 
90
176
  ```
91
- poolprobe 1.0.0 · sweep
177
+ poolprobe 1.2.0 · sweep
92
178
  http://127.0.0.1:8317/v1 · generic adapter (read-only)
93
179
 
94
180
  5 models · 4 alive · 0 starved · 1 dead · 0 prunable · median 5.43s · run #1
@@ -150,7 +236,10 @@ a tool like this, so the guards live in one function, checked in one place:
150
236
 
151
237
  1. **dead verdict** — `EMPTY`, `ERR`, or `TIMEOUT` (`OK`/`EMPTY-LENGTH` never qualify);
152
238
  2. **kind `PERMANENT`** — the upstream does not know this model;
153
- 3. **fail streak** — dead for at least `--streak-min` consecutive *runs* (needs `--state`);
239
+ 3. **fail streak** — dead for at least `--streak-min` consecutive *runs* (needs `--state`)
240
+ **and** for at least `--min-dead-for` seconds of wall time (default 1800), so a
241
+ burst of runs during one outage counts as one observation (`--min-interval SECONDS`
242
+ is the matching probe budget: models probed that recently are skipped, not re-asked);
154
243
  4. **not protected** — absent from `--protect` (a namespaced alias's base name matches too);
155
244
  5. **not a generation modality** — image/video/tts/music models legitimately fail a chat probe.
156
245
 
@@ -171,12 +260,16 @@ ran out — alive) · `ERR` (non-2xx) · `TIMEOUT`.
171
260
  |------|---------|:---------:|
172
261
  | `ALIVE` | Answered. | no |
173
262
  | `ALIVE-STARVED` | Answered `EMPTY-LENGTH` — raise `--max-tokens`. | no |
174
- | `PERMANENT` | Upstream does not know this model (`404`, "does not exist"). | **yes** |
263
+ | `PERMANENT` | Explicit model-error evidence, such as structured `model_not_found`; a bare/route `404` is not enough. | **yes** |
175
264
  | `TRANSIENT` | Timeout, `429`, `5xx` — weather. Burst-confirm before believing it. | no |
176
265
  | `CREDENTIAL` | `401`/`403`, "invalid api key" — rotate the key, never the model. | no |
177
266
  | `MODALITY` | Model lives on another endpoint (`/v1/images/…`), not chat. | no |
178
267
  | `UNKNOWN` | None of the above; reported, never acted on. | no |
179
268
 
269
+ Structured model-error codes are classified before the display note is truncated
270
+ to 160 characters. An ambiguous `404` stays `UNKNOWN`; a permanent classification
271
+ alone never bypasses the five prune guards or makes an adapter editable.
272
+
180
273
  Three renderers, one report: the terminal table (ANSI, honours `NO_COLOR` /
181
274
  `FORCE_COLOR` / `TERM=dumb` / isatty), a machine-readable `--json`, and a
182
275
  self-contained `--html` you can attach to a post. They cannot disagree about
@@ -37,7 +37,7 @@ CLI use::
37
37
 
38
38
  from __future__ import annotations
39
39
 
40
- __version__ = "1.0.0"
40
+ __version__ = "1.2.0"
41
41
 
42
42
  # Submodules first: probe, bench and adapter all resolve ``taxonomy`` through
43
43
  # the package attribute at import time, and a partially initialised package must
@@ -8,7 +8,8 @@ Two adapters, one interface:
8
8
  against OpenRouter, one-api, LiteLLM, vLLM, cliproxy or a bare vendor endpoint.
9
9
  * :class:`CliproxyAdapter` — adds attribution (which upstream provider serves
10
10
  this alias, is it disabled, what is the real model name) by reading the
11
- gateway's own YAML config, and can plan a removal.
11
+ gateway's own YAML config. Only unambiguous, unprefixed legacy layouts can
12
+ use the existing removal planner; v8 attribution stays read-only.
12
13
 
13
14
  The split between *reading* and *removing* is deliberate:
14
15
 
@@ -31,9 +32,16 @@ from pathlib import Path
31
32
 
32
33
  from .probe import Gateway, GatewayError, list_models
33
34
 
34
- #: Top-level YAML keys whose list items are *provider blocks* holding models.
35
- #: Kept as a tuple so a different gateway layout is a one-line extension.
35
+ #: Supported legacy sections and their v8 ``api-keys`` family names.
36
36
  PROVIDER_SECTIONS: tuple[str, ...] = ("openai-compatibility", "claude-api-key")
37
+ V8_FAMILIES: dict[str, str] = {
38
+ "openai-compatibility": "openai-compatibility",
39
+ "claude-api-key": "claude",
40
+ }
41
+ _SHARED_KEY_FIELDS = frozenset({
42
+ "priority", "prefix", "proxy-url", "headers", "models", "excluded-models",
43
+ "disable-cooling", "request-retry", "request-scoped-errors",
44
+ })
37
45
 
38
46
  #: A list item is a model entry only when its nearest lower-indented ancestor
39
47
  #: line is one of these keys. Provider blocks also carry ``name:`` and would
@@ -72,6 +80,66 @@ def effective_alias(name: str, alias: str = "") -> str:
72
80
  return (alias or name or "").strip()
73
81
 
74
82
 
83
+ def _v8_blocks(groups: object, family: str) -> list[dict]:
84
+ """Expand the two supported v8 families, without interpreting credentials."""
85
+ if not isinstance(groups, list):
86
+ raise ConfigError(f"api-keys.{family} must be a list")
87
+ blocks = []
88
+ for group in groups:
89
+ if not isinstance(group, dict) or not isinstance(group.get("keys"), list):
90
+ raise ConfigError(f"api-keys.{family} groups need a keys list")
91
+ keys = group["keys"]
92
+ if any(not isinstance(key, dict) for key in keys):
93
+ raise ConfigError(f"api-keys.{family} keys must be mappings")
94
+ if family == "openai-compatibility":
95
+ # Upstream keeps the group's models/prefix on the provider block;
96
+ # individual keys do not override them for this family.
97
+ blocks.append({key: value for key, value in group.items() if key != "keys"})
98
+ continue
99
+ allowed = _SHARED_KEY_FIELDS | {"name", "base-url", "keys"}
100
+ if set(group) - allowed:
101
+ raise ConfigError(f"unsupported api-keys.{family} group fields")
102
+ for key in keys:
103
+ if "base-url" in key:
104
+ raise ConfigError(f"api-keys.{family}: base-url belongs to the group")
105
+ block = {
106
+ name: value for name, value in group.items()
107
+ if name == "base-url" or name in _SHARED_KEY_FIELDS
108
+ }
109
+ block.update({
110
+ name: value for name, value in key.items()
111
+ if name not in {"auth_index", "auth-index"} and value is not None
112
+ })
113
+ blocks.append(block)
114
+ return blocks
115
+
116
+
117
+ def _has_models(value: object) -> bool:
118
+ """An unhandled models subtree must not be eligible for text surgery."""
119
+ pending = [value]
120
+ seen = set()
121
+ while pending:
122
+ item = pending.pop()
123
+ if id(item) in seen:
124
+ continue # YAML aliases may share or even cycle containers.
125
+ seen.add(id(item))
126
+ if isinstance(item, dict):
127
+ if "models" in item:
128
+ return True
129
+ pending.extend(item.values())
130
+ elif isinstance(item, list):
131
+ pending.extend(item)
132
+ return False
133
+
134
+
135
+ def _prefix(value: object) -> str:
136
+ if not isinstance(value, str):
137
+ raise ConfigError("model prefix must be a string")
138
+ prefix = value.strip().strip("/")
139
+ # CLIProxyAPI normalizes an empty or multi-segment prefix away.
140
+ return "" if not prefix or "/" in prefix else prefix.strip()
141
+
142
+
75
143
  # --- attribution ------------------------------------------------------------
76
144
 
77
145
 
@@ -180,7 +248,7 @@ class CliproxyAdapter(Adapter):
180
248
  """
181
249
 
182
250
  name = "cliproxy"
183
- removable = True
251
+ removable = False
184
252
 
185
253
  def __init__(self, gateway: Gateway, config_path: str | Path):
186
254
  self.gateway = gateway
@@ -192,51 +260,97 @@ class CliproxyAdapter(Adapter):
192
260
  # -- reading ---------------------------------------------------------
193
261
 
194
262
  def _parse(self) -> dict[str, Attribution]:
263
+ self.removable = False
195
264
  doc = _load_yaml(self.config_path)
196
265
  if not isinstance(doc, dict):
197
266
  raise ConfigError(f"{self.config_path}: expected a mapping at top level")
267
+ if "config-version" in doc and (
268
+ type(doc["config-version"]) is not int or doc["config-version"] != 8
269
+ ):
270
+ raise ConfigError("unsupported config-version (expected 8)")
271
+ families = doc.get("api-keys")
272
+ if families is not None and not isinstance(families, (dict, list)):
273
+ raise ConfigError("api-keys must be a client-key list or a v8 provider mapping")
274
+ v8 = families if isinstance(families, dict) else {}
275
+ force = doc.get("force-model-prefix", False)
276
+ routing = doc.get("routing", {})
277
+ if not isinstance(routing, dict):
278
+ raise ConfigError("routing must be a mapping")
279
+ if "force-model-prefix" in routing:
280
+ force = routing["force-model-prefix"]
281
+ if not isinstance(force, bool):
282
+ raise ConfigError("force-model-prefix must be a boolean")
198
283
 
199
284
  attr: dict[str, Attribution] = {}
200
285
  providers = 0
286
+ recognized = False
287
+ editable = (
288
+ "config-version" not in doc and not isinstance(families, dict) and not force
289
+ )
290
+ for key, value in doc.items():
291
+ if key not in PROVIDER_SECTIONS and _has_models(value):
292
+ editable = False
201
293
  for section in PROVIDER_SECTIONS:
202
- blocks = doc.get(section)
203
- if not isinstance(blocks, list):
294
+ family = V8_FAMILIES[section]
295
+ if family in v8:
296
+ # Presence, not truthiness: [] replaces a legacy section too.
297
+ blocks = _v8_blocks(v8[family], family)
298
+ editable = False
299
+ elif section in doc:
300
+ blocks = doc[section]
301
+ else:
204
302
  continue
303
+ recognized = True
304
+ if not isinstance(blocks, list):
305
+ raise ConfigError(f"{section} must be a list")
205
306
  for block in blocks:
206
307
  if not isinstance(block, dict):
207
- continue
308
+ raise ConfigError(f"{section} entries must be mappings")
309
+ if any(_has_models(value) for key, value in block.items() if key != "models"):
310
+ editable = False
208
311
  providers += 1
209
312
  provider = str(block.get("name") or section)
210
313
  base_url = str(block.get("base-url") or block.get("base_url") or "")
211
314
  disabled = bool(block.get("disabled"))
212
- entries = block.get("models")
315
+ raw_prefix = block.get("prefix", "")
316
+ prefix = _prefix(raw_prefix)
317
+ if raw_prefix:
318
+ editable = False
319
+ entries = block.get("models", [])
213
320
  if not isinstance(entries, list):
214
- continue
321
+ raise ConfigError(f"{section}.models must be a list")
215
322
  for entry in entries:
216
323
  if isinstance(entry, str):
217
324
  name, alias = entry, ""
218
325
  elif isinstance(entry, dict):
219
- name = str(entry.get("name") or "")
220
- alias = str(entry.get("alias") or "")
326
+ name = entry.get("name") or ""
327
+ alias = entry.get("alias") or ""
328
+ if not isinstance(name, str) or not isinstance(alias, str):
329
+ raise ConfigError("model name and alias must be strings")
221
330
  else:
222
- continue
331
+ raise ConfigError("models entries must be names or mappings")
223
332
  key = effective_alias(name, alias)
224
333
  if not key:
334
+ editable = False
225
335
  continue
226
- # First declaration wins: a duplicate alias later in the
227
- # file is a config bug, and silently overwriting attribution
228
- # would hide it from the report.
229
- attr.setdefault(
230
- key,
231
- Attribution(
336
+ routes = [key] if not prefix or not force or prefix == key else []
337
+ if prefix:
338
+ routes.append(f"{prefix}/{key}")
339
+ for route in routes:
340
+ # Keep the first hint, but ambiguity cannot enable edits.
341
+ if route in attr:
342
+ editable = False
343
+ attr.setdefault(route, Attribution(
232
344
  provider=provider,
233
345
  upstream=name.strip(),
234
346
  base_url=base_url,
235
347
  disabled=disabled,
236
348
  source="config",
237
- ),
238
- )
349
+ ))
350
+ if not recognized:
351
+ raise ConfigError("no supported CLIProxyAPI provider layout")
239
352
  self._providers = providers
353
+ self.removable = editable and bool(attr)
240
354
  return attr
241
355
 
242
356
  def attribution_map(self) -> dict[str, Attribution]:
@@ -279,21 +393,21 @@ class CliproxyAdapter(Adapter):
279
393
  # we never measured. An empty list (gateway down) is a real 0 and shows.
280
394
  if self._live is not None:
281
395
  head += f", {len(self._live)} published"
282
- return head + ")"
396
+ return head + (")" if self.removable else ", read-only)")
283
397
 
284
398
  # -- removal ---------------------------------------------------------
285
399
 
286
400
  def plan_removal(self, aliases: Sequence[str]) -> EditPlan:
287
401
  """Cut model entries out of the config text, preserving everything else.
288
402
 
289
- Removal is driven by the text scan, which locates any ``- name:/alias:``
290
- item under a ``models:`` key in the file. Attribution is narrower — it
291
- only reads :data:`PROVIDER_SECTIONS` — so an alias from a section this
292
- adapter does not know can still be cut, and the diff shown to the
293
- operator is the only authority. Aliases that appear nowhere in the text
294
- (published by a running process, or already removed) come back in
295
- ``not_found`` and are left alone.
403
+ The existing text scan is limited to unambiguous, unprefixed legacy
404
+ layouts. v8, mixed, prefixed and otherwise ambiguous attribution is
405
+ read-only: successful parsing is not proof that this scanner can edit it.
406
+ Aliases absent from the text come back in ``not_found`` unchanged.
296
407
  """
408
+ self.attribution_map()
409
+ if not self.removable:
410
+ raise ConfigError("this config layout supports attribution only (read-only)")
297
411
  wanted = [a for a in dict.fromkeys(aliases) if a]
298
412
  before = read_text(self.config_path)
299
413
  entries = {entry.alias: entry for entry in iter_model_entries(before)}