ai_api_unified 2.35.0__tar.gz → 3.2.0__tar.gz

{ai_api_unified-2.35.0 → ai_api_unified-3.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.3
 Name: ai_api_unified
-Version: 2.35.0
+Version: 3.2.0
 Summary: Unified access layer for AI completions (LLM), embedding (semantic), image, video, and voice services
 License: MIT
 Keywords: llm,openai,anthropic,claude,bedrock,gemini,vertex,embeddings,completion,video,voice,api
@@ -77,7 +77,7 @@ Project-URL: Issues, https://github.com/davecthomas/ai-api-unified/issues
 Project-URL: Repository, https://github.com/davecthomas/ai-api-unified
 Description-Content-Type: text/markdown
 
-# ai-api-unified 2.35.0
+# ai-api-unified 3.2.0
 
 [![CI](https://github.com/davecthomas/ai-api-unified/actions/workflows/ci.yml/badge.svg)](https://github.com/davecthomas/ai-api-unified/actions/workflows/ci.yml)
 [![PyPI](https://img.shields.io/pypi/v/ai-api-unified.svg)](https://pypi.org/project/ai-api-unified/)
@@ -190,6 +190,7 @@ The public entry points are the stable base interfaces and factories:
 | ----------- | ------------------- | ----------------------------------------------------------------------------------------------------------------------------- | ---------------------------------------------------- |
 | Completions | `AIBaseCompletions` | `openai`, `openai-responses`, `openai-compatible`, `claude`, `google-gemini`, Bedrock-routed aliases such as `nova`, `anthropic`, `llama`, `mistral`, `cohere`, `ai21`, `rerank` | `openai`, `anthropic`, `google_gemini`, `bedrock`    |
 | Embeddings  | `AIBaseEmbeddings`  | `openai`, `titan`, `google-gemini`, `voyage`                                                                                  | `openai`, `bedrock`, `google_gemini`, `voyage`       |
+| Classification | `AIBaseClassifier` | `openai-decisions`, and `openai`, `openai-responses`, `openai-compatible`, `claude`, `google-gemini`, `bedrock`, `anthropic` through structured output | `openai`, `anthropic`, `google_gemini`, `bedrock` |
 | Images      | `AIBaseImages`      | `openai`, `google-gemini`, `nova-canvas` and Bedrock image aliases                                                            | `openai`, `google_gemini`, `bedrock`                 |
 | Videos      | `AIBaseVideos`      | `openai`, `google-gemini`, `nova-reel` and Bedrock video aliases                                                              | `openai`, `google_gemini`, `bedrock`                 |
 | Voice TTS   | `AIVoiceBase`       | `openai`, `google`, `azure`, `elevenlabs`                                                                                     | `openai`, `google_gemini`, `azure_tts`, `elevenlabs` |
@@ -900,6 +901,22 @@ Anthropic serves none.
   `AiProviderRequestError` carrying `status_code`, matching the completions
   clients.
 
+
+### Classification
+
+```python
+from ai_api_unified import AIFactory, AIClassificationChoice
+
+client = AIFactory.get_ai_classifier_client(classifier_engine="claude")
+result = client.classify_choice(
+    "Summarize this ticket in one line.",
+    [AIClassificationChoice(value="easy"), AIClassificationChoice(value="hard")],
+)
+print(result.choice)  # "easy"
+```
+
+See the Classification section for the question types and the engines that
+support them.
 ### Multimodal Embeddings
 
 Every embeddings client exposes a `capabilities` descriptor stating which input
@@ -1108,6 +1125,96 @@ than ignoring them.
 A blank retry-policy setting (`COMPLETIONS_RETRY_POLICY=`) is treated as
 unconfigured and falls back to `default` across every OpenAI-backed capability.
 
+## Classification
+
+A classifier answers one narrow question about a piece of text: which of
+your options fits it, whether a condition holds, or where it sits on a
+scale. It stands beside completions and embeddings as its own capability,
+so you can use it for routing, triage, or moderation without writing a
+prompt and parsing an answer. The model router (a later section) is one consumer.
+
+### Quickstart
+
+```python
+from ai_api_unified import AIFactory, AIClassificationChoice
+
+client = AIFactory.get_ai_classifier_client()   # CLASSIFIER_ENGINE from env
+result = client.classify_choice(
+    "My invoice shows two charges for one order.",
+    [
+        AIClassificationChoice(value="billing", description="Payments, invoices, refunds."),
+        AIClassificationChoice(value="technical", description="Errors using the product."),
+        AIClassificationChoice(value="other", description="Anything else."),
+    ],
+    instructions="Which team should handle this message?",
+)
+print(result.choice, result.confidence, result.usage.input_tokens)
+```
+
+Three question types share one result model, `AIClassificationResult`:
+
+| Method | Question | Result fields |
+| ------ | -------- | ------------- |
+| `classify_choice(text, choices)` | Which option fits? | `choice`, `confidence`, `probabilities` |
+| `classify_predicate(text, instructions=...)` | Does the condition hold? | `probability` |
+| `classify_score(text, levels)` | Where on the scale, lowest level first? | `score` in [0, 1], `choice` (the level label) |
+
+Every result also carries `refused` (the model declined), `usage`,
+`token_source`, `provider_engine`, `model_name`, and `latency_ms`. Async
+twins (`aclassify_choice` and so on) exist where `capabilities.supports_async`
+is True. `capabilities` reports which question types the engine answers,
+`max_choices`, `max_input_chars`, and `pricing`.
+
+Before the call, the text goes through the PII middleware when it is
+enabled, then is cut to `capabilities.max_input_chars` (6,000 by default)
+keeping its head and tail: a label rarely needs more, and the provider
+bills every input token.
+
+### Models that can classify
+
+| Engine | How it classifies | Models |
+| ------ | ----------------- | ------ |
+| `openai-decisions` | OpenAI's Decisions API (`POST /v1/decisions`). Native choice, predicate, and score answers with probabilities. About ten times faster than a chat completion. Bills input tokens only, at $0.10 per 1M. | `gpt-6-luna` (public beta) |
+| `openai`, `openai-responses`, `openai-compatible` | Structured output (a one-field enum schema) | Every model the engine lists |
+| `claude` | Structured output | Every model the engine lists |
+| `google-gemini` | Structured output | Every model the engine lists |
+| `bedrock`, `anthropic` | Structured output | Models whose capabilities include `supports_structured_output` (Claude, DeepSeek, Qwen, GLM); Nova cannot classify |
+
+Through structured output, a predicate answer is a hard yes or no
+(`probability` 1.0 or 0.0) and a score is the picked level's position on the
+scale; engines with native classification report real probabilities.
+Picking an engine without structured output fails at build with
+`AiProviderCapabilityUnsupportedError`.
+
+### Cost and metering
+
+Every classification call runs inside the same observability lifecycle as
+completions, so with observability on it emits the usual input, output,
+error, and cost events, with `operation` set to `classify_choice`,
+`classify_predicate`, or `classify_score`. The Decisions engine prices
+under the registry key `decisions/gpt-6-luna`, since its rates differ from
+the model's chat rates; when a response carries no `usage` block the engine
+estimates input tokens from the request text and reports
+`token_source="estimated"`. The completions engine a
+structured-output classifier wraps meters the call, and that call carries
+the tag `classifier_role` (`choice`, `predicate`, or `score`) so a dashboard can
+split classifier spend from the engine's other traffic. `token_source` on
+the result says where the counts came from: `provider`, `estimated`, or
+`none`. A call that could not be metered logs a WARNING with
+`ai_classifier_event=classifier_unmetered` on the `ai_api_unified.ai_base`
+logger; it is never a silent zero.
+
+### Configuration
+
+| Setting | Meaning |
+| ------- | ------- |
+| `CLASSIFIER_ENGINE` | Engine token. When unset, `openai-decisions` is used if `OPENAI_API_KEY` is set; otherwise the factory raises. |
+| `CLASSIFIER_MODEL_NAME` | Model override; blank falls back to `COMPLETIONS_MODEL_NAME` for structured-output engines, then the engine's default (`gpt-6-luna` for `openai-decisions`). |
+| `AIFactory.get_ai_classifier_client(classifier_engine=, model_name=, base_url=)` | Arguments override the settings. |
+
+Exports: `AIBaseClassifier`, `AIClassifierCapabilitiesBase`,
+`AIClassificationChoice`, `AIClassificationLevel`, `AIClassificationResult`.
+
 ## Model fallback
 
 A fallback chain keeps completions flowing when a model cannot serve a
@@ -1358,16 +1465,289 @@ poetry run pytest -m nonmock tests/test_model_fallback_nonmock.py -q
 It needs `ANTHROPIC_API_KEY` and `OPENAI_API_KEY` in `.env` and skips without
 them.
 
+## Model routing
+
+A router sends each request to a tier of models that matches how demanding
+the request is. A cheap classifier reads the latest user message, places it
+in one of your tiers (easy, medium, hard, or names of your own), and the
+request runs on that tier's model chain. A trivial question stops costing
+flagship rates; a hard one still gets the strongest model.
+
+### Why it exists
+
+One configured model serves every prompt, whatever its difficulty. Routing
+by complexity lowers cost and latency on the easy majority without giving
+up quality on the hard minority, and it does so in the library, so an
+application changes nothing but configuration.
+
+### Setting up a router
+
+Write the tiers as JSON, ascending capability. Each tier lists an ordered
+`engine:model` chain: the first entry is the tier's primary and the rest
+are its fallback chain. Tiers named `easy`, `medium`, and `hard` have
+built-in descriptions; any other name needs a `description` the classifier
+can read.
+
+```json
+{
+  "tiers": [
+    {"name": "easy",   "models": ["openai:gpt-4.1-mini", "google-gemini:gemini-2.5-flash-lite"]},
+    {"name": "medium", "models": ["claude:claude-sonnet-5", "openai:gpt-5.6-luna"]},
+    {"name": "hard",   "description": "Multi-step reasoning, long synthesis, cross-file code, or anything where a wrong answer is costly.",
+                       "models": ["claude:claude-opus-5-5"]}
+  ],
+  "classifier": {"engine": "openai-decisions", "model": "gpt-6-luna", "timeout_seconds": 5},
+  "default_tier": "medium",
+  "on_exhausted": "escalate"
+}
+```
+
+Point `COMPLETIONS_ROUTER_CONFIG` at the file (or set it to the JSON
+itself), keep `COMPLETIONS_ENGINE` and `COMPLETIONS_MODEL_NAME` as they
+are, and call the factory as usual:
+
+```python
+client = AIFactory.get_ai_completions_client()      # an AiRouterCompletions
+print(client.send_prompt("What is 2 + 2?"))          # routed to the easy tier
+print(client.last_tier, client.last_decision)
+```
+
+In code, pass `router=AIRouterConfig(...)` instead. The configured
+completions LLM, with its `COMPLETIONS_FALLBACKS` chain, is the **default
+route**: where calls go when no classifier is available, when the
+classifier fails, or when `default_tier` is unset.
+
+### The classifier
+
+The router uses a classification engine (see Classification). Resolution
+order: `COMPLETIONS_ROUTER_CLASSIFIER` (`engine:model`), then the config's
+`classifier`, then the classification factory's own resolution
+(`CLASSIFIER_ENGINE`, then `openai-decisions` when `OPENAI_API_KEY` is
+set). Set either to `none` to route without a classifier; if the
+classifier cannot be built, the router logs `classifier_unavailable` once
+and degrades to the default route, and never raises at startup.
+
+The classifier reads the latest user message only, cut to the engine's
+`max_input_chars`. The system prompt is left out: it is long, constant,
+and carries no complexity signal. The tier descriptions carry the policy,
+and `classifier.instructions` prepends your own guidance.
+
+### What happens on a request
+
+1. A per-call `route` option or `for_tier()` pins the tier (method
+   `override`).
+2. A history that already holds engine-shaped entries (tool turns) skips
+   the classifier and stays in a tier whose engines share that family,
+   preferring the tier that served the previous turn (method `pinned`).
+3. Otherwise the classifier names a tier (method `classifier`). No
+   classifier, an error, a refusal, or an unknown tier name routes to
+   `default_tier` or the default route (method `default`) and logs
+   `classifier_failed`.
+4. The call runs on the tier's chain, which fails over inside the tier as
+   any chain does.
+5. If the chain runs out of candidates on a trigger reason
+   (`fallback_on`), the router escalates to the next tier up and logs
+   `tier_exhausted`; with `on_exhausted: raise`, on a non-trigger error, or
+   on the top tier, the error propagates; if a tier's primary cannot be
+   built, the router logs `tier_unbuildable` once and skips the tier the
+   same way.
+
+### Seeing where a request went
+
+Every call logs a `routed` record at INFO before it runs, on the logger
+`ai_api_unified.completions.ai_router_completions`, with an
+`ai_router_event` field so a log processor can count and alert without
+parsing the message:
+
+| Level | `ai_router_event` | When |
+| ----- | ----------------- | ---- |
+| INFO | `routed` | Every call. Fields: `route_id`, `tier`, `tier_index`, `method`, `operation`, `classifier_engine`, `classifier_model`, `confidence`, `probabilities`, `classifier_latency_ms`, `classifier_input_tokens`, `classifier_token_source`, `classifier_usd_cost`, `caller_id`, `session_id`, `workflow_id`. |
+| WARNING | `classifier_unavailable` | Once at build: no classifier, with `reason` and `routed_to`. |
+| WARNING | `classifier_failed` | The classifier raised, refused, or named no tier; the call went to `routed_to`. Fields: `error`, `fallback_reason`, `latency_ms`. |
+| ERROR | `tier_exhausted` | A tier's chain ran out of candidates on a trigger reason; `escalate_to` names the next tier. |
+| ERROR | `tier_unbuildable` | A tier's primary could not be built; the tier is out for this process. |
+
+The router also pushes `route_id`, `route_tier`, and `route_method` onto
+the observability context around the downstream call, so the engine's own
+input, output, and cost events carry them as `tag_route_tier` and so on:
+cost by tier needs no extra work. The classifier call itself carries
+`tag_router_role=classifier`.
+
+### Per-call control
+
+- `provider_options={"route": "hard"}` pins one structured-output or
+  conversation call to a tier; `"default"` pins it to the default route.
+  The router removes the key before options reach the engine.
+- `client.for_tier("hard")` returns that tier's chain client, for
+  `send_prompt` and streaming, which take no `provider_options`.
+- `client.last_tier`, `client.last_decision`, and `client.last_route_client`
+  report the most recent call.
+
+### Configuration reference
+
+| Setting | Meaning |
+| ------- | ------- |
+| `COMPLETIONS_ROUTER_CONFIG` | Path to a JSON file, or inline JSON, with `tiers`, optional `classifier`, `default_tier`, `on_exhausted`, `fallback_on`. |
+| `COMPLETIONS_ROUTER_CLASSIFIER` | `engine:model` for the router's classifier, or `none`. Overrides the config's `classifier`. |
+| `AIFactory.get_ai_completions_client(router=AIRouterConfig(...))` | Code-first configuration; overrides the setting. |
+
+Exports: `AiRouterCompletions`, `AIRouterConfig`, `AIRouterTier`,
+`AIRouterClassifierConfig`, `load_router_config`.
+
+## Configured-model check
+
+A fallback chain or a router names models the process expects to call
+later. A typo, a retired id, or a model not offered in your region would
+otherwise surface only when that model is finally tried, in the middle of
+a failover. So whenever you configure a chain or a router, the factory
+verifies every configured model before it builds the client: the
+completions LLM, each chain candidate, each tier candidate, and the
+router's classifier.
+
+### What it does
+
+- One free metadata call per model, never a completion: OpenAI
+  `models.retrieve`, Anthropic `models.retrieve`, Gemini `models.get` on
+  the raw id, Bedrock `get_foundation_model` (or `get_inference_profile`
+  for `us.`, `eu.`, `apac.`, and `global.` ids) in `AWS_REGION`, an
+  OpenAI-compatible server's `/models` list, and the Decisions API's fixed
+  model set.
+- Calls run in parallel, and the checker probes each `(engine, model,
+  base_url)` once per process however many clients, chains, or tiers name
+  it. Startup
+  pays one round trip, and later factory calls pay nothing.
+- The factory prunes a model the provider reports **missing** from the
+  effective configuration with a WARNING: `configured model openai:gpt-nope
+  is not available and will be ignored`. A chain drops the candidate; a
+  router tier drops the candidate, and the router drops a tier left with no
+  models. A request for a dropped tier (by the classifier, a `route`
+  override, or `for_tier`) goes to the next **higher** surviving tier, never
+  a lower one, and to the default route when nothing is above. A dropped
+  `default_tier` moves the same way.
+- The primary completions model cannot be pruned: it is the default
+  route, so when it is missing the factory raises.
+- A missing classifier model logs `classifier_unavailable`, and the router
+  degrades to its default route.
+- A model the probe cannot ask about (no credentials, no extra installed,
+  credentials rejected, a server with no model catalog, Vertex auth) is
+  **unverifiable** and kept, logged at INFO. A probe error (network, 5xx)
+  keeps the model and logs a WARNING. Neither counts as missing.
+
+Log records go to `ai_api_unified.util.ai_model_checker` with an
+`ai_model_check_event` field: `model_missing` (WARNING), `model_check_error`
+(WARNING), `model_unverifiable` (INFO), `model_available` (DEBUG); the
+factory adds `router_tier_pruned` and `router_disabled` (WARNING) on
+`ai_api_unified.ai_factory`. `AIFactory.last_model_check_report()` returns
+the results of the most recent check.
+
+### Configuration
+
+| Setting | Meaning |
+| ------- | ------- |
+| `AI_MODEL_CHECK` | `prune` (default): drop missing models with a warning. `strict`: raise on any missing model. `off`: skip the check. |
+
+A client with only a primary and no chain or router runs no check: the
+engine fails on its own at build.
+
+### Checking your own configuration
+
+```bash
+poetry run pytest -m nonmock tests/test_configured_models_nonmock.py -q -s --models-config ./models.json
+```
+
+prints every model the file names with its status and fails on any the
+provider reports missing. To watch the pruning itself:
+
+```bash
+poetry run pytest -m nonmock tests/test_model_check_nonmock.py -q -s
+```
+
+reads `tests/fixtures/models_missing_test.json`, whose chain and easy tier
+each list an id no provider serves, and proves both are reported missing
+and pruned with the warning while the configuration still builds; with
+`--models-config ./models.json` it does the same against your file,
+injecting a bogus id in memory.
+
 ## Configuration
 
+### models.json: one file for model selections
+
+Engines, models, the fallback chain, the router, the classifier, the
+configured-model check, and the allow/deny policy live in one file,
+`models.json` (or `.yaml`). Copy `models_sample.json`, which sets every
+section, and point `AI_MODELS_CONFIG` at it (or leave it unset: the library
+picks up `./models.json` when it exists; a blank value turns the file off;
+the value may also be inline JSON).
+
+```json
+{
+  "allow": ["claude:*", "google-gemini:*", "openai-decisions:*"],
+  "deny":  ["openai:gpt-4*", "bedrock"],
+  "completions": {"engine": "google-gemini", "model": "gemini-2.5-flash",
+                  "fallbacks": ["claude:claude-sonnet-5"],
+                  "fallback_on": "unavailable,rate_limited,quota_exhausted,auth_rejected"},
+  "classifier":  {"engine": "openai-decisions", "model": "gpt-6-luna"},
+  "router":      {"tiers": [{"name": "easy", "models": ["google-gemini:gemini-2.5-flash-lite"]},
+                            {"name": "hard", "models": ["claude:claude-opus-5-5"]}],
+                  "default_tier": "easy"},
+  "model_check": {"mode": "prune", "timeout_seconds": 10},
+  "embeddings":  {"engine": "google-gemini", "model": "gemini-embedding-001"},
+  "images":      {"engine": "google-gemini", "model": "gemini-3.1-flash-image"},
+  "videos":      {"engine": "google-gemini", "model": "veo-3.1-lite-generate-preview"},
+  "voice":       {"engine": "google", "tts_model": "gemini-2.5-pro-tts", "language": "en-US"}
+}
+```
+
+To convert an existing `.env`, run
+
+```bash
+poetry run python scripts/migrate2json.py            # writes ./models.json
+poetry run python scripts/migrate2json.py --comment-env   # and comments the moved keys out of .env
+```
+
+It places each selection key in its section, embeds the router file named
+by `COMPLETIONS_ROUTER_CONFIG`, fills structural defaults from
+`models_sample.json`, leaves the allow and deny lists empty, and never
+touches secrets.
+
+Each section stands in for the `.env` keys of the same capability
+(`completions.model` for `COMPLETIONS_MODEL_NAME`, `embeddings.engine` for
+`EMBEDDING_ENGINE`, `router` for `COMPLETIONS_ROUTER_CONFIG`, and so on),
+so nothing else in the library changes. Precedence, highest first: an
+explicit factory argument, a process environment variable, a `.env` value,
+the file, the library default. Every key in the tables below still works
+and overrides the file when set, which lets a deployment pin one value
+without editing the file. Secrets (API keys, AWS credentials), `AWS_REGION`,
+base-URL overrides, and `AI_MIDDLEWARE_CONFIG_PATH` stay in `.env`.
+
+### Allow and deny lists
+
+`allow` and `deny` hold `engine:model` globs: `claude:*`, `*:gpt-4*`,
+`openai-decisions:gpt-6-luna`; a bare engine token (`bedrock`) means
+`bedrock:*`. Deny wins over allow, and an empty allow list allows every
+model. The policy applies wherever a model enters: the completions LLM,
+each fallback candidate, each router tier candidate, the router's
+classifier, and the embeddings, image, and video selections.
+
+- The factory prunes a denied fallback, tier, or classifier model the way
+  it prunes a missing one (see Configured-model check) and warns
+  `configured fallback model openai:gpt-4o is denied by policy pattern 'openai:gpt-4*' and will be ignored`
+  (`ai_model_policy_event=model_denied`). A tier left with no models
+  routes to the next higher surviving tier.
+- A denied primary, or an explicit `model_name=` argument for a denied
+  model, raises `ValueError` naming the pattern. Runtime never calls a
+  denied model.
+
 ### Required Engine Selectors
 
-There is no implicit default provider. Set the selector for each capability you use.
+There is no implicit default provider. Set the selector for each capability
+you use, in `models.json` or as a variable:
 
 | Environment variable | Valid values                                                                                                                  |
 | -------------------- | ----------------------------------------------------------------------------------------------------------------------------- |
 | `COMPLETIONS_ENGINE` | `openai`, `openai-responses`, `openai-compatible`, `claude`, `google-gemini`, `bedrock`, Bedrock-routed aliases such as `nova`, `anthropic`, `llama`, `mistral`, `cohere`, `ai21`, `rerank` |
 | `EMBEDDING_ENGINE`   | `openai`, `titan`, `google-gemini`                                                                                            |
+| `CLASSIFIER_ENGINE`  | `openai-decisions`, `openai`, `openai-responses`, `openai-compatible`, `claude`, `google-gemini`, `bedrock`, `anthropic` (see Classification) |
 | `IMAGE_ENGINE`       | `openai`, `google-gemini`, `nova-canvas`, `bedrock`, `nova`                                                                   |
 | `VIDEO_ENGINE`       | `openai`, `google-gemini`, `bedrock`, `nova`, `nova-reel`                                                                     |
 | `AI_VOICE_ENGINE`    | `openai`, `google`, `azure`, `elevenlabs`                                                                                     |
@@ -1379,7 +1759,13 @@ There is no implicit default provider. Set the selector for each capability you
 | `COMPLETIONS_MODEL_NAME`    | Optional completions model override                                                         |
 | `COMPLETIONS_FALLBACKS`     | Optional ordered `engine:model` pairs tried after the primary fails (see Model fallback)     |
 | `COMPLETIONS_FALLBACK_ON`   | Optional comma-separated `AiFallbackReason` values that trigger a fallback; default `unavailable,rate_limited,quota_exhausted,auth_rejected` |
+| `AI_MODELS_CONFIG` | Path to `models.json`/`.yaml`, or inline JSON; unset uses `./models.json` when present; blank turns the file off |
+| `COMPLETIONS_ROUTER_CONFIG` | Override of the file's `router` section: a JSON file path or inline JSON mapping tiers to `engine:model` chains (see Model routing) |
+| `COMPLETIONS_ROUTER_CLASSIFIER` | Optional `engine:model` for the router's classifier, or `none`; default is the classification factory's resolution |
+| `AI_MODEL_CHECK` | Optional: `prune` (default), `strict`, or `off` for the configured-model check (see Configured-model check) |
+| `AI_MODEL_CHECK_TIMEOUT_SECONDS` | Optional per-probe timeout for the configured-model check (default 10) |
 | `EMBEDDING_MODEL_NAME`      | Optional embeddings model override. `gemini-embedding-2` enables multimodal embeddings.     |
+| `CLASSIFIER_MODEL_NAME`     | Optional classification model override; blank falls back to `COMPLETIONS_MODEL_NAME`, then the engine default |
 | `IMAGE_MODEL_NAME`          | Optional image model override                                                               |
 | `VIDEO_MODEL_NAME`          | Optional video model override                                                               |
 | `VIDEO_OUTPUT_DIR`          | Optional local output directory for materialized video artifacts                            |
@@ -1927,6 +2313,15 @@ Live provider tests:
 poetry run pytest -m nonmock -s -vv
 ```
 
+Tests never read your own `models.json`: the suite blanks `AI_MODELS_CONFIG`.
+Live tests that need a models config read `tests/fixtures/models_test.json`,
+whose hard tier lists a model its own deny list rejects so the policy tests
+prove the deny list bites. To run them against any file:
+
+```bash
+poetry run pytest -m nonmock tests/test_model_policy_nonmock.py tests/test_configured_models_nonmock.py -q -s --models-config ./models.json
+```
+
 Notes for live tests:
 
 - install the matching provider extras first