@plurnk/plurnk-providers 5.0.2 → 6.0.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (108) hide show
  1. package/.env.defaults +11 -81
  2. package/README.md +21 -22
  3. package/SPEC.md +109 -1190
  4. package/dist/LeadingReasoning.js +1 -1
  5. package/dist/LeadingReasoning.js.map +1 -1
  6. package/dist/Mock.d.ts +2 -2
  7. package/dist/Mock.d.ts.map +1 -1
  8. package/dist/Mock.js +2 -2
  9. package/dist/Mock.js.map +1 -1
  10. package/dist/{AiSdkProvider.d.ts → ModelProvider.d.ts} +12 -21
  11. package/dist/ModelProvider.d.ts.map +1 -0
  12. package/dist/{AiSdkProvider.js → ModelProvider.js} +140 -188
  13. package/dist/ModelProvider.js.map +1 -0
  14. package/dist/{AiSdkRequestBody.d.ts → ModelRequestBody.d.ts} +6 -10
  15. package/dist/ModelRequestBody.d.ts.map +1 -0
  16. package/dist/{AiSdkRequestBody.js → ModelRequestBody.js} +17 -63
  17. package/dist/ModelRequestBody.js.map +1 -0
  18. package/dist/Pool.js +24 -24
  19. package/dist/Pool.js.map +1 -1
  20. package/dist/ProviderRegistry.d.ts.map +1 -1
  21. package/dist/ProviderRegistry.js +15 -15
  22. package/dist/ProviderRegistry.js.map +1 -1
  23. package/dist/RequestCapture.d.ts +1 -0
  24. package/dist/RequestCapture.d.ts.map +1 -1
  25. package/dist/RequestCapture.js +9 -6
  26. package/dist/RequestCapture.js.map +1 -1
  27. package/dist/accounting.d.ts +1 -1
  28. package/dist/accounting.d.ts.map +1 -1
  29. package/dist/accounting.js +4 -4
  30. package/dist/accounting.js.map +1 -1
  31. package/dist/capacity.js +4 -4
  32. package/dist/capacity.js.map +1 -1
  33. package/dist/catalogProvider.d.ts +5 -8
  34. package/dist/catalogProvider.d.ts.map +1 -1
  35. package/dist/catalogProvider.js +216 -66
  36. package/dist/catalogProvider.js.map +1 -1
  37. package/dist/compatibleProvider.d.ts.map +1 -1
  38. package/dist/compatibleProvider.js +8 -8
  39. package/dist/compatibleProvider.js.map +1 -1
  40. package/dist/discover.d.ts +1 -1
  41. package/dist/discover.d.ts.map +1 -1
  42. package/dist/discover.js +1 -8
  43. package/dist/discover.js.map +1 -1
  44. package/dist/env.d.ts +2 -2
  45. package/dist/env.d.ts.map +1 -1
  46. package/dist/env.js +20 -59
  47. package/dist/env.js.map +1 -1
  48. package/dist/errors.d.ts +48 -0
  49. package/dist/errors.d.ts.map +1 -1
  50. package/dist/errors.js +61 -4
  51. package/dist/errors.js.map +1 -1
  52. package/dist/index.d.ts +5 -6
  53. package/dist/index.d.ts.map +1 -1
  54. package/dist/index.js +6 -7
  55. package/dist/index.js.map +1 -1
  56. package/dist/{sdkModels.d.ts → modelAdapters.d.ts} +7 -9
  57. package/dist/modelAdapters.d.ts.map +1 -0
  58. package/dist/{sdkModels.js → modelAdapters.js} +194 -87
  59. package/dist/modelAdapters.js.map +1 -0
  60. package/dist/native-tools.d.ts +1 -2
  61. package/dist/native-tools.d.ts.map +1 -1
  62. package/dist/native-tools.js +9 -11
  63. package/dist/native-tools.js.map +1 -1
  64. package/dist/ollama.d.ts.map +1 -1
  65. package/dist/ollama.js +26 -12
  66. package/dist/ollama.js.map +1 -1
  67. package/dist/openai.d.ts +3 -3
  68. package/dist/openai.d.ts.map +1 -1
  69. package/dist/openai.js +1 -1
  70. package/dist/pi-transport.d.ts +69 -0
  71. package/dist/pi-transport.d.ts.map +1 -0
  72. package/dist/pi-transport.js +697 -0
  73. package/dist/pi-transport.js.map +1 -0
  74. package/dist/providerError.js +1 -1
  75. package/dist/providerError.js.map +1 -1
  76. package/dist/retry-deadline.d.ts +3 -0
  77. package/dist/retry-deadline.d.ts.map +1 -0
  78. package/dist/retry-deadline.js +96 -0
  79. package/dist/retry-deadline.js.map +1 -0
  80. package/dist/stream-forensics.d.ts +51 -0
  81. package/dist/stream-forensics.d.ts.map +1 -0
  82. package/dist/stream-forensics.js +403 -0
  83. package/dist/stream-forensics.js.map +1 -0
  84. package/dist/types.d.ts +29 -4
  85. package/dist/types.d.ts.map +1 -1
  86. package/dist/types.js.map +1 -1
  87. package/dist/usage.js +2 -2
  88. package/dist/usage.js.map +1 -1
  89. package/docs/models.md +3 -3
  90. package/package.json +8 -21
  91. package/dist/AiSdkProvider.d.ts.map +0 -1
  92. package/dist/AiSdkProvider.js.map +0 -1
  93. package/dist/AiSdkRequestBody.d.ts.map +0 -1
  94. package/dist/AiSdkRequestBody.js.map +0 -1
  95. package/dist/RequestFields.d.ts +0 -24
  96. package/dist/RequestFields.d.ts.map +0 -1
  97. package/dist/RequestFields.js +0 -237
  98. package/dist/RequestFields.js.map +0 -1
  99. package/dist/aiSdkTransport.d.ts +0 -84
  100. package/dist/aiSdkTransport.d.ts.map +0 -1
  101. package/dist/aiSdkTransport.js +0 -651
  102. package/dist/aiSdkTransport.js.map +0 -1
  103. package/dist/model-options.d.ts +0 -3
  104. package/dist/model-options.d.ts.map +0 -1
  105. package/dist/model-options.js +0 -26
  106. package/dist/model-options.js.map +0 -1
  107. package/dist/sdkModels.d.ts.map +0 -1
  108. package/dist/sdkModels.js.map +0 -1
package/SPEC.md CHANGED
@@ -1,1217 +1,136 @@
1
- # Provider Contract
1
+ # `@plurnk/plurnk-providers` Specification
2
2
 
3
- `@plurnk/plurnk-providers` adapts model endpoints to one stable PLURNK
4
- `Provider`. It does not maintain a parallel vendor registry or reproduce
5
- ordinary provider protocols.
3
+ Stable model-provider contract, capacity and monetary accounting authority, and transport adapter over `@earendil-works/pi-ai` (`@earendil-works/pi-ai/api/*` `stream`) and OpenAI-compatible HTTP endpoints.
6
4
 
7
- ## §1 Ownership
5
+ ## §provider-interface Provider and runtime-neutral contract
8
6
 
9
- <!-- map -->
10
- | Tag | Promise |
11
- | --- | --- |
12
- | {§provider-runtime-neutral-accounting} | `@plurnk/plurnk-providers/accounting` — Provider-request aggregation, Models.dev cost estimation, and their… |
13
- | {§provider-runtime-neutral-errors} | `@plurnk/plurnk-providers/errors` — Normalized `ProviderError`, `ProviderErrorKind`, and the types of its… |
14
- | {§provider-runtime-neutral-openai} | `@plurnk/plurnk-providers/openai` — The OpenAI-compatible provider: SDK transport, request capture,… |
15
- <!-- /map -->
7
+ `@plurnk/plurnk-providers` exposes three package entrypoints:
16
8
 
17
- The provider stack has four owners:
9
+ | Subpath | Surface | Runtime boundary |
10
+ |---|---|---|
11
+ | `@plurnk/plurnk-providers` | Full `Provider` contract, `ModelProvider`, `Pool`, `Mock`, `instantiateProvider`, `loadActiveProvider`, `discover`, and configuration helpers | Node >= 26 |
12
+ | `@plurnk/plurnk-providers/accounting` | `aggregateProviderAccounting`, `estimateProviderCost`, `providerCostNormalizer`, `resolveProviderCost`, `sumProviderCostsUsd`, `validateProviderUsage` | Runtime-neutral (browser-bundlable) |
13
+ | `@plurnk/plurnk-providers/errors` | `ProviderError`, `RETRYABLE_PROVIDER_KINDS`, `classifyProviderError`, `toProviderError`, `providerSource` | Runtime-neutral (browser-bundlable) |
18
14
 
19
- 1. Models.dev supplies a release-time snapshot of provider package, API
20
- endpoint, credential names, models, context/input/output limits, reasoning
21
- capability, and USD rates including distinct reasoning rates when supplied.
22
- 2. Official AI SDK providers own vendor request and response protocols.
23
- 3. This package owns the PLURNK contract: aliases, envelopes, normalized usage
24
- and errors, evidence, local capabilities, and first-party metadata.
25
- 4. The operator owns secrets, machine-specific endpoints, and deliberate
26
- metadata overrides through environment variables.
15
+ Every `Provider` instance exposes:
27
16
 
28
- Facts MUST have one owner. Do not copy a cataloged endpoint, credential name,
29
- model prefix, context window, price, or vendor request shape into a PLURNK
30
- table. A missing or wrong catalog fact is fixed upstream, overridden through a
31
- provider declaration, or left explicitly unknown.
17
+ | Member | Contract |
18
+ |---|---|
19
+ | Physical & policy getters | `model`, `servedModel`, `contextWindow`, `maxInputTokens`, `maxOutputTokens`, `outputBudget`, `outputFloor`, `reasoningBudget`, `inputCapacity`, `inputWall`, `supportedEfforts`, `inputModalities` (`ReadonlySet<"image" \| "audio" \| "video" \| "pdf">`), `constrainsOutput`, and `requiresOutputBudget` |
20
+ | `countPromptTokens(messages, signal?)` | Measures or estimates prompt token weight (`PromptTokenMeasurement`) |
21
+ | `assessRequestCapacity(messages, maxOutputTokens?, signal?)` | Evaluates `inputCapacity`, `inputWall`, admission `decision` (`"admit" \| "defer" \| "reject"`), and flexed `responseMax` |
22
+ | `generate(args)` | Requires non-empty `sessionId` (used for KV-cache and slot affinity), optional `callKind` (`"emission"` or `"bare"`), optional `grammar`, `maxOutputTokens`, `sampling`, `observeRequest`, and `observeReasoning`. Native tool calls are unconditionally disabled on every protocol |
23
+ | `observeRequest` | Opens before each physical HTTP attempt and settles once with `(requestAccounting, forensicEvidence)`. When `PLURNK_PROVIDERS_RAWBODY=1`, `forensicEvidence.request` records `{ origin, method, body }` with credentials and URL path/query secrets stripped |
24
+ | `observeReasoning` | Streams reasoning deltas live from structured reasoning events (`thinking_delta`) or leading tagged reasoning envelopes (`<think>...</think>` when `PLURNK_PROVIDERS_REASONING_RESPONSE_STYLE=think-tags`, or llama-server template envelopes on grammar-constrained turns) |
25
+ | `Pool` | Fronts $N \ge 1$ homogeneous backends sharing a `model` id: aggregates limits to conservative floors, pins each `sessionId` to one backend for KV-cache locality, and overflows to a healthy sibling on retryable availability failures (`network_failure`, `rate_limit`, `deadline_exceeded`) while never overflowing terminal errors or `resource_interrupted` attempts |
32
26
 
33
- The package exposes three runtime-neutral public surfaces:
27
+ ## §provider-capacity-admission Capacity and prompt measurement
28
+
29
+ `countPromptTokens` returns a `PromptTokenMeasurement` discriminated by `kind`:
34
30
 
35
- | Surface | Contract |
36
- | --- | --- |
37
- | §provider-runtime-neutral-accounting `@plurnk/plurnk-providers/accounting` | Provider-request aggregation, Models.dev cost estimation, and their wire types. |
38
- | §provider-runtime-neutral-errors `@plurnk/plurnk-providers/errors` | Normalized `ProviderError`, `ProviderErrorKind`, and the types of its attempt, accounting, and capacity evidence. |
39
- | §provider-runtime-neutral-openai `@plurnk/plurnk-providers/openai` | The OpenAI-compatible provider: SDK transport, request capture, accounting, capacity and error normalization, reaching no Node built-in. |
31
+ | `kind` | Source & semantics |
32
+ |---|---|
33
+ | `"exact"` | Measured by the serving vocabulary (such as llama-server `/v1/chat/completions/input_tokens` or `Mock`) |
34
+ | `"upper_bound"` | Conservative maximum across a `Pool` of exact counters |
35
+ | `"estimate"` | Character heuristic (`ceil(codePoints / 2)`, `source: "heuristic:chars2"`) when no exact endpoint exists or when the endpoint degrades |
36
+ | `"unavailable"` | No measurement available |
40
37
 
41
- All three re-export the package's sole implementations without evaluating
42
- Node-only provider discovery, filesystem defaults, or runtime construction.
43
- The package root remains the Node provider-runtime composition surface and is
44
- not a Worker entrypoint.
38
+ `assessRequestCapacity` computes:
45
39
 
46
- ## §2 Provider interface
40
+ - `inputCapacity = min(maxInputTokens, contextWindow - outputBudget)` (the soft curation reservation line).
41
+ - `inputWall = min(maxInputTokens, contextWindow - outputFloor)` (the hard physical wall; when `outputBudget` is tightened below `outputFloor`, the floor tightens with it).
42
+ - `decision`: `"reject"` only when `prompt.kind === "exact"` and `prompt.tokens > inputWall`; `"defer"` when a non-exact measurement exceeds `inputWall` or measurement is unavailable; `"admit"` otherwise.
43
+ - `responseMax`: the flexed wire output allowance (`min(maxOutputTokens, contextWindow - prompt.tokens - 256)` bounded below by `outputFloor` for exact measurements, or bounded above by `outputBudget` for estimates).
47
44
 
48
- <!-- map -->
49
- | Tag | Promise |
50
- | --- | --- |
51
- | {§provider-interface} | `Provider` exposes immutable model facts and one generation operation: |
52
- | {§provider-prompt-measurement} | `PromptTokenMeasurement` is a discriminated request-level result: |
53
- | {§provider-capacity-admission} | `assessRequestCapacity` intersects every known physical input constraint |
54
- | {§provider-monetary-evidence} | One precedence path converts each physical provider request into {§provider-cost} evidence before the request… |
55
- | {§provider-cache-identity} | `generate` requires a non-empty, stable, opaque `workerId`. |
56
- | {§provider-call-kind} | `callKind` is either `emission` (the response is a PLURNK turn emission) or `bare` (the response is… |
57
- | {§provider-native-tools-disabled} | Generation requests declare no native tools and select the SDK's portable `none` tool choice, for emissions… |
58
- | {§provider-generation-completion} | Stream consumption follows the provider's completion boundary, subject to caller cancellation and configured… |
59
- | {§provider-request-observer} | When a consumer supplies the request observer, the adapter opens one durable identity through it immediately… |
60
- | {§provider-dispatched-request} | With `PLURNK_PROVIDERS_RAWBODY` enabled, installed SDK transports also retain the exact serialized generation… |
61
- | {§provider-reasoning-observer} | When a consumer supplies `observeReasoning`, the provider synchronously delivers each exact, ordered,… |
62
- | {§provider-finish-reason} | The finish reason is the AI SDK's unified classification |
63
- | {§provider-tagged-reasoning} | Structured provider or SDK reasoning fields are authoritative. |
64
- <!-- /map -->
45
+ ## §provider-monetary-evidence Usage and monetary accounting
65
46
 
66
- §provider-interface `Provider` exposes immutable model facts and one generation
67
- operation:
47
+ Every physical attempt settles a `ProviderRequestAccounting` record (`{ provider, model, outcome, status?, usage?, cost }` under {§provider-request-accounting}):
68
48
 
69
- ```ts
70
- interface Provider {
71
- readonly model: string;
72
- readonly contextWindow: number | null;
73
- readonly maxInputTokens: number | null;
74
- readonly maxOutputTokens: number | null;
75
- readonly outputBudget: number | null;
76
- readonly reasoningBudget: number | null;
77
- readonly inputCapacity: number | null;
78
- readonly servedModel?: string;
79
- readonly constrainsOutput?: boolean;
80
- readonly requiresOutputBudget?: boolean;
49
+ - `normalizeUsage` validates non-negative safe integers, recovers additive reasoning when `prompt + completion + reasoning === total`, derives a single missing input cache partition (`noCacheTokens`, `cacheReadTokens`, `cacheWriteTokens`) when the other two and total are known, and rejects contradictory counters (`UsageDetailError`) while preserving valid outer totals.
50
+ - Cost precedence resolves in order:
51
+ 1. Direct provider charge evidence (`kind: "charged"`, e.g. OpenRouter `usage.cost` and BYOK `upstream_inference_cost`).
52
+ 2. Direct provider response estimate (`kind: "estimated"`, e.g. DeepInfra `usage.estimated_cost`).
53
+ 3. Operator rate override (`PLURNK_PROVIDERS_COST=input=...,output=...,reasoning=...,cacheRead=...,cacheWrite=...`) over catalog rates (`kind: "estimated"`).
54
+ 4. Catalog rates (`kind: "estimated"`). When a model has a distinct rate for a token category (`reasoning`, `cacheRead`, `cacheWrite`) that the response omitted, cost resolves to `kind: "unknown"` rather than guessing.
55
+ - `aggregateProviderAccounting` sums `usage` and `costUsd` across requests, returning `null` for `usage` or `costUsd` when any request's usage or cost is unknown while preserving `knownUsage` and `knownCostUsd` subtotals.
81
56
 
82
- countPromptTokens(
83
- messages: readonly ChatMessage[],
84
- signal?: AbortSignal,
85
- ): Promise<PromptTokenMeasurement>;
86
- assessRequestCapacity(
87
- messages: readonly ChatMessage[],
88
- maxOutputTokens?: number,
89
- signal?: AbortSignal,
90
- ): Promise<ProviderRequestCapacity>;
91
- tokenize?(text: string): Promise<number[]>;
92
- generate(args: GenerateArgs): Promise<ProviderResponse>;
93
- }
94
- ```
57
+ ## §provider-sdk-boundary SDK and transport boundary
95
58
 
96
- `contextWindow` is the effective total context envelope resolved under
97
- {§model-fact-resolution}: the minimum of known model capacity and any stricter
98
- operator cap. `null` means genuinely unknown; a consumer MUST NOT invent a
99
- stand-in. The context-window knob is a hard cap, never a model-facing curation
100
- pressure.
59
+ `ModelProvider.ts` delegates streaming, protocol framing, and vendor request assembly to `@earendil-works/pi-ai` (`@earendil-works/pi-ai/api/*` `stream`) across all supported APIs (`anthropic-messages`, `openai-completions`, `openai-responses`, `google-generative-ai`, `bedrock-converse-stream`):
101
60
 
102
- §provider-prompt-measurement `PromptTokenMeasurement` is a discriminated
103
- request-level result:
61
+ - Prompt cache affinity (`PLURNK_PROVIDERS_CACHE_AFFINITY=1`) projects each provider's documented session key (`sessionId` -> `prompt_cache_key` / `x-session-id` / `x-session-affinity`).
62
+ - Stable-system cache write policy (`PLURNK_PROVIDERS_CACHE_WRITE_POLICY=stable-system`) attaches ephemeral cache retention (`cache_control: { type: "ephemeral" }`) on providers that support prompt cache breakpoints (Anthropic and Anthropic routes on OpenRouter).
104
63
 
105
- | `kind` | Meaning | Capacity authority |
106
- | --- | --- | --- |
107
- | `exact` | Exact count for the complete provider request. | May prove fit or overflow. |
108
- | `upper_bound` | Proven upper bound for the complete provider request. | May prove fit; exceeding a limit does not prove overflow. |
109
- | `estimate` | Empirical prediction with required causal `detail`. | Cannot admit or reject. |
110
- | `unavailable` | No quantified measurement, with required causal `detail`. | Cannot admit or reject. |
64
+ ## §provider-effort Reasoning effort and readable trace
111
65
 
112
- Every result carries a non-empty `source`; quantified kinds carry non-negative
113
- integer `tokens`.
114
- `countPromptTokens` receives the same messages supplied to `generate` and may
115
- perform cancellable provider I/O. The common fallback is chars/2 over message
116
- content; it is announced once and reported honestly as an estimate because it
117
- knows neither the serving vocabulary nor provider-owned request framing.
118
- An adapter may retain that estimate when an optional counting endpoint fails,
119
- provided its detail names the cause; one unable to quantify anything returns
120
- `unavailable`. A malformed measurement is a provider contract violation and
121
- fails hard.
66
+ `PLURNK_PROVIDERS_EFFORT` selects one portable policy (`off`, `adaptive`, `low`, `medium`, `high`, `xhigh`, `max`, {§capability-policy-cascade}):
122
67
 
123
- §provider-capacity-admission **`assessRequestCapacity` intersects every known physical input constraint** — independent `maxInputTokens` and `contextWindow - outputFloor`, the input wall ({§provider-output-floor}) — and answers `admit`, `reject`, or `defer`, retaining the complete limit and measurement evidence. A consumer may assess a request before sending it; `generate` repeats the assessment as its own guard. Consumers may display or use that fact as policy, but MUST NOT substitute their own content heuristic for request-shaped admission.
68
+ - `catalogEfforts(providerInfo, modelInfo, env)` intersects the model's cataloged effort vocabulary (extended by `PLURNK_PROVIDERS_PROVIDER_<ID>_REASONING_EFFORTS` for reasoning models) with what the provider adapter can preserve on the wire.
69
+ - On `adaptive`, routes with native adaptive reasoning (such as Anthropic adaptive thinking or Gemini thinking) request their native adaptive posture; graded effort routes use `PLURNK_PROVIDERS_EFFORT_FALLBACK` (default `high`) when supported by the model's effort set.
70
+ - When `PLURNK_PROVIDERS_REASONING_BUDGET` is set alongside `adaptive`, budget-capable routes send the token budget (`thinkingBudgets` / `thinking.budget_tokens` / `thinkingConfig.thinkingBudget` / `reasoning.max_tokens`); setting a fixed effort alongside `PLURNK_PROVIDERS_REASONING_BUDGET` is rejected at construction. On Google routes where `maxOutputTokens` is shared across `thoughtsTokenCount` and `candidatesTokenCount`, `thinkingConfig.thinkingBudget` is bounded by `maxOutputTokens - outputFloor` (or `maxOutputTokens - 1` when tightened below `outputFloor`) so visible output headroom is reserved.
71
+ - On detected llama-server routes, `off` and `adaptive` send `chat_template_kwargs: { enable_thinking: false | true }`, and fixed efforts declared in `PLURNK_PROVIDERS_REASONING_EFFORTS` add `reasoning_effort`. When grammar constrains a template-reasoning turn, the adapter sends `reasoning_format: "none"` so the grammar governs the raw stream and projects the leading template block back into `assistant.reasoning`.
124
72
 
125
- | Measurement | Verdict |
126
- | --- | --- |
127
- | Exact fit | admit |
128
- | Exact overflow of the wall | reject |
129
- | A prompt between the curation reservation and the wall | admitted with its grant flexed down |
130
- | A proven upper bound | admits only when it fits |
131
- | Unknown limits, an upper bound above a limit, estimates, and unavailable measurements | defer to the upstream provider as capacity oracle |
73
+ ## §provider-configuration Environment configuration and envelope
132
74
 
133
- | Exposed | Meaning |
134
- | --- | --- |
135
- | `inputCapacity` | the curation reservation's intersection, `maxInputTokens` and `contextWindow - outputBudget`: the line Core packs the room against |
136
- | `inputWall` | the wall's; `null` means the available limits cannot establish one; a known combined context and output budget must leave positive input capacity |
75
+ | Knob | Default / Form | Effect |
76
+ |---|---|---|
77
+ | `PLURNK_PROVIDERS_OUTPUT_BUDGET` | `25%` (`<pct>%` or positive integer tokens) | Soft output reservation subtracted from `contextWindow` to form `inputCapacity` |
78
+ | `PLURNK_PROVIDERS_OUTPUT_FLOOR` | `25%` (`<pct>%` of effective output ceiling or integer tokens) | Hard minimum output allowance subtracted from `contextWindow` to form `inputWall` |
79
+ | `PLURNK_PROVIDERS_REASONING_BUDGET` | empty (`<pct>%` or integer tokens, `< outputBudget`) | Explicit thinking token budget under `adaptive` effort |
80
+ | `PLURNK_PROVIDERS_EFFORT` / `EFFORT_FALLBACK` | `adaptive` / `high` | Portable reasoning effort selector and graded-route fallback |
81
+ | `PLURNK_PROVIDERS_FETCH_TIMEOUT` / `OPERATION_TIMEOUT` | `30000` / `600000` ms | Per-attempt fetch/idle timeout and total operation wall-clock timeout |
82
+ | `PLURNK_PROVIDERS_RETRY_ATTEMPTS` | `3` | Maximum automatic retry attempts for retryable transport failures |
83
+ | `PLURNK_PROVIDERS_MAX_CONCURRENCY` | `-1` (unbounded or positive integer) | Per-endpoint FIFO concurrency gate ({§provider-inference-admission}) |
84
+ | `PLURNK_PROVIDERS_RAWBODY` | `0` (`0 \| 1`) | Retains redacted raw request/response forensic bodies in `forensicEvidence` |
137
85
 
138
- `tokenize` is the separate content-token capability and exists only when the
139
- endpoint exposes its real vocabulary. Content tokenization does not substitute
140
- for complete-request measurement.
86
+ - Every knob in `PROVIDERS_KNOBS` is documented in `plurnk-providers/.env.defaults` and supports case-insensitive per-alias suffix scoping (`<KNOB>_<ALIAS>`) via `scopeEnvToAlias(env, alias)`.
87
+ - Sampling knobs (`TEMPERATURE`, `TOP_P`, `TOP_K`, `PRESENCE_PENALTY`, `FREQUENCY_PENALTY`, `SEED`, `REPEAT_PENALTY`, `REPEAT_LAST_N`, `DRY_MULTIPLIER`, `DRY_BASE`, `DRY_ALLOWED_LENGTH`) validate at construction; caller `sampling` overrides configured defaults on a per-call basis, and `DRY_MULTIPLIER=0` suppresses all DRY fields.
88
+ - `instantiateProvider` resolves cataloged providers and `ollama` in-framework before scanning `node_modules` for third-party provider extensions (gated by `PLURNK_EXTENSIONS_TRUSTED_ONLY`). `providerReadiness(provider, env)` reports missing credentials or endpoint configuration without exposing secret values.
141
89
 
142
- §provider-monetary-evidence One precedence path converts each physical provider
143
- request into {§provider-cost} evidence before the request leaves the provider
144
- boundary:
90
+ ## §model-fact-resolution Route facts and local probes
145
91
 
146
- | Precedence | Evidence | Result |
147
- | --- | --- | --- |
148
- | 1 | A documented monetary field on that response or error | The adapter validates and preserves its documented `charged` or `estimated` character; its exact amount wins. |
149
- | 2 | Known response usage and the exact model's Models.dev rates | The adapter returns an exact decimal USD `estimated` amount only when every differently-priced applicable category is known. |
150
- | 3 | Neither | `unknown` with a concrete reason. |
92
+ | Route family | Fact authority |
93
+ |---|---|
94
+ | Cataloged cloud provider (`catalogProvider.ts`) | `resolveModel(provider, model)` from `@plurnk/plurnk-models`, capped by optional `PLURNK_PROVIDERS_CONTEXT_WINDOW`. Uncataloged models on a catalog provider require an explicit `PLURNK_PROVIDERS_CONTEXT_WINDOW`. Catalog credential names (`info.env`) or `PLURNK_PROVIDERS_PROVIDER_<ID>_API_KEY_ENV` (one exact variable name, never a comma list) supply authentication. |
95
+ | Local / compatible endpoint (`compatibleProvider.ts`) | Probes `GET <baseUrl>/models` for `n_ctx` / `max_model_len` / `context_length`. When llama-server is detected (`meta.n_ctx` or `PLURNK_PROVIDERS_LLAMA_SERVER=1`), probes `/props` for `total_slots` and `eos_token` and measures prompts via `/v1/chat/completions/input_tokens`. |
96
+ | Built-in Ollama rail (`ollama.ts`) | Probes `POST <origin>/api/show` for `*.context_length`, capped by optional `PLURNK_PROVIDERS_CONTEXT_WINDOW`, and dispatches chat completions to `<origin>/v1/chat/completions`. |
97
+ | Operator probe scripts (`scripts/`) | `providers-ping.mjs` probes one cheapest declared route per keyed provider and writes redacted JSON evidence; `token-envelope-probe.mjs` probes explicit aliases across configured or physical output limits. |
151
98
 
152
- Models.dev is the sole supported fallback rate table. Missing usage, a missing
153
- applicable category, or missing rates never proves zero. An exact zero rate
154
- produces an ordinary estimated amount of USD `0`. Rate calculation is internal
155
- to the provider request; Core, digest, ping, and clients never call a parallel
156
- pricing method.
99
+ ## §provider-request-authority Request controls and grammar transport
157
100
 
158
- Router accounting uses the complete cost to the caller, not just the router's
159
- fee. OpenRouter's response `usage` is authoritative; its SDK metadata projection
160
- omits the BYOK discriminator. Apply the same rule to successful and failed
161
- requests, independently of raw-body capture:
101
+ - Caller `sampling` and extension body fields cannot override protocol-owned fields (`model`, `messages`, `stream`, `tools`, `tool_choice`, `response_format`, `grammar`, `logprobs`, `top_logprobs`).
102
+ - `PLURNK_PROVIDERS_SERVICE_TIER` and `PLURNK_PROVIDERS_TOP_LOGPROBS` project onto native SDK requests only when the route's SDK documents the control; unsupported controls emit a single `PLURNK_REQUEST_CONTROL_UNSUPPORTED` process warning and are omitted from the wire.
103
+ - OpenRouter requests attach `HTTP-Referer` and `X-OpenRouter-Title` from `PLURNK_PROVIDERS_PROVIDER_OPENROUTER_APP_URL` (validated as an absolute HTTP(S) URL) and `PLURNK_PROVIDERS_PROVIDER_OPENROUTER_APP_NAME`.
104
+ - Grammar strings (`generate({ grammar })`) are transported only when `grammarStyle === "llamacpp"` (`constrainsOutput === true`), preserving `grammarEvidence` (`{ input, contentStart, transported }`) on the response. When `tokenize` is available on a grammar turn and billed `outputTokens` exceed visible content + reasoning tokens by more than 64, an `output_unaccounted` notice is attached ({§diagnostic-observation}).
105
+
106
+ ## §provider-failure-normalization Failure normalization and evidence
107
+
108
+ All transport and response failures normalize into `ProviderError` carrying an RFC 9457 Problem Details payload (`https://problems.plurnk.xyz/provider/<providerId>/<kind>`, {§diagnostic-observation}):
109
+
110
+ | `ProviderErrorKind` | HTTP / trigger | Problem status | `retryable` | Extensions |
111
+ |---|---|---|---|---|
112
+ | `unauthorized` | 401, 403 | 401 / 403 | `false` | `cause` |
113
+ | `quota_exceeded` | 402 or billing/quota exhaustion | 402 | `false` | `cause` |
114
+ | `rate_limit` | 429 | 429 | `true` | `cause` |
115
+ | `capacity_exceeded` | Preflight exact wall overflow, 413, or upstream context-length code | 413 | `false` | `cause` |
116
+ | `grammar_invalid` | 422 with `error.type === "grammar_invalid"` | 422 | `false` | `cause` |
117
+ | `request_rejected` | Other 400, 404, 405, 422 | 400..422 | `false` | `cause` |
118
+ | `network_failure` | 408, 409, 5xx, socket/TLS/proxy error, or mid-stream disconnect | 502 / 503 | `true` | `cause`, optional `attempt` |
119
+ | `deadline_exceeded` | `PLURNK_PROVIDERS_FETCH_TIMEOUT` or `PLURNK_PROVIDERS_OPERATION_TIMEOUT` | 504 | `true` | `cause`, optional `attempt` |
120
+ | `resource_interrupted` | `finishReason === "resource_interrupted"` | 503 | `true` | `attempt` |
121
+ | `generation_failed` | `finishReason === "error"` | 502 | `false` | `attempt` |
122
+ | `invalid_response` | Malformed 2xx payload, missing finish reason, invalid usage, or output exceeding granted allowance | 502 | `false` | `cause`, optional `attempt` |
123
+ | `model_refused` | Upstream safety/content refusal finish reason | 422 | `false` | `attempt` |
162
124
 
163
- | OpenRouter evidence | Monetary result |
164
- | --- | --- |
165
- | `is_byok: false` and `cost` | `charged`: `cost`; upstream inference is already included. |
166
- | `is_byok: true`, `cost`, and `cost_details.upstream_inference_cost` | `charged`: exact decimal sum of router fee and upstream charge. |
167
- | A monetary field without a BYOK discriminator, or BYOK missing either component | `unknown`; neither zero nor catalog pricing completes the reported charge. |
168
- | No reported charge and no BYOK indication | Ordinary Models.dev fallback above. |
169
-
170
- Explicit zero components are valid. Malformed monetary values are contract
171
- violations, never coerced. Raw routing and billing evidence remain available
172
- under {§provider-evidence}; Core and clients consume the existing single cost.
173
- See [OpenRouter usage accounting](https://openrouter.ai/docs/cookbook/administration/usage-accounting).
174
-
175
- ### Generation
176
-
177
- §provider-cache-identity `generate` requires a non-empty, stable, opaque
178
- `workerId`. A durable worker uses one globally unique value for its lifetime;
179
- independent databases and processes cannot mint the same local sequence. A
180
- `bare` call instead uses a fresh per-call value, preventing unrelated prompts
181
- from acquiring affinity with either the parent worker or another BARE call.
182
- Providers MUST NOT interpret either value.
183
-
184
- `generate` accepts:
185
-
186
- - `messages`: system, user, and assistant text messages;
187
- - caller cancellation through `signal`;
188
- - optional `grammar` and call-specific `maxOutputTokens` tightening;
189
- - standard `sampling` intent;
190
- - the caller-owned `callKind` output contract when one applies;
191
- - opaque attribution tags plus client, strike, workspace, loop, and turn metadata.
192
-
193
- §provider-call-kind `callKind` is either `emission` (the response is a PLURNK
194
- turn emission) or `bare` (the response is unconstrained answer text). The
195
- consumer states this semantic fact explicitly; providers MUST NOT infer it from
196
- message count, grammar presence, worker identity, or another incidental request
197
- shape. The signal never enters model-facing messages. Generic
198
- provider callers MAY omit it; Core supplies it for every model call.
199
-
200
- §provider-native-tools-disabled Generation requests declare no native tools and
201
- select the SDK's portable `none` tool choice, for emissions and BARE alike.
202
- The built-in transports preserve that intent after SDK serialization:
203
-
204
- | Protocol | No-tool wire representation |
205
- | --- | --- |
206
- | OpenAI-compatible Chat Completions and Responses | `tools: []`, `tool_choice: "none"` |
207
- | Anthropic Messages | `tools: []`, `tool_choice: {"type":"none"}` |
208
- | Gemini GenerateContent | `tools: []`, `toolConfig: {"functionCallingConfig":{"mode":"NONE"}}` |
209
- | Bedrock Converse | No tool configuration; the protocol has no `none` choice. |
210
- | Installed SDK provider extensions | Portable `none` intent; the extension owns its wire projection. |
211
-
212
- This is a text-generation contract, not provider/model tuning or prompt
213
- teaching. Callers cannot supply tools or override the choice through request
214
- extensions. Do not add deprecated function-call fields, dummy tools, or fields
215
- from another protocol. The opt-out neither constrains arbitrary prose nor
216
- proves that an endpoint honors it; unexpected native calls retain their full
217
- evidence and follow {§provider-native-tool-calls}.
218
-
219
- A successful return carries the model's raw content and reasoning, normalized
220
- finish reason, model identity, its ordered {§provider-request-accounting}, the
221
- request's `ProviderRequestCapacity`, opaque evidence, optional metadata, and
222
- optional notices. A `ProviderError` carries the same available capacity and
223
- accounting evidence. The provider transports and observes model
224
- output; it never retries, discards, or repairs an otherwise completed exchange
225
- because PLURNK grammar did not accept it.
226
-
227
- §provider-generation-completion Stream consumption follows the provider's
228
- completion boundary, subject to caller cancellation and configured deadlines.
229
- Repeated content or reasoning does not interrupt a response or request another
230
- generation. Preserve the complete response, its finish reason, and trailing
231
- usage and charge evidence before returning it to the consumer.
232
-
233
- §provider-request-observer When a consumer supplies the request observer, the
234
- adapter opens one durable identity through it immediately before each physical
235
- I/O and settles that identity with the resulting
236
- `ProviderRequestAccounting`. This applies to generation retries and capacity
237
- failover. The
238
- observer is a durability sink, not an alternate evidence representation; the
239
- same ordered records remain on the final generation result or
240
- error.
241
-
242
- The adapter also settles {§provider-request-evidence}. A successful request
243
- retains its normalized transport response; a failed request retains received
244
- SDK raw chunks (or the error response body), partial content and reasoning,
245
- available response identity/headers, usage/charge evidence, and its error and
246
- cancellation cause. Failure capture is independent of successful raw-body
247
- capture. These are the SDK's received values, not a claim of byte-for-byte
248
- HTTP capture. Settlement is durable before returning the failure; a process
249
- crash before settlement leaves a pending request, not an invented response.
250
-
251
- §provider-dispatched-request With `PLURNK_PROVIDERS_RAWBODY` enabled, installed
252
- SDK transports also retain the exact serialized generation request body at
253
- their injected fetch boundary, after all owned body transformations. Capture
254
- is scoped to the physical request and settles with its existing forensic
255
- evidence on success or failure; it does not reconstruct a body from settings.
256
- Only the HTTP method, origin and body are recorded: no request headers,
257
- userinfo, path, query or fragment. The body itself contains the supplied model
258
- input and must be treated as sensitive evidence. An external SDK model whose
259
- transport is not instrumented, disabled capture, or an unsettled request makes
260
- no capture claim. Existing accounting, failure evidence, cancellation and
261
- retention are unchanged.
262
-
263
- Native SDK construction binds each physical request's injected fetch after
264
- owned body transforms; compatible routes use the same request-scoped closure.
265
- Capture adds no global interception, runtime-specific context storage or
266
- cross-request mutable recorder. An externally supplied model instance remains
267
- opaque; a model factory can accept the injected fetch to participate.
268
-
269
- §provider-reasoning-observer When a consumer supplies `observeReasoning`, the
270
- provider synchronously delivers each exact, ordered, nonempty readable-reasoning
271
- delta as it becomes available. A transport without incremental reasoning emits
272
- the complete normalized value once before resolving. This is transient
273
- observation, not response authority: `ProviderResponse` or `ProviderError`
274
- remains the complete settled evidence. An automatically retried physical
275
- request may therefore expose a partial failed-attempt prefix before a later
276
- attempt settles. The consumer uses each `observeRequest` opening as the boundary
277
- between those physical-request reasoning streams: it preserves the failed
278
- prefix, gives the retry a distinct presentation identity, and never concatenates
279
- separate stochastic attempts into one reasoning message.
280
-
281
- Usage obeys {§provider-usage}:
282
-
283
- ```text
284
- totalTokens = inputTokens + outputTokens
285
- cacheReadTokens, cacheWriteTokens ⊆ inputTokens
286
- reasoningTokens ⊆ outputTokens
287
- ```
288
-
289
- Unknown fields remain absent.
290
-
291
- §provider-finish-reason **The finish reason is the AI SDK's unified classification**, never a
292
- second reading of the vendor's string: `stop`, `length`,
293
- `tool-calls` → `tool_calls`, `content-filter` → `content_filter`; `error` fails the attempt
294
- ({§provider-generation-failed}); `other` is `null`. The raw vendor value rides beside it as
295
- `rawFinishReason`. One upstream semantic the SDK folds into `other` stays distinct: the exact
296
- `insufficient_system_resource` is `resource_interrupted`, the failed-attempt disposition defined by
297
- {§provider-interrupted-attempt}.
298
-
299
- ### Tagged reasoning responses
300
-
301
- §provider-tagged-reasoning Structured provider or SDK reasoning fields are
302
- authoritative. Visible content is interpreted as tagged reasoning only under an
303
- explicit alias-scoped response style:
304
-
305
- | Effective style | Leading content | Normalized result |
306
- | --------------- | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
307
- | `verbatim` | Any bytes. | Content remains exact; only structured reasoning fields populate `reasoning`. |
308
- | `think-tags` | No exact leading `<think>`. | Content remains exact. |
309
- | `think-tags` | `<think>reasoning</think>visible`. | The first envelope body becomes reasoning; the exact suffix becomes content. Later tags remain literal. |
310
- | `think-tags` | `<think>reasoning` with no close, including a capped turn. | The complete post-open tail becomes reasoning; content is empty. |
311
-
312
- Tag projection never runs when readable structured reasoning is already
313
- present. Streamed and buffered transports use the same leading-envelope parser.
314
- An enabled tagged projection delivers readable deltas as content chunks arrive,
315
- holding only incomplete delimiter prefixes; it does not wait for the closing
316
- tag or response completion. An unfinished closing prefix is literal reasoning
317
- at EOF. Constrained template calls that retain the grammar sentence use the
318
- same path for `<think>\n…</think>` and `<|channel>thought\n…<channel|>`.
319
- Observation resets for each physical request and completed reasoning is not
320
- replayed. Neither the internal working copy nor its curation supplies this stream.
321
- When the upstream reports only combined output usage, that value remains
322
- `outputTokens` and its unavailable text/reasoning detail stays absent. The
323
- adapter never apportions tokens from character lengths.
324
-
325
- Grammar evidence retains the exact pre-projection sentence and its Unicode
326
- content offset. Response classification cannot rewrite what a transported GBNF
327
- rail observed.
328
-
329
- ## §3 AI SDK boundary
330
-
331
- <!-- map -->
332
- | Tag | Promise |
333
- | --- | --- |
334
- | {§provider-sdk-boundary} | Cataloged providers instantiate their Models.dev-declared AI SDK package. |
335
- | {§operator-cost-override} | Models.dev is the rate starting point; the operator may overlay it. |
336
- | {§provider-effort} | The portable vocabulary comes from {§effort-wire}. |
337
- | {§provider-readable-reasoning} | Route-owned options may request exposed reasoning text. |
338
- | {§provider-sdk-warning} | AI SDK compatibility, unsupported-feature, deprecation, and other call warnings become source-attributed… |
339
- | {§provider-cache-affinity} | Cache affinity is route-owned request projection. |
340
- | {§provider-model-options} | A model family's native options are a provider declaration. |
341
- | {§provider-cache-write-policy} | Cache-write policy is separate from affinity. |
342
- | {§deepseek-reasoning-request} | Direct DeepSeek uses the panel's request-field declarations under {§provider-wire-declaration}. |
343
- | {§provider-wire-declaration} | Request-field declarations |
344
- <!-- /map -->
345
-
346
- §provider-sdk-boundary Cataloged providers instantiate their
347
- Models.dev-declared AI SDK package.
348
- Standard request shaping, streaming, usage, and vendor error parsing belong to
349
- the SDK. PLURNK supplies cancellation and deadline signals and owns the sole
350
- cross-attempt scheduler so every physical request remains observable and
351
- accountable. The adapter hands the SDK the conversation as authored: a leading
352
- system message reaches the wire as the request's system message.
353
-
354
- | Usage input | Normalization |
355
- | --- | --- |
356
- | Native protocol | Preserve the SDK's inclusive totals and category counts; raw protocol counters are evidence, not interchangeable totals. |
357
- | OpenAI-compatible wire | Retain extended counters and exact total identities the SDK may omit. |
358
- | Compatible SDK input partition | Retain its uncached count when total input and an explicitly reported cache-read count agree with the wire; never promote SDK defaults over absent or contradictory wire counts. |
359
- | Input total plus two partition counts | Derive the remaining count exactly; otherwise preserve absence. |
360
-
361
- PLURNK maps its generic settings to AI SDK call settings:
362
-
363
- - `temperature`, `top_p`, `top_k`;
364
- - presence and frequency penalties;
365
- - stop sequences and seed;
366
- - output-token ceiling;
367
- - the supported efforts and optional numeric control under
368
- {§provider-effort}.
369
-
370
- Provider-specific options are permitted only where they preserve a documented
371
- PLURNK product contract the generic SDK surface cannot express.
372
-
373
- §operator-cost-override **Models.dev is the rate starting point; the operator may
374
- overlay it.** `PLURNK_PROVIDERS_COST` declares exact per-1M-token USD rates as
375
- comma-separated `key=value` over the catalog vocabulary (`input`, `output`,
376
- `reasoning`, `cacheRead`, `cacheWrite`), alias-scoped like every provider knob.
377
- Declared keys merge over the Models.dev cost block; undeclared keys keep the
378
- catalog figure. Without catalog rates the override must declare `input` and
379
- `output`. The estimate's `source` names the override, and a provider-reported
380
- response cost still outranks any estimate. Unknown keys, repeats, and negative
381
- or non-numeric rates refuse at construction.
382
-
383
- §provider-effort The portable vocabulary comes from
384
- {§effort-wire}. `adaptive` uses the first applicable projection:
385
-
386
- | Condition | Projection |
387
- | --- | --- |
388
- | Explicit adaptive declaration: `REASONING_ADAPTIVE_BODY`, or a native model family's `ADAPTIVE_OPTIONS` ({§provider-model-options}) | Preserve that mechanism; an explicit `{}` retains the enabled endpoint's default. |
389
- | The configured `PLURNK_PROVIDERS_EFFORT_FALLBACK` is supported by both model and transport | Send that exact effort. The shipped fallback is `high`, not the strongest available level. |
390
- | No supported fallback, including an empty fallback setting | Retain reasoning activation and the provider's default effort; never invent a level or escalate to another one. |
391
-
392
- Activation is distinct from effort: declared enable fields accompany active
393
- reasoning, and a toggle-only route uses its documented enable mechanism.
394
- Non-reasoning models receive no reasoning controls; `off` is the explicit opt-out.
395
- A fixed policy retains its exact name and is rejected before provider I/O when
396
- either the route or transport cannot represent it. Every provider exposes that
397
- exact intersection. A numeric reasoning budget constrains
398
- the generation envelope independently and never selects or changes policy. On
399
- routes whose controls are exclusive, fixed effort plus a numeric budget is
400
- rejected before I/O. Under `adaptive`, an explicit budget selects the numeric
401
- control; `off` suppresses it.
402
-
403
- `catalogEfforts` projects that same admission calculation from catalog
404
- facts and provider-wide environment declarations over the installed defaults,
405
- without constructing a model, requiring credentials, or performing provider I/O.
406
- Admission respects the installed projection: native SDK fixed efforts exclude
407
- `max`; declared transport vocabularies constrain per-call option projections.
408
-
409
- Models.dev's route-specific `reasoning_options` is the capability authority for
410
- what the daemon offers on its own; the daemon never adds vendor behaviour absent
411
- from the catalog. The operator may declare efforts a provider's reasoning routes
412
- accept (`PLURNK_PROVIDERS_PROVIDER_<PREFIX>_REASONING_EFFORTS`, comma-separated
413
- portable names); declared efforts join the catalog's for policy admission and the
414
- `adaptive` projection under the same transport rule, a declared `none` admits
415
- `off` where the transport can express it, and a host that rejects a declared
416
- effort answers on the first request rather than at boot. The declaration never
417
- turns a non-reasoning route into a reasoning one. The installed AI SDK or
418
- explicit compatible adapter owns the wire projection:
419
-
420
- | Catalog fact | `adaptive` projection | Portable policies admitted in addition to `adaptive` |
421
- | --- | --- | --- |
422
- | `reasoning: false` | No reasoning request | `off` |
423
- | `reasoning_options: []` | Provider default | None |
424
- | `effort.values` | Native dynamic mechanism, otherwise the supported configured fallback or provider default | Transportable fixed members of {§effort-wire}; `off` only when `none` is transportable |
425
- | `toggle` | Native or explicitly declared activation, otherwise provider default | `off` only when that transport owns the toggle wire |
426
- | `budget_tokens` | Does not select policy | None; an adapter may use its bounds when projecting the independent budget |
427
- | No catalog entry | Explicit adapter declaration | Only the declaration's exact subset |
428
- | Operator-declared efforts | Join the catalog's `effort.values` for that provider's reasoning routes | Their exact members, under the transport rule above |
429
-
430
- Models.dev identifies the route's controls but not a provider-specific toggle
431
- or budget field name. The adapter supplies that last-mile mechanism; it never
432
- invents a cataloged effort value.
433
-
434
- §provider-readable-reasoning Route-owned options may request exposed reasoning
435
- text. Normalization follows {§provider-open-reasoning}; Models.dev's reasoning
436
- bit is capability metadata, not a statement about returned text's fidelity.
437
- Billing without exposed text does not create a reasoning source.
438
-
439
- §provider-sdk-warning AI SDK compatibility, unsupported-feature, deprecation,
440
- and other call warnings become source-attributed provider Notices on the
441
- successful exchange. A lossy adapter projection is therefore observable rather
442
- than disappearing in transport internals.
443
-
444
- §provider-cache-affinity **Cache affinity is route-owned request projection.**
445
- When a provider documents a semantics-preserving conversation, session, or
446
- prompt-cache routing key, its catalog adapter projects `workerId` through that
447
- provider's documented header, body field, or native SDK option. The common
448
- transport neither guesses from protocol resemblance nor sends a generic cache
449
- field to an unknown provider. The operator may disable affinity globally or per
450
- alias; automatic provider caching without an affinity control remains untouched.
451
- The environment's `CACHE_AFFINITY_FIELD` declares that placement as
452
- `{"target":"header"|"body","name":"…"}` or
453
- `{"target":"provider-option","provider":"…","name":"…"}`. It follows
454
- the provider/route/alias precedence of {§provider-wire-declaration}; `null`
455
- clears the declaration. Placement is independent of the enable switch, cannot
456
- replace transport-owned fields, and does not imply support from a provider's name.
457
-
458
- §provider-model-options **A model family's native options are a provider declaration.** Models.dev
459
- does not say which models on a native SDK take adaptive reasoning or explicit cache writes, and the SDKs
460
- do not export their tables, so a provider declares them as data: `ADAPTIVE_OPTIONS` and
461
- `SYSTEM_CACHE_OPTIONS` are JSON arrays of `{"models":["<glob>",…],"options":{…}}` rules. The first rule
462
- with a glob matching the route's model id (`path.matchesGlob`) supplies its per-call provider options;
463
- without a match none apply. They follow the provider/route/alias precedence of
464
- {§provider-wire-declaration}, an empty value declares none, and a malformed value fails construction.
465
- None ships: Plurnk's shipped defaults serve open models only, and an operator who routes a closed model
466
- family declares its options in their own configuration.
467
-
468
- §provider-cache-write-policy **Cache-write policy is separate from affinity.**
469
- `PLURNK_PROVIDERS_CACHE_WRITE_POLICY` is `off` or `stable-system`. The latter
470
- marks only the final leading system instruction as an explicit reusable cache
471
- boundary, and only on routes whose provider declares the control in
472
- `SYSTEM_CACHE_OPTIONS` ({§provider-model-options}). It does
473
- not mark the changing user packet or enable an API-wide automatic cache mode.
474
- Unsupported routes receive no invented option. The default five-minute
475
- provider lifetime is used; a longer, differently priced lifetime is not an
476
- implicit transport choice.
477
-
478
- §deepseek-reasoning-request Direct DeepSeek uses the panel's request-field
479
- declarations under {§provider-wire-declaration}. Models.dev supplies each
480
- model's effort vocabulary; it is not a second fixed list in this adapter.
481
-
482
- The compatible transport is deliberately retained for:
483
-
484
- - `openai` local endpoints, including llama-server and vLLM;
485
- - `ollama`, after its native `/api/show` probe;
486
- - operator-declared `@ai-sdk/openai-compatible` providers.
487
-
488
- It carries PLURNK-only fields and raw wire evidence without reimplementing the
489
- SDK's ordinary transport.
490
-
491
- ### §provider-wire-declaration Request-field declarations
492
-
493
- Models.dev owns model capabilities, effort vocabulary, and numeric budget bounds.
494
- For endpoints whose field names it does not describe, the provider's
495
- environment panel supplies a bounded projection, independent of provider identity:
496
-
497
- | Declaration suffix | Meaning |
498
- | --- | --- |
499
- | `OPTIONS_NAMESPACE` | Native SDK's per-call `providerOptions` namespace. Without one, declared fields target the compatible request body. |
500
- | `OUTPUT_PATH` | RFC 6901 object-member pointer for the **inclusive** output ceiling; absent uses the compatible SDK's `max_tokens` field. |
501
- | `REASONING_EFFORT_PATH` / `REASONING_BUDGET_PATH` | Pointers for exact effort or numeric reasoning subset. No pointer means no such control. |
502
- | `REASONING_EFFORTS` | Additional declared effort values, unioned with Models.dev. |
503
- | `REASONING_TRANSPORT_EFFORTS` | Optional transport vocabulary, intersected with catalog/declaration efforts. It adds no model capability. |
504
- | `REASONING_CONTROLS` | Required when both pointers exist: `exclusive` refuses fixed effort plus budget; `combined` sends both. |
505
- | `REASONING_ON_BODY` | Static reasoning activation fields, merged with the selected control. |
506
- | `REASONING_OFF_BODY` | Explicit disable fields; otherwise a declared `none` effort can disable. |
507
- | `REASONING_ADAPTIVE_BODY` | Explicit adaptive fields, ahead of graded fallback; `{}` retains the enabled endpoint's default. |
508
- | `REASONING_TOGGLE_BODY` | Reasoning enable fields when Models.dev declares a toggle and neither an explicit adaptive body nor a supported fallback effort applies. |
509
-
510
- The existing provider declaration prefix is
511
- `PLURNK_PROVIDERS_PROVIDER_<NAME>_`; `PLURNK_PROVIDERS_<suffix>` overrides
512
- it for the route and accepts the ordinary alias suffix. These are data, not
513
- executable transformations. Static bodies cannot replace transport, sampling,
514
- or managed numeric controls; pointers cannot overlap. Configuration fails at
515
- construction before inference when a requested policy or numeric control is
516
- unrepresentable. A tighter per-call envelope reprojects the same declaration.
517
- Discovery and generation share the resolved policy set. A cataloged
518
- non-reasoning model receives no reasoning fields.
519
- Native SDKs retain their own output field; `OUTPUT_PATH` is incompatible with an
520
- options namespace. Request-local options are projected after the effective
521
- envelope is known, never frozen into SDK model construction. Streaming and
522
- non-streaming calls use the same projection.
523
-
524
- A native SDK's portable reasoning setting has no `max`, so a native route without
525
- a namespace cannot send an effort the catalog documents beyond it; construction
526
- refuses it and names the declaration that would. Native SDKs built on
527
- openai-compatible (DeepInfra, Together) write `reasoning_effort` from their own
528
- `reasoningEffort` option after spreading the others, overwriting a raw
529
- `reasoning_effort`; their shipped declarations therefore point at
530
- `/reasoningEffort`, with `{"reasoningEffort":"none"}` as the off body.
531
-
532
- ## §4 Operator configuration
533
-
534
- §provider-configuration Every operational value is an environment knob
535
- documented in `.env.defaults`.
536
- There are no hidden tuning constants. Every `PLURNK_PROVIDERS_*` knob may be
537
- scoped to an alias by appending `_<alias>`; the scoped value wins, and the alias
538
- segment folds case. The scoping takes a closed list of knob names: a consumer
539
- scopes its own knobs by passing its list, and neither list reaches the other's
540
- knobs.
541
- Each projection reads one fresh environment snapshot without modifying its
542
- source; later projections observe later configuration, never a cached view.
543
-
544
- The public Node provider registry applies this package's committed
545
- `.env.defaults` as a set-if-unset operational floor. Consumers pass their
546
- operator environment, not a manually composed copy of the provider floor.
547
- Explicit operator values always win and invalid explicit values fail at the
548
- knob's owning contract.
549
-
550
- The universal groups are:
551
-
552
- - reasoning activation and optional explicit budget;
553
- - explicit reasoning response-content style;
554
- - decode tuning;
555
- - whole-call generation, discovery/tokenizer HTTP, retry, and probe budgets;
556
- - local GBNF and llama-server capability pins;
557
- - context-window and generation-envelope overrides;
558
- - provider-documented cache affinity and explicit cache-write policy;
559
- - opt-in logprob and raw-body capture.
560
-
561
- Operator secrets and machine-specific values never belong in committed
562
- defaults.
563
-
564
- ## §5 Resolution
565
-
566
- <!-- map -->
567
- | Tag | Promise |
568
- | --- | --- |
569
- | {§provider-resolution} | `PLURNK_MODEL_<alias>=<provider>/<model-id>` declares an alias. |
570
- | {§model-catalog-readiness} | Catalog readiness and construction share one local configuration predicate. |
571
- | {§provider-input-modalities} | Native input parts |
572
- | {§model-fact-resolution} | Model fact precedence |
573
- | {§provider-fact-authority} | Provider declarations configure facts, not credentials, and Models.dev is authoritative for cataloged… |
574
- <!-- /map -->
575
-
576
- §provider-resolution `PLURNK_MODEL_<alias>=<provider>/<model-id>` declares an
577
- alias.
578
- `PLURNK_MODEL=<selector>` selects either a declared alias or an exact
579
- `<provider>/<model-id>` route. Model IDs may contain `/`; only the first slash
580
- separates provider from model. Exact routes carry no fabricated alias and use
581
- the global provider configuration. Declared aliases retain their provenance,
582
- endpoint override, and alias-scoped tuning.
583
- `PLURNK_BASEURL_<alias>` is a per-alias endpoint override.
584
-
585
- §model-catalog-readiness **Catalog readiness and construction share one local
586
- configuration predicate.** For each Models.dev provider, readiness evaluates
587
- the same effective credential names, endpoint template coordinates, base-URL
588
- precedence, and alternative Bedrock authentication sets used by construction.
589
- It makes no request and validates no credential value. A ready result therefore
590
- means only “configured enough to attempt”; missing causes contain environment
591
- names without values. Construction rejects the same missing requirements at
592
- the provider boundary instead of deferring a known configuration failure to a
593
- model request.
594
-
595
- ### §provider-input-modalities Native input parts
596
-
597
- A provider declares `inputModalities`, the set of native non-text inputs its model accepts, from the catalog's input modalities (Models.dev `modalities.input` minus `text`, kept to the vocabulary `image`, `pdf`, `audio`, `video`; empty when the model is unknown). Which parts actually ride a request is the service's decision per attachment ({§packet-attachment-parts} in the core specification).
598
-
599
- | Fact | Rule |
600
- | --- | --- |
601
- | The part | a user `ChatMessage` may carry text beside the AI SDK's current `{ type: "file", data: bytes, mediaType }` part; `mediaType` distinguishes an image, PDF, audio, or future native input without a second internal part shape |
602
- | The transport | the AI SDK transport forwards those parts as the model's native file inputs; it alone serializes the canonical model and messages before provider extension fields are merged |
603
- | Other messages | system and assistant messages stay text, and prompt-token estimates count text only, the provider's reported usage owning each part's cost |
604
- | Pools and the Mock | a pool declares a modality only when every backend does; the Mock declares them by option and records every request it receives |
605
- | Codecs | an accepted modality does not imply every codec/container is supported: SDK or endpoint format rejections remain provider errors, never silent omission or a text-only substitute |
606
-
607
- ### §model-fact-resolution Model fact precedence
608
-
609
- Provider and model facts resolve independently:
610
-
611
- | Fact | Natural source | Operator source | Effective value |
612
- | --- | --- | --- | --- |
613
- | Context window | Catalog metadata or local endpoint probe. | `PLURNK_PROVIDERS_CONTEXT_WINDOW`. | Minimum when both exist; sole value otherwise. Cataloged cloud miss fails construction; compatible probe miss remains `null` with one warning. |
614
- | Maximum input | Catalog `limit.input`; no generic live probe. | None. | Catalog value or `null`; never reconstructed from context and output. |
615
- | Maximum output | Catalog `limit.output`; no generic live probe. | None. | Minimum of catalog value and effective context, or `null`. |
616
- | Total output budget | None. | `PLURNK_PROVIDERS_OUTPUT_BUDGET`. | Curation reservation: percentage of effective context or absolute count, capped by known context/output limits; a call may only tighten it. The response grant flexes under {§provider-flexed-allowance}. |
617
- | Output floor | None. | `PLURNK_PROVIDERS_OUTPUT_FLOOR`, shipped as `10%`. | The least response room any request keeps: percentage of the effective output ceiling (`min(contextWindow, maxOutputTokens)` when both are known, otherwise `contextWindow`) or absolute count, capped by the output budget; `window − floor` is the input wall ({§provider-output-floor}). |
618
- | Effort | Catalog `reasoning_options` intersected with the installed adapter; explicit adapter declaration for uncataloged routes. | `PLURNK_PROVIDERS_EFFORT`, initially; durable worker selection thereafter. | A supported member of {§effort-wire}, projected under {§provider-effort}. The shipped selection is `adaptive`. |
619
- | Reasoning budget | None. | Optional `PLURNK_PROVIDERS_REASONING_BUDGET`. | Percentage of effective context or absolute count; valid only as a strict subset of total output and effective unless reasoning is `off`. |
620
- | Cost override | None. | Optional `PLURNK_PROVIDERS_COST`. | {§operator-cost-override} — comma-separated `key=value` per-1M-token USD rates over `input, output, reasoning, cacheRead, cacheWrite`; merges over the Models.dev catalog block (the catalog is the starting point), alias-scoped like every knob. Without catalog rates the override must declare `input` and `output`. The cost estimate's `source` names the override; a provider-reported response cost still outranks any estimate. |
621
- | Reasoning capability | Catalog `reasoning` and route-specific `reasoning_options`. | Adapter wire style only where the catalog cannot name the native field. | Catalog controls determine admissible policy; the adapter determines its wire projection. |
622
- | Estimated USD rates | Models.dev input, output, optional reasoning, and optional cache rates. | None. | Missing differently-priced usage or rates produces unknown; exact all-zero rates produces estimated USD zero. |
623
-
624
- Models.dev cache-read and cache-write rates default to the input rate, and the
625
- reasoning rate defaults to the output rate, when omitted. A differently priced
626
- category requires its corresponding usage detail or the request estimate is
627
- unknown.
628
-
629
- The provider segment of a model route IS the Models.dev id (#459) —
630
- configurations align with the catalog end to end (`fireworks-ai/…`,
631
- `cloudflare-workers-ai/@cf/…`, `togetherai/…`), and the
632
- `PLURNK_PROVIDERS_PROVIDER_<PREFIX>_*` prefix is that id uppercased with
633
- non-alphanumerics as underscores (`FIREWORKS_AI`). Custom declared providers
634
- keep their operator-chosen names; the built-in local `ollama` rail keeps its
635
- name (the catalog's `ollama-cloud` is a different provider).
636
-
637
- `instantiateProvider` resolves in this order:
638
-
639
- 1. A Models.dev provider and model, using its declared AI SDK package.
640
- 2. An operator provider declaration:
641
- `PLURNK_PROVIDERS_PROVIDER_<NAME>_{NPM,BASE_URL,API_KEY_ENV}`.
642
- 3. The local `openai` or `ollama` adapter.
643
- 4. A discovered AI SDK provider extension.
644
- 5. A precise unknown-provider error.
645
-
646
- Earlier sources are authoritative. Installed extensions cannot shadow a cataloged
647
- or explicitly declared name. This remains true when a named model is absent
648
- from the Models.dev snapshot: construction requires an explicit
649
- `PLURNK_PROVIDERS_CONTEXT_WINDOW` and never falls through to a same-name
650
- extension.
651
-
652
- Model IDs resolve exactly first. A unique catalog suffix is accepted to avoid
653
- forcing a vendor-owned resource prefix into PLURNK aliases. Ambiguous suffixes
654
- fail to resolve.
655
-
656
- The catalog package identifies the protocol family, not a mandatory client
657
- implementation. OpenRouter charges and DeepInfra estimates use cost normalizers
658
- over documented response fields, retaining their distinct monetary character
659
- under {§provider-monetary-evidence}.
660
-
661
- §provider-fact-authority Provider declarations configure facts, not
662
- credentials, and Models.dev is authoritative for cataloged providers: package
663
- defaults never redefine a cataloged provider's NPM package, endpoint, or
664
- credential names, and one declaration's `API_KEY_ENV` holds exactly one
665
- environment name — an ordered fallback list would paper over an
666
- operator/catalog naming mismatch instead of reconciling it at its owning
667
- boundary. A comma-separated value is rejected at construction.
668
-
669
- ```dotenv
670
- PLURNK_PROVIDERS_PROVIDER_ACME_NPM=@ai-sdk/openai-compatible
671
- PLURNK_PROVIDERS_PROVIDER_ACME_BASE_URL=https://api.acme.example/v1
672
- PLURNK_PROVIDERS_PROVIDER_ACME_API_KEY_ENV=ACME_API_KEY
673
- ```
674
-
675
- The named secret remains in the operator environment. `${ENV_NAME}` inside a
676
- catalog or declared endpoint is expanded at construction and fails clearly
677
- when absent.
678
-
679
- ## §6 Provider extensions
680
-
681
- Provider extensions are the escape hatch for a protocol binding not represented by
682
- Models.dev, installed SDK providers, or a declaration. Most extensibility
683
- belongs in MCP, schemes, executors, or mimetypes instead.
684
-
685
- A provider extension:
686
-
687
- 1. declares the exact string `plurnk: { kind: "provider", name }` in `package.json` ({§extension-kind});
688
- 2. may declare always-on package-level `plurnk.attribution` and/or implement the
689
- synchronous runtime `attributions(context)` hook under {§extension-attribution};
690
- 3. may use any npm scope;
691
- 4. default-exports an AI SDK provider with `languageModel(modelId)`;
692
- 5. peers on compatible `ai` and `@plurnk/plurnk-providers` majors.
693
-
694
- §provider-grammar-transport An extension whose backend accepts a llama.cpp-style
695
- GBNF grammar may declare `plurnk.grammarStyle: "llamacpp"` beside its kind and
696
- name; the discovery records it and the adapted Provider carries the capability,
697
- so an operator's grammar file ({§operator-grammar}) rides the wire exactly as
698
- on a probed llama-server. Absence or `"none"` keeps the grammar off the wire;
699
- any other value fails discovery loudly. The declaration is the extension author's
700
- fact about their backend; the transport evidence on each response
701
- ({§provider-grammar-evidence}) is where a wrong declaration shows, never by
702
- degrading admission.
703
-
704
- PLURNK adapts the returned language model. The extension does not implement the
705
- PLURNK `Provider`, read PLURNK tuning knobs, or reproduce transport policy.
706
-
707
- Discovery is scope-agnostic and memoized per process. Duplicate names fail hard.
708
- The common extension trust gate applies before import ({§extension-trust-boundary}). An extension absent from
709
- Models.dev requires an explicit context-window pin because PLURNK will not guess
710
- model physics. `Discovery.packageAttributions` carries the canonical package map.
711
-
712
- ## §7 Local capabilities
713
-
714
- <!-- map -->
715
- | Tag | Promise |
716
- | --- | --- |
717
- | {§provider-local-capabilities} | The `openai` local adapter probes `/v1/models`. |
718
- | {§llama-template-effort} | A detected llama-server rail admits `off` and `adaptive`, and a fixed effort only where the operator declares… |
719
- | {§llama-reasoning-request} | The allowance is cumulative across the complete response. |
720
- <!-- /map -->
721
-
722
- §provider-local-capabilities The `openai` local adapter probes `/v1/models`. A
723
- llama-server fingerprint may
724
- also expose:
725
-
726
- - the actual served model and per-slot context window;
727
- - request-scoped reasoning parsing and a cumulative response-wide allowance;
728
- - GBNF constrained sampling;
729
- - slot count and worker-sticky slot affinity;
730
- - EOS marker removal;
731
- - exact complete-request counting through `/v1/chat/completions/input_tokens`;
732
- - exact content token IDs through `/tokenize`;
733
- - the requirement that the adapter apply a finite output budget.
734
-
735
- `PLURNK_PROVIDERS_LLAMA_SERVER` may force or disable detection. Probe attempts
736
- and delay are knobs. A failed probe does not silently assert capabilities.
737
-
738
- Ollama probes `/api/show` for its model context and uses its documented
739
- OpenAI-compatible generation endpoint through the SDK adapter.
740
-
741
- ### llama-server reasoning
742
-
743
- For a detected llama-server, PLURNK sends the complete reasoning contract on
744
- every request:
745
-
746
- | Posture | Template activation | Template `reasoning_effort` | `thinking_budget_tokens` |
747
- |---|---:|---|---:|
748
- | `off` | false | none | `0` |
749
- | `adaptive` | true | none | configured reasoning subset, otherwise omitted |
750
- | A declared fixed effort | true | the effort's name | configured reasoning subset, otherwise omitted |
751
-
752
- §llama-template-effort A detected llama-server rail admits `off` and `adaptive`, and a fixed
753
- effort only where the operator declares it (`REASONING_EFFORTS`,
754
- {§provider-wire-declaration}). The chat template decides what each level means and which
755
- levels it accepts; PLURNK infers neither from the template nor from the model's name, and a
756
- template that refuses a declared level answers on the first request.
757
-
758
- The allowance is contained by the request's total output budget. Template calls
759
- normally use `reasoning_format: "auto"` for a separate
760
- readable channel. A GBNF-bearing call uses `"none"` so the exact constrained
761
- sentence survives response projection; the adapter separates its leading
762
- reasoning enclosure only after preserving grammar evidence. Process-wide
763
- llama-server flags are fallback server configuration, not part of the PLURNK
764
- contract and need not be synchronized with an alias.
765
-
766
- §llama-reasoning-request The allowance is cumulative across the complete response. Opening a second or
767
- later reasoning block does not replenish it. Template parsing, the reasoning
768
- sampler, normalized usage, and the returned reasoning channel MUST agree on that
769
- response boundary.
770
-
771
- ## Probes
772
-
773
- §provider-ping-probe **`npm run probe:providers` reaches every keyed provider once, by one cheapest declared route.** Planning selects, per provider that has a credential, its cheapest declared alias; a provider-named alias resolves that provider's own unpriced route without guessing another model, a catalog key with no declared route stays explicitly red, and several unpriced aliases of one provider are one ambiguous provider, never guessed routes. Execution invokes every selected route exactly once and preserves red outcomes. Each attempted provider leaves one safe JSON record: response evidence keeps its structure and no provider scalar values, retained diagnostics redact credentials, account identity and URL authority, and secret discovery never mistakes a token-limit control for a credential.
774
-
775
- §token-envelope-probe **`npm run probe:tokens` measures output-token envelopes for the aliases named explicitly, and no other.** Its output-token modes distinguish the configured policy from the model's physics.
776
-
777
- ## §8 Request authority
778
-
779
- <!-- map -->
780
- | Tag | Promise |
781
- | --- | --- |
782
- | {§provider-request-authority} | The caller's `sampling` bag expresses sampling intent. |
783
- | {§provider-request-controls} | A configured service tier or logprobs count reaches a native route only through the per-call option its SDK… |
784
- | {§openrouter-app-attribution} | A route on the OpenRouter SDK identifies the calling application only as its provider declares. |
785
- <!-- /map -->
786
-
787
- §provider-request-authority The caller's `sampling` bag expresses sampling
788
- intent. It cannot override:
789
-
790
- - model or messages;
791
- - stream mode;
792
- - grammar or response format;
793
- - backend slot;
794
- - data-capture settings;
795
- - tool, modality, or multi-choice behavior;
796
- - the consumer-owned output envelope;
797
- - cache affinity identity or cache-write policy.
798
-
799
- Generic AI SDK calls accept only settings represented by the SDK's portable
800
- surface. Compatible endpoints may carry additional sampling keys after reserved
801
- keys are removed.
802
-
803
- §provider-request-controls **A configured service tier or logprobs count reaches a
804
- native route only through the per-call option its SDK documents.** On a compatible
805
- route, `PLURNK_PROVIDERS_SERVICE_TIER` and `PLURNK_PROVIDERS_TOP_LOGPROBS` are the
806
- body fields `service_tier`, `logprobs` and `top_logprobs`. A native route sends them
807
- in its SDK's `providerOptions` namespace, serialized by that SDK. A control the SDK
808
- does not document is not sent, and one process warning,
809
- `PLURNK_REQUEST_CONTROL_UNSUPPORTED`, names it, so a configured choice never
810
- disappears in silence. The tier passes verbatim: the SDK owns its vocabulary, and a
811
- value it drops surfaces as its call warning ({§provider-sdk-warning}). A logprobs
812
- count of 0 asks for the chosen token only.
813
-
814
- | Native SDK | Namespace | Service tier | Logprobs |
815
- | --- | --- | --- | --- |
816
- | OpenAI | `openai` | sent | sent |
817
- | Groq | `groq` | sent | not sent |
818
- | Cerebras | `cerebras` | sent | not sent |
819
- | xAI | `xai` | sent | not sent |
820
- | Google | `google` | sent | not sent |
821
- | Anthropic | `anthropic` | sent | not sent |
822
- | Amazon Bedrock | `bedrock` | sent | not sent |
823
- | Mistral, Together, DeepInfra, OpenRouter | none | not sent | not sent |
824
-
825
- §openrouter-app-attribution **A route on the OpenRouter SDK identifies the calling
826
- application only as its provider declares.** `APP_URL`, an absolute HTTP(S) URL, and
827
- the optional `APP_NAME` are provider declarations (`PLURNK_PROVIDERS_PROVIDER_<NAME>_`,
828
- overridable per route or alias); the SDK sends them as `HTTP-Referer` and
829
- `X-OpenRouter-Title`. The shipped floor declares them for `openrouter` only, so
830
- another provider on the same SDK package sends none; an empty `APP_URL` suppresses
831
- both.
832
-
833
- ## §9 Failures, retries, and cancellation
834
-
835
- <!-- map -->
836
- | Tag | Promise |
837
- | --- | --- |
838
- | {§provider-failure-normalization} | Provider failures normalize to `ProviderError`. |
839
- | {§provider-retryable-truth} | `retryable` states what happens next. |
840
- | {§provider-failure-cause} | The wrapper's cause is evidence, bounded. |
841
- | {§provider-capacity-failure} | A proven exact preflight overflow and an upstream context rejection normalize to… |
842
- | {§provider-flexed-allowance} | The grant is the window's remainder, between the floor and the model's limit. |
843
- | {§provider-output-floor} | The floor is the wall's other side. |
844
- | {§provider-sampling-passthrough} | Unconfigured sampling retains endpoint defaults. |
845
- | {§provider-connectivity} | The provider adapter owns one attempt scheduler around the complete generation exchange; SDK-internal retries… |
846
- | {§provider-cancellation-evidence} | Received usage/charge evidence and available response identity survive cancellation and deadline… |
847
- | {§provider-native-tool-calls} | A response with native tool-call data, or a `tool_calls` finish, preserves its text, reasoning, structured… |
848
- | {§provider-completed-admission} | A completed exchange is admitted as the provider delivered it. |
849
- | {§provider-request-rejection} | An HTTP 4xx rejection not classified as authorization, quota, capacity, grammar, rate limit, or transient… |
850
- | {§provider-interrupted-attempt} | Provider-declared interruption |
851
- | {§provider-generation-failed} | Provider-declared generation error |
852
- <!-- /map -->
853
-
854
- §provider-failure-normalization Provider failures normalize to `ProviderError`.
855
- Its public contract is an RFC
856
- 9457 Problem Details object with an exact status, stable type, occurrence
857
- detail, and provider-kind extension; the original error remains its cause.
858
- A caught failure is surfaced or deliberately preserved; it is never converted
859
- into an empty model turn or reduced to a message plus a generic status.
860
- Upstream diagnostic text is bounded by
861
- `PLURNK_PROVIDERS_ERROR_DETAIL_LIMIT`; the committed `.env.defaults` owns its
862
- normal value. Retry exhaustion is preserved as `attempts` and
863
- `retryExhausted`; it does not change the Problem's `retryable`. A provider's
864
- producer id is `provider:<name>`, the name folded to the `source` grammar of
865
- `Notice.json` (lower-case words and hyphens; a leading digit is prefixed
866
- `p-`); an empty name is refused.
867
-
868
- §provider-retryable-truth **`retryable` states what happens next.** A Problem's
869
- `retryable` is exactly membership of its kind in the exported
870
- `RETRYABLE_PROVIDER_KINDS` — `rate_limit`, `network_failure`, `deadline_exceeded`,
871
- `resource_interrupted` — and Core's provider recovery re-issues
872
- exactly those kinds by reading the same set. It never copies the transport's own
873
- retry policy: a 502 or 524 that the transport will not replay is still re-issued by
874
- the consumer, so it reports `retryable: true`. Every other kind reports `false`.
875
-
876
- §provider-failure-cause **The wrapper's cause is evidence, bounded.** The SDK
877
- reports every processing failure of a 2xx body with one message ("Failed to
878
- process successful response") and keeps what happened as `cause`. A normalized
879
- Problem carries that cause as a bounded classification — `cause.causeKind` is
880
- `transport_terminated` (the body ended under a successful status),
881
- `invalid_json`, `schema_invalid`, `invalid_response_data`, or `internal`, with
882
- the cause's error name and one bounded line of its message — so forensics can
883
- tell an interrupted stream from a malformed body without the kind changing:
884
- classification of the Problem (`providerKind`, retryability, accounting) is
885
- untouched by this evidence. The SDK's parse and validation messages embed the
886
- payload, so those two report shape (character count, the validator's issue
887
- line), never response text; no headers, credentials, stack frames, or body
888
- dumps enter any Problem. Covered by `errors.test.ts` (#593).
889
-
890
- §provider-capacity-failure A proven exact preflight overflow and an upstream
891
- context rejection normalize to `ProviderError(kind="capacity_exceeded")` and
892
- an RFC 9457 status 413. `capacityStage` is `preflight` or `upstream`; a
893
- non-413 upstream status remains `providerStatus`, while physical request
894
- accounting retains the status actually received. Preflight rejection occurs
895
- before provider I/O and therefore opens no request identity and creates no
896
- request-accounting row. Capacity failures are not connectivity failures and are
897
- never retried by the provider scheduler; bounded packet recovery belongs to the
898
- consumer.
899
-
900
- §provider-flexed-allowance **The grant is the window's remainder, between the floor and the model's limit.** The configured output budget is the reservation curation packs the input against (`window − outputBudget`), and it is the only allowance the model is ever shown — the packet's disclosed `tokensResponseMax` stays the configured reservation so the response discipline it teaches never varies.
901
-
902
- | Prompt measurement | The wire grants |
903
- | --- | --- |
904
- | An exact prompt count (llama-server's input-token endpoint) | `window − promptTokens − 256`, clamped to no less than the output floor and no more than the model's own output limit, so a response overflowing the disclosed reservation completes whenever the window's unclaimed room can hold it, and a prompt that ate into the reservation still leaves the floor |
905
- | Every other measurement kind, and any pool | the reservation, clamped by the same remainder taken from the prompt's estimate so the wire never asks the window for more than it has; an estimate proves nothing about the true remainder, so it never rejects |
906
-
907
- §provider-output-floor **The floor is the wall's other side.** `PLURNK_PROVIDERS_OUTPUT_FLOOR`, shipped as `10%`, is the least response room any request keeps: a percentage resolves against the effective output ceiling (`min(contextWindow, maxOutputTokens)` when both are known, otherwise `contextWindow`), and the resolved floor is capped by the output budget; `window − floor`, intersected with `maxInputTokens`, is the physical input wall exposed as `inputWall`. A request under the curation reservation is the ordinary case; a request between the reservation and the wall is admitted with its grant flexed down to the remainder; an exact prompt over the wall is rejected before I/O; an estimate over the wall defers to the upstream provider as capacity oracle with its grant at the floor. Core consults the wall, never the reservation, to end a loop ({§context-wall} in the core contract). A response exceeding even the grant is cut,
908
- and the notice names the true per-call grant (`capacity.responseMax` on the
909
- response) — the tolerance's honest edge, never worse than the fixed allowance
910
- it forgives.
911
-
912
- §provider-sampling-passthrough **Unconfigured sampling retains endpoint defaults.**
913
- The provider panel owns tuning; Models.dev capabilities are not recommended
914
- sampling values. Every knob accepts the ordinary alias override
915
- ({§provider-configuration}). Both cataloged and local providers apply the same
916
- validation and precedence: configured values, then caller `sampling`, then
917
- transport-owned fields ({§provider-request-authority}).
918
-
919
- | `PLURNK_PROVIDERS_` suffix | Compatible wire / native SDK | Accepted configuration |
920
- | --- | --- | --- |
921
- | `TEMPERATURE` | `temperature` / `temperature` | Finite non-negative number; empty/unset omits. |
922
- | `TOP_P` | `top_p` / `topP` | Finite number in `[0,1]`; empty/unset omits. |
923
- | `TOP_K` | `top_k` / `topK` | Non-negative safe integer; empty/unset omits. Zero semantics are endpoint-owned. |
924
- | `PRESENCE_PENALTY` | `presence_penalty` / `presencePenalty` | Finite number in `[-2,2]`; empty/unset omits. |
925
- | `FREQUENCY_PENALTY` | `frequency_penalty` / `frequencyPenalty` | Required finite number in `[-2,2]`; the panel's zero selects no configured override. Negative values survive unchanged. |
926
- | `SEED` | `seed` / `seed` | Safe integer; empty/unset omits. No guarantee of reproducibility. |
927
-
928
- Configured zero is preserved except for the frequency knob's explicit
929
- no-override sentinel; a caller-supplied zero always overrides a configured value.
930
- Invalid configuration fails by knob name before inference. The SDK/endpoint
931
- owns narrower model restrictions and unsupported-setting diagnostics; Plurnk
932
- does not clamp values or invent model-specific sampling profiles.
933
- `REPEAT_PENALTY` and DRY remain explicit llama-server extensions, omitted when
934
- unconfigured. Their grammar protection remains at {§provider-grammar-transport}.
935
-
936
- §provider-connectivity The provider adapter owns one attempt scheduler around
937
- the complete generation exchange; SDK-internal retries are disabled.
938
- `PLURNK_PROVIDERS_RETRY_ATTEMPTS=N` permits at most `N + 1` physical requests.
939
- Generation has one configurable deadline; `0` disables it. Caller cancellation
940
- and the caller's task allowance remain authoritative. Initial silence, a pause
941
- between chunks, and buffered output do not acquire separate deadlines.
942
-
943
- | Layer | Operator knob | Boundary | Expiry |
944
- | --- | --- | --- | --- |
945
- | Operation | `PLURNK_PROVIDERS_OPERATION_TIMEOUT` | Complete logical call, including admission queueing, every attempt and retry delay. | Final `deadline_exceeded` Problem at 504 with `timeoutPhase=operation`; not retried inside this operation. Consumer recovery is separate. Enforced as a race, not only the advisory signal, so a wedged transport that never observes the abort cannot hang the loop past the deadline (#505); a well-behaved transport unwinds within a short grace and settles its own attempt evidence first. |
946
- | Discovery/tokenizer HTTP | `PLURNK_PROVIDERS_FETCH_TIMEOUT` | Non-generation endpoint discovery and tokenizer requests only. | Ordinary HTTP failure at the owning discovery or measurement boundary. Never interrupts generation. |
947
-
948
- Settled calls remove their deadline timers and cancellation subscriptions.
949
-
950
- Caller cancellation spans the operation and preserves the caller's reason.
951
-
952
- §provider-cancellation-evidence Received usage/charge evidence and available
953
- response identity survive cancellation and deadline normalization through
954
- physical-request settlement, independently of optional raw-body capture.
955
- This includes SDK abort events and non-Error cancellation reasons. Concurrent
956
- requests keep independent evidence even when cancelled by the same scope.
957
- Absent evidence remains absent; stopping a request neither invents usage nor
958
- retries generation to obtain it. Partial output is never a completed response.
959
-
960
- A 2xx exchange whose body cannot be processed (a provider invalid-response)
961
- classifies as the non-retryable 502 on the first failure unless an explicit
962
- `x-should-retry` directive says otherwise (#479). When a
963
- directive-driven retry sequence exhausts, `attempts` and `retryExhausted`
964
- are added and the classification is final. Every admitted physical attempt opens
965
- and settles exactly one ordered {§provider-request-accounting} record, including
966
- response-less network failures and timed-out attempts. A cancelled or expired
967
- admission wait opens no physical request.
968
-
969
- Each streamed physical request assembles its own response. Failed partial answer
970
- bytes never enter a later request's completed `ProviderResponse`; recovery is a
971
- complete re-emission, not continuation or salvage. When a retry succeeds after
972
- a failed request emitted nonempty text or reasoning, the accepted response
973
- carries one `provider_retry` warning identifying that fact. A retry before any
974
- semantic model output remains silent while its physical failure stays cardinally
975
- accounted. Transient reasoning follows {§provider-reasoning-observer}.
976
-
977
- Only a provider-directed wait earns a transport retry (#479): HTTP 429, any
978
- response carrying `Retry-After` (RFC 9110 defines it on 503 exactly as a
979
- directed wait) — header-level facts the consumer's recovery layer never sees —
980
- and an explicit `X-Should-Retry: true`. That header stays authoritative both
981
- ways, and endpoint control responses 520–527 stay final. HTTP 408, 409, and
982
- ordinary 5xx surface on the first failure as consumer-recoverable kinds; one
983
- retry authority — the consumer's own provider-recovery machinery — owns
984
- re-issue, backoff, and park above the transport.
985
-
986
- §provider-native-tool-calls A response with native tool-call data, or a
987
- `tool_calls` finish, preserves its text, reasoning, structured wire evidence,
988
- finish reason and accounting and emits one provider warning. It is not a
989
- transport failure and does not trigger provider recovery. The adapter executes
990
- no call and invents no textual operation; the consumer owns admission and
991
- no-operation handling. The normalized assistant exposes `nativeToolCalls: true`
992
- when either wire calls or a `tool_calls` finish is present, independently of the
993
- preserved finish reason; consumers need not mine wire evidence.
994
-
995
- §provider-completed-admission **A completed exchange is admitted as the provider delivered it.**
996
- Billed output tokens are accounting, never evidence of missing text: a provider may bill hidden or
997
- summarized reasoning, bill reasoning inside its text count, or tokenize differently from what it
998
- streams, so no comparison of token counts with streamed characters voids a response. What the
999
- provider reports as its finish ({§provider-finish-reason}) decides the attempt's disposition; what
1000
- the response says is the consumer's to judge ({§emission-admission}).
1001
-
1002
- §provider-request-rejection An HTTP 4xx rejection not classified as authorization,
1003
- quota, capacity, grammar, rate limit, or transient transport failure is
1004
- `request_rejected`: preserve the upstream status and detail, with `retryable: false`.
1005
- It is not an `invalid_response`; that kind describes a malformed successful
1006
- response. Consumers terminate a rejected request with its exact Problem rather
1007
- than retrying unchanged input or assigning response-contract strikes.
1008
-
1009
- ### §provider-interrupted-attempt Provider-declared interruption
1010
-
1011
- A successful transport response can still declare that inference did not
1012
- complete. That response is evidence for one failed provider attempt, never a
1013
- completed exchange.
1014
-
1015
- | Concern | Contract |
1016
- | -------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
1017
- | Normalized finish reason | Exact `insufficient_system_resource` becomes `resource_interrupted`; the raw value remains in `assistantRaw`. |
1018
- | `generate` outcome | Throw `ProviderError(kind="resource_interrupted")` at local status 503 with the normalized attempt on `error.attempt` and the same accounting on `error.accounting`. |
1019
- | Partial response | Preserve content, reasoning, usage, model, metadata, optional raw body, and other evidence without admitting it as success. |
1020
- | Automatic replay | None inside the provider; AI SDK retry scheduling has already completed. The Problem has `retryable: true`: the consumer re-issues it ({§provider-retryable-truth}). |
1021
- | Capacity-pool overflow | None under the existing routing policy; when other overflow-eligible failures do reach a sibling, the pool concatenates their request accounting. |
1022
- | Consumer admission | Persist the evidence as an unaccepted attempt; never parse it into executable work, even when its frame looks complete. |
1023
-
1024
- ### §provider-generation-failed Provider-declared generation error
1025
-
1026
- A finish the SDK classifies as `error` (Google's `MALFORMED_FUNCTION_CALL`, for one) is a failed
1027
- attempt, `ProviderError(kind="generation_failed")` at local status 502 with the normalized attempt
1028
- on `error.attempt` and its accounting on `error.accounting`: content, reasoning, usage, model, the
1029
- raw finish reason and any raw body stay evidence, never a completed exchange. It is not a transport
1030
- outage: `retryable: false`, so the consumer's provider recovery does not re-issue it; Core settles
1031
- it on the provider response contract — the turn is complete, the strike rail rules, and the next
1032
- turn asks again ({§provider-retryable-truth}).
1033
-
1034
- ## §10 Grammar
1035
-
1036
- GBNF is a local llama-server capability, not a generic provider expectation.
1037
- The consumer chooses whether to supply a grammar and reads it from the
1038
- operator's own file; the provider never creates, ships, rewrites, validates, or
1039
- grades one (#588). Transport is `{§provider-grammar-transport}`.
1040
-
1041
- §provider-grammar-evidence When a grammar-capable adapter receives a grammar,
1042
- `ProviderResponse` carries `grammarEvidence: { input, contentStart, transported }`.
1043
- `input` is the exact pre-projection sentence represented by the response,
1044
- `contentStart` is its Unicode-code-point offset to `assistant.content`, and
1045
- `transported` says whether the grammar was actually sent. Active llama-server
1046
- template reasoning requests the unprojected sentence, records it, and then
1047
- separates its leading enclosure; an empty body therefore remains observable. An
1048
- endpoint that projects despite that request supplies no independent evidence.
1049
- For an unsplit response, `input` is `content` and `contentStart` is zero. This
1050
- is transport evidence so a digest can prove a run was constrained; nothing in
1051
- the service issues a verdict on it.
1052
-
1053
- ## §11 Evidence and metadata
1054
-
1055
- <!-- map -->
1056
- | Tag | Promise |
1057
- | --- | --- |
1058
- | {§provider-evidence} | `assistantRaw` is an opaque normalized transport record. |
1059
- | {§provider-wire-emission} | The emission is retained as the wire carried it, on every response. |
1060
- | {§provider-usage-refusal} | The provider's bookkeeping is not the exchange. |
1061
- | {§provider-open-reasoning} | Plurnk requires endpoints with open, verbatim reasoning. |
1062
- <!-- /map -->
1063
-
1064
- §provider-evidence `assistantRaw` is an opaque normalized transport record.
1065
- Provider top-level
1066
- metadata is forwarded as an open bag without reinterpreting currencies or
1067
- vendor fields.
1068
-
1069
- Logprobs and raw capture are opt-in, alias-scoped diagnostic features.
1070
- The former requests per-token logprobs and preserves returned alternatives.
1071
- The latter retains SDK response bodies/chunks and the dispatched request body
1072
- under {§provider-dispatched-request}, without requesting extra model output.
1073
- Neither changes unconditional accounting or failed-response evidence.
1074
-
1075
- §provider-wire-emission **The emission is retained as the wire carried it, on every response.** Beside the normalized `content` and `reasoning`, the transport record keeps `wire`. Nothing is opt-in and nothing is reinterpreted: a blank emission reads as thirteen chunks that carried nothing, or as a tool-call section the model emitted into a channel the protocol does not read, instead of being guessed at from token counts. This required evidence is independent of the optional complete `rawBody` dataset capture. Covered by `aiSdkTransport.test.ts` and core's `Digest.wire-evidence.test.ts`.
1076
-
1077
- | `wire` holds | What |
1078
- | --- | --- |
1079
- | Chunks | the number of raw chunks, and the chunks that carried no field at all |
1080
- | Fields | every delta or message field the stream carried with the number of chunks carrying it |
1081
- | Tool calls | merged by index with their arguments as streamed |
1082
- | Finish reasons | those seen |
1083
- | Text | the verbatim text of every string field the record does not already hold |
1084
- | `wire.unmappedChunks` | frames outside the recognized OpenAI-compatible envelope, verbatim; the recorder never discards an unrecognized/native payload or invents another provider parser |
1085
-
1086
- §provider-usage-refusal **The provider's bookkeeping is not the exchange.** Usage
1087
- normalization is exact; a refusal never fails or retries the response. The same
1088
- rules apply to usage inside failure evidence:
1089
-
1090
- | Contradiction | Normalized evidence |
1091
- | --- | --- |
1092
- | Invalid aggregate or total inconsistent with its parts | No `usage`. |
1093
- | Invalid optional counter | Omit that counter; retain independently valid quantities. |
1094
- | Cache or reasoning breakdown inconsistent with its aggregate | Omit that breakdown; retain valid aggregates and the other breakdown. |
1095
-
1096
- Every refusal retains `usageRefusal`: the reason and the original wire counters.
1097
- No value is clamped, zeroed, or estimated. Cost uses authoritative charges when
1098
- available; catalog estimation requires all quantities its rates need, otherwise
1099
- cost is `unknown`. Covered by `usage.test.ts`, `aiSdkTransport.test.ts`, and
1100
- `AiSdkProvider.test.ts`.
1101
-
1102
- §provider-open-reasoning **Plurnk requires endpoints with open, verbatim reasoning.**
1103
- The adapter retains the reasoning text its transport exposes; fidelity is not
1104
- inferred from a model name or catalog flag. Hidden or summarized reasoning is not
1105
- a supported substitute, and encrypted or redacted payloads are never reconstructed
1106
- into text. A single response without reasoning does not establish endpoint
1107
- incapability: its `reasoning://` reads empty, while billed reasoning tokens remain
1108
- visible in accounting. Exposed operations follow {§reasoning-operations}; no
1109
- compatibility teaching or alternate admission mode is selected for a route.
1110
- Unrecognized detail shapes are omitted at this normalization boundary. Core
1111
- may preserve normalized items as forensic evidence, but a client protocol must
1112
- correlate them to an entity it actually created rather than reusing `id`.
1113
-
1114
- ## §12 Generation envelopes
1115
-
1116
- §provider-generation-envelope Every request has at most one total output
1117
- budget. It includes visible output and hidden reasoning. An optional reasoning
1118
- budget is a strict subset of that total, never an additive reserve. The
1119
- configured total is a percentage of effective context or an absolute count;
1120
- percentages resolve to the nearest whole token with a one-token minimum. It is
1121
- capped by known context and model-output limits; `generate.maxOutputTokens` may
1122
- only tighten this reservation for one call. The effective reasoning subset tightens
1123
- with that total and remains strictly smaller. Exact prompt measurements may expand
1124
- the response grant beyond the reservation under {§provider-flexed-allowance}.
1125
-
1126
- The adapter owns native projection. A backend whose generic SDK maximum already
1127
- includes reasoning receives the total directly. When a native SDK instead adds
1128
- an explicit reasoning allowance to its generic visible-output maximum, the
1129
- adapter sends `total - reasoning` through the generic field and the reasoning
1130
- subset through the documented provider option. Core and other callers never
1131
- reconstruct this arithmetic. When such a backend has only a manual allowance
1132
- and no numeric subset is configured, the adapter derives that allowance from
1133
- the durable policy inside the total using the native SDK's effort proportions
1134
- and provider minimum; an envelope too small to represent the minimum fails
1135
- before provider I/O.
1136
-
1137
- §provider-output-budget-conformance When a completed response reports
1138
- normalized output-token usage greater than its response grant ({§provider-flexed-allowance}),
1139
- the exchange is an `invalid_response` at 502 rather than an admitted result or
1140
- a prompt-capacity 413. Its complete failed-attempt evidence and settled charged
1141
- request remain available. The violation is final and is never automatically
1142
- replayed. Missing output usage cannot prove a violation.
1143
-
1144
- `PLURNK_PROVIDERS_OUTPUT_BUDGET` is required for standard providers and ships
1145
- as `35%`. `PLURNK_PROVIDERS_REASONING_BUDGET` is optional; leaving it unset
1146
- preserves provider-adaptive depth. A backend known to decode without a finite
1147
- limit advertises `requiresOutputBudget` and fails construction when no total can
1148
- be resolved.
1149
-
1150
- ## §13 Inference capacity
1151
-
1152
- ### Admission
1153
-
1154
- §provider-inference-admission `PLURNK_PROVIDERS_MAX_CONCURRENCY` controls physical
1155
- generation attempts per resolved endpoint within one process: `-1` is unrestricted;
1156
- a positive safe integer admits that many concurrent attempts. It follows ordinary
1157
- alias scoping. Zero, other negative values and non-integers are invalid.
1158
-
1159
- | Boundary | Contract |
1160
- | --- | --- |
1161
- | Identity | Provider instances and aliases sharing the resolved API base URL share one allowance. SDK- or extension-owned endpoints without a resolved URL share their provider identity. Conflicting limits for one identity fail construction, naming the identity and both values; they never create independent queues. The daemon reads its environment once at boot, so an identity's limit cannot change within a process: reconstructing a provider from that environment reuses its allowance, and in-flight leases keep counting. |
1162
- | Admission | FIFO among live waiters. The lease begins before the physical request observer and ends after the complete response or transport failure settles, including streamed bodies. |
1163
- | Cancellation | A queued abort removes that waiter and preserves the caller's reason. It opens no physical request or accounting row. In-flight cancellation signals the transport; capacity is released when that attempt unwinds. A transport still running despite abort does not authorize exceeding the limit. |
1164
- | Retries | Backoff holds no lease. Each retry rejoins admission as a new physical attempt. |
1165
- | Deadline | The whole-call operation deadline includes queueing, every physical attempt and retry wait. Transport adds no generation deadline. |
1166
- | Scope | Workers, tools, messages, waits, token measurement and endpoint discovery are not serialized by inference admission. Independent endpoints progress independently. No cross-process or machine-wide capacity guarantee is implied. |
1167
-
1168
- Admission neither changes worker lifecycle nor selects another model or endpoint.
1169
- Core consumes the same asynchronous Provider contract. A waiting worker holds no
1170
- inference lease; BARE and ordinary generation use the same physical boundary.
1171
-
1172
- ### Pool
1173
-
1174
- §provider-capacity-pool `Pool` fronts interchangeable `Provider` instances. It
1175
- keeps workers sticky for
1176
- cache locality, selects a healthy sibling for overflow, and preserves the same
1177
- Provider contract. Whether endpoints are interchangeable is a consumer
1178
- decision, not inferred from provider names. Overflow is limited to transport
1179
- availability and rate-limit failures that carry no normalized response attempt;
1180
- {§provider-interrupted-attempt} propagates without overflow.
1181
-
1182
- Prompt measurement covers every backend that could receive the request. The
1183
- pool takes the largest quantified result; differing exact counts or any proven
1184
- bound yield an `upper_bound`, any estimate makes the aggregate an estimate, and
1185
- any unavailable backend makes it unavailable. Physical limits and budgets are
1186
- independent safe minima across the pool. `inputCapacity` is the minimum of each
1187
- backend's complete derived input capacity, never a synthetic subtraction across
1188
- minima from different backends; request-specific output tightening repeats the
1189
- complete-envelope derivation per backend before taking the minimum.
1190
-
1191
- ## §14 Conformance
1192
-
1193
- Coverage MUST prove:
1194
-
1195
- - catalog, declaration, local, and extension resolution;
1196
- - exact and unique-suffix model lookup;
1197
- - native SDK request mapping and normalized responses;
1198
- - compatible extension preservation;
1199
- - timeout, retry, cancellation, interrupted-attempt, and final-error behavior;
1200
- - shared endpoint inference admission, FIFO queueing, queued/in-flight cancellation,
1201
- full-stream lease lifetime, retry release, and independent endpoint progress;
1202
- - local capability probes and pins;
1203
- - exact, bounded, estimated, and unavailable complete-request measurements;
1204
- - independent input/context/output limits, asymmetric admission, and normalized
1205
- local/upstream capacity failures;
1206
- - one total output budget and native additive-reasoning projection;
1207
- - provider-reported output beyond that budget failing once with complete
1208
- attempt and accounting evidence;
1209
- - local reasoning activation, response-wide allowance, and GBNF coexistence;
1210
- - explicit tagged-reasoning projection across streamed, buffered, capped, and
1211
- literal-tag responses;
1212
- - usage, costs, evidence, metadata isolation, and grammar observation;
1213
- - alias scoping and fail-hard invalid configuration.
1214
-
1215
- Mock-only green tests do not establish a vendor integration. Live drills and
1216
- integration tests complement this contract; they do not replace its unit-level
1217
- proof.
125
+ - Outbound HTTP requests honor `HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY`, and `NO_PROXY`.
126
+ - When an upstream stream terminates with an error or length finish reason (`MALFORMED_RESPONSE`, `MALFORMED_FUNCTION_CALL`, `MAX_TOKENS`) after delivering a complete, balanced fenced emission with no unclosed fence, no native tool calls, and no trailing unfenced text, `finishReason` normalizes to `"stop"` while preserving `rawFinishReason` in forensic evidence.
127
+ - Transient HTTP failures, upstream malformed-decoder aborts (`MALFORMED_RESPONSE`, `MALFORMED_FUNCTION_CALL`, `UNEXPECTED_TOOL_CALL`) without a complete fenced emission, and reasoning-only `MAX_TOKENS` cutoffs with empty visible content retry up to `PLURNK_PROVIDERS_RETRY_ATTEMPTS` inside the provider (stepping down Google thinking posture on retry when reasoning starved visible output); if a stream fails after emitting partial visible output and a retry succeeds, a `provider_retry` notice is attached.
128
+ - Problem `detail` and structured `cause.causeMessage` strings are bounded to `PLURNK_PROVIDERS_ERROR_DETAIL_LIMIT` characters without leaking raw payloads or headers.
129
+
130
+ ## §provider-inference-admission Inference admission
131
+
132
+ `InferenceAdmission.forEndpoint(endpoint, maxConcurrency)` coordinates in-flight generations across all `Provider` instances targeting the same canonical endpoint URL (`PLURNK_PROVIDERS_MAX_CONCURRENCY`, default `-1` unbounded, or a positive safe integer):
133
+
134
+ - Waiters acquire slots in strict FIFO order before opening a physical request (`observeRequest` does not fire while queued) and hold the slot until the response stream or error settles.
135
+ - Aborted waiters are removed from the queue immediately without consuming capacity; retry backoff releases the slot before sleeping and re-acquires it for the next attempt.
136
+ - Conflicting positive/unbounded `PLURNK_PROVIDERS_MAX_CONCURRENCY` declarations for the same canonical endpoint throw at construction.