@plurnk/plurnk-providers 5.0.1 → 6.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.defaults +11 -81
- package/README.md +21 -22
- package/SPEC.md +109 -1190
- package/dist/LeadingReasoning.js +1 -1
- package/dist/LeadingReasoning.js.map +1 -1
- package/dist/Mock.d.ts +2 -2
- package/dist/Mock.d.ts.map +1 -1
- package/dist/Mock.js +2 -2
- package/dist/Mock.js.map +1 -1
- package/dist/{AiSdkProvider.d.ts → ModelProvider.d.ts} +12 -21
- package/dist/ModelProvider.d.ts.map +1 -0
- package/dist/{AiSdkProvider.js → ModelProvider.js} +140 -188
- package/dist/ModelProvider.js.map +1 -0
- package/dist/{AiSdkRequestBody.d.ts → ModelRequestBody.d.ts} +6 -10
- package/dist/ModelRequestBody.d.ts.map +1 -0
- package/dist/{AiSdkRequestBody.js → ModelRequestBody.js} +17 -63
- package/dist/ModelRequestBody.js.map +1 -0
- package/dist/Pool.js +24 -24
- package/dist/Pool.js.map +1 -1
- package/dist/ProviderRegistry.d.ts.map +1 -1
- package/dist/ProviderRegistry.js +15 -15
- package/dist/ProviderRegistry.js.map +1 -1
- package/dist/RequestCapture.d.ts +1 -0
- package/dist/RequestCapture.d.ts.map +1 -1
- package/dist/RequestCapture.js +9 -6
- package/dist/RequestCapture.js.map +1 -1
- package/dist/accounting.d.ts +1 -1
- package/dist/accounting.d.ts.map +1 -1
- package/dist/accounting.js +4 -4
- package/dist/accounting.js.map +1 -1
- package/dist/capacity.js +4 -4
- package/dist/capacity.js.map +1 -1
- package/dist/catalogProvider.d.ts +5 -8
- package/dist/catalogProvider.d.ts.map +1 -1
- package/dist/catalogProvider.js +216 -66
- package/dist/catalogProvider.js.map +1 -1
- package/dist/compatibleProvider.d.ts.map +1 -1
- package/dist/compatibleProvider.js +8 -8
- package/dist/compatibleProvider.js.map +1 -1
- package/dist/discover.d.ts +1 -1
- package/dist/discover.d.ts.map +1 -1
- package/dist/discover.js +1 -8
- package/dist/discover.js.map +1 -1
- package/dist/env.d.ts +2 -2
- package/dist/env.d.ts.map +1 -1
- package/dist/env.js +20 -59
- package/dist/env.js.map +1 -1
- package/dist/errors.d.ts +48 -0
- package/dist/errors.d.ts.map +1 -1
- package/dist/errors.js +61 -4
- package/dist/errors.js.map +1 -1
- package/dist/index.d.ts +5 -6
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +6 -7
- package/dist/index.js.map +1 -1
- package/dist/{sdkModels.d.ts → modelAdapters.d.ts} +7 -9
- package/dist/modelAdapters.d.ts.map +1 -0
- package/dist/{sdkModels.js → modelAdapters.js} +194 -87
- package/dist/modelAdapters.js.map +1 -0
- package/dist/native-tools.d.ts +1 -2
- package/dist/native-tools.d.ts.map +1 -1
- package/dist/native-tools.js +9 -11
- package/dist/native-tools.js.map +1 -1
- package/dist/ollama.d.ts.map +1 -1
- package/dist/ollama.js +26 -12
- package/dist/ollama.js.map +1 -1
- package/dist/openai.d.ts +3 -3
- package/dist/openai.d.ts.map +1 -1
- package/dist/openai.js +1 -1
- package/dist/pi-transport.d.ts +69 -0
- package/dist/pi-transport.d.ts.map +1 -0
- package/dist/pi-transport.js +697 -0
- package/dist/pi-transport.js.map +1 -0
- package/dist/providerError.js +1 -1
- package/dist/providerError.js.map +1 -1
- package/dist/retry-deadline.d.ts +3 -0
- package/dist/retry-deadline.d.ts.map +1 -0
- package/dist/retry-deadline.js +96 -0
- package/dist/retry-deadline.js.map +1 -0
- package/dist/stream-forensics.d.ts +51 -0
- package/dist/stream-forensics.d.ts.map +1 -0
- package/dist/stream-forensics.js +403 -0
- package/dist/stream-forensics.js.map +1 -0
- package/dist/types.d.ts +29 -4
- package/dist/types.d.ts.map +1 -1
- package/dist/types.js.map +1 -1
- package/dist/usage.js +2 -2
- package/dist/usage.js.map +1 -1
- package/docs/models.md +3 -3
- package/package.json +8 -21
- package/dist/AiSdkProvider.d.ts.map +0 -1
- package/dist/AiSdkProvider.js.map +0 -1
- package/dist/AiSdkRequestBody.d.ts.map +0 -1
- package/dist/AiSdkRequestBody.js.map +0 -1
- package/dist/RequestFields.d.ts +0 -24
- package/dist/RequestFields.d.ts.map +0 -1
- package/dist/RequestFields.js +0 -237
- package/dist/RequestFields.js.map +0 -1
- package/dist/aiSdkTransport.d.ts +0 -84
- package/dist/aiSdkTransport.d.ts.map +0 -1
- package/dist/aiSdkTransport.js +0 -651
- package/dist/aiSdkTransport.js.map +0 -1
- package/dist/model-options.d.ts +0 -3
- package/dist/model-options.d.ts.map +0 -1
- package/dist/model-options.js +0 -26
- package/dist/model-options.js.map +0 -1
- package/dist/sdkModels.d.ts.map +0 -1
- package/dist/sdkModels.js.map +0 -1
package/SPEC.md
CHANGED
|
@@ -1,1217 +1,136 @@
|
|
|
1
|
-
#
|
|
1
|
+
# `@plurnk/plurnk-providers` Specification
|
|
2
2
|
|
|
3
|
-
`@
|
|
4
|
-
`Provider`. It does not maintain a parallel vendor registry or reproduce
|
|
5
|
-
ordinary provider protocols.
|
|
3
|
+
Stable model-provider contract, capacity and monetary accounting authority, and transport adapter over `@earendil-works/pi-ai` (`@earendil-works/pi-ai/api/*` `stream`) and OpenAI-compatible HTTP endpoints.
|
|
6
4
|
|
|
7
|
-
## §
|
|
5
|
+
## §provider-interface Provider and runtime-neutral contract
|
|
8
6
|
|
|
9
|
-
|
|
10
|
-
| Tag | Promise |
|
|
11
|
-
| --- | --- |
|
|
12
|
-
| {§provider-runtime-neutral-accounting} | `@plurnk/plurnk-providers/accounting` — Provider-request aggregation, Models.dev cost estimation, and their… |
|
|
13
|
-
| {§provider-runtime-neutral-errors} | `@plurnk/plurnk-providers/errors` — Normalized `ProviderError`, `ProviderErrorKind`, and the types of its… |
|
|
14
|
-
| {§provider-runtime-neutral-openai} | `@plurnk/plurnk-providers/openai` — The OpenAI-compatible provider: SDK transport, request capture,… |
|
|
15
|
-
<!-- /map -->
|
|
7
|
+
`@plurnk/plurnk-providers` exposes three package entrypoints:
|
|
16
8
|
|
|
17
|
-
|
|
9
|
+
| Subpath | Surface | Runtime boundary |
|
|
10
|
+
|---|---|---|
|
|
11
|
+
| `@plurnk/plurnk-providers` | Full `Provider` contract, `ModelProvider`, `Pool`, `Mock`, `instantiateProvider`, `loadActiveProvider`, `discover`, and configuration helpers | Node >= 26 |
|
|
12
|
+
| `@plurnk/plurnk-providers/accounting` | `aggregateProviderAccounting`, `estimateProviderCost`, `providerCostNormalizer`, `resolveProviderCost`, `sumProviderCostsUsd`, `validateProviderUsage` | Runtime-neutral (browser-bundlable) |
|
|
13
|
+
| `@plurnk/plurnk-providers/errors` | `ProviderError`, `RETRYABLE_PROVIDER_KINDS`, `classifyProviderError`, `toProviderError`, `providerSource` | Runtime-neutral (browser-bundlable) |
|
|
18
14
|
|
|
19
|
-
|
|
20
|
-
endpoint, credential names, models, context/input/output limits, reasoning
|
|
21
|
-
capability, and USD rates including distinct reasoning rates when supplied.
|
|
22
|
-
2. Official AI SDK providers own vendor request and response protocols.
|
|
23
|
-
3. This package owns the PLURNK contract: aliases, envelopes, normalized usage
|
|
24
|
-
and errors, evidence, local capabilities, and first-party metadata.
|
|
25
|
-
4. The operator owns secrets, machine-specific endpoints, and deliberate
|
|
26
|
-
metadata overrides through environment variables.
|
|
15
|
+
Every `Provider` instance exposes:
|
|
27
16
|
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
17
|
+
| Member | Contract |
|
|
18
|
+
|---|---|
|
|
19
|
+
| Physical & policy getters | `model`, `servedModel`, `contextWindow`, `maxInputTokens`, `maxOutputTokens`, `outputBudget`, `outputFloor`, `reasoningBudget`, `inputCapacity`, `inputWall`, `supportedEfforts`, `inputModalities` (`ReadonlySet<"image" \| "audio" \| "video" \| "pdf">`), `constrainsOutput`, and `requiresOutputBudget` |
|
|
20
|
+
| `countPromptTokens(messages, signal?)` | Measures or estimates prompt token weight (`PromptTokenMeasurement`) |
|
|
21
|
+
| `assessRequestCapacity(messages, maxOutputTokens?, signal?)` | Evaluates `inputCapacity`, `inputWall`, admission `decision` (`"admit" \| "defer" \| "reject"`), and flexed `responseMax` |
|
|
22
|
+
| `generate(args)` | Requires non-empty `sessionId` (used for KV-cache and slot affinity), optional `callKind` (`"emission"` or `"bare"`), optional `grammar`, `maxOutputTokens`, `sampling`, `observeRequest`, and `observeReasoning`. Native tool calls are unconditionally disabled on every protocol |
|
|
23
|
+
| `observeRequest` | Opens before each physical HTTP attempt and settles once with `(requestAccounting, forensicEvidence)`. When `PLURNK_PROVIDERS_RAWBODY=1`, `forensicEvidence.request` records `{ origin, method, body }` with credentials and URL path/query secrets stripped |
|
|
24
|
+
| `observeReasoning` | Streams reasoning deltas live from structured reasoning events (`thinking_delta`) or leading tagged reasoning envelopes (`<think>...</think>` when `PLURNK_PROVIDERS_REASONING_RESPONSE_STYLE=think-tags`, or llama-server template envelopes on grammar-constrained turns) |
|
|
25
|
+
| `Pool` | Fronts $N \ge 1$ homogeneous backends sharing a `model` id: aggregates limits to conservative floors, pins each `sessionId` to one backend for KV-cache locality, and overflows to a healthy sibling on retryable availability failures (`network_failure`, `rate_limit`, `deadline_exceeded`) while never overflowing terminal errors or `resource_interrupted` attempts |
|
|
32
26
|
|
|
33
|
-
|
|
27
|
+
## §provider-capacity-admission Capacity and prompt measurement
|
|
28
|
+
|
|
29
|
+
`countPromptTokens` returns a `PromptTokenMeasurement` discriminated by `kind`:
|
|
34
30
|
|
|
35
|
-
|
|
|
36
|
-
|
|
37
|
-
|
|
|
38
|
-
|
|
|
39
|
-
|
|
|
31
|
+
| `kind` | Source & semantics |
|
|
32
|
+
|---|---|
|
|
33
|
+
| `"exact"` | Measured by the serving vocabulary (such as llama-server `/v1/chat/completions/input_tokens` or `Mock`) |
|
|
34
|
+
| `"upper_bound"` | Conservative maximum across a `Pool` of exact counters |
|
|
35
|
+
| `"estimate"` | Character heuristic (`ceil(codePoints / 2)`, `source: "heuristic:chars2"`) when no exact endpoint exists or when the endpoint degrades |
|
|
36
|
+
| `"unavailable"` | No measurement available |
|
|
40
37
|
|
|
41
|
-
|
|
42
|
-
Node-only provider discovery, filesystem defaults, or runtime construction.
|
|
43
|
-
The package root remains the Node provider-runtime composition surface and is
|
|
44
|
-
not a Worker entrypoint.
|
|
38
|
+
`assessRequestCapacity` computes:
|
|
45
39
|
|
|
46
|
-
|
|
40
|
+
- `inputCapacity = min(maxInputTokens, contextWindow - outputBudget)` (the soft curation reservation line).
|
|
41
|
+
- `inputWall = min(maxInputTokens, contextWindow - outputFloor)` (the hard physical wall; when `outputBudget` is tightened below `outputFloor`, the floor tightens with it).
|
|
42
|
+
- `decision`: `"reject"` only when `prompt.kind === "exact"` and `prompt.tokens > inputWall`; `"defer"` when a non-exact measurement exceeds `inputWall` or measurement is unavailable; `"admit"` otherwise.
|
|
43
|
+
- `responseMax`: the flexed wire output allowance (`min(maxOutputTokens, contextWindow - prompt.tokens - 256)` bounded below by `outputFloor` for exact measurements, or bounded above by `outputBudget` for estimates).
|
|
47
44
|
|
|
48
|
-
|
|
49
|
-
| Tag | Promise |
|
|
50
|
-
| --- | --- |
|
|
51
|
-
| {§provider-interface} | `Provider` exposes immutable model facts and one generation operation: |
|
|
52
|
-
| {§provider-prompt-measurement} | `PromptTokenMeasurement` is a discriminated request-level result: |
|
|
53
|
-
| {§provider-capacity-admission} | `assessRequestCapacity` intersects every known physical input constraint |
|
|
54
|
-
| {§provider-monetary-evidence} | One precedence path converts each physical provider request into {§provider-cost} evidence before the request… |
|
|
55
|
-
| {§provider-cache-identity} | `generate` requires a non-empty, stable, opaque `workerId`. |
|
|
56
|
-
| {§provider-call-kind} | `callKind` is either `emission` (the response is a PLURNK turn emission) or `bare` (the response is… |
|
|
57
|
-
| {§provider-native-tools-disabled} | Generation requests declare no native tools and select the SDK's portable `none` tool choice, for emissions… |
|
|
58
|
-
| {§provider-generation-completion} | Stream consumption follows the provider's completion boundary, subject to caller cancellation and configured… |
|
|
59
|
-
| {§provider-request-observer} | When a consumer supplies the request observer, the adapter opens one durable identity through it immediately… |
|
|
60
|
-
| {§provider-dispatched-request} | With `PLURNK_PROVIDERS_RAWBODY` enabled, installed SDK transports also retain the exact serialized generation… |
|
|
61
|
-
| {§provider-reasoning-observer} | When a consumer supplies `observeReasoning`, the provider synchronously delivers each exact, ordered,… |
|
|
62
|
-
| {§provider-finish-reason} | The finish reason is the AI SDK's unified classification |
|
|
63
|
-
| {§provider-tagged-reasoning} | Structured provider or SDK reasoning fields are authoritative. |
|
|
64
|
-
<!-- /map -->
|
|
45
|
+
## §provider-monetary-evidence Usage and monetary accounting
|
|
65
46
|
|
|
66
|
-
|
|
67
|
-
operation:
|
|
47
|
+
Every physical attempt settles a `ProviderRequestAccounting` record (`{ provider, model, outcome, status?, usage?, cost }` under {§provider-request-accounting}):
|
|
68
48
|
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
readonly reasoningBudget: number | null;
|
|
77
|
-
readonly inputCapacity: number | null;
|
|
78
|
-
readonly servedModel?: string;
|
|
79
|
-
readonly constrainsOutput?: boolean;
|
|
80
|
-
readonly requiresOutputBudget?: boolean;
|
|
49
|
+
- `normalizeUsage` validates non-negative safe integers, recovers additive reasoning when `prompt + completion + reasoning === total`, derives a single missing input cache partition (`noCacheTokens`, `cacheReadTokens`, `cacheWriteTokens`) when the other two and total are known, and rejects contradictory counters (`UsageDetailError`) while preserving valid outer totals.
|
|
50
|
+
- Cost precedence resolves in order:
|
|
51
|
+
1. Direct provider charge evidence (`kind: "charged"`, e.g. OpenRouter `usage.cost` and BYOK `upstream_inference_cost`).
|
|
52
|
+
2. Direct provider response estimate (`kind: "estimated"`, e.g. DeepInfra `usage.estimated_cost`).
|
|
53
|
+
3. Operator rate override (`PLURNK_PROVIDERS_COST=input=...,output=...,reasoning=...,cacheRead=...,cacheWrite=...`) over catalog rates (`kind: "estimated"`).
|
|
54
|
+
4. Catalog rates (`kind: "estimated"`). When a model has a distinct rate for a token category (`reasoning`, `cacheRead`, `cacheWrite`) that the response omitted, cost resolves to `kind: "unknown"` rather than guessing.
|
|
55
|
+
- `aggregateProviderAccounting` sums `usage` and `costUsd` across requests, returning `null` for `usage` or `costUsd` when any request's usage or cost is unknown while preserving `knownUsage` and `knownCostUsd` subtotals.
|
|
81
56
|
|
|
82
|
-
|
|
83
|
-
messages: readonly ChatMessage[],
|
|
84
|
-
signal?: AbortSignal,
|
|
85
|
-
): Promise<PromptTokenMeasurement>;
|
|
86
|
-
assessRequestCapacity(
|
|
87
|
-
messages: readonly ChatMessage[],
|
|
88
|
-
maxOutputTokens?: number,
|
|
89
|
-
signal?: AbortSignal,
|
|
90
|
-
): Promise<ProviderRequestCapacity>;
|
|
91
|
-
tokenize?(text: string): Promise<number[]>;
|
|
92
|
-
generate(args: GenerateArgs): Promise<ProviderResponse>;
|
|
93
|
-
}
|
|
94
|
-
```
|
|
57
|
+
## §provider-sdk-boundary SDK and transport boundary
|
|
95
58
|
|
|
96
|
-
`
|
|
97
|
-
{§model-fact-resolution}: the minimum of known model capacity and any stricter
|
|
98
|
-
operator cap. `null` means genuinely unknown; a consumer MUST NOT invent a
|
|
99
|
-
stand-in. The context-window knob is a hard cap, never a model-facing curation
|
|
100
|
-
pressure.
|
|
59
|
+
`ModelProvider.ts` delegates streaming, protocol framing, and vendor request assembly to `@earendil-works/pi-ai` (`@earendil-works/pi-ai/api/*` `stream`) across all supported APIs (`anthropic-messages`, `openai-completions`, `openai-responses`, `google-generative-ai`, `bedrock-converse-stream`):
|
|
101
60
|
|
|
102
|
-
|
|
103
|
-
|
|
61
|
+
- Prompt cache affinity (`PLURNK_PROVIDERS_CACHE_AFFINITY=1`) projects each provider's documented session key (`sessionId` -> `prompt_cache_key` / `x-session-id` / `x-session-affinity`).
|
|
62
|
+
- Stable-system cache write policy (`PLURNK_PROVIDERS_CACHE_WRITE_POLICY=stable-system`) attaches ephemeral cache retention (`cache_control: { type: "ephemeral" }`) on providers that support prompt cache breakpoints (Anthropic and Anthropic routes on OpenRouter).
|
|
104
63
|
|
|
105
|
-
|
|
106
|
-
| --- | --- | --- |
|
|
107
|
-
| `exact` | Exact count for the complete provider request. | May prove fit or overflow. |
|
|
108
|
-
| `upper_bound` | Proven upper bound for the complete provider request. | May prove fit; exceeding a limit does not prove overflow. |
|
|
109
|
-
| `estimate` | Empirical prediction with required causal `detail`. | Cannot admit or reject. |
|
|
110
|
-
| `unavailable` | No quantified measurement, with required causal `detail`. | Cannot admit or reject. |
|
|
64
|
+
## §provider-effort Reasoning effort and readable trace
|
|
111
65
|
|
|
112
|
-
|
|
113
|
-
integer `tokens`.
|
|
114
|
-
`countPromptTokens` receives the same messages supplied to `generate` and may
|
|
115
|
-
perform cancellable provider I/O. The common fallback is chars/2 over message
|
|
116
|
-
content; it is announced once and reported honestly as an estimate because it
|
|
117
|
-
knows neither the serving vocabulary nor provider-owned request framing.
|
|
118
|
-
An adapter may retain that estimate when an optional counting endpoint fails,
|
|
119
|
-
provided its detail names the cause; one unable to quantify anything returns
|
|
120
|
-
`unavailable`. A malformed measurement is a provider contract violation and
|
|
121
|
-
fails hard.
|
|
66
|
+
`PLURNK_PROVIDERS_EFFORT` selects one portable policy (`off`, `adaptive`, `low`, `medium`, `high`, `xhigh`, `max`, {§capability-policy-cascade}):
|
|
122
67
|
|
|
123
|
-
|
|
68
|
+
- `catalogEfforts(providerInfo, modelInfo, env)` intersects the model's cataloged effort vocabulary (extended by `PLURNK_PROVIDERS_PROVIDER_<ID>_REASONING_EFFORTS` for reasoning models) with what the provider adapter can preserve on the wire.
|
|
69
|
+
- On `adaptive`, routes with native adaptive reasoning (such as Anthropic adaptive thinking or Gemini thinking) request their native adaptive posture; graded effort routes use `PLURNK_PROVIDERS_EFFORT_FALLBACK` (default `high`) when supported by the model's effort set.
|
|
70
|
+
- When `PLURNK_PROVIDERS_REASONING_BUDGET` is set alongside `adaptive`, budget-capable routes send the token budget (`thinkingBudgets` / `thinking.budget_tokens` / `thinkingConfig.thinkingBudget` / `reasoning.max_tokens`); setting a fixed effort alongside `PLURNK_PROVIDERS_REASONING_BUDGET` is rejected at construction. On Google routes where `maxOutputTokens` is shared across `thoughtsTokenCount` and `candidatesTokenCount`, `thinkingConfig.thinkingBudget` is bounded by `maxOutputTokens - outputFloor` (or `maxOutputTokens - 1` when tightened below `outputFloor`) so visible output headroom is reserved.
|
|
71
|
+
- On detected llama-server routes, `off` and `adaptive` send `chat_template_kwargs: { enable_thinking: false | true }`, and fixed efforts declared in `PLURNK_PROVIDERS_REASONING_EFFORTS` add `reasoning_effort`. When grammar constrains a template-reasoning turn, the adapter sends `reasoning_format: "none"` so the grammar governs the raw stream and projects the leading template block back into `assistant.reasoning`.
|
|
124
72
|
|
|
125
|
-
|
|
126
|
-
| --- | --- |
|
|
127
|
-
| Exact fit | admit |
|
|
128
|
-
| Exact overflow of the wall | reject |
|
|
129
|
-
| A prompt between the curation reservation and the wall | admitted with its grant flexed down |
|
|
130
|
-
| A proven upper bound | admits only when it fits |
|
|
131
|
-
| Unknown limits, an upper bound above a limit, estimates, and unavailable measurements | defer to the upstream provider as capacity oracle |
|
|
73
|
+
## §provider-configuration Environment configuration and envelope
|
|
132
74
|
|
|
133
|
-
|
|
|
134
|
-
|
|
135
|
-
| `
|
|
136
|
-
| `
|
|
75
|
+
| Knob | Default / Form | Effect |
|
|
76
|
+
|---|---|---|
|
|
77
|
+
| `PLURNK_PROVIDERS_OUTPUT_BUDGET` | `25%` (`<pct>%` or positive integer tokens) | Soft output reservation subtracted from `contextWindow` to form `inputCapacity` |
|
|
78
|
+
| `PLURNK_PROVIDERS_OUTPUT_FLOOR` | `25%` (`<pct>%` of effective output ceiling or integer tokens) | Hard minimum output allowance subtracted from `contextWindow` to form `inputWall` |
|
|
79
|
+
| `PLURNK_PROVIDERS_REASONING_BUDGET` | empty (`<pct>%` or integer tokens, `< outputBudget`) | Explicit thinking token budget under `adaptive` effort |
|
|
80
|
+
| `PLURNK_PROVIDERS_EFFORT` / `EFFORT_FALLBACK` | `adaptive` / `high` | Portable reasoning effort selector and graded-route fallback |
|
|
81
|
+
| `PLURNK_PROVIDERS_FETCH_TIMEOUT` / `OPERATION_TIMEOUT` | `30000` / `600000` ms | Per-attempt fetch/idle timeout and total operation wall-clock timeout |
|
|
82
|
+
| `PLURNK_PROVIDERS_RETRY_ATTEMPTS` | `3` | Maximum automatic retry attempts for retryable transport failures |
|
|
83
|
+
| `PLURNK_PROVIDERS_MAX_CONCURRENCY` | `-1` (unbounded or positive integer) | Per-endpoint FIFO concurrency gate ({§provider-inference-admission}) |
|
|
84
|
+
| `PLURNK_PROVIDERS_RAWBODY` | `0` (`0 \| 1`) | Retains redacted raw request/response forensic bodies in `forensicEvidence` |
|
|
137
85
|
|
|
138
|
-
`
|
|
139
|
-
|
|
140
|
-
for
|
|
86
|
+
- Every knob in `PROVIDERS_KNOBS` is documented in `plurnk-providers/.env.defaults` and supports case-insensitive per-alias suffix scoping (`<KNOB>_<ALIAS>`) via `scopeEnvToAlias(env, alias)`.
|
|
87
|
+
- Sampling knobs (`TEMPERATURE`, `TOP_P`, `TOP_K`, `PRESENCE_PENALTY`, `FREQUENCY_PENALTY`, `SEED`, `REPEAT_PENALTY`, `REPEAT_LAST_N`, `DRY_MULTIPLIER`, `DRY_BASE`, `DRY_ALLOWED_LENGTH`) validate at construction; caller `sampling` overrides configured defaults on a per-call basis, and `DRY_MULTIPLIER=0` suppresses all DRY fields.
|
|
88
|
+
- `instantiateProvider` resolves cataloged providers and `ollama` in-framework before scanning `node_modules` for third-party provider extensions (gated by `PLURNK_EXTENSIONS_TRUSTED_ONLY`). `providerReadiness(provider, env)` reports missing credentials or endpoint configuration without exposing secret values.
|
|
141
89
|
|
|
142
|
-
§
|
|
143
|
-
request into {§provider-cost} evidence before the request leaves the provider
|
|
144
|
-
boundary:
|
|
90
|
+
## §model-fact-resolution Route facts and local probes
|
|
145
91
|
|
|
146
|
-
|
|
|
147
|
-
|
|
148
|
-
|
|
|
149
|
-
|
|
|
150
|
-
|
|
|
92
|
+
| Route family | Fact authority |
|
|
93
|
+
|---|---|
|
|
94
|
+
| Cataloged cloud provider (`catalogProvider.ts`) | `resolveModel(provider, model)` from `@plurnk/plurnk-models`, capped by optional `PLURNK_PROVIDERS_CONTEXT_WINDOW`. Uncataloged models on a catalog provider require an explicit `PLURNK_PROVIDERS_CONTEXT_WINDOW`. Catalog credential names (`info.env`) or `PLURNK_PROVIDERS_PROVIDER_<ID>_API_KEY_ENV` (one exact variable name, never a comma list) supply authentication. |
|
|
95
|
+
| Local / compatible endpoint (`compatibleProvider.ts`) | Probes `GET <baseUrl>/models` for `n_ctx` / `max_model_len` / `context_length`. When llama-server is detected (`meta.n_ctx` or `PLURNK_PROVIDERS_LLAMA_SERVER=1`), probes `/props` for `total_slots` and `eos_token` and measures prompts via `/v1/chat/completions/input_tokens`. |
|
|
96
|
+
| Built-in Ollama rail (`ollama.ts`) | Probes `POST <origin>/api/show` for `*.context_length`, capped by optional `PLURNK_PROVIDERS_CONTEXT_WINDOW`, and dispatches chat completions to `<origin>/v1/chat/completions`. |
|
|
97
|
+
| Operator probe scripts (`scripts/`) | `providers-ping.mjs` probes one cheapest declared route per keyed provider and writes redacted JSON evidence; `token-envelope-probe.mjs` probes explicit aliases across configured or physical output limits. |
|
|
151
98
|
|
|
152
|
-
|
|
153
|
-
applicable category, or missing rates never proves zero. An exact zero rate
|
|
154
|
-
produces an ordinary estimated amount of USD `0`. Rate calculation is internal
|
|
155
|
-
to the provider request; Core, digest, ping, and clients never call a parallel
|
|
156
|
-
pricing method.
|
|
99
|
+
## §provider-request-authority Request controls and grammar transport
|
|
157
100
|
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
101
|
+
- Caller `sampling` and extension body fields cannot override protocol-owned fields (`model`, `messages`, `stream`, `tools`, `tool_choice`, `response_format`, `grammar`, `logprobs`, `top_logprobs`).
|
|
102
|
+
- `PLURNK_PROVIDERS_SERVICE_TIER` and `PLURNK_PROVIDERS_TOP_LOGPROBS` project onto native SDK requests only when the route's SDK documents the control; unsupported controls emit a single `PLURNK_REQUEST_CONTROL_UNSUPPORTED` process warning and are omitted from the wire.
|
|
103
|
+
- OpenRouter requests attach `HTTP-Referer` and `X-OpenRouter-Title` from `PLURNK_PROVIDERS_PROVIDER_OPENROUTER_APP_URL` (validated as an absolute HTTP(S) URL) and `PLURNK_PROVIDERS_PROVIDER_OPENROUTER_APP_NAME`.
|
|
104
|
+
- Grammar strings (`generate({ grammar })`) are transported only when `grammarStyle === "llamacpp"` (`constrainsOutput === true`), preserving `grammarEvidence` (`{ input, contentStart, transported }`) on the response. When `tokenize` is available on a grammar turn and billed `outputTokens` exceed visible content + reasoning tokens by more than 64, an `output_unaccounted` notice is attached ({§diagnostic-observation}).
|
|
105
|
+
|
|
106
|
+
## §provider-failure-normalization Failure normalization and evidence
|
|
107
|
+
|
|
108
|
+
All transport and response failures normalize into `ProviderError` carrying an RFC 9457 Problem Details payload (`https://problems.plurnk.xyz/provider/<providerId>/<kind>`, {§diagnostic-observation}):
|
|
109
|
+
|
|
110
|
+
| `ProviderErrorKind` | HTTP / trigger | Problem status | `retryable` | Extensions |
|
|
111
|
+
|---|---|---|---|---|
|
|
112
|
+
| `unauthorized` | 401, 403 | 401 / 403 | `false` | `cause` |
|
|
113
|
+
| `quota_exceeded` | 402 or billing/quota exhaustion | 402 | `false` | `cause` |
|
|
114
|
+
| `rate_limit` | 429 | 429 | `true` | `cause` |
|
|
115
|
+
| `capacity_exceeded` | Preflight exact wall overflow, 413, or upstream context-length code | 413 | `false` | `cause` |
|
|
116
|
+
| `grammar_invalid` | 422 with `error.type === "grammar_invalid"` | 422 | `false` | `cause` |
|
|
117
|
+
| `request_rejected` | Other 400, 404, 405, 422 | 400..422 | `false` | `cause` |
|
|
118
|
+
| `network_failure` | 408, 409, 5xx, socket/TLS/proxy error, or mid-stream disconnect | 502 / 503 | `true` | `cause`, optional `attempt` |
|
|
119
|
+
| `deadline_exceeded` | `PLURNK_PROVIDERS_FETCH_TIMEOUT` or `PLURNK_PROVIDERS_OPERATION_TIMEOUT` | 504 | `true` | `cause`, optional `attempt` |
|
|
120
|
+
| `resource_interrupted` | `finishReason === "resource_interrupted"` | 503 | `true` | `attempt` |
|
|
121
|
+
| `generation_failed` | `finishReason === "error"` | 502 | `false` | `attempt` |
|
|
122
|
+
| `invalid_response` | Malformed 2xx payload, missing finish reason, invalid usage, or output exceeding granted allowance | 502 | `false` | `cause`, optional `attempt` |
|
|
123
|
+
| `model_refused` | Upstream safety/content refusal finish reason | 422 | `false` | `attempt` |
|
|
162
124
|
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
### Generation
|
|
176
|
-
|
|
177
|
-
§provider-cache-identity `generate` requires a non-empty, stable, opaque
|
|
178
|
-
`workerId`. A durable worker uses one globally unique value for its lifetime;
|
|
179
|
-
independent databases and processes cannot mint the same local sequence. A
|
|
180
|
-
`bare` call instead uses a fresh per-call value, preventing unrelated prompts
|
|
181
|
-
from acquiring affinity with either the parent worker or another BARE call.
|
|
182
|
-
Providers MUST NOT interpret either value.
|
|
183
|
-
|
|
184
|
-
`generate` accepts:
|
|
185
|
-
|
|
186
|
-
- `messages`: system, user, and assistant text messages;
|
|
187
|
-
- caller cancellation through `signal`;
|
|
188
|
-
- optional `grammar` and call-specific `maxOutputTokens` tightening;
|
|
189
|
-
- standard `sampling` intent;
|
|
190
|
-
- the caller-owned `callKind` output contract when one applies;
|
|
191
|
-
- opaque attribution tags plus client, strike, workspace, loop, and turn metadata.
|
|
192
|
-
|
|
193
|
-
§provider-call-kind `callKind` is either `emission` (the response is a PLURNK
|
|
194
|
-
turn emission) or `bare` (the response is unconstrained answer text). The
|
|
195
|
-
consumer states this semantic fact explicitly; providers MUST NOT infer it from
|
|
196
|
-
message count, grammar presence, worker identity, or another incidental request
|
|
197
|
-
shape. The signal never enters model-facing messages. Generic
|
|
198
|
-
provider callers MAY omit it; Core supplies it for every model call.
|
|
199
|
-
|
|
200
|
-
§provider-native-tools-disabled Generation requests declare no native tools and
|
|
201
|
-
select the SDK's portable `none` tool choice, for emissions and BARE alike.
|
|
202
|
-
The built-in transports preserve that intent after SDK serialization:
|
|
203
|
-
|
|
204
|
-
| Protocol | No-tool wire representation |
|
|
205
|
-
| --- | --- |
|
|
206
|
-
| OpenAI-compatible Chat Completions and Responses | `tools: []`, `tool_choice: "none"` |
|
|
207
|
-
| Anthropic Messages | `tools: []`, `tool_choice: {"type":"none"}` |
|
|
208
|
-
| Gemini GenerateContent | `tools: []`, `toolConfig: {"functionCallingConfig":{"mode":"NONE"}}` |
|
|
209
|
-
| Bedrock Converse | No tool configuration; the protocol has no `none` choice. |
|
|
210
|
-
| Installed SDK provider extensions | Portable `none` intent; the extension owns its wire projection. |
|
|
211
|
-
|
|
212
|
-
This is a text-generation contract, not provider/model tuning or prompt
|
|
213
|
-
teaching. Callers cannot supply tools or override the choice through request
|
|
214
|
-
extensions. Do not add deprecated function-call fields, dummy tools, or fields
|
|
215
|
-
from another protocol. The opt-out neither constrains arbitrary prose nor
|
|
216
|
-
proves that an endpoint honors it; unexpected native calls retain their full
|
|
217
|
-
evidence and follow {§provider-native-tool-calls}.
|
|
218
|
-
|
|
219
|
-
A successful return carries the model's raw content and reasoning, normalized
|
|
220
|
-
finish reason, model identity, its ordered {§provider-request-accounting}, the
|
|
221
|
-
request's `ProviderRequestCapacity`, opaque evidence, optional metadata, and
|
|
222
|
-
optional notices. A `ProviderError` carries the same available capacity and
|
|
223
|
-
accounting evidence. The provider transports and observes model
|
|
224
|
-
output; it never retries, discards, or repairs an otherwise completed exchange
|
|
225
|
-
because PLURNK grammar did not accept it.
|
|
226
|
-
|
|
227
|
-
§provider-generation-completion Stream consumption follows the provider's
|
|
228
|
-
completion boundary, subject to caller cancellation and configured deadlines.
|
|
229
|
-
Repeated content or reasoning does not interrupt a response or request another
|
|
230
|
-
generation. Preserve the complete response, its finish reason, and trailing
|
|
231
|
-
usage and charge evidence before returning it to the consumer.
|
|
232
|
-
|
|
233
|
-
§provider-request-observer When a consumer supplies the request observer, the
|
|
234
|
-
adapter opens one durable identity through it immediately before each physical
|
|
235
|
-
I/O and settles that identity with the resulting
|
|
236
|
-
`ProviderRequestAccounting`. This applies to generation retries and capacity
|
|
237
|
-
failover. The
|
|
238
|
-
observer is a durability sink, not an alternate evidence representation; the
|
|
239
|
-
same ordered records remain on the final generation result or
|
|
240
|
-
error.
|
|
241
|
-
|
|
242
|
-
The adapter also settles {§provider-request-evidence}. A successful request
|
|
243
|
-
retains its normalized transport response; a failed request retains received
|
|
244
|
-
SDK raw chunks (or the error response body), partial content and reasoning,
|
|
245
|
-
available response identity/headers, usage/charge evidence, and its error and
|
|
246
|
-
cancellation cause. Failure capture is independent of successful raw-body
|
|
247
|
-
capture. These are the SDK's received values, not a claim of byte-for-byte
|
|
248
|
-
HTTP capture. Settlement is durable before returning the failure; a process
|
|
249
|
-
crash before settlement leaves a pending request, not an invented response.
|
|
250
|
-
|
|
251
|
-
§provider-dispatched-request With `PLURNK_PROVIDERS_RAWBODY` enabled, installed
|
|
252
|
-
SDK transports also retain the exact serialized generation request body at
|
|
253
|
-
their injected fetch boundary, after all owned body transformations. Capture
|
|
254
|
-
is scoped to the physical request and settles with its existing forensic
|
|
255
|
-
evidence on success or failure; it does not reconstruct a body from settings.
|
|
256
|
-
Only the HTTP method, origin and body are recorded: no request headers,
|
|
257
|
-
userinfo, path, query or fragment. The body itself contains the supplied model
|
|
258
|
-
input and must be treated as sensitive evidence. An external SDK model whose
|
|
259
|
-
transport is not instrumented, disabled capture, or an unsettled request makes
|
|
260
|
-
no capture claim. Existing accounting, failure evidence, cancellation and
|
|
261
|
-
retention are unchanged.
|
|
262
|
-
|
|
263
|
-
Native SDK construction binds each physical request's injected fetch after
|
|
264
|
-
owned body transforms; compatible routes use the same request-scoped closure.
|
|
265
|
-
Capture adds no global interception, runtime-specific context storage or
|
|
266
|
-
cross-request mutable recorder. An externally supplied model instance remains
|
|
267
|
-
opaque; a model factory can accept the injected fetch to participate.
|
|
268
|
-
|
|
269
|
-
§provider-reasoning-observer When a consumer supplies `observeReasoning`, the
|
|
270
|
-
provider synchronously delivers each exact, ordered, nonempty readable-reasoning
|
|
271
|
-
delta as it becomes available. A transport without incremental reasoning emits
|
|
272
|
-
the complete normalized value once before resolving. This is transient
|
|
273
|
-
observation, not response authority: `ProviderResponse` or `ProviderError`
|
|
274
|
-
remains the complete settled evidence. An automatically retried physical
|
|
275
|
-
request may therefore expose a partial failed-attempt prefix before a later
|
|
276
|
-
attempt settles. The consumer uses each `observeRequest` opening as the boundary
|
|
277
|
-
between those physical-request reasoning streams: it preserves the failed
|
|
278
|
-
prefix, gives the retry a distinct presentation identity, and never concatenates
|
|
279
|
-
separate stochastic attempts into one reasoning message.
|
|
280
|
-
|
|
281
|
-
Usage obeys {§provider-usage}:
|
|
282
|
-
|
|
283
|
-
```text
|
|
284
|
-
totalTokens = inputTokens + outputTokens
|
|
285
|
-
cacheReadTokens, cacheWriteTokens ⊆ inputTokens
|
|
286
|
-
reasoningTokens ⊆ outputTokens
|
|
287
|
-
```
|
|
288
|
-
|
|
289
|
-
Unknown fields remain absent.
|
|
290
|
-
|
|
291
|
-
§provider-finish-reason **The finish reason is the AI SDK's unified classification**, never a
|
|
292
|
-
second reading of the vendor's string: `stop`, `length`,
|
|
293
|
-
`tool-calls` → `tool_calls`, `content-filter` → `content_filter`; `error` fails the attempt
|
|
294
|
-
({§provider-generation-failed}); `other` is `null`. The raw vendor value rides beside it as
|
|
295
|
-
`rawFinishReason`. One upstream semantic the SDK folds into `other` stays distinct: the exact
|
|
296
|
-
`insufficient_system_resource` is `resource_interrupted`, the failed-attempt disposition defined by
|
|
297
|
-
{§provider-interrupted-attempt}.
|
|
298
|
-
|
|
299
|
-
### Tagged reasoning responses
|
|
300
|
-
|
|
301
|
-
§provider-tagged-reasoning Structured provider or SDK reasoning fields are
|
|
302
|
-
authoritative. Visible content is interpreted as tagged reasoning only under an
|
|
303
|
-
explicit alias-scoped response style:
|
|
304
|
-
|
|
305
|
-
| Effective style | Leading content | Normalized result |
|
|
306
|
-
| --------------- | ---------------------------------------------------------- | --------------------------------------------------------------------------------------------------------- |
|
|
307
|
-
| `verbatim` | Any bytes. | Content remains exact; only structured reasoning fields populate `reasoning`. |
|
|
308
|
-
| `think-tags` | No exact leading `<think>`. | Content remains exact. |
|
|
309
|
-
| `think-tags` | `<think>reasoning</think>visible`. | The first envelope body becomes reasoning; the exact suffix becomes content. Later tags remain literal. |
|
|
310
|
-
| `think-tags` | `<think>reasoning` with no close, including a capped turn. | The complete post-open tail becomes reasoning; content is empty. |
|
|
311
|
-
|
|
312
|
-
Tag projection never runs when readable structured reasoning is already
|
|
313
|
-
present. Streamed and buffered transports use the same leading-envelope parser.
|
|
314
|
-
An enabled tagged projection delivers readable deltas as content chunks arrive,
|
|
315
|
-
holding only incomplete delimiter prefixes; it does not wait for the closing
|
|
316
|
-
tag or response completion. An unfinished closing prefix is literal reasoning
|
|
317
|
-
at EOF. Constrained template calls that retain the grammar sentence use the
|
|
318
|
-
same path for `<think>\n…</think>` and `<|channel>thought\n…<channel|>`.
|
|
319
|
-
Observation resets for each physical request and completed reasoning is not
|
|
320
|
-
replayed. Neither the internal working copy nor its curation supplies this stream.
|
|
321
|
-
When the upstream reports only combined output usage, that value remains
|
|
322
|
-
`outputTokens` and its unavailable text/reasoning detail stays absent. The
|
|
323
|
-
adapter never apportions tokens from character lengths.
|
|
324
|
-
|
|
325
|
-
Grammar evidence retains the exact pre-projection sentence and its Unicode
|
|
326
|
-
content offset. Response classification cannot rewrite what a transported GBNF
|
|
327
|
-
rail observed.
|
|
328
|
-
|
|
329
|
-
## §3 AI SDK boundary
|
|
330
|
-
|
|
331
|
-
<!-- map -->
|
|
332
|
-
| Tag | Promise |
|
|
333
|
-
| --- | --- |
|
|
334
|
-
| {§provider-sdk-boundary} | Cataloged providers instantiate their Models.dev-declared AI SDK package. |
|
|
335
|
-
| {§operator-cost-override} | Models.dev is the rate starting point; the operator may overlay it. |
|
|
336
|
-
| {§provider-effort} | The portable vocabulary comes from {§effort-wire}. |
|
|
337
|
-
| {§provider-readable-reasoning} | Route-owned options may request exposed reasoning text. |
|
|
338
|
-
| {§provider-sdk-warning} | AI SDK compatibility, unsupported-feature, deprecation, and other call warnings become source-attributed… |
|
|
339
|
-
| {§provider-cache-affinity} | Cache affinity is route-owned request projection. |
|
|
340
|
-
| {§provider-model-options} | A model family's native options are a provider declaration. |
|
|
341
|
-
| {§provider-cache-write-policy} | Cache-write policy is separate from affinity. |
|
|
342
|
-
| {§deepseek-reasoning-request} | Direct DeepSeek uses the panel's request-field declarations under {§provider-wire-declaration}. |
|
|
343
|
-
| {§provider-wire-declaration} | Request-field declarations |
|
|
344
|
-
<!-- /map -->
|
|
345
|
-
|
|
346
|
-
§provider-sdk-boundary Cataloged providers instantiate their
|
|
347
|
-
Models.dev-declared AI SDK package.
|
|
348
|
-
Standard request shaping, streaming, usage, and vendor error parsing belong to
|
|
349
|
-
the SDK. PLURNK supplies cancellation and deadline signals and owns the sole
|
|
350
|
-
cross-attempt scheduler so every physical request remains observable and
|
|
351
|
-
accountable. The adapter hands the SDK the conversation as authored: a leading
|
|
352
|
-
system message reaches the wire as the request's system message.
|
|
353
|
-
|
|
354
|
-
| Usage input | Normalization |
|
|
355
|
-
| --- | --- |
|
|
356
|
-
| Native protocol | Preserve the SDK's inclusive totals and category counts; raw protocol counters are evidence, not interchangeable totals. |
|
|
357
|
-
| OpenAI-compatible wire | Retain extended counters and exact total identities the SDK may omit. |
|
|
358
|
-
| Compatible SDK input partition | Retain its uncached count when total input and an explicitly reported cache-read count agree with the wire; never promote SDK defaults over absent or contradictory wire counts. |
|
|
359
|
-
| Input total plus two partition counts | Derive the remaining count exactly; otherwise preserve absence. |
|
|
360
|
-
|
|
361
|
-
PLURNK maps its generic settings to AI SDK call settings:
|
|
362
|
-
|
|
363
|
-
- `temperature`, `top_p`, `top_k`;
|
|
364
|
-
- presence and frequency penalties;
|
|
365
|
-
- stop sequences and seed;
|
|
366
|
-
- output-token ceiling;
|
|
367
|
-
- the supported efforts and optional numeric control under
|
|
368
|
-
{§provider-effort}.
|
|
369
|
-
|
|
370
|
-
Provider-specific options are permitted only where they preserve a documented
|
|
371
|
-
PLURNK product contract the generic SDK surface cannot express.
|
|
372
|
-
|
|
373
|
-
§operator-cost-override **Models.dev is the rate starting point; the operator may
|
|
374
|
-
overlay it.** `PLURNK_PROVIDERS_COST` declares exact per-1M-token USD rates as
|
|
375
|
-
comma-separated `key=value` over the catalog vocabulary (`input`, `output`,
|
|
376
|
-
`reasoning`, `cacheRead`, `cacheWrite`), alias-scoped like every provider knob.
|
|
377
|
-
Declared keys merge over the Models.dev cost block; undeclared keys keep the
|
|
378
|
-
catalog figure. Without catalog rates the override must declare `input` and
|
|
379
|
-
`output`. The estimate's `source` names the override, and a provider-reported
|
|
380
|
-
response cost still outranks any estimate. Unknown keys, repeats, and negative
|
|
381
|
-
or non-numeric rates refuse at construction.
|
|
382
|
-
|
|
383
|
-
§provider-effort The portable vocabulary comes from
|
|
384
|
-
{§effort-wire}. `adaptive` uses the first applicable projection:
|
|
385
|
-
|
|
386
|
-
| Condition | Projection |
|
|
387
|
-
| --- | --- |
|
|
388
|
-
| Explicit adaptive declaration: `REASONING_ADAPTIVE_BODY`, or a native model family's `ADAPTIVE_OPTIONS` ({§provider-model-options}) | Preserve that mechanism; an explicit `{}` retains the enabled endpoint's default. |
|
|
389
|
-
| The configured `PLURNK_PROVIDERS_EFFORT_FALLBACK` is supported by both model and transport | Send that exact effort. The shipped fallback is `high`, not the strongest available level. |
|
|
390
|
-
| No supported fallback, including an empty fallback setting | Retain reasoning activation and the provider's default effort; never invent a level or escalate to another one. |
|
|
391
|
-
|
|
392
|
-
Activation is distinct from effort: declared enable fields accompany active
|
|
393
|
-
reasoning, and a toggle-only route uses its documented enable mechanism.
|
|
394
|
-
Non-reasoning models receive no reasoning controls; `off` is the explicit opt-out.
|
|
395
|
-
A fixed policy retains its exact name and is rejected before provider I/O when
|
|
396
|
-
either the route or transport cannot represent it. Every provider exposes that
|
|
397
|
-
exact intersection. A numeric reasoning budget constrains
|
|
398
|
-
the generation envelope independently and never selects or changes policy. On
|
|
399
|
-
routes whose controls are exclusive, fixed effort plus a numeric budget is
|
|
400
|
-
rejected before I/O. Under `adaptive`, an explicit budget selects the numeric
|
|
401
|
-
control; `off` suppresses it.
|
|
402
|
-
|
|
403
|
-
`catalogEfforts` projects that same admission calculation from catalog
|
|
404
|
-
facts and provider-wide environment declarations over the installed defaults,
|
|
405
|
-
without constructing a model, requiring credentials, or performing provider I/O.
|
|
406
|
-
Admission respects the installed projection: native SDK fixed efforts exclude
|
|
407
|
-
`max`; declared transport vocabularies constrain per-call option projections.
|
|
408
|
-
|
|
409
|
-
Models.dev's route-specific `reasoning_options` is the capability authority for
|
|
410
|
-
what the daemon offers on its own; the daemon never adds vendor behaviour absent
|
|
411
|
-
from the catalog. The operator may declare efforts a provider's reasoning routes
|
|
412
|
-
accept (`PLURNK_PROVIDERS_PROVIDER_<PREFIX>_REASONING_EFFORTS`, comma-separated
|
|
413
|
-
portable names); declared efforts join the catalog's for policy admission and the
|
|
414
|
-
`adaptive` projection under the same transport rule, a declared `none` admits
|
|
415
|
-
`off` where the transport can express it, and a host that rejects a declared
|
|
416
|
-
effort answers on the first request rather than at boot. The declaration never
|
|
417
|
-
turns a non-reasoning route into a reasoning one. The installed AI SDK or
|
|
418
|
-
explicit compatible adapter owns the wire projection:
|
|
419
|
-
|
|
420
|
-
| Catalog fact | `adaptive` projection | Portable policies admitted in addition to `adaptive` |
|
|
421
|
-
| --- | --- | --- |
|
|
422
|
-
| `reasoning: false` | No reasoning request | `off` |
|
|
423
|
-
| `reasoning_options: []` | Provider default | None |
|
|
424
|
-
| `effort.values` | Native dynamic mechanism, otherwise the supported configured fallback or provider default | Transportable fixed members of {§effort-wire}; `off` only when `none` is transportable |
|
|
425
|
-
| `toggle` | Native or explicitly declared activation, otherwise provider default | `off` only when that transport owns the toggle wire |
|
|
426
|
-
| `budget_tokens` | Does not select policy | None; an adapter may use its bounds when projecting the independent budget |
|
|
427
|
-
| No catalog entry | Explicit adapter declaration | Only the declaration's exact subset |
|
|
428
|
-
| Operator-declared efforts | Join the catalog's `effort.values` for that provider's reasoning routes | Their exact members, under the transport rule above |
|
|
429
|
-
|
|
430
|
-
Models.dev identifies the route's controls but not a provider-specific toggle
|
|
431
|
-
or budget field name. The adapter supplies that last-mile mechanism; it never
|
|
432
|
-
invents a cataloged effort value.
|
|
433
|
-
|
|
434
|
-
§provider-readable-reasoning Route-owned options may request exposed reasoning
|
|
435
|
-
text. Normalization follows {§provider-open-reasoning}; Models.dev's reasoning
|
|
436
|
-
bit is capability metadata, not a statement about returned text's fidelity.
|
|
437
|
-
Billing without exposed text does not create a reasoning source.
|
|
438
|
-
|
|
439
|
-
§provider-sdk-warning AI SDK compatibility, unsupported-feature, deprecation,
|
|
440
|
-
and other call warnings become source-attributed provider Notices on the
|
|
441
|
-
successful exchange. A lossy adapter projection is therefore observable rather
|
|
442
|
-
than disappearing in transport internals.
|
|
443
|
-
|
|
444
|
-
§provider-cache-affinity **Cache affinity is route-owned request projection.**
|
|
445
|
-
When a provider documents a semantics-preserving conversation, session, or
|
|
446
|
-
prompt-cache routing key, its catalog adapter projects `workerId` through that
|
|
447
|
-
provider's documented header, body field, or native SDK option. The common
|
|
448
|
-
transport neither guesses from protocol resemblance nor sends a generic cache
|
|
449
|
-
field to an unknown provider. The operator may disable affinity globally or per
|
|
450
|
-
alias; automatic provider caching without an affinity control remains untouched.
|
|
451
|
-
The environment's `CACHE_AFFINITY_FIELD` declares that placement as
|
|
452
|
-
`{"target":"header"|"body","name":"…"}` or
|
|
453
|
-
`{"target":"provider-option","provider":"…","name":"…"}`. It follows
|
|
454
|
-
the provider/route/alias precedence of {§provider-wire-declaration}; `null`
|
|
455
|
-
clears the declaration. Placement is independent of the enable switch, cannot
|
|
456
|
-
replace transport-owned fields, and does not imply support from a provider's name.
|
|
457
|
-
|
|
458
|
-
§provider-model-options **A model family's native options are a provider declaration.** Models.dev
|
|
459
|
-
does not say which models on a native SDK take adaptive reasoning or explicit cache writes, and the SDKs
|
|
460
|
-
do not export their tables, so a provider declares them as data: `ADAPTIVE_OPTIONS` and
|
|
461
|
-
`SYSTEM_CACHE_OPTIONS` are JSON arrays of `{"models":["<glob>",…],"options":{…}}` rules. The first rule
|
|
462
|
-
with a glob matching the route's model id (`path.matchesGlob`) supplies its per-call provider options;
|
|
463
|
-
without a match none apply. They follow the provider/route/alias precedence of
|
|
464
|
-
{§provider-wire-declaration}, an empty value declares none, and a malformed value fails construction.
|
|
465
|
-
None ships: Plurnk's shipped defaults serve open models only, and an operator who routes a closed model
|
|
466
|
-
family declares its options in their own configuration.
|
|
467
|
-
|
|
468
|
-
§provider-cache-write-policy **Cache-write policy is separate from affinity.**
|
|
469
|
-
`PLURNK_PROVIDERS_CACHE_WRITE_POLICY` is `off` or `stable-system`. The latter
|
|
470
|
-
marks only the final leading system instruction as an explicit reusable cache
|
|
471
|
-
boundary, and only on routes whose provider declares the control in
|
|
472
|
-
`SYSTEM_CACHE_OPTIONS` ({§provider-model-options}). It does
|
|
473
|
-
not mark the changing user packet or enable an API-wide automatic cache mode.
|
|
474
|
-
Unsupported routes receive no invented option. The default five-minute
|
|
475
|
-
provider lifetime is used; a longer, differently priced lifetime is not an
|
|
476
|
-
implicit transport choice.
|
|
477
|
-
|
|
478
|
-
§deepseek-reasoning-request Direct DeepSeek uses the panel's request-field
|
|
479
|
-
declarations under {§provider-wire-declaration}. Models.dev supplies each
|
|
480
|
-
model's effort vocabulary; it is not a second fixed list in this adapter.
|
|
481
|
-
|
|
482
|
-
The compatible transport is deliberately retained for:
|
|
483
|
-
|
|
484
|
-
- `openai` local endpoints, including llama-server and vLLM;
|
|
485
|
-
- `ollama`, after its native `/api/show` probe;
|
|
486
|
-
- operator-declared `@ai-sdk/openai-compatible` providers.
|
|
487
|
-
|
|
488
|
-
It carries PLURNK-only fields and raw wire evidence without reimplementing the
|
|
489
|
-
SDK's ordinary transport.
|
|
490
|
-
|
|
491
|
-
### §provider-wire-declaration Request-field declarations
|
|
492
|
-
|
|
493
|
-
Models.dev owns model capabilities, effort vocabulary, and numeric budget bounds.
|
|
494
|
-
For endpoints whose field names it does not describe, the provider's
|
|
495
|
-
environment panel supplies a bounded projection, independent of provider identity:
|
|
496
|
-
|
|
497
|
-
| Declaration suffix | Meaning |
|
|
498
|
-
| --- | --- |
|
|
499
|
-
| `OPTIONS_NAMESPACE` | Native SDK's per-call `providerOptions` namespace. Without one, declared fields target the compatible request body. |
|
|
500
|
-
| `OUTPUT_PATH` | RFC 6901 object-member pointer for the **inclusive** output ceiling; absent uses the compatible SDK's `max_tokens` field. |
|
|
501
|
-
| `REASONING_EFFORT_PATH` / `REASONING_BUDGET_PATH` | Pointers for exact effort or numeric reasoning subset. No pointer means no such control. |
|
|
502
|
-
| `REASONING_EFFORTS` | Additional declared effort values, unioned with Models.dev. |
|
|
503
|
-
| `REASONING_TRANSPORT_EFFORTS` | Optional transport vocabulary, intersected with catalog/declaration efforts. It adds no model capability. |
|
|
504
|
-
| `REASONING_CONTROLS` | Required when both pointers exist: `exclusive` refuses fixed effort plus budget; `combined` sends both. |
|
|
505
|
-
| `REASONING_ON_BODY` | Static reasoning activation fields, merged with the selected control. |
|
|
506
|
-
| `REASONING_OFF_BODY` | Explicit disable fields; otherwise a declared `none` effort can disable. |
|
|
507
|
-
| `REASONING_ADAPTIVE_BODY` | Explicit adaptive fields, ahead of graded fallback; `{}` retains the enabled endpoint's default. |
|
|
508
|
-
| `REASONING_TOGGLE_BODY` | Reasoning enable fields when Models.dev declares a toggle and neither an explicit adaptive body nor a supported fallback effort applies. |
|
|
509
|
-
|
|
510
|
-
The existing provider declaration prefix is
|
|
511
|
-
`PLURNK_PROVIDERS_PROVIDER_<NAME>_`; `PLURNK_PROVIDERS_<suffix>` overrides
|
|
512
|
-
it for the route and accepts the ordinary alias suffix. These are data, not
|
|
513
|
-
executable transformations. Static bodies cannot replace transport, sampling,
|
|
514
|
-
or managed numeric controls; pointers cannot overlap. Configuration fails at
|
|
515
|
-
construction before inference when a requested policy or numeric control is
|
|
516
|
-
unrepresentable. A tighter per-call envelope reprojects the same declaration.
|
|
517
|
-
Discovery and generation share the resolved policy set. A cataloged
|
|
518
|
-
non-reasoning model receives no reasoning fields.
|
|
519
|
-
Native SDKs retain their own output field; `OUTPUT_PATH` is incompatible with an
|
|
520
|
-
options namespace. Request-local options are projected after the effective
|
|
521
|
-
envelope is known, never frozen into SDK model construction. Streaming and
|
|
522
|
-
non-streaming calls use the same projection.
|
|
523
|
-
|
|
524
|
-
A native SDK's portable reasoning setting has no `max`, so a native route without
|
|
525
|
-
a namespace cannot send an effort the catalog documents beyond it; construction
|
|
526
|
-
refuses it and names the declaration that would. Native SDKs built on
|
|
527
|
-
openai-compatible (DeepInfra, Together) write `reasoning_effort` from their own
|
|
528
|
-
`reasoningEffort` option after spreading the others, overwriting a raw
|
|
529
|
-
`reasoning_effort`; their shipped declarations therefore point at
|
|
530
|
-
`/reasoningEffort`, with `{"reasoningEffort":"none"}` as the off body.
|
|
531
|
-
|
|
532
|
-
## §4 Operator configuration
|
|
533
|
-
|
|
534
|
-
§provider-configuration Every operational value is an environment knob
|
|
535
|
-
documented in `.env.defaults`.
|
|
536
|
-
There are no hidden tuning constants. Every `PLURNK_PROVIDERS_*` knob may be
|
|
537
|
-
scoped to an alias by appending `_<alias>`; the scoped value wins, and the alias
|
|
538
|
-
segment folds case. The scoping takes a closed list of knob names: a consumer
|
|
539
|
-
scopes its own knobs by passing its list, and neither list reaches the other's
|
|
540
|
-
knobs.
|
|
541
|
-
Each projection reads one fresh environment snapshot without modifying its
|
|
542
|
-
source; later projections observe later configuration, never a cached view.
|
|
543
|
-
|
|
544
|
-
The public Node provider registry applies this package's committed
|
|
545
|
-
`.env.defaults` as a set-if-unset operational floor. Consumers pass their
|
|
546
|
-
operator environment, not a manually composed copy of the provider floor.
|
|
547
|
-
Explicit operator values always win and invalid explicit values fail at the
|
|
548
|
-
knob's owning contract.
|
|
549
|
-
|
|
550
|
-
The universal groups are:
|
|
551
|
-
|
|
552
|
-
- reasoning activation and optional explicit budget;
|
|
553
|
-
- explicit reasoning response-content style;
|
|
554
|
-
- decode tuning;
|
|
555
|
-
- whole-call generation, discovery/tokenizer HTTP, retry, and probe budgets;
|
|
556
|
-
- local GBNF and llama-server capability pins;
|
|
557
|
-
- context-window and generation-envelope overrides;
|
|
558
|
-
- provider-documented cache affinity and explicit cache-write policy;
|
|
559
|
-
- opt-in logprob and raw-body capture.
|
|
560
|
-
|
|
561
|
-
Operator secrets and machine-specific values never belong in committed
|
|
562
|
-
defaults.
|
|
563
|
-
|
|
564
|
-
## §5 Resolution
|
|
565
|
-
|
|
566
|
-
<!-- map -->
|
|
567
|
-
| Tag | Promise |
|
|
568
|
-
| --- | --- |
|
|
569
|
-
| {§provider-resolution} | `PLURNK_MODEL_<alias>=<provider>/<model-id>` declares an alias. |
|
|
570
|
-
| {§model-catalog-readiness} | Catalog readiness and construction share one local configuration predicate. |
|
|
571
|
-
| {§provider-input-modalities} | Native input parts |
|
|
572
|
-
| {§model-fact-resolution} | Model fact precedence |
|
|
573
|
-
| {§provider-fact-authority} | Provider declarations configure facts, not credentials, and Models.dev is authoritative for cataloged… |
|
|
574
|
-
<!-- /map -->
|
|
575
|
-
|
|
576
|
-
§provider-resolution `PLURNK_MODEL_<alias>=<provider>/<model-id>` declares an
|
|
577
|
-
alias.
|
|
578
|
-
`PLURNK_MODEL=<selector>` selects either a declared alias or an exact
|
|
579
|
-
`<provider>/<model-id>` route. Model IDs may contain `/`; only the first slash
|
|
580
|
-
separates provider from model. Exact routes carry no fabricated alias and use
|
|
581
|
-
the global provider configuration. Declared aliases retain their provenance,
|
|
582
|
-
endpoint override, and alias-scoped tuning.
|
|
583
|
-
`PLURNK_BASEURL_<alias>` is a per-alias endpoint override.
|
|
584
|
-
|
|
585
|
-
§model-catalog-readiness **Catalog readiness and construction share one local
|
|
586
|
-
configuration predicate.** For each Models.dev provider, readiness evaluates
|
|
587
|
-
the same effective credential names, endpoint template coordinates, base-URL
|
|
588
|
-
precedence, and alternative Bedrock authentication sets used by construction.
|
|
589
|
-
It makes no request and validates no credential value. A ready result therefore
|
|
590
|
-
means only “configured enough to attempt”; missing causes contain environment
|
|
591
|
-
names without values. Construction rejects the same missing requirements at
|
|
592
|
-
the provider boundary instead of deferring a known configuration failure to a
|
|
593
|
-
model request.
|
|
594
|
-
|
|
595
|
-
### §provider-input-modalities Native input parts
|
|
596
|
-
|
|
597
|
-
A provider declares `inputModalities`, the set of native non-text inputs its model accepts, from the catalog's input modalities (Models.dev `modalities.input` minus `text`, kept to the vocabulary `image`, `pdf`, `audio`, `video`; empty when the model is unknown). Which parts actually ride a request is the service's decision per attachment ({§packet-attachment-parts} in the core specification).
|
|
598
|
-
|
|
599
|
-
| Fact | Rule |
|
|
600
|
-
| --- | --- |
|
|
601
|
-
| The part | a user `ChatMessage` may carry text beside the AI SDK's current `{ type: "file", data: bytes, mediaType }` part; `mediaType` distinguishes an image, PDF, audio, or future native input without a second internal part shape |
|
|
602
|
-
| The transport | the AI SDK transport forwards those parts as the model's native file inputs; it alone serializes the canonical model and messages before provider extension fields are merged |
|
|
603
|
-
| Other messages | system and assistant messages stay text, and prompt-token estimates count text only, the provider's reported usage owning each part's cost |
|
|
604
|
-
| Pools and the Mock | a pool declares a modality only when every backend does; the Mock declares them by option and records every request it receives |
|
|
605
|
-
| Codecs | an accepted modality does not imply every codec/container is supported: SDK or endpoint format rejections remain provider errors, never silent omission or a text-only substitute |
|
|
606
|
-
|
|
607
|
-
### §model-fact-resolution Model fact precedence
|
|
608
|
-
|
|
609
|
-
Provider and model facts resolve independently:
|
|
610
|
-
|
|
611
|
-
| Fact | Natural source | Operator source | Effective value |
|
|
612
|
-
| --- | --- | --- | --- |
|
|
613
|
-
| Context window | Catalog metadata or local endpoint probe. | `PLURNK_PROVIDERS_CONTEXT_WINDOW`. | Minimum when both exist; sole value otherwise. Cataloged cloud miss fails construction; compatible probe miss remains `null` with one warning. |
|
|
614
|
-
| Maximum input | Catalog `limit.input`; no generic live probe. | None. | Catalog value or `null`; never reconstructed from context and output. |
|
|
615
|
-
| Maximum output | Catalog `limit.output`; no generic live probe. | None. | Minimum of catalog value and effective context, or `null`. |
|
|
616
|
-
| Total output budget | None. | `PLURNK_PROVIDERS_OUTPUT_BUDGET`. | Curation reservation: percentage of effective context or absolute count, capped by known context/output limits; a call may only tighten it. The response grant flexes under {§provider-flexed-allowance}. |
|
|
617
|
-
| Output floor | None. | `PLURNK_PROVIDERS_OUTPUT_FLOOR`, shipped as `10%`. | The least response room any request keeps: percentage of the effective output ceiling (`min(contextWindow, maxOutputTokens)` when both are known, otherwise `contextWindow`) or absolute count, capped by the output budget; `window − floor` is the input wall ({§provider-output-floor}). |
|
|
618
|
-
| Effort | Catalog `reasoning_options` intersected with the installed adapter; explicit adapter declaration for uncataloged routes. | `PLURNK_PROVIDERS_EFFORT`, initially; durable worker selection thereafter. | A supported member of {§effort-wire}, projected under {§provider-effort}. The shipped selection is `adaptive`. |
|
|
619
|
-
| Reasoning budget | None. | Optional `PLURNK_PROVIDERS_REASONING_BUDGET`. | Percentage of effective context or absolute count; valid only as a strict subset of total output and effective unless reasoning is `off`. |
|
|
620
|
-
| Cost override | None. | Optional `PLURNK_PROVIDERS_COST`. | {§operator-cost-override} — comma-separated `key=value` per-1M-token USD rates over `input, output, reasoning, cacheRead, cacheWrite`; merges over the Models.dev catalog block (the catalog is the starting point), alias-scoped like every knob. Without catalog rates the override must declare `input` and `output`. The cost estimate's `source` names the override; a provider-reported response cost still outranks any estimate. |
|
|
621
|
-
| Reasoning capability | Catalog `reasoning` and route-specific `reasoning_options`. | Adapter wire style only where the catalog cannot name the native field. | Catalog controls determine admissible policy; the adapter determines its wire projection. |
|
|
622
|
-
| Estimated USD rates | Models.dev input, output, optional reasoning, and optional cache rates. | None. | Missing differently-priced usage or rates produces unknown; exact all-zero rates produces estimated USD zero. |
|
|
623
|
-
|
|
624
|
-
Models.dev cache-read and cache-write rates default to the input rate, and the
|
|
625
|
-
reasoning rate defaults to the output rate, when omitted. A differently priced
|
|
626
|
-
category requires its corresponding usage detail or the request estimate is
|
|
627
|
-
unknown.
|
|
628
|
-
|
|
629
|
-
The provider segment of a model route IS the Models.dev id (#459) —
|
|
630
|
-
configurations align with the catalog end to end (`fireworks-ai/…`,
|
|
631
|
-
`cloudflare-workers-ai/@cf/…`, `togetherai/…`), and the
|
|
632
|
-
`PLURNK_PROVIDERS_PROVIDER_<PREFIX>_*` prefix is that id uppercased with
|
|
633
|
-
non-alphanumerics as underscores (`FIREWORKS_AI`). Custom declared providers
|
|
634
|
-
keep their operator-chosen names; the built-in local `ollama` rail keeps its
|
|
635
|
-
name (the catalog's `ollama-cloud` is a different provider).
|
|
636
|
-
|
|
637
|
-
`instantiateProvider` resolves in this order:
|
|
638
|
-
|
|
639
|
-
1. A Models.dev provider and model, using its declared AI SDK package.
|
|
640
|
-
2. An operator provider declaration:
|
|
641
|
-
`PLURNK_PROVIDERS_PROVIDER_<NAME>_{NPM,BASE_URL,API_KEY_ENV}`.
|
|
642
|
-
3. The local `openai` or `ollama` adapter.
|
|
643
|
-
4. A discovered AI SDK provider extension.
|
|
644
|
-
5. A precise unknown-provider error.
|
|
645
|
-
|
|
646
|
-
Earlier sources are authoritative. Installed extensions cannot shadow a cataloged
|
|
647
|
-
or explicitly declared name. This remains true when a named model is absent
|
|
648
|
-
from the Models.dev snapshot: construction requires an explicit
|
|
649
|
-
`PLURNK_PROVIDERS_CONTEXT_WINDOW` and never falls through to a same-name
|
|
650
|
-
extension.
|
|
651
|
-
|
|
652
|
-
Model IDs resolve exactly first. A unique catalog suffix is accepted to avoid
|
|
653
|
-
forcing a vendor-owned resource prefix into PLURNK aliases. Ambiguous suffixes
|
|
654
|
-
fail to resolve.
|
|
655
|
-
|
|
656
|
-
The catalog package identifies the protocol family, not a mandatory client
|
|
657
|
-
implementation. OpenRouter charges and DeepInfra estimates use cost normalizers
|
|
658
|
-
over documented response fields, retaining their distinct monetary character
|
|
659
|
-
under {§provider-monetary-evidence}.
|
|
660
|
-
|
|
661
|
-
§provider-fact-authority Provider declarations configure facts, not
|
|
662
|
-
credentials, and Models.dev is authoritative for cataloged providers: package
|
|
663
|
-
defaults never redefine a cataloged provider's NPM package, endpoint, or
|
|
664
|
-
credential names, and one declaration's `API_KEY_ENV` holds exactly one
|
|
665
|
-
environment name — an ordered fallback list would paper over an
|
|
666
|
-
operator/catalog naming mismatch instead of reconciling it at its owning
|
|
667
|
-
boundary. A comma-separated value is rejected at construction.
|
|
668
|
-
|
|
669
|
-
```dotenv
|
|
670
|
-
PLURNK_PROVIDERS_PROVIDER_ACME_NPM=@ai-sdk/openai-compatible
|
|
671
|
-
PLURNK_PROVIDERS_PROVIDER_ACME_BASE_URL=https://api.acme.example/v1
|
|
672
|
-
PLURNK_PROVIDERS_PROVIDER_ACME_API_KEY_ENV=ACME_API_KEY
|
|
673
|
-
```
|
|
674
|
-
|
|
675
|
-
The named secret remains in the operator environment. `${ENV_NAME}` inside a
|
|
676
|
-
catalog or declared endpoint is expanded at construction and fails clearly
|
|
677
|
-
when absent.
|
|
678
|
-
|
|
679
|
-
## §6 Provider extensions
|
|
680
|
-
|
|
681
|
-
Provider extensions are the escape hatch for a protocol binding not represented by
|
|
682
|
-
Models.dev, installed SDK providers, or a declaration. Most extensibility
|
|
683
|
-
belongs in MCP, schemes, executors, or mimetypes instead.
|
|
684
|
-
|
|
685
|
-
A provider extension:
|
|
686
|
-
|
|
687
|
-
1. declares the exact string `plurnk: { kind: "provider", name }` in `package.json` ({§extension-kind});
|
|
688
|
-
2. may declare always-on package-level `plurnk.attribution` and/or implement the
|
|
689
|
-
synchronous runtime `attributions(context)` hook under {§extension-attribution};
|
|
690
|
-
3. may use any npm scope;
|
|
691
|
-
4. default-exports an AI SDK provider with `languageModel(modelId)`;
|
|
692
|
-
5. peers on compatible `ai` and `@plurnk/plurnk-providers` majors.
|
|
693
|
-
|
|
694
|
-
§provider-grammar-transport An extension whose backend accepts a llama.cpp-style
|
|
695
|
-
GBNF grammar may declare `plurnk.grammarStyle: "llamacpp"` beside its kind and
|
|
696
|
-
name; the discovery records it and the adapted Provider carries the capability,
|
|
697
|
-
so an operator's grammar file ({§operator-grammar}) rides the wire exactly as
|
|
698
|
-
on a probed llama-server. Absence or `"none"` keeps the grammar off the wire;
|
|
699
|
-
any other value fails discovery loudly. The declaration is the extension author's
|
|
700
|
-
fact about their backend; the transport evidence on each response
|
|
701
|
-
({§provider-grammar-evidence}) is where a wrong declaration shows, never by
|
|
702
|
-
degrading admission.
|
|
703
|
-
|
|
704
|
-
PLURNK adapts the returned language model. The extension does not implement the
|
|
705
|
-
PLURNK `Provider`, read PLURNK tuning knobs, or reproduce transport policy.
|
|
706
|
-
|
|
707
|
-
Discovery is scope-agnostic and memoized per process. Duplicate names fail hard.
|
|
708
|
-
The common extension trust gate applies before import ({§extension-trust-boundary}). An extension absent from
|
|
709
|
-
Models.dev requires an explicit context-window pin because PLURNK will not guess
|
|
710
|
-
model physics. `Discovery.packageAttributions` carries the canonical package map.
|
|
711
|
-
|
|
712
|
-
## §7 Local capabilities
|
|
713
|
-
|
|
714
|
-
<!-- map -->
|
|
715
|
-
| Tag | Promise |
|
|
716
|
-
| --- | --- |
|
|
717
|
-
| {§provider-local-capabilities} | The `openai` local adapter probes `/v1/models`. |
|
|
718
|
-
| {§llama-template-effort} | A detected llama-server rail admits `off` and `adaptive`, and a fixed effort only where the operator declares… |
|
|
719
|
-
| {§llama-reasoning-request} | The allowance is cumulative across the complete response. |
|
|
720
|
-
<!-- /map -->
|
|
721
|
-
|
|
722
|
-
§provider-local-capabilities The `openai` local adapter probes `/v1/models`. A
|
|
723
|
-
llama-server fingerprint may
|
|
724
|
-
also expose:
|
|
725
|
-
|
|
726
|
-
- the actual served model and per-slot context window;
|
|
727
|
-
- request-scoped reasoning parsing and a cumulative response-wide allowance;
|
|
728
|
-
- GBNF constrained sampling;
|
|
729
|
-
- slot count and worker-sticky slot affinity;
|
|
730
|
-
- EOS marker removal;
|
|
731
|
-
- exact complete-request counting through `/v1/chat/completions/input_tokens`;
|
|
732
|
-
- exact content token IDs through `/tokenize`;
|
|
733
|
-
- the requirement that the adapter apply a finite output budget.
|
|
734
|
-
|
|
735
|
-
`PLURNK_PROVIDERS_LLAMA_SERVER` may force or disable detection. Probe attempts
|
|
736
|
-
and delay are knobs. A failed probe does not silently assert capabilities.
|
|
737
|
-
|
|
738
|
-
Ollama probes `/api/show` for its model context and uses its documented
|
|
739
|
-
OpenAI-compatible generation endpoint through the SDK adapter.
|
|
740
|
-
|
|
741
|
-
### llama-server reasoning
|
|
742
|
-
|
|
743
|
-
For a detected llama-server, PLURNK sends the complete reasoning contract on
|
|
744
|
-
every request:
|
|
745
|
-
|
|
746
|
-
| Posture | Template activation | Template `reasoning_effort` | `thinking_budget_tokens` |
|
|
747
|
-
|---|---:|---|---:|
|
|
748
|
-
| `off` | false | none | `0` |
|
|
749
|
-
| `adaptive` | true | none | configured reasoning subset, otherwise omitted |
|
|
750
|
-
| A declared fixed effort | true | the effort's name | configured reasoning subset, otherwise omitted |
|
|
751
|
-
|
|
752
|
-
§llama-template-effort A detected llama-server rail admits `off` and `adaptive`, and a fixed
|
|
753
|
-
effort only where the operator declares it (`REASONING_EFFORTS`,
|
|
754
|
-
{§provider-wire-declaration}). The chat template decides what each level means and which
|
|
755
|
-
levels it accepts; PLURNK infers neither from the template nor from the model's name, and a
|
|
756
|
-
template that refuses a declared level answers on the first request.
|
|
757
|
-
|
|
758
|
-
The allowance is contained by the request's total output budget. Template calls
|
|
759
|
-
normally use `reasoning_format: "auto"` for a separate
|
|
760
|
-
readable channel. A GBNF-bearing call uses `"none"` so the exact constrained
|
|
761
|
-
sentence survives response projection; the adapter separates its leading
|
|
762
|
-
reasoning enclosure only after preserving grammar evidence. Process-wide
|
|
763
|
-
llama-server flags are fallback server configuration, not part of the PLURNK
|
|
764
|
-
contract and need not be synchronized with an alias.
|
|
765
|
-
|
|
766
|
-
§llama-reasoning-request The allowance is cumulative across the complete response. Opening a second or
|
|
767
|
-
later reasoning block does not replenish it. Template parsing, the reasoning
|
|
768
|
-
sampler, normalized usage, and the returned reasoning channel MUST agree on that
|
|
769
|
-
response boundary.
|
|
770
|
-
|
|
771
|
-
## Probes
|
|
772
|
-
|
|
773
|
-
§provider-ping-probe **`npm run probe:providers` reaches every keyed provider once, by one cheapest declared route.** Planning selects, per provider that has a credential, its cheapest declared alias; a provider-named alias resolves that provider's own unpriced route without guessing another model, a catalog key with no declared route stays explicitly red, and several unpriced aliases of one provider are one ambiguous provider, never guessed routes. Execution invokes every selected route exactly once and preserves red outcomes. Each attempted provider leaves one safe JSON record: response evidence keeps its structure and no provider scalar values, retained diagnostics redact credentials, account identity and URL authority, and secret discovery never mistakes a token-limit control for a credential.
|
|
774
|
-
|
|
775
|
-
§token-envelope-probe **`npm run probe:tokens` measures output-token envelopes for the aliases named explicitly, and no other.** Its output-token modes distinguish the configured policy from the model's physics.
|
|
776
|
-
|
|
777
|
-
## §8 Request authority
|
|
778
|
-
|
|
779
|
-
<!-- map -->
|
|
780
|
-
| Tag | Promise |
|
|
781
|
-
| --- | --- |
|
|
782
|
-
| {§provider-request-authority} | The caller's `sampling` bag expresses sampling intent. |
|
|
783
|
-
| {§provider-request-controls} | A configured service tier or logprobs count reaches a native route only through the per-call option its SDK… |
|
|
784
|
-
| {§openrouter-app-attribution} | A route on the OpenRouter SDK identifies the calling application only as its provider declares. |
|
|
785
|
-
<!-- /map -->
|
|
786
|
-
|
|
787
|
-
§provider-request-authority The caller's `sampling` bag expresses sampling
|
|
788
|
-
intent. It cannot override:
|
|
789
|
-
|
|
790
|
-
- model or messages;
|
|
791
|
-
- stream mode;
|
|
792
|
-
- grammar or response format;
|
|
793
|
-
- backend slot;
|
|
794
|
-
- data-capture settings;
|
|
795
|
-
- tool, modality, or multi-choice behavior;
|
|
796
|
-
- the consumer-owned output envelope;
|
|
797
|
-
- cache affinity identity or cache-write policy.
|
|
798
|
-
|
|
799
|
-
Generic AI SDK calls accept only settings represented by the SDK's portable
|
|
800
|
-
surface. Compatible endpoints may carry additional sampling keys after reserved
|
|
801
|
-
keys are removed.
|
|
802
|
-
|
|
803
|
-
§provider-request-controls **A configured service tier or logprobs count reaches a
|
|
804
|
-
native route only through the per-call option its SDK documents.** On a compatible
|
|
805
|
-
route, `PLURNK_PROVIDERS_SERVICE_TIER` and `PLURNK_PROVIDERS_TOP_LOGPROBS` are the
|
|
806
|
-
body fields `service_tier`, `logprobs` and `top_logprobs`. A native route sends them
|
|
807
|
-
in its SDK's `providerOptions` namespace, serialized by that SDK. A control the SDK
|
|
808
|
-
does not document is not sent, and one process warning,
|
|
809
|
-
`PLURNK_REQUEST_CONTROL_UNSUPPORTED`, names it, so a configured choice never
|
|
810
|
-
disappears in silence. The tier passes verbatim: the SDK owns its vocabulary, and a
|
|
811
|
-
value it drops surfaces as its call warning ({§provider-sdk-warning}). A logprobs
|
|
812
|
-
count of 0 asks for the chosen token only.
|
|
813
|
-
|
|
814
|
-
| Native SDK | Namespace | Service tier | Logprobs |
|
|
815
|
-
| --- | --- | --- | --- |
|
|
816
|
-
| OpenAI | `openai` | sent | sent |
|
|
817
|
-
| Groq | `groq` | sent | not sent |
|
|
818
|
-
| Cerebras | `cerebras` | sent | not sent |
|
|
819
|
-
| xAI | `xai` | sent | not sent |
|
|
820
|
-
| Google | `google` | sent | not sent |
|
|
821
|
-
| Anthropic | `anthropic` | sent | not sent |
|
|
822
|
-
| Amazon Bedrock | `bedrock` | sent | not sent |
|
|
823
|
-
| Mistral, Together, DeepInfra, OpenRouter | none | not sent | not sent |
|
|
824
|
-
|
|
825
|
-
§openrouter-app-attribution **A route on the OpenRouter SDK identifies the calling
|
|
826
|
-
application only as its provider declares.** `APP_URL`, an absolute HTTP(S) URL, and
|
|
827
|
-
the optional `APP_NAME` are provider declarations (`PLURNK_PROVIDERS_PROVIDER_<NAME>_`,
|
|
828
|
-
overridable per route or alias); the SDK sends them as `HTTP-Referer` and
|
|
829
|
-
`X-OpenRouter-Title`. The shipped floor declares them for `openrouter` only, so
|
|
830
|
-
another provider on the same SDK package sends none; an empty `APP_URL` suppresses
|
|
831
|
-
both.
|
|
832
|
-
|
|
833
|
-
## §9 Failures, retries, and cancellation
|
|
834
|
-
|
|
835
|
-
<!-- map -->
|
|
836
|
-
| Tag | Promise |
|
|
837
|
-
| --- | --- |
|
|
838
|
-
| {§provider-failure-normalization} | Provider failures normalize to `ProviderError`. |
|
|
839
|
-
| {§provider-retryable-truth} | `retryable` states what happens next. |
|
|
840
|
-
| {§provider-failure-cause} | The wrapper's cause is evidence, bounded. |
|
|
841
|
-
| {§provider-capacity-failure} | A proven exact preflight overflow and an upstream context rejection normalize to… |
|
|
842
|
-
| {§provider-flexed-allowance} | The grant is the window's remainder, between the floor and the model's limit. |
|
|
843
|
-
| {§provider-output-floor} | The floor is the wall's other side. |
|
|
844
|
-
| {§provider-sampling-passthrough} | Unconfigured sampling retains endpoint defaults. |
|
|
845
|
-
| {§provider-connectivity} | The provider adapter owns one attempt scheduler around the complete generation exchange; SDK-internal retries… |
|
|
846
|
-
| {§provider-cancellation-evidence} | Received usage/charge evidence and available response identity survive cancellation and deadline… |
|
|
847
|
-
| {§provider-native-tool-calls} | A response with native tool-call data, or a `tool_calls` finish, preserves its text, reasoning, structured… |
|
|
848
|
-
| {§provider-completed-admission} | A completed exchange is admitted as the provider delivered it. |
|
|
849
|
-
| {§provider-request-rejection} | An HTTP 4xx rejection not classified as authorization, quota, capacity, grammar, rate limit, or transient… |
|
|
850
|
-
| {§provider-interrupted-attempt} | Provider-declared interruption |
|
|
851
|
-
| {§provider-generation-failed} | Provider-declared generation error |
|
|
852
|
-
<!-- /map -->
|
|
853
|
-
|
|
854
|
-
§provider-failure-normalization Provider failures normalize to `ProviderError`.
|
|
855
|
-
Its public contract is an RFC
|
|
856
|
-
9457 Problem Details object with an exact status, stable type, occurrence
|
|
857
|
-
detail, and provider-kind extension; the original error remains its cause.
|
|
858
|
-
A caught failure is surfaced or deliberately preserved; it is never converted
|
|
859
|
-
into an empty model turn or reduced to a message plus a generic status.
|
|
860
|
-
Upstream diagnostic text is bounded by
|
|
861
|
-
`PLURNK_PROVIDERS_ERROR_DETAIL_LIMIT`; the committed `.env.defaults` owns its
|
|
862
|
-
normal value. Retry exhaustion is preserved as `attempts` and
|
|
863
|
-
`retryExhausted`; it does not change the Problem's `retryable`. A provider's
|
|
864
|
-
producer id is `provider:<name>`, the name folded to the `source` grammar of
|
|
865
|
-
`Notice.json` (lower-case words and hyphens; a leading digit is prefixed
|
|
866
|
-
`p-`); an empty name is refused.
|
|
867
|
-
|
|
868
|
-
§provider-retryable-truth **`retryable` states what happens next.** A Problem's
|
|
869
|
-
`retryable` is exactly membership of its kind in the exported
|
|
870
|
-
`RETRYABLE_PROVIDER_KINDS` — `rate_limit`, `network_failure`, `deadline_exceeded`,
|
|
871
|
-
`resource_interrupted` — and Core's provider recovery re-issues
|
|
872
|
-
exactly those kinds by reading the same set. It never copies the transport's own
|
|
873
|
-
retry policy: a 502 or 524 that the transport will not replay is still re-issued by
|
|
874
|
-
the consumer, so it reports `retryable: true`. Every other kind reports `false`.
|
|
875
|
-
|
|
876
|
-
§provider-failure-cause **The wrapper's cause is evidence, bounded.** The SDK
|
|
877
|
-
reports every processing failure of a 2xx body with one message ("Failed to
|
|
878
|
-
process successful response") and keeps what happened as `cause`. A normalized
|
|
879
|
-
Problem carries that cause as a bounded classification — `cause.causeKind` is
|
|
880
|
-
`transport_terminated` (the body ended under a successful status),
|
|
881
|
-
`invalid_json`, `schema_invalid`, `invalid_response_data`, or `internal`, with
|
|
882
|
-
the cause's error name and one bounded line of its message — so forensics can
|
|
883
|
-
tell an interrupted stream from a malformed body without the kind changing:
|
|
884
|
-
classification of the Problem (`providerKind`, retryability, accounting) is
|
|
885
|
-
untouched by this evidence. The SDK's parse and validation messages embed the
|
|
886
|
-
payload, so those two report shape (character count, the validator's issue
|
|
887
|
-
line), never response text; no headers, credentials, stack frames, or body
|
|
888
|
-
dumps enter any Problem. Covered by `errors.test.ts` (#593).
|
|
889
|
-
|
|
890
|
-
§provider-capacity-failure A proven exact preflight overflow and an upstream
|
|
891
|
-
context rejection normalize to `ProviderError(kind="capacity_exceeded")` and
|
|
892
|
-
an RFC 9457 status 413. `capacityStage` is `preflight` or `upstream`; a
|
|
893
|
-
non-413 upstream status remains `providerStatus`, while physical request
|
|
894
|
-
accounting retains the status actually received. Preflight rejection occurs
|
|
895
|
-
before provider I/O and therefore opens no request identity and creates no
|
|
896
|
-
request-accounting row. Capacity failures are not connectivity failures and are
|
|
897
|
-
never retried by the provider scheduler; bounded packet recovery belongs to the
|
|
898
|
-
consumer.
|
|
899
|
-
|
|
900
|
-
§provider-flexed-allowance **The grant is the window's remainder, between the floor and the model's limit.** The configured output budget is the reservation curation packs the input against (`window − outputBudget`), and it is the only allowance the model is ever shown — the packet's disclosed `tokensResponseMax` stays the configured reservation so the response discipline it teaches never varies.
|
|
901
|
-
|
|
902
|
-
| Prompt measurement | The wire grants |
|
|
903
|
-
| --- | --- |
|
|
904
|
-
| An exact prompt count (llama-server's input-token endpoint) | `window − promptTokens − 256`, clamped to no less than the output floor and no more than the model's own output limit, so a response overflowing the disclosed reservation completes whenever the window's unclaimed room can hold it, and a prompt that ate into the reservation still leaves the floor |
|
|
905
|
-
| Every other measurement kind, and any pool | the reservation, clamped by the same remainder taken from the prompt's estimate so the wire never asks the window for more than it has; an estimate proves nothing about the true remainder, so it never rejects |
|
|
906
|
-
|
|
907
|
-
§provider-output-floor **The floor is the wall's other side.** `PLURNK_PROVIDERS_OUTPUT_FLOOR`, shipped as `10%`, is the least response room any request keeps: a percentage resolves against the effective output ceiling (`min(contextWindow, maxOutputTokens)` when both are known, otherwise `contextWindow`), and the resolved floor is capped by the output budget; `window − floor`, intersected with `maxInputTokens`, is the physical input wall exposed as `inputWall`. A request under the curation reservation is the ordinary case; a request between the reservation and the wall is admitted with its grant flexed down to the remainder; an exact prompt over the wall is rejected before I/O; an estimate over the wall defers to the upstream provider as capacity oracle with its grant at the floor. Core consults the wall, never the reservation, to end a loop ({§context-wall} in the core contract). A response exceeding even the grant is cut,
|
|
908
|
-
and the notice names the true per-call grant (`capacity.responseMax` on the
|
|
909
|
-
response) — the tolerance's honest edge, never worse than the fixed allowance
|
|
910
|
-
it forgives.
|
|
911
|
-
|
|
912
|
-
§provider-sampling-passthrough **Unconfigured sampling retains endpoint defaults.**
|
|
913
|
-
The provider panel owns tuning; Models.dev capabilities are not recommended
|
|
914
|
-
sampling values. Every knob accepts the ordinary alias override
|
|
915
|
-
({§provider-configuration}). Both cataloged and local providers apply the same
|
|
916
|
-
validation and precedence: configured values, then caller `sampling`, then
|
|
917
|
-
transport-owned fields ({§provider-request-authority}).
|
|
918
|
-
|
|
919
|
-
| `PLURNK_PROVIDERS_` suffix | Compatible wire / native SDK | Accepted configuration |
|
|
920
|
-
| --- | --- | --- |
|
|
921
|
-
| `TEMPERATURE` | `temperature` / `temperature` | Finite non-negative number; empty/unset omits. |
|
|
922
|
-
| `TOP_P` | `top_p` / `topP` | Finite number in `[0,1]`; empty/unset omits. |
|
|
923
|
-
| `TOP_K` | `top_k` / `topK` | Non-negative safe integer; empty/unset omits. Zero semantics are endpoint-owned. |
|
|
924
|
-
| `PRESENCE_PENALTY` | `presence_penalty` / `presencePenalty` | Finite number in `[-2,2]`; empty/unset omits. |
|
|
925
|
-
| `FREQUENCY_PENALTY` | `frequency_penalty` / `frequencyPenalty` | Required finite number in `[-2,2]`; the panel's zero selects no configured override. Negative values survive unchanged. |
|
|
926
|
-
| `SEED` | `seed` / `seed` | Safe integer; empty/unset omits. No guarantee of reproducibility. |
|
|
927
|
-
|
|
928
|
-
Configured zero is preserved except for the frequency knob's explicit
|
|
929
|
-
no-override sentinel; a caller-supplied zero always overrides a configured value.
|
|
930
|
-
Invalid configuration fails by knob name before inference. The SDK/endpoint
|
|
931
|
-
owns narrower model restrictions and unsupported-setting diagnostics; Plurnk
|
|
932
|
-
does not clamp values or invent model-specific sampling profiles.
|
|
933
|
-
`REPEAT_PENALTY` and DRY remain explicit llama-server extensions, omitted when
|
|
934
|
-
unconfigured. Their grammar protection remains at {§provider-grammar-transport}.
|
|
935
|
-
|
|
936
|
-
§provider-connectivity The provider adapter owns one attempt scheduler around
|
|
937
|
-
the complete generation exchange; SDK-internal retries are disabled.
|
|
938
|
-
`PLURNK_PROVIDERS_RETRY_ATTEMPTS=N` permits at most `N + 1` physical requests.
|
|
939
|
-
Generation has one configurable deadline; `0` disables it. Caller cancellation
|
|
940
|
-
and the caller's task allowance remain authoritative. Initial silence, a pause
|
|
941
|
-
between chunks, and buffered output do not acquire separate deadlines.
|
|
942
|
-
|
|
943
|
-
| Layer | Operator knob | Boundary | Expiry |
|
|
944
|
-
| --- | --- | --- | --- |
|
|
945
|
-
| Operation | `PLURNK_PROVIDERS_OPERATION_TIMEOUT` | Complete logical call, including admission queueing, every attempt and retry delay. | Final `deadline_exceeded` Problem at 504 with `timeoutPhase=operation`; not retried inside this operation. Consumer recovery is separate. Enforced as a race, not only the advisory signal, so a wedged transport that never observes the abort cannot hang the loop past the deadline (#505); a well-behaved transport unwinds within a short grace and settles its own attempt evidence first. |
|
|
946
|
-
| Discovery/tokenizer HTTP | `PLURNK_PROVIDERS_FETCH_TIMEOUT` | Non-generation endpoint discovery and tokenizer requests only. | Ordinary HTTP failure at the owning discovery or measurement boundary. Never interrupts generation. |
|
|
947
|
-
|
|
948
|
-
Settled calls remove their deadline timers and cancellation subscriptions.
|
|
949
|
-
|
|
950
|
-
Caller cancellation spans the operation and preserves the caller's reason.
|
|
951
|
-
|
|
952
|
-
§provider-cancellation-evidence Received usage/charge evidence and available
|
|
953
|
-
response identity survive cancellation and deadline normalization through
|
|
954
|
-
physical-request settlement, independently of optional raw-body capture.
|
|
955
|
-
This includes SDK abort events and non-Error cancellation reasons. Concurrent
|
|
956
|
-
requests keep independent evidence even when cancelled by the same scope.
|
|
957
|
-
Absent evidence remains absent; stopping a request neither invents usage nor
|
|
958
|
-
retries generation to obtain it. Partial output is never a completed response.
|
|
959
|
-
|
|
960
|
-
A 2xx exchange whose body cannot be processed (a provider invalid-response)
|
|
961
|
-
classifies as the non-retryable 502 on the first failure unless an explicit
|
|
962
|
-
`x-should-retry` directive says otherwise (#479). When a
|
|
963
|
-
directive-driven retry sequence exhausts, `attempts` and `retryExhausted`
|
|
964
|
-
are added and the classification is final. Every admitted physical attempt opens
|
|
965
|
-
and settles exactly one ordered {§provider-request-accounting} record, including
|
|
966
|
-
response-less network failures and timed-out attempts. A cancelled or expired
|
|
967
|
-
admission wait opens no physical request.
|
|
968
|
-
|
|
969
|
-
Each streamed physical request assembles its own response. Failed partial answer
|
|
970
|
-
bytes never enter a later request's completed `ProviderResponse`; recovery is a
|
|
971
|
-
complete re-emission, not continuation or salvage. When a retry succeeds after
|
|
972
|
-
a failed request emitted nonempty text or reasoning, the accepted response
|
|
973
|
-
carries one `provider_retry` warning identifying that fact. A retry before any
|
|
974
|
-
semantic model output remains silent while its physical failure stays cardinally
|
|
975
|
-
accounted. Transient reasoning follows {§provider-reasoning-observer}.
|
|
976
|
-
|
|
977
|
-
Only a provider-directed wait earns a transport retry (#479): HTTP 429, any
|
|
978
|
-
response carrying `Retry-After` (RFC 9110 defines it on 503 exactly as a
|
|
979
|
-
directed wait) — header-level facts the consumer's recovery layer never sees —
|
|
980
|
-
and an explicit `X-Should-Retry: true`. That header stays authoritative both
|
|
981
|
-
ways, and endpoint control responses 520–527 stay final. HTTP 408, 409, and
|
|
982
|
-
ordinary 5xx surface on the first failure as consumer-recoverable kinds; one
|
|
983
|
-
retry authority — the consumer's own provider-recovery machinery — owns
|
|
984
|
-
re-issue, backoff, and park above the transport.
|
|
985
|
-
|
|
986
|
-
§provider-native-tool-calls A response with native tool-call data, or a
|
|
987
|
-
`tool_calls` finish, preserves its text, reasoning, structured wire evidence,
|
|
988
|
-
finish reason and accounting and emits one provider warning. It is not a
|
|
989
|
-
transport failure and does not trigger provider recovery. The adapter executes
|
|
990
|
-
no call and invents no textual operation; the consumer owns admission and
|
|
991
|
-
no-operation handling. The normalized assistant exposes `nativeToolCalls: true`
|
|
992
|
-
when either wire calls or a `tool_calls` finish is present, independently of the
|
|
993
|
-
preserved finish reason; consumers need not mine wire evidence.
|
|
994
|
-
|
|
995
|
-
§provider-completed-admission **A completed exchange is admitted as the provider delivered it.**
|
|
996
|
-
Billed output tokens are accounting, never evidence of missing text: a provider may bill hidden or
|
|
997
|
-
summarized reasoning, bill reasoning inside its text count, or tokenize differently from what it
|
|
998
|
-
streams, so no comparison of token counts with streamed characters voids a response. What the
|
|
999
|
-
provider reports as its finish ({§provider-finish-reason}) decides the attempt's disposition; what
|
|
1000
|
-
the response says is the consumer's to judge ({§emission-admission}).
|
|
1001
|
-
|
|
1002
|
-
§provider-request-rejection An HTTP 4xx rejection not classified as authorization,
|
|
1003
|
-
quota, capacity, grammar, rate limit, or transient transport failure is
|
|
1004
|
-
`request_rejected`: preserve the upstream status and detail, with `retryable: false`.
|
|
1005
|
-
It is not an `invalid_response`; that kind describes a malformed successful
|
|
1006
|
-
response. Consumers terminate a rejected request with its exact Problem rather
|
|
1007
|
-
than retrying unchanged input or assigning response-contract strikes.
|
|
1008
|
-
|
|
1009
|
-
### §provider-interrupted-attempt Provider-declared interruption
|
|
1010
|
-
|
|
1011
|
-
A successful transport response can still declare that inference did not
|
|
1012
|
-
complete. That response is evidence for one failed provider attempt, never a
|
|
1013
|
-
completed exchange.
|
|
1014
|
-
|
|
1015
|
-
| Concern | Contract |
|
|
1016
|
-
| -------------------------- | ---------------------------------------------------------------------------------------------------------------------------- |
|
|
1017
|
-
| Normalized finish reason | Exact `insufficient_system_resource` becomes `resource_interrupted`; the raw value remains in `assistantRaw`. |
|
|
1018
|
-
| `generate` outcome | Throw `ProviderError(kind="resource_interrupted")` at local status 503 with the normalized attempt on `error.attempt` and the same accounting on `error.accounting`. |
|
|
1019
|
-
| Partial response | Preserve content, reasoning, usage, model, metadata, optional raw body, and other evidence without admitting it as success. |
|
|
1020
|
-
| Automatic replay | None inside the provider; AI SDK retry scheduling has already completed. The Problem has `retryable: true`: the consumer re-issues it ({§provider-retryable-truth}). |
|
|
1021
|
-
| Capacity-pool overflow | None under the existing routing policy; when other overflow-eligible failures do reach a sibling, the pool concatenates their request accounting. |
|
|
1022
|
-
| Consumer admission | Persist the evidence as an unaccepted attempt; never parse it into executable work, even when its frame looks complete. |
|
|
1023
|
-
|
|
1024
|
-
### §provider-generation-failed Provider-declared generation error
|
|
1025
|
-
|
|
1026
|
-
A finish the SDK classifies as `error` (Google's `MALFORMED_FUNCTION_CALL`, for one) is a failed
|
|
1027
|
-
attempt, `ProviderError(kind="generation_failed")` at local status 502 with the normalized attempt
|
|
1028
|
-
on `error.attempt` and its accounting on `error.accounting`: content, reasoning, usage, model, the
|
|
1029
|
-
raw finish reason and any raw body stay evidence, never a completed exchange. It is not a transport
|
|
1030
|
-
outage: `retryable: false`, so the consumer's provider recovery does not re-issue it; Core settles
|
|
1031
|
-
it on the provider response contract — the turn is complete, the strike rail rules, and the next
|
|
1032
|
-
turn asks again ({§provider-retryable-truth}).
|
|
1033
|
-
|
|
1034
|
-
## §10 Grammar
|
|
1035
|
-
|
|
1036
|
-
GBNF is a local llama-server capability, not a generic provider expectation.
|
|
1037
|
-
The consumer chooses whether to supply a grammar and reads it from the
|
|
1038
|
-
operator's own file; the provider never creates, ships, rewrites, validates, or
|
|
1039
|
-
grades one (#588). Transport is `{§provider-grammar-transport}`.
|
|
1040
|
-
|
|
1041
|
-
§provider-grammar-evidence When a grammar-capable adapter receives a grammar,
|
|
1042
|
-
`ProviderResponse` carries `grammarEvidence: { input, contentStart, transported }`.
|
|
1043
|
-
`input` is the exact pre-projection sentence represented by the response,
|
|
1044
|
-
`contentStart` is its Unicode-code-point offset to `assistant.content`, and
|
|
1045
|
-
`transported` says whether the grammar was actually sent. Active llama-server
|
|
1046
|
-
template reasoning requests the unprojected sentence, records it, and then
|
|
1047
|
-
separates its leading enclosure; an empty body therefore remains observable. An
|
|
1048
|
-
endpoint that projects despite that request supplies no independent evidence.
|
|
1049
|
-
For an unsplit response, `input` is `content` and `contentStart` is zero. This
|
|
1050
|
-
is transport evidence so a digest can prove a run was constrained; nothing in
|
|
1051
|
-
the service issues a verdict on it.
|
|
1052
|
-
|
|
1053
|
-
## §11 Evidence and metadata
|
|
1054
|
-
|
|
1055
|
-
<!-- map -->
|
|
1056
|
-
| Tag | Promise |
|
|
1057
|
-
| --- | --- |
|
|
1058
|
-
| {§provider-evidence} | `assistantRaw` is an opaque normalized transport record. |
|
|
1059
|
-
| {§provider-wire-emission} | The emission is retained as the wire carried it, on every response. |
|
|
1060
|
-
| {§provider-usage-refusal} | The provider's bookkeeping is not the exchange. |
|
|
1061
|
-
| {§provider-open-reasoning} | Plurnk requires endpoints with open, verbatim reasoning. |
|
|
1062
|
-
<!-- /map -->
|
|
1063
|
-
|
|
1064
|
-
§provider-evidence `assistantRaw` is an opaque normalized transport record.
|
|
1065
|
-
Provider top-level
|
|
1066
|
-
metadata is forwarded as an open bag without reinterpreting currencies or
|
|
1067
|
-
vendor fields.
|
|
1068
|
-
|
|
1069
|
-
Logprobs and raw capture are opt-in, alias-scoped diagnostic features.
|
|
1070
|
-
The former requests per-token logprobs and preserves returned alternatives.
|
|
1071
|
-
The latter retains SDK response bodies/chunks and the dispatched request body
|
|
1072
|
-
under {§provider-dispatched-request}, without requesting extra model output.
|
|
1073
|
-
Neither changes unconditional accounting or failed-response evidence.
|
|
1074
|
-
|
|
1075
|
-
§provider-wire-emission **The emission is retained as the wire carried it, on every response.** Beside the normalized `content` and `reasoning`, the transport record keeps `wire`. Nothing is opt-in and nothing is reinterpreted: a blank emission reads as thirteen chunks that carried nothing, or as a tool-call section the model emitted into a channel the protocol does not read, instead of being guessed at from token counts. This required evidence is independent of the optional complete `rawBody` dataset capture. Covered by `aiSdkTransport.test.ts` and core's `Digest.wire-evidence.test.ts`.
|
|
1076
|
-
|
|
1077
|
-
| `wire` holds | What |
|
|
1078
|
-
| --- | --- |
|
|
1079
|
-
| Chunks | the number of raw chunks, and the chunks that carried no field at all |
|
|
1080
|
-
| Fields | every delta or message field the stream carried with the number of chunks carrying it |
|
|
1081
|
-
| Tool calls | merged by index with their arguments as streamed |
|
|
1082
|
-
| Finish reasons | those seen |
|
|
1083
|
-
| Text | the verbatim text of every string field the record does not already hold |
|
|
1084
|
-
| `wire.unmappedChunks` | frames outside the recognized OpenAI-compatible envelope, verbatim; the recorder never discards an unrecognized/native payload or invents another provider parser |
|
|
1085
|
-
|
|
1086
|
-
§provider-usage-refusal **The provider's bookkeeping is not the exchange.** Usage
|
|
1087
|
-
normalization is exact; a refusal never fails or retries the response. The same
|
|
1088
|
-
rules apply to usage inside failure evidence:
|
|
1089
|
-
|
|
1090
|
-
| Contradiction | Normalized evidence |
|
|
1091
|
-
| --- | --- |
|
|
1092
|
-
| Invalid aggregate or total inconsistent with its parts | No `usage`. |
|
|
1093
|
-
| Invalid optional counter | Omit that counter; retain independently valid quantities. |
|
|
1094
|
-
| Cache or reasoning breakdown inconsistent with its aggregate | Omit that breakdown; retain valid aggregates and the other breakdown. |
|
|
1095
|
-
|
|
1096
|
-
Every refusal retains `usageRefusal`: the reason and the original wire counters.
|
|
1097
|
-
No value is clamped, zeroed, or estimated. Cost uses authoritative charges when
|
|
1098
|
-
available; catalog estimation requires all quantities its rates need, otherwise
|
|
1099
|
-
cost is `unknown`. Covered by `usage.test.ts`, `aiSdkTransport.test.ts`, and
|
|
1100
|
-
`AiSdkProvider.test.ts`.
|
|
1101
|
-
|
|
1102
|
-
§provider-open-reasoning **Plurnk requires endpoints with open, verbatim reasoning.**
|
|
1103
|
-
The adapter retains the reasoning text its transport exposes; fidelity is not
|
|
1104
|
-
inferred from a model name or catalog flag. Hidden or summarized reasoning is not
|
|
1105
|
-
a supported substitute, and encrypted or redacted payloads are never reconstructed
|
|
1106
|
-
into text. A single response without reasoning does not establish endpoint
|
|
1107
|
-
incapability: its `reasoning://` reads empty, while billed reasoning tokens remain
|
|
1108
|
-
visible in accounting. Exposed operations follow {§reasoning-operations}; no
|
|
1109
|
-
compatibility teaching or alternate admission mode is selected for a route.
|
|
1110
|
-
Unrecognized detail shapes are omitted at this normalization boundary. Core
|
|
1111
|
-
may preserve normalized items as forensic evidence, but a client protocol must
|
|
1112
|
-
correlate them to an entity it actually created rather than reusing `id`.
|
|
1113
|
-
|
|
1114
|
-
## §12 Generation envelopes
|
|
1115
|
-
|
|
1116
|
-
§provider-generation-envelope Every request has at most one total output
|
|
1117
|
-
budget. It includes visible output and hidden reasoning. An optional reasoning
|
|
1118
|
-
budget is a strict subset of that total, never an additive reserve. The
|
|
1119
|
-
configured total is a percentage of effective context or an absolute count;
|
|
1120
|
-
percentages resolve to the nearest whole token with a one-token minimum. It is
|
|
1121
|
-
capped by known context and model-output limits; `generate.maxOutputTokens` may
|
|
1122
|
-
only tighten this reservation for one call. The effective reasoning subset tightens
|
|
1123
|
-
with that total and remains strictly smaller. Exact prompt measurements may expand
|
|
1124
|
-
the response grant beyond the reservation under {§provider-flexed-allowance}.
|
|
1125
|
-
|
|
1126
|
-
The adapter owns native projection. A backend whose generic SDK maximum already
|
|
1127
|
-
includes reasoning receives the total directly. When a native SDK instead adds
|
|
1128
|
-
an explicit reasoning allowance to its generic visible-output maximum, the
|
|
1129
|
-
adapter sends `total - reasoning` through the generic field and the reasoning
|
|
1130
|
-
subset through the documented provider option. Core and other callers never
|
|
1131
|
-
reconstruct this arithmetic. When such a backend has only a manual allowance
|
|
1132
|
-
and no numeric subset is configured, the adapter derives that allowance from
|
|
1133
|
-
the durable policy inside the total using the native SDK's effort proportions
|
|
1134
|
-
and provider minimum; an envelope too small to represent the minimum fails
|
|
1135
|
-
before provider I/O.
|
|
1136
|
-
|
|
1137
|
-
§provider-output-budget-conformance When a completed response reports
|
|
1138
|
-
normalized output-token usage greater than its response grant ({§provider-flexed-allowance}),
|
|
1139
|
-
the exchange is an `invalid_response` at 502 rather than an admitted result or
|
|
1140
|
-
a prompt-capacity 413. Its complete failed-attempt evidence and settled charged
|
|
1141
|
-
request remain available. The violation is final and is never automatically
|
|
1142
|
-
replayed. Missing output usage cannot prove a violation.
|
|
1143
|
-
|
|
1144
|
-
`PLURNK_PROVIDERS_OUTPUT_BUDGET` is required for standard providers and ships
|
|
1145
|
-
as `35%`. `PLURNK_PROVIDERS_REASONING_BUDGET` is optional; leaving it unset
|
|
1146
|
-
preserves provider-adaptive depth. A backend known to decode without a finite
|
|
1147
|
-
limit advertises `requiresOutputBudget` and fails construction when no total can
|
|
1148
|
-
be resolved.
|
|
1149
|
-
|
|
1150
|
-
## §13 Inference capacity
|
|
1151
|
-
|
|
1152
|
-
### Admission
|
|
1153
|
-
|
|
1154
|
-
§provider-inference-admission `PLURNK_PROVIDERS_MAX_CONCURRENCY` controls physical
|
|
1155
|
-
generation attempts per resolved endpoint within one process: `-1` is unrestricted;
|
|
1156
|
-
a positive safe integer admits that many concurrent attempts. It follows ordinary
|
|
1157
|
-
alias scoping. Zero, other negative values and non-integers are invalid.
|
|
1158
|
-
|
|
1159
|
-
| Boundary | Contract |
|
|
1160
|
-
| --- | --- |
|
|
1161
|
-
| Identity | Provider instances and aliases sharing the resolved API base URL share one allowance. SDK- or extension-owned endpoints without a resolved URL share their provider identity. Conflicting limits for one identity fail construction, naming the identity and both values; they never create independent queues. The daemon reads its environment once at boot, so an identity's limit cannot change within a process: reconstructing a provider from that environment reuses its allowance, and in-flight leases keep counting. |
|
|
1162
|
-
| Admission | FIFO among live waiters. The lease begins before the physical request observer and ends after the complete response or transport failure settles, including streamed bodies. |
|
|
1163
|
-
| Cancellation | A queued abort removes that waiter and preserves the caller's reason. It opens no physical request or accounting row. In-flight cancellation signals the transport; capacity is released when that attempt unwinds. A transport still running despite abort does not authorize exceeding the limit. |
|
|
1164
|
-
| Retries | Backoff holds no lease. Each retry rejoins admission as a new physical attempt. |
|
|
1165
|
-
| Deadline | The whole-call operation deadline includes queueing, every physical attempt and retry wait. Transport adds no generation deadline. |
|
|
1166
|
-
| Scope | Workers, tools, messages, waits, token measurement and endpoint discovery are not serialized by inference admission. Independent endpoints progress independently. No cross-process or machine-wide capacity guarantee is implied. |
|
|
1167
|
-
|
|
1168
|
-
Admission neither changes worker lifecycle nor selects another model or endpoint.
|
|
1169
|
-
Core consumes the same asynchronous Provider contract. A waiting worker holds no
|
|
1170
|
-
inference lease; BARE and ordinary generation use the same physical boundary.
|
|
1171
|
-
|
|
1172
|
-
### Pool
|
|
1173
|
-
|
|
1174
|
-
§provider-capacity-pool `Pool` fronts interchangeable `Provider` instances. It
|
|
1175
|
-
keeps workers sticky for
|
|
1176
|
-
cache locality, selects a healthy sibling for overflow, and preserves the same
|
|
1177
|
-
Provider contract. Whether endpoints are interchangeable is a consumer
|
|
1178
|
-
decision, not inferred from provider names. Overflow is limited to transport
|
|
1179
|
-
availability and rate-limit failures that carry no normalized response attempt;
|
|
1180
|
-
{§provider-interrupted-attempt} propagates without overflow.
|
|
1181
|
-
|
|
1182
|
-
Prompt measurement covers every backend that could receive the request. The
|
|
1183
|
-
pool takes the largest quantified result; differing exact counts or any proven
|
|
1184
|
-
bound yield an `upper_bound`, any estimate makes the aggregate an estimate, and
|
|
1185
|
-
any unavailable backend makes it unavailable. Physical limits and budgets are
|
|
1186
|
-
independent safe minima across the pool. `inputCapacity` is the minimum of each
|
|
1187
|
-
backend's complete derived input capacity, never a synthetic subtraction across
|
|
1188
|
-
minima from different backends; request-specific output tightening repeats the
|
|
1189
|
-
complete-envelope derivation per backend before taking the minimum.
|
|
1190
|
-
|
|
1191
|
-
## §14 Conformance
|
|
1192
|
-
|
|
1193
|
-
Coverage MUST prove:
|
|
1194
|
-
|
|
1195
|
-
- catalog, declaration, local, and extension resolution;
|
|
1196
|
-
- exact and unique-suffix model lookup;
|
|
1197
|
-
- native SDK request mapping and normalized responses;
|
|
1198
|
-
- compatible extension preservation;
|
|
1199
|
-
- timeout, retry, cancellation, interrupted-attempt, and final-error behavior;
|
|
1200
|
-
- shared endpoint inference admission, FIFO queueing, queued/in-flight cancellation,
|
|
1201
|
-
full-stream lease lifetime, retry release, and independent endpoint progress;
|
|
1202
|
-
- local capability probes and pins;
|
|
1203
|
-
- exact, bounded, estimated, and unavailable complete-request measurements;
|
|
1204
|
-
- independent input/context/output limits, asymmetric admission, and normalized
|
|
1205
|
-
local/upstream capacity failures;
|
|
1206
|
-
- one total output budget and native additive-reasoning projection;
|
|
1207
|
-
- provider-reported output beyond that budget failing once with complete
|
|
1208
|
-
attempt and accounting evidence;
|
|
1209
|
-
- local reasoning activation, response-wide allowance, and GBNF coexistence;
|
|
1210
|
-
- explicit tagged-reasoning projection across streamed, buffered, capped, and
|
|
1211
|
-
literal-tag responses;
|
|
1212
|
-
- usage, costs, evidence, metadata isolation, and grammar observation;
|
|
1213
|
-
- alias scoping and fail-hard invalid configuration.
|
|
1214
|
-
|
|
1215
|
-
Mock-only green tests do not establish a vendor integration. Live drills and
|
|
1216
|
-
integration tests complement this contract; they do not replace its unit-level
|
|
1217
|
-
proof.
|
|
125
|
+
- Outbound HTTP requests honor `HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY`, and `NO_PROXY`.
|
|
126
|
+
- When an upstream stream terminates with an error or length finish reason (`MALFORMED_RESPONSE`, `MALFORMED_FUNCTION_CALL`, `MAX_TOKENS`) after delivering a complete, balanced fenced emission with no unclosed fence, no native tool calls, and no trailing unfenced text, `finishReason` normalizes to `"stop"` while preserving `rawFinishReason` in forensic evidence.
|
|
127
|
+
- Transient HTTP failures, upstream malformed-decoder aborts (`MALFORMED_RESPONSE`, `MALFORMED_FUNCTION_CALL`, `UNEXPECTED_TOOL_CALL`) without a complete fenced emission, and reasoning-only `MAX_TOKENS` cutoffs with empty visible content retry up to `PLURNK_PROVIDERS_RETRY_ATTEMPTS` inside the provider (stepping down Google thinking posture on retry when reasoning starved visible output); if a stream fails after emitting partial visible output and a retry succeeds, a `provider_retry` notice is attached.
|
|
128
|
+
- Problem `detail` and structured `cause.causeMessage` strings are bounded to `PLURNK_PROVIDERS_ERROR_DETAIL_LIMIT` characters without leaking raw payloads or headers.
|
|
129
|
+
|
|
130
|
+
## §provider-inference-admission Inference admission
|
|
131
|
+
|
|
132
|
+
`InferenceAdmission.forEndpoint(endpoint, maxConcurrency)` coordinates in-flight generations across all `Provider` instances targeting the same canonical endpoint URL (`PLURNK_PROVIDERS_MAX_CONCURRENCY`, default `-1` unbounded, or a positive safe integer):
|
|
133
|
+
|
|
134
|
+
- Waiters acquire slots in strict FIFO order before opening a physical request (`observeRequest` does not fire while queued) and hold the slot until the response stream or error settles.
|
|
135
|
+
- Aborted waiters are removed from the queue immediately without consuming capacity; retry backoff releases the slot before sleeping and re-acquires it for the next attempt.
|
|
136
|
+
- Conflicting positive/unbounded `PLURNK_PROVIDERS_MAX_CONCURRENCY` declarations for the same canonical endpoint throw at construction.
|