@plurnk/plurnk-providers 6.1.1 → 6.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/.env.defaults +5 -4
- package/README.md +3 -3
- package/SPEC.md +18 -15
- package/dist/InferenceAdmission.d.ts +10 -1
- package/dist/InferenceAdmission.d.ts.map +1 -1
- package/dist/InferenceAdmission.js +79 -15
- package/dist/InferenceAdmission.js.map +1 -1
- package/dist/Mock.d.ts +9 -0
- package/dist/Mock.d.ts.map +1 -1
- package/dist/Mock.js +13 -1
- package/dist/Mock.js.map +1 -1
- package/dist/ModelProvider.d.ts +3 -1
- package/dist/ModelProvider.d.ts.map +1 -1
- package/dist/ModelProvider.js +27 -9
- package/dist/ModelProvider.js.map +1 -1
- package/dist/ModelRequestBody.js +3 -3
- package/dist/ModelRequestBody.js.map +1 -1
- package/dist/catalogProvider.d.ts +2 -2
- package/dist/catalogProvider.d.ts.map +1 -1
- package/dist/catalogProvider.js +2 -1
- package/dist/catalogProvider.js.map +1 -1
- package/dist/compatibleProvider.d.ts.map +1 -1
- package/dist/compatibleProvider.js +6 -3
- package/dist/compatibleProvider.js.map +1 -1
- package/dist/env.d.ts +3 -1
- package/dist/env.d.ts.map +1 -1
- package/dist/env.js +24 -2
- package/dist/env.js.map +1 -1
- package/dist/index.d.ts +5 -3
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +3 -2
- package/dist/index.js.map +1 -1
- package/dist/modelAdapters.d.ts +2 -2
- package/dist/modelAdapters.d.ts.map +1 -1
- package/dist/modelAdapters.js +53 -4
- package/dist/modelAdapters.js.map +1 -1
- package/dist/pi-transport.d.ts +11 -10
- package/dist/pi-transport.d.ts.map +1 -1
- package/dist/pi-transport.js +752 -93
- package/dist/pi-transport.js.map +1 -1
- package/dist/stream-forensics.d.ts +4 -8
- package/dist/stream-forensics.d.ts.map +1 -1
- package/dist/stream-forensics.js +61 -192
- package/dist/stream-forensics.js.map +1 -1
- package/dist/types.d.ts +38 -5
- package/dist/types.d.ts.map +1 -1
- package/dist/types.js +15 -3
- package/dist/types.js.map +1 -1
- package/package.json +4 -4
package/.env.defaults
CHANGED
|
@@ -16,8 +16,9 @@
|
|
|
16
16
|
# PLURNK_BASEURL_<alias>=<url>
|
|
17
17
|
|
|
18
18
|
# --- Capacity ---
|
|
19
|
-
# Simultaneous inference attempts per resolved endpoint in this process; -1 = unrestricted
|
|
20
|
-
#
|
|
19
|
+
# Simultaneous inference attempts per resolved endpoint in this process; -1 = unrestricted, or
|
|
20
|
+
# bounded automatically by probed physical slot capacity (e.g. llama-server GET /props total_slots).
|
|
21
|
+
# Positive limits queue excess calls and are capped by probed physical slots.
|
|
21
22
|
PLURNK_PROVIDERS_MAX_CONCURRENCY=-1
|
|
22
23
|
# PLURNK_PROVIDERS_MAX_CONCURRENCY_local=1
|
|
23
24
|
# Reserved output INCLUDING reasoning: positive tokens or context percentage in (0,100).
|
|
@@ -77,9 +78,9 @@ PLURNK_PROVIDERS_DRY_MULTIPLIER=0
|
|
|
77
78
|
# --- Cache and billing ---
|
|
78
79
|
# Stable Session affinity on supported routes: 1 = on; 0 = off. Unknown routes get no guessed field.
|
|
79
80
|
PLURNK_PROVIDERS_CACHE_AFFINITY=1
|
|
80
|
-
# Explicit cache writes on explicit-breakpoint routes (Anthropic, OpenRouter Anthropic, Bedrock Claude):
|
|
81
|
+
# Explicit cache writes on explicit-breakpoint routes (Anthropic, MiniMax Anthropic, OpenRouter Anthropic, Bedrock Claude, OpenAI GPT-5.6+):
|
|
81
82
|
# conversation = system + rolling conversation/log prefix before volatile turn status;
|
|
82
|
-
# stable-system = reusable system boundary only; off = no explicit writes. Distinct from affinity and affects billing.
|
|
83
|
+
# stable-system = reusable system boundary only; off = no explicit writes (suppresses paid implicit writes on OpenAI GPT-5.6+). Distinct from affinity and affects billing.
|
|
83
84
|
PLURNK_PROVIDERS_CACHE_WRITE_POLICY=conversation
|
|
84
85
|
# Provider service tier; unset = provider default. Usually set per alias: routing can change cost.
|
|
85
86
|
# Native routes send it only where the SDK documents one ({§provider-request-authority}); others warn.
|
package/README.md
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
# @plurnk/plurnk-providers
|
|
2
2
|
|
|
3
|
-
Model-provider contract and transport adapters over `@earendil-works/pi-ai` and OpenAI-compatible HTTP endpoints.
|
|
3
|
+
Model-provider contract and transport adapters over `@earendil-works/pi-ai` and OpenAI-compatible HTTP endpoints. Emission requests declare the single structured `mcp` wire tool (`MCP_WIRE_TOOL`) and replay `assistant` `toolCalls` and `tool` results natively across protocols; `"bare"` calls omit tools ({§provider-interface}, {§provider-request-authority}).
|
|
4
4
|
|
|
5
5
|
- **Contract:** [`SPEC.md`](SPEC.md)
|
|
6
6
|
- **Model-facing reference:** [`docs/models.md`](docs/models.md)
|
|
@@ -57,8 +57,8 @@ In-package `openai` and `ollama` adapters probe live context windows, `llama-ser
|
|
|
57
57
|
| Boundary | Rule |
|
|
58
58
|
| --- | --- |
|
|
59
59
|
| Effort & reasoning budget | `adaptive` requests dynamic reasoning or falls back to `PLURNK_PROVIDERS_EFFORT_FALLBACK` (`high`); explicit fixed efforts must be route-supported. Unset reasoning budget leaves reasoning enabled. |
|
|
60
|
-
| Output & floor (`PLURNK_PROVIDERS_OUTPUT_FLOOR`) | Shipped `
|
|
61
|
-
| Concurrency & deadlines |
|
|
60
|
+
| Output & floor (`PLURNK_PROVIDERS_OUTPUT_FLOOR`) | Shipped `10%` floor (`35%` output budget) sets the input wall (`contextWindow - floor`); packets past the output budget flex output down to the remainder ({§provider-capacity-admission}). |
|
|
61
|
+
| Concurrency & deadlines | Per-endpoint FIFO `MAX_CONCURRENCY` gate ({§provider-inference-admission}), automatically bounded by probed `llama-server` `total_slots` and resized live across `.env` reloads; one whole-call generation deadline across queueing and `Retry-After` retries. |
|
|
62
62
|
| Provider extensions | First-party `@plurnk/*` packages with `plurnk: { "kind": "provider", "name": "..." }` default-export `ProviderExtension` (`model(modelId)`, optional `attributions(context)`, {§extension-discovery}). |
|
|
63
63
|
|
|
64
64
|
## Live conformance & development
|
package/SPEC.md
CHANGED
|
@@ -19,7 +19,7 @@ Every `Provider` instance exposes:
|
|
|
19
19
|
| Physical & policy getters | `model`, `servedModel`, `contextWindow`, `maxInputTokens`, `maxOutputTokens`, `outputBudget`, `outputFloor`, `reasoningBudget`, `inputCapacity`, `inputWall`, `supportedEfforts`, `inputModalities` (`ReadonlySet<"image" \| "audio" \| "video" \| "pdf">`), `temperature`, `seed`, and `requiresOutputBudget` |
|
|
20
20
|
| `countPromptTokens(messages, signal?)` | Measures or estimates prompt token weight (`PromptTokenMeasurement`) |
|
|
21
21
|
| `assessRequestCapacity(messages, maxOutputTokens?, signal?)` | Evaluates `inputCapacity`, `inputWall`, admission `decision` (`"admit" \| "defer" \| "reject"`), and flexed `responseMax` |
|
|
22
|
-
| `generate(args)` | Requires non-empty `sessionId` (used for KV-cache and slot affinity), optional `callKind` (`"emission"` or `"bare"`), `maxOutputTokens`, `sampling`, `observeRequest`, and `observeReasoning`.
|
|
22
|
+
| `generate(args)` | Requires non-empty `sessionId` (used for KV-cache and slot affinity), optional `callKind` (`"emission"` or `"bare"`), `maxOutputTokens`, `sampling`, `observeRequest`, and `observeReasoning`. `"emission"` requests declare the universal `mcp` wire tool (`MCP_WIRE_TOOL`) and project structured `assistant` `toolCalls` and `tool` receipts across every protocol; `"bare"` calls omit all tool declarations |
|
|
23
23
|
| `observeRequest` | Opens before each physical HTTP attempt and settles once with `(requestAccounting, forensicEvidence)`. When `PLURNK_PROVIDERS_RAWBODY=1`, `forensicEvidence.request` records `{ origin, method, body }` with credentials and URL path/query secrets stripped |
|
|
24
24
|
| `observeReasoning` | Streams reasoning deltas live from structured reasoning events (`thinking_delta`) or leading tagged reasoning envelopes (`<think>...</think>` when `PLURNK_PROVIDERS_REASONING_RESPONSE_STYLE=think-tags`), and rejoins split reasoning tails when an upstream parser prematurely exits native thinking before `</think>` |
|
|
25
25
|
| `Pool` | Fronts $N \ge 1$ homogeneous backends sharing a `model` id: aggregates limits to conservative floors, pins each `sessionId` to one backend for KV-cache locality, and overflows to a healthy sibling on retryable availability failures (`network_failure`, `rate_limit`, `deadline_exceeded`) while never overflowing terminal errors or `resource_interrupted` attempts |
|
|
@@ -46,7 +46,7 @@ Every `Provider` instance exposes:
|
|
|
46
46
|
|
|
47
47
|
Every physical attempt settles a `ProviderRequestAccounting` record (`{ provider, model, outcome, status?, usage?, cost }` under {§provider-request-accounting}):
|
|
48
48
|
|
|
49
|
-
- `normalizeUsage` validates non-negative safe integers, recovers additive reasoning when `prompt + completion + reasoning === total`, derives a single missing input cache partition (`noCacheTokens`, `cacheReadTokens`, `cacheWriteTokens`) when the other two and total are known, and rejects contradictory counters (`UsageDetailError`) while preserving valid outer totals.
|
|
49
|
+
- `normalizeUsage` validates non-negative safe integers, recovers additive reasoning when `prompt + completion + reasoning === total`, derives a single missing input cache partition (`noCacheTokens`, `cacheReadTokens`, `cacheWriteTokens`) when the other two and total are known, and rejects contradictory counters (`UsageDetailError`) while preserving valid outer totals. On Google `usageMetadata` (proto3 JSON default-value omission), omitted `cachedContentTokenCount` alongside reported `promptTokenCount` records `0` cached tokens (`cache_write_tokens: 0`), and omitted `thoughtsTokenCount` when `totalTokenCount === promptTokenCount + candidatesTokenCount` records `0` reasoning tokens.
|
|
50
50
|
- Cost precedence resolves in order:
|
|
51
51
|
1. Direct provider charge evidence (`kind: "charged"`, e.g. OpenRouter `usage.cost` and BYOK `upstream_inference_cost`).
|
|
52
52
|
2. Direct provider response estimate (`kind: "estimated"`, e.g. DeepInfra `usage.estimated_cost`).
|
|
@@ -59,10 +59,10 @@ Every physical attempt settles a `ProviderRequestAccounting` record (`{ provider
|
|
|
59
59
|
`ModelProvider.ts` delegates streaming, protocol framing, vendor request assembly, model compatibility flags (`getBuiltinModels`), header normalization (`headersToRecord`), interruptible backoff (`sleep`), token estimation (`estimateMessageTokens`), and context-overflow pattern matching (`getOverflowPatterns`) to `@earendil-works/pi-ai` across all supported APIs (`anthropic-messages`, `openai-completions`, `openai-responses`, `google-generative-ai`, `bedrock-converse-stream`):
|
|
60
60
|
|
|
61
61
|
- Prompt cache affinity (`PLURNK_PROVIDERS_CACHE_AFFINITY=1`) projects each provider's documented session key (`sessionId` -> `prompt_cache_key` / `x-session-id` / `x-session-affinity`).
|
|
62
|
-
- Explicit prompt-cache write policy (`PLURNK_PROVIDERS_CACHE_WRITE_POLICY`, default `conversation`: `conversation | stable-system | off`) governs explicit cache breakpoints (`cache_control: { type: "ephemeral" }` on Anthropic and OpenRouter Anthropic routes; `cachePoint: { type: "default" }` on Bedrock Claude prompt-caching routes), suppressed on stateless `callKind: "bare"` calls:
|
|
63
|
-
- `off
|
|
62
|
+
- Explicit prompt-cache write policy (`PLURNK_PROVIDERS_CACHE_WRITE_POLICY`, default `conversation`: `conversation | stable-system | off`) governs explicit cache breakpoints (`cache_control: { type: "ephemeral" }` on Anthropic, MiniMax Anthropic, and OpenRouter Anthropic routes; `cachePoint: { type: "default" }` on Bedrock Claude prompt-caching routes; `prompt_cache_options: { mode: "explicit" }` with `prompt_cache_breakpoint: { mode: "explicit" }` on OpenAI GPT-5.6+ routes, including OpenRouter `openai/*`), suppressed on stateless `callKind: "bare"` calls:
|
|
63
|
+
- `off` (and `callKind: "bare"`): zero explicit cache breakpoints; on OpenAI GPT-5.6+ routes `prompt_cache_options: { mode: "explicit" }` is still sent with zero breakpoints so implicit paid cache writes are disabled.
|
|
64
64
|
- `stable-system`: one breakpoint on the final leading `system` instruction.
|
|
65
|
-
- `conversation`: breakpoints on the final leading `system` instruction, the penultimate conversation message, and the
|
|
65
|
+
- `conversation`: splits caller-delimited `userCachePrefixes` on historical `user` messages into separate content blocks (mapping source `ChatMessage` indices across coalesced `tool_result` + `user` wire blocks on Anthropic and Bedrock) so prior turns' cached durable log blocks retain their exact block boundaries. Ephemeral-breakpoint routes (`cache_control` / `cachePoint`) place breakpoints on the final leading `system` instruction, the penultimate conversation message, and the durable prefix boundary before any volatile per-turn status tail in the final `user` message (including on the preceding `tool_result` block when coalesced ahead of a zero-prefix status tail, or the full final `user` message when no volatile tail is delimited), staying within three breakpoints per request; OpenAI GPT-5.6+ explicit routes keep monotonic `prompt_cache_breakpoint` markers on the final leading `system` instruction and each durable `tool` / `user` prefix boundary across turns so prior segment boundaries remain stable while writing only new uncached prefixes.
|
|
66
66
|
- Output token allowance (`responseMax ?? outputBudget`) is enforced both in flight during streaming (aborting a runaway stream once mid-stream reported output tokens or conservative token lower-bound `floor(estimateMessageTokens(partial) / 2)` exceeds the granted allowance) and on final reported `usage.outputTokens`.
|
|
67
67
|
|
|
68
68
|
## §provider-effort Reasoning effort and readable trace
|
|
@@ -78,13 +78,13 @@ Every physical attempt settles a `ProviderRequestAccounting` record (`{ provider
|
|
|
78
78
|
|
|
79
79
|
| Knob | Default / Form | Effect |
|
|
80
80
|
|---|---|---|
|
|
81
|
-
| `PLURNK_PROVIDERS_OUTPUT_BUDGET` | `
|
|
82
|
-
| `PLURNK_PROVIDERS_OUTPUT_FLOOR` | `
|
|
81
|
+
| `PLURNK_PROVIDERS_OUTPUT_BUDGET` | `35%` (`<pct>%` or positive integer tokens) | Soft output reservation subtracted from `contextWindow` to form `inputCapacity` |
|
|
82
|
+
| `PLURNK_PROVIDERS_OUTPUT_FLOOR` | `10%` (`<pct>%` of effective output ceiling or integer tokens) | Hard minimum output allowance subtracted from `contextWindow` to form `inputWall` |
|
|
83
83
|
| `PLURNK_PROVIDERS_REASONING_BUDGET` | empty (`<pct>%` or integer tokens, `< outputBudget`) | Explicit thinking token budget under `adaptive` effort |
|
|
84
84
|
| `PLURNK_PROVIDERS_EFFORT` / `EFFORT_FALLBACK` | `adaptive` / `high` | Portable reasoning effort selector and graded-route fallback |
|
|
85
85
|
| `PLURNK_PROVIDERS_FETCH_TIMEOUT` / `OPERATION_TIMEOUT` | `30000` / `600000` ms | Per-attempt fetch/idle timeout and total operation wall-clock timeout |
|
|
86
86
|
| `PLURNK_PROVIDERS_RETRY_ATTEMPTS` | `3` | Maximum automatic retry attempts for retryable transport failures |
|
|
87
|
-
| `PLURNK_PROVIDERS_MAX_CONCURRENCY` | `-1` (unbounded or positive integer) | Per-endpoint FIFO concurrency gate ({§provider-inference-admission}) |
|
|
87
|
+
| `PLURNK_PROVIDERS_MAX_CONCURRENCY` | `-1` (unbounded when unprobed, or positive integer) | Per-endpoint FIFO concurrency gate, automatically bounded by probed physical endpoint slots ({§provider-inference-admission}) |
|
|
88
88
|
| `PLURNK_PROVIDERS_COST_<CATEGORY>` | empty (non-negative USD per 1M tokens for `INPUT`, `OUTPUT`, `REASONING`, `CACHE_READ`, `CACHE_WRITE`) | Per-category operator rate overrides merged over catalog rates ({§provider-monetary-evidence}) |
|
|
89
89
|
| `PLURNK_PROVIDERS_RAWBODY` | `0` (`0 \| 1`) | Retains redacted raw request/response forensic bodies in `forensicEvidence` |
|
|
90
90
|
|
|
@@ -97,13 +97,15 @@ Every physical attempt settles a `ProviderRequestAccounting` record (`{ provider
|
|
|
97
97
|
| Route family | Fact authority |
|
|
98
98
|
|---|---|
|
|
99
99
|
| Cataloged cloud provider (`catalogProvider.ts`) | `resolveModel(provider, model)` from `@plurnk/plurnk-models`, capped by optional `PLURNK_PROVIDERS_CONTEXT_WINDOW`. Uncataloged models on a catalog provider require an explicit `PLURNK_PROVIDERS_CONTEXT_WINDOW`. Catalog credential names (`info.env`) or `PLURNK_PROVIDERS_PROVIDER_<ID>_API_KEY_ENV` (one exact variable name, never a comma list) supply authentication. |
|
|
100
|
-
| Local / compatible endpoint (`compatibleProvider.ts`) | Probes `GET <baseUrl>/models` for `n_ctx` / `max_model_len` / `context_length`. When llama-server is detected (`meta.n_ctx` or `PLURNK_PROVIDERS_LLAMA_SERVER=1`), probes `/props` for `total_slots` and `eos_token` and measures prompts via `/v1/chat/completions/input_tokens`. |
|
|
100
|
+
| Local / compatible endpoint (`compatibleProvider.ts`) | Probes `GET <baseUrl>/models` for `n_ctx` / `max_model_len` / `context_length`. When llama-server is detected (`meta.n_ctx` or `PLURNK_PROVIDERS_LLAMA_SERVER=1`), probes `/props` for `total_slots` (bounding endpoint inference concurrency via {§provider-inference-admission}) and `eos_token` and measures prompts via `/v1/chat/completions/input_tokens`. |
|
|
101
101
|
| Built-in Ollama rail (`ollama.ts`) | Probes `POST <origin>/api/show` for `*.context_length`, capped by optional `PLURNK_PROVIDERS_CONTEXT_WINDOW`, and dispatches chat completions to `<origin>/v1/chat/completions`. |
|
|
102
102
|
| Operator probe scripts (`scripts/`) | `providers-ping.mjs` probes one cheapest declared route per keyed provider and writes redacted JSON evidence; `token-envelope-probe.mjs` probes explicit aliases across configured or physical output limits. |
|
|
103
103
|
|
|
104
104
|
## §provider-request-authority Request controls
|
|
105
105
|
|
|
106
|
-
- Caller `sampling`
|
|
106
|
+
- Caller `sampling` fields cannot override protocol-owned fields (`model`, `messages`, `stream`, `tools`, `tool_choice`, `toolConfig`, `response_format`, `logprobs`, `top_logprobs`).
|
|
107
|
+
- `"emission"` requests declare the single `mcp` wire tool (`MCP_WIRE_TOOL`), an empty `tools: []` array is never sent, and `"bare"` calls omit all tool fields (`tools`, `tool_choice`, `toolConfig`).
|
|
108
|
+
- Provider-signed assistant replay metadata (`ChatMessage.replay` carrying `{ api, provider, model, thinking?, textSignature? }` and `WireToolCall.thoughtSignature`, extracted into `assistantRaw.wire` from signed `thinking` / `redacted` blocks, Gemini `thought: true` summaries, `textSignature`, and `thoughtSignature` while excluding raw OpenAI-completions reasoning field tags) rehydrates onto `pi-ai` `AssistantMessage` blocks so same-provider/same-model turns replay native reasoning continuity (Anthropic/Bedrock signed `thinking` / `redacted_thinking`, OpenAI Responses encrypted reasoning items, OpenRouter `reasoning_details`, and Google `thought` + `thoughtSignature`), while cross-model turns strip incompatible signatures automatically and Google `model` history turns whose first `functionCall` part lacks a valid `thoughtSignature` (such as synthetic initialization turns, cross-model switches, or curated turns where the leading call changed) fall back to `"skip_thought_signature_validator"` on the first `functionCall` part; consecutive `user` messages on Anthropic, Bedrock, and Google (such as `tool_result` / `functionResponse` followed by `user` log/status, folded into the final `functionResponse` on Google so `functionResponse` turns remain pure) merge in order.
|
|
107
109
|
- `PLURNK_PROVIDERS_SERVICE_TIER` and `PLURNK_PROVIDERS_TOP_LOGPROBS` project onto native SDK requests only when the route's SDK documents the control; unsupported controls emit a single `PLURNK_REQUEST_CONTROL_UNSUPPORTED` process warning and are omitted from the wire.
|
|
108
110
|
- OpenRouter requests attach `HTTP-Referer` and `X-OpenRouter-Title` from `PLURNK_PROVIDERS_PROVIDER_OPENROUTER_APP_URL` (validated as an absolute HTTP(S) URL) and `PLURNK_PROVIDERS_PROVIDER_OPENROUTER_APP_NAME`.
|
|
109
111
|
|
|
@@ -126,14 +128,15 @@ All transport and response failures normalize into `ProviderError` carrying an R
|
|
|
126
128
|
| `model_refused` | Upstream safety/content refusal finish reason | 422 | `false` | `attempt` |
|
|
127
129
|
|
|
128
130
|
- Outbound HTTP requests honor `HTTPS_PROXY`, `HTTP_PROXY`, `ALL_PROXY`, and `NO_PROXY`.
|
|
129
|
-
-
|
|
130
|
-
- Transient HTTP failures, upstream malformed-decoder aborts (`MALFORMED_RESPONSE`, `MALFORMED_FUNCTION_CALL`, `UNEXPECTED_TOOL_CALL`) without complete Code Mode content, and reasoning-only `MAX_TOKENS` cutoffs with empty visible content retry up to `PLURNK_PROVIDERS_RETRY_ATTEMPTS` inside the provider (stepping down Google thinking posture on retry when reasoning starved visible output); if a stream fails after emitting partial visible output and a retry succeeds, a `provider_retry` notice is attached.
|
|
131
|
+
- Transient HTTP failures, upstream malformed-decoder aborts (`MALFORMED_RESPONSE`, `MALFORMED_FUNCTION_CALL`, `UNEXPECTED_TOOL_CALL`), and reasoning-only `MAX_TOKENS` cutoffs with empty visible content and zero tool calls retry up to `PLURNK_PROVIDERS_RETRY_ATTEMPTS` inside the provider (stepping down Google thinking posture on retry when reasoning starved visible output); if a stream fails after emitting partial visible output and a retry succeeds, a `provider_retry` notice is attached.
|
|
131
132
|
- Problem `detail` and structured `cause.causeMessage` strings are bounded to `PLURNK_PROVIDERS_ERROR_DETAIL_LIMIT` characters without leaking raw payloads or headers.
|
|
132
133
|
|
|
133
134
|
## §provider-inference-admission Inference admission
|
|
134
135
|
|
|
135
|
-
`InferenceAdmission.forEndpoint(endpoint, maxConcurrency)` coordinates in-flight generations across all `Provider` instances targeting the same canonical endpoint URL (`PLURNK_PROVIDERS_MAX_CONCURRENCY`, default `-1` unbounded, or a positive safe integer):
|
|
136
|
+
`InferenceAdmission.forEndpoint(endpoint, maxConcurrency, options?)` coordinates in-flight generations across all `Provider` instances targeting the same canonical endpoint URL (`PLURNK_PROVIDERS_MAX_CONCURRENCY`, default `-1` unbounded, or a positive safe integer):
|
|
136
137
|
|
|
138
|
+
- `effectiveMaxConcurrency(configuredLimit, physicalSlots)` reconciles operator policy with probed physical endpoint slot capacity (`GET /props` `total_slots` on llama-server routes, `null` when unprobed): `-1` adopts `physicalSlots` when probed (remaining `-1` when `null`), and a positive `configuredLimit` resolves to `min(configuredLimit, physicalSlots)` when probed.
|
|
137
139
|
- Waiters acquire slots in strict FIFO order before opening a physical request (`observeRequest` does not fire while queued) and hold the slot until the response stream or error settles.
|
|
138
|
-
- Aborted waiters are removed from the queue immediately without consuming capacity; retry backoff releases the slot before sleeping and re-acquires it for the next attempt.
|
|
139
|
-
-
|
|
140
|
+
- Aborted waiters are removed from the queue immediately in $O(1)$ without consuming capacity or retaining signal listeners; retry backoff releases the slot before sleeping and re-acquires it for the next attempt.
|
|
141
|
+
- Within one configuration generation, conflicting effective allowances across aliases targeting the same canonical endpoint throw at construction; aliases that resolve to the same effective allowance (such as `-1` alongside explicit `1` on a 1-slot llama-server) share the gate.
|
|
142
|
+
- Advancing the configuration generation (`InferenceAdmission.advanceGeneration`) on operator `.env` reload or observing a changed physical `total_slots` under unchanged operator policy resizes the existing endpoint semaphore in place: growing immediately wakes queued FIFO waiters up to the new allowance, and shrinking lets surplus in-flight leases finish naturally before admitting queued work.
|
|
@@ -1,7 +1,16 @@
|
|
|
1
|
+
export interface InferenceAdmissionOptions {
|
|
2
|
+
readonly configuredLimit?: number;
|
|
3
|
+
readonly generation?: number;
|
|
4
|
+
}
|
|
1
5
|
export default class InferenceAdmission {
|
|
2
6
|
#private;
|
|
3
|
-
static
|
|
7
|
+
static get currentGeneration(): number;
|
|
8
|
+
static advanceGeneration(generation?: number): number;
|
|
9
|
+
static forEndpoint(endpoint: string, limit: number, options?: InferenceAdmissionOptions): InferenceAdmission;
|
|
4
10
|
private constructor();
|
|
11
|
+
get limit(): number;
|
|
12
|
+
get activeCount(): number;
|
|
13
|
+
get waitingCount(): number;
|
|
5
14
|
acquire(signal?: AbortSignal): Promise<() => void>;
|
|
6
15
|
}
|
|
7
16
|
//# sourceMappingURL=InferenceAdmission.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"InferenceAdmission.d.ts","sourceRoot":"","sources":["../src/InferenceAdmission.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"InferenceAdmission.d.ts","sourceRoot":"","sources":["../src/InferenceAdmission.ts"],"names":[],"mappings":"AAAA,MAAM,WAAW,yBAAyB;IACtC,QAAQ,CAAC,eAAe,CAAC,EAAE,MAAM,CAAC;IAClC,QAAQ,CAAC,UAAU,CAAC,EAAE,MAAM,CAAC;CAChC;AASD,MAAM,CAAC,OAAO,OAAO,kBAAkB;;IAUnC,MAAM,KAAK,iBAAiB,IAAI,MAAM,CAErC;IAID,MAAM,CAAC,iBAAiB,CAAC,UAAU,CAAC,EAAE,MAAM,GAAG,MAAM,CAYpD;IAED,MAAM,CAAC,WAAW,CACd,QAAQ,EAAE,MAAM,EAChB,KAAK,EAAE,MAAM,EACb,OAAO,CAAC,EAAE,yBAAyB,GACpC,kBAAkB,CAiCpB;IAED,OAAO,eAIN;IAED,IAAI,KAAK,IAAI,MAAM,CAElB;IAED,IAAI,WAAW,IAAI,MAAM,CAExB;IAED,IAAI,YAAY,IAAI,MAAM,CAEzB;IAeD,OAAO,CAAC,MAAM,CAAC,EAAE,WAAW,GAAG,OAAO,CAAC,MAAM,IAAI,CAAC,CA0BjD;CACJ"}
|
|
@@ -1,37 +1,104 @@
|
|
|
1
|
-
|
|
1
|
+
const assertValidLimit = (limit) => {
|
|
2
|
+
if (!Number.isSafeInteger(limit) || limit !== -1 && limit < 1) {
|
|
3
|
+
throw new TypeError("PLURNK_PROVIDERS_MAX_CONCURRENCY must be -1 or a positive safe integer");
|
|
4
|
+
}
|
|
5
|
+
};
|
|
6
|
+
// {§provider-inference-admission} One process-local FIFO allowance per canonical endpoint.
|
|
2
7
|
export default class InferenceAdmission {
|
|
3
8
|
static #endpoints = new Map();
|
|
9
|
+
static #currentGeneration = 0;
|
|
4
10
|
#limit;
|
|
11
|
+
#configuredLimit;
|
|
12
|
+
#generation;
|
|
5
13
|
#waiting = new Set();
|
|
6
14
|
#active = 0;
|
|
7
|
-
static
|
|
8
|
-
|
|
9
|
-
|
|
15
|
+
static get currentGeneration() {
|
|
16
|
+
return this.#currentGeneration;
|
|
17
|
+
}
|
|
18
|
+
// Advance the configuration generation monotonically on operator environment reload so live
|
|
19
|
+
// endpoints resize in place while conflicting aliases within the same generation still fail fast.
|
|
20
|
+
static advanceGeneration(generation) {
|
|
21
|
+
if (generation === undefined) {
|
|
22
|
+
this.#currentGeneration += 1;
|
|
23
|
+
return this.#currentGeneration;
|
|
24
|
+
}
|
|
25
|
+
if (!Number.isSafeInteger(generation) || generation < 0) {
|
|
26
|
+
throw new TypeError("InferenceAdmission generation must be a non-negative safe integer");
|
|
27
|
+
}
|
|
28
|
+
if (generation > this.#currentGeneration) {
|
|
29
|
+
this.#currentGeneration = generation;
|
|
30
|
+
}
|
|
31
|
+
return this.#currentGeneration;
|
|
32
|
+
}
|
|
33
|
+
static forEndpoint(endpoint, limit, options) {
|
|
34
|
+
assertValidLimit(limit);
|
|
35
|
+
const configuredLimit = options?.configuredLimit ?? limit;
|
|
36
|
+
assertValidLimit(configuredLimit);
|
|
37
|
+
const generation = options?.generation ?? this.#currentGeneration;
|
|
38
|
+
if (!Number.isSafeInteger(generation) || generation < 0) {
|
|
39
|
+
throw new TypeError("InferenceAdmission generation must be a non-negative safe integer");
|
|
10
40
|
}
|
|
11
41
|
const key = new URL(endpoint).href.replace(/\/+$/, "");
|
|
12
42
|
const existing = this.#endpoints.get(key);
|
|
13
43
|
if (existing !== undefined) {
|
|
14
|
-
if (existing.#
|
|
15
|
-
|
|
44
|
+
if (generation > existing.#generation) {
|
|
45
|
+
existing.#generation = generation;
|
|
46
|
+
existing.#configuredLimit = configuredLimit;
|
|
47
|
+
existing.#resize(limit);
|
|
48
|
+
return existing;
|
|
49
|
+
}
|
|
50
|
+
if (existing.#limit === limit) {
|
|
51
|
+
if (configuredLimit !== -1)
|
|
52
|
+
existing.#configuredLimit = configuredLimit;
|
|
53
|
+
return existing;
|
|
54
|
+
}
|
|
55
|
+
// Same operator policy on a restarted endpoint whose probed physical slot count changed.
|
|
56
|
+
if (existing.#configuredLimit === configuredLimit) {
|
|
57
|
+
existing.#resize(limit);
|
|
58
|
+
return existing;
|
|
16
59
|
}
|
|
17
|
-
|
|
60
|
+
throw new TypeError(`Provider aliases resolving to ${key} have conflicting PLURNK_PROVIDERS_MAX_CONCURRENCY values ${existing.#limit} and ${limit}; one endpoint has one allowance`);
|
|
18
61
|
}
|
|
19
|
-
const admission = new InferenceAdmission(limit);
|
|
62
|
+
const admission = new InferenceAdmission(limit, configuredLimit, generation);
|
|
20
63
|
this.#endpoints.set(key, admission);
|
|
21
64
|
return admission;
|
|
22
65
|
}
|
|
23
|
-
constructor(limit) {
|
|
66
|
+
constructor(limit, configuredLimit, generation) {
|
|
24
67
|
this.#limit = limit;
|
|
68
|
+
this.#configuredLimit = configuredLimit;
|
|
69
|
+
this.#generation = generation;
|
|
70
|
+
}
|
|
71
|
+
get limit() {
|
|
72
|
+
return this.#limit;
|
|
73
|
+
}
|
|
74
|
+
get activeCount() {
|
|
75
|
+
return this.#active;
|
|
76
|
+
}
|
|
77
|
+
get waitingCount() {
|
|
78
|
+
return this.#waiting.size;
|
|
79
|
+
}
|
|
80
|
+
#resize(limit) {
|
|
81
|
+
this.#limit = limit;
|
|
82
|
+
this.#drainQueue();
|
|
83
|
+
}
|
|
84
|
+
#drainQueue() {
|
|
85
|
+
while (this.#waiting.size > 0 && (this.#limit === -1 || this.#active < this.#limit)) {
|
|
86
|
+
const next = this.#waiting.values().next().value;
|
|
87
|
+
this.#waiting.delete(next);
|
|
88
|
+
next();
|
|
89
|
+
}
|
|
25
90
|
}
|
|
26
91
|
acquire(signal) {
|
|
27
92
|
signal?.throwIfAborted();
|
|
28
93
|
return new Promise((resolve, reject) => {
|
|
94
|
+
let queued = false;
|
|
29
95
|
const abort = () => {
|
|
30
96
|
this.#waiting.delete(enter);
|
|
31
97
|
reject(signal.reason);
|
|
32
98
|
};
|
|
33
99
|
const enter = () => {
|
|
34
|
-
|
|
100
|
+
if (queued)
|
|
101
|
+
signal?.removeEventListener("abort", abort);
|
|
35
102
|
this.#active += 1;
|
|
36
103
|
let released = false;
|
|
37
104
|
resolve(() => {
|
|
@@ -39,16 +106,13 @@ export default class InferenceAdmission {
|
|
|
39
106
|
throw new Error("Inference admission lease was released twice");
|
|
40
107
|
released = true;
|
|
41
108
|
this.#active -= 1;
|
|
42
|
-
|
|
43
|
-
if (next !== undefined) {
|
|
44
|
-
this.#waiting.delete(next);
|
|
45
|
-
next();
|
|
46
|
-
}
|
|
109
|
+
this.#drainQueue();
|
|
47
110
|
});
|
|
48
111
|
};
|
|
49
112
|
if (this.#limit === -1 || this.#active < this.#limit)
|
|
50
113
|
enter();
|
|
51
114
|
else {
|
|
115
|
+
queued = true;
|
|
52
116
|
this.#waiting.add(enter);
|
|
53
117
|
signal?.addEventListener("abort", abort, { once: true });
|
|
54
118
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"InferenceAdmission.js","sourceRoot":"","sources":["../src/InferenceAdmission.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"InferenceAdmission.js","sourceRoot":"","sources":["../src/InferenceAdmission.ts"],"names":[],"mappings":"AAKA,MAAM,gBAAgB,GAAG,CAAC,KAAa,EAAQ,EAAE;IAC7C,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,KAAK,KAAK,CAAC,CAAC,IAAI,KAAK,GAAG,CAAC,EAAE,CAAC;QAC5D,MAAM,IAAI,SAAS,CAAC,wEAAwE,CAAC,CAAC;IAClG,CAAC;AACL,CAAC,CAAC;AAEF,2FAA2F;AAC3F,MAAM,CAAC,OAAO,OAAO,kBAAkB;IACnC,MAAM,CAAU,UAAU,GAAG,IAAI,GAAG,EAA8B,CAAC;IACnE,MAAM,CAAC,kBAAkB,GAAG,CAAC,CAAC;IAE9B,MAAM,CAAS;IACf,gBAAgB,CAAS;IACzB,WAAW,CAAS;IACX,QAAQ,GAAG,IAAI,GAAG,EAAc,CAAC;IAC1C,OAAO,GAAG,CAAC,CAAC;IAEZ,MAAM,KAAK,iBAAiB;QACxB,OAAO,IAAI,CAAC,kBAAkB,CAAC;IACnC,CAAC;IAED,4FAA4F;IAC5F,kGAAkG;IAClG,MAAM,CAAC,iBAAiB,CAAC,UAAmB;QACxC,IAAI,UAAU,KAAK,SAAS,EAAE,CAAC;YAC3B,IAAI,CAAC,kBAAkB,IAAI,CAAC,CAAC;YAC7B,OAAO,IAAI,CAAC,kBAAkB,CAAC;QACnC,CAAC;QACD,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,UAAU,CAAC,IAAI,UAAU,GAAG,CAAC,EAAE,CAAC;YACtD,MAAM,IAAI,SAAS,CAAC,mEAAmE,CAAC,CAAC;QAC7F,CAAC;QACD,IAAI,UAAU,GAAG,IAAI,CAAC,kBAAkB,EAAE,CAAC;YACvC,IAAI,CAAC,kBAAkB,GAAG,UAAU,CAAC;QACzC,CAAC;QACD,OAAO,IAAI,CAAC,kBAAkB,CAAC;IACnC,CAAC;IAED,MAAM,CAAC,WAAW,CACd,QAAgB,EAChB,KAAa,EACb,OAAmC;QAEnC,gBAAgB,CAAC,KAAK,CAAC,CAAC;QACxB,MAAM,eAAe,GAAG,OAAO,EAAE,eAAe,IAAI,KAAK,CAAC;QAC1D,gBAAgB,CAAC,eAAe,CAAC,CAAC;QAClC,MAAM,UAAU,GAAG,OAAO,EAAE,UAAU,IAAI,IAAI,CAAC,kBAAkB,CAAC;QAClE,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,UAAU,CAAC,IAAI,UAAU,GAAG,CAAC,EAAE,CAAC;YACtD,MAAM,IAAI,SAAS,CAAC,mEAAmE,CAAC,CAAC;QAC7F,CAAC;QACD,MAAM,GAAG,GAAG,IAAI,GAAG,CAAC,QAAQ,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,MAAM,EAAE,EAAE,CAAC,CAAC;QACvD,MAAM,QAAQ,GAAG,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;QAC1C,IAAI,QAAQ,KAAK,SAAS,EAAE,CAAC;YACzB,IAAI,UAAU,GAAG,QAAQ,CAAC,WAAW,EAAE,CAAC;gBACpC,QAAQ,CAAC,WAAW,GAAG,UAAU,CAAC;gBAClC,QAAQ,CAAC,gBAAgB,GAAG,eAAe,CAAC;gBAC5C,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;gBACxB,OAAO,QAAQ,CAAC;YACpB,CAAC;YACD,IAAI,QAAQ,CAAC,MAAM,KAAK,KAAK,EAAE,CAAC;gBAC5B,IAAI,eAAe,KAAK,CAAC,CAAC;oBAAE,QAAQ,CAAC,gBAAgB,GAAG,eAAe,CAAC;gBACxE,OAAO,QAAQ,CAAC;YACpB,CAAC;YACD,yFAAyF;YACzF,IAAI,QAAQ,CAAC,gBAAgB,KAAK,eAAe,EAAE,CAAC;gBAChD,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,CAAC;gBACxB,OAAO,QAAQ,CAAC;YACpB,CAAC;YACD,MAAM,IAAI,SAAS,CACf,iCAAiC,GAAG,6DAA6D,QAAQ,CAAC,MAAM,QAAQ,KAAK,kCAAkC,CAClK,CAAC;QACN,CAAC;QACD,MAAM,SAAS,GAAG,IAAI,kBAAkB,CAAC,KAAK,EAAE,eAAe,EAAE,UAAU,CAAC,CAAC;QAC7E,IAAI,CAAC,UAAU,CAAC,GAAG,CAAC,GAAG,EAAE,SAAS,CAAC,CAAC;QACpC,OAAO,SAAS,CAAC;IACrB,CAAC;IAED,YAAoB,KAAa,EAAE,eAAuB,EAAE,UAAkB;QAC1E,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC;QACpB,IAAI,CAAC,gBAAgB,GAAG,eAAe,CAAC;QACxC,IAAI,CAAC,WAAW,GAAG,UAAU,CAAC;IAClC,CAAC;IAED,IAAI,KAAK;QACL,OAAO,IAAI,CAAC,MAAM,CAAC;IACvB,CAAC;IAED,IAAI,WAAW;QACX,OAAO,IAAI,CAAC,OAAO,CAAC;IACxB,CAAC;IAED,IAAI,YAAY;QACZ,OAAO,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC;IAC9B,CAAC;IAED,OAAO,CAAC,KAAa;QACjB,IAAI,CAAC,MAAM,GAAG,KAAK,CAAC;QACpB,IAAI,CAAC,WAAW,EAAE,CAAC;IACvB,CAAC;IAED,WAAW;QACP,OAAO,IAAI,CAAC,QAAQ,CAAC,IAAI,GAAG,CAAC,IAAI,CAAC,IAAI,CAAC,MAAM,KAAK,CAAC,CAAC,IAAI,IAAI,CAAC,OAAO,GAAG,IAAI,CAAC,MAAM,CAAC,EAAE,CAAC;YAClF,MAAM,IAAI,GAAG,IAAI,CAAC,QAAQ,CAAC,MAAM,EAAE,CAAC,IAAI,EAAE,CAAC,KAAM,CAAC;YAClD,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC;YAC3B,IAAI,EAAE,CAAC;QACX,CAAC;IACL,CAAC;IAED,OAAO,CAAC,MAAoB;QACxB,MAAM,EAAE,cAAc,EAAE,CAAC;QACzB,OAAO,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;YACnC,IAAI,MAAM,GAAG,KAAK,CAAC;YACnB,MAAM,KAAK,GAAG,GAAS,EAAE;gBACrB,IAAI,CAAC,QAAQ,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC;gBAC5B,MAAM,CAAC,MAAO,CAAC,MAAM,CAAC,CAAC;YAC3B,CAAC,CAAC;YACF,MAAM,KAAK,GAAG,GAAS,EAAE;gBACrB,IAAI,MAAM;oBAAE,MAAM,EAAE,mBAAmB,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC;gBACxD,IAAI,CAAC,OAAO,IAAI,CAAC,CAAC;gBAClB,IAAI,QAAQ,GAAG,KAAK,CAAC;gBACrB,OAAO,CAAC,GAAG,EAAE;oBACT,IAAI,QAAQ;wBAAE,MAAM,IAAI,KAAK,CAAC,8CAA8C,CAAC,CAAC;oBAC9E,QAAQ,GAAG,IAAI,CAAC;oBAChB,IAAI,CAAC,OAAO,IAAI,CAAC,CAAC;oBAClB,IAAI,CAAC,WAAW,EAAE,CAAC;gBACvB,CAAC,CAAC,CAAC;YACP,CAAC,CAAC;YACF,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC,CAAC,IAAI,IAAI,CAAC,OAAO,GAAG,IAAI,CAAC,MAAM;gBAAE,KAAK,EAAE,CAAC;iBACzD,CAAC;gBACF,MAAM,GAAG,IAAI,CAAC;gBACd,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;gBACzB,MAAM,EAAE,gBAAgB,CAAC,OAAO,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC;YAC7D,CAAC;QACL,CAAC,CAAC,CAAC;IACP,CAAC;CACJ","sourcesContent":["export interface InferenceAdmissionOptions {\n readonly configuredLimit?: number;\n readonly generation?: number;\n}\n\nconst assertValidLimit = (limit: number): void => {\n if (!Number.isSafeInteger(limit) || limit !== -1 && limit < 1) {\n throw new TypeError(\"PLURNK_PROVIDERS_MAX_CONCURRENCY must be -1 or a positive safe integer\");\n }\n};\n\n// {§provider-inference-admission} One process-local FIFO allowance per canonical endpoint.\nexport default class InferenceAdmission {\n static readonly #endpoints = new Map<string, InferenceAdmission>();\n static #currentGeneration = 0;\n\n #limit: number;\n #configuredLimit: number;\n #generation: number;\n readonly #waiting = new Set<() => void>();\n #active = 0;\n\n static get currentGeneration(): number {\n return this.#currentGeneration;\n }\n\n // Advance the configuration generation monotonically on operator environment reload so live\n // endpoints resize in place while conflicting aliases within the same generation still fail fast.\n static advanceGeneration(generation?: number): number {\n if (generation === undefined) {\n this.#currentGeneration += 1;\n return this.#currentGeneration;\n }\n if (!Number.isSafeInteger(generation) || generation < 0) {\n throw new TypeError(\"InferenceAdmission generation must be a non-negative safe integer\");\n }\n if (generation > this.#currentGeneration) {\n this.#currentGeneration = generation;\n }\n return this.#currentGeneration;\n }\n\n static forEndpoint(\n endpoint: string,\n limit: number,\n options?: InferenceAdmissionOptions,\n ): InferenceAdmission {\n assertValidLimit(limit);\n const configuredLimit = options?.configuredLimit ?? limit;\n assertValidLimit(configuredLimit);\n const generation = options?.generation ?? this.#currentGeneration;\n if (!Number.isSafeInteger(generation) || generation < 0) {\n throw new TypeError(\"InferenceAdmission generation must be a non-negative safe integer\");\n }\n const key = new URL(endpoint).href.replace(/\\/+$/, \"\");\n const existing = this.#endpoints.get(key);\n if (existing !== undefined) {\n if (generation > existing.#generation) {\n existing.#generation = generation;\n existing.#configuredLimit = configuredLimit;\n existing.#resize(limit);\n return existing;\n }\n if (existing.#limit === limit) {\n if (configuredLimit !== -1) existing.#configuredLimit = configuredLimit;\n return existing;\n }\n // Same operator policy on a restarted endpoint whose probed physical slot count changed.\n if (existing.#configuredLimit === configuredLimit) {\n existing.#resize(limit);\n return existing;\n }\n throw new TypeError(\n `Provider aliases resolving to ${key} have conflicting PLURNK_PROVIDERS_MAX_CONCURRENCY values ${existing.#limit} and ${limit}; one endpoint has one allowance`,\n );\n }\n const admission = new InferenceAdmission(limit, configuredLimit, generation);\n this.#endpoints.set(key, admission);\n return admission;\n }\n\n private constructor(limit: number, configuredLimit: number, generation: number) {\n this.#limit = limit;\n this.#configuredLimit = configuredLimit;\n this.#generation = generation;\n }\n\n get limit(): number {\n return this.#limit;\n }\n\n get activeCount(): number {\n return this.#active;\n }\n\n get waitingCount(): number {\n return this.#waiting.size;\n }\n\n #resize(limit: number): void {\n this.#limit = limit;\n this.#drainQueue();\n }\n\n #drainQueue(): void {\n while (this.#waiting.size > 0 && (this.#limit === -1 || this.#active < this.#limit)) {\n const next = this.#waiting.values().next().value!;\n this.#waiting.delete(next);\n next();\n }\n }\n\n acquire(signal?: AbortSignal): Promise<() => void> {\n signal?.throwIfAborted();\n return new Promise((resolve, reject) => {\n let queued = false;\n const abort = (): void => {\n this.#waiting.delete(enter);\n reject(signal!.reason);\n };\n const enter = (): void => {\n if (queued) signal?.removeEventListener(\"abort\", abort);\n this.#active += 1;\n let released = false;\n resolve(() => {\n if (released) throw new Error(\"Inference admission lease was released twice\");\n released = true;\n this.#active -= 1;\n this.#drainQueue();\n });\n };\n if (this.#limit === -1 || this.#active < this.#limit) enter();\n else {\n queued = true;\n this.#waiting.add(enter);\n signal?.addEventListener(\"abort\", abort, { once: true });\n }\n });\n }\n}\n"]}
|
package/dist/Mock.d.ts
CHANGED
|
@@ -1,10 +1,19 @@
|
|
|
1
1
|
import type { InputModality, ChatMessage, FinishReason, PromptTokenMeasurement, Provider, ProviderAssistant, ProviderCost, ProviderRequestCapacity, ProviderResponse, ProviderUsage } from "./types.ts";
|
|
2
2
|
import { type Effort } from "@plurnk/plurnk-contracts";
|
|
3
|
+
type MockToolCall = {
|
|
4
|
+
readonly index?: number;
|
|
5
|
+
readonly id?: string;
|
|
6
|
+
readonly type?: string;
|
|
7
|
+
readonly name: string;
|
|
8
|
+
readonly arguments: Record<string, unknown> | string;
|
|
9
|
+
readonly thoughtSignature?: string;
|
|
10
|
+
};
|
|
3
11
|
export type MockAssistant = {
|
|
4
12
|
content: string;
|
|
5
13
|
reasoning: string | null;
|
|
6
14
|
finishReason?: FinishReason;
|
|
7
15
|
model?: string;
|
|
16
|
+
toolCalls?: readonly MockToolCall[];
|
|
8
17
|
ops?: unknown[];
|
|
9
18
|
};
|
|
10
19
|
export type MockResponse = {
|
package/dist/Mock.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"Mock.d.ts","sourceRoot":"","sources":["../src/Mock.ts"],"names":[],"mappings":"AAQA,OAAO,KAAK,EAAE,aAAa,EAAE,WAAW,EAAE,YAAY,EAAE,sBAAsB,EAAE,QAAQ,EAAE,iBAAiB,EAAE,YAAY,EAA6B,uBAAuB,EAAE,gBAAgB,
|
|
1
|
+
{"version":3,"file":"Mock.d.ts","sourceRoot":"","sources":["../src/Mock.ts"],"names":[],"mappings":"AAQA,OAAO,KAAK,EAAE,aAAa,EAAE,WAAW,EAAE,YAAY,EAAE,sBAAsB,EAAE,QAAQ,EAAE,iBAAiB,EAAE,YAAY,EAA6B,uBAAuB,EAAE,gBAAgB,EAAoB,aAAa,EAAE,MAAM,YAAY,CAAC;AAErP,OAAO,EAAW,KAAK,MAAM,EAAE,MAAM,0BAA0B,CAAC;AAKhE,KAAK,YAAY,GAAG;IAChB,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,CAAC;IACxB,QAAQ,CAAC,EAAE,CAAC,EAAE,MAAM,CAAC;IACrB,QAAQ,CAAC,IAAI,CAAC,EAAE,MAAM,CAAC;IACvB,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAC;IACtB,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,GAAG,MAAM,CAAC;IACrD,QAAQ,CAAC,gBAAgB,CAAC,EAAE,MAAM,CAAC;CACtC,CAAC;AAEF,MAAM,MAAM,aAAa,GAAG;IACxB,OAAO,EAAE,MAAM,CAAC;IAChB,SAAS,EAAE,MAAM,GAAG,IAAI,CAAC;IACzB,YAAY,CAAC,EAAE,YAAY,CAAC;IAC5B,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,SAAS,CAAC,EAAE,SAAS,YAAY,EAAE,CAAC;IACpC,GAAG,CAAC,EAAE,OAAO,EAAE,CAAC;CACnB,CAAC;AAEF,MAAM,MAAM,YAAY,GAAG;IACvB,SAAS,EAAE,aAAa,CAAC;IACzB,YAAY,CAAC,EAAE,OAAO,CAAC;IAEvB,KAAK,CAAC,EAAE,OAAO,CAAC,aAAa,CAAC,CAAC;IAC/B,IAAI,CAAC,EAAE,YAAY,CAAC;CACvB,CAAC;AAGF,MAAM,MAAM,qBAAqB,GAAG,iBAAiB,GAAG;IAAE,GAAG,CAAC,EAAE,OAAO,EAAE,CAAA;CAAE,CAAC;AAC5E,MAAM,MAAM,oBAAoB,GAAG,gBAAgB,GAAG;IAAE,SAAS,EAAE,qBAAqB,CAAA;CAAE,CAAC;AAE3F,QAAA,MAAM,UAAU,EAAE,aAIjB,CAAC;AACF,KAAK,gBAAgB,GAAG,IAAI,CAAC,UAAU,CAAC,QAAQ,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC,EAAE,WAAW,CAAC,GAAG;IAAE,SAAS,CAAC,EAAE,MAAM,CAAA;CAAE,CAAC;AAExG,MAAM,CAAC,OAAO,OAAO,IAAK,YAAW,QAAQ;;IAIzC,QAAQ,CAAC,QAAQ,EAAE,WAAW,EAAE,EAAE,CAAM;IAcxC,YAAY,EACR,aAAa,EACb,SAAS,EACT,eAAoB,EACpB,WAAW,EACX,IAAI,GACP,EAAE;QACC,aAAa,EAAE,MAAM,GAAG,IAAI,CAAC;QAC7B,SAAS,EAAE,YAAY,EAAE,CAAC;QAC1B,eAAe,CAAC,EAAE,QAAQ,CAAC,aAAa,CAAC,CAAC;QAC1C,WAAW,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;QAC5B,IAAI,CAAC,EAAE,MAAM,CAAC;KACjB,EAUA;IAED,IAAI,aAAa,IAAI,MAAM,GAAG,IAAI,CAAgC;IAClE,IAAI,eAAe,IAAI,WAAW,CAAC,aAAa,CAAC,CAAkC;IACnF,IAAI,cAAc,IAAI,MAAM,GAAG,IAAI,CAAiB;IACpD,IAAI,eAAe,IAAI,MAAM,GAAG,IAAI,CAAiB;IACrD,IAAI,YAAY,IAAI,MAAM,GAAG,IAAI,CAA+B;IAChE,IAAI,WAAW,IAAI,MAAM,GAAG,IAAI,CAA8B;IAC9D,IAAI,eAAe,IAAI,MAAM,GAAG,IAAI,CAAkC;IACtE,IAAI,gBAAgB,IAAI,SAAS,MAAM,EAAE,CAAoB;IAC7D,IAAI,aAAa,IAAI,MAAM,GAAG,IAAI,CAMjC;IACD,IAAI,SAAS,IAAI,MAAM,GAAG,IAAI,CAM7B;IACD,IAAI,KAAK,IAAI,MAAM,CAAmB;IACtC,IAAI,WAAW,IAAI,MAAM,GAAG,IAAI,GAAG,SAAS,CAA8B;IAC1E,IAAI,IAAI,IAAI,MAAM,GAAG,SAAS,CAAuB;IAK/C,iBAAiB,CAAC,QAAQ,EAAE,SAAS,WAAW,EAAE,GAAG,OAAO,CAAC,sBAAsB,CAAC,CAMzF;IAEK,qBAAqB,CACvB,QAAQ,EAAE,SAAS,WAAW,EAAE,EAChC,eAAe,CAAC,EAAE,MAAM,GACzB,OAAO,CAAC,uBAAuB,CAAC,CAoBlC;IAEK,QAAQ,CAAC,EAAE,QAAQ,EAAE,eAAe,EAAE,MAAM,EAAE,cAAc,EAAE,gBAAgB,EAAE,EAAE,gBAAgB,GAAG,OAAO,CAAC,oBAAoB,CAAC,CA2EvI;IAED,IAAI,SAAS,IAAI,MAAM,CAA+B;CACzD;AAED,OAAO,EAAE,UAAU,IAAI,gBAAgB,EAAE,CAAC"}
|
package/dist/Mock.js
CHANGED
|
@@ -144,8 +144,20 @@ export default class Mock {
|
|
|
144
144
|
const assistant = {
|
|
145
145
|
content: a.content,
|
|
146
146
|
reasoning: a.reasoning,
|
|
147
|
-
finishReason: a.finishReason ?? "stop",
|
|
147
|
+
finishReason: a.finishReason ?? (a.toolCalls && a.toolCalls.length > 0 ? "tool_calls" : "stop"),
|
|
148
148
|
model: a.model ?? "mock",
|
|
149
|
+
...(a.toolCalls !== undefined
|
|
150
|
+
? {
|
|
151
|
+
toolCalls: a.toolCalls.map((tc, index) => ({
|
|
152
|
+
index: tc.index ?? index,
|
|
153
|
+
id: tc.id ?? `mock_call_${index}`,
|
|
154
|
+
type: tc.type ?? "function",
|
|
155
|
+
name: tc.name,
|
|
156
|
+
arguments: typeof tc.arguments === "string" ? tc.arguments : JSON.stringify(tc.arguments),
|
|
157
|
+
...(tc.thoughtSignature !== undefined ? { thoughtSignature: tc.thoughtSignature } : {}),
|
|
158
|
+
})),
|
|
159
|
+
}
|
|
160
|
+
: {}),
|
|
149
161
|
...(a.ops !== undefined ? { ops: a.ops } : {}),
|
|
150
162
|
};
|
|
151
163
|
if (assistant.reasoning !== null && assistant.reasoning.length > 0) {
|
package/dist/Mock.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"Mock.js","sourceRoot":"","sources":["../src/Mock.ts"],"names":[],"mappings":"AAAA,2DAA2D;AAC3D,EAAE;AACF,wEAAwE;AACxE,wEAAwE;AACxE,wEAAwE;AACxE,2CAA2C;AAE3C,OAAO,EAAE,eAAe,EAAE,MAAM,YAAY,CAAC;AAE7C,OAAO,EAAE,gCAAgC,EAAE,MAAM,UAAU,CAAC;AAC5D,OAAO,EAAE,OAAO,EAAe,MAAM,0BAA0B,CAAC;AAChE,OAAO,EAAE,iCAAiC,EAAE,MAAM,iBAAiB,CAAC;AACpE,OAAO,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AAC5C,OAAO,EAAE,qBAAqB,EAAE,sBAAsB,EAAE,kBAAkB,EAAE,qBAAqB,EAAE,wBAAwB,EAAE,MAAM,eAAe,CAAC;AA2BnJ,MAAM,UAAU,GAAkB;IAC9B,WAAW,EAAE,CAAC;IACd,YAAY,EAAE,CAAC;IACf,WAAW,EAAE,CAAC;CACjB,CAAC;AAGF,MAAM,CAAC,OAAO,OAAO,IAAI;IACrB,cAAc,CAAgB;IAC9B,gBAAgB,CAA6B;IAC7C,sFAAsF;IAC7E,QAAQ,GAAoB,EAAE,CAAC;IACxC,aAAa,CAAgB;IAC7B,YAAY,CAAgB;IAC5B,gBAAgB,CAAgB;IAChC,YAAY,CAA4B;IACxC,KAAK,CAAqB;IAC1B,MAAM,CAAiB;IAEvB,8DAA8D;IAC9D,yEAAyE;IACzE,sEAAsE;IACtE,uEAAuE;IACvE,iFAAiF;IACjF,wBAAwB;IACxB,YAAY,EACR,aAAa,EACb,SAAS,EACT,eAAe,GAAG,EAAE,EACpB,WAAW,EACX,IAAI,GAOP;QACG,IAAI,CAAC,cAAc,GAAG,aAAa,CAAC;QACpC,IAAI,CAAC,gBAAgB,GAAG,IAAI,GAAG,CAAC,eAAe,CAAC,CAAC;QACjD,MAAM,QAAQ,GAAG,gCAAgC,CAAC,OAAO,CAAC,GAAG,EAAE,aAAa,CAAC,CAAC;QAC9E,IAAI,CAAC,aAAa,GAAG,QAAQ,CAAC,YAAY,CAAC;QAC3C,IAAI,CAAC,YAAY,GAAG,QAAQ,CAAC,WAAW,CAAC;QACzC,IAAI,CAAC,gBAAgB,GAAG,QAAQ,CAAC,eAAe,CAAC;QACjD,IAAI,CAAC,YAAY,GAAG,WAAW,CAAC;QAChC,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC;QAClB,IAAI,CAAC,MAAM,GAAG,CAAC,GAAG,SAAS,CAAC,CAAC;IACjC,CAAC;IAED,IAAI,aAAa,KAAoB,OAAO,IAAI,CAAC,cAAc,CAAC,CAAC,CAAC;IAClE,IAAI,eAAe,KAAiC,OAAO,IAAI,CAAC,gBAAgB,CAAC,CAAC,CAAC;IACnF,IAAI,cAAc,KAAoB,OAAO,IAAI,CAAC,CAAC,CAAC;IACpD,IAAI,eAAe,KAAoB,OAAO,IAAI,CAAC,CAAC,CAAC;IACrD,IAAI,YAAY,KAAoB,OAAO,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC;IAChE,IAAI,WAAW,KAAoB,OAAO,IAAI,CAAC,YAAY,CAAC,CAAC,CAAC;IAC9D,IAAI,eAAe,KAAoB,OAAO,IAAI,CAAC,gBAAgB,CAAC,CAAC,CAAC;IACtE,IAAI,gBAAgB,KAAwB,OAAO,OAAO,CAAC,CAAC,CAAC;IAC7D,IAAI,aAAa;QACb,OAAO,sBAAsB,CAAC;YAC1B,aAAa,EAAE,IAAI,CAAC,cAAc;YAClC,cAAc,EAAE,IAAI,CAAC,cAAc;YACnC,YAAY,EAAE,IAAI,CAAC,aAAa;SACnC,CAAC,CAAC;IACP,CAAC;IACD,IAAI,SAAS;QACT,OAAO,kBAAkB,CAAC;YACtB,aAAa,EAAE,IAAI,CAAC,cAAc;YAClC,cAAc,EAAE,IAAI,CAAC,cAAc;YACnC,WAAW,EAAE,IAAI,CAAC,YAAY;SACjC,CAAC,CAAC;IACP,CAAC;IACD,IAAI,KAAK,KAAa,OAAO,MAAM,CAAC,CAAC,CAAC;IACtC,IAAI,WAAW,KAAgC,OAAO,IAAI,CAAC,YAAY,CAAC,CAAC,CAAC;IAC1E,IAAI,IAAI,KAAyB,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC;IAErD,4EAA4E;IAC5E,8EAA8E;IAC9E,gFAAgF;IAChF,KAAK,CAAC,iBAAiB,CAAC,QAAgC;QACpD,OAAO;YACH,IAAI,EAAE,OAAO;YACb,MAAM,EAAE,QAAQ,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,OAAO,EAAE,EAAE,CAAC,GAAG,GAAG,IAAI,CAAC,IAAI,CAAC,eAAe,CAAC,OAAO,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC;YAClG,MAAM,EAAE,aAAa;SACxB,CAAC;IACN,CAAC;IAED,KAAK,CAAC,qBAAqB,CACvB,QAAgC,EAChC,eAAwB;QAExB,MAAM,YAAY,GAAG,qBAAqB,CAAC;YACvC,SAAS,EAAE,eAAe;YAC1B,UAAU,EAAE,IAAI,CAAC,aAAa;YAC9B,eAAe,EAAE,IAAI;YACrB,aAAa,EAAE,IAAI,CAAC,cAAc;SACrC,CAAC,CAAC;QACH,MAAM,eAAe,GAAG,wBAAwB,CAAC;YAC7C,UAAU,EAAE,IAAI,CAAC,gBAAgB;YACjC,YAAY;SACf,CAAC,CAAC;QACH,OAAO,qBAAqB,CAAC;YACzB,aAAa,EAAE,IAAI,CAAC,cAAc;YAClC,cAAc,EAAE,IAAI;YACpB,eAAe,EAAE,IAAI;YACrB,YAAY;YACZ,WAAW,EAAE,IAAI,CAAC,YAAY;YAC9B,eAAe;YACf,WAAW,EAAE,MAAM,IAAI,CAAC,iBAAiB,CAAC,QAAQ,CAAC;SACtD,CAAC,CAAC;IACP,CAAC;IAED,KAAK,CAAC,QAAQ,CAAC,EAAE,QAAQ,EAAE,eAAe,EAAE,MAAM,EAAE,cAAc,EAAE,gBAAgB,EAAoB;QACpG,oEAAoE;QACpE,qDAAqD;QACrD,uCAAuC;QACvC,MAAM,EAAE,cAAc,EAAE,CAAC;QACzB,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,EAAE,GAAG,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC;QAChE,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,qBAAqB,CAAC,QAAQ,EAAE,eAAe,CAAC,CAAC;QAC7E,IAAI,QAAQ,CAAC,QAAQ,KAAK,QAAQ,EAAE,CAAC;YACjC,MAAM,IAAI,aAAa,CAAC,MAAM,EAAE,mBAAmB,EAAE,sCAAsC,EAAE;gBACzF,QAAQ;gBACR,UAAU,EAAE,EAAE,aAAa,EAAE,WAAW,EAAE,QAAQ,EAAE;aACvD,CAAC,CAAC;QACP,CAAC;QACD,MAAM,MAAM,GAAG,MAAM,cAAc,EAAE,CAAC,EAAE,QAAQ,EAAE,eAAe,EAAE,KAAK,EAAE,IAAI,CAAC,KAAK,EAAE,CAAC,CAAC;QACxF,MAAM,IAAI,GAAG,IAAI,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC;QACjC,IAAI,IAAI,KAAK,SAAS,EAAE,CAAC;YACrB,MAAM,UAAU,GAAG,iCAAiC,CAAC;gBACjD,QAAQ,EAAE,eAAe;gBACzB,KAAK,EAAE,IAAI,CAAC,KAAK;gBACjB,OAAO,EAAE,OAAO;gBAChB,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,MAAM,EAAE,qDAAqD,EAAE;aAC3F,CAAC,CAAC;YACH,MAAM,MAAM,EAAE,CAAC,UAAU,CAAC,CAAC;YAC3B,MAAM,IAAI,aAAa,CACnB,MAAM,EACN,kBAAkB,EAClB,mDAAmD,EACnD,EAAE,UAAU,EAAE,CAAC,UAAU,CAAC,EAAE,QAAQ,EAAE,CACzC,CAAC;QACN,CAAC;QACD,MAAM,CAAC,GAAG,IAAI,CAAC,SAAS,CAAC;QACzB,MAAM,KAAK,GAAkB;YACzB,GAAG,UAAU;YACb,GAAG,IAAI,CAAC,KAAK;SAChB,CAAC;QACF,MAAM,iBAAiB,GAA8B,iCAAiC,CAAC;YACnF,QAAQ,EAAE,eAAe;YACzB,KAAK,EAAE,CAAC,CAAC,KAAK,IAAI,IAAI,CAAC,KAAK;YAC5B,OAAO,EAAE,UAAU;YACnB,KAAK;YACL,IAAI,EAAE,IAAI,CAAC,IAAI,IAAI;gBACf,IAAI,EAAE,WAAW;gBACjB,MAAM,EAAE,EAAE,MAAM,EAAE,GAAG,EAAE,QAAQ,EAAE,KAAK,EAAE;gBACxC,MAAM,EAAE,uBAAuB;aAClC;SACJ,CAAC,CAAC;QACH,MAAM,MAAM,EAAE,CAAC,iBAAiB,CAAC,CAAC;QAClC,MAAM,SAAS,GAA0B;YACrC,OAAO,EAAE,CAAC,CAAC,OAAO;YAClB,SAAS,EAAE,CAAC,CAAC,SAAS;YACtB,YAAY,EAAE,CAAC,CAAC,YAAY,IAAI,MAAM;YACtC,KAAK,EAAE,CAAC,CAAC,KAAK,IAAI,MAAM;YACxB,GAAG,CAAC,CAAC,CAAC,GAAG,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;SACjD,CAAC;QACF,IAAI,SAAS,CAAC,SAAS,KAAK,IAAI,IAAI,SAAS,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YACjE,gBAAgB,EAAE,CAAC,SAAS,CAAC,SAAS,CAAC,CAAC;QAC5C,CAAC;QACD,OAAO;YACH,SAAS;YACT,YAAY,EAAE,IAAI,CAAC,YAAY,IAAI,IAAI;YACvC,UAAU,EAAE,CAAC,iBAAiB,CAAC;YAC/B,QAAQ;SACX,CAAC;IACN,CAAC;IAED,IAAI,SAAS,KAAa,OAAO,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;CACzD;AAED,OAAO,EAAE,UAAU,IAAI,gBAAgB,EAAE,CAAC","sourcesContent":["// Mock provider — reference implementation + test fixture.\n//\n// Dual purpose: (a) plurnk-service intg suite uses it for deterministic\n// engine tests; (b) worked example for sibling authors implementing the\n// Provider contract. Production providers don't expose the `ops` escape\n// hatch — that's an intg-only convenience.\n\nimport { chatMessageText } from \"./types.ts\";\nimport type { InputModality, ChatMessage, FinishReason, PromptTokenMeasurement, Provider, ProviderAssistant, ProviderCost, ProviderRequestAccounting, ProviderRequestCapacity, ProviderResponse, ProviderUsage } from \"./types.ts\";\nimport { resolveGenerationEnvelopeFromEnv } from \"./env.ts\";\nimport { EFFORTS, type Effort } from \"@plurnk/plurnk-contracts\";\nimport { validateProviderRequestAccounting } from \"./accounting.ts\";\nimport { ProviderError } from \"./errors.ts\";\nimport { assessRequestCapacity, effectiveInputCapacity, effectiveInputWall, effectiveOutputBudget, effectiveReasoningBudget } from \"./capacity.ts\";\n\nexport type MockAssistant = {\n content: string;\n reasoning: string | null;\n finishReason?: FinishReason;\n model?: string;\n // Provider-normalized encrypted reasoning fixture.\n // Pre-parsed ops — intg-only escape hatch. Typed `unknown[]` so the\n // framework carries no parser dependency; plurnk-service\n // proves each against the statement schema on its side. Production providers never\n // include this field.\n ops?: unknown[];\n};\n\nexport type MockResponse = {\n assistant: MockAssistant;\n assistantRaw?: unknown;\n // Partial — omitted fields fall back to the deliberate zero fixture.\n usage?: Partial<ProviderUsage>;\n cost?: ProviderCost;\n};\n\n// Returned shape: ProviderAssistant + pre-parsed ops visible for tests.\nexport type MockReturnedAssistant = ProviderAssistant & { ops?: unknown[] };\nexport type MockReturnedResponse = ProviderResponse & { assistant: MockReturnedAssistant };\n\nconst MOCK_USAGE: ProviderUsage = {\n inputTokens: 0,\n outputTokens: 0,\n totalTokens: 0,\n};\ntype MockGenerateArgs = Omit<Parameters<Provider[\"generate\"]>[0], \"sessionId\"> & { sessionId?: string };\n\nexport default class Mock implements Provider {\n #contextWindow: number | null;\n #inputModalities: ReadonlySet<InputModality>;\n // Every request as received, newest last — the witness for what reached the provider.\n readonly received: ChatMessage[][] = [];\n #outputBudget: number | null;\n #outputFloor: number | null;\n #reasoningBudget: number | null;\n #temperature: number | null | undefined;\n #seed: number | undefined;\n #queue: MockResponse[];\n\n // {§provider-configuration} Mock resolves the same generation\n // envelope as a real provider, but tolerates absent policy because it is\n // also the universal test fixture. No output budget means its context\n // window alone cannot determine an input capacity; no output floor, no\n // input wall ({§provider-capacity-admission}). Mock has no alias identity, so it\n // reads the bare knobs.\n constructor({\n contextWindow,\n responses,\n inputModalities = [],\n temperature,\n seed,\n }: {\n contextWindow: number | null;\n responses: MockResponse[];\n inputModalities?: Iterable<InputModality>;\n temperature?: number | null;\n seed?: number;\n }) {\n this.#contextWindow = contextWindow;\n this.#inputModalities = new Set(inputModalities);\n const envelope = resolveGenerationEnvelopeFromEnv(process.env, contextWindow);\n this.#outputBudget = envelope.outputBudget;\n this.#outputFloor = envelope.outputFloor;\n this.#reasoningBudget = envelope.reasoningBudget;\n this.#temperature = temperature;\n this.#seed = seed;\n this.#queue = [...responses];\n }\n\n get contextWindow(): number | null { return this.#contextWindow; }\n get inputModalities(): ReadonlySet<InputModality> { return this.#inputModalities; }\n get maxInputTokens(): number | null { return null; }\n get maxOutputTokens(): number | null { return null; }\n get outputBudget(): number | null { return this.#outputBudget; }\n get outputFloor(): number | null { return this.#outputFloor; }\n get reasoningBudget(): number | null { return this.#reasoningBudget; }\n get supportedEfforts(): readonly Effort[] { return EFFORTS; }\n get inputCapacity(): number | null {\n return effectiveInputCapacity({\n contextWindow: this.#contextWindow,\n maxInputTokens: this.maxInputTokens,\n outputBudget: this.#outputBudget,\n });\n }\n get inputWall(): number | null {\n return effectiveInputWall({\n contextWindow: this.#contextWindow,\n maxInputTokens: this.maxInputTokens,\n outputFloor: this.#outputFloor,\n });\n }\n get model(): string { return \"mock\"; }\n get temperature(): number | null | undefined { return this.#temperature; }\n get seed(): number | undefined { return this.#seed; }\n\n // Mock's deliberately simple vocabulary defines each two content code units\n // as one token and has no hidden request framing. This is exact for the mock,\n // unlike a production adapter applying the same arithmetic to an unknown model.\n async countPromptTokens(messages: readonly ChatMessage[]): Promise<PromptTokenMeasurement> {\n return {\n kind: \"exact\",\n tokens: messages.reduce((sum, message) => sum + Math.ceil(chatMessageText(message).length / 2), 0),\n source: \"mock:chars2\",\n };\n }\n\n async assessRequestCapacity(\n messages: readonly ChatMessage[],\n maxOutputTokens?: number,\n ): Promise<ProviderRequestCapacity> {\n const outputBudget = effectiveOutputBudget({\n requested: maxOutputTokens,\n configured: this.#outputBudget,\n maxOutputTokens: null,\n contextWindow: this.#contextWindow,\n });\n const reasoningBudget = effectiveReasoningBudget({\n configured: this.#reasoningBudget,\n outputBudget,\n });\n return assessRequestCapacity({\n contextWindow: this.#contextWindow,\n maxInputTokens: null,\n maxOutputTokens: null,\n outputBudget,\n outputFloor: this.#outputFloor,\n reasoningBudget,\n measurement: await this.countPromptTokens(messages),\n });\n }\n\n async generate({ messages, maxOutputTokens, signal, observeRequest, observeReasoning }: MockGenerateArgs): Promise<MockReturnedResponse> {\n // Honor abort before consuming the queue — an aborted call makes no\n // \"wire call\" and must not exhaust a queued response\n // ({§provider-failure-normalization}).\n signal?.throwIfAborted();\n this.received.push(messages.map((message) => ({ ...message })));\n const capacity = await this.assessRequestCapacity(messages, maxOutputTokens);\n if (capacity.decision === \"reject\") {\n throw new ProviderError(\"mock\", \"capacity_exceeded\", \"Mock request exceeds its input wall.\", {\n capacity,\n extensions: { capacityStage: \"preflight\", capacity },\n });\n }\n const settle = await observeRequest?.({ provider: \"provider:mock\", model: this.model });\n const next = this.#queue.shift();\n if (next === undefined) {\n const accounting = validateProviderRequestAccounting({\n provider: \"provider:mock\",\n model: this.model,\n outcome: \"error\",\n cost: { kind: \"unknown\", reason: \"mock provider exhausted before producing a response\" },\n });\n await settle?.(accounting);\n throw new ProviderError(\n \"mock\",\n \"invalid_response\",\n \"Mock provider exhausted: no more queued responses\",\n { accounting: [accounting], capacity },\n );\n }\n const a = next.assistant;\n const usage: ProviderUsage = {\n ...MOCK_USAGE,\n ...next.usage,\n };\n const requestAccounting: ProviderRequestAccounting = validateProviderRequestAccounting({\n provider: \"provider:mock\",\n model: a.model ?? this.model,\n outcome: \"response\",\n usage,\n cost: next.cost ?? {\n kind: \"estimated\",\n amount: { amount: \"0\", currency: \"USD\" },\n source: \"mock provider fixture\",\n },\n });\n await settle?.(requestAccounting);\n const assistant: MockReturnedAssistant = {\n content: a.content,\n reasoning: a.reasoning,\n finishReason: a.finishReason ?? \"stop\",\n model: a.model ?? \"mock\",\n ...(a.ops !== undefined ? { ops: a.ops } : {}),\n };\n if (assistant.reasoning !== null && assistant.reasoning.length > 0) {\n observeReasoning?.(assistant.reasoning);\n }\n return {\n assistant,\n assistantRaw: next.assistantRaw ?? null,\n accounting: [requestAccounting],\n capacity,\n };\n }\n\n get remaining(): number { return this.#queue.length; }\n}\n\nexport { MOCK_USAGE as mockDefaultUsage };\n"]}
|
|
1
|
+
{"version":3,"file":"Mock.js","sourceRoot":"","sources":["../src/Mock.ts"],"names":[],"mappings":"AAAA,2DAA2D;AAC3D,EAAE;AACF,wEAAwE;AACxE,wEAAwE;AACxE,wEAAwE;AACxE,2CAA2C;AAE3C,OAAO,EAAE,eAAe,EAAE,MAAM,YAAY,CAAC;AAE7C,OAAO,EAAE,gCAAgC,EAAE,MAAM,UAAU,CAAC;AAC5D,OAAO,EAAE,OAAO,EAAe,MAAM,0BAA0B,CAAC;AAChE,OAAO,EAAE,iCAAiC,EAAE,MAAM,iBAAiB,CAAC;AACpE,OAAO,EAAE,aAAa,EAAE,MAAM,aAAa,CAAC;AAC5C,OAAO,EAAE,qBAAqB,EAAE,sBAAsB,EAAE,kBAAkB,EAAE,qBAAqB,EAAE,wBAAwB,EAAE,MAAM,eAAe,CAAC;AAgCnJ,MAAM,UAAU,GAAkB;IAC9B,WAAW,EAAE,CAAC;IACd,YAAY,EAAE,CAAC;IACf,WAAW,EAAE,CAAC;CACjB,CAAC;AAGF,MAAM,CAAC,OAAO,OAAO,IAAI;IACrB,cAAc,CAAgB;IAC9B,gBAAgB,CAA6B;IAC7C,sFAAsF;IAC7E,QAAQ,GAAoB,EAAE,CAAC;IACxC,aAAa,CAAgB;IAC7B,YAAY,CAAgB;IAC5B,gBAAgB,CAAgB;IAChC,YAAY,CAA4B;IACxC,KAAK,CAAqB;IAC1B,MAAM,CAAiB;IAEvB,8DAA8D;IAC9D,yEAAyE;IACzE,sEAAsE;IACtE,uEAAuE;IACvE,iFAAiF;IACjF,wBAAwB;IACxB,YAAY,EACR,aAAa,EACb,SAAS,EACT,eAAe,GAAG,EAAE,EACpB,WAAW,EACX,IAAI,GAOP;QACG,IAAI,CAAC,cAAc,GAAG,aAAa,CAAC;QACpC,IAAI,CAAC,gBAAgB,GAAG,IAAI,GAAG,CAAC,eAAe,CAAC,CAAC;QACjD,MAAM,QAAQ,GAAG,gCAAgC,CAAC,OAAO,CAAC,GAAG,EAAE,aAAa,CAAC,CAAC;QAC9E,IAAI,CAAC,aAAa,GAAG,QAAQ,CAAC,YAAY,CAAC;QAC3C,IAAI,CAAC,YAAY,GAAG,QAAQ,CAAC,WAAW,CAAC;QACzC,IAAI,CAAC,gBAAgB,GAAG,QAAQ,CAAC,eAAe,CAAC;QACjD,IAAI,CAAC,YAAY,GAAG,WAAW,CAAC;QAChC,IAAI,CAAC,KAAK,GAAG,IAAI,CAAC;QAClB,IAAI,CAAC,MAAM,GAAG,CAAC,GAAG,SAAS,CAAC,CAAC;IACjC,CAAC;IAED,IAAI,aAAa,KAAoB,OAAO,IAAI,CAAC,cAAc,CAAC,CAAC,CAAC;IAClE,IAAI,eAAe,KAAiC,OAAO,IAAI,CAAC,gBAAgB,CAAC,CAAC,CAAC;IACnF,IAAI,cAAc,KAAoB,OAAO,IAAI,CAAC,CAAC,CAAC;IACpD,IAAI,eAAe,KAAoB,OAAO,IAAI,CAAC,CAAC,CAAC;IACrD,IAAI,YAAY,KAAoB,OAAO,IAAI,CAAC,aAAa,CAAC,CAAC,CAAC;IAChE,IAAI,WAAW,KAAoB,OAAO,IAAI,CAAC,YAAY,CAAC,CAAC,CAAC;IAC9D,IAAI,eAAe,KAAoB,OAAO,IAAI,CAAC,gBAAgB,CAAC,CAAC,CAAC;IACtE,IAAI,gBAAgB,KAAwB,OAAO,OAAO,CAAC,CAAC,CAAC;IAC7D,IAAI,aAAa;QACb,OAAO,sBAAsB,CAAC;YAC1B,aAAa,EAAE,IAAI,CAAC,cAAc;YAClC,cAAc,EAAE,IAAI,CAAC,cAAc;YACnC,YAAY,EAAE,IAAI,CAAC,aAAa;SACnC,CAAC,CAAC;IACP,CAAC;IACD,IAAI,SAAS;QACT,OAAO,kBAAkB,CAAC;YACtB,aAAa,EAAE,IAAI,CAAC,cAAc;YAClC,cAAc,EAAE,IAAI,CAAC,cAAc;YACnC,WAAW,EAAE,IAAI,CAAC,YAAY;SACjC,CAAC,CAAC;IACP,CAAC;IACD,IAAI,KAAK,KAAa,OAAO,MAAM,CAAC,CAAC,CAAC;IACtC,IAAI,WAAW,KAAgC,OAAO,IAAI,CAAC,YAAY,CAAC,CAAC,CAAC;IAC1E,IAAI,IAAI,KAAyB,OAAO,IAAI,CAAC,KAAK,CAAC,CAAC,CAAC;IAErD,4EAA4E;IAC5E,8EAA8E;IAC9E,gFAAgF;IAChF,KAAK,CAAC,iBAAiB,CAAC,QAAgC;QACpD,OAAO;YACH,IAAI,EAAE,OAAO;YACb,MAAM,EAAE,QAAQ,CAAC,MAAM,CAAC,CAAC,GAAG,EAAE,OAAO,EAAE,EAAE,CAAC,GAAG,GAAG,IAAI,CAAC,IAAI,CAAC,eAAe,CAAC,OAAO,CAAC,CAAC,MAAM,GAAG,CAAC,CAAC,EAAE,CAAC,CAAC;YAClG,MAAM,EAAE,aAAa;SACxB,CAAC;IACN,CAAC;IAED,KAAK,CAAC,qBAAqB,CACvB,QAAgC,EAChC,eAAwB;QAExB,MAAM,YAAY,GAAG,qBAAqB,CAAC;YACvC,SAAS,EAAE,eAAe;YAC1B,UAAU,EAAE,IAAI,CAAC,aAAa;YAC9B,eAAe,EAAE,IAAI;YACrB,aAAa,EAAE,IAAI,CAAC,cAAc;SACrC,CAAC,CAAC;QACH,MAAM,eAAe,GAAG,wBAAwB,CAAC;YAC7C,UAAU,EAAE,IAAI,CAAC,gBAAgB;YACjC,YAAY;SACf,CAAC,CAAC;QACH,OAAO,qBAAqB,CAAC;YACzB,aAAa,EAAE,IAAI,CAAC,cAAc;YAClC,cAAc,EAAE,IAAI;YACpB,eAAe,EAAE,IAAI;YACrB,YAAY;YACZ,WAAW,EAAE,IAAI,CAAC,YAAY;YAC9B,eAAe;YACf,WAAW,EAAE,MAAM,IAAI,CAAC,iBAAiB,CAAC,QAAQ,CAAC;SACtD,CAAC,CAAC;IACP,CAAC;IAED,KAAK,CAAC,QAAQ,CAAC,EAAE,QAAQ,EAAE,eAAe,EAAE,MAAM,EAAE,cAAc,EAAE,gBAAgB,EAAoB;QACpG,oEAAoE;QACpE,qDAAqD;QACrD,uCAAuC;QACvC,MAAM,EAAE,cAAc,EAAE,CAAC;QACzB,IAAI,CAAC,QAAQ,CAAC,IAAI,CAAC,QAAQ,CAAC,GAAG,CAAC,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC,EAAE,GAAG,OAAO,EAAE,CAAC,CAAC,CAAC,CAAC;QAChE,MAAM,QAAQ,GAAG,MAAM,IAAI,CAAC,qBAAqB,CAAC,QAAQ,EAAE,eAAe,CAAC,CAAC;QAC7E,IAAI,QAAQ,CAAC,QAAQ,KAAK,QAAQ,EAAE,CAAC;YACjC,MAAM,IAAI,aAAa,CAAC,MAAM,EAAE,mBAAmB,EAAE,sCAAsC,EAAE;gBACzF,QAAQ;gBACR,UAAU,EAAE,EAAE,aAAa,EAAE,WAAW,EAAE,QAAQ,EAAE;aACvD,CAAC,CAAC;QACP,CAAC;QACD,MAAM,MAAM,GAAG,MAAM,cAAc,EAAE,CAAC,EAAE,QAAQ,EAAE,eAAe,EAAE,KAAK,EAAE,IAAI,CAAC,KAAK,EAAE,CAAC,CAAC;QACxF,MAAM,IAAI,GAAG,IAAI,CAAC,MAAM,CAAC,KAAK,EAAE,CAAC;QACjC,IAAI,IAAI,KAAK,SAAS,EAAE,CAAC;YACrB,MAAM,UAAU,GAAG,iCAAiC,CAAC;gBACjD,QAAQ,EAAE,eAAe;gBACzB,KAAK,EAAE,IAAI,CAAC,KAAK;gBACjB,OAAO,EAAE,OAAO;gBAChB,IAAI,EAAE,EAAE,IAAI,EAAE,SAAS,EAAE,MAAM,EAAE,qDAAqD,EAAE;aAC3F,CAAC,CAAC;YACH,MAAM,MAAM,EAAE,CAAC,UAAU,CAAC,CAAC;YAC3B,MAAM,IAAI,aAAa,CACnB,MAAM,EACN,kBAAkB,EAClB,mDAAmD,EACnD,EAAE,UAAU,EAAE,CAAC,UAAU,CAAC,EAAE,QAAQ,EAAE,CACzC,CAAC;QACN,CAAC;QACD,MAAM,CAAC,GAAG,IAAI,CAAC,SAAS,CAAC;QACzB,MAAM,KAAK,GAAkB;YACzB,GAAG,UAAU;YACb,GAAG,IAAI,CAAC,KAAK;SAChB,CAAC;QACF,MAAM,iBAAiB,GAA8B,iCAAiC,CAAC;YACnF,QAAQ,EAAE,eAAe;YACzB,KAAK,EAAE,CAAC,CAAC,KAAK,IAAI,IAAI,CAAC,KAAK;YAC5B,OAAO,EAAE,UAAU;YACnB,KAAK;YACL,IAAI,EAAE,IAAI,CAAC,IAAI,IAAI;gBACf,IAAI,EAAE,WAAW;gBACjB,MAAM,EAAE,EAAE,MAAM,EAAE,GAAG,EAAE,QAAQ,EAAE,KAAK,EAAE;gBACxC,MAAM,EAAE,uBAAuB;aAClC;SACJ,CAAC,CAAC;QACH,MAAM,MAAM,EAAE,CAAC,iBAAiB,CAAC,CAAC;QAClC,MAAM,SAAS,GAA0B;YACrC,OAAO,EAAE,CAAC,CAAC,OAAO;YAClB,SAAS,EAAE,CAAC,CAAC,SAAS;YACtB,YAAY,EAAE,CAAC,CAAC,YAAY,IAAI,CAAC,CAAC,CAAC,SAAS,IAAI,CAAC,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,YAAY,CAAC,CAAC,CAAC,MAAM,CAAC;YAC/F,KAAK,EAAE,CAAC,CAAC,KAAK,IAAI,MAAM;YACxB,GAAG,CAAC,CAAC,CAAC,SAAS,KAAK,SAAS;gBACzB,CAAC,CAAC;oBACE,SAAS,EAAE,CAAC,CAAC,SAAS,CAAC,GAAG,CAAC,CAAC,EAAE,EAAE,KAAK,EAAoB,EAAE,CAAC,CAAC;wBACzD,KAAK,EAAE,EAAE,CAAC,KAAK,IAAI,KAAK;wBACxB,EAAE,EAAE,EAAE,CAAC,EAAE,IAAI,aAAa,KAAK,EAAE;wBACjC,IAAI,EAAE,EAAE,CAAC,IAAI,IAAI,UAAU;wBAC3B,IAAI,EAAE,EAAE,CAAC,IAAI;wBACb,SAAS,EAAE,OAAO,EAAE,CAAC,SAAS,KAAK,QAAQ,CAAC,CAAC,CAAC,EAAE,CAAC,SAAS,CAAC,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,CAAC,SAAS,CAAC;wBACzF,GAAG,CAAC,EAAE,CAAC,gBAAgB,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,gBAAgB,EAAE,EAAE,CAAC,gBAAgB,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;qBAC1F,CAAC,CAAC;iBACN;gBACD,CAAC,CAAC,EAAE,CAAC;YACT,GAAG,CAAC,CAAC,CAAC,GAAG,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,GAAG,EAAE,CAAC,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;SACjD,CAAC;QACF,IAAI,SAAS,CAAC,SAAS,KAAK,IAAI,IAAI,SAAS,CAAC,SAAS,CAAC,MAAM,GAAG,CAAC,EAAE,CAAC;YACjE,gBAAgB,EAAE,CAAC,SAAS,CAAC,SAAS,CAAC,CAAC;QAC5C,CAAC;QACD,OAAO;YACH,SAAS;YACT,YAAY,EAAE,IAAI,CAAC,YAAY,IAAI,IAAI;YACvC,UAAU,EAAE,CAAC,iBAAiB,CAAC;YAC/B,QAAQ;SACX,CAAC;IACN,CAAC;IAED,IAAI,SAAS,KAAa,OAAO,IAAI,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC;CACzD;AAED,OAAO,EAAE,UAAU,IAAI,gBAAgB,EAAE,CAAC","sourcesContent":["// Mock provider — reference implementation + test fixture.\n//\n// Dual purpose: (a) plurnk-service intg suite uses it for deterministic\n// engine tests; (b) worked example for sibling authors implementing the\n// Provider contract. Production providers don't expose the `ops` escape\n// hatch — that's an intg-only convenience.\n\nimport { chatMessageText } from \"./types.ts\";\nimport type { InputModality, ChatMessage, FinishReason, PromptTokenMeasurement, Provider, ProviderAssistant, ProviderCost, ProviderRequestAccounting, ProviderRequestCapacity, ProviderResponse, ProviderToolCall, ProviderUsage } from \"./types.ts\";\nimport { resolveGenerationEnvelopeFromEnv } from \"./env.ts\";\nimport { EFFORTS, type Effort } from \"@plurnk/plurnk-contracts\";\nimport { validateProviderRequestAccounting } from \"./accounting.ts\";\nimport { ProviderError } from \"./errors.ts\";\nimport { assessRequestCapacity, effectiveInputCapacity, effectiveInputWall, effectiveOutputBudget, effectiveReasoningBudget } from \"./capacity.ts\";\n\ntype MockToolCall = {\n readonly index?: number;\n readonly id?: string;\n readonly type?: string;\n readonly name: string;\n readonly arguments: Record<string, unknown> | string;\n readonly thoughtSignature?: string;\n};\n\nexport type MockAssistant = {\n content: string;\n reasoning: string | null;\n finishReason?: FinishReason;\n model?: string;\n toolCalls?: readonly MockToolCall[];\n ops?: unknown[];\n};\n\nexport type MockResponse = {\n assistant: MockAssistant;\n assistantRaw?: unknown;\n // Partial — omitted fields fall back to the deliberate zero fixture.\n usage?: Partial<ProviderUsage>;\n cost?: ProviderCost;\n};\n\n// Returned shape: ProviderAssistant + pre-parsed ops visible for tests.\nexport type MockReturnedAssistant = ProviderAssistant & { ops?: unknown[] };\nexport type MockReturnedResponse = ProviderResponse & { assistant: MockReturnedAssistant };\n\nconst MOCK_USAGE: ProviderUsage = {\n inputTokens: 0,\n outputTokens: 0,\n totalTokens: 0,\n};\ntype MockGenerateArgs = Omit<Parameters<Provider[\"generate\"]>[0], \"sessionId\"> & { sessionId?: string };\n\nexport default class Mock implements Provider {\n #contextWindow: number | null;\n #inputModalities: ReadonlySet<InputModality>;\n // Every request as received, newest last — the witness for what reached the provider.\n readonly received: ChatMessage[][] = [];\n #outputBudget: number | null;\n #outputFloor: number | null;\n #reasoningBudget: number | null;\n #temperature: number | null | undefined;\n #seed: number | undefined;\n #queue: MockResponse[];\n\n // {§provider-configuration} Mock resolves the same generation\n // envelope as a real provider, but tolerates absent policy because it is\n // also the universal test fixture. No output budget means its context\n // window alone cannot determine an input capacity; no output floor, no\n // input wall ({§provider-capacity-admission}). Mock has no alias identity, so it\n // reads the bare knobs.\n constructor({\n contextWindow,\n responses,\n inputModalities = [],\n temperature,\n seed,\n }: {\n contextWindow: number | null;\n responses: MockResponse[];\n inputModalities?: Iterable<InputModality>;\n temperature?: number | null;\n seed?: number;\n }) {\n this.#contextWindow = contextWindow;\n this.#inputModalities = new Set(inputModalities);\n const envelope = resolveGenerationEnvelopeFromEnv(process.env, contextWindow);\n this.#outputBudget = envelope.outputBudget;\n this.#outputFloor = envelope.outputFloor;\n this.#reasoningBudget = envelope.reasoningBudget;\n this.#temperature = temperature;\n this.#seed = seed;\n this.#queue = [...responses];\n }\n\n get contextWindow(): number | null { return this.#contextWindow; }\n get inputModalities(): ReadonlySet<InputModality> { return this.#inputModalities; }\n get maxInputTokens(): number | null { return null; }\n get maxOutputTokens(): number | null { return null; }\n get outputBudget(): number | null { return this.#outputBudget; }\n get outputFloor(): number | null { return this.#outputFloor; }\n get reasoningBudget(): number | null { return this.#reasoningBudget; }\n get supportedEfforts(): readonly Effort[] { return EFFORTS; }\n get inputCapacity(): number | null {\n return effectiveInputCapacity({\n contextWindow: this.#contextWindow,\n maxInputTokens: this.maxInputTokens,\n outputBudget: this.#outputBudget,\n });\n }\n get inputWall(): number | null {\n return effectiveInputWall({\n contextWindow: this.#contextWindow,\n maxInputTokens: this.maxInputTokens,\n outputFloor: this.#outputFloor,\n });\n }\n get model(): string { return \"mock\"; }\n get temperature(): number | null | undefined { return this.#temperature; }\n get seed(): number | undefined { return this.#seed; }\n\n // Mock's deliberately simple vocabulary defines each two content code units\n // as one token and has no hidden request framing. This is exact for the mock,\n // unlike a production adapter applying the same arithmetic to an unknown model.\n async countPromptTokens(messages: readonly ChatMessage[]): Promise<PromptTokenMeasurement> {\n return {\n kind: \"exact\",\n tokens: messages.reduce((sum, message) => sum + Math.ceil(chatMessageText(message).length / 2), 0),\n source: \"mock:chars2\",\n };\n }\n\n async assessRequestCapacity(\n messages: readonly ChatMessage[],\n maxOutputTokens?: number,\n ): Promise<ProviderRequestCapacity> {\n const outputBudget = effectiveOutputBudget({\n requested: maxOutputTokens,\n configured: this.#outputBudget,\n maxOutputTokens: null,\n contextWindow: this.#contextWindow,\n });\n const reasoningBudget = effectiveReasoningBudget({\n configured: this.#reasoningBudget,\n outputBudget,\n });\n return assessRequestCapacity({\n contextWindow: this.#contextWindow,\n maxInputTokens: null,\n maxOutputTokens: null,\n outputBudget,\n outputFloor: this.#outputFloor,\n reasoningBudget,\n measurement: await this.countPromptTokens(messages),\n });\n }\n\n async generate({ messages, maxOutputTokens, signal, observeRequest, observeReasoning }: MockGenerateArgs): Promise<MockReturnedResponse> {\n // Honor abort before consuming the queue — an aborted call makes no\n // \"wire call\" and must not exhaust a queued response\n // ({§provider-failure-normalization}).\n signal?.throwIfAborted();\n this.received.push(messages.map((message) => ({ ...message })));\n const capacity = await this.assessRequestCapacity(messages, maxOutputTokens);\n if (capacity.decision === \"reject\") {\n throw new ProviderError(\"mock\", \"capacity_exceeded\", \"Mock request exceeds its input wall.\", {\n capacity,\n extensions: { capacityStage: \"preflight\", capacity },\n });\n }\n const settle = await observeRequest?.({ provider: \"provider:mock\", model: this.model });\n const next = this.#queue.shift();\n if (next === undefined) {\n const accounting = validateProviderRequestAccounting({\n provider: \"provider:mock\",\n model: this.model,\n outcome: \"error\",\n cost: { kind: \"unknown\", reason: \"mock provider exhausted before producing a response\" },\n });\n await settle?.(accounting);\n throw new ProviderError(\n \"mock\",\n \"invalid_response\",\n \"Mock provider exhausted: no more queued responses\",\n { accounting: [accounting], capacity },\n );\n }\n const a = next.assistant;\n const usage: ProviderUsage = {\n ...MOCK_USAGE,\n ...next.usage,\n };\n const requestAccounting: ProviderRequestAccounting = validateProviderRequestAccounting({\n provider: \"provider:mock\",\n model: a.model ?? this.model,\n outcome: \"response\",\n usage,\n cost: next.cost ?? {\n kind: \"estimated\",\n amount: { amount: \"0\", currency: \"USD\" },\n source: \"mock provider fixture\",\n },\n });\n await settle?.(requestAccounting);\n const assistant: MockReturnedAssistant = {\n content: a.content,\n reasoning: a.reasoning,\n finishReason: a.finishReason ?? (a.toolCalls && a.toolCalls.length > 0 ? \"tool_calls\" : \"stop\"),\n model: a.model ?? \"mock\",\n ...(a.toolCalls !== undefined\n ? {\n toolCalls: a.toolCalls.map((tc, index): ProviderToolCall => ({\n index: tc.index ?? index,\n id: tc.id ?? `mock_call_${index}`,\n type: tc.type ?? \"function\",\n name: tc.name,\n arguments: typeof tc.arguments === \"string\" ? tc.arguments : JSON.stringify(tc.arguments),\n ...(tc.thoughtSignature !== undefined ? { thoughtSignature: tc.thoughtSignature } : {}),\n })),\n }\n : {}),\n ...(a.ops !== undefined ? { ops: a.ops } : {}),\n };\n if (assistant.reasoning !== null && assistant.reasoning.length > 0) {\n observeReasoning?.(assistant.reasoning);\n }\n return {\n assistant,\n assistantRaw: next.assistantRaw ?? null,\n accounting: [requestAccounting],\n capacity,\n };\n }\n\n get remaining(): number { return this.#queue.length; }\n}\n\nexport { MOCK_USAGE as mockDefaultUsage };\n"]}
|
package/dist/ModelProvider.d.ts
CHANGED
|
@@ -11,6 +11,7 @@ export type CacheAffinity = {
|
|
|
11
11
|
readonly name: string;
|
|
12
12
|
};
|
|
13
13
|
export type CacheBreakpoints = Exclude<CacheWritePolicy, "off">;
|
|
14
|
+
export type PromptCacheStyle = "ephemeral" | "openai-explicit";
|
|
14
15
|
export type ModelProviderConfig = {
|
|
15
16
|
inferenceAdmission?: InferenceAdmission;
|
|
16
17
|
model: string;
|
|
@@ -42,6 +43,7 @@ export type ModelProviderConfig = {
|
|
|
42
43
|
source?: string;
|
|
43
44
|
cacheAffinity?: CacheAffinity;
|
|
44
45
|
cacheBreakpoints?: CacheBreakpoints;
|
|
46
|
+
promptCacheStyle?: PromptCacheStyle;
|
|
45
47
|
serviceTier?: string;
|
|
46
48
|
requestControls?: NativeRequestControls;
|
|
47
49
|
apiKeyRejectedMessage?: string;
|
|
@@ -96,6 +98,6 @@ export default class ModelProvider implements Provider {
|
|
|
96
98
|
get requiresOutputBudget(): boolean | undefined;
|
|
97
99
|
countPromptTokens(messages: readonly ChatMessage[], signal?: AbortSignal): Promise<PromptTokenMeasurement>;
|
|
98
100
|
assessRequestCapacity(messages: readonly ChatMessage[], maxOutputTokens?: number, signal?: AbortSignal): Promise<ProviderRequestCapacity>;
|
|
99
|
-
generate({ messages, sessionId, signal, maxOutputTokens, sampling, observeRequest, observeReasoning, callKind,
|
|
101
|
+
generate({ messages, sessionId, signal, maxOutputTokens, sampling, observeRequest, observeReasoning, callKind, userCachePrefixes }: ProviderGenerateArgs): Promise<ProviderResponse>;
|
|
100
102
|
}
|
|
101
103
|
//# sourceMappingURL=ModelProvider.d.ts.map
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"ModelProvider.d.ts","sourceRoot":"","sources":["../src/ModelProvider.ts"],"names":[],"mappings":"AAQA,OAAO,KAAK,EACR,WAAW,EACX,MAAM,EACN,aAAa,EACb,sBAAsB,EACtB,QAAQ,EAER,sBAAsB,EACtB,oBAAoB,EACpB,aAAa,EAEb,uBAAuB,EAEvB,gBAAgB,EAChB,aAAa,EAChB,MAAM,YAAY,CAAC;AACpB,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,0BAA0B,CAAC;AAE7D,OAAO,KAAK,EAAE,gBAAgB,EAAE,aAAa,EAAE,sBAAsB,EAAE,MAAM,UAAU,CAAC;AAMxF,OAAO,EAAiB,KAAK,oBAAoB,EAAE,KAAK,2BAA2B,EAAE,MAAM,qBAAqB,CAAC;AAOjH,OAAO,KAAK,kBAAkB,MAAM,yBAAyB,CAAC;AAM9D,MAAM,MAAM,aAAa,GAAG,OAAO,UAAU,CAAC,KAAK,CAAC;AAGpD,MAAM,MAAM,cAAc,GAAG,MAAM,GAAG,OAAO,GAAG,UAAU,CAAC;AAE3D,MAAM,MAAM,YAAY,GAAG,SAAS,GAAG,KAAK,GAAG,QAAQ,GAAG,MAAM,GAAG,OAAO,CAAC;AAE3E,MAAM,MAAM,aAAa,GAAG;IAAE,QAAQ,CAAC,MAAM,EAAE,QAAQ,GAAG,MAAM,CAAC;IAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;CAAE,CAAC;AAC1F,MAAM,MAAM,gBAAgB,GAAG,OAAO,CAAC,gBAAgB,EAAE,KAAK,CAAC,CAAC;
|
|
1
|
+
{"version":3,"file":"ModelProvider.d.ts","sourceRoot":"","sources":["../src/ModelProvider.ts"],"names":[],"mappings":"AAQA,OAAO,KAAK,EACR,WAAW,EACX,MAAM,EACN,aAAa,EACb,sBAAsB,EACtB,QAAQ,EAER,sBAAsB,EACtB,oBAAoB,EACpB,aAAa,EAEb,uBAAuB,EAEvB,gBAAgB,EAChB,aAAa,EAChB,MAAM,YAAY,CAAC;AACpB,OAAO,KAAK,EAAE,YAAY,EAAE,MAAM,0BAA0B,CAAC;AAE7D,OAAO,KAAK,EAAE,gBAAgB,EAAE,aAAa,EAAE,sBAAsB,EAAE,MAAM,UAAU,CAAC;AAMxF,OAAO,EAAiB,KAAK,oBAAoB,EAAE,KAAK,2BAA2B,EAAE,MAAM,qBAAqB,CAAC;AAOjH,OAAO,KAAK,kBAAkB,MAAM,yBAAyB,CAAC;AAM9D,MAAM,MAAM,aAAa,GAAG,OAAO,UAAU,CAAC,KAAK,CAAC;AAGpD,MAAM,MAAM,cAAc,GAAG,MAAM,GAAG,OAAO,GAAG,UAAU,CAAC;AAE3D,MAAM,MAAM,YAAY,GAAG,SAAS,GAAG,KAAK,GAAG,QAAQ,GAAG,MAAM,GAAG,OAAO,CAAC;AAE3E,MAAM,MAAM,aAAa,GAAG;IAAE,QAAQ,CAAC,MAAM,EAAE,QAAQ,GAAG,MAAM,CAAC;IAAC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;CAAE,CAAC;AAC1F,MAAM,MAAM,gBAAgB,GAAG,OAAO,CAAC,gBAAgB,EAAE,KAAK,CAAC,CAAC;AAChE,MAAM,MAAM,gBAAgB,GAAG,WAAW,GAAG,iBAAiB,CAAC;AAE/D,MAAM,MAAM,mBAAmB,GAAG;IAC9B,kBAAkB,CAAC,EAAE,kBAAkB,CAAC;IACxC,KAAK,EAAE,MAAM,CAAC;IACd,GAAG,CAAC,EAAE,MAAM,CAAC;IACb,aAAa,CAAC,EAAE,aAAa,CAAC;IAC9B,YAAY,CAAC,EAAE,CAAC,OAAO,EAAE,2BAA2B,KAAK,oBAAoB,CAAC;IAC9E,cAAc,EAAE,MAAM,CAAC;IACvB,kBAAkB,EAAE,MAAM,CAAC;IAC3B,OAAO,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACjC,KAAK,CAAC,EAAE,aAAa,CAAC;IACtB,aAAa,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC9B,eAAe,CAAC,EAAE,WAAW,CAAC,aAAa,CAAC,CAAC;IAC7C,cAAc,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC/B,eAAe,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAChC,YAAY,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC7B,WAAW,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,eAAe,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAChC,gBAAgB,CAAC,EAAE,SAAS,MAAM,EAAE,CAAC;IAIrC,cAAc,CAAC,EAAE,YAAY,GAAG,kBAAkB,CAAC;IACnD,sBAAsB,CAAC,EAAE,OAAO,CAAC;IAKjC,yBAAyB,CAAC,EAAE,WAAW,GAAG,SAAS,CAAC;IACpD,cAAc,CAAC,EAAE,cAAc,CAAC;IAChC,gBAAgB,CAAC,EAAE,YAAY,GAAG,uBAAuB,CAAC;IAC1D,oBAAoB,CAAC,EAAE,CAAC,IAAI,EAAE,MAAM,EAAE,eAAe,EAAE,MAAM,GAAG,IAAI,KAAK,MAAM,CAAC,MAAM,EAAE,OAAO,CAAC,CAAC;IACjG,sBAAsB,CAAC,EAAE,sBAAsB,CAAC;IAChD,iBAAiB,CAAC,EAAE,CAAC,QAAQ,EAAE,SAAS,WAAW,EAAE,EAAE,MAAM,CAAC,EAAE,WAAW,KAAK,sBAAsB,GAAG,OAAO,CAAC,sBAAsB,CAAC,CAAC;IACzI,YAAY,CAAC,EAAE,CAAC,KAAK,EAAE,aAAa,GAAG,SAAS,KAAK,YAAY,CAAC;IAClE,aAAa,CAAC,EAAE,sBAAsB,CAAC;IACvC,MAAM,CAAC,EAAE,MAAM,CAAC;IAGhB,aAAa,CAAC,EAAE,aAAa,CAAC;IAE9B,gBAAgB,CAAC,EAAE,gBAAgB,CAAC;IACpC,gBAAgB,CAAC,EAAE,gBAAgB,CAAC;IAIpC,WAAW,CAAC,EAAE,MAAM,CAAC;IAGrB,eAAe,CAAC,EAAE,qBAAqB,CAAC;IACxC,qBAAqB,CAAC,EAAE,MAAM,CAAC;IAC/B,OAAO,CAAC,EAAE,MAAM,CAAC;IAEjB,mBAAmB,CAAC,EAAE,OAAO,CAAC;IAC9B,SAAS,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAI1B,WAAW,CAAC,EAAE,MAAM,CAAC;IAGrB,eAAe,CAAC,EAAE,MAAM,CAAC;IAKzB,WAAW,CAAC,EAAE,MAAM,CAAC;IAIrB,oBAAoB,CAAC,EAAE,OAAO,CAAC;IAG/B,MAAM,EAAE,aAAa,CAAC;IAGtB,WAAW,EAAE,MAAM,GAAG,IAAI,CAAC;IAC3B,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,IAAI,CAAC,EAAE,MAAM,CAAC;IACd,aAAa,EAAE,MAAM,GAAG,IAAI,CAAC;IAE7B,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAK1B,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,WAAW,CAAC,EAAE,MAAM,CAAC;IAKrB,aAAa,EAAE,MAAM,CAAC;IAItB,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAQ1B,WAAW,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,OAAO,CAAC,EAAE,OAAO,CAAC;CACrB,CAAC;AAmCF,MAAM,WAAW,qBAAqB;IAClC,QAAQ,CAAC,SAAS,EAAE,MAAM,CAAC;IAC3B,QAAQ,CAAC,WAAW,EAAE,OAAO,CAAC;IAC9B,QAAQ,CAAC,QAAQ,EAAE,OAAO,CAAC;CAC9B;AA+BD,MAAM,CAAC,OAAO,OAAO,aAAc,YAAW,QAAQ;;IAoDlD,QAAQ,CAAC,YAAY,CAAC,EAAE,CAAC,OAAO,EAAE,2BAA2B,KAAK,oBAAoB,CAAC;IAMvF,QAAQ,CAAC,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,OAAO,CAAC,MAAM,EAAE,CAAC,CAAC;IAE/C,YAAY,MAAM,EAAE,mBAAmB,EA6JtC;IAED,IAAI,aAAa,IAAI,MAAM,GAAG,IAAI,CAAgC;IAClE,IAAI,eAAe,IAAI,WAAW,CAAC,aAAa,CAAC,CAAkC;IACnF,IAAI,cAAc,IAAI,MAAM,GAAG,IAAI,CAAiC;IACpE,IAAI,eAAe,IAAI,MAAM,GAAG,IAAI,CAAkC;IACtE,IAAI,YAAY,IAAI,MAAM,GAAG,IAAI,CAA+B;IAChE,IAAI,WAAW,IAAI,MAAM,GAAG,IAAI,CAA8B;IAC9D,IAAI,eAAe,IAAI,MAAM,GAAG,IAAI,CAAkC;IACtE,IAAI,gBAAgB,IAAI,SAAS,MAAM,EAAE,CAAmC;IAC5E,IAAI,aAAa,IAAI,MAAM,GAAG,IAAI,CAMjC;IAED,IAAI,SAAS,IAAI,MAAM,GAAG,IAAI,CAM7B;IACD,IAAI,KAAK,IAAI,MAAM,CAAwB;IAC3C,IAAI,WAAW,IAAI,MAAM,GAAG,IAAI,CAAuD;IACvF,IAAI,IAAI,IAAI,MAAM,GAAG,SAAS,CAAwC;IAEtE,IAAI,WAAW,IAAI,MAAM,GAAG,SAAS,CAA8B;IAEnE,IAAI,oBAAoB,IAAI,OAAO,GAAG,SAAS,CAAuC;IAEhF,iBAAiB,CACnB,QAAQ,EAAE,SAAS,WAAW,EAAE,EAChC,MAAM,CAAC,EAAE,WAAW,GACrB,OAAO,CAAC,sBAAsB,CAAC,CA8DjC;IAEK,qBAAqB,CACvB,QAAQ,EAAE,SAAS,WAAW,EAAE,EAChC,eAAe,CAAC,EAAE,MAAM,EACxB,MAAM,CAAC,EAAE,WAAW,GACrB,OAAO,CAAC,uBAAuB,CAAC,CAoBlC;IA4BK,QAAQ,CAAC,EAAE,QAAQ,EAAE,SAAS,EAAE,MAAM,EAAE,eAAe,EAAE,QAAQ,EAAE,cAAc,EAAE,gBAAgB,EAAE,QAAQ,EAAE,iBAAiB,EAAE,EAAE,oBAAoB,GAAG,OAAO,CAAC,gBAAgB,CAAC,CAqbzL;CAEJ"}
|