@plurnk/plurnk-providers 1.21.0 → 1.22.0

package/.env.defaults

@@ -11,11 +11,15 @@
 # PLURNK_MODEL_local=openai/model-name-from-endpoint
 # PLURNK_BASEURL_local=http://127.0.0.1:8080/v1
 # PLURNK_MODEL=cloud
-# PLURNK_MODEL=google/gemini-3-flash
+# PLURNK_MODEL=deepseek/deepseek-v4-flash
 
 # --- Capacity ---
-# Total output envelope INCLUDING reasoning: positive tokens or context percentage in (0,100).
-# Known model limits cap it. Must leave input capacity; the context budget is derived, not set here.
+# Simultaneous inference attempts per resolved endpoint in this process; -1 = unrestricted.
+# Positive limits queue excess calls. Aliases sharing an endpoint must agree on its limit.
+PLURNK_PROVIDERS_MAX_CONCURRENCY=-1
+# PLURNK_PROVIDERS_MAX_CONCURRENCY_local=1
+# Reserved output INCLUDING reasoning: positive tokens or context percentage in (0,100).
+# Known model limits cap it; input capacity is derived. Exact-counting calls may grant unused context.
 PLURNK_PROVIDERS_OUTPUT_BUDGET=35%
 # Optional reasoning subset of output, using the same units; must be smaller than total output.
 # Unset/empty = provider-adaptive budget, not reasoning off.
@@ -26,23 +30,44 @@ PLURNK_PROVIDERS_OUTPUT_BUDGET=35%
 
 # --- Reasoning ---
 # off | adaptive | low | medium | high | xhigh | max. Seed for durable Worker reasoning.
-# adaptive uses native dynamic reasoning where supported, otherwise the supported high posture.
-PLURNK_PROVIDERS_REASONING=adaptive
+# adaptive prefers native dynamic reasoning, then the supported fallback below.
+PLURNK_PROVIDERS_EFFORT=adaptive
+# Fixed effort when native adaptation is unavailable; empty/unsupported = enabled provider default.
+PLURNK_PROVIDERS_EFFORT_FALLBACK=high
 # Response interpretation: verbatim = trust structured reasoning fields; think-tags = extract one
 # leading <think> envelope. Select only for an endpoint that emits it; independent of activation.
 PLURNK_PROVIDERS_REASONING_RESPONSE_STYLE=verbatim
-# Reasoning wire for one route, over PLURNK_PROVIDERS_PROVIDER_<NAME>_REASONING_STYLE; usually
-# alias-scoped. thinking_config = Gemini through an OpenAI-compatible endpoint (readable thoughts,
-# levels low/medium/high, no off). Empty = the provider's declaration.
-# PLURNK_PROVIDERS_REASONING_STYLE=
-# PLURNK_PROVIDERS_REASONING_STYLE_<alias>=thinking_config
+# Compatible wire declarations below supplement Models.dev; native SDKs own their protocols.
+# Route knobs use the same suffix, e.g. PLURNK_PROVIDERS_REASONING_BUDGET_PATH_<alias>.
+# PLURNK_PROVIDERS_OUTPUT_PATH_<alias>=/max_completion_tokens
+# PLURNK_PROVIDERS_REASONING_EFFORT_PATH_<alias>=/reasoning_effort
+# PLURNK_PROVIDERS_REASONING_CONTROLS_<alias>=exclusive
+# PLURNK_PROVIDERS_REASONING_ON_BODY_<alias>='{"enable_thinking":true}'
+# PLURNK_PROVIDERS_REASONING_OFF_BODY_<alias>='{"enable_thinking":false}'
+# PLURNK_PROVIDERS_REASONING_EFFORTS_<alias>=low,medium,high
+# PLURNK_PROVIDERS_REASONING_TRANSPORT_EFFORTS_<alias>=none,low,medium,high
+# PLURNK_PROVIDERS_OPTIONS_NAMESPACE_<alias>=openrouter
+# PLURNK_PROVIDERS_REASONING_ADAPTIVE_BODY_<alias>='{}'
+# PLURNK_PROVIDERS_REASONING_TOGGLE_BODY_<alias>='{"reasoning":{"enabled":true}}'
+# PLURNK_PROVIDERS_ADAPTIVE_OPTIONS_<alias>='[{"models":["*"],"options":{"<sdk-namespace>":{…}}}]'
+# PLURNK_PROVIDERS_SYSTEM_CACHE_OPTIONS_<alias>=
+# PLURNK_PROVIDERS_APP_URL_<alias>=https://example.test/app
+# PLURNK_PROVIDERS_APP_NAME_<alias>=Example
 
 # --- Sampling ---
 # Temperature; empty = omit, retaining the provider's sampling default.
 PLURNK_PROVIDERS_TEMPERATURE=
+# Nucleus probability mass, 0..1; empty = omit.
+PLURNK_PROVIDERS_TOP_P=
+# Token shortlist, non-negative integer; empty = omit. Support and zero semantics vary by endpoint.
+PLURNK_PROVIDERS_TOP_K=
+# Presence penalty, -2..2; empty = omit.
+PLURNK_PROVIDERS_PRESENCE_PENALTY=
+# Sampling seed, safe integer; empty = omit. Reproducibility depends on the endpoint.
+PLURNK_PROVIDERS_SEED=
 # Repetition penalty; empty = omit. Values above 1 can penalize required verbatim repetition.
 PLURNK_PROVIDERS_REPEAT_PENALTY=
-# Frequency penalty; 0 = off. Support and effect depend on the endpoint.
+# Frequency penalty, -2..2; 0 = no configured override. Support depends on the endpoint.
 PLURNK_PROVIDERS_FREQUENCY_PENALTY=0
 # Repeat-penalty window, tokens; unset = endpoint default.
 # PLURNK_PROVIDERS_REPEAT_LAST_N=512
@@ -56,7 +81,15 @@ PLURNK_PROVIDERS_DRY_MULTIPLIER=0
 # --- Cache and billing ---
 # Stable Worker affinity on supported routes: 1 = on; 0 = off. Unknown routes get no guessed field.
 PLURNK_PROVIDERS_CACHE_AFFINITY=1
-# Explicit cache writes: stable-system = reusable system boundary on supported Claude routes;
+# FIELD declares placement, independent of the enable switch; null clears it for an alias.
+# target is header or body with name, or provider-option with provider and name.
+# PLURNK_PROVIDERS_CACHE_AFFINITY_FIELD_<alias>='{"target":"header","name":"x-session-id"}'
+PLURNK_PROVIDERS_PROVIDER_OPENAI_CACHE_AFFINITY_FIELD='{"target":"provider-option","provider":"openai","name":"promptCacheKey"}'
+PLURNK_PROVIDERS_PROVIDER_DEEPINFRA_CACHE_AFFINITY_FIELD='{"target":"provider-option","provider":"deepinfra","name":"prompt_cache_key"}'
+PLURNK_PROVIDERS_PROVIDER_OPENROUTER_CACHE_AFFINITY_FIELD='{"target":"header","name":"x-session-id"}'
+PLURNK_PROVIDERS_PROVIDER_CLOUDFLARE_WORKERS_AI_CACHE_AFFINITY_FIELD='{"target":"header","name":"x-session-affinity"}'
+PLURNK_PROVIDERS_PROVIDER_FIREWORKS_AI_CACHE_AFFINITY_FIELD='{"target":"header","name":"x-session-affinity"}'
+# Explicit cache writes: stable-system = reusable system boundary where SYSTEM_CACHE_OPTIONS declares it;
 # off = no explicit writes. This is distinct from affinity and can affect billing.
 PLURNK_PROVIDERS_CACHE_WRITE_POLICY=stable-system
 # Provider service tier; unset = provider default. Usually set per alias: routing can change cost.
@@ -67,14 +100,21 @@ PLURNK_PROVIDERS_CACHE_WRITE_POLICY=stable-system
 # PLURNK_PROVIDERS_COST=input=0.22,output=0.66,reasoning=0.66,cacheRead=0.007
 
 # --- Connectivity (milliseconds; each deadline accepts 0 = off) ---
-# Logical generation deadline, including physical attempts and retry delays.
+# Logical generation deadline, including admission queues, physical attempts and retry delays.
 PLURNK_PROVIDERS_OPERATION_TIMEOUT=2700000
 # Deadline per physical generation attempt; a streamed attempt is bounded only until content flows.
 PLURNK_PROVIDERS_FETCH_TIMEOUT=600000
-# Wait from stream start to first semantic content; metadata/empty deltas do not count.
+# Wait from dispatch (after admission, before response headers) to first semantic content;
+# metadata/empty deltas do not count.
 PLURNK_PROVIDERS_FIRST_CONTENT_TIMEOUT=180000
 # Maximum silence between semantic content chunks after content begins.
 PLURNK_PROVIDERS_STREAM_IDLE_TIMEOUT=120000
+# A line of 16+ characters streamed this many times stops the response as a repetition; 0 = off.
+PLURNK_PROVIDERS_REPEATED_LINE_LIMIT=32
+# A response billed for this many more output tokens than it streamed characters (text and reasoning)
+# never delivered its output; it fails as output_dropped and is re-issued (a tool-call finish always
+# does). Reasoning billed but not streamed is exempt. 0 = off.
+PLURNK_PROVIDERS_DROPPED_OUTPUT_TOKENS=32
 # Retries for 429, Retry-After, or X-Should-Retry:true; 0 = none. Other failures return immediately
 # to Core's separate recovery window. Retry-After takes precedence over backoff.
 PLURNK_PROVIDERS_RETRY_ATTEMPTS=3
@@ -104,14 +144,60 @@ PLURNK_PROVIDERS_PROBE_DELAY=250
 # PLURNK_PROVIDERS_PROVIDER_ACME_NPM=@ai-sdk/openai-compatible
 # PLURNK_PROVIDERS_PROVIDER_ACME_BASE_URL=https://api.acme.example/v1
 # PLURNK_PROVIDERS_PROVIDER_ACME_API_KEY_ENV=ACME_API_KEY
-# Reasoning wire adaptations that are not expressed by catalog metadata:
-# Fireworks requires an explicit off effort; Cloudflare graded models require an effort.
-PLURNK_PROVIDERS_PROVIDER_FIREWORKS_AI_REASONING_STYLE=effort_explicit
-PLURNK_PROVIDERS_PROVIDER_CLOUDFLARE_WORKERS_AI_REASONING_STYLE=effort_required
+# Compatible request fields use RFC 6901 pointers; output defaults to the SDK's max_tokens.
+# Effort vocabulary and budget bounds come from Models.dev; EFFORTS adds declared values.
+# ON_BODY applies whenever reasoning is active. OFF_BODY disables it. ADAPTIVE_BODY replaces
+# the graded fallback; {} leaves the enabled endpoint at its native default.
+# TOGGLE_BODY enables a catalog toggle when no adaptive body or supported fallback applies.
+# Bodies contain only reasoning controls, not model/messages, sampling, or token ceilings.
+# With both paths declared, CONTROLS must say exclusive or combined. Fixed effort plus a numeric
+# budget on an exclusive route fails before inference; adaptive + budget selects the numeric dial.
+# PLURNK_PROVIDERS_PROVIDER_ACME_OUTPUT_PATH=/max_completion_tokens
+# PLURNK_PROVIDERS_PROVIDER_ACME_REASONING_EFFORT_PATH=/reasoning/effort
+# PLURNK_PROVIDERS_PROVIDER_ACME_REASONING_BUDGET_PATH=/reasoning/max_tokens
+# PLURNK_PROVIDERS_PROVIDER_ACME_REASONING_CONTROLS=exclusive
+# PLURNK_PROVIDERS_PROVIDER_ACME_REASONING_EFFORTS=low,medium,high
+# PLURNK_PROVIDERS_PROVIDER_ACME_REASONING_ON_BODY='{"reasoning":{"enabled":true}}'
+# PLURNK_PROVIDERS_PROVIDER_ACME_REASONING_OFF_BODY='{"reasoning":{"enabled":false}}'
+# PLURNK_PROVIDERS_PROVIDER_ACME_REASONING_ADAPTIVE_BODY='{}'
+PLURNK_PROVIDERS_PROVIDER_FIREWORKS_AI_REASONING_EFFORT_PATH=/reasoning_effort
+PLURNK_PROVIDERS_PROVIDER_FIREWORKS_AI_REASONING_OFF_BODY='{"reasoning_effort":"none"}'
+PLURNK_PROVIDERS_PROVIDER_FIREWORKS_AI_REASONING_TOGGLE_BODY='{"reasoning_effort":true}'
+PLURNK_PROVIDERS_PROVIDER_CLOUDFLARE_WORKERS_AI_REASONING_EFFORT_PATH=/reasoning_effort
 # Additional supported Cloudflare effort values, comma-separated; empty = Models.dev only.
 PLURNK_PROVIDERS_PROVIDER_CLOUDFLARE_WORKERS_AI_REASONING_EFFORTS=
 # Direct DeepSeek thinking activation and effort.
-PLURNK_PROVIDERS_PROVIDER_DEEPSEEK_REASONING_STYLE=thinking_effort
+PLURNK_PROVIDERS_PROVIDER_DEEPSEEK_REASONING_EFFORT_PATH=/reasoning_effort
+PLURNK_PROVIDERS_PROVIDER_DEEPSEEK_REASONING_ON_BODY='{"thinking":{"type":"enabled"}}'
+PLURNK_PROVIDERS_PROVIDER_DEEPSEEK_REASONING_OFF_BODY='{"thinking":{"type":"disabled"}}'
+# OpenRouter's native SDK accepts these same fields in per-call provider options.
+PLURNK_PROVIDERS_PROVIDER_OPENROUTER_OPTIONS_NAMESPACE=openrouter
+PLURNK_PROVIDERS_PROVIDER_OPENROUTER_REASONING_EFFORT_PATH=/reasoning/effort
+PLURNK_PROVIDERS_PROVIDER_OPENROUTER_REASONING_BUDGET_PATH=/reasoning/max_tokens
+PLURNK_PROVIDERS_PROVIDER_OPENROUTER_REASONING_CONTROLS=exclusive
+PLURNK_PROVIDERS_PROVIDER_OPENROUTER_REASONING_TRANSPORT_EFFORTS=none,minimal,low,medium,high,xhigh,max
+PLURNK_PROVIDERS_PROVIDER_OPENROUTER_REASONING_ON_BODY='{"reasoning":{"enabled":true}}'
+PLURNK_PROVIDERS_PROVIDER_OPENROUTER_REASONING_OFF_BODY='{"reasoning":{"effort":"none"}}'
+# DeepInfra's and Together's native SDKs extend openai-compatible, which overwrites a raw `reasoning_effort`
+# option; the effort rides the SDK's own `reasoningEffort`, so every catalog effort, max included, reaches the wire.
+PLURNK_PROVIDERS_PROVIDER_DEEPINFRA_OPTIONS_NAMESPACE=deepinfra
+PLURNK_PROVIDERS_PROVIDER_DEEPINFRA_REASONING_EFFORT_PATH=/reasoningEffort
+PLURNK_PROVIDERS_PROVIDER_DEEPINFRA_REASONING_OFF_BODY='{"reasoningEffort":"none"}'
+PLURNK_PROVIDERS_PROVIDER_TOGETHERAI_OPTIONS_NAMESPACE=togetherai
+PLURNK_PROVIDERS_PROVIDER_TOGETHERAI_REASONING_EFFORT_PATH=/reasoningEffort
+PLURNK_PROVIDERS_PROVIDER_TOGETHERAI_REASONING_OFF_BODY='{"reasoningEffort":"none"}'
+# Model-family provider options: a JSON array of {"models":["<glob>",…],"options":{…}} rules matched
+# against the route's model id, first match wins ({§provider-model-options}). ADAPTIVE_OPTIONS replaces the
+# adaptive policy's projection; SYSTEM_CACHE_OPTIONS marks the stable system prompt for cache writes.
+# None ship: the shipped defaults serve open models only; declare a closed family's options yourself.
+# Alibaba DashScope (OpenAI-compatible): max_tokens is answer-only, max_completion_tokens is inclusive;
+# effort and thinking budget are alternatives; enable_thinking switches reasoning.
+PLURNK_PROVIDERS_PROVIDER_ALIBABA_OUTPUT_PATH=/max_completion_tokens
+PLURNK_PROVIDERS_PROVIDER_ALIBABA_REASONING_EFFORT_PATH=/reasoning_effort
+PLURNK_PROVIDERS_PROVIDER_ALIBABA_REASONING_BUDGET_PATH=/thinking_budget
+PLURNK_PROVIDERS_PROVIDER_ALIBABA_REASONING_CONTROLS=exclusive
+PLURNK_PROVIDERS_PROVIDER_ALIBABA_REASONING_ON_BODY='{"enable_thinking":true}'
+PLURNK_PROVIDERS_PROVIDER_ALIBABA_REASONING_OFF_BODY='{"enable_thinking":false}'
 # Uncataloged compatible providers: SDK, endpoint, and credential-variable name.
 PLURNK_PROVIDERS_PROVIDER_BAICHUAN_NPM=@ai-sdk/openai-compatible
 PLURNK_PROVIDERS_PROVIDER_BAICHUAN_BASE_URL=https://api.baichuan-ai.com/v1
@@ -124,8 +210,8 @@ PLURNK_PROVIDERS_PROVIDER_QIANFAN_API_KEY_ENV=QIANFAN_API_KEY
 # OpenAI-compatible fallback endpoint; PLURNK_BASEURL_<alias> overrides it for one declared alias.
 OPENAI_BASE_URL=https://api.openai.com/v1
 
-# --- OpenRouter application attribution ---
-# Public application URL; empty suppresses attribution, including the title.
-OPENROUTER_HTTP_REFERER=https://github.com/plurnk/plurnk-service
-# Application display name, sent only with a nonempty URL.
-OPENROUTER_APP_TITLE=Plurnk
+# --- Application attribution (routes on the OpenRouter SDK) ---
+# Public application URL the SDK sends as HTTP-Referer; empty suppresses attribution, including the name.
+PLURNK_PROVIDERS_PROVIDER_OPENROUTER_APP_URL=https://github.com/plurnk/plurnk-service
+# Application display name the SDK sends as X-OpenRouter-Title, only with a nonempty URL.
+PLURNK_PROVIDERS_PROVIDER_OPENROUTER_APP_NAME=Plurnk

package/README.md

@@ -22,6 +22,8 @@ Model facts do not share one fallback chain. Context windows, output envelopes,
 reasoning activation, and estimated prices resolve independently
 ({§model-fact-resolution}). PLURNK does not fetch live per-token prices, and the
 local estimate is not an authoritative relay-settled charge.
+Router accounting includes separately billed BYOK inference, not merely the
+router fee; incomplete billing evidence remains unknown ({§provider-monetary-evidence}).
 
 ## Runtime-neutral contracts
 
@@ -45,16 +47,16 @@ discovery and environment-file defaults.
 Select a catalog route directly:
 
 ```dotenv
-PLURNK_MODEL=google/gemini-3-flash
-GEMINI_API_KEY=...
+PLURNK_MODEL=deepseek/deepseek-v4-flash
+DEEPSEEK_API_KEY=...
 ```
 
 Declare an alias when the route needs a reusable name or scoped tuning:
 
 ```dotenv
-PLURNK_MODEL_fast=openai/gpt-5-mini
+PLURNK_MODEL_fast=deepinfra/zai-org/GLM-5.3-Flash
 PLURNK_MODEL=fast
-OPENAI_API_KEY=...
+DEEPINFRA_API_KEY=...
 ```
 
 Cataloged providers need no endpoint declaration. To add an
@@ -88,6 +90,13 @@ changing a startup default is not a command to retarget existing Workers.
 Clients expose explicit model, reasoning, and child-model controls; their
 available effort choices come from the selected route.
 
+Sampling uses endpoint defaults unless configured. `TEMPERATURE`, `TOP_P`,
+`TOP_K`, `PRESENCE_PENALTY`, `FREQUENCY_PENALTY`, and `SEED` use the same
+`PLURNK_PROVIDERS_<KNOB>_<alias>` form. Supported values depend on the endpoint;
+Models.dev does not supply a recommended sampling profile. See the
+[sampling contract](SPEC.md) ({§provider-sampling-passthrough}) and
+[environment panel](.env.defaults) for ranges and omission rules.
+
 ## Capacity and reasoning
 
 ```mermaid
@@ -99,8 +108,9 @@ flowchart TD
 ```
 
 - **Effort** and **budget** are independent. `adaptive` requests native dynamic
-  reasoning where supported, otherwise the supported high posture. Fixed effort
-  names must be supported by the route; an unsupported request is not silently downgraded.
+  reasoning where supported, otherwise `PLURNK_PROVIDERS_EFFORT_FALLBACK`
+  (default `high`). An unavailable fallback retains the reasoning-enabled provider default.
+  Fixed effort names must be supported by the route; unsupported requests are not silently downgraded.
 - **Output** accepts positive tokens or a percentage of context; known model
   limits cap it. An explicit reasoning budget must be smaller than total output.
   Leaving the reasoning budget unset does not disable reasoning.
@@ -157,6 +167,8 @@ PLURNK_MODEL=local
 # Optional explicit generation allowances, in tokens:
 # PLURNK_PROVIDERS_OUTPUT_BUDGET_local=8192
 # PLURNK_PROVIDERS_REASONING_BUDGET_local=4096
+# Optional: serialize inference on a single-slot endpoint without serializing workers:
+# PLURNK_PROVIDERS_MAX_CONCURRENCY_local=1
 ```
 
 Endpoint probing supplies served-model capacity and llama-server capabilities.
@@ -175,6 +187,7 @@ opt-in for endpoints that emit a leading `<think>` envelope, not a reasoning swi
 
 | Concern | Boundary |
 | --- | --- |
+| Inference concurrency | Optional per-endpoint, process-local admission; queued calls remain cancellable. The default is unrestricted ({§provider-inference-admission}). |
 | Attempt, first-content, idle deadlines | Provider transport; a timeout surfaces a failure, not a fabricated response. |
 | Provider-directed waits | Bounded retries honor `Retry-After`; other recoverable failures return to Core. |
 | Loop recovery | Core reissues within its recovery window, then parks for a prompt or wake. |