@zhivex-ai/gateway 1.2.2 → 1.4.0

package/README.md

@@ -1,5 +1,33 @@
 # @zhivex-ai/gateway
 
+## Vertex partner routing
+
+Register one Vertex adapter with Google Cloud credentials. The publisher stays
+inside `modelId`; the gateway provider remains `vertex` for Google-hosted models.
+
+```ts
+import { createGateway } from "@zhivex-ai/gateway";
+import { createVertex } from "@zhivex-ai/vertex";
+
+const gateway = createGateway({
+  adapters: { vertex: createVertex({
+    projectId: process.env.GOOGLE_CLOUD_PROJECT,
+    location: "global"
+  }) },
+  maxRetries: 0
+});
+
+const result = await gateway.generate({
+  primary: { provider: "vertex", modelId: "openai/gpt-oss-120b-maas" },
+  fallbacks: [{ provider: "vertex", modelId: "gemini-3.7-flash" }],
+  messages: [{ role: "user", content: "Explain a binary search." }]
+});
+```
+
+This requires working ADC or another bearer credential resolved by Vertex.
+Model permissions, regional availability and quotas apply separately to each
+target. See the [Vertex provider](../vertex/README.md) for route-specific limits.
+
 Routing and fallback package for Zhivex AI SDK.
 
 The gateway now supports:
@@ -18,10 +46,10 @@ For agent routing, the gateway can also filter by `agentCapabilities`, such as p
 ## Install
 
 ```bash
-bun add @zhivex-ai/gateway @zhivex-ai/core @zhivex-ai/anthropic @zhivex-ai/openai @zhivex-ai/ollama
+bun add @zhivex-ai/gateway @zhivex-ai/core @zhivex-ai/anthropic @zhivex-ai/openai @zhivex-ai/ollama @zhivex-ai/vertex
 ```
 
-Install the provider packages used by your own adapter map; the examples below use OpenAI, Ollama and Anthropic.
+Install the provider packages used by your own adapter map; these examples use OpenAI, Ollama, Anthropic and Vertex.
 
 ## Usage
 
@@ -75,7 +103,7 @@ The gateway also supports `streamText()`, `generateObject()`, and `streamObject(
 - Requests containing image attachments only route to models that declare `capabilities.vision: true`. The gateway never removes images to make a target appear compatible; if one target cannot accept the original request, it is skipped in favor of a compatible fallback.
 - `scoreTarget(context)` can replace the built-in name-based heuristic with application metrics. It must return a finite number; higher scores route first.
 - Routing amplification is bounded even if a request is assembled from external input: requests accept at most `maxFallbacks` targets (default 8, hard maximum 32), `maxRetries` cannot exceed 5, and `maxTotalAttempts` caps provider calls across the whole routed operation including later agent steps (default 32, hard maximum 128). Model IDs are non-empty, limited to 256 characters, and cannot contain control characters. `maxCostPer1kTokens` and configured/catalog costs must be finite and non-negative.
-- `onAttempt` and `onAgentRoute` are best-effort observers. They receive an `abortSignal` and are allowed `observerTimeoutMs` to finish (default 1 second); rejection, timeout, or request cancellation cannot retry successful provider work or block routing indefinitely.
+- `onAttempt` and `onAgentRoute` are best-effort observers. Legacy callbacks run detached; `observerMode: "await"` explicitly awaits attempt callbacks and `"background"` queues them with bounded capacity. They receive an `abortSignal` and are allowed `observerTimeoutMs` to finish (default 1 second); rejection, timeout, or request cancellation cannot retry successful provider work or block routing indefinitely.
 
 Keep primary/fallback selection and these ceilings under application control when mapping an HTTP request into `GatewayRequest`. `maxCostPer1kTokens` limits model price, not the final invoice; use `maxTotalAttempts`, provider-side spend limits, authentication, and rate limiting for a complete cost boundary.
 
@@ -160,7 +188,7 @@ messages. System messages must precede conversation turns.
 | --- | --- |
 | `generate`, `streamText` | Supported, including usage estimates, attempts, finish reasons, cancellation and routing |
 | `generateObject`, `streamObject` | Same input validation and routing; destination must also support the requested object mode |
-| `runAgent`, `streamAgent` | Legacy `GatewayMessage[]` only; canonical input is explicitly rejected |
+| `runAgent`, `streamAgent` | Fresh canonical history imports; durable resumes use state/runId without resupplying messages |
 | Anthropic Messages | Text, user images, assistant tool calls, tool results and native `is_error` |
 | OpenAI Chat / Responses | Callable tool history, including multiple results; explicit JSON success/error envelopes |
 | DeepSeek Chat | Same JSON envelope support; portable replay defaults to non-thinking; reasoning-only models are excluded |
@@ -209,3 +237,220 @@ candidate core/gateway/provider tarballs without checkout links. Add
 authorized credential and perform two bounded live calls for one provider. A missing credential is a blocked live check, not
 a passing certification. Publication evidence and the actual version are recorded
 in [the delivery record](../../docs/GATEWAY_TOOL_HISTORY_DELIVERY.md).
+
+## Usage accounting
+
+`generate`, `generateObject`, `streamText().collect()` and `streamObject().collect()` preserve all reported `TokenUsage` fields, including `cachedInputTokens`, `cacheWriteTokens`, `reasoningTokens` and `speed`. Reported zeros remain zeros; missing optional details remain absent. Only missing `inputTokens`/`outputTokens` are estimated from text and a missing `totalTokens` is derived from those base counters. `estimated` is true if any of those three base counters was missing, including a derived total. Reasoning/cache details are never added again to the total. Agent results retain their existing usage aggregation without an `estimated` flag.
+
+## Detailed cost valuation
+
+Opt in with `costAccounting: { unknownCostPolicy: "allow" }` and an immutable `modelCatalog` carrying pricing metadata. `routeDecision.estimatedCosts` contains cold-cache preflight estimates from input length and `expectedOutputTokens` (or request `maxTokens`). They are estimates, not spending limits. With `unknownCostPolicy: "reject"`, destinations with unknown quotes are excluded before a call. The existing `maxCostPer1kTokens` budget remains independent.
+
+Each actual `attempt` carries its reported `usage` and `cost`, including provider/model, currency, catalog/pricing versions, optional provenance source, components, assumptions and unknown reasons. A failed attempt without usage has `amount: null`; a successful fallback does not make that failure free. Sum amounts only when every actual attempt is known or explicitly estimated; otherwise total cost remains unknown. Streams use terminal usage, and partial stream failures preserve any usage already received.
+
+`calculateModelCost()` is also exported from Core and SDK for standalone valuation. Input tokens must include cached reads and cache writes; these are subtracted from ordinary input before applying their separate prices. Missing cache counters remain unknown unless `cacheAssumption: "none"` explicitly assumes zeros. Missing relevant prices or base usage never become a free estimate. Long-context multipliers apply above the catalog threshold to all input classes and output. Zero tokens incur zero cost even when that unused category has no rate; its rate remains null.
+
+Reasoning semantics vary between adapters. Set `reasoningAccounting: { openai: "included", gemini: "additional" }` only for the provider protocols you have verified: included leaves the output count unchanged, additional adds reasoning once. Without an explicit setting, a positive reasoning counter produces an unknown valuation. For standalone valuation this setting is a single string. The catalog does not describe fast-tier pricing, so `speed: "fast"` remains unknown. Invalid reported counters produce unknown accounting without retrying a successful model call. This API is neither billing nor a strict monetary budget.
+
+## Local destination metrics
+
+Pass `metrics: createGatewayMetrics({ windowMs: 60_000, maxTargets: 128, maxSamplesPerTarget: 256 })` to opt in. Query `metrics.snapshot({ provider, modelId })` for in-flight count, recent successes/errors/cancellations, full-attempt p50/p95 and first-text p95. Cold destinations return undefined; expired samples have no latency estimates. Client cancellation is excluded from provider error and latency samples. The clock is injectable through `now` for deterministic tests.
+
+Storage is bounded by targets and samples. The least recently touched idle destination is evicted when full; if all target slots are active, a new destination remains untracked until capacity is available. No prompts, outputs, error payloads or credentials are retained. The default store is local to the process; applications may inject the synchronous `GatewayMetricsStore` contract. Hook failures are best effort. Do not put expensive synchronous work in a metrics hook.
+
+With metrics enabled, explicitly returning an event/text/object stream iterator cancels that routed operation and releases the in-flight slot. This applies to all consumers of that operation, including collect. Consume the stream fully if you also need its final result. Caller abort and timeout also release slots even when a provider ignores its signal. An operation with no explicit return/abort continues until completion or timeout; the SDK cannot detect garbage collection as abandonment.
+
+## Circuit breaker and adaptive routing
+
+Pass `circuitBreaker: createGatewayCircuitBreaker({ failureThreshold: 5, cooldownMs: 30_000 })` to opt in to local closed/open/half-open circuits. Retryable errors (including 429/5xx and attempt timeouts) accumulate; validation, authentication and client cancellation do not. Open circuits receive no calls. Recovery allows one concurrent probe by default; successful probes close the circuit, failed probes reopen it. Late results from an older circuit epoch cannot close a newly opened circuit. When all eligible circuits are unavailable the operation raises `GatewayCircuitOpenError`.
+
+Cooldown is `min(maxCooldownMs, max(cooldownMs, Retry-After))`, with defaults of 60 seconds maximum and 30 seconds minimum. The circuit never sleeps until recovery: it skips the target. Existing bounded retry backoff applies only while the circuit remains closed. Target storage is bounded; closed idle entries may be evicted, while capacity exhausted by active/open entries refuses admission. State observers receive only target IDs, state and timestamps, and run best effort. Metrics/circuit-enabled stream iterator return cancels the operation. Neither feature coordinates across processes.
+
+`adaptiveRouting` is an explicit alternative to legacy scoring. Supply a policy version, nonnegative weights for `latency`, `cost`, `quality`, `load`, `errorRate`, positive `latencyScaleMs`/`costScale`, and explicit `coldStart`, `unknownCost`, `missingQuality` policies (`allow`/`reject`). Quality profiles carry a target, task `intent`, score in [0,1] and evaluation version. There are no model-name quality heuristics in this mode.
+
+The score is quality reward minus normalized p95 latency, estimated request cost, in-flight load and observed error rate penalties. Enable `metrics` and `costAccounting` for those signals. Missing penalty signals under `allow` receive a conservative normalized penalty (default 1), missing quality/throughput receive no reward, and all remain listed as missing; `reject` excludes the destination. Expired samples become cold start. Capabilities and circuits are filtered before ordering, and circuits are rechecked atomically before each call. Ties preserve primary/fallback input order. `routeDecision.adaptive` records policy version, signals, profile versions and exclusions. Do not configure both `scoreTarget` and `adaptiveRouting`. These rules are transparent heuristics, not a claim of globally optimal routing.
+
+## Composing configured agents
+
+Pass `agent: configuredAgent` to `runAgent` or `streamAgent`. The gateway substitutes a routed LanguageModel and uses the existing Core runtime. Context schema, guardrails, output schema, compactor, approval signer, subagents, harness binding, environment, hooks and store remain on the definition. Supply ephemeral `context` on each invocation. Defined request options override definition defaults; policy and metadata merge shallowly. `compaction: false` explicitly disables the default for an invocation. The original definition is not mutated.
+
+Configured gateway runs persist a `gatewayAgentRouteBinding` metadata value for agent ID, primary/fallback targets, routing policy version and harness fingerprint. Resume must retain that binding; changing it fails before provider work. An older direct run without this binding requires a separately designed migration instead of silent adoption. Core still validates harness/environment fingerprints. Reserve `gatewayAgentRouteBinding` and `gatewayPortableHistory` for runtime use.
+
+For canonical history, import a fresh run with `messages` only. Do not combine import with prompt, state, runId, handoff, approvals or idempotencyKey. Resolved historical tools are input, never effects to replay. Then resume with store/runId or state and approvals as needed, without messages. Portable-history capability checks survive store reloads and compaction. Legacy inputs remain supported. Provider-specific approval data stays in durable state; it is not part of the portable history import format.
+
+
+## Production controls
+
+All controls below are optional and compose with text, object and agent routing.
+They do not require provider-specific changes. Existing default adapter maps and
+legacy scoring remain supported. Adaptive missing-data penalties are intentionally
+more conservative than the previous zero-penalty behavior.
+
+```ts
+import {
+  createGateway, createGatewayAdmissionController, createGatewayBudgetStore,
+  createGatewayMetrics, createGatewayRoutingPolicy
+} from "@zhivex-ai/gateway";
+import { createInMemoryGenerateCache, createModelCatalog } from "@zhivex-ai/core";
+import { createOpenAI } from "@zhivex-ai/openai";
+
+// Supply a verified catalog snapshot for your actual models before enabling cost routing.
+const catalog = createModelCatalog([{
+  provider: "openai", modelId: "your-model",
+  inputCostPer1kTokens: 0.001, outputCostPer1kTokens: 0.003
+}], { snapshotVersion: "example-only", pricing: {
+  version: "example-only", currency: "USD", unit: "per_1k_tokens"
+} });
+const budgets = createGatewayBudgetStore({ limit: 20, currency: "USD" });
+const gateway = createGateway({
+  adapters: { openai: createOpenAI() },
+  modelCatalog: catalog,
+  costAccounting: { unknownCostPolicy: "reject", cacheAssumption: "none" },
+  timeoutMs: 20_000,
+  metrics: createGatewayMetrics(),
+  adaptiveRouting: createGatewayRoutingPolicy("interactive"),
+  admission: createGatewayAdmissionController({
+    maxConcurrent: 16, requestsPerMinute: 600, tokensPerMinute: 100_000,
+    maxQueue: 32, queueTimeoutMs: 500
+  }),
+  budget: { store: budgets, currency: "USD", reserveAmount: 0.10 },
+  cache: { store: createInMemoryGenerateCache(), scope: "credential-revision-1" },
+  affinity: { ttlMs: 300_000, maxEntries: 1000, maxScoreLoss: 0.1 },
+  observerMode: "background", observerQueueCapacity: 256
+});
+const result = await gateway.generate({
+  primary: { provider: "openai", modelId: "your-model" },
+  messages: [{ role: "user", content: "Explain binary search." }],
+  maxTokens: 200, budgetScope: "tenant-123:2026-09", cacheScope: "tenant-123",
+  affinityKey: "conversation-456"
+});
+await gateway.flushObservers();
+await gateway.flushControls();
+console.log(result.text, gateway.diagnostics(), budgets.snapshot("tenant-123:2026-09"));
+```
+
+### Deadline and admission
+
+`timeoutMs` covers one invocation, including queueing, retries, backoff, tools,
+observers and streamed output. A request may shorten the configured deadline but
+cannot extend it. Expiration raises `GatewayDeadlineError` and aborts downstream
+signals. Providers/tools must cooperate to stop external effects; ignoring abort
+cannot keep the gateway response pending indefinitely. Durable resumes start a
+new invocation deadline. Normal completion removes timers and signal listeners.
+
+Admission is per provider/model/deployment and happens before every upstream
+call, including retries and later agent steps. Concurrency slots are released on
+completion, abort or stream iterator return. The global queue is bounded and
+waits at most `queueTimeoutMs`; the local implementation polls at up to 10 ms
+intervals and does not promise strict FIFO fairness. Capacity denial skips to an
+eligible fallback without marking the destination unhealthy.
+
+RPM and TPM use fixed 60-second windows. TPM reserves serialized input length / 4
+plus `maxTokens`, and requires `maxTokens`; this is an estimate, not provider token
+parity. Reported total tokens reconcile reservations in the same window; unknown
+usage keeps the reservation. RPM is not refunded after admission. Capacity
+pressure never evicts an unexpired quota window. Set upstream provider limits as
+the final boundary for usage the provider may continue after cancellation.
+
+### Monetary reservations
+
+Each dispatched model attempt reserves `budget.reserveAmount` in the trusted
+`budgetScope`. Successful known usage reconciles that amount against catalog
+pricing; failed, cancelled, partial or unpriceable usage retains the reservation
+as uncertain spend. No reservation is made for an exact cache hit. Budget denial
+stops the operation without retry or fallback. The configured currency must
+match the catalog valuation to release any unused reservation.
+
+This limits **admitted reservations**, not the exact provider invoice. Choose a
+conservative per-attempt reservation alongside `maxTokens`; actual reported cost
+can exceed the reservation and is recorded fully, blocking subsequent work when
+the scope is exhausted. Unknown spend remains blocked until the application
+reconciles its accounting externally. Local scopes never evict or automatically
+reset; use explicit period scopes and bounded `maxScopes`. Do not use arbitrary
+client-supplied scope values. For authoritative billing, implement the store
+against the service's durable ledger.
+
+`GatewayBudgetStore` and `GatewayAdmissionController` are asynchronous injection
+contracts for shared backends. The supplied factories are process-local; they do
+not implement Redis or cross-process coordination. Remote implementations must
+reserve atomically, expire abandoned concurrency leases, honor abort and make
+settlement/release idempotent. Settlement runs outside the provider response path. Failed or timed-out settlement
+cannot retry successful provider work; diagnostics report it and further budgeted dispatches fail closed on that
+gateway instance. Recover the backend/ledger before replacing the instance.
+
+### Adaptive policies and destination identity
+
+`createGatewayRoutingPolicy("interactive" | "economy" | "quality")` returns an
+explicit versioned starting policy. Configure quality profiles for economy and
+quality presets, which reject missing quality. Calibrate costs and latency scales
+on your workload; presets are not measured service-level guarantees.
+
+`latencyMetric: "ttft"` uses first-text latency; `"total"` retains full-attempt
+latency. Optional `weights.throughput` uses measured median output tokens/second
+with `throughputScale` (default 100). Throughput is an approximation using reported
+output tokens minus the first token and elapsed time after first text. Cache hits
+and cancellations do not supply provider latency evidence. `minSamples` prevents
+single samples from being treated as mature health evidence, and `minQuality`
+enforces an evaluated quality floor before ranking.
+
+Allowed missing latency/cost/load/error signals incur `missingSignalPenalty`
+(default 1, minimum 1). They are never represented as measured zero cost/latency.
+`explorationEvery` optionally probes one eligible cold destination every N
+operations; presets use 20. Exploration rotates candidates and cannot bypass
+capability, cost, quality-floor or circuit exclusions. It is local and deterministic,
+not a learned optimal policy.
+
+A target may supply `deploymentId`. Register it in
+`deployments: { "region-a": { provider: "openai", adapter } }` with an adapter
+configured for that endpoint, region and credential. Unknown or mismatched IDs
+are skipped without silently using the default adapter. Deduplication, metrics,
+circuits, quotas, quality profiles, cache partitions and attempt diagnostics
+include deployment identity. Catalog pricing still uses provider/model identity;
+deployment-specific negotiated tariffs require a separate pricing design.
+
+Optional affinity requires adaptive routing and both `cacheScope` and
+`affinityKey`. After a successful attempt, subsequent invocations may reuse that
+destination only if it remains eligible and within `maxScoreLoss` of the best
+score. Expired entries are ignored; capacity evicts the oldest stored affinity.
+Exploration takes precedence. This promotes provider prompt-cache locality; it
+does not prove an upstream cache hit or change provider caching parameters.
+
+### Exact cache and observer lifecycle
+
+Gateway caching reuses Core's canonical cache key middleware and configured store.
+It requires both a configured authentication scope and a per-request `cacheScope`;
+keys additionally include budget scope and deployment. Rotate the configured scope
+when credentials/endpoints change. Store TTL/retention belongs to the selected
+Core cache implementation. No semantic similarity cache is enabled.
+
+Only non-streaming text-only model steps without tools or `providerOptions` are
+eligible. Tool effects, portable tool history, images and provider state bypass
+this cache. Identical simultaneous misses share one upstream call (up to 1024
+active keys). One caller cancelling does not cancel the other subscribers; all
+subscribers leaving aborts the shared call. Cache reads time out after `cache.timeoutMs` (default 50 ms); writes are detached,
+bounded to 256 pending operations and use the same timeout. Cache failures do not
+retry successful provider work. `flushControls()` waits for the current cache-write
+and resource-settlement batch; resource settlements have `resourceTimeoutMs`
+(default 1000 ms). Pending counts and dropped cache writes are available in diagnostics. `attempt.cacheHit` identifies reuse, and detailed attempt valuation
+uses zero new upstream tokens; response usage still describes the reused output.
+A coalesced follower is also a reuse; correlate requests with your application
+ledger when attributing the originating call's cost.
+
+The legacy observer behavior stays detached. `observerMode: "background"` adds a
+bounded FIFO attempt queue, `diagnostics().droppedObservers`, and
+`flushObservers()` for the currently queued batch. Every callback remains bounded
+by `observerTimeoutMs`. `"await"` explicitly awaits attempt observers. None of
+these modes can preempt synchronous CPU work inside a callback. Route-selection
+observers keep their existing detached behavior.
+
+
+### Verification
+
+Run `bun run test packages/gateway/tests` for routing and resource-lifetime
+regressions. After `bun run build`, run
+`bun scripts/benchmarks/gateway-controls.mjs /tmp/gateway-controls.json` to compare
+local default/control overhead and verify exact-cache/shared-miss provider call
+counts. The benchmark asserts correct answers and actual fixture calls, includes
+runtime/source/artifact fingerprints, and explicitly excludes live provider
+performance or competitive claims. Its five measured trials are a smoke baseline,
+not a statistically reliable production tail-latency estimate.

package/dist/adaptive-routing.d.ts

@@ -0,0 +1,52 @@
+import { type GatewayModelTarget, type GatewayTaskIntent } from "./types.js";
+import type { GatewayMetricsSnapshot } from "./metrics.js";
+import type { ModelCostValuation } from "@zhivex-ai/core";
+export interface GatewayQualityProfile {
+    target: GatewayModelTarget;
+    intent: GatewayTaskIntent;
+    score: number;
+    version: string;
+}
+export interface GatewayAdaptiveRoutingPolicy {
+    version: string;
+    /** Select latency relevant to the workload; full attempt is the legacy default. */
+    latencyMetric?: "total" | "ttft";
+    minSamples?: number;
+    /** Probe one allowed cold destination every N operations. Disabled unless configured. */
+    explorationEvery?: number;
+    throughputScale?: number;
+    /** Conservative normalized penalties used for allowed missing signals. */
+    missingSignalPenalty?: number;
+    minQuality?: number;
+    weights: {
+        latency: number;
+        cost: number;
+        quality: number;
+        load: number;
+        errorRate: number;
+        throughput?: number;
+    };
+    latencyScaleMs: number;
+    costScale: number;
+    coldStart: "allow" | "reject";
+    unknownCost: "allow" | "reject";
+    missingQuality: "allow" | "reject";
+    qualityProfiles?: GatewayQualityProfile[];
+}
+export interface GatewayAdaptiveCandidate {
+    target: GatewayModelTarget;
+    score?: number;
+    exclusions: string[];
+    missingSignals: string[];
+    metrics?: GatewayMetricsSnapshot;
+    cost?: ModelCostValuation;
+    quality?: {
+        score: number;
+        version: string;
+    };
+}
+export declare const validateAdaptivePolicy: (policy: GatewayAdaptiveRoutingPolicy) => void;
+export declare const scoreAdaptiveTarget: (policy: GatewayAdaptiveRoutingPolicy, target: GatewayModelTarget, intent: GatewayTaskIntent, metrics?: GatewayMetricsSnapshot, cost?: ModelCostValuation) => GatewayAdaptiveCandidate;
+/** Explicit starting policies. Calibrate scales and quality profiles against your own workloads. */
+export declare const createGatewayRoutingPolicy: (preset: "interactive" | "economy" | "quality", overrides?: Partial<GatewayAdaptiveRoutingPolicy>) => GatewayAdaptiveRoutingPolicy;
+//# sourceMappingURL=adaptive-routing.d.ts.map

package/dist/adaptive-routing.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"adaptive-routing.d.ts","sourceRoot":"","sources":["../src/adaptive-routing.ts"],"names":[],"mappings":"AACA,OAAO,EAAgB,KAAK,kBAAkB,EAAE,KAAK,iBAAiB,EAAE,MAAM,YAAY,CAAC;AAC3F,OAAO,KAAK,EAAE,sBAAsB,EAAE,MAAM,cAAc,CAAC;AAC3D,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,iBAAiB,CAAC;AAC1D,MAAM,WAAW,qBAAqB;IAAG,MAAM,EAAE,kBAAkB,CAAC;IAAC,MAAM,EAAE,iBAAiB,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,OAAO,EAAE,MAAM,CAAC;CAAE;AACjI,MAAM,WAAW,4BAA4B;IAC3C,OAAO,EAAE,MAAM,CAAC;IAChB,mFAAmF;IACnF,aAAa,CAAC,EAAE,OAAO,GAAG,MAAM,CAAC;IACjC,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,yFAAyF;IACzF,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,0EAA0E;IAC1E,oBAAoB,CAAC,EAAE,MAAM,CAAC;IAC9B,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,OAAO,EAAE;QAAE,OAAO,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,SAAS,EAAE,MAAM,CAAC;QAAC,UAAU,CAAC,EAAE,MAAM,CAAA;KAAE,CAAC;IAClH,cAAc,EAAE,MAAM,CAAC;IACvB,SAAS,EAAE,MAAM,CAAC;IAClB,SAAS,EAAE,OAAO,GAAG,QAAQ,CAAC;IAC9B,WAAW,EAAE,OAAO,GAAG,QAAQ,CAAC;IAChC,cAAc,EAAE,OAAO,GAAG,QAAQ,CAAC;IACnC,eAAe,CAAC,EAAE,qBAAqB,EAAE,CAAC;CAC3C;AACD,MAAM,WAAW,wBAAwB;IACvC,MAAM,EAAE,kBAAkB,CAAC;IAC3B,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,UAAU,EAAE,MAAM,EAAE,CAAC;IACrB,cAAc,EAAE,MAAM,EAAE,CAAC;IACzB,OAAO,CAAC,EAAE,sBAAsB,CAAC;IACjC,IAAI,CAAC,EAAE,kBAAkB,CAAC;IAC1B,OAAO,CAAC,EAAE;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,OAAO,EAAE,MAAM,CAAA;KAAE,CAAC;CAC9C;AACD,eAAO,MAAM,sBAAsB,WAAY,4BAA4B,SAkB1E,CAAC;AACF,eAAO,MAAM,mBAAmB,WAAY,4BAA4B,UAAU,kBAAkB,UAAU,iBAAiB,YAAY,sBAAsB,SAAS,kBAAkB,KAAG,wBAwB9L,CAAC;AAEF,oGAAoG;AACpG,eAAO,MAAM,0BAA0B,WAAY,aAAa,GAAG,SAAS,GAAG,SAAS,cAAa,OAAO,CAAC,4BAA4B,CAAC,KAAQ,4BAWjJ,CAAC"}

package/dist/adaptive-routing.js

@@ -0,0 +1,84 @@
+import { targetKey, sameTarget } from "./target.js";
+import { GatewayError } from "./types.js";
+export const validateAdaptivePolicy = (policy) => {
+    if (!policy.version?.trim())
+        throw new GatewayError("Adaptive policy requires a version.", false);
+    if (![policy.latencyScaleMs, policy.costScale].every(x => Number.isFinite(x) && x > 0))
+        throw new GatewayError("Adaptive scales must be positive.", false);
+    const values = [policy.weights?.latency, policy.weights?.cost, policy.weights?.quality, policy.weights?.load, policy.weights?.errorRate, policy.weights?.throughput ?? 0];
+    if (!values.every(x => Number.isFinite(x) && x >= 0) || values.every(x => x === 0))
+        throw new GatewayError("Adaptive weights must be nonnegative with at least one positive weight.", false);
+    if (![policy.coldStart, policy.unknownCost, policy.missingQuality].every(x => x === "allow" || x === "reject"))
+        throw new GatewayError("Adaptive missing-data policies must be explicit.", false);
+    if (policy.latencyMetric !== undefined && !["total", "ttft"].includes(policy.latencyMetric))
+        throw new GatewayError("Invalid latencyMetric.", false);
+    if (policy.minSamples !== undefined && (!Number.isSafeInteger(policy.minSamples) || policy.minSamples < 1))
+        throw new GatewayError("Invalid minSamples.", false);
+    if (policy.missingSignalPenalty !== undefined && (!Number.isFinite(policy.missingSignalPenalty) || policy.missingSignalPenalty < 1))
+        throw new GatewayError("missingSignalPenalty must be at least one.", false);
+    if (policy.minQuality !== undefined && (!Number.isFinite(policy.minQuality) || policy.minQuality < 0 || policy.minQuality > 1))
+        throw new GatewayError("Invalid minQuality.", false);
+    if (policy.throughputScale !== undefined && (!Number.isFinite(policy.throughputScale) || policy.throughputScale <= 0))
+        throw new GatewayError("Invalid throughputScale.", false);
+    if (policy.explorationEvery !== undefined && (!Number.isSafeInteger(policy.explorationEvery) || policy.explorationEvery < 2))
+        throw new GatewayError("explorationEvery must be at least two.", false);
+    const keys = new Set();
+    for (const profile of policy.qualityProfiles ?? []) {
+        const key = JSON.stringify([targetKey(profile.target), profile.intent]);
+        if (keys.has(key) || !profile.version?.trim() || !Number.isFinite(profile.score) || profile.score < 0 || profile.score > 1)
+            throw new GatewayError("Invalid or ambiguous quality profile.", false);
+        keys.add(key);
+    }
+};
+export const scoreAdaptiveTarget = (policy, target, intent, metrics, cost) => {
+    const result = { target: { ...target }, exclusions: [], missingSignals: [], ...(metrics ? { metrics } : {}), ...(cost ? { cost } : {}) };
+    const quality = policy.qualityProfiles?.find(x => sameTarget(x.target, target) && x.intent === intent);
+    if (quality)
+        result.quality = { score: quality.score, version: quality.version };
+    const missing = (signal, action) => { result.missingSignals.push(signal); if (action === "reject")
+        result.exclusions.push(`${signal}-unavailable`); };
+    const healthy = !!metrics && metrics.successes + metrics.errors >= (policy.minSamples ?? 1);
+    const latency = policy.latencyMetric === "ttft" ? metrics?.p95TtftMs : metrics?.p95LatencyMs;
+    if ((policy.weights.latency || policy.weights.errorRate) && !healthy)
+        missing("health", policy.coldStart);
+    if (policy.weights.latency && healthy && latency === undefined)
+        missing("latency", policy.coldStart);
+    if (policy.weights.throughput && (!healthy || metrics?.p50TokensPerSecond === undefined))
+        missing("throughput", policy.coldStart);
+    if (policy.weights.load && !metrics)
+        missing("load", policy.coldStart);
+    if (policy.weights.cost && (!cost || cost.amount === null))
+        missing("cost", policy.unknownCost);
+    if (policy.weights.quality && !quality)
+        missing("quality", policy.missingQuality);
+    if (policy.minQuality !== undefined && (!quality || quality.score < policy.minQuality))
+        result.exclusions.push("quality-floor");
+    if (result.exclusions.length)
+        return result;
+    const penalty = policy.missingSignalPenalty ?? 1;
+    const errorRate = healthy ? metrics.errors / (metrics.successes + metrics.errors) : penalty;
+    const score = (quality?.score ?? 0) * policy.weights.quality
+        + (healthy ? metrics?.p50TokensPerSecond ?? 0 : 0) / (policy.throughputScale ?? 100) * (policy.weights.throughput ?? 0)
+        - (healthy && latency !== undefined ? latency / policy.latencyScaleMs : penalty) * policy.weights.latency
+        - (cost?.amount != null ? cost.amount / policy.costScale : penalty) * policy.weights.cost
+        - (metrics?.inFlight ?? penalty) * policy.weights.load - errorRate * policy.weights.errorRate;
+    if (!Number.isFinite(score))
+        throw new GatewayError("Adaptive signals produced a nonfinite score.", false);
+    result.score = score;
+    return result;
+};
+/** Explicit starting policies. Calibrate scales and quality profiles against your own workloads. */
+export const createGatewayRoutingPolicy = (preset, overrides = {}) => {
+    const policy = {
+        version: `gateway-${preset}-v1`, latencyMetric: preset === "interactive" ? "ttft" : "total",
+        latencyScaleMs: 1000, costScale: 0.01, minSamples: 5, explorationEvery: 20,
+        coldStart: "allow", unknownCost: "reject", missingQuality: "reject",
+        weights: preset === "interactive" ? { latency: 2, cost: 1, quality: 0, load: 1, errorRate: 3 }
+            : preset === "economy" ? { latency: .25, cost: 3, quality: 1, load: 1, errorRate: 3 }
+                : { latency: .25, cost: .25, quality: 3, load: 1, errorRate: 3 },
+        ...overrides
+    };
+    validateAdaptivePolicy(policy);
+    return policy;
+};
+//# sourceMappingURL=adaptive-routing.js.map

package/dist/adaptive-routing.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"adaptive-routing.js","sourceRoot":"","sources":["../src/adaptive-routing.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,SAAS,EAAE,UAAU,EAAE,MAAM,aAAa,CAAC;AACpD,OAAO,EAAE,YAAY,EAAmD,MAAM,YAAY,CAAC;AAgC3F,MAAM,CAAC,MAAM,sBAAsB,GAAG,CAAC,MAAoC,EAAE,EAAE;IAC7E,IAAI,CAAC,MAAM,CAAC,OAAO,EAAE,IAAI,EAAE;QAAE,MAAM,IAAI,YAAY,CAAC,qCAAqC,EAAE,KAAK,CAAC,CAAC;IAClG,IAAI,CAAC,CAAC,MAAM,CAAC,cAAc,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,mCAAmC,EAAE,KAAK,CAAC,CAAC;IAC3J,MAAM,MAAM,GAAG,CAAC,MAAM,CAAC,OAAO,EAAE,OAAO,EAAE,MAAM,CAAC,OAAO,EAAE,IAAI,EAAE,MAAM,CAAC,OAAO,EAAE,OAAO,EAAE,MAAM,CAAC,OAAO,EAAE,IAAI,EAAE,MAAM,CAAC,OAAO,EAAE,SAAS,EAAE,MAAM,CAAC,OAAO,EAAE,UAAU,IAAI,CAAC,CAAC,CAAC;IAC1K,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,yEAAyE,EAAE,KAAK,CAAC,CAAC;IAC7L,IAAI,CAAC,CAAC,MAAM,CAAC,SAAS,EAAE,MAAM,CAAC,WAAW,EAAE,MAAM,CAAC,cAAc,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,KAAK,OAAO,IAAI,CAAC,KAAK,QAAQ,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,kDAAkD,EAAE,KAAK,CAAC,CAAC;IAClM,IAAI,MAAM,CAAC,aAAa,KAAK,SAAS,IAAI,CAAC,CAAC,OAAO,EAAE,MAAM,CAAC,CAAC,QAAQ,CAAC,MAAM,CAAC,aAAa,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,wBAAwB,EAAE,KAAK,CAAC,CAAC;IACrJ,IAAI,MAAM,CAAC,UAAU,KAAK,SAAS,IAAI,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC,MAAM,CAAC,UAAU,CAAC,IAAI,MAAM,CAAC,UAAU,GAAG,CAAC,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,qBAAqB,EAAE,KAAK,CAAC,CAAC;IACjK,IAAI,MAAM,CAAC,oBAAoB,KAAK,SAAS,IAAI,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,oBAAoB,CAAC,IAAI,MAAM,CAAC,oBAAoB,GAAG,CAAC,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,4CAA4C,EAAE,KAAK,CAAC,CAAC;IACjN,IAAI,MAAM,CAAC,UAAU,KAAK,SAAS,IAAI,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,UAAU,CAAC,IAAI,MAAM,CAAC,UAAU,GAAG,CAAC,IAAI,MAAM,CAAC,UAAU,GAAG,CAAC,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,qBAAqB,EAAE,KAAK,CAAC,CAAC;IACrL,IAAI,MAAM,CAAC,eAAe,KAAK,SAAS,IAAI,CAAC,CAAC,MAAM,CAAC,QAAQ,CAAC,MAAM,CAAC,eAAe,CAAC,IAAI,MAAM,CAAC,eAAe,IAAI,CAAC,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,0BAA0B,EAAE,KAAK,CAAC,CAAC;IACjL,IAAI,MAAM,CAAC,gBAAgB,KAAK,SAAS,IAAI,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC,MAAM,CAAC,gBAAgB,CAAC,IAAI,MAAM,CAAC,gBAAgB,GAAG,CAAC,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,wCAAwC,EAAE,KAAK,CAAC,CAAC;IACtM,MAAM,IAAI,GAAG,IAAI,GAAG,EAAU,CAAC;IAC/B,KAAK,MAAM,OAAO,IAAI,MAAM,CAAC,eAAe,IAAI,EAAE,EAAE,CAAC;QACnD,MAAM,GAAG,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,SAAS,CAAC,OAAO,CAAC,MAAM,CAAC,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;QACxE,IAAI,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,OAAO,CAAC,OAAO,EAAE,IAAI,EAAE,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,OAAO,CAAC,KAAK,GAAG,CAAC,IAAI,OAAO,CAAC,KAAK,GAAG,CAAC;YAAE,MAAM,IAAI,YAAY,CAAC,uCAAuC,EAAE,KAAK,CAAC,CAAC;QACnM,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IAChB,CAAC;AACH,CAAC,CAAC;AACF,MAAM,CAAC,MAAM,mBAAmB,GAAG,CAAC,MAAoC,EAAE,MAA0B,EAAE,MAAyB,EAAE,OAAgC,EAAE,IAAyB,EAA4B,EAAE;IACxN,MAAM,MAAM,GAA6B,EAAE,MAAM,EAAE,EAAE,GAAG,MAAM,EAAE,EAAE,UAAU,EAAE,EAAE,EAAE,cAAc,EAAE,EAAE,EAAE,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC;IACnK,MAAM,OAAO,GAAG,MAAM,CAAC,eAAe,EAAE,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,UAAU,CAAC,CAAC,CAAC,MAAM,EAAE,MAAM,CAAC,IAAI,CAAC,CAAC,MAAM,KAAK,MAAM,CAAC,CAAC;IACvG,IAAI,OAAO;QAAE,MAAM,CAAC,OAAO,GAAG,EAAE,KAAK,EAAE,OAAO,CAAC,KAAK,EAAE,OAAO,EAAE,OAAO,CAAC,OAAO,EAAE,CAAC;IACjF,MAAM,OAAO,GAAG,CAAC,MAAc,EAAE,MAA0B,EAAE,EAAE,GAAG,MAAM,CAAC,cAAc,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,MAAM,KAAK,QAAQ;QAAE,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC,GAAG,MAAM,cAAc,CAAC,CAAC,CAAC,CAAC,CAAC;IAClL,MAAM,OAAO,GAAG,CAAC,CAAC,OAAO,IAAI,OAAO,CAAC,SAAS,GAAG,OAAO,CAAC,MAAM,IAAI,CAAC,MAAM,CAAC,UAAU,IAAI,CAAC,CAAC,CAAC;IAC5F,MAAM,OAAO,GAAG,MAAM,CAAC,aAAa,KAAK,MAAM,CAAC,CAAC,CAAC,OAAO,EAAE,SAAS,CAAC,CAAC,CAAC,OAAO,EAAE,YAAY,CAAC;IAC7F,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,OAAO,IAAI,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,IAAI,CAAC,OAAO;QAAE,OAAO,CAAC,QAAQ,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;IAC1G,IAAI,MAAM,CAAC,OAAO,CAAC,OAAO,IAAI,OAAO,IAAI,OAAO,KAAK,SAAS;QAAE,OAAO,CAAC,SAAS,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;IACrG,IAAI,MAAM,CAAC,OAAO,CAAC,UAAU,IAAI,CAAC,CAAC,OAAO,IAAI,OAAO,EAAE,kBAAkB,KAAK,SAAS,CAAC;QAAE,OAAO,CAAC,YAAY,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;IAClI,IAAI,MAAM,CAAC,OAAO,CAAC,IAAI,IAAI,CAAC,OAAO;QAAE,OAAO,CAAC,MAAM,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;IACvE,IAAI,MAAM,CAAC,OAAO,CAAC,IAAI,IAAI,CAAC,CAAC,IAAI,IAAI,IAAI,CAAC,MAAM,KAAK,IAAI,CAAC;QAAE,OAAO,CAAC,MAAM,EAAE,MAAM,CAAC,WAAW,CAAC,CAAC;IAChG,IAAI,MAAM,CAAC,OAAO,CAAC,OAAO,IAAI,CAAC,OAAO;QAAE,OAAO,CAAC,SAAS,EAAE,MAAM,CAAC,cAAc,CAAC,CAAC;IAClF,IAAI,MAAM,CAAC,UAAU,KAAK,SAAS,IAAI,CAAC,CAAC,OAAO,IAAI,OAAO,CAAC,KAAK,GAAG,MAAM,CAAC,UAAU,CAAC;QAAE,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC,eAAe,CAAC,CAAC;IAChI,IAAI,MAAM,CAAC,UAAU,CAAC,MAAM;QAAE,OAAO,MAAM,CAAC;IAC5C,MAAM,OAAO,GAAG,MAAM,CAAC,oBAAoB,IAAI,CAAC,CAAC;IACjD,MAAM,SAAS,GAAG,OAAO,CAAC,CAAC,CAAC,OAAQ,CAAC,MAAM,GAAG,CAAC,OAAQ,CAAC,SAAS,GAAG,OAAQ,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC;IAC/F,MAAM,KAAK,GAAG,CAAC,OAAO,EAAE,KAAK,IAAI,CAAC,CAAC,GAAG,MAAM,CAAC,OAAO,CAAC,OAAO;UACxD,CAAC,OAAO,CAAC,CAAC,CAAC,OAAO,EAAE,kBAAkB,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,eAAe,IAAI,GAAG,CAAC,GAAG,CAAC,MAAM,CAAC,OAAO,CAAC,UAAU,IAAI,CAAC,CAAC;UACrH,CAAC,OAAO,IAAI,OAAO,KAAK,SAAS,CAAC,CAAC,CAAC,OAAO,GAAG,MAAM,CAAC,cAAc,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,MAAM,CAAC,OAAO,CAAC,OAAO;UACvG,CAAC,IAAI,EAAE,MAAM,IAAI,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC,SAAS,CAAC,CAAC,CAAC,OAAO,CAAC,GAAG,MAAM,CAAC,OAAO,CAAC,IAAI;UACvF,CAAC,OAAO,EAAE,QAAQ,IAAI,OAAO,CAAC,GAAG,MAAM,CAAC,OAAO,CAAC,IAAI,GAAG,SAAS,GAAG,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC;IAChG,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,8CAA8C,EAAE,KAAK,CAAC,CAAC;IAC3G,MAAM,CAAC,KAAK,GAAG,KAAK,CAAC;IAAC,OAAO,MAAM,CAAC;AACtC,CAAC,CAAC;AAEF,oGAAoG;AACpG,MAAM,CAAC,MAAM,0BAA0B,GAAG,CAAC,MAA6C,EAAE,SAAS,GAA0C,EAAE,EAAgC,EAAE;IAC/K,MAAM,MAAM,GAAiC;QAC3C,OAAO,EAAE,WAAW,MAAM,KAAK,EAAE,aAAa,EAAE,MAAM,KAAK,aAAa,CAAC,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,OAAO;QAC3F,cAAc,EAAE,IAAI,EAAE,SAAS,EAAE,IAAI,EAAE,UAAU,EAAE,CAAC,EAAE,gBAAgB,EAAE,EAAE;QAC1E,SAAS,EAAE,OAAO,EAAE,WAAW,EAAE,QAAQ,EAAE,cAAc,EAAE,QAAQ;QACnE,OAAO,EAAE,MAAM,KAAK,aAAa,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,SAAS,EAAE,CAAC,EAAE;YAC5F,CAAC,CAAC,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,GAAG,EAAE,IAAI,EAAE,CAAC,EAAE,OAAO,EAAE,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,SAAS,EAAE,CAAC,EAAE;gBACrF,CAAC,CAAC,EAAE,OAAO,EAAE,GAAG,EAAE,IAAI,EAAE,GAAG,EAAE,OAAO,EAAE,CAAC,EAAE,IAAI,EAAE,CAAC,EAAE,SAAS,EAAE,CAAC,EAAE;QAClE,GAAG,SAAS;KACb,CAAC;IACF,sBAAsB,CAAC,MAAM,CAAC,CAAC;IAAC,OAAO,MAAM,CAAC;AAChD,CAAC,CAAC"}

package/dist/admission.d.ts

@@ -0,0 +1,26 @@
+import { GatewayError, type GatewayModelTarget } from "./types.js";
+export declare class GatewayAdmissionError extends GatewayError {
+    constructor();
+}
+export interface GatewayAdmissionLease {
+    release(actualTokens?: number): void | Promise<void>;
+}
+/** Remote implementations must reserve atomically, honor abort, and expire abandoned leases. */
+export interface GatewayAdmissionController {
+    acquire(input: {
+        target: GatewayModelTarget;
+        tokens?: number;
+        signal?: AbortSignal;
+    }): Promise<GatewayAdmissionLease>;
+}
+export interface GatewayAdmissionOptions {
+    maxConcurrent: number;
+    requestsPerMinute?: number;
+    tokensPerMinute?: number;
+    maxQueue?: number;
+    queueTimeoutMs?: number;
+    maxTargets?: number;
+    now?: () => number;
+}
+export declare const createGatewayAdmissionController: (options: GatewayAdmissionOptions) => GatewayAdmissionController;
+//# sourceMappingURL=admission.d.ts.map

package/dist/admission.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"admission.d.ts","sourceRoot":"","sources":["../src/admission.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,KAAK,kBAAkB,EAAE,MAAM,YAAY,CAAC;AAEnE,qBAAa,qBAAsB,SAAQ,YAAY;IACrD,cAAyH;CAC1H;AACD,MAAM,WAAW,qBAAqB;IAAG,OAAO,CAAC,YAAY,CAAC,EAAE,MAAM,GAAG,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;CAAE;AAChG,gGAAgG;AAChG,MAAM,WAAW,0BAA0B;IACzC,OAAO,CAAC,KAAK,EAAE;QAAE,MAAM,EAAE,kBAAkB,CAAC;QAAC,MAAM,CAAC,EAAE,MAAM,CAAC;QAAC,MAAM,CAAC,EAAE,WAAW,CAAA;KAAE,GAAG,OAAO,CAAC,qBAAqB,CAAC,CAAC;CACvH;AACD,MAAM,WAAW,uBAAuB;IACtC,aAAa,EAAE,MAAM,CAAC;IACtB,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,eAAe,CAAC,EAAE,MAAM,CAAC;IACzB,QAAQ,CAAC,EAAE,MAAM,CAAC;IAClB,cAAc,CAAC,EAAE,MAAM,CAAC;IACxB,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,GAAG,CAAC,EAAE,MAAM,MAAM,CAAC;CACpB;AACD,eAAO,MAAM,gCAAgC,YAAa,uBAAuB,KAAG,0BAqDnF,CAAC"}

package/dist/admission.js

@@ -0,0 +1,88 @@
+import { GatewayError } from "./types.js";
+import { targetKey } from "./target.js";
+export class GatewayAdmissionError extends GatewayError {
+    constructor() { super("Gateway destination admission capacity exhausted.", false); this.name = "GatewayAdmissionError"; }
+}
+export const createGatewayAdmissionController = (options) => {
+    const maxQueue = options.maxQueue ?? 0, queueTimeout = options.queueTimeoutMs ?? 1000, maxTargets = options.maxTargets ?? 128;
+    for (const n of [options.maxConcurrent, queueTimeout, maxTargets, options.requestsPerMinute ?? 1, options.tokensPerMinute ?? 1]) {
+        if (!Number.isSafeInteger(n) || n < 1)
+            throw new GatewayError("Admission limits must be positive safe integers.", false);
+    }
+    if (!Number.isSafeInteger(maxQueue) || maxQueue < 0 || maxQueue > 10000 || maxTargets > 10000 || queueTimeout > 60000)
+        throw new GatewayError("Invalid admission queue/capacity.", false);
+    const now = options.now ?? Date.now;
+    const entries = new Map();
+    let queued = 0;
+    return {
+        async acquire({ target, tokens, signal }) {
+            if ((tokens !== undefined && (!Number.isSafeInteger(tokens) || tokens < 0)) || (options.tokensPerMinute !== undefined && tokens === undefined))
+                throw new GatewayError("Invalid admission token reservation.", false);
+            if (options.tokensPerMinute !== undefined && tokens > options.tokensPerMinute)
+                throw new GatewayAdmissionError();
+            const reservedTokens = tokens ?? 0;
+            const id = targetKey(target), deadline = Date.now() + queueTimeout;
+            let waiting = false;
+            try {
+                for (;;) {
+                    signal?.throwIfAborted();
+                    const at = now();
+                    let entry = entries.get(id);
+                    if (!entry) {
+                        if (entries.size >= maxTargets) {
+                            const disposable = [...entries].find(([, value]) => value.active === 0 && at >= value.resetAt);
+                            if (disposable)
+                                entries.delete(disposable[0]);
+                            else
+                                throw new GatewayAdmissionError();
+                        }
+                        entry = { active: 0, requests: 0, tokens: 0, resetAt: at + 60000 };
+                        entries.set(id, entry);
+                    }
+                    if (at >= entry.resetAt) {
+                        entry.requests = 0;
+                        entry.tokens = 0;
+                        entry.resetAt = at + 60000;
+                    }
+                    if (entry.active < options.maxConcurrent && entry.requests < (options.requestsPerMinute ?? Infinity) && entry.tokens + reservedTokens <= (options.tokensPerMinute ?? Infinity)) {
+                        entry.active++;
+                        entry.requests++;
+                        entry.tokens += reservedTokens;
+                        const window = entry.resetAt;
+                        let released = false;
+                        return { release(actualTokens) {
+                                if (!released) {
+                                    released = true;
+                                    entry.active--;
+                                    if (entry.resetAt === window && actualTokens !== undefined && Number.isSafeInteger(actualTokens) && actualTokens >= 0)
+                                        entry.tokens = Math.max(0, entry.tokens + actualTokens - reservedTokens);
+                                }
+                            } };
+                    }
+                    if (!waiting) {
+                        if (queued >= maxQueue)
+                            throw new GatewayAdmissionError();
+                        queued++;
+                        waiting = true;
+                    }
+                    const remaining = deadline - Date.now();
+                    if (remaining <= 0)
+                        throw new GatewayAdmissionError();
+                    await new Promise((resolve, reject) => {
+                        const clean = () => { clearTimeout(timer); signal?.removeEventListener("abort", abort); };
+                        const abort = () => { clean(); reject(signal.reason); };
+                        const timer = setTimeout(() => { clean(); resolve(); }, Math.min(10, remaining));
+                        signal?.addEventListener("abort", abort, { once: true });
+                        if (signal?.aborted)
+                            abort();
+                    });
+                }
+            }
+            finally {
+                if (waiting)
+                    queued--;
+            }
+        }
+    };
+};
+//# sourceMappingURL=admission.js.map

package/dist/admission.js.map

@@ -0,0 +1 @@
+{"version":3,"file":"admission.js","sourceRoot":"","sources":["../src/admission.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAA2B,MAAM,YAAY,CAAC;AACnE,OAAO,EAAE,SAAS,EAAE,MAAM,aAAa,CAAC;AACxC,MAAM,OAAO,qBAAsB,SAAQ,YAAY;IACrD,gBAAgB,KAAK,CAAC,mDAAmD,EAAE,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,GAAG,uBAAuB,CAAC,CAAC,CAAC;CAC1H;AAeD,MAAM,CAAC,MAAM,gCAAgC,GAAG,CAAC,OAAgC,EAA8B,EAAE;IAC/G,MAAM,QAAQ,GAAG,OAAO,CAAC,QAAQ,IAAI,CAAC,EAAE,YAAY,GAAG,OAAO,CAAC,cAAc,IAAI,IAAI,EAAE,UAAU,GAAG,OAAO,CAAC,UAAU,IAAI,GAAG,CAAC;IAC9H,KAAK,MAAM,CAAC,IAAI,CAAC,OAAO,CAAC,aAAa,EAAE,YAAY,EAAE,UAAU,EAAE,OAAO,CAAC,iBAAiB,IAAI,CAAC,EAAE,OAAO,CAAC,eAAe,IAAI,CAAC,CAAC,EAAE,CAAC;QAChI,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC;YAAE,MAAM,IAAI,YAAY,CAAC,kDAAkD,EAAE,KAAK,CAAC,CAAC;IAC3H,CAAC;IACD,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,QAAQ,CAAC,IAAI,QAAQ,GAAG,CAAC,IAAI,QAAQ,GAAG,KAAK,IAAI,UAAU,GAAG,KAAK,IAAI,YAAY,GAAG,KAAK;QAAE,MAAM,IAAI,YAAY,CAAC,mCAAmC,EAAE,KAAK,CAAC,CAAC;IAC1L,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,IAAI,IAAI,CAAC,GAAG,CAAC;IAEpC,MAAM,OAAO,GAAG,IAAI,GAAG,EAAiB,CAAC;IACzC,IAAI,MAAM,GAAG,CAAC,CAAC;IACf,OAAO;QACL,KAAK,CAAC,OAAO,CAAC,EAAE,MAAM,EAAE,MAAM,EAAE,MAAM,EAAE;YACtC,IAAI,CAAC,MAAM,KAAK,SAAS,IAAI,CAAC,CAAC,MAAM,CAAC,aAAa,CAAC,MAAM,CAAC,IAAI,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,eAAe,KAAK,SAAS,IAAI,MAAM,KAAK,SAAS,CAAC;gBAAE,MAAM,IAAI,YAAY,CAAC,sCAAsC,EAAE,KAAK,CAAC,CAAC;YACtN,IAAI,OAAO,CAAC,eAAe,KAAK,SAAS,IAAI,MAAO,GAAG,OAAO,CAAC,eAAe;gBAAE,MAAM,IAAI,qBAAqB,EAAE,CAAC;YAClH,MAAM,cAAc,GAAG,MAAM,IAAI,CAAC,CAAC;YACnC,MAAM,EAAE,GAAG,SAAS,CAAC,MAAM,CAAC,EAAE,QAAQ,GAAG,IAAI,CAAC,GAAG,EAAE,GAAG,YAAY,CAAC;YACnE,IAAI,OAAO,GAAG,KAAK,CAAC;YACpB,IAAI,CAAC;gBACH,SAAS,CAAC;oBACR,MAAM,EAAE,cAAc,EAAE,CAAC;oBACzB,MAAM,EAAE,GAAG,GAAG,EAAE,CAAC;oBACjB,IAAI,KAAK,GAAG,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;oBAC5B,IAAI,CAAC,KAAK,EAAE,CAAC;wBACX,IAAI,OAAO,CAAC,IAAI,IAAI,UAAU,EAAE,CAAC;4BAC/B,MAAM,UAAU,GAAG,CAAC,GAAG,OAAO,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,KAAK,CAAC,EAAE,EAAE,CAAC,KAAK,CAAC,MAAM,KAAK,CAAC,IAAI,EAAE,IAAI,KAAK,CAAC,OAAO,CAAC,CAAC;4BAC/F,IAAI,UAAU;gCAAE,OAAO,CAAC,MAAM,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,CAAC;;gCACzC,MAAM,IAAI,qBAAqB,EAAE,CAAC;wBACzC,CAAC;wBACD,KAAK,GAAG,EAAE,MAAM,EAAE,CAAC,EAAE,QAAQ,EAAE,CAAC,EAAE,MAAM,EAAE,CAAC,EAAE,OAAO,EAAE,EAAE,GAAG,KAAK,EAAE,CAAC;wBAAC,OAAO,CAAC,GAAG,CAAC,EAAE,EAAE,KAAK,CAAC,CAAC;oBAC7F,CAAC;oBACD,IAAI,EAAE,IAAI,KAAK,CAAC,OAAO,EAAE,CAAC;wBAAC,KAAK,CAAC,QAAQ,GAAG,CAAC,CAAC;wBAAC,KAAK,CAAC,MAAM,GAAG,CAAC,CAAC;wBAAC,KAAK,CAAC,OAAO,GAAG,EAAE,GAAG,KAAK,CAAC;oBAAC,CAAC;oBAC9F,IAAI,KAAK,CAAC,MAAM,GAAG,OAAO,CAAC,aAAa,IAAI,KAAK,CAAC,QAAQ,GAAG,CAAC,OAAO,CAAC,iBAAiB,IAAI,QAAQ,CAAC,IAAI,KAAK,CAAC,MAAM,GAAG,cAAc,IAAI,CAAC,OAAO,CAAC,eAAe,IAAI,QAAQ,CAAC,EAAE,CAAC;wBAC/K,KAAK,CAAC,MAAM,EAAE,CAAC;wBAAC,KAAK,CAAC,QAAQ,EAAE,CAAC;wBAAC,KAAK,CAAC,MAAM,IAAI,cAAc,CAAC;wBACjE,MAAM,MAAM,GAAG,KAAK,CAAC,OAAO,CAAC;wBAC7B,IAAI,QAAQ,GAAG,KAAK,CAAC;wBACrB,OAAO,EAAE,OAAO,CAAC,YAAY;gCAAI,IAAI,CAAC,QAAQ,EAAE,CAAC;oCAAC,QAAQ,GAAG,IAAI,CAAC;oCAAC,KAAM,CAAC,MAAM,EAAE,CAAC;oCACjF,IAAI,KAAM,CAAC,OAAO,KAAK,MAAM,IAAI,YAAY,KAAK,SAAS,IAAI,MAAM,CAAC,aAAa,CAAC,YAAY,CAAC,IAAI,YAAY,IAAI,CAAC;wCAAE,KAAM,CAAC,MAAM,GAAG,IAAI,CAAC,GAAG,CAAC,CAAC,EAAE,KAAM,CAAC,MAAM,GAAG,YAAY,GAAG,cAAc,CAAC,CAAC;gCACrM,CAAC;4BAAC,CAAC,EAAE,CAAC;oBACR,CAAC;oBACD,IAAI,CAAC,OAAO,EAAE,CAAC;wBAAC,IAAI,MAAM,IAAI,QAAQ;4BAAE,MAAM,IAAI,qBAAqB,EAAE,CAAC;wBAAC,MAAM,EAAE,CAAC;wBAAC,OAAO,GAAG,IAAI,CAAC;oBAAC,CAAC;oBACtG,MAAM,SAAS,GAAG,QAAQ,GAAG,IAAI,CAAC,GAAG,EAAE,CAAC;oBACxC,IAAI,SAAS,IAAI,CAAC;wBAAE,MAAM,IAAI,qBAAqB,EAAE,CAAC;oBACtD,MAAM,IAAI,OAAO,CAAO,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;wBAC1C,MAAM,KAAK,GAAG,GAAG,EAAE,GAAG,YAAY,CAAC,KAAK,CAAC,CAAC,CAAC,MAAM,EAAE,mBAAmB,CAAC,OAAO,EAAE,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC;wBAC1F,MAAM,KAAK,GAAG,GAAG,EAAE,GAAG,KAAK,EAAE,CAAC,CAAC,MAAM,CAAC,MAAO,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC;wBACzD,MAAM,KAAK,GAAG,UAAU,CAAC,GAAG,EAAE,GAAG,KAAK,EAAE,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,IAAI,CAAC,GAAG,CAAC,EAAE,EAAE,SAAS,CAAC,CAAC,CAAC;wBACjF,MAAM,EAAE,gBAAgB,CAAC,OAAO,EAAE,KAAK,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC;wBACzD,IAAI,MAAM,EAAE,OAAO;4BAAE,KAAK,EAAE,CAAC;oBAC/B,CAAC,CAAC,CAAC;gBACL,CAAC;YACH,CAAC;oBAAS,CAAC;gBAAC,IAAI,OAAO;oBAAE,MAAM,EAAE,CAAC;YAAC,CAAC;QACtC,CAAC;KACF,CAAC;AACJ,CAAC,CAAC"}

package/dist/budget.d.ts

@@ -0,0 +1,33 @@
+import { GatewayError } from "./types.js";
+export declare class GatewayBudgetError extends GatewayError {
+    constructor();
+}
+export interface GatewayBudgetReservation {
+    /** null retains the entire reservation as uncertain spend; settlement is idempotent. */
+    settle(actualAmount: number | null): void | Promise<void>;
+    /** Only use when no upstream request was dispatched. */
+    cancel(): void | Promise<void>;
+}
+export interface GatewayBudgetStore {
+    reserve(input: {
+        scope: string;
+        currency: string;
+        amount: number;
+        signal?: AbortSignal;
+    }): Promise<GatewayBudgetReservation>;
+}
+export interface GatewayBudgetSnapshot {
+    spent: number;
+    reserved: number;
+    uncertain: number;
+    remaining: number;
+}
+/** Process-local lifetime budgets. Use a new store/scope for a new accounting period. */
+export declare const createGatewayBudgetStore: (options: {
+    limit: number;
+    currency: string;
+    maxScopes?: number;
+}) => GatewayBudgetStore & {
+    snapshot(scope: string): GatewayBudgetSnapshot;
+};
+//# sourceMappingURL=budget.d.ts.map

package/dist/budget.d.ts.map

@@ -0,0 +1 @@
+{"version":3,"file":"budget.d.ts","sourceRoot":"","sources":["../src/budget.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC1C,qBAAa,kBAAmB,SAAQ,YAAY;IAClD,cAAsH;CACvH;AACD,MAAM,WAAW,wBAAwB;IACvC,wFAAwF;IACxF,MAAM,CAAC,YAAY,EAAE,MAAM,GAAG,IAAI,GAAG,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;IAC1D,wDAAwD;IACxD,MAAM,IAAI,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;CAChC;AACD,MAAM,WAAW,kBAAkB;IACjC,OAAO,CAAC,KAAK,EAAE;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,QAAQ,EAAE,MAAM,CAAC;QAAC,MAAM,EAAE,MAAM,CAAC;QAAC,MAAM,CAAC,EAAE,WAAW,CAAA;KAAE,GAAG,OAAO,CAAC,wBAAwB,CAAC,CAAC;CAC9H;AACD,MAAM,WAAW,qBAAqB;IAAG,KAAK,EAAE,MAAM,CAAC;IAAC,QAAQ,EAAE,MAAM,CAAC;IAAC,SAAS,EAAE,MAAM,CAAC;IAAC,SAAS,EAAE,MAAM,CAAC;CAAE;AACjH,yFAAyF;AACzF,eAAO,MAAM,wBAAwB,YAAa;IAAE,KAAK,EAAE,MAAM,CAAC;IAAC,QAAQ,EAAE,MAAM,CAAC;IAAC,SAAS,CAAC,EAAE,MAAM,CAAA;CAAE,KAAG,kBAAkB,GAAG;IAAE,QAAQ,CAAC,KAAK,EAAE,MAAM,GAAG,qBAAqB,CAAA;CAiChL,CAAC"}