@zhivex-ai/gateway 1.2.2 → 1.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +41 -1
- package/dist/adaptive-routing.d.ts +40 -0
- package/dist/adaptive-routing.d.ts.map +1 -0
- package/dist/adaptive-routing.js +47 -0
- package/dist/adaptive-routing.js.map +1 -0
- package/dist/circuit-breaker.d.ts +36 -0
- package/dist/circuit-breaker.d.ts.map +1 -0
- package/dist/circuit-breaker.js +85 -0
- package/dist/circuit-breaker.js.map +1 -0
- package/dist/index.d.ts +8 -3
- package/dist/index.d.ts.map +1 -1
- package/dist/index.js +284 -19
- package/dist/index.js.map +1 -1
- package/dist/metrics.d.ts +30 -0
- package/dist/metrics.d.ts.map +1 -0
- package/dist/metrics.js +63 -0
- package/dist/metrics.js.map +1 -0
- package/dist/types.d.ts +30 -4
- package/dist/types.d.ts.map +1 -1
- package/dist/types.js.map +1 -1
- package/package.json +2 -2
package/README.md
CHANGED
|
@@ -160,7 +160,7 @@ messages. System messages must precede conversation turns.
|
|
|
160
160
|
| --- | --- |
|
|
161
161
|
| `generate`, `streamText` | Supported, including usage estimates, attempts, finish reasons, cancellation and routing |
|
|
162
162
|
| `generateObject`, `streamObject` | Same input validation and routing; destination must also support the requested object mode |
|
|
163
|
-
| `runAgent`, `streamAgent` |
|
|
163
|
+
| `runAgent`, `streamAgent` | Fresh canonical history imports; durable resumes use state/runId without resupplying messages |
|
|
164
164
|
| Anthropic Messages | Text, user images, assistant tool calls, tool results and native `is_error` |
|
|
165
165
|
| OpenAI Chat / Responses | Callable tool history, including multiple results; explicit JSON success/error envelopes |
|
|
166
166
|
| DeepSeek Chat | Same JSON envelope support; portable replay defaults to non-thinking; reasoning-only models are excluded |
|
|
@@ -209,3 +209,43 @@ candidate core/gateway/provider tarballs without checkout links. Add
|
|
|
209
209
|
authorized credential and perform two bounded live calls for one provider. A missing credential is a blocked live check, not
|
|
210
210
|
a passing certification. Publication evidence and the actual version are recorded
|
|
211
211
|
in [the delivery record](../../docs/GATEWAY_TOOL_HISTORY_DELIVERY.md).
|
|
212
|
+
|
|
213
|
+
## Usage accounting
|
|
214
|
+
|
|
215
|
+
`generate`, `generateObject`, `streamText().collect()` and `streamObject().collect()` preserve all reported `TokenUsage` fields, including `cachedInputTokens`, `cacheWriteTokens`, `reasoningTokens` and `speed`. Reported zeros remain zeros; missing optional details remain absent. Only missing `inputTokens`/`outputTokens` are estimated from text and a missing `totalTokens` is derived from those base counters. `estimated` is true if any of those three base counters was missing, including a derived total. Reasoning/cache details are never added again to the total. Agent results retain their existing usage aggregation without an `estimated` flag.
|
|
216
|
+
|
|
217
|
+
## Detailed cost valuation
|
|
218
|
+
|
|
219
|
+
Opt in with `costAccounting: { unknownCostPolicy: "allow" }` and an immutable `modelCatalog` carrying pricing metadata. `routeDecision.estimatedCosts` contains cold-cache preflight estimates from input length and `expectedOutputTokens` (or request `maxTokens`). They are estimates, not spending limits. With `unknownCostPolicy: "reject"`, destinations with unknown quotes are excluded before a call. The existing `maxCostPer1kTokens` budget remains independent.
|
|
220
|
+
|
|
221
|
+
Each actual `attempt` carries its reported `usage` and `cost`, including provider/model, currency, catalog/pricing versions, optional provenance source, components, assumptions and unknown reasons. A failed attempt without usage has `amount: null`; a successful fallback does not make that failure free. Sum amounts only when every actual attempt is known or explicitly estimated; otherwise total cost remains unknown. Streams use terminal usage, and partial stream failures preserve any usage already received.
|
|
222
|
+
|
|
223
|
+
`calculateModelCost()` is also exported from Core and SDK for standalone valuation. Input tokens must include cached reads and cache writes; these are subtracted from ordinary input before applying their separate prices. Missing cache counters remain unknown unless `cacheAssumption: "none"` explicitly assumes zeros. Missing relevant prices or base usage never become a free estimate. Long-context multipliers apply above the catalog threshold to all input classes and output. Zero tokens incur zero cost even when that unused category has no rate; its rate remains null.
|
|
224
|
+
|
|
225
|
+
Reasoning semantics vary between adapters. Set `reasoningAccounting: { openai: "included", gemini: "additional" }` only for the provider protocols you have verified: included leaves the output count unchanged, additional adds reasoning once. Without an explicit setting, a positive reasoning counter produces an unknown valuation. For standalone valuation this setting is a single string. The catalog does not describe fast-tier pricing, so `speed: "fast"` remains unknown. Invalid reported counters produce unknown accounting without retrying a successful model call. This API is neither billing nor a strict monetary budget.
|
|
226
|
+
|
|
227
|
+
## Local destination metrics
|
|
228
|
+
|
|
229
|
+
Pass `metrics: createGatewayMetrics({ windowMs: 60_000, maxTargets: 128, maxSamplesPerTarget: 256 })` to opt in. Query `metrics.snapshot({ provider, modelId })` for in-flight count, recent successes/errors/cancellations, full-attempt p50/p95 and first-text p95. Cold destinations return undefined; expired samples have no latency estimates. Client cancellation is excluded from provider error and latency samples. The clock is injectable through `now` for deterministic tests.
|
|
230
|
+
|
|
231
|
+
Storage is bounded by targets and samples. The least recently touched idle destination is evicted when full; if all target slots are active, a new destination remains untracked until capacity is available. No prompts, outputs, error payloads or credentials are retained. The default store is local to the process; applications may inject the synchronous `GatewayMetricsStore` contract. Hook failures are best effort. Do not put expensive synchronous work in a metrics hook.
|
|
232
|
+
|
|
233
|
+
With metrics enabled, explicitly returning an event/text/object stream iterator cancels that routed operation and releases the in-flight slot. This applies to all consumers of that operation, including collect. Consume the stream fully if you also need its final result. Caller abort and timeout also release slots even when a provider ignores its signal. An operation with no explicit return/abort continues until completion or timeout; the SDK cannot detect garbage collection as abandonment.
|
|
234
|
+
|
|
235
|
+
## Circuit breaker and adaptive routing
|
|
236
|
+
|
|
237
|
+
Pass `circuitBreaker: createGatewayCircuitBreaker({ failureThreshold: 5, cooldownMs: 30_000 })` to opt in to local closed/open/half-open circuits. Retryable errors (including 429/5xx and attempt timeouts) accumulate; validation, authentication and client cancellation do not. Open circuits receive no calls. Recovery allows one concurrent probe by default; successful probes close the circuit, failed probes reopen it. Late results from an older circuit epoch cannot close a newly opened circuit. When all eligible circuits are unavailable the operation raises `GatewayCircuitOpenError`.
|
|
238
|
+
|
|
239
|
+
Cooldown is `min(maxCooldownMs, max(cooldownMs, Retry-After))`, with defaults of 60 seconds maximum and 30 seconds minimum. The circuit never sleeps until recovery: it skips the target. Existing bounded retry backoff applies only while the circuit remains closed. Target storage is bounded; closed idle entries may be evicted, while capacity exhausted by active/open entries refuses admission. State observers receive only target IDs, state and timestamps, and run best effort. Metrics/circuit-enabled stream iterator return cancels the operation. Neither feature coordinates across processes.
|
|
240
|
+
|
|
241
|
+
`adaptiveRouting` is an explicit alternative to legacy scoring. Supply a policy version, nonnegative weights for `latency`, `cost`, `quality`, `load`, `errorRate`, positive `latencyScaleMs`/`costScale`, and explicit `coldStart`, `unknownCost`, `missingQuality` policies (`allow`/`reject`). Quality profiles carry a target, task `intent`, score in [0,1] and evaluation version. There are no model-name quality heuristics in this mode.
|
|
242
|
+
|
|
243
|
+
The score is quality reward minus normalized p95 latency, estimated request cost, in-flight load and observed error rate penalties. Enable `metrics` and `costAccounting` for those signals. Missing signals under `allow` omit that score term and remain listed as missing; `reject` excludes the destination. Expired samples become cold start. Capabilities and circuits are filtered before ordering, and circuits are rechecked atomically before each call. Ties preserve primary/fallback input order. `routeDecision.adaptive` records policy version, signals, profile versions and exclusions. Do not configure both `scoreTarget` and `adaptiveRouting`. These rules are transparent heuristics, not a claim of globally optimal routing.
|
|
244
|
+
|
|
245
|
+
## Composing configured agents
|
|
246
|
+
|
|
247
|
+
Pass `agent: configuredAgent` to `runAgent` or `streamAgent`. The gateway substitutes a routed LanguageModel and uses the existing Core runtime. Context schema, guardrails, output schema, compactor, approval signer, subagents, harness binding, environment, hooks and store remain on the definition. Supply ephemeral `context` on each invocation. Defined request options override definition defaults; policy and metadata merge shallowly. `compaction: false` explicitly disables the default for an invocation. The original definition is not mutated.
|
|
248
|
+
|
|
249
|
+
Configured gateway runs persist a `gatewayAgentRouteBinding` metadata value for agent ID, primary/fallback targets, routing policy version and harness fingerprint. Resume must retain that binding; changing it fails before provider work. An older direct run without this binding requires a separately designed migration instead of silent adoption. Core still validates harness/environment fingerprints. Reserve `gatewayAgentRouteBinding` and `gatewayPortableHistory` for runtime use.
|
|
250
|
+
|
|
251
|
+
For canonical history, import a fresh run with `messages` only. Do not combine import with prompt, state, runId, handoff, approvals or idempotencyKey. Resolved historical tools are input, never effects to replay. Then resume with store/runId or state and approvals as needed, without messages. Portable-history capability checks survive store reloads and compaction. Legacy inputs remain supported. Provider-specific approval data stays in durable state; it is not part of the portable history import format.
|
|
@@ -0,0 +1,40 @@
|
|
|
1
|
+
import { type GatewayModelTarget, type GatewayTaskIntent } from "./types.js";
|
|
2
|
+
import type { GatewayMetricsSnapshot } from "./metrics.js";
|
|
3
|
+
import type { ModelCostValuation } from "@zhivex-ai/core";
|
|
4
|
+
export interface GatewayQualityProfile {
|
|
5
|
+
target: GatewayModelTarget;
|
|
6
|
+
intent: GatewayTaskIntent;
|
|
7
|
+
score: number;
|
|
8
|
+
version: string;
|
|
9
|
+
}
|
|
10
|
+
export interface GatewayAdaptiveRoutingPolicy {
|
|
11
|
+
version: string;
|
|
12
|
+
weights: {
|
|
13
|
+
latency: number;
|
|
14
|
+
cost: number;
|
|
15
|
+
quality: number;
|
|
16
|
+
load: number;
|
|
17
|
+
errorRate: number;
|
|
18
|
+
};
|
|
19
|
+
latencyScaleMs: number;
|
|
20
|
+
costScale: number;
|
|
21
|
+
coldStart: "allow" | "reject";
|
|
22
|
+
unknownCost: "allow" | "reject";
|
|
23
|
+
missingQuality: "allow" | "reject";
|
|
24
|
+
qualityProfiles?: GatewayQualityProfile[];
|
|
25
|
+
}
|
|
26
|
+
export interface GatewayAdaptiveCandidate {
|
|
27
|
+
target: GatewayModelTarget;
|
|
28
|
+
score?: number;
|
|
29
|
+
exclusions: string[];
|
|
30
|
+
missingSignals: string[];
|
|
31
|
+
metrics?: GatewayMetricsSnapshot;
|
|
32
|
+
cost?: ModelCostValuation;
|
|
33
|
+
quality?: {
|
|
34
|
+
score: number;
|
|
35
|
+
version: string;
|
|
36
|
+
};
|
|
37
|
+
}
|
|
38
|
+
export declare const validateAdaptivePolicy: (policy: GatewayAdaptiveRoutingPolicy) => void;
|
|
39
|
+
export declare const scoreAdaptiveTarget: (policy: GatewayAdaptiveRoutingPolicy, target: GatewayModelTarget, intent: GatewayTaskIntent, metrics?: GatewayMetricsSnapshot, cost?: ModelCostValuation) => GatewayAdaptiveCandidate;
|
|
40
|
+
//# sourceMappingURL=adaptive-routing.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"adaptive-routing.d.ts","sourceRoot":"","sources":["../src/adaptive-routing.ts"],"names":[],"mappings":"AAAA,OAAO,EAAgB,KAAK,kBAAkB,EAAE,KAAK,iBAAiB,EAAE,MAAM,YAAY,CAAC;AAC3F,OAAO,KAAK,EAAE,sBAAsB,EAAE,MAAM,cAAc,CAAC;AAC3D,OAAO,KAAK,EAAE,kBAAkB,EAAE,MAAM,iBAAiB,CAAC;AAC1D,MAAM,WAAW,qBAAqB;IAAG,MAAM,EAAE,kBAAkB,CAAC;IAAC,MAAM,EAAE,iBAAiB,CAAC;IAAC,KAAK,EAAE,MAAM,CAAC;IAAC,OAAO,EAAE,MAAM,CAAC;CAAE;AACjI,MAAM,WAAW,4BAA4B;IAC3C,OAAO,EAAE,MAAM,CAAC;IAChB,OAAO,EAAE;QAAE,OAAO,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,OAAO,EAAE,MAAM,CAAC;QAAC,IAAI,EAAE,MAAM,CAAC;QAAC,SAAS,EAAE,MAAM,CAAA;KAAE,CAAC;IAC7F,cAAc,EAAE,MAAM,CAAC;IACvB,SAAS,EAAE,MAAM,CAAC;IAClB,SAAS,EAAE,OAAO,GAAG,QAAQ,CAAC;IAC9B,WAAW,EAAE,OAAO,GAAG,QAAQ,CAAC;IAChC,cAAc,EAAE,OAAO,GAAG,QAAQ,CAAC;IACnC,eAAe,CAAC,EAAE,qBAAqB,EAAE,CAAC;CAC3C;AACD,MAAM,WAAW,wBAAwB;IACvC,MAAM,EAAE,kBAAkB,CAAC;IAC3B,KAAK,CAAC,EAAE,MAAM,CAAC;IACf,UAAU,EAAE,MAAM,EAAE,CAAC;IACrB,cAAc,EAAE,MAAM,EAAE,CAAC;IACzB,OAAO,CAAC,EAAE,sBAAsB,CAAC;IACjC,IAAI,CAAC,EAAE,kBAAkB,CAAC;IAC1B,OAAO,CAAC,EAAE;QAAE,KAAK,EAAE,MAAM,CAAC;QAAC,OAAO,EAAE,MAAM,CAAA;KAAE,CAAC;CAC9C;AACD,eAAO,MAAM,sBAAsB,WAAY,4BAA4B,SAY1E,CAAC;AACF,eAAO,MAAM,mBAAmB,WAAY,4BAA4B,UAAU,kBAAkB,UAAU,iBAAiB,YAAY,sBAAsB,SAAS,kBAAkB,KAAG,wBAiB9L,CAAC"}
|
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
import { GatewayError } from "./types.js";
|
|
2
|
+
export const validateAdaptivePolicy = (policy) => {
|
|
3
|
+
if (!policy.version?.trim())
|
|
4
|
+
throw new GatewayError("Adaptive policy requires a version.", false);
|
|
5
|
+
if (![policy.latencyScaleMs, policy.costScale].every(x => Number.isFinite(x) && x > 0))
|
|
6
|
+
throw new GatewayError("Adaptive scales must be positive.", false);
|
|
7
|
+
const values = [policy.weights?.latency, policy.weights?.cost, policy.weights?.quality, policy.weights?.load, policy.weights?.errorRate];
|
|
8
|
+
if (!values.every(x => Number.isFinite(x) && x >= 0) || values.every(x => x === 0))
|
|
9
|
+
throw new GatewayError("Adaptive weights must be nonnegative with at least one positive weight.", false);
|
|
10
|
+
if (![policy.coldStart, policy.unknownCost, policy.missingQuality].every(x => x === "allow" || x === "reject"))
|
|
11
|
+
throw new GatewayError("Adaptive missing-data policies must be explicit.", false);
|
|
12
|
+
const keys = new Set();
|
|
13
|
+
for (const profile of policy.qualityProfiles ?? []) {
|
|
14
|
+
const key = JSON.stringify([profile.target.provider, profile.target.modelId, profile.intent]);
|
|
15
|
+
if (keys.has(key) || !profile.version?.trim() || !Number.isFinite(profile.score) || profile.score < 0 || profile.score > 1)
|
|
16
|
+
throw new GatewayError("Invalid or ambiguous quality profile.", false);
|
|
17
|
+
keys.add(key);
|
|
18
|
+
}
|
|
19
|
+
};
|
|
20
|
+
export const scoreAdaptiveTarget = (policy, target, intent, metrics, cost) => {
|
|
21
|
+
const result = { target: { ...target }, exclusions: [], missingSignals: [], ...(metrics ? { metrics } : {}), ...(cost ? { cost } : {}) };
|
|
22
|
+
const quality = policy.qualityProfiles?.find(x => x.target.provider === target.provider && x.target.modelId === target.modelId && x.intent === intent);
|
|
23
|
+
if (quality)
|
|
24
|
+
result.quality = { score: quality.score, version: quality.version };
|
|
25
|
+
const missing = (signal, action) => { result.missingSignals.push(signal); if (action === "reject")
|
|
26
|
+
result.exclusions.push(`${signal}-unavailable`); };
|
|
27
|
+
if ((policy.weights.latency || policy.weights.errorRate) && (!metrics || metrics.successes + metrics.errors === 0))
|
|
28
|
+
missing("health", policy.coldStart);
|
|
29
|
+
if (policy.weights.load && !metrics)
|
|
30
|
+
missing("load", policy.coldStart);
|
|
31
|
+
if (policy.weights.cost && (!cost || cost.amount === null))
|
|
32
|
+
missing("cost", policy.unknownCost);
|
|
33
|
+
if (policy.weights.quality && !quality)
|
|
34
|
+
missing("quality", policy.missingQuality);
|
|
35
|
+
if (result.exclusions.length)
|
|
36
|
+
return result;
|
|
37
|
+
const errorRate = metrics && metrics.successes + metrics.errors > 0 ? metrics.errors / (metrics.successes + metrics.errors) : 0;
|
|
38
|
+
const score = (quality?.score ?? 0) * policy.weights.quality
|
|
39
|
+
- (metrics?.p95LatencyMs ?? 0) / policy.latencyScaleMs * policy.weights.latency
|
|
40
|
+
- (cost?.amount ?? 0) / policy.costScale * policy.weights.cost
|
|
41
|
+
- (metrics?.inFlight ?? 0) * policy.weights.load - errorRate * policy.weights.errorRate;
|
|
42
|
+
if (!Number.isFinite(score))
|
|
43
|
+
throw new GatewayError("Adaptive signals produced a nonfinite score.", false);
|
|
44
|
+
result.score = score;
|
|
45
|
+
return result;
|
|
46
|
+
};
|
|
47
|
+
//# sourceMappingURL=adaptive-routing.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"adaptive-routing.js","sourceRoot":"","sources":["../src/adaptive-routing.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAmD,MAAM,YAAY,CAAC;AAuB3F,MAAM,CAAC,MAAM,sBAAsB,GAAG,CAAC,MAAoC,EAAE,EAAE;IAC7E,IAAI,CAAC,MAAM,CAAC,OAAO,EAAE,IAAI,EAAE;QAAE,MAAM,IAAI,YAAY,CAAC,qCAAqC,EAAE,KAAK,CAAC,CAAC;IAClG,IAAI,CAAC,CAAC,MAAM,CAAC,cAAc,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,GAAG,CAAC,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,mCAAmC,EAAE,KAAK,CAAC,CAAC;IAC3J,MAAM,MAAM,GAAG,CAAC,MAAM,CAAC,OAAO,EAAE,OAAO,EAAE,MAAM,CAAC,OAAO,EAAE,IAAI,EAAE,MAAM,CAAC,OAAO,EAAE,OAAO,EAAE,MAAM,CAAC,OAAO,EAAE,IAAI,EAAE,MAAM,CAAC,OAAO,EAAE,SAAS,CAAC,CAAC;IACzI,IAAI,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,MAAM,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAC,IAAI,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,KAAK,CAAC,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,yEAAyE,EAAE,KAAK,CAAC,CAAC;IAC7L,IAAI,CAAC,CAAC,MAAM,CAAC,SAAS,EAAE,MAAM,CAAC,WAAW,EAAE,MAAM,CAAC,cAAc,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,KAAK,OAAO,IAAI,CAAC,KAAK,QAAQ,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,kDAAkD,EAAE,KAAK,CAAC,CAAC;IAClM,MAAM,IAAI,GAAG,IAAI,GAAG,EAAU,CAAC;IAC/B,KAAK,MAAM,OAAO,IAAI,MAAM,CAAC,eAAe,IAAI,EAAE,EAAE,CAAC;QACnD,MAAM,GAAG,GAAG,IAAI,CAAC,SAAS,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC,QAAQ,EAAE,OAAO,CAAC,MAAM,CAAC,OAAO,EAAE,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC;QAC9F,IAAI,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,IAAI,CAAC,OAAO,CAAC,OAAO,EAAE,IAAI,EAAE,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,OAAO,CAAC,KAAK,CAAC,IAAI,OAAO,CAAC,KAAK,GAAG,CAAC,IAAI,OAAO,CAAC,KAAK,GAAG,CAAC;YAAE,MAAM,IAAI,YAAY,CAAC,uCAAuC,EAAE,KAAK,CAAC,CAAC;QACnM,IAAI,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC;IAChB,CAAC;AACH,CAAC,CAAC;AACF,MAAM,CAAC,MAAM,mBAAmB,GAAG,CAAC,MAAoC,EAAE,MAA0B,EAAE,MAAyB,EAAE,OAAgC,EAAE,IAAyB,EAA4B,EAAE;IACxN,MAAM,MAAM,GAA6B,EAAE,MAAM,EAAE,EAAE,GAAG,MAAM,EAAE,EAAE,UAAU,EAAE,EAAE,EAAE,cAAc,EAAE,EAAE,EAAE,GAAG,CAAC,OAAO,CAAC,CAAC,CAAC,EAAE,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,GAAG,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,CAAC;IACnK,MAAM,OAAO,GAAG,MAAM,CAAC,eAAe,EAAE,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,MAAM,CAAC,QAAQ,KAAK,MAAM,CAAC,QAAQ,IAAI,CAAC,CAAC,MAAM,CAAC,OAAO,KAAK,MAAM,CAAC,OAAO,IAAI,CAAC,CAAC,MAAM,KAAK,MAAM,CAAC,CAAC;IACvJ,IAAI,OAAO;QAAE,MAAM,CAAC,OAAO,GAAG,EAAE,KAAK,EAAE,OAAO,CAAC,KAAK,EAAE,OAAO,EAAE,OAAO,CAAC,OAAO,EAAE,CAAC;IACjF,MAAM,OAAO,GAAG,CAAC,MAAc,EAAE,MAA0B,EAAE,EAAE,GAAG,MAAM,CAAC,cAAc,CAAC,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC,IAAI,MAAM,KAAK,QAAQ;QAAE,MAAM,CAAC,UAAU,CAAC,IAAI,CAAC,GAAG,MAAM,cAAc,CAAC,CAAC,CAAC,CAAC,CAAC;IAClL,IAAI,CAAC,MAAM,CAAC,OAAO,CAAC,OAAO,IAAI,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC,IAAI,CAAC,CAAC,OAAO,IAAI,OAAO,CAAC,SAAS,GAAG,OAAO,CAAC,MAAM,KAAK,CAAC,CAAC;QAAE,OAAO,CAAC,QAAQ,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;IACxJ,IAAI,MAAM,CAAC,OAAO,CAAC,IAAI,IAAI,CAAC,OAAO;QAAE,OAAO,CAAC,MAAM,EAAE,MAAM,CAAC,SAAS,CAAC,CAAC;IACvE,IAAI,MAAM,CAAC,OAAO,CAAC,IAAI,IAAI,CAAC,CAAC,IAAI,IAAI,IAAI,CAAC,MAAM,KAAK,IAAI,CAAC;QAAE,OAAO,CAAC,MAAM,EAAE,MAAM,CAAC,WAAW,CAAC,CAAC;IAChG,IAAI,MAAM,CAAC,OAAO,CAAC,OAAO,IAAI,CAAC,OAAO;QAAE,OAAO,CAAC,SAAS,EAAE,MAAM,CAAC,cAAc,CAAC,CAAC;IAClF,IAAI,MAAM,CAAC,UAAU,CAAC,MAAM;QAAE,OAAO,MAAM,CAAC;IAC5C,MAAM,SAAS,GAAG,OAAO,IAAI,OAAO,CAAC,SAAS,GAAG,OAAO,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,MAAM,GAAG,CAAC,OAAO,CAAC,SAAS,GAAG,OAAO,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IAChI,MAAM,KAAK,GAAG,CAAC,OAAO,EAAE,KAAK,IAAI,CAAC,CAAC,GAAG,MAAM,CAAC,OAAO,CAAC,OAAO;UACxD,CAAC,OAAO,EAAE,YAAY,IAAI,CAAC,CAAC,GAAG,MAAM,CAAC,cAAc,GAAG,MAAM,CAAC,OAAO,CAAC,OAAO;UAC7E,CAAC,IAAI,EAAE,MAAM,IAAI,CAAC,CAAC,GAAG,MAAM,CAAC,SAAS,GAAG,MAAM,CAAC,OAAO,CAAC,IAAI;UAC5D,CAAC,OAAO,EAAE,QAAQ,IAAI,CAAC,CAAC,GAAG,MAAM,CAAC,OAAO,CAAC,IAAI,GAAG,SAAS,GAAG,MAAM,CAAC,OAAO,CAAC,SAAS,CAAC;IAC1F,IAAI,CAAC,MAAM,CAAC,QAAQ,CAAC,KAAK,CAAC;QAAE,MAAM,IAAI,YAAY,CAAC,8CAA8C,EAAE,KAAK,CAAC,CAAC;IAC3G,MAAM,CAAC,KAAK,GAAG,KAAK,CAAC;IAAC,OAAO,MAAM,CAAC;AACtC,CAAC,CAAC"}
|
|
@@ -0,0 +1,36 @@
|
|
|
1
|
+
import { GatewayError, type GatewayModelTarget } from "./types.js";
|
|
2
|
+
export type GatewayCircuitState = "closed" | "open" | "half-open";
|
|
3
|
+
export interface GatewayCircuitSnapshot {
|
|
4
|
+
state: GatewayCircuitState;
|
|
5
|
+
failures: number;
|
|
6
|
+
inFlight: number;
|
|
7
|
+
openUntil?: number;
|
|
8
|
+
}
|
|
9
|
+
export interface GatewayCircuitPermit {
|
|
10
|
+
end(outcome: "success" | "retryable-error" | "neutral", retryAfterMs?: number): void;
|
|
11
|
+
}
|
|
12
|
+
export interface GatewayCircuitBreaker {
|
|
13
|
+
acquire(target: GatewayModelTarget): GatewayCircuitPermit | undefined;
|
|
14
|
+
snapshot(target: GatewayModelTarget): GatewayCircuitSnapshot | undefined;
|
|
15
|
+
canAttempt(target: GatewayModelTarget): boolean;
|
|
16
|
+
}
|
|
17
|
+
export interface GatewayCircuitOptions {
|
|
18
|
+
failureThreshold?: number;
|
|
19
|
+
cooldownMs?: number;
|
|
20
|
+
maxCooldownMs?: number;
|
|
21
|
+
halfOpenMaxProbes?: number;
|
|
22
|
+
maxTargets?: number;
|
|
23
|
+
now?: () => number;
|
|
24
|
+
onStateChange?: (event: {
|
|
25
|
+
target: GatewayModelTarget;
|
|
26
|
+
from: GatewayCircuitState;
|
|
27
|
+
to: GatewayCircuitState;
|
|
28
|
+
at: number;
|
|
29
|
+
openUntil?: number;
|
|
30
|
+
}) => void | Promise<void>;
|
|
31
|
+
}
|
|
32
|
+
export declare class GatewayCircuitOpenError extends GatewayError {
|
|
33
|
+
constructor();
|
|
34
|
+
}
|
|
35
|
+
export declare const createGatewayCircuitBreaker: (options?: GatewayCircuitOptions) => GatewayCircuitBreaker;
|
|
36
|
+
//# sourceMappingURL=circuit-breaker.d.ts.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"circuit-breaker.d.ts","sourceRoot":"","sources":["../src/circuit-breaker.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAAE,KAAK,kBAAkB,EAAE,MAAM,YAAY,CAAC;AACnE,MAAM,MAAM,mBAAmB,GAAG,QAAQ,GAAG,MAAM,GAAG,WAAW,CAAC;AAClE,MAAM,WAAW,sBAAsB;IAAG,KAAK,EAAE,mBAAmB,CAAC;IAAC,QAAQ,EAAE,MAAM,CAAC;IAAC,QAAQ,EAAE,MAAM,CAAC;IAAC,SAAS,CAAC,EAAE,MAAM,CAAC;CAAE;AAC/H,MAAM,WAAW,oBAAoB;IAAG,GAAG,CAAC,OAAO,EAAE,SAAS,GAAG,iBAAiB,GAAG,SAAS,EAAE,YAAY,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;CAAE;AAC/H,MAAM,WAAW,qBAAqB;IACpC,OAAO,CAAC,MAAM,EAAE,kBAAkB,GAAG,oBAAoB,GAAG,SAAS,CAAC;IACtE,QAAQ,CAAC,MAAM,EAAE,kBAAkB,GAAG,sBAAsB,GAAG,SAAS,CAAC;IACzE,UAAU,CAAC,MAAM,EAAE,kBAAkB,GAAG,OAAO,CAAC;CACjD;AACD,MAAM,WAAW,qBAAqB;IACpC,gBAAgB,CAAC,EAAE,MAAM,CAAC;IAC1B,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,aAAa,CAAC,EAAE,MAAM,CAAC;IACvB,iBAAiB,CAAC,EAAE,MAAM,CAAC;IAC3B,UAAU,CAAC,EAAE,MAAM,CAAC;IACpB,GAAG,CAAC,EAAE,MAAM,MAAM,CAAC;IACnB,aAAa,CAAC,EAAE,CAAC,KAAK,EAAE;QAAE,MAAM,EAAE,kBAAkB,CAAC;QAAC,IAAI,EAAE,mBAAmB,CAAC;QAAC,EAAE,EAAE,mBAAmB,CAAC;QAAC,EAAE,EAAE,MAAM,CAAC;QAAC,SAAS,CAAC,EAAE,MAAM,CAAA;KAAE,KAAK,IAAI,GAAG,OAAO,CAAC,IAAI,CAAC,CAAC;CACrK;AACD,qBAAa,uBAAwB,SAAQ,YAAY;IACvD,cAAqJ;CACtJ;AACD,eAAO,MAAM,2BAA2B,aAAa,qBAAqB,KAAQ,qBAuDjF,CAAC"}
|
|
@@ -0,0 +1,85 @@
|
|
|
1
|
+
import { GatewayError } from "./types.js";
|
|
2
|
+
export class GatewayCircuitOpenError extends GatewayError {
|
|
3
|
+
constructor() { super("All eligible destination circuits are open or have exhausted probe capacity.", true); this.name = "GatewayCircuitOpenError"; }
|
|
4
|
+
}
|
|
5
|
+
export const createGatewayCircuitBreaker = (options = {}) => {
|
|
6
|
+
const threshold = options.failureThreshold ?? 5, cooldown = options.cooldownMs ?? 30_000, maxCooldown = options.maxCooldownMs ?? 60_000, probes = options.halfOpenMaxProbes ?? 1, capacity = options.maxTargets ?? 128;
|
|
7
|
+
for (const [value, max] of [[threshold, 1000], [cooldown, 86_400_000], [maxCooldown, 86_400_000], [probes, 100], [capacity, 10_000]]) {
|
|
8
|
+
if (!Number.isSafeInteger(value) || value < 1 || value > max)
|
|
9
|
+
throw new GatewayError("Invalid circuit breaker limits.", false);
|
|
10
|
+
}
|
|
11
|
+
if (cooldown > maxCooldown)
|
|
12
|
+
throw new GatewayError("cooldownMs cannot exceed maxCooldownMs.", false);
|
|
13
|
+
const now = options.now ?? Date.now;
|
|
14
|
+
const entries = new Map();
|
|
15
|
+
const key = (target) => JSON.stringify([target.provider, target.modelId]);
|
|
16
|
+
const transition = (entry, target, to) => {
|
|
17
|
+
const from = entry.state;
|
|
18
|
+
entry.state = to;
|
|
19
|
+
entry.epoch++;
|
|
20
|
+
if (from === to || !options.onStateChange)
|
|
21
|
+
return;
|
|
22
|
+
const event = { target: { ...target }, from, to, at: now(), openUntil: entry.openUntil };
|
|
23
|
+
queueMicrotask(() => { try {
|
|
24
|
+
void Promise.resolve(options.onStateChange(event)).catch(() => undefined);
|
|
25
|
+
}
|
|
26
|
+
catch { /* Observer failure cannot change policy. */ } });
|
|
27
|
+
};
|
|
28
|
+
return {
|
|
29
|
+
canAttempt(target) {
|
|
30
|
+
const entry = entries.get(key(target));
|
|
31
|
+
return !entry || entry.state === "closed" || ((entry.state === "half-open" || now() >= entry.openUntil) && entry.inFlight < probes);
|
|
32
|
+
},
|
|
33
|
+
acquire(target) {
|
|
34
|
+
const at = now(), id = key(target);
|
|
35
|
+
let entry = entries.get(id);
|
|
36
|
+
if (!entry) {
|
|
37
|
+
if (entries.size >= capacity) {
|
|
38
|
+
const idle = [...entries].filter(([, x]) => x.inFlight === 0 && x.state === "closed").sort((a, b) => a[1].touched - b[1].touched)[0];
|
|
39
|
+
if (!idle)
|
|
40
|
+
return undefined;
|
|
41
|
+
entries.delete(idle[0]);
|
|
42
|
+
}
|
|
43
|
+
entry = { state: "closed", failures: 0, inFlight: 0, epoch: 0, touched: at };
|
|
44
|
+
entries.set(id, entry);
|
|
45
|
+
}
|
|
46
|
+
if (entry.state === "open") {
|
|
47
|
+
if (at < entry.openUntil)
|
|
48
|
+
return undefined;
|
|
49
|
+
transition(entry, target, "half-open");
|
|
50
|
+
}
|
|
51
|
+
if (entry.state === "half-open" && entry.inFlight >= probes)
|
|
52
|
+
return undefined;
|
|
53
|
+
entry.inFlight++;
|
|
54
|
+
entry.touched = at;
|
|
55
|
+
const epoch = entry.epoch;
|
|
56
|
+
let ended = false;
|
|
57
|
+
return { end(outcome, retryAfterMs) {
|
|
58
|
+
if (ended)
|
|
59
|
+
return;
|
|
60
|
+
ended = true;
|
|
61
|
+
entry.inFlight--;
|
|
62
|
+
entry.touched = now();
|
|
63
|
+
if (epoch !== entry.epoch)
|
|
64
|
+
return;
|
|
65
|
+
if (outcome === "success") {
|
|
66
|
+
entry.failures = 0;
|
|
67
|
+
entry.openUntil = undefined;
|
|
68
|
+
if (entry.state !== "closed")
|
|
69
|
+
transition(entry, target, "closed");
|
|
70
|
+
}
|
|
71
|
+
if (outcome === "retryable-error") {
|
|
72
|
+
entry.failures++;
|
|
73
|
+
if (entry.state === "half-open" || entry.failures >= threshold) {
|
|
74
|
+
const serverDelay = Number.isFinite(retryAfterMs) && retryAfterMs > 0 ? retryAfterMs : 0;
|
|
75
|
+
entry.openUntil = now() + Math.min(maxCooldown, Math.max(cooldown, serverDelay));
|
|
76
|
+
transition(entry, target, "open");
|
|
77
|
+
}
|
|
78
|
+
}
|
|
79
|
+
} };
|
|
80
|
+
},
|
|
81
|
+
snapshot(target) { const entry = entries.get(key(target)); if (!entry)
|
|
82
|
+
return undefined; const { state, failures, inFlight, openUntil } = entry; return { state, failures, inFlight, openUntil }; }
|
|
83
|
+
};
|
|
84
|
+
};
|
|
85
|
+
//# sourceMappingURL=circuit-breaker.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"circuit-breaker.js","sourceRoot":"","sources":["../src/circuit-breaker.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,YAAY,EAA2B,MAAM,YAAY,CAAC;AAkBnE,MAAM,OAAO,uBAAwB,SAAQ,YAAY;IACvD,gBAAgB,KAAK,CAAC,8EAA8E,EAAE,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,GAAG,yBAAyB,CAAC,CAAC,CAAC;CACtJ;AACD,MAAM,CAAC,MAAM,2BAA2B,GAAG,CAAC,OAAO,GAA0B,EAAE,EAAyB,EAAE;IACxG,MAAM,SAAS,GAAG,OAAO,CAAC,gBAAgB,IAAI,CAAC,EAAE,QAAQ,GAAG,OAAO,CAAC,UAAU,IAAI,MAAM,EAAE,WAAW,GAAG,OAAO,CAAC,aAAa,IAAI,MAAM,EAAE,MAAM,GAAG,OAAO,CAAC,iBAAiB,IAAI,CAAC,EAAE,QAAQ,GAAG,OAAO,CAAC,UAAU,IAAI,GAAG,CAAC;IACvN,KAAK,MAAM,CAAC,KAAK,EAAE,GAAG,CAAC,IAAI,CAAC,CAAC,SAAS,EAAE,IAAI,CAAC,EAAE,CAAC,QAAQ,EAAE,UAAU,CAAC,EAAE,CAAC,WAAW,EAAE,UAAU,CAAC,EAAE,CAAC,MAAM,EAAE,GAAG,CAAC,EAAE,CAAC,QAAQ,EAAE,MAAM,CAAC,CAAC,EAAE,CAAC;QACrI,IAAI,CAAC,MAAM,CAAC,aAAa,CAAC,KAAK,CAAC,IAAI,KAAM,GAAG,CAAC,IAAI,KAAM,GAAG,GAAI;YAAE,MAAM,IAAI,YAAY,CAAC,iCAAiC,EAAE,KAAK,CAAC,CAAC;IACpI,CAAC;IACD,IAAI,QAAQ,GAAG,WAAW;QAAE,MAAM,IAAI,YAAY,CAAC,yCAAyC,EAAE,KAAK,CAAC,CAAC;IACrG,MAAM,GAAG,GAAG,OAAO,CAAC,GAAG,IAAI,IAAI,CAAC,GAAG,CAAC;IAEpC,MAAM,OAAO,GAAG,IAAI,GAAG,EAAiB,CAAC;IACzC,MAAM,GAAG,GAAG,CAAC,MAA0B,EAAE,EAAE,CAAC,IAAI,CAAC,SAAS,CAAC,CAAC,MAAM,CAAC,QAAQ,EAAE,MAAM,CAAC,OAAO,CAAC,CAAC,CAAC;IAC9F,MAAM,UAAU,GAAG,CAAC,KAAY,EAAE,MAA0B,EAAE,EAAuB,EAAE,EAAE;QACvF,MAAM,IAAI,GAAG,KAAK,CAAC,KAAK,CAAC;QAAC,KAAK,CAAC,KAAK,GAAG,EAAE,CAAC;QAAC,KAAK,CAAC,KAAK,EAAE,CAAC;QAC1D,IAAI,IAAI,KAAK,EAAE,IAAI,CAAC,OAAO,CAAC,aAAa;YAAE,OAAO;QAClD,MAAM,KAAK,GAAG,EAAE,MAAM,EAAE,EAAE,GAAG,MAAM,EAAE,EAAE,IAAI,EAAE,EAAE,EAAE,EAAE,EAAE,GAAG,EAAE,EAAE,SAAS,EAAE,KAAK,CAAC,SAAS,EAAE,CAAC;QACzF,cAAc,CAAC,GAAG,EAAE,GAAG,IAAI,CAAC;YAAC,KAAK,OAAO,CAAC,OAAO,CAAC,OAAO,CAAC,aAAc,CAAC,KAAK,CAAC,CAAC,CAAC,KAAK,CAAC,GAAG,EAAE,CAAC,SAAS,CAAC,CAAC;QAAC,CAAC;QAAC,MAAM,CAAC,CAAC,4CAA4C,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC;IACvK,CAAC,CAAC;IACF,OAAO;QACL,UAAU,CAAC,MAAM;YACf,MAAM,KAAK,GAAG,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC;YACvC,OAAO,CAAC,KAAK,IAAI,KAAK,CAAC,KAAK,KAAK,QAAQ,IAAI,CAAC,CAAC,KAAK,CAAC,KAAK,KAAK,WAAW,IAAI,GAAG,EAAE,IAAI,KAAK,CAAC,SAAU,CAAC,IAAI,KAAK,CAAC,QAAQ,GAAG,MAAM,CAAC,CAAC;QACvI,CAAC;QACD,OAAO,CAAC,MAAM;YACZ,MAAM,EAAE,GAAG,GAAG,EAAE,EAAE,EAAE,GAAG,GAAG,CAAC,MAAM,CAAC,CAAC;YACnC,IAAI,KAAK,GAAG,OAAO,CAAC,GAAG,CAAC,EAAE,CAAC,CAAC;YAC5B,IAAI,CAAC,KAAK,EAAE,CAAC;gBACX,IAAI,OAAO,CAAC,IAAI,IAAI,QAAQ,EAAE,CAAC;oBAC7B,MAAM,IAAI,GAAG,CAAC,GAAG,OAAO,CAAC,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,QAAQ,KAAK,CAAC,IAAI,CAAC,CAAC,KAAK,KAAK,QAAQ,CAAC,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,OAAO,CAAC,CAAC,CAAC,CAAC,CAAC;oBACrI,IAAI,CAAC,IAAI;wBAAE,OAAO,SAAS,CAAC;oBAC5B,OAAO,CAAC,MAAM,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC,CAAC;gBAC1B,CAAC;gBACD,KAAK,GAAG,EAAE,KAAK,EAAE,QAAQ,EAAE,QAAQ,EAAE,CAAC,EAAE,QAAQ,EAAE,CAAC,EAAE,KAAK,EAAE,CAAC,EAAE,OAAO,EAAE,EAAE,EAAE,CAAC;gBAAC,OAAO,CAAC,GAAG,CAAC,EAAE,EAAE,KAAK,CAAC,CAAC;YACvG,CAAC;YACD,IAAI,KAAK,CAAC,KAAK,KAAK,MAAM,EAAE,CAAC;gBAC3B,IAAI,EAAE,GAAG,KAAK,CAAC,SAAU;oBAAE,OAAO,SAAS,CAAC;gBAC5C,UAAU,CAAC,KAAK,EAAE,MAAM,EAAE,WAAW,CAAC,CAAC;YACzC,CAAC;YACD,IAAI,KAAK,CAAC,KAAK,KAAK,WAAW,IAAI,KAAK,CAAC,QAAQ,IAAI,MAAM;gBAAE,OAAO,SAAS,CAAC;YAC9E,KAAK,CAAC,QAAQ,EAAE,CAAC;YAAC,KAAK,CAAC,OAAO,GAAG,EAAE,CAAC;YACrC,MAAM,KAAK,GAAG,KAAK,CAAC,KAAK,CAAC;YAAC,IAAI,KAAK,GAAG,KAAK,CAAC;YAC7C,OAAO,EAAE,GAAG,CAAC,OAAO,EAAE,YAAY;oBAChC,IAAI,KAAK;wBAAE,OAAO;oBAAC,KAAK,GAAG,IAAI,CAAC;oBAAC,KAAM,CAAC,QAAQ,EAAE,CAAC;oBAAC,KAAM,CAAC,OAAO,GAAG,GAAG,EAAE,CAAC;oBAC3E,IAAI,KAAK,KAAK,KAAM,CAAC,KAAK;wBAAE,OAAO;oBACnC,IAAI,OAAO,KAAK,SAAS,EAAE,CAAC;wBAAC,KAAM,CAAC,QAAQ,GAAG,CAAC,CAAC;wBAAC,KAAM,CAAC,SAAS,GAAG,SAAS,CAAC;wBAAC,IAAI,KAAM,CAAC,KAAK,KAAK,QAAQ;4BAAE,UAAU,CAAC,KAAM,EAAE,MAAM,EAAE,QAAQ,CAAC,CAAC;oBAAC,CAAC;oBACtJ,IAAI,OAAO,KAAK,iBAAiB,EAAE,CAAC;wBAClC,KAAM,CAAC,QAAQ,EAAE,CAAC;wBAClB,IAAI,KAAM,CAAC,KAAK,KAAK,WAAW,IAAI,KAAM,CAAC,QAAQ,IAAI,SAAS,EAAE,CAAC;4BACjE,MAAM,WAAW,GAAG,MAAM,CAAC,QAAQ,CAAC,YAAY,CAAC,IAAI,YAAa,GAAG,CAAC,CAAC,CAAC,CAAC,YAAa,CAAC,CAAC,CAAC,CAAC,CAAC;4BAC3F,KAAM,CAAC,SAAS,GAAG,GAAG,EAAE,GAAG,IAAI,CAAC,GAAG,CAAC,WAAW,EAAE,IAAI,CAAC,GAAG,CAAC,QAAQ,EAAE,WAAW,CAAC,CAAC,CAAC;4BAClF,UAAU,CAAC,KAAM,EAAE,MAAM,EAAE,MAAM,CAAC,CAAC;wBACrC,CAAC;oBACH,CAAC;gBACH,CAAC,EAAE,CAAC;QACN,CAAC;QACD,QAAQ,CAAC,MAAM,IAAI,MAAM,KAAK,GAAG,OAAO,CAAC,GAAG,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK;YAAE,OAAO,SAAS,CAAC,CAAC,MAAM,EAAE,KAAK,EAAE,QAAQ,EAAE,QAAQ,EAAE,SAAS,EAAE,GAAG,KAAK,CAAC,CAAC,OAAO,EAAE,KAAK,EAAE,QAAQ,EAAE,QAAQ,EAAE,SAAS,EAAE,CAAC,CAAC,CAAC;KACpM,CAAC;AACJ,CAAC,CAAC"}
|
package/dist/index.d.ts
CHANGED
|
@@ -1,13 +1,18 @@
|
|
|
1
1
|
import type { ZodTypeAny } from "zod";
|
|
2
|
+
export type { GatewayAdaptiveCandidate, GatewayAdaptiveRoutingPolicy, GatewayQualityProfile } from "./adaptive-routing.js";
|
|
3
|
+
export { createGatewayCircuitBreaker, GatewayCircuitOpenError } from "./circuit-breaker.js";
|
|
4
|
+
export type { GatewayCircuitBreaker, GatewayCircuitOptions, GatewayCircuitPermit, GatewayCircuitSnapshot, GatewayCircuitState } from "./circuit-breaker.js";
|
|
5
|
+
export { createGatewayMetrics } from "./metrics.js";
|
|
6
|
+
export type { GatewayMetricsHandle, GatewayMetricsOptions, GatewayMetricsSnapshot, GatewayMetricsStore, GatewayMetricOutcome } from "./metrics.js";
|
|
2
7
|
import { type GatewayAgentRequest, type GatewayAgentResponse, type GatewayAgentStreamResult, type GatewayConfig, type GatewayGenerateObjectRequest, type GatewayObjectResponse, type GatewayRequest, type GatewayResponse, type GatewayStreamObjectResult, type GatewayStreamTextResult } from "./types.js";
|
|
3
8
|
export { GatewayError } from "./types.js";
|
|
4
9
|
export type { GatewayAgentRequest, GatewayAgentResponse, GatewayAgentStreamResult, GatewayAttempt, GatewayAttemptReasonCode, GatewayConfig, GatewayGenerateObjectRequest, GatewayImageAttachment, GatewayInputMessage, GatewayMessage, GatewayModelTarget, GatewayObjectResponse, GatewayProviderId, GatewayRequest, GatewayResponse, GatewayRouteDecisionReasonCode, GatewayRoutingMode, GatewayRoutingScoreContext, GatewayStreamObjectResult, GatewayStreamTextResult, GatewayTaskIntent, GatewayUnknownCostPolicy } from "./types.js";
|
|
5
10
|
export declare const createGateway: (config: GatewayConfig) => {
|
|
6
11
|
generate(request: GatewayRequest): Promise<GatewayResponse>;
|
|
7
|
-
streamText(request: GatewayRequest): GatewayStreamTextResult;
|
|
8
12
|
generateObject<TSchema extends ZodTypeAny>(request: GatewayGenerateObjectRequest<TSchema>): Promise<GatewayObjectResponse<TSchema>>;
|
|
9
|
-
streamObject<TSchema extends ZodTypeAny>(request: GatewayGenerateObjectRequest<TSchema>): GatewayStreamObjectResult<TSchema>;
|
|
10
13
|
runAgent(request: GatewayAgentRequest): Promise<GatewayAgentResponse>;
|
|
11
|
-
|
|
14
|
+
streamText: (request: GatewayRequest) => GatewayStreamTextResult;
|
|
15
|
+
streamObject: <TSchema extends ZodTypeAny>(request: GatewayGenerateObjectRequest<TSchema>) => GatewayStreamObjectResult<TSchema>;
|
|
16
|
+
streamAgent: (request: GatewayAgentRequest) => GatewayAgentStreamResult;
|
|
12
17
|
};
|
|
13
18
|
//# sourceMappingURL=index.d.ts.map
|
package/dist/index.d.ts.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"
|
|
1
|
+
{"version":3,"file":"index.d.ts","sourceRoot":"","sources":["../src/index.ts"],"names":[],"mappings":"AA8BA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,KAAK,CAAC;AAGtC,YAAY,EAAE,wBAAwB,EAAE,4BAA4B,EAAE,qBAAqB,EAAE,MAAM,uBAAuB,CAAC;AAC3H,OAAO,EAAE,2BAA2B,EAAE,uBAAuB,EAAE,MAAM,sBAAsB,CAAC;AAC5F,YAAY,EAAE,qBAAqB,EAAE,qBAAqB,EAAE,oBAAoB,EAAE,sBAAsB,EAAE,mBAAmB,EAAE,MAAM,sBAAsB,CAAC;AAE5J,OAAO,EAAE,oBAAoB,EAAE,MAAM,cAAc,CAAC;AACpD,YAAY,EAAE,oBAAoB,EAAE,qBAAqB,EAAE,sBAAsB,EAAE,mBAAmB,EAAE,oBAAoB,EAAE,MAAM,cAAc,CAAC;AAInJ,OAAO,EAEL,KAAK,mBAAmB,EACxB,KAAK,oBAAoB,EACzB,KAAK,wBAAwB,EAG7B,KAAK,aAAa,EAClB,KAAK,4BAA4B,EAEjC,KAAK,qBAAqB,EAE1B,KAAK,cAAc,EACnB,KAAK,eAAe,EAEpB,KAAK,yBAAyB,EAC9B,KAAK,uBAAuB,EAE7B,MAAM,YAAY,CAAC;AAEpB,OAAO,EAAE,YAAY,EAAE,MAAM,YAAY,CAAC;AAC1C,YAAY,EACV,mBAAmB,EACnB,oBAAoB,EACpB,wBAAwB,EACxB,cAAc,EACd,wBAAwB,EACxB,aAAa,EACb,4BAA4B,EAC5B,sBAAsB,EACtB,mBAAmB,EACnB,cAAc,EACd,kBAAkB,EAClB,qBAAqB,EACrB,iBAAiB,EACjB,cAAc,EACd,eAAe,EACf,8BAA8B,EAC9B,kBAAkB,EAClB,0BAA0B,EAC1B,yBAAyB,EACzB,uBAAuB,EACvB,iBAAiB,EACjB,wBAAwB,EACzB,MAAM,YAAY,CAAC;AA45BpB,eAAO,MAAM,aAAa,WAAY,aAAa;sBA6lBvB,cAAc,GAAG,OAAO,CAAC,eAAe,CAAC;mBAoC5C,OAAO,SAAS,UAAU,WACpC,4BAA4B,CAAC,OAAO,CAAC,GAC7C,OAAO,CAAC,qBAAqB,CAAC,OAAO,CAAC,CAAC;sBAmElB,mBAAmB,GAAG,OAAO,CAAC,oBAAoB,CAAC;0BAyJrD,cAAc;mBACrB,OAAO,SAAS,UAAU,WAAW,4BAA4B,CAAC,OAAO,CAAC;2BAClE,mBAAmB;CAE7C,CAAC"}
|