@plurnk/plurnk-providers 0.5.0 → 0.7.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +40 -3
- package/SPEC.md +2 -1
- package/dist/OpenAICompat.d.ts +1 -0
- package/dist/OpenAICompat.d.ts.map +1 -1
- package/dist/OpenAICompat.js +47 -14
- package/dist/OpenAICompat.js.map +1 -1
- package/dist/standardProviders.d.ts.map +1 -1
- package/dist/standardProviders.js +20 -0
- package/dist/standardProviders.js.map +1 -1
- package/package.json +2 -1
package/README.md
CHANGED
|
@@ -5,15 +5,52 @@ Framework + contract for `@plurnk/plurnk-providers-*` sibling packages (LLM tran
|
|
|
5
5
|
## Documentation
|
|
6
6
|
|
|
7
7
|
- [`SPEC.md`](./SPEC.md) — author-facing contract for sibling implementers.
|
|
8
|
-
- Constellation: [plurnk-grammar](https://github.com/plurnk/plurnk-grammar) (HEREDOC + AST), [plurnk-mimetypes](https://github.com/plurnk/plurnk-mimetypes), [plurnk-schemes](https://github.com/plurnk/plurnk-schemes), [plurnk-execs](https://github.com/plurnk/plurnk-execs).
|
|
8
|
+
- Constellation: [plurnk-grammar](https://github.com/plurnk/plurnk-grammar) (HEREDOC + AST), [plurnk-mimetypes](https://github.com/plurnk/plurnk-mimetypes), [plurnk-schemes](https://github.com/plurnk/plurnk-schemes), [plurnk-execs](https://github.com/plurnk/plurnk-execs) (the reference family this one mirrors).
|
|
9
|
+
|
|
10
|
+
## Write a provider
|
|
11
|
+
|
|
12
|
+
Ship a provider by publishing a package — **under any scope** (`@acme/whatever`; discovery keys on `plurnk.kind`, not the `@plurnk` scope) — that declares its name and default-exports a `fromEnv` factory. (A plain OpenAI-compatible endpoint with no probe or wire quirk needs *no* package at all for first-party use — it's a frozen `STANDARD_PROVIDERS` entry; the package path is for bespoke providers and any third party.)
|
|
13
|
+
|
|
14
|
+
### 1. Declare the name in `package.json`
|
|
15
|
+
|
|
16
|
+
```json
|
|
17
|
+
{
|
|
18
|
+
"plurnk": { "kind": "provider", "name": "acme" }
|
|
19
|
+
}
|
|
20
|
+
```
|
|
21
|
+
|
|
22
|
+
One package is **one** provider identity — the `<name>` segment of `PLURNK_MODEL_<alias>=<name>/<model>`. (Unlike execs' `runtimes[]` array; a provider is singular.)
|
|
23
|
+
|
|
24
|
+
### 2. Default-export a `fromEnv` factory
|
|
25
|
+
|
|
26
|
+
The framework calls `YourClass.fromEnv(env, model)` (sync or async) and expects a `Provider`. Two ways in:
|
|
27
|
+
|
|
28
|
+
- **OpenAI-compatible backends** (the common case): `fromEnv` reads its env (base URL, key), probes whatever it needs (catalog, context window, pricing), and returns **`new OpenAICompatProvider(config)`**. You write a `fromEnv` and a config object — the transport spine (SSE, usage normalization, `finishReason`, grammar transport, slot affinity) is inherited. See `OpenAICompatConfig` / SPEC §11.
|
|
29
|
+
- **Non-OpenAI backends**: `implements Provider` directly — `generate`, `contextSize`, `model`, `countTokens(text)`, `costFor(usage)`.
|
|
30
|
+
|
|
31
|
+
`fromEnv` **MUST fail fast with a named error** when required env is missing — name the var the operator must set. (Why a factory, not a base-class constructor like execs/mimes: a provider often async-probes at construction — SPEC §3.)
|
|
32
|
+
|
|
33
|
+
### 3. What `generate` receives — and returns
|
|
34
|
+
|
|
35
|
+
`generate({ messages, runId, signal?, grammar?, maxTokens? }) → Promise<ProviderResponse>`. Return **raw** wire output: `content` unparsed (the consumer parses the plurnk DSL — never parse it yourself), `reasoning` is the wire-reported CoT only. Honor `signal`. The provider never mutates `messages` or injects turns. `grammar` (GBNF) is attached only by backends that support it; all others ignore it (SPEC §13).
|
|
36
|
+
|
|
37
|
+
## Discovery & trust
|
|
38
|
+
|
|
39
|
+
`discover(options?)` scans **every installed package** under `<cwd>/node_modules` — scope-agnostic — for `plurnk.kind === "provider"`, returning `{ registry, skipped }` (name → package specifier).
|
|
40
|
+
|
|
41
|
+
- **Name collisions are fail-hard.** Two packages claiming the same provider name throw at discovery, naming both.
|
|
42
|
+
- **The standard table wins.** A scanned package whose name duplicates a built-in standard provider (`openai`, `groq`, …) is shadowed — tier 1 resolves first.
|
|
43
|
+
- **Trust gate.** `discover()` honors **`PLURNK_PLUGINS_TRUSTED_ONLY`** (host posture, plurnk-service#229): unset/`""`/`0` → every package registers (default, no regression); any value → `@plurnk/*` always trusted plus a comma-separated allowlist (`1` = first-party only). An untrusted package is discovered but **not** registered (returned in `Discovery.skipped`), so requesting its name yields a precise *untrusted* error — never a crash.
|
|
44
|
+
|
|
45
|
+
First-party daughters install flat via [`@plurnk/plurnk-providers-all`](https://github.com/plurnk/plurnk-providers-all) so the scan finds them; a third party publishes under their own scope and installs alongside.
|
|
9
46
|
|
|
10
47
|
## Exports
|
|
11
48
|
|
|
12
|
-
- `Provider`, `ChatMessage`, `ProviderResponse`, `ProviderAssistant`, `ProviderUsage`, `FinishReason`, `ProviderFactory` — types.
|
|
49
|
+
- `Provider`, `ChatMessage`, `ProviderResponse`, `ProviderAssistant`, `ProviderUsage`, `FinishReason`, `ProviderFactory` (+ `Discovery`, `DiscoverOptions`) — types.
|
|
13
50
|
- `parseAliasesFromEnv`, `resolveActiveAlias`, `instantiateProvider`, `loadActiveProvider`, `discover` — alias-cascade resolution + two-tier provider instantiation. Tier 1 is the standard table; tier 2 is a scope-agnostic `node_modules` scan for `plurnk.kind:"provider"` packages — first-party daughters (flat via `@plurnk/plurnk-providers-all`) and third-party providers under any scope, gated by the host `PLURNK_PLUGINS_TRUSTED_ONLY` allowlist. The framework is contract-only (SPEC §5).
|
|
14
51
|
- `OpenAICompatProvider` (+ `OpenAICompatConfig`, `ReasoningStyle`, `effortFromBudget`) — shared OpenAI-compatible transport spine; siblings extend it (SPEC §11). Transports GBNF grammar-constrained sampling for capable backends (SPEC §13).
|
|
15
52
|
- `chatCompletionStream`, `OpenAiHttpError`, `StreamResponse` — the shared SSE client.
|
|
16
|
-
- `parseRequiredInt`, `parseOptionalInt`, `requireEnv`, `reasoningBudgetFromEnv
|
|
53
|
+
- `parseRequiredInt`, `parseOptionalInt`, `requireEnv`, `reasoningBudgetFromEnv` — env helpers (SPEC §4; all required-with-named-errors, no in-code defaults).
|
|
17
54
|
- `normalizeUsage`, `computeCost` (+ `RawUsage`, `TokenRates`) — usage normalization to the §2 invariant and the single cost formula (SPEC §11).
|
|
18
55
|
- `ProviderError`, `classifyProviderError`, `toProviderError`, `providerSource` (+ `TelemetryEvent`, `ProviderTelemetryKind`) — the TelemetryEvent envelope for transport failures (SPEC §12).
|
|
19
56
|
- `tokenizerFor`, `tokenizerByPublisher`, `parseTokenizerFamily` (+ `TokenizerFamily`, `CountTokens`) — synchronous tokenizer strategies.
|
package/SPEC.md
CHANGED
|
@@ -107,7 +107,8 @@ Each provider's `fromEnv` reads these:
|
|
|
107
107
|
- **`PLURNK_PROVIDERS_REASONING_BUDGET`** — REQUIRED integer `>= -1`. One knob carries the whole side-channel-reasoning space: **`0`** off, **`-1`** on/adaptive (no cap — model/backend decides depth), **`N>0`** on/capped at `N`. The provider maps this single intent to the active backend's mechanism — llama-server `chat_template_kwargs: { enable_thinking }` (always emitted; the explicit FALSE is the only working off-switch — llama-server ignores `think` and per-request budgets, and its `--reasoning-budget` default otherwise keeps the channel live, fatal under an active grammar, §13), Ollama `think`, relay `include_reasoning`, cloud `reasoning_effort` (tier from `N`; adaptive `-1` omits the field, letting the API pick its depth), Anthropic `thinking: { type: "adaptive" }` (`-1`) / `budget_tokens` (`N`). For native backends the magnitude is irrelevant — only zero vs non-zero matters. Consumers state intent, never mechanism. (There is deliberately **no** knob for in-DSL `<<PLAN>>` reasoning — PLAN is forced by the grammar itself, the `plurnk-plan.gbnf` variant whose root opens with `<<PLAN:`, plus a consumer prompt line; it has no provider footprint. The former `PLURNK_PLAN` assistant-prefill was removed in 0.4.0 — it broke under grammar-constrained generation, stacking malformed plans, since the prefill is prompt text the grammar never sees. Plan-forcing now lives entirely in the grammar (`plurnk-plan.gbnf`) and the service prompt, #16.)
|
|
108
108
|
|
|
109
109
|
Read via `reasoningBudgetFromEnv` and **fail hard when unset** — configuration lives in the operator's env (the consumer's `.env.example` declares every var); the framework never defaults a knob in code.
|
|
110
|
-
- **`PLURNK_FETCH_TIMEOUT`** — service-wide ms ceiling on any single outbound request. Each `fromEnv` reads and passes as `AbortSignal.timeout`. Per-provider override envs are NOT part of the contract.
|
|
110
|
+
- **`PLURNK_FETCH_TIMEOUT`** — service-wide ms ceiling on any single outbound request (**per attempt**, not shared across retries). Each `fromEnv` reads and passes as `AbortSignal.timeout`. Per-provider override envs are NOT part of the contract.
|
|
111
|
+
- **`PLURNK_PROVIDER_RETRY_ATTEMPTS`** — REQUIRED non-negative integer (read via `parseRequiredInt`). The transient-failure retry budget: **`0`** surfaces the first failure; **`N`** retries up to `N` times on a *transient* classification only (`rate_limit` / `network_failure` — 429, 5xx, timeout, connection reset). Terminal kinds (`unauthorized`, `quota_exceeded`, `invalid_response`, `model_refused`) are never retried. Backoff is exponential from a `2000ms` base (`base * 2^(attempt-1)`), unless the server sent a `Retry-After` (which wins). The caller's `signal` aborts both the in-flight request and the backoff sleep. Lives in the shared `OpenAICompatProvider` so every provider inherits it uniformly; rides on the existing `classifyProviderError` (#18).
|
|
111
112
|
- **`PLURNK_PROVIDER_CONTEXT_SIZE`** — optional positive-integer override for the model's reported context window. Resolution: this env var → provider probe/config/table → `null`.
|
|
112
113
|
|
|
113
114
|
## §5 Alias cascade resolution
|
package/dist/OpenAICompat.d.ts
CHANGED
|
@@ -14,6 +14,7 @@ export type OpenAICompatConfig = {
|
|
|
14
14
|
supportsSlotPinning?: boolean;
|
|
15
15
|
slotCount?: number | null;
|
|
16
16
|
reasoningBudget: number;
|
|
17
|
+
retryAttempts: number;
|
|
17
18
|
};
|
|
18
19
|
export declare const effortFromBudget: (budget: number) => "low" | "medium" | "high";
|
|
19
20
|
export default class OpenAICompatProvider implements Provider {
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"OpenAICompat.d.ts","sourceRoot":"","sources":["../src/OpenAICompat.ts"],"names":[],"mappings":"AAUA,OAAO,KAAK,EAAE,WAAW,EAAgB,QAAQ,EAAE,gBAAgB,EAAE,aAAa,EAAE,MAAM,YAAY,CAAC;AAiBvG,MAAM,MAAM,cAAc,GAAG,MAAM,GAAG,OAAO,GAAG,mBAAmB,GAAG,QAAQ,GAAG,UAAU,CAAC;AAE5F,MAAM,MAAM,kBAAkB,GAAG;IAC7B,KAAK,EAAE,MAAM,CAAC;IACd,GAAG,EAAE,MAAM,CAAC;IACZ,cAAc,EAAE,MAAM,CAAC;IACvB,OAAO,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACjC,WAAW,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,cAAc,CAAC,EAAE,cAAc,CAAC;IAChC,WAAW,CAAC,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,MAAM,CAAC;IACvC,OAAO,CAAC,EAAE,CAAC,KAAK,EAAE,aAAa,KAAK,MAAM,CAAC;IAC3C,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,eAAe,CAAC,EAAE,OAAO,CAAC;IAE1B,mBAAmB,CAAC,EAAE,OAAO,CAAC;IAC9B,SAAS,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAK1B,eAAe,EAAE,MAAM,CAAC;
|
|
1
|
+
{"version":3,"file":"OpenAICompat.d.ts","sourceRoot":"","sources":["../src/OpenAICompat.ts"],"names":[],"mappings":"AAUA,OAAO,KAAK,EAAE,WAAW,EAAgB,QAAQ,EAAE,gBAAgB,EAAE,aAAa,EAAE,MAAM,YAAY,CAAC;AAiBvG,MAAM,MAAM,cAAc,GAAG,MAAM,GAAG,OAAO,GAAG,mBAAmB,GAAG,QAAQ,GAAG,UAAU,CAAC;AAE5F,MAAM,MAAM,kBAAkB,GAAG;IAC7B,KAAK,EAAE,MAAM,CAAC;IACd,GAAG,EAAE,MAAM,CAAC;IACZ,cAAc,EAAE,MAAM,CAAC;IACvB,OAAO,CAAC,EAAE,MAAM,CAAC,MAAM,EAAE,MAAM,CAAC,CAAC;IACjC,WAAW,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAC5B,cAAc,CAAC,EAAE,cAAc,CAAC;IAChC,WAAW,CAAC,EAAE,CAAC,IAAI,EAAE,MAAM,KAAK,MAAM,CAAC;IACvC,OAAO,CAAC,EAAE,CAAC,KAAK,EAAE,aAAa,KAAK,MAAM,CAAC;IAC3C,MAAM,CAAC,EAAE,MAAM,CAAC;IAChB,eAAe,CAAC,EAAE,OAAO,CAAC;IAE1B,mBAAmB,CAAC,EAAE,OAAO,CAAC;IAC9B,SAAS,CAAC,EAAE,MAAM,GAAG,IAAI,CAAC;IAK1B,eAAe,EAAE,MAAM,CAAC;IAIxB,aAAa,EAAE,MAAM,CAAC;CACzB,CAAC;AAqCF,eAAO,MAAM,gBAAgB,GAAI,QAAQ,MAAM,KAAG,KAAK,GAAG,QAAQ,GAAG,MAIpE,CAAC;AAIF,MAAM,CAAC,OAAO,OAAO,oBAAqB,YAAW,QAAQ;;gBAgB7C,MAAM,EAAE,kBAAkB;IAiBtC,IAAI,WAAW,IAAI,MAAM,GAAG,IAAI,CAA8B;IAC9D,IAAI,KAAK,IAAI,MAAM,CAAwB;IAE3C,WAAW,CAAC,IAAI,EAAE,MAAM,GAAG,MAAM;IACjC,OAAO,CAAC,KAAK,EAAE,aAAa,GAAG,MAAM;IAkD/B,QAAQ,CAAC,EAAE,QAAQ,EAAE,KAAK,EAAE,MAAM,EAAE,OAAO,EAAE,SAAS,EAAE,EAAE;QAAE,QAAQ,EAAE,WAAW,EAAE,CAAC;QAAC,KAAK,EAAE,MAAM,CAAC;QAAC,MAAM,CAAC,EAAE,WAAW,CAAC;QAAC,OAAO,CAAC,EAAE,MAAM,CAAC;QAAC,SAAS,CAAC,EAAE,MAAM,CAAA;KAAE,GAAG,OAAO,CAAC,gBAAgB,CAAC;CAiDrM"}
|
package/dist/OpenAICompat.js
CHANGED
|
@@ -7,15 +7,35 @@
|
|
|
7
7
|
// A sibling's fromEnv probes whatever it needs (catalog, pricing, context
|
|
8
8
|
// window), builds the config, and returns `new OpenAICompatProvider(config)`.
|
|
9
9
|
// Pure-config providers come from ./standardProviders.ts with no sibling at all.
|
|
10
|
-
import { chatCompletionStream } from "./openaiStream.js";
|
|
10
|
+
import { chatCompletionStream, OpenAiHttpError } from "./openaiStream.js";
|
|
11
11
|
import { normalizeUsage } from "./usage.js";
|
|
12
|
-
import { toProviderError } from "./telemetry.js";
|
|
12
|
+
import { toProviderError, classifyProviderError } from "./telemetry.js";
|
|
13
13
|
// Sampling guard under an active grammar (SPEC §13): greedy decoding under
|
|
14
14
|
// hard constraint masks degenerates into repetition loops at the server
|
|
15
15
|
// default of 1.0, so the floor rides per-request with every attached grammar —
|
|
16
16
|
// never rely on server launch flags. Probed on llama.cpp b894 + gemma-4-26B
|
|
17
17
|
// (plurnk-providers#9; reference: plurnk-grammar test/llama/gbnf-live.test.ts).
|
|
18
18
|
const GRAMMAR_REPEAT_PENALTY_FLOOR = 1.15;
|
|
19
|
+
// Exponential-backoff base for transient-failure retries (#18). Attempt N waits
|
|
20
|
+
// RETRY_BASE_DELAY_MS * 2^(N-1), unless the server sent a Retry-After (which
|
|
21
|
+
// wins). The magnitude is mechanism, not operator intent — the COUNT is the
|
|
22
|
+
// knob (PLURNK_PROVIDER_RETRY_ATTEMPTS); the base stays a constant.
|
|
23
|
+
const RETRY_BASE_DELAY_MS = 2000;
|
|
24
|
+
// Only these two classifications are transient and worth retrying: rate_limit
|
|
25
|
+
// (429) and network_failure (5xx, timeout, connection reset). unauthorized,
|
|
26
|
+
// quota_exceeded, invalid_response, model_refused are terminal — retrying just
|
|
27
|
+
// burns time and budget.
|
|
28
|
+
const RETRYABLE = new Set(["rate_limit", "network_failure"]);
|
|
29
|
+
// Sleep that rejects the moment `signal` aborts (caller cancellation must not
|
|
30
|
+
// wait out a backoff). Resolves normally on timeout.
|
|
31
|
+
const sleepWithAbort = (ms, signal) => new Promise((resolve, reject) => {
|
|
32
|
+
if (signal?.aborted) {
|
|
33
|
+
reject(signal.reason);
|
|
34
|
+
return;
|
|
35
|
+
}
|
|
36
|
+
const timer = setTimeout(resolve, ms);
|
|
37
|
+
signal?.addEventListener("abort", () => { clearTimeout(timer); reject(signal.reason); }, { once: true });
|
|
38
|
+
});
|
|
19
39
|
// SPEC §2 closed set. Wire values outside it (provider-specific or absent)
|
|
20
40
|
// collapse to null — the consumer treats null as "no signal".
|
|
21
41
|
const FINISH_REASONS = new Set(["stop", "length", "tool_calls", "content_filter"]);
|
|
@@ -43,6 +63,7 @@ export default class OpenAICompatProvider {
|
|
|
43
63
|
#supportsGrammar;
|
|
44
64
|
#supportsSlotPinning;
|
|
45
65
|
#slotCount;
|
|
66
|
+
#retryAttempts;
|
|
46
67
|
constructor(config) {
|
|
47
68
|
this.#model = config.model;
|
|
48
69
|
this.#url = config.url;
|
|
@@ -50,6 +71,7 @@ export default class OpenAICompatProvider {
|
|
|
50
71
|
this.#headers = config.headers ?? {};
|
|
51
72
|
this.#contextSize = config.contextSize ?? null;
|
|
52
73
|
this.#reasoningBudget = config.reasoningBudget;
|
|
74
|
+
this.#retryAttempts = config.retryAttempts;
|
|
53
75
|
this.#reasoningStyle = config.reasoningStyle ?? "none";
|
|
54
76
|
this.#countTokens = config.countTokens ?? heuristicTokens;
|
|
55
77
|
this.#costFor = config.costFor ?? (() => 0);
|
|
@@ -115,8 +137,6 @@ export default class OpenAICompatProvider {
|
|
|
115
137
|
throw new Error("generate: runId is required — the run's stable, opaque identity");
|
|
116
138
|
// Reject before any wire call when already aborted (SPEC §10.8).
|
|
117
139
|
signal?.throwIfAborted();
|
|
118
|
-
const timeoutSignal = AbortSignal.timeout(this.#fetchTimeoutMs);
|
|
119
|
-
const effectiveSignal = signal !== undefined ? AbortSignal.any([signal, timeoutSignal]) : timeoutSignal;
|
|
120
140
|
const body = {
|
|
121
141
|
model: this.#model,
|
|
122
142
|
messages,
|
|
@@ -125,17 +145,30 @@ export default class OpenAICompatProvider {
|
|
|
125
145
|
...(maxTokens !== undefined ? { max_tokens: maxTokens } : {}),
|
|
126
146
|
...this.#slotBody(runId),
|
|
127
147
|
};
|
|
148
|
+
// Transient-failure retry (#18). Each attempt gets a FRESH fetch timeout
|
|
149
|
+
// (the budget is per-request, not shared across retries); the caller's
|
|
150
|
+
// signal spans them all. Retry only the transient classifications, prefer
|
|
151
|
+
// a server Retry-After over the backoff, and let the caller's abort cut
|
|
152
|
+
// through both the in-flight request and the backoff sleep.
|
|
128
153
|
let raw;
|
|
129
|
-
|
|
130
|
-
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
|
|
134
|
-
|
|
135
|
-
|
|
136
|
-
|
|
137
|
-
|
|
138
|
-
|
|
154
|
+
for (let attempt = 0;; attempt++) {
|
|
155
|
+
const timeoutSignal = AbortSignal.timeout(this.#fetchTimeoutMs);
|
|
156
|
+
const effectiveSignal = signal !== undefined ? AbortSignal.any([signal, timeoutSignal]) : timeoutSignal;
|
|
157
|
+
try {
|
|
158
|
+
raw = await chatCompletionStream({ url: this.#url, headers: this.#headers, body, signal: effectiveSignal });
|
|
159
|
+
break;
|
|
160
|
+
}
|
|
161
|
+
catch (err) {
|
|
162
|
+
// Caller-initiated abort is cancellation — never retried or wrapped.
|
|
163
|
+
if (signal?.aborted)
|
|
164
|
+
throw err;
|
|
165
|
+
const { kind } = classifyProviderError(err);
|
|
166
|
+
// Terminal kind, or budget spent → surface the classified failure.
|
|
167
|
+
if (!RETRYABLE.has(kind) || attempt >= this.#retryAttempts)
|
|
168
|
+
throw toProviderError(err, this.#source);
|
|
169
|
+
const retryAfter = err instanceof OpenAiHttpError ? err.retryAfter : null;
|
|
170
|
+
await sleepWithAbort(retryAfter ?? RETRY_BASE_DELAY_MS * 2 ** attempt, signal);
|
|
171
|
+
}
|
|
139
172
|
}
|
|
140
173
|
return {
|
|
141
174
|
assistant: {
|
package/dist/OpenAICompat.js.map
CHANGED
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"OpenAICompat.js","sourceRoot":"","sources":["../src/OpenAICompat.ts"],"names":[],"mappings":"AAAA,yEAAyE;AACzE,oEAAoE;AACpE,wEAAwE;AACxE,EAAE;AACF,4EAA4E;AAC5E,2EAA2E;AAC3E,0EAA0E;AAC1E,8EAA8E;AAC9E,iFAAiF;AAGjF,OAAO,EAAE,oBAAoB,EAAE,MAAM,mBAAmB,CAAC;
|
|
1
|
+
{"version":3,"file":"OpenAICompat.js","sourceRoot":"","sources":["../src/OpenAICompat.ts"],"names":[],"mappings":"AAAA,yEAAyE;AACzE,oEAAoE;AACpE,wEAAwE;AACxE,EAAE;AACF,4EAA4E;AAC5E,2EAA2E;AAC3E,0EAA0E;AAC1E,8EAA8E;AAC9E,iFAAiF;AAGjF,OAAO,EAAE,oBAAoB,EAAE,eAAe,EAAE,MAAM,mBAAmB,CAAC;AAC1E,OAAO,EAAE,cAAc,EAAE,MAAM,YAAY,CAAC;AAC5C,OAAO,EAAE,eAAe,EAAE,qBAAqB,EAAE,MAAM,gBAAgB,CAAC;AAyCxE,2EAA2E;AAC3E,wEAAwE;AACxE,+EAA+E;AAC/E,4EAA4E;AAC5E,gFAAgF;AAChF,MAAM,4BAA4B,GAAG,IAAI,CAAC;AAE1C,gFAAgF;AAChF,6EAA6E;AAC7E,4EAA4E;AAC5E,oEAAoE;AACpE,MAAM,mBAAmB,GAAG,IAAI,CAAC;AAEjC,8EAA8E;AAC9E,4EAA4E;AAC5E,+EAA+E;AAC/E,yBAAyB;AACzB,MAAM,SAAS,GAAwB,IAAI,GAAG,CAAC,CAAC,YAAY,EAAE,iBAAiB,CAAC,CAAC,CAAC;AAElF,8EAA8E;AAC9E,qDAAqD;AACrD,MAAM,cAAc,GAAG,CAAC,EAAU,EAAE,MAA+B,EAAiB,EAAE,CAClF,IAAI,OAAO,CAAC,CAAC,OAAO,EAAE,MAAM,EAAE,EAAE;IAC5B,IAAI,MAAM,EAAE,OAAO,EAAE,CAAC;QAAC,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC;QAAC,OAAO;IAAC,CAAC;IACvD,MAAM,KAAK,GAAG,UAAU,CAAC,OAAO,EAAE,EAAE,CAAC,CAAC;IACtC,MAAM,EAAE,gBAAgB,CAAC,OAAO,EAAE,GAAG,EAAE,GAAG,YAAY,CAAC,KAAK,CAAC,CAAC,CAAC,MAAM,CAAC,MAAM,CAAC,MAAM,CAAC,CAAC,CAAC,CAAC,EAAE,EAAE,IAAI,EAAE,IAAI,EAAE,CAAC,CAAC;AAC7G,CAAC,CAAC,CAAC;AAEP,2EAA2E;AAC3E,8DAA8D;AAC9D,MAAM,cAAc,GAAwB,IAAI,GAAG,CAAC,CAAC,MAAM,EAAE,QAAQ,EAAE,YAAY,EAAE,gBAAgB,CAAC,CAAC,CAAC;AACxG,MAAM,qBAAqB,GAAG,CAAC,GAAkB,EAAgB,EAAE,CAC/D,GAAG,KAAK,IAAI,IAAI,cAAc,CAAC,GAAG,CAAC,GAAG,CAAC,CAAC,CAAC,CAAE,GAAoB,CAAC,CAAC,CAAC,IAAI,CAAC;AAE3E,0EAA0E;AAC1E,MAAM,CAAC,MAAM,gBAAgB,GAAG,CAAC,MAAc,EAA6B,EAAE;IAC1E,IAAI,MAAM,IAAI,IAAI;QAAE,OAAO,KAAK,CAAC;IACjC,IAAI,MAAM,IAAI,IAAI;QAAE,OAAO,QAAQ,CAAC;IACpC,OAAO,MAAM,CAAC;AAClB,CAAC,CAAC;AAEF,MAAM,eAAe,GAAG,CAAC,IAAY,EAAU,EAAE,CAAC,CAAC,IAAI,CAAC,MAAM,KAAK,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC;AAEvG,MAAM,CAAC,OAAO,OAAO,oBAAoB;IACrC,MAAM,CAAS;IACf,IAAI,CAAS;IACb,eAAe,CAAS;IACxB,QAAQ,CAAyB;IACjC,YAAY,CAAgB;IAC5B,gBAAgB,CAAS;IACzB,eAAe,CAAiB;IAChC,YAAY,CAA2B;IACvC,QAAQ,CAAmC;IAC3C,OAAO,CAAS;IAChB,gBAAgB,CAAU;IAC1B,oBAAoB,CAAU;IAC9B,UAAU,CAAgB;IAC1B,cAAc,CAAS;IAEvB,YAAY,MAA0B;QAClC,IAAI,CAAC,MAAM,GAAG,MAAM,CAAC,KAAK,CAAC;QAC3B,IAAI,CAAC,IAAI,GAAG,MAAM,CAAC,GAAG,CAAC;QACvB,IAAI,CAAC,eAAe,GAAG,MAAM,CAAC,cAAc,CAAC;QAC7C,IAAI,CAAC,QAAQ,GAAG,MAAM,CAAC,OAAO,IAAI,EAAE,CAAC;QACrC,IAAI,CAAC,YAAY,GAAG,MAAM,CAAC,WAAW,IAAI,IAAI,CAAC;QAC/C,IAAI,CAAC,gBAAgB,GAAG,MAAM,CAAC,eAAe,CAAC;QAC/C,IAAI,CAAC,cAAc,GAAG,MAAM,CAAC,aAAa,CAAC;QAC3C,IAAI,CAAC,eAAe,GAAG,MAAM,CAAC,cAAc,IAAI,MAAM,CAAC;QACvD,IAAI,CAAC,YAAY,GAAG,MAAM,CAAC,WAAW,IAAI,eAAe,CAAC;QAC1D,IAAI,CAAC,QAAQ,GAAG,MAAM,CAAC,OAAO,IAAI,CAAC,GAAG,EAAE,CAAC,CAAC,CAAC,CAAC;QAC5C,IAAI,CAAC,OAAO,GAAG,MAAM,CAAC,MAAM,IAAI,UAAU,CAAC;QAC3C,IAAI,CAAC,gBAAgB,GAAG,MAAM,CAAC,eAAe,IAAI,KAAK,CAAC;QACxD,IAAI,CAAC,oBAAoB,GAAG,MAAM,CAAC,mBAAmB,IAAI,KAAK,CAAC;QAChE,IAAI,CAAC,UAAU,GAAG,MAAM,CAAC,SAAS,IAAI,IAAI,CAAC;IAC/C,CAAC;IAED,IAAI,WAAW,KAAoB,OAAO,IAAI,CAAC,YAAY,CAAC,CAAC,CAAC;IAC9D,IAAI,KAAK,KAAa,OAAO,IAAI,CAAC,MAAM,CAAC,CAAC,CAAC;IAE3C,WAAW,CAAC,IAAY,IAAY,OAAO,IAAI,CAAC,YAAY,CAAC,IAAI,CAAC,CAAC,CAAC,CAAC;IACrE,OAAO,CAAC,KAAoB,IAAY,OAAO,IAAI,CAAC,QAAQ,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC;IAEtE,cAAc;QACV,MAAM,CAAC,GAAG,IAAI,CAAC,gBAAgB,CAAC,CAAG,iCAAiC;QACpE,MAAM,EAAE,GAAG,CAAC,KAAK,CAAC,CAAC;QACnB,QAAQ,IAAI,CAAC,eAAe,EAAE,CAAC;YAC3B,gEAAgE;YAChE,0DAA0D;YAC1D,4EAA4E;YAC5E,KAAK,UAAU,CAAC,CAAC,OAAO,EAAE,oBAAoB,EAAE,EAAE,eAAe,EAAE,EAAE,EAAE,EAAE,CAAC;YAC1E,KAAK,OAAO,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC/C,KAAK,mBAAmB,CAAC,CAAC,OAAO,EAAE,CAAC,CAAC,CAAC,EAAE,iBAAiB,EAAE,IAAI,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YACvE,mEAAmE;YACnE,wDAAwD;YACxD,KAAK,QAAQ,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,EAAE,gBAAgB,EAAE,gBAAgB,CAAC,CAAC,CAAC,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC7E,KAAK,MAAM,CAAC,CAAC,OAAO,EAAE,CAAC;QAC3B,CAAC;IACL,CAAC;IAED,0EAA0E;IAC1E,2EAA2E;IAC3E,0EAA0E;IAC1E,uEAAuE;IACvE,4EAA4E;IAC5E,SAAS,GAAG,IAAI,GAAG,EAAkB,CAAC;IACtC,SAAS,GAAG,CAAC,CAAC;IAEd,SAAS,CAAC,KAAa;QACnB,IAAI,CAAC,IAAI,CAAC,oBAAoB,IAAI,IAAI,CAAC,UAAU,KAAK,IAAI,IAAI,IAAI,CAAC,UAAU,GAAG,CAAC;YAAE,OAAO,EAAE,CAAC;QAC7F,IAAI,IAAI,GAAG,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,KAAK,CAAC,CAAC;QACrC,IAAI,IAAI,KAAK,SAAS,EAAE,CAAC;YACrB,IAAI,GAAG,IAAI,CAAC,SAAS,EAAE,GAAG,IAAI,CAAC,UAAU,CAAC;YAC1C,IAAI,IAAI,CAAC,SAAS,CAAC,IAAI,IAAI,IAAI,CAAC,UAAU,GAAG,CAAC,EAAE,CAAC;gBAC7C,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI,EAAE,CAAC,IAAI,EAAE,CAAC,KAAe,CAAC,CAAC;YACxE,CAAC;QACL,CAAC;aAAM,CAAC;YACJ,IAAI,CAAC,SAAS,CAAC,MAAM,CAAC,KAAK,CAAC,CAAC,CAAC,mCAAmC;QACrE,CAAC;QACD,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,KAAK,EAAE,IAAI,CAAC,CAAC;QAChC,OAAO,EAAE,OAAO,EAAE,IAAI,EAAE,CAAC;IAC7B,CAAC;IAED,yEAAyE;IACzE,2EAA2E;IAC3E,2EAA2E;IAC3E,YAAY,CAAC,OAA2B;QACpC,IAAI,OAAO,KAAK,SAAS,IAAI,CAAC,IAAI,CAAC,gBAAgB;YAAE,OAAO,EAAE,CAAC;QAC/D,OAAO,EAAE,OAAO,EAAE,cAAc,EAAE,4BAA4B,EAAE,CAAC;IACrE,CAAC;IAED,KAAK,CAAC,QAAQ,CAAC,EAAE,QAAQ,EAAE,KAAK,EAAE,MAAM,EAAE,OAAO,EAAE,SAAS,EAA0G;QAClK,+DAA+D;QAC/D,IAAI,KAAK,KAAK,SAAS,IAAI,KAAK,CAAC,MAAM,KAAK,CAAC;YAAE,MAAM,IAAI,KAAK,CAAC,iEAAiE,CAAC,CAAC;QAClI,iEAAiE;QACjE,MAAM,EAAE,cAAc,EAAE,CAAC;QAEzB,MAAM,IAAI,GAA4B;YAClC,KAAK,EAAE,IAAI,CAAC,MAAM;YAClB,QAAQ;YACR,GAAG,IAAI,CAAC,cAAc,EAAE;YACxB,GAAG,IAAI,CAAC,YAAY,CAAC,OAAO,CAAC;YAC7B,GAAG,CAAC,SAAS,KAAK,SAAS,CAAC,CAAC,CAAC,EAAE,UAAU,EAAE,SAAS,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;YAC7D,GAAG,IAAI,CAAC,SAAS,CAAC,KAAK,CAAC;SAC3B,CAAC;QAEF,yEAAyE;QACzE,uEAAuE;QACvE,0EAA0E;QAC1E,wEAAwE;QACxE,4DAA4D;QAC5D,IAAI,GAAG,CAAC;QACR,KAAK,IAAI,OAAO,GAAG,CAAC,GAAI,OAAO,EAAE,EAAE,CAAC;YAChC,MAAM,aAAa,GAAG,WAAW,CAAC,OAAO,CAAC,IAAI,CAAC,eAAe,CAAC,CAAC;YAChE,MAAM,eAAe,GAAG,MAAM,KAAK,SAAS,CAAC,CAAC,CAAC,WAAW,CAAC,GAAG,CAAC,CAAC,MAAM,EAAE,aAAa,CAAC,CAAC,CAAC,CAAC,CAAC,aAAa,CAAC;YACxG,IAAI,CAAC;gBACD,GAAG,GAAG,MAAM,oBAAoB,CAAC,EAAE,GAAG,EAAE,IAAI,CAAC,IAAI,EAAE,OAAO,EAAE,IAAI,CAAC,QAAQ,EAAE,IAAI,EAAE,MAAM,EAAE,eAAe,EAAE,CAAC,CAAC;gBAC5G,MAAM;YACV,CAAC;YAAC,OAAO,GAAG,EAAE,CAAC;gBACX,qEAAqE;gBACrE,IAAI,MAAM,EAAE,OAAO;oBAAE,MAAM,GAAG,CAAC;gBAC/B,MAAM,EAAE,IAAI,EAAE,GAAG,qBAAqB,CAAC,GAAG,CAAC,CAAC;gBAC5C,mEAAmE;gBACnE,IAAI,CAAC,SAAS,CAAC,GAAG,CAAC,IAAI,CAAC,IAAI,OAAO,IAAI,IAAI,CAAC,cAAc;oBAAE,MAAM,eAAe,CAAC,GAAG,EAAE,IAAI,CAAC,OAAO,CAAC,CAAC;gBACrG,MAAM,UAAU,GAAG,GAAG,YAAY,eAAe,CAAC,CAAC,CAAC,GAAG,CAAC,UAAU,CAAC,CAAC,CAAC,IAAI,CAAC;gBAC1E,MAAM,cAAc,CAAC,UAAU,IAAI,mBAAmB,GAAG,CAAC,IAAI,OAAO,EAAE,MAAM,CAAC,CAAC;YACnF,CAAC;QACL,CAAC;QAED,OAAO;YACH,SAAS,EAAE;gBACP,OAAO,EAAE,GAAG,CAAC,OAAO;gBACpB,SAAS,EAAE,GAAG,CAAC,iBAAiB,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,iBAAiB,CAAC,CAAC,CAAC,IAAI;gBAC1E,KAAK,EAAE,cAAc,CAAC,GAAG,CAAC,KAAK,CAAC;gBAChC,YAAY,EAAE,qBAAqB,CAAC,GAAG,CAAC,aAAa,CAAC;gBACtD,KAAK,EAAE,GAAG,CAAC,KAAK,IAAI,IAAI,CAAC,MAAM;aAClC;YACD,YAAY,EAAE,GAAG;SACpB,CAAC;IACN,CAAC;CACJ"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"standardProviders.d.ts","sourceRoot":"","sources":["../src/standardProviders.ts"],"names":[],"mappings":"AAUA,OAAO,KAAK,EAAE,QAAQ,
|
|
1
|
+
{"version":3,"file":"standardProviders.d.ts","sourceRoot":"","sources":["../src/standardProviders.ts"],"names":[],"mappings":"AAUA,OAAO,KAAK,EAAE,QAAQ,EAAiB,MAAM,YAAY,CAAC;AAC1D,OAA6B,EAAE,KAAK,cAAc,EAAE,MAAM,mBAAmB,CAAC;AAE9E,OAAO,EAAsC,KAAK,eAAe,EAAE,MAAM,iBAAiB,CAAC;AAK3F,KAAK,oBAAoB,GAAG;IAGxB,SAAS,EAAE,MAAM,CAAC;IAClB,cAAc,EAAE,OAAO,CAAC;IAGxB,OAAO,CAAC,EAAE,MAAM,CAAC;IACjB,UAAU,CAAC,EAAE,MAAM,CAAC;IAEpB,QAAQ,EAAE,MAAM,CAAC;IAIjB,aAAa,CAAC,EAAE,OAAO,CAAC;IACxB,cAAc,EAAE,cAAc,CAAC;IAC/B,gBAAgB,EAAE,eAAe,CAAC;IAClC,eAAe,EAAE,MAAM,CAAC;IAOxB,SAAS,CAAC,EAAE,OAAO,CAAC;CACvB,CAAC;AAGF,eAAO,MAAM,kBAAkB,EAAE,QAAQ,CAAC,MAAM,CAAC,MAAM,EAAE,oBAAoB,CAAC,CAwC5E,CAAC;AAEH,eAAO,MAAM,kBAAkB,GAAI,MAAM,MAAM,KAAG,OAAqC,CAAC;AAwDxF,eAAO,MAAM,uBAAuB,GAAU,MAAM,MAAM,EAAE,KAAK,MAAM,CAAC,UAAU,EAAE,OAAO,MAAM,KAAG,OAAO,CAAC,QAAQ,GAAG,IAAI,CAmE1H,CAAC"}
|
|
@@ -11,6 +11,8 @@ import OpenAICompatProvider from "./OpenAICompat.js";
|
|
|
11
11
|
import { parseRequiredInt, parseOptionalInt, requireEnv, reasoningBudgetFromEnv } from "./env.js";
|
|
12
12
|
import { parseTokenizerFamily, tokenizerFor } from "./tokenizers.js";
|
|
13
13
|
import { providerSource } from "./telemetry.js";
|
|
14
|
+
import { computeCost } from "./usage.js";
|
|
15
|
+
import { lookup } from "@plurnk/plurnk-models";
|
|
14
16
|
// Frozen so a downstream can't mutate the shared table.
|
|
15
17
|
export const STANDARD_PROVIDERS = Object.freeze({
|
|
16
18
|
// Generic OpenAI-compatible endpoint (OpenAI proper, llama-server, vLLM,
|
|
@@ -133,6 +135,22 @@ export const standardProviderFromEnv = async (name, env, model) => {
|
|
|
133
135
|
if (probe.llamaServer)
|
|
134
136
|
slotCount = await probeSlotCount(url, headers, fetchTimeoutMs);
|
|
135
137
|
}
|
|
138
|
+
// Vendored-snapshot FALLBACK (#19) — live always wins. contextSize already
|
|
139
|
+
// preferred env then the live probe; the catalog only fills a still-null
|
|
140
|
+
// window for a known cloud model (groq/deepseek/mistral/…, which don't
|
|
141
|
+
// probe). A local llama-server model misses the catalog and keeps its
|
|
142
|
+
// probed n_ctx. Standard providers carry NO live pricing, so the catalog is
|
|
143
|
+
// the sole — never shadowing — cost source; per-1M USD → pico-USD/token (×1e6).
|
|
144
|
+
const fallback = lookup(name, model);
|
|
145
|
+
contextSize ??= fallback?.contextWindow ?? null;
|
|
146
|
+
const cost = fallback?.cost;
|
|
147
|
+
const costFor = cost === undefined
|
|
148
|
+
? undefined
|
|
149
|
+
: (usage) => computeCost(usage, {
|
|
150
|
+
input: cost.inputPer1M * 1e6,
|
|
151
|
+
output: cost.outputPer1M * 1e6,
|
|
152
|
+
cached: (cost.cacheReadPer1M ?? cost.inputPer1M) * 1e6,
|
|
153
|
+
});
|
|
136
154
|
return new OpenAICompatProvider({
|
|
137
155
|
model,
|
|
138
156
|
url,
|
|
@@ -140,8 +158,10 @@ export const standardProviderFromEnv = async (name, env, model) => {
|
|
|
140
158
|
contextSize,
|
|
141
159
|
fetchTimeoutMs,
|
|
142
160
|
reasoningBudget: reasoningBudgetFromEnv(env, name),
|
|
161
|
+
retryAttempts: parseRequiredInt(env.PLURNK_PROVIDER_RETRY_ATTEMPTS, "PLURNK_PROVIDER_RETRY_ATTEMPTS", name),
|
|
143
162
|
reasoningStyle,
|
|
144
163
|
countTokens: tokenizerFor(family),
|
|
164
|
+
costFor,
|
|
145
165
|
source: providerSource(name),
|
|
146
166
|
supportsGrammar,
|
|
147
167
|
// The same fingerprint backs both llama-server dialect extensions.
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"standardProviders.js","sourceRoot":"","sources":["../src/standardProviders.ts"],"names":[],"mappings":"AAAA,8EAA8E;AAC9E,8EAA8E;AAC9E,2EAA2E;AAC3E,8EAA8E;AAC9E,6DAA6D;AAC7D,EAAE;AACF,4EAA4E;AAC5E,8EAA8E;AAC9E,2EAA2E;AAG3E,OAAO,oBAA6C,MAAM,mBAAmB,CAAC;AAC9E,OAAO,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,UAAU,EAAE,sBAAsB,EAAE,MAAM,UAAU,CAAC;AAClG,OAAO,EAAE,oBAAoB,EAAE,YAAY,EAAwB,MAAM,iBAAiB,CAAC;AAC3F,OAAO,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;
|
|
1
|
+
{"version":3,"file":"standardProviders.js","sourceRoot":"","sources":["../src/standardProviders.ts"],"names":[],"mappings":"AAAA,8EAA8E;AAC9E,8EAA8E;AAC9E,2EAA2E;AAC3E,8EAA8E;AAC9E,6DAA6D;AAC7D,EAAE;AACF,4EAA4E;AAC5E,8EAA8E;AAC9E,2EAA2E;AAG3E,OAAO,oBAA6C,MAAM,mBAAmB,CAAC;AAC9E,OAAO,EAAE,gBAAgB,EAAE,gBAAgB,EAAE,UAAU,EAAE,sBAAsB,EAAE,MAAM,UAAU,CAAC;AAClG,OAAO,EAAE,oBAAoB,EAAE,YAAY,EAAwB,MAAM,iBAAiB,CAAC;AAC3F,OAAO,EAAE,cAAc,EAAE,MAAM,gBAAgB,CAAC;AAChD,OAAO,EAAE,WAAW,EAAE,MAAM,YAAY,CAAC;AACzC,OAAO,EAAE,MAAM,EAAE,MAAM,uBAAuB,CAAC;AA6B/C,wDAAwD;AACxD,MAAM,CAAC,MAAM,kBAAkB,GAAmD,MAAM,CAAC,MAAM,CAAC;IAC5F,yEAAyE;IACzE,wEAAwE;IACxE,wEAAwE;IACxE,MAAM,EAAE;QACJ,SAAS,EAAE,gBAAgB,EAAE,cAAc,EAAE,KAAK;QAClD,UAAU,EAAE,iBAAiB,EAAE,QAAQ,EAAE,sBAAsB,EAAE,aAAa,EAAE,IAAI;QACpF,cAAc,EAAE,OAAO,EAAE,gBAAgB,EAAE,WAAW,EAAE,eAAe,EAAE,kBAAkB;QAC3F,SAAS,EAAE,IAAI;KAClB;IACD,IAAI,EAAE;QACF,SAAS,EAAE,cAAc,EAAE,cAAc,EAAE,IAAI;QAC/C,OAAO,EAAE,gCAAgC,EAAE,UAAU,EAAE,eAAe,EAAE,QAAQ,EAAE,mBAAmB;QACrG,cAAc,EAAE,QAAQ,EAAE,gBAAgB,EAAE,WAAW,EAAE,eAAe,EAAE,gBAAgB;KAC7F;IACD,QAAQ,EAAE;QACN,SAAS,EAAE,kBAAkB,EAAE,cAAc,EAAE,IAAI;QACnD,OAAO,EAAE,6BAA6B,EAAE,UAAU,EAAE,mBAAmB,EAAE,QAAQ,EAAE,mBAAmB;QACtG,cAAc,EAAE,MAAM,EAAE,gBAAgB,EAAE,WAAW,EAAE,eAAe,EAAE,oBAAoB;KAC/F;IACD,OAAO,EAAE;QACL,SAAS,EAAE,iBAAiB,EAAE,cAAc,EAAE,IAAI;QAClD,OAAO,EAAE,2BAA2B,EAAE,UAAU,EAAE,kBAAkB,EAAE,QAAQ,EAAE,mBAAmB;QACnG,cAAc,EAAE,MAAM,EAAE,gBAAgB,EAAE,WAAW,EAAE,eAAe,EAAE,mBAAmB;KAC9F;IACD,QAAQ,EAAE;QACN,SAAS,EAAE,kBAAkB,EAAE,cAAc,EAAE,IAAI;QACnD,OAAO,EAAE,6BAA6B,EAAE,UAAU,EAAE,mBAAmB,EAAE,QAAQ,EAAE,mBAAmB;QACtG,cAAc,EAAE,MAAM,EAAE,gBAAgB,EAAE,WAAW,EAAE,eAAe,EAAE,oBAAoB;KAC/F;IACD,SAAS,EAAE;QACP,SAAS,EAAE,mBAAmB,EAAE,cAAc,EAAE,IAAI;QACpD,OAAO,EAAE,uCAAuC,EAAE,UAAU,EAAE,oBAAoB,EAAE,QAAQ,EAAE,mBAAmB;QACjH,cAAc,EAAE,MAAM,EAAE,gBAAgB,EAAE,WAAW,EAAE,eAAe,EAAE,qBAAqB;KAChG;IACD,SAAS,EAAE;QACP,SAAS,EAAE,mBAAmB,EAAE,cAAc,EAAE,IAAI;QACpD,OAAO,EAAE,qCAAqC,EAAE,UAAU,EAAE,oBAAoB,EAAE,QAAQ,EAAE,mBAAmB;QAC/G,cAAc,EAAE,MAAM,EAAE,gBAAgB,EAAE,WAAW,EAAE,eAAe,EAAE,qBAAqB;KAChG;CACJ,CAAC,CAAC;AAEH,MAAM,CAAC,MAAM,kBAAkB,GAAG,CAAC,IAAY,EAAW,EAAE,CAAC,IAAI,IAAI,kBAAkB,CAAC;AAExF,MAAM,UAAU,GAAG,CAAC,IAA0B,EAAE,GAAsB,EAAE,KAAa,EAAU,EAAE;IAC7F,MAAM,QAAQ,GAAG,IAAI,CAAC,UAAU,KAAK,SAAS,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,UAAU,CAAC,CAAC,CAAC,CAAC,SAAS,CAAC;IAClF,MAAM,IAAI,GAAG,QAAQ,KAAK,SAAS,IAAI,QAAQ,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,CAAC,QAAQ,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC;IACrF,IAAI,IAAI,KAAK,SAAS,IAAI,IAAI,CAAC,MAAM,KAAK,CAAC,EAAE,CAAC;QAC1C,MAAM,IAAI,KAAK,CAAC,GAAG,KAAK,cAAc,IAAI,CAAC,UAAU,IAAI,UAAU,cAAc,CAAC,CAAC;IACvF,CAAC;IACD,MAAM,OAAO,GAAG,IAAI,CAAC,aAAa,KAAK,IAAI,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,UAAU,EAAE,EAAE,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,OAAO,CAAC,KAAK,EAAE,EAAE,CAAC,CAAC;IACrG,OAAO,GAAG,OAAO,GAAG,IAAI,CAAC,QAAQ,EAAE,CAAC;AACxC,CAAC,CAAC;AAUF,MAAM,WAAW,GAAG,KAAK,EAAE,OAAe,EAAE,OAA+B,EAAE,KAAa,EAAE,cAAsB,EAA0B,EAAE;IAC1I,MAAM,SAAS,GAAG,OAAO,CAAC,OAAO,CAAC,sBAAsB,EAAE,SAAS,CAAC,CAAC;IACrE,IAAI,CAAC;QACD,MAAM,GAAG,GAAG,MAAM,KAAK,CAAC,SAAS,EAAE,EAAE,OAAO,EAAE,MAAM,EAAE,WAAW,CAAC,OAAO,CAAC,cAAc,CAAC,EAAE,CAAC,CAAC;QAC7F,IAAI,CAAC,GAAG,CAAC,EAAE;YAAE,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;QACvD,MAAM,IAAI,GAAG,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAAiF,CAAC;QAChH,MAAM,IAAI,GAAG,IAAI,CAAC,IAAI,IAAI,EAAE,CAAC;QAC7B,MAAM,GAAG,GAAG,IAAI,CAAC,IAAI,CAAC,CAAC,CAAC,EAAE,EAAE,CAAC,CAAC,CAAC,EAAE,KAAK,KAAK,CAAC,IAAI,IAAI,CAAC,CAAC,CAAC,CAAC;QACxD,MAAM,CAAC,GAAG,GAAG,EAAE,IAAI,EAAE,KAAK,IAAI,GAAG,EAAE,KAAK,CAAC;QACzC,OAAO;YACH,IAAI,EAAE,OAAO,CAAC,KAAK,QAAQ,IAAI,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,CAAC,IAAI;YAC/C,WAAW,EAAE,GAAG,EAAE,IAAI,KAAK,SAAS;SACvC,CAAC;IACN,CAAC;IAAC,MAAM,CAAC;QACL,OAAO,EAAE,IAAI,EAAE,IAAI,EAAE,WAAW,EAAE,KAAK,EAAE,CAAC;IAC9C,CAAC;AACL,CAAC,CAAC;AAEF,0EAA0E;AAC1E,6EAA6E;AAC7E,0DAA0D;AAC1D,MAAM,cAAc,GAAG,KAAK,EAAE,OAAe,EAAE,OAA+B,EAAE,cAAsB,EAA0B,EAAE;IAC9H,MAAM,QAAQ,GAAG,OAAO,CAAC,OAAO,CAAC,0BAA0B,EAAE,QAAQ,CAAC,CAAC;IACvE,IAAI,CAAC;QACD,MAAM,GAAG,GAAG,MAAM,KAAK,CAAC,QAAQ,EAAE,EAAE,OAAO,EAAE,MAAM,EAAE,WAAW,CAAC,OAAO,CAAC,cAAc,CAAC,EAAE,CAAC,CAAC;QAC5F,IAAI,CAAC,GAAG,CAAC,EAAE;YAAE,OAAO,IAAI,CAAC;QACzB,MAAM,IAAI,GAAG,CAAC,MAAM,GAAG,CAAC,IAAI,EAAE,CAA6B,CAAC;QAC5D,OAAO,OAAO,IAAI,CAAC,WAAW,KAAK,QAAQ,IAAI,IAAI,CAAC,WAAW,GAAG,CAAC,CAAC,CAAC,CAAC,IAAI,CAAC,WAAW,CAAC,CAAC,CAAC,IAAI,CAAC;IAClG,CAAC;IAAC,MAAM,CAAC;QACL,OAAO,IAAI,CAAC;IAChB,CAAC;AACL,CAAC,CAAC;AAEF,uEAAuE;AACvE,8EAA8E;AAC9E,iEAAiE;AACjE,MAAM,CAAC,MAAM,uBAAuB,GAAG,KAAK,EAAE,IAAY,EAAE,GAAsB,EAAE,KAAa,EAA4B,EAAE;IAC3H,MAAM,IAAI,GAAG,kBAAkB,CAAC,IAAI,CAAC,CAAC;IACtC,IAAI,IAAI,KAAK,SAAS;QAAE,OAAO,IAAI,CAAC;IAEpC,MAAM,MAAM,GAAG,IAAI,CAAC,cAAc;QAC9B,CAAC,CAAC,UAAU,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,EAAE,IAAI,CAAC,SAAS,EAAE,IAAI,CAAC;QACvD,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,SAAS,CAAC,IAAI,EAAE,CAAC;IAEhC,MAAM,OAAO,GAA2B,EAAE,CAAC;IAC3C,IAAI,MAAM,CAAC,MAAM,GAAG,CAAC;QAAE,OAAO,CAAC,aAAa,GAAG,UAAU,MAAM,EAAE,CAAC;IAElE,MAAM,MAAM,GAAG,oBAAoB,CAAC,GAAG,CAAC,IAAI,CAAC,eAAe,CAAC,EAAE,IAAI,CAAC,gBAAgB,EAAE,IAAI,CAAC,eAAe,EAAE,IAAI,CAAC,CAAC;IAClH,MAAM,GAAG,GAAG,UAAU,CAAC,IAAI,EAAE,GAAG,EAAE,IAAI,CAAC,CAAC;IACxC,MAAM,cAAc,GAAG,gBAAgB,CAAC,GAAG,CAAC,oBAAoB,EAAE,sBAAsB,EAAE,IAAI,CAAC,CAAC;IAEhG,0EAA0E;IAC1E,yEAAyE;IACzE,qEAAqE;IACrE,IAAI,WAAW,GAAG,gBAAgB,CAAC,GAAG,CAAC,4BAA4B,EAAE,8BAA8B,EAAE,IAAI,CAAC,CAAC;IAC3G,IAAI,eAAe,GAAG,KAAK,CAAC;IAC5B,IAAI,SAAS,GAAkB,IAAI,CAAC;IACpC,IAAI,cAAc,GAAG,IAAI,CAAC,cAAc,CAAC;IACzC,IAAI,IAAI,CAAC,SAAS,KAAK,IAAI,EAAE,CAAC;QAC1B,MAAM,KAAK,GAAG,MAAM,WAAW,CAAC,GAAG,EAAE,OAAO,EAAE,KAAK,EAAE,cAAc,CAAC,CAAC;QACrE,eAAe,GAAG,KAAK,CAAC,WAAW,CAAC;QACpC,WAAW,KAAK,KAAK,CAAC,IAAI,CAAC;QAC3B,qEAAqE;QACrE,qEAAqE;QACrE,kEAAkE;QAClE,IAAI,KAAK,CAAC,WAAW,IAAI,cAAc,KAAK,OAAO;YAAE,cAAc,GAAG,UAAU,CAAC;QACjF,IAAI,KAAK,CAAC,WAAW;YAAE,SAAS,GAAG,MAAM,cAAc,CAAC,GAAG,EAAE,OAAO,EAAE,cAAc,CAAC,CAAC;IAC1F,CAAC;IAED,2EAA2E;IAC3E,yEAAyE;IACzE,uEAAuE;IACvE,sEAAsE;IACtE,4EAA4E;IAC5E,gFAAgF;IAChF,MAAM,QAAQ,GAAG,MAAM,CAAC,IAAI,EAAE,KAAK,CAAC,CAAC;IACrC,WAAW,KAAK,QAAQ,EAAE,aAAa,IAAI,IAAI,CAAC;IAChD,MAAM,IAAI,GAAG,QAAQ,EAAE,IAAI,CAAC;IAC5B,MAAM,OAAO,GAAG,IAAI,KAAK,SAAS;QAC9B,CAAC,CAAC,SAAS;QACX,CAAC,CAAC,CAAC,KAAoB,EAAU,EAAE,CAAC,WAAW,CAAC,KAAK,EAAE;YACnD,KAAK,EAAE,IAAI,CAAC,UAAU,GAAG,GAAG;YAC5B,MAAM,EAAE,IAAI,CAAC,WAAW,GAAG,GAAG;YAC9B,MAAM,EAAE,CAAC,IAAI,CAAC,cAAc,IAAI,IAAI,CAAC,UAAU,CAAC,GAAG,GAAG;SACzD,CAAC,CAAC;IAEP,OAAO,IAAI,oBAAoB,CAAC;QAC5B,KAAK;QACL,GAAG;QACH,OAAO;QACP,WAAW;QACX,cAAc;QACd,eAAe,EAAE,sBAAsB,CAAC,GAAG,EAAE,IAAI,CAAC;QAClD,aAAa,EAAE,gBAAgB,CAAC,GAAG,CAAC,8BAA8B,EAAE,gCAAgC,EAAE,IAAI,CAAC;QAC3G,cAAc;QACd,WAAW,EAAE,YAAY,CAAC,MAAM,CAAC;QACjC,OAAO;QACP,MAAM,EAAE,cAAc,CAAC,IAAI,CAAC;QAC5B,eAAe;QACf,mEAAmE;QACnE,mBAAmB,EAAE,eAAe;QACpC,SAAS;KACZ,CAAC,CAAC;AACP,CAAC,CAAC"}
|
package/package.json
CHANGED
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@plurnk/plurnk-providers",
|
|
3
|
-
"version": "0.
|
|
3
|
+
"version": "0.7.0",
|
|
4
4
|
"description": "Framework + contract for the @plurnk/plurnk-providers-* LLM transport family.",
|
|
5
5
|
"keywords": [
|
|
6
6
|
"plurnk",
|
|
@@ -50,6 +50,7 @@
|
|
|
50
50
|
"typescript": "^6.0.3"
|
|
51
51
|
},
|
|
52
52
|
"dependencies": {
|
|
53
|
+
"@plurnk/plurnk-models": "0.1.0",
|
|
53
54
|
"gpt-tokenizer": "^3.4.0",
|
|
54
55
|
"llama-tokenizer-js": "^1.2.2"
|
|
55
56
|
}
|