@bivelio/savings-layer 0.2.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +32 -36
- package/dist/index.d.ts +504 -135
- package/dist/index.js +23 -21
- package/package.json +19 -24
package/dist/index.d.ts
CHANGED
|
@@ -1,17 +1,8 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* Environment-agnostic UUID generation.
|
|
3
|
-
* ---------------------------------------------------------------------------
|
|
4
|
-
* Trace IDs only need to be unique, not cryptographically unpredictable. We
|
|
5
|
-
* prefer the platform `crypto.randomUUID` (present in modern Node, browsers,
|
|
6
|
-
* Deno and edge runtimes) and fall back to a v4-shaped id when it is absent, so
|
|
7
|
-
* BV-SALA runs everywhere without a `node:crypto` import that breaks bundlers.
|
|
8
|
-
*/
|
|
9
|
-
declare function randomId(): string;
|
|
10
|
-
|
|
11
1
|
/**
|
|
12
2
|
* The four-ledger savings accounting.
|
|
13
3
|
*
|
|
14
|
-
* These ledgers are kept strictly separate on purpose
|
|
4
|
+
* These ledgers are kept strictly separate on purpose: each answers a different
|
|
5
|
+
* question, and summing them would double-count.
|
|
15
6
|
* The ledger is a pure accumulator: the pipeline records events into it, and it
|
|
16
7
|
* produces the numbers that populate the SavingsReport. It never guesses — every
|
|
17
8
|
* increment is driven by an explicit pipeline decision.
|
|
@@ -61,26 +52,32 @@ declare class SavingsLedger {
|
|
|
61
52
|
/**
|
|
62
53
|
* ¿El estimador que produjo las dos cifras anteriores cuenta con la regla del
|
|
63
54
|
* proveedor? Lo fija el pipeline al resolver el tokenizador del modelo.
|
|
64
|
-
*
|
|
55
|
+
* Defaults to NO: exactness is claimed only once it has been verified.
|
|
65
56
|
*/
|
|
66
57
|
estimatorExact: boolean;
|
|
67
58
|
/** Salida del reintento de reparación. Es coste NUESTRO, nunca del baseline. */
|
|
68
59
|
repairOutput: number;
|
|
69
60
|
/**
|
|
70
|
-
*
|
|
61
|
+
* INPUT of the repair retry. Mirrors `repairOutput`.
|
|
71
62
|
*
|
|
72
|
-
*
|
|
73
|
-
*
|
|
74
|
-
*
|
|
75
|
-
* cliente: una petición que costó el doble que el camino naive se publicaba con
|
|
76
|
-
* ahorro POSITIVO. Medido: netSavings = -0,2857 cuando la verdad era -100 %, y
|
|
77
|
-
* con `structuredData` llegaba a +36,4 % — y encima con `basis: 'measured'`,
|
|
78
|
-
* que es justo lo que el panel presenta como contrastable con la factura.
|
|
79
|
-
*
|
|
80
|
-
* Para la salida ya existía la corrección (`baselineOutputHonesto`); para la
|
|
81
|
-
* entrada no había nada.
|
|
63
|
+
* Both are OUR cost, never the customer's, so both are subtracted from the
|
|
64
|
+
* reconstructed baseline. The baseline is what the naive path would have
|
|
65
|
+
* cost; work the layer does to fix its own output has no place in it.
|
|
82
66
|
*/
|
|
83
67
|
repairInput: number;
|
|
68
|
+
/**
|
|
69
|
+
* INPUT of the schema-on-demand discovery probe. Held apart exactly like
|
|
70
|
+
* {@link repairInput}, and for the same reason.
|
|
71
|
+
*
|
|
72
|
+
* The layer ADDS this call; the naive path would never make it. Letting its
|
|
73
|
+
* tokens into the reconstructed baseline would make the probe pay for itself,
|
|
74
|
+
* and a request that cost MORE than going without the layer would publish a
|
|
75
|
+
* positive saving. It still counts in full towards `actualInput` — that is
|
|
76
|
+
* money genuinely spent — it just cannot inflate what that is compared against.
|
|
77
|
+
*/
|
|
78
|
+
discoveryInput: number;
|
|
79
|
+
/** OUTPUT of the discovery probe. Our cost, kept out of the baseline. */
|
|
80
|
+
discoveryOutput: number;
|
|
84
81
|
/**
|
|
85
82
|
* Lo que de verdad hizo la compresión, para publicarlo en
|
|
86
83
|
* `SavingsReport.compression`. El campo estaba declarado en el contrato y no lo
|
|
@@ -119,6 +116,53 @@ declare class SavingsLedger {
|
|
|
119
116
|
};
|
|
120
117
|
}
|
|
121
118
|
|
|
119
|
+
/**
|
|
120
|
+
* LIST PRICES PER MODEL.
|
|
121
|
+
*
|
|
122
|
+
* The layer reports savings in money, and money needs a price. This is the
|
|
123
|
+
* built-in catalogue of published list prices, so the dollar figures mean
|
|
124
|
+
* something without any configuration.
|
|
125
|
+
*
|
|
126
|
+
* ⚠️ A WRONG PRICE HERE IS WORSE THAN NO CATALOGUE. A model that is ABSENT falls
|
|
127
|
+
* back to the illustrative default AND warns, so the customer knows that column
|
|
128
|
+
* is not their invoice. A model that is PRESENT with the wrong price silences
|
|
129
|
+
* that warning and hands over a confident, wrong number. So only models whose
|
|
130
|
+
* published price is known with certainty belong here, and {@link PRECIOS_AL_DIA}
|
|
131
|
+
* records when they were last checked.
|
|
132
|
+
*
|
|
133
|
+
* Prices are USD per 1,000 tokens — the unit of `CostModel` — even though
|
|
134
|
+
* providers publish per million: 1M to 1K is a division by a thousand.
|
|
135
|
+
*
|
|
136
|
+
* `costModels` on the layer config ALWAYS wins over this. A customer with volume
|
|
137
|
+
* discounts, negotiated rates or a different region sets their own and this
|
|
138
|
+
* catalogue stays out of the way.
|
|
139
|
+
*/
|
|
140
|
+
/** When these prices were last checked against the providers' public rates. */
|
|
141
|
+
declare const PRECIOS_AL_DIA = "2026-08-13";
|
|
142
|
+
/**
|
|
143
|
+
* Key = the model id as the provider publishes it, without a vendor prefix.
|
|
144
|
+
* Lookup tries the exact id, then the id without its provider prefix, then the
|
|
145
|
+
* LONGEST matching prefix — so `gpt-4o-2024-08-06` inherits `gpt-4o` without
|
|
146
|
+
* having to enumerate every dated snapshot.
|
|
147
|
+
*/
|
|
148
|
+
declare const CATALOGO: Readonly<Record<string, CostModel>>;
|
|
149
|
+
/**
|
|
150
|
+
* The list price for a model, or `undefined` when it is not in the catalogue.
|
|
151
|
+
*
|
|
152
|
+
* @param model the id as the caller passes it, with or without a vendor prefix.
|
|
153
|
+
*/
|
|
154
|
+
declare function precioDeLista(model: string): CostModel | undefined;
|
|
155
|
+
|
|
156
|
+
/**
|
|
157
|
+
* Environment-agnostic UUID generation.
|
|
158
|
+
* ---------------------------------------------------------------------------
|
|
159
|
+
* Trace IDs only need to be unique, not cryptographically unpredictable. We
|
|
160
|
+
* prefer the platform `crypto.randomUUID` (present in modern Node, browsers,
|
|
161
|
+
* Deno and edge runtimes) and fall back to a v4-shaped id when it is absent, so
|
|
162
|
+
* BV-SALA runs everywhere without a `node:crypto` import that breaks bundlers.
|
|
163
|
+
*/
|
|
164
|
+
declare function randomId(): string;
|
|
165
|
+
|
|
122
166
|
/**
|
|
123
167
|
* Core contracts for BV-SALA (bivelio-savings-layer).
|
|
124
168
|
*
|
|
@@ -132,8 +176,8 @@ declare class SavingsLedger {
|
|
|
132
176
|
* - provider_cached_tokens tokens still in the prompt that the provider may
|
|
133
177
|
* bill or process differently (prefix/KV caching).
|
|
134
178
|
*
|
|
135
|
-
* A cache hit eliminates a whole call.
|
|
136
|
-
*
|
|
179
|
+
* A cache hit eliminates a whole call. A compact encoding re-encodes a structured payload.
|
|
180
|
+
* A context compressor removes redundant words. A provider prefix cache makes tokens
|
|
137
181
|
* cheaper without removing them. Mixing these produces misleading metrics.
|
|
138
182
|
*/
|
|
139
183
|
/** Roles understood by the provider-neutral chat interface. */
|
|
@@ -193,7 +237,7 @@ interface OutputContract {
|
|
|
193
237
|
version?: string;
|
|
194
238
|
}
|
|
195
239
|
/** Serialization formats the format-selector can choose between. */
|
|
196
|
-
type SerializationFormat = "json-compact" | "
|
|
240
|
+
type SerializationFormat = "json-compact" | "compact-rows" | "csv";
|
|
197
241
|
interface RetrievalOptions {
|
|
198
242
|
mode: "off" | "auto" | "single" | "multi";
|
|
199
243
|
maxRounds?: number;
|
|
@@ -207,6 +251,61 @@ interface SavingsToggles {
|
|
|
207
251
|
singleflight?: boolean;
|
|
208
252
|
deterministicResolvers?: boolean;
|
|
209
253
|
router?: boolean;
|
|
254
|
+
/**
|
|
255
|
+
* OPT-IN. Send the provider a compacted form of your output contract — short
|
|
256
|
+
* property keys, short enum codes — so the model spends fewer tokens saying
|
|
257
|
+
* the same thing, and expand the reply back to YOUR shape before it is
|
|
258
|
+
* validated, cached or returned. What you receive in `data`, `text` and
|
|
259
|
+
* `rendered` is identical either way.
|
|
260
|
+
*
|
|
261
|
+
* It is applied only when the layer can prove the expansion is exact for that
|
|
262
|
+
* schema and only when it counts as a net token win on both sides (the
|
|
263
|
+
* compact contract itself travels as input). Otherwise your schema is sent
|
|
264
|
+
* unchanged. Off by default because it changes what the provider sees, and
|
|
265
|
+
* whether a model answers as WELL from a compact contract is a question only
|
|
266
|
+
* an A/B on your own traffic can settle. Check `report.traces` for what it
|
|
267
|
+
* decided and why.
|
|
268
|
+
*/
|
|
269
|
+
outputCodebook?: boolean;
|
|
270
|
+
/**
|
|
271
|
+
* OPT-IN, and it changes the latency contract: where it applies, one request
|
|
272
|
+
* becomes TWO provider calls, one after the other.
|
|
273
|
+
*
|
|
274
|
+
* With a large tool catalogue the full JSON schema of every candidate travels
|
|
275
|
+
* on every request, even though the model will use one tool or none. With this
|
|
276
|
+
* on, the layer may first send lightweight descriptors (name + one line, no
|
|
277
|
+
* `parameters`), read which tools the model asks for, and only then send those
|
|
278
|
+
* full schemas.
|
|
279
|
+
*
|
|
280
|
+
* It is applied only when the token arithmetic for THIS request, priced with
|
|
281
|
+
* your cost model, says the second call pays for itself with margin — so a
|
|
282
|
+
* small catalogue silently keeps the single-call path.
|
|
283
|
+
*
|
|
284
|
+
* THE MARGIN IS NOT A GUARANTEE THAT EVERY REQUEST GETS CHEAPER. When the
|
|
285
|
+
* model names no tool in the first call, the request still goes out with every
|
|
286
|
+
* schema and the first call is pure added cost: that request costs MORE, and
|
|
287
|
+
* the report says so (`cost.netSavings` goes negative). The margin required is
|
|
288
|
+
* exactly the cost of that outcome, so the pattern breaks even while it
|
|
289
|
+
* happens on at most one request in two. If your traffic asks for a tool less
|
|
290
|
+
* often than that — a chat assistant with a big catalogue answers most turns
|
|
291
|
+
* directly — leave this off. Only an A/B against your own traffic settles it.
|
|
292
|
+
*
|
|
293
|
+
* IT ALSO GIVES UP THE PROVIDER'S PREFIX CACHE, on every request, not just the
|
|
294
|
+
* ones where it applies. Tools render FIRST in a provider request, and with
|
|
295
|
+
* this on, which schemas travel is decided by a probe whose answer is not
|
|
296
|
+
* deterministic — so the front of the prompt varies between requests that
|
|
297
|
+
* would otherwise share a cacheable prefix. Marking one would pay the write
|
|
298
|
+
* premium (1.25x on Anthropic) for a prefix that cannot hit. If your prompts
|
|
299
|
+
* carry a long fixed system header and your traffic repeats it, that discount
|
|
300
|
+
* is very likely worth more than this lever: measure both before choosing.
|
|
301
|
+
* `report.traces` says which one is off and why.
|
|
302
|
+
*
|
|
303
|
+
* When the second call happened, `report.calls.actualLlm` is 2. The extra
|
|
304
|
+
* call's tokens are counted as spend and never credited to your baseline.
|
|
305
|
+
* Write and destructive tools always travel with their full schema, so no
|
|
306
|
+
* action can be lost to a discovery step.
|
|
307
|
+
*/
|
|
308
|
+
toolDiscovery?: boolean;
|
|
210
309
|
}
|
|
211
310
|
interface RequestContext {
|
|
212
311
|
tenantId?: string;
|
|
@@ -258,12 +357,9 @@ interface SavingsResult<T = unknown> {
|
|
|
258
357
|
* `data` NO son la respuesta final: el modelo devolvió `content` vacío y pide
|
|
259
358
|
* ejecutar la acción (`finishReason === "tool_calls"`).
|
|
260
359
|
*
|
|
261
|
-
*
|
|
262
|
-
*
|
|
263
|
-
*
|
|
264
|
-
* campo — el cliente recibía "" como si fuera la respuesta buena y la
|
|
265
|
-
* invocación se perdía entera. La palanca de herramientas que vende el README
|
|
266
|
-
* era inservible en el paquete publicado.
|
|
360
|
+
* `content: null` + `tool_calls` is the NORMAL shape as soon as tools are in
|
|
361
|
+
* play, so treat an empty `text` together with a populated `toolCalls` as a
|
|
362
|
+
* tool invocation to execute — not as an empty answer.
|
|
267
363
|
*/
|
|
268
364
|
toolCalls?: ToolCallRequest[];
|
|
269
365
|
/**
|
|
@@ -275,6 +371,42 @@ interface SavingsResult<T = unknown> {
|
|
|
275
371
|
report: SavingsReport;
|
|
276
372
|
}
|
|
277
373
|
type CacheOutcome = "miss" | "exact" | "semantic" | "singleflight" | "deterministic";
|
|
374
|
+
/**
|
|
375
|
+
* The quality claims this layer is able to DEMONSTRATE for the answer it served.
|
|
376
|
+
*
|
|
377
|
+
* A boolean cannot carry this. `constraints.structurallyValid` puts an answer
|
|
378
|
+
* computed by a deterministic resolver — exact by construction — in the same box
|
|
379
|
+
* as one the model wrote while nobody checked anything but its shape. Those two
|
|
380
|
+
* are the difference between "this is the result" and "this is what the model
|
|
381
|
+
* said", and a product that claims savings owes the reader that distinction.
|
|
382
|
+
*
|
|
383
|
+
* THE RULE BEHIND THIS VOCABULARY: never report "no loss" on the strength of
|
|
384
|
+
* cosine similarity, BLEU, ROUGE or another model's judgement. Those scores stay
|
|
385
|
+
* high while a figure, a negation or an exception quietly disappears — precisely
|
|
386
|
+
* the fields that decide a legal or financial document. A claim of equivalence
|
|
387
|
+
* rests on a deterministic gate, or it is not made.
|
|
388
|
+
*
|
|
389
|
+
* - `exact` exact by construction: a deterministic resolver computed
|
|
390
|
+
* the answer, or an exact-cache hit replayed the answer
|
|
391
|
+
* already produced for this very same request.
|
|
392
|
+
* - `task_equivalent` not byte-identical, but it cleared a deterministic gate
|
|
393
|
+
* before being served — numbers and dates as a multiset,
|
|
394
|
+
* negation parity and a key-entity floor all had to agree,
|
|
395
|
+
* on top of any similarity score.
|
|
396
|
+
* - `not_verified` the model was called and nothing beyond the shape of the
|
|
397
|
+
* answer was checked. The honest default, and the common case.
|
|
398
|
+
* - `rejected` what was served does not satisfy its output contract, and
|
|
399
|
+
* a repair either failed or would have been a wasted call.
|
|
400
|
+
* The answer is still returned: this is the layer's verdict
|
|
401
|
+
* on it, not a refusal to hand it over.
|
|
402
|
+
*
|
|
403
|
+
* The list is deliberately short. A status nobody produces is a promise the
|
|
404
|
+
* reader cannot check, so levers that do not exist yet bring their own status
|
|
405
|
+
* the day they can demonstrate it. Read this as an OPEN set — keep a default
|
|
406
|
+
* branch, exactly as with a licence status.
|
|
407
|
+
*/
|
|
408
|
+
declare const QUALITY_STATUSES: readonly ["exact", "task_equivalent", "not_verified", "rejected"];
|
|
409
|
+
type QualityStatus = (typeof QUALITY_STATUSES)[number];
|
|
278
410
|
interface SavingsReport {
|
|
279
411
|
requestId: string;
|
|
280
412
|
model: string;
|
|
@@ -337,8 +469,27 @@ interface SavingsReport {
|
|
|
337
469
|
*/
|
|
338
470
|
structurallyValid?: boolean;
|
|
339
471
|
};
|
|
472
|
+
/**
|
|
473
|
+
* How the served answer earned its place, in one word. Optional and additive:
|
|
474
|
+
* a report built by an older SDK simply does not carry it, so treat an absent
|
|
475
|
+
* value as "unknown", never as a pass.
|
|
476
|
+
*
|
|
477
|
+
* It only ever DEGRADES what the rest of the report claims. An answer that
|
|
478
|
+
* fails its output contract is `rejected` however it was produced — a resolver
|
|
479
|
+
* that computes the right number for the wrong contract does not get to keep
|
|
480
|
+
* the `exact` label.
|
|
481
|
+
*/
|
|
482
|
+
qualityStatus?: QualityStatus;
|
|
340
483
|
quality: {
|
|
341
484
|
taskPassed?: boolean;
|
|
485
|
+
/**
|
|
486
|
+
* Similarity score of the retained answer, when one was computed.
|
|
487
|
+
*
|
|
488
|
+
* A number here is NOT evidence of equivalence: see {@link QualityStatus}.
|
|
489
|
+
* Cosine (and BLEU, ROUGE, or an LLM judge) can sit above 0.95 across a
|
|
490
|
+
* changed figure or a dropped negation, so it informs a decision — it never
|
|
491
|
+
* licenses a "no loss" claim.
|
|
492
|
+
*/
|
|
342
493
|
semanticScore?: number;
|
|
343
494
|
freshnessPassed?: boolean;
|
|
344
495
|
};
|
|
@@ -419,7 +570,7 @@ interface TraceEvent {
|
|
|
419
570
|
* be re-explained in free-form natural language at every hop. BV-SALA replaces
|
|
420
571
|
* that with a compact, typed handoff object. Each consumer receives a
|
|
421
572
|
* projection of the state, not the whole conversation. The serializer then
|
|
422
|
-
* decides whether to send it as compact JSON or
|
|
573
|
+
* decides whether to send it as compact JSON or compact rows.
|
|
423
574
|
*/
|
|
424
575
|
interface HandoffFact {
|
|
425
576
|
id: string;
|
|
@@ -469,7 +620,7 @@ interface TokenEstimator {
|
|
|
469
620
|
* El heurístico vale para RANKEAR formatos candidatos (le basta con acertar el
|
|
470
621
|
* orden). No vale para restar tokens entre formatos distintos: cobra cada
|
|
471
622
|
* símbolo suelto como un token entero, así que infla el JSON denso en símbolos
|
|
472
|
-
* mucho más que el
|
|
623
|
+
* mucho más que el the compact encoder, y la diferencia se acredita como ahorro inexistente.
|
|
473
624
|
*/
|
|
474
625
|
readonly exact: boolean;
|
|
475
626
|
/** Estimate the number of tokens in a string. */
|
|
@@ -526,22 +677,13 @@ declare const defaultTokenizerRegistry: TokenizerRegistry;
|
|
|
526
677
|
* contamos aquí. Con el estimador heurístico esos dos sumandos vienen de reglas
|
|
527
678
|
* distintas, y la diferencia NO es ruido: el heurístico cobra cada símbolo suelto
|
|
528
679
|
* como un token entero, así que infla el JSON (40% de símbolos) mucho más que el
|
|
529
|
-
*
|
|
530
|
-
*
|
|
531
|
-
*
|
|
532
|
-
* Cómo se instala. `gpt-tokenizer` son ~53 MB desempaquetados, así que NO es
|
|
533
|
-
* dependencia dura: imponérsela a todo el que instale el SDK, cuando la mayoría
|
|
534
|
-
* no necesita contabilidad medida, sería abusivo. Es peerDependency opcional:
|
|
535
|
-
*
|
|
536
|
-
* pnpm add gpt-tokenizer
|
|
537
|
-
*
|
|
538
|
-
* Sin ella el layer sigue funcionando con el heurístico, y el `basis` de cada
|
|
539
|
-
* informe lo dice — "estimated" en vez de "measured". Esa es la diferencia
|
|
540
|
-
* importante: el producto no deja de funcionar, deja de AFIRMAR de más.
|
|
680
|
+
* An estimator that is not calibrated for the destination model can miss by
|
|
681
|
+
* enough to publish a baseline several times the real one, which would credit a
|
|
682
|
+
* saving that never existed. That is why `basis` degrades instead of claiming.
|
|
541
683
|
*/
|
|
542
684
|
|
|
543
685
|
/** Lo que necesitamos de gpt-tokenizer. Tipado aquí para no depender de sus tipos. */
|
|
544
|
-
interface
|
|
686
|
+
interface BpeEncoder {
|
|
545
687
|
encode(text: string): number[] | {
|
|
546
688
|
length: number;
|
|
547
689
|
};
|
|
@@ -549,19 +691,19 @@ interface CodificadorBpe {
|
|
|
549
691
|
/**
|
|
550
692
|
* Estimador exacto sobre una codificación BPE ya cargada.
|
|
551
693
|
*
|
|
552
|
-
* Se construye con el
|
|
694
|
+
* Se construye con el encoder ya resuelto en vez de importarlo dentro: así
|
|
553
695
|
* este módulo no obliga a nadie a tener el paquete, y quien lo tiene decide qué
|
|
554
696
|
* codificación carga (o200k_base para gpt-4o/4.1/5, cl100k_base para 3.5/4).
|
|
555
697
|
*/
|
|
556
698
|
declare class ExactTokenizer implements TokenEstimator {
|
|
557
|
-
private readonly
|
|
699
|
+
private readonly encoder;
|
|
558
700
|
readonly id: string;
|
|
559
701
|
readonly exact = true;
|
|
560
|
-
constructor(id: string,
|
|
702
|
+
constructor(id: string, encoder: BpeEncoder);
|
|
561
703
|
count(text: string): number;
|
|
562
704
|
}
|
|
563
705
|
/** Modelos que usan cada codificación. Sirve para mapear el registro de una vez. */
|
|
564
|
-
declare const
|
|
706
|
+
declare const MODELS_BY_ENCODING: Readonly<Record<string, readonly string[]>>;
|
|
565
707
|
/**
|
|
566
708
|
* Carga `gpt-tokenizer` si está instalado y registra el tokenizador exacto para
|
|
567
709
|
* los modelos de OpenAI. Devuelve `false` —sin lanzar— cuando no está: la
|
|
@@ -572,12 +714,20 @@ declare const MODELOS_POR_CODIFICACION: Readonly<Record<string, readonly string[
|
|
|
572
714
|
* expone un punto de entrada por codificación y cargarlas todas serían decenas
|
|
573
715
|
* de MB en memoria sin motivo.
|
|
574
716
|
*/
|
|
575
|
-
declare function
|
|
717
|
+
declare function registerExactTokenizer(registro: {
|
|
576
718
|
register(e: TokenEstimator): unknown;
|
|
577
719
|
mapModel(model: string, tokenizerId: string): unknown;
|
|
578
720
|
}, opciones?: {
|
|
579
721
|
codificaciones?: readonly string[];
|
|
580
722
|
}): Promise<boolean>;
|
|
723
|
+
/**
|
|
724
|
+
* @deprecated Spanish-named aliases kept for one minor version. Use
|
|
725
|
+
* {@link MODELS_BY_ENCODING} and {@link registerExactTokenizer}. The public
|
|
726
|
+
* surface of this package is English-only; these leaked and are on the way out.
|
|
727
|
+
*/
|
|
728
|
+
declare const MODELOS_POR_CODIFICACION: Readonly<Record<string, readonly string[]>>;
|
|
729
|
+
/** @deprecated Use {@link registerExactTokenizer}. */
|
|
730
|
+
declare const registrarTokenizadorExacto: typeof registerExactTokenizer;
|
|
581
731
|
|
|
582
732
|
/**
|
|
583
733
|
* Canonicalization turns a raw request into a normalized, hashable form. This
|
|
@@ -589,7 +739,11 @@ interface CanonicalRequest {
|
|
|
589
739
|
model: string;
|
|
590
740
|
modelFamily: string;
|
|
591
741
|
messages: ChatMessage[];
|
|
592
|
-
/**
|
|
742
|
+
/**
|
|
743
|
+
* Text of the user-visible turns, normalized so that trivial near-duplicates
|
|
744
|
+
* collide: Unicode form, case, whitespace, typographic marks and sentence
|
|
745
|
+
* punctuation. Numbers, dates, quantities and negations are preserved exactly.
|
|
746
|
+
*/
|
|
593
747
|
normalizedText: string;
|
|
594
748
|
systemPromptHash: string;
|
|
595
749
|
toolsetHash: string;
|
|
@@ -617,6 +771,14 @@ interface CanonicalRequest {
|
|
|
617
771
|
tokenizer: string | undefined;
|
|
618
772
|
original: SavingsRequest;
|
|
619
773
|
}
|
|
774
|
+
/**
|
|
775
|
+
* Normalized form of a turn's text, as it enters the cache key.
|
|
776
|
+
*
|
|
777
|
+
* Folded away: Unicode form, case, whitespace, typographic marks, Latin
|
|
778
|
+
* diacritics and sentence punctuation. Preserved exactly: numbers, decimal and
|
|
779
|
+
* thousands separators, times, dates, units, negations, math operators and any
|
|
780
|
+
* combining mark outside the Latin alphabet.
|
|
781
|
+
*/
|
|
620
782
|
declare function normalizeText(text: string): string;
|
|
621
783
|
declare function sha256(input: string): string;
|
|
622
784
|
/** Stable JSON stringify with sorted keys so hashes are order-independent. */
|
|
@@ -638,14 +800,10 @@ declare function buildExactKey(c: CanonicalRequest): string;
|
|
|
638
800
|
* need freshness bucketing, since the work is happening *now*), but must still
|
|
639
801
|
* isolate tenants and permissions.
|
|
640
802
|
*
|
|
641
|
-
* `schema`
|
|
642
|
-
*
|
|
643
|
-
*
|
|
644
|
-
*
|
|
645
|
-
* `structurallyValid: true`, porque `asCoalescedFollower` clona el informe del
|
|
646
|
-
* líder sin recalcular `constraints` ni pasar por la validación. `buildExactKey`
|
|
647
|
-
* ya lo incluía y la caché semántica lo filtra a mano; esta era la única de las
|
|
648
|
-
* tres puertas que no lo miraba.
|
|
803
|
+
* `schema` is REQUIRED here, even though `contract` is already part of the key:
|
|
804
|
+
* two concurrent calls that differ only in `schemaVersion` are different
|
|
805
|
+
* requests and must never coalesce. All three doors — exact key, in-flight key
|
|
806
|
+
* and semantic filter — key on the same identity, by design.
|
|
649
807
|
*/
|
|
650
808
|
declare function buildInflightKey(c: CanonicalRequest): string;
|
|
651
809
|
|
|
@@ -780,23 +938,20 @@ interface SemanticRecord {
|
|
|
780
938
|
knowledgeEpoch: string;
|
|
781
939
|
contractId: string | null;
|
|
782
940
|
/**
|
|
783
|
-
*
|
|
941
|
+
* Who made the request (session/principal), or null when anonymous.
|
|
784
942
|
*
|
|
785
|
-
*
|
|
786
|
-
*
|
|
787
|
-
*
|
|
788
|
-
* calculada para el primero. `tenantId` separa organizaciones, no personas.
|
|
943
|
+
* Isolation boundary: `tenantId` separates organizations, `principal`
|
|
944
|
+
* separates people within one. Both are hard filters applied BEFORE any
|
|
945
|
+
* similarity scoring — a semantic match never crosses either.
|
|
789
946
|
*/
|
|
790
947
|
principal: string | null;
|
|
791
948
|
/**
|
|
792
|
-
*
|
|
949
|
+
* The COMPLETE identity of the request that produced this answer.
|
|
793
950
|
*
|
|
794
|
-
*
|
|
795
|
-
*
|
|
796
|
-
*
|
|
797
|
-
*
|
|
798
|
-
* calculada sobre la tabla de otro. `model` y `systemPromptHash` van detrás:
|
|
799
|
-
* un modelo o un system prompt distintos son otra respuesta.
|
|
951
|
+
* Every field here is a hard filter, checked before similarity is scored, and
|
|
952
|
+
* the set is identical to the one the exact cache keys on. Two questions
|
|
953
|
+
* worded alike but carrying different attached data, a different model or a
|
|
954
|
+
* different system prompt are different questions, and never match.
|
|
800
955
|
*/
|
|
801
956
|
model: string;
|
|
802
957
|
systemPromptHash: string;
|
|
@@ -829,7 +984,7 @@ declare function cosine(a: number[], b: number[]): number;
|
|
|
829
984
|
declare function keyTokensMatch(a: string, b: string): boolean;
|
|
830
985
|
/**
|
|
831
986
|
* In-memory reference implementation. A production build would back this with a
|
|
832
|
-
* real vector store
|
|
987
|
+
* real vector store and metadata filters pushed down
|
|
833
988
|
* to the store. The verification logic — the part that makes it *safe* — stays
|
|
834
989
|
* identical regardless of the backend.
|
|
835
990
|
*/
|
|
@@ -944,6 +1099,17 @@ interface ToolSelection {
|
|
|
944
1099
|
tools: ToolDescriptor[];
|
|
945
1100
|
droppedIds: string[];
|
|
946
1101
|
reason: string;
|
|
1102
|
+
/**
|
|
1103
|
+
* Whether the SET of tools is determined by something stable across requests,
|
|
1104
|
+
* rather than by this request's text.
|
|
1105
|
+
*
|
|
1106
|
+
* It is not cosmetic. Tools render FIRST in a provider request (tools ->
|
|
1107
|
+
* system -> messages), so a tool set that changes with the question moves the
|
|
1108
|
+
* very front of the prompt and invalidates the whole prefix cache behind it.
|
|
1109
|
+
* A caller whose prompt caching silently never hits, on traffic with tools, is
|
|
1110
|
+
* usually looking at this.
|
|
1111
|
+
*/
|
|
1112
|
+
setIsStable: boolean;
|
|
947
1113
|
}
|
|
948
1114
|
interface ToolSelectorOptions {
|
|
949
1115
|
/** Max tools to include (schema-token budget guard). */
|
|
@@ -1080,7 +1246,7 @@ interface Serializer {
|
|
|
1080
1246
|
encode(value: unknown): string;
|
|
1081
1247
|
/**
|
|
1082
1248
|
* The instruction text a model needs to interpret this format. For JSON this
|
|
1083
|
-
* is empty; for
|
|
1249
|
+
* is empty; for row/CSV it is the per-request cost that must be amortized
|
|
1084
1250
|
* before the format is worthwhile.
|
|
1085
1251
|
*/
|
|
1086
1252
|
instruction(value: unknown): string;
|
|
@@ -1098,60 +1264,60 @@ interface Serializer {
|
|
|
1098
1264
|
* Compact JSON: the lossless baseline every other format is measured against.
|
|
1099
1265
|
* No indentation, no incidental whitespace. This is frequently the *winner* for
|
|
1100
1266
|
* deeply nested or non-uniform data, which is exactly why it is the baseline and
|
|
1101
|
-
*
|
|
1267
|
+
* The compact encoder is only ever a challenger.
|
|
1102
1268
|
*/
|
|
1103
1269
|
declare const jsonCompactSerializer: Serializer;
|
|
1104
1270
|
|
|
1105
1271
|
/**
|
|
1106
|
-
*
|
|
1272
|
+
* Compact row encoder.
|
|
1107
1273
|
*
|
|
1108
|
-
*
|
|
1274
|
+
* Its strength is UNIFORM ARRAYS OF FLAT OBJECTS: it declares the column
|
|
1109
1275
|
* names once and then emits one row per object, which removes the repeated keys
|
|
1110
1276
|
* that make JSON expensive for tabular data. On deeply nested or non-uniform
|
|
1111
|
-
* data
|
|
1277
|
+
* data it can be *larger* than compact JSON — so this is only ever a candidate
|
|
1112
1278
|
* the selector may pick, never a default.
|
|
1113
1279
|
*
|
|
1114
1280
|
* This is an in-house, pinned encoder placed behind the {@link Serializer}
|
|
1115
1281
|
* interface (as recommended: do not couple the system to an upstream `main`).
|
|
1116
|
-
* It emits a well-defined subset and ships a matching decoder so the JSON<->
|
|
1282
|
+
* It emits a well-defined subset and ships a matching decoder so the JSON<->rows
|
|
1117
1283
|
* round-trip is deterministic and testable.
|
|
1118
1284
|
*/
|
|
1119
|
-
interface
|
|
1285
|
+
interface CompactRowsOptions {
|
|
1120
1286
|
/** Field delimiter within tabular/inline rows. */
|
|
1121
1287
|
delimiter?: "," | "\t";
|
|
1122
1288
|
/** Indentation unit. */
|
|
1123
1289
|
indent?: string;
|
|
1124
1290
|
}
|
|
1125
|
-
declare function
|
|
1291
|
+
declare function encodeCompactRows(value: unknown, options?: CompactRowsOptions): string;
|
|
1126
1292
|
/** Return shared column names iff every element is a flat, scalar-valued object with identical keys. */
|
|
1127
1293
|
declare function uniformColumns(arr: unknown[]): string[] | null;
|
|
1128
1294
|
/**
|
|
1129
|
-
* The instruction a model needs to read
|
|
1130
|
-
* is the per-request overhead the selector must amortize before
|
|
1295
|
+
* The instruction a model needs to read the compact encoder. Kept short on purpose — this text
|
|
1296
|
+
* is the per-request overhead the selector must amortize before it wins.
|
|
1131
1297
|
*/
|
|
1132
|
-
declare const
|
|
1133
|
-
declare function
|
|
1134
|
-
declare const
|
|
1298
|
+
declare const COMPACT_ROWS_INSTRUCTION: string;
|
|
1299
|
+
declare function createCompactRowsSerializer(options?: CompactRowsOptions): Serializer;
|
|
1300
|
+
declare const compactRowsSerializer: Serializer;
|
|
1135
1301
|
|
|
1136
1302
|
/**
|
|
1137
|
-
*
|
|
1303
|
+
* Decoder for the subset emitted by {@link encodeCompactRows}.
|
|
1138
1304
|
*
|
|
1139
1305
|
* Supports: scalar fields, nested objects, empty arrays, inline scalar arrays
|
|
1140
1306
|
* (`key[N]: a,b,c`) and tabular object arrays (`key[N]{c1,c2}:` + rows). This
|
|
1141
|
-
* covers the cases the format selector actually chooses
|
|
1142
|
-
* JSON<->
|
|
1307
|
+
* covers the cases the format selector actually chooses it for, and makes the
|
|
1308
|
+
* JSON<->rows round-trip deterministic and testable. List-form arrays of nested
|
|
1143
1309
|
* objects are intentionally out of scope for decoding (the selector never picks
|
|
1144
|
-
*
|
|
1310
|
+
* it for those, since JSON wins there anyway).
|
|
1145
1311
|
*/
|
|
1146
|
-
interface
|
|
1312
|
+
interface CompactRowsDecodeOptions {
|
|
1147
1313
|
delimiter?: "," | "\t";
|
|
1148
1314
|
indent?: string;
|
|
1149
1315
|
}
|
|
1150
|
-
declare function
|
|
1316
|
+
declare function decodeCompactRows(text: string, options?: CompactRowsDecodeOptions): unknown;
|
|
1151
1317
|
|
|
1152
1318
|
/**
|
|
1153
1319
|
* CSV serializer — only valid for genuinely tabular data (a uniform array of
|
|
1154
|
-
* flat objects). CSV can be marginally smaller than
|
|
1320
|
+
* flat objects). CSV can be marginally smaller than the compact rows on pure tables, but it
|
|
1155
1321
|
* does NOT preserve the JSON data model in general, so `supports()` is strict.
|
|
1156
1322
|
*/
|
|
1157
1323
|
declare const csvSerializer: Serializer;
|
|
@@ -1159,7 +1325,7 @@ declare const csvSerializer: Serializer;
|
|
|
1159
1325
|
/**
|
|
1160
1326
|
* Format selector.
|
|
1161
1327
|
*
|
|
1162
|
-
*
|
|
1328
|
+
* It is a candidate, never a default. For each request we count, USING THE
|
|
1163
1329
|
* DESTINATION MODEL'S TOKENIZER:
|
|
1164
1330
|
*
|
|
1165
1331
|
* tokens(encoded payload) + tokens(instruction to understand it)
|
|
@@ -1178,7 +1344,7 @@ interface FormatSelectorOptions {
|
|
|
1178
1344
|
minimumSavingsRatio?: number;
|
|
1179
1345
|
/**
|
|
1180
1346
|
* Per-(model,format) quality penalty in tokens. Encodes "this model is worse
|
|
1181
|
-
* at reading
|
|
1347
|
+
* at reading the compact rows" as an additive cost, learned from production traces.
|
|
1182
1348
|
*/
|
|
1183
1349
|
qualityPenalty?: (model: string, format: SerializationFormat) => number;
|
|
1184
1350
|
}
|
|
@@ -1282,7 +1448,7 @@ declare function resolveConstraintBackend(model: string, schema: JsonSchema, bac
|
|
|
1282
1448
|
declare function providerNativeBackend(isSupported?: (model: string, schema: JsonSchema) => boolean): ConstraintBackend;
|
|
1283
1449
|
|
|
1284
1450
|
/**
|
|
1285
|
-
* Context compression
|
|
1451
|
+
* Context compression, as an OPTIONAL sidecar.
|
|
1286
1452
|
*
|
|
1287
1453
|
* Two rules make this safe and worthwhile:
|
|
1288
1454
|
*
|
|
@@ -1351,9 +1517,60 @@ declare function shouldCompress(input: CompressionPolicyInput): {
|
|
|
1351
1517
|
* worse than no compressor at all.
|
|
1352
1518
|
*/
|
|
1353
1519
|
declare function withProtectedSpans(inner: Compressor): Compressor;
|
|
1354
|
-
/** No-op compressor: the safe default until a real
|
|
1520
|
+
/** No-op compressor: the safe default until a real compressor sidecar is wired in. */
|
|
1355
1521
|
declare const noopCompressor: Compressor;
|
|
1356
1522
|
|
|
1523
|
+
/** What the layer needs to know about a provider's prefix cache to price a decision. */
|
|
1524
|
+
interface CacheTier {
|
|
1525
|
+
/**
|
|
1526
|
+
* Minimum prompt tokens before the provider caches the prefix at all;
|
|
1527
|
+
* `null` means it never does.
|
|
1528
|
+
*
|
|
1529
|
+
* The built-in tiers assume 1024 for every model. That is an ASSUMPTION, not a
|
|
1530
|
+
* verified figure: real minimums differ per model and family. Set your model's
|
|
1531
|
+
* own minimum here when you know it.
|
|
1532
|
+
*/
|
|
1533
|
+
minimumCacheTokens: number | null;
|
|
1534
|
+
/** Price of a cache-read token as a fraction of a normal one (e.g. 0.5, 0.1). */
|
|
1535
|
+
cacheReadMultiplier: number;
|
|
1536
|
+
/** Surcharge for WRITING the cache the first time; some providers bill above 1. */
|
|
1537
|
+
cacheWriteMultiplier: number;
|
|
1538
|
+
}
|
|
1539
|
+
|
|
1540
|
+
/** Tunes when compression is worth its money. Every field is optional. */
|
|
1541
|
+
interface CompressionEconomics {
|
|
1542
|
+
/**
|
|
1543
|
+
* The provider's prefix-cache tier. Derived from the request's model when
|
|
1544
|
+
* omitted; set it to state your model's real cache minimum and multipliers.
|
|
1545
|
+
*/
|
|
1546
|
+
cacheTier?: CacheTier;
|
|
1547
|
+
/**
|
|
1548
|
+
* YOUR MEASURED prefix-cache economics: how many prompt tokens are a stable
|
|
1549
|
+
* prefix across requests, and how often that prefix arrives already warm
|
|
1550
|
+
* (`report.tokens.providerCachedInput` is the numerator for the rate).
|
|
1551
|
+
*
|
|
1552
|
+
* Both are required together — a hit rate over an unknown prefix says nothing
|
|
1553
|
+
* about how much money is at stake. Declaring them unlocks the full decision,
|
|
1554
|
+
* including knowingly trading the prefix cache for a smaller prompt when that
|
|
1555
|
+
* is the cheaper side. Left out, the layer refuses to compress a prompt out of
|
|
1556
|
+
* a cacheable tier rather than assume a rate on your behalf.
|
|
1557
|
+
*/
|
|
1558
|
+
prefix?: {
|
|
1559
|
+
stableTokens: number;
|
|
1560
|
+
hitRate: number;
|
|
1561
|
+
};
|
|
1562
|
+
/**
|
|
1563
|
+
* What running the compressor costs, in equivalent tokens (default 0, which
|
|
1564
|
+
* fits a local deterministic compressor). A compressor that is itself another
|
|
1565
|
+
* model call is not free, and unaccounted spend shows up nowhere else.
|
|
1566
|
+
*/
|
|
1567
|
+
compressorCostTokens?: number;
|
|
1568
|
+
/** Do not compress contexts below this many tokens (default 2000). */
|
|
1569
|
+
minimumContextTokens?: number;
|
|
1570
|
+
/** Do not apply a compression that saves fewer net tokens than this (default 200). */
|
|
1571
|
+
minimumNetTokens?: number;
|
|
1572
|
+
}
|
|
1573
|
+
|
|
1357
1574
|
/**
|
|
1358
1575
|
* Provider-neutral generation interface. Every provider (OpenAI-compatible,
|
|
1359
1576
|
* Anthropic-compatible, Google-compatible, self-hosted) is adapted to this
|
|
@@ -1370,17 +1587,52 @@ interface ProviderRequest {
|
|
|
1370
1587
|
seed?: number;
|
|
1371
1588
|
/** Backend id selected by the constraints layer, for observability. */
|
|
1372
1589
|
constraintBackend?: string;
|
|
1590
|
+
/**
|
|
1591
|
+
* How many LEADING messages form the stable, cacheable prefix — the part the
|
|
1592
|
+
* provider may reuse across requests. `undefined` claims nothing and adapters
|
|
1593
|
+
* mark no cache breakpoint.
|
|
1594
|
+
*
|
|
1595
|
+
* Adapters that need EXPLICIT breakpoints (Anthropic) place them at this
|
|
1596
|
+
* boundary; adapters that cache automatically (OpenAI) may ignore it. The
|
|
1597
|
+
* count is a claim about byte-for-byte stability, so a caller that rewrites an
|
|
1598
|
+
* earlier turn must lower it.
|
|
1599
|
+
*/
|
|
1600
|
+
stablePrefixMessages?: number;
|
|
1601
|
+
/**
|
|
1602
|
+
* Routing hint so requests sharing a prefix land on the same cache shard
|
|
1603
|
+
* (OpenAI `prompt_cache_key`). It is a hint, never an identity: it must not
|
|
1604
|
+
* carry tenant or user data, because it does not isolate anything.
|
|
1605
|
+
*/
|
|
1606
|
+
promptCacheKey?: string;
|
|
1373
1607
|
}
|
|
1374
1608
|
interface ProviderUsage {
|
|
1375
1609
|
inputTokens: number;
|
|
1376
1610
|
outputTokens: number;
|
|
1377
1611
|
/** Provider-reported prefix/KV-cached input tokens (ledger 4), if any. */
|
|
1378
1612
|
cachedInputTokens?: number;
|
|
1613
|
+
/**
|
|
1614
|
+
* Input tokens billed at a PREMIUM to WRITE the provider's prefix cache
|
|
1615
|
+
* (Anthropic `cache_creation_input_tokens`: 1.25x the normal rate).
|
|
1616
|
+
*
|
|
1617
|
+
* This is a COST, not a discount, and it is reported separately for exactly
|
|
1618
|
+
* that reason: `cachedInputTokens` is a saving to publish in ledger 4, this
|
|
1619
|
+
* is the price paid to obtain it. Folding the two together would report a
|
|
1620
|
+
* bill INCREASE as a saving.
|
|
1621
|
+
*/
|
|
1622
|
+
cacheWriteInputTokens?: number;
|
|
1379
1623
|
}
|
|
1380
1624
|
interface ProviderResponse {
|
|
1381
1625
|
text: string;
|
|
1382
1626
|
usage: ProviderUsage;
|
|
1383
|
-
/**
|
|
1627
|
+
/**
|
|
1628
|
+
* Whether the provider enforced the schema natively.
|
|
1629
|
+
*
|
|
1630
|
+
* An explicit `false` is a claim about the guarantee that was obtained, not a
|
|
1631
|
+
* detail: the pipeline reads it and reports `constraints.backend: "none"`, so
|
|
1632
|
+
* an adapter that had to ask for JSON in the prompt (for instance because the
|
|
1633
|
+
* model rejects `response_format`) must set it. Leaving it undefined asserts
|
|
1634
|
+
* nothing and keeps the planned backend.
|
|
1635
|
+
*/
|
|
1384
1636
|
structuredNatively?: boolean;
|
|
1385
1637
|
/**
|
|
1386
1638
|
* Por qué paró de generar el modelo. `"length"` significa que se quedó sin
|
|
@@ -1397,13 +1649,10 @@ interface ProviderResponse {
|
|
|
1397
1649
|
* Llamadas a herramienta que pidió el modelo, tal cual las devolvió el
|
|
1398
1650
|
* proveedor.
|
|
1399
1651
|
*
|
|
1400
|
-
*
|
|
1401
|
-
*
|
|
1402
|
-
*
|
|
1403
|
-
*
|
|
1404
|
-
* modo que el caso no es exótico: es el camino normal en cuanto hay tools.
|
|
1405
|
-
* Mismo tipo que expone `SavingsResult.toolCalls`: el pipeline lo transmite
|
|
1406
|
-
* tal cual hasta el llamante (no solo hasta aquí).
|
|
1652
|
+
* When the model invokes a tool it returns `content: null` + `tool_calls`, so
|
|
1653
|
+
* this is the normal shape as soon as tools are in play — not an edge case.
|
|
1654
|
+
* Same type as `SavingsResult.toolCalls`: the pipeline forwards it unchanged
|
|
1655
|
+
* all the way to the caller.
|
|
1407
1656
|
*/
|
|
1408
1657
|
toolCalls?: ToolCallRequest[];
|
|
1409
1658
|
raw?: unknown;
|
|
@@ -1447,6 +1696,13 @@ declare function createMockProvider(options?: MockProviderOptions): Provider;
|
|
|
1447
1696
|
* with a JSON schema when a contract is present, which is the provider-native
|
|
1448
1697
|
* constrained-decoding path preferred by the layer.
|
|
1449
1698
|
*
|
|
1699
|
+
* Models that predate structured outputs reject `response_format` with a 400. The
|
|
1700
|
+
* adapter degrades for those: it repeats the request once with the schema carried
|
|
1701
|
+
* in the prompt instead, remembers the model so later requests skip the rejected
|
|
1702
|
+
* attempt, and reports `structuredNatively: false` so the savings report states
|
|
1703
|
+
* `constraints.backend: "none"` rather than claiming a guarantee the provider
|
|
1704
|
+
* never gave.
|
|
1705
|
+
*
|
|
1450
1706
|
* Network calls use global `fetch` (Node >= 18) so there is no SDK dependency.
|
|
1451
1707
|
*/
|
|
1452
1708
|
interface OpenAICompatibleOptions {
|
|
@@ -1486,8 +1742,74 @@ declare class OpenAICompatibleProvider implements Provider {
|
|
|
1486
1742
|
readonly id: string;
|
|
1487
1743
|
private readonly opts;
|
|
1488
1744
|
private readonly fetchImpl;
|
|
1745
|
+
/** Models of this endpoint already known to reject `response_format`. */
|
|
1746
|
+
private readonly sinResponseFormat;
|
|
1489
1747
|
constructor(opts: OpenAICompatibleOptions);
|
|
1490
1748
|
generate(req: ProviderRequest): Promise<ProviderResponse>;
|
|
1749
|
+
/** Build the request body and post it. `nativo` selects the `response_format` path. */
|
|
1750
|
+
private enviar;
|
|
1751
|
+
/** Translate a 2xx endpoint response into the provider-neutral shape. */
|
|
1752
|
+
private leer;
|
|
1753
|
+
}
|
|
1754
|
+
|
|
1755
|
+
/**
|
|
1756
|
+
* Anthropic Messages API adapter.
|
|
1757
|
+
*
|
|
1758
|
+
* Speaks `POST /v1/messages`: the system prompt travels in its own top-level
|
|
1759
|
+
* `system` field (not as a message), tools are `{name, description,
|
|
1760
|
+
* input_schema}` with no `function` wrapper, and prefix caching is requested
|
|
1761
|
+
* with EXPLICIT `cache_control` breakpoints — Anthropic does not cache on its
|
|
1762
|
+
* own. A cached read is billed at 0.1x the input rate, which is the largest
|
|
1763
|
+
* single lever on the input side of a customer's bill.
|
|
1764
|
+
*
|
|
1765
|
+
* Network calls use global `fetch` (Node >= 18) so there is no SDK dependency.
|
|
1766
|
+
*/
|
|
1767
|
+
/** Render order of the request, which is also the order the cache prefix grows in. */
|
|
1768
|
+
interface AnthropicOptions {
|
|
1769
|
+
apiKey?: string;
|
|
1770
|
+
/** Defaults to the public API. Override for a gateway or a proxy. */
|
|
1771
|
+
baseUrl?: string;
|
|
1772
|
+
/** Wire version pinned in the `anthropic-version` header. */
|
|
1773
|
+
apiVersion?: string;
|
|
1774
|
+
/** Extra headers (e.g. for a gateway). */
|
|
1775
|
+
headers?: Record<string, string>;
|
|
1776
|
+
/** Injectable fetch for testing. */
|
|
1777
|
+
fetchImpl?: typeof fetch;
|
|
1778
|
+
/**
|
|
1779
|
+
* `max_tokens` is REQUIRED by this API — unlike the OpenAI shape, where it is
|
|
1780
|
+
* optional. A request without one is a 400, so the adapter has to supply a
|
|
1781
|
+
* value. It is deliberately generous: a low default would truncate answers
|
|
1782
|
+
* and the truncation would look like a model failure rather than an adapter
|
|
1783
|
+
* default.
|
|
1784
|
+
*/
|
|
1785
|
+
defaultMaxOutputTokens?: number;
|
|
1786
|
+
/**
|
|
1787
|
+
* When true (recommended for measurement), a response whose `usage` block is
|
|
1788
|
+
* missing its counters is treated as a HARD ERROR instead of being coerced to
|
|
1789
|
+
* 0 tokens. An absent usage payload otherwise becomes `actual cost = 0` → a
|
|
1790
|
+
* fabricated 100% saving.
|
|
1791
|
+
*/
|
|
1792
|
+
requireUsage?: boolean;
|
|
1793
|
+
/**
|
|
1794
|
+
* How many `cache_control` breakpoints the adapter may spend.
|
|
1795
|
+
*
|
|
1796
|
+
* The API caps them at 4 per request. The adapter never needs more than 2
|
|
1797
|
+
* (end of system, end of the stable message prefix) and the cap is exposed so
|
|
1798
|
+
* a gateway with a stricter limit can lower it. Raising it above 4 is
|
|
1799
|
+
* pointless: the extra breakpoints are rejected by the API, not ignored.
|
|
1800
|
+
*/
|
|
1801
|
+
maxCacheBreakpoints?: number;
|
|
1802
|
+
}
|
|
1803
|
+
declare class AnthropicProvider implements Provider {
|
|
1804
|
+
readonly id: string;
|
|
1805
|
+
private readonly opts;
|
|
1806
|
+
private readonly fetchImpl;
|
|
1807
|
+
constructor(opts?: AnthropicOptions);
|
|
1808
|
+
generate(req: ProviderRequest): Promise<ProviderResponse>;
|
|
1809
|
+
private construirCuerpo;
|
|
1810
|
+
private marcarCache;
|
|
1811
|
+
/** Traduce una respuesta 2xx a la forma neutra del layer. */
|
|
1812
|
+
private leer;
|
|
1491
1813
|
}
|
|
1492
1814
|
|
|
1493
1815
|
/**
|
|
@@ -1525,7 +1847,7 @@ declare class LicenseError extends Error {
|
|
|
1525
1847
|
* against BiVelio's public key, decode the payload, and apply the same grace
|
|
1526
1848
|
* policy the license-client uses. Verification is **browser-safe**: it uses
|
|
1527
1849
|
* `@noble/curves` (pure JS) + Web primitives (`atob`, `TextEncoder`) rather than
|
|
1528
|
-
* `node:crypto`, so the shared
|
|
1850
|
+
* `node:crypto`, so the shared entry point still compiles into the browser
|
|
1529
1851
|
* demo bundle. The private signing key never leaves the control plane; only the
|
|
1530
1852
|
* public key is embedded here, so a token cannot be forged or tampered with.
|
|
1531
1853
|
*/
|
|
@@ -1737,7 +2059,7 @@ declare function liveGateFromKey(licenseKey: string, publicKey: string | License
|
|
|
1737
2059
|
* hole the old one-shot resolution left open. `isActive()` is true while active or
|
|
1738
2060
|
* in grace; `isEntitled` additionally requires the feature in the token's grant.
|
|
1739
2061
|
* Online revocation is added by pairing {@link liveGateFromKey} with a
|
|
1740
|
-
* poller-fed snapshot
|
|
2062
|
+
* poller-fed snapshot, which `createSavingsLayer` wires up for you.
|
|
1741
2063
|
*/
|
|
1742
2064
|
declare function gateFromKey(licenseKey: string, publicKey: string | LicenseKeyset, now?: () => number): ResolvedGate;
|
|
1743
2065
|
|
|
@@ -1788,6 +2110,12 @@ interface RevocationPollerOptions {
|
|
|
1788
2110
|
* silencio, durante días. Quien tenga observabilidad propia puede apagarlos;
|
|
1789
2111
|
* quien no la tenga, que es casi todo el mundo, se entera igual.
|
|
1790
2112
|
*/
|
|
2113
|
+
silenceWarnings?: boolean;
|
|
2114
|
+
/**
|
|
2115
|
+
* @deprecated Spanish name that leaked into the public options. Use
|
|
2116
|
+
* {@link RevocationPollerOptions.silenceWarnings}. Honoured for one minor
|
|
2117
|
+
* version so nobody gets their warnings back by surprise.
|
|
2118
|
+
*/
|
|
1791
2119
|
silenciarAvisos?: boolean;
|
|
1792
2120
|
}
|
|
1793
2121
|
/**
|
|
@@ -1827,23 +2155,22 @@ declare class RevocationPoller {
|
|
|
1827
2155
|
* last-known-good. Only a transport/abort/parse error counts as a failure.
|
|
1828
2156
|
*/
|
|
1829
2157
|
pollOnce(): Promise<void>;
|
|
2158
|
+
/** El interruptor, aceptando todavía el nombre viejo. */
|
|
2159
|
+
private callado;
|
|
1830
2160
|
/** Un aviso por causa y por proceso, respetando el interruptor. */
|
|
1831
2161
|
private avisar;
|
|
1832
2162
|
/**
|
|
1833
|
-
*
|
|
1834
|
-
*
|
|
1835
|
-
* Este es el fallo silencioso del SDK: el sondeo es fail-open a propósito —una
|
|
1836
|
-
* red caída no puede tumbar la producción de un cliente— pero la consecuencia
|
|
1837
|
-
* es que un endpoint mal configurado, un cortafuegos nuevo o una URL cambiada
|
|
1838
|
-
* no producen NINGÚN síntoma. La licencia sigue funcionando, las peticiones
|
|
1839
|
-
* siguen pasando, y lo único que ocurre es que la lista de revocaciones se
|
|
1840
|
-
* queda congelada. Durante días.
|
|
2163
|
+
* THE CONTROL PLANE HAS BEEN UNREACHABLE FOR A WHILE.
|
|
1841
2164
|
*
|
|
1842
|
-
*
|
|
1843
|
-
*
|
|
2165
|
+
* Polling is fail-open by design: a network outage must never take down the
|
|
2166
|
+
* customer's production traffic. The trade-off is that an unreachable control
|
|
2167
|
+
* plane has no visible symptom — the license keeps working and requests keep
|
|
2168
|
+
* flowing while the revocation list quietly goes stale.
|
|
1844
2169
|
*
|
|
1845
|
-
*
|
|
1846
|
-
*
|
|
2170
|
+
* So the warning goes to the console BY DEFAULT, not only through the optional
|
|
2171
|
+
* `onEvent` observer. It fires on the third consecutive failure and then every
|
|
2172
|
+
* twelfth: a lost request is normal, a week-long outage must keep reminding.
|
|
2173
|
+
* Silence it with `silenceWarnings` if you have your own observability.
|
|
1847
2174
|
*/
|
|
1848
2175
|
private avisarFalloDeRed;
|
|
1849
2176
|
/** Fire one non-blocking warm poll, then self-schedule. Idempotent. */
|
|
@@ -1880,6 +2207,13 @@ interface UsageEmitterOptions {
|
|
|
1880
2207
|
* exactamente lo mismo que se ve cuando no hay tráfico—.
|
|
1881
2208
|
*/
|
|
1882
2209
|
onAuthError?: (status: number) => void;
|
|
2210
|
+
/**
|
|
2211
|
+
* Silence the console warnings. They are ON by default, exactly as in the
|
|
2212
|
+
* revocation poller: metering is best-effort, so a broken ingest breaks
|
|
2213
|
+
* nothing visible — usage simply stops arriving, silently, and the customer's
|
|
2214
|
+
* dashboard stays empty forever.
|
|
2215
|
+
*/
|
|
2216
|
+
silenceWarnings?: boolean;
|
|
1883
2217
|
}
|
|
1884
2218
|
/**
|
|
1885
2219
|
* Buffers counts-only usage and flushes it to the control plane. DETECTIVE
|
|
@@ -1899,6 +2233,13 @@ declare class UsageEmitter {
|
|
|
1899
2233
|
private ci;
|
|
1900
2234
|
private timer;
|
|
1901
2235
|
private flushing;
|
|
2236
|
+
/**
|
|
2237
|
+
* CONSECUTIVE failed flushes. Warn on the third and then every twelfth: one
|
|
2238
|
+
* lost batch is normal, a whole hour is not. Same cadence as the revocation
|
|
2239
|
+
* poller on purpose — two different rhythms for the same class of failure
|
|
2240
|
+
* only confuse whoever reads the logs.
|
|
2241
|
+
*/
|
|
2242
|
+
private fallosSeguidos;
|
|
1902
2243
|
constructor(options: UsageEmitterOptions);
|
|
1903
2244
|
/** Set the resolved seat identity (derivation is async; records emitted before
|
|
1904
2245
|
* it resolves are stamped at flush time, so none are lost). */
|
|
@@ -1914,6 +2255,19 @@ declare class UsageEmitter {
|
|
|
1914
2255
|
* since retrying a rejected payload would loop forever.
|
|
1915
2256
|
*/
|
|
1916
2257
|
flush(): Promise<void>;
|
|
2258
|
+
/**
|
|
2259
|
+
* THE INGEST HAS BEEN FAILING FOR A WHILE AND NOBODY NOTICED.
|
|
2260
|
+
*
|
|
2261
|
+
* Metering is best-effort by design — it must never take down the customer's
|
|
2262
|
+
* request — and that turns every failure into SILENT data loss. Without this,
|
|
2263
|
+
* the only symptom is an empty dashboard, which is indistinguishable from
|
|
2264
|
+
* "you haven't sent anything yet": the symptom points the opposite way from
|
|
2265
|
+
* the cause.
|
|
2266
|
+
*
|
|
2267
|
+
* Warns on the third consecutive failure and then every twelfth. Not the
|
|
2268
|
+
* first — a lost request is normal — and not every one, which becomes noise.
|
|
2269
|
+
*/
|
|
2270
|
+
private avisarDeFallo;
|
|
1917
2271
|
private requeue;
|
|
1918
2272
|
private toWire;
|
|
1919
2273
|
}
|
|
@@ -1959,6 +2313,17 @@ interface SavingsLayerConfig {
|
|
|
1959
2313
|
schemaRegistry?: SchemaRegistry;
|
|
1960
2314
|
constraintBackends?: ConstraintBackend[];
|
|
1961
2315
|
compressor?: Compressor;
|
|
2316
|
+
/**
|
|
2317
|
+
* The numbers behind "compress only when it pays for itself" — in money, not
|
|
2318
|
+
* in tokens.
|
|
2319
|
+
*
|
|
2320
|
+
* All optional, and the default never invents a saving: the layer compresses
|
|
2321
|
+
* when the net token saving justifies it, and holds back when compressing
|
|
2322
|
+
* would push the prompt below the provider's cacheable minimum — where fewer
|
|
2323
|
+
* tokens can mean a larger bill. Declare {@link CompressionEconomics.prefix}
|
|
2324
|
+
* with your own measurements to unlock the full decision in both directions.
|
|
2325
|
+
*/
|
|
2326
|
+
compressionEconomics?: CompressionEconomics;
|
|
1962
2327
|
retriever?: Retriever;
|
|
1963
2328
|
reranker?: Reranker;
|
|
1964
2329
|
/**
|
|
@@ -2112,6 +2477,16 @@ interface SavingsLayerConfig {
|
|
|
2112
2477
|
* in tests/benchmarks that want internal errors to surface.
|
|
2113
2478
|
*/
|
|
2114
2479
|
failOpen?: boolean;
|
|
2480
|
+
/**
|
|
2481
|
+
* Silence BV-SALA's console warnings. They are ON by default, and they only
|
|
2482
|
+
* ever report a condition whose symptom is otherwise invisible: a saving knob
|
|
2483
|
+
* asked for but not wired up, usage that stops reaching the control plane, a
|
|
2484
|
+
* revocation list going stale. Nothing here throws and nothing here changes
|
|
2485
|
+
* what the layer does — turning it on only removes the console noise, so set
|
|
2486
|
+
* it when you have your own observability. Applies to the pipeline, the usage
|
|
2487
|
+
* emitter and the revocation poller alike.
|
|
2488
|
+
*/
|
|
2489
|
+
silenceWarnings?: boolean;
|
|
2115
2490
|
/** Injectable clock for deterministic tests. */
|
|
2116
2491
|
now?: () => number;
|
|
2117
2492
|
/** Injectable id generator for deterministic tests. */
|
|
@@ -2153,7 +2528,7 @@ interface SavingsLayer {
|
|
|
2153
2528
|
* Create a savings layer. The only hard requirement is a `provider`; a default
|
|
2154
2529
|
* heuristic tokenizer and in-memory implementations of every other component
|
|
2155
2530
|
* are wired automatically. Replace any of them for production (Redis exact
|
|
2156
|
-
* cache,
|
|
2531
|
+
* cache, a vector-backed semantic cache, a real tokenizer, a compressor sidecar,
|
|
2157
2532
|
* XGrammar/llguidance backends, etc.).
|
|
2158
2533
|
*
|
|
2159
2534
|
* @example
|
|
@@ -2184,20 +2559,14 @@ declare function createSavingsLayer(config: Omit<SavingsLayerConfig, "tokenizer"
|
|
|
2184
2559
|
* misleading metrics.
|
|
2185
2560
|
*/
|
|
2186
2561
|
/**
|
|
2187
|
-
*
|
|
2188
|
-
* package.json a propósito: el bundle publicado no incluye package.json, así que
|
|
2189
|
-
* un `require` aquí rompería en tiempo de ejecución en casa del cliente.
|
|
2562
|
+
* The version the SDK reports for itself.
|
|
2190
2563
|
*
|
|
2191
|
-
*
|
|
2192
|
-
*
|
|
2193
|
-
*
|
|
2194
|
-
*
|
|
2195
|
-
* distinguir a quien ya actualizó de quien no.
|
|
2196
|
-
*
|
|
2197
|
-
* Por eso hay un test que la ata a package.json (`test/version-declarada.test.ts`).
|
|
2198
|
-
* Si subes la versión del paquete y olvidas esta línea, la suite se pone roja
|
|
2199
|
-
* antes de que la Release llegue a npm.
|
|
2564
|
+
* Written by hand and deliberately NOT read from package.json: the published
|
|
2565
|
+
* bundle does not ship package.json, so a `require` here would fail at runtime
|
|
2566
|
+
* on the customer's machine. `test/version-declarada.test.ts` ties this
|
|
2567
|
+
* constant to package.json so the two can never drift.
|
|
2200
2568
|
*/
|
|
2201
|
-
declare const BV_SALA_VERSION = "0.2.1";
|
|
2202
2569
|
|
|
2203
|
-
|
|
2570
|
+
declare const BV_SALA_VERSION = "0.3.0";
|
|
2571
|
+
|
|
2572
|
+
export { type AnthropicOptions, AnthropicProvider, BV_SALA_VERSION, type BivelioHandoff, CATALOGO, COMPACT_ROWS_INSTRUCTION, type CacheOutcome, type CacheTier, type CachedAnswer, type CanonicalRequest, type ChatMessage, type CompactRowsDecodeOptions, type CompactRowsOptions, type CompactVerification, type CompressionEconomics, type CompressionPolicyInput, type CompressionRequest, type CompressionResult, type Compressor, type ConstraintBackend, type ConstraintBackendId, type ConstraintPlan, type ConversationState, type CostBasis, type CostModel, DEFAULT_COST_MODEL, type DeterministicResolver, EMBEDDED_PUBLIC_KEY, type EmbeddingFn, type EvidenceDoc, type ExactCache, ExactTokenizer, type ExecutionClass, ExecutionRouter, type FormatChoice, FormatSelector, type FormatSelectorOptions, type HandoffFact, HeuristicEstimator, InMemoryExactCache, InMemorySemanticCache, type JsonSchema, LicenseError, type LicenseGate, type LicenseKeyset, type LicenseStatus, type LicenseTokenPayload, type LocalClassifier, MODELOS_POR_CODIFICACION, MODELS_BY_ENCODING, type MemoryDelta, type MessageRole, MockProvider, type MockProviderOptions, type OpenAICompatibleOptions, OpenAICompatibleProvider, type OutputContract, PRECIOS_AL_DIA, type Policy, type ProtectedSpan, type Provider, type ProviderRequest, type ProviderResponse, type ProviderUsage, QUALITY_STATUSES, type QualityStatus, type RequestContext, type Reranker, type ResolvedDeps, type ResolvedGate, type ResolvedLicense, ResolverRegistry, type ResolverResult, type RetrievalOptions, type RetrievalOutcome, type RetrievalPlan, type Retriever, type RevocationListBody, type RevocationListVerification, type RevocationPollEvent, RevocationPoller, type RevocationPollerOptions, RevocationSnapshot, type RouteDecision, type SavingsLayer, type SavingsLayerConfig, SavingsLedger, type SavingsReport, type SavingsRequest, type SavingsResult, type SavingsToggles, SchemaRegistry, type SemanticCacheOptions, type SemanticHit, type SemanticRecord, type SemanticVerifier, type SerializationFormat, type Serializer, Singleflight, type TokenEstimator, TokenizerRegistry, type ToolCallRequest, type ToolDescriptor, type ToolRisk, type ToolSelection, ToolSelector, type ToolSelectorOptions, TraceCollector, type TraceEvent, UsageEmitter, type UsageEmitterOptions, type ValidationError, type ValidationResult, adaptiveK, arithmeticResolver, buildExactKey, buildInflightKey, canonicalize, compactRowsSerializer, computeNextDelay, cosine, createCompactRowsSerializer, createDefaultResolvers, createHandoff, createMockProvider, createSavingsLayer, csvSerializer, decodeCompactRows, dedupe, defaultTokenizerRegistry, deriveSeatId, deriveStatus, detectCi, emptyState, encodeCompactRows, evalArithmetic, evaluatePolicy, executeRetrievalPlan, gateFromKey, heuristicEstimator, isKeyset, isStrictCompatible, jsonCompactSerializer, keyTokensMatch, liveGateFromKey, mergeDelta, noopCompressor, normalizeText, precioDeLista, projectContext, providerNativeBackend, randomId, registerExactTokenizer, registrarTokenizadorExacto, renderState, resolveConstraintBackend, resolveDependencies, resolveLicense, sha256, shouldCompress, stableStringify, uniformColumns, validateAgainstSchema, verifyCompactSignature, verifyLicenseToken, verifyRevocationList, withProtectedSpans };
|