@bivelio/savings-layer 0.5.1 → 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +115 -0
- package/LICENSE +183 -163
- package/LICENSE-CLOUD.md +174 -0
- package/NOTICE +11 -0
- package/README.md +228 -126
- package/dist/adapters/ai-sdk.js +78 -78
- package/dist/adapters/anthropic-sdk.js +88 -88
- package/dist/adapters/langchain.js +58 -58
- package/dist/adapters/mastra.js +70 -70
- package/dist/gateway/bin.js +1291 -1082
- package/dist/index.d.ts +367 -197
- package/dist/index.js +1717 -1641
- package/docs/terminales/README.md +39 -33
- package/docs/terminales/aider.md +1 -1
- package/docs/terminales/claude-code.md +2 -2
- package/docs/terminales/cline.md +1 -1
- package/docs/terminales/codex.md +3 -3
- package/docs/terminales/copilot.md +1 -1
- package/docs/terminales/crush.md +1 -1
- package/docs/terminales/cursor.md +32 -0
- package/docs/terminales/deepseek.md +28 -0
- package/docs/terminales/droid.md +1 -1
- package/docs/terminales/gemini.md +38 -0
- package/docs/terminales/goose.md +2 -2
- package/docs/terminales/kilo.md +2 -2
- package/docs/terminales/no-soportados.md +16 -27
- package/docs/terminales/ollama.md +19 -0
- package/docs/terminales/opencode.md +2 -2
- package/docs/terminales/qwen-code.md +2 -2
- package/package.json +48 -20
package/dist/index.d.ts
CHANGED
|
@@ -1,161 +1,3 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* The four-ledger savings accounting.
|
|
3
|
-
*
|
|
4
|
-
* These ledgers are kept strictly separate on purpose: each answers a different
|
|
5
|
-
* question, and summing them would double-count.
|
|
6
|
-
* The ledger is a pure accumulator: the pipeline records events into it, and it
|
|
7
|
-
* produces the numbers that populate the SavingsReport. It never guesses — every
|
|
8
|
-
* increment is driven by an explicit pipeline decision.
|
|
9
|
-
*/
|
|
10
|
-
/**
|
|
11
|
-
* Where the baseline being compared against comes from.
|
|
12
|
-
*
|
|
13
|
-
* - `measured` — reconstructed from this request's provider counter. Real money.
|
|
14
|
-
* - `reference` — MEASURED BY REFERENCE: the call was avoided (a repeat served
|
|
15
|
-
* without calling the provider) and a byte-identical request had
|
|
16
|
-
* already run through the layer untouched. The baseline is the
|
|
17
|
-
* provider's own counter for that request, valued at the
|
|
18
|
-
* provider's cached-input rate. Output is credited only under
|
|
19
|
-
* deterministic sampling, on a model with a list price in the
|
|
20
|
-
* catalog that does not reason and honors that setting, when
|
|
21
|
-
* the call that ran carried no hidden reasoning (zero reasoning
|
|
22
|
-
* tokens reported, or no reasoning block where the provider
|
|
23
|
-
* does not break them out), and then at 95% of its counter.
|
|
24
|
-
* With an output schema, only when the schema traveled exactly
|
|
25
|
-
* as sent, in strict mode, in the provider's native
|
|
26
|
-
* response-format parameter. Valued low on purpose. Enters the
|
|
27
|
-
* commission base.
|
|
28
|
-
* - `estimated` — the call was AVOIDED: it never ran, so there is no provider
|
|
29
|
-
* counter and the baseline comes from the local estimator. Still
|
|
30
|
-
* the best figure available, but not measured.
|
|
31
|
-
* - `unmeasured` — the provider reported no usage. Nothing is claimed.
|
|
32
|
-
* - `shadow` — HYPOTHETICAL: what would have happened with a lever turned on
|
|
33
|
-
* (the shadow lane). Never enters the commission base, never
|
|
34
|
-
* added to measured or to any total; shadow reports are built
|
|
35
|
-
* separately and `estimateCost` must never produce this value.
|
|
36
|
-
*
|
|
37
|
-
* Summing `measured` and `estimated` into a single "money saved" figure is
|
|
38
|
-
* exactly what a verify-don't-trust product cannot do — and adding `shadow` to
|
|
39
|
-
* either would be making the savings up.
|
|
40
|
-
*/
|
|
41
|
-
type CostBasis = "measured" | "reference" | "estimated" | "unmeasured" | "shadow";
|
|
42
|
-
interface CostModel {
|
|
43
|
-
/** Cost per 1K input tokens, in the caller's currency unit. */
|
|
44
|
-
inputPer1k: number;
|
|
45
|
-
/** Cost per 1K output tokens. */
|
|
46
|
-
outputPer1k: number;
|
|
47
|
-
/** Optional discounted rate for provider-cached (prefix) input tokens. */
|
|
48
|
-
cachedInputPer1k?: number;
|
|
49
|
-
}
|
|
50
|
-
/**
|
|
51
|
-
* Why an avoided call (a cache or coalesced hit) could not be valued by
|
|
52
|
-
* reference, and so stays `estimated`.
|
|
53
|
-
*
|
|
54
|
-
* - `absent` — the stored answer carries no provider counter.
|
|
55
|
-
* - `malformed` — it carries one, but not in a readable shape.
|
|
56
|
-
* - `witness` — the counter is not bound to the answer being served.
|
|
57
|
-
* - `model-mismatch` — the counter belongs to another model.
|
|
58
|
-
* - `expired` — the counter is older than one billing period.
|
|
59
|
-
* - `different-request` — the request that ran was not byte-identical to this one.
|
|
60
|
-
* - `quality` — the answer being served fails its output contract.
|
|
61
|
-
* - `no-cached-price` — no documented cached-input rate for this exact model:
|
|
62
|
-
* its own catalogue row (or a dated snapshot of it) or
|
|
63
|
-
* `costModels`, never a row inherited from a sibling id.
|
|
64
|
-
* - `no-credit` — nothing left to credit once the allowances are applied.
|
|
65
|
-
*/
|
|
66
|
-
type ReferenceRejection = "absent" | "malformed" | "witness" | "model-mismatch" | "expired" | "different-request" | "quality" | "no-cached-price" | "no-credit";
|
|
67
|
-
/**
|
|
68
|
-
* The provider counter an avoided call is valued against, and every figure
|
|
69
|
-
* needed to recompute its baseline:
|
|
70
|
-
* `tierFactor * (creditedInputTokens / 1000 * cachedInputPer1k + creditedOutputTokens / 1000 * outputPer1k)`.
|
|
71
|
-
*/
|
|
72
|
-
interface ReferenceCredit {
|
|
73
|
-
/** Id of the report of the call that ran (its report.requestId). */
|
|
74
|
-
requestId: string;
|
|
75
|
-
/** Epoch ms when that call's provider counter was read. */
|
|
76
|
-
at: number;
|
|
77
|
-
/** Input tokens the provider billed for that call. */
|
|
78
|
-
inputTokens: number;
|
|
79
|
-
/** Output tokens the provider billed for that call (the smaller of the first answer and its repair). */
|
|
80
|
-
outputTokens: number;
|
|
81
|
-
/** Input tokens credited to this repeat: inputTokens minus the allowance for what the layer inserted (messages, the native schema wrapper, empty tool parameters). */
|
|
82
|
-
creditedInputTokens: number;
|
|
83
|
-
/**
|
|
84
|
-
* Output tokens credited: a share of outputTokens capped by this request's
|
|
85
|
-
* output limit. 0 unless this request asked for deterministic sampling
|
|
86
|
-
* (`temperature: 0`), the model has a list price in the catalog, does not
|
|
87
|
-
* reason and honors that setting, and the call that ran carried no hidden
|
|
88
|
-
* reasoning (zero reasoning tokens reported, or no reasoning block where the
|
|
89
|
-
* provider does not break them out).
|
|
90
|
-
*/
|
|
91
|
-
creditedOutputTokens: number;
|
|
92
|
-
/** Rate applied to every credited input token, per 1K: the provider's cached-input rate. */
|
|
93
|
-
cachedInputPer1k: number;
|
|
94
|
-
/** Rate applied to credited output tokens, per 1K. */
|
|
95
|
-
outputPer1k: number;
|
|
96
|
-
/** Fraction of the standard price applied: 1, or the discounted tier the call ran on or this request declared. */
|
|
97
|
-
tierFactor: number;
|
|
98
|
-
}
|
|
99
|
-
|
|
100
|
-
/**
|
|
101
|
-
* LIST PRICES PER MODEL.
|
|
102
|
-
*
|
|
103
|
-
* The layer reports savings in money, and money needs a price. This is the
|
|
104
|
-
* built-in catalogue of published list prices, so the dollar figures mean
|
|
105
|
-
* something without any configuration.
|
|
106
|
-
*
|
|
107
|
-
* ⚠️ A WRONG PRICE HERE IS WORSE THAN NO CATALOGUE. A model that is ABSENT falls
|
|
108
|
-
* back to the illustrative default AND warns, so the customer knows that column
|
|
109
|
-
* is not their invoice. A model that is PRESENT with the wrong price silences
|
|
110
|
-
* that warning and hands over a confident, wrong number. So only models whose
|
|
111
|
-
* published price is known with certainty belong here, and {@link PRECIOS_AL_DIA}
|
|
112
|
-
* records when they were last checked.
|
|
113
|
-
*
|
|
114
|
-
* Prices are USD per 1,000 tokens — the unit of `CostModel` — even though
|
|
115
|
-
* providers publish per million: 1M to 1K is a division by a thousand.
|
|
116
|
-
*
|
|
117
|
-
* `costModels` on the layer config ALWAYS wins over this. A customer with volume
|
|
118
|
-
* discounts, negotiated rates or a different region sets their own and this
|
|
119
|
-
* catalogue stays out of the way.
|
|
120
|
-
*/
|
|
121
|
-
/**
|
|
122
|
-
* Version of this price catalogue: the UTC day its content last changed (rows,
|
|
123
|
-
* announced price steps, cache-write premiums, tier rates). Two computations
|
|
124
|
-
* citing the same version were priced with the same content.
|
|
125
|
-
*/
|
|
126
|
-
declare const PRECIOS_AL_DIA = "2026-09-29";
|
|
127
|
-
/**
|
|
128
|
-
* Key = the model id as the provider publishes it, without a vendor prefix.
|
|
129
|
-
* Lookup tries the exact id, then the id without its provider prefix, then the
|
|
130
|
-
* LONGEST matching prefix — so `gpt-4o-2024-08-06` inherits `gpt-4o` without
|
|
131
|
-
* having to enumerate every dated snapshot.
|
|
132
|
-
*
|
|
133
|
-
* These are the rows in force on {@link PRECIOS_AL_DIA}. A price change a
|
|
134
|
-
* provider has already announced is applied by {@link precioDeLista} from its
|
|
135
|
-
* effective date, so a request is always priced at the rate of its own day.
|
|
136
|
-
*/
|
|
137
|
-
declare const CATALOGO: Readonly<Record<string, CostModel>>;
|
|
138
|
-
/**
|
|
139
|
-
* The list price for a model on the day of the request, or `undefined` when
|
|
140
|
-
* the catalogue has no price for it in force that day.
|
|
141
|
-
*
|
|
142
|
-
* @param model the id as the caller passes it, with or without a vendor prefix.
|
|
143
|
-
* @param at when the request was made (epoch ms or a `Date`); defaults to now.
|
|
144
|
-
* Prices resolve per UTC day, so a price change the provider has announced
|
|
145
|
-
* applies from its effective date.
|
|
146
|
-
*/
|
|
147
|
-
declare function precioDeLista(model: string, at?: Date | number): CostModel | undefined;
|
|
148
|
-
|
|
149
|
-
/**
|
|
150
|
-
* The version the SDK reports for itself.
|
|
151
|
-
*
|
|
152
|
-
* Written by hand and deliberately NOT read from package.json: the published
|
|
153
|
-
* bundle does not ship package.json, so a `require` here would fail at runtime
|
|
154
|
-
* on the customer's machine. `test/version-declarada.test.ts` ties this
|
|
155
|
-
* constant to package.json so the two can never drift.
|
|
156
|
-
*/
|
|
157
|
-
declare const BV_SALA_VERSION = "0.5.1";
|
|
158
|
-
|
|
159
1
|
/** How the layer served a request that is eligible for a verification check. */
|
|
160
2
|
type VerificationStratum =
|
|
161
3
|
/** Answered without calling the provider (cache, resolver, identical request in flight). */
|
|
@@ -1067,10 +909,28 @@ type SavingsStreamEvent<T = unknown> =
|
|
|
1067
909
|
type: "done";
|
|
1068
910
|
result: SavingsResult<T>;
|
|
1069
911
|
};
|
|
912
|
+
/** Optional cancellation and total-deadline controls for one request. */
|
|
913
|
+
interface SavingsRequestOptions {
|
|
914
|
+
/** Cancels this request without cancelling other users of the same layer. */
|
|
915
|
+
signal?: AbortSignal;
|
|
916
|
+
/**
|
|
917
|
+
* Total budget from the public entry point until the answer is delivered, in
|
|
918
|
+
* milliseconds. It must be a positive finite integer no greater than
|
|
919
|
+
* 2,147,483,647 (about 24.8 days), the largest portable `setTimeout` delay.
|
|
920
|
+
*
|
|
921
|
+
* Expiry aborts the provider signal and throws `RequestTimeoutError`. A custom
|
|
922
|
+
* provider that ignores that signal is awaited until its work stops; if it never
|
|
923
|
+
* stops, rejection is delayed indefinitely too. No late cache or learning write
|
|
924
|
+
* is started after the provider finally settles.
|
|
925
|
+
*/
|
|
926
|
+
requestTimeoutMs?: number;
|
|
927
|
+
}
|
|
1070
928
|
/** Options for `generateStream`. */
|
|
1071
|
-
interface SavingsStreamOptions {
|
|
929
|
+
interface SavingsStreamOptions extends SavingsRequestOptions {
|
|
1072
930
|
/** Aborting it has the same effect as calling `abort()` on the stream. */
|
|
1073
931
|
signal?: AbortSignal;
|
|
932
|
+
/** Total budget for this stream; see `SavingsRequestOptions.requestTimeoutMs`. */
|
|
933
|
+
requestTimeoutMs?: number;
|
|
1074
934
|
}
|
|
1075
935
|
/**
|
|
1076
936
|
* The handle returned by `generateStream`. Iterate it (once) for the events, or
|
|
@@ -1487,6 +1347,34 @@ interface SavingsReport {
|
|
|
1487
1347
|
protectedSpans: number;
|
|
1488
1348
|
fidelityPassed: boolean;
|
|
1489
1349
|
};
|
|
1350
|
+
/**
|
|
1351
|
+
* Present only when tool-output compaction ran with `profile:
|
|
1352
|
+
* "lossless-data"`: one explained decision per tool output in the request,
|
|
1353
|
+
* applied or bypassed, in message order. Token counts come from the
|
|
1354
|
+
* estimator named in `estimator` and are estimates, never a verified saving.
|
|
1355
|
+
*/
|
|
1356
|
+
toolOutputCompaction?: {
|
|
1357
|
+
profile: "lossless-data";
|
|
1358
|
+
/** Always `"estimated"`: these counts are not the provider's. */
|
|
1359
|
+
tokenBasis: "estimated";
|
|
1360
|
+
/** Id of the token estimator that counted, and whether it uses the provider's rule. */
|
|
1361
|
+
estimator: {
|
|
1362
|
+
id: string;
|
|
1363
|
+
exact: boolean;
|
|
1364
|
+
};
|
|
1365
|
+
decisions: Array<{
|
|
1366
|
+
/** The tool call this output answers, unchanged. Absent when the message had none. */
|
|
1367
|
+
toolCallId?: string;
|
|
1368
|
+
status: "applied" | "bypassed";
|
|
1369
|
+
reason: "applied" | "not_json" | "invalid_json" | "duplicate_keys" | "invalid_unicode" | "not_smaller" | "below_threshold" | "protected";
|
|
1370
|
+
transformId: string;
|
|
1371
|
+
version: string;
|
|
1372
|
+
tokensBefore: number;
|
|
1373
|
+
/** What is sent: the compacted text when applied, the original otherwise. */
|
|
1374
|
+
tokensAfter: number;
|
|
1375
|
+
candidateTokens?: number;
|
|
1376
|
+
}>;
|
|
1377
|
+
};
|
|
1490
1378
|
constraints: {
|
|
1491
1379
|
backend: string;
|
|
1492
1380
|
schemaVersion: string;
|
|
@@ -1569,6 +1457,16 @@ interface SavingsReport {
|
|
|
1569
1457
|
cost: {
|
|
1570
1458
|
baselineEstimate: number;
|
|
1571
1459
|
actual: number;
|
|
1460
|
+
/**
|
|
1461
|
+
* Present, and `true`, only when a billed call of this request reached the
|
|
1462
|
+
* layer without its full provider counter (a stream cut before its final
|
|
1463
|
+
* usage event, an endpoint that omitted a counter). `actual` then prices
|
|
1464
|
+
* only the counters that did arrive, with the missing ones at 0: it is a
|
|
1465
|
+
* FLOOR of what the provider billed, possibly 0, never the spend. The
|
|
1466
|
+
* request is also `basis: "unmeasured"`. Do not add such an `actual` to
|
|
1467
|
+
* measured spend. Absent whenever every counter arrived.
|
|
1468
|
+
*/
|
|
1469
|
+
actualIsLowerBound?: true;
|
|
1572
1470
|
/**
|
|
1573
1471
|
* The saving as a RATIO of the baseline: `1 - actual / baselineEstimate`,
|
|
1574
1472
|
* rounded to six decimals (0 when nothing was saved or there is no baseline,
|
|
@@ -1664,13 +1562,10 @@ interface SavingsReport {
|
|
|
1664
1562
|
interface LicenseGate {
|
|
1665
1563
|
isEntitled(feature: string): boolean;
|
|
1666
1564
|
/**
|
|
1667
|
-
* Whether there is an overall active (or in-grace) license.
|
|
1668
|
-
*
|
|
1565
|
+
* Whether there is an overall active (or in-grace) license. The pipeline's
|
|
1566
|
+
* fail-closed entry check reads it.
|
|
1669
1567
|
*
|
|
1670
|
-
*
|
|
1671
|
-
* bypass: un gate al que le faltara el método dejaba pasar todas las
|
|
1672
|
-
* peticiones con solo un aviso por consola. Para desarrollo local está
|
|
1673
|
-
* `devUnlicensed: true`, que es explícito y no se cuela sin querer.
|
|
1568
|
+
* Required: a gate without it would let every request through.
|
|
1674
1569
|
*/
|
|
1675
1570
|
isActive(): boolean;
|
|
1676
1571
|
/**
|
|
@@ -1748,6 +1643,164 @@ interface TraceEvent {
|
|
|
1748
1643
|
atMs: number;
|
|
1749
1644
|
}
|
|
1750
1645
|
|
|
1646
|
+
/**
|
|
1647
|
+
* The four-ledger savings accounting.
|
|
1648
|
+
*
|
|
1649
|
+
* These ledgers are kept strictly separate on purpose: each answers a different
|
|
1650
|
+
* question, and summing them would double-count.
|
|
1651
|
+
* The ledger is a pure accumulator: the pipeline records events into it, and it
|
|
1652
|
+
* produces the numbers that populate the SavingsReport. It never guesses — every
|
|
1653
|
+
* increment is driven by an explicit pipeline decision.
|
|
1654
|
+
*/
|
|
1655
|
+
/**
|
|
1656
|
+
* Where the baseline being compared against comes from.
|
|
1657
|
+
*
|
|
1658
|
+
* - `measured` — reconstructed from this request's provider counter. Real money.
|
|
1659
|
+
* - `reference` — MEASURED BY REFERENCE: the call was avoided (a repeat served
|
|
1660
|
+
* without calling the provider) and a byte-identical request had
|
|
1661
|
+
* already run through the layer untouched. The baseline is the
|
|
1662
|
+
* provider's own counter for that request, valued at the
|
|
1663
|
+
* provider's cached-input rate. Output is credited only under
|
|
1664
|
+
* deterministic sampling, on a model with a list price in the
|
|
1665
|
+
* catalog that does not reason and honors that setting, when
|
|
1666
|
+
* the call that ran carried no hidden reasoning (zero reasoning
|
|
1667
|
+
* tokens reported, or no reasoning block where the provider
|
|
1668
|
+
* does not break them out), and then at 95% of its counter.
|
|
1669
|
+
* With an output schema, only when the schema traveled exactly
|
|
1670
|
+
* as sent, in strict mode, in the provider's native
|
|
1671
|
+
* response-format parameter. Valued low on purpose. Enters the
|
|
1672
|
+
* commission base.
|
|
1673
|
+
* - `estimated` — the call was AVOIDED: it never ran, so there is no provider
|
|
1674
|
+
* counter and the baseline comes from the local estimator. Still
|
|
1675
|
+
* the best figure available, but not measured.
|
|
1676
|
+
* - `unmeasured` — the provider reported no usage. Nothing is claimed.
|
|
1677
|
+
* - `shadow` — HYPOTHETICAL: what would have happened with a lever turned on
|
|
1678
|
+
* (the shadow lane). Never enters the commission base, never
|
|
1679
|
+
* added to measured or to any total; shadow reports are built
|
|
1680
|
+
* separately and `estimateCost` must never produce this value.
|
|
1681
|
+
*
|
|
1682
|
+
* Summing `measured` and `estimated` into a single "money saved" figure is
|
|
1683
|
+
* exactly what a verify-don't-trust product cannot do — and adding `shadow` to
|
|
1684
|
+
* either would be making the savings up.
|
|
1685
|
+
*/
|
|
1686
|
+
type CostBasis = "measured" | "reference" | "estimated" | "unmeasured" | "shadow";
|
|
1687
|
+
interface CostModel {
|
|
1688
|
+
/** Cost per 1K input tokens, in the caller's currency unit. */
|
|
1689
|
+
inputPer1k: number;
|
|
1690
|
+
/** Cost per 1K output tokens. */
|
|
1691
|
+
outputPer1k: number;
|
|
1692
|
+
/** Optional discounted rate for provider-cached (prefix) input tokens. */
|
|
1693
|
+
cachedInputPer1k?: number;
|
|
1694
|
+
}
|
|
1695
|
+
/**
|
|
1696
|
+
* Why an avoided call (a cache or coalesced hit) could not be valued by
|
|
1697
|
+
* reference, and so stays `estimated`.
|
|
1698
|
+
*
|
|
1699
|
+
* - `absent` — the stored answer carries no provider counter.
|
|
1700
|
+
* - `malformed` — it carries one, but not in a readable shape.
|
|
1701
|
+
* - `witness` — the counter is not bound to the answer being served.
|
|
1702
|
+
* - `model-mismatch` — the counter belongs to another model.
|
|
1703
|
+
* - `expired` — the counter is older than one billing period.
|
|
1704
|
+
* - `different-request` — the request that ran was not byte-identical to this one.
|
|
1705
|
+
* - `quality` — the answer being served fails its output contract.
|
|
1706
|
+
* - `no-cached-price` — no documented cached-input rate for this exact model:
|
|
1707
|
+
* its own catalogue row (or a dated snapshot of it) or
|
|
1708
|
+
* `costModels`, never a row inherited from a sibling id.
|
|
1709
|
+
* - `no-credit` — nothing left to credit once the allowances are applied.
|
|
1710
|
+
*/
|
|
1711
|
+
type ReferenceRejection = "absent" | "malformed" | "witness" | "model-mismatch" | "expired" | "different-request" | "quality" | "no-cached-price" | "no-credit";
|
|
1712
|
+
/**
|
|
1713
|
+
* The provider counter an avoided call is valued against, and every figure
|
|
1714
|
+
* needed to recompute its baseline:
|
|
1715
|
+
* `tierFactor * (creditedInputTokens / 1000 * cachedInputPer1k + creditedOutputTokens / 1000 * outputPer1k)`.
|
|
1716
|
+
*/
|
|
1717
|
+
interface ReferenceCredit {
|
|
1718
|
+
/** Id of the report of the call that ran (its report.requestId). */
|
|
1719
|
+
requestId: string;
|
|
1720
|
+
/** Epoch ms when that call's provider counter was read. */
|
|
1721
|
+
at: number;
|
|
1722
|
+
/** Input tokens the provider billed for that call. */
|
|
1723
|
+
inputTokens: number;
|
|
1724
|
+
/** Output tokens the provider billed for that call (the smaller of the first answer and its repair). */
|
|
1725
|
+
outputTokens: number;
|
|
1726
|
+
/** Input tokens credited to this repeat: inputTokens minus the allowance for what the layer inserted (messages, the native schema wrapper, empty tool parameters). */
|
|
1727
|
+
creditedInputTokens: number;
|
|
1728
|
+
/**
|
|
1729
|
+
* Output tokens credited: a share of outputTokens capped by this request's
|
|
1730
|
+
* output limit. 0 unless this request asked for deterministic sampling
|
|
1731
|
+
* (`temperature: 0`), the model has a list price in the catalog, does not
|
|
1732
|
+
* reason and honors that setting, and the call that ran carried no hidden
|
|
1733
|
+
* reasoning (zero reasoning tokens reported, or no reasoning block where the
|
|
1734
|
+
* provider does not break them out).
|
|
1735
|
+
*/
|
|
1736
|
+
creditedOutputTokens: number;
|
|
1737
|
+
/** Rate applied to every credited input token, per 1K: the provider's cached-input rate. */
|
|
1738
|
+
cachedInputPer1k: number;
|
|
1739
|
+
/** Rate applied to credited output tokens, per 1K. */
|
|
1740
|
+
outputPer1k: number;
|
|
1741
|
+
/** Fraction of the standard price applied: 1, or the discounted tier the call ran on or this request declared. */
|
|
1742
|
+
tierFactor: number;
|
|
1743
|
+
}
|
|
1744
|
+
|
|
1745
|
+
/**
|
|
1746
|
+
* LIST PRICES PER MODEL.
|
|
1747
|
+
*
|
|
1748
|
+
* The layer reports savings in money, and money needs a price. This is the
|
|
1749
|
+
* built-in catalogue of published list prices, so the dollar figures mean
|
|
1750
|
+
* something without any configuration.
|
|
1751
|
+
*
|
|
1752
|
+
* ⚠️ A WRONG PRICE HERE IS WORSE THAN NO CATALOGUE. A model that is ABSENT falls
|
|
1753
|
+
* back to the illustrative default AND warns, so the customer knows that column
|
|
1754
|
+
* is not their invoice. A model that is PRESENT with the wrong price silences
|
|
1755
|
+
* that warning and hands over a confident, wrong number. So only models whose
|
|
1756
|
+
* published price is known with certainty belong here, and {@link PRECIOS_AL_DIA}
|
|
1757
|
+
* records when they were last checked.
|
|
1758
|
+
*
|
|
1759
|
+
* Prices are USD per 1,000 tokens — the unit of `CostModel` — even though
|
|
1760
|
+
* providers publish per million: 1M to 1K is a division by a thousand.
|
|
1761
|
+
*
|
|
1762
|
+
* `costModels` on the layer config ALWAYS wins over this. A customer with volume
|
|
1763
|
+
* discounts, negotiated rates or a different region sets their own and this
|
|
1764
|
+
* catalogue stays out of the way.
|
|
1765
|
+
*/
|
|
1766
|
+
/**
|
|
1767
|
+
* Version of this price catalogue: the UTC day its content last changed (rows,
|
|
1768
|
+
* announced price steps, cache-write premiums, tier rates). Two computations
|
|
1769
|
+
* citing the same version were priced with the same content.
|
|
1770
|
+
*/
|
|
1771
|
+
declare const PRECIOS_AL_DIA = "2026-11-03";
|
|
1772
|
+
/**
|
|
1773
|
+
* Key = the model id as the provider publishes it, without a vendor prefix.
|
|
1774
|
+
* Lookup tries the exact id, then the id without its provider prefix, then the
|
|
1775
|
+
* LONGEST matching prefix — so `gpt-4o-2024-08-06` inherits `gpt-4o` without
|
|
1776
|
+
* having to enumerate every dated snapshot.
|
|
1777
|
+
*
|
|
1778
|
+
* These are the rows in force on {@link PRECIOS_AL_DIA}. A price change a
|
|
1779
|
+
* provider has already announced is applied by {@link precioDeLista} from its
|
|
1780
|
+
* effective date, so a request is always priced at the rate of its own day.
|
|
1781
|
+
*/
|
|
1782
|
+
declare const CATALOGO: Readonly<Record<string, CostModel>>;
|
|
1783
|
+
/**
|
|
1784
|
+
* The list price for a model on the day of the request, or `undefined` when
|
|
1785
|
+
* the catalogue has no price for it in force that day.
|
|
1786
|
+
*
|
|
1787
|
+
* @param model the id as the caller passes it, with or without a vendor prefix.
|
|
1788
|
+
* @param at when the request was made (epoch ms or a `Date`); defaults to now.
|
|
1789
|
+
* Prices resolve per UTC day, so a price change the provider has announced
|
|
1790
|
+
* applies from its effective date.
|
|
1791
|
+
*/
|
|
1792
|
+
declare function precioDeLista(model: string, at?: Date | number): CostModel | undefined;
|
|
1793
|
+
|
|
1794
|
+
/**
|
|
1795
|
+
* The version the SDK reports for itself.
|
|
1796
|
+
*
|
|
1797
|
+
* Written by hand and deliberately NOT read from package.json: the published
|
|
1798
|
+
* bundle does not ship package.json, so a `require` here would fail at runtime
|
|
1799
|
+
* on the customer's machine. `test/version-declarada.test.ts` ties this
|
|
1800
|
+
* constant to package.json so the two can never drift.
|
|
1801
|
+
*/
|
|
1802
|
+
declare const BV_SALA_VERSION = "1.0.0";
|
|
1803
|
+
|
|
1751
1804
|
/**
|
|
1752
1805
|
* Typed intermediate representation for LLM -> LLM handoffs.
|
|
1753
1806
|
*
|
|
@@ -1984,8 +2037,8 @@ declare class CalibrationStateError extends Error {
|
|
|
1984
2037
|
*/
|
|
1985
2038
|
type AnswerJudge = (stored: CachedAnswer, fresh: CachedAnswer) => boolean | Promise<boolean>;
|
|
1986
2039
|
/**
|
|
1987
|
-
*
|
|
1988
|
-
*
|
|
2040
|
+
* Calibration required for semantic reuse: the cache serves a reworded request
|
|
2041
|
+
* only at or above a floor that labeled
|
|
1989
2042
|
* traffic has certified, so that at most `maxErrorRate` of the reuses served
|
|
1990
2043
|
* from that floor up are wrong, at the stated `confidence`.
|
|
1991
2044
|
*
|
|
@@ -2056,6 +2109,8 @@ interface SemanticCalibrationState {
|
|
|
2056
2109
|
verifier: string | null;
|
|
2057
2110
|
/** `built_in:<version>` for {@link sameAnswer}, `custom:<judgeId>` for yours. */
|
|
2058
2111
|
judge: string;
|
|
2112
|
+
/** Storage backend whose observations produced this state; absent in states written before backend identities. */
|
|
2113
|
+
backend?: string;
|
|
2059
2114
|
config: {
|
|
2060
2115
|
maxErrorRate: number;
|
|
2061
2116
|
confidence: number;
|
|
@@ -2140,7 +2195,8 @@ declare function calibrateSemanticFloor(observations: readonly CalibrationObserv
|
|
|
2140
2195
|
* question. Therefore the headline metric here is NOT hit ratio — it is the
|
|
2141
2196
|
* PRECISION of hits. Every candidate must pass: tenant/permission filter,
|
|
2142
2197
|
* knowledge-epoch filter, a similarity threshold, a stronger local verifier, a
|
|
2143
|
-
* freshness check,
|
|
2198
|
+
* freshness check, an output-contract compatibility check, and a certified
|
|
2199
|
+
* calibration floor. Without calibration the cache is shadow-only and serves nothing.
|
|
2144
2200
|
*/
|
|
2145
2201
|
interface EmbeddingFn {
|
|
2146
2202
|
(text: string): Promise<number[]> | number[];
|
|
@@ -2154,6 +2210,8 @@ interface SemanticVerifier {
|
|
|
2154
2210
|
(query: string, candidateQuery: string): Promise<number> | number;
|
|
2155
2211
|
}
|
|
2156
2212
|
interface SemanticRecord {
|
|
2213
|
+
/** Stable backend identifier, when the store assigns one. */
|
|
2214
|
+
recordId?: string;
|
|
2157
2215
|
embedding: number[];
|
|
2158
2216
|
query: string;
|
|
2159
2217
|
answer: CachedAnswer;
|
|
@@ -2240,6 +2298,8 @@ interface SemanticHit {
|
|
|
2240
2298
|
*/
|
|
2241
2299
|
interface SemanticProbe {
|
|
2242
2300
|
reason: "calibrating" | "below_floor" | "record_floor" | "audit";
|
|
2301
|
+
/** Stable backend identifier for this observation, when the store assigns one. */
|
|
2302
|
+
observationId?: string;
|
|
2243
2303
|
/** The score the decision was made on: the lower of similarity and your verifier's score. */
|
|
2244
2304
|
score: number;
|
|
2245
2305
|
segment: "single_shot" | "agentic";
|
|
@@ -2250,6 +2310,15 @@ interface SemanticLookup {
|
|
|
2250
2310
|
hit: SemanticHit | null;
|
|
2251
2311
|
probe: SemanticProbe | null;
|
|
2252
2312
|
}
|
|
2313
|
+
/** Structural contract required by the pipeline; backends may add persistence and lifecycle methods. */
|
|
2314
|
+
interface SemanticCacheStore {
|
|
2315
|
+
readonly calibrated: boolean;
|
|
2316
|
+
lookup(c: CanonicalRequest, opts?: {
|
|
2317
|
+
agentic?: boolean;
|
|
2318
|
+
}): Promise<SemanticLookup>;
|
|
2319
|
+
store(c: CanonicalRequest, answer: CachedAnswer): Promise<void>;
|
|
2320
|
+
observe(probe: SemanticProbe, fresh: CachedAnswer): Promise<boolean | null>;
|
|
2321
|
+
}
|
|
2253
2322
|
interface SemanticCacheOptions {
|
|
2254
2323
|
/**
|
|
2255
2324
|
* How text becomes a vector. OPTIONAL: omit it and {@link localEmbedding} is
|
|
@@ -2283,15 +2352,16 @@ interface SemanticCacheOptions {
|
|
|
2283
2352
|
verifierId?: string;
|
|
2284
2353
|
/** Cosine similarity floor for a candidate to be considered at all. */
|
|
2285
2354
|
similarityThreshold?: number;
|
|
2286
|
-
/**
|
|
2355
|
+
/** Explicit verifier score floor applied in addition to the certified floor. */
|
|
2287
2356
|
acceptThreshold?: number;
|
|
2288
2357
|
ttlMs?: number;
|
|
2289
2358
|
maxRecords?: number;
|
|
2290
2359
|
now?: () => number;
|
|
2291
2360
|
/**
|
|
2292
|
-
*
|
|
2293
|
-
*
|
|
2294
|
-
* {@link
|
|
2361
|
+
* Required to serve semantic hits: your own labeled traffic certifies a floor
|
|
2362
|
+
* for a wrong-reuse rate you choose. Without it the cache is shadow-only and
|
|
2363
|
+
* may store answers, but {@link find} and {@link lookup} never return a hit.
|
|
2364
|
+
* See {@link SemanticCalibrationOptions}.
|
|
2295
2365
|
*/
|
|
2296
2366
|
calibration?: SemanticCalibrationOptions;
|
|
2297
2367
|
}
|
|
@@ -2301,7 +2371,7 @@ interface SemanticCacheOptions {
|
|
|
2301
2371
|
* to the store. The verification logic — the part that makes it *safe* — stays
|
|
2302
2372
|
* identical regardless of the backend.
|
|
2303
2373
|
*/
|
|
2304
|
-
declare class InMemorySemanticCache {
|
|
2374
|
+
declare class InMemorySemanticCache implements SemanticCacheStore {
|
|
2305
2375
|
private readonly records;
|
|
2306
2376
|
private readonly embed;
|
|
2307
2377
|
private readonly verify;
|
|
@@ -2312,7 +2382,7 @@ declare class InMemorySemanticCache {
|
|
|
2312
2382
|
private readonly now;
|
|
2313
2383
|
private readonly embebedorPropio;
|
|
2314
2384
|
private readonly calibracion;
|
|
2315
|
-
private readonly
|
|
2385
|
+
private readonly diario;
|
|
2316
2386
|
private readonly nombres;
|
|
2317
2387
|
private readonly pendientes;
|
|
2318
2388
|
private readonly aceptacionExplicita;
|
|
@@ -2324,21 +2394,22 @@ declare class InMemorySemanticCache {
|
|
|
2324
2394
|
* FIRST and hard — a candidate from another tenant, scope, or knowledge epoch
|
|
2325
2395
|
* is never even scored.
|
|
2326
2396
|
*
|
|
2327
|
-
*
|
|
2328
|
-
*
|
|
2329
|
-
*
|
|
2397
|
+
* Only a certified calibration floor can authorize a hit. Without calibration
|
|
2398
|
+
* this always returns null, including for direct callers. It labels nothing:
|
|
2399
|
+
* without {@link lookup} and {@link observe} (the layer calls both) the
|
|
2400
|
+
* calibration never gets a label and never certifies.
|
|
2330
2401
|
*/
|
|
2331
2402
|
find(c: CanonicalRequest): Promise<SemanticHit | null>;
|
|
2332
2403
|
/**
|
|
2333
2404
|
* The best candidate that passes every hard filter, the similarity floor and
|
|
2334
|
-
* the fact guard.
|
|
2335
|
-
* without calibration, unchanged); without it, calibration decides.
|
|
2405
|
+
* the fact guard. Calibration decides whether it can be served.
|
|
2336
2406
|
*/
|
|
2337
2407
|
private mejorCandidata;
|
|
2338
2408
|
/**
|
|
2339
2409
|
* Like {@link find}, and with calibration on it also returns the candidate it
|
|
2340
2410
|
* did not serve, so the caller can label it with {@link observe} once the
|
|
2341
|
-
* provider has answered. Without calibration, `probe`
|
|
2411
|
+
* provider has answered. Without calibration, both `hit` and `probe` are null:
|
|
2412
|
+
* stored answers remain available for future calibration but are never served.
|
|
2342
2413
|
*
|
|
2343
2414
|
* `agentic` picks the traffic segment: calibrations are kept apart for
|
|
2344
2415
|
* single-shot and agentic requests.
|
|
@@ -2390,7 +2461,6 @@ declare class InMemorySemanticCache {
|
|
|
2390
2461
|
*/
|
|
2391
2462
|
importCalibration(state: SemanticCalibrationState | string | unknown): void;
|
|
2392
2463
|
private identidad;
|
|
2393
|
-
private estadoDe;
|
|
2394
2464
|
store(c: CanonicalRequest, answer: CachedAnswer): Promise<void>;
|
|
2395
2465
|
/** Drop every record for a knowledge epoch (invalidation on re-index). */
|
|
2396
2466
|
invalidateEpoch(knowledgeEpoch: string): number;
|
|
@@ -2777,6 +2847,12 @@ interface ConstraintPlan {
|
|
|
2777
2847
|
/** A permissive backend used when the provider validates structured output natively. */
|
|
2778
2848
|
declare function providerNativeBackend(isSupported?: (model: string, schema: JsonSchema) => boolean): ConstraintBackend;
|
|
2779
2849
|
|
|
2850
|
+
/** Durable, asynchronous memory for value constraints learned from prior responses. */
|
|
2851
|
+
interface ConstraintMemoryStore {
|
|
2852
|
+
has(key: string): Promise<boolean>;
|
|
2853
|
+
remember(key: string): Promise<void>;
|
|
2854
|
+
}
|
|
2855
|
+
|
|
2780
2856
|
/**
|
|
2781
2857
|
* Context compression, as an OPTIONAL sidecar.
|
|
2782
2858
|
*
|
|
@@ -2864,6 +2940,17 @@ interface ToolOutputCompactionSettings {
|
|
|
2864
2940
|
* would not match the file on disk.
|
|
2865
2941
|
*/
|
|
2866
2942
|
compactFileViews?: boolean;
|
|
2943
|
+
/**
|
|
2944
|
+
* `"lossless-data"`: run only the JSON transformation, under a strict check.
|
|
2945
|
+
* The output must be strict JSON (RFC 8259) with no duplicate key at any
|
|
2946
|
+
* depth, and the compacted text must have exactly the same lexemes as the
|
|
2947
|
+
* original — same strings with the same escapes, same number literals, same
|
|
2948
|
+
* key order — with only insignificant whitespace removed. It is applied only
|
|
2949
|
+
* when it is also strictly smaller in estimated tokens. `transforms` is
|
|
2950
|
+
* ignored. Every tool output gets an explained decision. Default: unset (the
|
|
2951
|
+
* four transformations, as before).
|
|
2952
|
+
*/
|
|
2953
|
+
profile?: "lossless-data";
|
|
2867
2954
|
}
|
|
2868
2955
|
/** The outcome of compacting one tool output. */
|
|
2869
2956
|
interface ToolOutputCompaction {
|
|
@@ -2876,7 +2963,39 @@ interface ToolOutputCompaction {
|
|
|
2876
2963
|
compactChars: number;
|
|
2877
2964
|
/** Why the output was, or was not, compacted. */
|
|
2878
2965
|
reason: string;
|
|
2966
|
+
/**
|
|
2967
|
+
* Only with `profile: "lossless-data"`: the explained decision, with a stable
|
|
2968
|
+
* reason code. Token counts are estimates from the estimator in use, never a
|
|
2969
|
+
* verified saving.
|
|
2970
|
+
*/
|
|
2971
|
+
decision?: ToolOutputDecision;
|
|
2879
2972
|
}
|
|
2973
|
+
/**
|
|
2974
|
+
* Why a `lossless-data` decision went the way it did. `applied`: whitespace
|
|
2975
|
+
* removed. Bypass codes: `not_json` (the output is not a JSON object or array),
|
|
2976
|
+
* `invalid_json` (it looks like one but is not strict JSON), `duplicate_keys`,
|
|
2977
|
+
* `invalid_unicode` (an unpaired surrogate), `not_smaller` (no fewer tokens),
|
|
2978
|
+
* `below_threshold` (fewer than `minSavedChars` removed), `protected` (a file
|
|
2979
|
+
* view or an excepted tool).
|
|
2980
|
+
*/
|
|
2981
|
+
type ToolOutputDecisionReason = "applied" | "not_json" | "invalid_json" | "duplicate_keys" | "invalid_unicode" | "not_smaller" | "below_threshold" | "protected";
|
|
2982
|
+
/** One explained decision of the `lossless-data` profile. */
|
|
2983
|
+
interface ToolOutputDecision {
|
|
2984
|
+
status: "applied" | "bypassed";
|
|
2985
|
+
reason: ToolOutputDecisionReason;
|
|
2986
|
+
transformId: typeof LOSSLESS_JSON_TRANSFORM_ID;
|
|
2987
|
+
version: typeof LOSSLESS_JSON_VERSION;
|
|
2988
|
+
/** Estimated tokens of the original output. */
|
|
2989
|
+
tokensBefore: number;
|
|
2990
|
+
/** Estimated tokens of what is sent: the compacted text when applied, the original otherwise. */
|
|
2991
|
+
tokensAfter: number;
|
|
2992
|
+
/** Estimated tokens of the compacted candidate, when one was built (also when bypassed). */
|
|
2993
|
+
candidateTokens?: number;
|
|
2994
|
+
}
|
|
2995
|
+
/** Id of the `lossless-data` transformation, as it appears in a decision. */
|
|
2996
|
+
declare const LOSSLESS_JSON_TRANSFORM_ID = "json-lossless";
|
|
2997
|
+
/** Version of that transformation. A change of output for the same input bumps it. */
|
|
2998
|
+
declare const LOSSLESS_JSON_VERSION = "1";
|
|
2880
2999
|
/** Default minimum saving, in characters, for an output to be compacted. */
|
|
2881
3000
|
declare const DEFAULT_MIN_SAVED_CHARS = 256;
|
|
2882
3001
|
/**
|
|
@@ -2892,8 +3011,11 @@ declare const DEFAULT_MIN_SAVED_CHARS = 256;
|
|
|
2892
3011
|
* is produced, before you append it to your conversation history, and the
|
|
2893
3012
|
* history itself holds the compacted text — the provider's prefix cache is
|
|
2894
3013
|
* never disturbed.
|
|
3014
|
+
*
|
|
3015
|
+
* `countTokens` is used only by the `lossless-data` profile, for its token
|
|
3016
|
+
* gate; without it the built-in heuristic estimator counts.
|
|
2895
3017
|
*/
|
|
2896
|
-
declare function compactToolOutput(text: string, settings?: ToolOutputCompactionSettings): ToolOutputCompaction;
|
|
3018
|
+
declare function compactToolOutput(text: string, settings?: ToolOutputCompactionSettings, countTokens?: (text: string) => number): ToolOutputCompaction;
|
|
2897
3019
|
|
|
2898
3020
|
/** What the layer needs to know about a provider's prefix cache to price a decision. */
|
|
2899
3021
|
interface CacheTier {
|
|
@@ -3351,6 +3473,12 @@ interface SavingsLayerConfig {
|
|
|
3351
3473
|
* Solo guarda identificadores de contrato, nunca contenido de peticiones.
|
|
3352
3474
|
*/
|
|
3353
3475
|
constraintMemory?: Set<string>;
|
|
3476
|
+
/**
|
|
3477
|
+
* Optional durable backend. The local Set remains a write-through cache, so
|
|
3478
|
+
* both options compose. Durable keys also include tenant, scope, provider and
|
|
3479
|
+
* served model; store failures never repeat a provider call.
|
|
3480
|
+
*/
|
|
3481
|
+
constraintMemoryStore?: ConstraintMemoryStore;
|
|
3354
3482
|
provider: Provider;
|
|
3355
3483
|
tokenizer: TokenizerRegistry;
|
|
3356
3484
|
exactCache?: ExactCache;
|
|
@@ -3366,7 +3494,7 @@ interface SavingsLayerConfig {
|
|
|
3366
3494
|
* the per-request switch (`savings.semanticCache`) no longer share a name.
|
|
3367
3495
|
* See the 0.5 migration guide.
|
|
3368
3496
|
*/
|
|
3369
|
-
semanticCacheStore?:
|
|
3497
|
+
semanticCacheStore?: SemanticCacheStore;
|
|
3370
3498
|
/**
|
|
3371
3499
|
* Default for `savings.semanticCache` on every request of this layer. A
|
|
3372
3500
|
* request's own value always wins, `false` included. Unset, the semantic cache
|
|
@@ -4121,6 +4249,35 @@ declare function localEmbedding(text: string): number[];
|
|
|
4121
4249
|
*/
|
|
4122
4250
|
declare const LOCAL_EMBEDDING_THRESHOLD = 0.72;
|
|
4123
4251
|
|
|
4252
|
+
interface SemanticCalibrationIdentity {
|
|
4253
|
+
embedder: string;
|
|
4254
|
+
verifier: string | null;
|
|
4255
|
+
judge: string;
|
|
4256
|
+
maxErrorRate: number;
|
|
4257
|
+
confidence: number;
|
|
4258
|
+
similarityThreshold: number;
|
|
4259
|
+
backend?: string;
|
|
4260
|
+
}
|
|
4261
|
+
/** Pure, shared online calibration state, without storage or I/O. */
|
|
4262
|
+
declare class SemanticCalibrationJournal {
|
|
4263
|
+
private readonly identity;
|
|
4264
|
+
private readonly estados;
|
|
4265
|
+
constructor(identity: SemanticCalibrationIdentity);
|
|
4266
|
+
floorIndex(segment: SemanticProbe["segment"]): number | null;
|
|
4267
|
+
record(segment: SemanticProbe["segment"], score: number, correct: boolean): void;
|
|
4268
|
+
status(segment: SemanticProbe["segment"]): SemanticCalibrationStatus;
|
|
4269
|
+
export(): SemanticCalibrationState;
|
|
4270
|
+
import(state: unknown): void;
|
|
4271
|
+
private estadoDe;
|
|
4272
|
+
}
|
|
4273
|
+
|
|
4274
|
+
/** Shared deterministic safety operations for semantic-cache backends. */
|
|
4275
|
+
declare class SemanticCacheSafety {
|
|
4276
|
+
static cosine(a: number[], b: number[]): number;
|
|
4277
|
+
static keyTokensMatch(query: string, candidateQuery: string): boolean;
|
|
4278
|
+
static sameAnswer(stored: CachedAnswer, fresh: CachedAnswer): boolean;
|
|
4279
|
+
}
|
|
4280
|
+
|
|
4124
4281
|
/**
|
|
4125
4282
|
* Conversational memory by deltas.
|
|
4126
4283
|
*
|
|
@@ -4650,7 +4807,7 @@ declare class LicenseError extends Error {
|
|
|
4650
4807
|
|
|
4651
4808
|
/** The public handle returned by {@link createSavingsLayer}. */
|
|
4652
4809
|
interface SavingsLayer {
|
|
4653
|
-
generate<T = unknown>(request: SavingsRequest): Promise<SavingsResult<T>>;
|
|
4810
|
+
generate<T = unknown>(request: SavingsRequest, options?: SavingsRequestOptions): Promise<SavingsResult<T>>;
|
|
4654
4811
|
/**
|
|
4655
4812
|
* Same request as generate(), delivered as a stream of events.
|
|
4656
4813
|
*
|
|
@@ -4682,13 +4839,13 @@ interface SavingsLayer {
|
|
|
4682
4839
|
*/
|
|
4683
4840
|
readonly batches: SavingsBatches;
|
|
4684
4841
|
/**
|
|
4685
|
-
* Stop
|
|
4686
|
-
*
|
|
4687
|
-
*
|
|
4688
|
-
*
|
|
4689
|
-
*
|
|
4842
|
+
* Stop background work, drain usage batches and await a semantic store's
|
|
4843
|
+
* optional `settled()` method last. Backends should handle their own expected
|
|
4844
|
+
* observation failures; an unexpected `settled()` rejection is propagated
|
|
4845
|
+
* after the other components have closed rather than silently discarded.
|
|
4846
|
+
* Calling close() without await preserves fire-and-forget shutdown behavior.
|
|
4690
4847
|
*/
|
|
4691
|
-
close(): void
|
|
4848
|
+
close(): void | Promise<void>;
|
|
4692
4849
|
}
|
|
4693
4850
|
/**
|
|
4694
4851
|
* Create a savings layer. The only hard requirement is a `provider`; a default
|
|
@@ -4697,14 +4854,16 @@ interface SavingsLayer {
|
|
|
4697
4854
|
* cache, a vector-backed semantic cache, a real tokenizer, a compressor sidecar,
|
|
4698
4855
|
* XGrammar/llguidance backends, etc.).
|
|
4699
4856
|
*
|
|
4700
|
-
*
|
|
4701
|
-
*
|
|
4702
|
-
*
|
|
4857
|
+
* Open core: no `licenseKey` is needed. Without one the layer runs on the
|
|
4858
|
+
* COMMUNITY gate — every local saving lever on, zero telemetry, zero network.
|
|
4859
|
+
* A `licenseKey` (from savings.bivelio.com) turns on the paid surfaces: cloud
|
|
4860
|
+
* metering, license renewal/revocation and org entitlements. An INVALID or
|
|
4861
|
+
* EXPIRED key still fails closed with {@link LicenseError}; only its absence is
|
|
4862
|
+
* a working product.
|
|
4703
4863
|
*
|
|
4704
4864
|
* @example
|
|
4705
|
-
* //
|
|
4706
|
-
*
|
|
4707
|
-
* const layer = createSavingsLayer({ provider, licenseKey });
|
|
4865
|
+
* // No license: full savings, no telemetry.
|
|
4866
|
+
* const layer = createSavingsLayer({ provider });
|
|
4708
4867
|
* const { text, report } = await layer.generate({
|
|
4709
4868
|
* model: "openai/gpt-4o-mini",
|
|
4710
4869
|
* messages: [{ role: "user", content: "What is 2 + 2 * 3?" }],
|
|
@@ -4712,6 +4871,17 @@ interface SavingsLayer {
|
|
|
4712
4871
|
*/
|
|
4713
4872
|
declare function createSavingsLayer(configPublica: Omit<SavingsLayerConfig, "tokenizer"> & Partial<Pick<SavingsLayerConfig, "tokenizer">>): SavingsLayer;
|
|
4714
4873
|
|
|
4874
|
+
/**
|
|
4875
|
+
* The total request deadline expired. No partial result or zero-cost report is
|
|
4876
|
+
* returned: tokens already sent to the provider may still appear on its invoice.
|
|
4877
|
+
*/
|
|
4878
|
+
declare class RequestTimeoutError extends Error {
|
|
4879
|
+
readonly code = "REQUEST_TIMEOUT";
|
|
4880
|
+
/** True when at least one provider request had already been sent. */
|
|
4881
|
+
readonly requestSent: boolean;
|
|
4882
|
+
readonly timeoutMs: number;
|
|
4883
|
+
constructor(requestSent: boolean, timeoutMs: number);
|
|
4884
|
+
}
|
|
4715
4885
|
/**
|
|
4716
4886
|
* Rejection of `SavingsStream.result` when the stream was aborted before the
|
|
4717
4887
|
* provider finished its answer. Nothing was cached and no usage was recorded.
|
|
@@ -5140,4 +5310,4 @@ declare const listPrice: typeof precioDeLista;
|
|
|
5140
5310
|
/** When the catalogue prices were last checked against the providers' public rates (YYYY-MM-DD). */
|
|
5141
5311
|
declare const PRICES_AS_OF: typeof PRECIOS_AL_DIA;
|
|
5142
5312
|
|
|
5143
|
-
export { type AnswerJudge, type AnswerSource, type AnthropicOptions, AnthropicProvider, BV_SALA_VERSION, type BatchCounts, type BatchItemResult, type BatchJob, type BatchJobStatus, type BatchRequest, type BatchResults, BatchSubmitError, type BatchSubmitOptions, type BatchSummary, type BivelioHandoff, COMPACTION_TRANSFORMS, COMPACT_ROWS_INSTRUCTION, type CacheBreakpointStore, type CacheOutcome, type CachePlacement, type CacheTier, type CacheWriteReport, type CachedAnswer, type CalibrationObservation, CalibrationStateError, type ChatMessage, type CompactRowsDecodeOptions, type CompactRowsOptions, type CompactionTransform, type CompressionEconomics, type CompressionPolicyInput, type CompressionRequest, type CompressionResult, type Compressor, type ConstraintBackend, type ConstraintBackendId, type ConstraintPlan, type ConversationState, type CostBasis, type CostModel, type CsvValue, DEFAULT_COST_MODEL, DEFAULT_MIN_SAVED_CHARS, type DeferredExecutionReport, type DeterministicResolver, type EmbeddingFn, type EvidenceDoc, type ExactCache, ExactTokenizer, type ExecutionClass, ExecutionRouter, type FocusExportOptions, type FocusRow, type FormatChoice, FormatSelector, type FormatSelectorOptions, type GoogleOptions, GoogleProvider, type GoogleThinkingConfig, type HandoffFact, HeuristicEstimator, InMemoryCacheBreakpointStore, InMemoryExactCache, InMemorySemanticCache, InMemoryToolSetFreezer, type JsonSchema, LOCAL_EMBEDDING_THRESHOLD, type LangfuseGenerationUpdate, LicenseError, type LicenseGate, type LicenseKeyset, type LocalClassifier, MODELS_BY_ENCODING, type MeasureReport, type MemoryDelta, type MessageRole, MockProvider, type MockProviderOptions, type ModelCascadeMode, type ModelCascadeReport, type ModelCascadeSettings, type NativeSavingsReport, type OpenAICompatibleOptions, OpenAICompatibleProvider, type OutputContract, PRICES_AS_OF, PRICE_CATALOG, type Policy, type PriceLeverSaving, type PriceLeverSettings, type ProtectedSpan, type Provider, type ProviderRequest, type ProviderResponse, type ProviderStreamEvent, type ProviderStreamOptions, type ProviderToolSearch, type ProviderToolSearchUsage, type ProviderUsage, QUALITY_STATUSES, type QualityCacheMatch, type QualityCascadeCheck, type QualityLedger, type QualityRepairCheck, type QualitySchemaCheck, type QualityStatus, type QualityTruncationCheck, type ReasoningEffort, type ReferenceCredit, type ReferenceRejection, type RequestContext, type Reranker, ResolverRegistry, type ResolverResult, type RetrievalOptions, type RetrievalOutcome, type RetrievalPlan, type Retriever, type RouteDecision, type SavingsAdapterLevers, type SavingsAdapterOptions, type SavingsAttributeOptions, type SavingsBatches, type SavingsCallOptions, type SavingsLayer, type SavingsLayerConfig, type SavingsRecord, type SavingsReport, type SavingsReportForExport, type SavingsRequest, type SavingsResult, type SavingsSeriesOptions, type SavingsSeriesRow, type SavingsSpanAttributes, type SavingsSpanOptions, type SavingsStream, type SavingsStreamEvent, type SavingsStreamOptions, type SavingsToggles, type SavingsTracingOptions, SchemaRegistry, type SemanticCacheOptions, type SemanticCalibrationBound, type SemanticCalibrationOptions, type SemanticCalibrationSegmentState, type SemanticCalibrationState, type SemanticCalibrationStatus, type SemanticHit, type SemanticLookup, type SemanticProbe, type SemanticRecord, type SemanticVerifier, type SerializationFormat, type Serializer, Singleflight, type SpanAttributeValue, type SpanLike, type SpanStartOptionsLike, StreamAbortedError, type StreamingReport, type TokenEstimator, TokenizerRegistry, type ToolCallRequest, type ToolDescriptor, type ToolOutputCompaction, type ToolOutputCompactionSettings, type ToolRisk, type ToolSearchMode, type ToolSearchOptions, type ToolSelection, ToolSelector, type ToolSelectorOptions, type ToolSetFreezer, type TraceEvent, type TracerLike, type TracerProviderLike, type ValidationError, type ValidationResult, type VerificationCheck, type VerificationExclusion, type VerificationOutcome, type VerificationReport, type VerificationSettings, type VerificationSide, type VerificationStratum, arithmeticResolver, calibrateSemanticFloor, compactRowsSerializer, compactToolOutput, createCompactRowsSerializer, createDefaultResolvers, createHandoff, createMockProvider, createSavingsLayer, csvSerializer, decodeCompactRows, defaultTokenizerRegistry, emptyState, encodeCompactRows, heuristicEstimator, instrumentSavingsLayer, jsonCompactSerializer, listPrice, localEmbedding, mergeDelta, noopCompressor, providerNativeBackend, recordSavingsOnSpan, registerExactTokenizer, renderState, savingsSpanAttributes, toCsv, toFocusRows, toLangfuseGeneration, toSavingsSeries, validateAgainstSchema };
|
|
5313
|
+
export { type AnswerJudge, type AnswerSource, type AnthropicOptions, AnthropicProvider, BV_SALA_VERSION, type BatchCounts, type BatchItemResult, type BatchJob, type BatchJobStatus, type BatchRequest, type BatchResults, BatchSubmitError, type BatchSubmitOptions, type BatchSummary, type BivelioHandoff, COMPACTION_TRANSFORMS, COMPACT_ROWS_INSTRUCTION, type CacheBreakpointStore, type CacheOutcome, type CachePlacement, type CacheTier, type CacheWriteReport, type CachedAnswer, type CalibrationObservation, CalibrationStateError, type ChatMessage, type CompactRowsDecodeOptions, type CompactRowsOptions, type CompactionTransform, type CompressionEconomics, type CompressionPolicyInput, type CompressionRequest, type CompressionResult, type Compressor, type ConstraintBackend, type ConstraintBackendId, type ConstraintMemoryStore, type ConstraintPlan, type ConversationState, type CostBasis, type CostModel, type CsvValue, DEFAULT_COST_MODEL, DEFAULT_MIN_SAVED_CHARS, type DeferredExecutionReport, type DeterministicResolver, type EmbeddingFn, type EvidenceDoc, type ExactCache, ExactTokenizer, type ExecutionClass, ExecutionRouter, type FocusExportOptions, type FocusRow, type FormatChoice, FormatSelector, type FormatSelectorOptions, type GoogleOptions, GoogleProvider, type GoogleThinkingConfig, type HandoffFact, HeuristicEstimator, InMemoryCacheBreakpointStore, InMemoryExactCache, InMemorySemanticCache, InMemoryToolSetFreezer, type JsonSchema, LOCAL_EMBEDDING_THRESHOLD, type LangfuseGenerationUpdate, LicenseError, type LicenseGate, type LicenseKeyset, type LocalClassifier, MODELS_BY_ENCODING, type MeasureReport, type MemoryDelta, type MessageRole, MockProvider, type MockProviderOptions, type ModelCascadeMode, type ModelCascadeReport, type ModelCascadeSettings, type NativeSavingsReport, type OpenAICompatibleOptions, OpenAICompatibleProvider, type OutputContract, PRICES_AS_OF, PRICE_CATALOG, type Policy, type PriceLeverSaving, type PriceLeverSettings, type ProtectedSpan, type Provider, type ProviderRequest, type ProviderResponse, type ProviderStreamEvent, type ProviderStreamOptions, type ProviderToolSearch, type ProviderToolSearchUsage, type ProviderUsage, QUALITY_STATUSES, type QualityCacheMatch, type QualityCascadeCheck, type QualityLedger, type QualityRepairCheck, type QualitySchemaCheck, type QualityStatus, type QualityTruncationCheck, type ReasoningEffort, type ReferenceCredit, type ReferenceRejection, type RequestContext, RequestTimeoutError, type Reranker, ResolverRegistry, type ResolverResult, type RetrievalOptions, type RetrievalOutcome, type RetrievalPlan, type Retriever, type RouteDecision, type SavingsAdapterLevers, type SavingsAdapterOptions, type SavingsAttributeOptions, type SavingsBatches, type SavingsCallOptions, type SavingsLayer, type SavingsLayerConfig, type SavingsRecord, type SavingsReport, type SavingsReportForExport, type SavingsRequest, type SavingsRequestOptions, type SavingsResult, type SavingsSeriesOptions, type SavingsSeriesRow, type SavingsSpanAttributes, type SavingsSpanOptions, type SavingsStream, type SavingsStreamEvent, type SavingsStreamOptions, type SavingsToggles, type SavingsTracingOptions, SchemaRegistry, type SemanticCacheOptions, SemanticCacheSafety, type SemanticCacheStore, type SemanticCalibrationBound, type SemanticCalibrationIdentity, SemanticCalibrationJournal, type SemanticCalibrationOptions, type SemanticCalibrationSegmentState, type SemanticCalibrationState, type SemanticCalibrationStatus, type SemanticHit, type SemanticLookup, type SemanticProbe, type SemanticRecord, type SemanticVerifier, type SerializationFormat, type Serializer, Singleflight, type SpanAttributeValue, type SpanLike, type SpanStartOptionsLike, StreamAbortedError, type StreamingReport, type TokenEstimator, TokenizerRegistry, type ToolCallRequest, type ToolDescriptor, type ToolOutputCompaction, type ToolOutputCompactionSettings, type ToolRisk, type ToolSearchMode, type ToolSearchOptions, type ToolSelection, ToolSelector, type ToolSelectorOptions, type ToolSetFreezer, type TraceEvent, type TracerLike, type TracerProviderLike, type ValidationError, type ValidationResult, type VerificationCheck, type VerificationExclusion, type VerificationOutcome, type VerificationReport, type VerificationSettings, type VerificationSide, type VerificationStratum, arithmeticResolver, calibrateSemanticFloor, compactRowsSerializer, compactToolOutput, createCompactRowsSerializer, createDefaultResolvers, createHandoff, createMockProvider, createSavingsLayer, csvSerializer, decodeCompactRows, defaultTokenizerRegistry, emptyState, encodeCompactRows, heuristicEstimator, instrumentSavingsLayer, jsonCompactSerializer, listPrice, localEmbedding, mergeDelta, noopCompressor, providerNativeBackend, recordSavingsOnSpan, registerExactTokenizer, renderState, savingsSpanAttributes, toCsv, toFocusRows, toLangfuseGeneration, toSavingsSeries, validateAgainstSchema };
|