@bivelio/savings-layer 0.5.2 → 1.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +138 -0
- package/LICENSE +183 -163
- package/LICENSE-CLOUD.md +174 -0
- package/NOTICE +11 -0
- package/README.md +228 -122
- package/dist/adapters/ai-sdk.js +78 -78
- package/dist/adapters/anthropic-sdk.js +88 -88
- package/dist/adapters/langchain.js +58 -58
- package/dist/adapters/mastra.js +70 -70
- package/dist/gateway/bin.js +1291 -1082
- package/dist/index.d.ts +393 -194
- package/dist/index.js +1704 -1612
- package/docs/terminales/README.md +39 -33
- package/docs/terminales/aider.md +1 -1
- package/docs/terminales/claude-code.md +2 -2
- package/docs/terminales/cline.md +1 -1
- package/docs/terminales/codex.md +3 -3
- package/docs/terminales/copilot.md +1 -1
- package/docs/terminales/crush.md +1 -1
- package/docs/terminales/cursor.md +32 -0
- package/docs/terminales/deepseek.md +28 -0
- package/docs/terminales/droid.md +1 -1
- package/docs/terminales/gemini.md +38 -0
- package/docs/terminales/goose.md +2 -2
- package/docs/terminales/kilo.md +2 -2
- package/docs/terminales/no-soportados.md +16 -27
- package/docs/terminales/ollama.md +19 -0
- package/docs/terminales/opencode.md +2 -2
- package/docs/terminales/qwen-code.md +2 -2
- package/package.json +50 -20
package/dist/index.d.ts
CHANGED
|
@@ -1,161 +1,3 @@
|
|
|
1
|
-
/**
|
|
2
|
-
* The four-ledger savings accounting.
|
|
3
|
-
*
|
|
4
|
-
* These ledgers are kept strictly separate on purpose: each answers a different
|
|
5
|
-
* question, and summing them would double-count.
|
|
6
|
-
* The ledger is a pure accumulator: the pipeline records events into it, and it
|
|
7
|
-
* produces the numbers that populate the SavingsReport. It never guesses — every
|
|
8
|
-
* increment is driven by an explicit pipeline decision.
|
|
9
|
-
*/
|
|
10
|
-
/**
|
|
11
|
-
* Where the baseline being compared against comes from.
|
|
12
|
-
*
|
|
13
|
-
* - `measured` — reconstructed from this request's provider counter. Real money.
|
|
14
|
-
* - `reference` — MEASURED BY REFERENCE: the call was avoided (a repeat served
|
|
15
|
-
* without calling the provider) and a byte-identical request had
|
|
16
|
-
* already run through the layer untouched. The baseline is the
|
|
17
|
-
* provider's own counter for that request, valued at the
|
|
18
|
-
* provider's cached-input rate. Output is credited only under
|
|
19
|
-
* deterministic sampling, on a model with a list price in the
|
|
20
|
-
* catalog that does not reason and honors that setting, when
|
|
21
|
-
* the call that ran carried no hidden reasoning (zero reasoning
|
|
22
|
-
* tokens reported, or no reasoning block where the provider
|
|
23
|
-
* does not break them out), and then at 95% of its counter.
|
|
24
|
-
* With an output schema, only when the schema traveled exactly
|
|
25
|
-
* as sent, in strict mode, in the provider's native
|
|
26
|
-
* response-format parameter. Valued low on purpose. Enters the
|
|
27
|
-
* commission base.
|
|
28
|
-
* - `estimated` — the call was AVOIDED: it never ran, so there is no provider
|
|
29
|
-
* counter and the baseline comes from the local estimator. Still
|
|
30
|
-
* the best figure available, but not measured.
|
|
31
|
-
* - `unmeasured` — the provider reported no usage. Nothing is claimed.
|
|
32
|
-
* - `shadow` — HYPOTHETICAL: what would have happened with a lever turned on
|
|
33
|
-
* (the shadow lane). Never enters the commission base, never
|
|
34
|
-
* added to measured or to any total; shadow reports are built
|
|
35
|
-
* separately and `estimateCost` must never produce this value.
|
|
36
|
-
*
|
|
37
|
-
* Summing `measured` and `estimated` into a single "money saved" figure is
|
|
38
|
-
* exactly what a verify-don't-trust product cannot do — and adding `shadow` to
|
|
39
|
-
* either would be making the savings up.
|
|
40
|
-
*/
|
|
41
|
-
type CostBasis = "measured" | "reference" | "estimated" | "unmeasured" | "shadow";
|
|
42
|
-
interface CostModel {
|
|
43
|
-
/** Cost per 1K input tokens, in the caller's currency unit. */
|
|
44
|
-
inputPer1k: number;
|
|
45
|
-
/** Cost per 1K output tokens. */
|
|
46
|
-
outputPer1k: number;
|
|
47
|
-
/** Optional discounted rate for provider-cached (prefix) input tokens. */
|
|
48
|
-
cachedInputPer1k?: number;
|
|
49
|
-
}
|
|
50
|
-
/**
|
|
51
|
-
* Why an avoided call (a cache or coalesced hit) could not be valued by
|
|
52
|
-
* reference, and so stays `estimated`.
|
|
53
|
-
*
|
|
54
|
-
* - `absent` — the stored answer carries no provider counter.
|
|
55
|
-
* - `malformed` — it carries one, but not in a readable shape.
|
|
56
|
-
* - `witness` — the counter is not bound to the answer being served.
|
|
57
|
-
* - `model-mismatch` — the counter belongs to another model.
|
|
58
|
-
* - `expired` — the counter is older than one billing period.
|
|
59
|
-
* - `different-request` — the request that ran was not byte-identical to this one.
|
|
60
|
-
* - `quality` — the answer being served fails its output contract.
|
|
61
|
-
* - `no-cached-price` — no documented cached-input rate for this exact model:
|
|
62
|
-
* its own catalogue row (or a dated snapshot of it) or
|
|
63
|
-
* `costModels`, never a row inherited from a sibling id.
|
|
64
|
-
* - `no-credit` — nothing left to credit once the allowances are applied.
|
|
65
|
-
*/
|
|
66
|
-
type ReferenceRejection = "absent" | "malformed" | "witness" | "model-mismatch" | "expired" | "different-request" | "quality" | "no-cached-price" | "no-credit";
|
|
67
|
-
/**
|
|
68
|
-
* The provider counter an avoided call is valued against, and every figure
|
|
69
|
-
* needed to recompute its baseline:
|
|
70
|
-
* `tierFactor * (creditedInputTokens / 1000 * cachedInputPer1k + creditedOutputTokens / 1000 * outputPer1k)`.
|
|
71
|
-
*/
|
|
72
|
-
interface ReferenceCredit {
|
|
73
|
-
/** Id of the report of the call that ran (its report.requestId). */
|
|
74
|
-
requestId: string;
|
|
75
|
-
/** Epoch ms when that call's provider counter was read. */
|
|
76
|
-
at: number;
|
|
77
|
-
/** Input tokens the provider billed for that call. */
|
|
78
|
-
inputTokens: number;
|
|
79
|
-
/** Output tokens the provider billed for that call (the smaller of the first answer and its repair). */
|
|
80
|
-
outputTokens: number;
|
|
81
|
-
/** Input tokens credited to this repeat: inputTokens minus the allowance for what the layer inserted (messages, the native schema wrapper, empty tool parameters). */
|
|
82
|
-
creditedInputTokens: number;
|
|
83
|
-
/**
|
|
84
|
-
* Output tokens credited: a share of outputTokens capped by this request's
|
|
85
|
-
* output limit. 0 unless this request asked for deterministic sampling
|
|
86
|
-
* (`temperature: 0`), the model has a list price in the catalog, does not
|
|
87
|
-
* reason and honors that setting, and the call that ran carried no hidden
|
|
88
|
-
* reasoning (zero reasoning tokens reported, or no reasoning block where the
|
|
89
|
-
* provider does not break them out).
|
|
90
|
-
*/
|
|
91
|
-
creditedOutputTokens: number;
|
|
92
|
-
/** Rate applied to every credited input token, per 1K: the provider's cached-input rate. */
|
|
93
|
-
cachedInputPer1k: number;
|
|
94
|
-
/** Rate applied to credited output tokens, per 1K. */
|
|
95
|
-
outputPer1k: number;
|
|
96
|
-
/** Fraction of the standard price applied: 1, or the discounted tier the call ran on or this request declared. */
|
|
97
|
-
tierFactor: number;
|
|
98
|
-
}
|
|
99
|
-
|
|
100
|
-
/**
|
|
101
|
-
* LIST PRICES PER MODEL.
|
|
102
|
-
*
|
|
103
|
-
* The layer reports savings in money, and money needs a price. This is the
|
|
104
|
-
* built-in catalogue of published list prices, so the dollar figures mean
|
|
105
|
-
* something without any configuration.
|
|
106
|
-
*
|
|
107
|
-
* ⚠️ A WRONG PRICE HERE IS WORSE THAN NO CATALOGUE. A model that is ABSENT falls
|
|
108
|
-
* back to the illustrative default AND warns, so the customer knows that column
|
|
109
|
-
* is not their invoice. A model that is PRESENT with the wrong price silences
|
|
110
|
-
* that warning and hands over a confident, wrong number. So only models whose
|
|
111
|
-
* published price is known with certainty belong here, and {@link PRECIOS_AL_DIA}
|
|
112
|
-
* records when they were last checked.
|
|
113
|
-
*
|
|
114
|
-
* Prices are USD per 1,000 tokens — the unit of `CostModel` — even though
|
|
115
|
-
* providers publish per million: 1M to 1K is a division by a thousand.
|
|
116
|
-
*
|
|
117
|
-
* `costModels` on the layer config ALWAYS wins over this. A customer with volume
|
|
118
|
-
* discounts, negotiated rates or a different region sets their own and this
|
|
119
|
-
* catalogue stays out of the way.
|
|
120
|
-
*/
|
|
121
|
-
/**
|
|
122
|
-
* Version of this price catalogue: the UTC day its content last changed (rows,
|
|
123
|
-
* announced price steps, cache-write premiums, tier rates). Two computations
|
|
124
|
-
* citing the same version were priced with the same content.
|
|
125
|
-
*/
|
|
126
|
-
declare const PRECIOS_AL_DIA = "2026-09-29";
|
|
127
|
-
/**
|
|
128
|
-
* Key = the model id as the provider publishes it, without a vendor prefix.
|
|
129
|
-
* Lookup tries the exact id, then the id without its provider prefix, then the
|
|
130
|
-
* LONGEST matching prefix — so `gpt-4o-2024-08-06` inherits `gpt-4o` without
|
|
131
|
-
* having to enumerate every dated snapshot.
|
|
132
|
-
*
|
|
133
|
-
* These are the rows in force on {@link PRECIOS_AL_DIA}. A price change a
|
|
134
|
-
* provider has already announced is applied by {@link precioDeLista} from its
|
|
135
|
-
* effective date, so a request is always priced at the rate of its own day.
|
|
136
|
-
*/
|
|
137
|
-
declare const CATALOGO: Readonly<Record<string, CostModel>>;
|
|
138
|
-
/**
|
|
139
|
-
* The list price for a model on the day of the request, or `undefined` when
|
|
140
|
-
* the catalogue has no price for it in force that day.
|
|
141
|
-
*
|
|
142
|
-
* @param model the id as the caller passes it, with or without a vendor prefix.
|
|
143
|
-
* @param at when the request was made (epoch ms or a `Date`); defaults to now.
|
|
144
|
-
* Prices resolve per UTC day, so a price change the provider has announced
|
|
145
|
-
* applies from its effective date.
|
|
146
|
-
*/
|
|
147
|
-
declare function precioDeLista(model: string, at?: Date | number): CostModel | undefined;
|
|
148
|
-
|
|
149
|
-
/**
|
|
150
|
-
* The version the SDK reports for itself.
|
|
151
|
-
*
|
|
152
|
-
* Written by hand and deliberately NOT read from package.json: the published
|
|
153
|
-
* bundle does not ship package.json, so a `require` here would fail at runtime
|
|
154
|
-
* on the customer's machine. `test/version-declarada.test.ts` ties this
|
|
155
|
-
* constant to package.json so the two can never drift.
|
|
156
|
-
*/
|
|
157
|
-
declare const BV_SALA_VERSION = "0.5.2";
|
|
158
|
-
|
|
159
1
|
/** How the layer served a request that is eligible for a verification check. */
|
|
160
2
|
type VerificationStratum =
|
|
161
3
|
/** Answered without calling the provider (cache, resolver, identical request in flight). */
|
|
@@ -1067,10 +909,28 @@ type SavingsStreamEvent<T = unknown> =
|
|
|
1067
909
|
type: "done";
|
|
1068
910
|
result: SavingsResult<T>;
|
|
1069
911
|
};
|
|
912
|
+
/** Optional cancellation and total-deadline controls for one request. */
|
|
913
|
+
interface SavingsRequestOptions {
|
|
914
|
+
/** Cancels this request without cancelling other users of the same layer. */
|
|
915
|
+
signal?: AbortSignal;
|
|
916
|
+
/**
|
|
917
|
+
* Total budget from the public entry point until the answer is delivered, in
|
|
918
|
+
* milliseconds. It must be a positive finite integer no greater than
|
|
919
|
+
* 2,147,483,647 (about 24.8 days), the largest portable `setTimeout` delay.
|
|
920
|
+
*
|
|
921
|
+
* Expiry aborts the provider signal and throws `RequestTimeoutError`. A custom
|
|
922
|
+
* provider that ignores that signal is awaited until its work stops; if it never
|
|
923
|
+
* stops, rejection is delayed indefinitely too. No late cache or learning write
|
|
924
|
+
* is started after the provider finally settles.
|
|
925
|
+
*/
|
|
926
|
+
requestTimeoutMs?: number;
|
|
927
|
+
}
|
|
1070
928
|
/** Options for `generateStream`. */
|
|
1071
|
-
interface SavingsStreamOptions {
|
|
929
|
+
interface SavingsStreamOptions extends SavingsRequestOptions {
|
|
1072
930
|
/** Aborting it has the same effect as calling `abort()` on the stream. */
|
|
1073
931
|
signal?: AbortSignal;
|
|
932
|
+
/** Total budget for this stream; see `SavingsRequestOptions.requestTimeoutMs`. */
|
|
933
|
+
requestTimeoutMs?: number;
|
|
1074
934
|
}
|
|
1075
935
|
/**
|
|
1076
936
|
* The handle returned by `generateStream`. Iterate it (once) for the events, or
|
|
@@ -1487,6 +1347,34 @@ interface SavingsReport {
|
|
|
1487
1347
|
protectedSpans: number;
|
|
1488
1348
|
fidelityPassed: boolean;
|
|
1489
1349
|
};
|
|
1350
|
+
/**
|
|
1351
|
+
* Present only when tool-output compaction ran with `profile:
|
|
1352
|
+
* "lossless-data"`: one explained decision per tool output in the request,
|
|
1353
|
+
* applied or bypassed, in message order. Token counts come from the
|
|
1354
|
+
* estimator named in `estimator` and are estimates, never a verified saving.
|
|
1355
|
+
*/
|
|
1356
|
+
toolOutputCompaction?: {
|
|
1357
|
+
profile: "lossless-data";
|
|
1358
|
+
/** Always `"estimated"`: these counts are not the provider's. */
|
|
1359
|
+
tokenBasis: "estimated";
|
|
1360
|
+
/** Id of the token estimator that counted, and whether it uses the provider's rule. */
|
|
1361
|
+
estimator: {
|
|
1362
|
+
id: string;
|
|
1363
|
+
exact: boolean;
|
|
1364
|
+
};
|
|
1365
|
+
decisions: Array<{
|
|
1366
|
+
/** The tool call this output answers, unchanged. Absent when the message had none. */
|
|
1367
|
+
toolCallId?: string;
|
|
1368
|
+
status: "applied" | "bypassed";
|
|
1369
|
+
reason: "applied" | "not_json" | "invalid_json" | "duplicate_keys" | "invalid_unicode" | "not_smaller" | "below_threshold" | "protected";
|
|
1370
|
+
transformId: string;
|
|
1371
|
+
version: string;
|
|
1372
|
+
tokensBefore: number;
|
|
1373
|
+
/** What is sent: the compacted text when applied, the original otherwise. */
|
|
1374
|
+
tokensAfter: number;
|
|
1375
|
+
candidateTokens?: number;
|
|
1376
|
+
}>;
|
|
1377
|
+
};
|
|
1490
1378
|
constraints: {
|
|
1491
1379
|
backend: string;
|
|
1492
1380
|
schemaVersion: string;
|
|
@@ -1569,6 +1457,16 @@ interface SavingsReport {
|
|
|
1569
1457
|
cost: {
|
|
1570
1458
|
baselineEstimate: number;
|
|
1571
1459
|
actual: number;
|
|
1460
|
+
/**
|
|
1461
|
+
* Present, and `true`, only when a billed call of this request reached the
|
|
1462
|
+
* layer without its full provider counter (a stream cut before its final
|
|
1463
|
+
* usage event, an endpoint that omitted a counter). `actual` then prices
|
|
1464
|
+
* only the counters that did arrive, with the missing ones at 0: it is a
|
|
1465
|
+
* FLOOR of what the provider billed, possibly 0, never the spend. The
|
|
1466
|
+
* request is also `basis: "unmeasured"`. Do not add such an `actual` to
|
|
1467
|
+
* measured spend. Absent whenever every counter arrived.
|
|
1468
|
+
*/
|
|
1469
|
+
actualIsLowerBound?: true;
|
|
1572
1470
|
/**
|
|
1573
1471
|
* The saving as a RATIO of the baseline: `1 - actual / baselineEstimate`,
|
|
1574
1472
|
* rounded to six decimals (0 when nothing was saved or there is no baseline,
|
|
@@ -1745,6 +1643,164 @@ interface TraceEvent {
|
|
|
1745
1643
|
atMs: number;
|
|
1746
1644
|
}
|
|
1747
1645
|
|
|
1646
|
+
/**
|
|
1647
|
+
* The four-ledger savings accounting.
|
|
1648
|
+
*
|
|
1649
|
+
* These ledgers are kept strictly separate on purpose: each answers a different
|
|
1650
|
+
* question, and summing them would double-count.
|
|
1651
|
+
* The ledger is a pure accumulator: the pipeline records events into it, and it
|
|
1652
|
+
* produces the numbers that populate the SavingsReport. It never guesses — every
|
|
1653
|
+
* increment is driven by an explicit pipeline decision.
|
|
1654
|
+
*/
|
|
1655
|
+
/**
|
|
1656
|
+
* Where the baseline being compared against comes from.
|
|
1657
|
+
*
|
|
1658
|
+
* - `measured` — reconstructed from this request's provider counter. Real money.
|
|
1659
|
+
* - `reference` — MEASURED BY REFERENCE: the call was avoided (a repeat served
|
|
1660
|
+
* without calling the provider) and a byte-identical request had
|
|
1661
|
+
* already run through the layer untouched. The baseline is the
|
|
1662
|
+
* provider's own counter for that request, valued at the
|
|
1663
|
+
* provider's cached-input rate. Output is credited only under
|
|
1664
|
+
* deterministic sampling, on a model with a list price in the
|
|
1665
|
+
* catalog that does not reason and honors that setting, when
|
|
1666
|
+
* the call that ran carried no hidden reasoning (zero reasoning
|
|
1667
|
+
* tokens reported, or no reasoning block where the provider
|
|
1668
|
+
* does not break them out), and then at 95% of its counter.
|
|
1669
|
+
* With an output schema, only when the schema traveled exactly
|
|
1670
|
+
* as sent, in strict mode, in the provider's native
|
|
1671
|
+
* response-format parameter. Valued low on purpose. Enters the
|
|
1672
|
+
* commission base.
|
|
1673
|
+
* - `estimated` — the call was AVOIDED: it never ran, so there is no provider
|
|
1674
|
+
* counter and the baseline comes from the local estimator. Still
|
|
1675
|
+
* the best figure available, but not measured.
|
|
1676
|
+
* - `unmeasured` — the provider reported no usage. Nothing is claimed.
|
|
1677
|
+
* - `shadow` — HYPOTHETICAL: what would have happened with a lever turned on
|
|
1678
|
+
* (the shadow lane). Never enters the commission base, never
|
|
1679
|
+
* added to measured or to any total; shadow reports are built
|
|
1680
|
+
* separately and `estimateCost` must never produce this value.
|
|
1681
|
+
*
|
|
1682
|
+
* Summing `measured` and `estimated` into a single "money saved" figure is
|
|
1683
|
+
* exactly what a verify-don't-trust product cannot do — and adding `shadow` to
|
|
1684
|
+
* either would be making the savings up.
|
|
1685
|
+
*/
|
|
1686
|
+
type CostBasis = "measured" | "reference" | "estimated" | "unmeasured" | "shadow";
|
|
1687
|
+
interface CostModel {
|
|
1688
|
+
/** Cost per 1K input tokens, in the caller's currency unit. */
|
|
1689
|
+
inputPer1k: number;
|
|
1690
|
+
/** Cost per 1K output tokens. */
|
|
1691
|
+
outputPer1k: number;
|
|
1692
|
+
/** Optional discounted rate for provider-cached (prefix) input tokens. */
|
|
1693
|
+
cachedInputPer1k?: number;
|
|
1694
|
+
}
|
|
1695
|
+
/**
|
|
1696
|
+
* Why an avoided call (a cache or coalesced hit) could not be valued by
|
|
1697
|
+
* reference, and so stays `estimated`.
|
|
1698
|
+
*
|
|
1699
|
+
* - `absent` — the stored answer carries no provider counter.
|
|
1700
|
+
* - `malformed` — it carries one, but not in a readable shape.
|
|
1701
|
+
* - `witness` — the counter is not bound to the answer being served.
|
|
1702
|
+
* - `model-mismatch` — the counter belongs to another model.
|
|
1703
|
+
* - `expired` — the counter is older than one billing period.
|
|
1704
|
+
* - `different-request` — the request that ran was not byte-identical to this one.
|
|
1705
|
+
* - `quality` — the answer being served fails its output contract.
|
|
1706
|
+
* - `no-cached-price` — no documented cached-input rate for this exact model:
|
|
1707
|
+
* its own catalogue row (or a dated snapshot of it) or
|
|
1708
|
+
* `costModels`, never a row inherited from a sibling id.
|
|
1709
|
+
* - `no-credit` — nothing left to credit once the allowances are applied.
|
|
1710
|
+
*/
|
|
1711
|
+
type ReferenceRejection = "absent" | "malformed" | "witness" | "model-mismatch" | "expired" | "different-request" | "quality" | "no-cached-price" | "no-credit";
|
|
1712
|
+
/**
|
|
1713
|
+
* The provider counter an avoided call is valued against, and every figure
|
|
1714
|
+
* needed to recompute its baseline:
|
|
1715
|
+
* `tierFactor * (creditedInputTokens / 1000 * cachedInputPer1k + creditedOutputTokens / 1000 * outputPer1k)`.
|
|
1716
|
+
*/
|
|
1717
|
+
interface ReferenceCredit {
|
|
1718
|
+
/** Id of the report of the call that ran (its report.requestId). */
|
|
1719
|
+
requestId: string;
|
|
1720
|
+
/** Epoch ms when that call's provider counter was read. */
|
|
1721
|
+
at: number;
|
|
1722
|
+
/** Input tokens the provider billed for that call. */
|
|
1723
|
+
inputTokens: number;
|
|
1724
|
+
/** Output tokens the provider billed for that call (the smaller of the first answer and its repair). */
|
|
1725
|
+
outputTokens: number;
|
|
1726
|
+
/** Input tokens credited to this repeat: inputTokens minus the allowance for what the layer inserted (messages, the native schema wrapper, empty tool parameters). */
|
|
1727
|
+
creditedInputTokens: number;
|
|
1728
|
+
/**
|
|
1729
|
+
* Output tokens credited: a share of outputTokens capped by this request's
|
|
1730
|
+
* output limit. 0 unless this request asked for deterministic sampling
|
|
1731
|
+
* (`temperature: 0`), the model has a list price in the catalog, does not
|
|
1732
|
+
* reason and honors that setting, and the call that ran carried no hidden
|
|
1733
|
+
* reasoning (zero reasoning tokens reported, or no reasoning block where the
|
|
1734
|
+
* provider does not break them out).
|
|
1735
|
+
*/
|
|
1736
|
+
creditedOutputTokens: number;
|
|
1737
|
+
/** Rate applied to every credited input token, per 1K: the provider's cached-input rate. */
|
|
1738
|
+
cachedInputPer1k: number;
|
|
1739
|
+
/** Rate applied to credited output tokens, per 1K. */
|
|
1740
|
+
outputPer1k: number;
|
|
1741
|
+
/** Fraction of the standard price applied: 1, or the discounted tier the call ran on or this request declared. */
|
|
1742
|
+
tierFactor: number;
|
|
1743
|
+
}
|
|
1744
|
+
|
|
1745
|
+
/**
|
|
1746
|
+
* LIST PRICES PER MODEL.
|
|
1747
|
+
*
|
|
1748
|
+
* The layer reports savings in money, and money needs a price. This is the
|
|
1749
|
+
* built-in catalogue of published list prices, so the dollar figures mean
|
|
1750
|
+
* something without any configuration.
|
|
1751
|
+
*
|
|
1752
|
+
* ⚠️ A WRONG PRICE HERE IS WORSE THAN NO CATALOGUE. A model that is ABSENT falls
|
|
1753
|
+
* back to the illustrative default AND warns, so the customer knows that column
|
|
1754
|
+
* is not their invoice. A model that is PRESENT with the wrong price silences
|
|
1755
|
+
* that warning and hands over a confident, wrong number. So only models whose
|
|
1756
|
+
* published price is known with certainty belong here, and {@link PRECIOS_AL_DIA}
|
|
1757
|
+
* records when they were last checked.
|
|
1758
|
+
*
|
|
1759
|
+
* Prices are USD per 1,000 tokens — the unit of `CostModel` — even though
|
|
1760
|
+
* providers publish per million: 1M to 1K is a division by a thousand.
|
|
1761
|
+
*
|
|
1762
|
+
* `costModels` on the layer config ALWAYS wins over this. A customer with volume
|
|
1763
|
+
* discounts, negotiated rates or a different region sets their own and this
|
|
1764
|
+
* catalogue stays out of the way.
|
|
1765
|
+
*/
|
|
1766
|
+
/**
|
|
1767
|
+
* Version of this price catalogue: the UTC day its content last changed (rows,
|
|
1768
|
+
* announced price steps, cache-write premiums, tier rates). Two computations
|
|
1769
|
+
* citing the same version were priced with the same content.
|
|
1770
|
+
*/
|
|
1771
|
+
declare const PRECIOS_AL_DIA = "2026-11-03";
|
|
1772
|
+
/**
|
|
1773
|
+
* Key = the model id as the provider publishes it, without a vendor prefix.
|
|
1774
|
+
* Lookup tries the exact id, then the id without its provider prefix, then the
|
|
1775
|
+
* LONGEST matching prefix — so `gpt-4o-2024-08-06` inherits `gpt-4o` without
|
|
1776
|
+
* having to enumerate every dated snapshot.
|
|
1777
|
+
*
|
|
1778
|
+
* These are the rows in force on {@link PRECIOS_AL_DIA}. A price change a
|
|
1779
|
+
* provider has already announced is applied by {@link precioDeLista} from its
|
|
1780
|
+
* effective date, so a request is always priced at the rate of its own day.
|
|
1781
|
+
*/
|
|
1782
|
+
declare const CATALOGO: Readonly<Record<string, CostModel>>;
|
|
1783
|
+
/**
|
|
1784
|
+
* The list price for a model on the day of the request, or `undefined` when
|
|
1785
|
+
* the catalogue has no price for it in force that day.
|
|
1786
|
+
*
|
|
1787
|
+
* @param model the id as the caller passes it, with or without a vendor prefix.
|
|
1788
|
+
* @param at when the request was made (epoch ms or a `Date`); defaults to now.
|
|
1789
|
+
* Prices resolve per UTC day, so a price change the provider has announced
|
|
1790
|
+
* applies from its effective date.
|
|
1791
|
+
*/
|
|
1792
|
+
declare function precioDeLista(model: string, at?: Date | number): CostModel | undefined;
|
|
1793
|
+
|
|
1794
|
+
/**
|
|
1795
|
+
* The version the SDK reports for itself.
|
|
1796
|
+
*
|
|
1797
|
+
* Written by hand and deliberately NOT read from package.json: the published
|
|
1798
|
+
* bundle does not ship package.json, so a `require` here would fail at runtime
|
|
1799
|
+
* on the customer's machine. `test/version-declarada.test.ts` ties this
|
|
1800
|
+
* constant to package.json so the two can never drift.
|
|
1801
|
+
*/
|
|
1802
|
+
declare const BV_SALA_VERSION = "1.1.0";
|
|
1803
|
+
|
|
1748
1804
|
/**
|
|
1749
1805
|
* Typed intermediate representation for LLM -> LLM handoffs.
|
|
1750
1806
|
*
|
|
@@ -1981,8 +2037,8 @@ declare class CalibrationStateError extends Error {
|
|
|
1981
2037
|
*/
|
|
1982
2038
|
type AnswerJudge = (stored: CachedAnswer, fresh: CachedAnswer) => boolean | Promise<boolean>;
|
|
1983
2039
|
/**
|
|
1984
|
-
*
|
|
1985
|
-
*
|
|
2040
|
+
* Calibration required for semantic reuse: the cache serves a reworded request
|
|
2041
|
+
* only at or above a floor that labeled
|
|
1986
2042
|
* traffic has certified, so that at most `maxErrorRate` of the reuses served
|
|
1987
2043
|
* from that floor up are wrong, at the stated `confidence`.
|
|
1988
2044
|
*
|
|
@@ -2053,6 +2109,8 @@ interface SemanticCalibrationState {
|
|
|
2053
2109
|
verifier: string | null;
|
|
2054
2110
|
/** `built_in:<version>` for {@link sameAnswer}, `custom:<judgeId>` for yours. */
|
|
2055
2111
|
judge: string;
|
|
2112
|
+
/** Storage backend whose observations produced this state; absent in states written before backend identities. */
|
|
2113
|
+
backend?: string;
|
|
2056
2114
|
config: {
|
|
2057
2115
|
maxErrorRate: number;
|
|
2058
2116
|
confidence: number;
|
|
@@ -2137,7 +2195,8 @@ declare function calibrateSemanticFloor(observations: readonly CalibrationObserv
|
|
|
2137
2195
|
* question. Therefore the headline metric here is NOT hit ratio — it is the
|
|
2138
2196
|
* PRECISION of hits. Every candidate must pass: tenant/permission filter,
|
|
2139
2197
|
* knowledge-epoch filter, a similarity threshold, a stronger local verifier, a
|
|
2140
|
-
* freshness check,
|
|
2198
|
+
* freshness check, an output-contract compatibility check, and a certified
|
|
2199
|
+
* calibration floor. Without calibration the cache is shadow-only and serves nothing.
|
|
2141
2200
|
*/
|
|
2142
2201
|
interface EmbeddingFn {
|
|
2143
2202
|
(text: string): Promise<number[]> | number[];
|
|
@@ -2151,6 +2210,8 @@ interface SemanticVerifier {
|
|
|
2151
2210
|
(query: string, candidateQuery: string): Promise<number> | number;
|
|
2152
2211
|
}
|
|
2153
2212
|
interface SemanticRecord {
|
|
2213
|
+
/** Stable backend identifier, when the store assigns one. */
|
|
2214
|
+
recordId?: string;
|
|
2154
2215
|
embedding: number[];
|
|
2155
2216
|
query: string;
|
|
2156
2217
|
answer: CachedAnswer;
|
|
@@ -2237,6 +2298,8 @@ interface SemanticHit {
|
|
|
2237
2298
|
*/
|
|
2238
2299
|
interface SemanticProbe {
|
|
2239
2300
|
reason: "calibrating" | "below_floor" | "record_floor" | "audit";
|
|
2301
|
+
/** Stable backend identifier for this observation, when the store assigns one. */
|
|
2302
|
+
observationId?: string;
|
|
2240
2303
|
/** The score the decision was made on: the lower of similarity and your verifier's score. */
|
|
2241
2304
|
score: number;
|
|
2242
2305
|
segment: "single_shot" | "agentic";
|
|
@@ -2247,6 +2310,15 @@ interface SemanticLookup {
|
|
|
2247
2310
|
hit: SemanticHit | null;
|
|
2248
2311
|
probe: SemanticProbe | null;
|
|
2249
2312
|
}
|
|
2313
|
+
/** Structural contract required by the pipeline; backends may add persistence and lifecycle methods. */
|
|
2314
|
+
interface SemanticCacheStore {
|
|
2315
|
+
readonly calibrated: boolean;
|
|
2316
|
+
lookup(c: CanonicalRequest, opts?: {
|
|
2317
|
+
agentic?: boolean;
|
|
2318
|
+
}): Promise<SemanticLookup>;
|
|
2319
|
+
store(c: CanonicalRequest, answer: CachedAnswer): Promise<void>;
|
|
2320
|
+
observe(probe: SemanticProbe, fresh: CachedAnswer): Promise<boolean | null>;
|
|
2321
|
+
}
|
|
2250
2322
|
interface SemanticCacheOptions {
|
|
2251
2323
|
/**
|
|
2252
2324
|
* How text becomes a vector. OPTIONAL: omit it and {@link localEmbedding} is
|
|
@@ -2280,15 +2352,16 @@ interface SemanticCacheOptions {
|
|
|
2280
2352
|
verifierId?: string;
|
|
2281
2353
|
/** Cosine similarity floor for a candidate to be considered at all. */
|
|
2282
2354
|
similarityThreshold?: number;
|
|
2283
|
-
/**
|
|
2355
|
+
/** Explicit verifier score floor applied in addition to the certified floor. */
|
|
2284
2356
|
acceptThreshold?: number;
|
|
2285
2357
|
ttlMs?: number;
|
|
2286
2358
|
maxRecords?: number;
|
|
2287
2359
|
now?: () => number;
|
|
2288
2360
|
/**
|
|
2289
|
-
*
|
|
2290
|
-
*
|
|
2291
|
-
* {@link
|
|
2361
|
+
* Required to serve semantic hits: your own labeled traffic certifies a floor
|
|
2362
|
+
* for a wrong-reuse rate you choose. Without it the cache is shadow-only and
|
|
2363
|
+
* may store answers, but {@link find} and {@link lookup} never return a hit.
|
|
2364
|
+
* See {@link SemanticCalibrationOptions}.
|
|
2292
2365
|
*/
|
|
2293
2366
|
calibration?: SemanticCalibrationOptions;
|
|
2294
2367
|
}
|
|
@@ -2298,7 +2371,7 @@ interface SemanticCacheOptions {
|
|
|
2298
2371
|
* to the store. The verification logic — the part that makes it *safe* — stays
|
|
2299
2372
|
* identical regardless of the backend.
|
|
2300
2373
|
*/
|
|
2301
|
-
declare class InMemorySemanticCache {
|
|
2374
|
+
declare class InMemorySemanticCache implements SemanticCacheStore {
|
|
2302
2375
|
private readonly records;
|
|
2303
2376
|
private readonly embed;
|
|
2304
2377
|
private readonly verify;
|
|
@@ -2309,7 +2382,7 @@ declare class InMemorySemanticCache {
|
|
|
2309
2382
|
private readonly now;
|
|
2310
2383
|
private readonly embebedorPropio;
|
|
2311
2384
|
private readonly calibracion;
|
|
2312
|
-
private readonly
|
|
2385
|
+
private readonly diario;
|
|
2313
2386
|
private readonly nombres;
|
|
2314
2387
|
private readonly pendientes;
|
|
2315
2388
|
private readonly aceptacionExplicita;
|
|
@@ -2321,21 +2394,22 @@ declare class InMemorySemanticCache {
|
|
|
2321
2394
|
* FIRST and hard — a candidate from another tenant, scope, or knowledge epoch
|
|
2322
2395
|
* is never even scored.
|
|
2323
2396
|
*
|
|
2324
|
-
*
|
|
2325
|
-
*
|
|
2326
|
-
*
|
|
2397
|
+
* Only a certified calibration floor can authorize a hit. Without calibration
|
|
2398
|
+
* this always returns null, including for direct callers. It labels nothing:
|
|
2399
|
+
* without {@link lookup} and {@link observe} (the layer calls both) the
|
|
2400
|
+
* calibration never gets a label and never certifies.
|
|
2327
2401
|
*/
|
|
2328
2402
|
find(c: CanonicalRequest): Promise<SemanticHit | null>;
|
|
2329
2403
|
/**
|
|
2330
2404
|
* The best candidate that passes every hard filter, the similarity floor and
|
|
2331
|
-
* the fact guard.
|
|
2332
|
-
* without calibration, unchanged); without it, calibration decides.
|
|
2405
|
+
* the fact guard. Calibration decides whether it can be served.
|
|
2333
2406
|
*/
|
|
2334
2407
|
private mejorCandidata;
|
|
2335
2408
|
/**
|
|
2336
2409
|
* Like {@link find}, and with calibration on it also returns the candidate it
|
|
2337
2410
|
* did not serve, so the caller can label it with {@link observe} once the
|
|
2338
|
-
* provider has answered. Without calibration, `probe`
|
|
2411
|
+
* provider has answered. Without calibration, both `hit` and `probe` are null:
|
|
2412
|
+
* stored answers remain available for future calibration but are never served.
|
|
2339
2413
|
*
|
|
2340
2414
|
* `agentic` picks the traffic segment: calibrations are kept apart for
|
|
2341
2415
|
* single-shot and agentic requests.
|
|
@@ -2387,7 +2461,6 @@ declare class InMemorySemanticCache {
|
|
|
2387
2461
|
*/
|
|
2388
2462
|
importCalibration(state: SemanticCalibrationState | string | unknown): void;
|
|
2389
2463
|
private identidad;
|
|
2390
|
-
private estadoDe;
|
|
2391
2464
|
store(c: CanonicalRequest, answer: CachedAnswer): Promise<void>;
|
|
2392
2465
|
/** Drop every record for a knowledge epoch (invalidation on re-index). */
|
|
2393
2466
|
invalidateEpoch(knowledgeEpoch: string): number;
|
|
@@ -2774,6 +2847,12 @@ interface ConstraintPlan {
|
|
|
2774
2847
|
/** A permissive backend used when the provider validates structured output natively. */
|
|
2775
2848
|
declare function providerNativeBackend(isSupported?: (model: string, schema: JsonSchema) => boolean): ConstraintBackend;
|
|
2776
2849
|
|
|
2850
|
+
/** Durable, asynchronous memory for value constraints learned from prior responses. */
|
|
2851
|
+
interface ConstraintMemoryStore {
|
|
2852
|
+
has(key: string): Promise<boolean>;
|
|
2853
|
+
remember(key: string): Promise<void>;
|
|
2854
|
+
}
|
|
2855
|
+
|
|
2777
2856
|
/**
|
|
2778
2857
|
* Context compression, as an OPTIONAL sidecar.
|
|
2779
2858
|
*
|
|
@@ -2861,6 +2940,17 @@ interface ToolOutputCompactionSettings {
|
|
|
2861
2940
|
* would not match the file on disk.
|
|
2862
2941
|
*/
|
|
2863
2942
|
compactFileViews?: boolean;
|
|
2943
|
+
/**
|
|
2944
|
+
* `"lossless-data"`: run only the JSON transformation, under a strict check.
|
|
2945
|
+
* The output must be strict JSON (RFC 8259) with no duplicate key at any
|
|
2946
|
+
* depth, and the compacted text must have exactly the same lexemes as the
|
|
2947
|
+
* original — same strings with the same escapes, same number literals, same
|
|
2948
|
+
* key order — with only insignificant whitespace removed. It is applied only
|
|
2949
|
+
* when it is also strictly smaller in estimated tokens. `transforms` is
|
|
2950
|
+
* ignored. Every tool output gets an explained decision. Default: unset (the
|
|
2951
|
+
* four transformations, as before).
|
|
2952
|
+
*/
|
|
2953
|
+
profile?: "lossless-data";
|
|
2864
2954
|
}
|
|
2865
2955
|
/** The outcome of compacting one tool output. */
|
|
2866
2956
|
interface ToolOutputCompaction {
|
|
@@ -2873,7 +2963,39 @@ interface ToolOutputCompaction {
|
|
|
2873
2963
|
compactChars: number;
|
|
2874
2964
|
/** Why the output was, or was not, compacted. */
|
|
2875
2965
|
reason: string;
|
|
2966
|
+
/**
|
|
2967
|
+
* Only with `profile: "lossless-data"`: the explained decision, with a stable
|
|
2968
|
+
* reason code. Token counts are estimates from the estimator in use, never a
|
|
2969
|
+
* verified saving.
|
|
2970
|
+
*/
|
|
2971
|
+
decision?: ToolOutputDecision;
|
|
2876
2972
|
}
|
|
2973
|
+
/**
|
|
2974
|
+
* Why a `lossless-data` decision went the way it did. `applied`: whitespace
|
|
2975
|
+
* removed. Bypass codes: `not_json` (the output is not a JSON object or array),
|
|
2976
|
+
* `invalid_json` (it looks like one but is not strict JSON), `duplicate_keys`,
|
|
2977
|
+
* `invalid_unicode` (an unpaired surrogate), `not_smaller` (no fewer tokens),
|
|
2978
|
+
* `below_threshold` (fewer than `minSavedChars` removed), `protected` (a file
|
|
2979
|
+
* view or an excepted tool).
|
|
2980
|
+
*/
|
|
2981
|
+
type ToolOutputDecisionReason = "applied" | "not_json" | "invalid_json" | "duplicate_keys" | "invalid_unicode" | "not_smaller" | "below_threshold" | "protected";
|
|
2982
|
+
/** One explained decision of the `lossless-data` profile. */
|
|
2983
|
+
interface ToolOutputDecision {
|
|
2984
|
+
status: "applied" | "bypassed";
|
|
2985
|
+
reason: ToolOutputDecisionReason;
|
|
2986
|
+
transformId: typeof LOSSLESS_JSON_TRANSFORM_ID;
|
|
2987
|
+
version: typeof LOSSLESS_JSON_VERSION;
|
|
2988
|
+
/** Estimated tokens of the original output. */
|
|
2989
|
+
tokensBefore: number;
|
|
2990
|
+
/** Estimated tokens of what is sent: the compacted text when applied, the original otherwise. */
|
|
2991
|
+
tokensAfter: number;
|
|
2992
|
+
/** Estimated tokens of the compacted candidate, when one was built (also when bypassed). */
|
|
2993
|
+
candidateTokens?: number;
|
|
2994
|
+
}
|
|
2995
|
+
/** Id of the `lossless-data` transformation, as it appears in a decision. */
|
|
2996
|
+
declare const LOSSLESS_JSON_TRANSFORM_ID = "json-lossless";
|
|
2997
|
+
/** Version of that transformation. A change of output for the same input bumps it. */
|
|
2998
|
+
declare const LOSSLESS_JSON_VERSION = "1";
|
|
2877
2999
|
/** Default minimum saving, in characters, for an output to be compacted. */
|
|
2878
3000
|
declare const DEFAULT_MIN_SAVED_CHARS = 256;
|
|
2879
3001
|
/**
|
|
@@ -2889,8 +3011,11 @@ declare const DEFAULT_MIN_SAVED_CHARS = 256;
|
|
|
2889
3011
|
* is produced, before you append it to your conversation history, and the
|
|
2890
3012
|
* history itself holds the compacted text — the provider's prefix cache is
|
|
2891
3013
|
* never disturbed.
|
|
3014
|
+
*
|
|
3015
|
+
* `countTokens` is used only by the `lossless-data` profile, for its token
|
|
3016
|
+
* gate; without it the built-in heuristic estimator counts.
|
|
2892
3017
|
*/
|
|
2893
|
-
declare function compactToolOutput(text: string, settings?: ToolOutputCompactionSettings): ToolOutputCompaction;
|
|
3018
|
+
declare function compactToolOutput(text: string, settings?: ToolOutputCompactionSettings, countTokens?: (text: string) => number): ToolOutputCompaction;
|
|
2894
3019
|
|
|
2895
3020
|
/** What the layer needs to know about a provider's prefix cache to price a decision. */
|
|
2896
3021
|
interface CacheTier {
|
|
@@ -3348,6 +3473,12 @@ interface SavingsLayerConfig {
|
|
|
3348
3473
|
* Solo guarda identificadores de contrato, nunca contenido de peticiones.
|
|
3349
3474
|
*/
|
|
3350
3475
|
constraintMemory?: Set<string>;
|
|
3476
|
+
/**
|
|
3477
|
+
* Optional durable backend. The local Set remains a write-through cache, so
|
|
3478
|
+
* both options compose. Durable keys also include tenant, scope, provider and
|
|
3479
|
+
* served model; store failures never repeat a provider call.
|
|
3480
|
+
*/
|
|
3481
|
+
constraintMemoryStore?: ConstraintMemoryStore;
|
|
3351
3482
|
provider: Provider;
|
|
3352
3483
|
tokenizer: TokenizerRegistry;
|
|
3353
3484
|
exactCache?: ExactCache;
|
|
@@ -3363,7 +3494,7 @@ interface SavingsLayerConfig {
|
|
|
3363
3494
|
* the per-request switch (`savings.semanticCache`) no longer share a name.
|
|
3364
3495
|
* See the 0.5 migration guide.
|
|
3365
3496
|
*/
|
|
3366
|
-
semanticCacheStore?:
|
|
3497
|
+
semanticCacheStore?: SemanticCacheStore;
|
|
3367
3498
|
/**
|
|
3368
3499
|
* Default for `savings.semanticCache` on every request of this layer. A
|
|
3369
3500
|
* request's own value always wins, `false` included. Unset, the semantic cache
|
|
@@ -3814,6 +3945,16 @@ interface BatchSubmitOptions {
|
|
|
3814
3945
|
* OpenAI, by model).
|
|
3815
3946
|
*/
|
|
3816
3947
|
maxRequestsPerBatch?: number;
|
|
3948
|
+
/**
|
|
3949
|
+
* Deduplicate requests whose final provider body is byte-identical within THIS
|
|
3950
|
+
* submission: the first is sent and the rest are replayed from its result, so the
|
|
3951
|
+
* provider bills each unique request once. Defaults to false. Only turn it on when
|
|
3952
|
+
* identical bodies imply identical answers — deterministic sampling (temperature 0 or
|
|
3953
|
+
* unset), or a seed you control. With temperature > 0 and no seed the provider would
|
|
3954
|
+
* have sampled each copy independently, and deduplication hands every copy the first
|
|
3955
|
+
* one's answer.
|
|
3956
|
+
*/
|
|
3957
|
+
deduplicate?: boolean;
|
|
3817
3958
|
/** Cancels the HTTP calls of this submission. Batches already created stay created. */
|
|
3818
3959
|
signal?: AbortSignal;
|
|
3819
3960
|
}
|
|
@@ -3843,6 +3984,17 @@ interface BatchJob {
|
|
|
3843
3984
|
batch: number;
|
|
3844
3985
|
schema?: number;
|
|
3845
3986
|
}[];
|
|
3987
|
+
/**
|
|
3988
|
+
* Requests that were NOT sent because an identical one (same final provider body) was
|
|
3989
|
+
* already in this submission. Each entry names its own id, the id it duplicates, and
|
|
3990
|
+
* its model. `results()` replays the duplicate from the source's result; nothing about
|
|
3991
|
+
* the answer is stored here. Present only when at least one request was deduplicated.
|
|
3992
|
+
*/
|
|
3993
|
+
duplicates?: {
|
|
3994
|
+
customId: string;
|
|
3995
|
+
sameAs: string;
|
|
3996
|
+
model: string;
|
|
3997
|
+
}[];
|
|
3846
3998
|
/** The distinct output schemas of the job, stored once each. */
|
|
3847
3999
|
schemas: JsonSchema[];
|
|
3848
4000
|
}
|
|
@@ -3932,6 +4084,8 @@ interface BatchSummary {
|
|
|
3932
4084
|
missing: number;
|
|
3933
4085
|
/** Requests whose provider batch has not ended yet: not in `items`. */
|
|
3934
4086
|
pending: number;
|
|
4087
|
+
/** Requests dropped at submit() because an identical one was already in the job. */
|
|
4088
|
+
deduplicated?: number;
|
|
3935
4089
|
/** Succeeded requests with a `tierSaving`. */
|
|
3936
4090
|
credited: number;
|
|
3937
4091
|
/** Sum of every `tierSaving.amount`. Never more than the sum of each request's published difference. */
|
|
@@ -4118,6 +4272,35 @@ declare function localEmbedding(text: string): number[];
|
|
|
4118
4272
|
*/
|
|
4119
4273
|
declare const LOCAL_EMBEDDING_THRESHOLD = 0.72;
|
|
4120
4274
|
|
|
4275
|
+
interface SemanticCalibrationIdentity {
|
|
4276
|
+
embedder: string;
|
|
4277
|
+
verifier: string | null;
|
|
4278
|
+
judge: string;
|
|
4279
|
+
maxErrorRate: number;
|
|
4280
|
+
confidence: number;
|
|
4281
|
+
similarityThreshold: number;
|
|
4282
|
+
backend?: string;
|
|
4283
|
+
}
|
|
4284
|
+
/** Pure, shared online calibration state, without storage or I/O. */
|
|
4285
|
+
declare class SemanticCalibrationJournal {
|
|
4286
|
+
private readonly identity;
|
|
4287
|
+
private readonly estados;
|
|
4288
|
+
constructor(identity: SemanticCalibrationIdentity);
|
|
4289
|
+
floorIndex(segment: SemanticProbe["segment"]): number | null;
|
|
4290
|
+
record(segment: SemanticProbe["segment"], score: number, correct: boolean): void;
|
|
4291
|
+
status(segment: SemanticProbe["segment"]): SemanticCalibrationStatus;
|
|
4292
|
+
export(): SemanticCalibrationState;
|
|
4293
|
+
import(state: unknown): void;
|
|
4294
|
+
private estadoDe;
|
|
4295
|
+
}
|
|
4296
|
+
|
|
4297
|
+
/** Shared deterministic safety operations for semantic-cache backends. */
|
|
4298
|
+
declare class SemanticCacheSafety {
|
|
4299
|
+
static cosine(a: number[], b: number[]): number;
|
|
4300
|
+
static keyTokensMatch(query: string, candidateQuery: string): boolean;
|
|
4301
|
+
static sameAnswer(stored: CachedAnswer, fresh: CachedAnswer): boolean;
|
|
4302
|
+
}
|
|
4303
|
+
|
|
4121
4304
|
/**
|
|
4122
4305
|
* Conversational memory by deltas.
|
|
4123
4306
|
*
|
|
@@ -4647,7 +4830,7 @@ declare class LicenseError extends Error {
|
|
|
4647
4830
|
|
|
4648
4831
|
/** The public handle returned by {@link createSavingsLayer}. */
|
|
4649
4832
|
interface SavingsLayer {
|
|
4650
|
-
generate<T = unknown>(request: SavingsRequest): Promise<SavingsResult<T>>;
|
|
4833
|
+
generate<T = unknown>(request: SavingsRequest, options?: SavingsRequestOptions): Promise<SavingsResult<T>>;
|
|
4651
4834
|
/**
|
|
4652
4835
|
* Same request as generate(), delivered as a stream of events.
|
|
4653
4836
|
*
|
|
@@ -4679,13 +4862,13 @@ interface SavingsLayer {
|
|
|
4679
4862
|
*/
|
|
4680
4863
|
readonly batches: SavingsBatches;
|
|
4681
4864
|
/**
|
|
4682
|
-
* Stop
|
|
4683
|
-
*
|
|
4684
|
-
*
|
|
4685
|
-
*
|
|
4686
|
-
*
|
|
4865
|
+
* Stop background work, drain usage batches and await a semantic store's
|
|
4866
|
+
* optional `settled()` method last. Backends should handle their own expected
|
|
4867
|
+
* observation failures; an unexpected `settled()` rejection is propagated
|
|
4868
|
+
* after the other components have closed rather than silently discarded.
|
|
4869
|
+
* Calling close() without await preserves fire-and-forget shutdown behavior.
|
|
4687
4870
|
*/
|
|
4688
|
-
close(): void
|
|
4871
|
+
close(): void | Promise<void>;
|
|
4689
4872
|
}
|
|
4690
4873
|
/**
|
|
4691
4874
|
* Create a savings layer. The only hard requirement is a `provider`; a default
|
|
@@ -4694,14 +4877,16 @@ interface SavingsLayer {
|
|
|
4694
4877
|
* cache, a vector-backed semantic cache, a real tokenizer, a compressor sidecar,
|
|
4695
4878
|
* XGrammar/llguidance backends, etc.).
|
|
4696
4879
|
*
|
|
4697
|
-
*
|
|
4698
|
-
*
|
|
4699
|
-
*
|
|
4880
|
+
* Open core: no `licenseKey` is needed. Without one the layer runs on the
|
|
4881
|
+
* COMMUNITY gate — every local saving lever on, zero telemetry, zero network.
|
|
4882
|
+
* A `licenseKey` (from savings.bivelio.com) turns on the paid surfaces: cloud
|
|
4883
|
+
* metering, license renewal/revocation and org entitlements. An INVALID or
|
|
4884
|
+
* EXPIRED key still fails closed with {@link LicenseError}; only its absence is
|
|
4885
|
+
* a working product.
|
|
4700
4886
|
*
|
|
4701
4887
|
* @example
|
|
4702
|
-
* //
|
|
4703
|
-
*
|
|
4704
|
-
* const layer = createSavingsLayer({ provider, licenseKey });
|
|
4888
|
+
* // No license: full savings, no telemetry.
|
|
4889
|
+
* const layer = createSavingsLayer({ provider });
|
|
4705
4890
|
* const { text, report } = await layer.generate({
|
|
4706
4891
|
* model: "openai/gpt-4o-mini",
|
|
4707
4892
|
* messages: [{ role: "user", content: "What is 2 + 2 * 3?" }],
|
|
@@ -4709,6 +4894,17 @@ interface SavingsLayer {
|
|
|
4709
4894
|
*/
|
|
4710
4895
|
declare function createSavingsLayer(configPublica: Omit<SavingsLayerConfig, "tokenizer"> & Partial<Pick<SavingsLayerConfig, "tokenizer">>): SavingsLayer;
|
|
4711
4896
|
|
|
4897
|
+
/**
|
|
4898
|
+
* The total request deadline expired. No partial result or zero-cost report is
|
|
4899
|
+
* returned: tokens already sent to the provider may still appear on its invoice.
|
|
4900
|
+
*/
|
|
4901
|
+
declare class RequestTimeoutError extends Error {
|
|
4902
|
+
readonly code = "REQUEST_TIMEOUT";
|
|
4903
|
+
/** True when at least one provider request had already been sent. */
|
|
4904
|
+
readonly requestSent: boolean;
|
|
4905
|
+
readonly timeoutMs: number;
|
|
4906
|
+
constructor(requestSent: boolean, timeoutMs: number);
|
|
4907
|
+
}
|
|
4712
4908
|
/**
|
|
4713
4909
|
* Rejection of `SavingsStream.result` when the stream was aborted before the
|
|
4714
4910
|
* provider finished its answer. Nothing was cached and no usage was recorded.
|
|
@@ -5108,9 +5304,12 @@ declare function toCsv(rows: readonly Readonly<Record<string, CsvValue>>[], opti
|
|
|
5108
5304
|
/**
|
|
5109
5305
|
* bivelio-savings-layer (BV-SALA)
|
|
5110
5306
|
* ---------------------------------------------------------------------------
|
|
5111
|
-
*
|
|
5112
|
-
*
|
|
5113
|
-
*
|
|
5307
|
+
* Copyright (c) 2026 BiVelio Inc.
|
|
5308
|
+
* Licensed under the Apache License, Version 2.0 (the "License"); you may not
|
|
5309
|
+
* use this file except in compliance with the License. You may obtain a copy
|
|
5310
|
+
* of the License at https://www.apache.org/licenses/LICENSE-2.0 — the full
|
|
5311
|
+
* text ships in this repository's LICENSE file. The optional BiVelio cloud
|
|
5312
|
+
* services keep their own terms in LICENSE-CLOUD.md.
|
|
5114
5313
|
* ---------------------------------------------------------------------------
|
|
5115
5314
|
*
|
|
5116
5315
|
* A provider-neutral LLM execution optimizer. For every request it answers, in
|
|
@@ -5137,4 +5336,4 @@ declare const listPrice: typeof precioDeLista;
|
|
|
5137
5336
|
/** When the catalogue prices were last checked against the providers' public rates (YYYY-MM-DD). */
|
|
5138
5337
|
declare const PRICES_AS_OF: typeof PRECIOS_AL_DIA;
|
|
5139
5338
|
|
|
5140
|
-
export { type AnswerJudge, type AnswerSource, type AnthropicOptions, AnthropicProvider, BV_SALA_VERSION, type BatchCounts, type BatchItemResult, type BatchJob, type BatchJobStatus, type BatchRequest, type BatchResults, BatchSubmitError, type BatchSubmitOptions, type BatchSummary, type BivelioHandoff, COMPACTION_TRANSFORMS, COMPACT_ROWS_INSTRUCTION, type CacheBreakpointStore, type CacheOutcome, type CachePlacement, type CacheTier, type CacheWriteReport, type CachedAnswer, type CalibrationObservation, CalibrationStateError, type ChatMessage, type CompactRowsDecodeOptions, type CompactRowsOptions, type CompactionTransform, type CompressionEconomics, type CompressionPolicyInput, type CompressionRequest, type CompressionResult, type Compressor, type ConstraintBackend, type ConstraintBackendId, type ConstraintPlan, type ConversationState, type CostBasis, type CostModel, type CsvValue, DEFAULT_COST_MODEL, DEFAULT_MIN_SAVED_CHARS, type DeferredExecutionReport, type DeterministicResolver, type EmbeddingFn, type EvidenceDoc, type ExactCache, ExactTokenizer, type ExecutionClass, ExecutionRouter, type FocusExportOptions, type FocusRow, type FormatChoice, FormatSelector, type FormatSelectorOptions, type GoogleOptions, GoogleProvider, type GoogleThinkingConfig, type HandoffFact, HeuristicEstimator, InMemoryCacheBreakpointStore, InMemoryExactCache, InMemorySemanticCache, InMemoryToolSetFreezer, type JsonSchema, LOCAL_EMBEDDING_THRESHOLD, type LangfuseGenerationUpdate, LicenseError, type LicenseGate, type LicenseKeyset, type LocalClassifier, MODELS_BY_ENCODING, type MeasureReport, type MemoryDelta, type MessageRole, MockProvider, type MockProviderOptions, type ModelCascadeMode, type ModelCascadeReport, type ModelCascadeSettings, type NativeSavingsReport, type OpenAICompatibleOptions, OpenAICompatibleProvider, type OutputContract, PRICES_AS_OF, PRICE_CATALOG, type Policy, type PriceLeverSaving, type PriceLeverSettings, type ProtectedSpan, type Provider, type ProviderRequest, type ProviderResponse, type ProviderStreamEvent, type ProviderStreamOptions, type ProviderToolSearch, type ProviderToolSearchUsage, type ProviderUsage, QUALITY_STATUSES, type QualityCacheMatch, type QualityCascadeCheck, type QualityLedger, type QualityRepairCheck, type QualitySchemaCheck, type QualityStatus, type QualityTruncationCheck, type ReasoningEffort, type ReferenceCredit, type ReferenceRejection, type RequestContext, type Reranker, ResolverRegistry, type ResolverResult, type RetrievalOptions, type RetrievalOutcome, type RetrievalPlan, type Retriever, type RouteDecision, type SavingsAdapterLevers, type SavingsAdapterOptions, type SavingsAttributeOptions, type SavingsBatches, type SavingsCallOptions, type SavingsLayer, type SavingsLayerConfig, type SavingsRecord, type SavingsReport, type SavingsReportForExport, type SavingsRequest, type SavingsResult, type SavingsSeriesOptions, type SavingsSeriesRow, type SavingsSpanAttributes, type SavingsSpanOptions, type SavingsStream, type SavingsStreamEvent, type SavingsStreamOptions, type SavingsToggles, type SavingsTracingOptions, SchemaRegistry, type SemanticCacheOptions, type SemanticCalibrationBound, type SemanticCalibrationOptions, type SemanticCalibrationSegmentState, type SemanticCalibrationState, type SemanticCalibrationStatus, type SemanticHit, type SemanticLookup, type SemanticProbe, type SemanticRecord, type SemanticVerifier, type SerializationFormat, type Serializer, Singleflight, type SpanAttributeValue, type SpanLike, type SpanStartOptionsLike, StreamAbortedError, type StreamingReport, type TokenEstimator, TokenizerRegistry, type ToolCallRequest, type ToolDescriptor, type ToolOutputCompaction, type ToolOutputCompactionSettings, type ToolRisk, type ToolSearchMode, type ToolSearchOptions, type ToolSelection, ToolSelector, type ToolSelectorOptions, type ToolSetFreezer, type TraceEvent, type TracerLike, type TracerProviderLike, type ValidationError, type ValidationResult, type VerificationCheck, type VerificationExclusion, type VerificationOutcome, type VerificationReport, type VerificationSettings, type VerificationSide, type VerificationStratum, arithmeticResolver, calibrateSemanticFloor, compactRowsSerializer, compactToolOutput, createCompactRowsSerializer, createDefaultResolvers, createHandoff, createMockProvider, createSavingsLayer, csvSerializer, decodeCompactRows, defaultTokenizerRegistry, emptyState, encodeCompactRows, heuristicEstimator, instrumentSavingsLayer, jsonCompactSerializer, listPrice, localEmbedding, mergeDelta, noopCompressor, providerNativeBackend, recordSavingsOnSpan, registerExactTokenizer, renderState, savingsSpanAttributes, toCsv, toFocusRows, toLangfuseGeneration, toSavingsSeries, validateAgainstSchema };
|
|
5339
|
+
export { type AnswerJudge, type AnswerSource, type AnthropicOptions, AnthropicProvider, BV_SALA_VERSION, type BatchCounts, type BatchItemResult, type BatchJob, type BatchJobStatus, type BatchRequest, type BatchResults, BatchSubmitError, type BatchSubmitOptions, type BatchSummary, type BivelioHandoff, COMPACTION_TRANSFORMS, COMPACT_ROWS_INSTRUCTION, type CacheBreakpointStore, type CacheOutcome, type CachePlacement, type CacheTier, type CacheWriteReport, type CachedAnswer, type CalibrationObservation, CalibrationStateError, type ChatMessage, type CompactRowsDecodeOptions, type CompactRowsOptions, type CompactionTransform, type CompressionEconomics, type CompressionPolicyInput, type CompressionRequest, type CompressionResult, type Compressor, type ConstraintBackend, type ConstraintBackendId, type ConstraintMemoryStore, type ConstraintPlan, type ConversationState, type CostBasis, type CostModel, type CsvValue, DEFAULT_COST_MODEL, DEFAULT_MIN_SAVED_CHARS, type DeferredExecutionReport, type DeterministicResolver, type EmbeddingFn, type EvidenceDoc, type ExactCache, ExactTokenizer, type ExecutionClass, ExecutionRouter, type FocusExportOptions, type FocusRow, type FormatChoice, FormatSelector, type FormatSelectorOptions, type GoogleOptions, GoogleProvider, type GoogleThinkingConfig, type HandoffFact, HeuristicEstimator, InMemoryCacheBreakpointStore, InMemoryExactCache, InMemorySemanticCache, InMemoryToolSetFreezer, type JsonSchema, LOCAL_EMBEDDING_THRESHOLD, type LangfuseGenerationUpdate, LicenseError, type LicenseGate, type LicenseKeyset, type LocalClassifier, MODELS_BY_ENCODING, type MeasureReport, type MemoryDelta, type MessageRole, MockProvider, type MockProviderOptions, type ModelCascadeMode, type ModelCascadeReport, type ModelCascadeSettings, type NativeSavingsReport, type OpenAICompatibleOptions, OpenAICompatibleProvider, type OutputContract, PRICES_AS_OF, PRICE_CATALOG, type Policy, type PriceLeverSaving, type PriceLeverSettings, type ProtectedSpan, type Provider, type ProviderRequest, type ProviderResponse, type ProviderStreamEvent, type ProviderStreamOptions, type ProviderToolSearch, type ProviderToolSearchUsage, type ProviderUsage, QUALITY_STATUSES, type QualityCacheMatch, type QualityCascadeCheck, type QualityLedger, type QualityRepairCheck, type QualitySchemaCheck, type QualityStatus, type QualityTruncationCheck, type ReasoningEffort, type ReferenceCredit, type ReferenceRejection, type RequestContext, RequestTimeoutError, type Reranker, ResolverRegistry, type ResolverResult, type RetrievalOptions, type RetrievalOutcome, type RetrievalPlan, type Retriever, type RouteDecision, type SavingsAdapterLevers, type SavingsAdapterOptions, type SavingsAttributeOptions, type SavingsBatches, type SavingsCallOptions, type SavingsLayer, type SavingsLayerConfig, type SavingsRecord, type SavingsReport, type SavingsReportForExport, type SavingsRequest, type SavingsRequestOptions, type SavingsResult, type SavingsSeriesOptions, type SavingsSeriesRow, type SavingsSpanAttributes, type SavingsSpanOptions, type SavingsStream, type SavingsStreamEvent, type SavingsStreamOptions, type SavingsToggles, type SavingsTracingOptions, SchemaRegistry, type SemanticCacheOptions, SemanticCacheSafety, type SemanticCacheStore, type SemanticCalibrationBound, type SemanticCalibrationIdentity, SemanticCalibrationJournal, type SemanticCalibrationOptions, type SemanticCalibrationSegmentState, type SemanticCalibrationState, type SemanticCalibrationStatus, type SemanticHit, type SemanticLookup, type SemanticProbe, type SemanticRecord, type SemanticVerifier, type SerializationFormat, type Serializer, Singleflight, type SpanAttributeValue, type SpanLike, type SpanStartOptionsLike, StreamAbortedError, type StreamingReport, type TokenEstimator, TokenizerRegistry, type ToolCallRequest, type ToolDescriptor, type ToolOutputCompaction, type ToolOutputCompactionSettings, type ToolRisk, type ToolSearchMode, type ToolSearchOptions, type ToolSelection, ToolSelector, type ToolSelectorOptions, type ToolSetFreezer, type TraceEvent, type TracerLike, type TracerProviderLike, type ValidationError, type ValidationResult, type VerificationCheck, type VerificationExclusion, type VerificationOutcome, type VerificationReport, type VerificationSettings, type VerificationSide, type VerificationStratum, arithmeticResolver, calibrateSemanticFloor, compactRowsSerializer, compactToolOutput, createCompactRowsSerializer, createDefaultResolvers, createHandoff, createMockProvider, createSavingsLayer, csvSerializer, decodeCompactRows, defaultTokenizerRegistry, emptyState, encodeCompactRows, heuristicEstimator, instrumentSavingsLayer, jsonCompactSerializer, listPrice, localEmbedding, mergeDelta, noopCompressor, providerNativeBackend, recordSavingsOnSpan, registerExactTokenizer, renderState, savingsSpanAttributes, toCsv, toFocusRows, toLangfuseGeneration, toSavingsSeries, validateAgainstSchema };
|