@bivelio/savings-layer 0.4.32 → 0.4.34
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/LICENSE +22 -12
- package/README.md +150 -39
- package/dist/adapters/anthropic-sdk.js +72 -71
- package/dist/gateway/bin.js +902 -864
- package/dist/index.d.ts +611 -105
- package/dist/index.js +1604 -1493
- package/package.json +3 -2
package/LICENSE
CHANGED
|
@@ -10,18 +10,23 @@ not install or use the Software.
|
|
|
10
10
|
|
|
11
11
|
1. LICENSE GRANT. Subject to your continuous compliance with this Agreement and to
|
|
12
12
|
an active BiVelio subscription (paid, or granted by BiVelio at no charge, for
|
|
13
|
-
example through a promotional code
|
|
14
|
-
("License Key"), BiVelio grants you a limited, non-exclusive,
|
|
15
|
-
non-sublicensable, revocable license to install and use the
|
|
16
|
-
published, solely for your own internal business purposes, and only
|
|
17
|
-
number of seats covered by your subscription. The Software is licensed,
|
|
13
|
+
example through a promotional code, a pilot, or the free Measure license) with a
|
|
14
|
+
valid license key ("License Key"), BiVelio grants you a limited, non-exclusive,
|
|
15
|
+
non-transferable, non-sublicensable, revocable license to install and use the
|
|
16
|
+
Software, as published, solely for your own internal business purposes, and only
|
|
17
|
+
for the number of seats covered by your subscription. The Software is licensed,
|
|
18
|
+
not sold. Under a Measure license the Software runs in shadow mode only: it
|
|
19
|
+
forwards your requests to your provider unchanged, applies no optimization, and
|
|
20
|
+
reports estimated savings that are never billed.
|
|
18
21
|
|
|
19
22
|
2. LICENSE KEY; FAIL-CLOSED OPERATION. The Software requires a valid, active License
|
|
20
23
|
Key and does not operate without one. Expiry, non-payment, revocation, suspension,
|
|
21
24
|
or termination of your subscription immediately ends the license granted in
|
|
22
|
-
Section 1, and the Software will cease to function
|
|
23
|
-
|
|
24
|
-
|
|
25
|
+
Section 1, and the Software will cease to function; when a paid subscription
|
|
26
|
+
ends, BiVelio may instead grant you a Measure license, under which the Software
|
|
27
|
+
continues in shadow mode only. You will not circumvent, disable, remove, or tamper
|
|
28
|
+
with the Software's licensing, metering, or revocation mechanisms, nor use the
|
|
29
|
+
Software after your license has ended.
|
|
25
30
|
|
|
26
31
|
3. RESTRICTIONS. Except as expressly permitted above, you will not: (a) distribute,
|
|
27
32
|
publish, resell, rent, lease, lend, offer as a hosted or managed service, or
|
|
@@ -74,8 +79,11 @@ not install or use the Software.
|
|
|
74
79
|
(iv) cost figures computed by the Software from its price catalogue (or from
|
|
75
80
|
a cost model you supply): baseline cost, actual cost, the net-saving
|
|
76
81
|
ratio, the basis of the figures (measured, reference, estimated, shadow,
|
|
77
|
-
or unmeasured),
|
|
78
|
-
input tokens your provider reported as cleared from context
|
|
82
|
+
or unmeasured), the value, at the provider's cache-read price, of
|
|
83
|
+
input tokens your provider reported as cleared from context, and the
|
|
84
|
+
discount your provider's own prompt cache gave the request and the
|
|
85
|
+
cache-write premium it billed, which are shown apart and never used to
|
|
86
|
+
calculate the fees in Section 6.
|
|
79
87
|
Each batch of records also carries the Software's surface and version (for
|
|
80
88
|
example, "gateway/0.4.23"). BiVelio's ingest rejects any record that carries a
|
|
81
89
|
field not listed above. Separately, the Software fetches from BiVelio its current
|
|
@@ -102,8 +110,10 @@ not install or use the Software.
|
|
|
102
110
|
6. FEES. Your subscription is subject to BiVelio's pricing as published at
|
|
103
111
|
https://savings.bivelio.com/pricing (the "Published Pricing") for each billing
|
|
104
112
|
period:
|
|
105
|
-
(a) a subscription fee per seat
|
|
106
|
-
|
|
113
|
+
(a) a subscription fee, per seat or for a plan that includes a number of seats,
|
|
114
|
+
payable whether or not the Software produces any saving; and (b) only if your
|
|
115
|
+
subscription is on a plan that the Published Pricing lists as carrying a
|
|
116
|
+
commission, a commission, at tiered marginal rates, charged only on measured
|
|
107
117
|
savings, meaning savings whose basis is labelled "measured" or "reference" in
|
|
108
118
|
your usage records (Section 5(b)(iv)). A "measured" saving is established from
|
|
109
119
|
your LLM provider's own usage counter for that request; a "reference" saving, for
|
package/README.md
CHANGED
|
@@ -19,21 +19,19 @@
|
|
|
19
19
|
Every request your app sends to an AI is paid for by the text going in and out.
|
|
20
20
|
**Savings** sits on that path and lets through only what's needed — skipping calls
|
|
21
21
|
that don't have to happen, and trimming what does — so you pay for less. It is not a
|
|
22
|
-
promise you take on faith: **every saving we
|
|
22
|
+
promise you take on faith: **every saving we report is measured with your provider's
|
|
23
23
|
own usage counter, the one your bill is built from** (for a repeat served without
|
|
24
24
|
calling the provider, the counter of the identical call that ran), and every saving —
|
|
25
25
|
measured or estimated, and labelled as such — is handed back to you request by request.
|
|
26
26
|
|
|
27
|
-
**
|
|
28
|
-
|
|
29
|
-
|
|
30
|
-
|
|
31
|
-
|
|
32
|
-
|
|
33
|
-
|
|
34
|
-
|
|
35
|
-
seat fee is separate). A month with no measured saving is zero commission. And your
|
|
36
|
-
prompts and completions **never reach BiVelio**: what BiVelio receives is per‑request
|
|
27
|
+
**Measure for free, then pay per seat — with no commission on your savings.** The
|
|
28
|
+
free **Measure** license (no card) runs the layer in shadow mode on your own traffic and
|
|
29
|
+
shows what it would have saved, valued with your provider's counter. To turn the levers
|
|
30
|
+
on, **Pro** is **€15 / seat / month** or **€150 / seat / year**, and **Team** is **€129 / month
|
|
31
|
+
with 10 seats included** (€12 per extra seat; €1,290 / year, €120 per extra seat), EUR,
|
|
32
|
+
VAT not included — a fixed fee, paid whether or not you save, and no commission on what
|
|
33
|
+
you save. Enterprise (more seats, your own contract or invoicing terms) is by contract.
|
|
34
|
+
And your prompts and completions **never reach BiVelio**: what BiVelio receives is per‑request
|
|
37
35
|
usage — token counts, cost, the model id and opaque ids — never your content
|
|
38
36
|
([LICENSE](./LICENSE) §5 lists every field; [savings.bivelio.com/privacy](https://savings.bivelio.com/privacy)
|
|
39
37
|
explains each one and how long it is kept).
|
|
@@ -46,14 +44,16 @@ picks the cheapest execution that still meets your quality bar and hands you an
|
|
|
46
44
|
auditable `SavingsReport` — the same numbers your dashboard reads. Traffic still
|
|
47
45
|
goes straight from your app to your provider; the SDK is never a middleman server.
|
|
48
46
|
|
|
49
|
-
> **Commercial software — © 2026 BiVelio Inc.** Free to install, **paid to
|
|
50
|
-
> BV‑SALA is fail‑closed and executes only with an active BiVelio license key.
|
|
47
|
+
> **Commercial software — © 2026 BiVelio Inc.** Free to install, **paid to save**:
|
|
48
|
+
> BV‑SALA is fail‑closed and executes only with an active BiVelio license key. The
|
|
49
|
+
> free **Measure** license (no card) runs it in shadow mode: nothing you send is
|
|
50
|
+
> changed, nothing is served from cache, and you see what it would have saved.
|
|
51
51
|
> The license signature is verified **offline**; with `licenseServerUrl` set, the SDK
|
|
52
52
|
> also calls `savings.bivelio.com` for the revocation list, license renewal and usage
|
|
53
53
|
> metering. **Your prompts, completions and provider keys never reach BiVelio** — what
|
|
54
54
|
> travels is per‑request usage: token counts, cost, the model id, a request id and
|
|
55
55
|
> opaque seat and session ids (the exact list is in [LICENSE](./LICENSE) §5). Use is
|
|
56
|
-
> governed by the LICENSE, which also sets out the
|
|
56
|
+
> governed by the LICENSE, which also sets out the fees, and the
|
|
57
57
|
> BiVelio Terms. Not a wrapper around any third‑party library.
|
|
58
58
|
|
|
59
59
|
## Where it installs
|
|
@@ -119,6 +119,12 @@ bvsala gateway --upstream openai --port 8403 # one gateway per upstream
|
|
|
119
119
|
bvsala gateway # Anthropic, the default, on 8402
|
|
120
120
|
```
|
|
121
121
|
|
|
122
|
+
Flags and environment variables are in English (`--port`, `--project`, `--force`,
|
|
123
|
+
`BVSALA_GATEWAY_EFFORT`, `BVSALA_GATEWAY_SIBLING_MODEL`). The Spanish spellings from
|
|
124
|
+
earlier versions (`--puerto`, `--proyecto`, `--forzar`, `BVSALA_GATEWAY_ESFUERZO`,
|
|
125
|
+
`BVSALA_GATEWAY_MODELO_HERMANO`) keep working until 0.5 and print a deprecation
|
|
126
|
+
notice; `BVSALA_SILENCE_DEPRECATIONS=1` silences it. `bvsala --help` lists them all.
|
|
127
|
+
|
|
122
128
|
```python
|
|
123
129
|
from openai import OpenAI
|
|
124
130
|
client = OpenAI(base_url="http://127.0.0.1:8403/v1") # WITH /v1
|
|
@@ -159,8 +165,8 @@ provider, and a request without it is not changed at all:
|
|
|
159
165
|
What you can check before paying is what we have measured: the published benchmark,
|
|
160
166
|
trial by trial, with the raw CSV linked, at
|
|
161
167
|
**[savings.bivelio.com/en/how-it-works](https://savings.bivelio.com/en/how-it-works)**,
|
|
162
|
-
and the
|
|
163
|
-
|
|
168
|
+
and the free **Measure** license, which runs the layer in shadow mode on your own
|
|
169
|
+
traffic and shows what it would have saved, valued with your provider's counter. The verifier that replays the optimizer on your own data
|
|
164
170
|
lives in your dashboard once you sign in.
|
|
165
171
|
|
|
166
172
|
## Set it up in one command
|
|
@@ -195,7 +201,8 @@ saving where the layer removed something is reported as `estimated`, and estimat
|
|
|
195
201
|
savings are never billed — the SDK warns once when the tokenizer is missing.
|
|
196
202
|
|
|
197
203
|
**2 · The license** — from your [dashboard](https://savings.bivelio.com/dashboard/install)
|
|
198
|
-
(
|
|
204
|
+
(free **Measure** license, no card; **Pro** €15 / seat / month or €150 / seat / year;
|
|
205
|
+
**Team** €129 / month with 10 seats included — EUR)
|
|
199
206
|
|
|
200
207
|
```bash
|
|
201
208
|
export BIVELIO_LICENSE_KEY=… # your license key
|
|
@@ -210,6 +217,7 @@ import { createSavingsLayer, OpenAICompatibleProvider } from "@bivelio/savings-l
|
|
|
210
217
|
|
|
211
218
|
// Your provider. LLM traffic goes straight to it — the SDK never proxies it.
|
|
212
219
|
// Any OpenAI-compatible endpoint works: OpenAI, Azure, vLLM, SGLang, a gateway.
|
|
220
|
+
// For Gemini, use the native `GoogleProvider` instead (see "Gemini: the native adapter").
|
|
213
221
|
// Model ids are qualified (`openai/gpt-4o-mini`); the prefix is stripped on the
|
|
214
222
|
// wire for OpenAI, Azure and Google's Gemini API and kept for gateways — see
|
|
215
223
|
// `stripProviderPrefix`.
|
|
@@ -232,8 +240,8 @@ const { text, report } = await layer.generate({
|
|
|
232
240
|
|
|
233
241
|
console.log(text); // "8" (computed locally — no LLM call)
|
|
234
242
|
console.log(report.calls.avoidedLlm); // → 1: a whole call you never paid for
|
|
235
|
-
console.log(report.cost.
|
|
236
|
-
console.log(report.cost.
|
|
243
|
+
console.log(report.cost.netSavingsRatio); // → 0..1 ratio of the baseline you did not pay (1 here)
|
|
244
|
+
console.log(report.cost.netSavingsUsd); // → the same saving in USD (baselineEstimate − actual)
|
|
237
245
|
console.log(report.cost.basis); // → "estimated": nothing ran, so there is no provider counter
|
|
238
246
|
```
|
|
239
247
|
|
|
@@ -260,8 +268,8 @@ never do.
|
|
|
260
268
|
A repeat is valued by reference only when the call that ran reached the provider's
|
|
261
269
|
own API (`https://api.openai.com`, `https://api.anthropic.com` or
|
|
262
270
|
`https://generativelanguage.googleapis.com`), because the list price it is valued at
|
|
263
|
-
is the invoice only there. When `OpenAICompatibleProvider
|
|
264
|
-
points anywhere else — a proxy such as LiteLLM, a gateway, Azure, Bedrock, Vertex —
|
|
271
|
+
is the invoice only there. When `OpenAICompatibleProvider`, `AnthropicProvider` or
|
|
272
|
+
`GoogleProvider` points anywhere else — a proxy such as LiteLLM, a gateway, Azure, Bedrock, Vertex —
|
|
265
273
|
or when the Anthropic SDK or LangChain adapter reads that its client does, that call
|
|
266
274
|
leaves no reference: its repeats stay `estimated`, and the `reference` trace of the
|
|
267
275
|
call that ran says `unofficial-origin`. When the layer cannot tell where the call
|
|
@@ -277,7 +285,7 @@ tokens) with `stop_details.category` set to `cyber`, `general_harms` or `null`.
|
|
|
277
285
|
documents these as
|
|
278
286
|
[not billed](https://platform.claude.com/docs/en/build-with-claude/refusals-and-fallback#how-refusals-are-billed).
|
|
279
287
|
Its tokens still appear in `report.tokens`. `cost.actual`, `cost.baselineEstimate` and
|
|
280
|
-
`cost.
|
|
288
|
+
`cost.netSavingsRatio` are all 0, so the request credits no saving: not measured, not
|
|
281
289
|
estimated and not by reference. A concurrent identical request served from it credits
|
|
282
290
|
nothing either. The same rule applies in the gateway. Any other refusal is priced from
|
|
283
291
|
its `usage` as usual. That covers `bio`, `frontier_llm` and `reasoning_extraction`, a
|
|
@@ -530,6 +538,32 @@ ledgers** that are never summed into one flattering figure:
|
|
|
530
538
|
| `compressed_tokens` | information kept, but encoded with fewer tokens |
|
|
531
539
|
| `provider_cached_tokens` | tokens still in the prompt, billed/processed more cheaply |
|
|
532
540
|
|
|
541
|
+
**Your provider's own cache discount, shown apart and never billed.** With prompt
|
|
542
|
+
caching, most of what you save often comes from your provider's prefix cache
|
|
543
|
+
(Anthropic `cache_control`, OpenAI's automatic cache, Gemini's implicit cache), not
|
|
544
|
+
from the layer. `cost` prices cached tokens at the cached rate on both sides, so that
|
|
545
|
+
discount is never credited as the layer's saving. `report.nativeSavings` shows it on
|
|
546
|
+
its own, from your provider's counter at list price, so the report adds up to your
|
|
547
|
+
invoice:
|
|
548
|
+
|
|
549
|
+
```ts
|
|
550
|
+
report.nativeSavings;
|
|
551
|
+
// {
|
|
552
|
+
// provider: "anthropic", mechanism: "explicit", layerMarked: true,
|
|
553
|
+
// cacheReadTokens: 40000, cacheWriteTokens: 3000, cacheWrite1hTokens: 0,
|
|
554
|
+
// inputPer1k: 0.003, cachedInputPer1k: 0.0003, writeMultiplier: 1.25, write1hMultiplier: 2,
|
|
555
|
+
// readDiscountUsd: 0.108, // reads × (input − cached-input rate)
|
|
556
|
+
// writePremiumUsd: 0.00225, // writes × input × (multiplier − 1)
|
|
557
|
+
// netUsd: 0.10575, // can be negative: a cold write pays before any read
|
|
558
|
+
// basis: "measured",
|
|
559
|
+
// }
|
|
560
|
+
```
|
|
561
|
+
|
|
562
|
+
It is present only when a call ran and the provider reported cache reads or writes;
|
|
563
|
+
a cache hit or a coalesced request never carries it. It is not in `cost.netSavings`,
|
|
564
|
+
not in `cost.baselineEstimate`, and never in the commission base. Your dashboard shows
|
|
565
|
+
it as its own card, **Native savings unlocked**, next to the credited savings.
|
|
566
|
+
|
|
533
567
|
**Quality, kept apart from money.** `report.qualityLedger` records how the answer you
|
|
534
568
|
received earned its place: whether it came from your provider, a replay, or the answer
|
|
535
569
|
to a reworded question (with its similarity score); whether it was checked against your
|
|
@@ -616,7 +650,7 @@ changes the answer, and what they are worth is reported next to the four ledgers
|
|
|
616
650
|
never folded into `cost.*`, never part of any commission.
|
|
617
651
|
|
|
618
652
|
**`savings.deferrable`** asks for the provider's Flex tier: OpenAI, and Gemini through
|
|
619
|
-
its OpenAI‑compatible endpoint. Anthropic has no cheaper synchronous tier: its
|
|
653
|
+
`GoogleProvider` (`serviceTier: "flex"`) or its OpenAI‑compatible endpoint. Anthropic has no cheaper synchronous tier: its
|
|
620
654
|
discount lives only in Message Batches, which `layer.batches` (below) sends to.
|
|
621
655
|
Bedrock and DeepSeek off‑peak pricing are not supported.
|
|
622
656
|
|
|
@@ -801,7 +835,7 @@ Calibrated reuse and `cacheBreakpoints: "auto"` learn from your traffic, and by
|
|
|
801
835
|
what they learn lives in the process. On serverless platforms (Vercel Functions, AWS
|
|
802
836
|
Lambda) an instance may serve one request or live a few minutes, so neither would ever
|
|
803
837
|
get past its starting point. Give each one somewhere to keep it. Giving the layer a
|
|
804
|
-
`
|
|
838
|
+
`semanticCacheStore` only says where reworded questions are kept; `semanticCacheDefault: true`
|
|
805
839
|
is what switches reuse on for every request (a request's own `savings.semanticCache`
|
|
806
840
|
still wins), and without it, or `savings.semanticCache` on each request, the cache
|
|
807
841
|
stays empty and learns nothing:
|
|
@@ -823,7 +857,7 @@ async function serverlessLayer(kv: SharedKeyValue) {
|
|
|
823
857
|
const layer = createSavingsLayer({
|
|
824
858
|
provider,
|
|
825
859
|
licenseKey: process.env.BIVELIO_LICENSE_KEY,
|
|
826
|
-
semanticCache,
|
|
860
|
+
semanticCacheStore: semanticCache,
|
|
827
861
|
// Passing the cache does not switch it on; this asks for it on every request.
|
|
828
862
|
semanticCacheDefault: true,
|
|
829
863
|
cacheBreakpoints: "auto",
|
|
@@ -897,6 +931,60 @@ console.log(data.answer); // 8
|
|
|
897
931
|
|
|
898
932
|
</details>
|
|
899
933
|
|
|
934
|
+
<details>
|
|
935
|
+
<summary><b>Gemini: the native adapter</b></summary>
|
|
936
|
+
|
|
937
|
+
For Gemini, use `GoogleProvider`: it speaks Gemini's own API (`generateContent` and
|
|
938
|
+
`streamGenerateContent`), where what the layer measures is documented. Keep
|
|
939
|
+
`OpenAICompatibleProvider` for every other provider.
|
|
940
|
+
|
|
941
|
+
```ts
|
|
942
|
+
import { GoogleProvider } from "@bivelio/savings-layer";
|
|
943
|
+
|
|
944
|
+
const gemini = createSavingsLayer({
|
|
945
|
+
provider: new GoogleProvider({ apiKey: process.env.GEMINI_API_KEY, requireUsage: true }),
|
|
946
|
+
licenseKey: process.env.BIVELIO_LICENSE_KEY,
|
|
947
|
+
});
|
|
948
|
+
|
|
949
|
+
const { text: answer, report: geminiReport } = await gemini.generate({
|
|
950
|
+
model: "google/gemini-3.8-flash", // the `google/` prefix never travels
|
|
951
|
+
messages: [
|
|
952
|
+
{ role: "system", content: "Answer in one sentence." },
|
|
953
|
+
{ role: "user", content: "Why is the sky blue?" },
|
|
954
|
+
],
|
|
955
|
+
maxOutputTokens: 400,
|
|
956
|
+
reasoningEffort: "low", // → thinkingLevel "LOW" (Gemini 3) or a thinking budget (2.5)
|
|
957
|
+
});
|
|
958
|
+
console.log(answer, geminiReport.cost.actual);
|
|
959
|
+
```
|
|
960
|
+
|
|
961
|
+
What the native API gives the layer that the OpenAI-compatible endpoint hides:
|
|
962
|
+
|
|
963
|
+
- **Thinking is billed output.** Output is `candidatesTokenCount + thoughtsTokenCount`
|
|
964
|
+
(Google bills thinking as output) and the thinking is reported as reasoning. Thought
|
|
965
|
+
summaries (`thinkingConfig: { includeThoughts: true }` on the adapter) are never part
|
|
966
|
+
of `text`.
|
|
967
|
+
- **Refusals with their reason.** A blocked prompt (`promptFeedback.blockReason`) or a
|
|
968
|
+
filtered answer (`finishReason` `SAFETY`, `PROHIBITED_CONTENT`, `BLOCKLIST`, …) is
|
|
969
|
+
`finishReason: "content_filter"` with the native reason in `finishReasonDetail`:
|
|
970
|
+
measured from its usage, never cached, never credited.
|
|
971
|
+
- **Tools.** Calls come back with their `id` and `thoughtSignature`; send the assistant
|
|
972
|
+
turn back with its `toolCalls` as they came (as in the tool loop above) and the
|
|
973
|
+
signature travels back untouched. The results of one turn go back together.
|
|
974
|
+
- **Implicit caching.** Gemini 2.5 and later cache repeated prefixes on their own; the
|
|
975
|
+
cached part of the prompt (`cachedContentTokenCount`) is reported as provider-cached
|
|
976
|
+
tokens. Explicit context caching (`cachedContents`) is not used: it bills storage by
|
|
977
|
+
the hour, which a per-request report cannot price.
|
|
978
|
+
- **Output contract.** Your schema travels as written in `responseJsonSchema`. Google
|
|
979
|
+
does not document that every JSON Schema feature is enforced, so a repeat of a request
|
|
980
|
+
with a schema is not valued by reference.
|
|
981
|
+
- **Thinking level.** `reasoningEffort` follows Google's published mapping. A model that
|
|
982
|
+
rejects a level (`gemini-3.8-flash` answers 400 to `MINIMAL`) is asked again once at
|
|
983
|
+
`LOW`, and remembered; the rejected request was not billed. Thinking cannot be turned
|
|
984
|
+
off on Gemini 3 or 2.5 Pro, so `"none"` there is the lowest level, with a warning.
|
|
985
|
+
|
|
986
|
+
</details>
|
|
987
|
+
|
|
900
988
|
<details>
|
|
901
989
|
<summary><b>Output cap — with or without a contract</b></summary>
|
|
902
990
|
|
|
@@ -934,7 +1022,8 @@ own API, where its 400 is the answer. Gemini leaves its thinking tokens out of
|
|
|
934
1022
|
as reasoning); on OpenAI's API `completion_tokens` already include reasoning and nothing
|
|
935
1023
|
is added. Gemini stops a filtered answer with `finish_reason: "content_filter: OTHER"`
|
|
936
1024
|
(or another reason): it is `finishReason: "content_filter"` (the adapter keeps the
|
|
937
|
-
reason
|
|
1025
|
+
reason, which reaches you as `result.finishReasonDetail` and is named in the report's
|
|
1026
|
+
cache trace; an Anthropic refusal brings its `stop_details.category` there too), and like every
|
|
938
1027
|
provider refusal it is measured, never
|
|
939
1028
|
cached and never credited. Google does not document that such refusals are free, so
|
|
940
1029
|
they are costed from their `usage` like any call.
|
|
@@ -997,7 +1086,10 @@ console.log(res.text);
|
|
|
997
1086
|
the SDK warns once.
|
|
998
1087
|
- **Errors.** When a tool fails, put the error in `content` and set `isError: true`.
|
|
999
1088
|
Anthropic receives it as `is_error`; providers without such a flag get the text.
|
|
1000
|
-
- **Gemini 3's thought signature.**
|
|
1089
|
+
- **Gemini 3's thought signature.** With `GoogleProvider` it travels in each call's
|
|
1090
|
+
`thoughtSignature` and comes back in `call.extraContent` in the same shape as below,
|
|
1091
|
+
so a conversation can move between the two Gemini adapters. Through its
|
|
1092
|
+
OpenAI-compatible endpoint Gemini
|
|
1001
1093
|
returns each call with `extra_content.google.thought_signature` and rejects the next
|
|
1002
1094
|
turn with a 400 if the call comes back without it. It arrives in `call.extraContent`;
|
|
1003
1095
|
pushing `res.toolCalls` back as they came (as above) sends it back untouched. It is
|
|
@@ -1020,10 +1112,11 @@ difference.
|
|
|
1020
1112
|
**Which tools reach the model.** The layer never cuts a catalogue it cannot rank. If
|
|
1021
1113
|
your tools declare no `intents` and no `risk` (the usual shape of MCP and OpenAI tool
|
|
1022
1114
|
lists), or you pass `savings: { router: false }`, every tool goes to the provider
|
|
1023
|
-
exactly as you sent it, and the layer credits nothing for the tool block.
|
|
1024
|
-
|
|
1025
|
-
|
|
1026
|
-
|
|
1115
|
+
exactly as you sent it, and the layer credits nothing for the tool block. When your
|
|
1116
|
+
tools declare `intents`/`risk`, the selector holds back write tools from a request
|
|
1117
|
+
that is not an action and, since 0.4.33, sends every other tool: it prunes by
|
|
1118
|
+
relevance only if you ask for it with `new ToolSelector({ maxTools })` (up to 0.4.32
|
|
1119
|
+
that was the default, with 8). Whatever it leaves out, the model cannot call in that
|
|
1027
1120
|
request: the report's `tool-selector` trace lists it, the tokens count as eliminated
|
|
1028
1121
|
input, and the SDK warns once per process (`silenceWarnings` turns the warning off).
|
|
1029
1122
|
|
|
@@ -1103,7 +1196,7 @@ console.log(tracedReport.cost.basis); // the span carries the same figures
|
|
|
1103
1196
|
- **Money is kept apart by basis.** The money saved is
|
|
1104
1197
|
`com.bivelio.cost.saved.measured`, `.reference` or `.estimated` — one key per span,
|
|
1105
1198
|
the one of its basis — so a sum over one key never mixes measured and estimated
|
|
1106
|
-
money; `unmeasured` spans carry counts and no money. `report.cost.
|
|
1199
|
+
money; `unmeasured` spans carry counts and no money. `report.cost.netSavingsRatio` is a
|
|
1107
1200
|
ratio and goes out as `com.bivelio.cost.net_savings_ratio`; the money is
|
|
1108
1201
|
`baselineEstimate − actual`, and that is what `saved.*` holds.
|
|
1109
1202
|
- **None of this talks to BiVelio.** Spans go to your exporter; files stay where you
|
|
@@ -1138,7 +1231,7 @@ console.log(tracedReport.cost.basis); // the span carries the same figures
|
|
|
1138
1231
|
| `gen_ai.usage.input_tokens`, `.output_tokens`, `.cache_read.input_tokens` | your provider's counter; absent when it reported no usage |
|
|
1139
1232
|
| `com.bivelio.cost.basis`, `.currency`, `.baseline`, `.actual` | where the money comes from, its currency (USD by default), and both sides |
|
|
1140
1233
|
| `com.bivelio.cost.saved.measured` \| `.reference` \| `.estimated` | the money saved, in the key of its basis only |
|
|
1141
|
-
| `com.bivelio.cost.net_savings_ratio` | `cost.
|
|
1234
|
+
| `com.bivelio.cost.net_savings_ratio` | `cost.netSavingsRatio`, a 0..1 ratio — not money |
|
|
1142
1235
|
| `com.bivelio.calls.*`, `com.bivelio.tokens.*` | the four ledgers: avoided calls, eliminated, compressed and provider‑cached tokens |
|
|
1143
1236
|
| `com.bivelio.request_id`, `.model.id`, `.cache.result`, `.quality.status` | ids and outcomes |
|
|
1144
1237
|
|
|
@@ -1253,12 +1346,30 @@ BVSALA_SEAT_ID=worker-03 # stable per replica; [A-Za-z0-9._:/-], max 128 chars
|
|
|
1253
1346
|
|
|
1254
1347
|
## Licensing & privacy
|
|
1255
1348
|
|
|
1256
|
-
BV‑SALA is commercial software
|
|
1257
|
-
|
|
1258
|
-
|
|
1259
|
-
[LICENSE](./LICENSE) §6 and in
|
|
1260
|
-
|
|
1261
|
-
|
|
1349
|
+
BV‑SALA is commercial software. The plans are **Measure** (free), **Pro** (per seat),
|
|
1350
|
+
**Team** (a number of seats included, plus each extra seat) and **Enterprise** (by
|
|
1351
|
+
contract); Pro and Team are paid whether or not you save and carry **no commission** on
|
|
1352
|
+
your savings. The prices are at the top of this page, in [LICENSE](./LICENSE) §6 and in
|
|
1353
|
+
the Published Pricing it refers to,
|
|
1354
|
+
**[savings.bivelio.com/pricing](https://savings.bivelio.com/pricing)**. No trial. There is one free license,
|
|
1355
|
+
**Measure** — no card, one organization per verified account, up to 5 seats and 20,000
|
|
1356
|
+
requests a day: the layer runs in **shadow mode**. Every request goes to your provider
|
|
1357
|
+
exactly as you built it, nothing is served from any cache, no lever is applied, and
|
|
1358
|
+
`report.measure.wouldHaveSaved` (with `basis: "shadow"`) says what the exact cache and
|
|
1359
|
+
in‑flight coalescing would have saved on that request, valued with your provider's own
|
|
1360
|
+
counter — a lower bound, never part of any commission. The levers need a paid plan: once
|
|
1361
|
+
you upgrade, a layer with `serviceAccountKey` and `licenseServerUrl` picks up the paid
|
|
1362
|
+
license by itself and turns them on without a restart.
|
|
1363
|
+
|
|
1364
|
+
The earlier plan — a lower per‑seat fee plus a tiered commission on measured savings
|
|
1365
|
+
only — is **closed to new customers**; a subscription already on it keeps it unchanged.
|
|
1366
|
+
On that plan your first month carries no commission (the first calendar month, UTC, in
|
|
1367
|
+
which your organization reports usage); after that the commission is charged on the
|
|
1368
|
+
measured saving only — never from estimates — at a tiered rate from 10 % down to 4 %, so
|
|
1369
|
+
you keep at least 90 % of every euro saved (before rounding to the cent; the seat fee is
|
|
1370
|
+
separate).
|
|
1371
|
+
|
|
1372
|
+
The license is a signed token your SDK and the
|
|
1262
1373
|
gateway verify **offline**; the managed control plane and the revocation kill‑switch
|
|
1263
1374
|
are served from BiVelio. Your LLM traffic goes **straight to your provider** —
|
|
1264
1375
|
BiVelio receives per‑request usage (counts, cost, the model id and opaque ids), never
|