@bivelio/savings-layer 0.4.24 → 0.4.25

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/LICENSE CHANGED
@@ -9,11 +9,12 @@ copying, or using the Software, you agree to this Agreement. If you do not agree
9
9
  not install or use the Software.
10
10
 
11
11
  1. LICENSE GRANT. Subject to your continuous compliance with this Agreement and to
12
- an active, paid BiVelio subscription with a valid license key ("License Key"),
13
- BiVelio grants you a limited, non-exclusive, non-transferable, non-sublicensable,
14
- revocable license to install and use the Software, as published, solely for your
15
- own internal business purposes, and only for the number of seats covered by your
16
- subscription. The Software is licensed, not sold.
12
+ an active BiVelio subscription (paid, or granted by BiVelio at no charge, for
13
+ example through a promotional code or a pilot) with a valid license key
14
+ ("License Key"), BiVelio grants you a limited, non-exclusive, non-transferable,
15
+ non-sublicensable, revocable license to install and use the Software, as
16
+ published, solely for your own internal business purposes, and only for the
17
+ number of seats covered by your subscription. The Software is licensed, not sold.
17
18
 
18
19
  2. LICENSE KEY; FAIL-CLOSED OPERATION. The Software requires a valid, active License
19
20
  Key and does not operate without one. Expiry, non-payment, revocation, suspension,
@@ -25,13 +26,13 @@ not install or use the Software.
25
26
  3. RESTRICTIONS. Except as expressly permitted above, you will not: (a) distribute,
26
27
  publish, resell, rent, lease, lend, offer as a hosted or managed service, or
27
28
  otherwise make the Software available to any third party; (b) sublicense, assign,
28
- or transfer your rights; (c) use the Software for more seats than you have paid
29
- for, or share a License Key across more machines or service identities than your
30
- subscription covers; (d) remove, alter, or obscure any proprietary or copyright
31
- notice; (e) reverse engineer, decompile, or disassemble the Software, except to
32
- the limited extent applicable law expressly permits notwithstanding this
33
- restriction; or (f) use the Software to develop, train, or improve a product or
34
- service that competes with the Software.
29
+ or transfer your rights; (c) use the Software for more seats than your
30
+ subscription covers, or share a License Key across more machines or service
31
+ identities than it covers; (d) remove, alter, or obscure any proprietary or
32
+ copyright notice; (e) reverse engineer, decompile, or disassemble the Software,
33
+ except to the limited extent applicable law expressly permits notwithstanding
34
+ this restriction; or (f) use the Software to develop, train, or improve a product
35
+ or service that competes with the Software.
35
36
 
36
37
  4. OWNERSHIP. The Software is the exclusive property of BiVelio and its licensors and
37
38
  embodies valuable proprietary information and trade secrets. BiVelio retains all
@@ -39,27 +40,110 @@ not install or use the Software.
39
40
  rights therein. No rights are granted to you except as expressly set out in this
40
41
  Agreement, whether by implication, estoppel, or otherwise.
41
42
 
42
- 5. YOUR DATA; PRIVACY. Your prompts, completions, and content are processed within
43
- your own infrastructure and are not transmitted to BiVelio in order to serve a
44
- request. BiVelio's control plane receives only usage counts and cost metadata, as
45
- described in the BiVelio documentation and privacy terms.
43
+ 5. YOUR DATA; PRIVACY.
44
+ (a) What the Software never sends to BiVelio. The Software never transmits to
45
+ BiVelio the content of your requests or responses (including prompts, system
46
+ instructions, messages, tool definitions, tool arguments and results, retrieved
47
+ documents, and completions) or the credentials you use with your LLM provider
48
+ (API keys, OAuth tokens, or other authorization headers). That content and those
49
+ credentials travel only between your systems and the LLM provider you configure.
50
+ (b) What the Software sends to BiVelio. The SDK contacts BiVelio only when you
51
+ configure a license server or metering URL; the local gateway always does. With a
52
+ BiVelio service-account key, the Software sends BiVelio's control plane usage
53
+ records for the requests it meters, each limited to:
54
+ (i) identifiers: a request identifier generated on your side (at random, or by
55
+ your own generator); for a repeated request valued by reference, the
56
+ identifier and timestamp of the earlier request whose provider counter is
57
+ used; a seat identifier, which is either a value you set or an opaque value
58
+ computed on your machine as a keyed hash (HMAC-SHA-256 wherever the
59
+ runtime provides it), keyed with your service-account key, of the
60
+ platform, CPU architecture, runtime executable path, host name,
61
+ operating-system user name, and home directory (those underlying values
62
+ are never sent); for traffic through the local gateway,
63
+ the session and sub-agent identifiers that your coding agent attaches to
64
+ its own requests (for example, Claude Code's session id), only when they
65
+ are plain identifiers; an organization hint; and a flag marking CI or
66
+ ephemeral runners;
67
+ (ii) the identifier of the model that served the request (the model named in
68
+ the request or, where the Software sent the request to another model under
69
+ a setting you enabled, that model; for a self-hosted or Azure deployment,
70
+ the name you gave it) and the time of the request;
71
+ (iii) counts: LLM calls executed and avoided; input and output tokens before
72
+ and after the Software's optimizations, including tokens removed,
73
+ compressed, or read from the provider's cache; and the cache outcome; and
74
+ (iv) cost figures computed by the Software from its price catalogue (or from
75
+ a cost model you supply): baseline cost, actual cost, the net-saving
76
+ ratio, the basis of the figures (measured, reference, estimated, shadow,
77
+ or unmeasured), and the value, at the provider's cache-read price, of
78
+ input tokens your provider reported as cleared from context.
79
+ Each batch of records also carries the Software's surface and version (for
80
+ example, "gateway/0.4.23"). BiVelio's ingest rejects any record that carries a
81
+ field not listed above. Separately, the Software fetches from BiVelio its current
82
+ license token, the signed license revocation list and, for the gateway, your
83
+ organization's feature entitlements; these requests carry your service-account
84
+ key where authentication is required, and no request data. Like any HTTPS
85
+ request, all of the above also reveals ordinary network metadata, such as your
86
+ IP address. If you enable the SDK's optional verification, which is off by
87
+ default, the SDK also sends BiVelio, through a separate channel and for each
88
+ request it samples: a label marking the record as a verification sample; the
89
+ request identifier and time; the model requested and the model that served it;
90
+ the cache outcome; the category of the request and the outcome of the check;
91
+ the input, cached-input, and output token counts and the cost of the served
92
+ request and of the unoptimized copy the SDK sent to your LLM provider; how
93
+ that copy finished and whether it met the response contract; and the per-token
94
+ prices applied; together with daily counts of requests per category and a
95
+ random batch identifier. That channel likewise rejects any field not listed
96
+ here, and none of it is used to calculate the fees in Section 6.
97
+ (c) BiVelio uses this data to operate licensing and revocation, show your
98
+ dashboard, count billable seats, and calculate the fees in Section 6. Each
99
+ field, and how long BiVelio keeps it, is further described at
100
+ https://savings.bivelio.com/privacy.
46
101
 
47
- 6. TERM AND TERMINATION. This Agreement remains in effect while your subscription is
102
+ 6. FEES. Your subscription is subject to BiVelio's pricing as published at
103
+ https://savings.bivelio.com/pricing (the "Published Pricing") for each billing
104
+ period:
105
+ (a) a subscription fee per seat, payable whether or not the Software produces any
106
+ saving; and (b) a commission, at tiered marginal rates, charged only on measured
107
+ savings, meaning savings whose basis is labelled "measured" or "reference" in
108
+ your usage records (Section 5(b)(iv)). A "measured" saving is established from
109
+ your LLM provider's own usage counter for that request; a "reference" saving, for
110
+ a repeated request the Software served without calling your provider, from your
111
+ provider's counter for the identical earlier request that did run. Both are
112
+ valued at the per-token prices the Software applied to that request: those of a
113
+ cost model you configure, where you configure one, and otherwise your provider's
114
+ list prices in the price catalogue included in the Software. Savings labelled
115
+ "estimated", "shadow", or "unmeasured" never carry commission. Traffic the
116
+ Software identifies as a flat-rate provider subscription (a Claude or ChatGPT
117
+ subscription used through the local gateway) is metered but credited no measured
118
+ saving, so it carries no commission. A period with no measured savings carries no
119
+ commission. The first calendar month (UTC) in which your organization reports any
120
+ usage is a calibration period, however little of that month remains, and carries
121
+ no commission. Nor is commission charged for a period that BiVelio closes while
122
+ your account is exempt from it. Your account is exempt if BiVelio grants it an
123
+ exemption and, until a payment is made on the account, if its subscription was
124
+ activated with a promotional code that waives the subscription fee in full for
125
+ its entire term. For a calibration or exempt period the statement still shows the
126
+ commission that would have applied. Each period's commission is
127
+ itemized on a statement in your dashboard, together with the exchange rate
128
+ applied. Rates, tier thresholds, and currency are those of the Published Pricing
129
+ and are not restated here.
130
+
131
+ 7. TERM AND TERMINATION. This Agreement remains in effect while your subscription is
48
132
  active. BiVelio may terminate this Agreement immediately if you breach it. Upon
49
133
  expiry or termination you must cease all use of the Software and destroy your
50
- copies. Sections 2 through 9 survive termination.
134
+ copies. Sections 2 through 10 survive termination.
51
135
 
52
- 7. DISCLAIMER OF WARRANTY. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
136
+ 8. DISCLAIMER OF WARRANTY. THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY
53
137
  KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
54
138
  MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE, AND NONINFRINGEMENT.
55
139
 
56
- 8. LIMITATION OF LIABILITY. TO THE MAXIMUM EXTENT PERMITTED BY LAW, IN NO EVENT SHALL
140
+ 9. LIMITATION OF LIABILITY. TO THE MAXIMUM EXTENT PERMITTED BY LAW, IN NO EVENT SHALL
57
141
  BIVELIO INC. BE LIABLE FOR ANY INDIRECT, INCIDENTAL, SPECIAL, CONSEQUENTIAL, OR
58
142
  EXEMPLARY DAMAGES, OR FOR ANY LOSS OF PROFITS, REVENUE, OR DATA, ARISING FROM OR
59
143
  RELATED TO THE SOFTWARE OR THIS AGREEMENT. BIVELIO'S TOTAL LIABILITY SHALL NOT
60
144
  EXCEED THE FEES YOU PAID FOR THE SOFTWARE IN THE TWELVE MONTHS PRECEDING THE CLAIM.
61
145
 
62
- 9. GENERAL. This Agreement is governed by the laws of the State of Delaware, USA,
146
+ 10. GENERAL. This Agreement is governed by the laws of the State of Delaware, USA,
63
147
  without regard to its conflict-of-laws rules. If any provision is held
64
148
  unenforceable, the remaining provisions remain in effect. This Agreement, together
65
149
  with the applicable BiVelio subscription terms, constitutes the entire agreement
package/README.md CHANGED
@@ -19,17 +19,24 @@
19
19
  Every request your app sends to an AI is paid for by the text going in and out.
20
20
  **Savings** sits on that path and lets through only what's needed — skipping calls
21
21
  that don't have to happen, and trimming what does — so you pay for less. It is not a
22
- promise you take on faith: **every saving we charge on is measured with the provider's
23
- official meter, the same one on your bill**, and every saving — measured or estimated,
24
- and labelled as such — is handed back to you request by request.
25
-
26
- **We only earn when you save.** Your first month carries no commission at all: we
27
- measure everything and show you the statement of what it would have cost. After
28
- that, the commission comes out of the **measured** saving only — never out of your
29
- pocket, and never from estimates: a tiered rate from 10 % down to 4 % of what you
30
- saved, so **you always keep at least 90 % of every euro saved**, and a month with no
31
- saving is zero commission. And your prompts **never leave your infrastructure**:
32
- BiVelio only ever sees counts and cost, never your content.
22
+ promise you take on faith: **every saving we charge on is measured with your provider's
23
+ own usage counter, the one your bill is built from** (for a repeat served without
24
+ calling the provider, the counter of the identical call that ran), and every saving —
25
+ measured or estimated, and labelled as such — is handed back to you request by request.
26
+
27
+ **A small per‑seat fee, and a commission only on savings measured against your
28
+ provider's counter.** The seat fee is paid whether or not you save. Your
29
+ first month carries no commission at all — it is the first calendar month (UTC) in
30
+ which your organization reports usage, so starting late in a month makes it a short one —
31
+ and at its close you see the statement of what the commission would have been. After
32
+ that, the commission is charged on the **measured** saving only — never from estimates:
33
+ a tiered rate from 10 % down to 4 % of what you saved, so **you keep
34
+ at least 90 % of every euro saved** (before the invoice is rounded to the cent; the
35
+ seat fee is separate). A month with no measured saving is zero commission. And your
36
+ prompts and completions **never reach BiVelio**: what BiVelio receives is per‑request
37
+ usage — token counts, cost, the model id and opaque ids — never your content
38
+ ([LICENSE](./LICENSE) §5 lists every field; [savings.bivelio.com/privacy](https://savings.bivelio.com/privacy)
39
+ explains each one and how long it is kept).
33
40
 
34
41
  ### For engineers, in one line
35
42
 
@@ -43,9 +50,11 @@ goes straight from your app to your provider; the SDK is never a middleman serve
43
50
  > BV‑SALA is fail‑closed and executes only with an active BiVelio license key.
44
51
  > The license signature is verified **offline**; with `licenseServerUrl` set, the SDK
45
52
  > also calls `savings.bivelio.com` for the revocation list, license renewal and usage
46
- > metering. **Your prompts never leave your infrastructure** — what travels is token
47
- > counts, cost, the model id and opaque seat/session ids, never content. Use is governed by the [LICENSE](./LICENSE)
48
- > and the BiVelio Terms. Not a wrapper around any third‑party library.
53
+ > metering. **Your prompts, completions and provider keys never reach BiVelio** — what
54
+ > travels is per‑request usage: token counts, cost, the model id, a request id and
55
+ > opaque seat and session ids (the exact list is in [LICENSE](./LICENSE) §5). Use is
56
+ > governed by the LICENSE, which also sets out the seat fee and the commission, and the
57
+ > BiVelio Terms. Not a wrapper around any third‑party library.
49
58
 
50
59
  ## Where it installs
51
60
 
@@ -83,6 +92,20 @@ subscription there is no per-token invoice, so that traffic is metered at
83
92
  list-price equivalent and never carries commission; the per-seat license applies
84
93
  as for any other use.
85
94
 
95
+ **The gateway forwards what your terminal sends.** With no lever armed, your
96
+ provider receives the request body byte for byte and every header your client
97
+ sends (see the limits): `cache_control` and its 1-hour TTL, tool search and
98
+ `tool_reference` blocks, the advisor, every `anthropic-beta` value. An armed lever
99
+ changes only what it declares, and a request it cannot rewrite exactly travels
100
+ untouched. A test suite checks this against a fake upstream on every change
101
+ (`test/gateway-no-empeora.test.ts`; the repository is private, so the linked
102
+ section lists exactly what it checks). The gateway also logs, once per session,
103
+ requests that come out worse than your client asked for: cache markers the
104
+ provider ignored, a 1-hour TTL written at 5 minutes upstream, and tool search off
105
+ (Claude Code turns it off by default behind any proxy, this one included;
106
+ `bvsala claude` turns it back on). What it checks, and its limits:
107
+ [docs/terminales/README.md](./docs/terminales/README.md#what-the-gateway-guarantees-it-forwards-what-your-client-sends).
108
+
86
109
  ### Python and any other language: the local gateway
87
110
 
88
111
  The same gateway sits in front of any SDK that lets you set a base URL, whatever
@@ -228,6 +251,135 @@ still writing cancels its request, and nothing is cached or recorded; once its a
228
251
  is complete, or when it arrives in one piece, nothing is cancelled.
229
252
  `await stream.result` tells you which happened.
230
253
 
254
+ ## Inside your framework
255
+
256
+ Already on an agent framework? The layer plugs in at that framework's own extension
257
+ point, from a subpath of this same package. The four adapters behave alike:
258
+
259
+ - **They answer without calling the model when they can** — a computed answer (the
260
+ arithmetic above), an exact repeat of a request that already ran, an identical
261
+ request already in flight — and otherwise let your framework make exactly the call
262
+ it was going to make. They never rewrite what your framework sends.
263
+ - **Tools are treated as having side effects** unless you list them in
264
+ `readOnlyTools`: a request that offers any other tool is never answered from a
265
+ cache, because the action has to run.
266
+ - **Fail‑closed, like the SDK**: without an active license the call fails with
267
+ `LicenseError` (a framework may wrap it, keeping its name and message) and the
268
+ model is not called. **Put the adapter last**, so nothing sits between it and the
269
+ model.
270
+
271
+ Every model call gets the same `SavingsReport` as with the SDK — its `basis` says
272
+ whether a figure comes from your provider's own counter — and `onReport` receives it
273
+ too.
274
+
275
+ ### Vercel AI SDK
276
+
277
+ `pnpm add @bivelio/savings-layer ai @ai-sdk/openai` — a `LanguageModelMiddleware`
278
+ for `ai` 6 or 7:
279
+
280
+ ```ts
281
+ import { generateText, wrapLanguageModel } from "ai";
282
+ import { openai } from "@ai-sdk/openai";
283
+ import { savingsMiddleware, getSavingsReport } from "@bivelio/savings-layer/ai-sdk";
284
+
285
+ const savings = savingsMiddleware({
286
+ licenseKey: process.env.BIVELIO_LICENSE_KEY,
287
+ licenseServerUrl: "https://savings.bivelio.com",
288
+ serviceAccountKey: process.env.BIVELIO_SERVICE_KEY,
289
+ readOnlyTools: ["get_weather"],
290
+ });
291
+ const model = wrapLanguageModel({ model: openai("gpt-4o-mini"), middleware: savings }); // last in the list
292
+
293
+ const result = await generateText({
294
+ model,
295
+ prompt: "What is 2 + 2 * 3?",
296
+ providerOptions: { bivelio: { context: { sessionId: "user-42" } } },
297
+ });
298
+ console.log(getSavingsReport(result)?.cost.basis); // "estimated": answered without calling the model
299
+ ```
300
+
301
+ For `streamText`, read it with `getSavingsReport({ providerMetadata: await result.providerMetadata })`.
302
+
303
+ ### Mastra
304
+
305
+ `pnpm add @bivelio/savings-layer @mastra/core` — an input processor for
306
+ `@mastra/core` 1.x:
307
+
308
+ ```ts
309
+ import { Agent } from "@mastra/core/agent";
310
+ import { savingsProcessor } from "@bivelio/savings-layer/mastra";
311
+
312
+ const bivelio = savingsProcessor({
313
+ licenseKey: process.env.BIVELIO_LICENSE_KEY,
314
+ licenseServerUrl: "https://savings.bivelio.com",
315
+ serviceAccountKey: process.env.BIVELIO_SERVICE_KEY,
316
+ onReport: (r) => console.log(r.cost.basis),
317
+ });
318
+ const agent = new Agent({
319
+ id: "support",
320
+ name: "Support",
321
+ instructions: "Answer briefly.",
322
+ model: "openai/gpt-4o-mini",
323
+ inputProcessors: [bivelio], // last in the list
324
+ });
325
+ console.log((await agent.generate("What is 2 + 2 * 3?")).text); // "8"
326
+ ```
327
+
328
+ It covers `generate()` and `stream()`, not `generateLegacy()`/`streamLegacy()` or a
329
+ durable agent that resumes in another process; identical requests in flight are not
330
+ shared here. `emitReportPart: true` also streams each report as a
331
+ `data-bivelio-savings-report` part.
332
+
333
+ ### LangChain.js
334
+
335
+ `pnpm add @bivelio/savings-layer langchain @langchain/core` — an agent middleware for
336
+ `createAgent` in `langchain` 1.x:
337
+
338
+ ```ts
339
+ import { createAgent } from "langchain";
340
+ import { savingsAgentMiddleware, getSavingsReportFromMessage } from "@bivelio/savings-layer/langchain";
341
+
342
+ const agentSavings = savingsAgentMiddleware({
343
+ licenseKey: process.env.BIVELIO_LICENSE_KEY,
344
+ licenseServerUrl: "https://savings.bivelio.com",
345
+ serviceAccountKey: process.env.BIVELIO_SERVICE_KEY,
346
+ });
347
+ const lcAgent = createAgent({ model: "openai:gpt-4o-mini", tools: [], middleware: [agentSavings] }); // last
348
+ const state = await lcAgent.invoke({ messages: [{ role: "user", content: "What is 2 + 2 * 3?" }] });
349
+ console.log(getSavingsReportFromMessage(state.messages.at(-1))?.cost.basis);
350
+ ```
351
+
352
+ The model call is measured when it finishes; tokens you stream with
353
+ `streamMode: "messages"` keep coming from the model as usual. A chat model with its
354
+ own `cache` set is reported as `unmeasured`: its counter may be a cached copy.
355
+
356
+ ### Anthropic TypeScript SDK
357
+
358
+ `pnpm add @bivelio/savings-layer @anthropic-ai/sdk` — a client middleware for
359
+ `@anthropic-ai/sdk` 0.103 or later:
360
+
361
+ ```ts
362
+ import Anthropic from "@anthropic-ai/sdk";
363
+ import { anthropicSavings } from "@bivelio/savings-layer/anthropic-sdk";
364
+
365
+ const claudeSavings = anthropicSavings({
366
+ licenseKey: process.env.BIVELIO_LICENSE_KEY,
367
+ licenseServerUrl: "https://savings.bivelio.com",
368
+ serviceAccountKey: process.env.BIVELIO_SERVICE_KEY,
369
+ });
370
+ const client = new Anthropic({ middleware: [claudeSavings.middleware] }); // last in the list
371
+ const message = await client.messages.create({
372
+ model: "claude-opus-5",
373
+ max_tokens: 256,
374
+ messages: [{ role: "user", content: "What is 2 + 2 * 3?" }],
375
+ });
376
+ console.log(claudeSavings.reportFor(message)?.cost.basis);
377
+ ```
378
+
379
+ It runs once per HTTP attempt, inside the SDK's retries: an attempt the provider
380
+ rejects reaches the SDK unchanged and is not recorded. Only message creation goes
381
+ through the layer; `countTokens`, batches, models and files pass untouched.
382
+
231
383
  ## What you get on every request
232
384
 
233
385
  A `SavingsReport` with the numbers behind every claim, kept in **four separate
@@ -240,6 +392,16 @@ ledgers** that are never summed into one flattering figure:
240
392
  | `compressed_tokens` | information kept, but encoded with fewer tokens |
241
393
  | `provider_cached_tokens` | tokens still in the prompt, billed/processed more cheaply |
242
394
 
395
+ **Quality, kept apart from money.** `report.qualityLedger` records how the answer you
396
+ received earned its place: whether it came from your provider, a replay, or the answer
397
+ to a reworded question (with its similarity score); whether it was checked against your
398
+ output contract, repaired or cut off by the output limit; and what the model cascade
399
+ decided. It holds no money, is never added to the four ledgers, and never leaves your
400
+ process. Reusing answers to reworded questions is opt-in, and by default it covers
401
+ single-shot requests only: requests that declare tools, carry tool calls or results, or
402
+ hold more than one user turn are left out unless you set
403
+ `savings.semanticCache: "include-agentic"`.
404
+
243
405
  And a promise you can hold it to: **never worse than your JSON.** Payload encodings
244
406
  are chosen by counting real tokens with the destination model's own tokenizer, and
245
407
  BV‑SALA only moves away from plain JSON when the saving clearly clears the bar —
@@ -295,6 +457,65 @@ and every pair as a CSV download.
295
457
 
296
458
  </details>
297
459
 
460
+ <details>
461
+ <summary><b>Provider price levers (opt-in, off by default)</b></summary>
462
+
463
+ Two request flags ask your provider for a lower price on the same tokens. Neither
464
+ changes the answer, and what they are worth is reported next to the four ledgers:
465
+ never folded into `cost.*`, never part of any commission.
466
+
467
+ **`savings.deferrable`** asks for the provider's Flex tier: OpenAI, and Gemini through
468
+ its OpenAI‑compatible endpoint. Anthropic has no cheaper synchronous tier (its
469
+ discount lives only in Message Batches, which the layer does not support yet).
470
+ Bedrock and DeepSeek off‑peak pricing are not supported.
471
+
472
+ - **Latency.** OpenAI publishes no latency commitment for Flex; Google publishes a
473
+ target of minutes. Keep deferrable requests off user‑facing paths.
474
+ `report.deferred.latency` quotes what the provider documents.
475
+ - **Risk.** Without capacity the provider refuses the request (429, or 503 on Gemini)
476
+ instead of waiting; OpenAI documents that the refusal is not charged. With
477
+ `priceLevers: { onCapacityError: "standard" }` the layer repeats it once at the
478
+ standard tier and price, and says so in `report.deferred.capacityFallback`.
479
+ - **What it is worth.** Your provider's usage counter for that call, priced at the two
480
+ published rate cards (standard, and the tier that actually served it), component
481
+ by component, with the date the prices were checked. It is `measured` only when
482
+ the served tier was read back from the provider's own API, the prices are list
483
+ prices and the counter says how much of the input was read from (or written to)
484
+ the prompt cache; otherwise `estimated`, and `basisReason` says why. It never
485
+ exceeds the difference between the two published rate cards, even with your own
486
+ `costModels`. A tier the provider did not confirm is never credited.
487
+
488
+ **`savings.standalone`** (GPT‑5.6 and later, on OpenAI's own API) declares that nothing
489
+ after the stable prefix of this request will be sent again. The provider then writes
490
+ its prompt cache only at the end of that prefix, or nowhere, instead of at the end of
491
+ your message, which on these models costs more than plain input. The trade‑off:
492
+ repeats and continuations of this exact prompt lose the cache reads the provider
493
+ default would give them, so leave it off conversation turns you will continue. Its
494
+ figure, `report.cacheWrites.avoidedWritePremium`, is always `estimated`: the provider
495
+ default was not run.
496
+
497
+ ```ts
498
+ const leveredLayer = createSavingsLayer({
499
+ provider,
500
+ licenseKey: process.env.BIVELIO_LICENSE_KEY,
501
+ priceLevers: { deferrable: true, onCapacityError: "standard" }, // layer-wide defaults
502
+ });
503
+
504
+ const { report } = await leveredLayer.generate({
505
+ model: "openai/gpt-5.6-luna",
506
+ messages: [
507
+ { role: "system", content: "…your long, fixed instructions…" },
508
+ { role: "user", content: "Classify ticket 42." },
509
+ ],
510
+ savings: { standalone: true }, // a request's own value always wins
511
+ });
512
+
513
+ report.deferred?.tierSaving; // { amount, basis, standardCost, appliedCost, pricesCheckedOn, basisReason }
514
+ report.cacheWrites; // { mode, breakpoints, avoidedWritePremium?, reason }
515
+ ```
516
+
517
+ </details>
518
+
298
519
  <details>
299
520
  <summary><b>Structured output — validated <code>data</code>, not just text</b></summary>
300
521
 
@@ -368,6 +589,144 @@ an action that has not run yet cannot be a saving.
368
589
 
369
590
  </details>
370
591
 
592
+ <details>
593
+ <summary><b>Your ledgers in your tracing — OpenTelemetry, Langfuse, Datadog</b></summary>
594
+
595
+ Wrap the layer once and every request becomes a span on the OpenTelemetry pipeline
596
+ you already run (`@opentelemetry/api` is an optional peer; the SDK never imports it):
597
+
598
+ ```ts
599
+ import { trace } from "@opentelemetry/api";
600
+ import { instrumentSavingsLayer } from "@bivelio/savings-layer";
601
+
602
+ // One span per request, on the OpenTelemetry pipeline you already run.
603
+ const traced = instrumentSavingsLayer(layer, { tracer: trace });
604
+ const { report: tracedReport } = await traced.generate({
605
+ model: "openai/gpt-4o-mini",
606
+ messages: [{ role: "user", content: "What is 2 + 2 * 3?" }],
607
+ });
608
+ console.log(tracedReport.cost.basis); // the span carries the same figures
609
+ ```
610
+
611
+ - **Standard spans, plus your ledgers.** Spans follow the
612
+ [GenAI semantic conventions](https://github.com/open-telemetry/semantic-conventions/blob/v1.41.1/model/gen-ai/registry.yaml)
613
+ as last published (OpenTelemetry semantic conventions v1.41.1): `chat {model}`,
614
+ kind CLIENT, provider, model and usage from your provider's counter (0 on an avoided
615
+ call: no token was billed). On top, `com.bivelio.*` carries the four ledgers, the
616
+ `basis`, the cost and the ids; `com.bivelio.request_id` is the same id as your usage
617
+ record. Same results, same errors: `traced` hands back the very objects `layer` does.
618
+ - **Money is kept apart by basis.** The money saved is
619
+ `com.bivelio.cost.saved.measured`, `.reference` or `.estimated` — one key per span,
620
+ the one of its basis — so a sum over one key never mixes measured and estimated
621
+ money; `unmeasured` spans carry counts and no money. `report.cost.netSavings` is a
622
+ ratio and goes out as `com.bivelio.cost.net_savings_ratio`; the money is
623
+ `baselineEstimate − actual`, and that is what `saved.*` holds.
624
+ - **None of this talks to BiVelio.** Spans go to your exporter; files stay where you
625
+ write them. Your prompts, completions and provider keys never reach BiVelio, and no
626
+ span carries them either: ids, counts, money and enums only, and a failed request
627
+ records its `error.type`, never its message.
628
+ - **Langfuse.** Add the `LangfuseSpanProcessor` from `@langfuse/otel` to your tracer
629
+ provider. These spans carry `gen_ai.*`, so its default filter exports them
630
+ ([Langfuse OpenTelemetry docs](https://langfuse.com/integrations/native/opentelemetry)).
631
+ `instrumentSavingsLayer(layer, { tracer: trace, langfuse: { metadata: true } })`
632
+ makes the figures filterable metadata, and `langfuse: { cost: true }` hands Langfuse
633
+ the report's own cost (in USD) instead of its estimate. Tracing with `@langfuse/tracing` by
634
+ hand? Pass `toLangfuseGeneration(report)` to `generation.update()`.
635
+ - **Datadog LLM Observability** reads these spans over OTLP
636
+ ([Datadog OpenTelemetry docs](https://docs.datadoghq.com/llm_observability/instrumentation/otel_instrumentation/)):
637
+
638
+ ```bash
639
+ OTEL_EXPORTER_OTLP_TRACES_PROTOCOL=http/protobuf
640
+ OTEL_EXPORTER_OTLP_TRACES_ENDPOINT=<your Datadog site's OTLP traces endpoint>
641
+ OTEL_EXPORTER_OTLP_TRACES_HEADERS=dd-api-key=<your Datadog API key>,dd-otlp-source=llmobs
642
+ ```
643
+
644
+ The `com.bivelio.*` attributes arrive as tags; with `datadog: { metadata: true }`
645
+ they also arrive as span metadata. Datadog drops spans that carry no `gen_ai.*`
646
+ attribute, so use `instrumentSavingsLayer` there rather than `recordSavingsOnSpan`
647
+ (which writes `com.bivelio.*` on a span you already have, and no `gen_ai.*` unless
648
+ you ask for it with `genAi: true`).
649
+
650
+ | Attribute | What it holds |
651
+ | --- | --- |
652
+ | `gen_ai.operation.name`, `gen_ai.provider.name`, `gen_ai.request.model`, `gen_ai.response.model` | `chat`, the provider and the model (without a known provider prefix) |
653
+ | `gen_ai.usage.input_tokens`, `.output_tokens`, `.cache_read.input_tokens` | your provider's counter; absent when it reported no usage |
654
+ | `com.bivelio.cost.basis`, `.currency`, `.baseline`, `.actual` | where the money comes from, its currency (USD by default), and both sides |
655
+ | `com.bivelio.cost.saved.measured` \| `.reference` \| `.estimated` | the money saved, in the key of its basis only |
656
+ | `com.bivelio.cost.net_savings_ratio` | `cost.netSavings`, a 0..1 ratio — not money |
657
+ | `com.bivelio.calls.*`, `com.bivelio.tokens.*` | the four ledgers: avoided calls, eliminated, compressed and provider‑cached tokens |
658
+ | `com.bivelio.request_id`, `.model.id`, `.cache.result`, `.quality.status` | ids and outcomes |
659
+
660
+ </details>
661
+
662
+ <details>
663
+ <summary><b>Your ledgers in your FinOps tool — FOCUS rows and a daily savings series</b></summary>
664
+
665
+ Keep each report with the time it came back, and export whenever you like:
666
+
667
+ ```ts
668
+ import { writeFileSync } from "node:fs";
669
+ import { toFocusRows, toSavingsSeries, toCsv, type SavingsRecord } from "@bivelio/savings-layer";
670
+
671
+ const records: SavingsRecord[] = [];
672
+ const answer = await layer.generate({
673
+ model: "openai/gpt-4o-mini",
674
+ messages: [{ role: "user", content: "What is 2 + 2 * 3?" }],
675
+ });
676
+ records.push({ at: Date.now(), report: answer.report }); // keep reports as they come back
677
+
678
+ // FOCUS 1.4 rows, one per day, model and basis. Every cost column is 0 by design:
679
+ // your provider's bill already has the spend, so these rows add savings, never cost.
680
+ const rows = toFocusRows(records, { billingAccountId: "org-XXXXXXXX" }); // your provider billing account id
681
+ writeFileSync("bv-sala-focus.csv", toCsv(rows));
682
+
683
+ // A daily series for tools that take savings as a metric: one file per basis.
684
+ const measured = toSavingsSeries(records).filter((r) => r.measuredSavings !== null);
685
+ writeFileSync(
686
+ "vantage-measured-savings.csv",
687
+ toCsv(measured.map((r) => ({ date: r.date, amount: r.measuredSavings, model: r.model }))),
688
+ );
689
+ ```
690
+
691
+ The rows follow [FOCUS 1.4](https://github.com/FinOps-Open-Cost-and-Usage-Spec/FOCUS_Spec/tree/v1.4),
692
+ the FinOps Open Cost and Usage Specification: `ChargeCategory` is `Adjustment`, the
693
+ service provider is the one in the model id (`OpenAI`, `Anthropic`…; pass
694
+ `serviceProviderName`, `invoiceIssuerName` or `hostProviderName` when a cloud or a
695
+ gateway bills you), and no SKU or unit price is invented. Nothing is sent anywhere:
696
+ `toFocusRows`, `toSavingsSeries` and `toCsv` are plain functions over the reports you
697
+ kept. Where each file goes, as each tool documents it in September 2026:
698
+
699
+ - **A FOCUS data lake or warehouse** reads every column, the `x_BiVelio*` ones included.
700
+ - **[Finout](https://docs.finout.io/get-started-with-finout/cost-and-usage-types/focus-in-finout)**
701
+ loads a FOCUS CSV as a custom cost center; you send the file to Finout support.
702
+ - **[Vantage](https://docs.vantage.sh/connecting_custom_providers)** Custom Providers
703
+ accept FOCUS 1.1–1.3 files but use only a few columns for cost reporting, and say
704
+ nothing about `x_` columns, so the savings would not show. Upload the series as a
705
+ [Business Metric](https://docs.vantage.sh/per_unit_costs) instead
706
+ (`date,amount,<labels>`), one file per basis, so a metric never mixes bases.
707
+ Vantage rejects two rows with the same date and labels: with `dimensions`, add
708
+ them as label columns too.
709
+ - **[CloudZero](https://docs.cloudzero.com/docs/anycost-common-bill-format-cbf)** AnyCost
710
+ takes its own Common Bill Format; the series can go in as a
711
+ [Telemetry Stream](https://docs.cloudzero.com/docs/telemetry-streams).
712
+
713
+ The custom columns, in the currency you pass (`currency`, USD by default):
714
+
715
+ | Column | What it holds |
716
+ | --- | --- |
717
+ | `x_BiVelioBasis`, `x_BiVelioModelId` | the basis of every figure in the row, and the model id |
718
+ | `x_BiVelioRequestCount`, `x_BiVelioProviderCallCount`, `x_BiVelioAvoidedCallCount` | requests, calls that reached the provider, calls avoided |
719
+ | `x_BiVelioBaselineCost`, `x_BiVelioActualCost` | both sides of the saving; empty when the basis is `unmeasured` |
720
+ | `x_BiVelioMeasuredSavings`, `x_BiVelioReferenceSavings`, `x_BiVelioEstimatedSavings` | `baseline − actual`, in the column of the row's basis only; negative when the layer cost you money |
721
+ | `x_BiVelioEliminatedInputFloorCost` | the separate floor lane for context your provider cleared |
722
+ | `x_BiVelio…InputTokens`, `x_BiVelio…OutputTokens` | the token ledgers: baseline, actual, eliminated, compressed, provider‑cached |
723
+
724
+ `Tags` carries `bivelio/basis` and `bivelio/model`, plus any tags you return from the
725
+ `tags` option. Sum `measured` and `reference` for what you can check against your
726
+ provider invoice; `estimated` has its own column on purpose.
727
+
728
+ </details>
729
+
371
730
  <details>
372
731
  <summary><b>Running a fleet? Seats and renewals</b></summary>
373
732
 
@@ -390,6 +749,10 @@ BVSALA_SEAT_ID=worker-03 # stable per replica; [A-Za-z0-9._:/-], max 128 chars
390
749
 
391
750
  - **Node ≥ 20.** The package is **ESM‑only** (`"type": "module"`); from CommonJS,
392
751
  `require()` works on Node ≥ 22.
752
+ - Framework adapters: `ai` 6 or 7, `@mastra/core` ≥ 1.68 < 2, `langchain` ≥ 1.1 < 2
753
+ with `@langchain/core` ≥ 1.1 < 2, and `@anthropic-ai/sdk` ≥ 0.103 < 1 — optional
754
+ peers, needed only for the adapter you import. `ai` 7 and `@mastra/core` declare
755
+ Node ≥ 22 in their own `engines`.
393
756
  - Optional peer `gpt-tokenizer` for exact OpenAI token counts, loaded automatically
394
757
  when installed — without it a built‑in estimator is used. It improves the accuracy
395
758
  of estimated figures; what reaches your invoice is `measured` and `reference`, which
@@ -402,14 +765,20 @@ BVSALA_SEAT_ID=worker-03 # stable per replica; [A-Za-z0-9._:/-], max 128 chars
402
765
 
403
766
  ## Licensing & privacy
404
767
 
405
- BV‑SALA is commercial, per‑seat software (**€4 / seat / month** or **€39 / seat /
406
- year**, EUR; VAT not included). No free tier and no trial: what you can check before
407
- paying is the published measurement. The license is a signed token your SDK and the gateway verify **offline**; the
408
- managed control plane and the revocation kill‑switch are served from BiVelio. Your
409
- LLM traffic goes **straight to your provider** — BiVelio sees counts and cost,
410
- never prompts or completions.
411
-
412
- Get a key and read the terms at **[savings.bivelio.com](https://savings.bivelio.com)**.
768
+ BV‑SALA is commercial software: a per‑seat subscription (**€4 / seat / month** or
769
+ **€39 / seat / year**, EUR; VAT not included), paid whether or not you save, plus a
770
+ tiered commission on measured savings only, as described at the top of this page, in
771
+ [LICENSE](./LICENSE) §6 and in the Published Pricing it refers to,
772
+ **[savings.bivelio.com/pricing](https://savings.bivelio.com/pricing)**. No free tier and no trial: what you can check before
773
+ paying is the published measurement. The license is a signed token your SDK and the
774
+ gateway verify **offline**; the managed control plane and the revocation kill‑switch
775
+ are served from BiVelio. Your LLM traffic goes **straight to your provider** —
776
+ BiVelio receives per‑request usage (counts, cost, the model id and opaque ids), never
777
+ prompts, completions or provider keys; [LICENSE](./LICENSE) §5 lists every field, and
778
+ **[savings.bivelio.com/privacy](https://savings.bivelio.com/privacy)** says how long
779
+ each one is kept.
780
+
781
+ Get a key at **[savings.bivelio.com](https://savings.bivelio.com)**.
413
782
  Onboarding & general questions: **support@bivelio.com** · licensing & compliance:
414
783
  **privacy@bivelio.com** · technical: **team@bivelio.com**.
415
784