@bivelio/savings-layer 0.4.21 → 0.4.22
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/README.md +20 -8
- package/dist/gateway/bin.js +299 -299
- package/dist/index.d.ts +31 -3
- package/dist/index.js +537 -533
- package/package.json +1 -1
package/README.md
CHANGED
|
@@ -40,8 +40,10 @@ goes straight from your app to your provider; the SDK is never a middleman serve
|
|
|
40
40
|
|
|
41
41
|
> **Commercial software — © 2026 BiVelio Inc.** Free to install, **paid to run**:
|
|
42
42
|
> BV‑SALA is fail‑closed and executes only with an active BiVelio license key.
|
|
43
|
-
>
|
|
44
|
-
>
|
|
43
|
+
> The license signature is verified **offline**; with `licenseServerUrl` set, the SDK
|
|
44
|
+
> also calls `savings.bivelio.com` for the revocation list, license renewal and usage
|
|
45
|
+
> metering. **Your prompts never leave your infrastructure** — what travels is token
|
|
46
|
+
> counts, cost, the model id and opaque seat/session ids, never content. Use is governed by the [LICENSE](./LICENSE)
|
|
45
47
|
> and the BiVelio Terms. Not a wrapper around any third‑party library.
|
|
46
48
|
|
|
47
49
|
## Where it installs
|
|
@@ -99,13 +101,16 @@ export BIVELIO_LICENSE_KEY=… # your license key
|
|
|
99
101
|
export BIVELIO_SERVICE_KEY=… # service-account key: auto-renewal + seat metering
|
|
100
102
|
```
|
|
101
103
|
|
|
102
|
-
**3 · Wrap the call** — this is the entire code change:
|
|
104
|
+
**3 · Wrap the call** — this is the entire code change. The package is ESM only:
|
|
105
|
+
put this in a `"type": "module"` project or a `.mts` file.
|
|
103
106
|
|
|
104
107
|
```ts
|
|
105
108
|
import { createSavingsLayer, OpenAICompatibleProvider } from "@bivelio/savings-layer";
|
|
106
109
|
|
|
107
110
|
// Your provider. LLM traffic goes straight to it — the SDK never proxies it.
|
|
108
111
|
// Any OpenAI-compatible endpoint works: OpenAI, Azure, vLLM, SGLang, a gateway.
|
|
112
|
+
// Model ids are qualified (`openai/gpt-4o-mini`); the prefix is stripped on the
|
|
113
|
+
// wire for OpenAI and Azure and kept for gateways — see `stripProviderPrefix`.
|
|
109
114
|
const provider = new OpenAICompatibleProvider({
|
|
110
115
|
baseUrl: "https://api.openai.com/v1",
|
|
111
116
|
apiKey: process.env.OPENAI_API_KEY,
|
|
@@ -114,7 +119,7 @@ const provider = new OpenAICompatibleProvider({
|
|
|
114
119
|
const layer = createSavingsLayer({
|
|
115
120
|
provider,
|
|
116
121
|
licenseKey: process.env.BIVELIO_LICENSE_KEY, // required — from your savings.bivelio.com dashboard
|
|
117
|
-
licenseServerUrl: "https://savings.bivelio.com", //
|
|
122
|
+
licenseServerUrl: "https://savings.bivelio.com", // revocation list, renewal + metering — omit it and the two lines below do nothing
|
|
118
123
|
serviceAccountKey: process.env.BIVELIO_SERVICE_KEY, // seat metering + AUTO-RENEWAL pickup
|
|
119
124
|
});
|
|
120
125
|
|
|
@@ -124,12 +129,19 @@ const { text, report } = await layer.generate({
|
|
|
124
129
|
});
|
|
125
130
|
|
|
126
131
|
console.log(text); // "8" (computed locally — no LLM call)
|
|
127
|
-
console.log(report.calls.avoidedLlm); // → whole
|
|
128
|
-
console.log(report.cost.netSavings); // →
|
|
132
|
+
console.log(report.calls.avoidedLlm); // → 1: a whole call you never paid for
|
|
133
|
+
console.log(report.cost.netSavings); // → 0..1 ratio of the baseline you did not pay (1 here)
|
|
134
|
+
console.log(report.cost.baselineEstimate - report.cost.actual); // → the same saving in USD
|
|
135
|
+
console.log(report.cost.basis); // → "estimated": nothing ran, so there is no provider counter
|
|
129
136
|
```
|
|
130
137
|
|
|
131
|
-
That's it. From the first request, `report` carries the saving
|
|
132
|
-
|
|
138
|
+
That's it. From the first request, `report` carries the saving and says how it
|
|
139
|
+
was obtained: `actual` is what the provider metered; `basis` is `"measured"` when
|
|
140
|
+
the request went through untouched, `"estimated"` when the layer removed something
|
|
141
|
+
(the baseline is then reconstructed with the model's tokenizer, or with a
|
|
142
|
+
calibrated estimate where no exact tokenizer exists, as for Anthropic models),
|
|
143
|
+
and `"unmeasured"` when the provider returned no usage. Only measured figures
|
|
144
|
+
reach your invoice.
|
|
133
145
|
|
|
134
146
|
## What you get on every request
|
|
135
147
|
|