floe-guard 0.14.0 → 0.15.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/README.md CHANGED
@@ -64,6 +64,24 @@ const adv = guard.advisory();
64
64
  const model = adv.nearLimit ? openai("gpt-4o-mini") : openai("gpt-4o");
65
65
  ```
66
66
 
67
+ ## Request-sized estimates
68
+
69
+ To ensure the ceiling is enforced on the first run or for a call much larger than the previous one, you can price the actual incoming request using `estimateCall()` and pass the estimate to `reserve()` or `check()`:
70
+
71
+ ```ts
72
+ const est = guard.estimateCall("gpt-4o", 12_000, 4_096);
73
+ const handle = guard.reserve(est); // throws BudgetExceeded NOW if this call alone would cross
74
+ try {
75
+ const response = await callYourLlm({ model: "gpt-4o", ... });
76
+ guard.settle("gpt-4o", response.usage.promptTokens, response.usage.completionTokens, { reserved: handle });
77
+ } catch (err) {
78
+ guard.release(handle);
79
+ throw err;
80
+ }
81
+ ```
82
+
83
+ If the model is unpriceable, `estimateCall()` returns `undefined` and `reserve(undefined)` / `check(undefined)` fall back gracefully to the last-cost prediction.
84
+
67
85
  ## Tool spend under the same ceiling
68
86
 
69
87
  Paid tool calls (Apollo, Exa, scrapers) burn the same budget as tokens. The