bare-agent 0.36.1 → 0.38.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/bareagent.context.md +4 -4
- package/package.json +1 -1
- package/src/judge.d.ts +29 -3
- package/src/judge.js +19 -15
- package/src/loop.d.ts +82 -70
- package/src/loop.js +155 -56
- package/src/provider-clipipe-mcp.js +1 -0
- package/src/provider-clipipe.d.ts +5 -3
- package/src/provider-clipipe.js +10 -3
- package/types/index.d.ts +3 -1
package/bareagent.context.md
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# bareagent — Integration Guide
|
|
2
2
|
|
|
3
3
|
> For AI assistants and developers wiring bareagent into a project.
|
|
4
|
-
> v0.
|
|
4
|
+
> v0.38.1 | Node.js >= 18 | zero required deps (`bareguard >=0.9.0 <0.14.0` optional peer for governance) | Apache 2.0
|
|
5
5
|
>
|
|
6
6
|
> Full human guide with composition examples, design philosophy, and recipes: [Usage Guide](docs/02-features/usage-guide.md)
|
|
7
7
|
|
|
@@ -874,7 +874,7 @@ new CLIPipe({ command: 'claude', args: ['-p', '--model', 'sonnet'], toolProtocol
|
|
|
874
874
|
- `policy` — the SAME `(tool, args, ctx) => true|string` chokepoint as `Loop({policy})`, so a wired `wireGate(gate).policy` writes **audit rows of identical shape, zero gate changes**. A deny is a tool result (advisory); the handler never runs. **Required here** — a `Loop({policy})` in native mode would be a fence that is silently not there, so the Loop throws.
|
|
875
875
|
- `maxConsecutiveDenials` (3) / `maxIdenticalToolErrors` (3) — BA-11/BA-12 guards at the bridge, same narrowest triggers; end the session `denied:<tool>` / `stuck:<tool>`.
|
|
876
876
|
- `maxTurns` — a bound on **assistant/LLM turns**, the SAME unit as the Loop path, so one number means one thing on both surfaces (BA-17). NOT a tool-call count: one turn can fire a dozen parallel tool calls and still be one turn (measured: 12 calls across 2 turns, inside `--max-turns 3`). Enforced twice — the CLI's own `--max-turns` stops cleanly at N (and emits the result event that carries the session's real cost), plus a parent-side counter that kills the session if a turn beyond N is ever seen, since that flag is undocumented in `claude --help`. The stop is `error:'max_turns'` + `stopReason:'max_turns'` and **carries the last turn's text forward** — the CLI reports `result:null` on a bounded session, so an unfixed build returned `text:''`.
|
|
877
|
-
- `onTurn` — fires once per **assistant turn** (BA-17: the CLI emits a separate stream event per content *block*, all repeating that message's usage — firing per event inflated a caller's turn axis ~5–7× and its token axis 5.04×), carrying four cache tiers and `costUsd:null` since the CLI prices the session, not the turn. Then one closing `kind:'session'` event carrying the authoritative cost **and the token residual** — a turn's `message.usage` is a snapshot taken at its first block and never revised (a turn that emitted ~816 output tokens reported 2), so the closing event makes the streamed tiers add up to exactly the CLI's own session total. Shape mirrors `onLlmResult
|
|
877
|
+
- `onTurn` — fires once per **assistant turn** (BA-17: the CLI emits a separate stream event per content *block*, all repeating that message's usage — firing per event inflated a caller's turn axis ~5–7× and its token axis 5.04×), carrying four cache tiers and `costUsd:null`/`rateSource:null` since the CLI prices the session, not the turn. Then one closing `kind:'session'` event carrying the authoritative cost **and the token residual** — a turn's `message.usage` is a snapshot taken at its first block and never revised (a turn that emitted ~816 output tokens reported 2), so the closing event makes the streamed tiers add up to exactly the CLI's own session total. The session event stamps `rateSource:'provider'` when its cost is finite (BA-22 — the CLI's own `total_cost_usd`, no local rate table; a killed/unknown session carries `rateSource:null`, never a spurious `'provider'`). Shape mirrors `onLlmResult` (incl. `rateSource`), so `wireGate(gate).onLlmResult` drops in; when wired the Loop skips its own forward (billed once, never starved).
|
|
878
878
|
- `sessionTimeout` (600s) / `bridgeTimeoutMs` (120s) — whole-session and per-handler ceilings.
|
|
879
879
|
|
|
880
880
|
`GenerateResult.session` (`{turns, toolCalls, error, usageReported}`) carries what really happened; `metrics.sessionTurns` reports the real turn count — assistant *messages*, not stream events — so a 14-turn session never reads as one round, and a 2-turn session never reads as 14. A terminal the CLI detects inside the session (bound, guard, or a **broken tool bridge** — a dead bridge still ends `subtype:'success'`, so it is caught parent-side by attempted-vs-served tool calls) surfaces as the run's `error`, never a laundered clean finish. `assemble`/`trim`/`cacheMessages` and a Loop-level `policy` all THROW at construction in native mode (no silently-dead knobs — the CLI owns the transcript). The bridge is a unix socket (0600 in a 0700 dir), never a listening port. Claude-only for now; the CLI-specific parts live in `src/provider-clipipe-mcp.js` + `src/mcp-bridge-stub.js` behind the same seam as emulation.
|
|
@@ -891,7 +891,7 @@ All return `{ text, toolCalls, usage: { inputTokens, outputTokens }, model?, cos
|
|
|
891
891
|
|
|
892
892
|
**Plaintext-key warning (Unreleased):** the OpenAI provider's `baseUrl` accepts `http://` (for local/OpenAI-compatible endpoints), but a `Bearer` key sent over plaintext http to a **non-loopback** host is exposed on the wire. The provider now warns once when that happens. Loopback hosts (`localhost`/`127.0.0.0/8`/`::1` — local proxies, Ollama-style endpoints) stay silent, since that's the legitimate keyless-local case. The header is **not** stripped (some local proxies want a key), so use `https` for any remote endpoint, or drop `apiKey` when the local endpoint needs none.
|
|
893
893
|
|
|
894
|
-
**Cost estimation:** Loop automatically estimates USD cost per run based on model and token usage. The `cost` field appears in every `loop.run()` result and in `loop:done` stream events. Pricing
|
|
894
|
+
**Cost estimation:** Loop automatically estimates USD cost per run based on model and token usage. The `cost` field appears in every `loop.run()` result and in `loop:done` stream events. **Pricing honesty (BA-21, v0.37.0):** a token-bearing round is ALWAYS priced (guesstimate-and-run — a null/unknown model no longer forces `unpriced`); rates resolve as caller-supplied `new Loop({ rates: { in, out } })` (authoritative) → a recognized Claude tier from the model id (`haiku`/`sonnet`) → the **Sonnet-tier** ceiling default ($0.003/$0.015 per 1K, no per-model table). Every metering payload carries a `rateSource: 'provider' | 'caller' | 'tier' | 'default' | null` field so a consumer can tell an authoritative price from a guess — and (BA-21 follow-up, v0.38.0) a recognized-tier guess (`'tier'`) from a blind ceiling fallback (`'default'`); a round priced off either guesstimate emits ONE loud `console.warn` per Loop instance naming the actual source (silenced by passing `rates`). To set your own rates, construct `new Loop({ rates })` — there is no longer a `COST_PER_1K` table to edit. The model is resolved as `result.model || provider.model` (v0.16.1+) — providers now echo the model in their `generate()` result, so cost accounting holds even when `provider.model` is absent or varies per response, e.g. behind `FallbackProvider` or `CircuitBreaker.wrapProvider` (the wrapper also preserves `model`/`name` passthrough props). Wire `onLlmResult` (via `wireGate`) and a `budget.maxCostUsd` cap then halts on token-heavy workloads too.
|
|
895
895
|
|
|
896
896
|
## Store options
|
|
897
897
|
|
|
@@ -1047,7 +1047,7 @@ Stale example removed in 0.10.4: `examples/mcp-bridge-gov.js` (used a hard-coded
|
|
|
1047
1047
|
16. **`bin/cli.js` uses BA1 callbacks since v0.10.3** — children record LLM cost into bareguard's audit, thread `_ctx`, and pre-filter tools via `gate.allows`. Pre-0.10.3 children used the deprecated `wrapTools` path which silently dropped LLM cost from `budget.maxCostUsd` and printed a deprecation warning per first tool call. No config change needed; the upgrade is transparent.
|
|
1048
1048
|
17. **Halt-path `msgs` is sealed (v0.10.3+)** — when Loop catches `HaltError` mid-round, every dangling assistant `tool_calls.id` from the halted round gets a synthetic `{ role:'tool', tool_call_id, content: '[halted:<rule>]' }` appended so the returned `result.msgs` is valid OpenAI shape. Safe to feed back into another provider call without protocol errors. The `[halted:<rule>]` tag is lowercase — distinct from the legacy `[HALT:]` deny strings (removed in 0.10.0, do not match on the old form).
|
|
1049
1049
|
18. **`HaltError` with no `rule` resolves to `halt:unknown` (v0.10.3+)** — `new HaltError('msg')` without a `{ rule }` option still produces a stable `result.error = 'halt:unknown'` and `loop:done{halted:true, rule:'unknown'}`. Pre-0.10.3 produced the literal `'halt:null'` which broke string-matching consumers. The `_reportError('halt', ...)` extra carries the same `rule:'unknown'` token.
|
|
1050
|
-
19. **
|
|
1050
|
+
19. **Pricing is a two-tier guesstimate, not a per-model table (BA-21, v0.37.0)** — a recognized Claude tier in the model id prices at `haiku` ($0.001/$0.005) or `sonnet` ($0.003/$0.015) per 1K; anything else (incl. a null/unknown model) falls through to the **Sonnet-tier ceiling default** ($0.003/$0.015) and still gets priced (`rateSource: 'default'`), never silently `unpriced`. Both guesstimate cases emit one `console.warn` per Loop instance. The `rateSource` on every metering payload distinguishes them: `'tier'` (a recognized haiku/sonnet match — a confident guess) vs `'default'` (the blind ceiling for an unrecognized/absent model), alongside `'provider'`/`'caller'` (authoritative) and `null` (unpriced). If you use a model whose real rates differ and care about `result.cost` accuracy or `budget.maxCostUsd` enforcement via `onLlmResult`, pass `new Loop({ rates: { in, out, cacheReadMult?, cacheWriteMult? } })` — that's authoritative (`rateSource: 'caller'`) and silences the warning. There is no `COST_PER_1K` table to edit.
|
|
1051
1051
|
|
|
1052
1052
|
## Cross-language SDKs
|
|
1053
1053
|
|
package/package.json
CHANGED
package/src/judge.d.ts
CHANGED
|
@@ -14,6 +14,18 @@ export type JudgeOptions = {
|
|
|
14
14
|
* option, because the http providers build the request from `this.model` and would silently ignore them.
|
|
15
15
|
*/
|
|
16
16
|
provider: import("../types").Provider;
|
|
17
|
+
/**
|
|
18
|
+
* - BA-21: per-1K USD
|
|
19
|
+
* rates for the judge's own model, to price its call authoritatively (rateSource:'caller'). Omit → a flagged
|
|
20
|
+
* guesstimate (rateSource:'tier' for a recognized Claude tier, else 'default'). Bring your own rate, or take a
|
|
21
|
+
* guesstimate — never a silent guess.
|
|
22
|
+
*/
|
|
23
|
+
rates?: {
|
|
24
|
+
in: number;
|
|
25
|
+
out: number;
|
|
26
|
+
cacheReadMult?: number;
|
|
27
|
+
cacheWriteMult?: number;
|
|
28
|
+
} | undefined;
|
|
17
29
|
/**
|
|
18
30
|
* - Cap on the judge's own output. A verdict truncated at the cap floors to
|
|
19
31
|
* `broke`; 512 clears the mechanical `where` with headroom (measured), lower it only if you know the artifacts are small.
|
|
@@ -27,6 +39,7 @@ export type JudgeOptions = {
|
|
|
27
39
|
model: string | null;
|
|
28
40
|
kind: "judge";
|
|
29
41
|
costUsd: number | null;
|
|
42
|
+
rateSource: "provider" | "caller" | "tier" | "default" | null;
|
|
30
43
|
}) => any) | undefined;
|
|
31
44
|
};
|
|
32
45
|
/**
|
|
@@ -97,9 +110,15 @@ export type JudgeVerdict = {
|
|
|
97
110
|
*/
|
|
98
111
|
parseError: boolean;
|
|
99
112
|
/**
|
|
100
|
-
* -
|
|
113
|
+
* - Per-call cost. An HONEST null only when genuinely unpriced (never coerced to 0).
|
|
101
114
|
*/
|
|
102
115
|
costUsd: number | null;
|
|
116
|
+
/**
|
|
117
|
+
* - WHERE costUsd came from (BA-21): 'provider'
|
|
118
|
+
* (real), 'caller' (your `rates`), 'tier' (recognized Claude tier — a confident guesstimate), 'default' (blind
|
|
119
|
+
* ceiling guesstimate — discount or override either guess), null (unpriced).
|
|
120
|
+
*/
|
|
121
|
+
rateSource: "provider" | "caller" | "tier" | "default" | null;
|
|
103
122
|
/**
|
|
104
123
|
* - Neutral usage shape from the provider.
|
|
105
124
|
*/
|
|
@@ -121,9 +140,13 @@ export type JudgeVerdict = {
|
|
|
121
140
|
* @property {import('../types').Provider} provider - LLM provider (bare-agent already owns the transport). To judge
|
|
122
141
|
* on a different tier, construct the provider FOR that tier — there is deliberately no per-call `model`/`effort`
|
|
123
142
|
* option, because the http providers build the request from `this.model` and would silently ignore them.
|
|
143
|
+
* @property {{in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}} [rates] - BA-21: per-1K USD
|
|
144
|
+
* rates for the judge's own model, to price its call authoritatively (rateSource:'caller'). Omit → a flagged
|
|
145
|
+
* guesstimate (rateSource:'tier' for a recognized Claude tier, else 'default'). Bring your own rate, or take a
|
|
146
|
+
* guesstimate — never a silent guess.
|
|
124
147
|
* @property {number} [maxTokens=512] - Cap on the judge's own output. A verdict truncated at the cap floors to
|
|
125
148
|
* `broke`; 512 clears the mechanical `where` with headroom (measured), lower it only if you know the artifacts are small.
|
|
126
|
-
* @property {(payload: { usage: any, model: string|null, kind: 'judge', costUsd: number|null }) => any} [onLlmResult]
|
|
149
|
+
* @property {(payload: { usage: any, model: string|null, kind: 'judge', costUsd: number|null, rateSource: 'provider'|'caller'|'tier'|'default'|null }) => any} [onLlmResult]
|
|
127
150
|
* - Budget hook; each judge call forwards its usage/cost (mirror of Evaluator/remember).
|
|
128
151
|
*/
|
|
129
152
|
/**
|
|
@@ -164,7 +187,10 @@ export function judge(options?: JudgeOptions): Promise<JudgeVerdict>;
|
|
|
164
187
|
* outcome (criterion 5): the verdict is floored to `broke` (fail toward surfacing) but callers/harnesses must EXCLUDE a
|
|
165
188
|
* truncated result from any graded denominator — never counted as a miss, never as a pass.
|
|
166
189
|
* @property {boolean} parseError - The model did not return usable JSON. Floored to `broke`, flagged like truncation.
|
|
167
|
-
* @property {number|null} costUsd -
|
|
190
|
+
* @property {number|null} costUsd - Per-call cost. An HONEST null only when genuinely unpriced (never coerced to 0).
|
|
191
|
+
* @property {'provider'|'caller'|'tier'|'default'|null} rateSource - WHERE costUsd came from (BA-21): 'provider'
|
|
192
|
+
* (real), 'caller' (your `rates`), 'tier' (recognized Claude tier — a confident guesstimate), 'default' (blind
|
|
193
|
+
* ceiling guesstimate — discount or override either guess), null (unpriced).
|
|
168
194
|
* @property {any} usage - Neutral usage shape from the provider.
|
|
169
195
|
* @property {string} model - The model that produced the verdict.
|
|
170
196
|
* @property {string} raw - The model's raw text (for calibration/audit; scrub before persisting — contract 5).
|
package/src/judge.js
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
'use strict';
|
|
2
2
|
|
|
3
3
|
const { ValidationError, HaltError } = require('./errors');
|
|
4
|
-
const {
|
|
4
|
+
const { resolveRoundCost } = require('./loop');
|
|
5
5
|
|
|
6
6
|
/**
|
|
7
7
|
* The decisive return-time judge (BA-20 / bareguard E6i, PRD §9.2). It compares a
|
|
@@ -31,7 +31,10 @@ const { estimateCost, COST_PER_1K } = require('./loop');
|
|
|
31
31
|
* outcome (criterion 5): the verdict is floored to `broke` (fail toward surfacing) but callers/harnesses must EXCLUDE a
|
|
32
32
|
* truncated result from any graded denominator — never counted as a miss, never as a pass.
|
|
33
33
|
* @property {boolean} parseError - The model did not return usable JSON. Floored to `broke`, flagged like truncation.
|
|
34
|
-
* @property {number|null} costUsd -
|
|
34
|
+
* @property {number|null} costUsd - Per-call cost. An HONEST null only when genuinely unpriced (never coerced to 0).
|
|
35
|
+
* @property {'provider'|'caller'|'tier'|'default'|null} rateSource - WHERE costUsd came from (BA-21): 'provider'
|
|
36
|
+
* (real), 'caller' (your `rates`), 'tier' (recognized Claude tier — a confident guesstimate), 'default' (blind
|
|
37
|
+
* ceiling guesstimate — discount or override either guess), null (unpriced).
|
|
35
38
|
* @property {any} usage - Neutral usage shape from the provider.
|
|
36
39
|
* @property {string} model - The model that produced the verdict.
|
|
37
40
|
* @property {string} raw - The model's raw text (for calibration/audit; scrub before persisting — contract 5).
|
|
@@ -124,9 +127,13 @@ function normalizeWhere(where) {
|
|
|
124
127
|
* @property {import('../types').Provider} provider - LLM provider (bare-agent already owns the transport). To judge
|
|
125
128
|
* on a different tier, construct the provider FOR that tier — there is deliberately no per-call `model`/`effort`
|
|
126
129
|
* option, because the http providers build the request from `this.model` and would silently ignore them.
|
|
130
|
+
* @property {{in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}} [rates] - BA-21: per-1K USD
|
|
131
|
+
* rates for the judge's own model, to price its call authoritatively (rateSource:'caller'). Omit → a flagged
|
|
132
|
+
* guesstimate (rateSource:'tier' for a recognized Claude tier, else 'default'). Bring your own rate, or take a
|
|
133
|
+
* guesstimate — never a silent guess.
|
|
127
134
|
* @property {number} [maxTokens=512] - Cap on the judge's own output. A verdict truncated at the cap floors to
|
|
128
135
|
* `broke`; 512 clears the mechanical `where` with headroom (measured), lower it only if you know the artifacts are small.
|
|
129
|
-
* @property {(payload: { usage: any, model: string|null, kind: 'judge', costUsd: number|null }) => any} [onLlmResult]
|
|
136
|
+
* @property {(payload: { usage: any, model: string|null, kind: 'judge', costUsd: number|null, rateSource: 'provider'|'caller'|'tier'|'default'|null }) => any} [onLlmResult]
|
|
130
137
|
* - Budget hook; each judge call forwards its usage/cost (mirror of Evaluator/remember).
|
|
131
138
|
*/
|
|
132
139
|
|
|
@@ -184,20 +191,16 @@ async function judge(options = /** @type {JudgeOptions} */ ({})) {
|
|
|
184
191
|
const model = (out && out.model) || provider.model || null;
|
|
185
192
|
const usage = (out && out.usage) || null;
|
|
186
193
|
|
|
187
|
-
// costUsd
|
|
188
|
-
//
|
|
189
|
-
//
|
|
190
|
-
//
|
|
191
|
-
|
|
192
|
-
|
|
193
|
-
|
|
194
|
-
} else if (model && usage && Object.prototype.hasOwnProperty.call(COST_PER_1K, model)) {
|
|
195
|
-
const est = estimateCost(model, usage);
|
|
196
|
-
if (typeof est === 'number' && Number.isFinite(est)) costUsd = est;
|
|
197
|
-
}
|
|
194
|
+
// costUsd (BA-21 uniform rule — "bring your own rate, or take a flagged guesstimate"): prefer a finite
|
|
195
|
+
// provider-reported cost (rateSource:'provider'); else caller `options.rates` (rateSource:'caller');
|
|
196
|
+
// else a built-in guesstimate (rateSource:'tier' for a recognized Claude tier, else 'default' for the
|
|
197
|
+
// blind ceiling) — never a silent guess, always flagged, never a refuse. NEVER coerce to 0. A consumer
|
|
198
|
+
// that wants the judge austere (no guess) reads rateSource 'tier'/'default' and discounts it, or passes
|
|
199
|
+
// its own rates. `null` only when genuinely unpriced.
|
|
200
|
+
const { cost: costUsd, source: rateSource } = resolveRoundCost(out, model, usage, options.rates || null);
|
|
198
201
|
const onLlmResult = typeof options.onLlmResult === 'function' ? options.onLlmResult : null;
|
|
199
202
|
if (onLlmResult) {
|
|
200
|
-
await onLlmResult({ usage, model, kind: 'judge', costUsd });
|
|
203
|
+
await onLlmResult({ usage, model, kind: 'judge', costUsd, rateSource });
|
|
201
204
|
}
|
|
202
205
|
|
|
203
206
|
// Truncation is a DISTINCT flagged outcome (contract 2 / criterion 5): the API cut the round off, or it came
|
|
@@ -217,6 +220,7 @@ async function judge(options = /** @type {JudgeOptions} */ ({})) {
|
|
|
217
220
|
truncated,
|
|
218
221
|
parseError,
|
|
219
222
|
costUsd,
|
|
223
|
+
rateSource,
|
|
220
224
|
usage,
|
|
221
225
|
model: model || '',
|
|
222
226
|
raw: text,
|
package/src/loop.d.ts
CHANGED
|
@@ -48,9 +48,25 @@ export type LoopOptions = {
|
|
|
48
48
|
/**
|
|
49
49
|
* - async (event) => void after each LLM call; forwards usage to
|
|
50
50
|
* gate.record (via wireGate). `event.kind` discriminates the source: `'turn'` for a main-loop round,
|
|
51
|
-
* `'summarize'` for an out-of-band `ctx.summarize` call (R-C6). Both count against the budget.
|
|
51
|
+
* `'summarize'` for an out-of-band `ctx.summarize` call (R-C6). Both count against the budget. Each
|
|
52
|
+
* event also carries `rateSource` ('provider'|'caller'|'tier'|'default'|null, BA-21) beside the
|
|
53
|
+
* unchanged two-value `pricing` — a 'tier'/'default' cost is a flagged guesstimate, never a silent
|
|
54
|
+
* guess ('tier' = recognized Claude tier, 'default' = blind ceiling fallback).
|
|
52
55
|
*/
|
|
53
56
|
onLlmResult?: Function | undefined;
|
|
57
|
+
/**
|
|
58
|
+
* - BA-21:
|
|
59
|
+
* per-1K USD rates for THIS run's model (`in`/`out` required, finite, non-negative; cache multipliers
|
|
60
|
+
* default to Anthropic's 0.1×/1.25×). When set, every token-only round is priced from these
|
|
61
|
+
* (rateSource:'caller') instead of the built-in guesstimate. There is NO per-model rate table — bring
|
|
62
|
+
* your own rate, or take a flagged guesstimate (haiku/sonnet recognized, else the Sonnet-tier default).
|
|
63
|
+
*/
|
|
64
|
+
rates?: {
|
|
65
|
+
in: number;
|
|
66
|
+
out: number;
|
|
67
|
+
cacheReadMult?: number;
|
|
68
|
+
cacheWriteMult?: number;
|
|
69
|
+
} | undefined;
|
|
54
70
|
onToolResult?: Function | undefined;
|
|
55
71
|
/**
|
|
56
72
|
* - BA-12 safety net (default 3). Short-circuit the run when a
|
|
@@ -108,6 +124,12 @@ export class Loop {
|
|
|
108
124
|
trim: Function | null;
|
|
109
125
|
onLlmResult: Function | null;
|
|
110
126
|
onToolResult: Function | null;
|
|
127
|
+
rates: {
|
|
128
|
+
in: number;
|
|
129
|
+
out: number;
|
|
130
|
+
cacheReadMult?: number;
|
|
131
|
+
cacheWriteMult?: number;
|
|
132
|
+
} | null;
|
|
111
133
|
_stopped: boolean;
|
|
112
134
|
/** @type {Message[]} */
|
|
113
135
|
_history: Message[];
|
|
@@ -117,6 +139,9 @@ export class Loop {
|
|
|
117
139
|
* @param {Record<string, any>} [extra]
|
|
118
140
|
*/
|
|
119
141
|
_reportError(source: string, err: any, extra?: Record<string, any>): void;
|
|
142
|
+
/** @param {string|null} model @param {string} rateSource - 'tier' | 'default' */
|
|
143
|
+
_warnGuesstimateOnce(model: string | null, rateSource: string): void;
|
|
144
|
+
_warnedGuesstimate: boolean | undefined;
|
|
120
145
|
/** @param {{type: string, data?: any, ts?: string}} event */
|
|
121
146
|
_safeEmit(event: {
|
|
122
147
|
type: string;
|
|
@@ -225,82 +250,69 @@ export class Loop {
|
|
|
225
250
|
* Estimate the USD cost of one round's usage, pricing the FOUR token tiers separately (D9/L7):
|
|
226
251
|
* uncached input, output, cache-read, and cache-creation. Folding cache tokens into the full input
|
|
227
252
|
* rate mis-prices badly — a warm prompt is mostly cache-read (~0.1–0.5× input) and Anthropic's
|
|
228
|
-
* cache-creation is a ~1.25× premium — so each tier gets its own rate.
|
|
229
|
-
*
|
|
253
|
+
* cache-creation is a ~1.25× premium — so each tier gets its own rate. The rate comes from
|
|
254
|
+
* `resolveRates` (caller `rates` → recognized tier → Sonnet-tier ceiling default), so a token-bearing
|
|
255
|
+
* round is ALWAYS priced (guesstimate-and-run per BA-21) — a null model no longer forces `unpriced`.
|
|
256
|
+
* Returns null ONLY when usage is absent, or the arithmetic is non-finite (runaway ±Infinity), the
|
|
257
|
+
* genuinely-unpriceable cases that must stay `unpriced` / fail-closed.
|
|
230
258
|
* @param {string|null} model
|
|
231
259
|
* @param {Usage|null} usage
|
|
260
|
+
* @param {{in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}|null} [callerRates]
|
|
232
261
|
* @returns {number|null}
|
|
233
262
|
*/
|
|
234
|
-
export function estimateCost(model: string | null, usage: Usage | null
|
|
235
|
-
|
|
236
|
-
|
|
237
|
-
|
|
238
|
-
|
|
239
|
-
|
|
240
|
-
/**
|
|
241
|
-
|
|
242
|
-
|
|
243
|
-
|
|
244
|
-
|
|
245
|
-
|
|
263
|
+
export function estimateCost(model: string | null, usage: Usage | null, callerRates?: {
|
|
264
|
+
in: number;
|
|
265
|
+
out: number;
|
|
266
|
+
cacheReadMult?: number;
|
|
267
|
+
cacheWriteMult?: number;
|
|
268
|
+
} | null): number | null;
|
|
269
|
+
/**
|
|
270
|
+
* Resolve the effective per-1K rate for a round, and WHERE it came from. Caller-supplied `rates` win
|
|
271
|
+
* (authoritative to the caller, `'caller'`); else a recognized Claude tier matched by the model id
|
|
272
|
+
* (`'tier'` — a confident but non-caller-vouched guess); else the Sonnet-tier ceiling fallback
|
|
273
|
+
* (`'default'` — a blind guess for an unknown/absent model). A null model still resolves to the
|
|
274
|
+
* ceiling — we guesstimate and run, never refuse. `'tier'` and `'default'` are BOTH built-in
|
|
275
|
+
* guesstimates (see `isGuesstimateSource`); the split only lets a consumer tell a recognized-tier
|
|
276
|
+
* price from a blind ceiling in its own ledger (BA-21 follow-up, requested by the bareloop adopter).
|
|
277
|
+
* @param {string|null|undefined} model
|
|
278
|
+
* @param {{in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}|null} [callerRates]
|
|
279
|
+
* @returns {{rates: {in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}, source: 'caller'|'tier'|'default'}}
|
|
280
|
+
*/
|
|
281
|
+
export function resolveRates(model: string | null | undefined, callerRates?: {
|
|
282
|
+
in: number;
|
|
283
|
+
out: number;
|
|
284
|
+
cacheReadMult?: number;
|
|
285
|
+
cacheWriteMult?: number;
|
|
286
|
+
} | null): {
|
|
287
|
+
rates: {
|
|
288
|
+
in: number;
|
|
289
|
+
out: number;
|
|
290
|
+
cacheReadMult?: number;
|
|
291
|
+
cacheWriteMult?: number;
|
|
292
|
+
};
|
|
293
|
+
source: "caller" | "tier" | "default";
|
|
294
|
+
};
|
|
246
295
|
/**
|
|
247
|
-
*
|
|
248
|
-
*
|
|
249
|
-
*
|
|
250
|
-
*
|
|
251
|
-
*
|
|
252
|
-
*
|
|
253
|
-
*
|
|
254
|
-
*
|
|
255
|
-
* @
|
|
256
|
-
* @
|
|
257
|
-
* @
|
|
258
|
-
* @
|
|
259
|
-
* @
|
|
260
|
-
* window sent to the provider each round (e.g. a context-engineering library). Returns a VIEW — the
|
|
261
|
-
* canonical transcript is never mutated. Fail-open (a thrown error degrades to full context); a
|
|
262
|
-
* thrown HaltError propagates. `ctx` is the per-run opaque blob (`run(msgs, tools, { ctx })`), the
|
|
263
|
-
* same object forwarded to `policy`; litectx reads `ctx.task` (intent) and `ctx.budget`. The
|
|
264
|
-
* neutral-unit signature `assemble(units, ctx)` is provided by bareagent's msgs⇄units adapter
|
|
265
|
-
* (src/context-units.js), which composes over this msgs-level seam. When `ctx` is an object, the
|
|
266
|
-
* Loop also lends a provider-bound `ctx.summarize(excerpt, opts?) => Promise<string>` (R-C6,
|
|
267
|
-
* non-enumerable): assemble calls it to roll a summary window — bareagent makes the one model
|
|
268
|
-
* call, the consumer owns the trigger/N/splice. Its usage is forwarded to `onLlmResult` so the
|
|
269
|
-
* summary tokens count against the budget.
|
|
270
|
-
* @property {Function} [trim] - async (msgs, ctx) => msgs. DESTRUCTIVE transcript-trim chokepoint (RT-2),
|
|
271
|
-
* the opposite of `assemble`: it BOUNDS the canonical transcript — the Loop replaces `msgs` with what
|
|
272
|
-
* this returns, evicting old turns AFTER they are harvested. Runs once per round before `assemble`.
|
|
273
|
-
* So eviction never drops un-persisted history, wire it via `unitTrimmer({ trim, onHarvest, policy })`
|
|
274
|
-
* (src/context-units.js), which performs the harvest-before-evict interlock over litectx's `trim` verb.
|
|
275
|
-
* An optional `.flush(msgs, ctx)` method is called on clean completion for the residual-window harvest.
|
|
276
|
-
* Fail-open (a trim fault degrades to no eviction that round); a thrown HaltError propagates.
|
|
277
|
-
* @property {Function} [onLlmResult] - async (event) => void after each LLM call; forwards usage to
|
|
278
|
-
* gate.record (via wireGate). `event.kind` discriminates the source: `'turn'` for a main-loop round,
|
|
279
|
-
* `'summarize'` for an out-of-band `ctx.summarize` call (R-C6). Both count against the budget.
|
|
280
|
-
* @property {Function} [onToolResult]
|
|
281
|
-
* @property {number} [maxIdenticalToolErrors] - BA-12 safety net (default 3). Short-circuit the run when a
|
|
282
|
-
* tool's `execute` throws this many times IN A ROW for a BYTE-IDENTICAL call (same tool + same args). A
|
|
283
|
-
* tool error is deliberately fed back to the model so it can recover — that is the point of the feedback
|
|
284
|
-
* loop — but a model re-issuing the SAME impossible call verbatim can never succeed, and spins to the
|
|
285
|
-
* budget cap with no progress (observed live: `claude-sonnet-5` retried a rejected write 8/8 times).
|
|
286
|
-
* Deliberately the NARROWEST guard: any tool call that SUCCEEDS, or the same tool called with DIFFERENT
|
|
287
|
-
* arguments, resets the streak — a model adapting its input in response to an error is genuinely
|
|
288
|
-
* recovering and is never penalised. Returns cleanly with `error: 'stuck:<tool>'` (mirrors the deny/halt
|
|
289
|
-
* returns; never throws even under `throwOnError`; transcript sealed; the model's text preserved).
|
|
290
|
-
* `0`/`Infinity` disables (restores pre-BA-12 behavior: errors are advisory forever).
|
|
291
|
-
* @property {number} [maxConsecutiveDenials] - BA-11 safety net (default 3). Short-circuit the run when
|
|
292
|
-
* `policy` denies this many tool calls IN A ROW with no allowed call in between — a governance deny is
|
|
293
|
-
* not a recoverable tool error, so a model that keeps retrying variants of a denied action would
|
|
294
|
-
* otherwise burn the budget to the cap without progress (probe-16: 16 calls, sensor never reached). Any
|
|
295
|
-
* tool call that PASSES policy resets the streak, preserving allowlist-safe pivoting (deny X → allow Y).
|
|
296
|
-
* The run returns cleanly with `error: 'denied:<tool>'` (mirrors the halt return; never throws even under
|
|
297
|
-
* throwOnError). Set `0` or `Infinity` to disable (restores pre-BA-11 advisory-deny behavior).
|
|
298
|
-
* @property {number} [maxRounds] - Removed in v0.8; presence throws a migration error.
|
|
296
|
+
* Resolve the priced USD for a round AND its `rateSource`. A provider MAY report its own authoritative
|
|
297
|
+
* `costUsd` on the GenerateResult (e.g. CLIPipeProvider surfacing the claude CLI's `total_cost_usd` — a
|
|
298
|
+
* real price, `source:'provider'`). When present as a FINITE number it wins over any estimate —
|
|
299
|
+
* including `0`, a valid priced value (a subscription/marginal-$0 run), which stays priced. A non-finite
|
|
300
|
+
* provider cost (±Inf/NaN) is NOT a price → fall through to the rate-based estimate (`caller`/`default`).
|
|
301
|
+
* `source` is null only when the round is genuinely unpriced (no usage, or a non-finite estimate).
|
|
302
|
+
* A rate-estimated round is `'caller'` (caller rates), `'tier'` (recognized Claude tier), or `'default'`
|
|
303
|
+
* (blind ceiling fallback) — the last two are both built-in guesstimates (`isGuesstimateSource`).
|
|
304
|
+
* @param {any} result - the GenerateResult from provider.generate()
|
|
305
|
+
* @param {string|null} model
|
|
306
|
+
* @param {Usage|null} usage
|
|
307
|
+
* @param {{in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}|null} [callerRates]
|
|
308
|
+
* @returns {{cost: number|null, source: 'provider'|'caller'|'tier'|'default'|null}}
|
|
299
309
|
*/
|
|
300
|
-
|
|
301
|
-
export const COST_PER_1K: Record<string, {
|
|
310
|
+
export function resolveRoundCost(result: any, model: string | null, usage: Usage | null, callerRates?: {
|
|
302
311
|
in: number;
|
|
303
312
|
out: number;
|
|
304
313
|
cacheReadMult?: number;
|
|
305
314
|
cacheWriteMult?: number;
|
|
306
|
-
}
|
|
315
|
+
} | null): {
|
|
316
|
+
cost: number | null;
|
|
317
|
+
source: "provider" | "caller" | "tier" | "default" | null;
|
|
318
|
+
};
|
package/src/loop.js
CHANGED
|
@@ -48,7 +48,15 @@ const { classifyStopReason } = require('./provider-stop-reason');
|
|
|
48
48
|
* Fail-open (a trim fault degrades to no eviction that round); a thrown HaltError propagates.
|
|
49
49
|
* @property {Function} [onLlmResult] - async (event) => void after each LLM call; forwards usage to
|
|
50
50
|
* gate.record (via wireGate). `event.kind` discriminates the source: `'turn'` for a main-loop round,
|
|
51
|
-
* `'summarize'` for an out-of-band `ctx.summarize` call (R-C6). Both count against the budget.
|
|
51
|
+
* `'summarize'` for an out-of-band `ctx.summarize` call (R-C6). Both count against the budget. Each
|
|
52
|
+
* event also carries `rateSource` ('provider'|'caller'|'tier'|'default'|null, BA-21) beside the
|
|
53
|
+
* unchanged two-value `pricing` — a 'tier'/'default' cost is a flagged guesstimate, never a silent
|
|
54
|
+
* guess ('tier' = recognized Claude tier, 'default' = blind ceiling fallback).
|
|
55
|
+
* @property {{in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}} [rates] - BA-21:
|
|
56
|
+
* per-1K USD rates for THIS run's model (`in`/`out` required, finite, non-negative; cache multipliers
|
|
57
|
+
* default to Anthropic's 0.1×/1.25×). When set, every token-only round is priced from these
|
|
58
|
+
* (rateSource:'caller') instead of the built-in guesstimate. There is NO per-model rate table — bring
|
|
59
|
+
* your own rate, or take a flagged guesstimate (haiku/sonnet recognized, else the Sonnet-tier default).
|
|
52
60
|
* @property {Function} [onToolResult]
|
|
53
61
|
* @property {number} [maxIdenticalToolErrors] - BA-12 safety net (default 3). Short-circuit the run when a
|
|
54
62
|
* tool's `execute` throws this many times IN A ROW for a BYTE-IDENTICAL call (same tool + same args). A
|
|
@@ -70,41 +78,70 @@ const { classifyStopReason } = require('./provider-stop-reason');
|
|
|
70
78
|
* @property {number} [maxRounds] - Removed in v0.8; presence throws a migration error.
|
|
71
79
|
*/
|
|
72
80
|
|
|
73
|
-
//
|
|
74
|
-
//
|
|
75
|
-
//
|
|
76
|
-
//
|
|
77
|
-
//
|
|
78
|
-
//
|
|
79
|
-
//
|
|
80
|
-
//
|
|
81
|
+
// BA-21 — pricing is "bring your own rate, or take a flagged guesstimate", NOT a maintained per-model
|
|
82
|
+
// table (a table rots: every new model + every price change is a code edit, and a stale row silently
|
|
83
|
+
// misprices). No vanilla LLM API returns a price — every provider reports tokens only (the sole real $
|
|
84
|
+
// in the codebase comes from the CLIPipe/Claude-CLI harness's own `total_cost_usd`, handled via the
|
|
85
|
+
// provider `costUsd` path). So a token-only round is priced from a RATE, surfaced on every metering
|
|
86
|
+
// payload as `rateSource` (four non-null values — two authoritative, two built-in guesstimates):
|
|
87
|
+
// 'provider' — the provider reported an authoritative `costUsd` (real; e.g. the CLI harness).
|
|
88
|
+
// 'caller' — the caller passed `new Loop({ rates })` with its own model's numbers (authoritative to it).
|
|
89
|
+
// 'tier' — a recognized Claude tier (haiku/sonnet) matched by the model id: a confident but
|
|
90
|
+
// non-caller-vouched GUESSTIMATE (the hardcoded tier rate could drift from Anthropic's).
|
|
91
|
+
// 'default' — nobody supplied a rate and the model was unrecognized/absent, so we GUESSTIMATE and RUN
|
|
92
|
+
// off the ceiling, flagged as a blind guess (never a silent refuse — governance's job is
|
|
93
|
+
// to keep the user in the know, not to stop on a missing rate).
|
|
94
|
+
// 'tier' and 'default' are BOTH guesstimates (`isGuesstimateSource`): they drive the pass-but-warn and
|
|
95
|
+
// the `estimatedRounds` counter identically; the split only lets a consumer tell a recognized-tier price
|
|
96
|
+
// from a blind ceiling in its own ledger (BA-21 follow-up, requested by the bareloop adopter).
|
|
97
|
+
// The guesstimate recognizes the two Claude tiers in common use (haiku/sonnet) and otherwise falls to
|
|
98
|
+
// the Sonnet-tier default — the common workhorse rate, "usually more expensive" than the cheap models
|
|
99
|
+
// so a budget cap over- rather than under-reports on an unrecognized model, without the absurd
|
|
100
|
+
// over-report an Opus ceiling would inflict on a cheap one. Any 'default'-priced round ALSO emits a
|
|
101
|
+
// one-time loud warning per Loop instance (pass-but-warn) so the guess is never quietly relied on — a
|
|
102
|
+
// caller pricing a different/cheaper model passes `rates` to correct it (and silence the warning).
|
|
103
|
+
// Rates are USD per 1K tokens; `cacheReadMult`/`cacheWriteMult` are multipliers ON the input rate for
|
|
104
|
+
// the two cache tiers (default to Anthropic's 0.1×/1.25× when omitted).
|
|
105
|
+
/** @type {{in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}} */
|
|
106
|
+
const DEFAULT_RATES = { in: 0.003, out: 0.015 }; // Sonnet-tier: the common workhorse rate, the guesstimate default
|
|
81
107
|
/** @type {Record<string, {in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}>} */
|
|
82
|
-
const
|
|
83
|
-
|
|
84
|
-
|
|
85
|
-
'gpt-4o-mini': { in: 0.00015, out: 0.0006, cacheReadMult: 0.5 },
|
|
86
|
-
'gpt-4.1': { in: 0.002, out: 0.008, cacheReadMult: 0.5 },
|
|
87
|
-
'gpt-4.1-mini': { in: 0.0004, out: 0.0016, cacheReadMult: 0.5 },
|
|
88
|
-
'gpt-4.1-nano': { in: 0.0001, out: 0.0004, cacheReadMult: 0.5 },
|
|
89
|
-
'o3-mini': { in: 0.0011, out: 0.0044, cacheReadMult: 0.5 },
|
|
90
|
-
// Anthropic — Claude current generation (2026-06). Cache tiers use the default 0.1×/1.25×.
|
|
91
|
-
'claude-fable-5': { in: 0.01, out: 0.05 },
|
|
92
|
-
'claude-opus-4-8': { in: 0.005, out: 0.025 },
|
|
93
|
-
'claude-opus-4-7': { in: 0.005, out: 0.025 },
|
|
94
|
-
'claude-opus-4-6': { in: 0.005, out: 0.025 },
|
|
95
|
-
'claude-sonnet-4-6': { in: 0.003, out: 0.015 },
|
|
96
|
-
'claude-haiku-4-5-20251001': { in: 0.001, out: 0.005 },
|
|
97
|
-
'claude-haiku-4-5': { in: 0.001, out: 0.005 },
|
|
98
|
-
// Anthropic — earlier snapshots (the original Opus 4 / Sonnet 4 generation, genuinely different rates)
|
|
99
|
-
'claude-sonnet-4-20250514': { in: 0.003, out: 0.015 },
|
|
100
|
-
'claude-opus-4-20250514': { in: 0.015, out: 0.075 },
|
|
101
|
-
// Google Gemini — cached content ~0.25× (no write tier). Native provider lands in a following piece.
|
|
102
|
-
'gemini-2.5-flash': { in: 0.0003, out: 0.0025, cacheReadMult: 0.25 },
|
|
103
|
-
'gemini-2.5-pro': { in: 0.00125, out: 0.01, cacheReadMult: 0.25 },
|
|
104
|
-
// Fallback average across popular models (~$0.002 in, ~$0.008 out per 1K)
|
|
105
|
-
'_default': { in: 0.002, out: 0.008 },
|
|
108
|
+
const TIER_RATES = {
|
|
109
|
+
haiku: { in: 0.001, out: 0.005 }, // low tier (cheaper than the default — recognized so it isn't over-reported)
|
|
110
|
+
sonnet: { in: 0.003, out: 0.015 }, // middle tier (== the default; standing rate, over-reports vs the intro rate)
|
|
106
111
|
};
|
|
107
112
|
|
|
113
|
+
/**
|
|
114
|
+
* Resolve the effective per-1K rate for a round, and WHERE it came from. Caller-supplied `rates` win
|
|
115
|
+
* (authoritative to the caller, `'caller'`); else a recognized Claude tier matched by the model id
|
|
116
|
+
* (`'tier'` — a confident but non-caller-vouched guess); else the Sonnet-tier ceiling fallback
|
|
117
|
+
* (`'default'` — a blind guess for an unknown/absent model). A null model still resolves to the
|
|
118
|
+
* ceiling — we guesstimate and run, never refuse. `'tier'` and `'default'` are BOTH built-in
|
|
119
|
+
* guesstimates (see `isGuesstimateSource`); the split only lets a consumer tell a recognized-tier
|
|
120
|
+
* price from a blind ceiling in its own ledger (BA-21 follow-up, requested by the bareloop adopter).
|
|
121
|
+
* @param {string|null|undefined} model
|
|
122
|
+
* @param {{in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}|null} [callerRates]
|
|
123
|
+
* @returns {{rates: {in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}, source: 'caller'|'tier'|'default'}}
|
|
124
|
+
*/
|
|
125
|
+
function resolveRates(model, callerRates) {
|
|
126
|
+
if (callerRates) return { rates: callerRates, source: 'caller' };
|
|
127
|
+
if (model) {
|
|
128
|
+
const m = String(model).toLowerCase();
|
|
129
|
+
if (m.includes('haiku')) return { rates: TIER_RATES.haiku, source: 'tier' };
|
|
130
|
+
if (m.includes('sonnet')) return { rates: TIER_RATES.sonnet, source: 'tier' };
|
|
131
|
+
}
|
|
132
|
+
return { rates: DEFAULT_RATES, source: 'default' };
|
|
133
|
+
}
|
|
134
|
+
|
|
135
|
+
// BA-21 follow-up: a built-in guesstimate is either a recognized-tier match (`'tier'`) or the blind
|
|
136
|
+
// ceiling fallback (`'default'`) — both are non-authoritative (unlike `'provider'`/`'caller'`), so both
|
|
137
|
+
// drive the pass-but-warn and the `estimatedRounds` counter. One predicate keeps those two call sites
|
|
138
|
+
// from drifting apart if the vocabulary grows again. Typed as a TS predicate so a guarded call narrows
|
|
139
|
+
// `rateSource` to the two guesstimate literals (drops `null`) before it reaches the warn.
|
|
140
|
+
/** @param {string|null} source @returns {source is 'tier'|'default'} */
|
|
141
|
+
function isGuesstimateSource(source) {
|
|
142
|
+
return source === 'tier' || source === 'default';
|
|
143
|
+
}
|
|
144
|
+
|
|
108
145
|
// Internal safety net only — real iteration bounds come from a wired bareguard
|
|
109
146
|
// Gate via `limits.maxTurns`. If you hit this without bareguard wired, you have
|
|
110
147
|
// no governance and the LLM loop is unbounded by design — wire bareguard.
|
|
@@ -143,15 +180,19 @@ function sealDanglingToolCalls(msgs, marker) {
|
|
|
143
180
|
* Estimate the USD cost of one round's usage, pricing the FOUR token tiers separately (D9/L7):
|
|
144
181
|
* uncached input, output, cache-read, and cache-creation. Folding cache tokens into the full input
|
|
145
182
|
* rate mis-prices badly — a warm prompt is mostly cache-read (~0.1–0.5× input) and Anthropic's
|
|
146
|
-
* cache-creation is a ~1.25× premium — so each tier gets its own rate.
|
|
147
|
-
*
|
|
183
|
+
* cache-creation is a ~1.25× premium — so each tier gets its own rate. The rate comes from
|
|
184
|
+
* `resolveRates` (caller `rates` → recognized tier → Sonnet-tier ceiling default), so a token-bearing
|
|
185
|
+
* round is ALWAYS priced (guesstimate-and-run per BA-21) — a null model no longer forces `unpriced`.
|
|
186
|
+
* Returns null ONLY when usage is absent, or the arithmetic is non-finite (runaway ±Infinity), the
|
|
187
|
+
* genuinely-unpriceable cases that must stay `unpriced` / fail-closed.
|
|
148
188
|
* @param {string|null} model
|
|
149
189
|
* @param {Usage|null} usage
|
|
190
|
+
* @param {{in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}|null} [callerRates]
|
|
150
191
|
* @returns {number|null}
|
|
151
192
|
*/
|
|
152
|
-
function estimateCost(model, usage) {
|
|
153
|
-
if (!usage
|
|
154
|
-
const rates =
|
|
193
|
+
function estimateCost(model, usage, callerRates) {
|
|
194
|
+
if (!usage) return null;
|
|
195
|
+
const { rates } = resolveRates(model, callerRates || null);
|
|
155
196
|
const readMult = rates.cacheReadMult ?? 0.1; // Anthropic convention when unspecified
|
|
156
197
|
const writeMult = rates.cacheWriteMult ?? 1.25;
|
|
157
198
|
const cost = (
|
|
@@ -160,29 +201,35 @@ function estimateCost(model, usage) {
|
|
|
160
201
|
(usage.cacheReadTokens || 0) * rates.in * readMult +
|
|
161
202
|
(usage.cacheCreationTokens || 0) * rates.in * writeMult
|
|
162
203
|
) / 1000;
|
|
163
|
-
// A non-finite cost (±Infinity from runaway token counts, NaN from a garbage rate
|
|
204
|
+
// A non-finite cost (±Infinity from runaway token counts, NaN from a garbage caller rate) is a
|
|
164
205
|
// COULDN'T-PRICE, not a price. Return null so the round is marked `unpriced` and the value never
|
|
165
206
|
// poisons `totalCost`, `result.metrics.costUsd`, or — via onLlmResult → the gate — `spentUsd`. The
|
|
166
207
|
// last is the dangerous one: `NaN >= cap` is false, which would DISABLE a budget cap, not just
|
|
167
|
-
// under-count it.
|
|
208
|
+
// under-count it. This is the ONE fail-closed pricing case (§3.7).
|
|
168
209
|
return Number.isFinite(cost) ? cost : null;
|
|
169
210
|
}
|
|
170
211
|
|
|
171
212
|
/**
|
|
172
|
-
* Resolve the priced USD for a round
|
|
173
|
-
* GenerateResult (e.g. CLIPipeProvider
|
|
174
|
-
*
|
|
175
|
-
*
|
|
176
|
-
*
|
|
177
|
-
*
|
|
213
|
+
* Resolve the priced USD for a round AND its `rateSource`. A provider MAY report its own authoritative
|
|
214
|
+
* `costUsd` on the GenerateResult (e.g. CLIPipeProvider surfacing the claude CLI's `total_cost_usd` — a
|
|
215
|
+
* real price, `source:'provider'`). When present as a FINITE number it wins over any estimate —
|
|
216
|
+
* including `0`, a valid priced value (a subscription/marginal-$0 run), which stays priced. A non-finite
|
|
217
|
+
* provider cost (±Inf/NaN) is NOT a price → fall through to the rate-based estimate (`caller`/`default`).
|
|
218
|
+
* `source` is null only when the round is genuinely unpriced (no usage, or a non-finite estimate).
|
|
219
|
+
* A rate-estimated round is `'caller'` (caller rates), `'tier'` (recognized Claude tier), or `'default'`
|
|
220
|
+
* (blind ceiling fallback) — the last two are both built-in guesstimates (`isGuesstimateSource`).
|
|
178
221
|
* @param {any} result - the GenerateResult from provider.generate()
|
|
179
222
|
* @param {string|null} model
|
|
180
223
|
* @param {Usage|null} usage
|
|
181
|
-
* @
|
|
224
|
+
* @param {{in: number, out: number, cacheReadMult?: number, cacheWriteMult?: number}|null} [callerRates]
|
|
225
|
+
* @returns {{cost: number|null, source: 'provider'|'caller'|'tier'|'default'|null}}
|
|
182
226
|
*/
|
|
183
|
-
function resolveRoundCost(result, model, usage) {
|
|
184
|
-
if (result && Number.isFinite(result.costUsd)) return result.costUsd;
|
|
185
|
-
return
|
|
227
|
+
function resolveRoundCost(result, model, usage, callerRates) {
|
|
228
|
+
if (result && Number.isFinite(result.costUsd)) return { cost: result.costUsd, source: 'provider' };
|
|
229
|
+
if (!usage) return { cost: null, source: null };
|
|
230
|
+
const { source } = resolveRates(model, callerRates || null);
|
|
231
|
+
const cost = estimateCost(model, usage, callerRates || null);
|
|
232
|
+
return { cost, source: cost === null ? null : source };
|
|
186
233
|
}
|
|
187
234
|
|
|
188
235
|
// R-C6: default instruction for the provider-bound `ctx.summarize` lent to the assemble seam.
|
|
@@ -309,6 +356,16 @@ class Loop {
|
|
|
309
356
|
}
|
|
310
357
|
this.onLlmResult = options.onLlmResult || null;
|
|
311
358
|
this.onToolResult = options.onToolResult || null;
|
|
359
|
+
// BA-21: caller-supplied per-1K rates ({in, out, cacheReadMult?, cacheWriteMult?}) for THIS run's
|
|
360
|
+
// model. When set, they price every token-only round (rateSource:'caller') instead of the built-in
|
|
361
|
+
// guesstimate. `in`/`out` are required and must be finite non-negative numbers.
|
|
362
|
+
if (options.rates != null) {
|
|
363
|
+
const r = options.rates;
|
|
364
|
+
if (typeof r !== 'object' || !Number.isFinite(r.in) || !Number.isFinite(r.out) || r.in < 0 || r.out < 0) {
|
|
365
|
+
throw new Error('[Loop] options.rates must be { in, out, cacheReadMult?, cacheWriteMult? } with finite non-negative in/out (USD per 1K tokens)');
|
|
366
|
+
}
|
|
367
|
+
}
|
|
368
|
+
this.rates = options.rates || null;
|
|
312
369
|
this._stopped = false;
|
|
313
370
|
/** @type {Message[]} */
|
|
314
371
|
this._history = []; // for chat() stateful mode
|
|
@@ -334,6 +391,28 @@ class Loop {
|
|
|
334
391
|
}
|
|
335
392
|
}
|
|
336
393
|
|
|
394
|
+
// BA-21 pass-but-warn: the FIRST round priced off a built-in guesstimate (rateSource:'tier' or
|
|
395
|
+
// 'default' — no provider cost, no caller `rates`) emits ONE loud warning per Loop instance. Loud
|
|
396
|
+
// because a silent guess is exactly the honesty gap BA-21 closes; once-per-instance because per-round
|
|
397
|
+
// would be noise. Fires for a recognized-tier price too (it is still non-authoritative), and the
|
|
398
|
+
// message names the ACTUAL source so a 'tier' round isn't mislabelled 'default'. Silenced entirely by
|
|
399
|
+
// passing `new Loop({ rates })`. `console.warn` mirrors the temperature-degrade precedent; the
|
|
400
|
+
// structured signal for programmatic consumers is `rateSource` on every payload.
|
|
401
|
+
/** @param {string|null} model @param {string} rateSource - 'tier' | 'default' */
|
|
402
|
+
_warnGuesstimateOnce(model, rateSource) {
|
|
403
|
+
if (this._warnedGuesstimate) return;
|
|
404
|
+
this._warnedGuesstimate = true;
|
|
405
|
+
// `model` is provider-reported (`result.model || provider.model`) — strip control chars/ANSI and
|
|
406
|
+
// clamp before it reaches stderr, so a loose/hostile provider can't inject terminal escapes or a
|
|
407
|
+
// giant string into this diagnostic. Log-hygiene only; no secret ever rides in a model id.
|
|
408
|
+
const safeModel = model ? String(model).replace(/[\x00-\x1f\x7f]/g, '').slice(0, 80) : null;
|
|
409
|
+
console.warn(
|
|
410
|
+
`[Loop] pricing with a built-in GUESSTIMATE rate (rateSource:'${rateSource}'${safeModel ? `, model '${safeModel}'` : ''}) — `
|
|
411
|
+
+ 'cost/budget figures are approximate. Pass new Loop({ rates: { in, out } }) with your model\'s '
|
|
412
|
+
+ 'USD-per-1K rates for an authoritative price (and to silence this warning).'
|
|
413
|
+
);
|
|
414
|
+
}
|
|
415
|
+
|
|
337
416
|
// Swallow-proof stream emit: a throwing listener must not corrupt Loop state.
|
|
338
417
|
/** @param {{type: string, data?: any, ts?: string}} event */
|
|
339
418
|
_safeEmit(event) {
|
|
@@ -525,6 +604,10 @@ class Loop {
|
|
|
525
604
|
byTool: {},
|
|
526
605
|
tokens: { input: 0, output: 0, cacheCreation: 0, cacheRead: 0 },
|
|
527
606
|
unpricedRounds: 0,
|
|
607
|
+
// BA-21 — priced rounds whose rate was a built-in guesstimate (rateSource:'tier' or 'default'), i.e.
|
|
608
|
+
// no provider cost and no caller `rates`. Not unpriced (they DID price and count against budget); a
|
|
609
|
+
// guess flagged so a consumer can see how much of the spend was estimated vs a real/supplied rate.
|
|
610
|
+
estimatedRounds: 0,
|
|
528
611
|
// §3.6 CE-activity rollup — convenience counts derived in-place from the same events already
|
|
529
612
|
// on the Stream (loop:trim, loop:summarize), not a second source. `compactions` counts
|
|
530
613
|
// destructive trim evictions; `summaries` counts ctx.summarize calls; `tokensTrimmed` is an
|
|
@@ -561,6 +644,7 @@ class Loop {
|
|
|
561
644
|
tokens: { ...metrics.tokens },
|
|
562
645
|
costUsd: pricedAny ? totalCost : null,
|
|
563
646
|
unpricedRounds: metrics.unpricedRounds,
|
|
647
|
+
estimatedRounds: metrics.estimatedRounds, // BA-21 — rounds priced off the built-in guesstimate
|
|
564
648
|
spawned: metrics.byTool.spawn || 0, // §3.6 — spawn-tool invocations (byTool counts every call, incl. denied)
|
|
565
649
|
context: { ...metrics.context }, // §3.6 CE-activity rollup
|
|
566
650
|
memory: { ...metrics.memory }, // §3.6 memory footprint (stashed/episodes/recalls/stored/facts; see init note)
|
|
@@ -606,7 +690,8 @@ class Loop {
|
|
|
606
690
|
const result = await loop.provider.generate(prompt, [], { temperature: 0, ...genOpts });
|
|
607
691
|
const usage = (result && result.usage) || null;
|
|
608
692
|
const model = (result && result.model) || loop.provider.model || null;
|
|
609
|
-
const cost = resolveRoundCost(result, model, usage);
|
|
693
|
+
const { cost, source: rateSource } = resolveRoundCost(result, model, usage, loop.rates);
|
|
694
|
+
if (isGuesstimateSource(rateSource)) loop._warnGuesstimateOnce(model, rateSource);
|
|
610
695
|
if (cost !== null) { totalCost += cost; pricedAny = true; }
|
|
611
696
|
addUsage(usage); // summarize tokens are real spend → count them in the cumulative meter
|
|
612
697
|
metrics.context.summaries++; // §3.6 CE-activity rollup
|
|
@@ -619,6 +704,7 @@ class Loop {
|
|
|
619
704
|
usage,
|
|
620
705
|
costUsd: cost,
|
|
621
706
|
pricing: cost === null ? 'unpriced' : 'priced',
|
|
707
|
+
rateSource, // BA-21: 'provider'|'caller'|'tier'|'default'|null — 'tier'/'default' is a flagged guesstimate
|
|
622
708
|
durationMs: Date.now() - startedAt,
|
|
623
709
|
ctx,
|
|
624
710
|
kind: 'summarize',
|
|
@@ -764,15 +850,25 @@ class Loop {
|
|
|
764
850
|
// Prefer the model the response reports (robust when provider.model is absent or varies per
|
|
765
851
|
// response — e.g. FallbackProvider, or a CircuitBreaker-wrapped provider that drops .model).
|
|
766
852
|
const model = result.model || this.provider.model || null;
|
|
767
|
-
|
|
853
|
+
// BA-23: price THIS round's usage, never the stale `lastUsage` carry-over. `lastUsage` is a truthy
|
|
854
|
+
// zero-object seed (line 530) kept across null/absent-usage rounds (line 832) for the BA-5 returns +
|
|
855
|
+
// ctx.usage publish — but handing it to the resolver made its `if (!usage) return {cost:null}` branch
|
|
856
|
+
// (resolveRoundCost line ~229) DEAD: a genuinely no-usage round was laundered into costUsd:0/priced
|
|
857
|
+
// (against the "honest null if unpriced, never 0" contract), and a MID-RUN no-usage round was priced
|
|
858
|
+
// on the PREVIOUS round's tokens — a stale repeat charge. addUsage below already uses result.usage.
|
|
859
|
+
const { cost: roundCost, source: rateSource } = resolveRoundCost(result, model, result.usage ?? null, this.rates);
|
|
860
|
+
if (isGuesstimateSource(rateSource)) this._warnGuesstimateOnce(model, rateSource);
|
|
768
861
|
if (roundCost !== null) totalCost += roundCost;
|
|
769
862
|
|
|
770
863
|
// Meter this round: count the turn, accumulate the four token tiers, and classify pricing —
|
|
771
|
-
// an unpriced round (null cost: no
|
|
772
|
-
// unenforceable on budget rather than silently free (the #3 cost contract).
|
|
864
|
+
// an unpriced round (null cost: no usage, or a runaway non-finite estimate) is tallied so the run
|
|
865
|
+
// is observably unenforceable on budget rather than silently free (the #3 cost contract). A round
|
|
866
|
+
// priced off a built-in guesstimate (rateSource:'tier' or 'default') is counted as an estimate too,
|
|
867
|
+
// so a consumer can see how much of the spend was guessed (BA-21 — keep the user in the know).
|
|
773
868
|
metrics.turns++;
|
|
774
869
|
addUsage(result.usage);
|
|
775
870
|
if (roundCost === null) metrics.unpricedRounds++; else pricedAny = true;
|
|
871
|
+
if (isGuesstimateSource(rateSource)) metrics.estimatedRounds++;
|
|
776
872
|
// BA-16: a cycle-owning provider reports what really happened inside its session.
|
|
777
873
|
const session = (result.session && typeof result.session === 'object') ? result.session : null;
|
|
778
874
|
if (session) {
|
|
@@ -796,8 +892,11 @@ class Loop {
|
|
|
796
892
|
usage: result.usage || null,
|
|
797
893
|
costUsd: roundCost,
|
|
798
894
|
// Priced vs unpriced is explicit so the gate never mistakes "couldn't price" (null) for
|
|
799
|
-
// "free" (0) — the silent-zero that made #3's budget cap a no-op. (D5 / §3.7.)
|
|
895
|
+
// "free" (0) — the silent-zero that made #3's budget cap a no-op. (D5 / §3.7.) `rateSource`
|
|
896
|
+
// is the honesty axis BESIDE it: 'default' means the number is a flagged guesstimate, never
|
|
897
|
+
// a silent guess stamped as if it were a real rate (BA-21). `pricing` keeps its two values.
|
|
800
898
|
pricing: roundCost === null ? 'unpriced' : 'priced',
|
|
899
|
+
rateSource, // 'provider'|'caller'|'tier'|'default'|null
|
|
801
900
|
durationMs: Date.now() - llmStartedAt,
|
|
802
901
|
ctx,
|
|
803
902
|
kind: 'turn',
|
|
@@ -1213,4 +1312,4 @@ class Loop {
|
|
|
1213
1312
|
}
|
|
1214
1313
|
}
|
|
1215
1314
|
|
|
1216
|
-
module.exports = { Loop, estimateCost,
|
|
1315
|
+
module.exports = { Loop, estimateCost, resolveRates, resolveRoundCost };
|
|
@@ -462,6 +462,7 @@ function createSessionStream({ onTurn, ctx, startedAt, onHalt, turnCap = null, o
|
|
|
462
462
|
usage,
|
|
463
463
|
costUsd: null, // the CLI prices the SESSION, not the turn — explicitly unpriced, never a synthetic 0.
|
|
464
464
|
pricing: 'unpriced',
|
|
465
|
+
rateSource: null, // BA-22: no price to vouch for → null, NEVER 'provider' (negative control); rides every payload
|
|
465
466
|
durationMs: Date.now() - startedAt,
|
|
466
467
|
ctx, // what a wired gate records spend against — same as the Loop's onLlmResult.
|
|
467
468
|
kind: 'turn',
|
|
@@ -59,9 +59,11 @@ export type CLIPipeOptions = {
|
|
|
59
59
|
policy?: ((tool: string, args: any, ctx?: any) => any) | undefined;
|
|
60
60
|
/**
|
|
61
61
|
* - (native mode) Called with `{model, provider, usage, costUsd, pricing,
|
|
62
|
-
* durationMs, ctx, kind}` for EACH completed CLI turn as it arrives (`kind:'turn'`, four
|
|
63
|
-
* `costUsd:null` — the CLI prices the session, not the turn), then once
|
|
64
|
-
* (`kind:'session'`) carrying the authoritative total cost with zero usage
|
|
62
|
+
* rateSource, durationMs, ctx, kind}` for EACH completed CLI turn as it arrives (`kind:'turn'`, four
|
|
63
|
+
* cache tiers, `costUsd:null`/`rateSource:null` — the CLI prices the session, not the turn), then once
|
|
64
|
+
* at session end (`kind:'session'`) carrying the authoritative total cost with zero usage and
|
|
65
|
+
* `rateSource:'provider'` when that cost is finite (BA-22 — the CLI's own `total_cost_usd`, no rate
|
|
66
|
+
* table; `null` cost → `rateSource:null`, never a spurious 'provider'). Streaming, never
|
|
65
67
|
* sum-at-end: a session that dies mid-run must already have surfaced every completed turn's spend or
|
|
66
68
|
* the gate loses all of it. The event shape mirrors `Loop({onLlmResult})`, so `wireGate(gate).onLlmResult`
|
|
67
69
|
* drops straight in — and when it is wired the Loop skips its own forward, so nothing is billed twice.
|
package/src/provider-clipipe.js
CHANGED
|
@@ -37,9 +37,11 @@ const { createBridge, resolveSessionError, runSession } = require('./provider-cl
|
|
|
37
37
|
* Loop — a `Loop({policy})` would be a fence that is silently not there (the Loop throws instead).
|
|
38
38
|
* Wiring the same `wireGate(gate).policy` keeps audit rows byte-shape-identical, with zero gate changes.
|
|
39
39
|
* @property {Function} [onTurn] - (native mode) Called with `{model, provider, usage, costUsd, pricing,
|
|
40
|
-
* durationMs, ctx, kind}` for EACH completed CLI turn as it arrives (`kind:'turn'`, four
|
|
41
|
-
* `costUsd:null` — the CLI prices the session, not the turn), then once
|
|
42
|
-
* (`kind:'session'`) carrying the authoritative total cost with zero usage
|
|
40
|
+
* rateSource, durationMs, ctx, kind}` for EACH completed CLI turn as it arrives (`kind:'turn'`, four
|
|
41
|
+
* cache tiers, `costUsd:null`/`rateSource:null` — the CLI prices the session, not the turn), then once
|
|
42
|
+
* at session end (`kind:'session'`) carrying the authoritative total cost with zero usage and
|
|
43
|
+
* `rateSource:'provider'` when that cost is finite (BA-22 — the CLI's own `total_cost_usd`, no rate
|
|
44
|
+
* table; `null` cost → `rateSource:null`, never a spurious 'provider'). Streaming, never
|
|
43
45
|
* sum-at-end: a session that dies mid-run must already have surfaced every completed turn's spend or
|
|
44
46
|
* the gate loses all of it. The event shape mirrors `Loop({onLlmResult})`, so `wireGate(gate).onLlmResult`
|
|
45
47
|
* drops straight in — and when it is wired the Loop skips its own forward, so nothing is billed twice.
|
|
@@ -369,6 +371,11 @@ class CLIPipeProvider {
|
|
|
369
371
|
usage: residual,
|
|
370
372
|
costUsd,
|
|
371
373
|
pricing: costUsd === null ? 'unpriced' : 'priced',
|
|
374
|
+
// BA-22: the session cost is the claude CLI's own `total_cost_usd` (no local rate table) — a
|
|
375
|
+
// provider reporting its authoritative cost, exactly what rateSource:'provider' means. Stamp it
|
|
376
|
+
// ONLY when finite (costUsd is finite-or-null per the guard above); a null cost never claims
|
|
377
|
+
// 'provider' provenance. Native mode bypasses the Loop's resolveRoundCost, so it stamps its own.
|
|
378
|
+
rateSource: costUsd === null ? null : 'provider',
|
|
372
379
|
durationMs: r.ms,
|
|
373
380
|
ctx: options.ctx,
|
|
374
381
|
kind: 'session',
|
package/types/index.d.ts
CHANGED
|
@@ -47,8 +47,10 @@ export interface RunMetrics {
|
|
|
47
47
|
tokens: { input: number; output: number; cacheCreation: number; cacheRead: number };
|
|
48
48
|
/** Cumulative USD over priced rounds; null ONLY if nothing could be priced (explicit-unknown, not free). */
|
|
49
49
|
costUsd: number | null;
|
|
50
|
-
/** Count of rounds
|
|
50
|
+
/** Count of rounds genuinely unpriceable (no usage, or a runaway non-finite estimate) — the loud-unpriced signal. */
|
|
51
51
|
unpricedRounds: number;
|
|
52
|
+
/** BA-21: count of priced rounds whose rate was a built-in guesstimate (rateSource:'tier' or 'default'), not a real/supplied rate. */
|
|
53
|
+
estimatedRounds: number;
|
|
52
54
|
/** Spawn-tool invocations this run (the `spawn` tool count — counts every call, incl. denied/failed). */
|
|
53
55
|
spawned: number;
|
|
54
56
|
/**
|