@zerotal/ai 1.11.1 → 1.11.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +92 -1
- package/README.md +3 -1
- package/api-surface.md +15 -13
- package/package.json +5 -6
- package/src/AiManager.ts +27 -5
- package/src/config.ts +24 -10
- package/src/drivers/AiDriver.ts +6 -1
- package/src/drivers/AnthropicDriver.ts +78 -7
- package/src/drivers/OllamaDriver.ts +4 -2
- package/src/drivers/OpenAiDriver.ts +4 -2
- package/src/errors.ts +22 -0
- package/src/index.ts +2 -0
- package/src/modelCapabilities.ts +105 -0
- package/src/pricing.ts +10 -14
- package/src/schema.ts +7 -1
- package/src/spend.ts +25 -3
- package/src/stats.ts +4 -1
- package/src/types.ts +14 -0
package/CHANGELOG.md
CHANGED
|
@@ -4,10 +4,101 @@ All notable changes to this package are documented here. The format is
|
|
|
4
4
|
based on [Keep a Changelog](https://keepachangelog.com/); this package
|
|
5
5
|
follows the Zerotal monorepo's unified versioning.
|
|
6
6
|
|
|
7
|
-
**Maturity: `
|
|
7
|
+
**Maturity: `stable`**
|
|
8
8
|
|
|
9
9
|
## [Unreleased]
|
|
10
10
|
|
|
11
|
+
## [1.11.2] — 2026-08-31
|
|
12
|
+
|
|
13
|
+
### Changed
|
|
14
|
+
|
|
15
|
+
- **Promoted to `stable`.** The public API now follows the compatibility promise in
|
|
16
|
+
[the support policy](https://zerotal.dev/docs/support-policy): anything importable
|
|
17
|
+
without an `@internal` marker is covered.
|
|
18
|
+
|
|
19
|
+
The precondition was met rather than waived. This package shipped `experimental`
|
|
20
|
+
with a stated one — _it graduates in the release after its first real users_ — and
|
|
21
|
+
its first production users sent a field review of the driver running against
|
|
22
|
+
Anthropic. Five bugs came back with it, all fixed below. A `stable` promise about an
|
|
23
|
+
API that nothing has pushed against is a promise nobody has tested.
|
|
24
|
+
|
|
25
|
+
**The surface was narrowed first**, because narrowing after `stable` is itself a
|
|
26
|
+
breaking change. `toSchema`, `strippedConstraints`, `resetSpend` and `resetStats` are
|
|
27
|
+
`@internal` — still exported, so nothing breaks at runtime, but no longer promised.
|
|
28
|
+
`translateSchema` stayed public despite having no caller outside this package, for
|
|
29
|
+
the same reason `AiDriver` is public: the point of a driver contract is that someone
|
|
30
|
+
else implements it, and implementing structured output means translating a schema.
|
|
31
|
+
`AiDelivery` stayed too — it is the element type of `recentGenerations()`, and
|
|
32
|
+
marking the return type of a public function internal would be the exact lie the
|
|
33
|
+
review is meant to prevent.
|
|
34
|
+
|
|
35
|
+
Then the two modules it would have been embarrassing to freeze untested: the SSE
|
|
36
|
+
parser, which reads a remote provider's framing off the network, and prompt
|
|
37
|
+
redaction, which is the only thing between a user's prompt and a log that outlives
|
|
38
|
+
the request.
|
|
39
|
+
|
|
40
|
+
- **An app with no AI configured now boots.** `AiConfig` threw when no driver was
|
|
41
|
+
declared, and threw again on an empty `apiKey`, so a deployment with no key could
|
|
42
|
+
not express itself either way — one app declared an Ollama server it did not run
|
|
43
|
+
purely to satisfy the validator, with a comment explaining that the config was lying.
|
|
44
|
+
Declaring no driver is now legal and means AI is off; the first call raises
|
|
45
|
+
`AiDriverUnavailableError`, whose `transient` is already `false`, so a caller that
|
|
46
|
+
latches itself off on a permanent error gets the right behaviour for free. Declaring
|
|
47
|
+
a driver still means you want it, so an incomplete one is still refused at boot.
|
|
48
|
+
|
|
49
|
+
- **`countTokens` returns `null` where a provider cannot count**, rather than `0`. Only
|
|
50
|
+
Anthropic has a counting endpoint; `0` is also a real count for an empty prompt, so
|
|
51
|
+
the old return value was a number you could divide by and budget against without ever
|
|
52
|
+
being told it meant "unsupported".
|
|
53
|
+
|
|
54
|
+
### Fixed
|
|
55
|
+
|
|
56
|
+
- **Sonnet 5 was priced as Sonnet 4.6** — 3/15 instead of 2/10, 50% high. The same
|
|
57
|
+
table feeds `limits.perRequestUsd` and `perDayUsd`, so a Sonnet 5 app was refused
|
|
58
|
+
requests comfortably inside its budget and the daily ceiling tripped a third early.
|
|
59
|
+
The error said "spend limit", which sends someone to their config rather than to the
|
|
60
|
+
row that is wrong. `AiSpendLimitError` now quotes the rate it used and names
|
|
61
|
+
`registerModelPrice()`, so a bad table is legible from the refusal and correctable
|
|
62
|
+
without waiting for a release.
|
|
63
|
+
|
|
64
|
+
- **`effort` and `thinking` are model-aware.** Both were sent on every call. `effort`
|
|
65
|
+
is a 400 on the 4.5 generation, and those models want an explicit thinking budget
|
|
66
|
+
rather than `{ type: "adaptive" }` — so the package advertised `claude-haiku-4-5` in
|
|
67
|
+
its pricing table while the driver could not successfully call it. `modelCapabilities()`
|
|
68
|
+
now answers what a model takes, and the driver builds the request that model accepts.
|
|
69
|
+
|
|
70
|
+
The table lists the models that _differ_ and treats anything unrecognised as current
|
|
71
|
+
generation. An allowlist would need an edit every time a model ships and would treat
|
|
72
|
+
each new one as legacy until that edit landed; the models that differ are a closed
|
|
73
|
+
set that ages out.
|
|
74
|
+
|
|
75
|
+
- **`temperature` never reached the API, on any model.** The driver warned about
|
|
76
|
+
dropping it and had no branch that set it, so `AiRequest.temperature` and the
|
|
77
|
+
configured default were both inert everywhere — including on the 4.6 models, which
|
|
78
|
+
accept it. The old sampling predicate hid this by warning for almost every model,
|
|
79
|
+
which made the silence look deliberate on the few it did not warn for. Sampling
|
|
80
|
+
parameters are now sent where the model takes them.
|
|
81
|
+
|
|
82
|
+
- **`modelRejectsSampling` was too broad**, reading as "every Claude model except 4.6",
|
|
83
|
+
which is wrong for the 4.5 generation. It compounded with the above: on Haiku 4.5 the
|
|
84
|
+
driver dropped a legal `temperature` and advised `effort` instead — the one parameter
|
|
85
|
+
guaranteed to fail there. The advice is now only given where `effort` exists, in the
|
|
86
|
+
driver and in `validateAiConfig` alike.
|
|
87
|
+
|
|
88
|
+
- **The streamed `thinking` chunk was always empty.** The API omits thinking text by
|
|
89
|
+
default on the current generation, so a documented stream chunk fired forever with
|
|
90
|
+
`text: ""` and no error — and a "thinking…" view built against the 4.6 models, where
|
|
91
|
+
it defaulted to on, silently stopped working as users moved to 5.
|
|
92
|
+
`drivers.anthropic.thinkingDisplay` defaults to `"summarized"`; set `"omitted"` for
|
|
93
|
+
the API's own default. The thinking happens and is billed either way.
|
|
94
|
+
|
|
95
|
+
### Added
|
|
96
|
+
|
|
97
|
+
- **`modelCapabilities()` and `ModelCapabilities`** — what a model accepts: sampling,
|
|
98
|
+
`effort`, and which of the three thinking shapes it takes.
|
|
99
|
+
- **`drivers.anthropic.thinkingDisplay`** — whether a streamed `thinking` chunk carries
|
|
100
|
+
text.
|
|
101
|
+
|
|
11
102
|
## [1.11.0] — 2026-08-31
|
|
12
103
|
|
|
13
104
|
### Fixed
|
package/README.md
CHANGED
|
@@ -6,7 +6,9 @@ One way to talk to a language model: `Ai.text()` for a completion, `Ai.stream()`
|
|
|
6
6
|
|
|
7
7
|
Part of the [Zerotal](../../README.md) framework. Requires **Bun ≥ 1.3.14**.
|
|
8
8
|
|
|
9
|
-
**Maturity: `
|
|
9
|
+
**Maturity: `stable`** — the public API follows the compatibility promise in
|
|
10
|
+
[the support policy](https://zerotal.dev/docs/support-policy). Anything importable
|
|
11
|
+
that does not carry an `@internal` marker is covered.
|
|
10
12
|
|
|
11
13
|
## Installation
|
|
12
14
|
|
package/api-surface.md
CHANGED
|
@@ -31,6 +31,7 @@ class AiConfigError = {
|
|
|
31
31
|
|
|
32
32
|
class AiDriverUnavailableError = {
|
|
33
33
|
new (driver: string, packageName: string): AiDriverUnavailableError
|
|
34
|
+
static notConfigured: (requested: string) => AiDriverUnavailableError
|
|
34
35
|
readonly code: string
|
|
35
36
|
readonly context?: Record<string, unknown> | undefined
|
|
36
37
|
readonly status: number
|
|
@@ -91,7 +92,7 @@ class AiGenerated = {
|
|
|
91
92
|
class AiManager = {
|
|
92
93
|
new (config: AiConfigShape): AiManager
|
|
93
94
|
agent: (request: AiAgentRequest) => Promise<AiAgentResult>
|
|
94
|
-
countTokens: (request: AiRequest | string) => Promise<number>
|
|
95
|
+
countTokens: (request: AiRequest | string) => Promise<number | null>
|
|
95
96
|
driver: (name?: string) => AiDriver
|
|
96
97
|
drivers: () => string[]
|
|
97
98
|
embed: (input: string | string[], options?: Omit<AiEmbedRequest, 'input'>) => Promise<AiEmbedResponse>
|
|
@@ -195,7 +196,7 @@ class AiToolCalled = {
|
|
|
195
196
|
|
|
196
197
|
class AnthropicDriver = {
|
|
197
198
|
new (config: AnthropicConfigShape, inject?: LoadedAnthropic): AnthropicDriver
|
|
198
|
-
countTokens: (request: AiRequest) => Promise<number>
|
|
199
|
+
countTokens: (request: AiRequest) => Promise<number | null>
|
|
199
200
|
model: string
|
|
200
201
|
object: <T>(request: AiRequest, schema: SchemaInput) => Promise<AiObjectResponse<T>>
|
|
201
202
|
readonly name: 'anthropic'
|
|
@@ -206,7 +207,7 @@ class AnthropicDriver = {
|
|
|
206
207
|
|
|
207
208
|
class OllamaDriver = {
|
|
208
209
|
new (config: OllamaConfigShape, fetchImpl?: typeof fetch): OllamaDriver
|
|
209
|
-
countTokens: (_request: AiRequest) => Promise<number>
|
|
210
|
+
countTokens: (_request: AiRequest) => Promise<number | null>
|
|
210
211
|
model: string
|
|
211
212
|
object: <T>(request: AiRequest, schema: SchemaInput) => Promise<AiObjectResponse<T>>
|
|
212
213
|
readonly name: 'ollama'
|
|
@@ -224,7 +225,7 @@ class OllamaEmbeddingsDriver = {
|
|
|
224
225
|
|
|
225
226
|
class OpenAiDriver = {
|
|
226
227
|
new (config: OpenAiConfigShape, fetchImpl?: typeof fetch): OpenAiDriver
|
|
227
|
-
countTokens: (_request: AiRequest) => Promise<number>
|
|
228
|
+
countTokens: (_request: AiRequest) => Promise<number | null>
|
|
228
229
|
model: string
|
|
229
230
|
object: <T>(request: AiRequest, schema: SchemaInput) => Promise<AiObjectResponse<T>>
|
|
230
231
|
readonly name: 'openai'
|
|
@@ -256,6 +257,8 @@ function AiConfigFromEnv = () => AiConfigShape
|
|
|
256
257
|
|
|
257
258
|
function estimateCost = (model: string, usage: AiUsage) => number
|
|
258
259
|
|
|
260
|
+
function modelCapabilities = (model: string) => ModelCapabilities
|
|
261
|
+
|
|
259
262
|
function modelPrice = (model: string) => ModelPrice | undefined
|
|
260
263
|
|
|
261
264
|
function modelRejectsSampling = (model: string) => boolean
|
|
@@ -268,18 +271,10 @@ function refusalRate = () => number
|
|
|
268
271
|
|
|
269
272
|
function registerModelPrice = (model: string, price: ModelPrice) => void
|
|
270
273
|
|
|
271
|
-
function resetSpend = () => void
|
|
272
|
-
|
|
273
|
-
function resetStats = () => void
|
|
274
|
-
|
|
275
274
|
function spentToday = () => number
|
|
276
275
|
|
|
277
|
-
function strippedConstraints = (input: SchemaInput) => string[]
|
|
278
|
-
|
|
279
276
|
function tool = <I extends Record<string, unknown> = Record<string, unknown>>(options: { name: string; description: string; input: ((rule: RuleBuilder) => Record<string, FieldRule>) | SchemaInput; handle: (input: I, ctx: AiToolContext) => Promise<unknown> | unknown;}) => AiTool
|
|
280
277
|
|
|
281
|
-
function toSchema = (input: SchemaInput) => Schema
|
|
282
|
-
|
|
283
278
|
function translateSchema = (input: SchemaInput) => JsonSchema
|
|
284
279
|
|
|
285
280
|
interface AgentOptions = {
|
|
@@ -356,7 +351,7 @@ interface AiDelivery = {
|
|
|
356
351
|
|
|
357
352
|
interface AiDriver = {
|
|
358
353
|
agent?: (request: AiRequest, options: AgentOptions) => Promise<AiAgentResult>
|
|
359
|
-
countTokens: (request: AiRequest) => Promise<number>
|
|
354
|
+
countTokens: (request: AiRequest) => Promise<number | null>
|
|
360
355
|
object: <T>(request: AiRequest, schema: SchemaInput) => Promise<AiObjectResponse<T>>
|
|
361
356
|
readonly model: string
|
|
362
357
|
readonly name: string
|
|
@@ -470,6 +465,7 @@ interface AnthropicConfigShape = {
|
|
|
470
465
|
model: string
|
|
471
466
|
streamMaxTokens: number
|
|
472
467
|
temperature?: number | undefined
|
|
468
|
+
thinkingDisplay: 'summarized' | 'omitted'
|
|
473
469
|
timeout: number
|
|
474
470
|
}
|
|
475
471
|
|
|
@@ -509,6 +505,12 @@ interface JsonSchema = {
|
|
|
509
505
|
type?: 'string' | 'number' | 'boolean' | 'object' | 'array' | 'integer' | 'null'
|
|
510
506
|
}
|
|
511
507
|
|
|
508
|
+
interface ModelCapabilities = {
|
|
509
|
+
effort: boolean
|
|
510
|
+
sampling: boolean
|
|
511
|
+
thinking: 'adaptive' | 'budget' | null
|
|
512
|
+
}
|
|
513
|
+
|
|
512
514
|
interface ModelPrice = {
|
|
513
515
|
input: number
|
|
514
516
|
output: number
|
package/package.json
CHANGED
|
@@ -1,9 +1,8 @@
|
|
|
1
1
|
{
|
|
2
2
|
"name": "@zerotal/ai",
|
|
3
|
-
"version": "1.11.
|
|
3
|
+
"version": "1.11.2",
|
|
4
4
|
"license": "MIT",
|
|
5
|
-
"maturity": "
|
|
6
|
-
"maturityReview": "1.11.0",
|
|
5
|
+
"maturity": "stable",
|
|
7
6
|
"private": false,
|
|
8
7
|
"type": "module",
|
|
9
8
|
"main": "./src/index.ts",
|
|
@@ -31,9 +30,9 @@
|
|
|
31
30
|
"typecheck": "tsc --noEmit"
|
|
32
31
|
},
|
|
33
32
|
"dependencies": {
|
|
34
|
-
"@zerotal/core": "1.11.
|
|
35
|
-
"@zerotal/validator": "1.11.
|
|
36
|
-
"@zerotal/queue": "1.11.
|
|
33
|
+
"@zerotal/core": "1.11.2",
|
|
34
|
+
"@zerotal/validator": "1.11.2",
|
|
35
|
+
"@zerotal/queue": "1.11.2"
|
|
37
36
|
},
|
|
38
37
|
"peerDependencies": {
|
|
39
38
|
"@anthropic-ai/sdk": ">=0.70.0"
|
package/src/AiManager.ts
CHANGED
|
@@ -8,7 +8,13 @@ import { OpenAiEmbeddingsDriver } from "./drivers/embeddings/OpenAiEmbeddingsDri
|
|
|
8
8
|
import type { EmbeddingsDriver } from "./drivers/embeddings/EmbeddingsDriver.ts";
|
|
9
9
|
import type { AiDriver, DriverStatus } from "./drivers/AiDriver.ts";
|
|
10
10
|
import { normalizeMessages, promptText } from "./drivers/AiDriver.ts";
|
|
11
|
-
import {
|
|
11
|
+
import {
|
|
12
|
+
AiCancelledError,
|
|
13
|
+
AiConfigError,
|
|
14
|
+
AiDriverUnavailableError,
|
|
15
|
+
AiRefusedError,
|
|
16
|
+
UnknownAiDriverError,
|
|
17
|
+
} from "./errors.ts";
|
|
12
18
|
import { AiGenerated, AiRefused } from "./events.ts";
|
|
13
19
|
import { estimateCost } from "./pricing.ts";
|
|
14
20
|
import { redactPrompt } from "./redact.ts";
|
|
@@ -144,7 +150,16 @@ export class AiManager {
|
|
|
144
150
|
if (existing) return existing;
|
|
145
151
|
|
|
146
152
|
const resolver = this._resolvers.get(key);
|
|
147
|
-
if (!resolver)
|
|
153
|
+
if (!resolver) {
|
|
154
|
+
// No driver at all is a different situation from a driver named wrongly, and
|
|
155
|
+
// it is now a legal config: an app can ship with AI off and no key. Saying
|
|
156
|
+
// "unknown driver 'anthropic', configured: (none)" invites someone to go
|
|
157
|
+
// looking for a typo. Both errors are permanent (`transient: false`), so a
|
|
158
|
+
// caller latching itself off behaves correctly either way — this only decides
|
|
159
|
+
// which sentence it reads on the way.
|
|
160
|
+
if (this._resolvers.size === 0) throw AiDriverUnavailableError.notConfigured(key);
|
|
161
|
+
throw new UnknownAiDriverError(key, [...this._resolvers.keys()]);
|
|
162
|
+
}
|
|
148
163
|
|
|
149
164
|
const instance = resolver();
|
|
150
165
|
this._drivers.set(key, instance);
|
|
@@ -358,8 +373,12 @@ export class AiManager {
|
|
|
358
373
|
return result;
|
|
359
374
|
}
|
|
360
375
|
|
|
361
|
-
/**
|
|
362
|
-
|
|
376
|
+
/**
|
|
377
|
+
* Count the prompt's tokens with the provider's own tokenizer.
|
|
378
|
+
*
|
|
379
|
+
* `null` when the configured provider cannot count — only Anthropic can, today.
|
|
380
|
+
*/
|
|
381
|
+
async countTokens(request: AiRequest | string): Promise<number | null> {
|
|
363
382
|
const normalized = normalize(request);
|
|
364
383
|
return this.driver(normalized.driver).countTokens(normalized);
|
|
365
384
|
}
|
|
@@ -458,7 +477,10 @@ export class AiManager {
|
|
|
458
477
|
let inputTokens = 0;
|
|
459
478
|
|
|
460
479
|
if (limits.perRequestUsd > 0) {
|
|
461
|
-
|
|
480
|
+
// A provider that cannot count, or one whose count failed, falls back to the
|
|
481
|
+
// approximation. Both were already the same branch when this returned 0 — the
|
|
482
|
+
// difference is that the ambiguity is no longer visible to callers.
|
|
483
|
+
inputTokens = (await driver.countTokens(request).catch(() => null)) ?? 0;
|
|
462
484
|
if (inputTokens === 0) inputTokens = approximateTokens(request);
|
|
463
485
|
}
|
|
464
486
|
|
package/src/config.ts
CHANGED
|
@@ -8,7 +8,7 @@ import type {
|
|
|
8
8
|
OllamaConfigShape,
|
|
9
9
|
OpenAiConfigShape,
|
|
10
10
|
} from "./types.ts";
|
|
11
|
-
import {
|
|
11
|
+
import { modelCapabilities } from "./modelCapabilities.ts";
|
|
12
12
|
|
|
13
13
|
/**
|
|
14
14
|
* What {@link AiConfig} accepts — every key optional, all the way down. The
|
|
@@ -109,6 +109,7 @@ function applyDriverDefaults(config: AiConfigShape): void {
|
|
|
109
109
|
// Streaming has no HTTP-timeout ceiling to respect, so give it room.
|
|
110
110
|
streamMaxTokens: given.streamMaxTokens ?? 64000,
|
|
111
111
|
effort: given.effort ?? "high",
|
|
112
|
+
thinkingDisplay: given.thinkingDisplay ?? "summarized",
|
|
112
113
|
fallbacks: given.fallbacks ?? true,
|
|
113
114
|
cacheSystem: given.cacheSystem ?? true,
|
|
114
115
|
timeout: given.timeout ?? 600_000,
|
|
@@ -192,13 +193,19 @@ export function validateAiConfig(config: AiConfigShape): void {
|
|
|
192
193
|
(name) => drivers[name as keyof typeof drivers] !== undefined,
|
|
193
194
|
);
|
|
194
195
|
|
|
195
|
-
|
|
196
|
-
|
|
197
|
-
|
|
198
|
-
|
|
199
|
-
|
|
200
|
-
|
|
201
|
-
|
|
196
|
+
// "AI is off" is a coherent deployment, and it used to be inexpressible: naming no
|
|
197
|
+
// driver was a boot failure and naming one without a key was also a boot failure,
|
|
198
|
+
// so an app with no key on this machine had to declare a driver it did not run
|
|
199
|
+
// purely to satisfy this function. One did — an Ollama block it had no server for,
|
|
200
|
+
// with a comment explaining that the config was lying to get past the validator.
|
|
201
|
+
//
|
|
202
|
+
// So the split is by intent. *Declaring* a driver still means "I want this", and an
|
|
203
|
+
// incomplete one is still a misconfiguration worth refusing at boot. Declaring
|
|
204
|
+
// none — or pointing `default` at one you did not declare — now means AI is not
|
|
205
|
+
// available here, which is a fact rather than an error. The first call says so,
|
|
206
|
+
// through `AiDriverUnavailableError`, whose `transient` is already `false`: a caller
|
|
207
|
+
// that latches itself off on a permanent error gets the right behaviour for free.
|
|
208
|
+
if (configured.length > 0 && !configured.includes(config.default)) {
|
|
202
209
|
throw new AiConfigError(
|
|
203
210
|
`default is '${config.default}' but that driver has no block. Configured: ${configured.join(", ")}.`,
|
|
204
211
|
{ default: config.default, configured },
|
|
@@ -224,13 +231,20 @@ export function validateAiConfig(config: AiConfigShape): void {
|
|
|
224
231
|
{ maxTokens: a.maxTokens, streamMaxTokens: a.streamMaxTokens },
|
|
225
232
|
);
|
|
226
233
|
}
|
|
227
|
-
|
|
234
|
+
const capabilities = modelCapabilities(a.model);
|
|
235
|
+
if (a.temperature !== undefined && !capabilities.sampling) {
|
|
228
236
|
// A warning rather than a throw: the driver drops it and the request still
|
|
229
237
|
// succeeds. Throwing would break an app whose config merely carries a
|
|
230
238
|
// leftover from a model that accepted it.
|
|
231
239
|
console.warn(
|
|
232
240
|
`[Zerotal/ai] drivers.anthropic.temperature is set, but ${a.model} rejects temperature/top_p/top_k ` +
|
|
233
|
-
`with a 400. The driver drops it
|
|
241
|
+
`with a 400. The driver drops it.` +
|
|
242
|
+
// Only where effort exists. This suggestion used to be unconditional and
|
|
243
|
+
// fired on models that accept the temperature it was telling them to
|
|
244
|
+
// replace — recommending the one parameter that 400s there.
|
|
245
|
+
(capabilities.effort
|
|
246
|
+
? ` Use effort ('low' … 'max') to trade thoroughness for cost instead.`
|
|
247
|
+
: ""),
|
|
234
248
|
);
|
|
235
249
|
}
|
|
236
250
|
}
|
package/src/drivers/AiDriver.ts
CHANGED
|
@@ -61,8 +61,13 @@ export interface AiDriver {
|
|
|
61
61
|
* Count the tokens this request would consume, using the provider's own
|
|
62
62
|
* tokenizer. Never an estimate from another vendor's tokenizer — the spend
|
|
63
63
|
* panel is built on this number.
|
|
64
|
+
*
|
|
65
|
+
* `null` when the provider has no counting endpoint, which is most of them.
|
|
66
|
+
* It used to be `0`, and a zero that means "cannot count" is indistinguishable
|
|
67
|
+
* from a zero that means "empty prompt" — easy to divide by, easy to budget
|
|
68
|
+
* against, and wrong in the direction that costs money.
|
|
64
69
|
*/
|
|
65
|
-
countTokens(request: AiRequest): Promise<number>;
|
|
70
|
+
countTokens(request: AiRequest): Promise<number | null>;
|
|
66
71
|
|
|
67
72
|
/** Reach the provider once and report what came back. Backs `zt ai:test`. */
|
|
68
73
|
verify(): Promise<DriverStatus>;
|
|
@@ -7,7 +7,8 @@ import {
|
|
|
7
7
|
AiSchemaError,
|
|
8
8
|
} from "../errors.ts";
|
|
9
9
|
import { recheckAgainstSchema, translateSchema, type SchemaInput } from "../schema.ts";
|
|
10
|
-
import {
|
|
10
|
+
import { modelCapabilities } from "../modelCapabilities.ts";
|
|
11
|
+
import type { ModelCapabilities } from "../modelCapabilities.ts";
|
|
11
12
|
import type {
|
|
12
13
|
AiMessage,
|
|
13
14
|
AiObjectResponse,
|
|
@@ -35,6 +36,14 @@ import type {
|
|
|
35
36
|
/** Opts into the server-side refusal fallback. */
|
|
36
37
|
const FALLBACK_BETA = "server-side-fallback-2026-07-01";
|
|
37
38
|
|
|
39
|
+
/**
|
|
40
|
+
* The API's floor for an explicit thinking budget, on the models that take one.
|
|
41
|
+
*
|
|
42
|
+
* A budget below this is rejected, so a `max_tokens` too small to hold both a
|
|
43
|
+
* budget and an answer means thinking is dropped rather than asked for invalidly.
|
|
44
|
+
*/
|
|
45
|
+
const MIN_THINKING_BUDGET = 1024;
|
|
46
|
+
|
|
38
47
|
/**
|
|
39
48
|
* Roughly the shortest system prompt worth a cache breakpoint.
|
|
40
49
|
*
|
|
@@ -165,10 +174,10 @@ export class AnthropicDriver implements AiDriver {
|
|
|
165
174
|
};
|
|
166
175
|
}
|
|
167
176
|
|
|
168
|
-
async countTokens(request: AiRequest): Promise<number> {
|
|
177
|
+
async countTokens(request: AiRequest): Promise<number | null> {
|
|
169
178
|
const { client } = await this._load();
|
|
170
179
|
const api = this._api(client);
|
|
171
|
-
if (!api.countTokens) return
|
|
180
|
+
if (!api.countTokens) return null;
|
|
172
181
|
|
|
173
182
|
const params = this._params(request, false);
|
|
174
183
|
// Counting is about the prompt; the response ceiling and sampling knobs are
|
|
@@ -217,16 +226,62 @@ export class AnthropicDriver implements AiDriver {
|
|
|
217
226
|
: { timeout: this.config.timeout };
|
|
218
227
|
}
|
|
219
228
|
|
|
229
|
+
/**
|
|
230
|
+
* The `thinking` block for this model, or `null` to omit the field.
|
|
231
|
+
*
|
|
232
|
+
* Three shapes, because the models take three. The 4.5 generation wants an
|
|
233
|
+
* explicit `budget_tokens`; everything current takes `adaptive` and decides for
|
|
234
|
+
* itself; a non-Anthropic id gets nothing. Sending the wrong one is a 400, which
|
|
235
|
+
* is why this returns `null` rather than guessing.
|
|
236
|
+
*
|
|
237
|
+
* `display` is set on the adaptive form because the API omits thinking text by
|
|
238
|
+
* default there — see {@link AnthropicConfigShape.thinkingDisplay}.
|
|
239
|
+
*
|
|
240
|
+
* @param capabilities - What the model accepts.
|
|
241
|
+
* @param maxTokens - The request's ceiling, which thinking shares with the answer.
|
|
242
|
+
*/
|
|
243
|
+
private _thinking(
|
|
244
|
+
capabilities: ModelCapabilities,
|
|
245
|
+
maxTokens: number,
|
|
246
|
+
): Record<string, unknown> | null {
|
|
247
|
+
if (capabilities.thinking === null) return null;
|
|
248
|
+
|
|
249
|
+
if (capabilities.thinking === "budget") {
|
|
250
|
+
// The API requires at least 1024 thinking tokens, and the budget shares
|
|
251
|
+
// `max_tokens` with the answer. A ceiling too low to hold both is a request
|
|
252
|
+
// that would be rejected for asking, so thinking is dropped instead — the
|
|
253
|
+
// answer is what the caller wanted.
|
|
254
|
+
const budget = Math.floor(maxTokens / 2);
|
|
255
|
+
if (budget < MIN_THINKING_BUDGET) return null;
|
|
256
|
+
return { type: "enabled", budget_tokens: budget };
|
|
257
|
+
}
|
|
258
|
+
|
|
259
|
+
// Defaulted here as well as in `AiConfig`, because a config object built by
|
|
260
|
+
// hand — a test, a driver constructed directly — would otherwise send
|
|
261
|
+
// `display: undefined`, which drops out of the JSON and silently restores the
|
|
262
|
+
// API's own default of omitting the text.
|
|
263
|
+
return { type: "adaptive", display: this.config.thinkingDisplay ?? "summarized" };
|
|
264
|
+
}
|
|
265
|
+
|
|
220
266
|
private _params(request: AiRequest, streaming: boolean): Record<string, unknown> {
|
|
221
267
|
const model = request.model ?? this.config.model;
|
|
222
268
|
const maxTokens =
|
|
223
269
|
request.maxTokens ?? (streaming ? this.config.streamMaxTokens : this.config.maxTokens);
|
|
224
270
|
|
|
225
|
-
|
|
271
|
+
const capabilities = modelCapabilities(model);
|
|
272
|
+
|
|
273
|
+
if (request.temperature !== undefined && !capabilities.sampling && !this._warnedSampling) {
|
|
226
274
|
this._warnedSampling = true;
|
|
227
275
|
console.warn(
|
|
228
276
|
`[Zerotal/ai] temperature was supplied but ${model} rejects temperature/top_p/top_k with ` +
|
|
229
|
-
`a 400. Dropping it
|
|
277
|
+
`a 400. Dropping it.` +
|
|
278
|
+
// Only suggest effort where effort exists. This advice used to be
|
|
279
|
+
// unconditional, and on the 4.5 models it named the one parameter that
|
|
280
|
+
// is guaranteed to 400 there — on models that accept the temperature it
|
|
281
|
+
// had just dropped.
|
|
282
|
+
(capabilities.effort
|
|
283
|
+
? ` Use effort ('low' … 'max') to trade thoroughness for cost.`
|
|
284
|
+
: ""),
|
|
230
285
|
);
|
|
231
286
|
}
|
|
232
287
|
|
|
@@ -236,10 +291,26 @@ export class AnthropicDriver implements AiDriver {
|
|
|
236
291
|
// by default on Claude Opus 5, so a budget sized for the prose truncates.
|
|
237
292
|
max_tokens: maxTokens,
|
|
238
293
|
messages: toAnthropicMessages(normalizeMessages(request)),
|
|
239
|
-
thinking: { type: "adaptive" },
|
|
240
|
-
output_config: { effort: request.effort ?? this.config.effort },
|
|
241
294
|
};
|
|
242
295
|
|
|
296
|
+
// Sent where the model takes it. It never was, on any model: this driver
|
|
297
|
+
// warned about dropping `temperature` and then had no branch that set it, so
|
|
298
|
+
// `AiRequest.temperature` and the configured default were both inert. The old
|
|
299
|
+
// sampling predicate hid it by warning for almost every model, which made the
|
|
300
|
+
// silence look deliberate on the few it did not warn for.
|
|
301
|
+
if (capabilities.sampling) {
|
|
302
|
+
const temperature = request.temperature ?? this.config.temperature;
|
|
303
|
+
if (temperature !== undefined) params["temperature"] = temperature;
|
|
304
|
+
}
|
|
305
|
+
|
|
306
|
+
// Both of these are 400s on a model that does not take them, so they are added
|
|
307
|
+
// only where they are accepted rather than sent everywhere and hoped for.
|
|
308
|
+
if (capabilities.effort) {
|
|
309
|
+
params["output_config"] = { effort: request.effort ?? this.config.effort };
|
|
310
|
+
}
|
|
311
|
+
const thinking = this._thinking(capabilities, maxTokens);
|
|
312
|
+
if (thinking) params["thinking"] = thinking;
|
|
313
|
+
|
|
243
314
|
const system = this._system(request);
|
|
244
315
|
if (system) params["system"] = system;
|
|
245
316
|
|
|
@@ -128,8 +128,10 @@ export class OllamaDriver implements AiDriver {
|
|
|
128
128
|
}
|
|
129
129
|
|
|
130
130
|
/** Ollama reports `prompt_eval_count` only after generating. 0 means unknown. */
|
|
131
|
-
async countTokens(_request: AiRequest): Promise<number> {
|
|
132
|
-
|
|
131
|
+
async countTokens(_request: AiRequest): Promise<number | null> {
|
|
132
|
+
// Ollama exposes no token-counting endpoint. `null`, not 0 — the caller can
|
|
133
|
+
// tell "cannot count" from "counted nothing".
|
|
134
|
+
return null;
|
|
133
135
|
}
|
|
134
136
|
|
|
135
137
|
async verify(): Promise<DriverStatus> {
|
|
@@ -153,8 +153,10 @@ export class OpenAiDriver implements AiDriver {
|
|
|
153
153
|
* tokenizer would be a guess dressed as a number. 0 means "unknown"; the spend
|
|
154
154
|
* guard falls back to a labelled character approximation.
|
|
155
155
|
*/
|
|
156
|
-
async countTokens(_request: AiRequest): Promise<number> {
|
|
157
|
-
|
|
156
|
+
async countTokens(_request: AiRequest): Promise<number | null> {
|
|
157
|
+
// OpenAI exposes no token-counting endpoint. `null`, not 0 — the caller can
|
|
158
|
+
// tell "cannot count" from "counted nothing".
|
|
159
|
+
return null;
|
|
158
160
|
}
|
|
159
161
|
|
|
160
162
|
async verify(): Promise<DriverStatus> {
|
package/src/errors.ts
CHANGED
|
@@ -94,6 +94,28 @@ export class AiDriverUnavailableError extends AiError {
|
|
|
94
94
|
false,
|
|
95
95
|
);
|
|
96
96
|
}
|
|
97
|
+
|
|
98
|
+
/**
|
|
99
|
+
* The variant for an app that has configured no AI at all.
|
|
100
|
+
*
|
|
101
|
+
* Deliberately this class rather than a new one. "No driver is installed" and "no
|
|
102
|
+
* driver is configured" are the same fact to a caller — AI cannot answer here, and
|
|
103
|
+
* will not start being able to mid-process — and every consumer's handling is
|
|
104
|
+
* already written against the four permanent classes. A fifth would fall outside
|
|
105
|
+
* it silently, which is the failure mode a shared error taxonomy exists to prevent.
|
|
106
|
+
*
|
|
107
|
+
* @param requested - The driver name that was asked for.
|
|
108
|
+
*/
|
|
109
|
+
static notConfigured(requested: string): AiDriverUnavailableError {
|
|
110
|
+
const error = new AiDriverUnavailableError(requested, "a provider SDK");
|
|
111
|
+
return Object.assign(error, {
|
|
112
|
+
message:
|
|
113
|
+
`[Zerotal/ai] No AI driver is configured, so there is nothing to generate with. ` +
|
|
114
|
+
`This is a supported state — an app can ship with AI off. Add a driver under ` +
|
|
115
|
+
`'drivers' in config/ai.ts to turn it on.`,
|
|
116
|
+
context: { requested, configured: [] as string[] },
|
|
117
|
+
});
|
|
118
|
+
}
|
|
97
119
|
}
|
|
98
120
|
|
|
99
121
|
/**
|
package/src/index.ts
CHANGED
|
@@ -58,6 +58,8 @@ export { AiGenerationJob } from "./AiGenerationJob.ts";
|
|
|
58
58
|
|
|
59
59
|
// Cost estimation — extend the table for a model this package does not price.
|
|
60
60
|
export { estimateCost, modelPrice, registerModelPrice, modelRejectsSampling } from "./pricing.ts";
|
|
61
|
+
export { modelCapabilities } from "./modelCapabilities.ts";
|
|
62
|
+
export type { ModelCapabilities } from "./modelCapabilities.ts";
|
|
61
63
|
export type { ModelPrice } from "./pricing.ts";
|
|
62
64
|
|
|
63
65
|
// Spend ledger
|
|
@@ -0,0 +1,105 @@
|
|
|
1
|
+
/**
|
|
2
|
+
* What each Anthropic model will actually accept on a request.
|
|
3
|
+
*
|
|
4
|
+
* The driver used to send `output_config.effort` and `thinking: { type: "adaptive" }`
|
|
5
|
+
* on every call, and decide sampling with one regex. That was right for the current
|
|
6
|
+
* generation and wrong for everything older, in a way that had the package
|
|
7
|
+
* contradicting itself: `claude-haiku-4-5` is in the pricing table — so the package
|
|
8
|
+
* presents it as supported — and the driver could not successfully call it, because
|
|
9
|
+
* `effort` is a 400 on that model and `adaptive` thinking is not a shape it takes.
|
|
10
|
+
*
|
|
11
|
+
* The two mistakes compounded. On Haiku 4.5 the driver dropped a `temperature` the
|
|
12
|
+
* model accepts perfectly well, and told the user to reach for `effort` instead —
|
|
13
|
+
* the one parameter guaranteed to fail there.
|
|
14
|
+
*
|
|
15
|
+
* ## Why the exceptions are listed and the default is current
|
|
16
|
+
*
|
|
17
|
+
* The obvious fix is an allowlist of models that behave the modern way, and it is the
|
|
18
|
+
* wrong one: it needs an edit every time a model ships, and until that edit lands a
|
|
19
|
+
* brand-new model is treated as legacy. Anthropic's direction of travel is *towards*
|
|
20
|
+
* this shape, so the models that differ are a closed set that ages out rather than an
|
|
21
|
+
* open one that grows.
|
|
22
|
+
*
|
|
23
|
+
* So: the older models are named, and anything unrecognised is assumed to behave like
|
|
24
|
+
* the current generation. A new model works on the day it ships.
|
|
25
|
+
*
|
|
26
|
+
* @module
|
|
27
|
+
*/
|
|
28
|
+
|
|
29
|
+
/** What a model accepts on a request. */
|
|
30
|
+
export interface ModelCapabilities {
|
|
31
|
+
/**
|
|
32
|
+
* Accepts `temperature` / `top_p` / `top_k`.
|
|
33
|
+
*
|
|
34
|
+
* These became a 400 on the current generation; they are fine on 4.6 and below.
|
|
35
|
+
*/
|
|
36
|
+
sampling: boolean;
|
|
37
|
+
/**
|
|
38
|
+
* Accepts `output_config.effort`.
|
|
39
|
+
*
|
|
40
|
+
* A 400 on the 4.5 generation, which has no equivalent knob.
|
|
41
|
+
*/
|
|
42
|
+
effort: boolean;
|
|
43
|
+
/**
|
|
44
|
+
* How this model takes extended thinking.
|
|
45
|
+
*
|
|
46
|
+
* - `"adaptive"` — `{ type: "adaptive" }`, the model decides how much to spend.
|
|
47
|
+
* - `"budget"` — `{ type: "enabled", budget_tokens: N }`, an explicit allowance.
|
|
48
|
+
* - `null` — the model has no thinking mode and the field must be omitted.
|
|
49
|
+
*/
|
|
50
|
+
thinking: "adaptive" | "budget" | null;
|
|
51
|
+
}
|
|
52
|
+
|
|
53
|
+
/**
|
|
54
|
+
* The current generation's shape, and the default for anything unrecognised.
|
|
55
|
+
*
|
|
56
|
+
* A model this file has never heard of is far more likely to be newer than these
|
|
57
|
+
* than older, because the older ones are already named below.
|
|
58
|
+
*/
|
|
59
|
+
const CURRENT: ModelCapabilities = { sampling: false, effort: true, thinking: "adaptive" };
|
|
60
|
+
|
|
61
|
+
/**
|
|
62
|
+
* Models that differ from {@link CURRENT}, by exact id.
|
|
63
|
+
*
|
|
64
|
+
* A closed set: these age out of use, and nothing new joins them. That is the whole
|
|
65
|
+
* reason the table is written as exceptions rather than as an allowlist.
|
|
66
|
+
*/
|
|
67
|
+
const EXCEPTIONS: Record<string, ModelCapabilities> = {
|
|
68
|
+
// 4.6 was the last generation to accept sampling parameters. Effort and adaptive
|
|
69
|
+
// thinking both work.
|
|
70
|
+
"claude-opus-4-6": { sampling: true, effort: true, thinking: "adaptive" },
|
|
71
|
+
"claude-sonnet-4-6": { sampling: true, effort: true, thinking: "adaptive" },
|
|
72
|
+
|
|
73
|
+
// The 4.5 generation predates `output_config` entirely and wants an explicit
|
|
74
|
+
// thinking budget. Sampling is fine.
|
|
75
|
+
"claude-opus-4-5": { sampling: true, effort: false, thinking: "budget" },
|
|
76
|
+
"claude-sonnet-4-5": { sampling: true, effort: false, thinking: "budget" },
|
|
77
|
+
"claude-haiku-4-5": { sampling: true, effort: false, thinking: "budget" },
|
|
78
|
+
};
|
|
79
|
+
|
|
80
|
+
/**
|
|
81
|
+
* What this model accepts.
|
|
82
|
+
*
|
|
83
|
+
* Non-Anthropic model ids are reported as accepting sampling and nothing
|
|
84
|
+
* Anthropic-specific, because the Anthropic request shape is the only thing this
|
|
85
|
+
* describes — an Ollama or OpenAI model reaches a different driver that builds its
|
|
86
|
+
* own request.
|
|
87
|
+
*
|
|
88
|
+
* @param model - Exact model id, e.g. `claude-sonnet-5`.
|
|
89
|
+
*/
|
|
90
|
+
export function modelCapabilities(model: string): ModelCapabilities {
|
|
91
|
+
if (!model.startsWith("claude-")) {
|
|
92
|
+
return { sampling: true, effort: false, thinking: null };
|
|
93
|
+
}
|
|
94
|
+
return EXCEPTIONS[model] ?? CURRENT;
|
|
95
|
+
}
|
|
96
|
+
|
|
97
|
+
/**
|
|
98
|
+
* Whether a model rejects `temperature` / `top_p` / `top_k` with a 400.
|
|
99
|
+
*
|
|
100
|
+
* @param model - Exact model id.
|
|
101
|
+
* @returns `true` when sampling parameters must be dropped.
|
|
102
|
+
*/
|
|
103
|
+
export function modelRejectsSampling(model: string): boolean {
|
|
104
|
+
return !modelCapabilities(model).sampling;
|
|
105
|
+
}
|
package/src/pricing.ts
CHANGED
|
@@ -30,7 +30,10 @@ const PRICES: Record<string, ModelPrice> = {
|
|
|
30
30
|
"claude-opus-4-8": { input: 5, output: 25 },
|
|
31
31
|
"claude-opus-4-7": { input: 5, output: 25 },
|
|
32
32
|
"claude-opus-4-6": { input: 5, output: 25 },
|
|
33
|
-
|
|
33
|
+
// Sonnet 5 carried Sonnet 4.6's row — 3/15 — until 1.11.2. Every other family
|
|
34
|
+
// got its own number and this one was copied, so the ceiling refused requests
|
|
35
|
+
// that were 50% inside their budget and blamed the budget.
|
|
36
|
+
"claude-sonnet-5": { input: 2, output: 10 },
|
|
34
37
|
"claude-sonnet-4-6": { input: 3, output: 15 },
|
|
35
38
|
"claude-haiku-4-5": { input: 1, output: 5 },
|
|
36
39
|
};
|
|
@@ -75,16 +78,9 @@ export function estimateCost(model: string, usage: AiUsage): number {
|
|
|
75
78
|
);
|
|
76
79
|
}
|
|
77
80
|
|
|
78
|
-
|
|
79
|
-
|
|
80
|
-
|
|
81
|
-
|
|
82
|
-
|
|
83
|
-
|
|
84
|
-
* every single request against a model released after this line was written.
|
|
85
|
-
*/
|
|
86
|
-
export function modelRejectsSampling(model: string): boolean {
|
|
87
|
-
// Opus 4.6 and Sonnet 4.6 were the last Claude models to accept sampling
|
|
88
|
-
// parameters; everything before them predates the models this package targets.
|
|
89
|
-
return !/^claude-(opus|sonnet)-4-6\b/.test(model) && model.startsWith("claude-");
|
|
90
|
-
}
|
|
81
|
+
// Re-exported from its new home so existing imports keep working. The predicate is
|
|
82
|
+
// derived from the capability table now rather than from a regex over model ids —
|
|
83
|
+
// see `modelCapabilities.ts` for why the exceptions are listed and the default is
|
|
84
|
+
// the current generation.
|
|
85
|
+
export { modelRejectsSampling, modelCapabilities } from "./modelCapabilities.ts";
|
|
86
|
+
export type { ModelCapabilities } from "./modelCapabilities.ts";
|
package/src/schema.ts
CHANGED
|
@@ -44,7 +44,11 @@ function defOf(value: FieldRule | FieldRuleDefinition): FieldRuleDefinition {
|
|
|
44
44
|
return "_def" in value ? value._def : value;
|
|
45
45
|
}
|
|
46
46
|
|
|
47
|
-
/** Normalise either input shape to raw definitions.
|
|
47
|
+
/** Normalise either input shape to raw definitions.
|
|
48
|
+
*
|
|
49
|
+
* @internal Normalisation between the two accepted input shapes. No caller anywhere, in this
|
|
50
|
+
* package or out of it — an app declares a schema; it never converts one.
|
|
51
|
+
*/
|
|
48
52
|
export function toSchema(input: SchemaInput): Schema {
|
|
49
53
|
const out: Schema = {};
|
|
50
54
|
for (const [key, value] of Object.entries(input)) out[key] = defOf(value);
|
|
@@ -212,6 +216,8 @@ function withStringKeywords(base: JsonSchema, def: FieldRuleDefinition): JsonSch
|
|
|
212
216
|
* @example
|
|
213
217
|
* strippedConstraints({ title: rule.string().min(3).max(80) });
|
|
214
218
|
* // → ["title: min", "title: max"]
|
|
219
|
+
*
|
|
220
|
+
* @internal A diagnostic for the translation layer, called by nothing but its own test.
|
|
215
221
|
*/
|
|
216
222
|
export function strippedConstraints(input: SchemaInput): string[] {
|
|
217
223
|
const out: string[] = [];
|
package/src/spend.ts
CHANGED
|
@@ -48,7 +48,13 @@ export function recordSpend(model: string, usage: AiUsage): number {
|
|
|
48
48
|
return cost;
|
|
49
49
|
}
|
|
50
50
|
|
|
51
|
-
/**
|
|
51
|
+
/**
|
|
52
|
+
* Reset the ledger.
|
|
53
|
+
*
|
|
54
|
+
* @internal Backs `ai:spend --reset` and this package's own tests. An app's test
|
|
55
|
+
* wanting spend isolation should reach for `AiFake`, which is the seam built for
|
|
56
|
+
* it; this one reaches past the manager into module state.
|
|
57
|
+
*/
|
|
52
58
|
export function resetSpend(): void {
|
|
53
59
|
_day = today();
|
|
54
60
|
_spentUsd = 0;
|
|
@@ -90,11 +96,27 @@ export function assertWithinLimits(
|
|
|
90
96
|
});
|
|
91
97
|
|
|
92
98
|
if (worstCase > limits.perRequestUsd) {
|
|
99
|
+
// The price is named because the table is a thing that can be wrong, and when
|
|
100
|
+
// it is, this refusal is the only symptom. A row carrying its predecessor's
|
|
101
|
+
// number once made the ceiling reject requests comfortably inside a budget, and
|
|
102
|
+
// the message said "spend limit" — which sends someone to their config rather
|
|
103
|
+
// than to the row that is 50% high. Quoting the rate makes a bad table legible
|
|
104
|
+
// from the error, and `registerModelPrice` is how it gets corrected without
|
|
105
|
+
// waiting for a release.
|
|
106
|
+
const price = modelPrice(model)!;
|
|
93
107
|
throw new AiSpendLimitError(
|
|
94
108
|
`This request could cost up to $${worstCase.toFixed(4)}, over the per-request ceiling of ` +
|
|
95
109
|
`$${limits.perRequestUsd.toFixed(4)}. Shorten the prompt, lower maxTokens, or raise ` +
|
|
96
|
-
`limits.perRequestUsd in config/ai.ts
|
|
97
|
-
|
|
110
|
+
`limits.perRequestUsd in config/ai.ts. ` +
|
|
111
|
+
`Priced at $${price.input}/$${price.output} per million tokens for ${model} — if that is ` +
|
|
112
|
+
`not what you are billed, correct it with registerModelPrice() rather than raising the ceiling.`,
|
|
113
|
+
{
|
|
114
|
+
model,
|
|
115
|
+
estimatedInputTokens,
|
|
116
|
+
maxOutputTokens,
|
|
117
|
+
worstCaseUsd: worstCase,
|
|
118
|
+
priceUsdPerMillion: price,
|
|
119
|
+
},
|
|
98
120
|
);
|
|
99
121
|
}
|
|
100
122
|
}
|
package/src/stats.ts
CHANGED
|
@@ -96,7 +96,10 @@ export function refusalRate(): number {
|
|
|
96
96
|
return _deliveries.filter((d) => d.refused).length / _deliveries.length;
|
|
97
97
|
}
|
|
98
98
|
|
|
99
|
-
/** Reset the buffer. Tests.
|
|
99
|
+
/** Reset the buffer. Tests.
|
|
100
|
+
*
|
|
101
|
+
* @internal A test helper, as `resetSpend` is.
|
|
102
|
+
*/
|
|
100
103
|
export function resetStats(): void {
|
|
101
104
|
_deliveries.length = 0;
|
|
102
105
|
}
|
package/src/types.ts
CHANGED
|
@@ -232,6 +232,20 @@ export interface AnthropicConfigShape {
|
|
|
232
232
|
/** Cap for streaming calls, where HTTP timeouts are not a concern. */
|
|
233
233
|
streamMaxTokens: number;
|
|
234
234
|
effort: AiEffort;
|
|
235
|
+
/**
|
|
236
|
+
* Whether a streamed `thinking` chunk carries text.
|
|
237
|
+
*
|
|
238
|
+
* The API defaults this to `"omitted"` on the current generation, which is why
|
|
239
|
+
* the documented `thinking` stream chunk used to fire forever with `text: ""` —
|
|
240
|
+
* thinking happened, and was billed, and nothing was emitted. It defaulted to
|
|
241
|
+
* `"summarized"` on the 4.6 models, so a "thinking…" view built against those
|
|
242
|
+
* worked and then silently stopped working as people moved forward.
|
|
243
|
+
*
|
|
244
|
+
* Defaults to `"summarized"` here, because a documented channel that is always
|
|
245
|
+
* empty is worse than one that costs a little to fill. Set `"omitted"` to get
|
|
246
|
+
* the API's own default back — the thinking still happens either way.
|
|
247
|
+
*/
|
|
248
|
+
thinkingDisplay: "summarized" | "omitted";
|
|
235
249
|
/** Route a safety refusal to Anthropic's recommended fallback model. */
|
|
236
250
|
fallbacks: boolean;
|
|
237
251
|
/** Mark the system prompt cacheable. The cheapest win available. */
|