@namzu/sdk 42.0.2 → 43.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +34 -0
- package/dist/provider/conformance.d.ts.map +1 -1
- package/dist/provider/conformance.js +13 -1
- package/dist/provider/conformance.js.map +1 -1
- package/dist/runtime/query/callback-inference.d.ts.map +1 -1
- package/dist/runtime/query/callback-inference.js +10 -4
- package/dist/runtime/query/callback-inference.js.map +1 -1
- package/dist/types/provider/model.d.ts +26 -2
- package/dist/types/provider/model.d.ts.map +1 -1
- package/dist/types/run/prepare-step.d.ts +12 -1
- package/dist/types/run/prepare-step.d.ts.map +1 -1
- package/package.json +1 -1
- package/src/provider/conformance.ts +13 -1
- package/src/runtime/query/callback-inference.ts +10 -7
- package/src/types/provider/model.ts +26 -2
- package/src/types/run/prepare-step.ts +12 -1
package/CHANGELOG.md
CHANGED
|
@@ -1,5 +1,39 @@
|
|
|
1
1
|
# Changelog
|
|
2
2
|
|
|
3
|
+
## 43.0.0
|
|
4
|
+
|
|
5
|
+
### Major Changes
|
|
6
|
+
|
|
7
|
+
- e83dfe5: `ModelInfo.inputPrice` and `ModelInfo.outputPrice` are now optional. Six drivers wrote `0` wherever the vendor listing carries no rate; they omit the field instead.
|
|
8
|
+
|
|
9
|
+
A price of zero is not "I do not know" — it is a billing fact. It says the model is free, and it reaches every consumer as a quote: a menu prints `(free)`, a total reports the model as costing nothing, and an operator believes both. The drivers had no way to say anything else, because the type required a number.
|
|
10
|
+
|
|
11
|
+
This is the defect `9d6c482c` removed from `contextWindow`, in the field where a wrong value is money rather than a compaction pass. The distinction was already the house rule one level down: `resolveModelPricing` returns `undefined` for a rate nobody has and `{ inputCostPer1M: 0, outputCostPer1M: 0 }` for a driver that genuinely bills nothing, and its own docblock says a caller that flattens the two reproduces the defect the module exists to remove. `ModelInfo` carried the flattened version to every consumer that never reached that module.
|
|
12
|
+
|
|
13
|
+
**What breaks.** Code reading `model.inputPrice` or `model.outputPrice` as a `number` must handle `undefined`. That is the point — the value was already absent in fact, and the type was asserting otherwise. Treat `undefined` as unknown rather than as free: a caller that renders a price should say "unknown" rather than `$0.00`, and only `0` means the model is free.
|
|
14
|
+
|
|
15
|
+
**What does not change.** A driver that knows a rate of zero still writes `0`. `ollama` and `lmstudio` bill nothing by construction, and the Zen catalogue's free tier is priced from a source that names those models free. Values that were genuinely known are untouched.
|
|
16
|
+
|
|
17
|
+
### Minor Changes
|
|
18
|
+
|
|
19
|
+
- 9661d16: Stop an optional inference from ending the run that made it.
|
|
20
|
+
|
|
21
|
+
`PreparationTextRequest.timeoutMs` is deprecated and no longer bounds the request. An
|
|
22
|
+
auxiliary provider request that ends without its final usage receipt leaves the run's shared
|
|
23
|
+
token ledger unresolved, and an account with unresolved spend admits nothing further — so a
|
|
24
|
+
deadline that fired in normal use did not bound the call, it stopped the run. Against a
|
|
25
|
+
reasoning model whose auxiliary answer took 17 s, every turn after the first ended with
|
|
26
|
+
"usage for a model request could not be confirmed" before the model was ever asked, so no
|
|
27
|
+
tool call in the run could ever execute.
|
|
28
|
+
|
|
29
|
+
What a host observes: a run that used to stop after its first turn now continues. A
|
|
30
|
+
preparation or review inference is bounded by the provider's own request timeout and by the
|
|
31
|
+
run's cancellation — the same two bounds every other model request in the run has — instead
|
|
32
|
+
of by a 10 s deadline of its own; set `runConfig.timeoutMs` to bound the whole turn. Stop
|
|
33
|
+
still cancels an in-flight auxiliary call, and a cancelled call still leaves its unresolved
|
|
34
|
+
receipt visible in `/cost`. Passing `timeoutMs` changes nothing and is still validated; the
|
|
35
|
+
field will be removed in a later major.
|
|
36
|
+
|
|
3
37
|
## 42.0.2
|
|
4
38
|
|
|
5
39
|
### Patch Changes
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"conformance.d.ts","sourceRoot":"","sources":["../../src/provider/conformance.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACX,mBAAmB,EACnB,iBAAiB,EACjB,aAAa,EACb,MAAM,6BAA6B,CAAA;AACpC,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,gCAAgC,CAAA;AACjE,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,YAAY,CAAA;AAErD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAuCG;AAEH,sEAAsE;AACtE,eAAO,MAAM,gCAAgC,IAAI,CAAA;AAEjD,MAAM,WAAW,gCAAgC;IAChD,QAAQ,CAAC,QAAQ,EAAE,mBAAmB,CAAA;IACtC,QAAQ,CAAC,EAAE,EAAE,aAAa,CAAA;IAC1B,QAAQ,CAAC,MAAM,EAAE,iBAAiB,CAAA;IAClC;;;;;OAKG;IACH,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAA;IAC7B;;;;OAIG;IACH,QAAQ,CAAC,YAAY,EAAE,MAAM,WAAW,GAAG,OAAO,CAAC,WAAW,CAAC,CAAA;IAC/D;;;;;;;;OAQG;IACH,QAAQ,CAAC,aAAa,EAAE,OAAO,CAAC,mBAAmB,CAAC,GAAG,SAAS,CAAA;IAChE;;;;;;;;;OASG;IACH,QAAQ,CAAC,WAAW,EAAE;QAAE,IAAI,EAAE,QAAQ,CAAA;KAAE,GAAG;QAAE,IAAI,EAAE,aAAa,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAA;IAClF,sEAAsE;IACtE,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,CAAA;CACvB;AAED,wBAAgB,+BAA+B,CAAC,OAAO,EAAE,gCAAgC,GAAG,IAAI,
|
|
1
|
+
{"version":3,"file":"conformance.d.ts","sourceRoot":"","sources":["../../src/provider/conformance.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EACX,mBAAmB,EACnB,iBAAiB,EACjB,aAAa,EACb,MAAM,6BAA6B,CAAA;AACpC,OAAO,KAAK,EAAE,WAAW,EAAE,MAAM,gCAAgC,CAAA;AACjE,OAAO,KAAK,EAAE,mBAAmB,EAAE,MAAM,YAAY,CAAA;AAErD;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAuCG;AAEH,sEAAsE;AACtE,eAAO,MAAM,gCAAgC,IAAI,CAAA;AAEjD,MAAM,WAAW,gCAAgC;IAChD,QAAQ,CAAC,QAAQ,EAAE,mBAAmB,CAAA;IACtC,QAAQ,CAAC,EAAE,EAAE,aAAa,CAAA;IAC1B,QAAQ,CAAC,MAAM,EAAE,iBAAiB,CAAA;IAClC;;;;;OAKG;IACH,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAA;IAC7B;;;;OAIG;IACH,QAAQ,CAAC,YAAY,EAAE,MAAM,WAAW,GAAG,OAAO,CAAC,WAAW,CAAC,CAAA;IAC/D;;;;;;;;OAQG;IACH,QAAQ,CAAC,aAAa,EAAE,OAAO,CAAC,mBAAmB,CAAC,GAAG,SAAS,CAAA;IAChE;;;;;;;;;OASG;IACH,QAAQ,CAAC,WAAW,EAAE;QAAE,IAAI,EAAE,QAAQ,CAAA;KAAE,GAAG;QAAE,IAAI,EAAE,aAAa,CAAC;QAAC,MAAM,EAAE,MAAM,CAAA;KAAE,CAAA;IAClF,sEAAsE;IACtE,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,CAAA;CACvB;AAED,wBAAgB,+BAA+B,CAAC,OAAO,EAAE,gCAAgC,GAAG,IAAI,CA0G/F"}
|
|
@@ -89,6 +89,13 @@ export function defineProviderDriverConformance(options) {
|
|
|
89
89
|
// window, and it reads to every consumer as a measurement of a model
|
|
90
90
|
// that can hold nothing.
|
|
91
91
|
//
|
|
92
|
+
// The same reasoning makes `inputPrice`/`outputPrice` optional, and a
|
|
93
|
+
// price is the more expensive of the two to get wrong: a zero window
|
|
94
|
+
// is a nonsensical measurement, where a zero PRICE is a plain claim
|
|
95
|
+
// that a model is free — one a menu will print, and one a `(free)`
|
|
96
|
+
// marker reads as a fact. Six drivers made that claim on every model
|
|
97
|
+
// they could not price.
|
|
98
|
+
//
|
|
92
99
|
// The obvious place to enforce that is here, reading `listModels`.
|
|
93
100
|
// It was here, and it was worse than nothing twice over. For every
|
|
94
101
|
// in-tree driver `listModels` needs a live service, so the rule
|
|
@@ -100,7 +107,12 @@ export function defineProviderDriverConformance(options) {
|
|
|
100
107
|
//
|
|
101
108
|
// The reintroduction of a literal zero is caught by a source scan in
|
|
102
109
|
// `provider/__tests__/conformance-fails-a-wrong-driver.test.ts`,
|
|
103
|
-
// which depends on reaching nothing.
|
|
110
|
+
// which depends on reaching nothing. It holds two further cases for
|
|
111
|
+
// prices, and the sharper of them is not the literal: a price
|
|
112
|
+
// DEFAULTED to zero cannot be written without a `??` or a `||`, and
|
|
113
|
+
// that is catchable without deciding whether any particular `0` was
|
|
114
|
+
// meant — which a text scan never can, since ollama's honest zero and
|
|
115
|
+
// a placeholder zero are the same characters.
|
|
104
116
|
it('declares the retry defaults its conformance run says it does', async () => {
|
|
105
117
|
// The option above forces the decision; this asserts the answer is
|
|
106
118
|
// TRUE of the driver. Without it a package could declare one thing
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"conformance.js","sourceRoot":"","sources":["../../src/provider/conformance.ts"],"names":[],"mappings":"AAQA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAuCG;AAEH,sEAAsE;AACtE,MAAM,CAAC,MAAM,gCAAgC,GAAG,CAAC,CAAA;AA4CjD,MAAM,UAAU,+BAA+B,CAAC,OAAyC;IACxF,MAAM,EAAE,QAAQ,EAAE,EAAE,EAAE,MAAM,EAAE,YAAY,EAAE,YAAY,EAAE,GAAG,OAAO,CAAA;IACpE,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,IAAI,iBAAiB,CAAA;IAEhD,QAAQ,CAAC,GAAG,KAAK,uBAAuB,gCAAgC,EAAE,EAAE,GAAG,EAAE;QAChF,EAAE,CAAC,6BAA6B,EAAE,KAAK,IAAI,EAAE;YAC5C,4DAA4D;YAC5D,iEAAiE;YACjE,qDAAqD;YACrD,MAAM,QAAQ,GAAG,MAAM,YAAY,EAAE,CAAA;YAErC,MAAM,CAAC,OAAO,QAAQ,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAA;YACzC,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAA;YACzC,MAAM,CAAC,OAAO,QAAQ,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAA;YAC3C,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAA;QAC5C,CAAC,CAAC,CAAA;QAEF,EAAE,CAAC,sDAAsD,EAAE,KAAK,IAAI,EAAE;YACrE,gEAAgE;YAChE,iEAAiE;YACjE,iEAAiE;YACjE,2BAA2B;YAC3B,MAAM,QAAQ,GAAG,MAAM,YAAY,EAAE,CAAA;YAErC,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,YAAY,CAAC,CAAA;QACvC,CAAC,CAAC,CAAA;QAEF,EAAE,CAAC,kCAAkC,EAAE,KAAK,IAAI,EAAE;YACjD,gEAAgE;YAChE,iEAAiE;YACjE,2CAA2C;YAC3C,MAAM,QAAQ,GAAG,MAAM,YAAY,EAAE,CAAA;YAErC,MAAM,CAAC,OAAO,QAAQ,CAAC,UAAU,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAA;QACpD,CAAC,CAAC,CAAA;QAEF,EAAE,CAAC,8CAA8C,EAAE,KAAK,IAAI,EAAE;YAC7D,4DAA4D;YAC5D,8DAA8D;YAC9D,iEAAiE;YACjE,iEAAiE;YACjE,mEAAmE;YACnE,MAAM,QAAQ,GAAG,MAAM,YAAY,EAAE,CAAA;YACrC,IAAI,QAAQ,CAAC,YAAY,KAAK,SAAS;gBAAE,OAAM;YAE/C,MAAM,CAAC,OAAO,QAAQ,CAAC,YAAY,CAAC,aAAa,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAA;YAClE,MAAM,CAAC,OAAO,QAAQ,CAAC,YAAY,CAAC,iBAAiB,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAA;YACtE,MAAM,CAAC,OAAO,QAAQ,CAAC,YAAY,CAAC,uBAAuB,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAA;QAC7E,CAAC,CAAC,CAAA;QAEF,oDAAoD;QACpD,EAAE;QACF,mEAAmE;QACnE,+DAA+D;QAC/D,qEAAqE;QACrE,yBAAyB;QACzB,EAAE;QACF,mEAAmE;QACnE,mEAAmE;QACnE,gEAAgE;QAChE,oEAAoE;QACpE,qEAAqE;QACrE,oEAAoE;QACpE,oEAAoE;QACpE,iBAAiB;QACjB,EAAE;QACF,qEAAqE;QACrE,iEAAiE;QACjE,
|
|
1
|
+
{"version":3,"file":"conformance.js","sourceRoot":"","sources":["../../src/provider/conformance.ts"],"names":[],"mappings":"AAQA;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;GAuCG;AAEH,sEAAsE;AACtE,MAAM,CAAC,MAAM,gCAAgC,GAAG,CAAC,CAAA;AA4CjD,MAAM,UAAU,+BAA+B,CAAC,OAAyC;IACxF,MAAM,EAAE,QAAQ,EAAE,EAAE,EAAE,MAAM,EAAE,YAAY,EAAE,YAAY,EAAE,GAAG,OAAO,CAAA;IACpE,MAAM,KAAK,GAAG,OAAO,CAAC,KAAK,IAAI,iBAAiB,CAAA;IAEhD,QAAQ,CAAC,GAAG,KAAK,uBAAuB,gCAAgC,EAAE,EAAE,GAAG,EAAE;QAChF,EAAE,CAAC,6BAA6B,EAAE,KAAK,IAAI,EAAE;YAC5C,4DAA4D;YAC5D,iEAAiE;YACjE,qDAAqD;YACrD,MAAM,QAAQ,GAAG,MAAM,YAAY,EAAE,CAAA;YAErC,MAAM,CAAC,OAAO,QAAQ,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAA;YACzC,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAA;YACzC,MAAM,CAAC,OAAO,QAAQ,CAAC,IAAI,CAAC,CAAC,IAAI,CAAC,QAAQ,CAAC,CAAA;YAC3C,MAAM,CAAC,QAAQ,CAAC,IAAI,CAAC,MAAM,GAAG,CAAC,CAAC,CAAC,IAAI,CAAC,IAAI,CAAC,CAAA;QAC5C,CAAC,CAAC,CAAA;QAEF,EAAE,CAAC,sDAAsD,EAAE,KAAK,IAAI,EAAE;YACrE,gEAAgE;YAChE,iEAAiE;YACjE,iEAAiE;YACjE,2BAA2B;YAC3B,MAAM,QAAQ,GAAG,MAAM,YAAY,EAAE,CAAA;YAErC,MAAM,CAAC,QAAQ,CAAC,EAAE,CAAC,CAAC,IAAI,CAAC,YAAY,CAAC,CAAA;QACvC,CAAC,CAAC,CAAA;QAEF,EAAE,CAAC,kCAAkC,EAAE,KAAK,IAAI,EAAE;YACjD,gEAAgE;YAChE,iEAAiE;YACjE,2CAA2C;YAC3C,MAAM,QAAQ,GAAG,MAAM,YAAY,EAAE,CAAA;YAErC,MAAM,CAAC,OAAO,QAAQ,CAAC,UAAU,CAAC,CAAC,IAAI,CAAC,UAAU,CAAC,CAAA;QACpD,CAAC,CAAC,CAAA;QAEF,EAAE,CAAC,8CAA8C,EAAE,KAAK,IAAI,EAAE;YAC7D,4DAA4D;YAC5D,8DAA8D;YAC9D,iEAAiE;YACjE,iEAAiE;YACjE,mEAAmE;YACnE,MAAM,QAAQ,GAAG,MAAM,YAAY,EAAE,CAAA;YACrC,IAAI,QAAQ,CAAC,YAAY,KAAK,SAAS;gBAAE,OAAM;YAE/C,MAAM,CAAC,OAAO,QAAQ,CAAC,YAAY,CAAC,aAAa,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAA;YAClE,MAAM,CAAC,OAAO,QAAQ,CAAC,YAAY,CAAC,iBAAiB,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAA;YACtE,MAAM,CAAC,OAAO,QAAQ,CAAC,YAAY,CAAC,uBAAuB,CAAC,CAAC,IAAI,CAAC,SAAS,CAAC,CAAA;QAC7E,CAAC,CAAC,CAAA;QAEF,oDAAoD;QACpD,EAAE;QACF,mEAAmE;QACnE,+DAA+D;QAC/D,qEAAqE;QACrE,yBAAyB;QACzB,EAAE;QACF,sEAAsE;QACtE,qEAAqE;QACrE,oEAAoE;QACpE,mEAAmE;QACnE,qEAAqE;QACrE,wBAAwB;QACxB,EAAE;QACF,mEAAmE;QACnE,mEAAmE;QACnE,gEAAgE;QAChE,oEAAoE;QACpE,qEAAqE;QACrE,oEAAoE;QACpE,oEAAoE;QACpE,iBAAiB;QACjB,EAAE;QACF,qEAAqE;QACrE,iEAAiE;QACjE,oEAAoE;QACpE,8DAA8D;QAC9D,oEAAoE;QACpE,oEAAoE;QACpE,sEAAsE;QACtE,8CAA8C;QAE9C,EAAE,CAAC,8DAA8D,EAAE,KAAK,IAAI,EAAE;YAC7E,mEAAmE;YACnE,mEAAmE;YACnE,4DAA4D;YAC5D,kDAAkD;YAClD,MAAM,QAAQ,GAAG,MAAM,YAAY,EAAE,CAAA;YAErC,MAAM,CAAC,IAAI,CAAC,SAAS,CAAC,QAAQ,CAAC,aAAa,IAAI,IAAI,CAAC,CAAC,CAAC,IAAI,CAC1D,IAAI,CAAC,SAAS,CAAC,OAAO,CAAC,aAAa,IAAI,IAAI,CAAC,CAC7C,CAAA;QACF,CAAC,CAAC,CAAA;QAEF,EAAE,CAAC,mDAAmD,EAAE,KAAK,IAAI,EAAE;YAClE,gEAAgE;YAChE,kEAAkE;YAClE,4DAA4D;YAC5D,+DAA+D;YAC/D,qCAAqC;YACrC,MAAM,CAAC,GAAG,MAAM,YAAY,EAAE,CAAA;YAC9B,MAAM,CAAC,GAAG,MAAM,YAAY,EAAE,CAAA;YAE9B,MAAM,CAAC,CAAC,KAAK,CAAC,CAAC,CAAC,IAAI,CAAC,KAAK,CAAC,CAAA;QAC5B,CAAC,CAAC,CAAA;IACH,CAAC,CAAC,CAAA;AACH,CAAC"}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"callback-inference.d.ts","sourceRoot":"","sources":["../../../src/runtime/query/callback-inference.ts"],"names":[],"mappings":"AAIA,OAAO,KAAK,EAAE,sBAAsB,EAAE,qBAAqB,EAAE,MAAM,iCAAiC,CAAA;AACpG,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,+BAA+B,CAAA;AAWrE,oFAAoF;AACpF,wBAAgB,uBAAuB,CACtC,GAAG,EAAE,gBAAgB,EACrB,KAAK,EAAE,MAAM,EACb,KAAK,EAAE,aAAa,GAAG,QAAQ;;0BAQF,sBAAsB,GAAG,OAAO,CAAC,qBAAqB,CAAC;
|
|
1
|
+
{"version":3,"file":"callback-inference.d.ts","sourceRoot":"","sources":["../../../src/runtime/query/callback-inference.ts"],"names":[],"mappings":"AAIA,OAAO,KAAK,EAAE,sBAAsB,EAAE,qBAAqB,EAAE,MAAM,iCAAiC,CAAA;AACpG,OAAO,KAAK,EAAE,gBAAgB,EAAE,MAAM,+BAA+B,CAAA;AAWrE,oFAAoF;AACpF,wBAAgB,uBAAuB,CACtC,GAAG,EAAE,gBAAgB,EACrB,KAAK,EAAE,MAAM,EACb,KAAK,EAAE,aAAa,GAAG,QAAQ;;0BAQF,sBAAsB,GAAG,OAAO,CAAC,qBAAqB,CAAC;EAoEpF"}
|
|
@@ -29,12 +29,19 @@ export function createCallbackInference(ctx, model, phase) {
|
|
|
29
29
|
if (used)
|
|
30
30
|
throw new Error(`${phase === 'preparation' ? 'A' : 'An'} ${owner} may make only one inference call.`);
|
|
31
31
|
used = true;
|
|
32
|
-
|
|
33
|
-
|
|
32
|
+
// No deadline of our own on the request — see the note on
|
|
33
|
+
// `PreparationTextRequest.timeoutMs`. The two bounds that remain are
|
|
34
|
+
// the ones every other model request in the run has: the run's own
|
|
35
|
+
// cancellation, and the provider's request timeout. An expired local
|
|
36
|
+
// deadline here would abort a CONTACTED request, and a request that
|
|
37
|
+
// ends without its usage receipt leaves the shared ledger unresolved,
|
|
38
|
+
// which stops the whole run — so a stage that gave up on its own
|
|
39
|
+
// optional inference would take the operator's turn with it. That is
|
|
40
|
+
// not hypothetical: it is what a 10s deadline did to every turn
|
|
41
|
+
// against a reasoning model slower than 10s per auxiliary answer.
|
|
34
42
|
const signal = AbortSignal.any([
|
|
35
43
|
ctx.abortController.signal,
|
|
36
44
|
lifetime.signal,
|
|
37
|
-
deadline.signal,
|
|
38
45
|
...(requestedSignal ? [requestedSignal] : []),
|
|
39
46
|
]);
|
|
40
47
|
const requested = ctx.servingMember?.() ?? { index: 0, providerId: ctx.provider.id };
|
|
@@ -79,7 +86,6 @@ export function createCallbackInference(ctx, model, phase) {
|
|
|
79
86
|
return { text: text.text, usage, servedBy: route() };
|
|
80
87
|
}
|
|
81
88
|
finally {
|
|
82
|
-
clearTimeout(timer);
|
|
83
89
|
if (usage)
|
|
84
90
|
ctx.runMgr.accumulateUsage(usage, route());
|
|
85
91
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"callback-inference.js","sourceRoot":"","sources":["../../../src/runtime/query/callback-inference.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAA;AACvB,OAAO,EAAE,qBAAqB,EAAE,MAAM,+BAA+B,CAAA;AACrE,OAAO,EAAmB,eAAe,EAAE,MAAM,6BAA6B,CAAA;AAC9E,OAAO,EAAE,mBAAmB,EAAE,iBAAiB,EAAE,MAAM,8BAA8B,CAAA;AAIrF,MAAM,aAAa,GAAG,CAAC;KACrB,MAAM,CAAC;IACP,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC;IACrC,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC;IACrC,SAAS,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC;IACzD,SAAS,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC;CAC9D,CAAC;KACD,MAAM,EAAE,CAAA;AAEV,oFAAoF;AACpF,MAAM,UAAU,uBAAuB,CACtC,GAAqB,EACrB,KAAa,EACb,KAA+B;IAE/B,MAAM,KAAK,GAAG,KAAK,KAAK,aAAa,CAAC,CAAC,CAAC,aAAa,CAAC,CAAC,CAAC,QAAQ,CAAA;IAChE,MAAM,KAAK,GAAG,KAAK,KAAK,aAAa,CAAC,CAAC,CAAC,mBAAmB,CAAC,CAAC,CAAC,iBAAiB,CAAA;IAC/E,MAAM,QAAQ,GAAG,IAAI,eAAe,EAAE,CAAA;IACtC,IAAI,IAAI,GAAG,KAAK,CAAA;IAChB,OAAO;QACN,KAAK,EAAE,GAAG,EAAE,CAAC,QAAQ,CAAC,KAAK,CAAC,IAAI,KAAK,CAAC,OAAO,KAAK,aAAa,CAAC,CAAC;QACjE,KAAK,CAAC,YAAY,CAAC,OAA+B;YACjD,QAAQ,CAAC,MAAM,CAAC,cAAc,EAAE,CAAA;YAChC,GAAG,CAAC,eAAe,CAAC,MAAM,CAAC,cAAc,EAAE,CAAA;YAC3C,OAAO,CAAC,MAAM,EAAE,cAAc,EAAE,CAAA;YAChC,MAAM,EAAE,MAAM,EAAE,eAAe,EAAE,GAAG,MAAM,EAAE,GAAG,OAAO,CAAA;YACtD,MAAM,KAAK,GAAG,aAAa,CAAC,KAAK,CAAC,MAAM,CAAC,CAAA;YACzC,IAAI,KAAK,CAAC,MAAM,CAAC,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC,MAAM,GAAG,MAAM;gBACrD,MAAM,IAAI,KAAK,CAAC,GAAG,KAAK,6CAA6C,CAAC,CAAA;YACvE,IAAI,IAAI;gBACP,MAAM,IAAI,KAAK,CACd,GAAG,KAAK,KAAK,aAAa,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,IAAI,KAAK,oCAAoC,CACpF,CAAA;YACF,IAAI,GAAG,IAAI,CAAA;YACX,
|
|
1
|
+
{"version":3,"file":"callback-inference.js","sourceRoot":"","sources":["../../../src/runtime/query/callback-inference.ts"],"names":[],"mappings":"AAAA,OAAO,EAAE,CAAC,EAAE,MAAM,KAAK,CAAA;AACvB,OAAO,EAAE,qBAAqB,EAAE,MAAM,+BAA+B,CAAA;AACrE,OAAO,EAAmB,eAAe,EAAE,MAAM,6BAA6B,CAAA;AAC9E,OAAO,EAAE,mBAAmB,EAAE,iBAAiB,EAAE,MAAM,8BAA8B,CAAA;AAIrF,MAAM,aAAa,GAAG,CAAC;KACrB,MAAM,CAAC;IACP,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC;IACrC,MAAM,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC;IACrC,SAAS,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,IAAI,CAAC,CAAC,OAAO,CAAC,GAAG,CAAC;IACzD,SAAS,EAAE,CAAC,CAAC,MAAM,EAAE,CAAC,GAAG,EAAE,CAAC,GAAG,CAAC,CAAC,CAAC,CAAC,GAAG,CAAC,MAAM,CAAC,CAAC,OAAO,CAAC,MAAM,CAAC;CAC9D,CAAC;KACD,MAAM,EAAE,CAAA;AAEV,oFAAoF;AACpF,MAAM,UAAU,uBAAuB,CACtC,GAAqB,EACrB,KAAa,EACb,KAA+B;IAE/B,MAAM,KAAK,GAAG,KAAK,KAAK,aAAa,CAAC,CAAC,CAAC,aAAa,CAAC,CAAC,CAAC,QAAQ,CAAA;IAChE,MAAM,KAAK,GAAG,KAAK,KAAK,aAAa,CAAC,CAAC,CAAC,mBAAmB,CAAC,CAAC,CAAC,iBAAiB,CAAA;IAC/E,MAAM,QAAQ,GAAG,IAAI,eAAe,EAAE,CAAA;IACtC,IAAI,IAAI,GAAG,KAAK,CAAA;IAChB,OAAO;QACN,KAAK,EAAE,GAAG,EAAE,CAAC,QAAQ,CAAC,KAAK,CAAC,IAAI,KAAK,CAAC,OAAO,KAAK,aAAa,CAAC,CAAC;QACjE,KAAK,CAAC,YAAY,CAAC,OAA+B;YACjD,QAAQ,CAAC,MAAM,CAAC,cAAc,EAAE,CAAA;YAChC,GAAG,CAAC,eAAe,CAAC,MAAM,CAAC,cAAc,EAAE,CAAA;YAC3C,OAAO,CAAC,MAAM,EAAE,cAAc,EAAE,CAAA;YAChC,MAAM,EAAE,MAAM,EAAE,eAAe,EAAE,GAAG,MAAM,EAAE,GAAG,OAAO,CAAA;YACtD,MAAM,KAAK,GAAG,aAAa,CAAC,KAAK,CAAC,MAAM,CAAC,CAAA;YACzC,IAAI,KAAK,CAAC,MAAM,CAAC,MAAM,GAAG,KAAK,CAAC,MAAM,CAAC,MAAM,GAAG,MAAM;gBACrD,MAAM,IAAI,KAAK,CAAC,GAAG,KAAK,6CAA6C,CAAC,CAAA;YACvE,IAAI,IAAI;gBACP,MAAM,IAAI,KAAK,CACd,GAAG,KAAK,KAAK,aAAa,CAAC,CAAC,CAAC,GAAG,CAAC,CAAC,CAAC,IAAI,IAAI,KAAK,oCAAoC,CACpF,CAAA;YACF,IAAI,GAAG,IAAI,CAAA;YACX,0DAA0D;YAC1D,qEAAqE;YACrE,mEAAmE;YACnE,qEAAqE;YACrE,oEAAoE;YACpE,sEAAsE;YACtE,iEAAiE;YACjE,qEAAqE;YACrE,gEAAgE;YAChE,kEAAkE;YAClE,MAAM,MAAM,GAAG,WAAW,CAAC,GAAG,CAAC;gBAC9B,GAAG,CAAC,eAAe,CAAC,MAAM;gBAC1B,QAAQ,CAAC,MAAM;gBACf,GAAG,CAAC,eAAe,CAAC,CAAC,CAAC,CAAC,eAAe,CAAC,CAAC,CAAC,CAAC,EAAE,CAAC;aAC7C,CAAC,CAAA;YACF,MAAM,SAAS,GAAG,GAAG,CAAC,aAAa,EAAE,EAAE,IAAI,EAAE,KAAK,EAAE,CAAC,EAAE,UAAU,EAAE,GAAG,CAAC,QAAQ,CAAC,EAAE,EAAE,CAAA;YACpF,MAAM,KAAK,GAAG,GAAG,EAAE;gBAClB,MAAM,MAAM,GAAG,GAAG,CAAC,aAAa,EAAE,EAAE,IAAI,SAAS,CAAA;gBACjD,OAAO;oBACN,UAAU,EAAE,MAAM,CAAC,UAAU;oBAC7B,KAAK,EAAE,MAAM,CAAC,KAAK,IAAI,KAAK;oBAC5B,UAAU,EAAE,MAAM,CAAC,KAAK;iBACxB,CAAA;YACF,CAAC,CAAA;YACD,MAAM,IAAI,GAAG,IAAI,qBAAqB,EAAE,CAAA;YACxC,IAAI,aAAiC,CAAA;YACrC,IAAI,KAA6B,CAAA;YACjC,IAAI,CAAC;gBACJ,IAAI,KAAK,EAAE,MAAM,KAAK,IAAI,GAAG,CAAC,QAAQ,CAAC,UAAU,CAAC;oBACjD,KAAK;oBACL,aAAa,EAAE,KAAK,EAAE;oBACtB,QAAQ,EAAE,CAAC,mBAAmB,CAAC,KAAK,CAAC,MAAM,CAAC,EAAE,iBAAiB,CAAC,KAAK,CAAC,MAAM,CAAC,CAAC;oBAC9E,SAAS,EAAE,KAAK,CAAC,SAAS;oBAC1B,GAAG,CAAC,GAAG,CAAC,SAAS,CAAC,MAAM,CAAC,CAAC,CAAC,EAAE,MAAM,EAAE,GAAG,CAAC,SAAS,CAAC,MAAM,EAAE,CAAC,CAAC,CAAC,EAAE,CAAC;oBACjE,MAAM;iBACN,CAAC,EAAE,CAAC;oBACJ,IAAI,KAAK,CAAC,KAAK;wBAAE,KAAK,GAAG,KAAK,CAAC,CAAC,CAAC,eAAe,CAAC,KAAK,EAAE,KAAK,CAAC,KAAK,CAAC,CAAC,CAAC,CAAC,EAAE,GAAG,KAAK,CAAC,KAAK,EAAE,CAAA;oBACzF,IAAI,KAAK,CAAC,KAAK;wBAAE,MAAM,IAAI,KAAK,CAAC,KAAK,CAAC,KAAK,CAAC,CAAA;oBAC7C,MAAM,CAAC,cAAc,EAAE,CAAA;oBACvB,IAAI,KAAK,CAAC,KAAK,CAAC,SAAS,EAAE,MAAM;wBAAE,aAAa,GAAG,GAAG,KAAK,+BAA+B,CAAA;oBAC1F,IAAI,CAAC,aAAa,EAAE,CAAC;wBACpB,IAAI,CAAC,IAAI,CAAC,KAAK,CAAC,CAAA;wBAChB,IAAI,IAAI,CAAC,UAAU,GAAG,IAAI;4BACzB,aAAa,GAAG,GAAG,KAAK,6CAA6C,CAAA;oBACvE,CAAC;gBACF,CAAC;gBACD,MAAM,CAAC,cAAc,EAAE,CAAA;gBACvB,IAAI,CAAC,KAAK;oBAAE,MAAM,IAAI,KAAK,CAAC,GAAG,KAAK,iCAAiC,CAAC,CAAA;gBACtE,IAAI,aAAa;oBAAE,MAAM,IAAI,KAAK,CAAC,aAAa,CAAC,CAAA;gBACjD,OAAO,EAAE,IAAI,EAAE,IAAI,CAAC,IAAI,EAAE,KAAK,EAAE,QAAQ,EAAE,KAAK,EAAE,EAAE,CAAA;YACrD,CAAC;oBAAS,CAAC;gBACV,IAAI,KAAK;oBAAE,GAAG,CAAC,MAAM,CAAC,eAAe,CAAC,KAAK,EAAE,KAAK,EAAE,CAAC,CAAA;YACtD,CAAC;QACF,CAAC;KACD,CAAA;AACF,CAAC"}
|
|
@@ -33,8 +33,32 @@ export interface ModelInfo {
|
|
|
33
33
|
contextWindow?: number;
|
|
34
34
|
/** Same, and absent for the same reason. */
|
|
35
35
|
maxOutputTokens?: number;
|
|
36
|
-
|
|
37
|
-
|
|
36
|
+
/**
|
|
37
|
+
* USD per million input tokens, when the driver knows the rate.
|
|
38
|
+
*
|
|
39
|
+
* OPTIONAL, and the optionality is the fix — the same one `contextWindow`
|
|
40
|
+
* got, applied to the field where a zero is most expensive. Six drivers
|
|
41
|
+
* wrote `0` wherever the vendor listing carries no rate, and a price of
|
|
42
|
+
* zero is not "I do not know", it is a billing fact: "this model is
|
|
43
|
+
* free". A consumer summing these under-reports a bill; one offering a
|
|
44
|
+
* `(free)` marker — the read that found this — labels every paid model
|
|
45
|
+
* on a driver that never learned its rates.
|
|
46
|
+
*
|
|
47
|
+
* The distinction is not new here, only lost in transit.
|
|
48
|
+
* `resolveModelPricing` already returns `undefined` for a rate nobody
|
|
49
|
+
* has and `{0, 0}` for a driver that genuinely bills nothing, and says
|
|
50
|
+
* in as many words that a caller flattening the two reproduces the
|
|
51
|
+
* defect the pricing module exists to remove. This field carried the
|
|
52
|
+
* flattened version to every consumer that never reached that module.
|
|
53
|
+
*
|
|
54
|
+
* Absent means unknown. `0` means free, and is only written by a driver
|
|
55
|
+
* that knows it: the local servers, and a catalogue whose source names
|
|
56
|
+
* the model as free. A caller that must show a number should render
|
|
57
|
+
* "unknown" rather than `$0.00`, which is the conclusion being fixed.
|
|
58
|
+
*/
|
|
59
|
+
inputPrice?: number;
|
|
60
|
+
/** Same, and absent for the same reason. */
|
|
61
|
+
outputPrice?: number;
|
|
38
62
|
supportsToolUse: boolean;
|
|
39
63
|
supportsStreaming: boolean;
|
|
40
64
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"model.d.ts","sourceRoot":"","sources":["../../../src/types/provider/model.ts"],"names":[],"mappings":"AAAA,kDAAkD;AAClD,MAAM,MAAM,kBAAkB,GAAG,MAAM,GAAG,OAAO,GAAG,UAAU,CAAA;AAE9D,MAAM,WAAW,SAAS;IACzB,EAAE,EAAE,MAAM,CAAA;IACV,IAAI,EAAE,MAAM,CAAA;IACZ;;;;;;;;OAQG;IACH,eAAe,CAAC,EAAE,SAAS,kBAAkB,EAAE,CAAA;IAC/C,2GAA2G;IAC3G,qBAAqB,CAAC,EAAE,SAAS,OAAO,WAAW,EAAE,eAAe,EAAE,CAAA;IACtE,uGAAuG;IACvG,sBAAsB,CAAC,EAAE,OAAO,WAAW,EAAE,eAAe,CAAA;IAC5D;;;;;;;;;;;;OAYG;IACH,aAAa,CAAC,EAAE,MAAM,CAAA;IACtB,4CAA4C;IAC5C,eAAe,CAAC,EAAE,MAAM,CAAA;IACxB,UAAU,EAAE,MAAM,CAAA;
|
|
1
|
+
{"version":3,"file":"model.d.ts","sourceRoot":"","sources":["../../../src/types/provider/model.ts"],"names":[],"mappings":"AAAA,kDAAkD;AAClD,MAAM,MAAM,kBAAkB,GAAG,MAAM,GAAG,OAAO,GAAG,UAAU,CAAA;AAE9D,MAAM,WAAW,SAAS;IACzB,EAAE,EAAE,MAAM,CAAA;IACV,IAAI,EAAE,MAAM,CAAA;IACZ;;;;;;;;OAQG;IACH,eAAe,CAAC,EAAE,SAAS,kBAAkB,EAAE,CAAA;IAC/C,2GAA2G;IAC3G,qBAAqB,CAAC,EAAE,SAAS,OAAO,WAAW,EAAE,eAAe,EAAE,CAAA;IACtE,uGAAuG;IACvG,sBAAsB,CAAC,EAAE,OAAO,WAAW,EAAE,eAAe,CAAA;IAC5D;;;;;;;;;;;;OAYG;IACH,aAAa,CAAC,EAAE,MAAM,CAAA;IACtB,4CAA4C;IAC5C,eAAe,CAAC,EAAE,MAAM,CAAA;IACxB;;;;;;;;;;;;;;;;;;;;;;OAsBG;IACH,UAAU,CAAC,EAAE,MAAM,CAAA;IACnB,4CAA4C;IAC5C,WAAW,CAAC,EAAE,MAAM,CAAA;IACpB,eAAe,EAAE,OAAO,CAAA;IACxB,iBAAiB,EAAE,OAAO,CAAA;CAC1B"}
|
|
@@ -11,7 +11,18 @@ export interface PreparationTextRequest {
|
|
|
11
11
|
readonly prompt: string;
|
|
12
12
|
/** Default 256; maximum 1,024. A provider limit, not a hard billing ceiling. */
|
|
13
13
|
readonly maxTokens?: number;
|
|
14
|
-
/**
|
|
14
|
+
/**
|
|
15
|
+
* Accepted and validated; no longer bounds the request.
|
|
16
|
+
*
|
|
17
|
+
* @deprecated An auxiliary request that ends without its final usage
|
|
18
|
+
* receipt leaves the run's shared ledger with unresolved spend, which stops
|
|
19
|
+
* the run — so a deadline short enough to fire in normal use does not bound
|
|
20
|
+
* the call, it ends the run that made it. The call is bounded by the
|
|
21
|
+
* provider's own request timeout and by the run's cancellation instead, the
|
|
22
|
+
* same two bounds every other model request in the run has. Passing this
|
|
23
|
+
* field changes nothing; it is retained so an existing caller keeps
|
|
24
|
+
* compiling until it is removed in a later major.
|
|
25
|
+
*/
|
|
15
26
|
readonly timeoutMs?: number;
|
|
16
27
|
readonly signal?: AbortSignal;
|
|
17
28
|
}
|
|
@@ -1 +1 @@
|
|
|
1
|
-
{"version":3,"file":"prepare-step.d.ts","sourceRoot":"","sources":["../../../src/types/run/prepare-step.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,oBAAoB,CAAA;AACpD,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,iBAAiB,CAAA;AAC5C,OAAO,KAAK,EAAE,OAAO,EAAE,WAAW,EAAE,MAAM,qBAAqB,CAAA;AAC/D,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,qBAAqB,CAAA;AACrD,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,oBAAoB,CAAA;AAC/C,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,WAAW,CAAA;AAC3C,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,WAAW,CAAA;AAE/C,0FAA0F;AAC1F,MAAM,WAAW,sBAAsB;IACtC,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;IACvB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;IACvB,gFAAgF;IAChF,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAA;IAC3B
|
|
1
|
+
{"version":3,"file":"prepare-step.d.ts","sourceRoot":"","sources":["../../../src/types/run/prepare-step.ts"],"names":[],"mappings":"AAAA,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,oBAAoB,CAAA;AACpD,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,iBAAiB,CAAA;AAC5C,OAAO,KAAK,EAAE,OAAO,EAAE,WAAW,EAAE,MAAM,qBAAqB,CAAA;AAC/D,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,qBAAqB,CAAA;AACrD,OAAO,KAAK,EAAE,KAAK,EAAE,MAAM,oBAAoB,CAAA;AAC/C,OAAO,KAAK,EAAE,UAAU,EAAE,MAAM,WAAW,CAAA;AAC3C,OAAO,KAAK,EAAE,cAAc,EAAE,MAAM,WAAW,CAAA;AAE/C,0FAA0F;AAC1F,MAAM,WAAW,sBAAsB;IACtC,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;IACvB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;IACvB,gFAAgF;IAChF,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAA;IAC3B;;;;;;;;;;;OAWG;IACH,QAAQ,CAAC,SAAS,CAAC,EAAE,MAAM,CAAA;IAC3B,QAAQ,CAAC,MAAM,CAAC,EAAE,WAAW,CAAA;CAC7B;AAED,iFAAiF;AACjF,MAAM,WAAW,qBAAqB;IACrC,QAAQ,CAAC,IAAI,EAAE,MAAM,CAAA;IACrB,QAAQ,CAAC,KAAK,EAAE,UAAU,CAAA;IAC1B,QAAQ,CAAC,QAAQ,EAAE,cAAc,CAAA;CACjC;AAED;;;;;;;;;;GAUG;AACH,MAAM,WAAW,kBAAkB;IAClC,QAAQ,CAAC,KAAK,EAAE,KAAK,CAAA;IACrB;;;;;;;OAOG;IACH,QAAQ,CAAC,YAAY,CAAC,EAAE,CAAC,OAAO,EAAE,sBAAsB,KAAK,OAAO,CAAC,qBAAqB,CAAC,CAAA;IAC3F;;;;;OAKG;IACH,QAAQ,CAAC,kBAAkB,CAAC,EAAE,CAC7B,YAAY,CAAC,EAAE,MAAM,EACrB,MAAM,CAAC,EAAE,WAAW,KAChB,OAAO,CAAC,OAAO,+BAA+B,EAAE,qBAAqB,GAAG,SAAS,CAAC,CAAA;IACvF,mEAAmE;IACnE,QAAQ,CAAC,UAAU,EAAE,MAAM,CAAA;IAC3B,uEAAuE;IACvE,QAAQ,CAAC,QAAQ,EAAE,SAAS,OAAO,EAAE,CAAA;IACrC;;;;OAIG;IACH,QAAQ,CAAC,iBAAiB,CAAC,EAAE,WAAW,CAAA;IACxC,2EAA2E;IAC3E,QAAQ,CAAC,MAAM,CAAC,EAAE,WAAW,CAAA;IAC7B;;;;;OAKG;IACH,QAAQ,CAAC,aAAa,CAAC,EAAE;QACxB,QAAQ,CAAC,eAAe,EAAE,MAAM,CAAA;QAChC,QAAQ,CAAC,YAAY,EAAE,MAAM,CAAA;KAC7B,CAAA;IACD,sCAAsC;IACtC,QAAQ,CAAC,KAAK,EAAE,SAAS,UAAU,EAAE,CAAA;IAErC;;;;;;;OAOG;IACH,QAAQ,CAAC,QAAQ,EAAE,QAAQ,CAAC,iBAAiB,CAAC,CAAA;CAC9C;AAED;;;;GAIG;AACH,MAAM,WAAW,iBAAiB;IACjC;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;OAiCG;IACH,QAAQ,CAAC,WAAW,CAAC,EAAE,SAAS,MAAM,EAAE,CAAA;IAExC;;;;;;;;;;;;;;;;;;;;OAoBG;IACH,QAAQ,CAAC,UAAU,CAAC,EAAE,UAAU,CAAA;IAEhC;;;;;;;;;;;;;;;;;;;;;;;OAuBG;IACH,QAAQ,CAAC,MAAM,CAAC,EAAE,SAAS,KAAK,EAAE,CAAA;IAElC,2CAA2C;IAC3C,QAAQ,CAAC,KAAK,CAAC,EAAE,MAAM,CAAA;IAEvB;;;;;;;OAOG;IACH,QAAQ,CAAC,MAAM,CAAC,EAAE,MAAM,CAAA;IAExB;;;;;;;;;;OAUG;IACH,QAAQ,CAAC,OAAO,CAAC,EAAE,MAAM,CAAA;IAEzB,QAAQ,CAAC,WAAW,CAAC,EAAE,MAAM,CAAA;IAC7B,QAAQ,CAAC,iBAAiB,CAAC,EAAE,MAAM,CAAA;CACnC;AAED;;;;;;;;GAQG;AACH,MAAM,MAAM,WAAW,GAAG,CACzB,OAAO,EAAE,kBAAkB,KACvB,iBAAiB,GAAG,SAAS,GAAG,OAAO,CAAC,iBAAiB,GAAG,SAAS,CAAC,CAAA;AAE3E;;;;;;;;GAQG;AACH,MAAM,WAAW,QAAQ;IACxB,QAAQ,CAAC,MAAM,EAAE,MAAM,CAAA;CACvB;AAED;;;;;;;;;;;;;;;GAeG;AACH,MAAM,MAAM,UAAU,GAAG,CACxB,OAAO,EAAE,kBAAkB,KACvB,QAAQ,GAAG,SAAS,GAAG,OAAO,CAAC,QAAQ,GAAG,SAAS,CAAC,CAAA;AAEzD;;;;;;;;;;;;;;;;;;GAkBG;AACH,MAAM,MAAM,gBAAgB,GAAG,WAAW,GAAG,SAAS,WAAW,EAAE,CAAA"}
|
package/package.json
CHANGED
|
@@ -149,6 +149,13 @@ export function defineProviderDriverConformance(options: ProviderDriverConforman
|
|
|
149
149
|
// window, and it reads to every consumer as a measurement of a model
|
|
150
150
|
// that can hold nothing.
|
|
151
151
|
//
|
|
152
|
+
// The same reasoning makes `inputPrice`/`outputPrice` optional, and a
|
|
153
|
+
// price is the more expensive of the two to get wrong: a zero window
|
|
154
|
+
// is a nonsensical measurement, where a zero PRICE is a plain claim
|
|
155
|
+
// that a model is free — one a menu will print, and one a `(free)`
|
|
156
|
+
// marker reads as a fact. Six drivers made that claim on every model
|
|
157
|
+
// they could not price.
|
|
158
|
+
//
|
|
152
159
|
// The obvious place to enforce that is here, reading `listModels`.
|
|
153
160
|
// It was here, and it was worse than nothing twice over. For every
|
|
154
161
|
// in-tree driver `listModels` needs a live service, so the rule
|
|
@@ -160,7 +167,12 @@ export function defineProviderDriverConformance(options: ProviderDriverConforman
|
|
|
160
167
|
//
|
|
161
168
|
// The reintroduction of a literal zero is caught by a source scan in
|
|
162
169
|
// `provider/__tests__/conformance-fails-a-wrong-driver.test.ts`,
|
|
163
|
-
// which depends on reaching nothing.
|
|
170
|
+
// which depends on reaching nothing. It holds two further cases for
|
|
171
|
+
// prices, and the sharper of them is not the literal: a price
|
|
172
|
+
// DEFAULTED to zero cannot be written without a `??` or a `||`, and
|
|
173
|
+
// that is catchable without deciding whether any particular `0` was
|
|
174
|
+
// meant — which a text scan never can, since ollama's honest zero and
|
|
175
|
+
// a placeholder zero are the same characters.
|
|
164
176
|
|
|
165
177
|
it('declares the retry defaults its conformance run says it does', async () => {
|
|
166
178
|
// The option above forces the decision; this asserts the answer is
|
|
@@ -39,15 +39,19 @@ export function createCallbackInference(
|
|
|
39
39
|
`${phase === 'preparation' ? 'A' : 'An'} ${owner} may make only one inference call.`,
|
|
40
40
|
)
|
|
41
41
|
used = true
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
42
|
+
// No deadline of our own on the request — see the note on
|
|
43
|
+
// `PreparationTextRequest.timeoutMs`. The two bounds that remain are
|
|
44
|
+
// the ones every other model request in the run has: the run's own
|
|
45
|
+
// cancellation, and the provider's request timeout. An expired local
|
|
46
|
+
// deadline here would abort a CONTACTED request, and a request that
|
|
47
|
+
// ends without its usage receipt leaves the shared ledger unresolved,
|
|
48
|
+
// which stops the whole run — so a stage that gave up on its own
|
|
49
|
+
// optional inference would take the operator's turn with it. That is
|
|
50
|
+
// not hypothetical: it is what a 10s deadline did to every turn
|
|
51
|
+
// against a reasoning model slower than 10s per auxiliary answer.
|
|
47
52
|
const signal = AbortSignal.any([
|
|
48
53
|
ctx.abortController.signal,
|
|
49
54
|
lifetime.signal,
|
|
50
|
-
deadline.signal,
|
|
51
55
|
...(requestedSignal ? [requestedSignal] : []),
|
|
52
56
|
])
|
|
53
57
|
const requested = ctx.servingMember?.() ?? { index: 0, providerId: ctx.provider.id }
|
|
@@ -86,7 +90,6 @@ export function createCallbackInference(
|
|
|
86
90
|
if (invalidOutput) throw new Error(invalidOutput)
|
|
87
91
|
return { text: text.text, usage, servedBy: route() }
|
|
88
92
|
} finally {
|
|
89
|
-
clearTimeout(timer)
|
|
90
93
|
if (usage) ctx.runMgr.accumulateUsage(usage, route())
|
|
91
94
|
}
|
|
92
95
|
},
|
|
@@ -34,8 +34,32 @@ export interface ModelInfo {
|
|
|
34
34
|
contextWindow?: number
|
|
35
35
|
/** Same, and absent for the same reason. */
|
|
36
36
|
maxOutputTokens?: number
|
|
37
|
-
|
|
38
|
-
|
|
37
|
+
/**
|
|
38
|
+
* USD per million input tokens, when the driver knows the rate.
|
|
39
|
+
*
|
|
40
|
+
* OPTIONAL, and the optionality is the fix — the same one `contextWindow`
|
|
41
|
+
* got, applied to the field where a zero is most expensive. Six drivers
|
|
42
|
+
* wrote `0` wherever the vendor listing carries no rate, and a price of
|
|
43
|
+
* zero is not "I do not know", it is a billing fact: "this model is
|
|
44
|
+
* free". A consumer summing these under-reports a bill; one offering a
|
|
45
|
+
* `(free)` marker — the read that found this — labels every paid model
|
|
46
|
+
* on a driver that never learned its rates.
|
|
47
|
+
*
|
|
48
|
+
* The distinction is not new here, only lost in transit.
|
|
49
|
+
* `resolveModelPricing` already returns `undefined` for a rate nobody
|
|
50
|
+
* has and `{0, 0}` for a driver that genuinely bills nothing, and says
|
|
51
|
+
* in as many words that a caller flattening the two reproduces the
|
|
52
|
+
* defect the pricing module exists to remove. This field carried the
|
|
53
|
+
* flattened version to every consumer that never reached that module.
|
|
54
|
+
*
|
|
55
|
+
* Absent means unknown. `0` means free, and is only written by a driver
|
|
56
|
+
* that knows it: the local servers, and a catalogue whose source names
|
|
57
|
+
* the model as free. A caller that must show a number should render
|
|
58
|
+
* "unknown" rather than `$0.00`, which is the conclusion being fixed.
|
|
59
|
+
*/
|
|
60
|
+
inputPrice?: number
|
|
61
|
+
/** Same, and absent for the same reason. */
|
|
62
|
+
outputPrice?: number
|
|
39
63
|
supportsToolUse: boolean
|
|
40
64
|
supportsStreaming: boolean
|
|
41
65
|
}
|
|
@@ -12,7 +12,18 @@ export interface PreparationTextRequest {
|
|
|
12
12
|
readonly prompt: string
|
|
13
13
|
/** Default 256; maximum 1,024. A provider limit, not a hard billing ceiling. */
|
|
14
14
|
readonly maxTokens?: number
|
|
15
|
-
/**
|
|
15
|
+
/**
|
|
16
|
+
* Accepted and validated; no longer bounds the request.
|
|
17
|
+
*
|
|
18
|
+
* @deprecated An auxiliary request that ends without its final usage
|
|
19
|
+
* receipt leaves the run's shared ledger with unresolved spend, which stops
|
|
20
|
+
* the run — so a deadline short enough to fire in normal use does not bound
|
|
21
|
+
* the call, it ends the run that made it. The call is bounded by the
|
|
22
|
+
* provider's own request timeout and by the run's cancellation instead, the
|
|
23
|
+
* same two bounds every other model request in the run has. Passing this
|
|
24
|
+
* field changes nothing; it is retained so an existing caller keeps
|
|
25
|
+
* compiling until it is removed in a later major.
|
|
26
|
+
*/
|
|
16
27
|
readonly timeoutMs?: number
|
|
17
28
|
readonly signal?: AbortSignal
|
|
18
29
|
}
|