weflayr 0.32.0 → 0.32.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
package/dist/package.json
CHANGED
|
@@ -214,7 +214,7 @@ export type BenchmarkConfigInput = {
|
|
|
214
214
|
/**
|
|
215
215
|
* Auto Disallow Too Costly Reasoning Efforts
|
|
216
216
|
*
|
|
217
|
-
* Stop testing a model at a reasoning effort that turns out far more expensive than its medium effort, and leave its answers out of the results: once the first 3 examples are done, then after every example, an effort costing on average more than 6 times the model's medium on the same calls (its default effort when it has no medium), while writing at least twice as many output tokens as the baseline, is removed.
|
|
217
|
+
* Stop testing a model at a reasoning effort that turns out far more expensive than its medium effort, and leave its answers out of the results: once the first 3 examples are done, then after every example, an effort costing on average more than 6 times the model's medium on the same calls (its default effort when it has no medium), while writing at least twice as many output tokens as the baseline, is removed. A model is also dropped from the benchmark when, at the effort it is first tested at, it writes at least 250,000 output tokens to a call whose logged answer wrote fewer than 10,000.
|
|
218
218
|
*/
|
|
219
219
|
auto_disallow_too_costly_reasoning_efforts?: boolean;
|
|
220
220
|
/**
|