dsh-llm-verifier 0.9.3 → 0.9.5
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +47 -0
- package/README.md +6 -4
- package/icon.svg +13 -0
- package/lib/{caller-Bfp4HPK_.js → caller-CPbFWOz7.js} +54 -4
- package/lib/caller-CPbFWOz7.js.map +1 -0
- package/lib/caller.js +1 -1
- package/lib/client.js +3 -3
- package/lib/client.js.map +1 -1
- package/lib/core.js +1 -1
- package/lib/core.js.map +1 -1
- package/lib/index.js +6 -6
- package/lib/index.js.map +1 -1
- package/lib/types/caller.d.ts.map +1 -1
- package/lib/types/caller.js +16 -3
- package/lib/types/caller.js.map +1 -1
- package/lib/types/client-fields.js +1 -1
- package/lib/types/config.js +2 -2
- package/lib/types/core.d.ts +1 -1
- package/lib/types/core.d.ts.map +1 -1
- package/lib/types/core.js +1 -1
- package/lib/types/core.js.map +1 -1
- package/lib/types/engine.js +1 -1
- package/lib/types/top-logprobs.d.ts +15 -0
- package/lib/types/top-logprobs.d.ts.map +1 -1
- package/lib/types/top-logprobs.js +40 -1
- package/lib/types/top-logprobs.js.map +1 -1
- package/package.json +12 -6
- package/src/cache.test.ts +13 -1
- package/src/caller.test.ts +43 -1
- package/src/caller.ts +16 -3
- package/src/client-fields.ts +1 -1
- package/src/config.test.ts +3 -3
- package/src/config.ts +2 -2
- package/src/core.test.ts +16 -0
- package/src/core.ts +1 -1
- package/src/engine.test.ts +3 -3
- package/src/engine.ts +1 -1
- package/src/top-logprobs.test.ts +51 -1
- package/src/top-logprobs.ts +38 -1
- package/src/topic-storage.test.ts +18 -8
- package/lib/caller-Bfp4HPK_.js.map +0 -1
package/CHANGELOG.md
ADDED
|
@@ -0,0 +1,47 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## Unreleased
|
|
4
|
+
|
|
5
|
+
## 0.9.5 - 2026-10-15
|
|
6
|
+
|
|
7
|
+
- **[UI / 打包] 补上插件包图标**
|
|
8
|
+
- 新增根目录 `icon.svg`:与侧栏徽章同一套紫→蓝→青渐变,白色护盾与对勾,48×48 viewBox 覆盖卡片 36px 与列表行 30px 两种渲染尺寸
|
|
9
|
+
- `package.json` 新增 `icon` 字段并把 `icon.svg` 加入 `files`;宿主插件管理器按该字段为插件卡片、列表行与详情页渲染封面,缺省时仍回落宿主缺省图
|
|
10
|
+
- **[对齐上游 v0.3.0] 判官默认输出预算 32768 → 65536**
|
|
11
|
+
- 上游 `llm-as-a-verifier` 自 `8db8a114`(v0.2.0)推进到 `90e8fc21`(v0.3.0),**算法零改动**(评分、锦标赛与进度模块逐文件未变),只改了接入层三处:判官模型 id 改用滚动别名端点、输出预算翻倍、以及"没有答案 logprobs"时的错误分类
|
|
12
|
+
- 本插件据此把判官默认 `maxTokens` 对齐到 **65536**(`config.ts` 的 schema 默认值与 `resolveConfig` 兜底、`client-fields.ts` 的 `CONFIG_DEFAULTS` 三处同步):`deepseek-flash` 推理开销比上一代更高,旧预算会让百分之几的比较拿不到分数
|
|
13
|
+
- 评分缓存身份版本随之 **6 → 7**(`engine.ts` 的 `identity`),避免旧预算写入的评分被新默认值命中
|
|
14
|
+
- `GENERATION_MAX_TOKENS`(草稿上限,16384)**不受影响**:它受"实测单份推理用量约 2 倍"与"两份草稿仍留在 24 万字符证据上限内"两个独立约束,与判官预算无关
|
|
15
|
+
- **[修复] 判官拿不到 logprobs 时区分"预算耗尽"与"能力不足"**
|
|
16
|
+
- 此前两者都抛 `TopLogprobsUnsupportedError`,而该类型正是降级判据:命中即 `markUnsupported(provider, model)` 把路由**记进通道能力记忆**(24h)并回退到显式标签通道。因此一次"推理吃满输出预算"的配置失误,会被误记成"该模型不支持 logprobs",让该路由在 TTL 内持续以单字母采样(A–T 每档约 5.3%)的精度出分
|
|
17
|
+
- 现按响应自身的 `finish_reason` 分流:`'length'` 抛新增的 `TopLogprobsBudgetError`(**直接继承 `Error`,不是 `TopLogprobsUnsupportedError` 子类**,故不触发降级、不写能力记忆),消息带出实际预算与两条可操作建议;其余成因(含字段缺失回落 `unknown`)保持原能力探测语义与降级行为
|
|
18
|
+
- **[修复] 重试判据的数字边界与确定性失败排除**
|
|
19
|
+
- `RETRYABLE_MESSAGE` 的 `5\d\d` 无数字边界,任何"以 5 开头的三位数字"都被读成 HTTP 5xx;判官预算升到 65536 后,`65536` 的子串 `553` 命中该规则,使这条**确定性**失败被判成可重试,在默认 `maxRetries: 3` 下白跑 4 次满额推理调用
|
|
20
|
+
- 现按类型排除预算错误,并把 5xx 规则锚定到数字边界(`"HTTP 503"` 仍命中,`"65536"` 不再命中);两条修改相互独立
|
|
21
|
+
- **[改进] Ground Truth Note 补强**
|
|
22
|
+
- 融合上游 v0.3.0 新增的反奖励黑客指引:漂亮的总结、整齐的表格与"已完全验证"的声称在失败的运行里同样常见;缺少自认问题**不是**证据,但自认的问题**就是**证据并须据此计权;含糊其辞、表达不确定与轨迹长度均属中性;判实质与如何被检查,不判语气
|
|
23
|
+
- **[文档] 上游核对基准更新为 v0.3.0**
|
|
24
|
+
- README 的核对基准注释由 `8db8a114`(v0.2.0)更新为 `90e8fc21`(v0.3.0),并补一段本次对齐说明;「最大输出 Token」条目补默认值与预算吃满时的行为;「自适应降级」条目区分"供应商未回传概率"与"推理吃满预算";备选模型池示例由 `deepseek-v4-flash` 更新为滚动别名 `deepseek-flash`
|
|
25
|
+
- **[测试] 新增 7 条回归断言**
|
|
26
|
+
- `top-logprobs.test.ts`:三种成因分流(`length` 走预算错误、`stop` 走能力探测、字段缺失回落 `unknown`)
|
|
27
|
+
- `caller.test.ts`:预算错误不重试且**不污染能力记忆**、纯消息边界(引用大额数字的普通失败仍只跑一次)、`HTTP 503` 仍可重试
|
|
28
|
+
- `core.test.ts`:Ground Truth Note 的金丝雀断言,防止后续被裁回旧版单句
|
|
29
|
+
|
|
30
|
+
## 0.9.4 - 2026-10-10
|
|
31
|
+
|
|
32
|
+
- **[构建 / 发布] `lib/` 转为构建产物,不再入库**
|
|
33
|
+
- `lib/` 加入 `.gitignore` 并解除跟踪;发布时由 `prepublishOnly`(`verify:release`)重建,再经 `files` 字段打进 tarball
|
|
34
|
+
- `files` 新增 `CHANGELOG.md`,并固定 `publishConfig.access` 为 `public`
|
|
35
|
+
- **[自动化] 接入 GitHub Actions 自动发版工作流与发版技能**
|
|
36
|
+
- `v*` tag 触发:typecheck → build → test → 从 CHANGELOG 提取 release notes → `pnpm pack` 打包 → 发布 npm 与 GitHub Release
|
|
37
|
+
- 附带 CI 工作流,在 Ubuntu 与 Windows 双平台验证
|
|
38
|
+
- **[修复] 发版工作流的三处失败点**
|
|
39
|
+
- 锁定 pnpm 11:pnpm 9 无法解析 `pnpm-workspace.yaml` 的 `minimumReleaseAgeExclude`,会把每条命令都误报成 `packages field missing or empty`
|
|
40
|
+
- 移除会在任何测试运行前就失败的 pnpm 缓存步骤
|
|
41
|
+
- `pnpm pack` 的完整清单曾被整体写入 `$GITHUB_OUTPUT`,导致 Release 收到多行 blob;现只取 `.tgz` 那一行,并在 `NPM_TOKEN` 缺失时直接给出补救提示,而非只抛裸 `ENEEDAUTH`
|
|
42
|
+
- **[测试] 两处断言改为平台无关表达**:缓存路径断言与 topic-storage fixture 改用平台辅助函数构造,不再依赖 POSIX 形式路径
|
|
43
|
+
- **[杂项] 清理失效的 `.claude/skills` 指针与临时截图**,`AGENTS.md` 硬性规矩 1 同步更正为「`lib/` 不入库」
|
|
44
|
+
|
|
45
|
+
## 0.9.3 - 2026-10-05
|
|
46
|
+
|
|
47
|
+
- **[UI / Client] 新增 LLM Verifier Web 界面组件与源码映射**
|
package/README.md
CHANGED
|
@@ -135,10 +135,12 @@ typecheck 门禁按 `package.json` 的 `devDependencies` 解析,已随宿主
|
|
|
135
135
|
|
|
136
136
|
```python
|
|
137
137
|
# llm_verifier/fine_grained_reward.py:678
|
|
138
|
-
# 核对基准:上游
|
|
138
|
+
# 核对基准:上游 v0.3.0 @ 90e8fc215606447d6d31e9e24bff7200fb0c0994(pyproject version 0.3.0)
|
|
139
139
|
probs[val] = max(probs.get(val, 0.0), p)
|
|
140
140
|
```
|
|
141
141
|
|
|
142
|
+
> 上游 v0.3.0 对算法零改动:评分、锦标赛与进度模块逐文件未变,动的只有接入层三处——判官模型 id(改用滚动别名端点)、输出预算,以及"没有答案 logprobs"时的错误分类(区分"推理吃满输出预算"与"模型没有产出答案")。本插件据此后把判官默认输出预算对齐到 65536,并补强了 Ground Truth Note。
|
|
143
|
+
|
|
142
144
|
`max` 会系统性低估"概率质量被拆到两种写法上"的字母(例:`" A"=0.4, "A"=0.4, " T"=0.5, "T"=0.1` 时,求和得 ≈0.5714,取最大值只有 ≈0.4444,足以跨过通过阈值),因此这里选择求和。该聚合在上游也没有被测试或文档固定下来,属于可自由取舍的实现细节;一致性测试的 fixture 有意不覆盖该情形(否则会必然失败)。
|
|
143
145
|
|
|
144
146
|
### 赛制(probabilistic pivot tournament)
|
|
@@ -215,7 +217,7 @@ T = 明确且无可争议的失败
|
|
|
215
217
|
字母按等距标尺线性映射到 `[0, 1]` 区间。在打分计算上,插件采用**自适应评分机制**:
|
|
216
218
|
|
|
217
219
|
- **优先概率期望评分**:若当前模型路由为官方 `deepseek-official` 或显式配置了兼容 OpenAI Chat Completions 且支持 `top_logprobs` 的端点,插件将基于 A–T 候选 Token 的完整概率分布计算加权期望值;
|
|
218
|
-
- **自适应降级显式标签**:若当前路由不支持、供应商拒绝返回 `logprobs`,或响应中缺失 Token 概率,则平滑回退至模型最终输出的 A–T
|
|
220
|
+
- **自适应降级显式标签**:若当前路由不支持、供应商拒绝返回 `logprobs`,或响应中缺失 Token 概率,则平滑回退至模型最终输出的 A–T 显式标签。此处的"缺失 Token 概率"指供应商**没有回传概率**,而不是**推理吃满了输出预算**——后者是配置问题,会直接报错并提示调大 `最大输出 Token` 或调低推理强度,且不写入通道能力记忆(否则一次配置失误会让该路由被误判 24h"不支持 logprobs")。
|
|
219
221
|
- *原则:插件不会仅凭供应商或模型名称预设其能力,始终以运行时实际响应结果为准。*
|
|
220
222
|
|
|
221
223
|
> [!NOTE]
|
|
@@ -281,7 +283,7 @@ flowchart LR
|
|
|
281
283
|
| **接入真实文件改动证据** | 默认**开启**。会话验收(显式 `verifier_current_session` 与自动最终验收)把**宿主自己记录的**本轮文件改动作为证据交给裁判:改动文件清单(路径 + 增删行数)与逐文件前后对比(`@@` 块 + 带 `+`/`-`/空格前缀的行),经参考上下文分隔块进入提示词(因此改动不同必然重新评分,评分缓存键包含该文本)。它修的是「裁判只能从 `tool/call` 的参数反推 Agent 声称改了什么」——改完没落盘、或事后又被改写,过去与真实改动无从区分。限长沿用同一套预算:整块不超过「单项字符上限」(截断提示计入),最多渲染 8 个文件(超出在表头注明),其余字符按文件数用 `itemBudget` 分摊、逐文件按行截断并注明省略行数;`binary` / `oversized` 文件只列一行说明、不读取对比。宿主未提供该服务(0.1.1/0.1.5)、会话已销毁、`summary` 或 `diff` 报错时一律**静默降级为不带该证据**并在宿主日志告警,绝不阻断验收。需要 DSH 0.1.6+ 宿主 |
|
|
282
284
|
| **过程选优** | 三档(旧布尔值兼容:`true`→`recovery`,`false`/缺省→`off`):`off`(默认)完全不进入该路径(不生成候选、不调用裁判、不缓冲响应);`recovery`(恢复触发)仅当同一任务最近两次已完成的验证运行**都失败**(「失败」从运行自己的判决读:`[exit code: N≠0]`、`N failed`、`error TS…`、`test result: FAILED`、`FAIL` 都算,工具级报错也算;`0 failed` 不算失败)时,为**下一次主模型请求**额外生成备选回复,每任务最多 1 个周期;`every-step`(每步选优)为**每个主循环请求**登记选优意图,无需失败信号,每任务最多 `maxProcessCyclesPerTask` 个周期。三档均**仅 smart 模式**生效(manual/strict 不进入该路径),触发后机制相同:额外生成备选(默认 N=2:原回复=候选 0,新增一份=候选 1;`autoProcessCandidates` 可提到 3 或 4,那时按锦标赛判定;采样温度不低于生成档位,避免低温会话买到原回复的副本),按 **process 判据**(失败靶向 / 与已失败尝试不同 / 可验证的一步)比较后**只回放胜者**。过程周期有自己的每任务上限(独立于路由与最终验收的第三本账),同时也消耗「每任务/每会话最多路由」额度,并受任务/会话模型调用预算与最终验收保留额度共同限制;生成与比较共用一个 `timeoutMs` 阶段截止时间。裁判视图里的任务、约束、最近失败证据与两份候选(含工具名与参数)**先脱敏再限长**,并按实际渲染文本计量总量,工具动作无法完整容纳时回退原回复。选优**不是**验收:胜者交给宿主后,最终验收照常覆盖随后产生的实际工作 |
|
|
283
285
|
| **备选带上失败证据** | 默认**开启**,仅在过程选优非 `off` 时生效。`recovery` 档把触发本周期的那两次失败验证运行(先 `sanitizeVerifierText` 脱敏,再按单项/总量双层限长,总量 4000 字符)作为一条插件 user 消息附在备选请求之后,让额外生成的候选是针对真实失败的一次不同尝试,而不是同一提示词的再抽样;`every-step` 档只在任务恰好出现两连败信号时附带,无信号则不带;关闭即对照组(两份候选掌握的信息完全相同,周期照买)。裁判**不被告知**哪份候选带了证据,避免按来源而不是按内容打分;统计行的 `route.alternativeAugmented` 记录该差异,供真实任务对照读取。 |
|
|
284
|
-
| **备选用另一个模型** | 默认留空:备选由当前会话模型再采样。可填**逗号分隔**的 `provider/model` 列表(例如 `deepseek-official/deepseek-
|
|
286
|
+
| **备选用另一个模型** | 默认留空:备选由当前会话模型再采样。可填**逗号分隔**的 `provider/model` 列表(例如 `deepseek-official/deepseek-flash, openai/gpt-5`):第 i 份备选使用第 i 项,列表短于备选数时循环取用,候选之间因此是真正的不同假设;跨 provider 时**不继承**原请求的推理强度(适配器可能不认识该 id),`maxTokens` 与生成温度下限照旧,归一化后的整个列表记进 `route.alternativeModel`。代价是比较同时也在比「哪个模型更好」,与任务本身的质量不可分——所以它是实验臂,不是推荐配置。 |
|
|
285
287
|
| **每任务选优周期上限** | 仅 `every-step` 档生效:单个任务最多购买的过程选优周期数(1–32,默认 4)。它是独立于路由额度与最终验收额度的第三本账;但每个过程周期仍消耗一次「每任务/每会话最多路由」额度(会话默认上限 8),因此每步选优的实际步数上限取两者较小者。周期计数同时保存在路由内存计数与话题侧车 `verifier/process-selection-v1.json`(取两者较大值),插件重载后不失忆 |
|
|
286
288
|
| **过程选优候选数** | 默认 **2**(原回复 + 1 份备选,判一对)。设为 3 或 4 时按 `[原回复, 备选 1, …]` 跑锦标赛,判官调用随之上升(N=3 约 5 对,约为单对的 5 倍),预算从同一份任务/会话模型调用额度里出——预约用引擎自己的 `estimateRoutedCalls()` 估算对局数,不会和实际跑的对数脱节。完全相同的候选不会送给判官:引擎先去重,全部相同则 0 调用并回放原回复。**不采纳**上游「多数票直接返回未评判候选」的语义。 |
|
|
287
289
|
| **进度完成阈值** | `track` **最新**检查点低于该值时 steering 要求继续执行,否则改为"准备最终交付证据;最终会话验收是强制的"。默认 `0.684` = 判官标尺上 **N("偏成功")的起点**(进度分 = 字母序号 ÷ 19,N = 13/19 ≈ 68.4%),与提示词里"N–S = leans YES"的档位描述自洽;旧的 0.8 要求 Q 以上,等于"说得再肯定也不算过"。更早的检查点只随消息展示:每个任务的第一个检查点都是"刚写完计划",用"任一检查点低于阈值"判定会让继续分支恒真(实测 5/5)。**最新检查点达到阈值时还会置一个"下个边界直接进最终验收"的偏好**:steering 文案不变,但下一停止边界会跳过新的自动路由(不再为同一句提示重复购买 `track`);偏好由最终验收的预约消耗,验收失败后路由自动恢复 |
|
|
@@ -304,7 +306,7 @@ flowchart LR
|
|
|
304
306
|
| **裁判标签 (Label)** | 主裁判在工具返回的 `judges[].label` 中显示的名称,留空则使用模型名 |
|
|
305
307
|
| **附加裁判 (Extra judges)** | 在主裁判之外追加最多 4 个裁判模型(各自可指定 Provider / Model / 推理强度 / 显示名),同一 Provider + Model 不允许重复;留空即为单裁判模式,行为与旧版本逐位一致 |
|
|
306
308
|
| **推理强度 (Reasoning Effort)** | 使用 Adapter 为该模型声明的思考强度,或保留模型默认值 |
|
|
307
|
-
| **最大输出 Token** | 单次裁判请求的输出 Token
|
|
309
|
+
| **最大输出 Token** | 单次裁判请求的输出 Token 上限,默认 **65536**(对齐上游 v0.3.0 的判官输出预算)。注意这是**上限**而非预留:判官把预算全部用在推理上、正文一个字没输出时,直连概率通道会报"推理吃满了 N token 预算"并提示调大该值或调低推理强度,而**不会**把这条路由误记成"不支持 logprobs" |
|
|
308
310
|
| **判官温度 (Temperature)** | 判官采样的温度,范围 0–2,默认 **0.2**(旧版固定为 1)。自动路由默认只跑 1 轮、最终验收默认 2 轮(偶数轮交换 A/B 位置),温度越低判决越可复现;部分推理模型只接受默认温度,遇到适配器报错时改回 1 |
|
|
309
311
|
| **最大并发数** | 所有 Verifier 调用共享的全局并发请求限制 |
|
|
310
312
|
| **最大重试次数** | 遇到偶发网络或服务短暂故障时的额外重试次数 |
|
package/icon.svg
ADDED
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
<svg xmlns="http://www.w3.org/2000/svg" viewBox="0 0 48 48" width="48" height="48" role="img" aria-label="DSH LLM Verifier">
|
|
2
|
+
<title>DSH LLM Verifier</title>
|
|
3
|
+
<defs>
|
|
4
|
+
<linearGradient id="llm-verifier-icon" x1="3" y1="2" x2="45" y2="46" gradientUnits="userSpaceOnUse">
|
|
5
|
+
<stop stop-color="#8A63FF"/>
|
|
6
|
+
<stop offset="0.5" stop-color="#4F7BFF"/>
|
|
7
|
+
<stop offset="1" stop-color="#2FC5C9"/>
|
|
8
|
+
</linearGradient>
|
|
9
|
+
</defs>
|
|
10
|
+
<rect width="48" height="48" rx="15" fill="url(#llm-verifier-icon)"/>
|
|
11
|
+
<path d="M24 8.8 36.8 13.8v10.4c0 7.2-5.4 12.8-12.8 15.2C16.6 37 11.2 31.4 11.2 24.2V13.8L24 8.8Z" fill="#FFFFFF" fill-opacity="0.94"/>
|
|
12
|
+
<path d="M17.6 24.6 22 29l8.6-9.6" fill="none" stroke="url(#llm-verifier-icon)" stroke-width="4.2" stroke-linecap="round" stroke-linejoin="round"/>
|
|
13
|
+
</svg>
|
|
@@ -23,6 +23,47 @@ var TopLogprobsRouteError = class extends TopLogprobsUnsupportedError {
|
|
|
23
23
|
this.name = "TopLogprobsRouteError";
|
|
24
24
|
}
|
|
25
25
|
};
|
|
26
|
+
/**
|
|
27
|
+
* A deterministic, non-transient failure of the direct transport: the provider DID answer, but the
|
|
28
|
+
* answer cannot be scored and no second attempt can change that. Upstream v0.3.0 splits exactly this
|
|
29
|
+
* condition out of the capability path for the same reason.
|
|
30
|
+
*
|
|
31
|
+
* Deliberately NOT a {@link TopLogprobsUnsupportedError}: `callAutomatic` catches that type to
|
|
32
|
+
* `markUnsupported(provider, model)`, so a route whose reasoning merely ate the output budget would be
|
|
33
|
+
* remembered for 24h as a model that cannot do logprobs at all — a capability-cache lie that outlives
|
|
34
|
+
* the misconfiguration that caused it. It is also excluded from the shared retry heuristic, because
|
|
35
|
+
* the message quotes the budget figure and that figure can read as a transient HTTP 5xx code while the
|
|
36
|
+
* failure itself is perfectly reproducible.
|
|
37
|
+
*/
|
|
38
|
+
var TopLogprobsBudgetError = class extends Error {
|
|
39
|
+
constructor(message) {
|
|
40
|
+
super(message);
|
|
41
|
+
this.name = "TopLogprobsBudgetError";
|
|
42
|
+
}
|
|
43
|
+
};
|
|
44
|
+
/**
|
|
45
|
+
* Classify a 200 response that carries no token logprob row.
|
|
46
|
+
*
|
|
47
|
+
* Different causes hide behind "no rows", and reporting them as one is how a configuration mistake
|
|
48
|
+
* turns into a 24h capability-cache entry:
|
|
49
|
+
*
|
|
50
|
+
* 1. `finish_reason: 'length'` — reasoning consumed the whole output budget before the answer
|
|
51
|
+
* started. Raising `maxTokens` or lowering the reasoning effort fixes it, so this surfaces as
|
|
52
|
+
* {@link TopLogprobsBudgetError} and never reaches the capability memory.
|
|
53
|
+
* 2. Every other finish reason — the route keeps its capability-probe meaning
|
|
54
|
+
* ({@link TopLogprobsUnsupportedError}): a provider that answers while ignoring `logprobs` is
|
|
55
|
+
* exactly the answer the probe exists to collect.
|
|
56
|
+
*
|
|
57
|
+
* The finish reason is recorded either way, so "this model produced no answer" stays readable apart
|
|
58
|
+
* from "this model cannot report logprobs".
|
|
59
|
+
* @param finishReason - `choices[0].finish_reason` as a trimmed string, when the provider sent one.
|
|
60
|
+
* @param maxTokens - the output budget actually requested, quoted back in the budget message.
|
|
61
|
+
* @returns The error to throw for a response that carries no logprob rows.
|
|
62
|
+
*/
|
|
63
|
+
function noLogprobsError(finishReason, maxTokens) {
|
|
64
|
+
if (finishReason === "length") return new TopLogprobsBudgetError(`llm-verifier: top_logprobs returned no answer logprobs (finish_reason='length', reasoning consumed the ${maxTokens}-token budget); raise the judge maxTokens or lower the reasoning effort`);
|
|
65
|
+
return new TopLogprobsUnsupportedError("provider returned no token logprobs (finish_reason=" + (finishReason ?? "unknown") + ", no answer emitted)");
|
|
66
|
+
}
|
|
26
67
|
function object(value) {
|
|
27
68
|
return typeof value === "object" && value !== null && !Array.isArray(value) ? value : void 0;
|
|
28
69
|
}
|
|
@@ -151,7 +192,7 @@ async function callTopLogprobs(route, model, prompt, maxTokens, reasoningEffort,
|
|
|
151
192
|
const answer = typeof message?.content === "string" ? message.content : "";
|
|
152
193
|
const logprobs = object(choice?.logprobs);
|
|
153
194
|
const rows = Array.isArray(logprobs?.content) ? logprobs.content : [];
|
|
154
|
-
if (!rows.length) throw
|
|
195
|
+
if (!rows.length) throw noLogprobsError(text(choice?.finish_reason), maxTokens);
|
|
155
196
|
const tokens = [];
|
|
156
197
|
const positions = [];
|
|
157
198
|
for (const rawRow of rows) {
|
|
@@ -427,7 +468,16 @@ function usage(attempts, value = {}) {
|
|
|
427
468
|
};
|
|
428
469
|
}
|
|
429
470
|
/** Transient failures worth another attempt; anything else fails fast. */
|
|
430
|
-
|
|
471
|
+
/**
|
|
472
|
+
* Transient-failure heuristic applied to a thrown message.
|
|
473
|
+
*
|
|
474
|
+
* The `5\d\d` arm matches a bare HTTP 5xx status and is therefore ANCHORED to digit boundaries:
|
|
475
|
+
* an unanchored `5\d\d` matches any three-digit run starting with 5, so an ordinary figure like a
|
|
476
|
+
* 65536-token output budget hit "553" and made a perfectly reproducible configuration failure look
|
|
477
|
+
* retryable — four full-budget judge calls before reporting the same error. "HTTP 503" still
|
|
478
|
+
* matches (the space is a boundary); "65536" no longer does (both sides of "553" are digits).
|
|
479
|
+
*/
|
|
480
|
+
const RETRYABLE_MESSAGE = /rate|quota|timeout|timed out|temporar|network|fetch|socket|(?:^|[^0-9])5\d\d(?:[^0-9]|$)/i;
|
|
431
481
|
/**
|
|
432
482
|
* Run one logical verifier request under the configured timeout, retrying
|
|
433
483
|
* transient failures with exponential backoff.
|
|
@@ -511,7 +561,7 @@ async function retrying(config, signal, run) {
|
|
|
511
561
|
} catch (error) {
|
|
512
562
|
const billed = partialUsage(error);
|
|
513
563
|
if (signal?.aborted) cancel(signal.reason ?? error, billed, unknownFailure || billed === void 0);
|
|
514
|
-
const retryable = timedOut || error instanceof Error && RETRYABLE_MESSAGE.test(error.message);
|
|
564
|
+
const retryable = timedOut || error instanceof Error && !(error instanceof TopLogprobsBudgetError) && RETRYABLE_MESSAGE.test(error.message);
|
|
515
565
|
if (attempt > config.maxRetries || !retryable) {
|
|
516
566
|
if (billed === void 0) unknownFailure = true;
|
|
517
567
|
const failure = typeof error === "object" && error !== null ? error : new Error(String(error));
|
|
@@ -785,4 +835,4 @@ function emptyUsage() {
|
|
|
785
835
|
//#endregion
|
|
786
836
|
export { attachUsage as a, emptyUsage as c, generationClient as d, partialUsage as f, resolveCapabilityFile as g, TopLogprobCapabilityCache as h, addUsage as i, fallbackDeepFreeze as l, requestAttempts as m, GENERATION_TEMPERATURE as n, callVerifier as o, predictScoringChannel as p, RequestLimiter as r, callVerifierText as s, GENERATION_MAX_TOKENS as t, generateCandidate as u };
|
|
787
837
|
|
|
788
|
-
//# sourceMappingURL=caller-
|
|
838
|
+
//# sourceMappingURL=caller-CPbFWOz7.js.map
|
|
@@ -0,0 +1 @@
|
|
|
1
|
+
{"version":3,"file":"caller-CPbFWOz7.js","names":[],"sources":["../src/top-logprobs.ts","../src/caller.ts"],"sourcesContent":["import type { Context } from '@deepseek-ai/cordis'\nimport { credentialRef } from '@deepseek-ai/dsh-credentials'\nimport { mkdir, readFile, rename, unlink, writeFile } from 'node:fs/promises'\nimport { dirname, isAbsolute, join, resolve } from 'node:path'\nimport type { CompletionLogprobs, TokenAlternative } from './core.ts'\nimport type { UsageStats, VerifierImage } from './caller.ts'\n\n/**\n * The two scoring channels are the logprob expectation (this file) and the explicit tag reply\n * (caller.ts). Upstream has a THIRD one worth knowing about before inventing anything new:\n * when an open-model server does not emit the score tags at all, it prefills the tag itself and\n * constrains the next token to the twenty scale letters, then reads that token's distribution\n * (llm-as-a-verifier `_score_tags_by_prefill`: vLLM/SGLang `continue_final_message` plus\n * `structured_outputs`). That turns ANY OpenAI-compatible server into a distribution-scoring\n * judge, which is strictly more informative than our explicit-tag channel's single sampled letter\n * — it is why upstream can run K=1 and we average repeats instead.\n *\n * NOT implemented here, deliberately: the direct transport below speaks to whichever\n * OpenAI-compatible endpoint a provider declares, and the prefill/grammar parameters are\n * vLLM/SGLang extensions that a normal endpoint rejects. Two preconditions before adding it:\n * (1) the DSH stream adapter exposes prefill or a constrained-decoding option, or the route is\n * known to be a local inference server; (2) the capability probe can detect support at runtime and\n * degrade to explicit tags — NEVER keyed off a provider or model name (AGENTS.md rule 7). If it\n * lands, it belongs in `callTopLogprobs` as an opt-in third mode, and `autoTrackRepeats` could\n * then drop.\n */\nexport interface TopLogprobRoute {\n baseURL: string\n apiKey?: string\n headers?: Record<string, string>\n deepSeekThinking: boolean\n}\nexport interface TopLogprobCompletion extends CompletionLogprobs { usage: UsageStats; scoringMode: 'top-logprobs' }\nexport class TopLogprobsUnsupportedError extends Error { constructor(message: string) { super(message); this.name = 'TopLogprobsUnsupportedError' } }\n/**\n * A provider-level rejection of the direct transport that is not a logprobs\n * capability answer (bad request, auth, quota, malformed body). It downgrades\n * this topic to the DSH stream instead of failing the whole verification.\n */\nexport class TopLogprobsRouteError extends TopLogprobsUnsupportedError { constructor(message: string, readonly status?: number) { super(message); this.name = 'TopLogprobsRouteError' } }\n/**\n * A deterministic, non-transient failure of the direct transport: the provider DID answer, but the\n * answer cannot be scored and no second attempt can change that. Upstream v0.3.0 splits exactly this\n * condition out of the capability path for the same reason.\n *\n * Deliberately NOT a {@link TopLogprobsUnsupportedError}: `callAutomatic` catches that type to\n * `markUnsupported(provider, model)`, so a route whose reasoning merely ate the output budget would be\n * remembered for 24h as a model that cannot do logprobs at all — a capability-cache lie that outlives\n * the misconfiguration that caused it. It is also excluded from the shared retry heuristic, because\n * the message quotes the budget figure and that figure can read as a transient HTTP 5xx code while the\n * failure itself is perfectly reproducible.\n */\nexport class TopLogprobsBudgetError extends Error { constructor(message: string) { super(message); this.name = 'TopLogprobsBudgetError' } }\n\n/**\n * Classify a 200 response that carries no token logprob row.\n *\n * Different causes hide behind \"no rows\", and reporting them as one is how a configuration mistake\n * turns into a 24h capability-cache entry:\n *\n * 1. `finish_reason: 'length'` — reasoning consumed the whole output budget before the answer\n * started. Raising `maxTokens` or lowering the reasoning effort fixes it, so this surfaces as\n * {@link TopLogprobsBudgetError} and never reaches the capability memory.\n * 2. Every other finish reason — the route keeps its capability-probe meaning\n * ({@link TopLogprobsUnsupportedError}): a provider that answers while ignoring `logprobs` is\n * exactly the answer the probe exists to collect.\n *\n * The finish reason is recorded either way, so \"this model produced no answer\" stays readable apart\n * from \"this model cannot report logprobs\".\n * @param finishReason - `choices[0].finish_reason` as a trimmed string, when the provider sent one.\n * @param maxTokens - the output budget actually requested, quoted back in the budget message.\n * @returns The error to throw for a response that carries no logprob rows.\n */\nfunction noLogprobsError(finishReason: string | undefined, maxTokens: number): Error {\n if (finishReason === 'length') return new TopLogprobsBudgetError(`llm-verifier: top_logprobs returned no answer logprobs (finish_reason='length', reasoning consumed the ${maxTokens}-token budget); raise the judge maxTokens or lower the reasoning effort`)\n return new TopLogprobsUnsupportedError('provider returned no token logprobs (finish_reason=' + (finishReason ?? 'unknown') + ', no answer emitted)')\n}\n\nfunction object(value: unknown): Record<string, unknown> | undefined { return typeof value === 'object' && value !== null && !Array.isArray(value) ? value as Record<string, unknown> : undefined }\nfunction text(value: unknown): string | undefined { return typeof value === 'string' && value.trim() ? value.trim() : undefined }\nfunction endpoint(baseURL: string): string { return baseURL.replace(/\\/+$/, '') + '/chat/completions' }\nfunction dataUrl(image: VerifierImage): string { return 'data:' + image.mediaType + ';base64,' + Buffer.from(image.data.buffer, image.data.byteOffset, image.data.byteLength).toString('base64') }\n\nasync function credential(ctx: Context, name: string | undefined): Promise<string | undefined> {\n if (!name) return undefined\n const provider = ctx.get('credentials')\n return (await provider?.resolve(credentialRef(name)))?.value\n}\n\n/**\n * Read one host settings namespace as a plain object.\n *\n * DSH 0.1.1-0.1.6 answered this with `settings.get(ns)` on the register-based seam.\n * DSH 0.1.7 replaced that seam with `SettingsForms`, which has no `get` method at\n * all: the only public read is `describe()`, whose descriptors carry each\n * namespace's projected live value. Probing both keeps the direct logprob route\n * working across the whole supported line, and a seam that throws degrades to the\n * explicit-tag channel instead of failing the verification outright.\n * @param ctx - Plugin context carrying the optional settings service.\n * @param ns - Host namespace id, e.g. `llm-deepseek`.\n * @returns The namespace value, or `undefined` when it cannot be read.\n */\nfunction settingsNamespace(ctx: Context, ns: string): Record<string, unknown> | undefined {\n const settings = ctx.get('settings') as unknown as { get?: unknown; describe?: unknown } | undefined\n if (!settings) return undefined\n try {\n if (typeof settings.get === 'function') return object((settings.get as (ns: string) => unknown)(ns))\n if (typeof settings.describe !== 'function') return undefined\n const rows: unknown = (settings.describe as () => unknown)()\n if (!Array.isArray(rows)) return undefined\n const row = rows.map(object).find(candidate => candidate?.ns === ns)\n return object(row?.value)\n } catch {\n return undefined\n }\n}\n\nexport async function resolveTopLogprobRoute(ctx: Context, provider: string): Promise<TopLogprobRoute | undefined> {\n if (provider === 'deepseek-official') {\n const value = settingsNamespace(ctx, 'llm-deepseek') ?? {}\n const apiKeyEnv = text(value.apiKeyEnv) ?? 'DEEPSEEK_API_KEY'\n const apiKey = await credential(ctx, apiKeyEnv)\n if (!apiKey) return undefined\n return { baseURL: text(value.baseURL) ?? 'https://api.deepseek.com', apiKey, deepSeekThinking: true }\n }\n const root = settingsNamespace(ctx, 'llm-pi-ai')\n if (root === undefined) return undefined\n const profiles = object(root?.providers)\n const profile = object(profiles?.[provider])\n // Only explicitly OpenAI-compatible profiles are safe to serialize directly.\n // Other DSH adapters keep their private protocol and use the explicit-tag fallback.\n if (!profile || profile.api !== 'openai-completions') return undefined\n const baseURL = text(profile.baseURL)\n if (!baseURL || !/^https:\\/\\//i.test(baseURL)) return undefined\n const apiKey = await credential(ctx, text(profile.apiKeyEnv))\n const rawHeaders = object(profile.headers)\n const headers = rawHeaders === undefined ? undefined : Object.fromEntries(Object.entries(rawHeaders).filter((entry): entry is [string, string] => typeof entry[1] === 'string'))\n return { baseURL, ...(apiKey ? { apiKey } : {}), ...(headers ? { headers } : {}), deepSeekThinking: false }\n}\n\nexport async function callTopLogprobs(route: TopLogprobRoute, model: string, prompt: string, maxTokens: number, reasoningEffort: string | undefined, signal?: AbortSignal, images?: readonly VerifierImage[], attempt = 1, temperature = 0.2): Promise<TopLogprobCompletion> {\n const content: string | Record<string, unknown>[] = images?.length ? [{ type: 'text', text: prompt }, ...images.map(image => ({ type: 'image_url', image_url: { url: dataUrl(image) } }))] : prompt\n const thinking = route.deepSeekThinking && reasoningEffort ? reasoningEffort === 'off' ? { thinking: { type: 'disabled' } } : { thinking: { type: 'enabled' }, reasoning_effort: reasoningEffort } : {}\n const response = await fetch(endpoint(route.baseURL), {\n method: 'POST', redirect: 'error', signal,\n headers: { 'content-type': 'application/json', ...(route.apiKey ? { authorization: 'Bearer ' + route.apiKey } : {}), ...route.headers },\n body: JSON.stringify({ model, messages: [{ role: 'user', content }], max_tokens: maxTokens, temperature, logprobs: true, top_logprobs: 20, ...thinking }),\n })\n const raw = await response.text()\n if (!response.ok) {\n const excerpt = raw.slice(0, 1000)\n const message = 'llm-verifier: top_logprobs request failed with HTTP ' + response.status + ': ' + excerpt\n if ([400, 404, 405, 415, 422].includes(response.status) && /logprob|top_logprobs|unsupported|unknown (?:field|parameter)|unrecognized (?:field|parameter)|not support/i.test(excerpt)) throw new TopLogprobsUnsupportedError('provider rejected top_logprobs: HTTP ' + response.status + ' ' + excerpt)\n // Throttling and server faults stay generic so the shared retry wrapper can\n // attempt again; every other rejection downgrades to the DSH stream.\n if (response.status === 408 || response.status === 409 || response.status === 429 || response.status >= 500) throw new Error(message)\n throw new TopLogprobsRouteError(message, response.status)\n }\n let body: Record<string, unknown>\n try { body = object(JSON.parse(raw)) ?? {} } catch { throw new TopLogprobsRouteError('llm-verifier: top_logprobs endpoint returned invalid JSON', response.status) }\n const choices = Array.isArray(body.choices) ? body.choices : []\n const choice = object(choices[0])\n const message = object(choice?.message)\n const answer = typeof message?.content === 'string' ? message.content : ''\n const logprobs = object(choice?.logprobs)\n const rows = Array.isArray(logprobs?.content) ? logprobs.content : []\n if (!rows.length) throw noLogprobsError(text(choice?.finish_reason), maxTokens)\n const tokens: string[] = []\n const positions: TokenAlternative[][] = []\n for (const rawRow of rows) {\n const row = object(rawRow) ?? {}\n const token = typeof row.token === 'string' ? row.token : ''\n tokens.push(token)\n const top = Array.isArray(row.top_logprobs) ? row.top_logprobs : []\n const alternatives = top.flatMap(value => { const item = object(value); return item && typeof item.token === 'string' && typeof item.logprob === 'number' ? [{ token: item.token, logprob: item.logprob }] : [] })\n if (!alternatives.length && typeof row.logprob === 'number') alternatives.push({ token, logprob: row.logprob })\n positions.push(alternatives)\n }\n const rawUsage = object(body.usage) ?? {}\n const promptDetails = object(rawUsage.prompt_tokens_details) ?? {}\n const completionDetails = object(rawUsage.completion_tokens_details) ?? {}\n const cached = Number(rawUsage.prompt_cache_hit_tokens ?? promptDetails.cached_tokens ?? 0) || 0\n const input = Number(rawUsage.prompt_tokens ?? 0) || 0\n return { text: answer, tokens, positions, scoringMode: 'top-logprobs', usage: { calls: 1, attempts: attempt, retries: attempt - 1, inputTokens: Math.max(0, input - cached), cachedInputTokens: cached, outputTokens: Number(rawUsage.completion_tokens ?? 0) || 0, reasoningTokens: Number(completionDetails.reasoning_tokens ?? 0) || 0 } }\n}\n\n/** Marks older than this are dropped on hydration so a provider that later gains logprobs support is re-probed. */\nexport const CAPABILITY_TTL_MS = 24 * 60 * 60 * 1000\n\ninterface CapabilityDocument { version: 1; entries: Record<string, number> }\n\n/** Resolves the capability memory file beside the score cache inside the topic verifier directory. */\nexport function resolveCapabilityFile(cacheDir: string, cwd = process.cwd()): string {\n const root = isAbsolute(cacheDir) ? cacheDir : resolve(cwd, cacheDir)\n return join(root, 'capabilities-v1.json')\n}\n\n/** Replace failures that are a transient lock rather than a verdict about the capability file directory. */\nconst TRANSIENT_REPLACE_CODES = new Set(['EPERM', 'EACCES', 'EBUSY'])\n\nlet replaceOrdinal = 0\n\nasync function replaceFile(temporary: string, target: string): Promise<void> {\n for (let attempt = 1; ; attempt += 1) {\n try {\n await rename(temporary, target)\n return\n } catch (error) {\n const code = (error as NodeJS.ErrnoException).code\n if (attempt >= 5 || code === undefined || !TRANSIENT_REPLACE_CODES.has(code)) throw error\n await new Promise(resolve => setTimeout(resolve, attempt * 10))\n }\n }\n}\n\nexport class TopLogprobCapabilityCache {\n private readonly unsupported = new Map<string, number>()\n private loaded = false\n private hydrating: Promise<void> | undefined\n private writing: Promise<void> = Promise.resolve()\n\n constructor(private readonly file?: string, private readonly now: () => number = Date.now) {}\n\n /** Expired marks are dropped so a provider that later gains logprobs support is re-probed. */\n isUnsupported(provider: string, model: string): boolean {\n const key = provider + '\\0' + model\n const markedAt = this.unsupported.get(key)\n if (markedAt === undefined) return false\n if (this.now() - markedAt > CAPABILITY_TTL_MS) { this.unsupported.delete(key); return false }\n return true\n }\n\n /** Hydrates persisted marks once; in-process marks always win over file contents. */\n async ensureLoaded(): Promise<void> {\n if (this.loaded || this.file === undefined) return\n this.hydrating ??= (async () => {\n try {\n const document = JSON.parse(await readFile(this.file!, 'utf8')) as CapabilityDocument\n const entries = document !== null && typeof document === 'object' && document.version === 1 && typeof document.entries === 'object' && document.entries !== null ? document.entries : {}\n const now = this.now()\n for (const [key, markedAt] of Object.entries(entries)) {\n if (typeof markedAt !== 'number' || !Number.isFinite(markedAt) || markedAt < 0 || now - markedAt > CAPABILITY_TTL_MS) continue\n // Max-merge: never let an older persisted stamp clobber a fresher in-process probe.\n const existing = this.unsupported.get(key)\n if (existing === undefined || existing < markedAt) this.unsupported.set(key, markedAt)\n }\n } catch { /* a missing or unreadable file simply starts empty */ }\n this.loaded = true\n })()\n await this.hydrating\n }\n\n markUnsupported(provider: string, model: string): void {\n this.unsupported.set(provider + '\\0' + model, this.now())\n this.persist()\n }\n\n /**\n * Forget one provider/model mark so the next call re-probes instead of replaying a cached answer.\n *\n * The judge probe exists to answer \"which scoring channel is this judge on\". Replaying a mark\n * written up to {@link CAPABILITY_TTL_MS} ago — possibly under a different provider\n * configuration — would answer a different, stale question. The removal is persisted, so a host\n * restart cannot resurrect the old mark.\n *\n * The delete is repeated INSIDE the serialized write: hydration max-merges the file into memory,\n * so deleting only beforehand would let the very write we schedule put the entry back.\n * @param provider - provider id the mark belongs to.\n * @param model - model id the mark belongs to.\n */\n forget(provider: string, model: string): void {\n const key = provider + '\\0' + model\n this.unsupported.delete(key)\n if (this.file === undefined) return\n const next = this.writing\n .catch(() => {})\n .then(() => this.ensureLoaded())\n .then(() => { this.unsupported.delete(key); return this.writeDocument() })\n this.writing = next\n next.catch(() => { /* capability memory is best-effort */ })\n }\n\n /** Serialize behind hydration so an early mark never clobbers not-yet-loaded entries. */\n private persist(): void {\n if (this.file === undefined) return\n const next = this.writing\n .catch(() => {})\n .then(() => this.ensureLoaded())\n .then(() => this.writeDocument())\n this.writing = next\n next.catch(() => { /* capability memory is best-effort; the next mark rewrites the file */ })\n }\n\n private async writeDocument(): Promise<void> {\n const snapshot: CapabilityDocument = { version: 1, entries: Object.fromEntries(this.unsupported) }\n await mkdir(dirname(this.file!), { recursive: true })\n const temporary = this.file! + '.tmp-' + process.pid + '-' + (++replaceOrdinal)\n await writeFile(temporary, JSON.stringify(snapshot), 'utf8')\n try { await replaceFile(temporary, this.file!) } catch (error) { await unlink(temporary).catch(() => {}); throw error }\n }\n\n /** Resolves once the trailing persistence attempt settles; exposed for tests. */\n flush(): Promise<void> { return this.writing }\n}\n","import { BlockAssembler, ReasoningEffortId, createUserMessage, type ContentBlock, type FinishReason, type LlmRuntime } from '@deepseek-ai/dsh-llm'\nimport * as LlmModule from '@deepseek-ai/dsh-llm'\n// Supplies the `'llm-verifier'` MessageSourceMap member named by the judge request below.\nimport type {} from './message-source.ts'\n\n/**\n * Deep-freeze a call-options graph, leaving any live AbortSignal mutable.\n *\n * The host re-exports its own `deepFreeze` from `@deepseek-ai/dsh-llm` on the 0.1.1 line; the\n * 0.1.5 line moved the helper to `@deepseek-ai/dsh-util-values` and no longer re-exports it, so\n * the fallback below is what actually runs there. Both implementations must carry the same rule\n * (\"leaving live AbortSignal objects mutable\" — the host's own wording), because freezing a\n * signal breaks the transport twice over: Node >= 26.5 initializes the signal's internal event\n * map lazily, so the first `addEventListener` throws \"Cannot assign to read only property\n * 'Symbol(kEvents)'\", and on every Node version the attempt timeout's `controller.abort()` throws\n * the same way on 'Symbol(kAborted)'. The signal belongs to this call's retry loop and is still\n * unsubscribed when the options are frozen, which is what makes the ordering fatal.\n */\nfunction deepFreeze<T>(value: T): T {\n const mod = LlmModule as unknown as Record<string, unknown>\n if (typeof mod.deepFreeze === 'function') {\n return (mod.deepFreeze as Function)(value)\n }\n return fallbackDeepFreeze(value)\n}\n\n/**\n * The graph walk used when the host exports no `deepFreeze`; exported for tests.\n * @param value - value to freeze.\n * @returns The same value, with every reachable enumerable child frozen except live AbortSignals.\n */\nexport function fallbackDeepFreeze<T>(value: T): T {\n // See deepFreeze(): a frozen signal fails on the transport's first subscription.\n if (value instanceof AbortSignal) return value\n if (value && typeof value === 'object' && !Object.isFrozen(value)) {\n Object.freeze(value)\n for (const key of Object.keys(value)) {\n fallbackDeepFreeze((value as Record<string, unknown>)[key])\n }\n }\n return value\n}\nimport type { AttachmentStore } from '@deepseek-ai/dsh-attachment'\nimport type { CompletionLogprobs } from './core.ts'\nimport { TopLogprobCapabilityCache, TopLogprobsBudgetError, TopLogprobsUnsupportedError, callTopLogprobs, resolveTopLogprobRoute } from './top-logprobs.ts'\nimport type { Context } from '@deepseek-ai/cordis'\n\nexport interface VerifierClientConfig {\n ctx: Context\n llm: LlmRuntime\n attachments: AttachmentStore\n topLogprobCapabilities: TopLogprobCapabilityCache\n provider: string\n model: string\n /** User-facing judge name for tool output; cosmetic only, never part of the scoring cache identity. */\n label?: string\n reasoningEffort?: string\n maxTokens: number\n temperature: number\n timeoutMs: number\n maxRetries: number\n retryBaseDelayMs: number\n limiter?: RequestLimiter\n}\n\nexport interface VerifierImage {\n data: Uint8Array\n mediaType: 'image/png' | 'image/jpeg' | 'image/webp' | 'image/gif'\n}\n\nexport interface UsageStats {\n calls: number\n attempts: number\n retries: number\n inputTokens: number\n cachedInputTokens: number\n outputTokens: number\n reasoningTokens: number\n /**\n * At least one attempt's usage is UNKNOWN (an earlier attempt failed and was retried, or a\n * response came back unusable). The token counts are a floor, never a confident total.\n */\n usageIncomplete?: boolean\n}\n\nexport type ScoringMode = 'top-logprobs' | 'explicit-tag'\nexport interface VerifierCompletion extends CompletionLogprobs {\n usage: UsageStats\n scoringMode: ScoringMode\n /**\n * The direct logprob transport was attempted and the provider rejected it, so this\n * answer came from the explicit-tag fallback. Diagnostics only: it never changes the\n * score, and it is recorded so a silent downgrade is visible in the statistics.\n */\n channelFallback?: boolean\n}\n\n/**\n * Where a finally-failed request records how many attempts it already spent.\n *\n * A failure that never returned usage leaves the tokens UNKNOWN, not zero. The attempt\n * count is the one fact the transport does know, so it is carried on the error itself\n * (the engine catches per-judge errors) instead of being discarded.\n */\nconst REQUEST_ATTEMPTS = Symbol('llm-verifier.requestAttempts')\n\n/** Attempts one failed verifier request already spent; 0 when the error carries none. */\nexport function requestAttempts(error: unknown): number {\n if (typeof error !== 'object' || error === null) return 0\n const value = (error as { [REQUEST_ATTEMPTS]?: unknown })[REQUEST_ATTEMPTS]\n return typeof value === 'number' && Number.isSafeInteger(value) && value > 0 ? value : 0\n}\n\n/**\n * Usage the transport DID return for a response that then turned out unusable.\n *\n * A parse failure or a truncated judge answer happened after the request was billed, so the\n * engine must not fold it into a zero-cost failure. The usage rides on the error next to the\n * attempt count; one error can carry both (a retried call whose final answer also failed).\n */\nconst PARTIAL_USAGE = Symbol('llm-verifier.partialUsage')\n\n/** Usage a completed-but-unusable response already cost; undefined when the error carries none. */\nexport function partialUsage<T = UsageStats>(error: unknown): T | undefined {\n if (typeof error !== 'object' || error === null) return undefined\n const value = (error as { [PARTIAL_USAGE]?: unknown })[PARTIAL_USAGE]\n return typeof value === 'object' && value !== null ? value as T : undefined\n}\n\n/** Attach the usage a completed-but-unusable response already cost to its error. */\nexport function attachUsage(error: unknown, usage: UsageStats): void {\n if (typeof error === 'object' && error !== null) (error as { [PARTIAL_USAGE]?: UsageStats })[PARTIAL_USAGE] = usage\n}\n\n/**\n * An error for a response that came back and was billed but produced no usable verdict.\n *\n * The usage is attached so the failure row reports known requests and tokens instead of zero.\n */\nfunction unusable(reason: string, attempt: number, raw: { inputTokens?: number; outputTokens?: number; cacheReadTokens?: number; cacheWriteTokens?: number; reasoningTokens?: number } | undefined): Error {\n const error = new Error('llm-verifier: ' + reason)\n attachUsage(error, usage(attempt, raw ?? {}))\n return error\n}\n\n/**\n * Describe why a finished stream is unusable.\n *\n * `FinishReason.failure` is the host's SERIALIZABLE facts record (`{message, code, status?,\n * providerRetryAfterMs?, requestId?}`), not the error the adapter threw: `normalizeLlmFailure`\n * detaches the facts at the adapter boundary, so the stack and the cause are already gone before\n * this plugin sees them. The machine code and the provider status ARE carried, and dropping them\n * left the board with one line and no way to tell an auth failure from a transport one.\n */\nfunction failureMessage(finish: FinishReason): string | undefined {\n if (finish.kind === 'error' || finish.kind === 'aborted') {\n const failure = finish.failure\n const facts = [\n failure.code,\n failure.status === undefined ? undefined : 'HTTP ' + failure.status,\n failure.requestId === undefined ? undefined : 'request ' + failure.requestId,\n ].filter((fact): fact is string => typeof fact === 'string' && fact.length > 0)\n return facts.length > 0 ? failure.message + ' [' + facts.join(', ') + ']' : failure.message\n }\n if (finish.kind === 'max-tokens') return 'verifier response reached max tokens before completing its answer'\n return undefined\n}\n\nasync function delay(ms: number, signal?: AbortSignal): Promise<void> {\n if (signal?.aborted) throw signal.reason\n await new Promise<void>((resolve, reject) => {\n const timer = setTimeout(resolve, ms)\n const abort = () => { clearTimeout(timer); reject(signal?.reason) }\n signal?.addEventListener('abort', abort, { once: true })\n })\n}\n\nfunction usage(attempts: number, value = {} as { inputTokens?: number; outputTokens?: number; cacheReadTokens?: number; cacheWriteTokens?: number; reasoningTokens?: number }): UsageStats {\n return { calls: 1, attempts, retries: attempts - 1, inputTokens: value.inputTokens ?? 0, cachedInputTokens: (value.cacheReadTokens ?? 0) + (value.cacheWriteTokens ?? 0), outputTokens: value.outputTokens ?? 0, reasoningTokens: value.reasoningTokens ?? 0 }\n}\n\n/** Transient failures worth another attempt; anything else fails fast. */\n/**\n * Transient-failure heuristic applied to a thrown message.\n *\n * The `5\\d\\d` arm matches a bare HTTP 5xx status and is therefore ANCHORED to digit boundaries:\n * an unanchored `5\\d\\d` matches any three-digit run starting with 5, so an ordinary figure like a\n * 65536-token output budget hit \"553\" and made a perfectly reproducible configuration failure look\n * retryable — four full-budget judge calls before reporting the same error. \"HTTP 503\" still\n * matches (the space is a boundary); \"65536\" no longer does (both sides of \"553\" are digits).\n */\nconst RETRYABLE_MESSAGE = /rate|quota|timeout|timed out|temporar|network|fetch|socket|(?:^|[^0-9])5\\d\\d(?:[^0-9]|$)/i\n\n/**\n * Run one logical verifier request under the configured timeout, retrying\n * transient failures with exponential backoff.\n *\n * Every channel goes through this wrapper — including the direct top_logprobs\n * transport, which used to bypass both the timeout and the retry budget.\n * @param config - resolved verifier client configuration.\n * @param signal - caller's abort signal (tool call or turn boundary).\n * @param run - one attempt; receives its own deadline signal and 1-based attempt number.\n * @returns The first successful completion.\n */\nfunction attachAttempts(error: unknown, attempt: number): void {\n if (typeof error === 'object' && error !== null) (error as { [REQUEST_ATTEMPTS]?: number })[REQUEST_ATTEMPTS] = attempt\n}\n\n/**\n * A fresh error for one aborted call.\n *\n * Never annotate and rethrow the shared `signal.reason`: several concurrent requests share it,\n * and the engine attaches its accumulator to the thrown error, so one object carries several\n * accumulators and they get merged repeatedly. The wrapper preserves the message and name.\n */\nfunction abortFailure(reason: unknown, attempt: number): Error {\n const source = reason instanceof Error ? reason : new Error(reason === undefined ? 'llm-verifier: request aborted' : String(reason))\n const error = new Error(source.message)\n error.name = source.name\n attachAttempts(error, attempt)\n return error\n}\n\n/** Token-only addition: call/attempt counters already describe the logical request. */\nfunction addTokens(target: Pick<UsageStats, 'inputTokens' | 'cachedInputTokens' | 'outputTokens' | 'reasoningTokens'>, source: Pick<UsageStats, 'inputTokens' | 'cachedInputTokens' | 'outputTokens' | 'reasoningTokens'>): void {\n target.inputTokens += source.inputTokens\n target.cachedInputTokens += source.cachedInputTokens\n target.outputTokens += source.outputTokens\n target.reasoningTokens += source.reasoningTokens\n}\n\nasync function retrying<T>(config: VerifierClientConfig, signal: AbortSignal | undefined, run: (signal: AbortSignal, attempt: number) => Promise<T>): Promise<T> {\n let attempt = 0\n // Tokens an earlier attempt already paid for but never delivered a usable result. They are\n // real cost, so they follow the chain into whichever exit happens (success, final failure or\n // cancellation) instead of being discarded when no attempt ultimately succeeds.\n const carried = { inputTokens: 0, cachedInputTokens: 0, outputTokens: 0, reasoningTokens: 0 }\n let unknownFailure = false\n // The usage the chain must be able to report: everything already spent plus this attempt's own\n // billed response. `unknown` (some attempt's tokens never came back) is what makes it a floor.\n const spent = (billed: UsageStats | undefined, unknown: boolean): UsageStats => ({\n calls: billed === undefined ? 0 : billed.calls,\n attempts: attempt,\n retries: Math.max(0, attempt - 1),\n inputTokens: carried.inputTokens + (billed?.inputTokens ?? 0),\n cachedInputTokens: carried.cachedInputTokens + (billed?.cachedInputTokens ?? 0),\n outputTokens: carried.outputTokens + (billed?.outputTokens ?? 0),\n reasoningTokens: carried.reasoningTokens + (billed?.reasoningTokens ?? 0),\n ...(unknown ? { usageIncomplete: true } : {}),\n })\n const cancel = (reason: unknown, billed: UsageStats | undefined, unknown: boolean): never => {\n const error = abortFailure(reason, attempt)\n attachUsage(error, spent(billed, unknown))\n throw error\n }\n while (true) {\n if (signal?.aborted) cancel(signal.reason, undefined, false)\n attempt += 1\n const controller = new AbortController()\n let timedOut = false\n const timeout = setTimeout(() => { timedOut = true; controller.abort(new Error('llm-verifier: request timed out')) }, config.timeoutMs)\n const abort = () => controller.abort(signal?.reason)\n signal?.addEventListener('abort', abort, { once: true })\n try {\n const value = await run(controller.signal, attempt)\n if (typeof value === 'object' && value !== null) {\n const usage = (value as { usage?: UsageStats }).usage\n if (usage !== undefined) {\n addTokens(usage, carried)\n // Only a failed attempt whose tokens never came back makes the total a floor.\n if (unknownFailure) usage.usageIncomplete = true\n }\n }\n return value\n } catch (error) {\n const billed = partialUsage<UsageStats>(error)\n if (signal?.aborted) cancel(signal.reason ?? error, billed, unknownFailure || billed === undefined)\n // A deadline abort is retryable even when the adapter wraps the reason in\n // its own error type with an unrelated message.\n // TopLogprobsBudgetError is excluded on purpose: it quotes the judge's output budget, and a\n // budget like 65536 contains \"553\" — which RETRYABLE_MESSAGE reads as an HTTP 5xx. The\n // underlying failure (reasoning ate the whole budget) reproduces identically on every attempt,\n // so retrying would only buy 4x the reasoning bill before reporting the same error.\n const retryable = timedOut || (error instanceof Error && !(error instanceof TopLogprobsBudgetError) && RETRYABLE_MESSAGE.test(error.message))\n if (attempt > config.maxRetries || !retryable) {\n // Every attempt of the chain goes onto the thrown error: the last response alone would\n // report a fraction of the requests and tokens the invocation really spent.\n if (billed === undefined) unknownFailure = true\n const failure = typeof error === 'object' && error !== null ? error : new Error(String(error))\n attachAttempts(failure, attempt)\n attachUsage(failure, spent(billed, unknownFailure || billed === undefined))\n throw failure\n }\n if (billed === undefined) unknownFailure = true\n else addTokens(carried, billed)\n try {\n await delay(Math.min(30000, config.retryBaseDelayMs * 2 ** (attempt - 1) * (0.8 + Math.random() * 0.4)), signal)\n } catch (waitError) {\n // Cancelled while waiting to retry: the attempts and the tokens already spent are real.\n cancel(signal?.reason ?? waitError, undefined, unknownFailure)\n }\n } finally {\n clearTimeout(timeout)\n signal?.removeEventListener('abort', abort)\n }\n }\n}\n\n/** One attachment per image object per process: retries must not duplicate attachments. */\nconst imageRefs = new WeakMap<VerifierImage, Promise<unknown>>()\n\n/**\n * One plain-text completion through the DSH stream.\n *\n * Shared by the explicit-tag judge path and by best-of-N generation, which differ in exactly one\n * place: a `max-tokens` finish. A truncated JUDGE answer is unusable — its verdict tags may never\n * have been emitted — so it stays a hard error. A truncated DRAFT is still a candidate the operator\n * already paid for, and the judge can see that the text stops mid-sentence and score it accordingly,\n * so generation receives the text plus a `truncated` flag.\n * @param config - resolved verifier client configuration.\n * @param prompt - the rendered prompt.\n * @param signal - the attempt's deadline signal.\n * @param images - attachments to send, if any.\n * @param attempt - 1-based attempt number for usage accounting.\n * @param tolerateTruncation - return the text of a max-tokens finish instead of throwing.\n * @returns The completion (with `truncated`) when it produced text.\n */\nasync function callTextCompletion(config: VerifierClientConfig, prompt: string, signal: AbortSignal | undefined, images: readonly VerifierImage[] | undefined, attempt: number, tolerateTruncation: boolean): Promise<VerifierCompletion & { truncated: boolean }> {\n const content: ContentBlock[] = [{ type: 'text', text: prompt }]\n for (const image of images ?? []) {\n let pending = imageRefs.get(image)\n if (pending === undefined) {\n const promise = Promise.resolve()\n .then(() => config.attachments.saveImage({ data: image.data, mediaType: image.mediaType }))\n .catch(error => {\n if (imageRefs.get(image) === pending) {\n imageRefs.delete(image)\n }\n throw error\n })\n pending = promise\n imageRefs.set(image, pending)\n }\n content.push({ type: 'image', attachment: await pending as never })\n }\n const messages = [createUserMessage({ content, source: { kind: 'llm-verifier' } })]\n const assembler = new BlockAssembler()\n const options = deepFreeze({\n provider: config.provider,\n model: config.model,\n ...(config.reasoningEffort ? { reasoningEffort: ReasoningEffortId(config.reasoningEffort) } : {}),\n messages,\n maxTokens: config.maxTokens,\n temperature: config.temperature,\n signal,\n })\n for await (const chunk of config.llm.stream(options)) assembler.push(chunk)\n let truncated = false\n if (assembler.finish.kind === 'max-tokens') {\n if (!tolerateTruncation) throw unusable('model call failed: ' + failureMessage(assembler.finish), attempt, assembler.usage)\n truncated = true\n } else {\n const failed = failureMessage(assembler.finish)\n if (failed !== undefined) throw unusable('model call failed: ' + failed, attempt, assembler.usage)\n }\n const text = assembler.blocks().filter((block): block is Extract<ContentBlock, { type: 'text' }> => block.type === 'text').map(block => block.text).join('')\n if (!text.trim()) throw unusable('selected DSH model produced no text', attempt, assembler.usage)\n // DSH adapters expose provider-neutral text/usage but not top-logprob candidates.\n // extractScore() therefore uses the model's explicit final A–T tags.\n return { text, tokens: [], positions: [], scoringMode: 'explicit-tag', usage: usage(attempt, assembler.usage), truncated }\n}\n\n/** Explicit-tag judge completion: fail-closed, so a truncated answer is an error and never a score. */\nasync function callExplicitTag(config: VerifierClientConfig, prompt: string, signal: AbortSignal | undefined, images: readonly VerifierImage[] | undefined, attempt: number): Promise<VerifierCompletion> {\n return callTextCompletion(config, prompt, signal, images, attempt, false)\n}\n\nexport class RequestLimiter {\n private active = 0\n private readonly queue: Array<() => void> = []\n constructor(readonly limit: number) { if (!Number.isSafeInteger(limit) || limit < 1) throw new Error('llm-verifier: request concurrency limit must be a positive integer') }\n async run<T>(operation: () => Promise<T>, signal?: AbortSignal): Promise<T> {\n if (signal?.aborted) throw signal.reason\n if (this.active >= this.limit) await new Promise<void>((resolve, reject) => {\n const enter = () => { signal?.removeEventListener('abort', abort); resolve() }\n const abort = () => { const index = this.queue.indexOf(enter); if (index >= 0) this.queue.splice(index, 1); reject(signal?.reason) }\n this.queue.push(enter); signal?.addEventListener('abort', abort, { once: true })\n })\n if (signal?.aborted) {\n this.queue.shift()?.()\n throw signal.reason\n }\n this.active += 1\n try { return await operation() } finally { this.active -= 1; this.queue.shift()?.() }\n }\n}\n\n/** Best-effort pre-call channel prediction for cache identity only; callAutomatic() stays the runtime source of truth. */\nexport async function predictScoringChannel(config: VerifierClientConfig): Promise<ScoringMode> {\n await config.topLogprobCapabilities.ensureLoaded()\n return config.topLogprobCapabilities.isUnsupported(config.provider, config.model) ? 'explicit-tag' : 'top-logprobs'\n}\n\nasync function callAutomatic(config: VerifierClientConfig, prompt: string, signal: AbortSignal | undefined, images: readonly VerifierImage[] | undefined, attempt: number): Promise<VerifierCompletion> {\n await config.topLogprobCapabilities.ensureLoaded()\n let fellBack = false\n if (!config.topLogprobCapabilities.isUnsupported(config.provider, config.model)) {\n const route = await resolveTopLogprobRoute(config.ctx, config.provider)\n if (route !== undefined) {\n try { return await callTopLogprobs(route, config.model, prompt, config.maxTokens, config.reasoningEffort, signal, images, attempt, config.temperature) }\n catch (error) {\n // Both a capability rejection and a non-retryable provider rejection of\n // the direct transport fall back to the DSH stream instead of failing\n // the whole verification.\n if (!(error instanceof TopLogprobsUnsupportedError)) throw error\n config.topLogprobCapabilities.markUnsupported(config.provider, config.model)\n // A runtime downgrade AFTER a live attempt, unlike the capability probe that\n // finds no route at all: only this one is a fallback worth counting.\n fellBack = true\n }\n } else config.topLogprobCapabilities.markUnsupported(config.provider, config.model)\n }\n const completion = await callExplicitTag(config, prompt, signal, images, attempt)\n return fellBack ? { ...completion, channelFallback: true } : completion\n}\n\nexport async function callVerifier(config: VerifierClientConfig, prompt: string, signal?: AbortSignal, images?: readonly VerifierImage[]): Promise<VerifierCompletion> {\n const invoke = () => retrying(config, signal, (attemptSignal, attempt) => callAutomatic(config, prompt, attemptSignal, images, attempt))\n return config.limiter === undefined ? invoke() : config.limiter.run(invoke, signal)\n}\n\n/**\n * Best-of-N generation sampling constants.\n *\n * Deliberately NOT the judge's temperature: best-of-N only pays off when the drafts actually\n * differ, and the judge's low default (0.2) would produce N near-copies — the tool would then\n * spend N times the money choosing between the same answer.\n */\nexport const GENERATION_TEMPERATURE = 1\n/**\n * Output ceiling for one draft.\n *\n * 4096 was a guess in the plan, and the first real end-to-end acceptance disproved it: a\n * \"function + 12 test cases + a note\" task truncated ALL THREE drafts, so the tool returned\n * nothing at all. The measured cause is reasoning tokens: on a trivial 2-draft run the session\n * model (deepseek-official/deepseek-flash) spent 16363 reasoning tokens out of 17254 output\n * tokens — roughly 8k of reasoning PER DRAFT, which eats any 4096 budget before the answer starts.\n *\n * 16384 is about 2x that measured per-draft spend, and it is also the largest ceiling that keeps\n * a pairwise judge prompt inside the plugin's own evidence limit: two drafts at 16384 tokens are\n * roughly 130k characters against a 240k explicit-evidence ceiling (EXPLICIT_MAX_TOTAL_CHARS).\n * `maxTokens` is a ceiling rather than a reservation, so a draft that needs less room costs\n * exactly what it did before. A task whose answer genuinely needs more is too large for this tool,\n * and its truncation is REPORTED instead of being silently returned.\n */\nexport const GENERATION_MAX_TOKENS = 16384\n\nexport interface GenerationTarget {\n provider: string\n model: string\n reasoningEffort?: string\n}\n\n/** One draft: plain text, never scored, plus whether it ran into the output ceiling. */\nexport interface GeneratedCandidate extends VerifierCompletion { truncated: boolean }\n\n/**\n * Re-point an existing verifier client at the model that should draft the candidates.\n *\n * Everything transport-shaped (context, llm runtime, attachments, capability memory, limiter,\n * timeout, retry budget) is inherited; only the route and the two generation-specific scalars\n * are replaced. `reasoningEffort` comes from the target and is never inherited from the base,\n * because the base is a judge: a leftover judge effort would silently draft with the wrong model\n * settings.\n * @param base - any configured verifier client (the primary judge is the cheapest source).\n * @param target - the model that writes the drafts.\n * @param maxTokens - output ceiling for this attempt.\n * @returns A client config safe to pass to {@link callGeneratedText}.\n */\nexport function generationClient(base: VerifierClientConfig, target: GenerationTarget, maxTokens = GENERATION_MAX_TOKENS): VerifierClientConfig {\n return {\n ctx: base.ctx,\n llm: base.llm,\n attachments: base.attachments,\n topLogprobCapabilities: base.topLogprobCapabilities,\n provider: target.provider,\n model: target.model,\n label: target.provider + '/' + target.model,\n ...(target.reasoningEffort === undefined ? {} : { reasoningEffort: target.reasoningEffort }),\n maxTokens,\n temperature: GENERATION_TEMPERATURE,\n timeoutMs: base.timeoutMs,\n maxRetries: base.maxRetries,\n retryBaseDelayMs: base.retryBaseDelayMs,\n ...(base.limiter === undefined ? {} : { limiter: base.limiter }),\n }\n}\n\n/**\n * One best-of-N draft.\n *\n * A named seam over the plain-text path: generation has no A–T contract and no scoring channel, and\n * the name keeps it distinct from the judge calls at every call site and in the decision snapshot.\n *\n * Hitting the output ceiling is deliberately NOT an error here. The operator already paid for those\n * tokens, and the judge can see for itself that the text stops mid-sentence. That is exactly why the\n * judge's fail-closed max-tokens handling was split out of this path: before the split, one long\n * task made the whole tool return zero candidates.\n * @param base - any configured verifier client (the primary judge is the cheapest source).\n * @param target - the model that writes the draft (normally the session model).\n * @param prompt - the rendered drafting prompt.\n * @param signal - caller's abort signal.\n * @returns The draft text, token usage, and whether it ran into the ceiling.\n */\nexport async function generateCandidate(base: VerifierClientConfig, target: GenerationTarget, prompt: string, signal?: AbortSignal): Promise<GeneratedCandidate> {\n return callGeneratedText(generationClient(base, target), prompt, signal)\n}\n\n/** Generation transport: the judge's timeout/retry/limiter wrappers, with truncation as a result. */\nasync function callGeneratedText(config: VerifierClientConfig, prompt: string, signal: AbortSignal | undefined): Promise<GeneratedCandidate> {\n const invoke = () => retrying(config, signal, (attemptSignal, attempt) => callTextCompletion(config, prompt, attemptSignal, undefined, attempt, true))\n return config.limiter === undefined ? invoke() : config.limiter.run(invoke, signal)\n}\n\n/** Plain-text verifier call for conservative JSON routing; probability labels are intentionally bypassed. */\nexport async function callVerifierText(config: VerifierClientConfig, prompt: string, signal?: AbortSignal): Promise<VerifierCompletion> {\n const invoke = () => retrying(config, signal, (attemptSignal, attempt) => callExplicitTag(config, prompt, attemptSignal, undefined, attempt))\n return config.limiter === undefined ? invoke() : config.limiter.run(invoke, signal)\n}\nexport function addUsage(target: UsageStats, source: UsageStats): void { for (const key of ['calls', 'attempts', 'retries', 'inputTokens', 'cachedInputTokens', 'outputTokens', 'reasoningTokens'] as const) target[key] += source[key] ?? 0 }\nexport function emptyUsage(): UsageStats { return { calls: 0, attempts: 0, retries: 0, inputTokens: 0, cachedInputTokens: 0, outputTokens: 0, reasoningTokens: 0 } }\n"],"mappings":";;;;;;AAiCA,IAAa,8BAAb,cAAiD,MAAM;CAAE,YAAY,SAAiB;EAAE,MAAM,OAAO;EAAG,KAAK,OAAO;CAA8B;AAAE;;;;;;AAMpJ,IAAa,wBAAb,cAA2C,4BAA4B;CAAwC;CAAtC,YAAY,SAAiB,QAA0B;EAAE,MAAM,OAAO;EAAhC,KAAA,SAAA;EAAmC,KAAK,OAAO;CAAwB;AAAE;;;;;;;;;;;;;AAaxL,IAAa,yBAAb,cAA4C,MAAM;CAAE,YAAY,SAAiB;EAAE,MAAM,OAAO;EAAG,KAAK,OAAO;CAAyB;AAAE;;;;;;;;;;;;;;;;;;;;AAqB1I,SAAS,gBAAgB,cAAkC,WAA0B;CACnF,IAAI,iBAAiB,UAAU,OAAO,IAAI,uBAAuB,0GAA0G,UAAU,wEAAwE;CAC7P,OAAO,IAAI,4BAA4B,yDAAyD,gBAAgB,aAAa,sBAAsB;AACrJ;AAEA,SAAS,OAAO,OAAqD;CAAE,OAAO,OAAO,UAAU,YAAY,UAAU,QAAQ,CAAC,MAAM,QAAQ,KAAK,IAAI,QAAmC,KAAA;AAAU;AAClM,SAAS,KAAK,OAAoC;CAAE,OAAO,OAAO,UAAU,YAAY,MAAM,KAAK,IAAI,MAAM,KAAK,IAAI,KAAA;AAAU;AAChI,SAAS,SAAS,SAAyB;CAAE,OAAO,QAAQ,QAAQ,QAAQ,EAAE,IAAI;AAAoB;AACtG,SAAS,QAAQ,OAA8B;CAAE,OAAO,UAAU,MAAM,YAAY,aAAa,OAAO,KAAK,MAAM,KAAK,QAAQ,MAAM,KAAK,YAAY,MAAM,KAAK,UAAU,CAAC,CAAC,SAAS,QAAQ;AAAE;AAEjM,eAAe,WAAW,KAAc,MAAuD;CAC7F,IAAI,CAAC,MAAM,OAAO,KAAA;CAElB,QAAQ,MADS,IAAI,IAAI,aACJ,CAAC,EAAE,QAAQ,cAAc,IAAI,CAAC,EAAA,EAAI;AACzD;;;;;;;;;;;;;;AAeA,SAAS,kBAAkB,KAAc,IAAiD;CACxF,MAAM,WAAW,IAAI,IAAI,UAAU;CACnC,IAAI,CAAC,UAAU,OAAO,KAAA;CACtB,IAAI;EACF,IAAI,OAAO,SAAS,QAAQ,YAAY,OAAO,OAAQ,SAAS,IAAgC,EAAE,CAAC;EACnG,IAAI,OAAO,SAAS,aAAa,YAAY,OAAO,KAAA;EACpD,MAAM,OAAiB,SAAS,SAA2B;EAC3D,IAAI,CAAC,MAAM,QAAQ,IAAI,GAAG,OAAO,KAAA;EAEjC,OAAO,OADK,KAAK,IAAI,MAAM,CAAC,CAAC,MAAK,cAAa,WAAW,OAAO,EACjD,CAAC,EAAE,KAAK;CAC1B,QAAQ;EACN;CACF;AACF;AAEA,eAAsB,uBAAuB,KAAc,UAAwD;CACjH,IAAI,aAAa,qBAAqB;EACpC,MAAM,QAAQ,kBAAkB,KAAK,cAAc,KAAK,CAAC;EAEzD,MAAM,SAAS,MAAM,WAAW,KADd,KAAK,MAAM,SAAS,KAAK,kBACG;EAC9C,IAAI,CAAC,QAAQ,OAAO,KAAA;EACpB,OAAO;GAAE,SAAS,KAAK,MAAM,OAAO,KAAK;GAA4B;GAAQ,kBAAkB;EAAK;CACtG;CACA,MAAM,OAAO,kBAAkB,KAAK,WAAW;CAC/C,IAAI,SAAS,KAAA,GAAW,OAAO,KAAA;CAE/B,MAAM,UAAU,OADC,OAAO,MAAM,SACA,CAAC,GAAG,SAAS;CAG3C,IAAI,CAAC,WAAW,QAAQ,QAAQ,sBAAsB,OAAO,KAAA;CAC7D,MAAM,UAAU,KAAK,QAAQ,OAAO;CACpC,IAAI,CAAC,WAAW,CAAC,eAAe,KAAK,OAAO,GAAG,OAAO,KAAA;CACtD,MAAM,SAAS,MAAM,WAAW,KAAK,KAAK,QAAQ,SAAS,CAAC;CAC5D,MAAM,aAAa,OAAO,QAAQ,OAAO;CACzC,MAAM,UAAU,eAAe,KAAA,IAAY,KAAA,IAAY,OAAO,YAAY,OAAO,QAAQ,UAAU,CAAC,CAAC,QAAQ,UAAqC,OAAO,MAAM,OAAO,QAAQ,CAAC;CAC/K,OAAO;EAAE;EAAS,GAAI,SAAS,EAAE,OAAO,IAAI,CAAC;EAAI,GAAI,UAAU,EAAE,QAAQ,IAAI,CAAC;EAAI,kBAAkB;CAAM;AAC5G;AAEA,eAAsB,gBAAgB,OAAwB,OAAe,QAAgB,WAAmB,iBAAqC,QAAsB,QAAmC,UAAU,GAAG,cAAc,IAAoC;CAC3Q,MAAM,UAA8C,QAAQ,SAAS,CAAC;EAAE,MAAM;EAAQ,MAAM;CAAO,GAAG,GAAG,OAAO,KAAI,WAAU;EAAE,MAAM;EAAa,WAAW,EAAE,KAAK,QAAQ,KAAK,EAAE;CAAE,EAAE,CAAC,IAAI;CAC7L,MAAM,WAAW,MAAM,oBAAoB,kBAAkB,oBAAoB,QAAQ,EAAE,UAAU,EAAE,MAAM,WAAW,EAAE,IAAI;EAAE,UAAU,EAAE,MAAM,UAAU;EAAG,kBAAkB;CAAgB,IAAI,CAAC;CACtM,MAAM,WAAW,MAAM,MAAM,SAAS,MAAM,OAAO,GAAG;EACpD,QAAQ;EAAQ,UAAU;EAAS;EACnC,SAAS;GAAE,gBAAgB;GAAoB,GAAI,MAAM,SAAS,EAAE,eAAe,YAAY,MAAM,OAAO,IAAI,CAAC;GAAI,GAAG,MAAM;EAAQ;EACtI,MAAM,KAAK,UAAU;GAAE;GAAO,UAAU,CAAC;IAAE,MAAM;IAAQ;GAAQ,CAAC;GAAG,YAAY;GAAW;GAAa,UAAU;GAAM,cAAc;GAAI,GAAG;EAAS,CAAC;CAC1J,CAAC;CACD,MAAM,MAAM,MAAM,SAAS,KAAK;CAChC,IAAI,CAAC,SAAS,IAAI;EAChB,MAAM,UAAU,IAAI,MAAM,GAAG,GAAI;EACjC,MAAM,UAAU,yDAAyD,SAAS,SAAS,OAAO;EAClG,IAAI;GAAC;GAAK;GAAK;GAAK;GAAK;EAAG,CAAC,CAAC,SAAS,SAAS,MAAM,KAAK,6GAA6G,KAAK,OAAO,GAAG,MAAM,IAAI,4BAA4B,0CAA0C,SAAS,SAAS,MAAM,OAAO;EAGtS,IAAI,SAAS,WAAW,OAAO,SAAS,WAAW,OAAO,SAAS,WAAW,OAAO,SAAS,UAAU,KAAK,MAAM,IAAI,MAAM,OAAO;EACpI,MAAM,IAAI,sBAAsB,SAAS,SAAS,MAAM;CAC1D;CACA,IAAI;CACJ,IAAI;EAAE,OAAO,OAAO,KAAK,MAAM,GAAG,CAAC,KAAK,CAAC;CAAE,QAAQ;EAAE,MAAM,IAAI,sBAAsB,6DAA6D,SAAS,MAAM;CAAE;CAEnK,MAAM,SAAS,QADC,MAAM,QAAQ,KAAK,OAAO,IAAI,KAAK,UAAU,CAAC,EAAA,CAChC,EAAE;CAChC,MAAM,UAAU,OAAO,QAAQ,OAAO;CACtC,MAAM,SAAS,OAAO,SAAS,YAAY,WAAW,QAAQ,UAAU;CACxE,MAAM,WAAW,OAAO,QAAQ,QAAQ;CACxC,MAAM,OAAO,MAAM,QAAQ,UAAU,OAAO,IAAI,SAAS,UAAU,CAAC;CACpE,IAAI,CAAC,KAAK,QAAQ,MAAM,gBAAgB,KAAK,QAAQ,aAAa,GAAG,SAAS;CAC9E,MAAM,SAAmB,CAAC;CAC1B,MAAM,YAAkC,CAAC;CACzC,KAAK,MAAM,UAAU,MAAM;EACzB,MAAM,MAAM,OAAO,MAAM,KAAK,CAAC;EAC/B,MAAM,QAAQ,OAAO,IAAI,UAAU,WAAW,IAAI,QAAQ;EAC1D,OAAO,KAAK,KAAK;EAEjB,MAAM,gBADM,MAAM,QAAQ,IAAI,YAAY,IAAI,IAAI,eAAe,CAAC,EAAA,CACzC,SAAQ,UAAS;GAAE,MAAM,OAAO,OAAO,KAAK;GAAG,OAAO,QAAQ,OAAO,KAAK,UAAU,YAAY,OAAO,KAAK,YAAY,WAAW,CAAC;IAAE,OAAO,KAAK;IAAO,SAAS,KAAK;GAAQ,CAAC,IAAI,CAAC;EAAE,CAAC;EACjN,IAAI,CAAC,aAAa,UAAU,OAAO,IAAI,YAAY,UAAU,aAAa,KAAK;GAAE;GAAO,SAAS,IAAI;EAAQ,CAAC;EAC9G,UAAU,KAAK,YAAY;CAC7B;CACA,MAAM,WAAW,OAAO,KAAK,KAAK,KAAK,CAAC;CACxC,MAAM,gBAAgB,OAAO,SAAS,qBAAqB,KAAK,CAAC;CACjE,MAAM,oBAAoB,OAAO,SAAS,yBAAyB,KAAK,CAAC;CACzE,MAAM,SAAS,OAAO,SAAS,2BAA2B,cAAc,iBAAiB,CAAC,KAAK;CAC/F,MAAM,QAAQ,OAAO,SAAS,iBAAiB,CAAC,KAAK;CACrD,OAAO;EAAE,MAAM;EAAQ;EAAQ;EAAW,aAAa;EAAgB,OAAO;GAAE,OAAO;GAAG,UAAU;GAAS,SAAS,UAAU;GAAG,aAAa,KAAK,IAAI,GAAG,QAAQ,MAAM;GAAG,mBAAmB;GAAQ,cAAc,OAAO,SAAS,qBAAqB,CAAC,KAAK;GAAG,iBAAiB,OAAO,kBAAkB,oBAAoB,CAAC,KAAK;EAAE;CAAE;AAC9U;;AAQA,SAAgB,sBAAsB,UAAkB,MAAM,QAAQ,IAAI,GAAW;CAEnF,OAAO,KADM,WAAW,QAAQ,IAAI,WAAW,QAAQ,KAAK,QAAQ,GAClD,sBAAsB;AAC1C;;AAGA,MAAM,0CAA0B,IAAI,IAAI;CAAC;CAAS;CAAU;AAAO,CAAC;AAEpE,IAAI,iBAAiB;AAErB,eAAe,YAAY,WAAmB,QAA+B;CAC3E,KAAK,IAAI,UAAU,IAAK,WAAW,GACjC,IAAI;EACF,MAAM,OAAO,WAAW,MAAM;EAC9B;CACF,SAAS,OAAO;EACd,MAAM,OAAQ,MAAgC;EAC9C,IAAI,WAAW,KAAK,SAAS,KAAA,KAAa,CAAC,wBAAwB,IAAI,IAAI,GAAG,MAAM;EACpF,MAAM,IAAI,SAAQ,YAAW,WAAW,SAAS,UAAU,EAAE,CAAC;CAChE;AAEJ;AAEA,IAAa,4BAAb,MAAuC;CAMR;CAAgC;CAL7D,8BAA+B,IAAI,IAAoB;CACvD,SAAiB;CACjB;CACA,UAAiC,QAAQ,QAAQ;CAEjD,YAAY,MAAgC,MAAqC,KAAK,KAAK;EAA9D,KAAA,OAAA;EAAgC,KAAA,MAAA;CAA+B;;CAG5F,cAAc,UAAkB,OAAwB;EACtD,MAAM,MAAM,WAAW,OAAO;EAC9B,MAAM,WAAW,KAAK,YAAY,IAAI,GAAG;EACzC,IAAI,aAAa,KAAA,GAAW,OAAO;EACnC,IAAI,KAAK,IAAI,IAAI,WAAA,OAA8B;GAAE,KAAK,YAAY,OAAO,GAAG;GAAG,OAAO;EAAM;EAC5F,OAAO;CACT;;CAGA,MAAM,eAA8B;EAClC,IAAI,KAAK,UAAU,KAAK,SAAS,KAAA,GAAW;EAC5C,KAAK,eAAe,YAAY;GAC9B,IAAI;IACF,MAAM,WAAW,KAAK,MAAM,MAAM,SAAS,KAAK,MAAO,MAAM,CAAC;IAC9D,MAAM,UAAU,aAAa,QAAQ,OAAO,aAAa,YAAY,SAAS,YAAY,KAAK,OAAO,SAAS,YAAY,YAAY,SAAS,YAAY,OAAO,SAAS,UAAU,CAAC;IACvL,MAAM,MAAM,KAAK,IAAI;IACrB,KAAK,MAAM,CAAC,KAAK,aAAa,OAAO,QAAQ,OAAO,GAAG;KACrD,IAAI,OAAO,aAAa,YAAY,CAAC,OAAO,SAAS,QAAQ,KAAK,WAAW,KAAK,MAAM,WAAA,OAA8B;KAEtH,MAAM,WAAW,KAAK,YAAY,IAAI,GAAG;KACzC,IAAI,aAAa,KAAA,KAAa,WAAW,UAAU,KAAK,YAAY,IAAI,KAAK,QAAQ;IACvF;GACF,QAAQ,CAAyD;GACjE,KAAK,SAAS;EAChB,EAAA,CAAG;EACH,MAAM,KAAK;CACb;CAEA,gBAAgB,UAAkB,OAAqB;EACrD,KAAK,YAAY,IAAI,WAAW,OAAO,OAAO,KAAK,IAAI,CAAC;EACxD,KAAK,QAAQ;CACf;;;;;;;;;;;;;;CAeA,OAAO,UAAkB,OAAqB;EAC5C,MAAM,MAAM,WAAW,OAAO;EAC9B,KAAK,YAAY,OAAO,GAAG;EAC3B,IAAI,KAAK,SAAS,KAAA,GAAW;EAC7B,MAAM,OAAO,KAAK,QACf,YAAY,CAAC,CAAC,CAAC,CACf,WAAW,KAAK,aAAa,CAAC,CAAC,CAC/B,WAAW;GAAE,KAAK,YAAY,OAAO,GAAG;GAAG,OAAO,KAAK,cAAc;EAAE,CAAC;EAC3E,KAAK,UAAU;EACf,KAAK,YAAY,CAAyC,CAAC;CAC7D;;CAGA,UAAwB;EACtB,IAAI,KAAK,SAAS,KAAA,GAAW;EAC7B,MAAM,OAAO,KAAK,QACf,YAAY,CAAC,CAAC,CAAC,CACf,WAAW,KAAK,aAAa,CAAC,CAAC,CAC/B,WAAW,KAAK,cAAc,CAAC;EAClC,KAAK,UAAU;EACf,KAAK,YAAY,CAA0E,CAAC;CAC9F;CAEA,MAAc,gBAA+B;EAC3C,MAAM,WAA+B;GAAE,SAAS;GAAG,SAAS,OAAO,YAAY,KAAK,WAAW;EAAE;EACjG,MAAM,MAAM,QAAQ,KAAK,IAAK,GAAG,EAAE,WAAW,KAAK,CAAC;EACpD,MAAM,YAAY,KAAK,OAAQ,UAAU,QAAQ,MAAM,MAAO,EAAE;EAChE,MAAM,UAAU,WAAW,KAAK,UAAU,QAAQ,GAAG,MAAM;EAC3D,IAAI;GAAE,MAAM,YAAY,WAAW,KAAK,IAAK;EAAE,SAAS,OAAO;GAAE,MAAM,OAAO,SAAS,CAAC,CAAC,YAAY,CAAC,CAAC;GAAG,MAAM;EAAM;CACxH;;CAGA,QAAuB;EAAE,OAAO,KAAK;CAAQ;AAC/C;;;;;;;;;;;;;;;;AC7RA,SAAS,WAAc,OAAa;CAClC,MAAM,MAAM;CACZ,IAAI,OAAO,IAAI,eAAe,YAC5B,OAAQ,IAAI,WAAwB,KAAK;CAE3C,OAAO,mBAAmB,KAAK;AACjC;;;;;;AAOA,SAAgB,mBAAsB,OAAa;CAEjD,IAAI,iBAAiB,aAAa,OAAO;CACzC,IAAI,SAAS,OAAO,UAAU,YAAY,CAAC,OAAO,SAAS,KAAK,GAAG;EACjE,OAAO,OAAO,KAAK;EACnB,KAAK,MAAM,OAAO,OAAO,KAAK,KAAK,GACjC,mBAAoB,MAAkC,IAAI;CAE9D;CACA,OAAO;AACT;;;;;;;;AA+DA,MAAM,mBAAmB,OAAO,8BAA8B;;AAG9D,SAAgB,gBAAgB,OAAwB;CACtD,IAAI,OAAO,UAAU,YAAY,UAAU,MAAM,OAAO;CACxD,MAAM,QAAS,MAA2C;CAC1D,OAAO,OAAO,UAAU,YAAY,OAAO,cAAc,KAAK,KAAK,QAAQ,IAAI,QAAQ;AACzF;;;;;;;;AASA,MAAM,gBAAgB,OAAO,2BAA2B;;AAGxD,SAAgB,aAA6B,OAA+B;CAC1E,IAAI,OAAO,UAAU,YAAY,UAAU,MAAM,OAAO,KAAA;CACxD,MAAM,QAAS,MAAwC;CACvD,OAAO,OAAO,UAAU,YAAY,UAAU,OAAO,QAAa,KAAA;AACpE;;AAGA,SAAgB,YAAY,OAAgB,OAAyB;CACnE,IAAI,OAAO,UAAU,YAAY,UAAU,MAAM,MAA4C,iBAAiB;AAChH;;;;;;AAOA,SAAS,SAAS,QAAgB,SAAiB,KAAwJ;CACzM,MAAM,wBAAQ,IAAI,MAAM,mBAAmB,MAAM;CACjD,YAAY,OAAO,MAAM,SAAS,OAAO,CAAC,CAAC,CAAC;CAC5C,OAAO;AACT;;;;;;;;;;AAWA,SAAS,eAAe,QAA0C;CAChE,IAAI,OAAO,SAAS,WAAW,OAAO,SAAS,WAAW;EACxD,MAAM,UAAU,OAAO;EACvB,MAAM,QAAQ;GACZ,QAAQ;GACR,QAAQ,WAAW,KAAA,IAAY,KAAA,IAAY,UAAU,QAAQ;GAC7D,QAAQ,cAAc,KAAA,IAAY,KAAA,IAAY,aAAa,QAAQ;EACrE,CAAC,CAAC,QAAQ,SAAyB,OAAO,SAAS,YAAY,KAAK,SAAS,CAAC;EAC9E,OAAO,MAAM,SAAS,IAAI,QAAQ,UAAU,OAAO,MAAM,KAAK,IAAI,IAAI,MAAM,QAAQ;CACtF;CACA,IAAI,OAAO,SAAS,cAAc,OAAO;AAE3C;AAEA,eAAe,MAAM,IAAY,QAAqC;CACpE,IAAI,QAAQ,SAAS,MAAM,OAAO;CAClC,MAAM,IAAI,SAAe,SAAS,WAAW;EAC3C,MAAM,QAAQ,WAAW,SAAS,EAAE;EACpC,MAAM,cAAc;GAAE,aAAa,KAAK;GAAG,OAAO,QAAQ,MAAM;EAAE;EAClE,QAAQ,iBAAiB,SAAS,OAAO,EAAE,MAAM,KAAK,CAAC;CACzD,CAAC;AACH;AAEA,SAAS,MAAM,UAAkB,QAAQ,CAAC,GAAiJ;CACzL,OAAO;EAAE,OAAO;EAAG;EAAU,SAAS,WAAW;EAAG,aAAa,MAAM,eAAe;EAAG,oBAAoB,MAAM,mBAAmB,MAAM,MAAM,oBAAoB;EAAI,cAAc,MAAM,gBAAgB;EAAG,iBAAiB,MAAM,mBAAmB;CAAE;AAC/P;;;;;;;;;;;AAYA,MAAM,oBAAoB;;;;;;;;;;;;AAa1B,SAAS,eAAe,OAAgB,SAAuB;CAC7D,IAAI,OAAO,UAAU,YAAY,UAAU,MAAM,MAA2C,oBAAoB;AAClH;;;;;;;;AASA,SAAS,aAAa,QAAiB,SAAwB;CAC7D,MAAM,SAAS,kBAAkB,QAAQ,SAAS,IAAI,MAAM,WAAW,KAAA,IAAY,kCAAkC,OAAO,MAAM,CAAC;CACnI,MAAM,QAAQ,IAAI,MAAM,OAAO,OAAO;CACtC,MAAM,OAAO,OAAO;CACpB,eAAe,OAAO,OAAO;CAC7B,OAAO;AACT;;AAGA,SAAS,UAAU,QAAoG,QAA0G;CAC/N,OAAO,eAAe,OAAO;CAC7B,OAAO,qBAAqB,OAAO;CACnC,OAAO,gBAAgB,OAAO;CAC9B,OAAO,mBAAmB,OAAO;AACnC;AAEA,eAAe,SAAY,QAA8B,QAAiC,KAAuE;CAC/J,IAAI,UAAU;CAId,MAAM,UAAU;EAAE,aAAa;EAAG,mBAAmB;EAAG,cAAc;EAAG,iBAAiB;CAAE;CAC5F,IAAI,iBAAiB;CAGrB,MAAM,SAAS,QAAgC,aAAkC;EAC/E,OAAO,WAAW,KAAA,IAAY,IAAI,OAAO;EACzC,UAAU;EACV,SAAS,KAAK,IAAI,GAAG,UAAU,CAAC;EAChC,aAAa,QAAQ,eAAe,QAAQ,eAAe;EAC3D,mBAAmB,QAAQ,qBAAqB,QAAQ,qBAAqB;EAC7E,cAAc,QAAQ,gBAAgB,QAAQ,gBAAgB;EAC9D,iBAAiB,QAAQ,mBAAmB,QAAQ,mBAAmB;EACvE,GAAI,UAAU,EAAE,iBAAiB,KAAK,IAAI,CAAC;CAC7C;CACA,MAAM,UAAU,QAAiB,QAAgC,YAA4B;EAC3F,MAAM,QAAQ,aAAa,QAAQ,OAAO;EAC1C,YAAY,OAAO,MAAM,QAAQ,OAAO,CAAC;EACzC,MAAM;CACR;CACA,OAAO,MAAM;EACX,IAAI,QAAQ,SAAS,OAAO,OAAO,QAAQ,KAAA,GAAW,KAAK;EAC3D,WAAW;EACX,MAAM,aAAa,IAAI,gBAAgB;EACvC,IAAI,WAAW;EACf,MAAM,UAAU,iBAAiB;GAAE,WAAW;GAAM,WAAW,sBAAM,IAAI,MAAM,iCAAiC,CAAC;EAAE,GAAG,OAAO,SAAS;EACtI,MAAM,cAAc,WAAW,MAAM,QAAQ,MAAM;EACnD,QAAQ,iBAAiB,SAAS,OAAO,EAAE,MAAM,KAAK,CAAC;EACvD,IAAI;GACF,MAAM,QAAQ,MAAM,IAAI,WAAW,QAAQ,OAAO;GAClD,IAAI,OAAO,UAAU,YAAY,UAAU,MAAM;IAC/C,MAAM,QAAS,MAAiC;IAChD,IAAI,UAAU,KAAA,GAAW;KACvB,UAAU,OAAO,OAAO;KAExB,IAAI,gBAAgB,MAAM,kBAAkB;IAC9C;GACF;GACA,OAAO;EACT,SAAS,OAAO;GACd,MAAM,SAAS,aAAyB,KAAK;GAC7C,IAAI,QAAQ,SAAS,OAAO,OAAO,UAAU,OAAO,QAAQ,kBAAkB,WAAW,KAAA,CAAS;GAOlG,MAAM,YAAY,YAAa,iBAAiB,SAAS,EAAE,iBAAiB,2BAA2B,kBAAkB,KAAK,MAAM,OAAO;GAC3I,IAAI,UAAU,OAAO,cAAc,CAAC,WAAW;IAG7C,IAAI,WAAW,KAAA,GAAW,iBAAiB;IAC3C,MAAM,UAAU,OAAO,UAAU,YAAY,UAAU,OAAO,QAAQ,IAAI,MAAM,OAAO,KAAK,CAAC;IAC7F,eAAe,SAAS,OAAO;IAC/B,YAAY,SAAS,MAAM,QAAQ,kBAAkB,WAAW,KAAA,CAAS,CAAC;IAC1E,MAAM;GACR;GACA,IAAI,WAAW,KAAA,GAAW,iBAAiB;QACtC,UAAU,SAAS,MAAM;GAC9B,IAAI;IACF,MAAM,MAAM,KAAK,IAAI,KAAO,OAAO,mBAAmB,MAAM,UAAU,MAAM,KAAM,KAAK,OAAO,IAAI,GAAI,GAAG,MAAM;GACjH,SAAS,WAAW;IAElB,OAAO,QAAQ,UAAU,WAAW,KAAA,GAAW,cAAc;GAC/D;EACF,UAAU;GACR,aAAa,OAAO;GACpB,QAAQ,oBAAoB,SAAS,KAAK;EAC5C;CACF;AACF;;AAGA,MAAM,4BAAY,IAAI,QAAyC;;;;;;;;;;;;;;;;;AAkB/D,eAAe,mBAAmB,QAA8B,QAAgB,QAAiC,QAA8C,SAAiB,oBAAmF;CACjQ,MAAM,UAA0B,CAAC;EAAE,MAAM;EAAQ,MAAM;CAAO,CAAC;CAC/D,KAAK,MAAM,SAAS,UAAU,CAAC,GAAG;EAChC,IAAI,UAAU,UAAU,IAAI,KAAK;EACjC,IAAI,YAAY,KAAA,GAAW;GASzB,UARgB,QAAQ,QAAQ,CAAC,CAC9B,WAAW,OAAO,YAAY,UAAU;IAAE,MAAM,MAAM;IAAM,WAAW,MAAM;GAAU,CAAC,CAAC,CAAC,CAC1F,OAAM,UAAS;IACd,IAAI,UAAU,IAAI,KAAK,MAAM,SAC3B,UAAU,OAAO,KAAK;IAExB,MAAM;GACR,CACc;GAChB,UAAU,IAAI,OAAO,OAAO;EAC9B;EACA,QAAQ,KAAK;GAAE,MAAM;GAAS,YAAY,MAAM;EAAiB,CAAC;CACpE;CACA,MAAM,WAAW,CAAC,kBAAkB;EAAE;EAAS,QAAQ,EAAE,MAAM,eAAe;CAAE,CAAC,CAAC;CAClF,MAAM,YAAY,IAAI,eAAe;CACrC,MAAM,UAAU,WAAW;EACzB,UAAU,OAAO;EACjB,OAAO,OAAO;EACd,GAAI,OAAO,kBAAkB,EAAE,iBAAiB,kBAAkB,OAAO,eAAe,EAAE,IAAI,CAAC;EAC/F;EACA,WAAW,OAAO;EAClB,aAAa,OAAO;EACpB;CACF,CAAC;CACD,WAAW,MAAM,SAAS,OAAO,IAAI,OAAO,OAAO,GAAG,UAAU,KAAK,KAAK;CAC1E,IAAI,YAAY;CAChB,IAAI,UAAU,OAAO,SAAS,cAAc;EAC1C,IAAI,CAAC,oBAAoB,MAAM,SAAS,wBAAwB,eAAe,UAAU,MAAM,GAAG,SAAS,UAAU,KAAK;EAC1H,YAAY;CACd,OAAO;EACL,MAAM,SAAS,eAAe,UAAU,MAAM;EAC9C,IAAI,WAAW,KAAA,GAAW,MAAM,SAAS,wBAAwB,QAAQ,SAAS,UAAU,KAAK;CACnG;CACA,MAAM,OAAO,UAAU,OAAO,CAAC,CAAC,QAAQ,UAA4D,MAAM,SAAS,MAAM,CAAC,CAAC,KAAI,UAAS,MAAM,IAAI,CAAC,CAAC,KAAK,EAAE;CAC3J,IAAI,CAAC,KAAK,KAAK,GAAG,MAAM,SAAS,uCAAuC,SAAS,UAAU,KAAK;CAGhG,OAAO;EAAE;EAAM,QAAQ,CAAC;EAAG,WAAW,CAAC;EAAG,aAAa;EAAgB,OAAO,MAAM,SAAS,UAAU,KAAK;EAAG;CAAU;AAC3H;;AAGA,eAAe,gBAAgB,QAA8B,QAAgB,QAAiC,QAA8C,SAA8C;CACxM,OAAO,mBAAmB,QAAQ,QAAQ,QAAQ,QAAQ,SAAS,KAAK;AAC1E;AAEA,IAAa,iBAAb,MAA4B;CAGL;CAFrB,SAAiB;CACjB,QAA4C,CAAC;CAC7C,YAAY,OAAwB;EAAf,KAAA,QAAA;EAAiB,IAAI,CAAC,OAAO,cAAc,KAAK,KAAK,QAAQ,GAAG,MAAM,IAAI,MAAM,oEAAoE;CAAE;CAC3K,MAAM,IAAO,WAA6B,QAAkC;EAC1E,IAAI,QAAQ,SAAS,MAAM,OAAO;EAClC,IAAI,KAAK,UAAU,KAAK,OAAO,MAAM,IAAI,SAAe,SAAS,WAAW;GAC1E,MAAM,cAAc;IAAE,QAAQ,oBAAoB,SAAS,KAAK;IAAG,QAAQ;GAAE;GAC7E,MAAM,cAAc;IAAE,MAAM,QAAQ,KAAK,MAAM,QAAQ,KAAK;IAAG,IAAI,SAAS,GAAG,KAAK,MAAM,OAAO,OAAO,CAAC;IAAG,OAAO,QAAQ,MAAM;GAAE;GACnI,KAAK,MAAM,KAAK,KAAK;GAAG,QAAQ,iBAAiB,SAAS,OAAO,EAAE,MAAM,KAAK,CAAC;EACjF,CAAC;EACD,IAAI,QAAQ,SAAS;GACnB,KAAK,MAAM,MAAM,CAAC,GAAG;GACrB,MAAM,OAAO;EACf;EACA,KAAK,UAAU;EACf,IAAI;GAAE,OAAO,MAAM,UAAU;EAAE,UAAU;GAAE,KAAK,UAAU;GAAG,KAAK,MAAM,MAAM,CAAC,GAAG;EAAE;CACtF;AACF;;AAGA,eAAsB,sBAAsB,QAAoD;CAC9F,MAAM,OAAO,uBAAuB,aAAa;CACjD,OAAO,OAAO,uBAAuB,cAAc,OAAO,UAAU,OAAO,KAAK,IAAI,iBAAiB;AACvG;AAEA,eAAe,cAAc,QAA8B,QAAgB,QAAiC,QAA8C,SAA8C;CACtM,MAAM,OAAO,uBAAuB,aAAa;CACjD,IAAI,WAAW;CACf,IAAI,CAAC,OAAO,uBAAuB,cAAc,OAAO,UAAU,OAAO,KAAK,GAAG;EAC/E,MAAM,QAAQ,MAAM,uBAAuB,OAAO,KAAK,OAAO,QAAQ;EACtE,IAAI,UAAU,KAAA,GACZ,IAAI;GAAE,OAAO,MAAM,gBAAgB,OAAO,OAAO,OAAO,QAAQ,OAAO,WAAW,OAAO,iBAAiB,QAAQ,QAAQ,SAAS,OAAO,WAAW;EAAE,SAChJ,OAAO;GAIZ,IAAI,EAAE,iBAAiB,8BAA8B,MAAM;GAC3D,OAAO,uBAAuB,gBAAgB,OAAO,UAAU,OAAO,KAAK;GAG3E,WAAW;EACb;OACK,OAAO,uBAAuB,gBAAgB,OAAO,UAAU,OAAO,KAAK;CACpF;CACA,MAAM,aAAa,MAAM,gBAAgB,QAAQ,QAAQ,QAAQ,QAAQ,OAAO;CAChF,OAAO,WAAW;EAAE,GAAG;EAAY,iBAAiB;CAAK,IAAI;AAC/D;AAEA,eAAsB,aAAa,QAA8B,QAAgB,QAAsB,QAAgE;CACrK,MAAM,eAAe,SAAS,QAAQ,SAAS,eAAe,YAAY,cAAc,QAAQ,QAAQ,eAAe,QAAQ,OAAO,CAAC;CACvI,OAAO,OAAO,YAAY,KAAA,IAAY,OAAO,IAAI,OAAO,QAAQ,IAAI,QAAQ,MAAM;AACpF;;;;;;;;AASA,MAAa,yBAAyB;;;;;;;;;;;;;;;;;AAiBtC,MAAa,wBAAwB;;;;;;;;;;;;;;AAwBrC,SAAgB,iBAAiB,MAA4B,QAA0B,YAAY,uBAA6C;CAC9I,OAAO;EACL,KAAK,KAAK;EACV,KAAK,KAAK;EACV,aAAa,KAAK;EAClB,wBAAwB,KAAK;EAC7B,UAAU,OAAO;EACjB,OAAO,OAAO;EACd,OAAO,OAAO,WAAW,MAAM,OAAO;EACtC,GAAI,OAAO,oBAAoB,KAAA,IAAY,CAAC,IAAI,EAAE,iBAAiB,OAAO,gBAAgB;EAC1F;EACA,aAAA;EACA,WAAW,KAAK;EAChB,YAAY,KAAK;EACjB,kBAAkB,KAAK;EACvB,GAAI,KAAK,YAAY,KAAA,IAAY,CAAC,IAAI,EAAE,SAAS,KAAK,QAAQ;CAChE;AACF;;;;;;;;;;;;;;;;;AAkBA,eAAsB,kBAAkB,MAA4B,QAA0B,QAAgB,QAAmD;CAC/J,OAAO,kBAAkB,iBAAiB,MAAM,MAAM,GAAG,QAAQ,MAAM;AACzE;;AAGA,eAAe,kBAAkB,QAA8B,QAAgB,QAA8D;CAC3I,MAAM,eAAe,SAAS,QAAQ,SAAS,eAAe,YAAY,mBAAmB,QAAQ,QAAQ,eAAe,KAAA,GAAW,SAAS,IAAI,CAAC;CACrJ,OAAO,OAAO,YAAY,KAAA,IAAY,OAAO,IAAI,OAAO,QAAQ,IAAI,QAAQ,MAAM;AACpF;;AAGA,eAAsB,iBAAiB,QAA8B,QAAgB,QAAmD;CACtI,MAAM,eAAe,SAAS,QAAQ,SAAS,eAAe,YAAY,gBAAgB,QAAQ,QAAQ,eAAe,KAAA,GAAW,OAAO,CAAC;CAC5I,OAAO,OAAO,YAAY,KAAA,IAAY,OAAO,IAAI,OAAO,QAAQ,IAAI,QAAQ,MAAM;AACpF;AACA,SAAgB,SAAS,QAAoB,QAA0B;CAAE,KAAK,MAAM,OAAO;EAAC;EAAS;EAAY;EAAW;EAAe;EAAqB;EAAgB;CAAiB,GAAY,OAAO,QAAQ,OAAO,QAAQ;AAAE;AAC7O,SAAgB,aAAyB;CAAE,OAAO;EAAE,OAAO;EAAG,UAAU;EAAG,SAAS;EAAG,aAAa;EAAG,mBAAmB;EAAG,cAAc;EAAG,iBAAiB;CAAE;AAAE"}
|
package/lib/caller.js
CHANGED
|
@@ -1,2 +1,2 @@
|
|
|
1
|
-
import { a as attachUsage, c as emptyUsage, d as generationClient, f as partialUsage, i as addUsage, l as fallbackDeepFreeze, m as requestAttempts, n as GENERATION_TEMPERATURE, o as callVerifier, p as predictScoringChannel, r as RequestLimiter, s as callVerifierText, t as GENERATION_MAX_TOKENS, u as generateCandidate } from "./caller-
|
|
1
|
+
import { a as attachUsage, c as emptyUsage, d as generationClient, f as partialUsage, i as addUsage, l as fallbackDeepFreeze, m as requestAttempts, n as GENERATION_TEMPERATURE, o as callVerifier, p as predictScoringChannel, r as RequestLimiter, s as callVerifierText, t as GENERATION_MAX_TOKENS, u as generateCandidate } from "./caller-CPbFWOz7.js";
|
|
2
2
|
export { GENERATION_MAX_TOKENS, GENERATION_TEMPERATURE, RequestLimiter, addUsage, attachUsage, callVerifier, callVerifierText, emptyUsage, fallbackDeepFreeze, generateCandidate, generationClient, partialUsage, predictScoringChannel, requestAttempts };
|
package/lib/client.js
CHANGED
|
@@ -1161,7 +1161,7 @@ window.__ModuleLoader__.load({
|
|
|
1161
1161
|
"Use only the " + target + " value(s) and evidence labels shown in this prompt; never invent one. Omit every finding line when nothing is locatable — do not guess, and do not restate the whole task."
|
|
1162
1162
|
].join("\n");
|
|
1163
1163
|
}
|
|
1164
|
-
const DEFAULT_GROUND_TRUTH_NOTE = "**IMPORTANT:** Focus on observed tool and terminal output as ground truth. Do NOT trust the agent's self-assessment or claims of success.";
|
|
1164
|
+
const DEFAULT_GROUND_TRUTH_NOTE = "**IMPORTANT:** Focus on observed tool and terminal output as ground truth. Do NOT trust the agent's self-assessment or claims of success: a polished summary, a neat table, or an assertion that everything was verified turns up just as often in a run that failed. Problems the agent admits ARE evidence and must weigh accordingly, but the absence of admitted problems is not evidence either way; hedging, expressed uncertainty and trajectory length are all neutral. Judge the substance and how it was checked, not the tone.";
|
|
1165
1165
|
/**
|
|
1166
1166
|
* Deterministic per-prompt delimiter token.
|
|
1167
1167
|
*
|
|
@@ -2729,7 +2729,7 @@ window.__ModuleLoader__.load({
|
|
|
2729
2729
|
provider: import_lib.default.string().default("deepseek-official"),
|
|
2730
2730
|
model: import_lib.default.string().default("deepseek-flash"),
|
|
2731
2731
|
reasoningEffort: import_lib.default.string(),
|
|
2732
|
-
maxTokens: import_lib.default.number().step(1).min(1).default(
|
|
2732
|
+
maxTokens: import_lib.default.number().step(1).min(1).default(65536),
|
|
2733
2733
|
temperature: import_lib.default.number().min(0).max(2).default(.2),
|
|
2734
2734
|
label: import_lib.default.string(),
|
|
2735
2735
|
timeoutMs: import_lib.default.number().step(1).min(1).default(3e5),
|
|
@@ -2795,7 +2795,7 @@ window.__ModuleLoader__.load({
|
|
|
2795
2795
|
criteriaFile: "",
|
|
2796
2796
|
provider: "",
|
|
2797
2797
|
model: "",
|
|
2798
|
-
maxTokens:
|
|
2798
|
+
maxTokens: 65536,
|
|
2799
2799
|
temperature: .2,
|
|
2800
2800
|
maxConcurrency: 8,
|
|
2801
2801
|
maxRetries: 3,
|